Package smile.io

Class Arrow

java.lang.Object
smile.io.Arrow
public class Arrow extends Object
Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware.

  • Constructor Details

    • Arrow

      public Arrow()
      Constructor.

    • Arrow

      public Arrow(int batch)
      Constructor.
      Parameters:
      batch - the number of records in a record batch.

  • Method Details

    • allocate

      public static void allocate(long limit)
      Creates the root allocator. The RootAllocator is responsible for being the master bookeeper for memory allocations.
      Parameters:
      limit - the memory allocation limit in bytes.

    • read

      public DataFrame read(Path path) throws IOException
      Reads an arrow file.
      Parameters:
      path - the input file path.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.

    • read

      public DataFrame read(Path path, int limit) throws IOException
      Reads an arrow file.
      Parameters:
      path - the input file path.
      limit - the number of records to read.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.

    • read

      public DataFrame read(String path) throws IOException, URISyntaxException
      Reads a limited number of records from an arrow file.
      Parameters:
      path - the input file path.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.
      URISyntaxException - when the file path syntax is wrong.

    • read

      public DataFrame read(String path, int limit) throws IOException, URISyntaxException
      Reads a limited number of records from an arrow file.
      Parameters:
      path - the input file path.
      limit - the number of records to read.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.
      URISyntaxException - when the file path syntax is wrong.

    • read

      public DataFrame read(InputStream input, int limit) throws IOException
      Reads a limited number of records from an arrow file.
      Parameters:
      input - the input stream.
      limit - the number of records to read.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.

    • write

      public void write(DataFrame data, Path path) throws IOException
      Writes the data frame to an arrow file.
      Parameters:
      data - the data frame.
      path - the output file path.
      Throws:
      IOException - when fails to write the file.

    • toDataType

      public static DataType toDataType(org.apache.arrow.vector.types.pojo.Field field)
      Converts an arrow field to smile data type.
      Parameters:
      field - an arrow field.
      Returns:
      the data type.

    • toStructField

      public static StructField toStructField(org.apache.arrow.vector.types.pojo.Field field)
      Converts an arrow field to smile field.
      Parameters:
      field - an arrow field.
      Returns:
      the struct field.

    • toArrow

      public static org.apache.arrow.vector.types.pojo.Field toArrow(StructField field)
      Converts a smile struct field to arrow field.
      Parameters:
      field - smile struct field.
      Returns:
      the arrow field.

    • toStructType

      public static StructType toStructType(org.apache.arrow.vector.types.pojo.Schema schema)
      Converts an arrow schema to smile schema.
      Parameters:
      schema - an arrow schema.
      Returns:
      the struct type.

    • toArrow

      public static org.apache.arrow.vector.types.pojo.Schema toArrow(StructType schema)
      Converts smile schema to an arrow schema.
      Parameters:
      schema - smile schema.
      Returns:
      the arrow schema.