tree: eb75fed05e94cc452bbc5b6d21b26d615873c9ad [search]
  1. Apache.Arrow/
  2. Apache.Arrow.Compression/
  3. Apache.Arrow.Flight/
  4. Apache.Arrow.Flight.AspNetCore/
  5. Apache.Arrow.Flight.Sql/
  6. Apache.Arrow.Operations/
  7. Apache.Arrow.Scalars/
  8. README.md
src/README.md

Apache Arrow .NET

Apache Arrow is a universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics. This package is part of the .NET implementation of Apache Arrow.

Packages

  • Apache.Arrow: the core library (arrays, schemas, record batches and the Arrow IPC stream and file formats).
  • Apache.Arrow.Compression: LZ4 frame and Zstandard codecs for compressed Arrow IPC data.
  • Apache.Arrow.Flight: Arrow Flight client and server.
  • Apache.Arrow.Flight.AspNetCore: ASP.NET Core integration for Arrow Flight servers.
  • Apache.Arrow.Flight.Sql: Arrow Flight SQL client and server.
  • Apache.Arrow.Operations: codecs for the Variant binary format, including a JSON reader and writer.
  • Apache.Arrow.Scalars: scalar types, including readers, writers and an object model for the Parquet Variant binary format.

Usage

Example of reading all record batches from an Arrow IPC file:

using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using Apache.Arrow;
using Apache.Arrow.Ipc;

public static async Task<List<RecordBatch>> ReadArrowFileAsync(string filename)
{
    var batches = new List<RecordBatch>();
    using (var stream = File.OpenRead(filename))
    using (var reader = new ArrowFileReader(stream))
    {
        // ReadNextRecordBatchAsync() returns null once the reader is drained.
        // The nullable local keeps this sample warning-free in projects that
        // enable nullable reference types.
        RecordBatch? batch;
        while ((batch = await reader.ReadNextRecordBatchAsync()) != null)
        {
            batches.Add(batch);
        }
    }
    return batches;
}

Links