Apache Arrow is a universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics. This package is part of the .NET implementation of Apache Arrow.
Apache.Arrow: the core library (arrays, schemas, record batches and the Arrow IPC stream and file formats).Apache.Arrow.Compression: LZ4 frame and Zstandard codecs for compressed Arrow IPC data.Apache.Arrow.Flight: Arrow Flight client and server.Apache.Arrow.Flight.AspNetCore: ASP.NET Core integration for Arrow Flight servers.Apache.Arrow.Flight.Sql: Arrow Flight SQL client and server.Apache.Arrow.Operations: codecs for the Variant binary format, including a JSON reader and writer.Apache.Arrow.Scalars: scalar types, including readers, writers and an object model for the Parquet Variant binary format.Example of reading all record batches from an Arrow IPC file:
using System.Collections.Generic; using System.IO; using System.Threading.Tasks; using Apache.Arrow; using Apache.Arrow.Ipc; public static async Task<List<RecordBatch>> ReadArrowFileAsync(string filename) { var batches = new List<RecordBatch>(); using (var stream = File.OpenRead(filename)) using (var reader = new ArrowFileReader(stream)) { // ReadNextRecordBatchAsync() returns null once the reader is drained. // The nullable local keeps this sample warning-free in projects that // enable nullable reference types. RecordBatch? batch; while ((batch = await reader.ReadNextRecordBatchAsync()) != null) { batches.Add(batch); } } return batches; }