Helpers for Arrow C Data & Arrow C Stream interfaces

Clone this repo:
  1. 0247fa9 ArrowArray consumer buffer helpers (#19) by Dewey Dunnington · 3 days ago main
  2. 3b30507 Buffer element appenders (#17) by Dewey Dunnington · 4 days ago
  3. df62a5e More realistic cmake user presets file (#15) by Dewey Dunnington · 6 days ago
  4. 89b5932 Add metadata builder functions (#12) by Dewey Dunnington · 6 days ago
  5. 1257834 Owning/mutable `struct ArrowArray` (#14) by Dewey Dunnington · 7 days ago

nanoarrow

Codecov test coverage

The nanoarrow library is a set of helper functions to interpret and generate Arrow C Data Interface and Arrow C Stream Interface structures. The library is in active development and should currently be used only for entertainment purposes. Everything from the name of the project to the variable names are up for grabs (i.e., suggest/pull request literally any ideas you may have!).

Whereas the current suite of Arrow implementations provide the basis for a comprehensive data analysis toolkit, this library is intended to support clients that wish to produce or interpret Arrow C Data and/or Arrow C Stream structures. The library will:

  • Create, copy, parse, and validate struct ArrowSchema objects (all types mentioned in the C Data interface specification)
  • Create and validate struct ArrowArray/struct ArrowSchema pairs for (all types mentioned in the C Data interface specification)
  • Iterate over struct ArrowArrays element-wise (non-nested types) (i.e., is the ith element null; get the ith element).
  • Build Arrays element-wise (non-nested types) (i.e., basic Array Builder logic).

While it will not provide full support for nested types, it should provide enough infrastructure that an extension library with a similar format could implement such support.

Usage

You can use nanoarrow in your project in two ways:

  1. Copy contents of the src/nanoarrow/ into your favourite include directory and #include <nanoarrow/nanoarrow.c> somewhere in your project exactly once.
  2. Clone and use cmake, cmake --build, and cmake --install to build/install the static library and add -L/path/to/nanoarrow/lib -lnanoarrow to your favourite linker flag configuration.

All public functions and types are declared in nanoarrow/nanoarrow.h.

In all cases you will want to copy this project or pin your build to a specific commit since it will change rapidly and regularly. The nanoarrow library does not and will not provide ABI stability (i.e., you must vendor or link to a private version of the static library).

Background

The design of nanoarrow reflects the needs of a few previous libraries/prototypes requiring a library with a similar scope: