layout: post title: “Apache Arrow nanoarrow 0.3.0 Release” date: “2023-10-03 00:00:00” author: pmc categories: [release]

The Apache Arrow team is pleased to announce the 0.3.0 release of Apache Arrow nanoarrow. This release covers 42 resolved issues from 4 contributors.

Release Highlights

See the Changelog for a detailed list of contributions to this release.

C library

The nanoarrow 0.3.0 release includes a number of bugfixes and improvements to the core C library and IPC C extension:

  • Improved bit packing/unpacking utilities and performance in tandem with an experiment to use bitmasks in pandas
  • Added support for building and consuming interval arrays
  • Fixed full validation of offset buffers for specific types of corrupted data that caused overflow
  • Fixed crashes that occurred for certain types of corrupted data and improved error messages that were misleading for other types of corrupted data.

R bindings

The nanoarrow R bindings are distributed as the nanoarrow package on CRAN. The 0.3.0 release of the R bindings includes improvements in type support and stability. Notably:

  • Conversion to/from bit64::integer64() is now supported
  • Warnings and errors for conversions that are invalid or may loose accuracy were improved
  • Extension types and extension type registration are now supported
  • Conversion of dictionary-encoded arrays to R vectors is now supported
  • Conversion of map arrays to R vectors is now supported
  • Improved performance of stream conversion to R vectors for streams of indeterminate length (e.g., database results from ADBC).

Experimental non-CPU support

With the addition of the Arrow C Device data interface comes an opportunity for nanoarrow to handle non-CPU (e.g., GPU) data. The nanoarrow_device extension was drafted in tandem with the specification and Arrow C++ implementation. While it is not an official part of the 0.3.0 release, it is available in an experimental state for those interested in its use and/or development.

Contributor experience

The 0.3.0 release features adoption of pre-commit hooks to improve consistency and contributor experience with a complex repository. Special thanks to @WillAyd for setting this up!

Contributors

This release consists of contributions from 4 contributors in addition to the invaluable advice and support of the Apache Arrow developer mailing list.

$ git shortlog -sn d4f038ce58655ba6e996cdae165f1b33c3919d51..apache-arrow-nanoarrow-0.3.0 | grep -v "GitHub Actions"
    40  Dewey Dunnington
     7  William Ayd
     2  Bryce Mecum
     1  Dane Pitkin