Google Git
Sign in
apache / arrow-nanoarrow / d9f428dec04f1a610b1c76b58f454be560a004df
commitd9f428dec04f1a610b1c76b58f454be560a004df[log] [tgz]
authorDewey Dunnington <dewey@dunnington.ca>Wed May 24 21:35:03 2023 -0400
committerGitHub <noreply@github.com>Wed May 24 22:35:03 2023 -0300
tree6f6ed29d70fbbde32959afa906979fa9e919c1f3
parent80904754895c7709ef4bee6c5e6f7d11ecdfc6cf [diff]
refactor: Unify `ArrowArrayView` and `ArrowArray` validation (#201)

The initial motivation for this refactor was to make it possible to go
straight from Arrow IPC to ArrowArrayView with zero heap allocations and
improve IPC validation; however, the changes needed for that (basically,
give every ArrowArrayView its own offset, length, and null_count)
allowed unifying the validation that we do when building arrays and the
validation we do when wrapping a foregin ArrowArray in an
ArrowArrayView. This PR ensures that no internal ArrowArrayView
functions actually refer to the `array` member since there is no
guarantee that an ArrowArray backs the buffers. Along the way, error
messages and documentation were improved.
6 files changed
tree: 6f6ed29d70fbbde32959afa906979fa9e919c1f3
  1. .github/
  2. ci/
  3. dev/
  4. dist/
  5. docs/
  6. examples/
  7. extensions/
  8. python/
  9. r/
  10. src/
  11. .asf.yaml
  12. .clang-format
  13. .env
  14. .gitattributes
  15. .gitignore
  16. CHANGELOG.md
  17. CMakeLists.txt
  18. CMakePresets.json
  19. CMakeUserPresets.json.example
  20. docker-compose.yml
  21. LICENSE.txt
  22. NOTICE.txt
  23. README.md
  24. valgrind.supp
README.md

nanoarrow

Codecov test coverage Documentation nanoarrow on GitHub

The nanoarrow library is a set of helper functions to interpret and generate Arrow C Data Interface and Arrow C Stream Interface structures. The library is in active early development and users should update regularly from the main branch of this repository.

Whereas the current suite of Arrow implementations provide the basis for a comprehensive data analysis toolkit, this library is intended to support clients that wish to produce or interpret Arrow C Data and/or Arrow C Stream structures where linking to a higher level Arrow binding is difficult or impossible.

Using the C library

The nanoarrow C library is intended to be copied and vendored. This can be done using CMake or by using the bundled nanoarrow.h/nanorrow.c distribution available in the dist/ directory in this repository. Examples of both can be found in the examples/ directory in this repository.

A simple producer example:

#include "nanoarrow.h"

int make_simple_array(struct ArrowArray* array_out, struct ArrowSchema* schema_out) {
  struct ArrowError error;
  array_out->release = NULL;
  schema_out->release = NULL;

  NANOARROW_RETURN_NOT_OK(ArrowArrayInitFromType(array_out, NANOARROW_TYPE_INT32));

  NANOARROW_RETURN_NOT_OK(ArrowArrayStartAppending(array_out));
  NANOARROW_RETURN_NOT_OK(ArrowArrayAppendInt(array_out, 1));
  NANOARROW_RETURN_NOT_OK(ArrowArrayAppendInt(array_out, 2));
  NANOARROW_RETURN_NOT_OK(ArrowArrayAppendInt(array_out, 3));
  NANOARROW_RETURN_NOT_OK(ArrowArrayFinishBuildingDefault(array_out, &error));

  NANOARROW_RETURN_NOT_OK(ArrowSchemaInit(schema_out, NANOARROW_TYPE_INT32));

  return NANOARROW_OK;
}

A simple consumer example:

#include <stdio.h>

#include "nanoarrow.h"

int print_simple_array(struct ArrowArray* array, struct ArrowSchema* schema) {
  struct ArrowError error;
  struct ArrowArrayView array_view;
  NANOARROW_RETURN_NOT_OK(ArrowArrayViewInitFromSchema(&array_view, schema, &error));

  if (array_view.storage_type != NANOARROW_TYPE_INT32) {
    printf("Array has storage that is not int32\n");
  }

  int result = ArrowArrayViewSetArray(&array_view, array, &error);
  if (result != NANOARROW_OK) {
    ArrowArrayViewReset(&array_view);
    return result;
  }

  for (int64_t i = 0; i < array->length; i++) {
    printf("%d\n", (int)ArrowArrayViewGetIntUnsafe(&array_view, i));
  }

  ArrowArrayViewReset(&array_view);
  return NANOARROW_OK;
}