Google Git
Sign in
apache / arrow-nanoarrow / 5756b76697d47820675da196916ca89c0ab499d3
commit5756b76697d47820675da196916ca89c0ab499d3[log] [tgz]
authorDewey Dunnington <dewey@dunnington.ca>Thu Mar 07 14:46:48 2024 -0400
committerGitHub <noreply@github.com>Thu Mar 07 14:46:48 2024 -0400
treefe7d3eae133cac0c001ee40268a2027b0fe08dd0
parentc97720003ff863b81805bcdb9f7c91306ab6b6a8 [diff]
chore: Add basic benchmark suite to C library (#393)

This PR adds an initial set of benchmarks covering some realistic usage
patterns. The general approach is to use doxygen comments to document
the benchmarks, which will run against the released version and the
previous version. I'm not sure exactly what the output format will be
but I'd like the benchmarks to be written in such a way that there's a
path to programatically generating a report (maybe using conbench, maybe
just a Quarto document).

Work in progress!
5 files changed
tree: fe7d3eae133cac0c001ee40268a2027b0fe08dd0
  1. .github/
  2. ci/
  3. dev/
  4. dist/
  5. docs/
  6. examples/
  7. extensions/
  8. python/
  9. r/
  10. src/
  11. thirdparty/
  12. .asf.yaml
  13. .clang-format
  14. .cmake-format
  15. .env
  16. .flake8
  17. .gitattributes
  18. .gitignore
  19. .isort.cfg
  20. .pre-commit-config.yaml
  21. CHANGELOG.md
  22. CMakeLists.txt
  23. CMakePresets.json
  24. CMakeUserPresets.json.example
  25. docker-compose.yml
  26. LICENSE.txt
  27. NOTICE.txt
  28. README.md
  29. valgrind.supp
README.md

nanoarrow

Codecov test coverage Documentation nanoarrow on GitHub

The nanoarrow library is a set of helper functions to interpret and generate Arrow C Data Interface and Arrow C Stream Interface structures. The library is in active early development and users should update regularly from the main branch of this repository.

Whereas the current suite of Arrow implementations provide the basis for a comprehensive data analysis toolkit, this library is intended to support clients that wish to produce or interpret Arrow C Data and/or Arrow C Stream structures where linking to a higher level Arrow binding is difficult or impossible.

Using the C library

The nanoarrow C library is intended to be copied and vendored. This can be done using CMake or by using the bundled nanoarrow.h/nanorrow.c distribution available in the dist/ directory in this repository. Examples of both can be found in the examples/ directory in this repository.

A simple producer example:

#include "nanoarrow.h"

int make_simple_array(struct ArrowArray* array_out, struct ArrowSchema* schema_out) {
  struct ArrowError error;
  array_out->release = NULL;
  schema_out->release = NULL;

  NANOARROW_RETURN_NOT_OK(ArrowArrayInitFromType(array_out, NANOARROW_TYPE_INT32));

  NANOARROW_RETURN_NOT_OK(ArrowArrayStartAppending(array_out));
  NANOARROW_RETURN_NOT_OK(ArrowArrayAppendInt(array_out, 1));
  NANOARROW_RETURN_NOT_OK(ArrowArrayAppendInt(array_out, 2));
  NANOARROW_RETURN_NOT_OK(ArrowArrayAppendInt(array_out, 3));
  NANOARROW_RETURN_NOT_OK(ArrowArrayFinishBuildingDefault(array_out, &error));

  NANOARROW_RETURN_NOT_OK(ArrowSchemaInitFromType(schema_out, NANOARROW_TYPE_INT32));

  return NANOARROW_OK;
}

A simple consumer example:

#include <stdio.h>

#include "nanoarrow.h"

int print_simple_array(struct ArrowArray* array, struct ArrowSchema* schema) {
  struct ArrowError error;
  struct ArrowArrayView array_view;
  NANOARROW_RETURN_NOT_OK(ArrowArrayViewInitFromSchema(&array_view, schema, &error));

  if (array_view.storage_type != NANOARROW_TYPE_INT32) {
    printf("Array has storage that is not int32\n");
  }

  int result = ArrowArrayViewSetArray(&array_view, array, &error);
  if (result != NANOARROW_OK) {
    ArrowArrayViewReset(&array_view);
    return result;
  }

  for (int64_t i = 0; i < array->length; i++) {
    printf("%d\n", (int)ArrowArrayViewGetIntUnsafe(&array_view, i));
  }

  ArrowArrayViewReset(&array_view);
  return NANOARROW_OK;
}