Google Git
Sign in
apache/arrow-experiments/HEAD/./data/rand-many-types
tree: e77cfcf1a5c2b465deb4595a47b23ebeff8eb871
  1. arrows-to-arrow.py
  2. arrows-to-parquet.py
  3. generate.py
  4. Makefile
  5. parquet-to-duckdb.sql
  6. parquet-to-sqlite.sql
  7. random.arrow
  8. random.arrows
  9. random.duckdb
  10. random.parquet
  11. random.sqlite
  12. README.md
  13. requirements.txt
data/rand-many-types/README.md

rand-many-types

This directory contains a file random.arrows in Arrow IPC stream format with randomly generated values in 20+ columns exercising many different Arrow data types. The Python script generate.py that generated the data file is included.

The same data is also included as a file in Arrow IPC file format (random.arrow), as a Parquet file (random.parquet), and as a DuckDB database file (random.duckdb) as the table named random. The Python and SQL used to generate these files is included.

To re-generate the data files (for example, if you change generate.py),

  1. Make sure duckdb is in your path and activate a Python environment with the packages in ./requirements.txt
  2. Run make