tree: 5b0a028a39d73347ca8f7e8d657702179b749292 [path history] [tgz]
  1. alltypes_dictionary.parquet
  2. alltypes_plain.parquet
  3. alltypes_plain.snappy.parquet
  4. binary.parquet
  5. bloom_filter.bin
  6. byte_array_decimal.parquet
  7. datapage_v2.snappy.parquet
  8. dict-page-offset-zero.parquet
  9. encrypt_columns_and_footer.parquet.encrypted
  10. encrypt_columns_and_footer_aad.parquet.encrypted
  11. encrypt_columns_and_footer_ctr.parquet.encrypted
  12. encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted
  13. encrypt_columns_plaintext_footer.parquet.encrypted
  14. fixed_length_decimal.parquet
  15. fixed_length_decimal_legacy.parquet
  16. int32_decimal.parquet
  17. int64_decimal.parquet
  18. nation.dict-malformed.parquet
  19. nested_lists.snappy.parquet
  20. nested_maps.snappy.parquet
  21. nonnullable.impala.parquet
  22. nullable.impala.parquet
  23. nulls.snappy.parquet
  24. README.md
  25. repeated_no_annotation.parquet
  26. uniform_encryption.parquet.encrypted
data/README.md

Test data files for Parquet compatibility and regression testing

TODO: Document what each file is

Encrypted Files

Tests files with .parquet.encrypted suffix are encrypted using Parquet Modular Encryption.

A detailed description of the Parquet Modular Encryption specification can be found here:

 https://github.com/apache/parquet-format/blob/encryption/Encryption.md

Following are the keys and key ids (when using key_retriever) used to encrypt the encrypted columns and footer in the all the encrypted files:

  • Encrypted/Signed Footer:
    • key: {0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5}
    • key_id: “kf”
  • Encrypted column named double_field:
    • key: {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,0}
    • key_id: “kc1”
  • Encrypted column named float_field:
    • key: {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,1}
    • key_id: “kc2”

The following files are encrypted with AAD prefix “tester”:

  1. encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted
  2. encrypt_columns_and_footer_aad.parquet.encrypted

A sample that reads and checks these files can be found at the following tests:

cpp/src/parquet/encryption-read-configurations-test.cc
cpp/src/parquet/test-encryption-util.h