tree: 5dee535b13ee0b2b8505c03b0c270661e8f61d80 [search]
  1. BTreeFixtureGenerator.java
  2. expected.json
  3. java-v1-bloom.btree
  4. java-v1-bloom.btree.meta
  5. java-v1-string-bloom.btree
  6. java-v1-string-bloom.btree.meta
  7. java-v1-string.btree
  8. java-v1-string.btree.meta
  9. java-v1.btree
  10. java-v1.btree.meta
  11. java-v2-bloom.btree
  12. java-v2-bloom.btree.meta
  13. java-v2-composite-ac.btree
  14. java-v2-composite-ac.btree.meta
  15. java-v2-composite-int.btree
  16. java-v2-composite-int.btree.meta
  17. java-v2-composite-keys.txt
  18. java-v2-composite-reversed.btree
  19. java-v2-composite-reversed.btree.meta
  20. java-v2-string-bloom.btree
  21. java-v2-string-bloom.btree.meta
  22. java-v2-string.btree
  23. java-v2-string.btree.meta
  24. java-v2.btree
  25. java-v2.btree.meta
  26. posting-1.bin
  27. posting-2.bin
  28. posting-3.bin
  29. posting-4.bin
  30. posting-5.bin
  31. posting-high-buckets.bin
  32. README.md
paimon-python/pypaimon/tests/resources/btree/README.md

Java BTree interoperability fixtures

These files are generated by BTreeFixtureGenerator.java using the repository's BTreeIndexWriter and BTreePostingList. They are not Python round-trip fixtures.

Each java-v{1,2}[-bloom].btree uses INT keys, no block compression, a 32-byte target block size, and an independent null bitmap containing row ID 999. The matching .meta file contains the writer's serialized index metadata. expected.json records the non-null row IDs for each key:

KeyRowsv2 encoding
110SINGLE (0)
220, 40, 4294967297DELTA_LIST (1)
31000 through 1127ROARING (2)
48589934592 through 8589934719ROARING (2)
59223372036854775680 through 9223372036854775807ROARING (2)

posting-{1..5}.bin contains the Java v2 posting for each key. The generator checks the actual encoding tag and the Java round-trip. Python file tests compare the postings extracted from the SST blocks against these bytes. Python posting tests check the tags and compare decoded row IDs against expected.json. posting-high-buckets.bin is an additional Java Roaring posting covering 4294967232 through 4294967359, across two high-word buckets in one payload.

java-v{1,2}-string[-bloom].btree and their .meta files use STRING keys ke, keep, keeper, key, and kf, with the same postings and null bitmap. These fixtures verify exact prefix queries across blocks, including all three v2 encodings and the exclusive prefix upper bound.

To regenerate from the repository root with JDK 8, first build and run the Java posting tests to obtain the reactor's exact classpath:

mvn -pl paimon-common -am -DwildcardSuites=none \
  -DfailIfNoTests=false -Dtest=BTreePostingListTest test

Then compile and run the generator without adding it to production sources:

from pathlib import Path
import subprocess
import tempfile
import xml.etree.ElementTree as ET

root = Path.cwd()
directory = root / 'paimon-python/pypaimon/tests/resources/btree'
report = root / 'paimon-common/target/surefire-reports/TEST-org.apache.paimon.globalindex.btree.BTreePostingListTest.xml'
properties = ET.parse(report).getroot().find('properties')
classpath = next(p.attrib['value'] for p in properties
                 if p.attrib['name'] == 'java.class.path')
with tempfile.TemporaryDirectory() as classes:
    subprocess.run(['javac', '-source', '8', '-target', '8', '-cp', classpath,
                    '-d', classes, str(directory / 'BTreeFixtureGenerator.java')], check=True)
    subprocess.run(['java', '-cp', classes + ':' + classpath,
                    'org.apache.paimon.globalindex.btree.BTreeFixtureGenerator',
                    str(directory)], check=True)

The fixtures were verified with Python 3.13 and pyroaring 1.1.0. These samples cover 64-bit portable Roaring payloads, including multiple high-word values and Long.MAX_VALUE. They do not establish compatibility with every pyroaring version.

The composite fixtures were generated by BTreeFixtureGenerator.java from Java source at e2cdaf54ffbfe84a86df7d0ca3fb1da2e2d0d8ba. Python tests attach field ID tuples for (a,b), (a,c), and (b,a); the key bytes contain ordered values, not field IDs. java-v2-composite-keys.txt checks mixed-type keys.