paimon-python/pypaimon/tests/resources/btree/README.md

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# Java BTree interoperability fixtures

These files are generated by `BTreeFixtureGenerator.java` using the repository's
`BTreeIndexWriter` and `BTreePostingList`. They are not Python round-trip fixtures.

Each `java-v{1,2}[-bloom].btree` uses INT keys, no block compression, a 32-byte
target block size, and an independent null bitmap containing row ID 999.
The matching `.meta` file contains the writer's serialized index metadata.
`expected.json` records the non-null row IDs for each key:

| Key | Rows | v2 encoding |
| --- | --- | --- |
| 1 | 10 | SINGLE (0) |
| 2 | 20, 40, 4294967297 | DELTA_LIST (1) |
| 3 | 1000 through 1127 | ROARING (2) |
| 4 | 8589934592 through 8589934719 | ROARING (2) |
| 5 | 9223372036854775680 through 9223372036854775807 | ROARING (2) |

`posting-{1..5}.bin` contains the Java v2 posting for each key. The generator
checks the actual encoding tag and the Java round-trip. Python file tests compare
the postings extracted from the SST blocks against these bytes. Python posting
tests check the tags and compare decoded row IDs against `expected.json`.
`posting-high-buckets.bin` is an additional Java Roaring posting covering
4294967232 through 4294967359, across two high-word buckets in one payload.

`java-v{1,2}-string[-bloom].btree` and their `.meta` files use STRING keys
`ke`, `keep`, `keeper`, `key`, and `kf`, with the same postings and null bitmap.
These fixtures verify exact prefix queries across blocks, including all three
v2 encodings and the exclusive prefix upper bound.

To regenerate from the repository root with JDK 8, first build and run the Java
posting tests to obtain the reactor's exact classpath:

```shell
mvn -pl paimon-common -am -DwildcardSuites=none \
  -DfailIfNoTests=false -Dtest=BTreePostingListTest test
```

Then compile and run the generator without adding it to production sources:

```python
from pathlib import Path
import subprocess
import tempfile
import xml.etree.ElementTree as ET

root = Path.cwd()
directory = root / 'paimon-python/pypaimon/tests/resources/btree'
report = root / 'paimon-common/target/surefire-reports/TEST-org.apache.paimon.globalindex.btree.BTreePostingListTest.xml'
properties = ET.parse(report).getroot().find('properties')
classpath = next(p.attrib['value'] for p in properties
                 if p.attrib['name'] == 'java.class.path')
with tempfile.TemporaryDirectory() as classes:
    subprocess.run(['javac', '-source', '8', '-target', '8', '-cp', classpath,
                    '-d', classes, str(directory / 'BTreeFixtureGenerator.java')], check=True)
    subprocess.run(['java', '-cp', classes + ':' + classpath,
                    'org.apache.paimon.globalindex.btree.BTreeFixtureGenerator',
                    str(directory)], check=True)
```

The fixtures were verified with Python 3.13 and pyroaring 1.1.0. These samples
cover 64-bit portable Roaring payloads, including multiple high-word values and
`Long.MAX_VALUE`. They do not establish compatibility with every pyroaring version.

The composite fixtures were generated by `BTreeFixtureGenerator.java` from
Java source at `e2cdaf54ffbfe84a86df7d0ca3fb1da2e2d0d8ba`. Python tests
attach field ID tuples for `(a,b)`, `(a,c)`, and `(b,a)`; the key bytes contain
ordered values, not field IDs. `java-v2-composite-keys.txt` checks mixed-type keys.