ORC-2214: [C++] Harden RLE integer decoders against buffer cursor overshoot ### What changes were proposed in this pull request? This is a small defense-in-depth hardening of the C++ RLE integer decoders so that a **malformed / truncated RLE stream can no longer walk the read cursor past the end of the internal stream buffer**. Two minimal changes: 1. **`RleDecoderV2::bufLength()` (`c++/src/RLEv2.hh`)** now returns `0` when the cursor has overshot the buffer (`bufferStart_ > bufferEnd_`) instead of the raw pointer difference. The difference is returned as an unsigned `uint64_t`; once `bufferStart_ > bufferEnd_` it underflows to a value near 2^64, which defeats the `std::min` clamps in the `UnpackDefault` bit-unpacking fast loops (`unrolledUnpack16/32/40/56/64` in `BpackingDefault.cc`) and lets them read `valuesRemaining * width` bytes off the end of the heap buffer. 2. **`RleDecoderV2::readByte()` (`c++/src/RleDecoderV2.cc`) and `RleDecoderV1::readByte()` (`c++/src/RLEv1.cc`)** now gate their buffer refill on `bufferStart_ >= bufferEnd_` instead of `bufferStart_ == bufferEnd_`. A cursor that has already overshot fails the strict-equality test and dereferences `*bufferStart_` out of bounds; `>=` makes an overshot cursor refill (and cleanly hit `ParseError` at EOF) instead. Both changes are behavior-neutral on valid data — for a well-formed stream `bufferStart_` never exceeds `bufferEnd_`, so `bufLength()` returns the same value and the refill guard fires at exactly the same point. They only bound the pathological overshoot path. ### Why are the changes needed? Reading a crafted/corrupt ORC file can currently drive the RLE integer decoders to read one or more bytes past the end of a heap stream buffer (AddressSanitizer: `heap-buffer-overflow READ of size 1`) via the public reader API (`orc::createReader` / `RowReader::next`). The effect is out-of-bounds **reads** / a crash of the reading process (no write / no code-execution demonstrated). The underlying cause is that the decode cursor `bufferStart_` is allowed to advance past the `bufferEnd_` sentinel when a run length / bit width taken from the file implies consuming more bytes than the current stream buffer holds; neither `bufLength()` nor the `readByte()` refill guard bounds the cursor once it has overshot. This was reported privately to `securityorc.apache.org`. Per the project's published security model (https://orc.apache.org/security/), a crash while reading an untrusted/malformed file is out of scope as a vulnerability, and Arnout Engelen (ASF Security) kindly invited us to submit the fix through the normal contribution channel as a hardening improvement — hence this PR. ### How was this patch tested? * Full existing C++ unit suite: **752/752 passed** (`orc-test`, `RelWithDebInfo`), including the 123 RLE / integer / byte-RLE / writer round-trip tests — no regressions. * Reproduced the original overreads against the **real file-based reader** (`readLocalFile` + `createReader` + `RowReader::next`) built with AddressSanitizer, on `main`: * **Before:** 31/31 malformed-file test cases produced `heap-buffer-overflow` in `RleDecoderV2::readByte`, `RleDecoderV1::readByte`, `UnpackDefault::unrolledUnpack32` and `unrolledUnpack64`. * **After:** 0/31 crash; every input now surfaces a handled exception (`ParseError` / decompression EOF) or decodes cleanly. No ASAN report. Representative before/after (`unrolledUnpack32` case): ``` before: AddressSanitizer: heap-buffer-overflow READ of size 1 #0 orc::UnpackDefault::unrolledUnpack32 BpackingDefault.cc:149 after: HANDLED-EXCEPTION: Buffer error in ZlibDecompressionStream::NextDecompress ``` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Anthropic Claude Opus 4.8) --- Contributed by **Roman Arce Brán — Cyfra Tech Solutions** (`arce.roman.bgmail.com`). Note on issue tracking: I don't have an Apache JIRA account to file the `ORC-xxxx` ticket that the PR template asks for, so this PR intentionally omits the `ORC-xxxx` prefix. Happy to add the reference (and reword the commit) once a JIRA is created/assigned — just let me know the number. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Closes #2691 from Roman-Da-Ripper/harden-rle-decoder-bounds. Authored-by: Roman Arce Brán <arce.roman.b@gmail.com> Signed-off-by: Gang Wu <ustcwg@gmail.com>
ORC is a self-describing type-aware columnar file format designed for Hadoop workloads. It is optimized for large streaming reads, but with integrated support for finding required rows quickly. Storing data in a columnar format lets the reader read, decompress, and process only the values that are required for the current query. Because ORC files are type-aware, the writer chooses the most appropriate encoding for the type and builds an internal index as the file is written. Predicate pushdown uses those indexes to determine which stripes in a file need to be read for a particular query and the row indexes can narrow the search to a particular set of 10,000 rows. ORC supports the complete set of types in Hive, including the complex types: structs, lists, maps, and unions.
This project includes both a Java library and a C++ library for reading and writing the Optimized Row Columnar (ORC) file format. The C++ and Java libraries are completely independent of each other and will each read all versions of ORC files.
Releases:
The current build status:
| Branch | Build Status |
|---|---|
| main |  |
| branch-2.3 |  |
| branch-2.2 |  |
| branch-2.1 |  |
| branch-2.0 |  |
| branch-1.9 |  |
Bug tracking: Apache Jira
The subdirectories are:
To build a release version with debug information:
% mkdir build % cd build % cmake .. % make package % make test-out
To build a debug version:
% mkdir build % cd build % cmake .. -DCMAKE_BUILD_TYPE=DEBUG % make package % make test-out
To build a release version without debug information:
% mkdir build % cd build % cmake .. -DCMAKE_BUILD_TYPE=RELEASE % make package % make test-out
To build only the Java library:
% cd java % ./mvnw package
To build only the C++ library:
% mkdir build % cd build % cmake .. -DBUILD_JAVA=OFF % make package % make test-out
To build the C++ library with AVX512 enabled:
export ORC_USER_SIMD_LEVEL=AVX512 % mkdir build % cd build % cmake .. -DBUILD_JAVA=OFF -DBUILD_ENABLE_AVX512=ON % make package % make test-out
Cmake option BUILD_ENABLE_AVX512 can be set to “ON” or (default value)“OFF” at the compile time. At compile time, it defines the SIMD level(AVX512) to be compiled into the binaries.
Environment variable ORC_USER_SIMD_LEVEL can be set to “AVX512” or (default value)“NONE” at the run time. At run time, it defines the SIMD level to dispatch the code which can apply SIMD optimization.
Note that if ORC_USER_SIMD_LEVEL is set to “NONE” at run time, AVX512 will not take effect at run time even if BUILD_ENABLE_AVX512 is set to “ON” at compile time.
While CMake is the official build system for orc, there is unofficial support for using Meson to build select parts of the project. To build a debug version of the library and test it using Meson, from the project root you can run:
meson setup build meson compile -C build meson test -C build
By default, Meson will build unoptimized libraries with debug symbols. By contrast, the CMake build system generates release libraries by default. If you would like to create release libraries ala CMake, you should set the buildtype option. You must either remove the existing build directory before changing that setting, or alternatively pass the --reconfigure flag:
meson setup build -Dbuildtype=release --reconfigure meson compile -C build meson test -C build
Meson supports running your test suite through valgrind out of the box:
meson test -C build --wrap=valgrind
If you'd like to enable sanitizers, you can leverage the -Db_sanitize= option. For example, to enable both ASAN and UBSAN, you can run:
meson setup build -Dbuildtype=debug -Db_sanitize=address,undefined --reconfigure meson compile -C build meson test
Meson takes care of detecting all dependencies on your system, and downloading missing ones as required through its Wrap system. The dependencies for the project are all stored in the subprojects directory in individual wrap files. The majority of these are system generated files created by running:
meson wrap install <depencency_name>
From the project root. If you are developing orc and need to add a new dependency in the future, be sure to check Meson's WrapDB to check if a pre-configured wrap entry exists. If not, you may still manually configure the dependency as outlined in the aforementioned Wrap system documentation.