[improvement](be) Optimize bit unpacking with PDEP (#65738)
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
`BitPacking::UnpackValues` currently decodes every complete 32-value
batch with scalar shifts and masks. This PR adds an x86 PDEP
implementation for `uint8_t`, `uint16_t`, and `uint32_t`, with AVX2
widening for narrow `uint16_t` and `uint32_t` values.
The optimized functions are compiled with `target("bmi2,avx2")`. Runtime
dispatch uses `__builtin_cpu_supports("bmi2")` and
`__builtin_cpu_supports("avx2")`; unsupported CPUs, architectures,
output types, and remainder values continue to use the generic scalar
implementation. Doris therefore does not require a global `-mbmi2` build
flag.
Production dispatch deliberately uses PDEP only when `BIT_WIDTH < 16`.
The generic PDEP implementation remains available to the benchmark for
all supported widths, but repeated measurements found non-monotonic,
CPU- and working-set-dependent results at 16-32 bits. A detailed code
comment explains why the scalar implementation is retained for these
widths instead of adding an irregular CPU-specific allowlist.
The PR also adds unit tests for every supported bit width, including
full batches and truncated/remainder input, plus a reproducible
benchmark comparing:
- the generic scalar kernel;
- the direct PDEP kernel;
- the actual `BitPacking::UnpackValues` dispatch path.
The benchmark covers 4K, 256K, and 1M values and validates optimized
output against the scalar reference before measuring.
#### Benchmark results
Hardware: Intel Xeon Platinum 8457C, 48 KiB L1D and 2 MiB private L2 per
core. Release build pinned to one CPU. Values below are CPU-time medians
in microseconds.
L1-sized working set: 4K `uint32_t` values, measured with 9 randomized
repetitions. Each iteration touches a 16 KiB output and 1-7 KiB of
packed input; including the benchmark's reference output, all buffers
total 33-39 KiB and fit in the 48 KiB L1D.
| bit width | scalar | PDEP | speedup |
| ---: | ---: | ---: | ---: |
| 2 | 0.628 | 0.338 | 1.86x |
| 4 | 0.551 | 0.336 | 1.64x |
| 6 | 0.969 | 0.355 | 2.73x |
| 8 | 0.224 | 0.221 | 1.01x |
| 10 | 1.216 | 0.565 | 2.15x |
| 12 | 0.870 | 0.564 | 1.54x |
| 14 | 0.828 | 0.737 | 1.12x |
L2-sized working set: 256K `uint32_t` values. The output is 1 MiB and
the packed input is 64-448 KiB.
| bit width | scalar | PDEP | speedup |
| ---: | ---: | ---: | ---: |
| 2 | 41.0 | 27.9 | 1.47x |
| 4 | 38.5 | 29.0 | 1.33x |
| 6 | 61.6 | 28.6 | 2.15x |
| 8 | 32.5 | 31.5 | 1.03x |
| 10 | 77.6 | 36.0 | 2.16x |
| 12 | 51.1 | 35.9 | 1.42x |
| 14 | 50.6 | 47.1 | 1.07x |
The result is workload- and bit-width-dependent. PDEP is faster for all
measured widths while the working set remains in L1 or L2, although the
gains at widths 8 and 14 are marginal. At 1M values, the actual PDEP
path is faster than the scalar path at widths 10, 12, and 14, but slower
at widths 2, 4, 6, and 8.
#### High-width dispatch validation
Widths 16-32 were measured twice independently. The following table is
the second run with 9 randomized repetitions; `scalar/PDEP` greater than
1 means PDEP is faster.
| bit width | 256K scalar | 256K PDEP | scalar/PDEP | 1M scalar | 1M
PDEP | scalar/PDEP |
| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 16 | 27.84 | 55.89 | 0.498x | 234.49 | 273.85 | 0.856x |
| 17 | 62.60 | 60.04 | 1.043x | 253.42 | 255.63 | 0.991x |
| 18 | 65.04 | 56.52 | 1.151x | 263.04 | 258.88 | 1.016x |
| 19 | 77.90 | 59.92 | 1.300x | 313.56 | 261.44 | 1.199x |
| 20 | 63.27 | 55.17 | 1.147x | 283.19 | 264.64 | 1.070x |
| 21 | 67.23 | 58.28 | 1.154x | 277.33 | 269.93 | 1.027x |
| 22 | 70.41 | 57.38 | 1.227x | 284.12 | 271.71 | 1.046x |
| 23 | 68.79 | 68.49 | 1.004x | 281.42 | 289.78 | 0.971x |
| 24 | 73.75 | 56.84 | 1.297x | 342.26 | 278.40 | 1.229x |
| 25 | 67.05 | 81.08 | 0.827x | 291.52 | 330.73 | 0.881x |
| 26 | 69.09 | 79.70 | 0.867x | 290.12 | 324.68 | 0.894x |
| 27 | 66.78 | 61.01 | 1.095x | 285.06 | 293.61 | 0.971x |
| 28 | 63.18 | 75.05 | 0.842x | 290.85 | 310.27 | 0.937x |
| 29 | 64.87 | 58.74 | 1.104x | 295.88 | 304.93 | 0.970x |
| 30 | 64.47 | 60.25 | 1.070x | 303.09 | 307.15 | 0.987x |
| 31 | 53.94 | 66.08 | 0.816x | 316.73 | 310.85 | 1.019x |
| 32 | 42.03 | 71.03 | 0.592x | 297.63 | 333.45 | 0.893x |
The first independent run showed the same material regressions at widths
16, 25, 26, 28, and 32. The `uint16_t` width-16 boundary was also slower
with PDEP at 4K, 256K, and 1M values. Some high widths improve, but the
profitable set changes with the working set and has no monotonic
boundary. The production path therefore conservatively retains scalar
unpacking for all widths at or above 16. After this change, the actual
entry point was benchmarked again: width 15 follows direct PDEP, while
widths 16-32 follow the scalar path and all outputs match the scalar
reference.
### Release note
Improve bit-packed integer decoding for bit widths below 16 on BMI2 and
AVX2 capable x86 CPUs.
### Check List (For Author)
- Test:
- [x] Unit Test: `./run-be-ut.sh -j 48 --run --filter=BitPackingTest.*`
- [x] Manual test: compiled and linked `benchmark_test`; ran scalar,
direct PDEP, and actual-path cases at 4K, 256K, and 1M values
- [x] `build-support/check-format.sh`
- [x] `build-support/run-clang-tidy.sh --build-dir be/build_Release
--files be/benchmark/benchmark_main.cpp
be/test/util/bit_packing_test.cpp`
- Behavior changed: Yes. Supported x86 CPUs use PDEP for complete
32-value batches only when `BIT_WIDTH < 16`; all other cases retain the
scalar path.
- Does this need documentation: No
---------
Co-authored-by: Dongyang Li <lidongyang@selectdb.com>English • العربية • বাংলা • Deutsch • Español • فارسی • Français • हिन्दी • Bahasa Indonesia • Italiano • 日本語 • 한국어 • Polski • Português • Română • Русский • Slovenščina • ไทย • Türkçe • Українська • Tiếng Việt • 简体中文 • 繁體中文
Apache Doris is an open-source, real-time analytics and search database built on MPP architecture. It provides fast SQL analytics, lakehouse query acceleration, and hybrid search across structured, text, and vector data.
Explore the official website for the latest product overview, use cases, ecosystem updates, blogs, and user stories. For version updates, see all release notes.
| Use Case | What it provides |
|---|---|
| Customer-Facing Analytics | Ship sub-second interactive analytics to external users. |
| Data Warehousing | Build one real-time warehouse across business domains. |
| Observability | Analyze high-throughput logs, events, and metrics with SQL. |
| Doris for AI | Use vector, text, JSON, and structured search in one SQL engine. |
Apache Doris is built around three core capabilities. The website is the source of truth for detailed product descriptions and examples.
| Capability | What it provides |
|---|---|
| Real-Time Analytics | Streaming ingestion, incremental transformation, and sub-second queries under high concurrency. |
| Lakehouse Analytics | Fast SQL analytics over open table formats such as Iceberg, Delta Lake, and Hudi. |
| Hybrid Search | SQL-native analytics across JSON, full-text, and vector data for AI and search workloads. |
Doris sits at the center of the modern data stack. It connects upstream databases, streaming systems, and lakehouse storage with downstream BI, AI, analytics, and observability tools.
For the latest ecosystem coverage, visit the official website and the connection and integration documentation.
Apache Doris supports both compute-storage coupled and compute-storage decoupled deployments. In decoupled mode, stateless compute groups run over shared object storage, so you can scale compute on demand and isolate workloads.
Learn more in the deployment guide and deployment mode guide.
| Resource | What it provides |
|---|---|
| Community Report | Weekly updates on community activity, merged PRs, contributors, and feature progress. |
| Roadmap 2026 | The 2026 planning discussion for AI and hybrid search, query engine, storage, and data lake work. |
Doris provides connectors and tools for common data engineering workflows.
Apache Doris is used in production by thousands of companies worldwide across internet services, finance, retail, logistics, manufacturing, energy, telecommunications, AI, and other industries.
Apache Doris graduated from the Apache Incubator and became an Apache Top-Level Project in June 2022. Thanks to all community contributors who help build Doris.
Note Some licenses of the third-party dependencies are not compatible with Apache 2.0 License. So you need to disable some Doris features to comply with Apache 2.0 License. For details, refer to the
thirdparty/LICENSE.txt