[fix](types) Fix binary value ownership and timestamp primitives (#68301) ### What problem does this PR solve? This ports #68297 to `master`, preserving the first of five planned extractions from #67784. Binary `Field` values can retain references to released source storage, and Hive binary text needs its own Base64 contract. TIMESTAMPTZ output can lose historical offset seconds, format invalid NULL payloads, or fail again while reporting a boundary cast error. - Own long binary Field values while keeping short values inline. Preserve execution type lengths and decoder bytes, and add Hive Base64 and hexadecimal decoding support. - Explicitly reject unsupported binary hash keys, IN, aggregates, predicates and computed partition transforms. Keep the existing FE comparison/group/join restrictions and existing binary scalar functions. Reject unsupported collection kernels in each function's legality check before coercion. - Preserve historical second offsets in both TIMESTAMPTZ formatting and parsing. Skip masked NULL payloads, reject unrepresentable local years, and preserve cast error/NULL behavior at boundaries. Arrow convertor migration, Parquet/ORC semantics, external writer changes and catalog mapping migration belong to the subsequent extractions. This PR does not enable native VARBINARY storage. ### Master adaptation - Retain the fixed-offset normalization and tests already present on master. - Use the current void-returning `VInPredicate::_prepare_zonemap_min_max` interface in both the guard and its test. - Retain master header cleanup and existing timestamp-nanosecond tests. - Retain the existing master binary-literal encoder and its StringView input contract; the older std::string-based caller fix is not applicable. ### Testing - TIMESTAMPTZ regression follow-up: reproduced both binary-output and stream-load failures using the master PR CI artifact, regenerated the two snapshots through `run-regression-test.sh`, and passed both suites in comparison mode from each branch checkout. Explicit `Asia/Shanghai` session settings were verified with the server default session zone set to UTC. Only historical offset seconds changed in the generated results. - Function-local validation update: 18 FE tests passed with Checkstyle enabled, covering direct legality checks, nested/mixed/variadic VARBINARY arguments, both `collect_set` arities, supported ordinary types, SQL analysis, and existing array rewrites. The new direct-legality tests reproduced missing rejection before the change. - BE ASAN build and **199 tests passed** across 17 suites using `run-be-ut.sh`, including binary lifetime/SerDe/rejection, timestamp parsing/casts, and existing Arrow/Variant serialization coverage. - `VarBinaryUnsupportedCollectionTest`: **passed** (13 unsupported expressions plus supported byte-preserving collection analysis). The FE test reactor and repository Checkstyle passed after cleaning stale branch build artifacts. - Repository clang-format 16 check and build-header hygiene checks: **passed**; 31 changed C++ source/header files. - Groovy compilation of the three regression suites: **passed**. Live SQL regression execution remains pending CI. - clang-tidy was attempted but could not complete because master already contains an unmatched `NOLINTEND` in `be/src/core/types.h`. A diagnostic run with the compiler resource directory corrected reproduced that blocker; the other reported findings in `column_varbinary.cpp` were outside changed lines. This is not a clean clang-tidy result. The focused BE test source list and local test/build settings were restored before committing. No build configuration changes are included. ### Release note Fix binary value lifetime and serialization, reject unsupported binary computation paths, and preserve TIMESTAMPTZ historical offsets and boundary error behavior. ### Check List (For Author) - Test - [x] Regression test (three self-checking suites added; execution pending CI) - [x] Unit Test - Behavior changed: - [x] Yes. Binary rejection and timestamp boundary behavior are described above. - Does this need documentation? - [x] No. This fixes existing type behavior without introducing a configuration option. ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label ### Scoped review follow-up This follow-up only fixes correctness/stability defects introduced by this PR. Compatibility preservation, pre-existing limitations, additional VARBINARY computation/validation, performance refactors, and test-style-only rewrites are excluded. - Separate historical TIMESTAMPTZ wire-offset parsing from session fixed-zone limits in both parser paths. - Use UTC diagnostics for the TIMESTAMP_NS cast/comparison failures affected by the new local-year formatting exception. - Validation: 31 focused ASAN BE tests passed, including ordinary DATE/DATETIME parsing. Three targeted tests failed before the fixes. clang-format 16 and build hygiene passed. Full clang-tidy remains affected by pre-existing diagnostics. - Branch-specific UTC/GMT normalization and FE folding fixes are handled in #68297; the corresponding master behavior predates this PR or already defers folding. ### CI test follow-up - Keep the binary literal test's owning Field alive while reading its StringView. Branch-4.1 now has the corresponding short/long embedded-NUL coverage using its execution API. - Replace the obsolete +15:00 rejection input with +24:00. Add generated historical-offset checks in both cast modes; all prior snapshot results are unchanged. - Validation: 34 focused ASAN BE tests passed on each branch. The lifetime error and the original SQL mismatch were reproduced. The complete cast regression suite passed in comparison mode from both branch checkouts against the reported master CI artifact. clang-format 16 passed; full clang-tidy remains blocked by pre-existing diagnostics. This follow-up changes tests only and retains the agreed scope: no compatibility work or additional binary computation support. Existing muted failures are outside this fix.
English • العربية • বাংলা • Deutsch • Español • فارسی • Français • हिन्दी • Bahasa Indonesia • Italiano • 日本語 • 한국어 • Polski • Português • Română • Русский • Slovenščina • ไทย • Türkçe • Українська • Tiếng Việt • 简体中文 • 繁體中文
Apache Doris is an open-source, real-time analytics and search database built on MPP architecture. It provides fast SQL analytics, lakehouse query acceleration, and hybrid search across structured, text, and vector data.
Explore the official website for the latest product overview, use cases, ecosystem updates, blogs, and user stories. For version updates, see all release notes.
| Use Case | What it provides |
|---|---|
| Customer-Facing Analytics | Ship sub-second interactive analytics to external users. |
| Data Warehousing | Build one real-time warehouse across business domains. |
| Observability | Analyze high-throughput logs, events, and metrics with SQL. |
| Doris for AI | Use vector, text, JSON, and structured search in one SQL engine. |
Apache Doris is built around three core capabilities. The website is the source of truth for detailed product descriptions and examples.
| Capability | What it provides |
|---|---|
| Real-Time Analytics | Streaming ingestion, incremental transformation, and sub-second queries under high concurrency. |
| Lakehouse Analytics | Fast SQL analytics over open table formats such as Iceberg, Delta Lake, and Hudi. |
| Hybrid Search | SQL-native analytics across JSON, full-text, and vector data for AI and search workloads. |
Doris sits at the center of the modern data stack. It connects upstream databases, streaming systems, and lakehouse storage with downstream BI, AI, analytics, and observability tools.
For the latest ecosystem coverage, visit the official website and the connection and integration documentation.
Apache Doris supports both compute-storage coupled and compute-storage decoupled deployments. In decoupled mode, stateless compute groups run over shared object storage, so you can scale compute on demand and isolate workloads.
Learn more in the deployment guide and deployment mode guide.
| Resource | What it provides |
|---|---|
| Community Report | Weekly updates on community activity, merged PRs, contributors, and feature progress. |
| Roadmap 2026 | The 2026 planning discussion for AI and hybrid search, query engine, storage, and data lake work. |
Doris provides connectors and tools for common data engineering workflows.
Apache Doris is used in production by thousands of companies worldwide across internet services, finance, retail, logistics, manufacturing, energy, telecommunications, AI, and other industries.
Apache Doris graduated from the Apache Incubator and became an Apache Top-Level Project in June 2022. Thanks to all community contributors who help build Doris.
Note Some licenses of the third-party dependencies are not compatible with Apache 2.0 License. So you need to disable some Doris features to comply with Apache 2.0 License. For details, refer to the
thirdparty/LICENSE.txt