[fix](types) Fix binary value ownership and timestamp primitives (#68301)

### What problem does this PR solve?

This ports #68297 to `master`, preserving the first of five planned
extractions from #67784.

Binary `Field` values can retain references to released source storage,
and Hive binary text needs its own Base64 contract. TIMESTAMPTZ output
can lose historical offset seconds, format invalid NULL payloads, or
fail again while reporting a boundary cast error.

- Own long binary Field values while keeping short values inline.
Preserve execution type lengths and decoder bytes, and add Hive Base64
and hexadecimal decoding support.
- Explicitly reject unsupported binary hash keys, IN, aggregates,
predicates and computed partition transforms. Keep the existing FE
comparison/group/join restrictions and existing binary scalar functions.
Reject unsupported collection kernels in each function's legality check
before coercion.
- Preserve historical second offsets in both TIMESTAMPTZ formatting and
parsing. Skip masked NULL payloads, reject unrepresentable local years,
and preserve cast error/NULL behavior at boundaries.

Arrow convertor migration, Parquet/ORC semantics, external writer
changes and catalog mapping migration belong to the subsequent
extractions. This PR does not enable native VARBINARY storage.

### Master adaptation

- Retain the fixed-offset normalization and tests already present on
master.
- Use the current void-returning
`VInPredicate::_prepare_zonemap_min_max` interface in both the guard and
its test.
- Retain master header cleanup and existing timestamp-nanosecond tests.
- Retain the existing master binary-literal encoder and its StringView
input contract; the older std::string-based caller fix is not
applicable.

### Testing
- TIMESTAMPTZ regression follow-up: reproduced both binary-output and
stream-load failures using the master PR CI artifact, regenerated the
two snapshots through `run-regression-test.sh`, and passed both suites
in comparison mode from each branch checkout. Explicit `Asia/Shanghai`
session settings were verified with the server default session zone set
to UTC. Only historical offset seconds changed in the generated results.
- Function-local validation update: 18 FE tests passed with Checkstyle
enabled, covering direct legality checks, nested/mixed/variadic
VARBINARY arguments, both `collect_set` arities, supported ordinary
types, SQL analysis, and existing array rewrites. The new
direct-legality tests reproduced missing rejection before the change.

- BE ASAN build and **199 tests passed** across 17 suites using
`run-be-ut.sh`, including binary lifetime/SerDe/rejection, timestamp
parsing/casts, and existing Arrow/Variant serialization coverage.
- `VarBinaryUnsupportedCollectionTest`: **passed** (13 unsupported
expressions plus supported byte-preserving collection analysis). The FE
test reactor and repository Checkstyle passed after cleaning stale
branch build artifacts.
- Repository clang-format 16 check and build-header hygiene checks:
**passed**; 31 changed C++ source/header files.
- Groovy compilation of the three regression suites: **passed**. Live
SQL regression execution remains pending CI.
- clang-tidy was attempted but could not complete because master already
contains an unmatched `NOLINTEND` in `be/src/core/types.h`. A diagnostic
run with the compiler resource directory corrected reproduced that
blocker; the other reported findings in `column_varbinary.cpp` were
outside changed lines. This is not a clean clang-tidy result.

The focused BE test source list and local test/build settings were
restored before committing. No build configuration changes are included.

### Release note

Fix binary value lifetime and serialization, reject unsupported binary
computation paths, and preserve TIMESTAMPTZ historical offsets and
boundary error behavior.

### Check List (For Author)

- Test
- [x] Regression test (three self-checking suites added; execution
pending CI)
  - [x] Unit Test
- Behavior changed:
- [x] Yes. Binary rejection and timestamp boundary behavior are
described above.
- Does this need documentation?
- [x] No. This fixes existing type behavior without introducing a
configuration option.

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label


### Scoped review follow-up

This follow-up only fixes correctness/stability defects introduced by
this PR. Compatibility preservation, pre-existing limitations,
additional VARBINARY computation/validation, performance refactors, and
test-style-only rewrites are excluded.

- Separate historical TIMESTAMPTZ wire-offset parsing from session
fixed-zone limits in both parser paths.
- Use UTC diagnostics for the TIMESTAMP_NS cast/comparison failures
affected by the new local-year formatting exception.
- Validation: 31 focused ASAN BE tests passed, including ordinary
DATE/DATETIME parsing. Three targeted tests failed before the fixes.
clang-format 16 and build hygiene passed. Full clang-tidy remains
affected by pre-existing diagnostics.
- Branch-specific UTC/GMT normalization and FE folding fixes are handled
in #68297; the corresponding master behavior predates this PR or already
defers folding.

### CI test follow-up

- Keep the binary literal test's owning Field alive while reading its
StringView. Branch-4.1 now has the corresponding short/long embedded-NUL
coverage using its execution API.
- Replace the obsolete +15:00 rejection input with +24:00. Add generated
historical-offset checks in both cast modes; all prior snapshot results
are unchanged.
- Validation: 34 focused ASAN BE tests passed on each branch. The
lifetime error and the original SQL mismatch were reproduced. The
complete cast regression suite passed in comparison mode from both
branch checkouts against the reported master CI artifact. clang-format
16 passed; full clang-tidy remains blocked by pre-existing diagnostics.

This follow-up changes tests only and retains the agreed scope: no
compatibility work or additional binary computation support. Existing
muted failures are outside this fix.
58 files changed
tree: 02ce1f1c6e0b4799927f916675847c906d848845
  1. .claude/
  2. .github/
  3. .idea/
  4. be/
  5. bin/
  6. build-support/
  7. cloud/
  8. common/
  9. conf/
  10. contrib/
  11. dist/
  12. docker/
  13. docs/
  14. extension/
  15. fe/
  16. fe_plugins/
  17. fs_brokers/
  18. gensrc/
  19. hooks/
  20. pytest/
  21. regression-test/
  22. samples/
  23. task_executor_simulator/
  24. thirdparty/
  25. tools/
  26. ui/
  27. webroot/
  28. .asf.yaml
  29. .clang-format
  30. .clang-format-ignore
  31. .clang-tidy
  32. .clangd
  33. .dockerignore
  34. .editorconfig
  35. .gitattributes
  36. .gitignore
  37. .gitleaks.toml
  38. .gitmodules
  39. .licenserc.yaml
  40. .rat-excludes
  41. .shellcheckrc
  42. AGENTS.md
  43. build-for-release.sh
  44. build-plugin.sh
  45. build.sh
  46. build_profile.sh
  47. CODE_OF_CONDUCT.md
  48. CONTRIBUTING.md
  49. CONTRIBUTING_CN.md
  50. doap_Doris.rdf
  51. env.sh
  52. generated-source.sh
  53. LICENSE.txt
  54. NOTICE.txt
  55. post-build.sh
  56. README.md
  57. reset_submodule.sh
  58. run-be-ut.sh
  59. run-cloud-ut.sh
  60. run-fe-ut.sh
  61. run-fs-env-test.sh
  62. run-regression-test.sh
  63. SECURITY.md
  64. sonar-project.properties
  65. threat-model.md
README.md

🌍 Read this in other language

EnglishالعربيةবাংলাDeutschEspañolفارسیFrançaisहिन्दीBahasa IndonesiaItaliano日本語한국어PolskiPortuguêsRomânăРусскийSlovenščinaไทยTürkçeУкраїнськаTiếng Việt简体中文繁體中文

Apache Doris

License GitHub release Slack EN doc CN doc

Official Website Quick Download


Apache Doris is an open-source, real-time analytics and search database built on MPP architecture. It provides fast SQL analytics, lakehouse query acceleration, and hybrid search across structured, text, and vector data.

Explore the official website for the latest product overview, use cases, ecosystem updates, blogs, and user stories. For version updates, see all release notes.

📈 Use Cases

Use CaseWhat it provides
Customer-Facing AnalyticsShip sub-second interactive analytics to external users.
Data WarehousingBuild one real-time warehouse across business domains.
ObservabilityAnalyze high-throughput logs, events, and metrics with SQL.
Doris for AIUse vector, text, JSON, and structured search in one SQL engine.

🚀 Core Capabilities

Apache Doris is built around three core capabilities. The website is the source of truth for detailed product descriptions and examples.

CapabilityWhat it provides
Real-Time AnalyticsStreaming ingestion, incremental transformation, and sub-second queries under high concurrency.
Lakehouse AnalyticsFast SQL analytics over open table formats such as Iceberg, Delta Lake, and Hudi.
Hybrid SearchSQL-native analytics across JSON, full-text, and vector data for AI and search workloads.

🔌 Ecosystem

Doris sits at the center of the modern data stack. It connects upstream databases, streaming systems, and lakehouse storage with downstream BI, AI, analytics, and observability tools.

For the latest ecosystem coverage, visit the official website and the connection and integration documentation.

👣 Get Started

🧱 Architecture

Apache Doris supports both compute-storage coupled and compute-storage decoupled deployments. In decoupled mode, stateless compute groups run over shared object storage, so you can scale compute on demand and isolate workloads.

Learn more in the deployment guide and deployment mode guide.

📣 Project Updates

ResourceWhat it provides
Community ReportWeekly updates on community activity, merged PRs, contributors, and feature progress.
Roadmap 2026The 2026 planning discussion for AI and hybrid search, query engine, storage, and data lake work.

🧩 Components

Doris provides connectors and tools for common data engineering workflows.

👨‍👩‍👧‍👦 Users

Apache Doris is used in production by thousands of companies worldwide across internet services, finance, retail, logistics, manufacturing, energy, telecommunications, AI, and other industries.

🙌 Contributors

Apache Doris graduated from the Apache Incubator and became an Apache Top-Level Project in June 2022. Thanks to all community contributors who help build Doris.

contrib graph

🌈 Community and Support

💬 Contact Us

🧰 Links

📜 License

Apache License, Version 2.0

Note Some licenses of the third-party dependencies are not compatible with Apache 2.0 License. So you need to disable some Doris features to comply with Apache 2.0 License. For details, refer to the thirdparty/LICENSE.txt