Fix OOB read in cdbcomponent_getComponentInfo() with partially registered mirrors

cdbcomponent_getComponentInfo() assumed that segment_db_info holds either
exactly one entry per content (no mirrors) or exactly two (fully mirrored)
and indexed the array with 2 * contentId in the latter case.  That
assumption breaks while mirrors are being registered one by one:
gpinitsystem's REGISTER_MIRRORS commits one gp_add_segment_mirror() per
content, so for several seconds gp_segment_configuration holds N
primaries plus k < N mirrors.

Any dispatching process that reads the catalog in that window computes an
index past the end of the palloc'd array.  The dtx recovery bgworker does
exactly that: every loop iteration it dispatches to all segments and then
destroys its component cache via DisconnectAndDestroyAllGangs(), so the
next iteration re-reads the half-populated catalog and crashes with
SIGSEGV.  The crash makes the postmaster set CleanupBackends, the restarted
dtx recovery immediately runs TerminateMppBackends(), crashes again, and
the coordinator loops forever producing core files, while gpinitsystem has
already failed with "failed to register mirror".  Larger clusters widen
the race window, which is why this surfaced on a 94+ segment cluster.

Replace the index arithmetic with a lower-bound binary search on segindex
over the sorted array, then pick the entry whose role matches what the QD
wants (active primary for a regular QD, mirror for a hot standby QD),
falling back to the last entry of the content as before.  Contents that do
not have a mirror yet are simply treated as single-copy.  All other
consumers of segment_db_info already iterate and filter by role, and FTS
already skips primaries without a peer, so this was the only place that
relied on the 2N layout.  Both the internal-FTS and external-FTS copies of
the function now share the new helper.

Verified on a demo cluster by deleting one mirror row in utility mode:
unpatched 2.1.0 reproduces the dtx recovery crash loop, patched main maps
every content to its own primary and "DTM Started" normally.

Fixes #2101
1 file changed
tree: 464b7f9bab927e5982da22213726d6dd63c535d3
  1. .abi-check/
  2. .github/
  3. config/
  4. contrib/
  5. devops/
  6. doc/
  7. gpAux/
  8. gpcontrib/
  9. gpMgmt/
  10. licenses/
  11. mcp-server/
  12. src/
  13. .asf.yaml
  14. .clang-tidy
  15. .dir-locals.el
  16. .editorconfig
  17. .git-blame-ignore-revs
  18. .gitattributes
  19. .gitignore
  20. .gitmessage
  21. .gitmodules
  22. aclocal.m4
  23. AGENTS.md.template
  24. AI_GUIDELINE.md
  25. CODE_OF_CONDUCT.md
  26. configure
  27. configure.ac
  28. CONTRIBUTING.md
  29. DISCLAIMER
  30. getversion
  31. GNUmakefile.in
  32. LICENSE
  33. Makefile
  34. meson.build
  35. meson_options.txt
  36. NOTICE
  37. pom.xml
  38. putversion
  39. python-dependencies.txt
  40. README.apache.md
  41. README.md
  42. SECURITY.md
  43. sonar-project.properties
README.md

Apache Cloudberry (Incubating)

Website Documentation Slack Twitter Follow WeChat Youtube GitHub Discussions GitHub commit activity(branch) GitHub contributors GitHub License Apache Cloudberry Build Ask DeepWiki Apache Rat Audit

Introduction

Apache Cloudberry (Incubating), created by the original developers of Greenplum® Database, is one advanced and mature open-source Massively Parallel Processing (MPP) database, which evolves from the open-source Greenplum® Database but features a newer PostgreSQL kernel and more advanced enterprise capabilities. It can serve as a data warehouse and can also be used for large-scale analytics and AI/ML workloads.

Build and try out

Build from source

You can follow these guides to build Cloudberry on Linux OS (including RHEL/Rocky Linux, and Ubuntu) and macOS.

Try out quickly

Welcome to try out Cloudberry via building one Docker-based Sandbox, which is tailored to help you gain a basic understanding of Cloudberry's capabilities and features.

Repositories

This is the main repository for Apache Cloudberry (Incubating). Alongside this, there are several ecosystem repositories for Cloudberry, including the website, extensions, connectors, adapters, and other utilities.

Community & Support

We have many channels for community members to discuss, ask for help, feedback, and chat:

TypeDescription
SlackClick to Join the real-time chat on Slack for QA, Dev, Events, and more. Don't miss out! Check out the Slack guide to learn more.
Q&AAsk for help when running/developing Cloudberry, visit GitHub Discussions - QA.
New ideas / Feature RequestsShare ideas for new features, visit GitHub Discussions - Ideas.
Report bugsProblems and issues in Apache Cloudberry core. If you find bugs, welcome to submit them here.
Report a security vulnerabilityView our security policy to learn how to report and contact us.
Community eventsIncluding meetups, webinars, conferences, and more events, visit the Events page and subscribe to the events calendar.
DocumentationOfficial documentation for Cloudberry. You can explore it to discover more details about us.

Contribution

Contributions can be diverse, such as code enhancements, bug fixes, feature proposals, documents, marketing, and so on. No contribution is too small, we encourage all types of contributions. Cloudberry community welcomes contributions from anyone, new and experienced! Our contribution guide will help you get started with the contribution.

TypeDescription
Code contributionLearn how to contribute code to the Cloudberry, including coding preparation, conventions, workflow, review, and checklist following the code contribution guide.
Submit the proposalProposing major changes to Cloudberry through proposal guide.
Doc contributionWe need you to join us to help us improve the documentation, see the doc contribution guide.
AI guidlineFor AI-assisted development, please review our AI guideline for advice on responsible AI usage.

Roadmap

You can check our Cloudberry Roadmap out to see the product plans and goals we want to achieve. Welcome to share your thoughts and ideas to join us in shaping the future of Apache Cloudberry (Incubating). (We will update the Roadmap after entering the Incubator.)

Acknowledgment

Thanks to PostgreSQL, Greenplum Database and other great open source projects to make Apache Cloudberry has a sound foundation.

License

Cloudberry is licensed under the Apache License, Version 2.0. For details, see the LICENSE.

Crypto Export Notice

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org for more information.

The Apache Software Foundation has classified this software as Export Commodity Control Number (ECCN) 5D002, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the “publicly available” Section 742.15(b) exemption (see the BIS Export Administration Regulations, Section 742.15(b)) for both object code and source code.

The following provides more details on the included cryptographic software:

  • Apache Cloudberry uses OpenSSL for TLS support on client and server connections, and for the SCRAM-SHA-256 authentication method.
  • Apache Cloudberry optionally uses GSSAPI (Kerberos) for authentication and connection encryption, and LDAP over TLS for authentication.
  • The bundled pgcrypto extension provides cryptographic functions to SQL, using either OpenSSL or its own built-in implementations.
  • The gpfdist external table file server can serve over HTTPS, using OpenSSL and X.509 certificates supplied through its --ssl <certificates_directory> option.

Apache Cloudberry does not ship OpenSSL itself; it links against the OpenSSL provided by the operating system.

ASF Incubator disclaimer

Apache Cloudberry is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required for all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.