Re-apply DROP DATABASE getting interrupted fix and adapt for MPP

In CBDB, DROP database is hazardous in few kill-9 scenarios. Most of them are already fixed in PostgreSQL, so bring this fix to our fork. Additional change here required to move DROP DATABASE on QE dispatch AFTER marking database as invalid on QD, otherwise we can get in case where QD did not mark database as invalid, and allows connections, while QE already removed datadirs. FAULT-INJECTION test hits this (without code fix)

This is re-apply of https://git.postgresql.org/cgit/postgresql.git/commit/?id=c66a7d75e652801043ece99b6a8f89fd9513eaaa with additional cbdb fixes & tests

Until now, when DROP DATABASE got interrupted in the wrong moment, the removal
of the pg_database row would also roll back, even though some irreversible
steps have already been taken. E.g. DropDatabaseBuffers() might have thrown
out dirty buffers, or files could have been unlinked. But we continued to
allow connections to such a corrupted database.

To fix this, mark databases invalid with an in-place update, just before
starting to perform irreversible steps. As we can't add a new column in the
back branches, we use pg_database.datconnlimit = -2 for this purpose.

An invalid database cannot be connected to anymore, but can still be
dropped.

Unfortunately we can't easily add output to psql's \l to indicate that some
database is invalid, it doesn't fit in any of the existing columns.

Add tests verifying that a interrupted DROP DATABASE is handled correctly in
the backend and in various tools.

In cloudberry, we also move segment DDL dispach after pg_database tuple
transaction commit, otherwise, if killed, segments will already drop
thier datadirs, while QD not.

Reported-by: Evgeny Morozov <postgresql3@realityexists.net>
Author: Andres Freund <andres@anarazel.de>
Co-authored-by: reshke <reshke@double.cloud> (Cloudberry part)
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20230509004637.cgvmfwrbht7xm7p6@awork3.anarazel.de
Discussion: https://postgr.es/m/20230314174521.74jl6ffqsee5mtug@awork3.anarazel.de
Backpatch: 11-, bug present in all supported versions
8 files changed
tree: 0ab433c2b21a69f4c03c7fadc8e12913d9a0fb20
  1. .abi-check/
  2. .github/
  3. config/
  4. contrib/
  5. dependency/
  6. devops/
  7. doc/
  8. gpAux/
  9. gpcontrib/
  10. gpMgmt/
  11. licenses/
  12. mcp-server/
  13. src/
  14. .asf.yaml
  15. .clang-tidy
  16. .dir-locals.el
  17. .editorconfig
  18. .git-blame-ignore-revs
  19. .gitattributes
  20. .gitignore
  21. .gitmessage
  22. .gitmodules
  23. aclocal.m4
  24. AGENTS.md.template
  25. AI_GUIDELINE.md
  26. CODE_OF_CONDUCT.md
  27. configure
  28. configure.ac
  29. CONTRIBUTING.md
  30. DISCLAIMER
  31. getversion
  32. GNUmakefile.in
  33. LICENSE
  34. Makefile
  35. meson.build
  36. meson_options.txt
  37. NOTICE
  38. pom.xml
  39. putversion
  40. python-dependencies.txt
  41. README.apache.md
  42. README.md
  43. SECURITY.md
  44. sonar-project.properties
README.md

Apache Cloudberry (Incubating)

Website Documentation Slack Twitter Follow WeChat Youtube GitHub Discussions GitHub commit activity(branch) GitHub contributors GitHub License Apache Cloudberry Build Ask DeepWiki Apache Rat Audit

Introduction

Apache Cloudberry (Incubating), created by the original developers of Greenplum Database, is one advanced and mature open-source Massively Parallel Processing (MPP) database, which evolves from the open-source version of the Pivotal Greenplum Database®️ but features a newer PostgreSQL kernel and more advanced enterprise capabilities. It can serve as a data warehouse and can also be used for large-scale analytics and AI/ML workloads.

Build and try out

Build from source

You can follow these guides to build Cloudberry on Linux OS (including RHEL/Rocky Linux, and Ubuntu) and macOS.

Try out quickly

Welcome to try out Cloudberry via building one Docker-based Sandbox, which is tailored to help you gain a basic understanding of Cloudberry's capabilities and features.

Repositories

This is the main repository for Apache Cloudberry (Incubating). Alongside this, there are several ecosystem repositories for Cloudberry, including the website, extensions, connectors, adapters, and other utilities.

Community & Support

We have many channels for community members to discuss, ask for help, feedback, and chat:

TypeDescription
SlackClick to Join the real-time chat on Slack for QA, Dev, Events, and more. Don't miss out! Check out the Slack guide to learn more.
Q&AAsk for help when running/developing Cloudberry, visit GitHub Discussions - QA.
New ideas / Feature RequestsShare ideas for new features, visit GitHub Discussions - Ideas.
Report bugsProblems and issues in Apache Cloudberry core. If you find bugs, welcome to submit them here.
Report a security vulnerabilityView our security policy to learn how to report and contact us.
Community eventsIncluding meetups, webinars, conferences, and more events, visit the Events page and subscribe to the events calendar.
DocumentationOfficial documentation for Cloudberry. You can explore it to discover more details about us.

Contribution

Contributions can be diverse, such as code enhancements, bug fixes, feature proposals, documents, marketing, and so on. No contribution is too small, we encourage all types of contributions. Cloudberry community welcomes contributions from anyone, new and experienced! Our contribution guide will help you get started with the contribution.

TypeDescription
Code contributionLearn how to contribute code to the Cloudberry, including coding preparation, conventions, workflow, review, and checklist following the code contribution guide.
Submit the proposalProposing major changes to Cloudberry through proposal guide.
Doc contributionWe need you to join us to help us improve the documentation, see the doc contribution guide.
AI guidlineFor AI-assisted development, please review our AI guideline for advice on responsible AI usage.

Roadmap

You can check our Cloudberry Roadmap out to see the product plans and goals we want to achieve. Welcome to share your thoughts and ideas to join us in shaping the future of Apache Cloudberry (Incubating). (We will update the Roadmap after entering the Incubator.)

Acknowledgment

Thanks to PostgreSQL, Greenplum Database and other great open source projects to make Apache Cloudberry has a sound foundation.

License

Cloudberry is licensed under the Apache License, Version 2.0. For details, see the LICENSE.

ASF Incubator disclaimer

Apache Cloudberry is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required for all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.