Add PGBouncer recommendation in "setup-database' doc. (#18399)
We were recommending using PGBouncer for all Postgres installation
for quite some time at least verbally but also in the Helm Chart
documentation. However we missed such recommendation in the
general Postgres area of 'Setting Up the database` doc.
This PR adds a note that we can refer to when explaining
problems with connections and stability to the users who
use Postgres without PGBouncer proxy (which is known to help
in such cases)
(cherry picked from commit e8667b6ac1d5409acddf1ba2a382ed30ee12f53d)
diff --git a/docs/apache-airflow/howto/set-up-database.rst b/docs/apache-airflow/howto/set-up-database.rst
index 6ed2982..216eeb3 100644
--- a/docs/apache-airflow/howto/set-up-database.rst
+++ b/docs/apache-airflow/howto/set-up-database.rst
@@ -218,6 +218,19 @@
For more information regarding setup of the PostgresSQL connection, see `PostgreSQL dialect <https://docs.sqlalchemy.org/en/13/dialects/postgresql.html>`__ in SQLAlchemy documentation.
+.. note::
+
+ Airflow is known - especially in high-performance setup - to open many connections to metadata database. This might cause problems for
+ Postgres resource usage, because in Postgres, each connection creates a new process and it makes Postgres resource-hungry when a lot
+ of connections are opened. Therefore we recommend to use `PGBouncer <https://www.pgbouncer.org/>`_ as database proxy for all Postgres
+ production installations. PGBouncer can handle connection pooling from multiple components, but also in case you have remote
+ database with potentially unstable connectivity, it will make your DB connectivity much more resilient to temporary network problems.
+ Example implementation of PGBouncer deployment can be found in the :doc:`helm-chart:index` where you can enable pre-configured
+ PGBouncer instance with flipping a boolean flag. You can take a look at the approach we have taken there and use it as
+ an inspiration, when you prepare your own Deployment, even if you do not use the Official Helm Chart.
+
+ See also :ref:`Helm Chart production guide <production-guide:pgbouncer>`
+
.. spelling::
hba
diff --git a/docs/helm-chart/production-guide.rst b/docs/helm-chart/production-guide.rst
index bd61808..0c2a2b5 100644
--- a/docs/helm-chart/production-guide.rst
+++ b/docs/helm-chart/production-guide.rst
@@ -43,6 +43,8 @@
port: ...
db: ...
+.. _production-guide:pgbouncer:
+
PgBouncer
---------
