Google Git
Sign in
apache/cloudberry/refs/heads/reshke-patch-1/./doc/src/sgml/protocol.sgml
blob: 4024fe5b84eff6b36d8257eb91a7c0f2b37d4190 [file]
<!-- doc/src/sgml/protocol.sgml -->
<chapter id="protocol">
<title>Frontend/Backend Protocol</title>
<indexterm zone="protocol">
<primary>protocol</primary>
<secondary>frontend-backend</secondary>
</indexterm>
<para>
<productname>PostgreSQL</productname> uses a message-based protocol
for communication between frontends and backends (clients and servers).
The protocol is supported over <acronym>TCP/IP</acronym> and also over
Unix-domain sockets. Port number 5432 has been registered with IANA as
the customary TCP port number for servers supporting this protocol, but
in practice any non-privileged port number can be used.
</para>
<para>
This document describes version 3.0 of the protocol, implemented in
<productname>PostgreSQL</productname> 7.4 and later. For descriptions
of the earlier protocol versions, see previous releases of the
<productname>PostgreSQL</productname> documentation. A single server
can support multiple protocol versions. The initial startup-request
message tells the server which protocol version the client is attempting to
use. If the major version requested by the client is not supported by
the server, the connection will be rejected (for example, this would occur
if the client requested protocol version 4.0, which does not exist as of
this writing). If the minor version requested by the client is not
supported by the server (e.g., the client requests version 3.1, but the
server supports only 3.0), the server may either reject the connection or
may respond with a NegotiateProtocolVersion message containing the highest
minor protocol version which it supports. The client may then choose either
to continue with the connection using the specified protocol version or
to abort the connection.
</para>
<para>
In order to serve multiple clients efficiently, the server launches
a new <quote>backend</quote> process for each client.
In the current implementation, a new child
process is created immediately after an incoming connection is detected.
This is transparent to the protocol, however. For purposes of the
protocol, the terms <quote>backend</quote> and <quote>server</quote> are
interchangeable; likewise <quote>frontend</quote> and <quote>client</quote>
are interchangeable.
</para>
<sect1 id="protocol-overview">
<title>Overview</title>
<para>
The protocol has separate phases for startup and normal operation.
In the startup phase, the frontend opens a connection to the server
and authenticates itself to the satisfaction of the server. (This might
involve a single message, or multiple messages depending on the
authentication method being used.) If all goes well, the server then sends
status information to the frontend, and finally enters normal operation.
Except for the initial startup-request message, this part of the
protocol is driven by the server.
</para>
<para>
During normal operation, the frontend sends queries and
other commands to the backend, and the backend sends back query results
and other responses. There are a few cases (such as <command>NOTIFY</command>)
wherein the
backend will send unsolicited messages, but for the most part this portion
of a session is driven by frontend requests.
</para>
<para>
Termination of the session is normally by frontend choice, but can be
forced by the backend in certain cases. In any case, when the backend
closes the connection, it will roll back any open (incomplete) transaction
before exiting.
</para>
<para>
Within normal operation, SQL commands can be executed through either of
two sub-protocols. In the <quote>simple query</quote> protocol, the frontend
just sends a textual query string, which is parsed and immediately
executed by the backend. In the <quote>extended query</quote> protocol,
processing of queries is separated into multiple steps: parsing,
binding of parameter values, and execution. This offers flexibility
and performance benefits, at the cost of extra complexity.
</para>
<para>
Normal operation has additional sub-protocols for special operations
such as <command>COPY</command>.
</para>
<sect2 id="protocol-message-concepts">
<title>Messaging Overview</title>
<para>
All communication is through a stream of messages. The first byte of a
message identifies the message type, and the next four bytes specify the
length of the rest of the message (this length count includes itself, but
not the message-type byte). The remaining contents of the message are
determined by the message type. For historical reasons, the very first
message sent by the client (the startup message) has no initial
message-type byte.
</para>
<para>
To avoid losing synchronization with the message stream, both servers and
clients typically read an entire message into a buffer (using the byte
count) before attempting to process its contents. This allows easy
recovery if an error is detected while processing the contents. In
extreme situations (such as not having enough memory to buffer the
message), the receiver can use the byte count to determine how much
input to skip before it resumes reading messages.
</para>
<para>
Conversely, both servers and clients must take care never to send an
incomplete message. This is commonly done by marshaling the entire message
in a buffer before beginning to send it. If a communications failure
occurs partway through sending or receiving a message, the only sensible
response is to abandon the connection, since there is little hope of
recovering message-boundary synchronization.
</para>
</sect2>
<sect2 id="protocol-query-concepts">
<title>Extended Query Overview</title>
<para>
In the extended-query protocol, execution of SQL commands is divided
into multiple steps. The state retained between steps is represented
by two types of objects: <firstterm>prepared statements</firstterm> and
<firstterm>portals</firstterm>. A prepared statement represents the result of
parsing and semantic analysis of a textual query string.
A prepared statement is not in itself ready to execute, because it might
lack specific values for <firstterm>parameters</firstterm>. A portal represents
a ready-to-execute or already-partially-executed statement, with any
missing parameter values filled in. (For <command>SELECT</command> statements,
a portal is equivalent to an open cursor, but we choose to use a different
term since cursors don't handle non-<command>SELECT</command> statements.)
</para>
<para>
The overall execution cycle consists of a <firstterm>parse</firstterm> step,
which creates a prepared statement from a textual query string; a
<firstterm>bind</firstterm> step, which creates a portal given a prepared
statement and values for any needed parameters; and an
<firstterm>execute</firstterm> step that runs a portal's query. In the case of
a query that returns rows (<command>SELECT</command>, <command>SHOW</command>, etc.),
the execute step can be told to fetch only
a limited number of rows, so that multiple execute steps might be needed
to complete the operation.
</para>
<para>
The backend can keep track of multiple prepared statements and portals
(but note that these exist only within a session, and are never shared
across sessions). Existing prepared statements and portals are
referenced by names assigned when they were created. In addition,
an <quote>unnamed</quote> prepared statement and portal exist. Although these
behave largely the same as named objects, operations on them are optimized
for the case of executing a query only once and then discarding it,
whereas operations on named objects are optimized on the expectation
of multiple uses.
</para>
</sect2>
<sect2 id="protocol-format-codes">
<title>Formats and Format Codes</title>
<para>
Data of a particular data type might be transmitted in any of several
different <firstterm>formats</firstterm>. As of <productname>PostgreSQL</productname> 7.4
the only supported formats are <quote>text</quote> and <quote>binary</quote>,
but the protocol makes provision for future extensions. The desired
format for any value is specified by a <firstterm>format code</firstterm>.
Clients can specify a format code for each transmitted parameter value
and for each column of a query result. Text has format code zero,
binary has format code one, and all other format codes are reserved
for future definition.
</para>
<para>
The text representation of values is whatever strings are produced
and accepted by the input/output conversion functions for the
particular data type. In the transmitted representation, there is
no trailing null character; the frontend must add one to received
values if it wants to process them as C strings.
(The text format does not allow embedded nulls, by the way.)
</para>
<para>
Binary representations for integers use network byte order (most
significant byte first). For other data types consult the documentation
or source code to learn about the binary representation. Keep in mind
that binary representations for complex data types might change across
server versions; the text format is usually the more portable choice.
</para>
</sect2>
</sect1>
<sect1 id="protocol-flow">
<title>Message Flow</title>
<para>
This section describes the message flow and the semantics of each
message type. (Details of the exact representation of each message
appear in <xref linkend="protocol-message-formats"/>.) There are
several different sub-protocols depending on the state of the
connection: start-up, query, function call,
<command>COPY</command>, and termination. There are also special
provisions for asynchronous operations (including notification
responses and command cancellation), which can occur at any time
after the start-up phase.
</para>
<sect2 id="protocol-flow-start-up">
<title>Start-up</title>
<para>
To begin a session, a frontend opens a connection to the server and sends
a startup message. This message includes the names of the user and of the
database the user wants to connect to; it also identifies the particular
protocol version to be used. (Optionally, the startup message can include
additional settings for run-time parameters.)
The server then uses this information and
the contents of its configuration files (such as
<filename>pg_hba.conf</filename>) to determine
whether the connection is provisionally acceptable, and what additional
authentication is required (if any).
</para>
<para>
The server then sends an appropriate authentication request message,
to which the frontend must reply with an appropriate authentication
response message (such as a password).
For all authentication methods except GSSAPI, SSPI and SASL, there is at
most one request and one response. In some methods, no response
at all is needed from the frontend, and so no authentication request
occurs. For GSSAPI, SSPI and SASL, multiple exchanges of packets may be
needed to complete the authentication.
</para>
<para>
The authentication cycle ends with the server either rejecting the
connection attempt (ErrorResponse), or sending AuthenticationOk.
</para>
<para>
The possible messages from the server in this phase are:
<variablelist>
<varlistentry>
<term>ErrorResponse</term>
<listitem>
<para>
The connection attempt has been rejected.
The server then immediately closes the connection.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationOk</term>
<listitem>
<para>
The authentication exchange is successfully completed.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationKerberosV5</term>
<listitem>
<para>
The frontend must now take part in a Kerberos V5
authentication dialog (not described here, part of the
Kerberos specification) with the server. If this is
successful, the server responds with an AuthenticationOk,
otherwise it responds with an ErrorResponse. This is no
longer supported.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationCleartextPassword</term>
<listitem>
<para>
The frontend must now send a PasswordMessage containing the
password in clear-text form. If
this is the correct password, the server responds with an
AuthenticationOk, otherwise it responds with an ErrorResponse.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationMD5Password</term>
<listitem>
<para>
The frontend must now send a PasswordMessage containing the
password (with user name) encrypted via MD5, then encrypted
again using the 4-byte random salt specified in the
AuthenticationMD5Password message. If this is the correct
password, the server responds with an AuthenticationOk,
otherwise it responds with an ErrorResponse. The actual
PasswordMessage can be computed in SQL as <literal>concat('md5',
md5(concat(md5(concat(password, username)), random-salt)))</literal>.
(Keep in mind the <function>md5()</function> function returns its
result as a hex string.)
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationGSS</term>
<listitem>
<para>
The frontend must now initiate a GSSAPI negotiation. The frontend
will send a GSSResponse message with the first part of the GSSAPI
data stream in response to this. If further messages are needed,
the server will respond with AuthenticationGSSContinue.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationSSPI</term>
<listitem>
<para>
The frontend must now initiate an SSPI negotiation. The frontend
will send a GSSResponse with the first part of the SSPI
data stream in response to this. If further messages are needed,
the server will respond with AuthenticationGSSContinue.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationGSSContinue</term>
<listitem>
<para>
This message contains the response data from the previous step
of GSSAPI or SSPI negotiation (AuthenticationGSS, AuthenticationSSPI
or a previous AuthenticationGSSContinue). If the GSSAPI
or SSPI data in this message
indicates more data is needed to complete the authentication,
the frontend must send that data as another GSSResponse message. If
GSSAPI or SSPI authentication is completed by this message, the server
will next send AuthenticationOk to indicate successful authentication
or ErrorResponse to indicate failure.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationSASL</term>
<listitem>
<para>
The frontend must now initiate a SASL negotiation, using one of the
SASL mechanisms listed in the message. The frontend will send a
SASLInitialResponse with the name of the selected mechanism, and the
first part of the SASL data stream in response to this. If further
messages are needed, the server will respond with
AuthenticationSASLContinue. See <xref linkend="sasl-authentication"/>
for details.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationSASLContinue</term>
<listitem>
<para>
This message contains challenge data from the previous step of SASL
negotiation (AuthenticationSASL, or a previous
AuthenticationSASLContinue). The frontend must respond with a
SASLResponse message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>AuthenticationSASLFinal</term>
<listitem>
<para>
SASL authentication has completed with additional mechanism-specific
data for the client. The server will next send AuthenticationOk to
indicate successful authentication, or an ErrorResponse to indicate
failure. This message is sent only if the SASL mechanism specifies
additional data to be sent from server to client at completion.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>NegotiateProtocolVersion</term>
<listitem>
<para>
The server does not support the minor protocol version requested
by the client, but does support an earlier version of the protocol;
this message indicates the highest supported minor version. This
message will also be sent if the client requested unsupported protocol
options (i.e., beginning with <literal>_pq_.</literal>) in the
startup packet. This message will be followed by an ErrorResponse or
a message indicating the success or failure of authentication.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
<para>
If the frontend does not support the authentication method
requested by the server, then it should immediately close the
connection.
</para>
<para>
After having received AuthenticationOk, the frontend must wait
for further messages from the server. In this phase a backend process
is being started, and the frontend is just an interested bystander.
It is still possible for the startup attempt
to fail (ErrorResponse) or the server to decline support for the requested
minor protocol version (NegotiateProtocolVersion), but in the normal case
the backend will send some ParameterStatus messages, BackendKeyData, and
finally ReadyForQuery.
</para>
<para>
During this phase the backend will attempt to apply any additional
run-time parameter settings that were given in the startup message.
If successful, these values become session defaults. An error causes
ErrorResponse and exit.
</para>
<para>
The possible messages from the backend in this phase are:
<variablelist>
<varlistentry>
<term>BackendKeyData</term>
<listitem>
<para>
This message provides secret-key data that the frontend must
save if it wants to be able to issue cancel requests later.
The frontend should not respond to this message, but should
continue listening for a ReadyForQuery message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>ParameterStatus</term>
<listitem>
<para>
This message informs the frontend about the current (initial)
setting of backend parameters, such as <xref
linkend="guc-client-encoding"/> or <xref linkend="guc-datestyle"/>.
The frontend can ignore this message, or record the settings
for its future use; see <xref linkend="protocol-async"/> for
more details. The frontend should not respond to this
message, but should continue listening for a ReadyForQuery
message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>ReadyForQuery</term>
<listitem>
<para>
Start-up is completed. The frontend can now issue commands.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>ErrorResponse</term>
<listitem>
<para>
Start-up failed. The connection is closed after sending this
message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>NoticeResponse</term>
<listitem>
<para>
A warning message has been issued. The frontend should
display the message but continue listening for ReadyForQuery
or ErrorResponse.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
<para>
The ReadyForQuery message is the same one that the backend will
issue after each command cycle. Depending on the coding needs of
the frontend, it is reasonable to consider ReadyForQuery as
starting a command cycle, or to consider ReadyForQuery as ending the
start-up phase and each subsequent command cycle.
</para>
</sect2>
<sect2 id="protocol-flow-simple-query">
<title>Simple Query</title>
<para>
A simple query cycle is initiated by the frontend sending a Query message
to the backend. The message includes an SQL command (or commands)
expressed as a text string.
The backend then sends one or more response
messages depending on the contents of the query command string,
and finally a ReadyForQuery response message. ReadyForQuery
informs the frontend that it can safely send a new command.
(It is not actually necessary for the frontend to wait for
ReadyForQuery before issuing another command, but the frontend must
then take responsibility for figuring out what happens if the earlier
command fails and already-issued later commands succeed.)
</para>
<para>
The possible response messages from the backend are:
<variablelist>
<varlistentry>
<term>CommandComplete</term>
<listitem>
<para>
An SQL command completed normally.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>CopyInResponse</term>
<listitem>
<para>
The backend is ready to copy data from the frontend to a
table; see <xref linkend="protocol-copy"/>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>CopyOutResponse</term>
<listitem>
<para>
The backend is ready to copy data from a table to the
frontend; see <xref linkend="protocol-copy"/>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>RowDescription</term>
<listitem>
<para>
Indicates that rows are about to be returned in response to
a <command>SELECT</command>, <command>FETCH</command>, etc. query.
The contents of this message describe the column layout of the rows.
This will be followed by a DataRow message for each row being returned
to the frontend.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>DataRow</term>
<listitem>
<para>
One of the set of rows returned by
a <command>SELECT</command>, <command>FETCH</command>, etc. query.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>EmptyQueryResponse</term>
<listitem>
<para>
An empty query string was recognized.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>ErrorResponse</term>
<listitem>
<para>
An error has occurred.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>ReadyForQuery</term>
<listitem>
<para>
Processing of the query string is complete. A separate
message is sent to indicate this because the query string might
contain multiple SQL commands. (CommandComplete marks the
end of processing one SQL command, not the whole string.)
ReadyForQuery will always be sent, whether processing
terminates successfully or with an error.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>NoticeResponse</term>
<listitem>
<para>
A warning message has been issued in relation to the query.
Notices are in addition to other responses, i.e., the backend
will continue processing the command.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
<para>
The response to a <command>SELECT</command> query (or other queries that
return row sets, such as <command>EXPLAIN</command> or <command>SHOW</command>)
normally consists of RowDescription, zero or more
DataRow messages, and then CommandComplete.
<command>COPY</command> to or from the frontend invokes special protocol
as described in <xref linkend="protocol-copy"/>.
All other query types normally produce only
a CommandComplete message.
</para>
<para>
Since a query string could contain several queries (separated by
semicolons), there might be several such response sequences before the
backend finishes processing the query string. ReadyForQuery is issued
when the entire string has been processed and the backend is ready to
accept a new query string.
</para>
<para>
If a completely empty (no contents other than whitespace) query string
is received, the response is EmptyQueryResponse followed by ReadyForQuery.
</para>
<para>
In the event of an error, ErrorResponse is issued followed by
ReadyForQuery. All further processing of the query string is aborted by
ErrorResponse (even if more queries remained in it). Note that this
might occur partway through the sequence of messages generated by an
individual query.
</para>
<para>
In simple Query mode, the format of retrieved values is always text,
except when the given command is a <command>FETCH</command> from a cursor
declared with the <literal>BINARY</literal> option. In that case, the
retrieved values are in binary format. The format codes given in
the RowDescription message tell which format is being used.
</para>
<para>
A frontend must be prepared to accept ErrorResponse and
NoticeResponse messages whenever it is expecting any other type of
message. See also <xref linkend="protocol-async"/> concerning messages
that the backend might generate due to outside events.
</para>
<para>
Recommended practice is to code frontends in a state-machine style
that will accept any message type at any time that it could make sense,
rather than wiring in assumptions about the exact sequence of messages.
</para>
<sect3 id="protocol-flow-multi-statement">
<title>Multiple Statements in a Simple Query</title>
<para>
When a simple Query message contains more than one SQL statement
(separated by semicolons), those statements are executed as a single
transaction, unless explicit transaction control commands are included
to force a different behavior. For example, if the message contains
<programlisting>
INSERT INTO mytable VALUES(1);
SELECT 1/0;
INSERT INTO mytable VALUES(2);
</programlisting>
then the divide-by-zero failure in the <command>SELECT</command> will force
rollback of the first <command>INSERT</command>. Furthermore, because
execution of the message is abandoned at the first error, the second
<command>INSERT</command> is never attempted at all.
</para>
<para>
If instead the message contains
<programlisting>
BEGIN;
INSERT INTO mytable VALUES(1);
COMMIT;
INSERT INTO mytable VALUES(2);
SELECT 1/0;
</programlisting>
then the first <command>INSERT</command> is committed by the
explicit <command>COMMIT</command> command. The second <command>INSERT</command>
and the <command>SELECT</command> are still treated as a single transaction,
so that the divide-by-zero failure will roll back the
second <command>INSERT</command>, but not the first one.
</para>
<para>
This behavior is implemented by running the statements in a
multi-statement Query message in an <firstterm>implicit transaction
block</firstterm> unless there is some explicit transaction block for them to
run in. The main difference between an implicit transaction block and
a regular one is that an implicit block is closed automatically at the
end of the Query message, either by an implicit commit if there was no
error, or an implicit rollback if there was an error. This is similar
to the implicit commit or rollback that happens for a statement
executed by itself (when not in a transaction block).
</para>
<para>
If the session is already in a transaction block, as a result of
a <command>BEGIN</command> in some previous message, then the Query message
simply continues that transaction block, whether the message contains
one statement or several. However, if the Query message contains
a <command>COMMIT</command> or <command>ROLLBACK</command> closing the existing
transaction block, then any following statements are executed in an
implicit transaction block.
Conversely, if a <command>BEGIN</command> appears in a multi-statement Query
message, then it starts a regular transaction block that will only be
terminated by an explicit <command>COMMIT</command> or <command>ROLLBACK</command>,
whether that appears in this Query message or a later one.
If the <command>BEGIN</command> follows some statements that were executed as
an implicit transaction block, those statements are not immediately
committed; in effect, they are retroactively included into the new
regular transaction block.
</para>
<para>
A <command>COMMIT</command> or <command>ROLLBACK</command> appearing in an implicit
transaction block is executed as normal, closing the implicit block;
however, a warning will be issued since a <command>COMMIT</command>
or <command>ROLLBACK</command> without a previous <command>BEGIN</command> might
represent a mistake. If more statements follow, a new implicit
transaction block will be started for them.
</para>
<para>
Savepoints are not allowed in an implicit transaction block, since
they would conflict with the behavior of automatically closing the
block upon any error.
</para>
<para>
Remember that, regardless of any transaction control commands that may
be present, execution of the Query message stops at the first error.
Thus for example given
<programlisting>
BEGIN;
SELECT 1/0;
ROLLBACK;
</programlisting>
in a single Query message, the session will be left inside a failed
regular transaction block, since the <command>ROLLBACK</command> is not
reached after the divide-by-zero error. Another <command>ROLLBACK</command>
will be needed to restore the session to a usable state.
</para>
<para>
Another behavior of note is that initial lexical and syntactic
analysis is done on the entire query string before any of it is
executed. Thus simple errors (such as a misspelled keyword) in later
statements can prevent execution of any of the statements. This
is normally invisible to users since the statements would all roll
back anyway when done as an implicit transaction block. However,
it can be visible when attempting to do multiple transactions within a
multi-statement Query. For instance, if a typo turned our previous
example into
<programlisting>
BEGIN;
INSERT INTO mytable VALUES(1);
COMMIT;
INSERT INTO mytable VALUES(2);
SELCT 1/0;<!-- this typo is intentional -->
</programlisting>
then none of the statements would get run, resulting in the visible
difference that the first <command>INSERT</command> is not committed.
Errors detected at semantic analysis or later, such as a misspelled
table or column name, do not have this effect.
</para>
</sect3>
</sect2>
<sect2 id="protocol-flow-ext-query">
<title>Extended Query</title>
<para>
The extended query protocol breaks down the above-described simple
query protocol into multiple steps. The results of preparatory
steps can be re-used multiple times for improved efficiency.
Furthermore, additional features are available, such as the possibility
of supplying data values as separate parameters instead of having to
insert them directly into a query string.
</para>
<para>
In the extended protocol, the frontend first sends a Parse message,
which contains a textual query string, optionally some information
about data types of parameter placeholders, and the
name of a destination prepared-statement object (an empty string
selects the unnamed prepared statement). The response is
either ParseComplete or ErrorResponse. Parameter data types can be
specified by OID; if not given, the parser attempts to infer the
data types in the same way as it would do for untyped literal string
constants.
</para>
<note>
<para>
A parameter data type can be left unspecified by setting it to zero,
or by making the array of parameter type OIDs shorter than the
number of parameter symbols (<literal>$</literal><replaceable>n</replaceable>)
used in the query string. Another special case is that a parameter's
type can be specified as <type>void</type> (that is, the OID of the
<type>void</type> pseudo-type). This is meant to allow parameter symbols
to be used for function parameters that are actually OUT parameters.
Ordinarily there is no context in which a <type>void</type> parameter
could be used, but if such a parameter symbol appears in a function's
parameter list, it is effectively ignored. For example, a function
call such as <literal>foo($1,$2,$3,$4)</literal> could match a function with
two IN and two OUT arguments, if <literal>$3</literal> and <literal>$4</literal>
are specified as having type <type>void</type>.
</para>
</note>
<note>
<para>
The query string contained in a Parse message cannot include more
than one SQL statement; else a syntax error is reported. This
restriction does not exist in the simple-query protocol, but it
does exist in the extended protocol, because allowing prepared
statements or portals to contain multiple commands would complicate
the protocol unduly.
</para>
</note>
<para>
If successfully created, a named prepared-statement object lasts till
the end of the current session, unless explicitly destroyed. An unnamed
prepared statement lasts only until the next Parse statement specifying
the unnamed statement as destination is issued. (Note that a simple
Query message also destroys the unnamed statement.) Named prepared
statements must be explicitly closed before they can be redefined by
another Parse message, but this is not required for the unnamed statement.
Named prepared statements can also be created and accessed at the SQL
command level, using <command>PREPARE</command> and <command>EXECUTE</command>.
</para>
<para>
Once a prepared statement exists, it can be readied for execution using a
Bind message. The Bind message gives the name of the source prepared
statement (empty string denotes the unnamed prepared statement), the name
of the destination portal (empty string denotes the unnamed portal), and
the values to use for any parameter placeholders present in the prepared
statement. The
supplied parameter set must match those needed by the prepared statement.
(If you declared any <type>void</type> parameters in the Parse message,
pass NULL values for them in the Bind message.)
Bind also specifies the format to use for any data returned
by the query; the format can be specified overall, or per-column.
The response is either BindComplete or ErrorResponse.
</para>
<note>
<para>
The choice between text and binary output is determined by the format
codes given in Bind, regardless of the SQL command involved. The
<literal>BINARY</literal> attribute in cursor declarations is irrelevant when
using extended query protocol.
</para>
</note>
<para>
Query planning typically occurs when the Bind message is processed.
If the prepared statement has no parameters, or is executed repeatedly,
the server might save the created plan and re-use it during subsequent
Bind messages for the same prepared statement. However, it will do so
only if it finds that a generic plan can be created that is not much
less efficient than a plan that depends on the specific parameter values
supplied. This happens transparently so far as the protocol is concerned.
</para>
<para>
If successfully created, a named portal object lasts till the end of the
current transaction, unless explicitly destroyed. An unnamed portal is
destroyed at the end of the transaction, or as soon as the next Bind
statement specifying the unnamed portal as destination is issued. (Note
that a simple Query message also destroys the unnamed portal.) Named
portals must be explicitly closed before they can be redefined by another
Bind message, but this is not required for the unnamed portal.
Named portals can also be created and accessed at the SQL
command level, using <command>DECLARE CURSOR</command> and <command>FETCH</command>.
</para>
<para>
Once a portal exists, it can be executed using an Execute message.
The Execute message specifies the portal name (empty string denotes the
unnamed portal) and
a maximum result-row count (zero meaning <quote>fetch all rows</quote>).
The result-row count is only meaningful for portals
containing commands that return row sets; in other cases the command is
always executed to completion, and the row count is ignored.
The possible
responses to Execute are the same as those described above for queries
issued via simple query protocol, except that Execute doesn't cause
ReadyForQuery or RowDescription to be issued.
</para>
<para>
If Execute terminates before completing the execution of a portal
(due to reaching a nonzero result-row count), it will send a
PortalSuspended message; the appearance of this message tells the frontend
that another Execute should be issued against the same portal to
complete the operation. The CommandComplete message indicating
completion of the source SQL command is not sent until
the portal's execution is completed. Therefore, an Execute phase is
always terminated by the appearance of exactly one of these messages:
CommandComplete, EmptyQueryResponse (if the portal was created from
an empty query string), ErrorResponse, or PortalSuspended.
</para>
<para>
At completion of each series of extended-query messages, the frontend
should issue a Sync message. This parameterless message causes the
backend to close the current transaction if it's not inside a
<command>BEGIN</command>/<command>COMMIT</command> transaction block (<quote>close</quote>
meaning to commit if no error, or roll back if error). Then a
ReadyForQuery response is issued. The purpose of Sync is to provide
a resynchronization point for error recovery. When an error is detected
while processing any extended-query message, the backend issues
ErrorResponse, then reads and discards messages until a Sync is reached,
then issues ReadyForQuery and returns to normal message processing.
(But note that no skipping occurs if an error is detected
<emphasis>while</emphasis> processing Sync &mdash; this ensures that there is one
and only one ReadyForQuery sent for each Sync.)
</para>
<note>
<para>
Sync does not cause a transaction block opened with <command>BEGIN</command>
to be closed. It is possible to detect this situation since the
ReadyForQuery message includes transaction status information.
</para>
</note>
<para>
In addition to these fundamental, required operations, there are several
optional operations that can be used with extended-query protocol.
</para>
<para>
The Describe message (portal variant) specifies the name of an existing
portal (or an empty string for the unnamed portal). The response is a
RowDescription message describing the rows that will be returned by
executing the portal; or a NoData message if the portal does not contain a
query that will return rows; or ErrorResponse if there is no such portal.
</para>
<para>
The Describe message (statement variant) specifies the name of an existing
prepared statement (or an empty string for the unnamed prepared
statement). The response is a ParameterDescription message describing the
parameters needed by the statement, followed by a RowDescription message
describing the rows that will be returned when the statement is eventually
executed (or a NoData message if the statement will not return rows).
ErrorResponse is issued if there is no such prepared statement. Note that
since Bind has not yet been issued, the formats to be used for returned
columns are not yet known to the backend; the format code fields in the
RowDescription message will be zeroes in this case.
</para>
<tip>
<para>
In most scenarios the frontend should issue one or the other variant
of Describe before issuing Execute, to ensure that it knows how to
interpret the results it will get back.
</para>
</tip>
<para>
The Close message closes an existing prepared statement or portal
and releases resources. It is not an error to issue Close against
a nonexistent statement or portal name. The response is normally
CloseComplete, but could be ErrorResponse if some difficulty is
encountered while releasing resources. Note that closing a prepared
statement implicitly closes any open portals that were constructed
from that statement.
</para>
<para>
The Flush message does not cause any specific output to be generated,
but forces the backend to deliver any data pending in its output
buffers. A Flush must be sent after any extended-query command except
Sync, if the frontend wishes to examine the results of that command before
issuing more commands. Without Flush, messages returned by the backend
will be combined into the minimum possible number of packets to minimize
network overhead.
</para>
<note>
<para>
The simple Query message is approximately equivalent to the series Parse,
Bind, portal Describe, Execute, Close, Sync, using the unnamed prepared
statement and portal objects and no parameters. One difference is that
it will accept multiple SQL statements in the query string, automatically
performing the bind/describe/execute sequence for each one in succession.
Another difference is that it will not return ParseComplete, BindComplete,
CloseComplete, or NoData messages.
</para>
</note>
</sect2>
<sect2 id="protocol-flow-pipelining">
<title>Pipelining</title>
<indexterm zone="protocol-flow-pipelining">
<primary>pipelining</primary>
<secondary>protocol specification</secondary>
</indexterm>
<para>
Use of the extended query protocol
allows <firstterm>pipelining</firstterm>, which means sending a series
of queries without waiting for earlier ones to complete. This reduces
the number of network round trips needed to complete a given series of
operations. However, the user must carefully consider the required
behavior if one of the steps fails, since later queries will already
be in flight to the server.
</para>
<para>
One way to deal with that is to make the whole query series be a
single transaction, that is wrap it in <command>BEGIN</command> ...
<command>COMMIT</command>. However, this does not help if one wishes
for some of the commands to commit independently of others.
</para>
<para>
The extended query protocol provides another way to manage this
concern, which is to omit sending Sync messages between steps that
are dependent. Since, after an error, the backend will skip command
messages until it finds Sync, this allows later commands in a pipeline
to be skipped automatically when an earlier one fails, without the
client having to manage that explicitly with <command>BEGIN</command>
and <command>COMMIT</command>. Independently-committable segments
of the pipeline can be separated by Sync messages.
</para>
<para>
If the client has not issued an explicit <command>BEGIN</command>,
then each Sync ordinarily causes an implicit <command>COMMIT</command>
if the preceding step(s) succeeded, or an
implicit <command>ROLLBACK</command> if they failed. However, there
are a few DDL commands (such as <command>CREATE DATABASE</command>)
that cannot be executed inside a transaction block. If one of
these is executed in a pipeline, it will fail unless it is the first
command in the pipeline. Furthermore, upon success it will force an
immediate commit to preserve database consistency. Thus a Sync
immediately following one of these commands has no effect except to
respond with ReadyForQuery.
</para>
<para>
When using this method, completion of the pipeline must be determined
by counting ReadyForQuery messages and waiting for that to reach the
number of Syncs sent. Counting command completion responses is
unreliable, since some of the commands may be skipped and thus not
produce a completion message.
</para>
</sect2>
<sect2 id="protocol-flow-function-call">
<title>Function Call</title>
<para>
The Function Call sub-protocol allows the client to request a direct
call of any function that exists in the database's
<structname>pg_proc</structname> system catalog. The client must have
execute permission for the function.
</para>
<note>
<para>
The Function Call sub-protocol is a legacy feature that is probably best
avoided in new code. Similar results can be accomplished by setting up
a prepared statement that does <literal>SELECT function($1, ...)</literal>.
The Function Call cycle can then be replaced with Bind/Execute.
</para>
</note>
<para>
A Function Call cycle is initiated by the frontend sending a
FunctionCall message to the backend. The backend then sends one
or more response messages depending on the results of the function
call, and finally a ReadyForQuery response message. ReadyForQuery
informs the frontend that it can safely send a new query or
function call.
</para>
<para>
The possible response messages from the backend are:
<variablelist>
<varlistentry>
<term>ErrorResponse</term>
<listitem>
<para>
An error has occurred.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>FunctionCallResponse</term>
<listitem>
<para>
The function call was completed and returned the result given
in the message.
(Note that the Function Call protocol can only handle a single
scalar result, not a row type or set of results.)
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>ReadyForQuery</term>
<listitem>
<para>
Processing of the function call is complete. ReadyForQuery
will always be sent, whether processing terminates
successfully or with an error.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>NoticeResponse</term>
<listitem>
<para>
A warning message has been issued in relation to the function
call. Notices are in addition to other responses, i.e., the
backend will continue processing the command.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</sect2>
<sect2 id="protocol-copy">
<title>COPY Operations</title>
<para>
The <command>COPY</command> command allows high-speed bulk data transfer
to or from the server. Copy-in and copy-out operations each switch
the connection into a distinct sub-protocol, which lasts until the
operation is completed.
</para>
<para>
Copy-in mode (data transfer to the server) is initiated when the
backend executes a <command>COPY FROM STDIN</command> SQL statement. The backend
sends a CopyInResponse message to the frontend. The frontend should
then send zero or more CopyData messages, forming a stream of input
data. (The message boundaries are not required to have anything to do
with row boundaries, although that is often a reasonable choice.)
The frontend can terminate the copy-in mode by sending either a CopyDone
message (allowing successful termination) or a CopyFail message (which
will cause the <command>COPY</command> SQL statement to fail with an
error). The backend then reverts to the command-processing mode it was
in before the <command>COPY</command> started, which will be either simple or
extended query protocol. It will next send either CommandComplete
(if successful) or ErrorResponse (if not).
</para>
<para>
In the event of a backend-detected error during copy-in mode (including
receipt of a CopyFail message), the backend will issue an ErrorResponse
message. If the <command>COPY</command> command was issued via an extended-query
message, the backend will now discard frontend messages until a Sync
message is received, then it will issue ReadyForQuery and return to normal
processing. If the <command>COPY</command> command was issued in a simple
Query message, the rest of that message is discarded and ReadyForQuery
is issued. In either case, any subsequent CopyData, CopyDone, or CopyFail
messages issued by the frontend will simply be dropped.
</para>
<para>
The backend will ignore Flush and Sync messages received during copy-in
mode. Receipt of any other non-copy message type constitutes an error
that will abort the copy-in state as described above. (The exception for
Flush and Sync is for the convenience of client libraries that always
send Flush or Sync after an Execute message, without checking whether
the command to be executed is a <command>COPY FROM STDIN</command>.)
</para>
<para>
Copy-out mode (data transfer from the server) is initiated when the
backend executes a <command>COPY TO STDOUT</command> SQL statement. The backend
sends a CopyOutResponse message to the frontend, followed by
zero or more CopyData messages (always one per row), followed by CopyDone.
The backend then reverts to the command-processing mode it was
in before the <command>COPY</command> started, and sends CommandComplete.
The frontend cannot abort the transfer (except by closing the connection
or issuing a Cancel request),
but it can discard unwanted CopyData and CopyDone messages.
</para>
<para>
In the event of a backend-detected error during copy-out mode,
the backend will issue an ErrorResponse message and revert to normal
processing. The frontend should treat receipt of ErrorResponse as
terminating the copy-out mode.
</para>
<para>
It is possible for NoticeResponse and ParameterStatus messages to be
interspersed between CopyData messages; frontends must handle these cases,
and should be prepared for other asynchronous message types as well (see
<xref linkend="protocol-async"/>). Otherwise, any message type other than
CopyData or CopyDone may be treated as terminating copy-out mode.
</para>
<para>
There is another Copy-related mode called copy-both, which allows
high-speed bulk data transfer to <emphasis>and</emphasis> from the server.
Copy-both mode is initiated when a backend in walsender mode
executes a <command>START_REPLICATION</command> statement. The
backend sends a CopyBothResponse message to the frontend. Both
the backend and the frontend may then send CopyData messages
until either end sends a CopyDone message. After the client
sends a CopyDone message, the connection goes from copy-both mode to
copy-out mode, and the client may not send any more CopyData messages.
Similarly, when the server sends a CopyDone message, the connection
goes into copy-in mode, and the server may not send any more CopyData
messages. After both sides have sent a CopyDone message, the copy mode
is terminated, and the backend reverts to the command-processing mode.
In the event of a backend-detected error during copy-both mode,
the backend will issue an ErrorResponse message, discard frontend messages
until a Sync message is received, and then issue ReadyForQuery and return
to normal processing. The frontend should treat receipt of ErrorResponse
as terminating the copy in both directions; no CopyDone should be sent
in this case. See <xref linkend="protocol-replication"/> for more
information on the subprotocol transmitted over copy-both mode.
</para>
<para>
The CopyInResponse, CopyOutResponse and CopyBothResponse messages
include fields that inform the frontend of the number of columns
per row and the format codes being used for each column. (As of
the present implementation, all columns in a given <command>COPY</command>
operation will use the same format, but the message design does not
assume this.)
</para>
</sect2>
<sect2 id="protocol-async">
<title>Asynchronous Operations</title>
<para>
There are several cases in which the backend will send messages that
are not specifically prompted by the frontend's command stream.
Frontends must be prepared to deal with these messages at any time,
even when not engaged in a query.
At minimum, one should check for these cases before beginning to
read a query response.
</para>
<para>
It is possible for NoticeResponse messages to be generated due to
outside activity; for example, if the database administrator commands
a <quote>fast</quote> database shutdown, the backend will send a NoticeResponse
indicating this fact before closing the connection. Accordingly,
frontends should always be prepared to accept and display NoticeResponse
messages, even when the connection is nominally idle.
</para>
<para>
ParameterStatus messages will be generated whenever the active
value changes for any of the parameters the backend believes the
frontend should know about. Most commonly this occurs in response
to a <command>SET</command> SQL command executed by the frontend, and
this case is effectively synchronous &mdash; but it is also possible
for parameter status changes to occur because the administrator
changed a configuration file and then sent the
<systemitem>SIGHUP</systemitem> signal to the server. Also,
if a <command>SET</command> command is rolled back, an appropriate
ParameterStatus message will be generated to report the current
effective value.
</para>
<para>
At present there is a hard-wired set of parameters for which
ParameterStatus will be generated. They are:
<simplelist type="vert" columns="2">
<member><varname>application_name</varname></member>
<member><varname>client_encoding</varname></member>
<member><varname>DateStyle</varname></member>
<member><varname>default_transaction_read_only</varname></member>
<member><varname>in_hot_standby</varname></member>
<member><varname>integer_datetimes</varname></member>
<member><varname>IntervalStyle</varname></member>
<member><varname>is_superuser</varname></member>
<member><varname>scram_iterations</varname></member>
<member><varname>server_encoding</varname></member>
<member><varname>server_version</varname></member>
<member><varname>session_authorization</varname></member>
<member><varname>standard_conforming_strings</varname></member>
<member><varname>TimeZone</varname></member>
</simplelist>
(<varname>server_encoding</varname>, <varname>TimeZone</varname>, and
<varname>integer_datetimes</varname> were not reported by releases before 8.0;
<varname>standard_conforming_strings</varname> was not reported by releases
before 8.1;
<varname>IntervalStyle</varname> was not reported by releases before 8.4;
<varname>application_name</varname> was not reported by releases before
9.0;
<varname>default_transaction_read_only</varname> and
<varname>in_hot_standby</varname> were not reported by releases before
14; <varname>scram_iterations</varname> was not reported by releases
before 16.)
Note that
<varname>server_version</varname>,
<varname>server_encoding</varname> and
<varname>integer_datetimes</varname>
are pseudo-parameters that cannot change after startup.
This set might change in the future, or even become configurable.
Accordingly, a frontend should simply ignore ParameterStatus for
parameters that it does not understand or care about.
</para>
<para>
If a frontend issues a <command>LISTEN</command> command, then the
backend will send a NotificationResponse message (not to be
confused with NoticeResponse!) whenever a
<command>NOTIFY</command> command is executed for the same
channel name.
</para>
<note>
<para>
At present, NotificationResponse can only be sent outside a
transaction, and thus it will not occur in the middle of a
command-response series, though it might occur just before ReadyForQuery.
It is unwise to design frontend logic that assumes that, however.
Good practice is to be able to accept NotificationResponse at any
point in the protocol.
</para>
</note>
</sect2>
<sect2 id="protocol-flow-canceling-requests">
<title>Canceling Requests in Progress</title>
<para>
During the processing of a query, the frontend might request
cancellation of the query. The cancel request is not sent
directly on the open connection to the backend for reasons of
implementation efficiency: we don't want to have the backend
constantly checking for new input from the frontend during query
processing. Cancel requests should be relatively infrequent, so
we make them slightly cumbersome in order to avoid a penalty in
the normal case.
</para>
<para>
To issue a cancel request, the frontend opens a new connection to
the server and sends a CancelRequest message, rather than the
StartupMessage message that would ordinarily be sent across a new
connection. The server will process this request and then close
the connection. For security reasons, no direct reply is made to
the cancel request message.
</para>
<para>
A CancelRequest message will be ignored unless it contains the
same key data (PID and secret key) passed to the frontend during
connection start-up. If the request matches the PID and secret
key for a currently executing backend, the processing of the
current query is aborted. (In the existing implementation, this is
done by sending a special signal to the backend process that is
processing the query.)
</para>
<para>
The cancellation signal might or might not have any effect &mdash; for
example, if it arrives after the backend has finished processing
the query, then it will have no effect. If the cancellation is
effective, it results in the current command being terminated
early with an error message.
</para>
<para>
The upshot of all this is that for reasons of both security and
efficiency, the frontend has no direct way to tell whether a
cancel request has succeeded. It must continue to wait for the
backend to respond to the query. Issuing a cancel simply improves
the odds that the current query will finish soon, and improves the
odds that it will fail with an error message instead of
succeeding.
</para>
<para>
Since the cancel request is sent across a new connection to the
server and not across the regular frontend/backend communication
link, it is possible for the cancel request to be issued by any
process, not just the frontend whose query is to be canceled.
This might provide additional flexibility when building
multiple-process applications. It also introduces a security
risk, in that unauthorized persons might try to cancel queries.
The security risk is addressed by requiring a dynamically
generated secret key to be supplied in cancel requests.
</para>
</sect2>
<sect2 id="protocol-flow-termination">
<title>Termination</title>
<para>
The normal, graceful termination procedure is that the frontend
sends a Terminate message and immediately closes the connection.
On receipt of this message, the backend closes the connection and
terminates.
</para>
<para>
In rare cases (such as an administrator-commanded database shutdown)
the backend might disconnect without any frontend request to do so.
In such cases the backend will attempt to send an error or notice message
giving the reason for the disconnection before it closes the connection.
</para>
<para>
Other termination scenarios arise from various failure cases, such as core
dump at one end or the other, loss of the communications link, loss of
message-boundary synchronization, etc. If either frontend or backend sees
an unexpected closure of the connection, it should clean
up and terminate. The frontend has the option of launching a new backend
by recontacting the server if it doesn't want to terminate itself.
Closing the connection is also advisable if an unrecognizable message type
is received, since this probably indicates loss of message-boundary sync.
</para>
<para>
For either normal or abnormal termination, any open transaction is
rolled back, not committed. One should note however that if a
frontend disconnects while a non-<command>SELECT</command> query
is being processed, the backend will probably finish the query
before noticing the disconnection. If the query is outside any
transaction block (<command>BEGIN</command> ... <command>COMMIT</command>
sequence) then its results might be committed before the
disconnection is recognized.
</para>
</sect2>
<sect2 id="protocol-flow-ssl">
<title><acronym>SSL</acronym> Session Encryption</title>
<para>
If <productname>PostgreSQL</productname> was built with
<acronym>SSL</acronym> support, frontend/backend communications
can be encrypted using <acronym>SSL</acronym>. This provides
communication security in environments where attackers might be
able to capture the session traffic. For more information on
encrypting <productname>PostgreSQL</productname> sessions with
<acronym>SSL</acronym>, see <xref linkend="ssl-tcp"/>.
</para>
<para>
To initiate an <acronym>SSL</acronym>-encrypted connection, the
frontend initially sends an SSLRequest message rather than a
StartupMessage. The server then responds with a single byte
containing <literal>S</literal> or <literal>N</literal>, indicating that it is
willing or unwilling to perform <acronym>SSL</acronym>,
respectively. The frontend might close the connection at this point
if it is dissatisfied with the response. To continue after
<literal>S</literal>, perform an <acronym>SSL</acronym> startup handshake
(not described here, part of the <acronym>SSL</acronym>
specification) with the server. If this is successful, continue
with sending the usual StartupMessage. In this case the
StartupMessage and all subsequent data will be
<acronym>SSL</acronym>-encrypted. To continue after
<literal>N</literal>, send the usual StartupMessage and proceed without
encryption.
(Alternatively, it is permissible to issue a GSSENCRequest message
after an <literal>N</literal> response to try to
use <acronym>GSSAPI</acronym> encryption instead
of <acronym>SSL</acronym>.)
</para>
<para>
The frontend should also be prepared to handle an ErrorMessage
response to SSLRequest from the server. The frontend should not display
this error message to the user/application, since the server has not been
authenticated
(<ulink url="https://www.postgresql.org/support/security/CVE-2024-10977/">CVE-2024-10977</ulink>).
In this case the connection must
be closed, but the frontend might choose to open a fresh connection
and proceed without requesting <acronym>SSL</acronym>.
</para>
<para>
When <acronym>SSL</acronym> encryption can be performed, the server
is expected to send only the single <literal>S</literal> byte and then
wait for the frontend to initiate an <acronym>SSL</acronym> handshake.
If additional bytes are available to read at this point, it likely
means that a man-in-the-middle is attempting to perform a
buffer-stuffing attack
(<ulink url="https://www.postgresql.org/support/security/CVE-2021-23222/">CVE-2021-23222</ulink>).
Frontends should be coded either to read exactly one byte from the
socket before turning the socket over to their SSL library, or to
treat it as a protocol violation if they find they have read additional
bytes.
</para>
<para>
An initial SSLRequest can also be used in a connection that is being
opened to send a CancelRequest message.
</para>
<para>
While the protocol itself does not provide a way for the server to
force <acronym>SSL</acronym> encryption, the administrator can
configure the server to reject unencrypted sessions as a byproduct
of authentication checking.
</para>
</sect2>
<sect2 id="protocol-flow-gssapi">
<title><acronym>GSSAPI</acronym> Session Encryption</title>
<para>
If <productname>PostgreSQL</productname> was built with
<acronym>GSSAPI</acronym> support, frontend/backend communications
can be encrypted using <acronym>GSSAPI</acronym>. This provides
communication security in environments where attackers might be
able to capture the session traffic. For more information on
encrypting <productname>PostgreSQL</productname> sessions with
<acronym>GSSAPI</acronym>, see <xref linkend="gssapi-enc"/>.
</para>
<para>
To initiate a <acronym>GSSAPI</acronym>-encrypted connection, the
frontend initially sends a GSSENCRequest message rather than a
StartupMessage. The server then responds with a single byte
containing <literal>G</literal> or <literal>N</literal>, indicating that it
is willing or unwilling to perform <acronym>GSSAPI</acronym> encryption,
respectively. The frontend might close the connection at this point
if it is dissatisfied with the response. To continue after
<literal>G</literal>, using the GSSAPI C bindings as discussed in
<ulink url="https://datatracker.ietf.org/doc/html/rfc2744">RFC 2744</ulink>
or equivalent, perform a <acronym>GSSAPI</acronym> initialization by
calling <function>gss_init_sec_context()</function> in a loop and sending
the result to the server, starting with an empty input and then with each
result from the server, until it returns no output. When sending the
results of <function>gss_init_sec_context()</function> to the server,
prepend the length of the message as a four byte integer in network byte
order.
To continue after
<literal>N</literal>, send the usual StartupMessage and proceed without
encryption.
(Alternatively, it is permissible to issue an SSLRequest message
after an <literal>N</literal> response to try to
use <acronym>SSL</acronym> encryption instead
of <acronym>GSSAPI</acronym>.)
</para>
<para>
The frontend should also be prepared to handle an ErrorMessage
response to GSSENCRequest from the server. The frontend should not display
this error message to the user/application, since the server has not been
authenticated
(<ulink url="https://www.postgresql.org/support/security/CVE-2024-10977/">CVE-2024-10977</ulink>).
In this case the connection must be closed, but the frontend might choose
to open a fresh connection and proceed without requesting
<acronym>GSSAPI</acronym> encryption.
</para>
<para>
When <acronym>GSSAPI</acronym> encryption can be performed, the server
is expected to send only the single <literal>G</literal> byte and then
wait for the frontend to initiate a <acronym>GSSAPI</acronym> handshake.
If additional bytes are available to read at this point, it likely
means that a man-in-the-middle is attempting to perform a
buffer-stuffing attack
(<ulink url="https://www.postgresql.org/support/security/CVE-2021-23222/">CVE-2021-23222</ulink>).
Frontends should be coded either to read exactly one byte from the
socket before turning the socket over to their GSSAPI library, or to
treat it as a protocol violation if they find they have read additional
bytes.
</para>
<para>
When <acronym>GSSAPI</acronym> encryption can be performed, the server
is expected to send only the single <literal>G</literal> byte and then
wait for the frontend to initiate a <acronym>GSSAPI</acronym> handshake.
If additional bytes are available to read at this point, it likely
means that a man-in-the-middle is attempting to perform a
buffer-stuffing attack
(<ulink url="https://www.postgresql.org/support/security/CVE-2021-23222/">CVE-2021-23222</ulink>).
Frontends should be coded either to read exactly one byte from the
socket before turning the socket over to their GSSAPI library, or to
treat it as a protocol violation if they find they have read additional
bytes.
</para>
<para>
An initial GSSENCRequest can also be used in a connection that is being
opened to send a CancelRequest message.
</para>
<para>
Once <acronym>GSSAPI</acronym> encryption has been successfully
established, use <function>gss_wrap()</function> to
encrypt the usual StartupMessage and all subsequent data, prepending the
length of the result from <function>gss_wrap()</function> as a four byte
integer in network byte order to the actual encrypted payload. Note that
the server will only accept encrypted packets from the client which are less
than 16kB; <function>gss_wrap_size_limit()</function> should be used by the
client to determine the size of the unencrypted message which will fit
within this limit and larger messages should be broken up into multiple
<function>gss_wrap()</function> calls. Typical segments are 8kB of
unencrypted data, resulting in encrypted packets of slightly larger than 8kB
but well within the 16kB maximum. The server can be expected to not send
encrypted packets of larger than 16kB to the client.
</para>
<para>
While the protocol itself does not provide a way for the server to
force <acronym>GSSAPI</acronym> encryption, the administrator can
configure the server to reject unencrypted sessions as a byproduct
of authentication checking.
</para>
</sect2>
</sect1>
<sect1 id="sasl-authentication">
<title>SASL Authentication</title>
<para>
<firstterm>SASL</firstterm> is a framework for authentication in connection-oriented
protocols. At the moment, <productname>PostgreSQL</productname> implements two SASL
authentication mechanisms, SCRAM-SHA-256 and SCRAM-SHA-256-PLUS. More
might be added in the future. The below steps illustrate how SASL
authentication is performed in general, while the next subsection gives
more details on SCRAM-SHA-256 and SCRAM-SHA-256-PLUS.
</para>
<procedure>
<title>SASL Authentication Message Flow</title>
<step id="sasl-auth-begin">
<para>
To begin a SASL authentication exchange, the server sends an
AuthenticationSASL message. It includes a list of SASL authentication
mechanisms that the server can accept, in the server's preferred order.
</para>
</step>
<step id="sasl-auth-initial-response">
<para>
The client selects one of the supported mechanisms from the list, and sends
a SASLInitialResponse message to the server. The message includes the name
of the selected mechanism, and an optional Initial Client Response, if the
selected mechanism uses that.
</para>
</step>
<step id="sasl-auth-continue">
<para>
One or more server-challenge and client-response message will follow. Each
server-challenge is sent in an AuthenticationSASLContinue message, followed
by a response from client in a SASLResponse message. The particulars of
the messages are mechanism specific.
</para>
</step>
<step id="sasl-auth-end">
<para>
Finally, when the authentication exchange is completed successfully, the
server sends an AuthenticationSASLFinal message, followed
immediately by an AuthenticationOk message. The AuthenticationSASLFinal
contains additional server-to-client data, whose content is particular to the
selected authentication mechanism. If the authentication mechanism doesn't
use additional data that's sent at completion, the AuthenticationSASLFinal
message is not sent.
</para>
</step>
</procedure>
<para>
On error, the server can abort the authentication at any stage, and send an
ErrorMessage.
</para>
<sect2 id="sasl-scram-sha-256">
<title>SCRAM-SHA-256 Authentication</title>
<para>
The implemented SASL mechanisms at the moment
are <literal>SCRAM-SHA-256</literal> and its variant with channel
binding <literal>SCRAM-SHA-256-PLUS</literal>. They are described in
detail in <ulink url="https://datatracker.ietf.org/doc/html/rfc7677">RFC 7677</ulink>
and <ulink url="https://datatracker.ietf.org/doc/html/rfc5802">RFC 5802</ulink>.
</para>
<para>
When SCRAM-SHA-256 is used in PostgreSQL, the server will ignore the user name
that the client sends in the <structname>client-first-message</structname>. The user name
that was already sent in the startup message is used instead.
<productname>PostgreSQL</productname> supports multiple character encodings, while SCRAM
dictates UTF-8 to be used for the user name, so it might be impossible to
represent the PostgreSQL user name in UTF-8.
</para>
<para>
The SCRAM specification dictates that the password is also in UTF-8, and is
processed with the <firstterm>SASLprep</firstterm> algorithm.
<productname>PostgreSQL</productname>, however, does not require UTF-8 to be used for
the password. When a user's password is set, it is processed with SASLprep
as if it was in UTF-8, regardless of the actual encoding used. However, if
it is not a legal UTF-8 byte sequence, or it contains UTF-8 byte sequences
that are prohibited by the SASLprep algorithm, the raw password will be used
without SASLprep processing, instead of throwing an error. This allows the
password to be normalized when it is in UTF-8, but still allows a non-UTF-8
password to be used, and doesn't require the system to know which encoding
the password is in.
</para>
<para>
<firstterm>Channel binding</firstterm> is supported in PostgreSQL builds with
SSL support. The SASL mechanism name for SCRAM with channel binding is
<literal>SCRAM-SHA-256-PLUS</literal>. The channel binding type used by
PostgreSQL is <literal>tls-server-end-point</literal>.
</para>
<para>
In <acronym>SCRAM</acronym> without channel binding, the server chooses
a random number that is transmitted to the client to be mixed with the
user-supplied password in the transmitted password hash. While this
prevents the password hash from being successfully retransmitted in
a later session, it does not prevent a fake server between the real
server and client from passing through the server's random value
and successfully authenticating.
</para>
<para>
<acronym>SCRAM</acronym> with channel binding prevents such
man-in-the-middle attacks by mixing the signature of the server's
certificate into the transmitted password hash. While a fake server can
retransmit the real server's certificate, it doesn't have access to the
private key matching that certificate, and therefore cannot prove it is
the owner, causing SSL connection failure.
</para>
<procedure>
<title>Example</title>
<step id="scram-begin">
<para>
The server sends an AuthenticationSASL message. It includes a list of
SASL authentication mechanisms that the server can accept.
This will be <literal>SCRAM-SHA-256-PLUS</literal>
and <literal>SCRAM-SHA-256</literal> if the server is built with SSL
support, or else just the latter.
</para>
</step>
<step id="scram-client-first">
<para>
The client responds by sending a SASLInitialResponse message, which
indicates the chosen mechanism, <literal>SCRAM-SHA-256</literal> or
<literal>SCRAM-SHA-256-PLUS</literal>. (A client is free to choose either
mechanism, but for better security it should choose the channel-binding
variant if it can support it.) In the Initial Client response field, the
message contains the SCRAM <structname>client-first-message</structname>.
The <structname>client-first-message</structname> also contains the channel
binding type chosen by the client.
</para>
</step>
<step id="scram-server-first">
<para>
Server sends an AuthenticationSASLContinue message, with a SCRAM
<structname>server-first-message</structname> as the content.
</para>
</step>
<step id="scram-client-final">
<para>
Client sends a SASLResponse message, with SCRAM
<structname>client-final-message</structname> as the content.
</para>
</step>
<step id="scram-server-final">
<para>
Server sends an AuthenticationSASLFinal message, with the SCRAM
<structname>server-final-message</structname>, followed immediately by
an AuthenticationOk message.
</para>
</step>
</procedure>
</sect2>
</sect1>
<sect1 id="protocol-replication">
<title>Streaming Replication Protocol</title>
<para>
To initiate streaming replication, the frontend sends the
<literal>replication</literal> parameter in the startup message. A Boolean
value of <literal>true</literal> (or <literal>on</literal>,
<literal>yes</literal>, <literal>1</literal>) tells the backend to go into
physical replication walsender mode, wherein a small set of replication
commands, shown below, can be issued instead of SQL statements.
</para>
<para>
Passing <literal>database</literal> as the value for the
<literal>replication</literal> parameter instructs the backend to go into
logical replication walsender mode, connecting to the database specified in
the <literal>dbname</literal> parameter. In logical replication walsender
mode, the replication commands shown below as well as normal SQL commands can
be issued.
</para>
<para>
In either physical replication or logical replication walsender mode, only the
simple query protocol can be used.
</para>
<para>
For the purpose of testing replication commands, you can make a replication
connection via <application>psql</application> or any other
<application>libpq</application>-using tool with a connection string including
the <literal>replication</literal> option,
e.g.:
<programlisting>
psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;"
</programlisting>
However, it is often more useful to use
<xref linkend="app-pgreceivewal"/> (for physical replication) or
<xref linkend="app-pgrecvlogical"/> (for logical replication).
</para>
<para>
Replication commands are logged in the server log when
<xref linkend="guc-log-replication-commands"/> is enabled.
</para>
<para>
The commands accepted in replication mode are:
<variablelist>
<varlistentry id="protocol-replication-identify-system">
<term><literal>IDENTIFY_SYSTEM</literal>
<indexterm><primary>IDENTIFY_SYSTEM</primary></indexterm>
</term>
<listitem>
<para>
Requests the server to identify itself. Server replies with a result
set of a single row, containing four fields:
</para>
<variablelist>
<varlistentry>
<term><literal>systemid</literal> (<type>text</type>)</term>
<listitem>
<para>
The unique system identifier identifying the cluster. This
can be used to check that the base backup used to initialize the
standby came from the same cluster.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>timeline</literal> (<type>int8</type>)</term>
<listitem>
<para>
Current timeline ID. Also useful to check that the standby is
consistent with the primary.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>xlogpos</literal> (<type>text</type>)</term>
<listitem>
<para>
Current WAL flush location. Useful to get a known location in the
write-ahead log where streaming can start.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>dbname</literal> (<type>text</type>)</term>
<listitem>
<para>
Database connected to or null.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-show">
<term><literal>SHOW</literal> <replaceable class="parameter">name</replaceable>
<indexterm><primary>SHOW</primary></indexterm>
</term>
<listitem>
<para>
Requests the server to send the current setting of a run-time parameter.
This is similar to the SQL command <xref linkend="sql-show"/>.
</para>
<variablelist>
<varlistentry>
<term><replaceable class="parameter">name</replaceable></term>
<listitem>
<para>
The name of a run-time parameter. Available parameters are documented
in <xref linkend="runtime-config"/>.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-timeline-history">
<term><literal>TIMELINE_HISTORY</literal> <replaceable class="parameter">tli</replaceable>
<indexterm><primary>TIMELINE_HISTORY</primary></indexterm>
</term>
<listitem>
<para>
Requests the server to send over the timeline history file for timeline
<replaceable class="parameter">tli</replaceable>. Server replies with a
result set of a single row, containing two fields. While the fields
are labeled as <type>text</type>, they effectively return raw bytes,
with no encoding conversion:
</para>
<variablelist>
<varlistentry>
<term><literal>filename</literal> (<type>text</type>)</term>
<listitem>
<para>
File name of the timeline history file, e.g., <filename>00000002.history</filename>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>content</literal> (<type>text</type>)</term>
<listitem>
<para>
Contents of the timeline history file.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-create-replication-slot" xreflabel="CREATE_REPLICATION_SLOT">
<term><literal>CREATE_REPLICATION_SLOT</literal> <replaceable class="parameter">slot_name</replaceable> [ <literal>TEMPORARY</literal> ] { <literal>PHYSICAL</literal> | <literal>LOGICAL</literal> <replaceable class="parameter">output_plugin</replaceable> } [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ]
<indexterm><primary>CREATE_REPLICATION_SLOT</primary></indexterm>
</term>
<listitem>
<para>
Create a physical or logical replication
slot. See <xref linkend="streaming-replication-slots"/> for more about
replication slots.
</para>
<variablelist>
<varlistentry>
<term><replaceable class="parameter">slot_name</replaceable></term>
<listitem>
<para>
The name of the slot to create. Must be a valid replication slot
name (see <xref linkend="streaming-replication-slots-manipulation"/>).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><replaceable class="parameter">output_plugin</replaceable></term>
<listitem>
<para>
The name of the output plugin used for logical decoding
(see <xref linkend="logicaldecoding-output-plugin"/>).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>TEMPORARY</literal></term>
<listitem>
<para>
Specify that this replication slot is a temporary one. Temporary
slots are not saved to disk and are automatically dropped on error
or when the session has finished.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>The following options are supported:</para>
<variablelist>
<varlistentry>
<term><literal>TWO_PHASE [ <replaceable class="parameter">boolean</replaceable> ]</literal></term>
<listitem>
<para>
If true, this logical replication slot supports decoding of two-phase
commit. With this option, commands related to two-phase commit such as
<literal>PREPARE TRANSACTION</literal>, <literal>COMMIT PREPARED</literal>
and <literal>ROLLBACK PREPARED</literal> are decoded and transmitted.
The transaction will be decoded and transmitted at
<literal>PREPARE TRANSACTION</literal> time.
The default is false.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>RESERVE_WAL [ <replaceable class="parameter">boolean</replaceable> ]</literal></term>
<listitem>
<para>
If true, this physical replication slot reserves <acronym>WAL</acronym>
immediately. Otherwise, <acronym>WAL</acronym> is only reserved upon
connection from a streaming replication client.
The default is false.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>SNAPSHOT { 'export' | 'use' | 'nothing' }</literal></term>
<listitem>
<para>
Decides what to do with the snapshot created during logical slot
initialization. <literal>'export'</literal>, which is the default,
will export the snapshot for use in other sessions. This option can't
be used inside a transaction. <literal>'use'</literal> will use the
snapshot for the current transaction executing the command. This
option must be used in a transaction, and
<literal>CREATE_REPLICATION_SLOT</literal> must be the first command
run in that transaction. Finally, <literal>'nothing'</literal> will
just use the snapshot for logical decoding as normal but won't do
anything else with it.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
In response to this command, the server will send a one-row result set
containing the following fields:
<variablelist>
<varlistentry>
<term><literal>slot_name</literal> (<type>text</type>)</term>
<listitem>
<para>
The name of the newly-created replication slot.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>consistent_point</literal> (<type>text</type>)</term>
<listitem>
<para>
The WAL location at which the slot became consistent. This is the
earliest location from which streaming can start on this replication
slot.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>snapshot_name</literal> (<type>text</type>)</term>
<listitem>
<para>
The identifier of the snapshot exported by the command. The
snapshot is valid until a new command is executed on this connection
or the replication connection is closed. Null if the created slot
is physical.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>output_plugin</literal> (<type>text</type>)</term>
<listitem>
<para>
The name of the output plugin used by the newly-created replication
slot. Null if the created slot is physical.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-create-replication-slot-legacy">
<term><literal>CREATE_REPLICATION_SLOT</literal> <replaceable class="parameter">slot_name</replaceable> [ <literal>TEMPORARY</literal> ] { <literal>PHYSICAL</literal> [ <literal>RESERVE_WAL</literal> ] | <literal>LOGICAL</literal> <replaceable class="parameter">output_plugin</replaceable> [ <literal>EXPORT_SNAPSHOT</literal> | <literal>NOEXPORT_SNAPSHOT</literal> | <literal>USE_SNAPSHOT</literal> | <literal>TWO_PHASE</literal> ] }
</term>
<listitem>
<para>
For compatibility with older releases, this alternative syntax for
the <literal>CREATE_REPLICATION_SLOT</literal> command is still supported.
</para>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-read-replication-slot">
<term><literal>READ_REPLICATION_SLOT</literal> <replaceable class="parameter">slot_name</replaceable>
<indexterm><primary>READ_REPLICATION_SLOT</primary></indexterm>
</term>
<listitem>
<para>
Read some information associated with a replication slot. Returns a tuple
with <literal>NULL</literal> values if the replication slot does not
exist. This command is currently only supported for physical replication
slots.
</para>
<para>
In response to this command, the server will return a one-row result set,
containing the following fields:
<variablelist>
<varlistentry>
<term><literal>slot_type</literal> (<type>text</type>)</term>
<listitem>
<para>
The replication slot's type, either <literal>physical</literal> or
<literal>NULL</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>restart_lsn</literal> (<type>text</type>)</term>
<listitem>
<para>
The replication slot's <literal>restart_lsn</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>restart_tli</literal> (<type>int8</type>)</term>
<listitem>
<para>
The timeline ID associated with <literal>restart_lsn</literal>,
following the current timeline history.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-start-replication">
<term><literal>START_REPLICATION</literal> [ <literal>SLOT</literal> <replaceable class="parameter">slot_name</replaceable> ] [ <literal>PHYSICAL</literal> ] <replaceable class="parameter">XXX/XXX</replaceable> [ <literal>TIMELINE</literal> <replaceable class="parameter">tli</replaceable> ]
<indexterm><primary>START_REPLICATION</primary></indexterm>
</term>
<listitem>
<para>
Instructs server to start streaming WAL, starting at
WAL location <replaceable class="parameter">XXX/XXX</replaceable>.
If <literal>TIMELINE</literal> option is specified,
streaming starts on timeline <replaceable class="parameter">tli</replaceable>;
otherwise, the server's current timeline is selected. The server can
reply with an error, for example if the requested section of WAL has already
been recycled. On success, the server responds with a CopyBothResponse
message, and then starts to stream WAL to the frontend.
</para>
<para>
If a slot's name is provided
via <replaceable class="parameter">slot_name</replaceable>, it will be updated
as replication progresses so that the server knows which WAL segments,
and if <varname>hot_standby_feedback</varname> is on which transactions,
are still needed by the standby.
</para>
<para>
If the client requests a timeline that's not the latest but is part of
the history of the server, the server will stream all the WAL on that
timeline starting from the requested start point up to the point where
the server switched to another timeline. If the client requests
streaming at exactly the end of an old timeline, the server skips COPY
mode entirely.
</para>
<para>
After streaming all the WAL on a timeline that is not the latest one,
the server will end streaming by exiting the COPY mode. When the client
acknowledges this by also exiting COPY mode, the server sends a result
set with one row and two columns, indicating the next timeline in this
server's history. The first column is the next timeline's ID (type <type>int8</type>), and the
second column is the WAL location where the switch happened (type <type>text</type>). Usually,
the switch position is the end of the WAL that was streamed, but there
are corner cases where the server can send some WAL from the old
timeline that it has not itself replayed before promoting. Finally, the
server sends two CommandComplete messages (one that ends the CopyData
and the other ends the <literal>START_REPLICATION</literal> itself), and
is ready to accept a new command.
</para>
<para>
WAL data is sent as a series of CopyData messages. (This allows
other information to be intermixed; in particular the server can send
an ErrorResponse message if it encounters a failure after beginning
to stream.) The payload of each CopyData message from server to the
client contains a message of one of the following formats:
</para>
<variablelist>
<varlistentry id="protocol-replication-xlogdata">
<term>XLogData (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('w')</term>
<listitem>
<para>
Identifies the message as WAL data.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The starting point of the WAL data in this message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The current end of WAL on the server.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The server's system clock at the time of transmission, as
microseconds since midnight on 2000-01-01.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte<replaceable>n</replaceable></term>
<listitem>
<para>
A section of the WAL data stream.
</para>
<para>
A single WAL record is never split across two XLogData messages.
When a WAL record crosses a WAL page boundary, and is therefore
already split using continuation records, it can be split at the page
boundary. In other words, the first main WAL record and its
continuation records can be sent in different XLogData messages.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-primary-keepalive-message">
<term>Primary keepalive message (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('k')</term>
<listitem>
<para>
Identifies the message as a sender keepalive.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The current end of WAL on the server.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The server's system clock at the time of transmission, as
microseconds since midnight on 2000-01-01.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte1</term>
<listitem>
<para>
1 means that the client should reply to this message as soon as
possible, to avoid a timeout disconnect. 0 otherwise.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
</variablelist>
<para>
The receiving process can send replies back to the sender at any time,
using one of the following message formats (also in the payload of a
CopyData message):
</para>
<variablelist>
<varlistentry id="protocol-replication-standby-status-update">
<term>Standby status update (F)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('r')</term>
<listitem>
<para>
Identifies the message as a receiver status update.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The location of the last WAL byte + 1 received and written to disk
in the standby.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The location of the last WAL byte + 1 flushed to disk in
the standby.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The location of the last WAL byte + 1 applied in the standby.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The client's system clock at the time of transmission, as
microseconds since midnight on 2000-01-01.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte1</term>
<listitem>
<para>
If 1, the client requests the server to reply to this message
immediately. This can be used to ping the server, to test if
the connection is still healthy.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-hot-standby-feedback-message">
<term>Hot standby feedback message (F)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('h')</term>
<listitem>
<para>
Identifies the message as a hot standby feedback message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem>
<para>
The client's system clock at the time of transmission, as
microseconds since midnight on 2000-01-01.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The standby's current global xmin, excluding the catalog_xmin from any
replication slots. If both this value and the following
catalog_xmin are 0 this is treated as a notification that hot standby
feedback will no longer be sent on this connection. Later non-zero
messages may reinitiate the feedback mechanism.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The epoch of the global xmin xid on the standby.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The lowest catalog_xmin of any replication slots on the standby. Set to 0
if no catalog_xmin exists on the standby or if hot standby feedback is being
disabled.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The epoch of the catalog_xmin xid on the standby.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-start-replication-slot-logical">
<term><literal>START_REPLICATION</literal> <literal>SLOT</literal> <replaceable class="parameter">slot_name</replaceable> <literal>LOGICAL</literal> <replaceable class="parameter">XXX/XXX</replaceable> [ ( <replaceable>option_name</replaceable> [ <replaceable>option_value</replaceable> ] [, ...] ) ]</term>
<listitem>
<para>
Instructs server to start streaming WAL for logical replication,
starting at either WAL location <replaceable
class="parameter">XXX/XXX</replaceable> or the slot's
<literal>confirmed_flush_lsn</literal> (see <xref
linkend="view-pg-replication-slots"/>), whichever is greater. This
behavior makes it easier for clients to avoid updating their local LSN
status when there is no data to process. However, starting at a
different LSN than requested might not catch certain kinds of client
errors; so the client may wish to check that
<literal>confirmed_flush_lsn</literal> matches its expectations before
issuing <literal>START_REPLICATION</literal>.
</para>
<para>
The server can reply with an error, for example if the
slot does not exist. On success, the server responds with a CopyBothResponse
message, and then starts to stream WAL to the frontend.
</para>
<para>
The messages inside the CopyBothResponse messages are of the same format
documented for <literal>START_REPLICATION ... PHYSICAL</literal>, including
two CommandComplete messages.
</para>
<para>
The output plugin associated with the selected slot is used
to process the output for streaming.
</para>
<variablelist>
<varlistentry>
<term><literal>SLOT</literal> <replaceable class="parameter">slot_name</replaceable></term>
<listitem>
<para>
The name of the slot to stream changes from. This parameter is required,
and must correspond to an existing logical replication slot created
with <literal>CREATE_REPLICATION_SLOT</literal> in
<literal>LOGICAL</literal> mode.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><replaceable class="parameter">XXX/XXX</replaceable></term>
<listitem>
<para>
The WAL location to begin streaming at.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><replaceable class="parameter">option_name</replaceable></term>
<listitem>
<para>
The name of an option passed to the slot's logical decoding output
plugin. See <xref linkend="protocol-logical-replication"/> for
options that are accepted by the standard (<literal>pgoutput</literal>)
plugin.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><replaceable class="parameter">option_value</replaceable></term>
<listitem>
<para>
Optional value, in the form of a string constant, associated with the
specified option.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-drop-replication-slot">
<term>
<literal>DROP_REPLICATION_SLOT</literal> <replaceable class="parameter">slot_name</replaceable> <optional> <literal>WAIT</literal> </optional>
<indexterm><primary>DROP_REPLICATION_SLOT</primary></indexterm>
</term>
<listitem>
<para>
Drops a replication slot, freeing any reserved server-side resources.
</para>
<variablelist>
<varlistentry>
<term><replaceable class="parameter">slot_name</replaceable></term>
<listitem>
<para>
The name of the slot to drop.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>WAIT</literal></term>
<listitem>
<para>
This option causes the command to wait if the slot is active until
it becomes inactive, instead of the default behavior of raising an
error.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-replication-base-backup" xreflabel="BASE_BACKUP">
<term><literal>BASE_BACKUP</literal> [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ]
<indexterm><primary>BASE_BACKUP</primary></indexterm>
</term>
<listitem>
<para>
Instructs the server to start streaming a base backup.
The system will automatically be put in backup mode before the backup
is started, and taken out of it when the backup is complete. The
following options are accepted:
<variablelist>
<varlistentry>
<term><literal>LABEL</literal> <replaceable>'label'</replaceable></term>
<listitem>
<para>
Sets the label of the backup. If none is specified, a backup label
of <literal>base backup</literal> will be used. The quoting rules
for the label are the same as a standard SQL string with
<xref linkend="guc-standard-conforming-strings"/> turned on.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>TARGET</literal> <replaceable>'target'</replaceable></term>
<listitem>
<para>
Tells the server where to send the backup. If the target is
<literal>client</literal>, which is the default, the backup data is
sent to the client. If it is <literal>server</literal>, the backup
data is written to the server at the pathname specified by the
<literal>TARGET_DETAIL</literal> option. If it is
<literal>blackhole</literal>, the backup data is not sent
anywhere; it is simply discarded.
</para>
<para>
The <literal>server</literal> target requires superuser privilege or
being granted the <literal>pg_write_server_files</literal> role.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>TARGET_DETAIL</literal> <replaceable>'detail'</replaceable></term>
<listitem>
<para>
Provides additional information about the backup target.
</para>
<para>
Currently, this option can only be used when the backup target is
<literal>server</literal>. It specifies the server directory
to which the backup should be written.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>PROGRESS [ <replaceable class="parameter">boolean</replaceable> ]</literal></term>
<listitem>
<para>
If set to true, request information required to generate a progress
report. This will send back an approximate size in the header of each
tablespace, which can be used to calculate how far along the stream
is done. This is calculated by enumerating all the file sizes once
before the transfer is even started, and might as such have a
negative impact on the performance. In particular, it might take
longer before the first data
is streamed. Since the database files can change during the backup,
the size is only approximate and might both grow and shrink between
the time of approximation and the sending of the actual files.
The default is false.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>CHECKPOINT { 'fast' | 'spread' }</literal></term>
<listitem>
<para>
Sets the type of checkpoint to be performed at the beginning of the
base backup. The default is <literal>spread</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>WAL [ <replaceable class="parameter">boolean</replaceable> ]</literal></term>
<listitem>
<para>
If set to true, include the necessary WAL segments in the backup.
This will include all the files between start and stop backup in the
<filename>pg_wal</filename> directory of the base directory tar
file. The default is false.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>WAIT [ <replaceable class="parameter">boolean</replaceable> ]</literal></term>
<listitem>
<para>
If set to true, the backup will wait until the last required WAL
segment has been archived, or emit a warning if WAL archiving is
not enabled. If false, the backup will neither wait nor warn,
leaving the client responsible for ensuring the required log is
available. The default is true.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>COMPRESSION</literal> <replaceable>'method'</replaceable></term>
<listitem>
<para>
Instructs the server to compress the backup using the specified
method. Currently, the supported methods are <literal>gzip</literal>,
<literal>lz4</literal>, and <literal>zstd</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>COMPRESSION_DETAIL</literal> <replaceable>detail</replaceable></term>
<listitem>
<para>
Specifies details for the chosen compression method. This should only
be used in conjunction with the <literal>COMPRESSION</literal>
option. If the value is an integer, it specifies the compression
level. Otherwise, it should be a comma-separated list of items,
each of the form <replaceable>keyword</replaceable> or
<replaceable>keyword=value</replaceable>. Currently, the supported
keywords are <literal>level</literal>, <literal>long</literal> and
<literal>workers</literal>.
</para>
<para>
The <literal>level</literal> keyword sets the compression level.
For <literal>gzip</literal> the compression level should be an
integer between <literal>1</literal> and <literal>9</literal>
(default <literal>Z_DEFAULT_COMPRESSION</literal> or
<literal>-1</literal>), for <literal>lz4</literal> an integer
between 1 and 12 (default <literal>0</literal> for fast compression
mode), and for <literal>zstd</literal> an integer between
<literal>ZSTD_minCLevel()</literal> (usually <literal>-131072</literal>)
and <literal>ZSTD_maxCLevel()</literal> (usually <literal>22</literal>),
(default <literal>ZSTD_CLEVEL_DEFAULT</literal> or
<literal>3</literal>).
</para>
<para>
The <literal>long</literal> keyword enables long-distance matching
mode, for improved compression ratio, at the expense of higher memory
use. Long-distance mode is supported only for
<literal>zstd</literal>.
</para>
<para>
The <literal>workers</literal> keyword sets the number of threads
that should be used for parallel compression. Parallel compression
is supported only for <literal>zstd</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>MAX_RATE</literal> <replaceable>rate</replaceable></term>
<listitem>
<para>
Limit (throttle) the maximum amount of data transferred from server
to client per unit of time. The expected unit is kilobytes per second.
If this option is specified, the value must either be equal to zero
or it must fall within the range from 32 kB through 1 GB (inclusive).
If zero is passed or the option is not specified, no restriction is
imposed on the transfer.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>TABLESPACE_MAP [ <replaceable class="parameter">boolean</replaceable> ]</literal></term>
<listitem>
<para>
If true, include information about symbolic links present in the
directory <filename>pg_tblspc</filename> in a file named
<filename>tablespace_map</filename>. The tablespace map file includes
each symbolic link name as it exists in the directory
<filename>pg_tblspc/</filename> and the full path of that symbolic link.
The default is false.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>VERIFY_CHECKSUMS [ <replaceable class="parameter">boolean</replaceable> ]</literal></term>
<listitem>
<para>
If true, checksums are verified during a base backup if they are
enabled. If false, this is skipped. The default is true.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>MANIFEST</literal> <replaceable>manifest_option</replaceable></term>
<listitem>
<para>
When this option is specified with a value of <literal>yes</literal>
or <literal>force-encode</literal>, a backup manifest is created
and sent along with the backup. The manifest is a list of every
file present in the backup with the exception of any WAL files that
may be included. It also stores the size, last modification time, and
optionally a checksum for each file.
A value of <literal>force-encode</literal> forces all filenames
to be hex-encoded; otherwise, this type of encoding is performed only
for files whose names are non-UTF8 octet sequences.
<literal>force-encode</literal> is intended primarily for testing
purposes, to be sure that clients which read the backup manifest
can handle this case. For compatibility with previous releases,
the default is <literal>MANIFEST 'no'</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>MANIFEST_CHECKSUMS</literal> <replaceable>checksum_algorithm</replaceable></term>
<listitem>
<para>
Specifies the checksum algorithm that should be applied to each file included
in the backup manifest. Currently, the available
algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
<literal>SHA224</literal>, <literal>SHA256</literal>,
<literal>SHA384</literal>, and <literal>SHA512</literal>.
The default is <literal>CRC32C</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
<para>
When the backup is started, the server will first send two
ordinary result sets, followed by one or more CopyOutResponse
results.
</para>
<para>
The first ordinary result set contains the starting position of the
backup, in a single row with two columns. The first column contains
the start position given in XLogRecPtr format, and the second column
contains the corresponding timeline ID.
</para>
<para>
The second ordinary result set has one row for each tablespace.
The fields in this row are:
<variablelist>
<varlistentry>
<term><literal>spcoid</literal> (<type>oid</type>)</term>
<listitem>
<para>
The OID of the tablespace, or null if it's the base
directory.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>spclocation</literal> (<type>text</type>)</term>
<listitem>
<para>
The full path of the tablespace directory, or null
if it's the base directory.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>size</literal> (<type>int8</type>)</term>
<listitem>
<para>
The approximate size of the tablespace, in kilobytes (1024 bytes),
if progress report has been requested; otherwise it's null.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
<para>
After the second regular result set, a CopyOutResponse will be sent.
The payload of each CopyData message will contain a message in one of
the following formats:
</para>
<variablelist>
<varlistentry>
<term>new archive (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('n')</term>
<listitem><para>
Identifies the message as indicating the start of a new archive.
There will be one archive for the main data directory and one
for each additional tablespace; each will use tar format
(following the <quote>ustar interchange format</quote> specified
in the POSIX 1003.1-2008 standard).
</para></listitem>
</varlistentry>
<varlistentry>
<term>String</term>
<listitem><para>
The file name for this archive.
</para></listitem>
</varlistentry>
<varlistentry>
<term>String</term>
<listitem><para>
For the main data directory, an empty string. For other
tablespaces, the full path to the directory from which this
archive was created.
</para></listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry>
<term>manifest (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('m')</term>
<listitem><para>
Identifies the message as indicating the start of the backup
manifest.
</para></listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry>
<term>archive or manifest data (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('d')</term>
<listitem><para>
Identifies the message as containing archive or manifest data.
</para></listitem>
</varlistentry>
<varlistentry>
<term>Byte<replaceable>n</replaceable></term>
<listitem><para>
Data bytes.
</para></listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry>
<term>progress report (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('p')</term>
<listitem><para>
Identifies the message as a progress report.
</para></listitem>
</varlistentry>
<varlistentry>
<term>Int64</term>
<listitem><para>
The number of bytes from the current tablespace for which
processing has been completed.
</para></listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
</variablelist>
<para>
After the CopyOutResponse, or all such responses, have been sent, a
final ordinary result set will be sent, containing the WAL end position
of the backup, in the same format as the start position.
</para>
<para>
The tar archive for the data directory and each tablespace will contain
all files in the directories, regardless of whether they are
<productname>PostgreSQL</productname> files or other files added to the same
directory. The only excluded files are:
<itemizedlist spacing="compact" mark="bullet">
<listitem>
<para>
<filename>postmaster.pid</filename>
</para>
</listitem>
<listitem>
<para>
<filename>postmaster.opts</filename>
</para>
</listitem>
<listitem>
<para>
<filename>pg_internal.init</filename> (found in multiple directories)
</para>
</listitem>
<listitem>
<para>
Various temporary files and directories created during the operation
of the PostgreSQL server, such as any file or directory beginning
with <filename>pgsql_tmp</filename> and temporary relations.
</para>
</listitem>
<listitem>
<para>
Unlogged relations, except for the init fork which is required to
recreate the (empty) unlogged relation on recovery.
</para>
</listitem>
<listitem>
<para>
<filename>pg_wal</filename>, including subdirectories. If the backup is run
with WAL files included, a synthesized version of <filename>pg_wal</filename> will be
included, but it will only contain the files necessary for the
backup to work, not the rest of the contents.
</para>
</listitem>
<listitem>
<para>
<filename>pg_dynshmem</filename>, <filename>pg_notify</filename>,
<filename>pg_replslot</filename>, <filename>pg_serial</filename>,
<filename>pg_snapshots</filename>, <filename>pg_stat_tmp</filename>, and
<filename>pg_subtrans</filename> are copied as empty directories (even if
they are symbolic links).
</para>
</listitem>
<listitem>
<para>
Files other than regular files and directories, such as symbolic
links (other than for the directories listed above) and special
device and operating system files, are skipped. (Symbolic links
in <filename>pg_tblspc</filename> are maintained.)
</para>
</listitem>
</itemizedlist>
Owner, group, and file mode are set if the underlying file system on
the server supports it.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</sect1>
<sect1 id="protocol-logical-replication">
<title>Logical Streaming Replication Protocol</title>
<para>
This section describes the logical replication protocol, which is the message
flow started by the <literal>START_REPLICATION</literal>
<literal>SLOT</literal> <replaceable class="parameter">slot_name</replaceable>
<literal>LOGICAL</literal> replication command.
</para>
<para>
The logical streaming replication protocol builds on the primitives of
the physical streaming replication protocol.
</para>
<para>
<productname>PostgreSQL</productname> logical decoding supports output
plugins. <literal>pgoutput</literal> is the standard one used for
the built-in logical replication.
</para>
<sect2 id="protocol-logical-replication-params">
<title>Logical Streaming Replication Parameters</title>
<para>
Using the <literal>START_REPLICATION</literal> command,
<literal>pgoutput</literal> accepts the following options:
<variablelist>
<varlistentry>
<term>
proto_version
</term>
<listitem>
<para>
Protocol version. Currently versions <literal>1</literal>, <literal>2</literal>,
<literal>3</literal>, and <literal>4</literal> are supported. A valid
version is required.
</para>
<para>
Version <literal>2</literal> is supported only for server version 14
and above, and it allows streaming of large in-progress transactions.
</para>
<para>
Version <literal>3</literal> is supported only for server version 15
and above, and it allows streaming of two-phase commits.
</para>
<para>
Version <literal>4</literal> is supported only for server version 16
and above, and it allows streams of large in-progress transactions to
be applied in parallel.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
publication_names
</term>
<listitem>
<para>
Comma separated list of publication names for which to subscribe
(receive changes). The individual publication names are treated
as standard objects names and can be quoted the same as needed.
At least one publication name is required.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
binary
</term>
<listitem>
<para>
Boolean option to use binary transfer mode. Binary mode is faster
than the text mode but slightly less robust.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
messages
</term>
<listitem>
<para>
Boolean option to enable sending the messages that are written
by <function>pg_logical_emit_message</function>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
streaming
</term>
<listitem>
<para>
Boolean option to enable streaming of in-progress transactions.
It accepts an additional value "parallel" to enable sending extra
information with some messages to be used for parallelisation.
Minimum protocol version 2 is required to turn it on. Minimum protocol
version 4 is required for the "parallel" option.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
two_phase
</term>
<listitem>
<para>
Boolean option to enable two-phase transactions. Minimum protocol
version 3 is required to turn it on.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
origin
</term>
<listitem>
<para>
Option to send changes by their origin. Possible values are "none"
to only send the changes that have no origin associated, or "any"
to send the changes regardless of their origin. This can be used
to avoid loops (infinite replication of the same data) among
replication nodes.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</sect2>
<sect2 id="protocol-logical-messages">
<title>Logical Replication Protocol Messages</title>
<para>
The individual protocol messages are discussed in the following
subsections. Individual messages are described in
<xref linkend="protocol-logicalrep-message-formats"/>.
</para>
<para>
All top-level protocol messages begin with a message type byte.
While represented in code as a character, this is a signed byte with no
associated encoding.
</para>
<para>
Since the streaming replication protocol supplies a message length there
is no need for top-level protocol messages to embed a length in their
header.
</para>
</sect2>
<sect2 id="protocol-logical-messages-flow">
<title>Logical Replication Protocol Message Flow</title>
<para>
With the exception of the <literal>START_REPLICATION</literal> command and
the replay progress messages, all information flows only from the backend
to the frontend.
</para>
<para>
The logical replication protocol sends individual transactions one by one.
This means that all messages between a pair of Begin and Commit messages
belong to the same transaction. Similarly, all messages between a pair of
Begin Prepare and Prepare messages belong to the same transaction.
It also sends changes of large in-progress transactions between a pair of
Stream Start and Stream Stop messages. The last stream of such a transaction
contains a Stream Commit or Stream Abort message.
</para>
<para>
Every sent transaction contains zero or more DML messages (Insert,
Update, Delete). In case of a cascaded setup it can also contain Origin
messages. The origin message indicates that the transaction originated on
different replication node. Since a replication node in the scope of logical
replication protocol can be pretty much anything, the only identifier
is the origin name. It's downstream's responsibility to handle this as
needed (if needed). The Origin message is always sent before any DML
messages in the transaction.
</para>
<para>
Every DML message contains a relation OID, identifying the publisher's
relation that was acted on. Before the first DML message for a given
relation OID, a Relation message will be sent, describing the schema of
that relation. Subsequently, a new Relation message will be sent if
the relation's definition has changed since the last Relation message
was sent for it. (The protocol assumes that the client is capable of
remembering this metadata for as many relations as needed.)
</para>
<para>
Relation messages identify column types by their OIDs. In the case
of a built-in type, it is assumed that the client can look up that
type OID locally, so no additional data is needed. For a non-built-in
type OID, a Type message will be sent before the Relation message,
to provide the type name associated with that OID. Thus, a client that
needs to specifically identify the types of relation columns should
cache the contents of Type messages, and first consult that cache to
see if the type OID is defined there. If not, look up the type OID
locally.
</para>
</sect2>
</sect1>
<sect1 id="protocol-message-types">
<title>Message Data Types</title>
<para>
This section describes the base data types used in messages.
</para>
<variablelist>
<varlistentry>
<term>Int<replaceable>n</replaceable>(<replaceable>i</replaceable>)</term>
<listitem>
<para>
An <replaceable>n</replaceable>-bit integer in network byte
order (most significant byte first).
If <replaceable>i</replaceable> is specified it
is the exact value that will appear, otherwise the value
is variable. Eg. Int16, Int32(42).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int<replaceable>n</replaceable>[<replaceable>k</replaceable>]</term>
<listitem>
<para>
An array of <replaceable>k</replaceable>
<replaceable>n</replaceable>-bit integers, each in network
byte order. The array length <replaceable>k</replaceable>
is always determined by an earlier field in the message.
Eg. Int16[M].
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>String(<replaceable>s</replaceable>)</term>
<listitem>
<para>
A null-terminated string (C-style string). There is no
specific length limitation on strings.
If <replaceable>s</replaceable> is specified it is the exact
value that will appear, otherwise the value is variable.
Eg. String, String("user").
</para>
<note>
<para>
<emphasis>There is no predefined limit</emphasis> on the length of a string
that can be returned by the backend. Good coding strategy for a frontend
is to use an expandable buffer so that anything that fits in memory can be
accepted. If that's not feasible, read the full string and discard trailing
characters that don't fit into your fixed-size buffer.
</para>
</note>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte<replaceable>n</replaceable>(<replaceable>c</replaceable>)</term>
<listitem>
<para>
Exactly <replaceable>n</replaceable> bytes. If the field
width <replaceable>n</replaceable> is not a constant, it is
always determinable from an earlier field in the message.
If <replaceable>c</replaceable> is specified it is the exact
value. Eg. Byte2, Byte1('\n').
</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 id="protocol-message-formats">
<title>Message Formats</title>
<para>
This section describes the detailed format of each message. Each is marked to
indicate that it can be sent by a frontend (F), a backend (B), or both
(F &amp; B).
Notice that although each message includes a byte count at the beginning,
the message format is defined so that the message end can be found without
reference to the byte count. This aids validity checking. (The CopyData
message is an exception, because it forms part of a data stream; the contents
of any individual CopyData message cannot be interpretable on their own.)
</para>
<variablelist>
<varlistentry id="protocol-message-formats-AuthenticationOk">
<term>AuthenticationOk (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(8)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(0)</term>
<listitem>
<para>
Specifies that the authentication was successful.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationKerberosV5">
<term>AuthenticationKerberosV5 (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(8)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(2)</term>
<listitem>
<para>
Specifies that Kerberos V5 authentication is required.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationCleartextPassword">
<term>AuthenticationCleartextPassword (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(8)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(3)</term>
<listitem>
<para>
Specifies that a clear-text password is required.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationMD5Password">
<term>AuthenticationMD5Password (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(12)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(5)</term>
<listitem>
<para>
Specifies that an MD5-encrypted password is required.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte4</term>
<listitem>
<para>
The salt to use when encrypting the password.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationGSS">
<term>AuthenticationGSS (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(8)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(7)</term>
<listitem>
<para>
Specifies that GSSAPI authentication is required.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationGSSContinue">
<term>AuthenticationGSSContinue (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(8)</term>
<listitem>
<para>
Specifies that this message contains GSSAPI or SSPI data.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte<replaceable>n</replaceable></term>
<listitem>
<para>
GSSAPI or SSPI authentication data.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationSSPI">
<term>AuthenticationSSPI (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(8)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(9)</term>
<listitem>
<para>
Specifies that SSPI authentication is required.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationSASL">
<term>AuthenticationSASL (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(10)</term>
<listitem>
<para>
Specifies that SASL authentication is required.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
The message body is a list of SASL authentication mechanisms, in the
server's order of preference. A zero byte is required as terminator after
the last authentication mechanism name. For each mechanism, there is the
following:
<variablelist>
<varlistentry>
<term>String</term>
<listitem>
<para>
Name of a SASL authentication mechanism.
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationSASLContinue">
<term>AuthenticationSASLContinue (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(11)</term>
<listitem>
<para>
Specifies that this message contains a SASL challenge.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte<replaceable>n</replaceable></term>
<listitem>
<para>
SASL data, specific to the SASL mechanism being used.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-AuthenticationSASLFinal">
<term>AuthenticationSASLFinal (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('R')</term>
<listitem>
<para>
Identifies the message as an authentication request.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(12)</term>
<listitem>
<para>
Specifies that SASL authentication has completed.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte<replaceable>n</replaceable></term>
<listitem>
<para>
SASL outcome "additional data", specific to the SASL mechanism
being used.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-BackendKeyData">
<term>BackendKeyData (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('K')</term>
<listitem>
<para>
Identifies the message as cancellation key data.
The frontend must save these values if it wishes to be
able to issue CancelRequest messages later.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(12)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The process ID of this backend.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The secret key of this backend.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-Bind">
<term>Bind (F)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('B')</term>
<listitem>
<para>
Identifies the message as a Bind command.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>String</term>
<listitem>
<para>
The name of the destination portal
(an empty string selects the unnamed portal).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>String</term>
<listitem>
<para>
The name of the source prepared statement
(an empty string selects the unnamed prepared statement).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int16</term>
<listitem>
<para>
The number of parameter format codes that follow
(denoted <replaceable>C</replaceable> below).
This can be zero to indicate that there are no parameters
or that the parameters all use the default format (text);
or one, in which case the specified format code is applied
to all parameters; or it can equal the actual number of
parameters.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int16[<replaceable>C</replaceable>]</term>
<listitem>
<para>
The parameter format codes. Each must presently be
zero (text) or one (binary).
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int16</term>
<listitem>
<para>
The number of parameter values that follow (possibly zero).
This must match the number of parameters needed by the query.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
Next, the following pair of fields appear for each parameter:
</para>
<variablelist>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The length of the parameter value, in bytes (this count
does not include itself). Can be zero.
As a special case, -1 indicates a NULL parameter value.
No value bytes follow in the NULL case.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte<replaceable>n</replaceable></term>
<listitem>
<para>
The value of the parameter, in the format indicated by the
associated format code.
<replaceable>n</replaceable> is the above length.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
After the last parameter, the following fields appear:
</para>
<variablelist>
<varlistentry>
<term>Int16</term>
<listitem>
<para>
The number of result-column format codes that follow
(denoted <replaceable>R</replaceable> below).
This can be zero to indicate that there are no result columns
or that the result columns should all use the default format
(text);
or one, in which case the specified format code is applied
to all result columns (if any); or it can equal the actual
number of result columns of the query.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int16[<replaceable>R</replaceable>]</term>
<listitem>
<para>
The result-column format codes. Each must presently be
zero (text) or one (binary).
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-BindComplete">
<term>BindComplete (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('2')</term>
<listitem>
<para>
Identifies the message as a Bind-complete indicator.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(4)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-CancelRequest">
<term>CancelRequest (F)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Int32(16)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(80877102)</term>
<listitem>
<para>
The cancel request code. The value is chosen to contain
<literal>1234</literal> in the most significant 16 bits, and <literal>5678</literal> in the
least significant 16 bits. (To avoid confusion, this code
must not be the same as any protocol version number.)
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The process ID of the target backend.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
The secret key for the target backend.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-Close">
<term>Close (F)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('C')</term>
<listitem>
<para>
Identifies the message as a Close command.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Byte1</term>
<listitem>
<para>
'<literal>S</literal>' to close a prepared statement; or
'<literal>P</literal>' to close a portal.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>String</term>
<listitem>
<para>
The name of the prepared statement or portal to close
(an empty string selects the unnamed prepared statement
or portal).
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-CloseComplete">
<term>CloseComplete (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('3')</term>
<listitem>
<para>
Identifies the message as a Close-complete indicator.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32(4)</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
</variablelist>
</listitem>
</varlistentry>
<varlistentry id="protocol-message-formats-CommandComplete">
<term>CommandComplete (B)</term>
<listitem>
<variablelist>
<varlistentry>
<term>Byte1('C')</term>
<listitem>
<para>
Identifies the message as a command-completed response.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>Int32</term>
<listitem>
<para>
Length of message contents in bytes, including self.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>String</term>
<listitem>
<para>
The command tag. This is usually a single
word that identifies which SQL command was completed.
</para>
<para>
For an <command>INSERT</command> command, the tag is
<literal>INSERT <replaceable>oid</replaceable>
<replaceable>rows</replaceable></literal>, where
<replaceable>rows</replaceable> is the number of rows
inserted. <replaceable>oid</replaceable> used to be the object ID
of the inserted row if <replaceable>rows</replaceable> was 1
and the target table had OIDs, but OIDs system columns are
not supported anymore; therefore <replaceable>oid</replaceable>
is always 0.
</para>
<para>
For a <command>DELETE</command> command, the tag is
<literal>DELETE <replaceable>rows</replaceable></literal> where
<replaceable>rows</replaceable> is the number of rows deleted.
</para>
<para>
For an <command>UPDATE</command> command, the tag is
<literal>UPDATE <replaceable>rows</replaceable></literal> where
<replaceable>rows</replaceable> is the number of rows updated.
</para>
<para>
For a <command>MERGE</command> command, the tag is
<literal>MERGE <replaceable>rows</replaceable></literal> where
<replaceable>rows</replaceable> is the number of rows inserted,
updated, or deleted.
</para>
<para>
For a <command>SELECT</command> or <command>CREATE TABLE AS</command>
command, the tag is <literal>SELECT <replaceable>rows</replaceable></literal>
where <replaceable>rows</replaceable> is the number of rows retrieved.
</para>
<para>
For a <command>MOVE</command> command, the tag is
<literal>MOVE <replaceable>rows</replaceable></literal> where
<replaceable>rows</replaceable> is the number of rows the
cursor's position has been changed by.
</para>
<para>
For a <command>FETCH</command> command, the tag is
<literal>FETCH <replaceable>rows</replaceable></literal> where
<replaceable>rows</replaceable> is the number of rows that
have been retrieved from the cursor.
</para>
<para>
For a <command>COPY</command> command, the tag is
<literal>COPY <replaceable>rows</replaceable></literal> where
<replaceable>rows</replaceable> is the number of rows copied.
(Note: the row count appears only in
<productname>PostgreSQL</productname> 8.2 and later.)
</para>
</chapter>