Setting up Streaming Replication in PostgreSQL 13 and Streaming Replication Internals

While migrating to PostgreSQL, some of the users may be curious about the type of replication offered by PostgreSQL. Some might want to know if they have to pay any additional enterprise license in order to achieve this requirement. This is because, a few commercial databases require a customer to take an enteprise license to setup replication. But, PostgreSQL, like advanced Open Source databases, automatically offers several types of replication methods built-in. This means, we never have to consider taking any additional license or an enterprise solution and stay away from vendor lock-in while using PostgreSQL. In this Article, we shall discuss the streaming replication internals and also see the detailed steps involved in setting up Streaming Replication in PostgreSQL 13.

PostgreSQL supports 2 types of replication. 

  • Streaming Replication (Synchronous and Asynchronous)
  • Logical Replication (Synchronous and Asynchronous)
Streaming Replication

Streaming replication may also be referred to as physical replication. One would generally hear the terms master and standby in postgres. A master is a server that accepts writes and streams changes to a standby. It can be compared with a primary database in some of the database technologies. Whereas, a standby replicates changes from a master and accepts only reads. It may be called a replica in some of the database technologies. 

Internals of streaming replication

To understand the internals of how streaming replication works, we need to start from ACID properties. The letter D in ACID means durability. Durability is said to be achieved when a relational database ensures that a committed transaction is never lost, even upon a crash. In general, anything in memory (or RAM) can be lost upon a crash but not anything on the disk.

Considering this fact, there exists a background utility process in PostgreSQL called WAL Writer that continuously writes the WAL records from memory to WAL (Write-Ahead logs) segments on the disk. The backend may also flush the WAL buffers to WAL segments. What this means is that a transaction is said to be successful only when it is written to a WAL segment on the disk. Of course, this default behavior can always be altered.

In the following Streaming Replication Architecture diagram, we see an example of how some blocks/buffers have become dirty blocks because of an UPDATE statement. Dirty buffers or dirty blocks are the modified buffers due to the changes to the tuples (or records) in that block.

Setting up Streaming Replication in PostgreSQLAs seen in the Streaming Replication Architecture diagram above, A WAL segment contains WAL records that include the changes done to data blocks (in the form of DML or DDL).

When we setup streaming replication, a new background process called WAL sender is started on the master. This process accepts requests from a standby and streams the WAL records continuously to its standby. Similarly, a background process called WAL receiver is started on the standby, that receives and applies these changes. More internals on Streaming Replication protocol can be seen here.

Let us now see the detailed steps involved in setting up streaming replication.

Steps to setup streaming replication in PostgreSQL

Step 1 : Create a replication user who has a replication role.
(Step to be executed on the master)

$ psql -p 5432 -c "CREATE USER replicator WITH REPLICATION ENCRYPTED PASSWORD 'secret'"

Step 2 : Allow the standby’s replication user to connect to the Primary. See this documentation on pg_hba.conf more details.
(Step to be executed on the master)

$ vi /var/lib/pgsql/13/data/pg_hba.conf

>>> Add entry

host  replication          replicator         192.169.12.1/32         md5

Replace 192.169.12.1 with the IP address of the standby that should be allowed to make replication connections to the master. 

Step 3 : Perform a reload to get the changes into effect.
(Step to be executed on the master)

$ psql -c "select pg_reload_conf()"

Step 4 : Following are the list of configuration parameters that enable us to perform a robust replication setup for production databases.
(Step to be executed on the master)

Please Note : Some of these parameters are not mandatory for streaming replication but only for a good setup. 

  • max_wal_senders : Number of WAL sender processes that can be started on the master. One for each replica. pg_basebackup may use one or 2 Wal senders which will be discussed later in a separate blog post. max_wal_senders generally defaults to 10 on all distributions. So, this parameter need not be modified unless we set up more than 5 standby to a master.
 ALTER SYSTEM SET max_wal_senders TO '10';
  • listen_addresses : This parameter determines the IP interfaces through which the connections are allowed. It defaults to localhost. It may be set to the public or a private IP interface or through everything using (*).
 ALTER SYSTEM SET listen_addresses TO '*';
  • archive_mode : This parameter must be set to ‘ON’ to enable archiving. When archiving is enabled, the WAL segments are copied to the archive location, before getting recycled.
    (archive_mode is not a mandatory parameter but suggested for production databases)
ALTER SYSTEM SET archive_mode TO 'ON';
  • archive_command : The shell command or a script using which the archiver should perform archiving of a full WAL segment. Following is an example setting that uses the cp command in linux. 
    For example,
ALTER SYSTEM SET archive_command TO 'cp %p /archives/%f';

%p is substituted by postgres with the path to the actual WAL segment.
%f  is substituted by postgres with the WAL file name.

(archive_command is not a mandatory parameter but suggested for production databases).
This parameter has no effect on standby unless archive_mode is set to ‘always’.

  • restore_command : The shell command or a script using which a standby copies a WAL segment from the archive location, when the WAL segment is not available with master (when a WAL segment is recycled upon its retention on the master). This parameter has no effect on master. This is only required on a standby or while performing a point-in-time recovery using a full backup and archives.
    For example,
ALTER SYSTEM SET restore_command TO 'cp /archives/%f %p';
  • wal_keep_size : Amount of WAL segments that are to be at least retained in the pg_wal directory before getting recycled.
ALTER SYSTEM SET wal_keep_size TO '1GB';
  • wal_compression : This is not a mandatory parameter but when it is set to ‘ON’, every full page before being written to a WAL segment will be compressed. This reduces the WAL writes and saves some IOPS on the disk.
ALTER SYSTEM SET wal_compression TO 'ON';

Step 5 : Reload or restart as necessary, upon the parameter changes.
(Step executed on master)

$ psql -p 5432 -c "select pg_reload_conf()"

$ psql -p 5432 -c "select name, setting, pending_restart from pg_settings where pending_restart IS true"

Restart if necessary. PGDATA is the environment variable for the Data Directory.

$ pg_ctl -D $PGDATA restart -mf

Step 6 : Take a full backup of the master to create a copy of the data directory on the standby. Replace the IP address with the IP or hostname of master as applicable.
(Step to be executed on the standby, if backup of the master is being taken remotely from the standby server)

$ pg_basebackup -h 192.169.12.1 -p 5432 -U replicator -D /var/lib/pgsql/13/data -Fp -R -Xs -P
  • -Fp : Plain copy of all sub-directories and their contents (datafiles, etc).
  • -R : Configure replication specific settings automatically in the postgresql.auto.conf file. 
  • -Xs : Using a separate channel/process, stream ongoing changes (WAL records) from master to standby, while the backup is in progress. 
  • -P : Show the progress of the backup. 
  • -c fast : This flag may be used to perform fast checkpoint and to avoid waiting until the lazy checkpoint is completed.

Step 7 : Validate if the following parameters are set. Else, set them appropriately on the standby.
(Validation to be performed on the standby)

primary_conninfo : When the backup is taken with -R flag as seen in step 6, primary_conninfo parameter is automatically added to the postgresql.auto.conf file in the data directory (seen in following log). A parameter set in this file is considered by postgres over postgresql.conf or other configuration files as part of include_dir.

$ cd $PGDATA

$ grep primary_conninfo postgresql.auto.conf 
primary_conninfo = 'user=replicator password=secret channel_binding=prefer host=192.169.12.1 port=5432 sslmode=prefer sslcompression=0 ssl_min_protocol_version=TLSv1.2 gssencmode=prefer krbsrvname=postgres target_session_attrs=any'

This parameter helps standby with the connection details of its master. Standby uses the user, host, port and other details added to this configuration parameter to start streaming WAL records from its master over the streaming replication protocol.

restore_command : This is the command used by the standby to restore a WAL segment from the archives, when the required WAL segment was not available with its master. This generally happens when the WAL segment has not been streamed by standby but the WAL segment has been recycled by the master. We could also use physical replication slots to avoid the master from recycling a WAL segment unless it was confirmed to be received by the standby. We need to ensure that the archive directory or server is accessible by the standby using the shell command or the script specified to the restore_command.

ALTER SYSTEM SET restore_command TO 'cp /archives/%f %p';

We can see more details on the above parameters in this documentation on configuration parameters for postgres streaming replication.

Step 8 : Create a signal file to mention that the server is a standby server. The standby.signal file must be created inside the data directory of the standby.
(Step to be executed on the standby)

$ touch /var/lib/pgsql/13/data/standby.signal

Step 9 : Start the standby to enable replication.
(Step to be executed on the standby)

$ pg_ctl -D /var/lib/pgsql/13/data start

Step 10 : Validate replication between master and standby using the following command on the master.
(Validation to be performed on the master)

$ psql -x -c "select * from pg_stat_replication"

The output looks like following –

$ psql -x -c "select * from pg_stat_replication"
-[ RECORD 1 ]----+------------------------------
pid | 28435
usesysid | 16384
usename | replicator
application_name | walreceiver
client_addr | 192.169.12.1
client_hostname | 
client_port | 49806
backend_start | 2021-03-31 12:39:22.876997+00
backend_xmin | 
state | streaming
sent_lsn | 0/3000148
write_lsn | 0/3000148
flush_lsn | 0/3000148
replay_lsn | 0/3000148
write_lag | 00:00:00.126533
flush_lag | 00:00:00.127325
replay_lag | 00:00:00.127386
sync_priority | 0
sync_state | async
reply_time | 2021-03-31 12:39:23.021395+00

So far, we have seen the steps involved in setting up Streaming Replication in PostgreSQL 13. In our future blogs, we shall see how to set up Synchronous replication and also see how we can perform a failover and a rewind of the demoted master to add it back to replication from the new master. Subscribe to our blog posts to stay up-do-date. Contact us to know more about how we can help you tune and set up a robust PostgreSQL production environment to help you migrate to PostgreSQL

3 thoughts on “Setting up Streaming Replication in PostgreSQL 13 and Streaming Replication Internals”

Leave a Comment

Your email address will not be published.

Scroll to Top