Use logical replication
Let Debezium, Airbyte or another PostgreSQL read the changes of your database through publications and replication slots.
Logical replication (CDC, CREATE SUBSCRIPTION) is off by default. Replace ewdb-0123456789ab
with your database's id and appdb with the database the slot decodes.
Important
Turning logical replication on or off restarts the database: a cluster restarts member by member with one switchover (a few seconds of failed writes); a single node restarts once (writes pause until it is back).
Prerequisites
- Permission for
rds:ModifyDBInstanceto turn it on,rds:CreateDBReplicationSlotandrds:DeleteDBReplicationSlotfor slots. - The database is available, for turning it on or off and for adding a slot. Otherwise the
request is refused with
409. - In the console, also
rds:DescribeDBInstancesto open the database's page (without it the page is not found).
Step 1: Turn on logical replication
You can also turn it on when creating the database (logicalReplication: true, Logical
replication in the create form).
- Open the database, then choose the Nodes tab.
- Under Logical replication, turn Logical replication on.
- In the browser's confirmation, which names the restart, choose OK.
dcloud rds replication set ewdb-0123456789ab --logical-replicationPATCH /rds/api/instances/ewdb-0123456789ab/replication
{ "logicalReplication": true }resource "dcloud_db_instance" "orders" {
identifier = "orders"
instance_class = "performance-2"
logical_replication = true
}This sets wal_level = logical. Activity (phase replication) shows the progress.
Step 2: Create a slot
- On the Nodes tab, under Logical replication, enter the slot name (for example
debezium) and the database (prefilled with the database's initial database). - Choose Add slot (pgoutput).
dcloud rds replication-slots create ewdb-0123456789ab --name debezium --database appdbPOST /rds/api/instances/ewdb-0123456789ab/replication-slots
{ "name": "debezium", "database": "appdb" }resource "dcloud_db_replication_slot" "debezium" {
db_instance_id = dcloud_db_instance.orders.id
name = "debezium"
database = "appdb"
}The plugin is pgoutput. A slot name uses a-z, 0-9 and _, starts with a letter and has at
most 63 characters. A slot is refused with 409 "Turn logical replication on first …" while it is
off, "Logical replication is still being switched on; try again once it is." during the restart,
"A slot named … already exists." for a duplicate, and 400 for a name the platform uses for its
own slots or a database that does not exist. A database holds at most four declared slots (a fixed
quota). On a cluster the platform keeps a declared slot on every member, so it
survives a failover and your consumer resumes from where it was. On PostgreSQL 17 and later, a slot
you create yourself with failover = true (CREATE SUBSCRIPTION … WITH (failover = true),
Debezium slot.failover=true) survives too; other slots made in SQL are lost on a failover.
Step 3: Connect the consumer
Connect the consumer to port 5432 (direct; the pooler on 6432 does not carry replication), with
a role that has REPLICATION, such as the master user, and CREATE PUBLICATION for the tables you
want. Delivery is at-least-once: after a failover a consumer can receive again some changes it
already had.
Delete a slot
- On the Nodes tab, under Logical replication, choose the delete icon on the slot's row.
- In the browser's confirmation, choose OK. A slot in use is deleted with its consumer disconnected, as the confirmation says.
dcloud rds replication-slots delete ewdb-0123456789ab debezium --force trueDELETE /rds/api/instances/ewdb-0123456789ab/replication-slots/debeziumRemove the dcloud_db_replication_slot resource and run terraform apply.
Through the CLI and API, a slot a consumer is still reading is refused with 409 "Slot … is in
use (pid …). Stop its consumer, or delete with force=true to end the connection reading it."
unless you pass force=true. The platform's own slots cannot be deleted (400). Whatever read
the slot has to start from a new snapshot.
Result
GET /rds/api/instances/{id}/replication-slots lists every logical slot on the leader, declared or made in SQL:
whether it is in use, how much WAL it holds, whether it survives a failover.
An unused slot keeps WAL on the leader until it is consumed. At 1 GiB or 10 % of the volume the database's health says so (Activity and mail); past the cap (a fifth of the volume, at least 1 GiB) PostgreSQL invalidates the slot and its consumer has to start from a new snapshot. Delete a slot you no longer need. Turning logical replication off is refused while any logical slot exists.
Related
- Quotas: replication slots per database.
POST /rds/api/instances/{id}/replication-slots·dcloud rds replication-slots create·dcloud_db_replication_slot