Monitoring Managed PostgreSQL
Metrics and alarms, replication lag, the health problems you are mailed about, and the Activity stream of every database.
Metrics and alarms
Every tab below is on the database's page, which needs rds:DescribeDBInstances to open (without it
the page is not found).
- Monitoring on the database page charts each node's connections and replication lag (every two
minutes), beside CPU, disk and network. Reading the charts needs
monitoring:GetMetricData, and the alarmsmonitoring:DescribeAlarms. - Alarms can use
connections,replication_lag_bytesandquery_slowdown.query_slowdownis how many times slower the most slowed-down statement ran in the last 15 minutes than over its own previous 7 days; see Find slow queries. - Nodes shows each replica's lag behind the leader, live, the PostgreSQL version each node runs, who leads now and whether synchronous replication is holding.
- Queries is query insights; see Find slow queries. The top statements
by time, calls or rows are also in
GET /rds/api/instances/{id}/query-stats. - Logs is the server log; see Read the server log.
Health problems you are mailed about
The platform checks every database every two minutes. A problem that lasts past its grace period
is written to Activity (phase health) and mailed to your organization's owners and admins,
again at most once a day while it stays open; another mail follows when everything you were told
about has resolved, and Activity records "Resolved: …".
| Problem | Grace |
|---|---|
| The database is unreachable | 3 minutes |
| The cluster has no leader | 2 minutes |
| No standby is replicating | 5 minutes |
| A replica is not replicating or is missing | 5 minutes |
| Synchronous replication is degraded | 2 minutes |
| A replica is 512 MB or more behind | 5 minutes |
| The pooler (port 6432) does not accept connections while PostgreSQL does | 5 minutes |
| Disk 85 % (warning) / 95 % (critical) full | none |
| WAL archiving failing for 10 minutes | none |
| An inactive replication slot holds 1 GiB or more, or 10 % of the volume | none |
| A replication slot was invalidated | none |
pg_wal holds 40 % of the volume or more |
none |
90 % of max_connections in use |
none |
| Transaction ID age at 1.2 billion (warning) / 1.8 billion (critical) | none |
| A transaction open for 6 hours or more | none |
| The server certificate expires within 14 days, or has expired | none |
An outage shorter than its grace period is not mailed. Separately, a daily sweep writes a tls
event to Activity once a day from 30 days before the server certificate expires, as an error
inside 7 days.
Activity
Activity is the event stream of a database: every step of provisioning, every certificate
action, every storage change, written by the machine itself or by the platform. Both the console
and the API show the newest 200 events, oldest first (the API takes limit up
to 500). To read what is new, pass since, a timestamp: only events after it are returned. The
console tab refreshes every three seconds while the database is creating, modifying, deleting or
backing up.
- Open the database, then choose the Activity tab.
dcloud rds instances events ewdb-0123456789ab --limit 100GET /rds/api/instances/ewdb-0123456789ab/events?limit=100Not available in Terraform. Events are read, not configured.
| Phase | Written by | Means |
|---|---|---|
provision |
platform | machines, network, address and certificate being created |
bootstrap |
machine | PostgreSQL and the pooler being installed and configured |
ready |
machine | the ports answer |
storage |
platform | a resize or an autoscaling step |
resize |
platform | a machine type change, node by node |
readers |
platform | a reader requested, added or removed |
repair |
platform | a lost machine rebuilt, or a stuck replica re-cloned |
reboot |
platform | a cluster reboot, machine by machine |
upgrade |
platform | a minor or major version upgrade, node by node |
replication |
platform | logical replication turned on or off |
parameters |
platform | parameters set or reset, and what waits for a restart |
logs |
machine | server log skipped because it was written too fast |
tls |
both | a certificate renewed, or one about to expire |
health |
platform | a health problem past its grace, or resolved |
failover |
platform | the leader moved: to which node, and what may be lost |
credentials |
platform | the master password rotated, or a rotation failed |
backup |
machine | a backup started, finished or failed |
extensions |
platform | an extension enabled or disabled, or a preload change |
logins |
platform | a login created, revoked, rotated, re-dated or expired |
settings |
platform | deletion protection or the TLS requirement changed |
reconcile |
platform | a resource with no owner was removed |
delete |
platform | teardown |
The statuses a database can be in are listed in Instances, clusters and endpoints.