Monitoring
Metrics of every machine, alarms that mail or call a webhook, and app logs.
What it is
Metrics, alarms and logs for the machines your resources run on, and uptime checks on any URL. Every compute instance, every managed database node and every app machine records CPU, disk and network once a minute, with nothing installed on the machine. An alarm watches one of those metrics and mails you or calls a webhook when it crosses a threshold. A deployed app's containers ship what they print, and you search it from the console. Open Monitoring in the service switcher; the charts are also on each instance, database and app page.
Concepts
- Metric — one measurement of one machine over time, recorded once a
minute:
cpu_percent,disk_read_iops,disk_write_iops,disk_read_bps,disk_write_bps,network_in_bps,network_out_bps. A database node also recordsdisk_used_percent, the fill level of its data volume, as the node reports it every few minutes. - Resource — what a metric belongs to: an
instance(a compute instance, including the machine an app runs on) or adb_node(one machine of a managed database; a cluster has one per member). Anapprecords its deployments: one point ofdeploy_duration_seconds(queued to live or failed) and one ofdeploys_succeededordeploys_failed(value 1) when each deployment finishes. An alarm ondeploys_failedwith statisticmax, threshold>= 1, a 30-minute period and 2 evaluation periods fires when deploys failed in two half-hours in a row. - Period and statistic — a chart or an alarm groups the per-minute values into periods and takes their average, maximum, minimum, sum or sample count.
- Custom metric — a series your own code publishes: a namespace, a metric name and dimensions. See Custom metrics.
- Alarm — a metric of one resource, a statistic, a comparison, a threshold,
a period and a number of periods. Its state is
OK,ALARMorINSUFFICIENT_DATA. - Log line — one line an app's container wrote to stdout or stderr, with
the time the container wrote it. Its stream is
<service>/stdoutor<service>/stderr; a single-image app's service isapp, a compose app's is the compose service name.
How to
Read a machine's metrics
Open the instance, database or app page and scroll to Monitoring. Pick a range from 1 hour to 14 days; a database shows one chart set per node. Over the API:
GET /monitoring/api/metrics/data?resourceType=instance&resourceId=i-1a2b3c4d
&metrics=cpu_percent,network_in_bps&start=2026-10-29T10:00:00Z&period=300&statistic=maxGET /monitoring/api/resources lists every resource that records metrics, with
the metrics each one has.
Create an alarm
- Monitoring → Alarms → Create alarm.
- Choose the machine and the metric, then the condition: the statistic, the comparison, the threshold, the period and for how many periods it must hold.
- Optionally add email addresses (up to ten) and an https webhook URL.
Over the API, POST /monitoring/api/alarms:
{
"name": "web-1 CPU high",
"resourceType": "instance",
"resourceId": "i-1a2b3c4d",
"metric": "cpu_percent",
"statistic": "avg",
"comparator": ">",
"threshold": 80,
"periodSeconds": 300,
"evaluationPeriods": 3,
"notifyEmails": ["ops@example.com"],
"webhookUrl": "https://hooks.example.com/ews"
}The CLI: dcloud monitoring alarms create.
Alarms
An alarm is evaluated every minute over its last evaluationPeriods whole
periods, ending two minutes before now so the newest minute has arrived:
| Datapoints in the evaluated periods | New state |
|---|---|
| None | INSUFFICIENT_DATA |
| At least one does not cross the threshold | OK |
| Every period has one, and every one crosses | ALARM |
| Some periods are empty, every one present crosses | unchanged |
Notifications go out when an alarm enters ALARM and when it leaves it. A new
alarm settling into OK, or a quiet machine moving between OK and
INSUFFICIENT_DATA, notifies nobody. The History of an alarm lists every
state change, its reason and what was sent.
The webhook receives a JSON POST:
{
"type": "ews.monitoring.alarm.state-change",
"alarm": { "id": "…", "name": "web-1 CPU high", "metric": "cpu_percent", "threshold": 80, "…": "…" },
"previousState": "OK",
"state": "ALARM",
"reason": "Threshold crossed: 3 datapoint(s) [91.20, 95.00, 97.40] were greater than the threshold (80).",
"at": "2026-10-29T12:08:00.000Z"
}A 2xx answer counts as delivered. Anything else, a redirect or no answer within 5 seconds is recorded as a failure in the history and not retried.
Stop an alarm's notifications
Alarms → bell icon on the alarm, or PUT /monitoring/api/alarms/{id} with
{ "enabled": false }. The alarm keeps being evaluated and its state stays
current; only the mail and the webhook stop.
Change an alarm
PUT /monitoring/api/alarms/{id} with the fields to change. Changing the
resource, the metric, the statistic, the comparison, the threshold or the
periods resets the alarm to INSUFFICIENT_DATA; changing its name or its
recipients does not.
Delete an alarm
Alarms → trash icon on the alarm, or DELETE /monitoring/api/alarms/{id}.
The alarm and its history are removed. The machine is not affected.
Uptime checks
An uptime check requests a URL on a schedule — any public http:// or
https:// address, not only what runs on Deutsche Cloud — and records whether it
answered as expected and how long the answer took. Monitoring → Uptime
lists every check with its state; a check's page shows its uptime, the
response time over the last hour to 30 days, its outage log and its latest
results.
A check is pending until its first run, then up, regional (see
Locations and confirmation) or down;
paused while it is paused. One run is one request, retried once after 3 seconds if it
fails, so a single dropped connection does not count. A run fails when:
- nothing answers within the timeout, the name does not resolve, the connection is refused or reset, or the certificate is invalid;
- the final status (after up to five redirects) is outside the expected range, 200–399 unless you set another;
- for a keyword check, the text is missing from (or, with
not_contains, present in) the first 1 MiB of the body.
The check goes down after the number of failed runs in a row you choose (two by default), and up again after two successful runs in a row. While a failure or a recovery is unconfirmed, the next run comes a minute later rather than after the full interval. Going down and coming back up are each mailed to the check's addresses and POSTed to its webhook, once per outage; the outage log records when it started, when it ended, why, and who was told.
The requests carry the user agent
EWS-Uptime/1 (+https://console.platform-development.deutschecloud.app/docs/monitoring#uptime-checks). A run
with no result — the platform itself was down — is neither up nor down, and
nobody is notified about it.
Locations and confirmation
Checks run from more than one place: the platform's control plane in Germany
(location control-plane) and small probe machines in other Hetzner
locations (Helsinki, …). Each run asks every location at the same time, and
the run counts as failed only when at least confirmations locations fail
together (two by default; never more than the locations that answered). When
fewer fail, the check is regional — up, with a network problem between one
location and your site — and nobody is notified, unless the check sets
alertOnRegional, which makes one failing location enough. The outage log and
the down mail name the locations that confirmed it; the check's page shows the
latest result from each location.
GET /monitoring/api/locations lists the locations, whether each is healthy
and the addresses its requests come from, for a firewall allowlist. A probe
that stops reporting is left out of new runs within two minutes, and the vote
counts only the locations that answered — a dead probe never blocks a
decision. The control plane does not check Deutsche Cloud's own addresses; checks on
EWS-hosted URLs are judged by the probes alone.
Create an uptime check
- Monitoring → Uptime → Create check.
- Give it a name and the URL. Choose Keyword to also require a text in the response body.
- Choose the interval (1 minute to 1 hour), after how many failed runs it is down, and the timeout.
- Optionally add up to ten email addresses and an https webhook.
Over the API:
POST /monitoring/api/checks
{ "name": "Shop", "url": "https://shop.example.com/health", "intervalSeconds": 60,
"notifyEmails": ["oncall@example.com"] }The CLI: dcloud monitoring checks create. The webhook body is
{ "type": "ews.monitoring.uptime.state-change", "check": {…}, "state": "DOWN" | "UP", "reason", "at", "downSince" },
under the same rules as an alarm's webhook.
Read a check's results
The check's page, or GET /monitoring/api/checks/{id}/results?start=…&end=…,
which answers the uptime percentage of the window (the share not covered by an
outage), the share of runs that succeeded, the average and 95th-percentile
response time, about 300 points of response time over the window, and the
latest raw results. GET /monitoring/api/checks/{id}/outages is the outage
log.
Pause an uptime check
Uptime → ⋯ → Pause, the Pause button on the check's page, or
POST /monitoring/api/checks/{id}/pause. A paused check sends no requests
and notifies nobody; an outage that was open is closed without a mail.
Resume (…/resume) starts it over as pending.
Change an uptime check
PATCH /monitoring/api/checks/{id} with the fields to change, or
dcloud monitoring checks update. Changing the URL, the type, the expected
status, the keyword, redirects or TLS verification starts the check over as
pending; changing its name, interval or recipients does not.
Delete an uptime check
Uptime → ⋯ → Delete check, the Danger zone on the check's page, or
DELETE /monitoring/api/checks/{id}. The check, its results and its outage
log are removed.
Custom metrics
Your own numbers — orders placed, queue depth, request latency — published by
your code and charted, kept and alarmed on like the built-in metrics. A
series is a namespace, a metric name and up to ten dimensions
(route=/pay, region=eu): the same name with other dimensions is another
series. Values are aggregated per minute on arrival, so every statistic —
average, sum, minimum, maximum and sample count — is exact whatever you send.
Open Monitoring → Custom metrics to chart them.
There are three ways in, all under the same limits.
The API. POST /monitoring/api/custom-metrics, needing
monitoring:PutMetricData:
{
"namespace": "Shop",
"metricData": [
{ "metricName": "orders_placed", "value": 1, "dimensions": { "region": "eu" } },
{ "metricName": "checkout_ms", "values": [120, 95, 310], "unit": "Milliseconds" },
{
"metricName": "batch_size",
"timestamp": "2026-11-10T12:00:00Z",
"statistics": { "count": 40, "sum": 812, "min": 3, "max": 61 }
}
]
}Each datum has exactly one of value, values (with optional counts) or
statistics. timestamp (ISO 8601 or epoch milliseconds) defaults to now. The
answer lists what was stored and every datum that was not, by index:
{ "accepted": 2, "rejected": [{ "index": 2, "reason": "timestamp: more than 14 days old" }] }.
Only a request where nothing was stored is a 400. The CLI takes one
--metric-data per datum:
dcloud monitoring custom-metrics put --namespace Shop --metric-data '{"metricName":"orders_placed","value":1}'.
OpenTelemetry. Point an OTLP/HTTP metrics exporter at
https://{console}/monitoring/api/otlp with the JSON encoding and your access
key as a header. For the OpenTelemetry Collector:
exporters:
otlphttp/ews:
metrics_endpoint: https://{console}/monitoring/api/otlp/v1/metrics
encoding: json
headers:
Authorization: 'EWS-ACCESS-KEY ${env:EWS_ACCESS_KEY_ID}:${env:EWS_SECRET_ACCESS_KEY}'An SDK without a collector: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT set to the
same URL, OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/json. The namespace is the
resource's service.namespace, else service.name, else otel; a data
point's attributes become dimensions (the first ten by name). Gauges and sums
become a value each, histograms their count, sum, minimum and maximum. Prefer
delta temporality (OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta):
a cumulative counter is stored as the running total it reports. Protobuf is
answered with 415.
StatsD from an app's containers. The agent on every app machine listens on
UDP port 8125 for the containers on that machine (nothing else can reach it).
Send to host.containers.internal:8125. Counters (c, with @rate), gauges
(g, +n/-n adjust), timers and histograms (ms, h, d) and sets (s)
are aggregated for a minute and stored under the namespace App/<app name>,
with an instance dimension naming the machine; DogStatsD tags (|#k:v)
become dimensions. A counter's per-minute total is its sum, a timer's
average its avg. A gauge is re-sent every minute for ten minutes after its
last update.
echo "orders_placed:1|c|#region:eu" | nc -u -w0 host.containers.internal 8125Read series over the API with GET /monitoring/api/custom-metrics?namespace=Shop
and their datapoints with
GET /monitoring/api/custom-metrics/data?seriesId=…&start=…&period=300&statistic=sum.
Alarms on custom metrics
An alarm on a custom series names it, rather than a machine: resourceType
custom, resourceId the namespace, metric the metric name and
dimensions exactly the series' dimensions.
{
"name": "Checkout slow",
"resourceType": "custom",
"resourceId": "Shop",
"metric": "checkout_ms",
"dimensions": {},
"statistic": "avg",
"comparator": ">",
"threshold": 500,
"periodSeconds": 300,
"evaluationPeriods": 2,
"notifyEmails": ["ops@example.com"]
}The series need not exist yet: until it has datapoints in the evaluated
periods the alarm is INSUFFICIENT_DATA. Missing data never breaches, so an
alarm cannot fire on a series that stops arriving altogether — publish a zero
(orders_placed with value: 0) when there is nothing to count, and alarm on
sum < 1. The CLI: dcloud monitoring alarms create --resource-type custom ….
Status pages
A status page tells your users whether your service works, at
https://{slug}.status.{root} — a public page with no sign-in, served by Deutsche Cloud.
Monitoring → Status pages → Create a status page asks for the address
(slug: 3–63 lower-case letters, digits and dashes) and a title; a few words
(dcloud, www, status, api, …) are reserved. At most five pages per
organization.
A page lists components. Each one either follows an uptime check — up is Operational, regional is Degraded performance, down is Major outage, paused or not yet run is No data — or has a state you set by hand. Components with the same group are shown together. Under each component that follows a check, 90 bars show its uptime per day (from the check's outage log) and the 90-day percentage. The banner at the top is the worst state on the page.
An incident is what you tell your users: a title, an impact (none,
minor, major, critical), a status (investigating, identified,
monitoring, resolved), the components it affects and a timeline of plain-text
updates. While it is open, it makes its components at least as bad as its
impact (minor → degraded, major → partial outage, critical → major outage).
Resolved incidents stay on the page for 14 days.
POST /monitoring/api/status-pages/{id}/incidents
{ "title": "Payments failing", "impact": "major", "body": "We are investigating.", "componentIds": ["…"] }
POST /monitoring/api/status-pages/{id}/incidents/{incidentId}/updates
{ "status": "resolved", "body": "Fixed: a certificate had expired." }Subscribers hear about every incident posted, updated or resolved, and about a component that follows a check going down and coming back:
- Email. Visitors enter an address on the page. Nothing is sent but a confirmation link until they open it; unconfirmed addresses are forgotten after a day. Every mail has a one-click unsubscribe link. You can invite an address from the console too — it confirms the same way.
- Webhook. Added by you (
kind: webhook, an https URL on a public address). It receives a JSONPOSTwithtypeews.monitoring.status-page.incident.created|updated|resolvedorews.monitoring.status-page.component.down|up, the page, and the incident or component.
At most 1000 subscribers per page; SMS is not offered yet. The page's data is
also served as JSON at https://{slug}.status.{root}/summary.json.
The status pages run on the same platform as the checks. Deutsche Cloud's own status
is served separately, from the probe machines outside the control plane, at
https://status.{root}, so it keeps answering when the console does not.
The CLI: dcloud monitoring status-pages create --slug acme --title "Acme status",
dcloud monitoring incidents create, dcloud monitoring subscribers add.
Read an app's logs
Open the app page and scroll to Logs, or Monitoring → Logs and pick the app. Lines appear about ten seconds after the container writes them, newest first. Type in the filter box to show only lines containing that text (case-insensitive). Over the API:
GET /monitoring/api/logs?resourceId=<app id>&contains=error&limit=500start defaults to a day before end, end to now.
Reference
| Limit | Value |
|---|---|
| Metric resolution | 1 minute |
| Metric retention | 14 days |
| Points per series in one answer | 1440 (widen period for longer ranges) |
| Metrics per data request | 8 |
| Alarm period | 1 minute to 1 day, whole minutes |
| Alarm look-back (period × periods) | at most 1 day |
| Email recipients per alarm | 10 |
| Webhook | https, port 443, public address, 5 s |
| Log retention | 7 days |
| Log line length | 16 KiB; longer lines end …[truncated] |
| Log lines per search | 1000 |
| Uptime checks per organization | 50 |
| Uptime check interval | 1, 2, 5, 10, 15, 30 or 60 minutes |
| Uptime check timeout | 1–30 s, 10 s by default |
| Runs against one host, all of Deutsche Cloud | 6 a minute, over every organization |
| Requests per run | one per location (2–3), each retried once |
| Uptime check ports | any except 25, 465 and 587 |
| Uptime results retention | 30 days; outages for the check's life |
| Custom metric resolution | 1 minute |
| Custom metric retention | 15 days per minute, 15 months per hour |
| Custom series per organization | 1000 with a datapoint in the last 15 d |
| Custom datums per request | 1000, body 1 MiB |
| Custom datums per minute | 20000 per organization |
| Custom datum age | at most 14 days old, 2 hours ahead |
| Dimensions per series | 10; names and values up to 255 chars |
cpu_percentis the provider's figure: 100 is one vCPU fully busy, so a machine with four vCPUs reaches 400.- Disk and network metrics are the machine's own disk and its public
interface. A database node's data volume is covered by
disk_used_percent, not by the disk metrics. - A stopped machine records nothing; its chart shows a gap, not zero.
- An app that prints faster than its machine can ship drops lines past 5000 waiting to be sent, and the app's activity feed says how many.
| Action | Grants |
|---|---|
monitoring:ListMetrics |
the resource list and a resource's metrics |
monitoring:GetMetricData |
the charts and their datapoints |
monitoring:PutMetricData |
publishing custom metrics (API, OTLP) |
monitoring:DescribeAlarms |
the alarm list and one alarm |
monitoring:DescribeAlarmHistory |
an alarm's history |
monitoring:PutMetricAlarm |
creating and changing an alarm |
monitoring:DeleteAlarms |
deleting an alarm |
monitoring:FilterLogEvents |
reading app logs |
monitoring:DescribeChecks |
the uptime check list and one check |
monitoring:GetCheckResults |
a check's results, response times, outages |
monitoring:CreateCheck |
creating an uptime check |
monitoring:UpdateCheck |
changing an uptime check |
monitoring:PauseChecks |
pausing an uptime check |
monitoring:ResumeChecks |
resuming a paused uptime check |
monitoring:DeleteChecks |
deleting an uptime check |
monitoring:DescribeStatusPages |
status pages, incidents and subscribers |
monitoring:CreateStatusPage |
creating a status page |
monitoring:UpdateStatusPage |
its title, address, components, webhooks |
monitoring:DeleteStatusPage |
deleting a status page |
monitoring:CreateIncident |
posting an incident |
monitoring:UpdateIncident |
posting an update, resolving it |
monitoring:DeleteIncident |
removing an incident |
Alarm actions can be granted on one alarm, rn:ews::{orgId}:monitoring:alarm/{id},
check actions on one check, rn:ews::{orgId}:monitoring:check/{id}, and status
page and incident actions on one page,
rn:ews::{orgId}:monitoring:status-page/{id} — so someone on call can post
incidents on one page and change nothing else. Metric and log reads cover every
resource of the organization.
What is not supported
- Metrics from inside a machine (memory, filesystem usage of an instance, process counts). Only what the hypervisor sees is recorded.
- Percentiles, sub-minute resolution and metric math on custom metrics; OTLP over protobuf or gRPC; deleting a custom series (it expires 15 months after its last datapoint).
- Treating missing data as breaching, or any missing-data rule other than "missing".
- Logs from compute instances you run yourself, and from managed databases.
- Alarms on a whole database or app rather than on one machine.
- Alarms on a combination of metrics, or on anomalies.
- Re-sending a failed webhook.
- Choosing which locations a check runs from, locations outside Hetzner, TCP, ping, heartbeat and DNS checks, custom request headers or bodies, methods other than GET and HEAD, and intervals under a minute.
- Custom domains and private (password or login) status pages, maintenance windows, Markdown or logos on a status page, and alerts or status updates by SMS.
- Confirming an alert recipient's address before it is mailed.
- Changing an uptime check's settings in the console (use the API or the CLI).
Troubleshooting
- A chart says "No datapoints in this range yet." The machine is stopped, was created in the last two minutes, or belongs to your own Hetzner project whose token has stopped working (Settings → Infrastructure says so).
- An alarm stays in Insufficient data. The machine records no points in
the evaluated periods — it is stopped, or the alarm's period is shorter than
the metric's interval (
disk_used_percentarrives every few minutes; use a period of 15 minutes or more). - The webhook shows "refusing to follow a redirect". The URL answers with a 3xx. Use the final URL.
- The webhook shows "resolves to a private or reserved address". The host name points at a private network. Webhooks only reach public addresses.
- An app's logs are empty. The app writes its logs to a file instead of stdout and stderr, or its machine's agent predates log shipping; the agent updates itself within a few minutes of a platform update.
- Lines from before a restart of the machine's agent are missing. The
agent ships what containers print from the moment it starts; earlier output
stays on the machine (
podman logs).