Monitoring

Metrics of every machine, alarms that mail or call a webhook, and app logs.

What it is

Metrics, alarms and logs for the machines your resources run on, and uptime checks on any URL. Every compute instance, every managed database node and every app machine records CPU, disk and network once a minute, with nothing installed on the machine. An alarm watches one of those metrics and mails you or calls a webhook when it crosses a threshold. A deployed app's containers ship what they print, and you search it from the console. Open Monitoring in the service switcher; the charts are also on each instance, database and app page.

Concepts

  • Metric — one measurement of one machine over time, recorded once a minute: cpu_percent, disk_read_iops, disk_write_iops, disk_read_bps, disk_write_bps, network_in_bps, network_out_bps. A database node also records disk_used_percent, the fill level of its data volume, as the node reports it every few minutes.
  • Resource — what a metric belongs to: an instance (a compute instance, including the machine an app runs on) or a db_node (one machine of a managed database; a cluster has one per member). An app records its deployments: one point of deploy_duration_seconds (queued to live or failed) and one of deploys_succeeded or deploys_failed (value 1) when each deployment finishes. An alarm on deploys_failed with statistic max, threshold >= 1, a 30-minute period and 2 evaluation periods fires when deploys failed in two half-hours in a row.
  • Period and statistic — a chart or an alarm groups the per-minute values into periods and takes their average, maximum, minimum, sum or sample count.
  • Custom metric — a series your own code publishes: a namespace, a metric name and dimensions. See Custom metrics.
  • Alarm — a metric of one resource, a statistic, a comparison, a threshold, a period and a number of periods. Its state is OK, ALARM or INSUFFICIENT_DATA.
  • Log line — one line an app's container wrote to stdout or stderr, with the time the container wrote it. Its stream is <service>/stdout or <service>/stderr; a single-image app's service is app, a compose app's is the compose service name.

How to

Read a machine's metrics

Open the instance, database or app page and scroll to Monitoring. Pick a range from 1 hour to 14 days; a database shows one chart set per node. Over the API:

GET /monitoring/api/metrics/data?resourceType=instance&resourceId=i-1a2b3c4d
    &metrics=cpu_percent,network_in_bps&start=2026-10-29T10:00:00Z&period=300&statistic=max

GET /monitoring/api/resources lists every resource that records metrics, with the metrics each one has.

Create an alarm

  1. Monitoring → Alarms → Create alarm.
  2. Choose the machine and the metric, then the condition: the statistic, the comparison, the threshold, the period and for how many periods it must hold.
  3. Optionally add email addresses (up to ten) and an https webhook URL.

Over the API, POST /monitoring/api/alarms:

{
  "name": "web-1 CPU high",
  "resourceType": "instance",
  "resourceId": "i-1a2b3c4d",
  "metric": "cpu_percent",
  "statistic": "avg",
  "comparator": ">",
  "threshold": 80,
  "periodSeconds": 300,
  "evaluationPeriods": 3,
  "notifyEmails": ["ops@example.com"],
  "webhookUrl": "https://hooks.example.com/ews"
}

The CLI: dcloud monitoring alarms create.

Alarms

An alarm is evaluated every minute over its last evaluationPeriods whole periods, ending two minutes before now so the newest minute has arrived:

Datapoints in the evaluated periods New state
None INSUFFICIENT_DATA
At least one does not cross the threshold OK
Every period has one, and every one crosses ALARM
Some periods are empty, every one present crosses unchanged

Notifications go out when an alarm enters ALARM and when it leaves it. A new alarm settling into OK, or a quiet machine moving between OK and INSUFFICIENT_DATA, notifies nobody. The History of an alarm lists every state change, its reason and what was sent.

The webhook receives a JSON POST:

{
  "type": "ews.monitoring.alarm.state-change",
  "alarm": { "id": "…", "name": "web-1 CPU high", "metric": "cpu_percent", "threshold": 80, "…": "…" },
  "previousState": "OK",
  "state": "ALARM",
  "reason": "Threshold crossed: 3 datapoint(s) [91.20, 95.00, 97.40] were greater than the threshold (80).",
  "at": "2026-10-29T12:08:00.000Z"
}

A 2xx answer counts as delivered. Anything else, a redirect or no answer within 5 seconds is recorded as a failure in the history and not retried.

Stop an alarm's notifications

Alarms → bell icon on the alarm, or PUT /monitoring/api/alarms/{id} with { "enabled": false }. The alarm keeps being evaluated and its state stays current; only the mail and the webhook stop.

Change an alarm

PUT /monitoring/api/alarms/{id} with the fields to change. Changing the resource, the metric, the statistic, the comparison, the threshold or the periods resets the alarm to INSUFFICIENT_DATA; changing its name or its recipients does not.

Delete an alarm

Alarms → trash icon on the alarm, or DELETE /monitoring/api/alarms/{id}. The alarm and its history are removed. The machine is not affected.

Uptime checks

An uptime check requests a URL on a schedule — any public http:// or https:// address, not only what runs on Deutsche Cloud — and records whether it answered as expected and how long the answer took. Monitoring → Uptime lists every check with its state; a check's page shows its uptime, the response time over the last hour to 30 days, its outage log and its latest results.

A check is pending until its first run, then up, regional (see Locations and confirmation) or down; paused while it is paused. One run is one request, retried once after 3 seconds if it fails, so a single dropped connection does not count. A run fails when:

  • nothing answers within the timeout, the name does not resolve, the connection is refused or reset, or the certificate is invalid;
  • the final status (after up to five redirects) is outside the expected range, 200–399 unless you set another;
  • for a keyword check, the text is missing from (or, with not_contains, present in) the first 1 MiB of the body.

The check goes down after the number of failed runs in a row you choose (two by default), and up again after two successful runs in a row. While a failure or a recovery is unconfirmed, the next run comes a minute later rather than after the full interval. Going down and coming back up are each mailed to the check's addresses and POSTed to its webhook, once per outage; the outage log records when it started, when it ended, why, and who was told.

The requests carry the user agent EWS-Uptime/1 (+https://console.platform-development.deutschecloud.app/docs/monitoring#uptime-checks). A run with no result — the platform itself was down — is neither up nor down, and nobody is notified about it.

Locations and confirmation

Checks run from more than one place: the platform's control plane in Germany (location control-plane) and small probe machines in other Hetzner locations (Helsinki, …). Each run asks every location at the same time, and the run counts as failed only when at least confirmations locations fail together (two by default; never more than the locations that answered). When fewer fail, the check is regional — up, with a network problem between one location and your site — and nobody is notified, unless the check sets alertOnRegional, which makes one failing location enough. The outage log and the down mail name the locations that confirmed it; the check's page shows the latest result from each location.

GET /monitoring/api/locations lists the locations, whether each is healthy and the addresses its requests come from, for a firewall allowlist. A probe that stops reporting is left out of new runs within two minutes, and the vote counts only the locations that answered — a dead probe never blocks a decision. The control plane does not check Deutsche Cloud's own addresses; checks on EWS-hosted URLs are judged by the probes alone.

Create an uptime check

  1. Monitoring → Uptime → Create check.
  2. Give it a name and the URL. Choose Keyword to also require a text in the response body.
  3. Choose the interval (1 minute to 1 hour), after how many failed runs it is down, and the timeout.
  4. Optionally add up to ten email addresses and an https webhook.

Over the API:

POST /monitoring/api/checks
{ "name": "Shop", "url": "https://shop.example.com/health", "intervalSeconds": 60,
  "notifyEmails": ["oncall@example.com"] }

The CLI: dcloud monitoring checks create. The webhook body is { "type": "ews.monitoring.uptime.state-change", "check": {…}, "state": "DOWN" | "UP", "reason", "at", "downSince" }, under the same rules as an alarm's webhook.

Read a check's results

The check's page, or GET /monitoring/api/checks/{id}/results?start=…&end=…, which answers the uptime percentage of the window (the share not covered by an outage), the share of runs that succeeded, the average and 95th-percentile response time, about 300 points of response time over the window, and the latest raw results. GET /monitoring/api/checks/{id}/outages is the outage log.

Pause an uptime check

Uptime → ⋯ → Pause, the Pause button on the check's page, or POST /monitoring/api/checks/{id}/pause. A paused check sends no requests and notifies nobody; an outage that was open is closed without a mail. Resume (…/resume) starts it over as pending.

Change an uptime check

PATCH /monitoring/api/checks/{id} with the fields to change, or dcloud monitoring checks update. Changing the URL, the type, the expected status, the keyword, redirects or TLS verification starts the check over as pending; changing its name, interval or recipients does not.

Delete an uptime check

Uptime → ⋯ → Delete check, the Danger zone on the check's page, or DELETE /monitoring/api/checks/{id}. The check, its results and its outage log are removed.

Custom metrics

Your own numbers — orders placed, queue depth, request latency — published by your code and charted, kept and alarmed on like the built-in metrics. A series is a namespace, a metric name and up to ten dimensions (route=/pay, region=eu): the same name with other dimensions is another series. Values are aggregated per minute on arrival, so every statistic — average, sum, minimum, maximum and sample count — is exact whatever you send. Open Monitoring → Custom metrics to chart them.

There are three ways in, all under the same limits.

The API. POST /monitoring/api/custom-metrics, needing monitoring:PutMetricData:

{
  "namespace": "Shop",
  "metricData": [
    { "metricName": "orders_placed", "value": 1, "dimensions": { "region": "eu" } },
    { "metricName": "checkout_ms", "values": [120, 95, 310], "unit": "Milliseconds" },
    {
      "metricName": "batch_size",
      "timestamp": "2026-11-10T12:00:00Z",
      "statistics": { "count": 40, "sum": 812, "min": 3, "max": 61 }
    }
  ]
}

Each datum has exactly one of value, values (with optional counts) or statistics. timestamp (ISO 8601 or epoch milliseconds) defaults to now. The answer lists what was stored and every datum that was not, by index: { "accepted": 2, "rejected": [{ "index": 2, "reason": "timestamp: more than 14 days old" }] }. Only a request where nothing was stored is a 400. The CLI takes one --metric-data per datum: dcloud monitoring custom-metrics put --namespace Shop --metric-data '{"metricName":"orders_placed","value":1}'.

OpenTelemetry. Point an OTLP/HTTP metrics exporter at https://{console}/monitoring/api/otlp with the JSON encoding and your access key as a header. For the OpenTelemetry Collector:

exporters:
  otlphttp/ews:
    metrics_endpoint: https://{console}/monitoring/api/otlp/v1/metrics
    encoding: json
    headers:
      Authorization: 'EWS-ACCESS-KEY ${env:EWS_ACCESS_KEY_ID}:${env:EWS_SECRET_ACCESS_KEY}'

An SDK without a collector: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT set to the same URL, OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/json. The namespace is the resource's service.namespace, else service.name, else otel; a data point's attributes become dimensions (the first ten by name). Gauges and sums become a value each, histograms their count, sum, minimum and maximum. Prefer delta temporality (OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta): a cumulative counter is stored as the running total it reports. Protobuf is answered with 415.

StatsD from an app's containers. The agent on every app machine listens on UDP port 8125 for the containers on that machine (nothing else can reach it). Send to host.containers.internal:8125. Counters (c, with @rate), gauges (g, +n/-n adjust), timers and histograms (ms, h, d) and sets (s) are aggregated for a minute and stored under the namespace App/<app name>, with an instance dimension naming the machine; DogStatsD tags (|#k:v) become dimensions. A counter's per-minute total is its sum, a timer's average its avg. A gauge is re-sent every minute for ten minutes after its last update.

echo "orders_placed:1|c|#region:eu" | nc -u -w0 host.containers.internal 8125

Read series over the API with GET /monitoring/api/custom-metrics?namespace=Shop and their datapoints with GET /monitoring/api/custom-metrics/data?seriesId=…&start=…&period=300&statistic=sum.

Alarms on custom metrics

An alarm on a custom series names it, rather than a machine: resourceType custom, resourceId the namespace, metric the metric name and dimensions exactly the series' dimensions.

{
  "name": "Checkout slow",
  "resourceType": "custom",
  "resourceId": "Shop",
  "metric": "checkout_ms",
  "dimensions": {},
  "statistic": "avg",
  "comparator": ">",
  "threshold": 500,
  "periodSeconds": 300,
  "evaluationPeriods": 2,
  "notifyEmails": ["ops@example.com"]
}

The series need not exist yet: until it has datapoints in the evaluated periods the alarm is INSUFFICIENT_DATA. Missing data never breaches, so an alarm cannot fire on a series that stops arriving altogether — publish a zero (orders_placed with value: 0) when there is nothing to count, and alarm on sum < 1. The CLI: dcloud monitoring alarms create --resource-type custom ….

Status pages

A status page tells your users whether your service works, at https://{slug}.status.{root} — a public page with no sign-in, served by Deutsche Cloud. Monitoring → Status pages → Create a status page asks for the address (slug: 3–63 lower-case letters, digits and dashes) and a title; a few words (dcloud, www, status, api, …) are reserved. At most five pages per organization.

A page lists components. Each one either follows an uptime check — up is Operational, regional is Degraded performance, down is Major outage, paused or not yet run is No data — or has a state you set by hand. Components with the same group are shown together. Under each component that follows a check, 90 bars show its uptime per day (from the check's outage log) and the 90-day percentage. The banner at the top is the worst state on the page.

An incident is what you tell your users: a title, an impact (none, minor, major, critical), a status (investigating, identified, monitoring, resolved), the components it affects and a timeline of plain-text updates. While it is open, it makes its components at least as bad as its impact (minor → degraded, major → partial outage, critical → major outage). Resolved incidents stay on the page for 14 days.

POST /monitoring/api/status-pages/{id}/incidents
{ "title": "Payments failing", "impact": "major", "body": "We are investigating.", "componentIds": ["…"] }

POST /monitoring/api/status-pages/{id}/incidents/{incidentId}/updates
{ "status": "resolved", "body": "Fixed: a certificate had expired." }

Subscribers hear about every incident posted, updated or resolved, and about a component that follows a check going down and coming back:

  • Email. Visitors enter an address on the page. Nothing is sent but a confirmation link until they open it; unconfirmed addresses are forgotten after a day. Every mail has a one-click unsubscribe link. You can invite an address from the console too — it confirms the same way.
  • Webhook. Added by you (kind: webhook, an https URL on a public address). It receives a JSON POST with type ews.monitoring.status-page.incident.created|updated|resolved or ews.monitoring.status-page.component.down|up, the page, and the incident or component.

At most 1000 subscribers per page; SMS is not offered yet. The page's data is also served as JSON at https://{slug}.status.{root}/summary.json.

The status pages run on the same platform as the checks. Deutsche Cloud's own status is served separately, from the probe machines outside the control plane, at https://status.{root}, so it keeps answering when the console does not.

The CLI: dcloud monitoring status-pages create --slug acme --title "Acme status", dcloud monitoring incidents create, dcloud monitoring subscribers add.

Read an app's logs

Open the app page and scroll to Logs, or Monitoring → Logs and pick the app. Lines appear about ten seconds after the container writes them, newest first. Type in the filter box to show only lines containing that text (case-insensitive). Over the API:

GET /monitoring/api/logs?resourceId=<app id>&contains=error&limit=500

start defaults to a day before end, end to now.

Reference

Limit Value
Metric resolution 1 minute
Metric retention 14 days
Points per series in one answer 1440 (widen period for longer ranges)
Metrics per data request 8
Alarm period 1 minute to 1 day, whole minutes
Alarm look-back (period × periods) at most 1 day
Email recipients per alarm 10
Webhook https, port 443, public address, 5 s
Log retention 7 days
Log line length 16 KiB; longer lines end …[truncated]
Log lines per search 1000
Uptime checks per organization 50
Uptime check interval 1, 2, 5, 10, 15, 30 or 60 minutes
Uptime check timeout 1–30 s, 10 s by default
Runs against one host, all of Deutsche Cloud 6 a minute, over every organization
Requests per run one per location (2–3), each retried once
Uptime check ports any except 25, 465 and 587
Uptime results retention 30 days; outages for the check's life
Custom metric resolution 1 minute
Custom metric retention 15 days per minute, 15 months per hour
Custom series per organization 1000 with a datapoint in the last 15 d
Custom datums per request 1000, body 1 MiB
Custom datums per minute 20000 per organization
Custom datum age at most 14 days old, 2 hours ahead
Dimensions per series 10; names and values up to 255 chars
  • cpu_percent is the provider's figure: 100 is one vCPU fully busy, so a machine with four vCPUs reaches 400.
  • Disk and network metrics are the machine's own disk and its public interface. A database node's data volume is covered by disk_used_percent, not by the disk metrics.
  • A stopped machine records nothing; its chart shows a gap, not zero.
  • An app that prints faster than its machine can ship drops lines past 5000 waiting to be sent, and the app's activity feed says how many.
Action Grants
monitoring:ListMetrics the resource list and a resource's metrics
monitoring:GetMetricData the charts and their datapoints
monitoring:PutMetricData publishing custom metrics (API, OTLP)
monitoring:DescribeAlarms the alarm list and one alarm
monitoring:DescribeAlarmHistory an alarm's history
monitoring:PutMetricAlarm creating and changing an alarm
monitoring:DeleteAlarms deleting an alarm
monitoring:FilterLogEvents reading app logs
monitoring:DescribeChecks the uptime check list and one check
monitoring:GetCheckResults a check's results, response times, outages
monitoring:CreateCheck creating an uptime check
monitoring:UpdateCheck changing an uptime check
monitoring:PauseChecks pausing an uptime check
monitoring:ResumeChecks resuming a paused uptime check
monitoring:DeleteChecks deleting an uptime check
monitoring:DescribeStatusPages status pages, incidents and subscribers
monitoring:CreateStatusPage creating a status page
monitoring:UpdateStatusPage its title, address, components, webhooks
monitoring:DeleteStatusPage deleting a status page
monitoring:CreateIncident posting an incident
monitoring:UpdateIncident posting an update, resolving it
monitoring:DeleteIncident removing an incident

Alarm actions can be granted on one alarm, rn:ews::{orgId}:monitoring:alarm/{id}, check actions on one check, rn:ews::{orgId}:monitoring:check/{id}, and status page and incident actions on one page, rn:ews::{orgId}:monitoring:status-page/{id} — so someone on call can post incidents on one page and change nothing else. Metric and log reads cover every resource of the organization.

What is not supported

  • Metrics from inside a machine (memory, filesystem usage of an instance, process counts). Only what the hypervisor sees is recorded.
  • Percentiles, sub-minute resolution and metric math on custom metrics; OTLP over protobuf or gRPC; deleting a custom series (it expires 15 months after its last datapoint).
  • Treating missing data as breaching, or any missing-data rule other than "missing".
  • Logs from compute instances you run yourself, and from managed databases.
  • Alarms on a whole database or app rather than on one machine.
  • Alarms on a combination of metrics, or on anomalies.
  • Re-sending a failed webhook.
  • Choosing which locations a check runs from, locations outside Hetzner, TCP, ping, heartbeat and DNS checks, custom request headers or bodies, methods other than GET and HEAD, and intervals under a minute.
  • Custom domains and private (password or login) status pages, maintenance windows, Markdown or logos on a status page, and alerts or status updates by SMS.
  • Confirming an alert recipient's address before it is mailed.
  • Changing an uptime check's settings in the console (use the API or the CLI).

Troubleshooting

  • A chart says "No datapoints in this range yet." The machine is stopped, was created in the last two minutes, or belongs to your own Hetzner project whose token has stopped working (Settings → Infrastructure says so).
  • An alarm stays in Insufficient data. The machine records no points in the evaluated periods — it is stopped, or the alarm's period is shorter than the metric's interval (disk_used_percent arrives every few minutes; use a period of 15 minutes or more).
  • The webhook shows "refusing to follow a redirect". The URL answers with a 3xx. Use the final URL.
  • The webhook shows "resolves to a private or reserved address". The host name points at a private network. Webhooks only reach public addresses.
  • An app's logs are empty. The app writes its logs to a file instead of stdout and stderr, or its machine's agent predates log shipping; the agent updates itself within a few minutes of a platform update.
  • Lines from before a restart of the machine's agent are missing. The agent ships what containers print from the moment it starts; earlier output stays on the machine (podman logs).
Did this page help you?