CI/CD pipeline
From a commit to a published image.
What it is
CI turns a commit into a published container image. Put a workflow file in .dcloud/workflows/ in your repository, and every push, pull request and version tag that matches its triggers starts a run on a fresh machine. Runs appear under CI in the console and per repository under Git → Repositories → Actions; a pull request's runs are the status checks branch protection waits for. Every key a workflow file accepts is listed in Workflow syntax — this page covers the pipeline around them.
Concepts
| Term | Meaning |
|---|---|
| Workflow | A YAML or JSON file under .dcloud/workflows/, read at the commit that triggered it — never from the default branch. |
| Run | One execution of one workflow at one commit, numbered per repository, on one machine of its own. |
| Job | A named group of steps inside a workflow. Jobs with no needs between them run at the same time. |
| Run identity | The service account a run acts as. One per repository, visible under IAM → Users as ci- and the repository's id. |
| Status check | A run's verdict for a commit. A protection rule that requires status checks reads the latest run per workflow name. |
How to
Add a workflow
Commit a file under .dcloud/workflows/ and push. The file is read at the commit you pushed, so a workflow takes effect on the commit that introduces it. A file that does not parse is reported in the console and never blocks the push.
Convert a GitHub Actions or GitLab CI file
Most repositories arrive with one already. There are two ways to bring it across, and both end at the same place: a file you read, edit and commit yourself.
In the console. CI → Workflows → Convert. Paste the file, say whether it came from GitHub Actions or GitLab CI, and convert. What comes back has already been through the same parser a push runs — invalid output is repaired once and re-checked, and whatever is still wrong is listed against the key it is wrong on, so you never commit a file that cannot run. Anything with no equivalent here (a matrix, a service container, a third-party action) is left out and recorded as a # note: comment at the top of the file, which is your to-do list. Read the result before committing it: a converter can translate docker build into a build-image step, but it cannot know that your test command needs a database.
In your own AI session. Fetch /docs/workflows/prompt.md — a public, unauthenticated file — and paste it into Claude Code, ChatGPT or whatever you use with the repository open, then paste your old CI file after it:
curl -s https://<your console host>/docs/workflows/prompt.mdThat document is generated from the workflow parser, so it lists every key the format has and nothing it does not, plus a mapping table for each of GitHub Actions and GitLab CI and three worked examples. Doing it this way is usually better for a repository the model can read, because it can see the Dockerfile, the lockfile and the scripts the workflow has to call. Check the result before you commit it with POST /ci/api/workflows/validate, or by pushing it — an unparsable file is reported and never blocks the push.
Converting needs an AI provider configured for the installation; without one, the console's Convert button and POST /ci/api/workflows/convert answer 503 and the prompt file is still there to use by hand.
Check every pull request
name: Pull request checks
on:
pull_request:
branches: [main]
cancelInProgress: true
jobs:
test:
name: Vet and test
image: docker.io/library/golang:1.23-alpine
steps:
- run: go vet ./...
- run: go test ./...
- build-image:
dockerfile: Dockerfile
tags: ['pr-${{ commit.short }}']A pull request builds the image without pushing it. cancelInProgress: true supersedes older in-flight runs of this workflow on the same branch, so a series of pushes does not hold a machine each.
Publish an image from the default branch
name: Publish from main
on:
push:
branches: [main]
jobs:
publish:
name: Build and publish
steps:
- build-image:
dockerfile: Dockerfile
buildArgs: { VERSION: '${{ commit.short }}' }
tags: ['${{ commit.short }}', latest]
- push-image:
repository: pipeline-demo
tags: ['${{ commit.short }}', latest]latest appears in both lists: push-image re-tags what build-image produced, so every tag you push must also have been built.
Publish a release from a version tag
The same file with a tag trigger and the version in the tags:
on:
push:
tags: ['v*']
jobs:
publish:
steps:
- build-image:
tags: ['${{ slice(tag, 1) }}']
- push-image:
repository: pipeline-demo
tags: ['${{ slice(tag, 1) }}']slice(tag, 1) drops the leading v, so git push origin v1.2.3 publishes pipeline-demo:1.2.3. Keep branch and tag publishing in separate files: a push trigger declaring only branches: never fires for a tag, and one declaring only tags: never fires for a branch.
Give a job a secret
Two things must both be true. List the name on the job — secrets: [NPM_TOKEN] — and grant that secret to the actions consumer under Secrets → the secret → Consumers, or:
POST /kms/api/secrets/{secretId}/grants
{ "consumer": "actions" }Each listed secret is exported into that job's run steps under its own name, and is available as ${{ secrets.NAME }} in a value. A sibling job that did not list it does not have it. Without the grant the run fails before any job starts, with this on the run itself:
Secret "NPM_TOKEN" is not granted to actions. Grant it under Secrets → Consumers.Require checks before a merge
Repository → Settings → Branch protection → Add rule, or:
POST /git/api/repositories/{repoId}/branch-protection
{ "branchPattern": "main", "requirePullRequest": true, "requireStatusChecks": true, "requiredChecks": ["CI"] }Which checks are required is decided by the protected branch, never by the pull request. A required check is a workflow file as it is on main; it passes only when a run of that version of the file succeeded on the commit being merged.
requiredChecks |
Required | Other runs |
|---|---|---|
names, e.g. [CI] |
the workflows of those names on the protected branch; a name that is no workflow's is a job of one (its name: or its key, as GitHub names a check run, e.g. checks, taint): every matrix leg of it must pass, a skipped job passes; a commit status of that context also satisfies it. A name the branch does not define blocks the merge and says so. |
shown, but do not block |
| empty (default) | every workflow on the protected branch that a pull request to it triggers (with its paths: filter applied to the pull request's changes; merge_group in the queue) |
must be green too, and at least one must run |
When the pull request leaves a required workflow alone, its own run is the check. When it changes, renames or deletes it, the platform also runs the protected branch's version on the pull request's commit, and only that run counts: a pull request cannot pass by replacing a test step with true. A change to CI therefore gates merges once it has landed. The same applies to merge queue builds.
Limit: local actions and local reusable workflows a required workflow calls (uses: ./…) are read from the commit under test. Protect them with review, as you protect the code itself.
Watch, cancel or re-run
Open the run under Git → Repositories → the repository → CI → the run, or from CI → Runs. Logs stream per step while the run is live. Cancel stops a run that has not finished; Re-run starts a new run from the same workflow source at the same commit, with the same trigger, event and the same identity as the run it repeats: who clicked Re-run is recorded (and audited) but grants nothing, so re-running someone else's run never hands it your secrets. Reopening a pull request runs its checks as the pull request's author, or with no identity (no secrets) when someone else reopens it. Only the latest run of a workflow counts as the check.
Run a job on your own machine
Every job runs on a machine the platform creates for the run and destroys after it — unless the job says otherwise:
jobs:
build-app:
runs-on: macos
steps:
- run: xcodebuild -scheme App -destination 'generic/platform=iOS' buildruns-on is linux (the default) or macos. A macos job runs on a self-hosted runner: a machine you own, registered with this organization, that asks the control plane for work matching its labels. There is nowhere to rent a Mac by the minute, so this is how macOS and iOS builds happen.
Steps in a macos job run natively, in a fresh workspace directory per job — there is no container engine on the machine, so image: on a macos job is a parse error rather than a setting that is quietly ignored.
Everything after the runner picks the run up is identical to a platform machine: the same plan, the same logs streaming into the console, the same artifacts, caches, secrets and status checks. Where a run executed changes nothing about what it produced.
Add a runner
- Open CI → Runners and choose Add runner. You need
ci:ManageRunners. - Choose the runner group and, optionally, its labels. Tick holds deploy or secret-manager credentials for a machine that will: the console then refuses any group that runs unprotected branches.
- Copy the install command it shows — it carries a one-time registration token, valid for 15 minutes, and it is shown once.
- Run it on the machine. It registers itself, reports its labels and capabilities (OS, architecture, cores), stores its credential in a root-only file, and starts asking for work.
The runner appears in the list with its group, labels, capabilities and when it was last seen. A runner whose capabilities include vm runs every job in its own disposable virtual machine, cloned from the image the value names; one without it runs jobs natively on the machine. Move to group changes which runs it may take from its next claim. Disable revokes its credential: it stops claiming runs and cannot reconnect, and the row stays so past runs still name the machine that ran them.
A registration token registers exactly one machine. Re-running the install command with a spent token registers nothing. A token issued for one repository registers a runner that serves that repository only. A token that names labels fixes them: the machine keeps the platform labels it reports (self-hosted, OS, architecture) and nothing else it declares, so a machine cannot call itself deploy to attract deploy jobs.
Runner groups
Labels route a job to a machine. A runner group decides whether a run's code may reach that machine at all:
| Setting | Effect |
|---|---|
| Repositories | all, or a list |
| Protected branches only | only branches a branch protection rule covers; a merge queue build counts as the branch it lands on |
| Refs | globs: main, release/*, refs/tags/v*; empty means any |
| Take pull-request runs | whether pull_request runs reach it |
A runner without a group is in Default: any repository, any branch, pull requests included. There are no forks on Deutsche Cloud, so whoever can push a branch decides what that branch's runs execute — through its push run as much as through a pull request. Keep deploy or secret-manager credentials only on runners in a group with protected branches only, and prefer --ephemeral for runners that take unprotected branches: one job, then the machine is retired, so nothing one job leaves behind (including the runner's own key) serves the next. A run that no allowed runner can take stays queued.
POST /ci/api/runner-groups
{ "name": "deploy", "protectedRefsOnly": true, "allowPullRequests": false }
POST /ci/api/runners/registration-tokens
{ "groupId": "<group id>", "labels": ["deploy"] }A group with runners cannot be deleted (409): its runners would otherwise fall back to Default.
Install the agent on a Mac
A Mac you own runs jobs that need macOS or Xcode. Get a registration token from CI → Runners → Add runner, then on the Mac:
curl -fsSL https://console.<your platform domain>/ci/api/agent/install.sh | sudo bash -s -- --token TOKENRequires macOS 13 or newer on Apple silicon and Node 20+ on PATH (brew install node; the installer will not install Homebrew or Node for you). Add labels beyond the default macos,arm64 with --labels gpu,xcode-16, and name it with --name; the default name is the Mac's own.
What the installer leaves behind:
| Path | What it is |
|---|---|
/usr/local/ews-runner/runner.env |
the runner credential, mode 600, root-only |
/usr/local/ews-runner/agent.cjs |
the agent, re-downloaded from the control plane every time the daemon starts |
/usr/local/ews-runner/work/<run id>/<job> |
one workspace per job, deleted when the job ends |
/usr/local/ews-runner/logs/runner.log |
what the daemon printed — claims, run outcomes, and why a heartbeat failed |
/Library/LaunchDaemons/app.deutschecloud.ews-runner.plist |
the daemon: starts at boot, restarts if the agent exits |
The daemon heartbeats every 30 seconds and asks for work every 5, backing off to every 15 while there is nothing to claim. Because it re-fetches the agent at each start, a runner follows the control plane's protocol without anyone updating it; sudo launchctl kickstart -k system/app.deutschecloud.ews-runner picks up a new version immediately.
tail -f /usr/local/ews-runner/logs/runner.log # follow it
sudo launchctl print system/app.deutschecloud.ews-runner # is it loaded
# stop it and remove its files — the same script, run the same way
curl -fsSL https://console.<your platform domain>/ci/api/agent/install.sh | sudo bash -s -- --uninstallUninstalling leaves the runner listed in the console until you delete it there.
Jobs run natively on the Mac, as root — there is no container around them. A step's environment is built from scratch (PATH, HOME, the job's env: and secrets, and the EWS_* run variables), so the runner's own credential is never visible to a build, but repository code still runs on the machine with your toolchain and your files. Only give a Mac to organizations whose repositories you would run on it anyway. Per-job isolation in a fresh Tart VM is the next phase. build-image and push-image steps are refused on a native job: nothing there can build an image.
Reference
Console pages
| Page | Shows |
|---|---|
/ci |
every run in the organization, filtered by status and trigger |
/ci/runners |
the organization's self-hosted runners |
/git/repositories/{repoId}/actions |
that repository's runs |
/git/repositories/{repoId}/actions/{runId} |
one run: its jobs, its steps, live logs, cancel and re-run |
Where a run executes
A run is executed by exactly one of two things, decided by its jobs' runs-on values:
Every job is linux |
Any job is not linux |
|---|---|
| The platform creates a machine for the run and destroys it afterwards. | The run stays queued until a registered runner claims it. |
Claiming is a single atomic update, so two runners polling at the same instant cannot both take one run, and a runner may only claim a run whose jobs' runs-on values are all within its own labels.
That last rule has a consequence worth knowing before you hit it: a workflow that mixes a linux job and a macos job in one run can only be claimed by a runner carrying both labels. A Mac registered with macos alone will not take it — the run is not split between a platform machine and your machine, it goes to one executor or the other. Put the macOS work in its own workflow file, or register the machine with linux as well once it can run those jobs.
A run nobody claims within 20 minutes fails with no online runner with label macos. That is deliberate: a machine in an office can be asleep, unplugged or not registered yet, and a pull request waiting forever on a check that will never arrive is worse than one that says what is missing. Turn the machine on, or register a runner, and re-run.
Run statuses
| Status | Meaning |
|---|---|
queued |
waiting for a machine, held behind the organization's ceiling, or waiting for a self-hosted runner to claim it |
provisioning |
the machine is being created |
running |
jobs are executing |
succeeded |
every job finished successfully |
failed |
a job failed, or the plan could not be built |
cancelled |
cancelled from the console, or superseded by a newer run |
timed_out |
the run passed the two-hour ceiling |
What starts a run
| Trigger | Fires on |
|---|---|
push with no filters |
any branch push; never a tag |
push with branches: |
a push to a matching branch |
push with tags: only |
a push of a matching tag; never a branch |
pull_request |
a pull request whose target branch matches, built from the source commit |
manual |
parses, but nothing dispatches it |
Merging a pull request, editing a file in the console, creating a branch and Update branch each dispatch what the equivalent push would, and the run is filed against the commit the ref now points at. An annotated tag is peeled first, so ${{ commit.short }} names the commit.
What a run may and may not do
| The run | Allowed |
|---|---|
| reads the repository it was triggered from | yes, read-only |
| pulls any image from your organization's registry | yes |
| publishes to the registry repositories that repository owns | yes |
| creates a registry repository on first push | yes, private |
| pushes to any Git repository | no |
| reads a secret it did not list, or any secret directly | no — values are resolved before the machine boots |
| deletes a published image | no |
| reports status or logs for another run | no — refused with 403 |
| writes a cache another branch restores | no — see "Cache scope" below |
| starts a machine, creates an IAM user, or reaches any other service | no |
Cache scope
A cache entry belongs to the repository and to the scope of the run that wrote it, as on GitHub:
| The run | Writes | Restores from, in order |
|---|---|---|
| push, tag, schedule, manual | its own ref | its own ref, then the default branch |
| pull request #n | the pull request (#n) |
the pull request, its target branch, the default branch |
| merge queue build | the queue branch | the queue branch, the protected branch, the default branch |
A pull request or an unmerged branch can therefore read the default branch's cache but never replace it: what the default branch restores was written by the default branch.
A repository owns the registry repository of its own name plus that name's -, . and _ suffixes: pipeline-demo publishes pipeline-demo, pipeline-demo-api and pipeline-demo-worker, and nothing else.
Credentials a run holds
| Credential | Scope | Life |
|---|---|---|
| Clone token | read-only, this repository | 6 hours |
| Registry login token | the run identity's grants | 12 hours |
| Registry bearer token | the repositories and actions that survived the check | 1 hour |
| Run token | one run, for status and logs | 6 hours |
| Cache and artifact upload | a presigned URL naming one object | per use |
There is no checkout step and no login step: the workspace is already the repository at the triggering commit, and registry credentials are injected.
Image tags
build-image builds locally under the tags it names; push-image re-tags those to <organization>.ecr.<platform domain>/<repository>:<tag> and pushes each one.
| Rule | Detail |
|---|---|
| Default tag | the short commit sha — the first 8 characters |
| Build before push | every tag push-image names must also appear in build-image |
| Repository name | lowercase letters, digits and . _ - separators, 2–128 characters |
| First push | creates the repository, private; publish it under ECR → Visibility |
Expressions in a tag are resolved before the machine boots, so the runner receives literal strings. A misspelled reference is a parse error on the workflow file, not an empty tag: jobs.publish.steps[0].push-image.tags[0]: unknown reference "taag".
Limits
| Limit | Value |
|---|---|
| Concurrent runs per org | 4; over the ceiling a run stays queued and retries |
| Run ceiling | 2 hours, regardless of per-job timeouts |
| Machines | one per run, not per job, destroyed when the run finishes |
| Metering | wall-clock minutes of the run, as ci.job.minute |
API
| Operation | Permission |
|---|---|
GET /ci/api/repositories/{repoId}/runs |
ci:ListRuns |
GET /ci/api/runs/{runId} |
ci:GetRun |
GET /ci/api/runs/{runId}/logs |
ci:GetRunLogs |
GET /ci/api/runs/{runId}/artifacts |
ci:ListArtifacts |
GET /ci/api/artifacts/{artifactId}/download |
ci:DownloadArtifact |
POST /ci/api/runs/{runId}/cancel |
ci:CancelRun |
POST /ci/api/runs/{runId}/rerun |
ci:RerunWorkflow |
POST /ci/api/workflows/validate |
ci:ListRuns |
POST /ci/api/workflows/convert |
ci:ListRuns |
GET /docs/workflows/prompt.md |
none — public |
Reading logs and downloading an artifact are sensitive: a denied attempt is recorded in the audit log, as are cancel and re-run.
The two workflows routes are POSTs that change nothing — the file is in the body because it does not fit in a query string, and the answer is returned rather than committed anywhere. Both take source (at most 65,536 characters); convert also takes from (github, gitlab or other) and an optional filename, and answers { yaml, valid, issues, notes }.
What is not supported
- A matrix built from an expression; fixed lists work (
strategy.matrix, see the workflow reference). - Reusable or composite workflows.
- Manual dispatch.
on: manual:parses and satisfies the "at least one trigger" rule, but only Re-run starts a run outside an event. - Service or sidecar containers alongside a job.
- Masking a transformed secret. A secret's value, its lines, and its base64 and URL-encoded forms are replaced with
***in the log; a step that reverses it, slices it or prints it character by character puts the result in the run log in plain text. - More than one attempt per run:
${{ run.attempt }}is always1.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
Secret "NPM_TOKEN" is not granted to actions |
the secret exists but has no actions grant, or the grant's TTL ran out |
grant it under Secrets → Consumers; the run fails before any job starts |
Secret "NPM_TOKEN" is not defined in this organization |
a typo, or a secret in another organization | check the name under Secrets |
| the push step fails while tagging | a tag push-image names was never built |
add the same tag to build-image |
| the merge stays blocked with no run reported | a paths: filter matched nothing on this pull request |
remove the filter from a workflow a protection rule requires |
a run sits in queued |
the organization is at its concurrent-run ceiling | wait, or cancel a run; the run page shows how many machines are in use |
a run sits in provisioning and never starts |
the control plane address is not reachable from a machine | it must be a public address, not localhost or a .localhost name |