AI models

An OpenAI-compatible model API in front of EU providers: IAM per model, residency labels, limits, usage on your bill.

Deutsche Cloud has an OpenAI-compatible model API in front of EU model providers, which also speaks Anthropic's Messages API for models whose provider does. You call it with any OpenAI or Anthropic SDK (or Claude Code), authenticate with a Deutsche Cloud access key, and pay for tokens on your Deutsche Cloud bill. Deutsche Cloud runs no models itself: each request is forwarded to the provider the model belongs to, and every model says where it is processed.

What Deutsche Cloud adds to calling a provider directly:

  • One IAM model: who may call which model is an IAM policy (ai:InvokeModel on rn:ews::{org}:ai:model/{id}), for users, groups, roles, CI jobs and OIDC role sessions alike. Every call and every refusal is in the audit log.
  • Residency on every model, and an organization-wide ceiling: by default only models from EU providers processing in the EU can be used at all.
  • Limits you set: requests per minute for the organization and per access key, and a hard monthly spend cap.
  • A usage log without your prompts: who called which model, when, tokens, latency and cost. Prompt and response bodies are not stored unless you switch prompt logging on.
  • One bill: tokens appear per model in the cost explorer next to everything else.

Call it

from openai import OpenAI

client = OpenAI(
    base_url="https://<your console>/ai/v1",
    api_key="EWS1XXXXXXXXXXXXXXXX:<secret>",  # an EWS access key
)
reply = client.chat.completions.create(
    model="mistral-small",
    messages=[{"role": "user", "content": "Say hello in German."}],
)
print(reply.choices[0].message.content)
import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'https://<your console>/ai/v1', apiKey: process.env.EWS_KEY });
const stream = await client.chat.completions.create({
  model: 'mistral-small',
  messages: [{ role: 'user', content: 'Count to five.' }],
  stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
curl https://<your console>/ai/v1/chat/completions \
  -H "Authorization: Bearer $EWS_ACCESS_KEY_ID:$EWS_SECRET_ACCESS_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"mistral-small","messages":[{"role":"user","content":"Hi"}]}'

Tools built on the OpenAI API (LangChain, LlamaIndex, Vercel AI SDK, most agents) work by setting OPENAI_BASE_URL and OPENAI_API_KEY.

Endpoint Does
GET /ai/v1/models The models you can call: allowed for the org and by your IAM
GET /ai/v1/models/{id} One of them
POST /ai/v1/chat/completions Chat, with streaming, tools, JSON mode and images as the model has
POST /ai/v1/embeddings Embeddings

Parameters Deutsche Cloud does not need to understand (temperature, tools, response_format, seed, …) are passed to the provider unchanged; what a model supports is the provider's. The answer carries the Deutsche Cloud model id, never the provider's internal name. Every response has an x-request-id, which is also on the usage log row and the audit row, and an x-dcloud-model-residency header.

Each model entry in /ai/v1/models has an dcloud object: kind (chat or embedding), residency, residency_label, location, provider, context_window and the price in EUR per million input and output tokens.

Credentials. The API key is an access key, EWS1…:<secret>, sent as Authorization: Bearer (what the SDKs send) or Authorization: EWS-ACCESS-KEY; on the Anthropic endpoints also as x-api-key. Keys from OIDC role sessions work the same way, so a CI job can call a model without a stored secret. A browser session cannot call the model API, and neither can a connected app's token (ews_at_…): those are read-only outside the MCP endpoint.

Anthropic Messages API

Models whose provider serves Anthropic's Messages API natively are also reachable in Anthropic's format, so the Anthropic SDKs and Claude Code work unchanged. Deutsche Cloud passes these requests through to the provider's own Messages endpoint and never translates them to another format: prompt caching, extended thinking, tool use, images, PDFs and citations behave exactly as the provider implements them. /ai/v1/models (with an anthropic-version header, which the SDKs send) lists the models reachable this way; in the OpenAI-format list they have "anthropic_messages": true.

Claude Code:

export ANTHROPIC_BASE_URL=https://<your console>/ai
export ANTHROPIC_API_KEY=EWS1XXXXXXXXXXXXXXXX:<secret>   # or ANTHROPIC_AUTH_TOKEN (sent as Bearer)
export ANTHROPIC_MODEL=<model id from /ai/v1/models>
export ANTHROPIC_DEFAULT_HAIKU_MODEL=<a small model id>  # Claude Code's background calls
claude

Model ids are the Deutsche Cloud ids in /ai/v1/models, not Anthropic's, so set both model variables; otherwise Claude Code asks for its own default and gets a 404.

from anthropic import Anthropic

client = Anthropic(
    base_url="https://<your console>/ai",       # the SDK appends /v1/messages
    api_key="EWS1XXXXXXXXXXXXXXXX:<secret>",
)
msg = client.messages.create(
    model="<model id>",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Say hello in German."}],
)
print(msg.content[0].text, msg.usage)

with client.messages.stream(model="<model id>", max_tokens=1024,
                            messages=[{"role": "user", "content": "Count to five."}]) as s:
    for text in s.text_stream:
        print(text, end="", flush=True)
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({ baseURL: 'https://<your console>/ai', apiKey: process.env.EWS_KEY });
const msg = await client.messages.create({
  model: '<model id>',
  max_tokens: 1024,
  tools: [{ name: 'get_weather', input_schema: { type: 'object', properties: { city: { type: 'string' } } } }],
  messages: [{ role: 'user', content: 'Weather in Berlin?' }],
});
const tokens = await client.messages.countTokens({
  model: '<model id>',
  messages: [{ role: 'user', content: 'How many tokens is this?' }],
});
curl https://<your console>/ai/v1/messages \
  -H "x-api-key: $EWS_ACCESS_KEY_ID:$EWS_SECRET_ACCESS_KEY" \
  -H 'anthropic-version: 2023-06-01' \
  -H 'content-type: application/json' \
  -d '{"model":"<model id>","max_tokens":256,"messages":[{"role":"user","content":"Hi"}]}'
Endpoint Does
POST /ai/v1/messages Messages, streaming or not; anthropic-beta and ?beta=true too
POST /ai/v1/messages/count_tokens Token counting, if the provider offers it; free, not logged
GET /ai/v1/models (Anthropic shape) With anthropic-version: {data: [{type, id, display_name}]}

What Deutsche Cloud changes: only the request's model (the Deutsche Cloud id becomes the provider's). What it does not change: the response. A JSON answer and a stream arrive byte for byte as the provider sent them, so the model field in them is the provider's name for the model. Streams are relayed with backpressure and without buffering. Requests may be up to 32 MiB.

  • Same rules as the OpenAI endpoints: ai:InvokeModel (counting tokens needs it too but is not audited), residency ceiling, allow-list, rate limits (counting has its own bucket of the same size), spend cap, usage log, prompt logging.
  • Not available: server-side tools (web_search_*, web_fetch_*, code_execution_*) and the MCP connector (mcp_servers) are refused with a 400: they run at the provider, outside the residency boundary and metering. Your own tools, bash, text editor and computer use work. Message Batches, Files and the rest of Anthropic's API answer 404.
  • Errors have Anthropic's shape, {"type": "error", "error": {"type", "message"}, "request_id"}, with Anthropic's statuses, so the SDKs raise and retry as usual. The spend cap is 402 billing_error, which the SDKs do not retry. The provider's own 400 message is passed on; its 529 overloaded_error stays a 529. Responses carry request-id (also on the usage row) and x-dcloud-model-residency.
  • Timeouts: 15 minutes for an answer, 30 minutes for a stream, and a stream that sends nothing for 5 minutes is ended with an error event.
  • Prompt caching is billed at cache prices. The provider reports cache writes and reads separately from input tokens; each is billed at its own price (see Billing). A second identical request with cache_control shows cache_read_input_tokens in its usage, in the usage log and on the bill.

Residency

Label Means
eu An EU-headquartered provider, processing in the EU
eu-processed Processed and stored in the EU by a provider under non-EU jurisdiction
non-eu Processed outside the EU

The organization's residency ceiling is eu until an administrator changes it to eu-processed or any. A model above the ceiling is not listed and a call to it is refused (403, code model_residency), whatever IAM says. The operator of the installation sets each model's label; there is no "unknown".

Providers the installation can be configured with: Mistral AI (Paris), IONOS AI Model Hub (Berlin), STACKIT AI Model Serving (Germany), Scaleway Generative APIs (Paris), and any other OpenAI-compatible endpoint. Which models an installation offers, at what price, is on the Models page of the console and in GET /ai/v1/models. OTC's LLM Hub is not offered: it forwards some models to hyperscalers outside the EU.

Who may call what

Action Allows
ai:InvokeModel Calling a model (chat and embeddings). Resource: ai:model/{id}
ai:ListModels Listing models and reading the organization's model access settings
ai:ConfigureModelAccess Changing the residency ceiling, allow-list, limits and prompt logging
ai:GetUsage Reading the usage log (never bodies)
ai:GetPromptLogs Reading logged prompts and responses (sensitive; every read is audited)

ai:InvokeModel is a write: it spends money. ReadOnlyAccess does not include it; AiFullAccess does. To let a team use only the cheap models:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ai:InvokeModel",
      "Resource": ["rn:ews::<org id>:ai:model/mistral-small", "rn:ews::<org id>:ai:model/*-embed"]
    }
  ]
}

On top of IAM, an organization can keep an allow-list: with it on, only the listed models can be called by anyone (403, code model_allow_list).

Limits

  • Requests per minute, per organization: 600 unless you set less. The installation sets the most you can set; ask support for more.
  • Requests per minute per access key, optional, never above the organization's.
  • Monthly spend cap in EUR (calendar month, UTC). Once the month's spend reaches it, calls are refused until the next month or until the cap is raised. Spend is summed every few seconds per server, so a burst can overshoot by what was spent in those seconds.

A refusal is 429 with Retry-After and code rate_limit_exceeded, or insufficient_quota for the spend cap. The OpenAI SDKs raise RateLimitError for both. A provider that rate-limits Deutsche Cloud is also a 429 (provider_rate_limited).

Usage log and prompt logging

Every request leaves one row: time, model, provider, residency, the IAM user and access key (and role session), status, input and output tokens, latency, time to first token, and cost at the model's price. Never the prompt or the answer. Rows are kept 90 days. Console → AI Models shows totals per model and the log; GET /ai/api/usage returns them.

On Anthropic Messages calls the row also has the cache tokens: cacheWriteTokens (5-minute cache), cacheWrite1hTokens and cacheReadTokens. They are in addition to inputTokens, which for these calls counts only the uncached input, as Anthropic reports it.

A stream the provider did not count is metered by estimate (four characters a token) and marked estimated. A client that disconnects mid-stream pays for what was generated until then: on the OpenAI endpoints by estimate; on Anthropic Messages the input and cache tokens exactly as the provider reported them when the stream began, and only the output tokens by estimate. Hanging up stops the generation at the provider.

Prompt logging is off by default. Turned on, each request's body and response are sealed under one of your KMS keys and kept 30 days; GET /ai/api/usage/{id}/prompt opens one, with ai:GetPromptLogs. It cannot be turned on without an enabled encrypt/decrypt key. Disabling the key makes the logs unreadable.

Billing

Tokens are billed per model and hour as ai.<model>.input and ai.<model>.output, in million tokens, at the model's price when the hour began; a price change applies from the next full hour. Prompt-cache tokens (Anthropic Messages) are separate lines: ai.<model>.cache_write (5-minute cache writes), .cache_write_1h and .cache_read. Unless the provider's cache prices were entered, they are Anthropic's ratios of the input price: 1.25×, 2× and 0.1×. /ai/v1/models shows all five prices per model. The usage log shows each request's cost at once; the cost explorer shows the billed tokens under AI Models about an hour later.

Settings as code

dcloud ai model-access get
dcloud ai model-access update --residency-ceiling eu --monthly-spend-cap-eur 200
dcloud ai models list
dcloud ai usage list --from 2026-11-01T00:00:00Z
resource "dcloud_ai_model_access" "this" {
  residency_ceiling     = "eu"
  allow_list_enabled    = true
  allowed_models        = ["mistral-small", "mistral-embed"]
  monthly_spend_cap_eur = 500
}

The same operations are MCP tools and SDK methods (ai.modelAccess.update, …).

Errors

On the OpenAI endpoints all errors have OpenAI's shape, {"error": {"message", "type", "code", "param"}} (Anthropic Messages errors: see above):

Status code Means
400 invalid_value, wrong_endpoint The body is wrong, or a chat model on /embeddings
400 provider_rejected The provider refused the request (its message given)
401 — No or wrong access key
403 access_denied IAM denies ai:InvokeModel on that model
403 model_residency, model_allow_list The organization does not allow that model
404 model_not_found No such model
429 rate_limit_exceeded, insufficient_quota A limit or the spend cap
502 provider_error, provider_auth_failed The provider failed; not your request
503 provider_unavailable The model's provider is not configured
504 provider_timeout No answer in time (5 min, streams 10 min)
Did this page help you?