Cloud GPU & Self-Hosted Models

No local GPU, but you want open-weight models instead of (or alongside) per-token frontier APIs? You have two options: rent a GPU and run your own inference server, or pay per token at an inference provider that hosts open models. Both connect to OpenClaw the same way, as an entry under models.providers.

tip

For running models on your own hardware, see Local Models. This page covers cloud-hosted inference.

Config is strict

OpenClaw validates openclaw.json against a strict schema; an unknown key stops the Gateway from starting. Run openclaw config validate after every edit.

Quick Decision Guide

GPU Cloud Providers

Dedicated GPU Cloud

Rent GPUs by the hour and run your own inference server (vLLM, SGLang, Ollama).

Prices are approximate on-demand list prices as of 2026-10-18, taken from each provider's pricing page. They change often and vary by region and availability; check the linked page before you budget.

Provider	GPU	VRAM	Price/hr	Monthly (24/7, ~730 h)	Notes
RunPod	RTX 4090	24 GB	$0.34 community / $0.89 secure	~$250 / ~$650	Per-second billing
RunPod	A100 80GB	80 GB	$1.19-$1.79	~$870-$1,310	70B quantized, 32B full precision
RunPod	H100 80GB	80 GB	$1.99-$3.99	~$1,450-$2,910	PCIe vs SXM, community vs secure
Lambda	A100 40GB	40 GB	$1.99	~$1,450	Simple pricing
Lambda	H100 80GB	80 GB	$3.29 PCIe / $3.99-$4.29 SXM	~$2,400 / ~$2,910-$3,130
Vast.ai	Various	Various	Marketplace (live)	Varies	Auction/marketplace model, usually cheapest, less predictable
CoreWeave	A100 / H100	80 GB	See pricing page		Enterprise, InfiniBand
AMD Developer Cloud	MI300X	192 GB	See site		Check the site for current credit offers

GPU Size Guide

GPU	VRAM	Fits (weights only, roughly)	Best For
RTX 4090	24 GB	8-14B at 16-bit, 32B at Q4	Daily driver, coding
A100 40GB	40 GB	14B at 16-bit, 70B at Q4 (tight)	Production serving
A100 / H100 80GB	80 GB	32B at 16-bit, 70B at 8-bit or Q4	Large models
MI300X	192 GB	70B at 16-bit	Largest models

A 70B model at 16-bit needs about 140 GB for weights alone, so it needs two 80 GB GPUs or one MI300X. Leave headroom for the KV cache: OpenClaw sends a large system prompt, tools and history on every turn.

RunPod Setup

RunPod has a vLLM deployment template:

Create an account at runpod.io.
Deploy a GPU Pod with the vLLM template.
Set the model name, for example Qwen/Qwen3-32B.
Set Max Model Length to the context you want (32768 is a reasonable start).
Set an API key for the server, and note the endpoint URL once it is up.

~/.openclaw/openclaw.json
{
  agents: {
    defaults: {
      model: { primary: "runpod/Qwen/Qwen3-32B" },
    },
  },
  models: {
    providers: {
      runpod: {
        baseUrl: "https://your-pod-id-8000.proxy.runpod.net/v1",
        apiKey: "${RUNPOD_VLLM_API_KEY}",
        api: "openai-completions",
        timeoutSeconds: 300,
        models: [
          { id: "Qwen/Qwen3-32B", name: "Qwen3 32B (RunPod)", contextWindow: 32768, maxTokens: 8192 },
        ],
      },
    },
  },
}

RUNPOD_VLLM_API_KEY here is whatever key you gave vLLM (--api-key), not your RunPod account key. Keep contextWindow equal to the server's --max-model-len.

RunPod Serverless

Serverless endpoints scale to zero when idle, which suits bursty use. As of 2026-10-18 RunPod lists serverless at about $4.79/h for H100 and $2.72/h for A100, billed only while a worker runs (pricing). Cold starts add latency to the first turn after an idle spell; raise the provider's timeoutSeconds.

Vast.ai Setup

Vast.ai is a marketplace: prices are often lower than RunPod, but host reliability varies.

Browse GPUs at vast.ai.
Pick a GPU and the Docker image vllm/vllm-openai:latest.
Set the start command with your model and --api-key.
Note the endpoint URL and configure it like the RunPod example above.

Inference-as-a-Service

These providers host open-weight models behind an API. You pay per token; there is no server to manage. Several are first-class OpenClaw providers, so you set a key and pick a model instead of writing a custom provider.

Provider	OpenClaw integration	Auth env var	Notes
Cerebras	Official plugin `@openclaw/cerebras-provider`; default model `cerebras/gemma-4-31b` (preview); `cerebras/gpt-oss-120b` is the production reasoning model	`CEREBRAS_API_KEY`	Very high output speed
Groq	Official plugin `@openclaw/groq-provider`; suggested `groq/openai/gpt-oss-120b`	`GROQ_API_KEY`	Low latency; also audio transcription
Together AI	Bundled `together` provider	`TOGETHER_API_KEY`	Llama, DeepSeek, Kimi, GLM and more
Fireworks AI	Official plugin `@openclaw/fireworks-provider`	`FIREWORKS_API_KEY`	Open-weight models and routers
OpenRouter	Bundled `openrouter` provider; refs `openrouter/<author>/<slug>`	`OPENROUTER_API_KEY`	One key, many upstream providers
Hugging Face	Bundled `huggingface` provider	`HF_TOKEN` or `HUGGINGFACE_HUB_TOKEN`	Routes to many inference providers; see Hugging Face
Modal	Custom provider	your key	Serverless GPUs running your own vLLM

Per-token prices differ per model and change often, so check the linked pricing pages. OpenClaw's usage view shows the cost estimates it applies (openclaw gateway usage-cost).

Setting API keys

Put keys in the global env file ~/.openclaw/.env (or export them in the Gateway's environment):

~/.openclaw/.env
CEREBRAS_API_KEY=your-key-here
GROQ_API_KEY=your-key-here
TOGETHER_API_KEY=your-key-here
OPENROUTER_API_KEY=your-key-here

Keep the file readable only by you (chmod 600 ~/.openclaw/.env). Onboarding can also store keys for you, for example openclaw onboard --auth-choice together-api-key.

Plugin providers

Cerebras, Groq and Fireworks ship as official plugins. Install, then onboard:

openclaw plugins install @openclaw/cerebras-provider
openclaw onboard --auth-choice cerebras-api-key

Then select a model:

~/.openclaw/openclaw.json
{
  agents: {
    defaults: {
      model: {
        primary: "anthropic/claude-sonnet-5-5",
        fallbacks: ["cerebras/gpt-oss-120b"],
      },
    },
  },
}

Upstream pages: Cerebras, Groq, Fireworks, Together, OpenRouter.

Custom OpenAI-Compatible Provider

For any other host with an OpenAI-compatible API:

~/.openclaw/openclaw.json
{
  models: {
    providers: {
      "my-host": {
        baseUrl: "https://inference.example.com/v1",
        apiKey: "${MY_HOST_API_KEY}",
        api: "openai-completions",
        models: [
          { id: "meta-llama/Llama-3.3-70B-Instruct", name: "Llama 3.3 70B" },
        ],
      },
    },
  },
}

Custom providers must declare baseUrl and a models array. If you omit api, it defaults to openai-completions.

Inference Server Options

If you rent your own GPU, you need an inference server. These expose OpenAI-compatible endpoints (Ollama also has its own native API, which OpenClaw prefers):

Server	Best For	Throughput	Setup
vLLM	Production, multi-user	Highest	`pip install vllm` or the `vllm/vllm-openai` image
SGLang	High throughput, RadixAttention	High	`pip install sglang`
Ollama	Quick start	Good (few users)	Single binary
llama.cpp (`llama-server`)	CPU-only, edge, GGUF	Good	Binary or build
TGI (Hugging Face)	Enterprise	High	Docker
TabbyAPI	Consumer GPUs (EXL2)	Fast	Python
LM Studio	Desktop with GUI	Good	App download

vLLM (Recommended for Cloud)

pip install vllm

vllm serve Qwen/Qwen3-32B \
  --host 0.0.0.0 \
  --port 8090 \
  --max-model-len 32768 \
  --api-key "$VLLM_API_KEY"

--host 0.0.0.0 exposes the server on every interface. On a cloud box, keep --api-key set and put TLS or a tunnel in front (see Security).

~/.openclaw/openclaw.json
{
  agents: {
    defaults: {
      model: { primary: "gpu-box/Qwen/Qwen3-32B" },
    },
  },
  models: {
    providers: {
      "gpu-box": {
        baseUrl: "https://your-server.example.com/v1",
        apiKey: "${VLLM_API_KEY}",
        api: "openai-completions",
        timeoutSeconds: 300,
        models: [
          { id: "Qwen/Qwen3-32B", name: "Qwen3 32B", contextWindow: 32768, maxTokens: 8192 },
        ],
      },
    },
  },
}

vLLM supports NVIDIA (CUDA) and AMD (ROCm) GPUs. More options (discovery, "vllm/*", serving limits): vLLM provider.

TabbyAPI (Consumer GPUs)

TabbyAPI uses ExLlamaV2/V3 for speed on RTX 3090/4090 with EXL2/GPTQ quantized models. Follow the TabbyAPI README for install; it exposes an OpenAI-compatible API that you configure like the custom provider above.

OpenClaw Configuration Notes

Key rules

baseUrl: OpenAI-compatible servers usually end in /v1; Ollama's native API must not. Copy it from the provider's docs.
api: optional on custom providers (defaults to openai-completions). Use openai-responses only if the server supports /v1/responses.
models[].id is provider-local, with no provider prefix; you select it as provider-id/model-id.
Keep contextWindow at or below the server's --max-model-len.
Put keys in ~/.openclaw/.env (mode 600) and reference them as ${ENV_VAR}.

How OpenClaw picks a model

There is no global provider ranking. OpenClaw uses the explicit agents.defaults.model.primary, then the fallbacks in order when a provider errors, and rotates auth profiles within a provider. Per-agent overrides go in agents.entries.<id>.model. See Models.

Cost Comparison

When Self-Hosting Wins

You run agents 24/7 and per-token spend would exceed the GPU rental
Privacy is a hard requirement: no data leaves infrastructure you control
You already have GPU hardware or credits
You want to fine-tune models for your tasks

When API Keys Win

You need frontier-quality reasoning
Usage is intermittent: a rented GPU bills by the hour whether or not the agent is busy
You value time over money: hosted APIs need no maintenance
Long multi-step agent tasks, where tool-calling reliability matters most

A 24/7 RTX 4090 pod costs about $250-$650 a month (above). Compare that with your actual API spend from openclaw gateway usage-cost --days 30 rather than with guesses. For per-token prices of hosted frontier models and how to work out heartbeat cost, see Cost Management.

Multi-Model Routing

Route frequent, simple work to a cheap or self-hosted model and keep a frontier model for hard tasks:

~/.openclaw/openclaw.json
{
  agents: {
    defaults: {
      model: {
        primary: "anthropic/claude-opus-5-5",
        fallbacks: ["anthropic/claude-sonnet-5-5"],
      },
      heartbeat: {
        every: "1h",
        model: "gpu-box/Qwen/Qwen3-32B",
        isolatedSession: true,
        lightContext: true,
      },
    },
  },
  models: {
    providers: {
      "gpu-box": {
        baseUrl: "https://your-server.example.com/v1",
        apiKey: "${VLLM_API_KEY}",
        api: "openai-completions",
        models: [{ id: "Qwen/Qwen3-32B", name: "Qwen3 32B", contextWindow: 32768 }],
      },
    },
  },
}

The heartbeat then runs on your GPU box at no per-token cost. See Model Selection for routing by agent.

Recommended Models

tip

For a full model selection guide covering cloud, local and hybrid setups, see the Model Selection Guide.

By VRAM Budget

VRAM	Model	Use Case
8 GB (minimum)	Qwen3 8B, Llama 3.1 8B	Basic tasks, simple chat
12-16 GB	Qwen3 14B, DeepSeek-R1-Distill-Qwen-14B	General use, coding
24 GB (RTX 4090)	Qwen3 32B (Q4), Qwen3.5 27B (Q4)	Daily driver
40-80 GB (A100/H100)	Qwen3 32B (16-bit), Llama 3.3 70B (Q4/8-bit)	Production quality
192 GB (MI300X)	Llama 3.3 70B (16-bit)	Largest open models

Community Picks

Qwen3 32B / Qwen3.5 for coding, reasoning and tool use
DeepSeek-R1-Distill-Qwen-32B for planning and reasoning-heavy work
Qwen 2.5 Coder 32B for code-heavy work

These are community experience, not benchmarks. Upstream's llama.cpp page has maintained model recipes.

warning

Models under about 14B parameters often struggle with multi-step tool calling, and those under 8B hallucinate tool calls. For reliable agent work use 30B+ or keep a hosted fallback. Test real tasks before you make an open model your default; the best results with OpenClaw still come from frontier hosted models.

Security

If you run inference servers on cloud GPUs:

Use TLS (reverse proxy) or a private tunnel (Tailscale, WireGuard, Cloudflare Tunnel); never expose a raw inference port
Set an API key even on self-hosted servers (vLLM --api-key)
Restrict network access to the OpenClaw Gateway's address
Run openclaw security audit after changing providers
Remember that open models have no hosted safety filters; keep tool approvals and sandboxing on

See Security Hardening for full guidance.

Quick Decision Guide​

GPU Cloud Providers​

Dedicated GPU Cloud​

GPU Size Guide​

RunPod Setup​

RunPod Serverless​

Vast.ai Setup​

Inference-as-a-Service​

Setting API keys​

Plugin providers​

Custom OpenAI-Compatible Provider​

Inference Server Options​

vLLM (Recommended for Cloud)​

TabbyAPI (Consumer GPUs)​

OpenClaw Configuration Notes​

Key rules​

How OpenClaw picks a model​

Cost Comparison​

When Self-Hosting Wins​

When API Keys Win​

Multi-Model Routing​

Recommended Models​

By VRAM Budget​

Community Picks​

Security​

See Also​