OmniscioGuidesClaude Code on a local model
Claude Code with a local model or an LLM gateway
The setup is three settings: a proxy address, a credential, and a model name, plus the model's context size. What costs people an evening is everything after that. A model that is not Claude does not announce what it cannot do, so prompt caching, exact token counts, web search and your subscription's limits all change behaviour without printing an error.
| What you want | Read this |
|---|---|
| The whole shape of it in one page | The short answer |
| Why a local model needs a translator | The three API formats |
| The settings, and naming the model | The variables that do the work |
| The model answers oddly | Unrecognised models |
| Caching, token counts and search went quiet | What stops working silently |
A 400 naming a field | The 400s |
| It keeps dropping the connection | Slower than the timeout |
| Traffic that skips the gateway | Background requests |
| Three checks before blaming the model | Verifying the setup |
| Doing this inside Omniscio | Custom Providers |
Sources read November 10, 2026. By the Omniscio team. Updated .
The short answer
Claude Code has no setting that says "use a local model". It has a base URL, and whatever answers at that URL is treated as the Claude API.
So the shape of every setup on this page is the same: run something that serves the Anthropic Messages API in front of the model you actually want, then set ANTHROPIC_BASE_URL and a credential. The proxy is doing the real work, because Claude Code sends an Anthropic-shaped request and a local model does not accept one.
Anthropic's own limit
Anthropic states the limit of this plainly in its gateway documentation: it does not endorse, maintain or audit third-party gateway products, and it does not support routing Claude Code to non-Claude models through any gateway.
That sentence is the honest frame for everything below. This is a setup people run successfully; it is not a configuration Anthropic will help you fix, and a bug you find in it is yours to work around.
What happens to your subscription
A gateway credential displaces your claude.ai login while it is active: requests carry the gateway's credential, the subscription's usage limits do not apply to them, and the tokens are billed to whoever owns the credential the gateway forwards.
Setting the base URL alone does not do that. With no credential variable set, the URL routes traffic but your saved login stays the active credential, so the gateway sees requests it cannot bill for and your limits still apply.
Why a local model needs a translator in front of it
A gateway has to expose one of three API formats for Claude Code to talk to it:
- The Anthropic Messages API at
/v1/messages, the general one, selected byANTHROPIC_BASE_URL. - Amazon Bedrock's InvokeModel shape.
- Google Cloud's Agent Platform shape.
There is no fourth format, and the OpenAI chat-completions API that almost every local server speaks is not on the list.
So something has to translate
Ollama's own endpoint answers on the OpenAI format, so it cannot be the thing ANTHROPIC_BASE_URL points at. A proxy that serves /v1/messages and forwards to Ollama can.
Anthropic names one in its environment-variable reference, calling LiteLLM an Anthropic-compatible gateway, which is why it appears in nearly every write-up of this setup. Any proxy that serves the Anthropic Messages format and can reach your model works the same way.
Two shapes that belong to another page
A gateway that speaks Bedrock's or Vertex's own format instead uses ANTHROPIC_BEDROCK_BASE_URL or ANTHROPIC_VERTEX_BASE_URL with the matching CLAUDE_CODE_USE_* switch. The Bedrock, Vertex and gateway guide covers those, the credential variables, and where to put them so a secret stays out of a committed file.
This page does not repeat that. It starts where the endpoint already speaks the Anthropic format and the model behind it is not Claude.
| What you want to run | What has to sit in front of it | The variable you set |
|---|---|---|
| A model on your own machine (Ollama, llama.cpp, vLLM) | A proxy that serves /v1/messages and reaches the local server | ANTHROPIC_BASE_URL pointing at the proxy, never at the model server |
| A hosted third-party model (OpenRouter, DeepSeek, GLM, Kimi) | The provider's own Anthropic-format endpoint, or a proxy that serves one | ANTHROPIC_BASE_URL plus that provider's key |
| Claude, through a model router you operate | Any gateway serving the Anthropic Messages format | ANTHROPIC_BASE_URL plus the gateway credential |
The variables that do the work
Four settings carry the whole configuration, and each one answers a different failure. The base URL says where requests go, the credential says who is asking, the model name says which model, and the context-window variable says how much room that model has.
The first two are the pair the Bedrock and gateway guide describes in full, including which credential variable matches which HTTP header and why a settings file beats a shell export. The last two are specific to running something other than Claude, so they are here.
export ANTHROPIC_BASE_URL=http://127.0.0.1:4000
export ANTHROPIC_AUTH_TOKEN=sk-your-gateway-key
export ANTHROPIC_MODEL=your-model-nameIf your proxy asks for no key at all, still set the credential variable: without one, Claude Code keeps your claude.ai login as the active credential and can ask you to sign in, and the request carries a token your local proxy will ignore.
Paste them into a shell for one session, or into the env block of your settings file to make them stick. The Omniscio section covers what the app does with them instead.
Name the model, or you get whatever the proxy defaults to
Claude Code sends a model ID with every request, and a proxy that does not recognise that ID either maps it or refuses it. Set ANTHROPIC_MODEL to the name your proxy serves, or pass --model for one session.
If the name is not one Claude Code knows, it prints a startup warning and assumes a 200,000-token context window, which is where the next problem comes from.
Tell Claude Code how big the window is
Claude Code auto-compacts a conversation when it reaches the window it assumes. So a model with a smaller real window starts failing before compaction ever triggers, and a model with a bigger one compacts far too early.
For an ID it does not recognise, CLAUDE_CODE_MAX_CONTEXT_TOKENS sets the real number in tokens. Append [1m] to the model name instead if the window is a million tokens: Claude Code assumes 1M for an ID carrying that suffix.
Which IDs the variable applies to is fiddly enough to have its own section on Anthropic's page, and the context window guide sets out all three cases.
Make the model appear in the picker, if its name lets you
By default /model lists Claude Code's built-in models only. Setting CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1 makes it query your gateway's /v1/models endpoint at startup and add those names to the picker.
Anthropic leaves it off by default because a gateway backed by one shared key would otherwise show every user every model that key can reach.
There is a catch that matters more here than anywhere else: discovery keeps an entry only when its id contains claude or anthropic, matched case-insensitively, and ignores everything else.
So qwen3-coder or deepseek-v4-flash never appear, however healthy your proxy is, while my-gateway-claude-sonnet-4-6 does. Give the model an alias containing one of those two words if you want it in the picker, and set it with ANTHROPIC_MODEL or --model if you would rather not.
If the picker is short a model for a different reason, the models missing from the picker guide covers the allowlist half.
The proxy's own config: LiteLLM's model_list
A gateway's config names the models it will answer to. In LiteLLM's own documented shape, each entry has a model_name that the client asks for and a litellm_params block that says which upstream serves it, and the proxy is started with litellm --config pointing at that file.
Fronting Ollama, a config of that shape looks like this, with LiteLLM's own Ollama prefix and the address your local server listens on. Installing Ollama and LiteLLM is outside this page; their own documentation covers it.
model_list:
- model_name: my-local-model
litellm_params:
model: ollama_chat/llama3.1
api_base: http://127.0.0.1:11434Then litellm --config config.yaml starts the proxy, ANTHROPIC_MODEL=my-local-model is the name Claude Code asks for, and ollama_chat/llama3.1 is what LiteLLM calls upstream. LiteLLM's own examples reach the proxy at port 4000, which is where the http://127.0.0.1:4000 in the exports above comes from.
The model is answering, and it is not doing what you expect
A model that was never trained on Claude Code's harness reads the same prompt differently, and the symptom is not an error. The harness is everything the CLI wraps around your message: the instructions, the skill listings and the tool descriptions.
An open report on Anthropic's issue tracker describes exactly this, with no mapping saying what the model should behave like: the local model misreads that injected content, while the CLI itself reports only that the model is unrecognised. The report's point is that the warning undersells the problem.
The nearest thing to an escape hatch: modelOverrides
The setting that tells Claude Code what an unrecognised ID is supposed to be is modelOverrides, or a modelPicker row with behavesAs set to a model Claude Code knows.
The entry maps a Claude model ID to yours, so "claude-sonnet-4-6": "your-model-name" makes Claude Code send your ID the request fields and capabilities it would send for Sonnet 4.6.
That is the closest thing to a supported escape hatch here, and it is a declaration you make, not a measurement: if the model does not really behave like the one you named, the mismatch shows up as odd answers rather than as an error.
Subagents and background tasks ask for a second model
Some requests are not your conversation. Session titles, summaries and other background work run on a model Claude Code chooses, and subagents run on theirs.
Behind an ANTHROPIC_BASE_URL gateway that background model is the main model, so your proxy sees the name you set. There is one exception, and it is easy to hit.
An apiKeyHelper is a command Claude Code runs to fetch your credential, and it counts as a key rather than as a token. So when the credential comes from one, or sits in ANTHROPIC_API_KEY, and ANTHROPIC_AUTH_TOKEN is unset, background work goes to Claude Code's default Haiku model instead.
A proxy that has never heard of that Claude ID then fails on a request you never typed. Putting the key in ANTHROPIC_AUTH_TOKEN avoids it, and CLAUDE_CODE_SUBAGENT_MODEL pins the subagent model to one your proxy serves.
What stops working without telling you
These four cost real money or real time and print nothing. Anthropic's gateway documentation lists each one with the symptom, and every symptom is the absence of something rather than an error.
- Prompt caching. Claude Code marks the parts of a request that should be cached. A proxy that drops the markers, or converts the message blocks to plain strings, still gets working answers, but every turn bills as fresh input. You see it as a high input-token count and no cache activity in the usage fields. The prompt cache guide covers what the markers do; on a local model there is often no cache to hit at all.
- Token counting. The exact count comes from a
count_tokensendpoint, which is the only optional endpoint in the format. Without it Claude Code estimates from characters, so/contextshows approximate figures. - Web search and fast mode. Feature availability behind a gateway matches the provider the gateway forwards to. Anthropic's own per-provider table marks web search and fast mode, its faster Opus setting, as unavailable on Amazon Bedrock, and a non-Claude upstream is further from the Anthropic API than that. Both are features Anthropic runs on its own side of the connection, so neither survives a model that answers anywhere else.
- Remote Control. While
ANTHROPIC_BASE_URLpoints at a host other thanapi.anthropic.com, Claude Code turns Remote Control off, whatever the gateway forwards. The Remote Control guide covers what returns when you unset it.
Prompt caching is the one to watch on a paid upstream. It is also the easiest of the four to check, since the usage numbers in a response carry the cache fields.
The 400s a non-Claude upstream sends back
Claude Code sends the request shape it sends to the Anthropic API, including fields that arrived in recent releases. A proxy that forwards the body unchanged passes them to a model that does not implement them, and the answer is a 400 naming a field you have never heard of.
Anthropic groups them in one place. The pattern to recognise is a 400 naming a field rather than a message about your prompt.
| The 400 names | What it means | The fix Anthropic documents |
|---|---|---|
thinking, or a tag such as adaptive | The upstream does not accept adaptive reasoning, which Claude Code requests for current models and for IDs it does not recognise | Move the upstream on, or set CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1 on Opus 4.6 or Sonnet 4.6 |
context_management, output_config, or Extra inputs are not permitted | A beta field arrived without the capability behind it | CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 suppresses most pre-release fields, but not effort, which is retried instead |
An unexpected system role in messages | Claude Code sends reminder text as mid-conversation system entries for unrecognised IDs | Claude Code retries once without them, and a modelOverrides entry pointing at a model released before Opus 4.8 stops them being sent at all |
| A context or token limit in the proxy's own words | The proxy enforces a smaller window than the model really has, so Claude Code does not recognise the error as a too-long prompt and does not compact | Run /compact, and set CLAUDE_CODE_AUTO_COMPACT_WINDOW to the proxy's real limit |
Three of them recover on their own
Three of these recover on their own on a current version. When the upstream rejects the thinking type, or rejects mid-conversation system messages, Claude Code retries once in the other shape and keeps that shape for the rest of the session.
It also retries once without context_management and the beta header that travels with it, from v2.1.304 on, and once without output_config.effort and its beta header.
A proxy too small to serve your model's real window does not recover, which is why the last row is the one to fix rather than re-run.
A local model is slower than the timeout
Claude Code reads a response as it streams, event by event, and expects the full sequence through the closing event. Two things follow.
- A proxy that buffers the whole answer before sending it makes Claude Code stall.
- A stream that ends before that final event reads as a dropped connection rather than a short answer.
Neither is a model problem, and neither shows up as a network error.
The timeout most likely to bite
The documented timeout most likely to bite a local model is the five-minute body idle timeout, which aborts a streaming response when no bytes arrive. Anthropic's own description of API_FORCE_IDLE_TIMEOUT gives a slow gateway or a local model pausing between chunks as the reason the variable exists, and setting it to 0 turns that timeout off.
The separate streaming idle watchdog still runs, so a long silent pause can still be aborted; the stream idle timeout guide has the detail.
Two smaller limits to set deliberately
CLAUDE_CODE_MAX_OUTPUT_TOKENS should sit under whatever your model or proxy will actually emit, or the request is rejected for asking for more than is available.
And if the proxy enforces its own context ceiling, CLAUDE_CODE_AUTO_COMPACT_WINDOW is how you make compaction happen before the proxy refuses instead of after.
The two context variables answer different problems. For an ID it does not recognise, CLAUDE_CODE_MAX_CONTEXT_TOKENS sets the real window in tokens. CLAUDE_CODE_AUTO_COMPACT_WINDOW moves the point where it compacts, for a proxy that refuses before that window is full.
The traffic that does not go through your gateway
A gateway carries model requests. Claude Code also makes background requests outside that path: version checks, telemetry, release notes. On a machine whose only egress is the gateway those fail and show up as blocked connections in egress monitoring, which reads like a broken setup.
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 turns that traffic off, and Anthropic notes it also disables auto-updates, so you need another way to update.
This is also the variable people reach for when they want a session that talks to nothing but their own model, and it does not promise that.
- It removes the nonessential calls, but the WebFetch domain safety check still calls
api.anthropic.com; turn that off separately withskipWebFetchPreflightif your network blocks the host. - Gateway model discovery is unaffected, and it queries only your gateway.
- The fast-mode check is suppressed rather than redirected. Unless fast mode was already enabled on that machine,
/fastreports it unavailable there.
Three checks before you blame the model
Start a session and run /status. It reports the base URL and which credential source is active, and those two lines answer most of the setup questions on this page: a missing base URL line means the variable did not reach the process, and a line naming a claude.ai account means no gateway credential was found.
- Which model is actually answering. The status output names the model for the session. If it is not the one you set, a proxy default or a pin elsewhere won the argument.
- Whether the window was assumed. A startup warning that the model is not recognised is the tell: Claude Code is running it as a 200,000-token model until you set
CLAUDE_CODE_MAX_CONTEXT_TOKENSor an override. - Whether caching is happening. One turn is enough. A response whose usage shows input tokens and no cache activity was not cached, and on a paid upstream that is the bill you will notice.
If a request never reaches the gateway at all, that is a different family of problem: a refused socket, a DNS failure or a certificate error. The connection errors guide covers those with the fix for each, and the Bedrock and gateway guide has a one-request test that proves a gateway is reachable and authenticated before you configure anything.
Running a local or third-party model in Omniscio
Omniscio is the desktop app we make for running many Claude Code sessions at once. It has a feature that fits this page directly: Custom Providers, where you add a display name, a base URL, an API key and the model IDs that endpoint serves, in either an OpenAI-compatible or an Anthropic-compatible format.
The provider then appears as a named entry in the session model picker, so one session can run on your local model while another runs on Claude.
The format you pick decides what the app does with your URL
An Anthropic-compatible entry is handed to the session as ANTHROPIC_BASE_URL, the same pair this page has been describing, so it needs a base URL that already serves /v1/messages.
An OpenAI-compatible entry goes through a local proxy the app runs, which is the translator in the section above: that is the format Ollama's own endpoint speaks, and it is the reason a model server at 127.0.0.1 can be added as a provider.
The base URL check allows plain HTTP only for a loopback address — localhost, the 127.x.x.x range, or ::1 on the same machine — because anything else would send your key in the clear; a model server on another machine in your house or office needs HTTPS.
The trade against the terminal route
The terminal gives you every variable Claude Code reads, and the app gives you one provider entry that every session can pick from, at the cost of the variables it does not expose.
OpenRouter is built into the app as a ready-made custom provider, so reaching the hosted models people usually mean by this question takes a key and no configuration. A custom provider can also be switched off entirely: the Custom Providers card in Settings carries a switch that hides every custom entry, OpenRouter included, and refuses a launch against one.
One behaviour that surprises people coming from a terminal
A session Omniscio starts does not inherit ANTHROPIC_BASE_URL from your shell: the app removes any ambient copy before it spawns the CLI, deliberately, because a leftover export would silently redirect every session to the wrong host.
So the export ANTHROPIC_BASE_URL=... from the top of this page has no effect on a session started in the app. Put the endpoint in Custom Providers, or in the env block of your settings file, which the CLI reads itself.
If you are comparing ways to run several sessions at once, the features page lists what the app does and the download page has the installers.
Sources
Every variable, message and behaviour on this page rests on one of these, with the two LiteLLM pages covering the proxy-side example rather than Claude Code itself. The date beside each is the day we read it.
- Other LLM gateways, for what a gateway provides, the statement that Anthropic does not support non-Claude models through one, and the subscription rule. Read November 10, 2026.
- Connect Claude Code to an LLM gateway, for the base URL, the credential variables, the settings-file rule, the request test, and Remote Control being unavailable behind a non-Anthropic host. Read November 10, 2026.
- Gateway compatibility guide, for the three API formats, the request fields a gateway must forward, prompt caching and token counting, streaming behaviour, and the model for unrecognised IDs. Read November 10, 2026.
- Environment variables, for
API_FORCE_IDLE_TIMEOUTand its local-model wording,CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERYand its LiteLLM example,CLAUDE_CODE_MAX_CONTEXT_TOKENS,CLAUDE_CODE_SUBAGENT_MODELandCLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC. Read November 10, 2026. - Model configuration, for the unrecognised-model warning,
modelOverridesandbehavesAs, and correcting the assumed context window. Read November 10, 2026. - Feature availability, for availability matching the provider behind a gateway, and for the features Claude Code turns off behind a non-Anthropic base URL. Read November 10, 2026.
- Errors, for the too-long-prompt wording and retry behaviour behind a gateway. Read November 10, 2026.
- LiteLLM: Anthropic /v1/messages and LiteLLM: Ollama, for the
model_listshape, thelitellm --configstart, and theollama_chat/model prefix andapi_basethe example uses. Read November 10, 2026. - GitHub issue 92449, open, for a local model misreading the harness content without an explicit mapping. Read November 10, 2026.
- GitHub issue 91746, closed, for a report that subscription auth ignored
ANTHROPIC_BASE_URLand requests never reached the proxy. Read November 10, 2026. - GitHub issue 68995, closed, for a multi-turn 400 when Claude Code was routed through OpenRouter to a third-party model while the first turn succeeded. Read November 10, 2026.
- One capture run on this machine on November 10, 2026:
claude --version, which printed2.1.319 (Claude Code).
This part of the product moves with every release, and the gateway pages carry version notes for a reason: the fields Claude Code sends, and what a gateway has to forward, change with the CLI. A copy of this page from a few months back is worth re-reading against the live pages before you trust a variable name.
Common questions
Can Claude Code run on Ollama or a local model?
Not directly. Ollama serves the OpenAI format, and Claude Code speaks only three API formats, the general one being the Anthropic Messages API at /v1/messages. A proxy such as LiteLLM serves that format and forwards to Ollama, and Claude Code points at the proxy. Anthropic does not support routing Claude Code to non-Claude models, so the setup works without being supported.
Does a Claude Pro or Max subscription pay for a local or third-party model?
No. A gateway credential replaces your claude.ai login for those requests, the subscription's limits stop applying, and the tokens bill to whoever owns the credential. The setup never spends a subscription's capacity on another vendor's model.
Why do my gateway requests come back with a 400 about thinking, effort or context_management?
The upstream does not implement a request field Claude Code sends to the Anthropic API. CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 removes most of them but not effort, CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1 removes the adaptive-thinking field on Opus 4.6 and Sonnet 4.6, and the table above matches each 400 to its own fix.
Why does the model answer the first message and then fail on the second?
Something needed on turn two was not needed on turn one. A thinking block has to survive across turns, and Claude Code sends mid-conversation system messages to model IDs it does not recognise. A rejected system message is retried once without it; a rejected thinking signature is handled by dropping the earlier thinking blocks and retrying; a proxy that rejects the field itself has no recovery, and the table above matches it to its fix.
Why does a slow local model keep dropping the connection?
No bytes arrived for too long. Claude Code has a five-minute body idle timeout and a separate streaming watchdog, and a local model can be silent longer than a hosted one. API_FORCE_IDLE_TIMEOUT=0 turns the first off; a proxy that buffers the whole answer instead of streaming it will still stall.