A backend is a model that does the investigating. This guide covers four cases, from the smallest change to the largest:
- Another model on a provider ask-kit already knows
- A provider ask-kit does not list
- A local model
- A new agent CLI
For choosing between models, see Choose the model. The keys a backend can have are
in the config reference, and the loop that runs an api backend
is in Backends.
1. Another model on a known provider
Any model id the provider serves works. Setup lists a few as suggestions, and model is a plain
string in the config.
With setup
Write the answers as JSON, with the model as <preset>:<model id>. This adds qwen/qwen3-coder
on OpenRouter as an extra backend and keeps the current default:
{
"sources": ["claude-code", "codex"],
"clients": ["claude-code", "codex"],
"investigator": "openrouter:z-ai/glm-5.3-flash",
"extras": ["openrouter:qwen/qwen3-coder"],
"domains": ["sessions", "code"],
"keys": { "openrouter": { "env": "OPENROUTER_API_KEY" } }
}
ask-kit apply answers.json --dry-run
ask-kit apply answers.json
apply writes the whole config again from the answers, so list every source, client and domain
you want. Run ask-kit questions first: its existing field shows the current config, and the
ask-kit-setup skill walks an agent through the same conversation.
The backend’s name is the model id after the last /, lowercased: qwen3-coder.
By hand
Add an entry under backends.list in config.json:
"qwen3-coder": {
"kind": "api",
"preset": "openrouter",
"model": "qwen/qwen3-coder",
"key": { "env": "OPENROUTER_API_KEY" }
}
Check it
ask-kit doctor
The line for the backend ends in a status:
| Status | Meaning |
|---|---|
ok |
The provider answered GET <baseUrl>/models and lists the model. |
reachable, but <model> is not in its model list |
The provider answered and does not list that id. Check the spelling. |
HTTP <code> |
The provider refused. 401 usually means the key. |
| an error message | The provider could not be reached. |
Then ask one question on it, without changing the default:
ask-kit ask --domain code --backend qwen3-coder "where is the config file read?"
A model that fails here usually fails in one of three ways, which the journal
and ask-kit report show: missteps (it wrote tool-call markup instead of calling the tool),
cut off (replies hit the length limit), or out of steps or time. For missteps, try
"nativeTools": false in the entry, which has the model write fenced js blocks instead of calling
a tool.
To make it the default for one domain, set domains.<name>.backend. To make it the default
everywhere, set backends.default.
To compare it with the others on real questions, use the benchmark.
2. A provider that is not listed
ask-kit speaks the OpenAI chat-completions protocol to any api backend. A provider needs:
POST <baseUrl>/chat/completions, withmessages,temperature,max_tokensand, for the default mode,tools,tool_choiceandparallel_tool_calls.- A reply with
choices[0].messageholdingcontentand, for a tool call,tool_calls, and afinish_reason. - For cost and cache numbers, a
usageobject withprompt_tokens,completion_tokensand optionallyprompt_tokens_details.cached_tokens,prompt_tokens_details.cache_write_tokensandcost. GET <baseUrl>/models, only forask-kit doctor.
Config only
Use the custom preset and give the base URL. The URL and key variable below are placeholders:
"acme-large": {
"kind": "api",
"preset": "custom",
"baseUrl": "https://llm.example.com/v1",
"model": "acme-large-1",
"key": { "env": "ACME_API_KEY" }
}
For an endpoint that needs no key, add "noKey": true instead of key. ask-kit setup cannot
write this entry, because the custom preset takes no base URL there. Add it to config.json
by hand.
A preset
If you want setup to offer the provider, add it to API_PRESETS in
src/providers.mjs:
acme: {
label: "Acme API",
baseUrl: "https://llm.example.com/v1",
keyEnv: "ACME_API_KEY",
models: [{ id: "acme-large-1", note: "small and cheap" }],
},
| Field | Meaning |
|---|---|
label |
Shown in setup and in errors. |
baseUrl |
Where requests go. |
keyEnv |
The variable detect looks for, and the one used when a backend’s key gives nothing. |
keyFiles |
Optional list of NAME=value files detect also searches for the key. |
reportsCost |
true when the reply’s usage.cost is in dollars, so the footer and journal show a price. Leave it out otherwise: the cost shows as unknown. |
models |
Suggestions, cheapest first. The first is setup’s default for this provider. Each can carry providerOrder. |
local |
true for a server on this machine that needs no key. detect probes <baseUrl>/models. |
Then:
ask-kit detectlists the provider under API keys, found ifACME_API_KEYis set.ask-kit questionsoffersacme:acme-large-1among the investigator choices.- Add the provider to the table in Choose the model and to the config reference.
Provider-specific behaviour (OpenRouter’s usage and session_id, Ollama’s own endpoint) is
keyed on the preset name in endpoint and askApi in
src/ask.mjs. A provider that needs
a special request field gets a branch there.
3. A local model
Ollama and LM Studio are presets. Start the server, pull a model and run:
ask-kit detect # "Local model servers" lists what it found
Setup offers the first five models of a running server, leaving out names that look like embedding, rerank, OCR or speech models. By hand:
"local-qwen": { "kind": "api", "preset": "ollama", "model": "qwen3.5:9b", "contextTokens": 32768 }
For Ollama, ask-kit calls Ollama’s own /api/chat, because the OpenAI-compatible /v1 endpoint
cannot raise the default context of 4096 tokens, and the investigator’s prefix alone is about 7000.
contextTokens is the num_ctx it asks for; the default is 32768. ask-kit doctor compares it with
the model’s limit and says holds only N tokens of context when the model tops out lower. Pick a
model with a larger context in that case.
Small local models make more mistakes in this loop. Run the benchmark on yours before you rely on it.
4. A new agent CLI as the investigator
An agent CLI backend runs a coding agent headless with the REPL as its only MCP server. This is the largest change, and it needs the CLI to support all of the following:
- a non-interactive mode that prints a result,
- an MCP server given on the command line or in a throwaway config,
- a way to turn off its own tools, so the investigator works through the REPL only (the investigator is read-only: see CONTRIBUTING),
- ideally a way to fork or resume a session, so the instructions are loaded once.
The three that exist are askClaude, askCodex and askCopilot in
src/ask.mjs. Read the one closest to
your CLI first. Then:
-
Describe it in
AGENT_CLISinsrc/providers.mjs:"acme-cli": { label: "Acme CLI (your Acme plan)", bin: "acme", forks: false, models: [{ id: "", note: "Acme's configured default" }], },binis whatdetectlooks for.forksistruewhen the CLI can copy a warm session, and decides what setup says about speed. A model withid: ""means the CLI’s own default. -
Write
askAcmeinregisterAsk, with the signature of the others,(b, question, timeoutMs, cont, signal). It must:- run the CLI with
run(be.bin ?? "acme", args, { cwd: WARM_CWD, env: childEnv(), timeoutMs, signal }), so it starts in the state folder (its own transcripts then land where the sessions domain skips them) and without the caller’s session variables; - give it the REPL server:
process.execPathwith[SERVER]and the environment fromreplEnv(inherit, ownId); - pass the domain’s instructions (
INSTRUCTIONS, a file) and, for the first question,D.primeif it can be warmed, or the instructions and question together asaskCopilotdoes; - append
rulesFor()to the question; - use
readLast(b)andwriteLast(b, ...)forcontinue: true, and thrownothing to continue: no earlier <b> question in this sessionwhen there is nothing to continue; - return
{ answer, ok, model, continued, ms, costUsd, usage: { input, cacheRead, output } }, andcellsandturnswhere the CLI reports them.costUsdisnullwhen the CLI does not report dollars.
- run the CLI with
-
Add it to
KINDSat the end ofregisterAsk:const KINDS = { "claude-cli": askClaude, "codex-cli": askCodex, "copilot-cli": askCopilot, "acme-cli": askAcme, api: askApi }; -
Probe its login in
detect()insrc/setup/detect.mjs: add an entry to theauthobject that returns{ ok, out }from a cheap status command, so setup can say whether it is signed in. Without one,signedInstays unknown and the choice is offered as ready. -
Check the names and the order.
backendForinsrc/setup/apply.mjsbuilds{ kind: "acme-cli", model }fromAGENT_CLISand needs no change.backendNamehas a case for each of the three existing CLIs (claude-haiku,codex,copilot); any other kind gets its model id as the name, or the kind when the model is empty. Add a case if you want another name.investigatorChoicesinsrc/setup/questions.mjsranks subscription CLIs with30 + 0for Claude,+ 2for Codex and+ 4for anything else, so yours ranks with Copilot. Change that expression to rank it elsewhere. -
Test it.
ask.mjshas no unit test, so use the real thing. Add the backend to a scratch config, run a question throughask-kit ask --domain code --backend acme-cli "<question>", a follow-up with--continue, and a cancel (Ctrl+C) to check the CLI process ends. Read the journal row:kind,ok,ms,usage. Then run the benchmark on it. -
Update the docs: Choose the model, the config reference’s
backendssection and Backends.
If you also want the same CLI’s transcripts readable, or the ask tool registered in it, that is a
separate step: Support a new coding agent.
Related
- Config file reference
- Run the benchmark: compare the new backend on the same questions.
- Find and fix a bad answer:
--backendreruns.