LLM inference

LLM inference is sold one chunk at a time. Each paid call returns the next chunk of a model's answer, about 32 tokens, so an agent that has enough text stops calling and stops paying.

The service has no model of its own. It streams from any OpenAI-compatible chat API: Ollama by default, a bundled llama.cpp server under Docker, or a hosted provider such as OpenRouter.

At a glance

How it works

1. Start. Send a prompt. The service checks the body before it asks for any money, so a bad request gets a 400 and costs nothing. A good one gets the 402.
2. Pay and read. Retry with a voucher?A message the agent signs for each call. It states the agent's running total, so the facilitator can check it against the deposit.. The service verifies it, starts the generation, waits for the first chunk, settles, and answers with chunk 0 and a stream id. Nothing is generated before this call is paid.
3. Continue. Send { "stream": "<id>" } with a new voucher for each further chunk. The last chunk has done: true.

What a chunk is

The service reads the provider's stream and counts each non-empty piece of text as one token. A chunk is released when it holds LLM_CHUNK_TOKENS pieces and the next one has arrived, or when the provider's stream ends. That makes done exact: the service never sends a full chunk marked done: false and then an empty one. A generation with no output at all gives one empty final chunk.

Chunks are delivered in order, and a stream's position moves only after a paid call was delivered. If a call fails after its chunk was ready, the next call gets the same chunk.

The route

Send prompt to start a stream, or stream to continue one, never both. Any other field is refused.

The answer is one chunk:

See the offer

An unpaid call shows the price. The 402 body is {}. The offer is base64 JSON in the PAYMENT-REQUIRED header.

curl -si http://127.0.0.1:4031/v1/generate \
  -H 'content-type: application/json' \
  -d '{"prompt":"Explain shielded payments in three sentences.","maxTokens":96}'

HTTP/1.1 402 Payment Required
Content-Type: application/json; charset=utf-8
PAYMENT-REQUIRED: eyJ4NDAyVmVyc2lvbiI6MiwiZXJyb3IiOiJQYXltZW50IHJlcXVpcmVkIiw...
Cache-Control: no-store

{}

{
  "x402Version": 2,
  "error": "Payment required",
  "resource": {
    "url": "http://localhost:4031/v1/generate",
    "description": "Streams qwen2.5:0.5b output, one chunk of about 32 tokens per paid call: send {prompt} to start, then {stream} to continue.",
    "mimeType": "application/json",
    "serviceName": "LLM inference",
    "tags": ["llm", "inference", "text"]
  },
  "accepts": [
    {
      "scheme": "prepaid-voucher",
      "network": "zcash:regtest",
      "amount": "200",
      "asset": "ZEC",
      "payTo": "uregtest1...",
      "maxTimeoutSeconds": 60,
      "extra": {
        "windowZat": "100000",
        "creditRatioBps": 2500,
        "creditCapZat": "200000",
        "zeroConfMinExpiryBlocks": 20,
        "termsVersion": "cb7f148af0fda19a"
      }
    }
  ],
  "extensions": {
    "z402": {
      "unit": "output chunk",
      "note": "The operator sees your prompts. Open-weights model."
    }
  }
}

The offer also carries the Bazaar's info (an example call) and a JSON Schema for the body under extensions.bazaar. See Protocol for every field. To read the header yourself, decode it:

curl -si http://127.0.0.1:4031/v1/generate -H 'content-type: application/json' -d '{"prompt":"hi"}' \
  | grep -i '^payment-required:' | cut -d' ' -f2 | base64 -d

Read a stream

These answers are illustrative. The stream id is random, and the text depends on the model.

{
  "stream": "jB8ajHfqaLW5NG0QGT0SeA",
  "index": 0,
  "text": "Shielded payments hide who pays whom and how much. For a software agent that matters, because a public ledger would show every purchase it makes",
  "done": false
}

{
  "stream": "jB8ajHfqaLW5NG0QGT0SeA",
  "index": 1,
  "text": ". Anyone could read its habits, its suppliers and its budget from the chain. A shielded payment keeps those details between the agent and the",
  "done": false
}

{
  "stream": "jB8ajHfqaLW5NG0QGT0SeA",
  "index": 2,
  "text": " seller, so the agent can buy what it needs without publishing a record of it.",
  "done": true,
  "finishReason": "stop"
}

Price and billing

One paid call is one chunk at 200 zat, so each token costs about 6 zat. Each call raises your voucher total by the price, and the vouchers draw down the deposit you made once for the window (see Prepaid vouchers).

Set maxTokens to what you need. It ends the stream at that many tokens, so it also caps the most a stream can cost:

Stopping early costs nothing more. The service keeps generating in the background up to maxTokens, so stopping saves your vouchers and not the operator's compute. A stream with no call for LLM_STREAM_IDLE_MS (10 minutes) is dropped and its generation is stopped.

Limits and refusals

Everything the service can decide without calling the model is refused on your unpaid first attempt, before any voucher exists. A refused call is never charged.

Failures after payment

A few failures cannot be seen earlier. The facilitator does not settle them, but the stock client cannot tell a failed call from a paid one, so it counts the call and its next voucher pays for it.

Make a paid call

The call buys one chunk at a time and stops when it has 200 characters. It needs a funded wallet and a running service: Make a paid call names the stack that paid-fetch.ts reaches, and Pay from an agent explains the setup.

import { close, paidFetch } from "./paid-fetch.js";

const url = "http://127.0.0.1:4031/v1/generate";

interface Chunk {
  stream: string;
  index: number;
  text: string;
  done: boolean;
  finishReason?: string;
}

async function next(body: { prompt: string; maxTokens?: number } | { stream: string }): Promise<Chunk> {
  const res = await paidFetch(url, {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify(body),
  });
  if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
  return (await res.json()) as Chunk;
}

try {
  // The first call opens the stream. Each later call buys the next chunk.
  let chunk = await next({ prompt: "Explain shielded payments in three sentences.", maxTokens: 96 });
  let text = chunk.text;
  while (!chunk.done && text.length < 200) {
    chunk = await next({ stream: chunk.stream });
    text += chunk.text;
  }
  console.log(text);
} finally {
  await close();
}

import { wrapFetchWithPayment, x402Client } from "@x402/fetch";
import { LightwalletdTxStatus, PrepaidVoucherScheme, ZalletRpcWallet } from "@z402-net/client";
import { createLightwalletdClient } from "@z402-net/lightwalletd";
import { NETWORK_REGTEST } from "@z402-net/protocol";

// Zallet pays the deposit and lightwalletd (Zaino) reports its status.
// These are the Z3 regtest defaults.
const wallet = new ZalletRpcWallet({
  url: "http://127.0.0.1:50232",
  user: "zebra",
  password: process.env.ZALLET_PASSWORD ?? "zebra",
});
const lwd = createLightwalletdClient({ url: "http://127.0.0.1:28137" });
const scheme = new PrepaidVoucherScheme({ wallet, chain: new LightwalletdTxStatus(lwd) });

// The one line that adds Zcash to the stock x402 client
export const paidFetch = wrapFetchWithPayment(fetch, new x402Client().register(NETWORK_REGTEST, scheme));

/** Lets a deposit in flight finish, then closes the wallet and the chain client. */
export async function close(): Promise<void> {
  await scheme.close();
  lwd.close();
}

The first paid call also makes the deposit, so it can take tens of seconds. Later calls only sign a voucher. See Prepaid vouchers.

Privacy

• The operator sees the prompt, the system prompt and the output. The model provider behind the service sees them too.
• The facilitator sees the payment, not the request: the voucher, the price and the route's public URL. It never receives your prompt.
• A stream id is 16 random bytes, base64url. It is the only thing that names a stream, so treat it as private for as long as the stream lives.
• Under the default model, the prompt goes to a small open-weights model that the operator runs. With a hosted LLM_BASE_URL, it also goes to that host.

See Privacy model for what the payment itself reveals.

Run it yourself

The service reads services/llm/.env and the environment, and a variable set in the environment wins. An empty value counts as unset. A bad value stops the start with an llm: ... line that names it.

ollama pull qwen2.5:0.5b
cp services/llm/.env.example services/llm/.env   # set the facilitator URL and key, and Z402_PAY_TO
pnpm --filter @z402/service-llm start

The ready line goes to stdout. Logs go to stderr.

llm listening on http://127.0.0.1:4031 model=qwen2.5:0.5b network=zcash:regtest

To use a hosted provider, change three lines:

LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=your-provider-key
LLM_MODEL=qwen/qwen-2.5-7b-instruct

Settings

Shared by every service:

Its own:

Under Docker, compose reads docker/env/llm.env first, then services/llm/.env, which wins. The Docker defaults point at the bundled llama.cpp server (qwen2.5-0.5b-instruct) and cap prompts at 1,600 characters so they fit its context. The first start downloads the model. See Deploy with Docker.

Readiness

The service is ready while the provider lists the model. It asks about every 10 seconds and logs each change. GET /health answers 200 when ready and 503 when not, and the service lists its route in the Bazaar only while it is ready. See Discovery.

{
  "ok": true,
  "service": "LLM inference",
  "ready": true,
  "network": "zcash:regtest",
  "priceZat": "200",
  "callsToday": 3,
  "routes": [{ "method": "POST", "path": "/v1/generate", "unit": "output chunk", "priceZat": "200" }]
}

callsToday counts paid calls delivered since 00:00 UTC. It is an aggregate and holds nothing per client.

Next steps

References

• services/llm/src/service.ts: the route, its checks and the paid handler
• services/llm/src/streams.ts: chunking, the stream registry and idle cleanup
• services/llm/src/provider.ts: the OpenAI-compatible client and model readiness
• services/llm/src/config.ts: the settings and their limits
• services/llm/README.md: the route, refusals and settings
• services/llm/.env.example: every variable with its default
• packages/service-kit/src/service.ts: the kit that adds the 402, /health and the Bazaar declarations
• docs/superpowers/specs/2026-10-27-z402-services-design.md: section 4.1, the service's contract
• docker/env/llm.env: the Docker defaults
• Ollama: OpenAI compatibility
• llama.cpp server