Jev Router: up to 42% fewer tokens, same solve rate

On one pytest issue, Jev Router used 42% fewer output tokens than fixed high reasoning effort and finished in about 40% less time, and both solved all six attempts. Across 44 SWE-bench Verified attempts with GPT-6 Astra, it averaged 14% fewer output tokens, 20% fewer input tokens, and 9% less time, and solved all 44, the same as fixed high.

Jev Router is an OpenCode plugin that picks the reasoning effort for every request a coding agent makes. You pick the model. Before every call, the router asks Jev how much reasoning the next step needs and sets the effort to match. It only works with OpenAI's GPT-6 models right now, for reasons covered below.

Why one effort level is the wrong setting

A coding agent makes dozens of model calls per task. Some are hard, like working out why a test fails only on one Python version. Most are routine. Read the next file, run the tests again, apply the edit you already planned. Fixed high effort spends a lot of reasoning on the routine calls. Fixed medium saves tokens and sometimes fails the hard ones.

Switching effort by hand per call isn't practical. Until recently it was also expensive. Changing effort rewrote the start of the prompt, so the next call reprocessed the entire history at the uncached rate. The previous post covers the new mechanism that fixes this. On GPT-6 you can leave the request's effort alone and add a configuration_update item to the conversation, and the cached prefix still matches. Jev Router is built on that.

How a request flows

The plugin registers a jev-router provider in OpenCode with Astra, Luna, and Sol models. Every Responses API request from OpenCode passes through the plugin before it reaches the upstream, which can be the OpenAI API or any gateway that serves the same /v1/responses endpoint.

For each request the router does four things.

1. Classify. It builds a small state from the request with the latest user text, recent assistant progress, up to eight tool results with their names and error flags, and a summary of recent failures. Each piece is truncated. It asks Jev one question, "Select the reasoning effort for the next model call," limited to the efforts that model supports.
2. Rewrite. The request-level reasoning.effort is pinned to a stable base (medium by default). If Jev's choice differs from the effort currently in effect, the router inserts a configuration_update before the current user message, or after the tool results when the agent is continuing a tool loop. If the effort hasn't changed, nothing is inserted.
3. Forward. The rewritten request goes upstream and the response streams back to OpenCode unchanged.
4. Log. An optional JSONL log records the selected effort, Jev's latency, any fallback, and the upstream's input, cached, and output token counts. It never records prompts, tool output, or credentials.

Jev added about 570 ms per call on average in the holdout below. That time is included in the reported task durations.

Keeping the prefix stable

The update items have to stay in place on every later request. OpenAI's docs say to replay each update in its original position. If one disappears, the prefix changes at that point and everything after it misses the cache.

OpenCode doesn't know about the updates. It builds each request from its own copy of the conversation, which doesn't contain the items the router inserted. So the router keeps its own record. For each session it stores hashes of the input items and the positions and values of the updates it inserted. On the next request it finds the longest stored history that the new input extends and puts the old updates back where they were. The record is kept in memory, holds up to 256 histories for ten minutes, and stores only hashes and positions.

Lineage can be lost. A router restart, a compacted or edited history, or a conversation that branches starts a fresh record. The next request can then miss the cache from the first missing update onward, and the new record keeps the prefix stable again after that.

To check that the replay worked, I ran a controlled comparison with a scripted effort sequence against the live upstream. Across 42 live requests, including warm-up passes, in two alternating fixed and adaptive trials, the cached share of input was 0.869 at fixed effort and 0.868 with effort changing. Each arm had one request with no cache hit.

Failure handling

The classifier is on the critical path, so it has a hard budget. One deadline, 4 seconds by default, covers the whole classification including one retry. If Jev times out, errors, or returns an effort the model doesn't support, the request goes out at fixed high effort. Fallback can also be set to reuse the previous decision, or to fail the request. If the client disconnects, the router cancels classification and never starts generation.

Results

The evaluation ran GPT-6 Astra in OpenCode on SWE-bench Verified tasks, each attempt in a fresh container with the same image, plugin, upstream, and grader. The main comparison has two parts, both against fixed high effort. A development set of four issues was run twice each, and a holdout of six issues was chosen before any results and run six times each.

Token counts are means per attempt. Output includes reasoning tokens. About 88% of input was cached for Jev and 89% for fixed high, so most of the input saving is in cheaper cached tokens.

The largest per-task saving was on pytest #5262. Both arms solved all six attempts, and Jev averaged 1,112 output tokens against 1,908 for fixed high, 42% fewer, in about 40% less time. The saving wasn't uniform. In an earlier two-task pilot, Jev was faster on a Flask issue (65 seconds against 126) but slower on a Django issue (232 against 221) while still using fewer output tokens.

Across the holdout, Jev chose high 375 times, medium 159 times, and low 42 times. It never chose xhigh. About a third of its calls ran below high.

A harder case

Every holdout attempt passed in both arms, so the holdout can't show whether Jev gives up quality on hard problems. I looked for a task where effort changed the outcome. Pytest #5787 was one, run five times per arm.

Jev ran 92 of its 127 calls on this task at high and matched high and xhigh on solves with less output than either. Two things weaken that. I picked the task after seeing medium fail on it, so it isn't an unbiased estimate of anything. And the classifier fell back 7 times in the Jev arm, which sends the call at high, so some of those high calls weren't Jev's choice.

A later rerun with fallback turned off didn't reproduce the gap. Medium solved 4 of 5, high 5 of 5, xhigh 4 of 5, and Jev 5 of 5. Jev used slightly less output than high (4,149 against 4,238) and took longer (248 seconds against 228). So this task shows Jev choosing high when a step looks hard. It doesn't show that medium reliably fails there.

What these numbers don't show

• Only Astra is in the main comparison. Sol ran in the two-task pilot, which is too small to conclude anything, and Luna hasn't been run on SWE-bench at all.
• SWE-bench Verified is public, so models may have seen these repositories and fixes. The task counts are small, and repeated attempts on one issue aren't independent tasks.
• The holdout issues were ones every arm solved every time. A workload with more hard failures could show a different tradeoff.
• Token counts aren't dollars. Cached and uncached input are priced differently, and a subscription upstream like Codex bills against a quota.

I also tried to improve the Jev prompt. The best candidate from a small search solved 35 of 36 on the holdout against 36 of 36 for the current prompt, with almost identical output, so the current prompt stayed.

Why only GPT-6

The router depends on configuration_update, and only OpenAI's GPT-6 models in standard, single-agent mode accept it. Pro mode, the pro model IDs, and automatic truncation are rejected before classification, because each one either doesn't support updates or can drop the inserted items.

Anthropic now has an equivalent, a per-message effort change on Claude Fable 5.1, Mythos 5.1, Opus 5.5, and Opus 5. Supporting it would mean a second rewrite path for the Messages API and a new cache measurement, since the OpenAI result doesn't transfer. Other providers don't document whether an effort change keeps the cache, so I'd have to measure each one before routing on it.

Try it

Add the plugin to your OpenCode config with a Jev key and a Responses upstream. The same config works in OpenCode 1.18.29 or newer and in OpenCode 2.

{
  "plugin": [
    [
      "@robertn702/opencode-jev-router@0.4.0",
      {
        "jevApiKey": "{env:JEV_ROUTER_API_KEY}",
        "jevBaseUrl": "https://ai-gateway.vercel.sh/typesafe",
        "upstreamBaseURL": "https://api.openai.com/v1",
        "upstreamApiKey": "{env:OPENAI_API_KEY}",
      },
    ],
  ],
  "model": "jev-router/gpt-6-astra",
}

This example classifies through Vercel AI Gateway. For a direct TypeSafe key, drop jevBaseUrl. The README covers the standalone proxy for other Responses clients, the decision log, and the full evaluation. The design borrows ideas from jev-codex-router and pi-jev-router.