Jev Engineering and Kimi K3: A Confidence Cascade for Cheaper Agents

Most agent bills are not created by the final answer. They are created by the small decisions an agent makes before the final answer exists: which tool to call, whether a source is relevant, whether a test passed, whether a command is safe, and whether the task is finished. Jev engineering is a practical way to move those bounded decisions out of a general-purpose language model, gate them with probabilities, and reserve expensive generation for the cases that actually need it.

The expensive part of an agent is often not creation

Open a real agent trace and mark every step by what it actually requires. Some steps create something: drafting a repair, writing a plan, summarizing evidence, explaining a failure. Some steps follow an exact rule: count retries, enforce a token budget, compare a date, check an allowlist. Some steps choose from a set that can be written down in advance: billing or support, relevant or irrelevant, safe or unsafe, done or not done. A small number of steps cannot be undone: send the email, publish the post, transfer the money, delete the record, change permissions.

The original article's central observation is that the third category gets billed at first-category prices. A frontier model is asked to read a large state and then make a narrow classification. It may produce one word, but the inference still carries the cost and latency of a generative model. If the agent makes that call after every turn, the nominally small decision becomes the dominant operating cost.

Jev is a system one model built around this distinction. It does not write prose. The caller sends state and typed questions, and the model returns structured answers with probabilities. TypeSafe describes most calls as completing in about 100 milliseconds. The current list price is $42 per billion input tokens, equivalent to $0.042 per million input tokens, with no output-token charge.

A four-bucket audit before choosing a model

The audit matters more than the model choice. If a task belongs in code, replacing a language model with another language model does not make the system more reliable. If a task belongs to a human, automating it because a classifier returned a number can make the system less safe. The useful unit of analysis is a decision boundary, not a whole agent.

Step type	Correct executor	Examples	Reason
Creation	General LLM	Draft, rewrite, explain, synthesize, plan	The output is open-ended and needs language generation.
Exact rule	Deterministic code	Retry count, arithmetic, budget, date ordering, permissions	A testable rule should not be estimated probabilistically.
Bounded judgment	Jev	Route, rank, keep or drop, safe or unsafe, done or not done	The answer can be represented as a Choice, Score, Noul, or Box question.
Irreversible side effect	Code plus human approval	Send, pay, publish, delete, grant access	A probabilistic judgment may recommend the action, but should not execute it alone.

A good migration starts with one decision that occurs dozens of times per day and already has a reviewable history. For each occurrence, record the state the agent saw, the decision it made, the confidence of the classifier, the downstream result, and whether a human corrected it. That log is more valuable than a generic benchmark because it measures the exact distribution the router will face.

Confidence is a routing signal, not a truth certificate

The important property of a system one model is not that every answer is correct. It is that the confidence can separate answers that should be acted on from answers that should be checked. TypeSafe describes Jev as trained for calibrated decisions rather than human-pleasing explanations. The original article frames this through Diogo Almeida's argument: a model that succeeds on 95% of cases but cannot identify the difficult 5% is hard to automate safely.

Current documentation is explicit that Jev's returned confidence measures the shape of the answer distribution. It is not the probability that the selected answer is correct. That distinction changes how a production system should route.

For a Choice question with probabilities p_i, two useful routing statistics are the selected probability and the separation between the first and second options:

$$C_{\mathrm{choice}}=\max_i p_i$$

$$M=p_{(1)}-p_{(2)}$$

A high selected probability with a small margin can indicate that the model concentrated most mass in two adjacent options. A lower selected probability with a large margin may still mean that the answer is uncertain, but in a less ambiguous way. Neither statistic proves correctness. Both can be calibrated against labeled examples for the specific action.

For a Score question with ordered levels, the expected level and the concentration of the distribution are useful separately. The expected score answers “where on the rubric is this case?” while confidence answers “how concentrated is the model?”

$$\mathbb{E}[s]=\sum_{j=1}^{n} j\,p_j$$

$$C_{\mathrm{score}}=\max_j p_j$$

For a Noul question, the result is the probability that the statement is true. The uncertainty is largest near 0.5 and falls toward either endpoint:

$$u_{\mathrm{noul}}=1-2\left|p_{\mathrm{true}}-0.5\right|$$

This is why naming a Noul is_not_safe is a design mistake. It inverts the meaning of the output while leaving the threshold logic looking plausible. Phrase every Noul so that a high value means the condition the router wants to detect is true.

The four current primitives

Jev 1.14 supports Choice, Score, Noul, and Box. The first three cover most agent decisions. Box is for locating a region in an image and is not a substitute for general visual reasoning. Current requests accept text and images, but image and Box requests require the TypeSafe-Danger-Zone-Images header.

Primitive	Question shape	Returned evidence	Operational rule
Choice	Pick one from a known set	choice, confidence, probabilities	Keep options mutually exclusive and describe them with observable situations.
Score	Place the case on an ordered rubric	score, confidence, legend, probabilities	Write every level as a standalone definition.
Noul	Is this statement true?	noul, the probability of true	Use for binary gates and pair it with a separate exit question when “other” matters.
Box	Locate a region	Normalized top, bottom, left, right, and confidence	Use only when a bounding region is the actual decision surface; verify geometry in code.

The question ID is never sent to the model. A field named is_fraud supplies no meaning by itself. All criteria must be in the instructions and option descriptions. Questions within one request observe the same state and run independently in parallel; one question cannot use the answer to another.

What the current API changed from the September snapshot

The original article documented the Jev 1.13 endpoint and several limits that were accurate for that release. A production integration should use the current contract rather than copying the historical example unchanged.

Topic	September source snapshot	Current verified behavior
Endpoint	POST /v1/systemone	POST https://api.typesafe.ai/v1/systemone
Pinned model	jev-1.13.0	jev-1.14.0; jev-latest tracks the moving alias
Primitives	Choice, Score, Noul	Choice, Score, Noul, and Box; Box is new in 1.14
Input modality	Text only	Text and images; images and Box need the danger-zone header
Default context	About 64K total and 32K for the longest question	Still 64K and 32K by default
Long context	Not part of the source snapshot	262K for both limits with TypeSafe-Danger-Zone-Long-Context: true; slower and less accurate
Rate limits	1,200 requests/s and 250,000 tokens/s	Organization-specific in the developer console; the historical figures are not universal
Latency	70 to 500 ms	Most calls complete in about 100 ms; tail latency still depends on state and load
Exit option	Always add other or none	Prefer a companion Noul instead of forcing every Choice to contain an artificial exit

The price remains $0.042 per million input tokens with output free. The header name is deliberately loud: long-context and image modes are risk controls, not free performance upgrades. Use them only after the smaller, typed request has been shown to need them.

Ask related questions in one call

Speculative fan-out is the largest structural saving in the original article. If several questions apply to the same state, send them together. Jev evaluates them in parallel against one copy of that state. The model can return a routing decision, a safety decision, and a completion check in one inference request without the caller making three separate trips with the same document.

The original source says the tested fan-out was about 10 times faster, while the current TypeSafe cookbook reports 12.2 times cheaper and 11.8 times faster across 13 questions over a 53,777-character document. The difference is not a contradiction that needs to be averaged away. They are measurement snapshots from different runs. Treat both as evidence that state reuse matters, then benchmark your own state size, question count, and provider load.

Two constraints decide whether fan-out is appropriate. First, related questions must actually share the same state; unrelated work belongs in separate calls. Second, questions cannot inspect one another's answers. If question B depends on newly fetched evidence produced after question A, the system needs another round with updated state.

Writing the question is the engineering work

Most failures attributed to Jev begin as specification failures. The model reads the question literally and returns a valid answer to the wrong question. The fix is usually a tighter state, an observable criterion, or a smaller unit of judgment.

1. Ask one judgment per question. “Is this urgent and from a paying customer?” is two facts and a rule. Ask both facts, then combine them in code.
2. Describe situations, not vague degrees. “Mentions cancelling, a chargeback, or a public review” is observable. “Very frustrated” is not.
3. Make every rubric level self-contained. Criteria are evaluated as descriptions, not as a conversation with the previous level.
4. Point to the exact field. Backticked paths such as order.charges reduce the search space.
5. Filter before inference. More irrelevant state can lower accuracy. Seven selected sources with their claims are often better than a research folder dumped into the request.
6. Turn extraction into selection where possible. If you need a vendor name, provide the known vendor list and ask which one appears. Do not expect a classifier to invent a value outside its schema.
7. Prefer a companion Noul to a universal “other” option. Current guidance recommends asking the primary Choice and a separate Noul such as “none of the listed categories fits.” This keeps the options meaningful while still giving uncertainty a route.

Where the model still breaks

Type safety protects the output shape, not the truth of the output. A schema-valid answer can still be the wrong valid answer. Jev is also not a calculator, a date engine, or a reliable multi-hop reasoner. Those operations should be computed in code and passed into the model as facts.

Failure mode	What happens	Harness response
Literal reading	A loose instruction gets a literal answer	Version questions, test them, and review behavior changes like code changes
Arithmetic	Counts and sums drift as the input grows	Compute totals in code and ask only for a bounded judgment over the result
Date ordering	“After March 3” is treated as text	Compute date comparisons before building state
Context rot	Irrelevant material lowers accuracy	Select and trim evidence for each state
Prompt injection in state	Adversarial text shifts the judgment	Keep permissions and deterministic checks outside the model
Need for explanation	A number cannot justify the decision to a human	Escalate to an LLM or a human when prose is required
Multi-hop inference	Long chains of inference become less stable	Break the chain into calls with newly materialized state

The confidence cascade

The practical architecture is a layered router. Deterministic code handles facts that must be exact. Jev handles bounded judgments and returns a probability. Code checks that probability against an action-specific threshold. Cases below the threshold go to Kimi K3, which can read a longer thread, inspect screenshots or scanned documents, and provide a reason. Cases that remain unresolved go to human review. Irreversible actions execute only after the appropriate approval gate.

state
  -> deterministic validation and feature extraction
  -> Jev: parallel typed questions
  -> router
       -> accept when selected probability and margin clear the action threshold
       -> escalate low-confidence cases to Kimi K3
       -> reject or request new state when the question is malformed
       -> human review when both models are uncertain
       -> execute irreversible actions only after an explicit approval step

Kimi K3 fits the escalation slot because an uncertain case often needs more than another short classification. It may need the full thread, screenshots, a scanned PDF, or a written reason. Current Kimi documentation advertises a 1M-token context with flat context pricing, which makes it suitable for the cases where Jev's compact state was insufficient. The model is available through hosted APIs and open weights, so the same routing contract can target different deployment backends.

The cascade should preserve Jev's answer as context for the larger model, but it should not treat that answer as evidence. A useful escalation message says: “The fast classifier selected X with a selected probability of 0.71 and a top-two margin of 0.08. Re-evaluate the original state against these criteria and explain any correction.” This lets Kimi challenge the first answer rather than anchor blindly to it.

One concrete cascade function

The following sketch keeps the policy visible. Model identifiers, headers, and retry behavior should be filled from the current SDK and documentation. The important part is the separation between classification, routing, and side effects.

from typesafe_sdk import Choice, Noul, TypeSafeClient

THRESHOLD = 0.95
MARGIN = 0.10

jev = TypeSafeClient(model="jev-1.14.0")

def classify_email(email):
    response = jev.system_one(
        state={"email": email},
        questions={
            "category": Choice(
                instructions="What does the sender of `email` want?",
                criteria={
                    "customer": "A current customer asks about an order, bill, or account",
                    "lead": "Someone asks about prices, a demo, or working together",
                    "vendor": "Someone is selling a product or service to us",
                    "fraud": "Urgent payment change, spoofed sender, or credential link",
                },
            ),
            "none": Noul(
                instructions="Does none of the listed categories fit `email`?"
            ),
        },
    )
    return route_j_email(response)

def route_j_email(response):
    answer = response.answers["category"]
    probs = sorted(answer.probabilities.values(), reverse=True)
    margin = probs[0] - probs[1] if len(probs) > 1 else 1.0
    none_probability = response.answers["none"].noul

    if (
        probs[0] >= THRESHOLD
        and margin >= MARGIN
        and none_probability < 0.5
    ):
        return {"status": "accepted", "category": answer.choice}

    return {
        "status": "escalate",
        "jev_guess": answer.choice,
        "selected_probability": probs[0],
        "margin": margin,
    }

This example deliberately refuses to send, delete, publish, or change permissions. Those operations belong after the router, behind a tool allowlist and a human approval path. A classifier can recommend an irreversible action; it should not be the authority that performs it.

Fraud screening: two runs, two different answers

The public jev-fraud repository provides the clearest evidence in the source article. It also demonstrates why a demo number should not be copied into production without inspecting the run. The original article reports one execution; the repository's checked-in Together AI trace records another.

Measurement	Original source run	Checked-in Together AI run
Dataset	100 emails, half legitimate and half fraud	100 emails from the same demo
Jev stage	Classified all 100	Accepted 70 and escalated 30
Jev alone	Not reported separately	90/100 correct
Kimi escalation	31 cases	30 cases
Final accuracy	96/100	92/100
Corrections	Not broken down	Two corrected, no regressions
Fraud recall	Not reported	84%
Jev latency	1.42 s for the batch stage	Aggregated into the 6.114 s total
Total latency	16 s	6.114 s
Recorded cost	About $0.07	$0.101775

The checked-in run configured Jev concurrency at 20, Kimi concurrency at 25, and a selected-probability threshold of 0.95. Its recorded Jev gateway cost was $0 because that route was included as a promotion or free allowance. The $0.101775 total is therefore not a portable estimate of Jev's production cost. A separate estimate for uncached Kimi input at $3 per million tokens and output at $15 per million tokens accounts for most of the economics in the source article.

The two runs differ because the cascade is not a deterministic rule engine with one global benchmark. Model versions, provider routing, prompt text, threshold behavior, concurrency, and the treatment of cached tokens can all move the result. The right conclusion is not “this pipeline is 96% accurate.” It is “this pipeline can be instrumented, and its accepted and escalated cohorts can be audited separately.”

Cost math without the marketing percentage

Use the source article's monthly email example to make the arithmetic visible. Assume 10,000 emails, about 1,000 Jev input tokens per email, 1,500 Kimi input tokens and 300 Kimi output tokens per escalation, and a 10% escalation rate. Kimi is priced here at $3 per million input tokens and $15 per million output tokens.

Path	Input basis	Output basis	Monthly estimate
Frontier model on every email	15M tokens at $10/M	3M tokens at $50/M	$300
Kimi K3 on every email	15M tokens at $3/M	3M tokens at $15/M	$90
Jev everywhere, Kimi on 10%	10M Jev tokens plus 1.5M Kimi tokens	0.3M Kimi output tokens	$9.42

The Jev component is:

$$\frac{10{,}000\times1{,}000}{1{,}000{,}000}\times0.042=0.42$$

The escalation component is:

$$\frac{1{,}000\times1{,}500}{1{,}000{,}000}\times3+\frac{1{,}000\times300}{1{,}000{,}000}\times15=9.00$$

The expected cost of a cascade with escalation probability e, Jev cost c_J, and escalated-model cost c_K is:

$$\mathbb{E}[C]=c_J+e\,c_K$$

If a fraction r of cases is rejected or sent back for more evidence rather than accepted, expected cost per accepted decision is:

$$\mathbb{E}[C_{\mathrm{accepted}}]=\frac{c_J+e\,c_K}{1-r}$$

Accuracy after escalation depends on both the first-stage accuracy A_J and the escalated model's accuracy A_K on the escalated subset:

$$A_{\mathrm{final}}=A_J+e\left(A_K-A_J\right)$$

These equations expose the tradeoff the “90% cheaper” headline hides. A lower escalation rate reduces cost but may reduce coverage or accuracy. A higher threshold can improve precision while sending more cases to the expensive model. The useful target is not the lowest bill; it is the lowest cost that satisfies a measured risk budget for that action.

Thresholds belong to actions, not models

A newsletter routing decision and a payment-change decision should not share a threshold. The first is reversible and cheap to correct. The second can create immediate financial loss. TypeSafe's guidance uses low-confidence bands for human review and much higher confidence before destructive actions, while explicitly treating those numbers as examples rather than universal defaults.

Action	Reversibility	Starting policy	Measure before promotion
Tag or route an internal item	Easy to undo	Use a lower threshold after shadow validation	Precision and reviewer correction rate by confidence band
Keep or drop a source	Usually reversible	Require a useful probability margin	Missed-evidence rate and downstream answer quality
Mark an invoice as fraudulent	High business impact	Use a high selected-probability threshold plus a second model	Fraud recall, false-positive cost, and escalation yield
Delete data or change permissions	Potentially irreversible	Human approval regardless of model confidence	Authorization checks and complete audit logs
Send money or publish externally	Irreversible	Approval gate outside the model	Dual control, idempotency, and rollback or recovery plan

Shadow mode is the rollout mechanism

Do not switch the production path on the first day. Run Jev in shadow mode while the existing system remains authoritative. For each decision, log the model ID, primitive, full probability distribution, selected probability, margin, threshold, escalation result, final human decision, and eventual outcome. Then evaluate by confidence band rather than by the model's global average.

1. Instrument one decision. Choose a bounded call with a clear ground truth and a review process that already exists.
2. Collect a representative shadow set. Include ordinary cases, adversarial cases, rare classes, and the failures the old system handles badly.
3. Measure calibration. For each confidence band, compute accuracy, precision for the action, coverage, and escalation rate. Do not assume the API's confidence number is calibrated for your data.
4. Promote the safest band first. If the 0.98+ band is consistently correct, automate that band while the rest continues to use the old path.
5. Add the second model only where it earns its cost. Compare final accuracy and review load with the escalation threshold varied across realistic values.
6. Pin and version. Pin a model version, version the question text, and treat criteria edits as production behavior changes.
7. Watch drift. Monitor state distribution, escalation rate, corrections, and the cost per accepted decision. A rising escalation rate can be an early signal that the input distribution changed.

flowchart TD
  A[Agent state and candidate evidence] --> B[Deterministic validation]
  B --> C[Parallel Jev questions]
  C --> D{Probability and margin above action threshold?}
  D -->|Yes| E[Accept and continue]
  D -->|No| F[Escalate to Kimi K3]
  F --> G{K3 has a supported answer?}
  G -->|Yes| H[Return reasoned decision]
  G -->|No| I[Human review]
  E --> J{Irreversible side effect?}
  H --> J
  I --> J
  J -->|No| K[Tool may execute]
  J -->|Yes| L[Require explicit approval]
  L --> K

Where Jev fits beside Kimi K3

The source article describes LangChain, Pydantic AI, and the Vercel AI SDK integrations. Those adapters matter because they move routing into the harness instead of forcing every application to reinvent the same protocol.

Integration	Useful boundary	What remains the application's responsibility
LangChain middleware	Model routing and risk checks around tool calls	Tool permissions, side-effect approval, and state redaction
Pydantic AI	Map output types to questions and use fallback models	Threshold selection and validation against the real distribution
Vercel AI SDK	Evaluate structured decisions in a TypeScript harness	Request construction, user consent, and downstream execution
Kimi K3 escalation	Long context, images, reasoning, and written explanations	Rechecking the answer against policy and retaining an audit trail

The boundary should remain stable even when the supplier changes. Code owns authorization, arithmetic, budgets, dates, idempotency, and execution. Jev owns a typed judgment over a bounded state. Kimi owns the harder generation or multimodal reasoning case. A person owns the irreversible decision. This separation is more portable than any one SDK call.

What to take away

The useful idea is not “put a smaller model everywhere.” It is to make the decision boundary explicit. A system one model is valuable when the work can be reduced to a question with a known answer space, the state can be trimmed to relevant evidence, and the probability can drive a routing policy. It is a poor fit for open-ended writing, exact computation, hidden multi-step reasoning, or authority over irreversible actions.

The original article's optimistic scenario can be cheaper by roughly an order of magnitude, but that result depends on token volume, escalation rate, provider price, and the amount of context that can be shared. The fraud demo's two traces prove both sides of the point: the cascade can produce strong accuracy at low cost, and small changes in model, provider, and accounting can move the observable result. Measure the cohort that falls below the threshold, not just the headline average.

Start with one decision your agent makes fifty times a day. Write down its four-bucket classification. Give it a versioned question and a shadow log. Calibrate a confidence band. Add a second model only for the cases that need it. Keep the irreversible action at a human or deterministic approval boundary. That is a more durable form of “Jev engineering” than copying a threshold from a demo.

Source: Mr. Buzzoni, “How to Cut Your Agent Bill by 90% With Jev Engineering and Kimi K3”, September 29, 2026. The article was translated, reorganized, fact-checked against current public documentation, and stripped of promotional material for RobotWorld.

Primary references

• TypeSafe system one API
• TypeSafe confidence guidance
• Confidence routing pattern
• Fan-out pattern
• Jev and Kimi K3 fraud demo
• Kimi platform