Routing a support ticket, assessing an agent’s proposed action, or rating a request’s urgency calls for a decision your code can act on. Language models can return structured decisions, but still produce them through text generation.

Jev, a System One model from TypeSafe AI, evaluates supplied state against typed questions and returns choices, scores, and boolean probabilities without generating prose. The AI SDK’s experimental_evaluate API makes these answers available directly in TypeScript via Vercel AI Gateway.

Your application decides what happens next: a department choice can select a support queue, a severity score can influence priority, and an uncertain result can trigger review. Keeping those rules in code lets you change how the application responds without redefining what you ask the model to assess.

Overview

Calling wasRefunded('The support agent issued a full refund to the customer.') returns a number close to 1.

One example of result.answers:

{

"refunded":{"type":"boolean","probability":0.99}

}

The criteria on a boolean question are optional, but they sharpen the decision by telling the model exactly what counts as true and false.

3. Answer several questions against structured state in one request

Jev evaluates all questions in a request in parallel, so adding questions barely changes latency. You can mix all three question types, and state accepts a JSON object or array as well as a string, which means you can pass a record or a message history without serializing it yourself.

The following route handler triages a support ticket into a department, a severity level, and a refund flag in one round trip.

Reading the response:

The providerOptions.gateway object is optional. Jev supports Zero Data Retention and No Training per request, and evaluation calls appear in AI Gateway logs and count toward budgets like any other model call.

4. Branch on probabilities and confidence

Use the returned probabilities to decide when your application should act automatically or request review, setting stricter thresholds where an incorrect decision would have greater consequences.

TypeSafe AI also returns a separate confidence statistic for choice and score answers in result.providerMetadata.typesafe.confidence, keyed by question ID. Confidence summarizes how concentrated the probability distribution is, from 0 (spread evenly across options) to 1 (all on one option). It differs from the selected option's probability, and it isn't returned for boolean answers.

The routing logic below uses two paths:

Condition

What your code does

Department confidence is 0.6 or higher and the selected option's probability is 0.7 or higher

Assign the ticket to that queue

Either value is below its floor

Send the ticket to a human

Move the evaluate call into a function that returns a decision, keeping the questions from step three. Its model parameter defaults to Jev and lets the tests in step five substitute a mock.

Describe options and levels instead of labeling them

Choice criteria are a map of option keys to descriptions, and score criteria are ordered descriptions from lowest to highest.

Writing 'Blocking with no workaround' gives the model far more to match against than 'high'. Descriptions can be strings, JSON objects, or arrays, so you can pass a list of example phrases for each option.

Keep state focused

Input tokens are the only thing you pay for, so pass the fields the decision depends on rather than an entire record. Adding questions to a request doesn't degrade the answers to existing ones, because each question is evaluated independently.

Set thresholds per action, not per model

Read-only actions like showing a screen can tolerate a wrong guess, so a probability of 0.7 might be enough. Destructive actions need a higher bar, closer to 0.9 or above, and a confirmation step below it. Encode that risk tolerance in your code, and keep a review path for anything under your floor.

Keep classification separate from authorization. Jev can tell you that a customer asked for a refund or that a command looks destructive. Whether to grant the refund or run the command depends on rules Jev doesn't see, such as account status, policy, and permissions, so make that a second check in your code.

Read distributions defensively

TypeSafe AI rounds probabilities and scores to two decimal places, and result.rounding reports that precision. Because of this rounding, a choice distribution may sum to 0.99 rather than exactly 1. The AI SDK accounts for this during validation, so don't renormalize the values yourself.

Troubleshooting

Authentication errors ( 401 or 403)

Expired local credentials or an unlinked project can prevent authentication. Run vercel env pull to refresh your credentials, using vercel link first if the directory isn’t linked to a project. If a 403 persists, check your access to the linked project and AI Gateway.

Unsupported question type

The evaluation model doesn’t support one of the requested question types. Check the model’s supported types and either adjust the questions or select a model that supports them. Jev supports Choice, Score, and Boolean questions.

NoSuchModelError

When modelType is 'evaluationModel', the provider couldn’t resolve the requested evaluation model or doesn’t support evaluation. Check that the model ID is typesafe-ai/jev and that the call resolves through AI Gateway or another evaluation-capable provider.