Decision Index: Jev 1.13 Tops Benchmark for Decision AI

👁 121 reads

On October 13, 2026, Artificial Analysis launched Decision Index v1.0, the first independent ranking of AI decision models - specialized systems that skip free-form text generation entirely, instead choosing from a set list of options and reporting a probability distribution across each choice. Leading the board is Jev 1.13 from TypeSafe with a score of 58 points, ahead of GLiDE (Fastino Labs) and Microsoft-Decision-1 (both at 54). The most budget-friendly option is Decider v1.1 27B from Perplexity at $17 per million tasks. Standard Claude Haiku 5.5 scores higher overall (61), but according to AA measurements, it is 11 times slower and 4.4 times more expensive than the index leader.

Disclosure: I was already familiar with Jev prior to this release; in September, I wrote a lightweight API client for it for use in my editorial workflows. TypeSafe did not sponsor or pay for this article, and Jev’s placement here is based entirely on the independent benchmarks published by Artificial Analysis.

What is an AI decision model?

A decision model takes two inputs: an observation state (such as the text of an incoming customer ticket) and a set of closed-ended questions. Instead of drafting a natural-language sentence, it outputs a probability distribution across every permitted choice. Artificial Analysis illustrates this with an example ticket - “payouts haven’t worked for 3 days and my rent is due tomorrow” - evaluated across three questions simultaneously:

• Target department: billing, tech support, or account management (categorical selection),
• Urgency: yes or no (binary classification),
• Customer sentiment: calm, frustrated, or angry (ordinal scale).

The decision model handles all three evaluations in a single inference pass and outputs an exact probability distribution. A standard LLM, by contrast, must generate JSON containing its choices, which application code must then parse, and it rarely provides calibrated confidence metrics. I explored this architecture in greater detail, along with tests on customer tickets, in my guide to AI decision models.

Artificial Analysis Decision Index v1.0 leaderboard

For comparison, general-purpose autoregressive LLMs tested on the identical evaluation suites recorded: Claude Haiku 5.5 (low) at 61 points with $184 and 1.4 s; GPT-6 Luna (low) at 55 points with $131 and 1.5 s. Latency reflects median duration per task, while cost represents the average task expense under current token pricing scaled to one million requests. Decision models incur costs exclusively for input tokens, whereas standard LLMs charge for both input and output tokens.

How the Decision Index is calculated

The Decision Index represents an equally weighted composite average across four core benchmarks, each contributing 25%:

• AA-Omniscience-Decision (knowledge and hallucination avoidance): 500 private, unreleased questions featuring five options - four concrete answers plus an explicit “I don’t know” fallback. The score weights accuracy and the rate of choosing “I don’t know” instead of guessing equally.
• HLE-MultiChoice (complex reasoning): 513 text-based multiple-choice prompts selected from Humanity’s Last Exam, among the most demanding benchmark suites in modern AI.
• When2Call (tool orchestration): NVIDIA’s benchmark evaluating whether an agent should formulate a direct answer, trigger an external tool, ask for missing details, or declare inability to execute.
• WildGuardTest (classification and guardrails): Allen Institute for AI (Ai2) benchmark testing whether an incoming prompt is harmful, whether the assistant reply constitutes a refusal, and whether that reply itself is harmful.

Key benchmark findings

• Jev wins on reasoning. In HLE-MultiChoice, it posted 21 points, while no other dedicated decision model surpassed 15. In factual knowledge tests, it achieved 35% accuracy (with Liquid AI’s d1 leading that specific category at 41%).
• Microsoft-Decision-1 is best at admitting uncertainty. It selected “I don’t know” appropriately 40% of the time (Jev scored 35%, Mercury Decide 32%, with the remainder under 17%). Liquid AI’s d1 reached peak knowledge accuracy, yet almost never deferred to uncertainty (scoring just 2).
• Decision models outperform standard LLMs in tool selection. In When2Call, Celeris-1 Decision and GLiDE achieved 84 points each, beating Claude Haiku 5.5 (low) at 73 and GPT-6 Luna (low) at 77.
• They also lead in classification accuracy. On WildGuardTest, GLiDE reached 95 points, while Jev and Decider reached 94, outpacing Claude Haiku 5.5 (low) at 93 and GPT-6 Luna (low) at 89.
• Every decision model displays overconfidence. AA compared stated confidence ratings against empirical accuracy: across knowledge and multi-step reasoning, observed accuracy routinely fell short of stated confidence. In classification and tool selection, however, calibration was assessed as relatively sound.

Why decision models cost less than standard LLMs

A decision model never generates sequential response tokens. It processes the full prompt context in a single evaluation run, produces probability distributions for every listed question at once, and terminates. This yields three tangible benefits demonstrated in the benchmark data:

• You pay strictly for input tokens. No output generation fees and zero reasoning token overhead.
• Multiple questions evaluated in one call, bypassing sequential JSON string building.
• Sub-second latency profiles. Execution times range from 0.06 to 0.50 s per task, compared to 1.4 to 1.5 s for smaller LLMs running in minimal reasoning mode.

These margins compound rapidly at scale. When evaluated solely on classification tasks (WildGuardTest), Artificial Analysis measured one million tasks at $12.10 for Decider and $22.80 for Jev, compared to $91.50 for Claude Haiku 5.5 (low).

When to use a decision model

Decision models deliver clear economic and performance advantages when problem spaces rely on fixed candidate options at high request volumes:

• Categorization, triage, and automated routing for inbound tickets, support emails, and documentation,
• Content moderation and safety checks against harmful user prompts,
• Tool selection and decision trees inside agent architectures (determining when to call an external API vs ask for clarification),
• Validation stages: approving output, initiating a revision run, or escalating an interaction to human review.

They are ill-suited when tasks demand broad world knowledge or open-ended text synthesis. On factual recall questions, Claude Haiku 5.5 (low) scored 52% accuracy and GPT-6 Luna (low) hit 66%, while dedicated decision models ranged between 28% and 41%. Furthermore, a decision model cannot draft an email response to a customer - that remains the job of a generative language model.

What to watch next

Artificial Analysis maintains this index as a continuous leaderboard and is accepting model submissions from additional developers. I have compiled comprehensive benchmark breakdowns across all four evaluation categories, API pricing for each provider, and practical classification tests in our dedicated guide to AI decision models. Previously, I covered TypeSafe Jev, Cloudflare Clef, and AWS Strands Decider 2B.

Sources

• Artificial Analysis: Artificial Analysis Decision Index v1.0 (Oct 13, 2026)
• Artificial Analysis: Decision Index - Live Leaderboard (data accessed evening of Oct 13, 2026)
• Artificial Analysis: Decision Index Methodology
• Artificial Analysis on X (announcement posted Oct 13, 2026)

// read moreAI Decision Models: Decision Index BenchmarkBenchmarksTypeSafe Jev: Decision Model Review, AA Benchmark & Hands-On TestAI ToolsLiquid AI d1-ultra: Decision Index Leader and Test ResultsBenchmarks

Kinetyra