LLMs

OpenAI launches Decisions API to compete in fast-inference decision-model market

OpenAI releases Decisions API widely, targeting TypeSafe's Jev model in a new category of fast, low-cost classifiers.

Last verified:

OpenAI enters decision-model market with Decisions API

According to TechCrunch, OpenAI released its Decisions API to all customers on October 23, entering a market category pioneered by TypeSafe’s Jev model, which turned the startup into a $7.6 billion company. Decision models function as ultra-fast classifiers: given a set of options, they select an outcome and assign a confidence score. Developers have rapidly adopted this category because decision models operate at dramatically lower cost and latency than general-purpose LLMs, making them suitable for high-throughput workflows where raw reasoning capability is unnecessary.

Latency and cost as primary value drivers

The core appeal of decision models lies in performance constraints, not capability. According to Nikunj Handa, OpenAI’s APIs product lead, the company’s Decisions API shrinks inference time from one to three seconds down to 100–120–180 milliseconds. Rather than training a specialized model, OpenAI reconfigured its existing ChatGPT Luna model to prioritize speed and cost efficiency.

Early customers validated the approach. Speak, an AI language-tutoring company led by CTO Andrew Hsu, reduced action latency from 1.4 seconds to 0.13 seconds using the Decisions API to replace OpenAI’s standard LLMs. OpenAI also deployed the API internally to improve responsiveness in its Dots voice assistant. The API now includes voice and visual modalities, with geographically targeted inference resources to minimize latency across regions.

Future training optimization remains speculative

Handa told TechCrunch that while the team currently uses off-the-shelf Luna configuration, OpenAI “may sort of like have specific training to make our models like even better in these ways” in the future. This indicates uncertainty about dedicated decision-model training — the technology is young enough that even the leading vendors are still determining whether specialized fine-tuning offers material gains over repurposed general models.

Why this matters

The decision-model category is redefining the efficiency frontier for AI applications. Teams previously forced to choose between LLM quality and latency can now select a category optimized explicitly for sub-second classification at lower token cost. This shifts purchasing logic away from model capability and toward workload fit. Organizations operating high-volume classification pipelines — content moderation, fraud detection, routing — should evaluate whether decision models reduce infrastructure costs relative to LLM-based approaches. The competitive intensity (with TypeSafe, Liquid, and now OpenAI all shipping variants within weeks) suggests rapid iteration on both architecture and benchmarking methodology.

Frequently Asked Questions

What is a decision model?

A decision model is a specialized classifier that rapidly selects among options and returns a confidence rating. It trades raw intelligence for speed and cost efficiency, operating in 100–180 milliseconds versus 1–3 seconds for standard LLMs.

How does OpenAI's Decisions API differ from Jev?

According to TechCrunch, OpenAI reconfigured ChatGPT Luna for speed and cost rather than training a dedicated decision model. OpenAI's version includes voice and visual modalities and optimized geographic inference.

Who is TypeSafe and why does it matter?

TypeSafe, founded by a former frontier-lab researcher, created the Jev decision model and became a $7.6 billion company. Its success prompted OpenAI and other labs to enter the decision-model space.

#decision-models #inference #api #competition #openai