Jev's 190-Test Challenge: When an AI Verdict Is Packaged as a Component

Deep News
Sep 21

In the first half of 2026, Kweichow Moutai reported operating revenue of 90.7 billion yuan, a rise of less than two percent, while net profit attributable to shareholders fell by two percent to 44.5 billion yuan, and operating cash flow surged nearly 5.5 times to 70.7 billion yuan. If this scenario were placed in a test with five descriptions to choose from, which one would you pick?

On September 15, the U.S. startup TypeSafe AI emerged from a two-year stealth period by releasing a model called Jev, capturing the attention of Silicon Valley and the AI community. After trying it, we fed the above question to the model; it chose the same description three times in a row, each time with full confidence. The entire task cost less than 0.001 U.S. dollars.

This model is unique because it writes no words, provides no explanations, generates no code, and does not chat. Given a piece of material and several candidate descriptions, it simply picks one and reports a number indicating its level of certainty. This subtractive tool was integrated by multiple developer platforms and model gateways like OpenRouter within a week, and within days, open-source replicas also appeared, sparking excitement across the tech and AI sectors.

Jev's founder, Diogo Almeida, spent four years at OpenAI, where he contributed to InstructGPT, the precursor to ChatGPT. DCVC led a 40 million U.S. dollar seed round for the company. The model's name references the 19th-century economist Jevons, a name that embeds the company's entire bet. Returning to the initial Moutai question, the correct options essentially paraphrase the facts of the material, so the task itself is not difficult, but the real question is: why has an AI that writes not a single word generated such a stir?

From Advisor to Switch

Anyone who has built AI products knows a common daily dilemma. A company's systems make countless trivial judgments every day: whether an email counts as a complaint, whether an announcement mentions a lawsuit, whether a refund should be auto-approved. The most common solution is to call a large model for each answer, but the core working mode of these models is generation, requiring word-by-word output. When you ask a yes or no question, it might write three paragraphs of analysis before concluding, charging by the word and taking seconds or even tens of seconds. The returned text still needs to be parsed and validated by programs, and any format error means starting over.

A bigger trouble is that when a large model says "I am very sure," that statement carries no scale. Sometimes "very sure" is like the "a pinch" in traditional cooking—whether it is 70 percent or 90 percent confident, and where the residual 10 to 30 percent uncertainty falls in terms of error direction, remains unknowable. Thus, many judgments that should be automated get stuck in a dilemma: using large models is too expensive and slow, while relying on hard-coded rules is not flexible enough.

Jev addresses this with a seemingly simple approach: it pre-writes the answers and lets the model only pick. By feeding both the material and candidate answers together, it scores each option's support level, returns the highest one, and because it avoids token-by-token generation, it is faster by more than an order of magnitude. Since it charges only for the input material and not for output, the cost per decision becomes almost negligible.

The key commercial difference here is the unit of measurement. Large models sell "words," while Jev sells "a single judgment." Word count is a service-based pricing method, while a judgment is priced like a component. Components can enter procurement lists, support unit economics, and be embedded into a line of code to run tens of thousands of times. If the component is cheap enough, software will spawn countless decision points where calling AI was previously not worth it.

TypeSafe conducted its own evaluations, comparing Jev with several large-model workflows across five business scenarios. In terms of accuracy, Jev only matched the mid-tier level, trailing the strongest model by six percentage points. All of its advantages are concentrated in unit price and speed: each judgment is hundreds of times cheaper and dozens of times faster.

Another selling point often mentioned is that it "does not hallucinate." Strictly speaking, Jev's answer can only fall within the options you provide, so it cannot fabricate a concept or number out of thin air. However, it can confidently select the wrong option, and the error shifts from "writing a non-existent statement" to "confidently ticking the wrong box," which is harder to detect. The founder himself admits that answering incorrectly with high confidence is possible.

TypeSafe has also done something uncommon among AI companies: it lists all known limitations of its model in the documentation. It notes that Jev reads dates like plain text, is not good at arithmetic, gets distracted by irrelevant content in the material, performs best in English, and works in Chinese but with weaker results, recommending that non-English users test with their own materials first. This documentation is worth writing because it precisely defines what we set out to do next.

Facing the Hog Cycle, It Hesitates

TypeSafe's documentation states its Chinese capabilities are weaker than English and advises users to self-test first. Since Jev's release, there have been remarkably few public Chinese-language tests. So after solving the API issue, we ran a round of tests using materials from domestic financial scenarios. The rules were simple: materials were sourced from regulatory bodies and companies' official interim report announcements, cross-checked with Wind, and reference answers were written and sealed before calling the model, with no revisions after. Each question was asked three times to see if the model would change its mind.

We prepared 15 questions covering policies from "two new" initiatives to reserve requirement ratio cuts and interest rate reductions, from coal supply shocks to automotive chip shortages, from photovoltaics to the hog cycle, plus interim reports from nine companies including CATL, BYD, China Vanke, Tencent, Moutai, Meituan, JD.com, Pop Mart, and Anta Sports. Across 45 judgments, all passed, with each question answered identically across all three rounds.

The least interesting part of this scorecard is the perfect score; the most interesting is question 15. In August 2021, the year-on-year hog inventory increased by nearly 30 percent, farmers lost an average of over 1,000 yuan per hog, and the inventory of breeding sows was still rising year-on-year despite a month-on-month decline. The overcapacity was not over, but signs of capacity reduction had appeared, placing this question right at a turning point. Jev chose "from overcapacity to rebalancing" in all three rounds, but this was the only question among the 15 where confidence did not reach full, reporting 85 percent, 88 percent, and 85 percent across the three rounds. The remaining uncertainty of over 10 percent was assigned to "still overcapacity," with no other wrong direction given.

A string of full-confidence answers proves nothing, but this hesitation shows that the model does form a certain degree of judgment. The perfect score is related to the test design. For example, growth rates like "year-on-year increase of 54.80 percent" or "year-on-year decrease of 7.13 percent" were pre-calculated and fed to it, so it never performed any addition or subtraction. In TypeSafe's own evaluations, Jev's worst performance was invoice reconciliation, which requires comparing amounts, quantities, and delivery dates, scoring only 61.8 percent. The documentation also states: keep arithmetic in code. We essentially designed a test paper according to the documentation, avoiding all its known weaknesses.

How well a model performs depends heavily on the design of the questions, which also means that the core competency for those building products around it likely lies in option design and data preprocessing. Then we moved to a second round of tests, handing the math back to it.

The Same Division, Wrong Ten Times

For the second round, we selected three companies and created 26 questions, each repeated five times, with answers sealed before calling as expected. We predicted 13 spots where it would fail and only hit two correctly. For instance, it managed to count correctly: in Pop Mart's interim report, among the revenue columns for 11 IPs and business lines, it identified which ones declined, and all five rounds produced correct ranges, though uncertainty increased with longer lists. TypeSafe's documentation says it cannot reliably count, but this task was passed.

Tsingtao Brewery's second-quarter profit fell while its first-half cumulative profit rose, with opposite directions mixed in the same material, but the model was not misled by the quarterly figure in any of the five rounds. Haier's 2024 revenue growth, part of which came from three acquisitions completed at year-end and part from organic growth, required subtracting acquisition contributions from total growth and calculating the ratio; it allocated correctly all five times. It also answered all growth-rate tiering questions for four companies correctly, such as for Yili's performance, where the actual growth was 4.13 percent and the cutoff was set at 4 percent, a gap of only 0.13 percentage points, with 97 percent confidence given.

Feeding Moutai's entire management discussion chapter yielded accurate results, and Pop Mart's English version with 44,000 characters was also processed, with ten out of ten correct. However, with the same full chapter of interim report text, Haier's Chinese version was rejected by the API interface five times. This shows that Chinese consumes significantly more billing units for the same length and is more likely to hit input limits.

Then the test moved to the model's boundaries. We provided CATL, Kingsoft Office, and Moutai with two years of revenue and costs, asking whether gross margin improved or worsened. These questions require calculating the gross margin for both years and comparing the change, which even a person would need a calculator for, and such tasks are basically impossible to get right. The model itself knew this, reporting confidence of under 20 percent, with probability almost evenly spread across six options, essentially admitting it was guessing.

Questions where the calculation result was far from the boundary and direction could be judged at a glance were mostly correct. Questions where the result grazed the boundary and required decimals to distinguish passed only one of three. This confirms the documentation's definition: Jev is not a calculator, and arithmetic should stay in code. For those who want to use it in financial products, the line is clear: tasks from data retrieval, caliber alignment, and ratio calculations must be done by programs, with Jev handling only the final step of semantic judgment.

One unexpected finding was a question it got wrong without knowing it. To test its performance on borderline questions, we used Midea's 2025 interim report as a control: operating costs rose 16.83 percent year-on-year, and our cutoff was set at 17 percent. The material listed the 2024 same-period operating costs under both an original caliber of about 158.4 billion yuan and an adjusted caliber of about 159.9 billion yuan. It chose "reached 17 percent" all five times, with confidence exceeding 80 percent. We initially thought the dual calibers had misled it—under the original caliber, the growth would indeed reach 17.9 percent, just above the line—so we removed the original caliber and adjustment amount, keeping only the two clean adjusted numbers, and asked five more times. The result was still all wrong, with even higher confidence. Two versions of the same question, asked ten times, wrong ten times. This was our third time guessing wrong about where it would fail.

Yili answered correctly near the line, while Midea answered incorrectly near the line, and the reason cannot be determined, mainly because Jev does not show its reasoning process and is untraceable. All we can say is to observe digit count and unit differences: Midea's numbers are nine digits in thousands of yuan, while Yili's have four decimal places in hundreds of millions of yuan. Gross margin errors alert on their own, as the model knows it is guessing, and users can set thresholds to route low-confidence cases to humans. What needs caution is the Midea question: it is very sure it is right, and in real products, the latter is more dangerous.

What is intriguing is what its "confidence" actually measures. We ran another set of tests for this: removing the correct answer and the "none of the above" option from the opening Moutai question, leaving only three descriptions that contradict the facts. It allocated almost all its support to the "revenue declined" option, but Moutai's revenue actually increased. A more direct comparison is that the same incorrect description received different treatment across two settings. When the "none of the above" option was present, it gave those descriptions almost no support; once removed, support jumped to over 80 percent, consistently across all three companies.

TypeSafe explained this mechanism in its documentation: confidence measures "how much the model favors one of these answers," not whether the answer is correct. A confidence of 95 percent means "the one that looks most like this among these," and reading it as "95 percent equals 100 percent" is a mistake. It only picks from the answers you write; no matter how firmly it chooses, it cannot pick an answer you did not write.

Where Derived Value Flows

After 190 judgments, at public pricing, the total cost was less than 0.1 yuan, meaning 1 yuan buys thousands of such decisions. In the 19th century, after steam engines became more coal-efficient, Britain burned more coal because it became cheap enough to be used in places where it was previously not worth burning—this is the starting point of the Jevons paradox. TypeSafe embedded this philosophy in the model's name: each time the cost of an AI judgment drops by an order of magnitude, the growth in call volume exceeds the savings. If judgments truly become component-priced, software will spawn countless call points where generative AI was never worth invoking.

Application scenarios will likely fall into two categories. One is adding a safety layer on top of large-model workflows: before an AI assistant takes any action, it judges whether human approval is needed; retrieved documents are pre-screened before being fed to the large model, and some developer platforms already publish tutorials for this. The other is a vast array of trivial judgments that were never worth using AI for, including ticket routing, refund reason categorization, device alert grading, product classification, and initial contract clause screening—huge in volume, low in per-unit value, and extremely sensitive to speed and cost.

But during testing, we also discovered that the real cost lies beyond the API bill. In our deliberately hard questions, setting the threshold to the most conservative level so that all approved judgments were error-free meant roughly 40 percent of judgments had to be routed to humans. Loosening the threshold slightly allowed errors to slip in, and even at the highest threshold, a checklist missing the correct answer could not be caught. Thus, derived commercial value will flow away from the model itself in five directions: arithmetic rules and caliber handling written into code, option templates designed by knowledgeable people with a "none of the above" outlet, thresholds calibrated repeatedly with one's own business data, and review teams that absorb the transferred human traffic.

A pipeline that can actually run would look something like this: programs handle data retrieval and accounting, Jev handles semantic judgment, low-confidence items go to humans, and generative large models explain the results to people. TypeSafe has chosen a clear commercial posture: integrating with multiple third-party platforms and gateways within a week while avoiding building its own application layer. Its bet is on "being integrated," running tens of thousands of times inside others' products and charging per usage. But there is an unverified risk here: TypeSafe itself admits it cannot prove that current pricing does not rely on subsidies.

The deeper impact on the tech industry may be a change in measurement systems. Over the past three years, the most common demand metric in the AI industry has been token consumption. If high-frequency, low-complexity judgments are carved away from general large models, this "word count"-anchored metric will begin to distort. The shape of inference compute demand will also diverge: "long outputs, tolerant of seconds" and "short inputs, high concurrency, sub-second returns" correspond to entirely different hardware and scheduling structures.

But Components May Not Be Scarce

A week after Jev's release, publicly available information shows no domestic Chinese model company has yet made "judge only, write nothing" a separately priced commercial product. However, similar capabilities already exist domestically. The LLaDA series from Ant Group and Renmin University also breaks out of the "write word by word" framework, but its goal is faster writing, while Jev simply does not write—a slight directional difference. Judgment capabilities exist widely in China: BGE from Zhipu AI, Qwen3's ranking models from Alibaba, and various content moderation and financial text tagging systems run countless times daily in search, risk control, and compliance pipelines. The key is that they are only used as called models; no one has packaged them into an independent model category with a dedicated pricing unit.

In fact, just days after Jev's release, open-source replicas appeared: the most notable, Laya, uses a pure encoder architecture based on ModernBERT-large (421 million parameters), while another project directly fine-tuned Alibaba's open-source Tongyi Qwen series models for the same decision tasks. Chinese teams' open-source models have become the base for replicating it, yet the commercial product comes from a San Francisco company. Laya's project documentation also states that TypeSafe's claimed calibration training method has not been reproduced. Calibration means "when it says 80 percent confidence, it is truly right 80 percent of the time," which is TypeSafe's claimed core differentiator. The shell is easy to copy, but how thick this layer actually is requires more independent testing to verify.

Our testing also left a finding more relevant to the domestic market: for the same length of financial report chapter, Chinese material consumes significantly more tokens than English and is more likely to hit the single-input limit. Of course, TypeSafe itself admits Chinese is weaker than English, so a Chinese-native judgment model has room on paper. The competitive narrative of domestic model companies is still locked on parameter scale, coding ability, and Agent leaderboards. The path of "driving the unit price of a single judgment to zero" has not yet been taken by anyone. But given that underlying capabilities exist, open-source foundations are ready, and application scenarios naturally connect with China's vast financial data and content moderation needs, this business model will likely be quickly followed. Fast or not depends on whether a team is willing to turn it from a component into a business.

Across our three rounds of testing, connecting the model API was easy, but the harder part was verifying the interim report data line by line for the nine companies and designing a set of non-omissive answers for each question. This business has pushed the price of a single judgment to nearly zero, and the remaining cost is, in fact, on the ledger of the question setter.

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Most Discussed

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10