The AI application layer is witnessing a significant shift as mounting API expenses from major model providers drive a growing number of startups to reclaim control over their costs. This pivot toward open-weight models is no longer an isolated experiment but a accelerating movement across multiple industries, putting direct pressure on the revenue streams of leading AI labs.

Legal AI unicorn Harvey serves as a stark example, with its gross margin plunging to -50% at one point this year. As API fees from OpenAI and Anthropic continue to climb, a wave of AI application startups are embracing open-weight models to regain cost control, with some already routing the majority of their traffic through proprietary systems.

This trend is gaining momentum rapidly. Startups spanning legal, medical, financial, and customer service sectors—including Harvey, Abridge, Decagon, and Ramp—have announced plans to develop or customize their own models, with certain companies already directing up to 80% of their traffic to in-house systems. Prominent investors such as Sequoia Capital and General Catalyst are actively backing this movement, which poses a direct challenge to OpenAI and Anthropic as both prepare for highly anticipated IPOs.

The Margin Collapse That Forced Harvey to Rethink Its Strategy

Harvey's situation best illustrates the underlying pressures. The legal AI startup, valued at $15.6 billion, has long relied on OpenAI's GPT-4 as the core of its product. After a major update to its AI agents in March, user usage surged dramatically, but this success came at a steep cost—gross margins collapsed from roughly 50% at the start of the year to -50% by June. Harvey's AI token consumption grew twentyfold during this period.

Co-founder and President Gabe Pereyra previously argued that application performance depended heavily on underlying model capabilities, making purchases of premium models from OpenAI and Anthropic a necessary expense. However, that logic has begun to crumble under the weight of rising costs. In August, Harvey launched its own model powered by Moonshot AI's Kimi K3, which delivers performance close to Anthropic's best offerings at a fraction of the cost. Combined with other strategic adjustments, Harvey's gross margins have since returned to positive territory.

Open-Source Adoption Spreads Across Multiple Verticals

Harvey's transformation is far from unique, with similar shifts emerging across various sectors. Healthcare technology startup Abridge recently announced plans to build its own foundational model for clinical applications based on Nvidia's open-source offerings. AI customer service startup Decagon reports that 80% of its query traffic now flows through its proprietary model. In fintech, Ramp and Rogo are exploring in-house model training for the first time. In the coding tools space, Cursor—now part of SpaceX—and Cognition, valued at $48 billion, were among the first AI application companies to release customized models.

Ramp Co-CEO Karim Atiyeh noted that training proprietary models was previously economically unjustifiable, but the dynamics have reversed following a $750 million funding round in June and substantial performance gains in open-weight systems. Dr. Lan Xuezhao, founder and managing partner of San Francisco venture firm Basis Set, took an even stronger stance: "If you're not optimizing costs and fine-tuning your own models, you're inefficient by definition. A company that doesn't consider building its own model might not secure funding at all."

Dual Pressures From Closed-Source Giants: Price Hikes and Market Encroachment

The driving force behind this transition isn't just cost—it's also the increasingly aggressive commercial strategies of OpenAI and Anthropic. Both companies have recently introduced additional usage-based fees for enterprise customers on top of existing subscription charges, effectively penalizing "token maximization" usage patterns. Reports indicate that Uber exhausted its entire annual AI budget by April after encouraging engineers to maximize their use of Anthropic's Claude Code.

Simultaneously, both OpenAI and Anthropic have been aggressively recruiting in legal, financial, and healthcare sectors this year, launching plugins and piloting industry-specific applications that directly compete with their own customer base. Startups also face the risk of having their access revoked entirely. Less than a week after SpaceX completed its acquisition of Cursor, OpenAI suspended model access to the coding tools startup, citing past violations of service terms by Musk's companies. Against this backdrop, an investor forum held at Anthropic's offices specifically addressed the trend of startups building their own models. Anthropic acknowledged that Harvey still depends on its most powerful Claude Opus model for the most complex tasks.

The Practical Challenges of the Open-Source Path

Transitioning to open-weight models comes with its own set of significant obstacles. Talent tops the list of hurdles. Matt Kraning, a partner at Menlo Ventures which backs Anthropic, points out that engineers capable of model fine-tuning command salaries in the millions and are highly susceptible to poaching by large institutions like OpenAI and Anthropic. Data presents another barrier, as training proprietary models requires substantial proprietary datasets. Harvey, unable to access sensitive client legal documents, was forced to purchase training data from AI data provider Mercor.

Infrastructure costs for open-source models are equally formidable. Andrew Dai, CEO of visual AI startup Elorian, notes that downloading open-weight models and managing computing infrastructure involves considerable expense. For early-stage companies with lower traffic volumes, pay-per-use closed-source models may actually prove more economical. Some companies have indeed tried and failed. Startup Salespeak previously announced plans to build its own large language model but abandoned the effort after several months of experimentation. Co-founder and CEO Omer Gotlieb stated they failed to see "significant advantages" compared to the ready-made models from Anthropic and OpenAI.

Continued Reliance and Ongoing Negotiation

Despite the wave of in-house development, most startups are not seeking to completely sever ties with OpenAI and Anthropic. Logan Bartlett, managing director at Redpoint Ventures who holds investments in Anthropic, Abridge, and Ramp, expects the industry's drive to reduce dependence on Anthropic's models to strengthen, yet acknowledges that startups will still pay for optimal performance when necessary. "They won't cut off their own hands out of spite," he said. Anthropic's own presentation materials confirm this reality—Harvey still requires Claude Opus for the most complex tasks, indicating that a hybrid approach combining open-source and closed-source models may represent the optimal strategy for most startups at this stage. This ongoing negotiation over AI costs and control fundamentally represents application-layer startups seeking to rebalance power with the model-layer giants. As open-source performance continues to close the gap with closed-source products, this tension will persistently test the boundaries of mutual interest as OpenAI and Anthropic proceed toward their IPOs.