In response to the increasing number of competitors in the AI server chip arena, NVIDIA has unveiled a new strategy: collaborating with its rivals.

According to an exclusive report, NVIDIA and AI chip startup d-Matrix will integrate their hardware to create a new computing system for large model inference.

This news comes just a month after another AI server chip maker, SambaNova, announced it had aligned its software and hardware with NVIDIA to allow its chips to run AI models in conjunction with NVIDIA GPUs.

NVIDIA has signaled that more such partnerships are in the pipeline. Dion Harris, Senior Director of High Performance Computing at NVIDIA, stated, "We cannot disclose other partnerships ahead of time."

Chips from companies like d-Matrix and SambaNova can natively connect to NVIDIA GPUs via standard Ethernet cables without formal collaboration. However, this deeper partnership involves NVIDIA engineers working directly with the rival chip teams to co-optimize GPU control software, significantly boosting cross-chip efficiency. Formal collaboration also makes it easier to convince customers to purchase these hybrid computing systems.

This approach reflects a strategic pivot by CEO Jensen Huang. As NVIDIA's dominant market share in AI chips faces mounting pressure, the company is choosing to partner proactively rather than engage in head-to-head battles. If these emerging chip companies achieve commercial success, NVIDIA stands to gain a share of the revenue.

Thomas Sommers, CTO of AI chip startup Positron — which is not yet partnered with NVIDIA but is open to it — commented, "NVIDIA's strategy represents a clear shift in industry perception. They are now choosing to extend an olive branch and build a heterogeneous computing ecosystem, rather than trying their utmost to suppress competitors."

Multiple Benefits of the Partnership Strategy

This strategy also helps address past market allegations. There have been customer complaints accusing NVIDIA of pressuring companies to buy only its hardware. A report from two years ago indicated the U.S. Department of Justice had launched an investigation into these claims, though no formal lawsuit has been filed.

Harris emphasized that NVIDIA's long-term vision is to be a comprehensive AI infrastructure provider, stating, "We are absolutely not just a chip company."

NVIDIA accelerated its ecosystem expansion last year by opening its NVLink high-speed interconnect, allowing third-party chips to connect and improve server cluster efficiency. Even if chip sales don't increase, this move allows NVIDIA to sell more networking hardware.

Harris put it bluntly: "It's better to have a product to sell than to have nothing."

In December, NVIDIA invested $20 billion to license technology from inference chip company Groq and absorb its core R&D team. The deal resembled a quasi-acquisition, though Groq remains independent. NVIDIA is concurrently developing a dedicated server rack that combines its GPUs with Groq chips, though market demand for this product remains unclear.

This was followed by the partnership with SambaNova. SambaNova CEO Rodrigo Liang stated that NVIDIA views the startup not just as a competitor, but more as a partner.

Data analysis shows that despite competition from giants like Google and Amazon, NVIDIA's share of the AI inference chip market has actually increased over the past two years. Huang has repeatedly emphasized that NVIDIA GPUs offer superior overall efficiency for various inference tasks. However, NVIDIA is not passively waiting for market forces; it is leveraging its strong balance sheet to provide financing to smaller cloud providers, lowering the barrier for them to purchase expensive NVIDIA AI chips, including underwriting compute leasing programs.

Concurrently, Microsoft, Meta, OpenAI, and the newly involved Anthropic are all developing or planning their own inference chips, which could potentially erode NVIDIA's market share in the long term.

An OpenAI spokesperson stated the company has not yet decided if it will pursue chip co-processing solutions with NVIDIA. NVIDIA has made a significant investment in OpenAI, and the two are also in talks about providing financial backing for OpenAI's Ohio-based mega data center project.

d-Matrix CEO Sid Sheth stated, "The future of the industry is moving towards multi-chip collaboration. NVIDIA is fully accepting of different chips handling different parts of the same AI task. The era of a single GPU monopolizing computing power is over."

While emerging chip companies once positioned themselves as "NVIDIA disruptors," NVIDIA has built formidable technological and ecosystem barriers. Startups now lean more towards complementing NVIDIA GPUs rather than replacing them entirely, and NVIDIA is actively extending partnership offers to these new players.

According to informed sources, this latest collaboration was initiated by NVIDIA, which approached d-Matrix, a company also headquartered in Santa Clara, California.

For a long time, AI companies like OpenAI have mixed different models of NVIDIA GPUs for a single task — manufacturers have long known that lower-power GPUs are better suited for specific workloads. However, cross-vendor heterogeneous chip co-processing for inference is a new architectural concept that has only emerged in the last couple of years.

NVIDIA is not alone in pursuing heterogeneous inference. Amazon, which develops its own Trainium chips (used by Anthropic and soon to be deployed by OpenAI), announced in March that it would partner with AI chip company Cerebras to launch an integrated AI inference server.

Understanding Heterogeneous Inference Technology

Splitting the large model inference process across chips of different architectures is known in the industry as Disaggregated Inference.

Taking NVIDIA's partnerships with SambaNova and Groq as examples: NVIDIA GPUs handle the Prefill stage — the most computationally intensive part of inference — while the startup's chips take on the subsequent Decode stage for token generation.

The d-Matrix solution employs a two-way division of labor: both types of chips simultaneously handle parts of the prefill and decode work. d-Matrix chips specialize in Speculative Decoding, significantly boosting large model response speed.

After a user submits a task, the d-Matrix chip runs a small draft model to predict the output tokens of the main model. The primary large model running on the NVIDIA GPU then verifies and accepts these predictions, thereby reducing overall inference latency.

Embracing a Multi-Chip Ecosystem

d-Matrix CEO Sid Sheth reiterated, "The future of the industry is moving towards multi-chip collaboration. NVIDIA is fully accepting of different chips handling different parts of the same AI task. The era of a single GPU monopolizing computing power is over."

Founded in 2019, d-Matrix completed a $275 million funding round last November, valuing the company at $2 billion. Three informed sources revealed the company has begun discussions for a new round of financing.

Sheth explained that d-Matrix's core differentiation lies in integrating computing units and memory on a single chip, and it does not use the same high-bandwidth memory (HBM) that is currently in short supply and used by NVIDIA. TSMC began mass production of d-Matrix chips this summer, and the company plans to achieve a production rate of thousands of chips per month by year-end.

He revealed the company's current annual revenue is only in the single-digit millions of dollars. The target for next year is for its chips to account for a total data center power consumption of 30-40 megawatts, supporting various inference services like AI code generation, speech, and video.

Emerging AI cloud provider Parasail, based in San Mateo, California, will be the first customer for the joint NVIDIA-d-Matrix server system, planning to offer this hybrid computing service to its tenants in the second half of this year. Parasail CEO Mike Henry stated that this combined server solution is highly attractive as it helps companies break free from high dependence on a single source of NVIDIA hardware.