The surge in artificial intelligence infrastructure within Silicon Valley has highlighted a significant physical constraint: the capacity for data transmission through copper wires is limited. Beyond a certain point, issues with heat, distance, and power consumption become unmanageable. This is the driving force behind the sudden, intense interest from investors, chipmakers, and cloud giants in photonics—the use of light rather than electrical signals to move data between AI chips and servers.

I recently visited the Silicon Valley headquarters of Lightmatter, a startup demonstrating its latest photonic hardware designed for AI data centers. Following the event, I spoke with Lightmatter's CEO, Nick Harris, to explore why optical technology might become essential infrastructure for the AI era. Harris, who holds a Ph.D. from MIT, appears remarkably youthful and intelligent. His company has also seen impressive success, raising $8.5 billion from major investors including Google, Fidelity, and T. Rowe Price.

On Tuesday, Lightmatter joined NVIDIA's NVLink Fusion ecosystem, a move expected to enhance the compatibility of the startup's technology with NVIDIA's dominant AI hardware. Below is an edited version of our conversation for clarity and brevity.

The AI Industry's Shift to Photonics

When asked why AI companies are suddenly interested in photonics, Harris explained that the industry has reached a stage where boosting performance is less about accelerating individual chips and more about efficiently connecting vast numbers of GPUs. Current AI systems heavily depend on copper wire links between GPUs. While this works adequately at smaller scales, it creates a bottleneck when companies link hundreds or thousands of GPUs to build cutting-edge models. Electrical signals degrade over relatively short distances and generate substantial heat.

Photonics utilizes light transmitted through fiber optics. This enables data to travel farther, faster, and with lower energy consumption. "Imagine you have 500 GPUs connected by copper wires to communicate within what's called a scale-up domain, running model training workloads," Harris told me. With copper, you might need four separate GPU server racks to achieve that scale. "But if you switch entirely to optics, you can directly connect all 500 GPU servers. This dramatically reduces the time to train an AI model. For a frontier model like Claude, you could see a threefold speed increase."

"The company that masters this technology first gains a faster cycle for launching new models in this frontier race. They face a choice: release a new model every month, or spend three months to release a much larger model," he added. Harris further illustrated the power efficiency: "Assume my power supply is limited. With the same power, I could get triple the performance. A 1-gigawatt supply would then feel like 3 gigawatts. Alternatively, you could consume the same amount of power in one-third the time."

Challenges with Copper in AI Data Centers

Addressing the specific problems with copper interconnects inside AI data centers, Harris noted, "Copper wires can only carry data about a meter" because electrical signals degrade rapidly. "The signal creates an electrical impulse in the wire, but it gets weaker and weaker with distance. After about a meter, the data is lost."

This physical limitation introduces another major issue: heat dissipation. Due to the short reach of copper, GPU server racks in AI data centers are now packed tightly together. "They're crammed in," Harris said. "The issue is I need them all stacked together so the copper can reach. The downside is it's extremely difficult to cool."

Photonics changes this dynamic because light signals can propagate much farther and faster without degradation, allowing GPU servers and racks to be spaced further apart. "Optics don't care about distance," Harris stated. "They could be a kilometer away." This grants data center operators greater flexibility in designing and cooling AI clusters, potentially saving significant power costs dedicated to cooling these systems.

The Significance of Bidirectional Communication

A more practical innovation Lightmatter is working on involves reducing the sheer volume of cabling required inside AI data centers. Harris indicated that some next-generation AI clusters need roughly 300 miles of cabling. Lightmatter aims to halve this length using a technology called BiDi, short for bidirectional communication.

"Typically, whether using copper or fiber, if I want a connection between two GPUs, I need two wires," Harris explained. "One is a transmit wire, and the other is a receive wire." Lightmatter's approach merges both directions into a single cable. For hyperscale data centers, reducing fiber length is critical as cables occupy space, generate heat, complicate maintenance, and increase costs. Lightmatter points out that cutting the total fiber requirement from 300 miles to 150 miles could significantly simplify the construction of massive AI clusters.

The Historical Barrier to Photonic Adoption

When questioned why photonics wasn't adopted earlier, Harris identified cost as the primary obstacle. "Photonics was too expensive," he said. He explained that this is changing due to improvements in manufacturing technology and the explosive growth in demand for AI infrastructure.

"The people designing these systems are chasing reliable 2x performance improvements. Doubling bandwidth, doubling performance, and they insist on achieving it regularly. Copper used to offer some incremental gains, but that well is now dry. Beyond that, there's a stronger driver: the realization that the company that adopts and deploys photonics first—and NVIDIA is likely one of them—holds a massive performance advantage."

"It has shifted from 'switch when you have to' to 'switch to gain a competitive edge,'" Harris concluded.