AI Computing to Consume Power of Multiple NYCs by 2026, Says Founder

Nvidia and Partners Expand AI Data Centers Worldwide

Nvidia, along with its partners and clients, has been actively expanding the size of computer facilities around the globe to meet the high computational demands of training massive artificial intelligence (AI) models like GPT-4. This expansion is set to become even more crucial as additional AI models are deployed into production, according to Thomas Graham, co-founder of the optical computing startup Lightmatter. In a recent interview in New York with Mandeep Singh, a senior technology analyst from Bloomberg Intelligence, Graham highlighted the growing need for computational resources.

"The demand for more compute isn't just about scaling laws; it's also about deploying these AI models now," Graham explained. Singh inquired about the future of large language models (LLMs) like GPT-4 and whether they would continue to grow in size. In response, Graham shifted the focus to the practical application of AI, emphasizing the importance of inferencing, or the deployment phase, which requires substantial computational power.

"If you consider training as R&D, inferencing is essentially deployment. As you deploy, you'll need large-scale computers to run your models," Graham stated during the "Gen AI: Can it deliver on the productivity promise?" conference hosted by Bloomberg Intelligence.

Graham's perspective aligns with that of Nvidia CEO Jensen Huang, who has emphasized to Wall Street that advancing "agentic" AI forms will necessitate not only more sophisticated training but also significantly enhanced inference capabilities, resulting in an exponential increase in compute requirements.

Lightmatter's Role in AI Infrastructure

Founded in 2018, Lightmatter is at the forefront of developing chip technology that uses optical connections to link multiple processors on a single semiconductor die. These optical interconnects can transfer data more efficiently than traditional copper wires, using less energy. This technology can streamline connections within and between data center racks, enhancing the overall efficiency and economy of the data center, according to Graham.

"By replacing copper traces in data centers—both on the server's printed circuit board and in the cabling between racks—with fiber optics, we can dramatically increase bandwidth," Graham told Singh. Lightmatter is currently collaborating with various tech companies on the design of new data centers, and Graham noted that these facilities are being built from the ground up. The company has already established a partnership with Global Foundries, a contract semiconductor manufacturer with operations in upstate New York, which serves clients like Advanced Micro Devices.

While Graham did not disclose specific partners and customers beyond this collaboration, he hinted that Lightmatter works with silicon providers such as Broadcom or Marvell to create custom components for tech giants like Google, Amazon, and Microsoft, who design their own data center processors.

The Scale and Future of AI Data Centers

To illustrate the magnitude of AI deployment, Graham pointed out that at least a dozen new AI data centers are either planned or under construction, each requiring a gigawatt of power. "For context, New York City uses about five gigawatts of power on an average day. So, we're talking about the power consumption of multiple New York Cities," he said. He predicts that by 2026, global AI processing will demand 40 gigawatts of power, equivalent to eight New York Cities, specifically for AI data centers.

Lightmatter recently secured a $400 million venture capital investment, valuing the company at $4.4 billion. Graham mentioned that the company aims to start production "over the next few years."

When asked about potential disruptions to Lightmatter's plans, Graham expressed confidence in the ongoing need for expanding AI computing infrastructure. However, he acknowledged that a breakthrough in AI algorithms requiring significantly less compute or achieving artificial general intelligence (AGI) more rapidly could challenge current assumptions about the need for exponential compute growth.