Advanced Micro Devices (AMD) has agreed to acquire Toronto-based AI startup Taalas, snapping up the inference startup ahead of rivals Intel and NVIDIA.
Taalas, founded in 2023 and emerging from stealth mode in 2024, is privately held, so terms of the transaction were not disclosed.
Founded in 2023, Taalas has taken an unconventional approach to AI hardware by designing chips that embed a neural network’s trained weights directly into silicon, rather than relying on programmable accelerators that repeatedly fetch model parameters from external memory.
The architecture is designed to eliminate one of the primary bottlenecks in AI inference—memory bandwidth. By putting portions of the models directly in silicon, the need to load the model from storage is bypassed, allowing the server to generate responses more quickly while consuming significantly less power.
Google is rumored to be working on a similar chip, and employees working on the project estimate this will make the chips anywhere from six to ten times more efficient than the current version of Google’s existing line of AI accelerators.
But there is a considerable downside, as large language models (LLMs) are constantly changing. Once the model is baked into silicon, you can’t change it, so the only parts of the model that can be put into the chip have to be immutable. How much of the model lends itself to this is unclear.
Still, if the idea works, it could give AMD a considerable advantage over Intel and NVIDIA. The acquisition extends AMD’s aggressive expansion beyond traditional GPUs and CPUs into a broader portfolio of AI infrastructure technologies. Over the past two years, the company has assembled a growing AI stack through acquisitions spanning software, networking, inference optimization, and systems integration.
“We founded Taalas to rethink AI inference from the ground up by building the hardware around the model,” said Ljubisa Bajic, co-founder and CEO of Taalas in a statement. “Our Canada-based team has combined deep technical expertise with a willingness to challenge conventional approaches. Joining AMD will give us the scale, engineering resources and global reach to accelerate our innovation.”
AMD said Taalas’ technology will complement its existing AI platform—including Instinct accelerators, EPYC processors, ROCm software, and Helios rack-scale systems—and will ultimately be integrated into its future accelerator roadmap. The company also plans to develop system-level AI solutions that combine Taalas’ specialized inference silicon with AMD Instinct GPUs.
“AMD is building a full-stack AI platform that gives customers the flexibility to deploy the right compute solutions for every AI workload,” Vamsi Boppana, senior vice president of AMD’s Artificial Intelligence Group, said in a statement. He added that Taalas’ technology strengthens AMD’s AI portfolio with differentiated inference performance and efficiency.
The purchase underscores why the industry has shifted its focus from AI training to inference. Training LLMs requires massive clusters of GPUs and an equally massive amount of power. But once the training is done, inference takes over the process.
Inference requires much less power than training, and on an individual basis, its power draw is minimal and the GPU is overkill for what you need. But inference is done at scale; millions of inference tokens are consumed every second, so in the end, inference ends up costing more power than training over time.
That’s why there has been such a concerted effort to develop inference processors. NVIDIA has basically won the training war, but GPUs can be overkill for many inference workloads. That’s where people are trying to build an alternative.
The strategy could allow AMD to address workloads where dedicated inference hardware delivers lower operating costs than general-purpose accelerators. Industry analysts increasingly expect hyperscale cloud providers to deploy mixtures of programmable GPUs for training and specialized inference chips for serving production AI applications.
