Google is reportedly developing a new custom AI chip designed specifically to run its Gemini family of artificial intelligence models more efficiently, marking the latest in a series of custom chips optimized for AI processing.

The chip, internally codenamed “Frozen v2,” is being engineered to accelerate AI inference in a very unique manner. According to The Information, the Frozen name comes from the idea of permanently etching part of the model into the silicon.

Google employees working on the chip estimate this will make the chip anywhere from six to ten times more efficient than the current version of Google’s existing line of homegrown AI chips when it launches, based on the number of tokens it can serve per unit of power.

The Information stated that the chip is still in the early design stages and that Google won’t be deploying them until at least 2028.

The effort also underscores the growing strategic importance of custom silicon in the AI race. The GPU may be good for training models but not for inference. GPUs are considered overkill for inference and are not the most power efficient design. There is considerably more effort in silicon startups around inference than there is around training. It would seem no one wants to take on NVIDIA or AMD on their turf.

Such a design is risky. Large language models are constantly in a state of flux and are being updated regularly. To bake it into silicon means you can’t change it so the only parts of the model that can be put into the chip have to be immutable. How much of the model lends itself to this is unclear, but it can be limiting.

The Information notes that this concept of baking the model into the silicon is not new. There is a Toronto-based semiconductor startup called Taalas that does the same thing.

Taalas emerged from stealth in March 2024 with $50 million in funding led by Quiet Capital and semiconductor veteran Pierre Lamond. The company was founded by Ljubisa Bajic, who previously founded AI chip startup Tenstorrent before leaving to start Taalas.

The company’s philosophy is summed up in its slogan: “The model is the computer.” Rather than treating an AI model as software that runs on hardware, Taalas argues that the model itself should become the hardware.

In inferencing, a model’s weights are stored in high-bandwidth memory (HBM). GPUs repeatedly fetch those weights from memory, so massive amounts of data move between memory and compute units, consuming power and limiting performance.

Taalas wants to eliminate this bottleneck by hardwiring both the neural network architecture and its trained weights directly into the chip. Instead of loading a model into memory, the chip itself embodies the model. This greatly reduces memory traffic that makes up inference workloads.

Taalas claims its technology could deliver up to 1,000x better efficiency than conventional software-based inference at a much lower latency. It also avoids the expensive HBM needed in GPU and AI accelerators.

But there is a trade-off. Modern foundation models evolve rapidly. If a company hardwires a model into silicon and a superior architecture appears six months later, that chip may become obsolete. This will be the challenge of both Google and Taalas.