AMD acquires Taalas: unlocking models and engraving them into silicon to get rid of HBM constraints

According to Kuai Technology News on August 7, AMD officially announced the acquisition of AI chip startup Taalas to further strengthen its strategic layout in the field of AI reasoning. Taalas was founded in 2023 and is headquartered in Toronto, Canada. Its core technology route is fundamentally different from data flow architectures such as mainstream GPU, Groq LPU and Cerebras wafer-level accelerator. Rather than relying on traditional high-bandwidth memory (HBM) to store model weights, the company etches the weights directly into silicon wafers to form model-specific integrated circuits. The chip is mainly composed of two functional areas: the mask ROM recall structure used to store model weights, and the SRAM recall structure used to store KV cache and fine-tuning adapters. This solution is expected to improve inference performance by an order of magnitude or more. In February this year, Taalas released its first test chip HC1, manufactured using TSMC's 6-nanometer process. Preliminary benchmark tests show that when the chip runs the Meta Llama 3.1 8B model, the speed reaches 16,960 tokens per second - a result that was approximately 48 times that of NVIDIA GPUs and 8.5 times that of Cerebras accelerators at the time. Although Llama 3.1 is no longer the latest model by current standards, the main purpose of this mask-sized chip is to prove the feasibility of the technology concept. Taalas plans to launch the second-generation HC2 chip this summer, with the goal of increasing the number of parameters supported by a single chip to 20 billion. Although this value itself is not huge, through a GPU-like pipeline parallelism, the weights can be distributed across multiple accelerators to run collaboratively. Based on a single chip with 20 billion parameters, only 50 accelerators can support a large model with trillions of parameters. AMD has a rack-level computing platform and internal system design team that are sufficient to meet this demand. For comparison, NVIDIA's recently released LPX system requires dozens of GPUs and at least 2,000 Groq LPUs to run a model of the same size. The Taalas solution has significant advantages in terms of space occupation and power consumption efficiency. According to people familiar with the matter