AI chip power consumption exceeds 1,000 watts! The liquid cooling penetration rate has soared to 53% and cannot be suppressed: heat dissipation has become the biggest bottleneck

Fast Technology reported on August 22 that Kim Jung-ho, a professor at the Korea Academy of Science and Technology, recently warned that the simultaneous use of 3D stacking technology for system semiconductors and memories will make heat dissipation challenges increasingly severe, which may limit the further expansion of AI computing power. This early warning points out the most urgent contradiction in the current AI industry: chip performance is soaring, but heat dissipation capacity cannot keep up. TrendForce data shows that the TDP (thermal design power consumption) of single AI chips from manufacturers such as Nvidia, AMD, and Google has generally exceeded 1,000 watts, and the power of cabinet-type AI servers has reached hundreds of kilowatts. In addition, the penetration rate of liquid cooling in AI chips is expected to increase from approximately 33% in 2025 to 53% in 2026, and to approach 60% in 2027. Even so, the heat dissipation pressure has not been fundamentally relieved. The penetration rate of liquid cooling is increasing, but the heat source of the chip itself is also expanding simultaneously. Traditional cooling methods can only respond passively and cannot actively curb the increase in heat sources. Storage and logic chips are simultaneously evolving towards three-dimensional stacking, and the heat generated per unit area continues to rise. Take Samsung as an example. Its zHBM technology vertically stacks memory chips on top of AI accelerators. Samsung's wafer bonding technology is expected to achieve more than ten times the storage density of HBM5 and reduce thermal resistance by more than 50%. As chips are stacked higher and higher, it becomes increasingly difficult for heat to escape. Facing the ever-expanding heat dissipation dilemma, traditional air cooling solutions have hit the physical ceiling. CPO (co-packaged optics) and STCO (system technology collaborative optimization) are regarded as the key to breaking the situation. CPO converts electrical signals into optical signals. Compared with traditional copper interconnects, this technology can improve performance and energy efficiency while reducing heat generation. STCO collaboratively optimizes computing, storage, interconnection, power supply and heat dissipation from the system level to avoid thermal bottleneck shifts caused by local optimization.