Tencent Hy3 programming evaluation is released: the parameters are only one-fifth of the opponent, but the coding ability is equal to DeepSeek-V4-Pro. SuperCLUE released the special programming evaluation data of Tencent Hy3 language model, and compared it with popular models such as DeepSeek-V4-Pro on the same stage to compete in coding ability.

Tencent Hy3 programming evaluation is released: the parameters are only one-fifth of the opponent, but the coding ability is tied with DeepSeek-V4-Pro. SuperCLUE released the programming special evaluation data of Tencent Hy3 language model, and compared it with popular models such as DeepSeek-V4-Pro on the same stage to compete in coding ability. Hy3 adopts MoE architecture, with a total parameter size of 295B, activation parameters of only 21B, and supports 256K context length. It is known as the strongest language model of Hunyuan. The results show that this model has much smaller parameters than its opponents, and it has delivered a surprising answer in programming scenarios. Comprehensive comparison of four major dimensions, Hy3 takes into account both performance and cost. This evaluation is specifically designed for real scenarios where domestic programmers write code every day. Each programming task requires dozens of rounds of back-and-forth communication, and code analysis, modification and verification are gradually completed, which is closer to the actual development and debugging process. The evaluation compares the four dimensions of usage cost, running speed, communication rounds and token consumption to help developers choose the appropriate model based on their own budget and efficiency needs. In terms of code ability score, the high-scoring version of Hy3 scored 47.37 points, which is exactly the same as DeepSeek-V4-Pro. You know, the parameter scale of many benchmark models is four or five times larger than that of Hy3, but they fail to widen the gap, which shows that a lot of optimization has been done to their model architecture and training methods. In terms of cost, Hy3 has more obvious advantages. It only costs 0.43 yuan to complete a programming question on average. Long-term high-frequency calls will not bring too much overhead, and the financial pressure of commercial implementation is greatly reduced. A single question takes less than 400 seconds, and 40 rounds of dialogue can complete the entire set of tasks. In terms of running speed, Hy3 takes an average of less than 400 seconds for a single question, ranking at the forefront of the evaluation. It is suitable for real-time coding and online debugging scenarios that require fast response. The communication efficiency is equally impressive, with an average of more than 40 rounds of dialogue to complete the entire set of tasks. There is no need to repeatedly adjust instructions and code corrections, and the convergence speed is faster. At the same time, it consumes only about 1.16 million Tokens per task, taking up less cloud computing power and lowering platform operating pressure. For programmers who often use AI-assisted development, Hy3 has found a rare balance between performance and cost. When parameter scale is no longer the only criterion for capabilities, models with "small size and high energy" are being proven with actual data: optimizing the architecture and training strategies can also compete with flagship opponents on the programming track. With Hy3 being fully open sourced under the Apache 2.0 protocol, this cost-effectiveness battle may have just begun. via AI News (author: AI Base)