Full-link practice of Taote end-side model from fine-tuning to application

This article summarizes Taote App’s full-link practice in client-side model construction from March to August. In order to solve the problems of real-time user intention recognition and the delay and cost of large cloud models, the team selected QWen3-0.6B as the basic model and improved the real machine accuracy to more than 80% through two generations of data set iterations. In terms of engineering implementation, LoRA fine-tuning, ShortGPT-based deep pruning and AWQ quantification technology are used to greatly optimize the model volume and inference performance; at the same time, the end-to-end delay is significantly reduced by combining prompt simplification, Prefix Cache and incremental inference. In the end, this terminal model was successfully implemented in the scenarios of information flow reordering and pre-loss equity retention, achieving positive growth of GMV.