8GB显存也能玩转128K超长上下文

8GB of video memory can also handle 128K ultra-long contexts! The open source local large model tool Strata 0.2.10 is released. The open source local AI tool Strata has officially received the 0.2.10 version update recently. As a local large model running artifact designed for consumer-grade graphics cards, this update brings breakthrough progress in lowering the memory threshold, increasing inference speed, and optimizing multi-graphics card collaboration, making it easier for ordinary computers to run large models with hundreds of billions of parameters. Significant reduction in video memory: How can an 8GB graphics card handle 128K ultra-long contexts? In previous local running scenarios, if users tried to open a 64K or higher context on a device with 8GB video memory, the startup would often fail due to exhaustion of the expert cache space, and they would even have to compromise down to around 48K. In Strata 0.2.10, the official optimized the underlying resource allocation mechanism: when the video memory is insufficient, the system will give priority to dynamically adjusting the data allocation between the video memory and the memory, and try to retain the context length set by the user as much as possible. This means that users can now successfully launch large models such as Qwen3.8IQ3_S on a device with 8GB of video memory, and maintain an ultra-long context of up to 128K (i.e. 131,072Token). The core principle is that Strata will automatically reduce the KV cache window residing in the GPU based on real-time video memory margin, and if necessary, will evenly spread part of the cache to system memory or hard disk. Official test data shows that when DFlash2 mode is turned on under the 8GB video memory limit, the generation speed can reach 63 to 73 Token/s; even with the Intel Arc A7508GB real graphics card, the 128K context can be started smoothly and the generation speed of 12.7 to 14.8 Token/s can be maintained. However, developers also need to pay attention: although the pressure on video memory hardware is shared through memory and solid-state drives, it also places higher requirements on the computer's memory configuration. Official recommendations suggest that the memory capacity should start at 32GB, with 48GB being more appropriate. Only with 64GB of memory can you truly cope with various complex scenarios. The DFlash2 accelerator is enabled by default: saving an additional 4GB of video memory. Another core highlight of this update is that the new DFlash2 is officially set as the default speculative decoder. As an efficient speculative decoding technology, the basic logic of DFlash2 is to let a lightweight auxiliary model predict the next generated content in advance, and then hand it over to the main model for batch verification. Compared with the previously used MTP speculation layer, which requires about 5.2GB of video memory space, DFlas