Block launches full-duplex AI voice model Dessa: full of real-life feeling, supports local operation on Apple MacBook

IT House reported on October 25 that Block, a company owned by Twitter co-founder Jack Dorsey, has launched a new voice model Dessa, which supports local operation and can simulate emotional expressions such as laughter, pauses, and choking, and the sound is highly anthropomorphic. According to the official introduction, Dessa is trained entirely based on human speech data, and the training process only uses a set of NVIDIA GPU racks. Dessa is a full-duplex model, and like GPT-Live released by OpenAI earlier this year, it can listen to the user's voice while speaking. Mainstream speech models generally need to be paired with large language models to retrieve knowledge; Block said that Dessa's own knowledge comes from the dialogue data used for its training. Block said that the existing voice AI experience is often stiff and inconsistent. The root cause is that the generated link must first be converted into text, and then the text drives speech synthesis. Many models such as ElevenLabs v4 and Fish Audio Drama 3 rely on voice tags to control emotions. Dorsey's company claims that emotional expression is built into the model natively, but it is unclear whether it still relies on some kind of label to regulate emotions. Block wrote in a blog: "The current voice AI interaction experience is really bad. It first converts your voice into text, and a large amount of information that cannot be carried by text is lost directly. Even the world's most powerful AI cannot hear your emotions. We think this is unreasonable. Not only is the experience poor, but it also has potential risks." The research team released multiple dialogue samples with Dessa. This intelligent agent can switch timbre, speaking speed and emotion. For example, in one conversation, the AI ​​said that it was very tired from work, and its voice also sounded tired. In another example, the AI ​​talked about how the other person made it feel at ease, with an emotional tone that was almost choking. Another demo featured Block employees playing a game with the voice assistant. According to official instructions, this model can be run locally on consumer-grade computers.