City of Rio de Janeiro Releases 397B Open-Source Language Model ‘Rio 3.5 Open’ Post-Trained on Qwen
English summary
The city of Rio de Janeiro has post-trained and released a massive language model named Rio 3.5 Open, with 397 billion parameters. It is built upon a Qwen base model—referred to as Qwen 7/2—and integrates SwiGLU activation and Rotary positional embeddings. The model is openly accessible, marking a rare public-sector contribution of a large-scale open LLM.
Chinese summary
里约热内卢市发布了一个名为Rio 3.5 Open的大语言模型，参数规模达397B。该模型基于Qwen（具体为Qwen 7/2变体）进行后训练，并集成了SwiGLU激活和旋转位置嵌入。模型以开放形式提供，是公共部门贡献大规模开源LLM的罕见案例。
Key points
Rio de Janeiro’s city government released Rio 3.5 Open, a 397B-parameter language model.
里约热内卢市政府发布了397B参数的语言模型Rio 3.5 Open。
The model is post-trained from a Qwen base (specifically Qwen 7/2, likely a 7B variant).
该模型基于Qwen基础模型（可能是Qwen 7/2，即一个7B变体）进行后训练。
It incorporates SwiGLU and Rotary embeddings.
模型采用了SwiGLU激活和旋转位置嵌入。
The release is open, giving the community access to a 397B model from a public institution.
模型以开源形式发布，使社区能够获取来自公共机构的397B模型。