Tencent Opens Hy4 Preview, a 770-Billion-Parameter Model Built for AI Agents

Hy4 preview activates 49 billion parameters per token, supports a one-million-token context window, and targets coding, office work, and scientific research.

Tencent has released Hy4 preview, its new open-weight language model. This initial version contains 770 billion parameters, with 49 billion activated for each token, and supports a context window of up to one million tokens. The company is primarily targeting agents handling long-running tasks in software development, office work, data analysis, game development, and scientific research.

The model uses a Mixture-of-Experts architecture. Of its 78 layers, 77 contain 256 routed experts and one shared expert. For every token, the system selects eight routed experts alongside the shared component. This setup limits the amount of compute used at each step without reducing the model’s overall size, which remains substantial.

Tencent has added a separate layer for speculative decoding, designed to anticipate several tokens and accelerate generation. Its attention system uses Gated DeepSeek Sparse Attention with IndexCache, a technique that reuses selected indexes across layers instead of repeatedly calculating them. Deep reasoning is enabled by default, although users can turn it off for more direct responses.

Hy4 preview was trained around professional scenarios supplied by Tencent software engineers, financial analysts, security specialists, and game developers. The model is intended to examine multiple files, produce documents, spreadsheets, or presentations, use external tools, and maintain continuity across complex, multi-step projects. Tencent has also integrated it into CodeBuddy and WorkBuddy.

In the evaluation table published by Tencent, Hy4 preview scores 82.9 on SWE-bench Multilingual, 65.7 on SWE-bench Pro, 64.3 on DeepSWE, and 85.4 on Terminal-Bench 2.1. It also reaches 83.7 on MCP-Atlas, which evaluates tool use, and 66.2 on OfficeQA Pro. On GPQA Diamond, a benchmark built around difficult scientific questions, it scores 92.3.

These results place Hy4 preview among the most competitive open-weight models, without making it the leader in every category. It remains behind Claude Opus 5 on several software-development and workplace-agent evaluations. Its performance more often falls within the same range as Qwen 3.8, GLM 5.3, and Kimi K3, with rankings changing considerably from one task to another.

Tencent also conducted an internal comparison involving 163 specialists and 203 engineering tasks. Hy4 preview finished slightly ahead of GLM 5.3 and Kimi K3. The differences were modest, however, and the study was conducted within Tencent. Several other scores in the comparison table come from internal evaluations or tests rerun by the company, so independent verification will still be necessary.

The standard and FP8 weights are available under an Apache 2.0 license on Hugging Face, ModelScope, GitCode, and CNB. Tencent also provides a fine-tuning pipeline, an OpenAI-compatible API, and deployment instructions for vLLM and SGLang. The official configuration distributes the FP8 version across eight accelerators, indicating that self-hosting remains aimed at organizations with significant computing infrastructure.

The model is also available through Tencent Cloud and OpenRouter. Pricing starts at $0.834 per million input tokens and $2.501 per million output tokens. Cached input costs $0.042 per million tokens.

Tencent openly describes the release as unfinished. Hy4 preview can spend longer than necessary reasoning through certain tasks and has a tendency to over-check its own work. The company chose to distribute this version before development was complete in order to gather feedback, following the approach previously used with Hy3. The “preview” label should therefore be taken literally: the weights are available, but the model will continue to change before its final release.