Baidu officially opens its latest Ernie 4.5 series, a powerful base model series designed to enhance language understanding, reasoning and power generation. This release includes ten model variants from compact 0.3B dense models to a large number of Experts (MOE) architectures, with the largest variants totaling 424B parameters. These models are now freely available for free for the global research and developer community by embracing the face, allowing open experimentation and wider access to cutting-edge Chinese and multilingual technologies.
Technical overview of Ernie 4.5 architecture
The Ernie 4.5 series is based on Baidu’s previous Ernie model iteration by introducing advanced model architectures (including dense and sparsely activated MOE designs). The MOE variants are particularly noteworthy for efficient scaling of parameter counts: the ERNIE 4.5-MOE-3B and ERNIE 4.5-MOE-47B variants only activate the expert subset of each input token (usually 2 out of 64 experts), maintaining the number of activity that manages model expression rates and routine rates.
The Ernie 4.5 model is trained using a mixture of supervised fine tuning (SFT), reinforcement learning with human feedback (RLHF) and contrast alignment techniques. The training corpus uses BAIDU’s proprietary multi-stage preprocessing pipeline, spanning 56 trillion tokens across a variety of fields in Chinese and English. The final model shows that high fidelity is shown in terms of tracking, multi-turn dialogue, long-form generation and reasoning benchmarks.
Model variants and open source versions
Ernie 4.5 version includes the following ten variants:
- Dense Model: Ernie 4.5-0.3b, 0.5b, 1.8b and 4b
- MOE model: ERNIE 4.5-MOE-3B, 4B, 6B, 15B, 47B and 424B total parameters (with different activity parameters)
For example, the MOE-47B variant only activates 3B parameters during inference, while the total is 47B. Similarly, the 424b model (the largest released by Baidu) adopts a sparse activation strategy to make the reasoning feasible and scalable. These models support FP16 and INT8 quantization for efficient deployment.
Performance Benchmark
The Ernie 4.5 model shows significant improvements on several key Chinese and multilingual NLP tasks. According to the official technical report:
- exist cmmluErnie 4.5 surpasses previous Ernie versions and achieves state-of-the-art accuracy in Chinese understanding.
- exist mmluMultilingual Benchmark Ernie 4.5-47B demonstrates competitive performance with other leading LLMSs such as GPT-4 and Claude.
- for Long formation generationErnie 4.5 can achieve higher coherence and fact scores when evaluated using Baidu’s internal metrics.
In the task of following instructions, the model benefits from contrast fine-tuning, with improved consistency with user intentions and reduced hallucination rates compared to earlier versions of Ernie.
Application and deployment
The Ernie 4.5 model is optimized for a wide range of applications:
- Chatbots and Assistants: Multilingual support and alignment following instructions make it suitable for AI assistants.
- Search and Q&A: High retrieval and generated fidelity allow integration with RAG pipelines.
- Content generation: Through a better factual basis, the generation of long texts and knowledge-rich content can be improved.
- Code and multi-mode extensions: Although the current release is focused on text, Baidu says Ernie 4.5 is compatible with multimodal extensions.
To support support for up to 128K context lengths in some variants, the Ernie 4.5 series can be used for tasks and reasoning that require long-term documents or sessions.
in conclusion
The Ernie 4.5 series represents an important step in open source AI development, providing a multifunctional model for scalable, multilingual and instruction-consistent tasks. Baidu’s decision to release a decision from a lightweight 0.3B variant to a 424B parameter MOE model highlights its commitment to inclusive and transparent AI research. With comprehensive documentation, embracing the open supply of faces, and support for effective deployment, Ernie 4.5 can accelerate global progress in natural language understanding and generation.
Check Embrace the paper and model on the face. All credits for this study are to the researchers on the project. Also, please stay tuned for us twitter And don’t forget to join us 100K+ ml reddit And subscribe Our newsletter.
Asif Razzaq is CEO of Marktechpost Media Inc. As a visionary entrepreneur and engineer, ASIF is committed to harnessing the potential of artificial intelligence to achieve social benefits. His recent effort is to launch Marktechpost, an artificial intelligence media platform that has an in-depth coverage of machine learning and deep learning news that can sound both technically, both through technical voices and be understood by a wide audience. The platform has over 2 million views per month, demonstrating its popularity among its audience.
