Ether0: A 24B LLM received advanced chemical reasoning task for reinforcement learning RL training

LLMS mainly improves accuracy by extending pretrained data and computing resources. However, due to limited data availability, attention has shifted toward alternative scaling. This includes test time training and inference calculation scaling. The inference model initially enhances performance by emitting a thought process before the answer. Recently, reinforcement learning (RL) training was used. The field of science provides ideal opportunities for inference models. The reason is that they involve “counter-problems” where solution quality assessment is simple, but solution generation is still challenging. Despite conceptual alignment between structured scientific reasoning and model capabilities, current methods still lack detailed approaches to scientific reasoning for multiple choice benchmarks.

Technical evolution of inference architecture

The inference model evolved from early-stage approaches such as COT, zero-ray CoT, and thought trees. They perform complex RL methods through group relative strategy optimization (GRPO) and inference time scaling. Furthermore, inference models in chemistry focus on knowledge-based benchmarks rather than complex inference tasks. Examples include returning synthesis or molecular design. Although datasets such as GPQA-D and MMLU evaluate chemistry knowledge, they are unable to evaluate complex chemical reasoning capabilities. Current scientific reasoning work is still scattered. Limited attempts include universal science omniscience, MED-R1 for medical visual language tasks, and biological seasons for genomic reasoning. However, there is no comprehensive framework for training on large-scale chemical reasoning models.

Ether0 architecture and design principles

Future House researchers propose ether0This is a new model of natural language and outputs molecular structures as smiling strings. It demonstrates the efficacy of inference models in chemistry tasks. It outperforms cutting-edge LLM, human experts and general chemistry models. The training method has performed several optimizations for vanilla RL. This includes distillation of reasoning behavior, dynamic courses and expert model initialization to improve efficiency and effectiveness. In addition, factors such as data efficiency, failure patterns and reasoning behavior were analyzed. This analysis provides a better understanding of the reasoning practicality of solving chemical problems.

Training pipeline: Distillation and GRPO integration

The model uses a multi-stage training program, alternating between the distillation and GRPO stages. This architecture introduces four special tokens. These tokens draw inferences and answer boundaries. Training begins with the SFT of the long COT sequence generated by DeepSeek-R1. These are filtered for effective smile formats and reasoning quality. Expert RL then used GRPO to optimize task-specific strategies for different problem categories. Distillation then merges the professional model into generalists. Merge is done through SFT, and the correct responses collected throughout the training are merged. The final stage applies generalist GRPO to the merge model. This includes continuous mass filtration to eliminate low-quality reasoning and poor molecular substructure.

Performance evaluation and benchmarks

Ether0 exhibits excellent performance with Claude and O1 (including ChemDFM and Txgemma) (including Claude and O1) as well as chemical specificity models (including Claude and O1). It achieves the highest accuracy in all open classification categories while maintaining competitive performance on multiple choice issues. For data efficiency, this model performs better than the traditional molecular transformer model. It trained only 60,000 responses compared to the Full USPTO dataset. ETHER0 achieves 70% accuracy after seeing 46,000 training examples. By comparison, molecular transformers achieved 64.1% on the complete dataset. Under one-time prompt condition, Ether0 exceeded all evaluated boundary models. The safe alignment procedure successfully filters 80% of the insecurity issues without degrading performance on core chemistry tasks.

Conclusion: Impact on Future Science LLM

In short, the researchers introduced Ether0, a 24B parameter model that trains ten challenging molecular tasks. It greatly outperforms cutting-edge LLM, domain experts and professional models. This is achieved through its interwoven RL and behavioral distillation pipelines. The model has excellent data efficiency and reasoning capabilities. It performs well in open chemistry tasks involving molecular design, completion, modification and synthesis. However, limitations include potential generalization challenges beyond organic chemistry. Additionally, general guidance tracking and missing integration of tool names are missing. The release of model weights, benchmark data and reward functions lays the foundation. This foundation helps advance scientific reasoning models in various fields.

View paper and technical details. All credits for this study are to the researchers on the project. Also, please feel free to follow us twitter And don’t forget to join us 99K+ ml reddit And subscribe Our newsletter.

▶ Want to promote your product/website/service to 1 million+ AI Engineer/Developer/Data Scientist/Architects/CTOS/CIO? Let your partner…

Sajjad Ansari is a final year undergraduate student from IIT Kharagpur. As a technology enthusiast, he delves into the practical application of AI, focusing on understanding AI technology and its real-world impact. He aims to express complex AI concepts in a clear and easy way.