Microsoft’s recent improved version of PHI-4 challenges key assumptions to establish artificial intelligence systems that can reason. Since the introduction of thoughtful reasoning in 2022, researchers believe that advanced reasoning requires very large language models with hundreds of billions of parameters. However, Microsoft’s new 14 billion parameter model, PHI-4 improved, questioned this belief. This model uses a data-centric approach instead of relying on pure computing power to achieve comparable performance to large systems. This breakthrough suggests that a data-centric approach can also be as effective for training inference models as conventional AI training. It provides smaller AI models with the possibility to achieve advanced inference by changing the way AI developers train inference models, from “bigger is better” to “better data” better.
Traditional reasoning paradigm
Thinking reasoning has become the standard for solving complex problems in artificial intelligence. This technology guides the language model through step-by-step reasoning, breaking down difficult problems into smaller, easy-to-manage steps. It imitates human thinking by “thinking loudly” in natural language before giving the answer.
However, this capability has important limitations. Researchers have consistently found that only when the language model is very large, the chain of promotion can work well after thinking. The reasoning ability seems to be directly associated with the model size, and larger models perform better on complex inference tasks. This discovery leads to competition for building large inference models in which companies work to transform their large language models into powerful inference engines.
The idea of incorporating reasoning capabilities into AI models comes primarily from observations that large language models can perform secret learning. Researchers observed that when showing examples of the model gradually solving problems, they learn to follow this pattern to solve new problems. This leads to the belief that larger models trained with large amounts of data will naturally develop more advanced inference. The strong connection between model size and inference performance becomes a recognized wisdom. The team invested huge resources to use reinforcement learning to expand inference capabilities, believing that computing power is the key to improving inference.
Understand data-centric approaches
The rise of data-centric AI has challenged the “bigger is better” mentality. This approach shifts the focus from model architecture to careful study of data used to train AI systems. A data-centric approach treats data as material that can be improved and optimized to improve AI performance, rather than treating data as fixed input.
Andrew Ng, a leader in the field, has promoted engineering practices to build systems to improve data quality, rather than just tuning code or scaling models. This philosophy recognizes that data quality and curation are often more important than model size. Companies that adopt this approach show that smaller, well-trained models can outperform larger models if trained with high-quality, well-prepared datasets.
The data-centric approach raises a different question: “How do we improve the data?” rather than “How do we make the model bigger?” This means creating better training data sets, improving data quality and developing system data engineering. In data-centric AI, the focus is on understanding what makes data efficient and not just collecting more data.
This approach shows great promise in training small and powerful AI models using small datasets, with much less computation. Microsoft’s PHI model is a great example of using a data-centric approach to training small language models. These models are trained using curriculum learning, which is mainly inspired by children’s learning through gradually difficult examples. Initially, these models were trained in simple examples and then gradually replaced by harder models. Microsoft constructs a dataset from textbooks, as explained in its paper, “All the textbooks you need.” This helps PHI-3 outperform models such as Google’s Gemma and GPT 3.5 in tasks such as language comprehension, general knowledge, elementary school math questions and medical question answering.
Despite the success of data-centric approaches, inference is often still a feature of large AI models. This is because reasoning requires complex patterns and knowledge, that is, large-scale models are easier to capture. However, this belief has recently been challenged by the development of the improved PHI-4 model.
PHI-4-REONING’s Breakthrough Strategy
The PHI-4 reaction shows how to use a data-centric approach to train small inference models. The model is constructed by fine-tuning the basic PHI-4 model of carefully selected “teachable” prompts and carefully selected “teachable” prompts and reasoning examples generated with OpenAI’s O3-Mini. The focus is on quality and specificity, not dataset size. The model is trained with about 1.4 million high-quality tips, rather than billions of general tips. The researchers filtered examples to cover different levels of difficulty and types of reasoning to ensure diversity. This careful planning makes each training example purposeful, teaching model-specific inference patterns, rather than just increasing the amount of data.
In fine-tuning of supervision, the model receives complete inference demonstration training involving a complete thinking process. These step-by-step reasoning chains help models learn how to build logical arguments and solve problems systematically. To further enhance the model’s inference capabilities, solutions that validate approximately 6,000 high-quality mathematical problems can be further improved. This suggests that even a small amount of focused enhanced learning can significantly improve inferences when curated data.
Performance beyond expectations
This data-centric approach proves effective. The Phi-4-Rounowing outperforms larger open models, such as the DeepSeek-R1-Distill-llalama-70b, which, although much smaller, almost matches the full DeepSeek-R1. In the AIME 2025 test (US Mathematics Olympic Qualifier), Phi-4-Rounowing beats DeepSeek-R1, which has 671 billion parameters.
These benefits are beyond the scope of mathematics and can solve scientific problem solving, coding, algorithms, planning and spatial tasks. Moving from careful data planning to improvements in general benchmarking suggests that this approach builds basic reasoning skills rather than task-specific skills.
PHI-4 reactions challenge the idea that advanced reasoning requires a lot of calculations. When trained with carefully planned data, a 14 billion parameter model can match dozens of times the performance of the model. This efficiency has important consequences for deploying inference AI with limited resources.
Impact on AI development
The success of Phi-4-Reoning marks a transformation in how AI inference models should be constructed. Instead of focusing mainly on increasing the size of the model, teams can invest in data quality and curate, they can increase the size of the model. This makes it easier for organizations to obtain advanced reasoning without a large calculated budget.
The data-centric approach also opens up new research paths. Future work can focus on finding better training tips, making richer inference demonstrations and knowing which data is best helpful in reasoning. These directions may be more productive than building larger models.
More broadly, this can help democratize AI. If smaller models trained with curated data can match large models, advanced AI will be used for more developers and organizations. This can also speed up AI adoption and innovation in areas where large models are not practical.
The future of inference model
PHI-4 reactions set new standards for inference model development. Future AI systems may balance careful data planning and building improvements. This approach acknowledges that data quality and model design are important, but improving data can lead to faster and more cost-effective benefits.
This also enables professional inference models trained with domain-specific data. Instead of general giants, teams can build great models in specific areas through targeted data planning. This will create more efficient AI for specific purposes.
With the advancement of AI, the lessons of PHI-4-curation will not only affect inference model training, but also the overall development of AI. The success of data planning to overcome size limitations shows that future progress lies in combining model innovation with intelligent data engineering, rather than just building larger architectures.
Bottom line
Microsoft’s PHI-4 backhaul has changed the general belief that advanced AI reasoning requires very large models. Instead of relying on larger sizes, the model uses a data-centric approach with high quality and carefully selected training data. Phi-4-Remoning has only 14 billion parameters, but performs a larger model on difficult inference tasks. This suggests that focusing on better data is more important than increasing the size of the model.
This new training method allows organizations without large amounts of computing resources to perform efficient AI efficiency more effectively. The success of the PHI-4 round points to a new direction for AI development. It focuses on improving data quality, smart training and careful engineering, not just making the model bigger.
This approach can help AI faster, reduce costs, and allow more people and companies to use powerful AI tools. In the future, AI may make advanced AI useful in many areas of expertise by combining better models with better data.
