Meet Argus: a scalable AI framework for training large recommended transformers to billions of parameters

Yandex introduced Argus (autoregressive generative user order modeling)a large-scale transformer framework for recommendation systems that can scale up to one billion parameters. This breakthrough led Yandex to list a small number of global technology leaders along with Google, Netflix and Meta, who have successfully overcome the long-term technical hurdles of extending recommended transformers.

Break technical barriers in recommendation systems

Recommendation systems have long struggled with three stubborn constraints: short-term memory, limited scalability, and poor adaptation to transfer user behavior. The conventional architecture narrows user history to a small window of recent interactions, discarding months or years of behavioral data. The result is a shallow view of intention, which misses long-term habits, subtle shifts in taste, and seasonal cycles. As the catalog expands to billions of projects, these truncated models not only lose accuracy, but also have to be stifled in accordance with large-scale personalized computing needs. The results are familiar: stale advice, lower engagement and fewer chances of stumbled upon.

Few companies have successfully extended their recommended transformers beyond experimental setups. Google, Netflix and Meta have made substantial investments in this space, reporting on the benefits of architectures such as YouTubednn, Pinnerformer and Meta’s generation referrals. Together with Argus, Yandex joins this selected company, showing recommended models of billions of parameters. By modeling the entire behavior schedule, the system can discover obvious and hidden correlations in user activities. This long-term perspective allows Argus to capture evolving intentions and periodic patterns with greater loyalty. For example, the model not only responds to recent purchases, but also learns to predict seasonal behavior, such as tennis brands that automatically surface when summer approaches – without requiring users to repeat the same signal year after year.

Technological innovation behind Argus

This framework introduces several key advances:

• Dual-target pre-training: Argus breaks down autoregressive learning into two subtasks – Next forecast and Feedback forecast. This combination not only improves imitation of historical system behavior and modeling of true user preferences.
• Scalable transformer encoder: The model has a consistent performance improvement of all indicators from 32m to 1B parameters. On the billion-parameter scale, the improvement in paired accuracy increased by 2.66%, indicating the emergence of the scaling law of recommended transformers.
• Extended context modeling: Argus handles user history of up to 8,192 interactions on a single pass, thus personalizing in months of behavior, not just the last few strokes.
• Effective fine-tuning: A two higher architecture allows offline computing embedding and scalable deployment, reducing inference costs relative to previous target-aware or impression-level online models.

Real-world deployment and measurement of benefits

Argus has already made large-scale deployments on Yandex’s music platform, serving millions of users. In production A/B testing, the system implements:

• +2.26% total hearing time (TLT) increase
  
• +6.37% chance of increased

For any recommendation model based on deep learning, these are the largest record quality improvements in the platform’s history.

The direction of the future

Yandex researchers plan to expand Argus to Real-time suggestion tasksexplore Functional engineering for paired rankingand adjust the frame to High mentality domain For example, large e-commerce and video platforms. Proof of scaling user sequence modeling through transformer architecture shows that recommendation systems are prepared to follow a scaling trajectory similar to natural language processing.

in conclusion

Together with Argus, Yandex has established itself as one of the few global leaders driving the latest recommendation systems. By sharing its breakthroughs publicly, the company can not only improve personalization in its services, but also accelerate the development of recommended technologies throughout the industry.

Check The paper is here. Thanks to Yandex team for their thought leadership/resources in this article.

Asif Razzaq is CEO of Marktechpost Media Inc. As a visionary entrepreneur and engineer, ASIF is committed to harnessing the potential of artificial intelligence to achieve social benefits. His recent effort is to launch Marktechpost, an artificial intelligence media platform that has an in-depth coverage of machine learning and deep learning news that can sound both technically, both through technical voices and be understood by a wide audience. The platform has over 2 million views per month, demonstrating its popularity among its audience.