The Allen Institute of Artificial Intelligence (AI2) has introduced Autods (automatic discovery through surprise discovery), a pioneering prototype engine for open and autonomous scientific discovery. Unlike traditional AI research assistants, relying on artificially defined goals or queries, automatically generate, test and iterate hypotheses, by quantifying and finding “Bayesian surprises”, this is a principle measure of true discovery that even exceeds the specific searches of humans.
From goal-driven query to open exploration
Traditional approaches to autonomous scientific discoveries (ASD) often revolve around answering pre-specified research questions: generating hypotheses related to a given question, and then experimentally verifying them. Autods fundamentally deviates from this paradigm. Drawing inspiration from the exploration of curiosity-driven human scientists, Autods operates in an open way – it decides What Ask questions, Which one Assumptions to pursue how Based on previous results, none of these have predefined goals.
Open discovery is challenging in nature, requiring traversing huge assumption spaces and priorities mechanisms. To address these challenges, Autods formalizes the concept of “surprise”, a measurable shift in belief in hypotheses before and after empirical evidence is obtained.
Quantify Bayesian surprises through large language models
At the heart of Autods is a new framework for estimating Bayesian surprises. For each generated hypothesis, state-of-the-art large language models (e.g., GPT-4O) act as probability observers, elicit their “belief” about hypothesis (in the form of probability) before and after empirical testing. These belief distributions are constructed by multiple judgments sampled from the LLM and modeled with a Beta distribution.
To detect meaningful findings, Autods calculated the Kullback-Leibler (KL) difference between the posterior (after evidence) and prior (previous evidence) β distribution, a formal measure of Bayesian surprise. Crucially, only beliefs shift to thresholds of evidence change (e.g., from possible to possible errors) are seen as truly surprising, focusing the system on substantial discovery rather than trivial updates of uncertainty.
Use MCT for valid hypothesis search
Effectively exploring huge hypothetical landscapes requires more than just childish sampling. Autods utilizes Monte Carlo Tree Search (MCT) for a gradual expansion to guide it in finding surprising findings. Each node in the search tree represents a hypothesis, and the branch corresponds to a new hypothesis for the previously discovered condition. This structure allows Autods to balance exploring new avenues and following up with fruitful prospects.
Unlike greed or beam search methods, MCTS discovery efficiency is very high under fixed calculations. Empirically, in 21 datasets from the fields of biology, economics, and behavioral sciences, automatic performance of repeated sampling, greed and beam search benchmarks exceeds the surprising assumption judged by LLM.
Modular multi-proxy LLM architecture
Autods carefully planned a series of professional LLM agents, each responsible for different parts of the autonomous scientific workflow:
- Assumptions are generated
- Experimental design
- Programming and executing
- Results analysis and revision
Deduplication of semantically similar assumptions uses a hierarchical clustering pipeline: LLM-based text embedding combined with paired semantic equivalence checks ensure that the final output set contains only truly different findings.
Human consistency and explanatory
Consistency with human scientific intuition is the key benchmark. In structured human assessments (reviewers with MS/PHD grade stemming background), 67% of hypothetical automatic drives are considered surprisingly surprising to domain name experts. Furthermore, with proxy metrics such as predicted “interesting” or “utility”, Autods’ Bayesian surprise metrics are more consistent with human judgments.
Interestingly, the nature and direction of surprising belief transfers vary by field of science, for example, confirmatory claims often require stronger evidence to be convincingly surprising rather than novel forgery.
Practical considerations and future prospects
Autods showed high implementation and experimental effectiveness, with more than 98% of the evaluations found to be correctly implemented by human reviewers. Although the current pipeline relies on API-driven LLM and therefore faces latency restrictions, the team also explores a “programmed search” implementation that results faster, albeit less conceptually.
Although Autods is currently a research prototype (forward-looking program), its architecture and experience success charts are scalable, compelling avenues for AI-driven science.
in conclusion
Autods represent a significant advance in autonomous scientific reasoning. By transitioning from goal-driven research to curiosity-based exploration and searching it in Bayesian surprises, it points to the path to future AI systems that can complement, accelerate, and even independently lead scientific discoveries.
Check Paper, github pages and blogs. All credits for this study are to the researchers on the project.
Sponsorship Opportunities: Attract the most influential AI developers in the United States and Europe. 1M+ monthly readers, 500K+ community builders, unlimited possibilities. [Explore Sponsorship]
Asif Razzaq is CEO of Marktechpost Media Inc. As a visionary entrepreneur and engineer, ASIF is committed to harnessing the potential of artificial intelligence to achieve social benefits. His recent effort is to launch Marktechpost, an artificial intelligence media platform that has an in-depth coverage of machine learning and deep learning news that can sound both technically, both through technical voices and be understood by a wide audience. The platform has over 2 million views per month, demonstrating its popularity among its audience.
