Difference between revisions of "Science Agents"
KevinYager (talk | contribs) (→Genuine Discoveries) |
KevinYager (talk | contribs) (→Science Benchmarks) |
||
| (25 intermediate revisions by the same user not shown) | |||
| Line 60: | Line 60: | ||
* 2025-11: [https://pubs.aip.org/aip/jcp/article/163/18/184110/3372267/A-foundation-model-for-atomistic-materials A foundation model for atomistic materials chemistry] | * 2025-11: [https://pubs.aip.org/aip/jcp/article/163/18/184110/3372267/A-foundation-model-for-atomistic-materials A foundation model for atomistic materials chemistry] | ||
* 2025-11: [https://arxiv.org/abs/2511.15684 Walrus: A Cross-Domain Foundation Model for Continuum Dynamics] | * 2025-11: [https://arxiv.org/abs/2511.15684 Walrus: A Cross-Domain Foundation Model for Continuum Dynamics] | ||
| + | * 2026-01: [https://www.science.org/doi/10.1126/science.ads9530 Deep contrastive learning enables genome-wide virtual screening] | ||
===Regression (Data Fitting)=== | ===Regression (Data Fitting)=== | ||
| Line 129: | Line 130: | ||
* 2025-09: [https://www.biorxiv.org/content/10.1101/2025.09.12.675911v1 Generative design of novel bacteriophages with genome language models] | * 2025-09: [https://www.biorxiv.org/content/10.1101/2025.09.12.675911v1 Generative design of novel bacteriophages with genome language models] | ||
* 2025-10: [https://www.science.org/doi/10.1126/science.adu8578 Strengthening nucleic acid biosecurity screening against generative protein design tools] | * 2025-10: [https://www.science.org/doi/10.1126/science.adu8578 Strengthening nucleic acid biosecurity screening against generative protein design tools] | ||
| + | * 2026-01: [https://www.nature.com/articles/s41586-025-10014-0 Advancing regulatory variant effect prediction with AlphaGenome] | ||
===Medicine=== | ===Medicine=== | ||
| Line 148: | Line 150: | ||
* 2025-02: [https://www.biorxiv.org/content/10.1101/2025.02.06.636901v1 From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models] | * 2025-02: [https://www.biorxiv.org/content/10.1101/2025.02.06.636901v1 From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models] | ||
* 2025-02: [https://www.goodfire.ai/blog/interpreting-evo-2 Interpreting Evo 2: Arc Institute's Next-Generation Genomic Foundation Model] | * 2025-02: [https://www.goodfire.ai/blog/interpreting-evo-2 Interpreting Evo 2: Arc Institute's Next-Generation Genomic Foundation Model] | ||
| + | * 2026-01: [https://www.goodfire.ai/research/interpretability-for-alzheimers-detection# Using Interpretability to Identify a Novel Class of Alzheimer's Biomarkers] | ||
===Uncertainty=== | ===Uncertainty=== | ||
| Line 165: | Line 168: | ||
** 2024-07: [https://arxiv.org/abs/2407.09413 SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers] | ** 2024-07: [https://arxiv.org/abs/2407.09413 SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers] | ||
** 2024-10: [https://neurips.cc/virtual/2024/98540 FEABench: Evaluating Language Models on Real World Physics Reasoning Ability] | ** 2024-10: [https://neurips.cc/virtual/2024/98540 FEABench: Evaluating Language Models on Real World Physics Reasoning Ability] | ||
| + | * 2026-02: [https://edisonscientific.com/ Edison]: [https://lab-bench.ai/ LABBench 2] | ||
=Science Agents= | =Science Agents= | ||
| Line 172: | Line 176: | ||
* 2025-07: [https://arxiv.org/abs/2507.01903 AI4Research: A Survey of Artificial Intelligence for Scientific Research] | * 2025-07: [https://arxiv.org/abs/2507.01903 AI4Research: A Survey of Artificial Intelligence for Scientific Research] | ||
* 2025-08: [https://arxiv.org/abs/2508.14111 From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery] | * 2025-08: [https://arxiv.org/abs/2508.14111 From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery] | ||
| + | |||
| + | ==Challenges== | ||
| + | * 2026-01: [https://arxiv.org/abs/2601.03315 Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts] | ||
==Specific== | ==Specific== | ||
| Line 188: | Line 195: | ||
* 2025-11: [https://arxiv.org/abs/2511.02824 Kosmos: An AI Scientist for Autonomous Discovery] | * 2025-11: [https://arxiv.org/abs/2511.02824 Kosmos: An AI Scientist for Autonomous Discovery] | ||
* 2025-11: [https://arxiv.org/abs/2511.08151 SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoning] | * 2025-11: [https://arxiv.org/abs/2511.08151 SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoning] | ||
| + | * 2026-02: [https://arxiv.org/abs/2601.23265 PaperBanana: Automating Academic Illustration for AI Scientists] | ||
==Science Multi-Agent Setups== | ==Science Multi-Agent Setups== | ||
| Line 200: | Line 208: | ||
** 2025-07: [https://arxiv.org/abs/2507.00964 Benchmarking the Discovery Engine] ([https://www.leap-labs.com/blog/how-we-replicated-five-peer-reviewed-papers-in-five-hours blog]) | ** 2025-07: [https://arxiv.org/abs/2507.00964 Benchmarking the Discovery Engine] ([https://www.leap-labs.com/blog/how-we-replicated-five-peer-reviewed-papers-in-five-hours blog]) | ||
* 2025-07: [https://www.preprints.org/manuscript/202507.1951/v1 Autonomous Scientific Discovery Through Hierarchical AI Scientist Systems] | * 2025-07: [https://www.preprints.org/manuscript/202507.1951/v1 Autonomous Scientific Discovery Through Hierarchical AI Scientist Systems] | ||
| + | * 2025-12: [https://arxiv.org/abs/2512.16969 Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows] | ||
| + | * 2026-01: [https://www.nature.com/articles/s43588-025-00906-6 SciSciGPT: advancing human–AI collaboration in the science of science] | ||
===Inorganic Materials Discovery=== | ===Inorganic Materials Discovery=== | ||
| Line 222: | Line 232: | ||
===Bio=== | ===Bio=== | ||
* 2025-07: [https://arxiv.org/abs/2507.01485 BioMARS: A Multi-Agent Robotic System for Autonomous Biological Experiments] | * 2025-07: [https://arxiv.org/abs/2507.01485 BioMARS: A Multi-Agent Robotic System for Autonomous Biological Experiments] | ||
| + | |||
| + | ===Physics=== | ||
| + | * 2025-12: [https://arxiv.org/abs/2512.19799 PhysMaster: Building an Autonomous AI Physicist for Theoretical and Computational Physics Research] | ||
==LLMs Optimized for Science== | ==LLMs Optimized for Science== | ||
| Line 234: | Line 247: | ||
** 2025-05: Retraction: [https://economics.mit.edu/news/assuring-accurate-research-record Assuring an accurate research record] | ** 2025-05: Retraction: [https://economics.mit.edu/news/assuring-accurate-research-record Assuring an accurate research record] | ||
* 2025-02: [https://arxiv.org/abs/2502.05151 Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation] | * 2025-02: [https://arxiv.org/abs/2502.05151 Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation] | ||
| + | * 2026-02: [https://arxiv.org/abs/2602.03837 Accelerating Scientific Research with Gemini: Case Studies and Common Techniques] | ||
=Related Tools= | =Related Tools= | ||
| Line 257: | Line 271: | ||
=Genuine Discoveries= | =Genuine Discoveries= | ||
* 2025-11: [https://cdn.openai.com/pdf/4a25f921-e4e0-479a-9b38-5367b47e8fd0/early-science-acceleration-experiments-with-gpt-5.pdf Early science acceleration experiments with GPT-5] | * 2025-11: [https://cdn.openai.com/pdf/4a25f921-e4e0-479a-9b38-5367b47e8fd0/early-science-acceleration-experiments-with-gpt-5.pdf Early science acceleration experiments with GPT-5] | ||
| − | + | * 2025-12: [https://andymasley.substack.com/p/ai-can-obviously-create-new-knowledge AI can obviously create new knowledge - But maybe not new concepts] | |
* '''Math:''' | * '''Math:''' | ||
** 2023-07: [https://www.nature.com/articles/s41586-023-06004-9?utm_source=chatgpt.com Faster sorting algorithms discovered using deep reinforcement learning] | ** 2023-07: [https://www.nature.com/articles/s41586-023-06004-9?utm_source=chatgpt.com Faster sorting algorithms discovered using deep reinforcement learning] | ||
| + | ** 2025-06: [https://arxiv.org/abs/2506.13131 AlphaEvolve: A coding agent for scientific and algorithmic discovery] | ||
** 2025-11: [https://arxiv.org/abs/2511.02864 Mathematical exploration and discovery at scale] | ** 2025-11: [https://arxiv.org/abs/2511.02864 Mathematical exploration and discovery at scale] | ||
** 2025-11: [https://www.nature.com/articles/s41586-025-09833-y Olympiad-level formal mathematical reasoning with reinforcement learning] | ** 2025-11: [https://www.nature.com/articles/s41586-025-09833-y Olympiad-level formal mathematical reasoning with reinforcement learning] | ||
| + | ** 2025-12: [https://arxiv.org/abs/2512.14575 Extremal descendant integrals on moduli spaces of curves: An inequality discovered and proved in collaboration with AI] | ||
| + | ** [https://github.com/teorth/erdosproblems/wiki/AI-contributions-to-Erd%C5%91s-problems AI Solving Erdős Problems]: | ||
| + | *** 2026-01: [https://www.erdosproblems.com/728 Erdős Problem #728] and [https://www.erdosproblems.com/729 #729] solved by Aristotle using ChatGPT 5.2 Pro | ||
| + | *** 2026-01: [https://www.erdosproblems.com/forum/thread/397 Erdős Problem #397] [https://x.com/neelsomani/status/2010215162146607128?s=20 solved] by [https://neelsomani.com/ Neel Somani] using ChatGPT 5.2 Pro | ||
| + | *** 2026-01: [https://www.erdosproblems.com/205 Erdős Problem #205] solved by Aristotle using ChatGPT 5.2 Pro | ||
| + | *** 2026-01: [https://www.erdosproblems.com/forum/thread/281 Erdős Problem #281] [https://x.com/neelsomani/status/2012695714187325745?s=20 solved] by [https://neelsomani.com/ Neel Somani] using ChatGPT 5.2 Pro | ||
| + | *** 2026-01: Google DeepMind: [https://arxiv.org/abs/2601.21442 Irrationality of rapidly converging series: a problem of Erdős and Graham] | ||
| + | **** [https://www.erdosproblems.com/1051 Erdős Problem #1051] [https://x.com/slow_developer/status/2018321002623901885?s=20 solved] by Google DeepMind Aletheia agent | ||
| + | *** 2026-01: Google DeepMind: [https://arxiv.org/abs/2601.22401 Semi-Autonomous Mathematics Discovery with Gemini: A Case Study on the Erdős Problems] | ||
| + | **** Attempted 700 problems, solved 13 open Erdős problems: 5 novel autonomous solutions, 8 through existing literature. | ||
| + | ** 2026-01: [https://arxiv.org/abs/2601.07222 The motivic class of the space of genus 0 maps to the flag variety] | ||
* '''Physics assistance:''' | * '''Physics assistance:''' | ||
| − | ** 2025-03: [https://arxiv.org/abs/2503.23758 Exact solution of the frustrated Potts model with next-nearest-neighbor interactions in one dimension via AI bootstrapping] | + | ** 2025-03: [https://arxiv.org/abs/2503.23758 Exact solution of the frustrated Potts model with next-nearest-neighbor interactions in one dimension via AI bootstrapping] ([https://www.bnl.gov/staff/wyin Weiguo Yin]) |
** 2025-12: [https://www.sciencedirect.com/science/article/pii/S0370269325008111 Relativistic covariance and nonlinear quantum mechanics: Tomonaga-Schwinger analysis] | ** 2025-12: [https://www.sciencedirect.com/science/article/pii/S0370269325008111 Relativistic covariance and nonlinear quantum mechanics: Tomonaga-Schwinger analysis] | ||
*** [https://x.com/hsu_steve/status/1996034522308026435?s=20 Steve Hsu], [https://drive.google.com/file/d/16sxJuwsHoi-fvTFbri9Bu8B9bqA6lr1H/view Theoretical Physics with Generative AI] | *** [https://x.com/hsu_steve/status/1996034522308026435?s=20 Steve Hsu], [https://drive.google.com/file/d/16sxJuwsHoi-fvTFbri9Bu8B9bqA6lr1H/view Theoretical Physics with Generative AI] | ||
* '''Literature exploration:''' | * '''Literature exploration:''' | ||
| − | ** 2025-11: [https://arxiv.org/abs/2511.02824 Kosmos: An AI Scientist for Autonomous Discovery] | + | ** 2025-11: [https://arxiv.org/abs/2511.02824 Kosmos: An AI Scientist for Autonomous Discovery] ([https://edisonscientific.com/ Edison]) |
*** [https://platform.edisonscientific.com/kosmos/c4bdef64-5e9b-43b9-a365-592dd1ed7587 Nucleotide metabolism in hypothermia] | *** [https://platform.edisonscientific.com/kosmos/c4bdef64-5e9b-43b9-a365-592dd1ed7587 Nucleotide metabolism in hypothermia] | ||
*** [https://platform.edisonscientific.com/kosmos/1fdbf827-be65-4d97-9b66-bf0da600091a Determinant of perovskite solar-cell failure] | *** [https://platform.edisonscientific.com/kosmos/1fdbf827-be65-4d97-9b66-bf0da600091a Determinant of perovskite solar-cell failure] | ||
| Line 279: | Line 305: | ||
** 2025-11: [https://www.nature.com/articles/s41586-025-09721-5 Atomically accurate de novo design of antibodies with RFdiffusion] | ** 2025-11: [https://www.nature.com/articles/s41586-025-09721-5 Atomically accurate de novo design of antibodies with RFdiffusion] | ||
** 2025-11: [https://deepmind.google/blog/alphafold-five-years-of-impact/ AlphaFold: Five years of impact] | ** 2025-11: [https://deepmind.google/blog/alphafold-five-years-of-impact/ AlphaFold: Five years of impact] | ||
| + | ** 2026-01: [https://www.goodfire.ai/research/interpretability-for-alzheimers-detection# Using Interpretability to Identify a Novel Class of Alzheimer's Biomarkers] | ||
* '''Material Discovery:''' | * '''Material Discovery:''' | ||
| − | ** 2023-11: | + | ** 2023-11: [https://doi.org/10.1038/s41586-023-06735-9 Scaling deep learning for materials discovery] |
=See Also= | =See Also= | ||
* [[AI agents]] | * [[AI agents]] | ||
* [https://nanobot.chat/ Nanobot.chat]: Intelligent AI for the labnetwork @ mtl.mit.edu forum | * [https://nanobot.chat/ Nanobot.chat]: Intelligent AI for the labnetwork @ mtl.mit.edu forum | ||
Latest revision as of 10:24, 6 February 2026
Contents
- 1 AI Use-cases for Science
- 2 Science Benchmarks
- 3 Science Agents
- 4 AI Science Systems
- 5 Impact of AI in Science
- 6 Related Tools
- 7 Science Datasets
- 8 Genuine Discoveries
- 9 See Also
AI Use-cases for Science
Literature
- alphaXiv | Explore: Understand arXiv papers
LLM extract data from papers
AI finding links in literature
- 2019-07: Unsupervised word embeddings capture latent knowledge from materials science literature
- 2024-11: Large language models surpass human experts in predicting neuroscience results
(Pre) Generate Articles
- 2022-12: Re3: Generating Longer Stories With Recursive Reprompting and Revision
- 2023-03: English essays: Artificial intelligence (AI) technology in OpenAI ChatGPT application: A review of ChatGPT in writing English essay
- 2023-01: Journalism: Collaborating With ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education
- 2023-07: Science writing: Artificial intelligence in scientific writing: a friend or a foe?
- 2024-02: Wikipedia style: Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
- 2024-02: LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs (code)
- 2024-08: Scientific papers: The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- 2024-09: PaperQA2: Language Models Achieve Superhuman Synthesis of Scientific Knowledge (𝕏 post, code)
- 2025-03: Reasoning to Learn from Latent Thoughts
- 2025-03: WikiAutoGen: Towards Multi-Modal Wikipedia-Style Article Generation
- 2025-04: Sleep-time Compute: Beyond Inference Scaling at Test-time
Explanation
- 2025-02: TheoremExplainAgent: Towards Multimodal Explanations for LLM Theorem Understanding (preprint)
- 2025-04: Do Two AI Scientists Agree?
Autonomous Ideation
- 2024-04: ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
- 2024-09: Mining Causality: AI-Assisted Search for Instrumental Variables
- 2024-12: Thinking Fast and Laterally: Multi-Agentic Approach for Reasoning about Uncertain Emerging Events
- 2024-12: LLMs can realize combinatorial creativity: generating creative ideas via LLMs for scientific research
- 2024-12: LiveIdeaBench: Evaluating LLMs' Scientific Creativity and Idea Generation with Minimal Context
- 2025-01: Hypothesis Generation for Materials Discovery and Design Using Goal-Driven and Constraint-Guided LLM Agents
- 2025-02: Agentic Deep Graph Reasoning Yields Self-Organizing Knowledge Networks
- 2025-06: Predicting Empirical AI Research Outcomes with Language Models
- 2025-06: The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
Adapting LLMs to Science
- 2023-06: Domain-specific chatbots for science using embeddings
- 2024-10: Personalization of Large Language Models: A Survey
- 2024-11: Adapting While Learning: Grounding LLMs for Scientific Problems with Intelligent Tool Usage Adaptation
AI/LLM Control of Scientific Instruments/Facilities
- 2023-12: Opportunities for retrieval and tool augmented large language models in scientific facilities
- 2023-12: Virtual Scientific Companion for Synchrotron Beamlines: A Prototype
- 2023-12: Autonomous chemical research with large language models
- 2024-01: Synergizing Human Expertise and AI Efficiency with Language Model for Microscopy Operation and Automated Experiment Design
- 2024-06: From Text to Test: AI-Generated Control Software for Materials Science Instruments
- 2024-12: VISION: A Modular AI Assistant for Natural Human-Instrument Interaction at Scientific User Facilities
- 2025-01: Large language models for human-machine collaborative particle accelerator tuning through natural language
- 2025-04: Operating Robotic Laboratories with Large Language Models and Teachable Agents
AI/ML Methods tailored to Science
Science Foundation Models
- 2025-08: Intern-S1: A Scientific Multimodal Foundation Model
- 2025-11: A foundation model for atomistic materials chemistry
- 2025-11: Walrus: A Cross-Domain Foundation Model for Continuum Dynamics
- 2026-01: Deep contrastive learning enables genome-wide virtual screening
Regression (Data Fitting)
- 2024-06: Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data: training on (x,y) pairs enables inferring underlying function (define it in code, invert it, compose it)
- 2024-12: OmniPred: Language Models as Universal Regressors
Tabular Classification/Regression
Symbolic Regression
Literature Discovery
- FutureHouse
- Lumina
- Automated-AI-Web-Researcher-Ollama
- 2025-01: Search-o1: Agentic Search-Enhanced Large Reasoning Models (project, code)
Commercial
- Sakana AI
- Cusp AI: Materials/AI
- Lila AI: Life sciences
- Radical AI: Material simulation/design
- Autoscience (Carl)
- Periodic Labs
- Edison Scientific (drug discovery, spinoff from FutureHouse)
Bio
AI/ML Methods in Science
Imaging
- 2025-05: Behind the Noise: Conformal Quantile Regression Reveals Emergent Representations (blog: Behind the Noise)
Materials
- 2024-12: Crystal structure generation with autoregressive large language modeling
- 2025-03: All-atom Diffusion Transformers: Unified generative modelling of molecules and materials
- 2022-11: Training-Free Active Learning Framework in Materials Science with Large Language Models
Chemistry
- 2025-01: Large language models for reticular chemistry
- 2025-02: Image-based generation for molecule design with SketchMol
- 2025-02: Large language models for scientific discovery in molecular property prediction
- 2025-03: Vant AI Neo-1: atomistic foundation model (small molecules, proteins, etc.)
- 2025-04: Compositional Flows for 3D Molecule and Synthesis Pathway Co-design
- 2025-07: General purpose models for the chemical sciences
- 2025-11: ChemTorch: A Deep Learning Framework for Benchmarking and Developing Chemical Reaction Property Prediction Models
Biology
- 2018: AlphaFold
- 2021-07: AlphaFold 2
- 2024-05: AlphaFold 3
- 2023-03: Evolutionary-scale prediction of atomic-level protein structure with a language model (ESMFold)
- 2023-11: Illuminating protein space with a programmable generative model
- 2024-11: Sequence modeling and design from molecular to genome scale with Evo (Evo)
- 2025-01: Targeting protein–ligand neosurfaces with a generalizable deep learning tool (Chroma)
- 2025-01: Simulating 500 million years of evolution with a language model (ESM 3 model)
- 2025-02: Genome modeling and design across all domains of life with Evo 2
- 2025-02: Exploring the structural changes driving protein function with BioEmu-1
- 2025-02: Protein Large Language Models: A Comprehensive Survey
- 2025-03: Vant AI Neo-1: atomistic foundation model (small molecules, proteins, etc.)
- 2025-03: Lyra: An Efficient and Expressive Subquadratic Architecture for Modeling Biological Sequences
- 2025-08: RosettaFold 3: Accelerating Biomolecular Modeling with AtomWorks and RF3
- 2025-09: Generative design of novel bacteriophages with genome language models
- 2025-10: Strengthening nucleic acid biosecurity screening against generative protein design tools
- 2026-01: Advancing regulatory variant effect prediction with AlphaGenome
Medicine
See: AI_Agents#Medicine
Successes
AI/ML Methods co-opted for Science
Mechanistic Interpretability
Train large model on science data. Then apply mechanistic interpretability (e.g. sparse autoencoders, SAE) to the feature/activation space.
- Mechanistic interpretability for protein language models (visualizer, code, SAE)
- Markov Bio: Through a Glass Darkly: Mechanistic Interpretability as the Bridge to End-to-End Biology (quick description, background info on recent bio progress)
- 2023-01: Tracr: Compiled Transformers as a Laboratory for Interpretability (code)
- 2024-10: An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation
- 2024-12: Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models
- 2024-12: InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoders
- 2025-01: Insights on Galaxy Evolution from Interpretable Sparse Feature Networks
- 2025-02: From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models
- 2025-02: Interpreting Evo 2: Arc Institute's Next-Generation Genomic Foundation Model
- 2026-01: Using Interpretability to Identify a Novel Class of Alzheimer's Biomarkers
Uncertainty
- 2024-10: entropix: Entropy Based Sampling and Parallel CoT Decoding
- 2024-10: Taming Overconfidence in LLMs: Reward Calibration in RLHF
Science Benchmarks
- 2024-07: SciCode: A Research Coding Benchmark Curated by Scientists (project)
- 2024-11: AidanBench: Evaluating Novel Idea Generation on Open-Ended Questions (code)
- 2024-12: LiveIdeaBench: Evaluating LLMs' Scientific Creativity and Idea Generation with Minimal Context
- 2025-01: Humanity's Last Exam
- ScienceAgentBench
- 2025-02: EAIRA: Establishing a Methodology for Evaluating AI Models as Scientific Research Assistants
- 2025-03: BixBench: Novel hypotheses (accept/reject)
- 2025-04: Google: Evaluating progress of LLMs on scientific problem-solving
- 2026-02: Edison: LABBench 2
Science Agents
Reviews
- 2024-10: Empowering biomedical discovery with AI agents
- 2025-01: A review of large language models and autonomous agents in chemistry (github)
- 2025-07: AI4Research: A Survey of Artificial Intelligence for Scientific Research
- 2025-08: From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery
Challenges
Specific
- 2024-01-13: ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterization (video)
- 2024-06-19: LLMatDesign: Autonomous Materials Discovery with Large Language Models
- 2024-08-12: Sakana AI: AI Scientist; The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery (code)
- 2024-09-09: SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning (code)
- 2024-09-11: PaperQA2: Language Models Achieve Superhuman Synthesis of Scientific Knowledge (𝕏 post, code)
- 2024-10-17: Rapid and Automated Alloy Design with Graph Neural Network-Powered LLM-Driven Multi-Agent Systems
- 2024-10-28: Large Language Model-Guided Prediction Toward Quantum Materials Synthesis
- 2024-12-06: The Virtual Lab: AI Agents Design New SARS-CoV-2 Nanobodies with Experimental Validation (writeup: Virtual lab powered by ‘AI scientists’ super-charges biomedical research: Could human–AI collaborations be the future of interdisciplinary studies?)
- 2024-12-30: Aviary: training language agents on challenging scientific tasks
- See also: AI Agents > Deep Research
- 2025-04-08: Sakana: The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search (code)
- 2025-07: DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulation
- 2025-11: Kosmos: An AI Scientist for Autonomous Discovery
- 2025-11: SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoning
- 2026-02: PaperBanana: Automating Academic Illustration for AI Scientists
Science Multi-Agent Setups
- 2025-01: Agent Laboratory: Using LLM Agents as Research Assistants
- 2025-04: Coordinated AI agents for advancing healthcare (pdf)
AI Science Systems
- 2025-01: Dolphin: Closed-loop Open-ended Auto-research through Thinking, Practice, and Feedback
- 2025-01: Hypothesis Generation for Materials Discovery and Design Using Goal-Driven and Constraint-Guided LLM Agents
- 2025-02: Towards an AI co-scientist (Google blog post: Accelerating scientific breakthroughs with an AI co-scientist)
- 2025-06: The Discovery Engine
- 2025-07: Benchmarking the Discovery Engine (blog)
- 2025-07: Autonomous Scientific Discovery Through Hierarchical AI Scientist Systems
- 2025-12: Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
- 2026-01: SciSciGPT: advancing human–AI collaboration in the science of science
Inorganic Materials Discovery
- 2023-11: Scaling deep learning for materials discovery
- 2023-11: An autonomous laboratory for the accelerated synthesis of novel materials
- 2024-09: HoneyComb: A Flexible LLM-Based Agent System for Materials Science
- 2024-10: Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models (code, datasets, checkpoints, blogpost)
- 2025-01: A generative model for inorganic materials design
- 2025-04: System of Agentic AI for the Discovery of Metal-Organic Frameworks
- 2025-05: The Open Molecules 2025 (OMol25) Dataset, Evaluations, and Models
Materials Characterization
Chemistry
- 2023-12: Autonomous chemical research with large language models (Coscientist)
- 2024-09: PNNL ChemAIst V0.2
- 2024-11: An automatic end-to-end chemical synthesis development platform powered by large language models
- 2025-06: Training a Scientific Reasoning Model for Chemistry
- 2025-06: ChemGraph: An Agentic Framework for Computational Chemistry Workflows (code)
Bio
Physics
- 2025-12: PhysMaster: Building an Autonomous AI Physicist for Theoretical and Computational Physics Research
LLMs Optimized for Science
- 2022-11: Galactica: A Large Language Model for Science
- 2024-12: Crystal structure generation with autoregressive large language modeling
- 2025-02: MatterChat: A Multi-Modal LLM for Material Science
- 2025-03: OmniScience: A Domain-Specialized LLM for Scientific Reasoning and Discovery
- 2025-03: Google TxGemma (2B, 9B, 27B): drug development
Impact of AI in Science
- 2024-11:
Artificial Intelligence, Scientific Discovery, and Product Innovation- 2025-05: Retraction: Assuring an accurate research record
- 2025-02: Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation
- 2026-02: Accelerating Scientific Research with Gemini: Case Studies and Common Techniques
Related Tools
Literature Search
Data Visualization
- 2024-10: Microsoft Data Formulator: Create Rich Visualization with AI iteratively (video, code)
- Julius AI: Analyze your data with computational AI
Generative
- 2025-03: StarVector 1B, 8B: text or image to SVG
Chemistry
Science Datasets
- Google Dataset Search
- Awesome Materials & Chemistry Datasets
- NIST Jarvis (simulations)
Genuine Discoveries
- 2025-11: Early science acceleration experiments with GPT-5
- 2025-12: AI can obviously create new knowledge - But maybe not new concepts
- Math:
- 2023-07: Faster sorting algorithms discovered using deep reinforcement learning
- 2025-06: AlphaEvolve: A coding agent for scientific and algorithmic discovery
- 2025-11: Mathematical exploration and discovery at scale
- 2025-11: Olympiad-level formal mathematical reasoning with reinforcement learning
- 2025-12: Extremal descendant integrals on moduli spaces of curves: An inequality discovered and proved in collaboration with AI
- AI Solving Erdős Problems:
- 2026-01: Erdős Problem #728 and #729 solved by Aristotle using ChatGPT 5.2 Pro
- 2026-01: Erdős Problem #397 solved by Neel Somani using ChatGPT 5.2 Pro
- 2026-01: Erdős Problem #205 solved by Aristotle using ChatGPT 5.2 Pro
- 2026-01: Erdős Problem #281 solved by Neel Somani using ChatGPT 5.2 Pro
- 2026-01: Google DeepMind: Irrationality of rapidly converging series: a problem of Erdős and Graham
- Erdős Problem #1051 solved by Google DeepMind Aletheia agent
- 2026-01: Google DeepMind: Semi-Autonomous Mathematics Discovery with Gemini: A Case Study on the Erdős Problems
- Attempted 700 problems, solved 13 open Erdős problems: 5 novel autonomous solutions, 8 through existing literature.
- 2026-01: The motivic class of the space of genus 0 maps to the flag variety
- Physics assistance:
- Literature exploration:
- Bio design:
- Material Discovery:
See Also
- AI agents
- Nanobot.chat: Intelligent AI for the labnetwork @ mtl.mit.edu forum
