The experiment-grounded
AI benchmark for
drug discovery.
Built on drug R&D and backed by an expert network from leading research institutions, we derive problems from real experiments, calibrate them with domain experts and validate them through dry- and wet-lab work to benchmark scientific AI agents.
An evaluation system
built on expert judgment
Our expert network spans drug discovery, chemistry, biology and experimental science. Experts select problems, define the key judgments and calibrate results.
- Tsinghua University清华大学
- Chinese Academy of Sciences中国科学院
- University of Science and Technology of China中国科学技术大学
- Fudan University复旦大学
- Beijing University of Posts and Telecommunications北京邮电大学
- Xidian University西安电子科技大学
and experts from many other leading universities and research institutes
Drug discovery & chemistry
Select problems and author scoring rubrics; assess the soundness of R&D decisions such as SAR and ADMET.
Biology & experimental science
Design experiments and controls, interpret dry-lab conclusions and wet-lab results, and define the scope of each conclusion.
Modeling & evaluation
Members come largely from teams at Tsinghua University, USTC and CAS, and build the Agent Harness, evaluation environments and grading.
Expert profiles are published with each expert's consent.
Grounded in real experiments,
calibrated by experts
Benchmark data is produced by agents and domain experts working together, not generated in bulk by models. Every item must trace back to a specific experiment and pass expert review.
From real experiments
Problems are distilled from published experimental evidence, real R&D projects and our own dry- and wet-lab work, and trace back to the original records.
Expert-calibrated
Domain experts confirm the key judgments, annotate evidence and write the scoring rubric. Items without expert review do not enter the benchmark.
Human-in-the-loop production
Agents gather materials, draft items, run trials and automated checks; experts decide and calibrate at key checkpoints, combining speed with quality.
From experimental evidence
to trusted benchmarks
Experimental data and expert knowledge feed a validation-aware loop. Only results that hold up under experimental and expert validation become benchmarks and feedback for model teams.
Experiment-grounded foundations
- Experimental data and literatureAssay readouts, papers and patents
- Structures, assays and protocolsTargets, molecules and experimental methods
- Expert knowledgeKey judgments and scoring rubrics
Validation-aware closed loop
- DesignHypotheses and molecules
- ComputeDry lab and modeling
- ExperimentWet lab with controls
- AnalyzeReadouts and review
- UpdateData and new problems
Each round's readouts feed the next design.
- Molecules and SAR
- Proteins and antibodies
- Assays and mechanisms
Trusted evaluation and translation
- Versioned benchmarksFrozen, verifiable and reproducible
- Agents for real R&DEvaluation reports that guide model and agent improvement
- Experimental validity
- Expert trust
A closed dry–wet loop
Dry-lab work leads today. Wet-lab submission, controls and result feedback are implemented in software, with lab integration under construction.
- 01
Scoping
ExpertsLab team
Select problems from R&D projects and lab records; define the data scope and validation boundaries.
- 02
Judgments
ExpertsAgent
Agents draft the task; experts confirm objectives, evidence requirements and scoring rubrics.
- 03
Dry lab
AgentPlatform
Analyze raw data with professional software in an isolated environment and run structure prediction.
In progress - 04
Wet lab
Lab teamPlatform
Blinded submissions, with positive and negative controls on every plate.
Under construction - 05
Feedback
ExpertsPlatform
Readouts are reconciled automatically; predicted and experimental scores are recorded separately, and experts define the scope of each result.
Under construction - 06
Benchmark
ExpertsPlatform
Reviewed, calibrated and frozen into versioned benchmarks; results seed new problems.
Agent Harness
and quality assurance
The production process follows the published practices of benchmarks such as HLE, GPQA, FrontierMath, HealthBench and PaperBench, covering authoring standards, difficulty screening, expert review, judge validation, contamination control and statistical reporting.
- Version freezeSplits and item hashes are written to the database and locked once frozen
- Isolated evaluationReference solutions and graders never enter the evaluation sandbox
- Two-gate checkReference solutions must score full marks; empty submissions must score zero
- Deterministic gradingGrading the same answer repeatedly yields the same score
- Experiments prevailPredicted and experimental scores are recorded separately; the experiment decides
- Statistical intervals95% confidence intervals from cluster bootstrap resampling
Roadmap
The benchmark has not been released yet. It will ship in versions that are verifiable and reproducible once frozen, alongside ongoing publications on experiment-validated evaluation, research-analysis agent tasks and the trustworthiness of scientific evaluation.
- In progress
Dry lab
Problems distilled from literature, databases and computational experiments; the Agent Harness and expert workflows are in place.
- Under construction
Wet-lab feedback
Submission, controls and result feedback are implemented in software; once connected, wet-lab readouts become the ground truth.
- Planned
Versioned releases
Release experiment-derived benchmarks and research problems with verifiable manifests and evaluation reports.
公开榜单
只公开已发布版本的 test 汇总 · 题目内容与逐题结果不出平台
Define the problems
worth evaluating.
For model and agent teams, domain experts and lab teams: start from a concrete scientific problem to discuss co-developed tasks, data partnerships and evaluation needs.
Zetta Benchmark · In development
- 01Discuss the domain, problem and evaluation goals
- 02Agree on data, tools and expert collaboration
- 03Scope a pilot or a follow-up evaluation
The benchmark is under development. Collaboration starts from the problem and data scope; the environment, acceptance rules and disclosure of results are agreed before any formal evaluation.