Zetta BenchmarkExperiment-grounded evaluation for scientific AI

The experiment-grounded
AI benchmark for
drug discovery.

Built on drug R&D and backed by an expert network from leading research institutions, we derive problems from real experiments, calibrate them with domain experts and validate them through dry- and wet-lab work to benchmark scientific AI agents.

An evaluation system
built on expert judgment

Our expert network spans drug discovery, chemistry, biology and experimental science. Experts select problems, define the key judgments and calibrate results.

  • Tsinghua University清华大学
  • Chinese Academy of Sciences中国科学院
  • University of Science and Technology of China中国科学技术大学
  • Fudan University复旦大学
  • Beijing University of Posts and Telecommunications北京邮电大学
  • Xidian University西安电子科技大学

and experts from many other leading universities and research institutes

Drug discovery & chemistry

Select problems and author scoring rubrics; assess the soundness of R&D decisions such as SAR and ADMET.

Biology & experimental science

Design experiments and controls, interpret dry-lab conclusions and wet-lab results, and define the scope of each conclusion.

Modeling & evaluation

Members come largely from teams at Tsinghua University, USTC and CAS, and build the Agent Harness, evaluation environments and grading.

Expert profiles are published with each expert's consent.

Grounded in real experiments,
calibrated by experts

Benchmark data is produced by agents and domain experts working together, not generated in bulk by models. Every item must trace back to a specific experiment and pass expert review.

From real experiments

Problems are distilled from published experimental evidence, real R&D projects and our own dry- and wet-lab work, and trace back to the original records.

Expert-calibrated

Domain experts confirm the key judgments, annotate evidence and write the scoring rubric. Items without expert review do not enter the benchmark.

Human-in-the-loop production

Agents gather materials, draft items, run trials and automated checks; experts decide and calibrate at key checkpoints, combining speed with quality.

From experimental evidence
to trusted benchmarks

Experimental data and expert knowledge feed a validation-aware loop. Only results that hold up under experimental and expert validation become benchmarks and feedback for model teams.

Experiment-grounded foundations

  • Experimental data and literatureAssay readouts, papers and patents
  • Structures, assays and protocolsTargets, molecules and experimental methods
  • Expert knowledgeKey judgments and scoring rubrics

Validation-aware closed loop

  1. DesignHypotheses and molecules
  2. ComputeDry lab and modeling
  3. ExperimentWet lab with controls
  4. AnalyzeReadouts and review
  5. UpdateData and new problems

Each round's readouts feed the next design.

  • Molecules and SAR
  • Proteins and antibodies
  • Assays and mechanisms

Trusted evaluation and translation

  • Versioned benchmarksFrozen, verifiable and reproducible
  • Agents for real R&DEvaluation reports that guide model and agent improvement
Discovery to benchmark
  • Experimental validity
  • Expert trust

A closed dry–wet loop

Dry-lab work leads today. Wet-lab submission, controls and result feedback are implemented in software, with lab integration under construction.

  1. 01

    Scoping

    ExpertsLab team

    Select problems from R&D projects and lab records; define the data scope and validation boundaries.

  2. 02

    Judgments

    ExpertsAgent

    Agents draft the task; experts confirm objectives, evidence requirements and scoring rubrics.

  3. 03

    Dry lab

    AgentPlatform

    Analyze raw data with professional software in an isolated environment and run structure prediction.

    In progress
  4. 04

    Wet lab

    Lab teamPlatform

    Blinded submissions, with positive and negative controls on every plate.

    Under construction
  5. 05

    Feedback

    ExpertsPlatform

    Readouts are reconciled automatically; predicted and experimental scores are recorded separately, and experts define the scope of each result.

    Under construction
  6. 06

    Benchmark

    ExpertsPlatform

    Reviewed, calibrated and frozen into versioned benchmarks; results seed new problems.

Agent Harness
and quality assurance

The production process follows the published practices of benchmarks such as HLE, GPQA, FrontierMath, HealthBench and PaperBench, covering authoring standards, difficulty screening, expert review, judge validation, contamination control and statistical reporting.

  • Version freezeSplits and item hashes are written to the database and locked once frozen
  • Isolated evaluationReference solutions and graders never enter the evaluation sandbox
  • Two-gate checkReference solutions must score full marks; empty submissions must score zero
  • Deterministic gradingGrading the same answer repeatedly yields the same score
  • Experiments prevailPredicted and experimental scores are recorded separately; the experiment decides
  • Statistical intervals95% confidence intervals from cluster bootstrap resampling

Roadmap

The benchmark has not been released yet. It will ship in versions that are verifiable and reproducible once frozen, alongside ongoing publications on experiment-validated evaluation, research-analysis agent tasks and the trustworthiness of scientific evaluation.

  1. In progress

    Dry lab

    Problems distilled from literature, databases and computational experiments; the Agent Harness and expert workflows are in place.

  2. Under construction

    Wet-lab feedback

    Submission, controls and result feedback are implemented in software; once connected, wet-lab readouts become the ground truth.

  3. Planned

    Versioned releases

    Release experiment-derived benchmarks and research problems with verifiable manifests and evaluation reports.