Kevin Musgrave 19bae5ac0f

feat(evaluation): Add placeholders to swe_gpt4.j2 (#11228) ...

Co-authored-by: Graham Neubig <neubig@gmail.com>

2025-10-13 22:15:05 +08:00

..

agent_bench

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

aider_bench

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

algotune

Add a new benchmark: AlgoTune (#10724)

2025-09-04 18:08:50 +00:00

biocoder

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

bird

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

browsing_delegation

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

commit0

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

discoverybench

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

EDA

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

gaia

Implement model routing support (#9738)

2025-09-08 16:19:34 +07:00

gorilla

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

gpqa

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

humanevalfix

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

lca_ci_build_repair

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

logic_reasoning

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

miniwob

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

mint

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

ml_bench

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

multi_swe_bench

Multi swe gym (#10605)

2025-09-22 15:56:26 -04:00

nocode_bench

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

scienceagentbench

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

swe_bench

feat(evaluation): Add placeholders to swe_gpt4.j2 (#11228)

2025-10-13 22:15:05 +08:00

swe_perf

Add inference generation of SWE-Perf Benchmark (#10246)

2025-09-22 20:35:30 +00:00

terminal_bench

Add README for terminal_bench evaluation harness (#9700)

2025-07-15 09:48:34 -04:00

testgeneval

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

the_agent_company

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

toolqa

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

versicode

chore(lint): Apply comprehensive linting and formatting fixes (#10287)

2025-08-13 21:13:19 +02:00

visual_swe_bench

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

visualwebarena

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00

webarena

Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)

2025-08-22 13:34:02 +00:00