Boxuan Li 4443417c75

A few fixes for TAC evaluation harness (#6586)

2025-02-14 21:01:57 -08:00

..

agent_bench

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

aider_bench

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

biocoder

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

bird

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

browsing_delegation

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

commit0_bench

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

discoverybench

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

EDA

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

gaia

Evaluation harness: Add agent config option (#6662)

2025-02-13 15:05:03 -05:00

gorilla

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

gpqa

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

humanevalfix

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

logic_reasoning

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

miniwob

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

mint

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

ml_bench

refactor(prompt): move runtime/repo info to user message and disable them in eval (#6291)

2025-01-16 17:53:10 +00:00

scienceagentbench

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

swe_bench

Better error logging in posthog (#6346)

2025-02-06 20:16:37 +00:00

the_agent_company

A few fixes for TAC evaluation harness (#6586)

2025-02-14 21:01:57 -08:00

toolqa

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

visualwebarena

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00

webarena

Fix for issue where retries continue on a closed runtime (#6564)

2025-02-03 08:44:09 -07:00