20 Commits

Author SHA1 Message Date
Xingyao Wang
4507a25b85
Evaluation: redirect sessions to repo-local .eval_sessions via helper; apply across entrypoints; add tests (#10540)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-22 13:34:02 +00:00
Engel Nyst
91d3d1d20a Fix: expose aggregated LLM metrics in State for evaluation scripts (#10537)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-21 17:43:09 +02:00
Robert Brennan
205f0234e8
Rename Conversation to ServerConversation and AppConfig to OpenHandsConfig (#8754)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-28 21:48:34 +02:00
Graham Neubig
689d3c9046
Update pre-commit hook versions to most recent versions (#8343)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-08 03:59:13 +00:00
Xingyao Wang
4be33a079b
Update SWE-Bench README.md about RemoteRuntime (#7108) 2025-03-05 23:00:54 +08:00
Engel Nyst
395c1ea9e3
[Refactor] split runtime initialization (create, connect, init) in cli scripts (#7036) 2025-03-03 00:19:25 +01:00
Engel Nyst
4f98bce6df
Add selected_repo to command line (#6949) 2025-02-26 20:42:59 +01:00
Mateusz Kwiatkowski
6562297615
Replace shebang with /usr/bin/env bash for improved portability (#6876)
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
2025-02-24 18:07:28 +00:00
Xingyao Wang
1a7003a705
Add sysbox support to remote runtime for eval; Add memory monitor, stress tests to help debug memory issue (#6684)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Graham Neubig <neubig@gmail.com>
2025-02-18 20:02:28 +00:00
tofarr
bbfdc62139
Fix for issue where retries continue on a closed runtime (#6564)
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2025-02-03 08:44:09 -07:00
Xingyao Wang
0c961bfd8b
refactor(prompt): move runtime/repo info to user message and disable them in eval (#6291) 2025-01-16 17:53:10 +00:00
Xingyao Wang
ec70af9412
refactor: Replace pexpect with libtmux in BashSession (#4881)
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Robert Brennan <accounts@rbren.io>
2025-01-04 05:22:13 +08:00
Xingyao Wang
9dd5463e06
Set default value of use_microagents to False to prevent breaking eval (#5976)
Co-authored-by: openhands <openhands@all-hands.dev>
2025-01-03 05:39:17 +08:00
Robert Brennan
0e4e1b3316
Factor out ActionExecutionClient (#5796) 2024-12-30 15:32:13 +00:00
Engel Nyst
3297e4d5a8
Use litellm's modify params (#5636) 2024-12-17 21:32:49 +01:00
Engel Nyst
b295f5775c
Revert "Fix issue #5609: Use litellm's modify_params with default True" (#5631) 2024-12-16 20:39:57 +00:00
OpenHands
09735c7869
Fix issue #5609: Use litellm's modify_params with default True (#5611)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-12-16 20:18:45 +01:00
Cheng Yang
8f47547b08
docs: fix markdown linting and broken links (#5401) 2024-12-05 01:28:04 +08:00
Xingyao Wang
9908e1b285
[Evaluation]: Log openhands version in eval output folder, instead of agent version (#5394) 2024-12-04 03:33:43 +00:00
OpenHands
678436da30
Fix issue #5222: [Refactor]: Refactor the evaluation directory (#5223)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-11-25 08:35:52 -05:00