github/OpenHands
1
0
Fork 0
mirror of https://github.com/OpenHands/OpenHands.git synced 2025-12-26 05:48:36 +08:00
Code Issues Packages Projects Releases Wiki Activity

Evaluation: Fix bug on python path on run.sh (#98)

Browse Source 
* Move regression tests to evaluation/

* use pythnon instead of docker in the script

* add model para

* change python to python3

* bug fix

* add python path

* add readme
This commit is contained in:
zch-cc 2024-03-22 09:01:48 -07:00 committed by GitHub
parent cfefc47439
commit e5a28cba2f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 16 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
evaluation/regression/README.md
View File

@ -16,3 +16,14 @@ echo "write a hello world script" >> ./agent/regression/cases/$name/task.txt
# Single out your test case using the TEST_CASE environment variable
TEST_CASE=$name ./agent/regression/run.sh
```
To add agent to regreesion test:
```bash
add agent pair to directory_class_pairs variable in run.sh
key is the directory name in /agenthub and value is the class name
```
To run regresion test:
```bash
./run.sh and enter DEBUG, OPENAI_API_KEY and Model name in the prompt.
```

5
evaluation/regression/run.sh
View File

@ -29,6 +29,11 @@ else
MODEL="$model"
fi
echo "Running with model: $MODEL"
# add python path
export PYTHONPATH="$PYTHONPATH:$SCRIPT_DIR/../../"
# hardcode pairs for directory to python class mapping
declare -A directory_class_pairs=(
[langchains_agent]="LangchainsAgent"