* modify api endpoint
* update frontend for backend
* fix fileservice
* rm file
* unskip test
* fix some more tests
* fix another test
* fix another test
* fix api call
* fix refresh for subdirs
* more tests passing
* more tests
* more tests
* another test
* logspam
* lint
* fix import
* logspam
* code review feedback
---------
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Disable Python linting by default, and turn it on for SWE Bench.
It is turned off by default since this behavior is weird and somewhat annoying to end users.
It is turned on for SWE Bench because linting python files gives LLM a chance to fix the indentations.
* Disable Python linting by default (fixes#1789)
* Try to simplify
* Return do nothing comment
* Disable linting for the javascript as well
* Apply suggestions from code review
---------
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
* fix: correct simple i18n key typo
* feat: refactor agent status component and add i18n support
---------
Co-authored-by: Jim Su <jimsu@protonmail.com>
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
* Update Dockerfile
creating the workspace directory after switching user to fix permission errors
https://github.com/OpenDevin/OpenDevin/issues/1560
* assign $workspace_base to user
instead of creating directory later, just fix the permissions so user can write to it
* another sudo
---------
Co-authored-by: Leo <ifuryst@gmail.com>
* Update README.md
* Fix WORKSPACE_MOUNT_PATH_IN_SANDBOX variable in regenerate.sh
* Regenerate prompts without calling real LLM
* Disable pytest warning capture
* Change planner agent prompt by a bit for demo
* Regenerate prompt files following prompt changes
* doc: elaborate on FORCE_USE_LLM
* Add another prompt change to monologue_agent for demo purpose
* Regenerate prompts with FORCE_USE_LLM=true
---------
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* Feat: add stream output to exec_run
* Using command timeout to control the exec_box's timeout.
* add bash -c to source command to compatible for sh.
Signed-off-by: ifuryst <ifuryst@gmail.com>
* Feat: add stream output to SSHBox execute
Signed-off-by: ifuryst <ifuryst@gmail.com>
* fix the test case fail.
Signed-off-by: ifuryst <ifuryst@gmail.com>
* fix the test case import wrong path for method.
Signed-off-by: ifuryst <ifuryst@gmail.com>
---------
Signed-off-by: ifuryst <ifuryst@gmail.com>
* Update msg_stack.py
1、[msg.to_dict() for msg in msgs], msg is not instanse of Message, it not has a func of to_dict(), so msg.to_dict() will accur JSONDecodeError;
2、json.dump(new_data, file), it appends new_data to the end of the file instead of overwriting from the beginning, Hence, it's necessary to first perform file.seek(0) and file.truncate().
* Update opendevin/server/session/msg_stack.py
---------
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
* create test todos
* extend to support uploading directories
* remove dir-upload logic and feature drag-and-drop
---------
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
* add draft dockerfile for build all
* add rsync for build
* add all-in-one docker
* update prepare scripts
* Update swe_env_box.py
* Add swe_entry.sh (buggy now)
* Parse the test command in swe_entry.sh
* Update README for instance eval in sandbox
* revert specialized config
* replace run_as_devin as an init arg
* set container & run_as_root via args
* update swe entry script
* update env
* remove mounting
* allow error after swe_entry
* update swe_env_box
* move file
* update gitignore
* get swe_env_box a working demo
* support faking user response & provide sandox ahead of time;
also return state for controller
* tweak main to support adding controller kwargs
* add module
* initialize plugin for provided sandbox
* add pip cache to plugin & fix jupyter kernel waiting
* better print Observation output
* add run infer scripts
* update readme
* add utility for getting diff patch
* use get_diff_patch in infer
* update readme
* support cost tracking for codeact
* add swe agent edit hack
* disable color in git diff
* fix git diff cmd
* fix state return
* support limit eval
* increase t imeout and export pip cache
* add eval limit config
* return state when hit turn limit
* save log to file; allow agent to give up
* run eval with max 50 turns
* add outputs to gitignore
* save swe_instance & instruction
* add uuid to swebench
* add streamlit dep
* fix save series
* fix the issue where session id might be duplicated
* allow setting temperature for llm (use 0 for eval)
* Get report from agent running log
* support evaluating task success right after inference.
* remove extra log
* comment out prompt for baseline
* add visualizer for eval
* use plaintext for instruction
* reduce timeout for all; only increase timeout for init
* reduce timeout for all; only increase timeout for init
* ignore sid for swe env
* close sandbox in each eval loop
* update visualizer instruction
* increase max chars
* add finish action to history too
* show test result in metrics
* add sidebars for visualizer
* also visualize swe_instance
* cleanup browser when agent controller finish runinng
* do not mount workspace for swe-eval to avoid accidentally overwrite files
* Revert "do not mount workspace for swe-eval to avoid accidentally overwrite files"
This reverts commit 8ef77390543e562e6f0a5a9992418014d8b3010c.
* Revert "Revert "do not mount workspace for swe-eval to avoid accidentally overwrite files""
This reverts commit 016cfbb9f0475f32bacbad5822996b4eaff24a5e.
* run jupyter command via copy to, instead of cp to mount
* only print mixin output when failed
* change ssh box logging
* add visualizer for pass rate
* add instance id to sandbox name
* only remove container we created
* use opendevin logger in main
* support multi-processing infer
* add back metadata, support keyboard interrupt
* remove container with startswith
* make pbar behave correctly
* update instruction w/ multi-processing
* show resolved rate by repo
* rename tmp dir name
* attempt to fix racing for copy to ssh_box
* fix script
* bump swe-bench-all version
* fix ipython with self-contained commands
* add jupyter demo to swe_env_box
* make resolved count two column
* increase height
* do not add glob to url params
* analyze obs length
* print instance id prior to removal handler
* add gold patch in visualizer
* fix interactive git by adding a git --no-pager as alias
* increase max_char to 10k to cover 98% of swe-bench obs cases
* allow parsing note
* prompt v2
* add iteration reminder
* adjust user response
* adjust order
* fix return eval
* fix typo
* add reminder before logging
* remove other resolve rate
* re adjust to new folder structure
* support adding eval note
* fix eval note path
* make sure first log of each instance is printed
* add eval note
* fix the display for visualizer
* tweak visualizer for better git patch reading
* exclude empty patch
* add retry mechanism for swe_env_box start
* fix ssh timeout issue
* add stat field for apply test patch success
* add visualization for fine-grained report
* attempt to support monologue agent by constraining it to single thread
* also log error msg when stopeed
* save error as well
* override WORKSPACE_MOUNT_PATH and WORKSPACE_BASE for monologue to work in mp
* add retry mechanism for sshbox
* remove retry for swe env box
* try to handle loop state stopped
* Add get report scripts
* Add script to convert agent output to swe-bench format
* Merge fine grained report for visualizer
* Update eval readme
* Update README.md
* Add CodeAct gpt4-1106 output and eval logs on swe-bench-lite
* Update the script to get model report
* Update get_model_report.sh
* Update get_agent_report.sh
* Update report merge script
* Add agent output conversion script
* Update swe_lite_env_setup.sh
* Add example swe-bench output files
* Update eval readme
* Remove redundant scripts
* set iteration count down to false by default
* fix: Issue where CodeAct agent was trying to log cost on local llm and throwing Undefined Model execption out of litellm (#1666)
* fix: Issue where CodeAct agent was trying to log cost on local llm and throwing Undefined Model execption out of litellm
* Review Feedback
* Missing None Check
* Review feedback and improved error handling
---------
Co-authored-by: Robert Brennan <accounts@rbren.io>
* fix prepare_swe_util scripts
* update builder images
* update setup script
* remove swe-bench build workflow
* update lock
* remove experiments since they are moved to hf
* remove visualizer (since it is moved to hf repo)
* simply jupyter execution via heredoc
* update ssh_box
* add initial docker readme
* add pkg-config as dependency
* add script for swe_bench all-in-one docker
* add rsync to builder
* rename var
* update commit
* update readme
* update lock
* support specify timeout for long running tasks
* fix path
* separate building of all deps and files
* support returning states at the end of controller
* remove return None
* support specify timeout for long running tasks
* add timeout for all existing sandbox impl
* fix swe_env_box for new codebase
* update llm config in config.py
* support pass sandbox in
* remove force set
* update eval script
* fix issue of overriding final state
* change default eval output to hf demo
* change default eval output to hf demo
* fix config
* only close it when it is NOT external sandbox
* add scripts
* tweak config
* only put in hostory when state has history attr
* fix agent controller on the case of run out interaction budget
* always assume state is always not none
* remove print of final state
* catch all exception when cannot compute completion cost
* Update README.md
* save source into json
* fix path
* update docker path
* return the final state on close
* merge AgentState with State
* fix integration test
* merge AgentState with State
* fix integration test
* add ChangeAgentStateAction to history in attempt to fix integration
* add back set agent state
* update tests
* update tests
* move scripts for setup
* update script and readme for infer
* do not reset logger when n processes == 1
* update eval_infer scripts and readme
* simplify readme
* copy over dir after eval
* copy over dir after eval
* directly return get state
* update lock
* fix output saving of infer
* replace print with logger
* update eval_infer script
* add back the missing .close
* increase timeout
* copy all swe_bench_format file
* attempt to fix output parsing
* log git commit id as metadata
* fix eval script
* update lock
* update unit tests
* fix argparser unit test
* fix lock
* the deps are now lightweight enough to be incude in make build
* add spaces for tests
* add eval outputs to gitignore
* remove git submodule
* readme
* tweak git email
* update upload instruction
* bump codeact version for eval
---------
Co-authored-by: Bowen Li <libowen.ne@gmail.com>
Co-authored-by: huybery <huybery@gmail.com>
Co-authored-by: Bart Shappee <bshappee@gmail.com>
Co-authored-by: Robert Brennan <accounts@rbren.io>
* enable browsing in codeact, and arbitrary browsergym DSL support
* fix
* fix unit test case
* update frontend for the new interactive browsing action
* bump ver
* Fix integration tests
---------
Co-authored-by: OpenDevinBot <bot@opendevin.com>
