Commit Graph

39 Commits

Author SHA1 Message Date
Shimada666
5972498e23 No longer chown -R the miniforge3 folder (#2566)
* No longer chown -R the miniforge3 folder

* change miniforge3 group permission
2024-06-21 17:33:00 +00:00
Shimada666
26fc3c886a Make plugins sandbox-agnostic (#2101)
* tmp

* tmp

* merge main

* feat: auto build image cache

* remove plugins

* use config file

* update mamba setup shell

* support agnostic sandbox image autobuild

* remove config

* Update .gitignore

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>

* update setup.sh

* readd sudo

* add sudo in dockerfile

* remove export

* move od-runtime dependencies to sandbox dockerfile

* factor out re-build logic into a separate util file

* tweak existing plugin to use OD specific sandbox

* update testcase

* attempt to fix unit test using image built in ghcr

* use cache tag

* try to fix unit tests

* add unittest

* add unittest

* add some unittests

* revert gh workflow changes

* feat: optimize sandbox image naming rule

* add pull latest image hint

* add opendevin python hint and use mamba to install gcc

* update docker image naming rule and fix mamba issue

* Update opendevin/runtime/docker/ssh_box.py

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* fix: opendevin user use correct pip

* fix lint issue

* fix custom sandbox base image

* rename test name

* add skipif

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Co-authored-by: tobitege <tobitege@gmx.de>
2024-06-19 19:58:07 -07:00
Boxuan Li
89cae8d6b8 Fix Docker tagging issue with upper case (#2512)
* Fix Docker tagging issue with upper case

* Update containers/build.sh

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>

* Use tr command which is available on both zsh and bash

* Lower image name

* Lower image name

* Update .github/workflows/ghcr.yml

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>

* Fix shell syntax

---------

Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
2024-06-19 00:11:18 -06:00
Boxuan Li
c2fa99b4a1 Split container image build & push (#2456)
* Split container image build & push

* Code cleanup

* Cleanup

* Add back useless docker_build_success step to make CI happy

* Revert "Cleanup"

This reverts commit 2a260791a9.

* Use fresh built sandbox image in integration test

* fix dependency

* DEBUG: only build

* Attempt to fix dependency

* Change dependency

* Combine both jobs

* Fix env

* Remove Mac integration tests as they are too unstable

* Move sandbox tests to ghcr

* Use loaded image
2024-06-19 01:44:51 +08:00
Shimada666
e4d78153fe feat: add environments in global bashrc file (#2486) 2024-06-18 02:36:35 +00:00
tobitege
823298e0d0 fix: Agentskills enhancements (#2384)
* avoid repeat logging of unneeded messages

* refactored append/edit_file (tests next)

* agentskills and unit test fixes

* testing

* more changes and test prompts

* smaller changes

* final test fixes

* remove dead code from test_agent.py

* reverting unneeded changes

* updated tests, more tweaks to skills

* refactor (#2442)

* chores: fix DelegatorAgent description (#2446)

* change

* change comments

* fix

* stopped container to prevent port issues. (#2447)

* chore: remove useless browsing code in CodeActSWEAgent (#2438)

* remove useless

* fix integration test

* Regenerate test_ipython_module artifacts for CodeActSWEAgent

---------

Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>

* Merge remote-tracking branch 'upstream/main' into agent-fileops

* unneeded tweak

* * fix edit_file to not introduce extra newline
* updated docstrings with more details for LLM
* fix legacy typo in prompts causing ]] instead of ]
* several mock files regenerated

* Regen'ed CodeActSWEAgent integration tests

* fix _print_window signature; explicit exception type in _is_valid_path

* splitlines with named param

---------

Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
2024-06-16 15:06:46 -04:00
Yufan Song
e04bde851e test (#2409)
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-06-13 21:10:17 +08:00
மனோஜ்குமார் பழனிச்சாமி
e0ad289483 Downgraded Python version to 3.12.3 (#2331)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-06-09 11:54:30 +05:30
மனோஜ்குமார் பழனிச்சாமி
971ad68431 Solved Hugging Face cache issue. (#2277) 2024-06-05 21:18:33 +05:30
Shimada666
b31f7701eb Integrate Multimodal tools to agentskills. (#2016)
* suport reading multimodal files

* move file

* update dependency

* remove useless pip install

* add comments

* update the comment

* Apply suggestions from code review

* Add unit test for TXTReader

* pre-commit hook corrupted utf16 test txt

* Revert unnecessary dependency upgrades

* feat: import some readers for agentskill

* add dependencies

* Integrate some multimodal tools

* add shell pip dependency

* update dependencies

* update dependencies

* update print window

* remove __main__

* locally import cv2

* add c library for opencv

* update lock file

* update prompt

* remove unuseful file

* add some unittest

* add unittest & remove excel-related parser

* rollback poetry lock

* remove markdown

* remove requests

* optimize parse_video output

* Fix integration tests for CodeActAgent

* remove test_parse_image unittest

* Add a TODO to containers/sandbox/Dockerfile

* update dependencies

* remove pyproject.toml useless package

* change document via openai key

* Fix prompts after removing some actions

---------

Co-authored-by: Mingchen Zhuge <mczhuge@gmail.com>
Co-authored-by: yufansong <yufan@risingwave-labs.com>
Co-authored-by: Boxuan Li <liboxuan@connect.hku.hk>
Co-authored-by: Mingchen Zhuge <64179323+mczhuge@users.noreply.github.com>
Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-05-25 18:58:49 +08:00
மனோஜ்குமார் பழனிச்சாமி
6ef7e6eb0f Checked for enduser in Docker (#1899) 2024-05-20 08:50:42 +05:30
John Tackman
171764469d Update Dockerfile to assign workspace directory properly to user (#1830)
* Update Dockerfile

creating the workspace directory after switching user to fix permission errors
https://github.com/OpenDevin/OpenDevin/issues/1560

* assign $workspace_base to user

instead of creating directory later, just fix the permissions so user can write to it

* another sudo

---------

Co-authored-by: Leo <ifuryst@gmail.com>
2024-05-17 20:11:23 -04:00
Xingyao Wang
c320d908e5 add iproute2 to sandbox (#1859) 2024-05-17 12:57:58 +08:00
Robert Brennan
135320861c set a higer UID_MAX (#1788)
Co-authored-by: Yufan Song <33971064+yufansong@users.noreply.github.com>
2024-05-15 00:28:37 +00:00
Xingyao Wang
1d58917bc8 remove swe-bench build workflow (#1726) 2024-05-12 06:56:20 +08:00
Robert Brennan
1cbb16cfc2 allow running app as root (#1651)
* allow running app as root

* better entrypoint mgmt

* add nosetup option

* remove comments

* create docker group if it doesnt exist

* better docker group mgmt

* cast bools better

* fix playwright

* fix playwright for root

* fix root source ~/.bashrc hangs by create clean bashrc

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-05-10 20:41:25 +00:00
Robert Brennan
7b6b4e3a11 be more dynamic around uid generation (#1584)
* be more dynamic around uid generation

* fix comment

* fix second uid add
2024-05-04 22:26:55 -04:00
Robert Brennan
1d92e9a27b Fix user permissions in docker (#1565)
* permissions mostly working

* fix browser
2024-05-04 13:08:43 +08:00
Robert Brennan
7d8e10aa18 Remove v prefix on tags (#1555) 2024-05-03 10:36:09 -04:00
Robert Brennan
7914258945 Fix docker build, integration test (#1533)
* fix docker build

* revert plugins

* fix imports

* fix import

* fix one more import
2024-05-02 13:49:25 -04:00
Robert Brennan
cfef3ee5c4 Fix docker push for non-forks (#1499)
* fix fork check

* minor docker tweaks

* remove caching

* try not chowinng

* fix chowns

* revert build.sh

* fix entrypoint user

* change message

* remove free disk space

* chown the entrypoint

* remove comments

* empty commit
2024-05-02 10:15:12 +08:00
Xingyao Wang
1c7cdbefdd feat(CodeActAgent): Support Agent-User Interaction during Task Execution and the Full Integration of CodeActAgent (#1290)
* initialize plugin definition

* initialize plugin definition

* simplify mixin

* further improve plugin mixin

* add cache dir for pip

* support clean up cache

* add script for setup jupyter and execution server

* integrate JupyterRequirement to ssh_box

* source bashrc at the end of plugin load

* add execute_cli that accept code via stdin

* make JUPYTER_EXEC_SERVER_PORT configurable via env var

* increase background cmd sleep time

* Update opendevin/sandbox/plugins/mixin.py

Co-authored-by: Robert Brennan <accounts@rbren.io>

* add mixin to base class

* make jupyter requirement a dataclass

* source plugins only when >0 requirements

* add `sandbox_plugins` for each agent & have controller take care of it

* update build.sh to make logs available in /opendevin/logs

* switch to use config for lib and cache dir

* Add SANDBOX_WORKSPACE_DIR into config

* Add SANDBOX_WORKSPACE_DIR into config

* fix occurence of /workspace

* fix permission issue with /workspace

* use python to implement execute_cli to avoid stdin escape issue

* add IPythonRunCellAction and get it working

* wait until jupyter is avaialble

* support plugin via copying instead of mounting

* add agent talk action

* support follow-up user language feedback

* add __str__ for action to be printed better

* only print PLAN at the beginning

* wip: update codeact agent

* get rid the initial messate

* update codeact agent to handle null action;
add thought to bash

* dispatch thought for RUN action as well

* fix weird behavior of pxssh where the output would not flush correctly

* make ssh box can handle exit_code properly as well

* add initial version of swe-agent plugin;

* rename swe cursors

* split setup script into two and create two requirements

* print SWE-agent command documentation

* update swe-agent to default to no custom docs

* add initial version of swe-agent plugin;

* rename swe cursors

* split setup script into two and create two requirements

* print SWE-agent command documentation

* update swe-agent to default to no custom docs

* update dockerfile with dependency from swe-agent

* make env setup a separate script for .bashrc source

* add wip prompt

* fix mount_dir for ssh_box

* update prompt

* fix mount_dir for ssh_box

* default to use host network

* default to use host network

* move prompt to a separate file

* fix swe-tool plugins;
add missing _split_string

* remove hostname from sshbox

* update the prompt with edit functionality

* fix swe-tool plugins;
add missing _split_string

* add awaiting into status bar

* fix the bug of additional send event

* remove some print action

* move logic to config.py

* remove debugging comments

* make host network as default

* make WORKSPACE_MOUNT_PATH as abspath

* implement execute_cli via file cp

* Revert "implement execute_cli via file cp"

This reverts commit 06f0155bc1.

* add codeact dependencies to default container

* add IPythonRunCellObservation

* add back cache dir and default to /tmp

* make USE_HOST_NETWORK a bool

* revert use host network to false

* add temporarily fix for IPython RUN action

* update prompt

* revert USE_HOST_NETWORK to true since it is not affecting anything

* attempt to fix lint

* remove newline

* fix jupyter execution server

* add `thought` to most action class

* fix unit tests for current action abstraction

* support user exit

* update test cases with the latest action format (added 'thought')

* fix integration test for CodeActAGent by mocking stdin

* only mock stdin for tests with user_responses.log

* remove -exec integration test for CodeActAgent since it is not supported

* remove specific stop word

* fix comments

* improve clarity of prompt

* fix py lint

* fix integration tests

* sandbox might failed in chown due to mounting, but it won't be fatal

* update debug instruction for sshbox

* fix typo

* get RUN_AS_DEVIN and network=host working with app sandbox

* get RUN_AS_DEVIN and network=host working with app sandbox

* attempt to fix the workspace base permission

* sandbox might failed in chown due to mounting, but it won't be fatal

* update sshbox instruction

* remove default user id since it will be passed in the instruction

* revert permission fix since it should be resolved by correct SANDBOX_USER_ID

* the permission issue can be fixed by simply provide correct env var

* remove log

* set sandbox user id to getuid by default

* move logging to initializer

* make the uid consistent across host, app container, and sandbox

* remove hostname as it causes sudo issue

* fix permission of entrypoint script

* make the uvicron app run as host user uid for jupyter plugin

* add warning message

* update dev md for instruction of running unit tests

* add back unit tests

* revert back to the original sandbox implementation to fix testcases

* revert use host network

* get docker socket gid and usermod instead of chmod 777

* allow unit test workflow to find docker.sock

* make sandbox test working via patch

* fix arg parser that's broken for some reason

* try to fix app build disk space issue

* fix integration test

* Revert "fix arg parser that's broken for some reason"

This reverts commit 6cc8961133.

* update Development.md

* cleanup intergration tests & add exception for CodeAct+execbox

* fix config

* implement user_message action

* fix doc

* fix event dict error

* fix frontend lint

* revert accidentally changes to integration tests

* revert accidentally changes to integration tests

---------

Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: Robert Brennan <contact@rbren.io>
2024-05-01 08:40:00 -04:00
Xingyao Wang
ccbbabac1c Get RUN_AS_DEVIN working with app sandbox (#1426)
* get RUN_AS_DEVIN and network=host working with app sandbox

* attempt to fix the workspace base permission

* sandbox might failed in chown due to mounting, but it won't be fatal

* update sshbox instruction

* remove default user id since it will be passed in the instruction

* revert permission fix since it should be resolved by correct SANDBOX_USER_ID

* the permission issue can be fixed by simply provide correct env var

* remove log

* set sandbox user id to getuid by default

* move logging to initializer

* make the uid consistent across host, app container, and sandbox

* remove hostname as it causes sudo issue

* fix permission of entrypoint script

* make the uvicron app run as host user uid for jupyter plugin

* revert use host network

* get docker socket gid and usermod instead of chmod 777

* try to fix app build disk space issue
2024-04-30 12:39:52 -04:00
Robert Brennan
11d48cc2f3 Run docker on forked pull requests (#1450)
* build docker on pull request

* run docker build on PRs

* remove if

* add permissions

* change ghcr login

* empty commit

* always use opendevin as org

* lowercase

* no client token

* dont push on forks

* remove env

* only cache-to if pushing

* fix org name

* fix owner

* Update containers/build.sh

Co-authored-by: Graham Neubig <neubig@gmail.com>

* lowercase

* remove tag prefix

* lowercase

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
2024-04-29 14:24:28 -04:00
Robert Brennan
ce47461c5a disable host network (#1404) 2024-04-27 08:07:25 -04:00
Robert Brennan
43214a69a0 Better version management (#1313)
* better version management

* Update opendevin/config.py
2024-04-24 21:53:49 +00:00
Xingyao Wang
fde0392457 Default for sandbox to use the host network (#1334)
* default to use host network

* make host network as default
2024-04-24 21:48:11 +00:00
Robert Brennan
8828d9836d use caching for docker builds (#1340)
* try pulling for cached layers

* use cache-from and cache-to

* empty commit

* add readme

* use specific cache tags

* also cache from main
2024-04-24 15:57:46 -04:00
Xingyao Wang
a0e8fcb19a Add SWE-agent tools as sandbox plugins (#1305)
* add initial version of swe-agent plugin;

* rename swe cursors

* split setup script into two and create two requirements

* print SWE-agent command documentation

* update swe-agent to default to no custom docs

* update dockerfile with dependency from swe-agent

* make env setup a separate script for .bashrc source

* fix swe-tool plugins;
add missing _split_string

---------

Co-authored-by: Robert Brennan <accounts@rbren.io>
2024-04-24 22:50:14 +08:00
மனோஜ்குமார் பழனிச்சாமி
99829f95fd Applied npm patch for build errors (#1326)
* npm patch

https://github.com/npm/cli/issues/7231#issuecomment-2062032218

* Delete huge unnecessary tools folder
2024-04-24 09:24:48 -04:00
Leo
0a2e90dece Disable additional Docker metadata to more compatible with OCI spec. (#1302) 2024-04-23 07:55:00 -04:00
Engel Nyst
960f17a565 Bug with sshd (#1297)
* Workaround for a bug with sshd

* Update containers/sandbox/Dockerfile

Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
2024-04-23 01:33:44 +00:00
Xingyao Wang
e84125de6f Symlink python3 to python in Sandbox (#1286)
* symlink py3 to py

* remove service ssh start
2024-04-23 01:05:05 +08:00
மனோஜ்குமார் பழனிச்சாமி
cfbd482498 Added playwright commands (#1184)
* post playwright commands

* post playwright commands with deps

* post playwright commands

* post playwright commands with deps

* Put playwright install in the correct place

---------

Co-authored-by: Graham Neubig <neubig@gmail.com>
2024-04-22 09:43:08 -04:00
Vasek Mlejnsky
76b81ca0ed Integrate E2B sandbox as an alternative to a Docker container (#727)
* add e2b sandbox [wip]

* Install e2b package

* Add basic E2B sandbox integration

* Update dependencies and fix command execution in E2BSandbox

* Udpate e2b

* Add comment

* Lint

* Remove unnecessary type conversion

* Lint

* Fix linting

* Resolve comments

* Update opendevin/action/fileop.py

* Update opendevin/action/fileop.py

* Fix log

* Update E2B readme

* poetry lock

---------

Co-authored-by: Robert Brennan <accounts@rbren.io>
2024-04-19 14:21:58 -04:00
Xingyao Wang
871eefe801 Revert "feat(sandbox): Add Jupyter Kernel for Interactive Python Interpreter for Sandbox (#1215)" (#1229)
This reverts commit 492feecb67.
2024-04-19 16:49:24 +00:00
Leo
34286fabcc fix ghcr workflow throw error when in the fork repo. (#1203)
* fix ghcr workflow throw error when in the fork repo.

* Split DOCKER_REPOSITORY for cleaner.
2024-04-19 11:06:25 -04:00
Xingyao Wang
492feecb67 feat(sandbox): Add Jupyter Kernel for Interactive Python Interpreter for Sandbox (#1215)
* add initial version of py interpreter

* fix bug

* fix async issue

* remove debugging print statement

* initialize kernel & update printing

* fix port mapping

* uncomment debug lines

* fix poetry lock

* make jupyter py interpreter into a subclass
2024-04-19 16:55:51 +08:00
Robert Brennan
516c9bf1e0 Revamp docker build process (#1121)
* refactor docker building

* change to buildx

* disable branch filter

* disable tags

* matrix for building

* fix branch filter

* rename workflow

* sanitize ref name

* fix sanitization

* fix source command

* fix source command

* add push arg

* enable for all branches

* logs

* empty commit

* try freeing disk space

* try disk clean again

* try alpine

* Update ghcr.yml

* Update ghcr.yml

* move checkout

* ignore .git

* add disk space debug

* add df h to build script

* remove pull

* try another failure bypass

* remove maximize build space step

* remove df -h debug

* add no-root

* multi-stage python build

* add ssh

* update readme

* remove references to config.toml
2024-04-15 19:10:38 -04:00