Xingyao Wang 1c7cdbefdd
feat(CodeActAgent): Support Agent-User Interaction during Task Execution and the Full Integration of CodeActAgent (#1290)
* initialize plugin definition

* initialize plugin definition

* simplify mixin

* further improve plugin mixin

* add cache dir for pip

* support clean up cache

* add script for setup jupyter and execution server

* integrate JupyterRequirement to ssh_box

* source bashrc at the end of plugin load

* add execute_cli that accept code via stdin

* make JUPYTER_EXEC_SERVER_PORT configurable via env var

* increase background cmd sleep time

* Update opendevin/sandbox/plugins/mixin.py

Co-authored-by: Robert Brennan <accounts@rbren.io>

* add mixin to base class

* make jupyter requirement a dataclass

* source plugins only when >0 requirements

* add `sandbox_plugins` for each agent & have controller take care of it

* update build.sh to make logs available in /opendevin/logs

* switch to use config for lib and cache dir

* Add SANDBOX_WORKSPACE_DIR into config

* Add SANDBOX_WORKSPACE_DIR into config

* fix occurence of /workspace

* fix permission issue with /workspace

* use python to implement execute_cli to avoid stdin escape issue

* add IPythonRunCellAction and get it working

* wait until jupyter is avaialble

* support plugin via copying instead of mounting

* add agent talk action

* support follow-up user language feedback

* add __str__ for action to be printed better

* only print PLAN at the beginning

* wip: update codeact agent

* get rid the initial messate

* update codeact agent to handle null action;
add thought to bash

* dispatch thought for RUN action as well

* fix weird behavior of pxssh where the output would not flush correctly

* make ssh box can handle exit_code properly as well

* add initial version of swe-agent plugin;

* rename swe cursors

* split setup script into two and create two requirements

* print SWE-agent command documentation

* update swe-agent to default to no custom docs

* add initial version of swe-agent plugin;

* rename swe cursors

* split setup script into two and create two requirements

* print SWE-agent command documentation

* update swe-agent to default to no custom docs

* update dockerfile with dependency from swe-agent

* make env setup a separate script for .bashrc source

* add wip prompt

* fix mount_dir for ssh_box

* update prompt

* fix mount_dir for ssh_box

* default to use host network

* default to use host network

* move prompt to a separate file

* fix swe-tool plugins;
add missing _split_string

* remove hostname from sshbox

* update the prompt with edit functionality

* fix swe-tool plugins;
add missing _split_string

* add awaiting into status bar

* fix the bug of additional send event

* remove some print action

* move logic to config.py

* remove debugging comments

* make host network as default

* make WORKSPACE_MOUNT_PATH as abspath

* implement execute_cli via file cp

* Revert "implement execute_cli via file cp"

This reverts commit 06f0155bc17d1f99097e71b83b2143f6e8092654.

* add codeact dependencies to default container

* add IPythonRunCellObservation

* add back cache dir and default to /tmp

* make USE_HOST_NETWORK a bool

* revert use host network to false

* add temporarily fix for IPython RUN action

* update prompt

* revert USE_HOST_NETWORK to true since it is not affecting anything

* attempt to fix lint

* remove newline

* fix jupyter execution server

* add `thought` to most action class

* fix unit tests for current action abstraction

* support user exit

* update test cases with the latest action format (added 'thought')

* fix integration test for CodeActAGent by mocking stdin

* only mock stdin for tests with user_responses.log

* remove -exec integration test for CodeActAgent since it is not supported

* remove specific stop word

* fix comments

* improve clarity of prompt

* fix py lint

* fix integration tests

* sandbox might failed in chown due to mounting, but it won't be fatal

* update debug instruction for sshbox

* fix typo

* get RUN_AS_DEVIN and network=host working with app sandbox

* get RUN_AS_DEVIN and network=host working with app sandbox

* attempt to fix the workspace base permission

* sandbox might failed in chown due to mounting, but it won't be fatal

* update sshbox instruction

* remove default user id since it will be passed in the instruction

* revert permission fix since it should be resolved by correct SANDBOX_USER_ID

* the permission issue can be fixed by simply provide correct env var

* remove log

* set sandbox user id to getuid by default

* move logging to initializer

* make the uid consistent across host, app container, and sandbox

* remove hostname as it causes sudo issue

* fix permission of entrypoint script

* make the uvicron app run as host user uid for jupyter plugin

* add warning message

* update dev md for instruction of running unit tests

* add back unit tests

* revert back to the original sandbox implementation to fix testcases

* revert use host network

* get docker socket gid and usermod instead of chmod 777

* allow unit test workflow to find docker.sock

* make sandbox test working via patch

* fix arg parser that's broken for some reason

* try to fix app build disk space issue

* fix integration test

* Revert "fix arg parser that's broken for some reason"

This reverts commit 6cc89611337bb74555fd16b4be78681fb7e36573.

* update Development.md

* cleanup intergration tests & add exception for CodeAct+execbox

* fix config

* implement user_message action

* fix doc

* fix event dict error

* fix frontend lint

* revert accidentally changes to integration tests

* revert accidentally changes to integration tests

---------

Co-authored-by: Robert Brennan <accounts@rbren.io>
Co-authored-by: Robert Brennan <contact@rbren.io>
2024-05-01 08:40:00 -04:00

154 lines
6.1 KiB
Python

import os
from dataclasses import dataclass
from pathlib import Path
from opendevin import config
from opendevin.observation import (
AgentErrorObservation,
FileReadObservation,
FileWriteObservation,
Observation,
)
from opendevin.sandbox import E2BBox
from opendevin.schema import ActionType
from opendevin.schema.config import ConfigType
from .base import ExecutableAction
def resolve_path(file_path, working_directory):
path_in_sandbox = Path(file_path)
# Apply working directory
if not path_in_sandbox.is_absolute():
path_in_sandbox = Path(working_directory) / path_in_sandbox
# Sanitize the path with respect to the root of the full sandbox
# (deny any .. path traversal to parent directories of the sandbox)
abs_path_in_sandbox = path_in_sandbox.resolve()
# If the path is outside the workspace, deny it
if not abs_path_in_sandbox.is_relative_to(config.get(ConfigType.WORKSPACE_MOUNT_PATH_IN_SANDBOX)):
raise PermissionError(f'File access not permitted: {file_path}')
# Get path relative to the root of the workspace inside the sandbox
path_in_workspace = abs_path_in_sandbox.relative_to(Path(config.get(ConfigType.WORKSPACE_MOUNT_PATH_IN_SANDBOX)))
# Get path relative to host
path_in_host_workspace = Path(config.get(ConfigType.WORKSPACE_BASE)) / path_in_workspace
return path_in_host_workspace
@dataclass
class FileReadAction(ExecutableAction):
"""
Reads a file from a given path.
Can be set to read specific lines using start and end
Default lines 0:-1 (whole file)
"""
path: str
start: int = 0
end: int = -1
thought: str = ''
action: str = ActionType.READ
def _read_lines(self, all_lines: list[str]):
if self.end == -1:
if self.start == 0:
return all_lines
else:
return all_lines[self.start:]
else:
num_lines = len(all_lines)
begin = max(0, min(self.start, num_lines - 2))
end = -1 if self.end > num_lines else max(begin + 1, self.end)
return all_lines[begin:end]
async def run(self, controller) -> Observation:
if isinstance(controller.action_manager.sandbox, E2BBox):
content = controller.action_manager.sandbox.filesystem.read(
self.path)
read_lines = self._read_lines(content.split('\n'))
code_view = ''.join(read_lines)
else:
try:
whole_path = resolve_path(self.path, controller.action_manager.sandbox.get_working_directory())
self.start = max(self.start, 0)
try:
with open(whole_path, 'r', encoding='utf-8') as file:
read_lines = self._read_lines(file.readlines())
code_view = ''.join(read_lines)
except FileNotFoundError:
return AgentErrorObservation(f'File not found: {self.path}')
except UnicodeDecodeError:
return AgentErrorObservation(f'File could not be decoded as utf-8: {self.path}')
except IsADirectoryError:
return AgentErrorObservation(f'Path is a directory: {self.path}. You can only read files')
except PermissionError:
return AgentErrorObservation(f'Malformed paths not permitted: {self.path}')
return FileReadObservation(path=self.path, content=code_view)
@property
def message(self) -> str:
return f'Reading file: {self.path}'
@dataclass
class FileWriteAction(ExecutableAction):
path: str
content: str
start: int = 0
end: int = -1
thought: str = ''
action: str = ActionType.WRITE
def _insert_lines(self, to_insert: list[str], original: list[str]):
"""
Insert the new content to the original content based on self.start and self.end
"""
new_lines = [''] if self.start == 0 else original[:self.start]
new_lines += [i + '\n' for i in to_insert]
new_lines += [''] if self.end == -1 else original[self.end:]
return new_lines
async def run(self, controller) -> Observation:
insert = self.content.split('\n')
if isinstance(controller.action_manager.sandbox, E2BBox):
files = controller.action_manager.sandbox.filesystem.list(self.path)
if self.path in files:
all_lines = controller.action_manager.sandbox.filesystem.read(self.path)
new_file = self._insert_lines(self.content.split('\n'), all_lines)
controller.action_manager.sandbox.filesystem.write(self.path, ''.join(new_file))
else:
return AgentErrorObservation(f'File not found: {self.path}')
else:
try:
whole_path = resolve_path(self.path, controller.action_manager.sandbox.get_working_directory())
mode = 'w' if not os.path.exists(whole_path) else 'r+'
try:
with open(whole_path, mode, encoding='utf-8') as file:
if mode != 'w':
all_lines = file.readlines()
new_file = self._insert_lines(insert, all_lines)
else:
new_file = [i + '\n' for i in insert]
file.seek(0)
file.writelines(new_file)
file.truncate()
except FileNotFoundError:
return AgentErrorObservation(f'File not found: {self.path}')
except IsADirectoryError:
return AgentErrorObservation(f'Path is a directory: {self.path}. You can only write to files')
except UnicodeDecodeError:
return AgentErrorObservation(f'File could not be decoded as utf-8: {self.path}')
except PermissionError:
return AgentErrorObservation(f'Malformed paths not permitted: {self.path}')
return FileWriteObservation(content='', path=self.path)
@property
def message(self) -> str:
return f'Writing file: {self.path}'