github/OpenHands
1
0
Fork 0
mirror of https://github.com/OpenHands/OpenHands.git synced 2026-03-22 13:47:19 +08:00
Code Issues Packages Projects Releases Wiki Activity

[bugfix] browse actions shouldn't change url and screenshot, only observations (#2311)

Browse Source 
* browse related actions shouldn't change url and screenshot, only the observations should

* fix linting

* fix integrat

* update integration test

---------

Co-authored-by: Xingyao Wang <xingyao6@illinois.edu>
This commit is contained in:
Frank Xu
2024-06-07 12:03:32 -04:00
committed by GitHub
parent 45ce09d70e
commit 4455260290
3 changed files with 6 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
frontend/src/services/actions.ts
View File

@@ -1,4 +1,3 @@
import { setScreenshotSrc, setUrl } from "#/state/browserSlice";
import { addAssistantMessage, addUserMessage } from "#/state/chatSlice";
import { setCode, setActiveFilepath } from "#/state/codeSlice";
import { appendInput } from "#/state/commandSlice";
@@ -13,18 +12,14 @@ import { getRootTask } from "./taskService";
const messageActions = {
[ActionType.BROWSE]: (message: ActionMessage) => {
const { url, screenshotSrc } = message.args;
store.dispatch(setUrl(url));
store.dispatch(setScreenshotSrc(screenshotSrc));
store.dispatch(addAssistantMessage(message.message));
},
[ActionType.BROWSE_INTERACTIVE]: (message: ActionMessage) => {
if (message.args.thought) {
store.dispatch(addAssistantMessage(message.args.thought));
} else {
store.dispatch(addAssistantMessage(message.message));
}
const { url, screenshotSrc } = message.args;
store.dispatch(setUrl(url));
store.dispatch(setScreenshotSrc(screenshotSrc));
},
[ActionType.WRITE]: (message: ActionMessage) => {
const { path, content } = message.args;

5
tests/integration/mock/CodeActAgent/test_browse_internet/prompt_003.log
View File

@@ -118,11 +118,10 @@ RootWebArea 'The Ultimate Answer', focused
[8] heading 'The Ultimate Answer'
[9] paragraph ''
StaticText 'Click the button to reveal the answer to life, the universe, and everything.'
[10] button 'Click me'
[10] button 'Click me', clickable
# Previous Actions
goto('http://localhost:8000')
noop()
Here is an example with chain of thought of a valid action when clicking on a button:
"

5
tests/integration/mock/CodeActAgent/test_browse_internet/prompt_004.log
View File

@@ -118,14 +118,13 @@ RootWebArea 'The Ultimate Answer', focused
[8] heading 'The Ultimate Answer'
[9] paragraph ''
StaticText 'Click the button to reveal the answer to life, the universe, and everything.'
[10] button 'Click me', focused
[10] button 'Click me', clickable, focused
StaticText 'The answer is OpenDevin is all you need!'
# Previous Actions
goto('http://localhost:8000')
noop()
click("10")
Here is an example with chain of thought of a valid action when clicking on a button:
"
In order to accomplish my goal I need to click on the button with bid 12