mirror of
https://github.com/OpenHands/OpenHands.git
synced 2026-03-22 13:47:19 +08:00
Doc: LM Studio guide (#2875)
This commit is contained in:
committed by
GitHub
parent
e45d46c993
commit
7cbf2d9f16
@@ -53,6 +53,7 @@ docker run \
|
||||
-e SANDBOX_USER_ID=$(id -u) \
|
||||
-e LLM_API_KEY="ollama" \
|
||||
-e LLM_BASE_URL="http://host.docker.internal:11434" \
|
||||
-e LLM_OLLAMA_BASE_URL="http://host.docker.internal:11434" \
|
||||
-e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
|
||||
-v $WORKSPACE_BASE:/opt/workspace_base \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
@@ -68,12 +69,16 @@ Use the instructions in [Development.md](https://github.com/OpenDevin/OpenDevin/
|
||||
Make sure `config.toml` is there by running `make setup-config` which will create one for you. In `config.toml`, enter the followings:
|
||||
|
||||
```
|
||||
LLM_MODEL="ollama/codellama:7b"
|
||||
LLM_API_KEY="ollama"
|
||||
LLM_EMBEDDING_MODEL="local"
|
||||
LLM_BASE_URL="http://localhost:11434"
|
||||
WORKSPACE_BASE="./workspace"
|
||||
WORKSPACE_DIR="$(pwd)/workspace"
|
||||
[core]
|
||||
workspace_base="./workspace"
|
||||
|
||||
[llm]
|
||||
model="ollama/codellama:7b"
|
||||
api_key="ollama"
|
||||
embedding_model="local"
|
||||
base_url="http://localhost:11434"
|
||||
ollama_base_url="http://localhost:11434"
|
||||
|
||||
```
|
||||
|
||||
Replace `LLM_MODEL` of your choice if you need to.
|
||||
@@ -142,3 +147,62 @@ ollama list # get list of installed models
|
||||
docker ps # get list of running docker containers, for most accurate test choose the open devin sandbox container.
|
||||
docker exec [CONTAINER ID] curl http://host.docker.internal:11434/api/generate -d '{"model":"[NAME]","prompt":"hi"}'
|
||||
```
|
||||
|
||||
|
||||
# Local LLM with LM Studio
|
||||
|
||||
Steps to set up LM Studio:
|
||||
1. Open LM Studio
|
||||
2. Go to the Local Server tab.
|
||||
3. Click the "Start Server" button.
|
||||
4. Select the model you want to use from the dropdown.
|
||||
|
||||
|
||||
Set the following configs:
|
||||
```bash
|
||||
LLM_MODEL="openai/lmstudio"
|
||||
LLM_BASE_URL="http://localhost:1234/v1"
|
||||
CUSTOM_LLM_PROVIDER="openai"
|
||||
```
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker run \
|
||||
-it \
|
||||
--pull=always \
|
||||
-e SANDBOX_USER_ID=$(id -u) \
|
||||
-e LLM_MODEL="openai/lmstudio"
|
||||
-e LLM_BASE_URL="http://host.docker.internal:1234/v1" \
|
||||
-e CUSTOM_LLM_PROVIDER="openai"
|
||||
-e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
|
||||
-v $WORKSPACE_BASE:/opt/workspace_base \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-p 3000:3000 \
|
||||
ghcr.io/opendevin/opendevin:main
|
||||
```
|
||||
|
||||
You should now be able to connect to `http://localhost:3000/`
|
||||
|
||||
In the development environment, you can set the following configs in the `config.toml` file:
|
||||
|
||||
```
|
||||
[core]
|
||||
workspace_base="./workspace"
|
||||
|
||||
[llm]
|
||||
model="openai/lmstudio"
|
||||
base_url="http://localhost:1234/v1"
|
||||
custom_llm_provider="openai"
|
||||
```
|
||||
|
||||
Done! Now you can start Devin by: `make run` without Docker. You now should be able to connect to `http://localhost:3000/`
|
||||
|
||||
# Note:
|
||||
|
||||
For WSL, run the following commands in cmd to set up the networking mode to mirrored:
|
||||
|
||||
```
|
||||
python -c "print('[wsl2]\nnetworkingMode=mirrored',file=open(r'%UserProfile%\.wslconfig','w'))"
|
||||
wsl --shutdown
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user