mirror of
https://github.com/OpenHands/OpenHands.git
synced 2025-12-26 05:48:36 +08:00
6aad23d35c
Co-authored-by: Mahdiglm <mahdiglm@users.noreply.github.com> Co-authored-by: Engel Nyst <engel.nyst@gmail.com> Co-authored-by: mamoodi <mamoodiha@gmail.com> Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
107 lines
4.4 KiB
Plaintext
107 lines
4.4 KiB
Plaintext
---
|
|
title: Overview
|
|
description: OpenHands can connect to any LLM supported by LiteLLM. However, it requires a powerful model to work.
|
|
---
|
|
|
|
<Note>
|
|
This section is for users who want to connect OpenHands to different LLMs.
|
|
</Note>
|
|
|
|
## Model Recommendations
|
|
|
|
Based on our evaluations of language models for coding tasks (using the SWE-bench dataset), we can provide some
|
|
recommendations for model selection. Our latest benchmarking results can be found in [this spreadsheet](https://docs.google.com/spreadsheets/d/1wOUdFCMyY6Nt0AIqF705KN4JKOWgeI4wUGUP60krXXs/edit?gid=0).
|
|
|
|
Based on these findings and community feedback, these are the latest models that have been verified to work reasonably well with OpenHands:
|
|
|
|
### Cloud / API-Based Models
|
|
|
|
- [anthropic/claude-sonnet-4-20250514](https://www.anthropic.com/api) (recommended)
|
|
- [openai/o4-mini](https://openai.com/index/introducing-o3-and-o4-mini/)
|
|
- [gemini/gemini-2.5-pro](https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/)
|
|
- [deepseek/deepseek-chat](https://api-docs.deepseek.com/)
|
|
|
|
If you have successfully run OpenHands with specific providers, we encourage you to open a PR to share your setup process
|
|
to help others using the same provider!
|
|
|
|
For a full list of the providers and models available, please consult the
|
|
[litellm documentation](https://docs.litellm.ai/docs/providers).
|
|
|
|
<Warning>
|
|
OpenHands will issue many prompts to the LLM you configure. Most of these LLMs cost money, so be sure to set spending
|
|
limits and monitor usage.
|
|
</Warning>
|
|
|
|
### Local / Self-Hosted Models
|
|
|
|
- [mistralai/devstral-small](https://www.all-hands.dev/blog/devstral-a-new-state-of-the-art-open-model-for-coding-agents) (20 May 2025) -- also available through [OpenRouter](https://openrouter.ai/mistralai/devstral-small:free)
|
|
- [all-hands/openhands-lm-32b-v0.1](https://www.all-hands.dev/blog/introducing-openhands-lm-32b----a-strong-open-coding-agent-model) (31 March 2025) -- also available through [OpenRouter](https://openrouter.ai/all-hands/openhands-lm-32b-v0.1)
|
|
|
|
<Note>
|
|
Most current local and open source models are not as powerful. When using such models, you may see long
|
|
wait times between messages, poor responses, or errors about malformed JSON. OpenHands can only be as powerful as the
|
|
models driving it. However, if you do find ones that work, please add them to the verified list above.
|
|
</Note>
|
|
|
|
## LLM Configuration
|
|
|
|
The following can be set in the OpenHands UI through the Settings:
|
|
|
|
- `LLM Provider`
|
|
- `LLM Model`
|
|
- `API Key`
|
|
- `Base URL` (through `Advanced` settings)
|
|
|
|
There are some settings that may be necessary for some LLMs/providers that cannot be set through the UI. Instead, these
|
|
can be set through environment variables passed to the docker run command when starting the app
|
|
using `-e`:
|
|
|
|
- `LLM_API_VERSION`
|
|
- `LLM_EMBEDDING_MODEL`
|
|
- `LLM_EMBEDDING_DEPLOYMENT_NAME`
|
|
- `LLM_DROP_PARAMS`
|
|
- `LLM_DISABLE_VISION`
|
|
- `LLM_CACHING_PROMPT`
|
|
|
|
We have a few guides for running OpenHands with specific model providers:
|
|
|
|
- [Azure](/usage/llms/azure-llms)
|
|
- [Google](/usage/llms/google-llms)
|
|
- [Groq](/usage/llms/groq)
|
|
- [Local LLMs with SGLang or vLLM](/usage/llms/local-llms)
|
|
- [LiteLLM Proxy](/usage/llms/litellm-proxy)
|
|
- [OpenAI](/usage/llms/openai-llms)
|
|
- [OpenRouter](/usage/llms/openrouter)
|
|
|
|
## Model Customization
|
|
|
|
LLM providers have specific settings that can be customized to optimize their performance with OpenHands, such as:
|
|
|
|
- **Custom Tokenizers**: For specialized models, you can add a suitable tokenizer
|
|
- **Native Tool Calling**: Toggle native function/tool calling capabilities
|
|
|
|
For detailed information about model customization, see [LLM Configuration Options](configuration-options#llm-customization).
|
|
|
|
### API retries and rate limits
|
|
|
|
LLM providers typically have rate limits, sometimes very low, and may require retries. OpenHands will automatically
|
|
retry requests if it receives a Rate Limit Error (429 error code).
|
|
|
|
You can customize these options as you need for the provider you're using. Check their documentation, and set the
|
|
following environment variables to control the number of retries and the time between retries:
|
|
|
|
- `LLM_NUM_RETRIES` (Default of 4 times)
|
|
- `LLM_RETRY_MIN_WAIT` (Default of 5 seconds)
|
|
- `LLM_RETRY_MAX_WAIT` (Default of 30 seconds)
|
|
- `LLM_RETRY_MULTIPLIER` (Default of 2)
|
|
|
|
If you are running OpenHands in development mode, you can also set these options in the `config.toml` file:
|
|
|
|
```toml
|
|
[llm]
|
|
num_retries = 4
|
|
retry_min_wait = 5
|
|
retry_max_wait = 30
|
|
retry_multiplier = 2
|
|
```
|