github/OpenHands
1
0
Fork 0
mirror of https://github.com/OpenHands/OpenHands.git synced 2025-12-26 05:48:36 +08:00
Code Issues Packages Projects Releases Wiki Activity
OpenHands/docs/modules/usage/llms/groq.md
mamoodi 4de6c782cc
Add doc style guide and make docs adhere to it (#5983) 
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2025-01-03 12:14:14 -05:00

1.2 KiB
Raw Blame History

Groq

OpenHands uses LiteLLM to make calls to chat models on Groq. You can find their documentation on using Groq as a provider here.

Configuration

When running OpenHands, you'll need to set the following in the OpenHands UI through the Settings:

  • LLM Provider to Groq
  • LLM Model to the model you will be using. Visit here to see the list of models that Groq hosts. If the model is not in the list, toggle Advanced Options, and enter it in Custom Model (e.g. groq/<model-name> like groq/llama3-70b-8192).
  • API key to your Groq API key. To find or create your Groq API Key, see here.

Using Groq as an OpenAI-Compatible Endpoint

The Groq endpoint for chat completion is mostly OpenAI-compatible. Therefore, you can access Groq models as you would access any OpenAI-compatible endpoint. In the OpenHands UI through the Settings:

  1. Enable Advanced Options
  2. Set the following:
    • Custom Model to the prefix openai/ + the model you will be using (e.g. openai/llama3-70b-8192)
    • Base URL to https://api.groq.com/openai/v1
    • API Key to your Groq API key