diff --git a/.agents/skills/update-sdk/SKILL.md b/.agents/skills/update-sdk/SKILL.md new file mode 100644 index 0000000000..a818db3641 --- /dev/null +++ b/.agents/skills/update-sdk/SKILL.md @@ -0,0 +1,123 @@ +--- +name: update-sdk +description: This skill should be used when the user asks to "update SDK", "bump SDK version", "pin SDK to a commit", "test unreleased SDK", "update agent-server image", "bump the version", "prepare a release", "what files change for a release", or needs to know how SDK packages are managed in the OpenHands repository. For detailed reference material, see references/docker-image-locations.md and references/sdk-pinning-examples.md in this skill directory. +--- + +# Update SDK + +Bump SDK packages (`openhands-sdk`, `openhands-agent-server`, `openhands-tools`), pin them to unreleased commits for testing, and cut an OpenHands release. + +## Quick Summary — How Many Files Change? + +| Activity | Manual edits | Auto-regenerated | Total | +|----------|:------------:|:----------------:|:-----:| +| **SDK bump** (released PyPI version) | 2 | 3 | **5** | +| **SDK pin** (unreleased git commit) | 3 | 3 | **6** | +| **Release commit** (version bump) | 3 | 0 | **3** | + +The 3 auto-regenerated files are always: `poetry.lock`, `uv.lock`, `enterprise/poetry.lock`. + +## SDK Package Bump — 2 Files + 3 Lock Files + +Land as a separate PR before the release. Examples: `929dcc3` (SDK 1.11.5), `cd235cc` (SDK 1.11.4). + +| File | What to change | +|------|----------------| +| `pyproject.toml` | `openhands-sdk`, `openhands-agent-server`, `openhands-tools` in **two** sections: the `dependencies` array (PEP 508) **and** `[tool.poetry.dependencies]` | +| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE` constant — set to `ghcr.io/openhands/agent-server:-python` | + +Then regenerate lock files: +```bash +poetry lock && uv lock && cd enterprise && poetry lock && cd .. +``` + +## Docker Image Locations — All Hardcoded References + +For the complete inventory of every file containing a hardcoded Docker image tag or repository, see `references/docker-image-locations.md`. Key files that must stay in sync during an SDK bump: + +| File | Image reference | Updated during SDK bump? | +|------|----------------|:------------------------:| +| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:-python'` | ✅ Yes | +| `docker-compose.yml` | `AGENT_SERVER_IMAGE_TAG` default | ✅ Should be | +| `containers/dev/compose.yml` | `AGENT_SERVER_IMAGE_REPOSITORY` + `_TAG` defaults | ✅ Should be | + +> **CI enforcement:** `.github/workflows/check-version-consistency.yml` validates version consistency and compose file image references on every PR and push to main. + +### ⚠️ Docker Image Tag Gotcha (merge-commit SHA) + +The SDK CI in `software-agent-sdk` repo tags Docker images with the **GitHub Actions merge-commit SHA**, NOT the PR head-commit SHA. When pinning to an SDK PR branch: + +1. Check the SDK PR description for the actual image tag (look for the `AGENT_SERVER_IMAGES` section) +2. Or query the CI logs: the "Consolidate Build Information" job prints `"short_sha": ""` +3. The merge-commit SHA differs from the head SHA shown in the PR + +For released SDK versions, images use a version tag (e.g., `1.12.0-python`) — no merge-commit ambiguity. + +## Cutting a Release — 3 Files + +A release commit updates the version string across 3 files. Gold-standard examples: 1.3.0 (`d063c8c`), 1.4.0 (`495f48b`). + +| File | What to change | +|------|----------------| +| `pyproject.toml` | `version = "X.Y.Z"` under `[tool.poetry]` | +| `frontend/package.json` | `"version": "X.Y.Z"` | +| `frontend/package-lock.json` | `"version": "X.Y.Z"` in **two** places (root object and `packages[""]`) | + +> **Note:** `openhands/version.py` reads the version from `pyproject.toml` at runtime — no manual edit needed there. + +### Compose Files (2 files) + +Both compose files should use `ghcr.io/openhands/agent-server` with the current SDK version tag. + +| File | What to verify | +|------|----------------| +| `docker-compose.yml` | `AGENT_SERVER_IMAGE_REPOSITORY` defaults to agent-server, `AGENT_SERVER_IMAGE_TAG` is current | +| `containers/dev/compose.yml` | Same — must use agent-server, not runtime | + +### Release Workflow + +#### Step 1: Verify the SDK bump has landed + +```bash +grep -n "openhands-sdk\|openhands-agent-server\|openhands-tools" pyproject.toml +grep -n "AGENT_SERVER_IMAGE" openhands/app_server/sandbox/sandbox_spec_service.py +grep "AGENT_SERVER_IMAGE_TAG" docker-compose.yml containers/dev/compose.yml +``` + +#### Step 2: Bump version numbers + +```bash +# Edit pyproject.toml, frontend/package.json, frontend/package-lock.json +git add pyproject.toml frontend/package.json frontend/package-lock.json +git commit -m "Release X.Y.Z" +git tag X.Y.Z +``` + +Create a `saas-rel-X.Y.Z` branch from the tagged commit for the SaaS deployment pipeline. + +#### Step 3: CI builds Docker images automatically + +The `ghcr-build.yml` workflow triggers on tag pushes and produces: +- `ghcr.io/openhands/openhands:X.Y.Z`, `X.Y`, `X`, `latest` +- `ghcr.io/openhands/runtime:X.Y.Z-nikolaik`, `X.Y-nikolaik` + +The tagging logic lives in `containers/build.sh` — when `GITHUB_REF_NAME` matches a semver pattern (`^[0-9]+\.[0-9]+\.[0-9]+$`), it auto-generates major, major.minor, and `latest` tags. + +## Development: Pin SDK to an Unreleased Commit + +For detailed examples of all pinning formats (commit, branch, uv-only), see `references/sdk-pinning-examples.md`. + +### Files to change (3 manual + 3 lock files) + +| File | What to change | +|------|----------------| +| `pyproject.toml` | Pin all 3 SDK packages in **both** `dependencies` and `[tool.poetry.dependencies]` | +| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE` — use the merge-commit SHA tag, NOT the head-commit SHA | +| `docker-compose.yml` | `AGENT_SERVER_IMAGE_TAG` default (for local development) | +| `poetry.lock` | Auto-regenerated via `poetry lock` | +| `uv.lock` | Auto-regenerated via `uv lock` | +| `enterprise/poetry.lock` | Auto-regenerated via `cd enterprise && poetry lock` | + +### CI guard + +The `check-package-versions.yml` workflow blocks merging to `main` if `[tool.poetry.dependencies]` contains any `rev` fields. This ensures unreleased SDK pins do not accidentally ship in a release. diff --git a/.agents/skills/update-sdk/references/docker-image-locations.md b/.agents/skills/update-sdk/references/docker-image-locations.md new file mode 100644 index 0000000000..c7a23bf43c --- /dev/null +++ b/.agents/skills/update-sdk/references/docker-image-locations.md @@ -0,0 +1,84 @@ +# Docker Image Locations — Complete Inventory + +Every file in the OpenHands repository containing a hardcoded Docker image tag, repository, or version-pinned image reference. Organized by update cadence. + +## Updated During SDK Bump (must change) + +These files contain image tags that **must** be updated whenever the SDK version or pinned commit changes. + +### `openhands/app_server/sandbox/sandbox_spec_service.py` +- **Line:** `AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:-python'` +- **Format:** `-python` for releases (e.g., `1.12.0-python`), `<7-char-commit-hash>-python` for dev pins +- **Source of truth** for which agent-server image the app server pulls at runtime +- **⚠️ Gotcha:** When pinning to an SDK PR, the image tag is the **merge-commit SHA** from GitHub Actions, not the PR head-commit SHA. Check the SDK PR description or CI logs for the correct tag. + +### `docker-compose.yml` +- **Lines:** + ```yaml + - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server} + - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:--python} + ``` +- Used by `docker compose up` for local development + +### `containers/dev/compose.yml` +- **Lines:** + ```yaml + - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server} + - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:--python} + ``` +- Used by the dev container setup +- **Known issue:** On main as of 1.4.0, this file still points to `ghcr.io/openhands/runtime` instead of `agent-server`, and the tag is `1.2-nikolaik` (stale from the V0 era). The `check-version-consistency.yml` CI workflow catches this. + +## Updated During Release Commit (version string only) + +### `pyproject.toml` +- **Line:** `version = "X.Y.Z"` under `[tool.poetry]` +- The Python version is derived from this at runtime via `openhands/version.py` + +### `frontend/package.json` +- **Line:** `"version": "X.Y.Z"` + +### `frontend/package-lock.json` +- **Two places:** root `"version": "X.Y.Z"` and `packages[""].version` + +## Dynamic References (auto-derived, no manual update) + +### `openhands/version.py` +- Reads version from `pyproject.toml` at runtime → `openhands.__version__` + +### `openhands/resolver/issue_resolver.py` +- Builds `ghcr.io/openhands/runtime:{openhands.__version__}-nikolaik` dynamically + +### `openhands/runtime/utils/runtime_build.py` +- Base repo URL `ghcr.io/openhands/runtime` is a constant; version comes from elsewhere + +### `.github/scripts/update_pr_description.sh` +- Uses `${SHORT_SHA}` variable at CI runtime, not hardcoded + +### `enterprise/Dockerfile` +- `ARG BASE="ghcr.io/openhands/openhands"` — base image, version supplied at build time + +## V0 Legacy Files (separate update cadence) + +These reference the V0 runtime image (`ghcr.io/openhands/runtime:X.Y-nikolaik`) for local Docker/Kubernetes paths. They are **not** updated as part of a V1 release but may be updated independently. + +### `Development.md` +- `export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/openhands/runtime:X.Y-nikolaik` + +### `openhands/runtime/impl/kubernetes/README.md` +- `runtime_container_image = "docker.openhands.dev/openhands/runtime:X.Y-nikolaik"` + +### `enterprise/enterprise_local/README.md` +- Uses `ghcr.io/openhands/runtime:main-nikolaik` (points to `main`, not versioned) + +### `third_party/runtime/impl/daytona/README.md` +- Uses `${OPENHANDS_VERSION}` variable, not hardcoded + +## Image Registries + +| Registry | Usage | +|----------|-------| +| `ghcr.io/openhands/agent-server` | V1 agent-server (sandbox) — built by SDK repo CI | +| `ghcr.io/openhands/openhands` | Main app image — built by `ghcr-build.yml` | +| `ghcr.io/openhands/runtime` | V0 runtime sandbox — built by `ghcr-build.yml` | +| `docker.openhands.dev/openhands/*` | Mirror/CDN for the above images | diff --git a/.agents/skills/update-sdk/references/sdk-pinning-examples.md b/.agents/skills/update-sdk/references/sdk-pinning-examples.md new file mode 100644 index 0000000000..8cb036a72d --- /dev/null +++ b/.agents/skills/update-sdk/references/sdk-pinning-examples.md @@ -0,0 +1,103 @@ +# SDK Pinning Examples + +Examples from real commits showing how to pin SDK packages to unreleased commits, branches, or released versions. + +## Pin to a Specific Commit + +Example from commit `169fb76` (pinning all 3 packages to SDK commit `100e9af`): + +### `dependencies` array (PEP 508 format) + +```toml +"openhands-agent-server @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-agent-server", +"openhands-sdk @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-sdk", +"openhands-tools @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-tools", +``` + +### `[tool.poetry.dependencies]` (Poetry format) + +```toml +openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-sdk" } +openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-agent-server" } +openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-tools" } +``` + +### `openhands/app_server/sandbox/sandbox_spec_service.py` + +```python +AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:-python' +``` + +**⚠️ Important:** The image tag is the **merge-commit SHA** from the SDK CI, not the commit hash used in `pyproject.toml`. Look up the correct tag from the SDK PR description or CI logs. + +## Pin to a Branch + +Example from commit `430ee1c` (pinning to branch `openhands/issue-2228-sdk-settings-schema`): + +### `[tool.poetry.dependencies]` + +```toml +openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-sdk" } +openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-agent-server" } +openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-tools" } +``` + +## Using `[tool.uv.sources]` Override + +When only `uv` needs the override (keep PyPI versions in the main arrays), add a `[tool.uv.sources]` section. Example from commit `1daca49`: + +```toml +[tool.uv.sources] +openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-sdk", rev = "4170cca" } +openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-agent-server", rev = "4170cca" } +openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-tools", rev = "4170cca" } +``` + +## Released PyPI Version (standard release) + +Example from commit `929dcc3` (SDK 1.11.5): + +### `dependencies` array + +```toml +"openhands-agent-server==1.11.5", +"openhands-sdk==1.11.5", +"openhands-tools==1.11.5", +``` + +### `[tool.poetry.dependencies]` + +```toml +openhands-sdk = "1.11.5" +openhands-agent-server = "1.11.5" +openhands-tools = "1.11.5" +``` + +### `openhands/app_server/sandbox/sandbox_spec_service.py` + +For released versions, the image tag uses the version number: + +```python +AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:1.11.5-python' +``` + +However, **some releases use a commit-hash tag** even for the released version. Check which tag format exists on GHCR. Example from `929dcc3`: + +```python +AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:010e847-python' +``` + +## Regenerate Lock Files + +After any change to `pyproject.toml`, always regenerate: + +```bash +poetry lock +uv lock +cd enterprise && poetry lock && cd .. +``` + +## CI Guards + +- **`check-package-versions.yml`**: Blocks merge to `main` if `[tool.poetry.dependencies]` contains `rev` fields (prevents shipping unreleased SDK pins) +- **`check-version-consistency.yml`**: Validates version strings match across `pyproject.toml`, `package.json`, `package-lock.json`, and verifies compose files use `agent-server` images diff --git a/.github/workflows/check-version-consistency.yml b/.github/workflows/check-version-consistency.yml new file mode 100644 index 0000000000..7df1a1764a --- /dev/null +++ b/.github/workflows/check-version-consistency.yml @@ -0,0 +1,122 @@ +name: Check Version Consistency + +on: + push: + branches: [main] + pull_request: + workflow_dispatch: + +jobs: + check-version-consistency: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: "3.12" + + - name: Check version and Docker image tag consistency + run: | + python - <<'PY' + import json + import re + import sys + import tomllib + + errors = [] + warnings = [] + + # ── 1. Extract the canonical version from pyproject.toml ────────── + with open("pyproject.toml", "rb") as f: + pyproject = tomllib.load(f) + version = pyproject["tool"]["poetry"]["version"] + major_minor = ".".join(version.split(".")[:2]) + print(f"📦 pyproject.toml version: {version} (major.minor: {major_minor})") + + # ── 2. Check frontend/package.json ──────────────────────────────── + with open("frontend/package.json") as f: + pkg = json.load(f) + if pkg["version"] != version: + errors.append( + f"frontend/package.json version is '{pkg['version']}', expected '{version}'" + ) + else: + print(f" ✔ frontend/package.json: {pkg['version']}") + + # ── 3. Check frontend/package-lock.json (2 places) ─────────────── + with open("frontend/package-lock.json") as f: + lock = json.load(f) + for key, val in [ + ("root.version", lock.get("version")), + ('packages[""].version', lock.get("packages", {}).get("", {}).get("version")), + ]: + if val != version: + errors.append( + f"frontend/package-lock.json {key} is '{val}', expected '{version}'" + ) + else: + print(f" ✔ frontend/package-lock.json {key}: {val}") + + # ── 4. Check compose files use agent-server images ───────────────── + # Both compose files should use ghcr.io/.../agent-server (not runtime). + # Agent-server tags use SDK version (e.g. "1.12.0-python") or commit + # hashes (e.g. "31536c8-python") — both are acceptable. + repo_pattern = re.compile(r"AGENT_SERVER_IMAGE_REPOSITORY[^}]*:-([^}]+)") + tag_pattern = re.compile(r"AGENT_SERVER_IMAGE_TAG:-([^}]+)") + + for filepath in ["docker-compose.yml", "containers/dev/compose.yml"]: + try: + with open(filepath) as f: + content = f.read() + except FileNotFoundError: + warnings.append(f"{filepath}: file not found") + continue + + repos = repo_pattern.findall(content) + tags = tag_pattern.findall(content) + + if not repos: + warnings.append(f"{filepath}: no AGENT_SERVER_IMAGE_REPOSITORY default found") + else: + repo = repos[0] + if "agent-server" not in repo: + errors.append( + f"{filepath}: AGENT_SERVER_IMAGE_REPOSITORY defaults to '{repo}', " + f"expected an agent-server image (not runtime)" + ) + else: + print(f" ✔ {filepath} image repository: {repo}") + + if not tags: + warnings.append(f"{filepath}: no AGENT_SERVER_IMAGE_TAG default found") + else: + tag = tags[0] + if not tag: + errors.append(f"{filepath}: AGENT_SERVER_IMAGE_TAG default is empty") + else: + print(f" ✔ {filepath} image tag: {tag}") + + # ── 5. Report ───────────────────────────────────────────────────── + print() + if warnings: + print("⚠ Warnings:") + for w in warnings: + print(f" {w}") + print() + + if errors: + print("❌ FAILED: Version inconsistencies found:\n") + for e in errors: + print(f" ✖ {e}") + print( + "\nAll version numbers and Docker image tags must be consistent." + "\nSee .agents/skills/update-sdk/SKILL.md for the full checklist." + ) + sys.exit(1) + else: + print("✅ All version numbers and Docker image tags are consistent.") + PY diff --git a/containers/dev/compose.yml b/containers/dev/compose.yml index 1adb7e6b3f..66b72e8a20 100644 --- a/containers/dev/compose.yml +++ b/containers/dev/compose.yml @@ -12,8 +12,8 @@ services: - SANDBOX_API_HOSTNAME=host.docker.internal - DOCKER_HOST_ADDR=host.docker.internal # - - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/runtime} - - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.2-nikolaik} + - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server} + - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.12.0-python} - SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} - WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace} ports: diff --git a/docker-compose.yml b/docker-compose.yml index 3fec30e057..089ed69c9a 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -8,7 +8,7 @@ services: container_name: openhands-app-${DATE:-} environment: - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server} - - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-31536c8-python} + - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.12.0-python} #- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} # enable this only if you want a specific non-root sandbox user but you will have to manually adjust permissions of ~/.openhands for this user - WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace} ports: