mirror of
https://github.com/OpenHands/OpenHands.git
synced 2026-10-07 12:58:49 +08:00
fix: bind static-server dual-stack to fix Docker E2E ECONNREFUSED on ::1 (#1032)
* fix: bind static-server dual-stack to fix Docker E2E ECONNREFUSED on ::1 The Docker mock-LLM E2E tests were flaky because static-server.mjs bound to 0.0.0.0 (IPv4 only), but Playwright and Chromium often resolve `localhost` to ::1 (IPv6) on Ubuntu CI runners, causing intermittent ECONNREFUSED. The non-Docker tests didn't have this problem because they go through ingress.mjs, which calls server.listen(port) without a host argument — Node.js defaults to :: (dual-stack: IPv4 + IPv6). Changes: - static-server.mjs: default host from "0.0.0.0" to null; when null, call server.listen(port) without host so Node binds to :: - docker/entrypoint.sh: drop --host 0.0.0.0 from both static-server invocations so they use the dual-stack default - playwright.mock-llm.config.ts: drop --host 0.0.0.0 from the public-mode static server (tests hit it directly via localhost) Callers behind ingress.mjs (dev-with-automation, dev-static) still pass --host 0.0.0.0 explicitly, which is fine since the ingress itself is already dual-stack. Co-authored-by: openhands <openhands@all-hands.dev> * fix: use 127.0.0.1 in Docker E2E config to avoid IPv6 ECONNREFUSED The Docker mock-LLM E2E tests fail with ECONNREFUSED ::1:18300 because Playwright/Chromium resolve `localhost` to ::1 (IPv6) on Ubuntu CI runners, but the Docker container's static-server binds to 0.0.0.0 (IPv4 only). The non-Docker tests don't have this issue because they go through ingress.mjs which binds to :: (dual-stack: IPv4 + IPv6). Rather than changing the server binding (which could have side-effects inside the Docker container), this fix changes the Docker Playwright config to use 127.0.0.1 directly for all URLs: INGRESS_URL, MOCK_LLM_BACKEND_URL, MOCK_LLM_PUBLIC_MODE_URL, and the webServer health-check probe. This bypasses DNS resolution entirely and connects via IPv4. Co-authored-by: openhands <openhands@all-hands.dev> * fix: keep Docker container alive when a backend service exits The Docker entrypoint used `wait -n` which exits the entire container when ANY child process exits. After heavy automation tests (which spawn multiple conversations), the agent-server or automation backend could exit, taking down the static-server with it — causing ECONNREFUSED for subsequent tests. In the non-Docker path, each service is an independent host process, so one crashing doesn't affect the others. The ingress proxy returns 502 for the dead backend but stays up. Change the entrypoint to monitor children in a loop: log crashes but keep the container running as long as any service is still alive. Only exit when ALL tracked children are dead. The SIGTERM trap still handles clean shutdown. Co-authored-by: openhands <openhands@all-hands.dev> * fix: simplify entrypoint keep-alive to avoid wait -n interaction issues Replace the complex PID monitoring loop with a simple sleep loop. The previous wait -n based loop regressed automation tests (5/14 vs 8/14 on the simpler IPv4-only commit), likely due to bash wait -n signal handling interacting poorly with child processes. The sleep loop keeps the container alive indefinitely. The existing SIGTERM/SIGINT trap handles clean shutdown. Co-authored-by: openhands <openhands@all-hands.dev> * fix: address review feedback on entrypoint and docs - Wait on STATIC_PID only (not all children): the static-server is the critical ingress process. If it dies the container exits with a meaningful exit code. Backend crashes (agent-server, automation) are tolerated — the proxy returns 502. (Copilot feedback) - Add `exit 0` to cleanup(): ensures the script terminates after a SIGTERM-triggered trap return instead of falling through. The `wait` builtin is signal-interruptible so SIGTERM is processed immediately — no stale sleep blocking delivery. (all-hands-bot) - Update AGENTS.md to match the actual behavior (wait on static-server PID, not a monitoring loop or infinite sleep). (Copilot feedback) Co-authored-by: openhands <openhands@all-hands.dev> * fix: use signal-safe sleep-wait loop with static-server liveness check The bare `wait "$STATIC_PID"` approach failed the same way as the original `wait -n` (8/14 — conversation/model-switch tests get ECONNREFUSED after automation). The `while true; do sleep 86400; done` pattern from the previous commit was the only one that passed 14/14, but had two issues flagged in review: 1. Bare `sleep` as foreground blocks SIGTERM delivery (all-hands-bot) 2. Container stays alive forever even if static-server dies (Copilot) This commit addresses both: - `sleep 10 & wait $!` — `wait` (builtin) is the foreground op, so SIGTERM interrupts it immediately and the trap fires. - `while kill -0 "$STATIC_PID"` — loop exits when the critical ingress process dies; container exits with a meaningful code. - `cleanup()` keeps `exit 0` so the script terminates after a signal-triggered trap return. Co-authored-by: openhands <openhands@all-hands.dev> --------- Co-authored-by: openhands <openhands@all-hands.dev>
This commit is contained in:
committed by
GitHub
co-authored by
openhands
parent
dbc3abfed3
commit
e408deea9e
@@ -168,6 +168,8 @@ you are running inside of — NOT the automation backend.
|
||||
|
||||
- The same test specs and helpers are reused to validate the Docker image via `playwright.mock-llm-docker.config.ts`. Run locally with `npm run test:e2e:mock-llm:docker` (requires Docker daemon and a built image).
|
||||
- **Architecture**: The Docker config replaces the npm path's `bin/agent-canvas.mjs` webServer with a `docker run --network host` command. The mock LLM server still runs on the host. On Linux (including CI), `--network host` lets the container share the host's network stack so all `127.0.0.1` URLs work identically. On macOS/Windows Docker Desktop (bridge networking), set `MOCK_LLM_AGENT_URL=http://host.docker.internal:<port>` so the agent-server inside Docker can reach the host-side mock LLM server.
|
||||
- **IPv4 URLs in Docker tests**: `playwright.mock-llm-docker.config.ts` uses `127.0.0.1` (not `localhost`) for all URLs — `INGRESS_URL`, `MOCK_LLM_BACKEND_URL`, `MOCK_LLM_PUBLIC_MODE_URL`, and webServer health-check probes. The Docker entrypoint's `static-server.mjs` binds to `0.0.0.0` (IPv4 only), but `localhost` can resolve to `::1` (IPv6) on Ubuntu CI runners, causing `ECONNREFUSED`. The non-Docker config can use `localhost` because its `ingress.mjs` binds to `::` (dual-stack). If you add new URLs or health-check probes to the Docker Playwright config, always use `127.0.0.1`.
|
||||
- **Entrypoint crash resilience**: `docker/entrypoint.sh` uses a `while kill -0 "$STATIC_PID"; do sleep 10 & wait $!; done` loop instead of `wait -n "${PIDS[@]}"` (any child). If the agent-server or automation backend exits mid-test, the static-server proxy stays up and returns 502s for backend routes — the container doesn't disappear with `ECONNREFUSED`. The container exits only when the static-server (ingress) dies or on SIGTERM/SIGINT. The `sleep & wait $!` pattern ensures `wait` (a bash builtin) is the foreground op, so trapped signals fire immediately. `cleanup()` includes `exit 0` so the script terminates after a signal-triggered trap return.
|
||||
- **URL split**: `mock-llm-helpers.ts` exports two mock LLM URL constants:
|
||||
- `MOCK_LLM_BASE_URL` — always `http://127.0.0.1:<port>`, used by tests for the mock LLM admin API (register/activate/reset trajectories).
|
||||
- `MOCK_LLM_AGENT_URL` — defaults to `MOCK_LLM_BASE_URL`, overridable via `MOCK_LLM_AGENT_URL` env var. Used when configuring the LLM profile (`base_url` field) — this is the URL the agent-server uses for inference calls. The npm path and Docker-with-`--network host` path use the same value; Docker on macOS needs the override.
|
||||
|
||||
+19
-6
@@ -137,6 +137,7 @@ cleanup() {
|
||||
kill "$pid" 2>/dev/null || true
|
||||
done
|
||||
wait 2>/dev/null || true
|
||||
exit 0
|
||||
}
|
||||
trap cleanup EXIT SIGINT SIGTERM
|
||||
|
||||
@@ -243,7 +244,8 @@ node /opt/agent-canvas/static-server.mjs \
|
||||
--route "/docs=http://127.0.0.1:${AGENT_SERVER_PORT}" \
|
||||
--route "/redoc=http://127.0.0.1:${AGENT_SERVER_PORT}" \
|
||||
--route "/openapi.json=http://127.0.0.1:${AGENT_SERVER_PORT}" &
|
||||
PIDS+=($!)
|
||||
STATIC_PID=$!
|
||||
PIDS+=("$STATIC_PID")
|
||||
|
||||
# ── 5. (Optional) Public-mode static server ─────────────────────────────────
|
||||
# When PUBLIC_MODE_PORT is set, start a second static-server instance that
|
||||
@@ -272,8 +274,19 @@ fi
|
||||
|
||||
log "All services started. Unified entry point: http://0.0.0.0:${PORT}/"
|
||||
|
||||
# Wait for any child to exit. If one dies, the trap will clean up the rest.
|
||||
wait -n "${PIDS[@]}" 2>/dev/null
|
||||
EXIT_CODE=$?
|
||||
log_error "A service exited with code $EXIT_CODE"
|
||||
exit "$EXIT_CODE"
|
||||
# Keep the container alive while the static-server (ingress) is running.
|
||||
# Backend crashes (agent-server, automation) are tolerated — the proxy
|
||||
# returns 502 for downed routes, matching the non-Docker path where each
|
||||
# service is an independent host process.
|
||||
#
|
||||
# Pattern: `sleep & wait $!` makes `wait` (a bash builtin) the foreground
|
||||
# operation. Unlike a bare `sleep`, the builtin `wait` is interrupted
|
||||
# immediately when a trapped signal (SIGTERM/SIGINT) arrives, so cleanup()
|
||||
# fires without delay. cleanup() calls `exit 0` to terminate after the
|
||||
# trap returns. The loop re-checks the static-server PID every 10 s so the
|
||||
# container exits promptly if the ingress process dies on its own.
|
||||
while kill -0 "$STATIC_PID" 2>/dev/null; do
|
||||
sleep 10 & wait $!
|
||||
done
|
||||
log_error "Static server (PID $STATIC_PID) exited"
|
||||
exit 1
|
||||
|
||||
@@ -58,7 +58,14 @@ const sessionApiKey =
|
||||
process.env.MOCK_LLM_SESSION_API_KEY = sessionApiKey;
|
||||
|
||||
// ── URLs ───────────────────────────────────────────────────────────────
|
||||
const INGRESS_URL = `http://localhost:${INGRESS_PORT}/`;
|
||||
// Use 127.0.0.1 (not localhost) everywhere so Playwright and Chromium
|
||||
// connect via IPv4 directly. The Docker container's static-server binds
|
||||
// to 0.0.0.0 (IPv4 only). `localhost` can resolve to ::1 (IPv6) on CI
|
||||
// runners, causing intermittent ECONNREFUSED when the server doesn't
|
||||
// listen on IPv6. The non-Docker config uses `localhost` because its
|
||||
// ingress.mjs binds to :: (dual-stack), but the Docker entrypoint uses
|
||||
// static-server.mjs which is IPv4-only.
|
||||
const INGRESS_URL = `http://127.0.0.1:${INGRESS_PORT}/`;
|
||||
const MOCK_LLM_URL = `http://127.0.0.1:${MOCK_LLM_PORT}`;
|
||||
|
||||
// Python binary for the mock server — defaults to "python3" but CI can
|
||||
@@ -68,9 +75,9 @@ const MOCK_LLM_PYTHON = process.env.MOCK_LLM_PYTHON ?? "python3";
|
||||
|
||||
// Export for the test helpers — BACKEND_URL points to the ingress (API
|
||||
// calls are proxied to the agent-server, so no direct backend port needed).
|
||||
process.env.MOCK_LLM_BACKEND_URL = `http://localhost:${INGRESS_PORT}`;
|
||||
process.env.MOCK_LLM_BACKEND_URL = `http://127.0.0.1:${INGRESS_PORT}`;
|
||||
process.env.MOCK_LLM_PORT = MOCK_LLM_PORT;
|
||||
process.env.MOCK_LLM_PUBLIC_MODE_URL = `http://localhost:${PUBLIC_MODE_PORT}`;
|
||||
process.env.MOCK_LLM_PUBLIC_MODE_URL = `http://127.0.0.1:${PUBLIC_MODE_PORT}`;
|
||||
process.env.VITE_SESSION_API_KEY = sessionApiKey;
|
||||
|
||||
// MOCK_LLM_AGENT_URL — the URL the agent-server inside Docker uses to
|
||||
@@ -86,9 +93,7 @@ export default defineConfig({
|
||||
testMatch: /.*\.spec\.ts/,
|
||||
fullyParallel: false,
|
||||
forbidOnly: !!process.env.CI,
|
||||
// One retry for transient Docker container startup failures (ECONNREFUSED).
|
||||
// The container health-check (webServer.url) confirms the stack is up, but
|
||||
// occasional races can still cause the first request to fail.
|
||||
// One retry for transient Docker container startup failures.
|
||||
retries: process.env.CI ? 1 : 0,
|
||||
workers: 1,
|
||||
timeout: 60_000,
|
||||
@@ -163,7 +168,9 @@ export default defineConfig({
|
||||
// up before tests start. GET /api/automation/v1 returns 200 (empty
|
||||
// list) without auth — the automation backend does not enforce
|
||||
// session-key auth on the list endpoint.
|
||||
url: `http://localhost:${INGRESS_PORT}/api/automation/v1`,
|
||||
// Use 127.0.0.1 (not localhost) to avoid IPv6 resolution — see URL
|
||||
// comment block above.
|
||||
url: `http://127.0.0.1:${INGRESS_PORT}/api/automation/v1`,
|
||||
timeout: 180_000, // Docker pull + all services startup
|
||||
reuseExistingServer: !process.env.CI,
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user