Files
OpenHands/playwright.live.config.ts
Rohit Malhotra 14b1b1e8ad feat: two auth modes — local (auto-key) and public (paste-key) (#790)
* feat: two auth modes — local (auto-key) and public (paste-key)

Local mode (agent-canvas, no flags):
- Ingress binds to 127.0.0.1 only
- Auto-generates session API key
- Writes /backends.json to static dir so frontend auto-authenticates
- Zero setup for localhost use

Public mode (agent-canvas --public):
- Ingress binds to 0.0.0.0 (all interfaces)
- Requires LOCAL_BACKEND_API_KEY env var
- Does NOT write /backends.json
- Frontend shows API key entry screen on 401

Co-authored-by: openhands <openhands@all-hands.dev>

* refactor: reuse BackendForm in public-mode API key entry screen

Replace the bespoke ApiKeyEntryScreen form with BackendForm configured
for the public-auth use case:

- Host field is auto-filled from window.location.origin and read-only
- Name field is hidden (auto-derived from the existing backend)
- Only the API key input is exposed to the user
- Uses the same SettingsInput / BrandButton components as the backend
  connection modals for visual consistency

BackendForm gains three optional props to support this:
- hideName: hides the name input and uses a fallback name
- hostReadOnly: disables the host input
- onSubmitPayload: receives the submitted payload for side-effects
  (the API key screen uses it to persist to agent-server-config and
  reload the page)

Co-authored-by: openhands <openhands@all-hands.dev>

* docs: update AGENTS.md with ApiKeyEntryScreen BackendForm reuse details

Co-authored-by: openhands <openhands@all-hands.dev>

* feat: implement public mode auth flow (--public flag)

- Add --public flag to dev-with-automation.mjs and bin/agent-canvas.mjs
- In public mode: require LOCAL_BACKEND_API_KEY, use as session key,
  don't bake into frontend (no VITE_SESSION_API_KEY / --session-api-key)
- Add isAgentServerAuthError() to detect 401 from /server_info probe
- root.tsx shows ApiKeyEntryScreen when 401 detected (lazy loaded)
- useConfig skips retries on 401 for instant auth screen display
- ApiKeyEntryScreen now has default export for React.lazy compatibility

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: use VITE_AUTH_REQUIRED flag instead of 401 detection for public mode

The 401-based approach was unreliable — /server_info may not require
auth on all server versions. Instead:

- dev-with-automation.mjs sets VITE_AUTH_REQUIRED=true in public mode
- isAuthRequiredAndMissing() checks the flag + localStorage for a key
- root.tsx gates on the flag BEFORE the /server_info probe, so the
  auth screen appears instantly with zero network round-trips
- 401 fallback kept as safety net for edge cases

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: handle stale key via 401 detection in public mode

When the server restarts with a new LOCAL_BACKEND_API_KEY, the browser
still has the old key in localStorage. isAuthRequiredAndMissing() returns
false (key exists), so the /server_info probe fires and 401s.

isAgentServerAuthError() now checks VITE_AUTH_REQUIRED=true AND 401
status, so it only triggers in public mode (a 401 in local mode is a
misconfiguration, not a key-rotation event). useConfig skips retries
on 401 to show the auth screen immediately.

Two gates, one screen:
- No key at all → flag check, instant, no network
- Stale key → /server_info 401, one round-trip

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: validate stale keys against GET /api/settings (protected)

/server_info is unprotected — it returns 200 even with a wrong key.
In public mode, after the /server_info probe succeeds, we now hit
GET /api/settings to verify the stored key is still valid. A 401
from that endpoint triggers the auth screen via isAgentServerAuthError().

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: rewrite ApiKeyEntryScreen — validate before save, always empty key, match add-modal UI

Three fixes:

1. Stale key conflict: The form now always starts with an empty API key
   field instead of pre-filling from the backend registry. Stale
   credentials from a previous session never bleed into the input.

2. Wrong key indicator: On submit, the key is validated against
   GET /api/settings (protected endpoint) BEFORE persisting. Wrong
   keys show an inline red status dot + 'Invalid API key' error
   via BackendStatusDot. Only validated keys trigger the reload.

3. UI parity with add-backend modal: Replaced BackendForm wrapper
   with direct SettingsInput fields matching ManualConnectionColumn's
   layout — host (read-only + helper text), API key (password with
   placeholder), status indicator, and Connect button. No cloud
   OAuth column.

New i18n key: AUTH$INVALID_KEY (all 15 languages).

Co-authored-by: openhands <openhands@all-hands.dev>

* feat: match add-backend modal UI + add test coverage

ApiKeyEntryScreen now renders the exact same card chrome as
BackendFormModal add-mode: same title ('Add a Backend'), same
Name/Host/API Key fields, same Connect button styling. Host is
pre-filled and read-only; no cloud OAuth column.

New tests (12 total):
- api-key-entry-screen.test.tsx (7 tests):
  - UI field parity with add-backend modal
  - Stale key wipe (empty API key field despite stale localStorage)
  - Connect disabled until name + key filled
  - Valid key: validates → persists → reloads
  - Invalid key: error indicator, no persist, no reload
  - Retry flow: wrong key → error → correct key → success
  - Stale key isolation: only fresh key persisted
- agent-server-config.test.ts (5 new tests for isAuthRequiredAndMissing):
  - Flag unset → false
  - Flag set, no key → true
  - Flag set, localStorage key → false
  - Flag set, VITE_SESSION_API_KEY → false
  - Flag not 'true' → false

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: distinguish 401 from other errors in ApiKeyEntryScreen

The catch-all was showing 'Invalid API key' for EVERY failure —
including 500s, network errors, and timeouts — even when the key
was correct. Now:

- 401 → 'Invalid API key. Please check the key and try again.'
- Anything else → 'Connection failed: <actual error message>'

This reveals the real problem when a correct key fails for a
non-auth reason (e.g. server misconfiguration, missing OH_SECRET_KEY).

New i18n key: AUTH$CONNECTION_FAILED (all 15 languages).
New test: non-401 errors show 'Connection failed' + detail.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: agent-server receives wrong session key in public mode

startAgentServer() called buildSafeDevConfig() which generated its
own random session key, ignoring config.sessionApiKey (which holds
LOCAL_BACKEND_API_KEY in public mode). The agent-server was started
with a random key while users were told to paste the LOCAL_BACKEND_API_KEY
value — every key was rejected with 401.

Fix: override OH_SESSION_API_KEYS_0 in the agent-server env with
config.sessionApiKey so both the agent-server and the frontend
agree on which key is valid.

Co-authored-by: openhands <openhands@all-hands.dev>

* refactor: address review comments on ApiKeyEntryScreen

1. Remove dead BackendFormProps (hideName, hostReadOnly, onSubmitPayload)
   — ApiKeyEntryScreen is standalone so no caller used these props.

2. Auto-generate backend name from window.location.hostname instead of
   requiring users to type one. Only the API key field is required now,
   reducing public-mode auth to a single-field flow.

3. Simplify redundant ternary: connectionStatus === 'success' ? true : false
   → connectionStatus === 'success'.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: address second round of review comments

1. Use shared isSdkHttpError() helper in ApiKeyEntryScreen instead of
   duplicating the SDK error shape check inline. Exported the helper
   from agent-server-compatibility.ts.

2. Add code comment acknowledging the edge case where a network hiccup
   between /server_info and getSettings() probes lets the app load with
   an unvalidated key. Acceptable since the window is narrow and a page
   refresh recovers.

3. Add --auth-required flag to static-server.mjs so pre-built static
   binaries (npx @openhands/agent-canvas --public) show the API key
   entry screen without needing VITE_AUTH_REQUIRED baked in at build
   time. The flag injects window.__AGENT_CANVAS_AUTH_REQUIRED__=true
   into index.html at runtime. Frontend isAuthRequired() checks both
   the build-time env var and the runtime window flag.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: use double cast (unknown) to satisfy strict TS on window flag access

window cannot be cast directly to Record<string, unknown> — TypeScript
requires going through unknown first for unrelated types.

  (window as unknown as Record<string, unknown>).__AGENT_CANVAS_AUTH_REQUIRED__

This fixes the CI typecheck failure introduced in aa76c01a.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: address remaining review comments on auth modes PR

- Use isAuthRequired() instead of raw import.meta.env.VITE_AUTH_REQUIRED
  in isAgentServerAuthError() so the runtime window flag injected by
  static-server.mjs in pre-built binaries is also honoured (bug fix).

- Preserve existing backend name during re-authentication flow in
  ApiKeyEntryScreen; only fall back to window.location.hostname for
  the initial entry so users don't lose custom labels on key rotation.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: restore MCP-to-integrations migration from main

A prior merge into this branch incorrectly kept the old
@openhands/extensions/mcps imports instead of the
@openhands/extensions/integrations paths introduced by d41bfe15
on main. Restore all affected files from origin/main so the
extensions package (which no longer exports ./mcps) resolves
correctly.

Files restored from main:
- src/utils/mcp-marketplace-utils.ts
- src/routes/mcp.tsx
- src/components/features/mcp-logo-badge.tsx
- src/components/features/mcp-page/* (6 files)
- src/components/features/automations/* (2 files)
- __tests__/ (4 test files)

Co-authored-by: openhands <openhands@all-hands.dev>

* style: fix prettier formatting and remove unused eslint-disable in api-key-entry-screen

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: address remaining PR review comments

- Extract isSdkHttpStatusError() helper in agent-server-compatibility.ts
  to DRY up the SDK error status check (review comment #3321202503).
  Both isAgentServerAuthError() and ApiKeyEntryScreen now use it.

- Use AUTH i18n keys in api-key-entry-screen.tsx:
  • Heading: AUTH$API_KEY_REQUIRED_TITLE ('API Key Required')
  • Description: AUTH$API_KEY_REQUIRED_DESCRIPTION added below heading
  • Button: AUTH$CONNECT ('Connect')
  (review comments #3325478721, #3325478729)

- Fix nested <main> landmark in root.tsx: remove the outer <main>
  wrapper since ApiKeyEntryScreen already provides its own semantic
  container (review comment #3325478704).

- Change ApiKeyEntryScreen root element from <main> to <div> so
  the Layout's own landmarks are not violated.

Co-authored-by: openhands <openhands@all-hands.dev>

* test: add coverage for window.__AGENT_CANVAS_AUTH_REQUIRED__ runtime flag

Add isAuthRequired() test block covering the window flag path used by
pre-built static binaries (static-server.mjs --auth-required). Also add
window-flag variants to isAuthRequiredAndMissing() tests.

Addresses review comment #3325587577.

Co-authored-by: openhands <openhands@all-hands.dev>

* refactor!: deduplicate SESSION_API_KEY into LOCAL_BACKEND_API_KEY

BREAKING CHANGE: The user-facing env var for setting the API key is now
`LOCAL_BACKEND_API_KEY` everywhere. The old `SESSION_API_KEY`,
`OH_SESSION_API_KEYS_0`, and `VITE_SESSION_API_KEY` env vars are no
longer read by launchers as user-facing configuration.

Internal plumbing (`config.sessionApiKey`, `VITE_SESSION_API_KEY` build
injection, `OH_SESSION_API_KEYS_0` agent-server env) is unchanged — only
the user-facing surface is unified into a single env var.

Changes:
- scripts/dev-safe.mjs: read LOCAL_BACKEND_API_KEY instead of
  SESSION_API_KEY / OH_SESSION_API_KEYS_0 / VITE_SESSION_API_KEY
- scripts/dev-with-automation.mjs: unify key resolution through
  buildSafeDevConfig for both public and local modes
- bin/agent-canvas.mjs: update CLI help text and examples
- docker/entrypoint.sh: read LOCAL_BACKEND_API_KEY, migrate legacy
  session-api-key.txt → api-key.txt
- scripts/static-server.mjs: add mutual-exclusion guard for
  --session-api-key + --auth-required flags
- playwright configs: pass LOCAL_BACKEND_API_KEY instead of the old trio
- test helpers: prefer LOCAL_BACKEND_API_KEY fallback chain
- Update tests and documentation

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: address review comments — narrow settings probe rethrow and use shared client options

- loadAgentServerInfo: narrow getSettings() catch to rethrow only 401
  errors. Other HTTP errors (403, 5xx) and non-HTTP errors (network,
  timeout) are now swallowed with a console.warn, since the server is
  confirmed up (via /server_info) and the probe is best-effort. This
  prevents misconfigured servers from silently falling through to
  <Outlet /> without showing either the auth or unavailable screen.

- ApiKeyEntryScreen: replace hand-rolled SettingsClient options with
  getAgentServerClientOptions() so transport-level settings (e.g.
  VITE_INSECURE_SKIP_VERIFY) are honoured. Uses the sessionApiKey
  override to pass the freshly-entered key.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: rewrite git+ssh to git+https for @openhands/extensions in lockfile

npm normalizes GitHub URLs to git+ssh:// in the lockfile, but machines
without SSH keys for GitHub (or with stale npm caches) can end up
installing a wrong version of the package. This causes the Vite resolve
error:

  "./integrations" is not exported under the conditions [...]

The same pattern was already fixed for @openhands/typescript-client
(see #384). vercel-install.sh already does a blanket sed rewrite, but
the committed lockfile itself should use git+https:// so local npm ci
works without SSH keys.

Co-authored-by: openhands <openhands@all-hands.dev>

* refactor: align public auth screen with Add Backend form layout

Replace the custom 'API Key Required' screen with the same form
layout used by the 'Add a Backend' left column in BackendFormModal:

- Heading changed from 'API Key Required' to 'Add a Backend'
- Added backend Name field (required, same as ManualConnectionColumn)
- Host field remains pre-filled and disabled (from window.location.origin)
- API Key field unchanged
- Submit button now uses BACKEND$CONNECT label (matching the modal)
- Removed the subtitle description paragraph for cleaner parity
- Name is persisted to the backend registry on submit

Tests updated: fillApiKey → fillRequiredFields (name + apiKey),
assertions cover the new name field and dual-field submit gating.

Co-authored-by: openhands <openhands@all-hands.dev>

* chore: remove unused AUTH$ i18n keys from this PR

The UI refactor (02faa0b0) switched ApiKeyEntryScreen to the
BACKEND$* keys. Drop the three AUTH$ entries that were introduced
and then superseded within this same PR:

- AUTH$API_KEY_REQUIRED_TITLE
- AUTH$API_KEY_REQUIRED_DESCRIPTION
- AUTH$CONNECT

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: sync stale session API key on boot when LOCAL_BACKEND_API_KEY changes

When a user restarts the stack with a different LOCAL_BACKEND_API_KEY,
the new VITE_SESSION_API_KEY is baked in correctly, but localStorage
may still hold the old key in two places:

  1. openhands-agent-server-config.sessionApiKey (written by onboarding
     or the Settings page)
  2. openhands-backends[].apiKey (seeded on first load, never re-synced)

The existing syncDefaultLocalBackendAuth() in storage.ts already tries
to fix #2 by comparing against makeDefaultLocalBackend(), but that
function reads through getConfiguredSessionApiKey() which hits #1
(stale localStorage) before falling back to VITE_SESSION_API_KEY.
So a stale #1 defeats the #2 sync.

Fix: add syncBakedSessionApiKey() which runs from readStoredBackends()
before any key resolution. When VITE_SESSION_API_KEY is set and the
stored key in openhands-agent-server-config differs, overwrite it.
This ensures getConfiguredSessionApiKey() and makeDefaultLocalBackend()
both return the correct key, and the downstream backend-registry sync
works as intended.

Also fix the static-server.mjs injection script to always overwrite a
stored key that differs from the runtime key (was guarded by
`if(!_c.sessionApiKey)` which skipped updates when any key existed).

Add mock-LLM E2E tests for:
- Key rotation recovery: seeds stale localStorage, verifies app loads
- Public-mode auth gate: tests auth screen visibility, wrong key
  rejection, and correct key acceptance

Co-authored-by: openhands <openhands@all-hands.dev>

* docs: document key rotation resilience in AGENTS.md

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: add syncBakedSessionApiKey to vi.mock stubs and use click-then-fill in E2E

Three test files mock #/api/agent-server-config without exporting
syncBakedSessionApiKey, which storage.ts now calls at import time.
Add the missing vi.fn() stub to all three.

Also fix the public-mode auth E2E test: use the click() → fill()
pattern for React controlled inputs (matching the established
convention in mock-llm-conversation.spec.ts) so the SettingsInput
onChange fires reliably in Playwright.

Co-authored-by: openhands <openhands@all-hands.dev>

* test: add public-mode key rotation E2E test

Simulates a server key rotation: localStorage holds a stale key from
a previous session, the server now has a new key. Verifies the app
detects the 401 from the stale key probe, shows the auth screen, and
accepts the new key.

Flow: stale key in localStorage → probe /server_info → 401 →
isAgentServerAuthError → ApiKeyEntryScreen → user pastes new key →
reload → app loads normally.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix(e2e): suppress consent modal in public-mode auth tests

The analytics consent modal overlays the auth screen on first visit
(clean localStorage). Playwright's click() on the form inputs was
intercepted by the modal overlay, causing a 60s timeout loop (121
retries). Add a beforeEach that seeds 'analytics-consent' and
'openhands-telemetry-consent' in localStorage before navigation.

Also deduplicate the consent seeding from the key-rotation test's
addInitScript since the beforeEach now handles it.

Co-authored-by: openhands <openhands@all-hands.dev>

* refactor: deduplicate readStoredConfig() call in syncBakedSessionApiKey

Capture the first readStoredConfig() result and reuse it in the
spread instead of hitting localStorage twice.

Co-authored-by: openhands <openhands@all-hands.dev>

* chore(docker): remove legacy session-api-key.txt migration

The backwards-compatibility shim that migrated the old
session-api-key.txt to api-key.txt is no longer needed — a breaking
change here is acceptable.

Co-authored-by: openhands <openhands@all-hands.dev>

* refactor: dedup ApiKeyEntryScreen against BackendForm

ApiKeyEntryScreen now renders BackendForm with three new props instead
of reimplementing the name/host/API-key inputs from scratch:

- hostReadOnly: locks the host field (pre-filled from window.origin)
- requireApiKey: forces a non-empty API key for local backends
- onSubmitOverride: replaces the default sync persist with async
  server-side validation (GET /api/settings) before persisting

The auth-gate-specific chrome (full-screen wrapper, connection status
indicator, validating/error state) stays in ApiKeyEntryScreen via the
existing renderActions slot.

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: chuckbutkus <chuck@openhands.dev>
2026-06-01 12:02:37 -06:00

75 lines
2.5 KiB
TypeScript

import { defineConfig, devices } from "@playwright/test";
import { randomBytes } from "node:crypto";
const configuredLiveE2ESessionApiKey =
process.env.LIVE_E2E_SESSION_API_KEY?.trim();
const liveE2ESessionApiKey =
configuredLiveE2ESessionApiKey || randomBytes(32).toString("hex");
process.env.LIVE_E2E_SESSION_API_KEY = liveE2ESessionApiKey;
const liveE2EFrontendPort = process.env.LIVE_E2E_FRONTEND_PORT ?? "3101";
const liveE2EBackendURL =
process.env.LIVE_E2E_BACKEND_URL ?? "http://127.0.0.1:18100";
let liveE2EBackendPort: string;
try {
liveE2EBackendPort = new URL(liveE2EBackendURL).port || "18100";
} catch {
throw new Error("Invalid LIVE_E2E_BACKEND_URL. Expected an absolute URL.");
}
const liveE2EFrontendURL = `http://localhost:${liveE2EFrontendPort}/`;
const liveE2EVideoMode =
process.env.LIVE_E2E_RECORD_VIDEO === "on" ? "on" : "retain-on-failure";
function shellQuote(value: string) {
return `'${value.replaceAll("'", "'\\''")}'`;
}
function envAssignment(name: string, value: string) {
return `${name}=${shellQuote(value)}`;
}
export default defineConfig({
testDir: "./tests/e2e/live",
testMatch: /.*\.spec\.ts/,
fullyParallel: false,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 1 : 0,
workers: 1,
timeout: 180_000,
reporter: [
["line"],
["json", { outputFile: "test-results-live/results.json" }],
["html", { outputFolder: "playwright-report-live", open: "never" }],
],
outputDir: "test-results-live",
use: {
baseURL: liveE2EFrontendURL,
screenshot: "only-on-failure",
trace: "off",
video: liveE2EVideoMode,
},
projects: [
{
name: "chromium",
use: { ...devices["Desktop Chrome"] },
},
],
// This live stack is intended for macOS/Linux shells and Ubuntu CI runners;
// the command below intentionally uses POSIX environment assignment syntax.
webServer: {
command:
"node -e \"const fs=require('node:fs'); for (const p of ['.tmp/live-e2e-state','node_modules/.vite']) fs.rmSync(p,{recursive:true,force:true});\" && " +
[
"OH_CANVAS_SAFE_STATE_DIR=.tmp/live-e2e-state",
envAssignment("LOCAL_BACKEND_API_KEY", liveE2ESessionApiKey),
envAssignment("OH_CANVAS_SAFE_BACKEND_PORT", liveE2EBackendPort),
"VITE_DO_NOT_TRACK=1",
"VITE_ENABLE_BROWSER_TOOLS=false",
envAssignment("VITE_FRONTEND_PORT", liveE2EFrontendPort),
"npm run dev:minimal",
].join(" "),
url: liveE2EFrontendURL,
timeout: 120_000,
reuseExistingServer: false,
},
});