fix: prevent snapshot CI runs from sending PostHog analytics events (#490)

* fix: separate PostHog keys for production and dev builds

The telemetry service (canvas_install events) was using a single
hardcoded PostHog key as a fallback in every build, so CI snapshot
tests sent events to the same project as real users, inflating the
unique-persons count and making install metrics unreliable.

Changes:

src/services/telemetry.ts
- POSTHOG_API_KEY is now string | null keyed off import.meta.env.PROD:
  - Production: VITE_POSTHOG_API_KEY || 'phc_REPLACE_WITH_PRODUCTION_KEY'
    (placeholder — swap in the real key before deploying)
  - Dev/test: VITE_POSTHOG_API_KEY || null
    PostHog never initialises when the key is null, so no events reach
    any PostHog project from dev servers or CI snapshot runs.
- initializePostHog() now returns null immediately when POSTHOG_API_KEY
  is null, before loading the posthog-js module at all.

src/mocks/analytics-handlers.ts
- Added https://z.openhands.dev/* intercept alongside the existing
  https://us.i.posthog.com/e handler. The library telemetry service
  routes through z.openhands.dev (the OpenHands reverse proxy), which
  MSW previously never saw.

Co-authored-by: openhands <openhands@all-hands.dev>

* feat: hardcode PostHog keys per deployment environment

Select the PostHog key based on hostname rather than a single constant or
env var override:
- app.all-hands.dev  → POSTHOG_PROD_KEY
- staging.app.all-hands.dev → POSTHOG_STAGING_KEY (shares prod key for
  now; swap to a dedicated project key when one is provisioned)
- all other origins   → null (no tracking)

Both the app-level PostHog (option-service / posthog-wrapper) and the
library telemetry service (telemetry.ts) follow the same pattern.
VITE_POSTHOG_API_KEY still works as an escape hatch for library consumers.

Drop VITE_DO_NOT_TRACK=1 from dev:mock: hostname detection already
returns a null key for localhost, so PostHog never initialises there.
Removing the flag also restores the TelemetryConsentBanner in snapshot
tests (the banner can be snapshotted correctly again).

Also adds PRODUCT_URL.STAGING to constants and a null guard in
initializePostHog for the key-null case.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: use prod PostHog project for now, drop placeholder staging URL

Remove the speculative staging.app.all-hands.dev hostname that doesn't
exist yet. Both POSTHOG_STAGING_KEY constants now alias POSTHOG_PROD_KEY
so staging events flow to the same project once the staging hostname is
wired up. The TODO comments mark exactly where to add the real staging
hostname and swap in a dedicated key.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: scope telemetry key to prod builds only, revert app-level PostHog changes

telemetry.ts: switch from hostname detection to import.meta.env.PROD as
the gate — this tracks local installs everywhere they are deployed, not
just on a specific hostname. Dev/test builds get null so no events reach
PostHog. Both POSTHOG_PROD_KEY and POSTHOG_STAGING_KEY are named
constants pointing at the same project for now; swap POSTHOG_STAGING_KEY
once a dedicated staging project exists.

option-service.api.ts: revert to posthog_client_key null. The app-level
PostHog wrapper belongs to the SaaS analytics path and is not relevant
for local install tracking.

constants.ts: drop the speculative STAGING URL addition.

Co-authored-by: openhands <openhands@all-hands.dev>

* feat: use VITE_APP_ENV to select staging vs prod PostHog key at bundle time

Both POSTHOG_PROD_KEY and POSTHOG_STAGING_KEY are now referenced in the
assignment. VITE_APP_ENV is a build-time constant: Vite replaces it with
a literal in the static bundle so the correct key is compiled in with no
runtime branching.

  VITE_APP_ENV=staging npm run build  → POSTHOG_STAGING_KEY
  (any other prod build)              → POSTHOG_PROD_KEY
  dev build (PROD=false)              → null

Set VITE_APP_ENV=staging in the staging deployment build config (Vercel
env vars, CI, etc.) to activate the staging key once it is provisioned.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: gate PostHog key on VITE_APP_ENV, not PROD

import.meta.env.PROD is true for any vite build output including
npm run dev (which runs a production static build via dev-docker.mjs),
so local developers would inadvertently compile in POSTHOG_PROD_KEY.

Switch to requiring VITE_APP_ENV to be explicitly set at bundle time:
  VITE_APP_ENV=production → POSTHOG_PROD_KEY
  VITE_APP_ENV=staging    → POSTHOG_STAGING_KEY
  unset (local dev, CI)   → null, PostHog never initialises

Document VITE_APP_ENV in .env.sample so it is visible to developers.

Co-authored-by: openhands <openhands@all-hands.dev>

* fix: bake VITE_APP_ENV=production into build:lib for npm releases

Without this, the published library bundle has POSTHOG_API_KEY=null
and library telemetry never fires for any consumer of the package.
A released npm package is a production artifact, so it should always
get POSTHOG_PROD_KEY compiled in.

Co-authored-by: openhands <openhands@all-hands.dev>

* refactor: simplify PostHog key to staging default, prod only when explicit

Drop the null/three-way branch. The key is now always a string:
- VITE_APP_ENV=production (hardcoded in build:lib and production CI) → POSTHOG_PROD_KEY
- everything else (local dev, CI, staging builds)                    → POSTHOG_STAGING_KEY

Since the key is never null, the null guard in initializePostHog is also removed.

Co-authored-by: openhands <openhands@all-hands.dev>

* Apply suggestions from code review

Co-authored-by: Rohit Malhotra <rohitvinodmalhotra@gmail.com>

---------

Co-authored-by: openhands <openhands@all-hands.dev>

.env.sample

@@ -16,6 +16,11 @@ VITE_FRONTEND_PORT="3001" # Port to run the frontend application
 VITE_USE_TLS="false" # Use HTTPS/WSS for proxied backend connections
 VITE_INSECURE_SKIP_VERIFY="false" # Skip TLS certificate verification for proxied backend requests
 
+# Deployment environment — controls which PostHog project key is compiled in.
+# Set to "production" or "staging" only in real deployment build pipelines.
+# Leave unset locally so developers never accidentally send telemetry.
+# VITE_APP_ENV="production"
+
 # Mocking / test helpers
 VITE_MOCK_API="false" # Enable/disable API mocking with MSW
 # VITE_GITHUB_TOKEN="" # GitHub token for repository access (used in some tests)

package.json

@@ -96,7 +96,7 @@
     "typecheck:staged": "react-router typegen && npx tsc --noEmit --skipLibCheck",
     "check-translation-completeness": "node scripts/check-translation-completeness.cjs",
     "build:app": "npm run make-i18n && react-router build",
-    "build:lib": "npm run make-i18n && react-router typegen && cross-env BUILD_LIB=true vite build && tsc -p tsconfig.lib.json"
+    "build:lib": "npm run make-i18n && react-router typegen && cross-env BUILD_LIB=true VITE_APP_ENV=production vite build && tsc -p tsconfig.lib.json"
   },
   "lint-staged": {
     "src/**/*.{ts,tsx,js}": [

src/mocks/analytics-handlers.ts

@@ -1,7 +1,13 @@
 import { http, HttpResponse } from "msw";
 
+// Block both the direct PostHog ingestion endpoint and the OpenHands reverse
+// proxy (z.openhands.dev) used by the library telemetry service so mock-mode
+// builds and snapshot test runs never send analytics events to PostHog.
 export const ANALYTICS_HANDLERS = [
   http.post("https://us.i.posthog.com/e", async () =>
     HttpResponse.json(null, { status: 200 }),
   ),
+  http.post("https://z.openhands.dev/*", async () =>
+    HttpResponse.json(null, { status: 200 }),
+  ),
 ];

src/services/telemetry.ts

@@ -32,12 +32,20 @@ const TELEMETRY_CONSENT_KEY = "openhands-telemetry-consent";
 const TELEMETRY_FIRST_USE_KEY = "openhands-telemetry-first-use";
 const TELEMETRY_SESSION_KEY = "openhands-telemetry-session";
 
-// PostHog configuration - configurable via env vars with OpenHands defaults
-// Note: The default API key sends telemetry to OpenHands' PostHog project.
-// Library consumers can override this with their own PostHog project key.
-const POSTHOG_API_KEY =
-  import.meta.env.VITE_POSTHOG_API_KEY ||
-  "phc_BgzfxKdgsYMLFTmJqt424ZoyVHvKFfrwttLimzdYTKFK";
+// PostHog project keys — one per deployment environment, hardcoded so they
+// are baked into the static bundle at build time and cannot drift at runtime.
+// Replace POSTHOG_STAGING_KEY with a dedicated project key once provisioned.
+const POSTHOG_PROD_KEY = "phc_BgzfxKdgsYMLFTmJqt424ZoyVHvKFfrwttLimzdYTKFK";
+const POSTHOG_STAGING_KEY = "phc_kBtz5nKmxVRRQ7HtPwr2QX9eMC5j65zE86QKocVNwb4U";
+
+// Always use the staging key unless VITE_APP_ENV is explicitly set to
+// "production" at bundle time (hardcoded in build:lib and production CI).
+// Library consumers can always override with VITE_POSTHOG_API_KEY.
+const POSTHOG_API_KEY: string =
+  (import.meta.env.VITE_POSTHOG_API_KEY as string | undefined) ||
+  (import.meta.env.VITE_APP_ENV === "production"
+    ? POSTHOG_PROD_KEY
+    : POSTHOG_STAGING_KEY);
 
 // Default to OpenHands' reverse proxy to bypass ad blockers.
 // The proxy at z.openhands.dev routes to PostHog's US region.