mirror of
https://github.com/OpenHands/OpenHands.git
synced 2026-10-07 16:19:05 +08:00
a248bf0e08
* feat: add critic result types, component, and event rendering Migrate critic visualization from OpenHands PR #14133. Types (src/types/agent-server/core/base/critic.ts): - CriticResult — score (0-1), message, metadata - CriticFeature — name, display_name, probability - CriticCategorizedFeatures — agent_behavioral_issues, user_followup_patterns, infrastructure_issues, other - CriticMetadata — wraps categorized features and event IDs - Added optional critic_result field to ActionEvent and MessageEvent Component (critic-result-display.tsx): - Star rating (0-5) with color coding: green ≥60%, yellow ≥40%, red <40% - Percentage display - Expandable categorized feature breakdown with per-feature probabilities - Iterative refinement hint when disabled in settings - Full i18n support (15 languages) Integration: - FinishEventMessage renders CriticResultDisplay below the finish message when critic_result is present - UserAssistantEventMessage renders CriticResultDisplay for agent messages when critic_result is present Tests: - 14 unit tests covering score rendering, star ratings, color coding, label rendering, expand/collapse, iterative refinement hint, and multiple feature categories Co-authored-by: openhands <openhands@all-hands.dev> * refactor: SdkSectionPage multi-source support + verification settings Refactor SdkSectionPage to accept a `settingsSources` array instead of single `settingsSource`/`sectionKeys` props, enabling a page to render fields from multiple schema sources (e.g. both agent_settings and conversation_settings). Key changes: - SdkSectionPage: new `settingsSources: SettingsSourceConfig[]` prop replaces `settingsSource`/`sectionKeys`; tracks values/dirty state per source; emits combined save payload with per-source diff keys (agent_settings_diff, conversation_settings_diff) - verification-settings: simplified to declarative multi-source config pulling critic fields from agent_settings and confirmation/security fields from conversation_settings - condenser-settings, llm-settings: updated to new `settingsSources` API - Mock handlers: merged critic fields into verification section; updated defaults to match upstream schema structure - All tests updated for new API shape; full suite passes (2313 tests) Co-authored-by: openhands <openhands@all-hands.dev> * feat(verification): require user-supplied API key when enabling the critic Instead of silently reusing the OpenHands provider's LLM API key, expose a dedicated verification.critic_api_key schema field that appears (required) once the critic is enabled. The hint underneath reuses the existing OpenHands Cloud copy from the LLM provider screen so users know any LLM API key — easiest, their OpenHands Cloud key — will power the critic. - Add field to mock agent_settings schema with secret/required/critical flags and depends_on: [verification.critic_enabled]. - Extend FIELD_HELP_LINKS with an optional suffixKey so the schema- driven help row can render OpenHands Cloud copy without forking it. - Add SCHEMA$VERIFICATION$CRITIC_API_KEY$LABEL and $DESCRIPTION across all 15 locales. - Cover both enabled (field + help link visible, password, required) and disabled (field hidden) states in __tests__/routes/verification-settings.test.tsx. Co-authored-by: openhands <openhands@all-hands.dev> * ui(verification): drop critic API key into a full-width row below the toggles The two-column settings grid was placing the critic API key beside Enable Critic, leaving a tall stretch of whitespace under the toggle and squeezing the help link copy. Instead: - Reorder the mock schema so the Critic API Key field comes after Enable Iterative Refinement, freeing the right column for the second toggle on the first row. - Introduce FIELD_FULL_WIDTH_KEYS in schema-field.tsx (small UI-only set, mirrors the FIELD_HELP_LINKS pattern) and have sdk-section-page apply xl:col-span-2 to those fields. The critic API key is the only entry for now. Result in Basic view with the critic enabled: - Row 1: Enable Critic | Enable Iterative Refinement - Row 2: Critic API Key (full width, with help link) Co-authored-by: openhands <openhands@all-hands.dev> * test(snapshots): update verification helper for schema-driven page The hand-written 'Enable Confirmation Mode' header was removed when verification-settings.tsx switched to a pure SdkSectionPage, and confirmation_mode is a prominence: 'major' schema field — so it only appears in Advanced/All views. The snapshot test helper was still waiting for the old text and using the old confirmation-mode-toggle testId, which made all three verification snapshots time out. - waitForVerificationPage now waits for 'Enable Critic' (the first critical-prominence field, always visible), then clicks the sdk-section-all-toggle to switch to the 'All' view, then waits for the rendered 'Confirmation Mode' label (i18n SCHEMA$…$LABEL gives it a capital M, not the schema's raw 'Confirmation mode'). - The on/off tests use the new sdk-settings-confirmation_mode testId emitted by SchemaField's SettingsSwitch wrapper. The label-click pattern is preserved (the <input type=checkbox> is hidden by SettingsSwitch). - Security-analyzer locator is now case-insensitive (/security analyzer/i) since the i18n label is 'Security Analyzer'. Baselines will needBaselines will needBaselines will needBaselines will needBaselull-width row, and switching to the 'All' view all change the rendered pixels. Apply the 'update-snapshots' label after this commit lands. Co-authored-by: openhands <openhands@all-hands.dev> * Clarify critic API key guidance * test: snapshot verification critic settings * fix: improve critic score rendering accessibility * test: cover multi-source settings save * test: cover verification settings dedupe * style: use strict null checks in critic helper * test: cover critic result e2e rendering * test: make live critic e2e start conversation directly * test: fix live e2e llm profile setup * chore: Update PR QA artifacts --------- Co-authored-by: openhands <openhands@all-hands.dev> Co-authored-by: Rohit Malhotra <rohitvinodmalhotra@gmail.com> Co-authored-by: allhands-bot <allhands-bot@users.noreply.github.com>