feat: support LLM profiles on cloud backends (#1532)

* feat: support LLM profiles on cloud backends

Cloud backends had no access to LLM profiles: the LLM was configured through the flat cloud settings form and the chat composer showed a plain model picker. The cloud app-server already exposes the full profile machinery under /api/v1/settings/profiles, so wire agent-canvas to it.

- ProfilesService branches to a new cloud service (src/api/cloud/profiles-service.api.ts) when the active backend is cloud, mirroring how SettingsService delegates to fetchCloudSettings; the profile hooks and the settings manager UI then work transparently.
- The LLM settings route renders the profile manager for both backends.
- Chat-level switching on cloud: the composer shows the profile switcher, /model lists/switches profiles, and per-conversation switching routes through the app-server's server-resolved /app-conversations/{id}/switch_profile endpoint.

* fix: gate cloud LLM profile management on org role (owner/admin)

Cloud org members (role=member) have VIEW_ORG_SETTINGS only: they may view but not create/edit/rename/delete/activate LLM profiles, which the app-server reserves for owner/admin (EDIT_ORG_SETTINGS). The cloud profile settings page exposed every mutating control to all members — reported in PR review.

Surface the caller's role from the existing GET /api/organizations/{orgId}/me call and add useCanManageLlmProfiles() (local backends always true; cloud only for owner/admin, reusing the /me query so no extra request). The settings profile manager hides Add and the per-row actions menu (edit/rename/duplicate/delete/activate) for members, rendering a read-only list.

Per-conversation profile switching in chat stays available to members: the app-server's /app-conversations/{id}/switch_profile route is not org-permission-gated, so switching one's own conversation is a permitted usage action, distinct from managing the org's profiles.

* fix: read profile-manage permission from the server, with role fallback

Review follow-up: instead of hardcoding the role->permission mapping on the client (role === owner||admin), useCanManageLlmProfiles now reads the server-defined `permissions` from GET /api/organizations/{orgId}/me and gates on `edit_org_settings`. Falls back to the previous role check when an older app-server doesn't return `permissions`, so it keeps working against either backend version.

Backend companion (adds `permissions` to /me): OpenHands/OpenHands#15048.

* fix: enforce LLM-profile permissions server-side via the org-gated routes

Route cloud profile CRUD/activate through /api/organizations/{orgId}/profiles, which require EDIT_ORG_SETTINGS server-side — so a member's mutation is rejected with 403 even on a direct API call, not just hidden by the client gate. Falls back to the ungated per-user /api/v1/settings/profiles route only when no org is bound (legacy keys).

A shared cloudProfilesTarget() picks the base path; get/activate normalize the org shapes (llm -> config / llm_applied). Completes the 'validate on both client and server' review point alongside the client gate (companion: OpenHands/OpenHands#15048 exposes the permission on /me).

---------

Co-authored-by: hieptl <hieptl.developer@gmail.com>
This commit is contained in:
Vasco Schiavo
2026-07-02 12:16:46 +00:00
committed by GitHub
co-authored by hieptl
parent 35fe8ac4c0
commit 9e787557fd
23 changed files with 766 additions and 108 deletions
@@ -818,7 +818,7 @@ describe("AgentServerConversationService", () => {
expect(mockSwitchLLM).not.toHaveBeenCalled();
});
it("rejects profile switching on cloud backends before any network call", async () => {
it("routes a cloud conversation switch through the app-server switch_profile endpoint", async () => {
const cloudBackend: Backend = {
id: "prod",
name: "Production",
@@ -828,15 +828,20 @@ describe("AgentServerConversationService", () => {
};
setRegisteredBackends([cloudBackend]);
setActiveSelection({ backendId: cloudBackend.id });
vi.mocked(axios.request).mockReset();
vi.mocked(axios.request).mockResolvedValue({ data: { success: true } });
await expect(
AgentServerConversationService.switchProfile("conv-1", "haiku"),
).rejects.toThrow(
"LLM profile switching is only supported for local agent-server backends.",
);
expect(mockActivateProfile).not.toHaveBeenCalled();
await AgentServerConversationService.switchProfile("conv-1", "haiku");
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({
method: "POST",
url: "https://app.all-hands.dev/api/v1/app-conversations/conv-1/switch_profile",
data: { profile_name: "haiku" },
});
// Cloud resolves the swap server-side: no client-side encrypted profile
// fetch and no direct switch_llm call.
expect(mockGetProfile).not.toHaveBeenCalled();
expect(mockSwitchProfile).not.toHaveBeenCalled();
expect(mockSwitchLLM).not.toHaveBeenCalled();
});
});
+29 -1
View File
@@ -40,6 +40,7 @@ describe("cloud organization /me", () => {
user_id: orgId,
email: "hieptl.developer@gmail.com",
role: "owner",
permissions: ["view_org_settings", "edit_org_settings"],
},
});
@@ -51,6 +52,33 @@ describe("cloud organization /me", () => {
method: "GET",
headers: { Authorization: "Bearer bearer-token" },
});
expect(result).toEqual({ orgId, userId: orgId });
expect(result).toEqual({
orgId,
userId: orgId,
role: "owner",
permissions: ["view_org_settings", "edit_org_settings"],
});
});
it("returns null permissions when the app-server omits them (older version)", async () => {
const orgId = "0b93b5f2-5396-49f2-8d98-61f906184270";
vi.mocked(axios.request).mockResolvedValue({
data: {
org_id: orgId,
user_id: orgId,
email: "x@example.com",
role: "member",
},
});
const result = await getCloudOrganizationMe(orgId);
// Absent `permissions` → null, so callers fall back to the role check.
expect(result).toEqual({
orgId,
userId: orgId,
role: "member",
permissions: null,
});
});
});
@@ -0,0 +1,200 @@
import axios from "axios";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
__resetActiveStoreForTests,
setActiveSelection,
setRegisteredBackends,
} from "#/api/backend-registry/active-store";
import type { Backend } from "#/api/backend-registry/types";
import ProfilesService, {
type SaveProfileRequest,
} from "#/api/profiles-service/profiles-service.api";
vi.mock("axios");
const cloudBackend: Backend = {
id: "prod",
name: "Production",
host: "https://app.all-hands.dev",
apiKey: "bearer-token",
kind: "cloud",
};
const ORG_ID = "org-1";
const ORG_BASE = `https://app.all-hands.dev/api/organizations/${ORG_ID}/profiles`;
const SETTINGS_BASE = "https://app.all-hands.dev/api/v1/settings/profiles";
beforeEach(() => {
window.localStorage.clear();
__resetActiveStoreForTests();
setRegisteredBackends([cloudBackend]);
vi.mocked(axios.request).mockReset();
});
afterEach(() => {
window.localStorage.clear();
__resetActiveStoreForTests();
});
// With an org bound, profile CRUD goes through the org-gated routes so the
// server enforces EDIT_ORG_SETTINGS (a member's mutation 403s, not just hidden).
describe("ProfilesService against a cloud org (gated org routes)", () => {
beforeEach(() => {
setActiveSelection({ backendId: cloudBackend.id, orgId: ORG_ID });
});
it("lists profiles via GET /api/organizations/{orgId}/profiles", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: {
profiles: [
{ name: "gpt", model: "gpt-4o", base_url: null, api_key_set: true },
],
active_profile: "gpt",
},
});
const res = await ProfilesService.listProfiles();
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({
method: "GET",
url: ORG_BASE,
headers: { Authorization: "Bearer bearer-token" },
});
expect(res.active_profile).toBe("gpt");
});
it("fetches a profile and maps the org `llm` onto `config`", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: { name: "my profile", llm: { model: "gpt-4o", api_key: null } },
});
const res = await ProfilesService.getProfile("my profile");
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({
method: "GET",
url: `${ORG_BASE}/my%20profile`,
});
expect(res).toEqual({
name: "my profile",
config: { model: "gpt-4o", api_key: null },
api_key_set: false,
});
});
it("saves a profile via POST .../{name} forwarding the request body", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: { name: "gpt", message: "Profile 'gpt' saved" },
});
await ProfilesService.saveProfile("gpt", {
llm: { model: "gpt-4o" } as SaveProfileRequest["llm"],
include_secrets: true,
});
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({
method: "POST",
url: `${ORG_BASE}/gpt`,
data: { llm: { model: "gpt-4o" }, include_secrets: true },
});
});
it("deletes a profile via DELETE .../{name}", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: { name: "gpt", message: "Profile 'gpt' deleted" },
});
await ProfilesService.deleteProfile("gpt");
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({ method: "DELETE", url: `${ORG_BASE}/gpt` });
});
it("renames a profile via POST .../{name}/rename with new_name", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: { name: "new", message: "renamed" },
});
await ProfilesService.renameProfile("old", "new");
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({
method: "POST",
url: `${ORG_BASE}/old/rename`,
data: { new_name: "new" },
});
});
it("activates a profile and maps the org `llm` onto `llm_applied`", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: {
name: "gpt",
message: "Switched to profile 'gpt'",
llm: { model: "gpt-4o" },
},
});
const res = await ProfilesService.activateProfile("gpt");
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({
method: "POST",
url: `${ORG_BASE}/gpt/activate`,
});
expect(res).toEqual({
name: "gpt",
message: "Switched to profile 'gpt'",
llm_applied: true,
});
});
});
// Legacy API keys have no org bound; CRUD falls back to the per-user settings
// route (ungated — there is no org role to enforce against).
describe("ProfilesService on a cloud backend with no org (fallback)", () => {
beforeEach(() => {
setActiveSelection({ backendId: cloudBackend.id });
});
it("lists via the per-user settings route", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: { profiles: [], active_profile: null },
});
await ProfilesService.listProfiles();
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({ method: "GET", url: SETTINGS_BASE });
});
it("saves via the per-user settings route", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: { name: "gpt", message: "saved" },
});
await ProfilesService.saveProfile("gpt", {
llm: { model: "gpt-4o" } as SaveProfileRequest["llm"],
include_secrets: true,
});
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({ method: "POST", url: `${SETTINGS_BASE}/gpt` });
});
it("activates via the per-user settings route and maps `model`", async () => {
vi.mocked(axios.request).mockResolvedValueOnce({
data: { name: "gpt", message: "ok", model: "gpt-4o" },
});
const res = await ProfilesService.activateProfile("gpt");
const [cfg] = vi.mocked(axios.request).mock.calls[0]!;
expect(cfg).toMatchObject({
method: "POST",
url: `${SETTINGS_BASE}/gpt/activate`,
});
expect(res.llm_applied).toBe(true);
});
});
@@ -111,6 +111,7 @@ beforeEach(() => {
vi.mocked(getCloudOrganizationMe).mockResolvedValue({
orgId: "",
userId: "",
role: null,
});
// Default to the legacy-key fallback so existing assertions about
// multiple orgs being visible still hold. Tests that exercise
@@ -310,6 +311,7 @@ describe("BackendSelector", () => {
vi.mocked(getCloudOrganizationMe).mockImplementation(async (orgId) => ({
orgId,
userId: orgId === personalOrgId ? personalOrgId : "some-user",
role: null,
}));
renderWithProviders(
@@ -351,6 +353,7 @@ describe("BackendSelector", () => {
vi.mocked(getCloudOrganizationMe).mockResolvedValue({
orgId: personalOrgId,
userId: personalOrgId,
role: null,
});
let cloudId = "";
@@ -102,6 +102,7 @@ beforeEach(() => {
vi.mocked(getCloudOrganizationMe).mockResolvedValue({
orgId: "",
userId: "",
role: null,
});
vi.mocked(getCurrentCloudApiKey).mockReset();
vi.mocked(getCurrentCloudApiKey).mockResolvedValue({
@@ -433,6 +434,7 @@ describe("ManageBackendsModal", () => {
vi.mocked(getCloudOrganizationMe).mockResolvedValue({
orgId: personalOrgId,
userId: personalOrgId,
role: null,
});
renderWithProviders(
@@ -100,7 +100,7 @@ describe("ChatInputActions", () => {
).not.toBeInTheDocument();
});
it("renders the active conversation model on a cloud backend", () => {
it("renders the SwitchProfileButton on a cloud backend", () => {
setRegisteredBackends([cloudBackend]);
setActiveSelection({ backendId: cloudBackend.id });
useActiveConversationMock.mockReturnValue({
@@ -113,15 +113,17 @@ describe("ChatInputActions", () => {
</ActiveBackendProvider>,
);
expect(screen.getByTestId("chat-input-llm-model")).toHaveTextContent(
"gpt-4o",
);
// Cloud now manages the LLM through profiles, so the composer shows the
// profile switcher (same as local), not the static cloud model label.
expect(
screen.queryByTestId("switch-profile-button-stub"),
screen.getByTestId("switch-profile-button-stub"),
).toBeInTheDocument();
expect(
screen.queryByTestId("chat-input-llm-model"),
).not.toBeInTheDocument();
});
it("omits the model label on cloud when the active conversation has no llm_model", () => {
it("renders the SwitchProfileButton on cloud even when the conversation has no llm_model", () => {
setRegisteredBackends([cloudBackend]);
setActiveSelection({ backendId: cloudBackend.id });
useActiveConversationMock.mockReturnValue({
@@ -134,11 +136,42 @@ describe("ChatInputActions", () => {
</ActiveBackendProvider>,
);
expect(
screen.getByTestId("switch-profile-button-stub"),
).toBeInTheDocument();
expect(
screen.queryByTestId("chat-input-llm-model"),
).not.toBeInTheDocument();
});
it("renders the static model label for cloud ACP conversations", () => {
setRegisteredBackends([cloudBackend]);
setActiveSelection({ backendId: cloudBackend.id });
useActiveConversationMock.mockReturnValue({
data: {
conversation_id: "test-conversation-id",
agent_kind: "acp",
llm_model: "claude-sonnet-4-6",
},
});
renderWithProviders(
<ActiveBackendProvider>
<ChatInputActions disabled={false} />
</ActiveBackendProvider>,
);
// ACP conversations keep the static model label on cloud too — the switch
// gate is now driven solely by isAcpContext, not the backend kind.
expect(screen.getByTestId("chat-input-llm-model")).toHaveAttribute(
"title",
"claude-sonnet-4-6",
);
expect(
screen.queryByTestId("switch-profile-button-stub"),
).not.toBeInTheDocument();
});
it("hides the Change Agent button on a local backend", () => {
renderWithProviders(<ChatInputActions disabled={false} />);
@@ -171,8 +204,6 @@ describe("ChatInputActions", () => {
{ navigation: { conversationId: null } },
);
expect(
screen.getByTestId("change-agent-button-stub"),
).toBeInTheDocument();
expect(screen.getByTestId("change-agent-button-stub")).toBeInTheDocument();
});
});
@@ -8,12 +8,12 @@ vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string) => {
const translations: Record<string, string> = {
"SETTINGS$PROFILE_ACTIVE": "Active",
"SETTINGS$PROFILE_MENU": "Profile menu",
"SETTINGS$PROFILE_EDIT": "Edit",
"BUTTON$RENAME": "Rename",
"SETTINGS$PROFILE_SET_ACTIVE": "Set as active",
"BUTTON$DELETE": "Delete",
SETTINGS$PROFILE_ACTIVE: "Active",
SETTINGS$PROFILE_MENU: "Profile menu",
SETTINGS$PROFILE_EDIT: "Edit",
BUTTON$RENAME: "Rename",
SETTINGS$PROFILE_SET_ACTIVE: "Set as active",
BUTTON$DELETE: "Delete",
};
return translations[key] || key;
},
@@ -30,6 +30,7 @@ const mockProfile: ProfileInfo = {
const defaultProps = {
profile: mockProfile,
isActive: false,
canManage: true,
onActivate: vi.fn(),
onEdit: vi.fn(),
onRename: vi.fn(),
@@ -72,7 +73,19 @@ describe("ProfileRow", () => {
it("does not show Active badge when isActive is false", () => {
render(<ProfileRow {...defaultProps} isActive={false} />);
expect(screen.queryByTestId("profile-active-badge")).not.toBeInTheDocument();
expect(
screen.queryByTestId("profile-active-badge"),
).not.toBeInTheDocument();
});
it("hides the actions menu when canManage is false (view-only members)", () => {
render(<ProfileRow {...defaultProps} canManage={false} />);
// The row still shows the profile, but offers no mutate actions.
expect(screen.getByText("gpt-4-profile")).toBeInTheDocument();
expect(
screen.queryByTestId("profile-menu-trigger"),
).not.toBeInTheDocument();
});
it("opens menu when trigger button is clicked", async () => {
@@ -95,10 +108,10 @@ describe("ProfileRow", () => {
render(<ProfileRow {...defaultProps} />);
const menuTrigger = screen.getByTestId("profile-menu-trigger");
// Menu should be closed initially
expect(screen.queryByText("Edit")).not.toBeInTheDocument();
// First click opens the menu
await user.click(menuTrigger);
expect(screen.getByText("Edit")).toBeInTheDocument();
@@ -7,14 +7,14 @@ vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string) => {
const translations: Record<string, string> = {
"SETTINGS$PROFILES_LOAD_ERROR": "Failed to load profiles",
"SETTINGS$PROFILES_EMPTY": "No profiles saved yet",
"SETTINGS$PROFILE_ACTIVE": "Active",
"SETTINGS$PROFILE_MENU": "Profile menu",
"SETTINGS$PROFILE_EDIT": "Edit",
"BUTTON$RENAME": "Rename",
"SETTINGS$PROFILE_SET_ACTIVE": "Set as active",
"BUTTON$DELETE": "Delete",
SETTINGS$PROFILES_LOAD_ERROR: "Failed to load profiles",
SETTINGS$PROFILES_EMPTY: "No profiles saved yet",
SETTINGS$PROFILE_ACTIVE: "Active",
SETTINGS$PROFILE_MENU: "Profile menu",
SETTINGS$PROFILE_EDIT: "Edit",
BUTTON$RENAME: "Rename",
SETTINGS$PROFILE_SET_ACTIVE: "Set as active",
BUTTON$DELETE: "Delete",
};
return translations[key] || key;
},
@@ -41,6 +41,7 @@ const defaultProps = {
loadError: null,
profiles: mockProfiles,
active: "gpt-4-profile",
canManage: true,
onActivate: vi.fn(),
onEdit: vi.fn(),
onRename: vi.fn(),
@@ -109,7 +110,9 @@ describe("ProfilesBody", () => {
);
expect(screen.getByTestId("loading-spinner")).toBeInTheDocument();
expect(screen.queryByText("Failed to load profiles")).not.toBeInTheDocument();
expect(
screen.queryByText("Failed to load profiles"),
).not.toBeInTheDocument();
});
it("error state takes priority over empty state", () => {
@@ -151,7 +151,7 @@ describe("useModelInterceptor", () => {
expect(mockSwitchAndLog).not.toHaveBeenCalled();
});
it("falls through on cloud backends", () => {
it("intercepts on cloud backends (cloud also manages the LLM via profiles)", () => {
setRegisteredBackends([cloudBackend]);
setActiveSelection({ backendId: cloudBackend.id });
const onSubmit = vi.fn();
@@ -163,8 +163,8 @@ describe("useModelInterceptor", () => {
act(() => result.current("/model haiku"));
expect(onSubmit).toHaveBeenCalledWith("/model haiku");
expect(mockSwitchAndLog).not.toHaveBeenCalled();
expect(mockSwitchAndLog).toHaveBeenCalledWith(CONVERSATION_ID, "haiku");
expect(onSubmit).not.toHaveBeenCalled();
});
it("activates the named profile globally even when no conversation is set", () => {
@@ -0,0 +1,111 @@
import React from "react";
import { renderHook, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { useCanManageLlmProfiles } from "#/hooks/use-can-manage-llm-profiles";
import * as orgService from "#/api/cloud/organization-service.api";
import * as activeBackendContext from "#/contexts/active-backend-context";
import type { Backend } from "#/api/backend-registry/types";
vi.mock("#/api/cloud/organization-service.api");
const localBackend: Backend = {
id: "local",
name: "Local",
host: "http://localhost:8000",
apiKey: "",
kind: "local",
};
const cloudBackend: Backend = {
id: "prod",
name: "Production",
host: "https://app.all-hands.dev",
apiKey: "bearer-token",
kind: "cloud",
};
function wrapper({ children }: { children: React.ReactNode }) {
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
return (
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
);
}
function mockBackend(backend: Backend, orgId: string | null) {
vi.spyOn(activeBackendContext, "useActiveBackend").mockReturnValue({
backend,
orgId,
});
}
describe("useCanManageLlmProfiles", () => {
beforeEach(() => {
vi.restoreAllMocks();
});
it("is always true on local backends (no /me call)", () => {
mockBackend(localBackend, null);
const { result } = renderHook(() => useCanManageLlmProfiles(), { wrapper });
expect(result.current).toBe(true);
expect(orgService.getCloudOrganizationMe).not.toHaveBeenCalled();
});
it("uses the server permission and ignores the role (permission grants)", async () => {
mockBackend(cloudBackend, "org-1");
// Role says member, but the permission set grants edit — permission wins.
vi.mocked(orgService.getCloudOrganizationMe).mockResolvedValue({
orgId: "org-1",
userId: "u",
role: "member",
permissions: ["view_org_settings", "edit_org_settings"],
});
const { result } = renderHook(() => useCanManageLlmProfiles(), { wrapper });
await waitFor(() => expect(result.current).toBe(true));
});
it("uses the server permission and ignores the role (permission denies)", async () => {
mockBackend(cloudBackend, "org-1");
// Role says owner, but the permission set lacks edit — permission wins.
vi.mocked(orgService.getCloudOrganizationMe).mockResolvedValue({
orgId: "org-1",
userId: "u",
role: "owner",
permissions: ["view_org_settings"],
});
const { result } = renderHook(() => useCanManageLlmProfiles(), { wrapper });
await waitFor(() =>
expect(orgService.getCloudOrganizationMe).toHaveBeenCalled(),
);
expect(result.current).toBe(false);
});
it("falls back to the role when the app-server omits permissions", async () => {
mockBackend(cloudBackend, "org-1");
vi.mocked(orgService.getCloudOrganizationMe).mockResolvedValue({
orgId: "org-1",
userId: "u",
role: "admin",
permissions: null,
});
const { result } = renderHook(() => useCanManageLlmProfiles(), { wrapper });
await waitFor(() => expect(result.current).toBe(true));
});
it("falls back to deny for a member when permissions are absent", async () => {
mockBackend(cloudBackend, "org-1");
vi.mocked(orgService.getCloudOrganizationMe).mockResolvedValue({
orgId: "org-1",
userId: "u",
role: "member",
permissions: null,
});
const { result } = renderHook(() => useCanManageLlmProfiles(), { wrapper });
await waitFor(() =>
expect(orgService.getCloudOrganizationMe).toHaveBeenCalled(),
);
expect(result.current).toBe(false);
});
});
+13 -10
View File
@@ -14,6 +14,11 @@ import * as useLlmProfilesHook from "#/hooks/query/use-llm-profiles";
import LLMSubscriptionService from "#/api/llm-subscription-service";
vi.mock("#/hooks/query/use-llm-profiles");
// The profile manager gates mutate controls on this hook; default to a user
// who can manage so the manager renders its full (editable) surface.
vi.mock("#/hooks/use-can-manage-llm-profiles", () => ({
useCanManageLlmProfiles: () => true,
}));
function buildSettings(overrides: Partial<Settings> = {}): Settings {
return {
@@ -171,9 +176,8 @@ describe("LlmSettingsScreen", () => {
await waitFor(() => expect(saveSettingsSpy).toHaveBeenCalled());
const payload = saveSettingsSpy.mock.calls[0][0] as Record<string, unknown>;
const llmPayload = (
payload.agent_settings_diff as Record<string, unknown>
).llm as Record<string, unknown>;
const llmPayload = (payload.agent_settings_diff as Record<string, unknown>)
.llm as Record<string, unknown>;
expect(llmPayload.api_key).toBe("test-api-key");
expect(llmPayload).not.toHaveProperty("base_url");
});
@@ -363,7 +367,7 @@ describe("LlmSettingsRoute - backend mode rendering", () => {
expect(screen.getByTestId("add-llm-profile")).toBeInTheDocument();
});
it("renders standard LlmSettingsScreen (no profiles) for cloud backends", async () => {
it("renders LlmSettingsLocalView (profile manager) for cloud backends", async () => {
vi.spyOn(activeBackendContext, "useActiveBackend").mockReturnValue({
backend: mockCloudBackend,
orgId: "org-123",
@@ -386,11 +390,10 @@ describe("LlmSettingsRoute - backend mode rendering", () => {
renderLlmSettingsRoute();
// Cloud mode shows the standard LLM settings form (not profile manager)
await screen.findByTestId("llm-settings-screen");
expect(screen.getByTestId("llm-settings-screen")).toBeInTheDocument();
// Should NOT show the "Add LLM Profile" button
expect(screen.queryByTestId("add-llm-profile")).not.toBeInTheDocument();
// Cloud now manages the LLM through profiles too (app-server
// /api/v1/settings/profiles), so the route renders the profile manager
// — same view as local — rather than the plain settings form.
await screen.findByTestId("add-llm-profile");
expect(screen.getByTestId("add-llm-profile")).toBeInTheDocument();
});
});
+22 -2
View File
@@ -88,13 +88,29 @@ export async function getCurrentCloudApiKey(
* whether `orgId` is the user's personal workspace — that's the cloud
* contract (the auto-generated personal-workspace org has the same id as
* the user).
*
* `role` is the caller's role in the org (`owner` | `admin` | `member`, or
* `null` if the upstream omits it). `permissions` is the server-defined
* permission set for that role (e.g. `edit_org_settings`); it is `null` on
* older app-servers that don't return it, so callers fall back to the role.
* See `useCanManageLlmProfiles`.
*/
export async function getCloudOrganizationMe(
orgId: string,
backend?: Backend,
): Promise<{ orgId: string; userId: string }> {
): Promise<{
orgId: string;
userId: string;
role: string | null;
permissions?: string[] | null;
}> {
const target = resolveBackend(backend);
const data = await callCloudProxy<{ org_id: string; user_id: string }>({
const data = await callCloudProxy<{
org_id: string;
user_id: string;
role?: string;
permissions?: string[];
}>({
backend: target,
method: "GET",
path: `/api/organizations/${encodeURIComponent(orgId)}/me`,
@@ -102,5 +118,9 @@ export async function getCloudOrganizationMe(
return {
orgId: data?.org_id ?? orgId,
userId: data?.user_id ?? "",
role: data?.role ?? null,
// `null` when the field is absent (older app-server) so callers can fall
// back to a role check; a present array is the server's source of truth.
permissions: Array.isArray(data?.permissions) ? data.permissions : null,
};
}
+140
View File
@@ -0,0 +1,140 @@
import type {
ActivateProfileResponse,
ProfileDetailResponse,
ProfileListResponse,
ProfileMutationResponse,
SaveProfileRequest,
} from "@openhands/typescript-client";
import { getActiveBackend } from "../backend-registry/active-store";
import type { Backend } from "../backend-registry/types";
import { callCloudProxy } from "./proxy";
/**
* Cloud LLM-profile service.
*
* Profile CRUD is routed to the org-scoped endpoints
* `/api/organizations/{orgId}/profiles`, which enforce `EDIT_ORG_SETTINGS`
* server-side — so a member's mutation is rejected with 403 even on a direct
* API call, not just hidden in the UI. When the active cloud backend has no org
* bound (legacy API keys), we fall back to the ungated per-user settings route
* `/api/v1/settings/profiles`, where there is no org role to enforce against.
*
* The two routes share shapes except `get` (org returns `llm`, settings returns
* `config`) and `activate` (org returns `llm`, settings returns `model`); both
* are normalized to the SDK profile types below. Neither route exposes secrets.
*/
const SETTINGS_PROFILES_PATH = "/api/v1/settings/profiles";
/**
* Resolve the backend + base path for the active cloud backend's profiles:
* the org-gated route when an org is bound, else the per-user settings route.
*/
function cloudProfilesTarget(): { backend: Backend; base: string } {
const { backend, orgId } = getActiveBackend();
if (backend.kind !== "cloud") {
throw new Error("Cloud profiles call requires a cloud backend.");
}
return {
backend,
base: orgId
? `/api/organizations/${encodeURIComponent(orgId)}/profiles`
: SETTINGS_PROFILES_PATH,
};
}
export async function fetchCloudProfiles(): Promise<ProfileListResponse> {
const { backend, base } = cloudProfilesTarget();
return callCloudProxy<ProfileListResponse>({
backend,
method: "GET",
path: base,
});
}
export async function fetchCloudProfile(
name: string,
): Promise<ProfileDetailResponse> {
const { backend, base } = cloudProfilesTarget();
// Org returns `{ name, llm }`; settings returns `{ name, config, api_key_set }`.
// Normalize to the SDK detail shape. Neither route exposes the key, and the
// GUI never reads the detail's `api_key_set` (only list-item `api_key_set`).
const result = await callCloudProxy<{
name: string;
config?: Record<string, unknown>;
llm?: Record<string, unknown>;
api_key_set?: boolean;
}>({
backend,
method: "GET",
path: `${base}/${encodeURIComponent(name)}`,
});
return {
name: result.name,
config: result.config ?? result.llm ?? {},
api_key_set: result.api_key_set ?? false,
};
}
export async function saveCloudProfile(
name: string,
request: SaveProfileRequest,
): Promise<ProfileMutationResponse> {
const { backend, base } = cloudProfilesTarget();
return callCloudProxy<ProfileMutationResponse>({
backend,
method: "POST",
path: `${base}/${encodeURIComponent(name)}`,
body: request,
});
}
export async function deleteCloudProfile(
name: string,
): Promise<ProfileMutationResponse> {
const { backend, base } = cloudProfilesTarget();
return callCloudProxy<ProfileMutationResponse>({
backend,
method: "DELETE",
path: `${base}/${encodeURIComponent(name)}`,
});
}
export async function renameCloudProfile(
name: string,
newName: string,
): Promise<ProfileMutationResponse> {
const { backend, base } = cloudProfilesTarget();
return callCloudProxy<ProfileMutationResponse>({
backend,
method: "POST",
path: `${base}/${encodeURIComponent(name)}/rename`,
body: { new_name: newName },
});
}
export async function activateCloudProfile(
name: string,
): Promise<ActivateProfileResponse> {
const { backend, base } = cloudProfilesTarget();
// Org returns `{ name, message, llm }`; settings returns `{ name, message,
// model }`. The SDK type carries `llm_applied`; derive it from whichever the
// route provided (consumers only read name/message — the hook just
// invalidates caches).
const result = await callCloudProxy<{
name: string;
message: string;
model?: string | null;
llm?: Record<string, unknown> | null;
}>({
backend,
method: "POST",
path: `${base}/${encodeURIComponent(name)}/activate`,
body: {},
});
return {
name: result.name,
message: result.message,
llm_applied: result.model != null || result.llm != null,
};
}
@@ -22,6 +22,7 @@ import {
getEffectiveLocalBackend,
} from "../backend-registry/active-store";
import { callCloudProxy } from "../cloud/proxy";
import ProfilesService from "../profiles-service/profiles-service.api";
import {
batchGetCloudConversations,
createCloudAppConversation,
@@ -684,15 +685,34 @@ class AgentServerConversationService {
* The per-conversation endpoint accepts only the profile name, so the UI does
* not need to fetch or forward profile secrets. That keeps switching working
* even when the agent server has no OH_SECRET_KEY for encrypted secret export.
*
* Cloud backends route to the app-server's per-conversation
* `/switch_profile`, which owns the profiles and resolves the swap
* server-side (base_url/api_key fixups, usage_id derivation, then the
* agent-server's switch_llm) — so the client only forwards the profile name,
* mirroring {@link switchAcpModel}.
*/
static async switchProfile(
conversationId: string | null,
profileName: string,
): Promise<void> {
if (getActiveBackend().backend.kind === "cloud") {
throw new Error(
"LLM profile switching is only supported for local agent-server backends.",
);
const { backend } = getActiveBackend();
if (backend.kind === "cloud") {
// No conversation (home page): activate globally so the next
// conversation starts with it. ProfilesService routes to the cloud
// activate endpoint.
if (!conversationId) {
await ProfilesService.activateProfile(profileName);
return;
}
await callCloudProxy({
backend,
method: "POST",
path: `/api/v1/app-conversations/${conversationId}/switch_profile`,
body: { profile_name: profileName },
});
return;
}
if (!conversationId) {
@@ -1,6 +1,12 @@
/**
* ProfilesService provides a thin wrapper around the SDK's ProfilesClient,
* creating a client per-call to pick up current backend configuration.
* ProfilesService is the single entry point for LLM-profile CRUD, routing per
* active backend so callers (hooks, the settings manager) stay backend-agnostic:
* - local agent-server: the SDK's ProfilesClient (`/api/profiles`), created
* per-call to pick up current backend configuration;
* - cloud app-server: `src/api/cloud/profiles-service.api.ts` (the org-gated
* `/api/organizations/{orgId}/profiles` routes, or the per-user settings
* route as a fallback) via the org-scoped cloud proxy.
* This mirrors how SettingsService branches to fetchCloudSettings().
*
* Uses ProfilesClient from @openhands/typescript-client v0.2.0+.
* All types are re-exported from the SDK for consumer convenience.
@@ -24,6 +30,15 @@ import type {
ExposeSecretsMode,
} from "@openhands/typescript-client";
import { getAgentServerClientOptions } from "../agent-server-client-options";
import { getActiveBackend } from "../backend-registry/active-store";
import {
activateCloudProfile,
deleteCloudProfile,
fetchCloudProfile,
fetchCloudProfiles,
renameCloudProfile,
saveCloudProfile,
} from "../cloud/profiles-service.api";
// Re-export SDK types for consumers
export type {
@@ -36,8 +51,13 @@ export type {
ExposeSecretsMode,
};
function isCloudBackend(): boolean {
return getActiveBackend().backend.kind === "cloud";
}
class ProfilesService {
static async listProfiles(): Promise<ProfileListResponse> {
if (isCloudBackend()) return fetchCloudProfiles();
return new ProfilesClient(getAgentServerClientOptions()).listProfiles();
}
@@ -45,6 +65,9 @@ class ProfilesService {
name: string,
exposeSecrets?: ExposeSecretsMode,
): Promise<ProfileDetailResponse> {
// Cloud never exposes profile secrets (api_key is always nulled with an
// api_key_set flag), so `exposeSecrets` is local-only.
if (isCloudBackend()) return fetchCloudProfile(name);
const options: GetProfileOptions = exposeSecrets ? { exposeSecrets } : {};
return new ProfilesClient(getAgentServerClientOptions()).getProfile(
name,
@@ -56,6 +79,7 @@ class ProfilesService {
name: string,
request: SaveProfileRequest,
): Promise<ProfileMutationResponse> {
if (isCloudBackend()) return saveCloudProfile(name, request);
return new ProfilesClient(getAgentServerClientOptions()).saveProfile(
name,
request,
@@ -63,6 +87,7 @@ class ProfilesService {
}
static async deleteProfile(name: string): Promise<ProfileMutationResponse> {
if (isCloudBackend()) return deleteCloudProfile(name);
return new ProfilesClient(getAgentServerClientOptions()).deleteProfile(
name,
);
@@ -72,6 +97,7 @@ class ProfilesService {
name: string,
newName: string,
): Promise<ProfileMutationResponse> {
if (isCloudBackend()) return renameCloudProfile(name, newName);
return new ProfilesClient(getAgentServerClientOptions()).renameProfile(
name,
newName,
@@ -79,6 +105,7 @@ class ProfilesService {
}
static async activateProfile(name: string): Promise<ActivateProfileResponse> {
if (isCloudBackend()) return activateCloudProfile(name);
return new ProfilesClient(getAgentServerClientOptions()).activateProfile(
name,
);
@@ -406,7 +406,7 @@ export function ChatInputActions({
</div>
)}
<div ref={modelRef} className={cn(!showModelInline && "hidden")}>
{isCloud || modelState.isAcpContext ? (
{modelState.isAcpContext ? (
<ChatInputModel />
) : (
<SwitchProfileButton />
@@ -11,6 +11,7 @@ import ProfilesService, {
import { useLlmProfiles } from "#/hooks/query/use-llm-profiles";
import { useActivateLlmProfile } from "#/hooks/mutation/use-activate-llm-profile";
import { useSaveLlmProfile } from "#/hooks/mutation/use-save-llm-profile";
import { useCanManageLlmProfiles } from "#/hooks/use-can-manage-llm-profiles";
import {
displayErrorToast,
displaySuccessToast,
@@ -30,6 +31,9 @@ export function LlmProfilesManager({
const { data, isLoading, error } = useLlmProfiles();
const activateProfile = useActivateLlmProfile();
const saveProfile = useSaveLlmProfile();
// Cloud members are view-only; only owners/admins (and all local users) may
// add, edit, rename, duplicate, delete, or activate profiles.
const canManage = useCanManageLlmProfiles();
const [profileToRename, setProfileToRename] = useState<ProfileInfo | null>(
null,
);
@@ -96,7 +100,7 @@ export function LlmProfilesManager({
<h2 className="text-base font-medium text-white">
{t(I18nKey.SETTINGS$AVAILABLE_PROFILES)}
</h2>
{onAddProfile ? (
{onAddProfile && canManage ? (
<BrandButton
testId="add-llm-profile"
type="button"
@@ -114,6 +118,7 @@ export function LlmProfilesManager({
loadError={error ?? null}
profiles={profiles}
active={active}
canManage={canManage}
onActivate={handleActivate}
onEdit={handleEdit}
onRename={setProfileToRename}
@@ -14,6 +14,8 @@ import {
interface ProfileRowProps {
profile: ProfileInfo;
isActive: boolean;
/** When false, the row is read-only and the actions menu is hidden. */
canManage: boolean;
onActivate: (name: string) => void;
onEdit: (profile: ProfileInfo) => void;
onRename: (profile: ProfileInfo) => void;
@@ -25,6 +27,7 @@ interface ProfileRowProps {
export function ProfileRow({
profile,
isActive,
canManage,
onActivate,
onEdit,
onRename,
@@ -65,28 +68,30 @@ export function ProfileRow({
</BrandBadge>
)}
</div>
<div className="relative shrink-0">
<EllipsisButton
ref={triggerRef}
onClick={() => setMenuOpen((open) => !open)}
ariaLabel={t(I18nKey.SETTINGS$PROFILE_MENU)}
testId="profile-menu-trigger"
className={settingsListIconActionButtonClassName}
/>
{menuOpen && (
<ProfileActionsMenu
anchorRef={triggerRef}
onEdit={() => onEdit(profile)}
onRename={() => onRename(profile)}
onDuplicate={() => onDuplicate(profile)}
onSetActive={() => onActivate(profile.name)}
onDelete={() => onDelete(profile)}
isActive={isActive}
isActivating={isActivating}
onClose={() => setMenuOpen(false)}
{canManage && (
<div className="relative shrink-0">
<EllipsisButton
ref={triggerRef}
onClick={() => setMenuOpen((open) => !open)}
ariaLabel={t(I18nKey.SETTINGS$PROFILE_MENU)}
testId="profile-menu-trigger"
className={settingsListIconActionButtonClassName}
/>
)}
</div>
{menuOpen && (
<ProfileActionsMenu
anchorRef={triggerRef}
onEdit={() => onEdit(profile)}
onRename={() => onRename(profile)}
onDuplicate={() => onDuplicate(profile)}
onSetActive={() => onActivate(profile.name)}
onDelete={() => onDelete(profile)}
isActive={isActive}
isActivating={isActivating}
onClose={() => setMenuOpen(false)}
/>
)}
</div>
)}
</div>
);
}
@@ -15,6 +15,8 @@ interface ProfilesBodyProps {
loadError: Error | null;
profiles: ProfileInfo[];
active: string | null;
/** When false, rows render read-only (no actions menu) — cloud members. */
canManage: boolean;
onActivate: (name: string) => void;
onEdit: (profile: ProfileInfo) => void;
onRename: (profile: ProfileInfo) => void;
@@ -28,6 +30,7 @@ export function ProfilesBody({
loadError,
profiles,
active,
canManage,
onActivate,
onEdit,
onRename,
@@ -83,6 +86,7 @@ export function ProfilesBody({
key={profile.name}
profile={profile}
isActive={profile.name === active}
canManage={canManage}
onActivate={onActivate}
onEdit={onEdit}
onRename={onRename}
+4 -6
View File
@@ -14,11 +14,11 @@ import { MODEL_COMMAND } from "#/utils/constants";
const MODEL_PREFIX = `${MODEL_COMMAND} `;
/**
* Intercepts "/model" submissions:
* Intercepts "/model" submissions (both local and cloud backends manage the
* LLM through saved profiles):
* - "/model" → render an inline list of saved profiles in the chat
* - "/model <name>" → switch the running conversation's LLM profile
* Anything else (or when on a cloud backend, which doesn't support profiles)
* falls through to `onSubmit`.
* Anything that isn't a "/model" command falls through to `onSubmit`.
*/
export const useModelInterceptor = (
conversationId: string | null | undefined,
@@ -28,7 +28,6 @@ export const useModelInterceptor = (
const queryClient = useQueryClient();
const { switchAndLog } = useSwitchLlmProfileAndLog();
const { backend, orgId } = useActiveBackend();
const isLocal = backend.kind === "local";
const { t } = useTranslation();
return useCallback(
@@ -36,7 +35,7 @@ export const useModelInterceptor = (
const trimmed = message.trim();
const isModel =
trimmed === MODEL_COMMAND || trimmed.startsWith(MODEL_PREFIX);
if (!isModel || !isLocal) {
if (!isModel) {
onSubmit(message);
return;
}
@@ -81,7 +80,6 @@ export const useModelInterceptor = (
},
[
conversationId,
isLocal,
onSubmit,
showProfiles,
queryClient,
+3 -8
View File
@@ -39,9 +39,7 @@ export const useSlashCommand = (
// slash menu lists the same project skills that were loaded into it.
const { data: skills, isLoading: isSkillsLoading } = useConversationSkills();
const isCloud = useActiveBackend().backend.kind === "cloud";
const { data: profilesData, isLoading: isProfilesLoading } = useLlmProfiles({
enabled: !isCloud,
});
const { data: profilesData, isLoading: isProfilesLoading } = useLlmProfiles();
const [isMenuOpen, setIsMenuOpen] = useState(false);
const [filterText, setFilterText] = useState("");
const [completionKind, setCompletionKind] =
@@ -51,13 +49,12 @@ export const useSlashCommand = (
// Build slash command items from built-in commands + skills:
// - Built-in commands (like /new) are included for V1 conversations
// - /new is cloud-only — local backends don't surface it
// - /model is local-only — cloud backends don't have profiles
// - /model lists/switches LLM profiles; both local and cloud support them
// - Skills with explicit "/" triggers use those triggers
// - AgentSkills without "/" triggers get a derived "/<name>" command
const slashItems = useMemo(() => {
const items: SlashCommandItem[] = BUILT_IN_COMMANDS.filter((cmd) => {
if (cmd.command === "/new") return isCloud;
if (cmd.command === MODEL_COMMAND) return !isCloud;
return true;
});
@@ -83,8 +80,6 @@ export const useSlashCommand = (
}, [skills, isSkillsLoading, isCloud]);
const modelProfileItems = useMemo<SlashCommandItem[]>(() => {
if (isCloud) return [];
return (profilesData?.profiles ?? []).map((profile) => {
const command = `${MODEL_COMMAND} ${profile.name}`;
return {
@@ -99,7 +94,7 @@ export const useSlashCommand = (
},
};
});
}, [profilesData?.profiles, isCloud]);
}, [profilesData?.profiles]);
// Filter items based on user input after "/"
const filteredItems = useMemo(() => {
+53
View File
@@ -0,0 +1,53 @@
import { useQuery } from "@tanstack/react-query";
import { getCloudOrganizationMe } from "#/api/cloud/organization-service.api";
import { useActiveBackend } from "#/contexts/active-backend-context";
/**
* Server-defined permission (mirrors the app-server's
* `authorization.Permission.EDIT_ORG_SETTINGS`) that gates LLM-profile
* mutations for an org.
*/
const EDIT_ORG_SETTINGS = "edit_org_settings";
/**
* Whether the current user may MUTATE LLM profiles on the active backend —
* create, edit, rename, delete, duplicate, or activate/switch.
*
* - Local agent-server (OSS): always `true`; the user owns their own profiles.
* - Cloud: profiles are org-scoped. The app-server grants every mutating
* profile action (save/delete/rename/activate) only to the `owner`/`admin`
* roles via the `edit_org_settings` permission; a `member` is view-only. We
* read the caller's server-defined permissions from
* `GET /api/organizations/{orgId}/me` — the same call `useCloudCurrentUserId`
* makes, reusing its query key so no extra request is issued — and fall back
* to a role check when an older app-server doesn't return `permissions`.
*
* Returns `false` while the role/permissions are still loading or unknown on
* cloud, so mutating controls never flash for a member before they resolve.
*/
export function useCanManageLlmProfiles(): boolean {
const { backend, orgId } = useActiveBackend();
const isCloud = backend.kind === "cloud";
// `backend` is identified by `backend.id`, which is already in the key; we
// keep the key byte-identical to useCloudCurrentUserId so React Query shares
// the cached /me result instead of firing a second request.
// eslint-disable-next-line @tanstack/query/exhaustive-deps
const { data } = useQuery({
queryKey: ["cloud-current-user", backend.id, orgId],
queryFn: () => getCloudOrganizationMe(orgId!, backend),
enabled: isCloud && !!orgId,
staleTime: 1000 * 60 * 5,
retry: false,
meta: { disableToast: true },
});
if (!isCloud) return true;
// Prefer the server-defined permission set; fall back to a role check for
// older app-servers whose /me doesn't return `permissions` yet, so this
// keeps working against either backend version.
if (data?.permissions) {
return data.permissions.includes(EDIT_ORG_SETTINGS);
}
return data?.role === "owner" || data?.role === "admin";
}
+6 -14
View File
@@ -17,7 +17,6 @@ import { LlmSettingsLocalView } from "#/components/features/settings/llm-profile
import { I18nKey } from "#/i18n/declaration";
import { Settings, SettingsSchema, SettingsScope } from "#/types/settings";
import { extractModelAndProvider } from "#/utils/extract-model-and-provider";
import { useActiveBackend } from "#/contexts/active-backend-context";
import {
inferInitialView,
type SettingsFormValues,
@@ -506,26 +505,19 @@ export function LlmSettingsScreen({
}
/**
* Default export for the route renders different views based on backend type:
* - Local backends: LlmSettingsLocalView with profile management
* - Cloud backends: Standard LlmSettingsScreen (profiles are not supported)
* Default export for the route renders the LLM-profile management view for both
* backend types. Both manage the LLM through named profiles — local via the
* agent-server (`/api/profiles`), cloud via the app-server
* (`/api/v1/settings/profiles`) — and the view is backend-agnostic because it
* goes through ProfilesService, which routes per active backend.
*
* The LlmSettingsScreen component is also exported for embedded use cases
* (e.g., onboarding, profile editing forms).
* (e.g., onboarding, the profile create/edit form).
*
* Note: This is a route file, only the router should import the default export.
* Other consumers should use the named export `LlmSettingsScreen` for embedded
* use cases.
*/
export default function LlmSettingsRoute() {
const { backend } = useActiveBackend();
const isCloud = backend.kind === "cloud";
// Cloud backends use the standard LLM settings form (no profiles support)
if (isCloud) {
return <LlmSettingsScreen />;
}
// Local backends use the profile management view
return <LlmSettingsLocalView />;
}