From c10ae9f9d2d15a66d3107248bb6597491cf33c34 Mon Sep 17 00:00:00 2001 From: Turtle_Li <282189765@qq.com> Date: Tue, 7 Jul 2026 16:21:25 +0800 Subject: [PATCH] chore: remove batch image draft reports --- .gitignore | 1 - rfcs/batch-image-issue-draft.md | 213 ------------------ .../claude-report.md | 91 -------- .../codex-report.md | 89 -------- .../pr-description.md | 58 ----- .../smoke-summary.txt | 23 -- .../batch-image-20260706-codex/test-case.md | 47 ---- .../codex-claude-fix-report.md | 55 ----- 8 files changed, 577 deletions(-) delete mode 100644 rfcs/batch-image-issue-draft.md delete mode 100644 test-reports/batch-image-20260706-codex/claude-report.md delete mode 100644 test-reports/batch-image-20260706-codex/codex-report.md delete mode 100644 test-reports/batch-image-20260706-codex/pr-description.md delete mode 100644 test-reports/batch-image-20260706-codex/smoke-summary.txt delete mode 100644 test-reports/batch-image-20260706-codex/test-case.md delete mode 100644 test-reports/batch-image-20260706-fix-verification/codex-claude-fix-report.md diff --git a/.gitignore b/.gitignore index f7ba576604..bd2e3e6ddf 100644 --- a/.gitignore +++ b/.gitignore @@ -130,7 +130,6 @@ deploy/docker-compose.override.yml .gocache/ vite.config.js docs/* -!docs/BATCH_IMAGE_MVP.md !docs/PAYMENT.md !docs/PAYMENT_CN.md !docs/ADMIN_PAYMENT_INTEGRATION_API.md diff --git a/rfcs/batch-image-issue-draft.md b/rfcs/batch-image-issue-draft.md deleted file mode 100644 index 0c8f32855c..0000000000 --- a/rfcs/batch-image-issue-draft.md +++ /dev/null @@ -1,213 +0,0 @@ -# RFC Issue Draft: Batch Image - -## Title - -```text -RFC: add asynchronous Gemini image batch generation with Gemini API key and Vertex providers -``` - -## Body - -```markdown -## Summary - -I would like to propose an MVP for asynchronous Gemini image batch generation in Sub2API. - -I want to add a new batch image gateway surface under `/v1/images/batches`, backed by Redis workers and PostgreSQL state, with two initial upstream providers: - -- Gemini Developer API / AI Studio API key accounts -- Vertex AI Gemini service-account accounts - -The goal is to expose one stable Sub2API batch interface while keeping provider-specific details such as Gemini file names, Vertex job names, GCS paths, and service-account credentials internal. - -## Why - -Sub2API already has most of the primitives needed for this: - -- Gemini accounts already support `platform=gemini,type=api_key`. -- Vertex service-account helpers already exist. -- Redis is already part of the runtime. -- PostgreSQL/Ent is already the source of truth. -- Existing usage billing already has idempotent billing via `usage_billing_dedup`. - -Gemini API and Vertex both support async batch generation, but their auth/storage/result mechanics are different. I want to keep one public API and put those differences behind a small provider abstraction. - -The main reason I want to build this is that the official Gemini Batch API is designed for asynchronous, non-urgent large-volume requests and is documented as running at 50% of the standard cost. For image generation, that makes batch mode useful both for higher-throughput workloads and for lowering user-facing cost compared with realtime generation. - -Official references: - -- Gemini Batch API: https://ai.google.dev/gemini-api/docs/batch-api -- Gemini image generation batch section: https://ai.google.dev/gemini-api/docs/image-generation#batch-api -- Vertex Gemini batch prediction: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/batch-prediction-gemini - -## MVP - -The MVP I want to build includes: - -- Batch submit -- Async worker execution -- Status query -- Result indexing -- Single image streaming download -- ZIP streaming download -- Basic hold -> settlement billing -- Idempotency and crash recovery -- First providers: `gemini_api` and `vertex` - -## API - -Gateway routes, API-key authenticated: - -```text -POST /v1/images/batches -GET /v1/images/batches/{id} -GET /v1/images/batches/{id}/items -GET /v1/images/batches/{id}/items/{custom_id}/content -GET /v1/images/batches/{id}/download -POST /v1/images/batches/{id}/cancel -DELETE /v1/images/batches/{id}/outputs -``` - -I want to use `/v1/images/batches` because this is a gateway/API-key feature rather than a dashboard/JWT-only feature under `/api/v1`. - -## Implementation Shape - -High-level shape: - -- Add `batch_image_jobs`, `batch_image_items`, and optional `batch_image_events`. -- Store `provider` as `gemini_api` or `vertex`. -- Store selected `account_id` on the job so worker retries are deterministic. -- Use Redis `LPUSH/BRPOP`, an active set, and per-job locks. -- Keep PostgreSQL as the source of truth. -- Stream downloads through Sub2API without writing image bytes to local disk. -- Keep Gemini file names, Vertex job names, GCS URIs, bucket names, and service-account details internal. - -Provider abstraction: - -```go -type BatchImageProvider interface { - Name() string - SupportsAccount(account *Account) bool - Submit(ctx context.Context, job *BatchImageJob, account *Account, input BatchImageInput) (*BatchProviderJob, error) - Get(ctx context.Context, job *BatchImageJob, account *Account) (*BatchProviderStatus, error) - Cancel(ctx context.Context, job *BatchImageJob, account *Account) error - OpenResult(ctx context.Context, job *BatchImageJob, item *BatchImageItem, account *Account) (io.ReadCloser, string, error) - Cleanup(ctx context.Context, job *BatchImageJob, account *Account, target CleanupTarget) error -} -``` - -Billing: - -- Estimate cost at submit time and place a hold. -- Charge only successful generated images. -- Failed items are not charged in the MVP. -- Settlement is idempotent. -- I want to reuse the existing `UsageBillingRepository.Apply` / `usage_billing_dedup` path with a synthetic request id like `batch_image_settlement:{job_id}`. - -## PR Split - -1. Schema, Ent models, repository CRUD, status machine -2. Redis queue, idempotency, active job recovery -3. Provider core plus both `gemini_api` and `vertex` providers -4. Settlement / billing integration -5. Download APIs -6. Cleanup worker - -## Questions for maintainers - -1. Is `/v1/images/batches` the right public route for this feature? -2. Is storing hold fields on the batch job acceptable for MVP, with final settlement reusing existing usage billing? -3. Would you prefer the first implementation to be API-only, or include dashboard pages from the beginning? -4. Do you prefer a different naming convention for provider names, table names, or statuses? - ---- - -## 中文版本 - -我想为 Sub2API 增加一个异步 Gemini 批量生图 MVP。 - -我想新增 `/v1/images/batches` 这一组网关 API，由 Redis worker 和 PostgreSQL 状态表驱动，首版支持两个上游 provider： - -- Gemini Developer API / AI Studio 的 API key 账号 -- Vertex AI Gemini 的 service account 账号 - -目标是让用户只调用一套 Sub2API batch 接口，同时把 Gemini file name、Vertex job name、GCS 路径、bucket、service account 等内部细节留在服务端。 - -### 为什么这样做 - -Sub2API 现有架构已经比较适合这个功能： - -- 现有账号模型已经支持 `platform=gemini,type=api_key`。 -- 代码里已有 Vertex service account token helper。 -- Redis 已经是运行时依赖。 -- PostgreSQL/Ent 已经是主要状态源。 -- 现有账务已经有 `usage_billing_dedup` 这种幂等扣费机制。 - -Gemini API 和 Vertex 都有异步 batch 能力，但认证、存储、结果读取方式不同。所以我想在内部加一个小的 provider 抽象，对外保持一套稳定 API。 - -我想做这个功能的主要原因是：Gemini 官方 Batch API 本身就是为异步、非实时的大批量请求设计的，而且官方文档写明成本是标准实时请求的 50%。对于批量生图场景，这既能提升大批量任务的可用性，也能让用户成本低于实时生成。 - -### MVP - -我想先实现： - -- 批量提交 -- 异步 worker 执行 -- 状态查询 -- 结果索引 -- 单图流式下载 -- ZIP 流式下载 -- 基础 hold -> settlement 计费 -- 幂等与 crash recovery -- 首批 provider：`gemini_api` 和 `vertex` - -### API - -这些路由走 API key 鉴权： - -```text -POST /v1/images/batches -GET /v1/images/batches/{id} -GET /v1/images/batches/{id}/items -GET /v1/images/batches/{id}/items/{custom_id}/content -GET /v1/images/batches/{id}/download -POST /v1/images/batches/{id}/cancel -DELETE /v1/images/batches/{id}/outputs -``` - -我想放在 `/v1/images/batches`，因为这是网关/API key 能力，不是只给后台面板用的 `/api/v1` JWT API。 - -### 实现方式 - -- 新增 `batch_image_jobs`、`batch_image_items`，以及可选的 `batch_image_events`。 -- job 记录 `provider=gemini_api|vertex`。 -- job 记录选中的 `account_id`，保证 worker 重试时不会换账号。 -- Redis 使用 `LPUSH/BRPOP`、active set 和 per-job lock。 -- PostgreSQL 作为事实状态源。 -- 下载经 Sub2API 流式返回，不把图片字节写入本地磁盘。 -- 不向用户暴露 Gemini file name、Vertex job name、GCS URI、bucket、service account 等细节。 - -计费： - -- 提交时估算费用并冻结额度。 -- 只对成功生成的图片收费。 -- MVP 中失败 item 不收费。 -- settlement 必须幂等。 -- 我想复用现有 `UsageBillingRepository.Apply` / `usage_billing_dedup`，使用类似 `batch_image_settlement:{job_id}` 的 synthetic request id。 - -### PR 拆分 - -1. Schema、Ent models、repository CRUD、状态机 -2. Redis queue、幂等、active job recovery -3. Provider core + `gemini_api` 和 `vertex` 两个 provider -4. Settlement / billing integration -5. Download APIs -6. Cleanup worker - -### 想请维护者确认的问题 - -1. `/v1/images/batches` 是否是合适的公开路由？ -2. MVP 中把 hold 字段先存在 batch job 表上，并在最终结算时复用现有 usage billing，是否可以接受？ -3. 首版做 API-only 是否可以，还是需要一开始就包含 dashboard 页面？ -4. provider 名称、表名、状态名是否有维护者偏好的命名规范？ -``` diff --git a/test-reports/batch-image-20260706-codex/claude-report.md b/test-reports/batch-image-20260706-codex/claude-report.md deleted file mode 100644 index 8efd46864a..0000000000 --- a/test-reports/batch-image-20260706-codex/claude-report.md +++ /dev/null @@ -1,91 +0,0 @@ -# Claude Code Batch Image QA Report - -Date: 2026-07-06 -Tester: Claude Code -Claude model selection: - -- Preferred for deep QA: `opus`, but the first run exceeded the initial budget before producing output. -- Practical model used for this recorded report: `sonnet` with `--safe-mode --effort low`, because it produced a bounded independent QA report at lower cost. - -## Original Claude Output - -> Batch Image 功能 QA 报告（只读探查） -> -> ## 范围 -> 后端计费冻结/结算/退款、状态机与异常兜底、前端批量生图说明文案。基于代码走查(Explore agent)+ 2 条本地 grep 命令验证，未修改任何文件，未执行且未查看任何密钥。 -> -> ## 执行命令 -> -> | # | 命令 | 目的 | -> |---|------|------| -> | 1 | `grep -n "FOR UPDATE\|Lock(" batch_image_settlement.go batch_image_repo.go` | 验证取消/结算并发是否有行锁保护 | -> | 2 | `grep -rn "SETTLEMENT_BILLING_FAILED\|enqueueBillingRetry\|MaxRetr" ...` | 验证结算失败重试是否有次数/退避上限 | -> -> ## 通过/失败表 -> -> | 检查项 | 结果 | 依据 | -> |---|---|---| -> | 状态转换行锁保护(防止取消/结算竞态) | ✅ 通过 | `batch_image_repo.go:193,322,415` 均用 `SELECT ... FOR UPDATE` | -> | 结算超额扣费保护 | ✅ 通过 | `batch_image_settlement.go:126-130`，`actualCost > holdAmount` 超万分之一即失败中止 | -> | 冻结→结算→释放状态机完整性 | ✅ 通过 | 冻结(billing_hold.go) → 结算(settlement.go) → 释放(processor.go:225-239) 链路闭合 | -> | 僵尸/未提交任务资金释放 | ✅ 通过 | `billing_recovery.go:22-62`，10分钟未提交自动 failed + 释放冻结 | -> | 非法状态转换保护 | ✅ 通过 | `batch_image.go:356-401` 终态不可逆流转 | -> | 部分失败正确计费(仅成功项扣费) | ✅ 通过 | `actualCost = successCount * unitPrice` | -> | 结算失败重试有界（次数/超时上限） | ⚠️ 未证实 | grep 未发现 `MaxRetr`/退避上限，仅见标记 `SETTLEMENT_BILLING_FAILED` 后重新入队，逻辑分散在其他文件未定位到边界 | -> | 前端费用/取消文案与后端逻辑一致 | ⚠️ 基本一致，措辞有偏差 | 见问题清单 P2 | -> -> ## 问题清单（按严重级别） -> -> **P1（无，未发现资金泄漏或重复扣款的确认性缺陷）** -> -> **P2 – 中** -> 1. 结算失败重试缺乏可见的次数/超时上限（`SETTLEMENT_BILLING_FAILED` 后 `enqueueBillingRetry`），存在长期卡在 `settling` 状态、资金持续冻结但不释放也不完成结算的风险；需要进一步定位重试调度代码确认是否有兜底超时释放。 -> 2. 前端取消提示文案（"已生成图片仍可能结算扣费"）与后端实际计费口径（以索引完成后统计的成功项为准）表述不完全对齐，可能造成用户对扣费范围的误解，建议澄清措辞而非改变逻辑。 -> -> **P3 – 低** -> 3. 结算过程中途宕机（`Settle()` 执行到一半进程重启）依赖外部定时任务/人工介入恢复，未在本次探查中确认是否有自动扫描 `settling` 超时状态的兜底任务。 -> -> ## 剩余风险 -> - 未验证"结算失败重试"的调度器代码（未在本次两条命令范围内），无法排除无限重试或永久悬挂的可能性。 -> - 未做真实并发压测，行锁存在但未验证高并发下取消+结算同时触发的实际表现（仅代码静态确认加锁点存在）。 -> - 前端文案审查仅基于关键字定位的片段，未通读整个 Guide 组件的所有分支文案。 -> -> ## 建议后续测试 -> 1. 定位并审查 `enqueueBillingRetry` 实际调度器（重试次数、退避策略、是否有最终告警/人工介入路径），必要时补充单测覆盖"结算持续失败"场景。 -> 2. 编写并发集成测试：同一 batch_id 同时发起"取消"与"结算完成回调"，验证最终状态与金额一致性（是否只释放或只结算一次）。 -> 3. 对 `settling` 状态增加超时巡检的专项测试（类比现有 10 分钟未提交巡检），确认是否已有等价机制，如无需评估是否要补齐。 -> 4. 前端文案走查+产品确认，将"取消后扣费口径"说明与后端"仅索引完成的成功项计费"对齐后再验收。 - -## Codex Follow-Up Note - -Codex spot-checked the first P2 after Claude's report. The current implementation has a bounded settlement billing retry path: - -- `batch_image_settlement.go` defines `batchImageSettlementMaxRetries = 5`. -- Repeated `SETTLEMENT_BILLING_FAILED` increments job retry state. -- Once the retry limit is reached, settlement fails the job and releases the remaining hold through the idempotent release path. -- `batch_image_settlement_test.go` covers transient settlement requeue, retry exhaustion release, and idempotent release after transition failure. - -So Claude's original "unbounded settlement retry" risk should be treated as resolved in the current PR state, not as an open blocker. - -## 2026-07-07 Follow-Up Addendum - -Claude Code was later used in a bounded pass to update the QA test-case matrix with the online verification scenarios. Codex performed the online API/database checks and fed the verified facts back into the report; this addendum does not claim Claude personally executed the paid online image runs. - -Additional scenarios now recorded in `test-case.md`: - -- `BI-ONLINE-001`: one-image success settlement balance closure. -- `BI-ONLINE-002`: immediate cancel after submit releases hold and charges zero. -- `BI-ONLINE-003`: Gemini API-key provider path is selectable/callable; the test key had no prepayment, so successful generation was not continued; failed submit released hold and charged zero. -- `BI-ONLINE-004`: two-item partial failure charged only the one successful image and included the failed item in `errors.json`. - -Current PR readiness view after follow-up: - -- `GO behind flag`: acceptable for upstream review and merge discussion while `BATCH_IMAGE_ENABLED` and `allow_batch_image_generation` remain opt-in. -- `Not GA by default`: do not enable for all groups until operators have monitored real traffic and provider/account configuration. -- Amount-sensitive paths now have online evidence for success, cancel, partial failure, failed submit release, and `frozen_balance` returning to zero. - -Remaining non-blocking gaps: - -- No high-concurrency online stress test was run because it would create unnecessary provider cost and operational pressure. -- API-key upstream path was not proven with a successful paid image because the available test key had no prepayment. -- A future integration test can still exercise simultaneous cancel vs settlement under load, even though Redis per-job locks, database row locks, and billing request idempotency are already present. diff --git a/test-reports/batch-image-20260706-codex/codex-report.md b/test-reports/batch-image-20260706-codex/codex-report.md deleted file mode 100644 index 3497f2d994..0000000000 --- a/test-reports/batch-image-20260706-codex/codex-report.md +++ /dev/null @@ -1,89 +0,0 @@ -# Codex Batch Image QA Report - -Date: 2026-07-06 -Tester: Codex -Baseline commits: - -- `8fab636 feat: complete batch image workflow` -- `5553d83 fix: localize antigravity image mapping labels` - -## Summary - -No blocking issue remains from the Codex-run checks. One frontend regression was found during testing: Antigravity image mapping preset labels displayed English `passthrough` while the existing UI/test expectation used Chinese `透传`. It was fixed in `5553d83`, and the full frontend suite then passed. - -## Commands Run - -| Area | Command | Result | -|---|---|---| -| Backend service tests | Docker Go 1.26.4: `go test ./internal/service -run "BatchImage|AdminService_.*BatchImage|GroupBatchImage|PricingService.*Batch|UsageBilling" -count=1 -timeout=10m` | Pass | -| Backend repository tests | Docker Go 1.26.4: `go test ./internal/repository -run "BatchImage|UsageBilling|Migrations" -count=1 -timeout=10m` | Pass | -| Backend server tests | Docker Go 1.26.4: `go test ./internal/server/... -run "APIContract|BatchImage|APIKey" -count=1 -timeout=10m` | Pass | -| Frontend typecheck | `pnpm --dir frontend typecheck` | Pass | -| Frontend build | `pnpm --dir frontend build` | Pass | -| Frontend full tests | `pnpm --dir frontend test:run` | Pass: 128 files, 803 tests | -| Local HTTP smoke | See `smoke-summary.txt` | Pass | - -## HTTP Smoke Result - -Source: `smoke-summary.txt` - -| Check | Result | -|---|---| -| Unauthorized batch list | `401 API_KEY_REQUIRED` | -| Model list | `200`, 2 models: `gemini-2.5-flash-image`, `gemini-3.1-flash-image` | -| Insufficient balance submit | `402 BATCH_IMAGE_INSUFFICIENT_BALANCE` | -| Completed batch detail | `200`, status `completed`, success `2`, fail `0`, actual cost `0.134` | -| Completed items | `200`, item count `2` | -| Completed download | `200 application/zip`, 1,602,237 bytes | -| Balance restoration after smoke | Original `1.86600000 / 0.00000000`; final `1.86600000 / 0.00000000` | - -## Findings - -| Severity | Finding | Status | -|---|---|---| -| P2 | Antigravity batch edit image mapping labels were mixed English/Chinese and failed existing UI expectation. | Fixed in `5553d83`; full frontend tests pass. | -| P3 | Frontend test output contains existing Vue/i18n warnings (`router-link`, `el-tooltip`, localstorage-file, Browserslist stale data). | Non-blocking; suite passes. | - -## Billing And Exception Coverage - -Covered by automated tests and smoke: - -- Balance reserve moves available funds to frozen funds. -- Insufficient balance returns 402 before provider submission. -- Capture rejects actual cost greater than hold. -- Capture below hold releases the remainder. -- Stale pre-provider jobs can be failed and released. -- Completed job download only returns successful outputs. - -## Access Control And Visibility - -The batch image feature has two independent gates: - -- Global runtime gate: `BATCH_IMAGE_ENABLED` controls whether `/v1/images/batches*` is available at all. If disabled, the backend returns `404 BATCH_IMAGE_DISABLED` regardless of group settings. This value is loaded at application startup, so changing the server environment requires restarting/redeploying the app container. -- Group/API-key gate: only Gemini groups with image generation enabled can enable `groups.allow_batch_image_generation`, which controls whether a user's API key may use the feature. If the global gate is enabled but the API key's group is not allowed, the backend returns `403 BATCH_IMAGE_GROUP_DISABLED`. - -Frontend visibility follows the same group/API-key gate for user-facing entry points: - -- Sidebar `/batch-image` entry is shown only when the current user has at least one active Gemini API key whose group has `allow_batch_image_generation=true`. -- User dashboard quick action is hidden under the same condition. -- Admin dashboard's shortcut to the user-facing batch image page is also hidden under the same current-user API-key condition; admin group configuration remains available under group management. -- The frontend check pages through active keys in batches of 100 and stops as soon as it finds an allowed key. The result is cached in a shared composable for sidebar/dashboard reuse, and API errors fail closed by hiding the entry. - -This frontend hiding is only a UX affordance. Backend authorization remains the source of truth, so direct API calls without an allowed group still fail. - -Quick action origin: - -- `UserDashboardQuickActions.vue` is an upstream dashboard component. The batch image button was added by the custom batch image work to fit into the existing quick action surface. -- The admin dashboard quick action block and the batch image shortcut inside it were added by the custom batch image work. -- The sidebar batch image module entry was added by the custom batch image work. - -## Residual Risks - -- Real provider failure combinations should still be tested with controlled fake/fixture provider outputs: malformed output JSONL, missing image bytes, provider cancelled after partial success, and delayed output indexing. -- Concurrent cancel vs settlement still benefits from a dedicated integration test with simultaneous requests to prove row-lock behavior under load, not only unit/static coverage. -- Google/Gemini API-key upstream success was not run because the available test key had no prepayment. The provider was verified as selectable/callable, and failed submit released hold. -- Online high-concurrency stress was intentionally skipped to avoid unnecessary provider cost; Redis per-job locks, database row locks, and billing request idempotency cover the core correctness path in code. - -## Recommendation - -Proceed to upstream review behind `BATCH_IMAGE_ENABLED` and `allow_batch_image_generation`. Before broad GA, add or run a dedicated cancel/settle concurrency integration test and a paid one-image API-key upstream success test with a properly prepaid Google key. diff --git a/test-reports/batch-image-20260706-codex/pr-description.md b/test-reports/batch-image-20260706-codex/pr-description.md deleted file mode 100644 index 2732109e43..0000000000 --- a/test-reports/batch-image-20260706-codex/pr-description.md +++ /dev/null @@ -1,58 +0,0 @@ -# PR Description Draft: Batch Image Generation MVP - -## Summary - -This PR adds an opt-in batch image generation MVP for Gemini image models through Sub2API. - -Main capabilities: - -- Public async batch image API under `/v1/images/batches*`. -- Provider support for Vertex-managed Gemini batch jobs and Gemini API batch jobs. -- Upstream account support is limited to Gemini `service_account` accounts for the Vertex provider and Gemini `apikey` accounts for the Gemini API provider. -- Redis-backed worker queue, delayed requeue, stale active recovery, and per-job locks. -- PostgreSQL job/item state, provider refs kept internal, and proxied item/ZIP downloads. -- Balance hold, capture, release, partial-failure settlement, and idempotent billing request ids. -- Frontend user batch image guide and gated navigation entry. -- Feature gates through global `BATCH_IMAGE_ENABLED`, Gemini-only group eligibility, image-generation enablement, and group-level `allow_batch_image_generation`. - -The feature is intentionally not GA by default. It should be enabled first through feature flag and group opt-in only. - -## Docs Included - -- `docs/BATCH_IMAGE_MVP.md`: API, lifecycle, billing, provider notes, config, official Google enablement, and operations checklist. -- `test-reports/batch-image-20260706-codex/test-case.md`: QA case matrix. -- `test-reports/batch-image-20260706-codex/codex-report.md`: Codex test report. -- `test-reports/batch-image-20260706-codex/claude-report.md`: Claude Code review report plus 2026-07-07 follow-up addendum. -- `test-reports/batch-image-20260706-codex/smoke-summary.txt`: local HTTP smoke result. - -## Validation - -Automated/local validation recorded in the test reports: - -- Backend batch image service/repository/server tests: pass. -- Frontend typecheck/build/full tests: pass. -- Local HTTP smoke: unauthenticated access, model listing, insufficient balance, completed status/items/download, and balance restoration. -- Settlement tests cover successful-image-only charging, zero-success completion, already-settled idempotency, billing crash idempotency, cost-over-hold rejection, pricing snapshot, bounded settlement retry, retry exhaustion release, and billing request ids. - -Online validation recorded on 2026-07-07: - -- One-image Vertex success: hold `0.0804`, actual `0.0737`, release `0.0067`, final `frozen_balance=0`. -- Immediate cancel after submit: hold released, charged `0`, no capture usage log. -- Two-item partial failure: one success, one failure, charged one image only, `errors.json` contains failed item, final `frozen_balance=0`. -- Gemini API-key provider path: provider selectable/callable; test key had no prepayment, so successful generation was not continued; failed submit released hold and charged `0`. - -## Remaining Non-Blocking Gaps - -- No high-concurrency online stress test was run because it would create unnecessary provider cost and production pressure. -- Gemini API-key upstream success still needs one paid/prepaid low-cost image test when such a key is available. -- Other Gemini login/account types were not tested and are not selected by the current providers unless they can expose equivalent service-account or API-key credentials through the same provider flow. -- A future integration test can exercise simultaneous cancel vs settlement under load, although Redis per-job locks, PostgreSQL row locks, and billing idempotency are already present. -- Optional object-storage download offload could be added later: store completed outputs in GCS/S3/R2 and issue short-lived signed links so large image/ZIP downloads do not consume Sub2API server bandwidth. This should remain opt-in because it adds storage credentials, lifecycle cleanup, signed-link expiry, and access-audit requirements. - -## Rollout Recommendation - -Merge/review behind flags only: - -- Keep `BATCH_IMAGE_ENABLED=false` by default. -- Enable only for selected Gemini groups after `allow_image_generation=true`, then set `allow_batch_image_generation=true`; non-Gemini groups are intentionally not eligible for this switch. -- Start with one controlled group and monitor job state, provider errors, hold/capture/release events, and download volume before broader enablement. diff --git a/test-reports/batch-image-20260706-codex/smoke-summary.txt b/test-reports/batch-image-20260706-codex/smoke-summary.txt deleted file mode 100644 index 30696744f1..0000000000 --- a/test-reports/batch-image-20260706-codex/smoke-summary.txt +++ /dev/null @@ -1,23 +0,0 @@ -base=http://127.0.0.1:8080 -unauthorized_status=401 -unauthorized_code=API_KEY_REQUIRED -models_status=200 -models_count=2 -models_models=gemini-2.5-flash-image,gemini-3.1-flash-image -insufficient_status=402 -insufficient_code=BATCH_IMAGE_INSUFFICIENT_BALANCE -insufficient_message=insufficient balance for batch image hold -latest_completed_batch=imgbatch_8944d988d7b92fcba158a9317fe3e699 -latest_completed_status=200 -latest_items_status=200 -latest_download_status=200 application/zip 1602237 -latest_id=imgbatch_8944d988d7b92fcba158a9317fe3e699 -latest_status=completed -latest_success_count=2 -latest_fail_count=0 -latest_actual_cost=0.134 -latest_item_count=2 -original_balance=1.86600000 -original_frozen_balance=0.00000000 -final_balance=1.86600000 -final_frozen_balance=0.00000000 diff --git a/test-reports/batch-image-20260706-codex/test-case.md b/test-reports/batch-image-20260706-codex/test-case.md deleted file mode 100644 index 361e858ada..0000000000 --- a/test-reports/batch-image-20260706-codex/test-case.md +++ /dev/null @@ -1,47 +0,0 @@ -# Batch Image QA Test Case - -Date: 2026-07-06 -Branch: `feature/batch-image-foundation` - -## Scope - -Validate the Sub2API batch image feature before broader external review: - -- Gateway API authentication and public response shape -- Available batch image model listing -- Balance hold failure path before upstream submission -- Completed job detail, item listing, and download path -- Billing hold, release, capture, settlement, and recovery unit coverage -- Frontend batch image page type/build/test health -- Agent-copy instruction text for slower polling and resume records -- PR docs/readiness materials for upstream review - -## Test Data - -- Local endpoint: `http://127.0.0.1:8080` -- Local completed batch used for read/download smoke: `imgbatch_8944d988d7b92fcba158a9317fe3e699` -- No API key or secret is stored in this report. - -## Cases - -| ID | Case | Expected | -|---|---|---| -| BI-API-001 | `GET /v1/images/batches` without key | `401`, `API_KEY_REQUIRED` | -| BI-API-002 | `GET /v1/images/batches/models` with key | `200`, returns supported image batch models | -| BI-API-003 | Submit with intentionally insufficient balance | `402`, `BATCH_IMAGE_INSUFFICIENT_BALANCE`, no provider submission | -| BI-API-004 | Fetch completed batch detail | `200`, terminal status and cost fields present | -| BI-API-005 | Fetch completed batch items | `200`, success/failure item summary present | -| BI-API-006 | Download completed successful images | `200 application/zip`, non-empty archive | -| BI-BILL-001 | Reserve balance hold | Available balance decreases, frozen balance increases | -| BI-BILL-002 | Capture hold with actual cost below hold | Remainder released, frozen balance returns to zero | -| BI-BILL-003 | Reject actual cost above hold | Settlement fails before over-capture | -| BI-BILL-004 | Release stale/unsubmitted hold | Stale job fails and frozen funds are released | -| BI-FE-001 | Frontend typecheck/build | Pass | -| BI-FE-002 | Full frontend test suite | Pass | -| BI-FE-003 | Batch image guide copy text | Includes slower polling and local resume-record requirements | -| BI-ONLINE-001 | One-image success settlement balance closure | Hold `0.0804`, actual `0.0737`, release `0.0067`; `frozen_balance` returns `0` | -| BI-ONLINE-002 | Immediate cancel after submit | Hold released, charged `0` | -| BI-ONLINE-003 | Google/Gemini API-key provider path | Account selectable/callable, models list returns `provider=gemini_api`; test key has no prepayment so no successful generation attempted; submit failure released hold, charged `0` | -| BI-ONLINE-004 | Two-item partial failure | One item succeeded, one item failed; charged one image only, `errors.json` contains failed item, `frozen_balance` returns `0` | -| BI-DOC-001 | Batch image MVP feature doc | Includes API surface, lifecycle, billing, provider notes, config, official Google enablement, and PR hygiene | -| BI-DOC-002 | PR description draft | Summarizes feature scope, tests, feature flags, and remaining non-blocking gaps for upstream review | diff --git a/test-reports/batch-image-20260706-fix-verification/codex-claude-fix-report.md b/test-reports/batch-image-20260706-fix-verification/codex-claude-fix-report.md deleted file mode 100644 index e5c51ffd51..0000000000 --- a/test-reports/batch-image-20260706-fix-verification/codex-claude-fix-report.md +++ /dev/null @@ -1,55 +0,0 @@ -# Batch Image Fix Verification Report - -Date: 2026-07-06 -Branch: feature/batch-image-foundation - -## Scope - -This pass fixes the remaining QA findings from the batch image reports: - -- Add a bounded settlement billing retry path for `SETTLEMENT_BILLING_FAILED`. -- Prevent jobs from staying in `settling` with frozen balance forever after repeated billing failures. -- Clarify cancel billing copy: only images indexed as successful are billed, and the remaining hold is released. -- Ask Claude Code to re-review the fix after Codex validation. - -## Codex Changes - -- `BatchImageSettlementService` now uses `batchImageSettlementMaxRetries = 5`. -- `SetBatchImageJobSettlementFailed` atomically increments and returns `retry_count` with `RETURNING retry_count`. -- When capture billing fails and reaches the retry limit, settlement releases the frozen hold and transitions the job to `failed` with `SETTLEMENT_BILLING_RETRY_EXHAUSTED`. -- The worker pipeline re-reads the job after settlement billing errors and acknowledges terminal jobs instead of requeueing forever. -- A transition-failure regression test verifies that release retry is idempotent when release succeeds but the failed-state transition fails. -- User-facing cancel copy and the copyable skill instructions now say indexed successful images are billed and the remaining hold is released. - -## Codex Verification - -| Check | Result | -|---|---| -| `go test -tags unit ./internal/service -run 'BatchImage(Settlement\|Pipeline\|Public\|Processor\|BillingRecovery)' -count=1 -timeout=8m` | Pass | -| `pnpm --dir frontend typecheck` | Pass | -| `pnpm --dir frontend build` | Pass, with existing Vite chunk/Browserslist warnings | -| `go test -tags integration ./internal/repository -run '^TestBatchImageRepository_SetBatchImageJobSettlementFailed$'` | Compiled; skipped inside Docker because Docker socket is unavailable to testcontainers | - -## Claude Code Verification - -Claude Code model used: `sonnet --safe-mode --effort low`. - -First review result: - -- No P1/P2 blocker found in the bounded retry and cancel-copy fix. -- Flagged one residual risk: if release succeeds but transition to `failed` fails, the next run could call release again; requested confirmation of idempotency. - -Codex follow-up: - -- Added `TestBatchImageSettlementRetryExhaustedReleaseIsIdempotentAfterTransitionFailure`. -- Confirmed the real `UsageBillingRepository` calls `claimUsageBillingRequest` before `ReleaseBatchImageBalance`; duplicate `BatchImageReleaseRequestID(batchID)` returns `Applied:false`. - -Second Claude review result: - -- Confirmed the new test closes the prior risk. -- No remaining P1/P2 issue. -- Remaining P3: repository integration should be run in an environment with Docker socket/testcontainers available. - -## Residual Risk - -- Repository integration was not fully executed in the Docker-based Go test container because testcontainers could not access Docker. The SQL change is small and compiled, but should be run once in an environment where repository integration tests can start containers.