github/GitNexus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(parsing): remove sequential-parser fallback (U20 design pivot)

Browse Source 
The worker pool's resilience layers — respawn budget, circuit breaker,
quarantine, slot-attribution, cumulative timeout — are now the SOLE
contract for handling worker failures. Two sequential-reparse paths
are removed from processParsing:

1. **U20.U1 sequential gap-fill for quarantined chunk files** (just
   added in commit 7dd489e9, now reverted). The pre-emptive rescue
   would re-run processParsingSequential on the file that ALREADY
   killed a worker — which for the most common quarantine cause
   (tree-sitter native SIGSEGV on a pathological file) re-triggers
   the same native crash on the main thread, killing the entire
   analyze. The "rescue" turned silent missing-symbols into a louder
   analyze-wide crash. Drop the rescue; accept the per-run gap.

2. **Pre-existing WorkerPoolDispatchError catch-block sequential
   fallback** (in production since PR #1693's resilience layer
   landed). Same risk class — when the pool exhausts its respawn
   budget / trips the circuit breaker, the failing files are
   precisely the ones likely to crash a sequential parser too. The
   "graceful degradation" hid pool failures behind degraded-but-
   completing analyze runs, making operational issues harder to
   surface and diagnose. Drop the catch-block; WorkerPoolDispatchError
   propagates to the analyze entry point where the user sees a clear
   hard signal.

What stays:
- The `skipWorkers: true` / small-repo path that uses
  `processParsingSequential` as the EXPLICIT primary path (not a
  fallback). Caller-driven opt-out and tiny-repo perf optimization
  are different intents.
- U2's chunk-cache write suppression in parse-impl.ts (commit
  7c9c9556). When quarantine fires, the chunk stays uncached so the
  next analyze with a fresh pool retries the file cleanly. That's
  the cross-run correctness Codex's adversarial review actually
  asked for.
- The per-chunk quarantine warn log (parsing-processor.ts) — operators
  see which files were skipped, both immediately and across runs.

What changed:
- `processParsing` worker-path try-block: unwrapped. The
  `processParsingWithWorkers` call is now direct (no try/catch
  wrapping); errors propagate to the chunk-loop caller.
- `parsing-worker-fallback.test.ts` rewritten: the previous 5 tests
  asserted graceful sequential-fallback behavior. Replaced with 3
  tests pinning the new contract — raw Error propagates, WorkerPool-
  DispatchError propagates with fallbackExcludePaths intact, normal
  quarantine signal does NOT throw and surfaces via progress detail.
- `parse-impl-quarantine-cache-skip.test.ts` (U20 integration test)
  updated: poison.ts is NOT in the post-run graph; surviving files
  are; chunk-cache stays empty; second pass re-dispatches and leaves
  cache empty.
- Plan doc updated to mark R1 as dropped and explain the U20 pivot
  in the Summary.

User decision: explicit directive ("let's remove the sequential
fallback entirely we must rely on entirely that the parallel process
is resilient enough to work itself through the code base"). The pool's
resilience layers are designed for this — respawn budget, circuit
breaker, quarantine, slot-generation, cumulative-timeout cap — and
adding a layer below them was redundant insurance with real downside.

Tests: 269/269 unit files (6135 tests) green. 31/31 worker-pool +
parse-impl integration tests green. The 2 reported "errors" in the
integration run are the pre-existing intentional-process.exit unhandled-
exception leaks from test workers — unchanged by U20.

References plan: docs/plans/2026-05-20-002-fix-chunk-cache-corruption-on-worker-quarantine-plan.md
This commit is contained in:
Gergo Magyar
2026-05-20 13:40:38 +01:00
parent 9d1436a2e3
commit be1f65cee8
3 changed files with 217 additions and 309 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gitnexus/src/core/ingestion/parsing-processor.ts
+59 -112
View File
@@ -37,7 +37,7 @@ import {
} from './utils/template-arguments.js';
import type { LanguageProvider } from './language-provider.js';
import type { ParsedFile } from 'gitnexus-shared';
import { WorkerPool, WorkerPoolDispatchError } from './workers/worker-pool.js';
import { WorkerPool } from './workers/worker-pool.js';
import { logger } from '../logger.js';
import type {
ParseWorkerResult,
@@ -882,128 +882,75 @@ export const processParsing = async (
`[scope-resolution prof] worker pool engaged for ${files.length} files — cross-phase tree cache will be empty; scope-resolution re-parses.`,
);
}
try {
const data = await processParsingWithWorkers(
graph,
files,
symbolTable,
astCache,
workerPool,
reportProgress,
outRawResults,
);
// Session-scoped quarantine (worker-pool resilience Layer 3): surface
// any files this pool has decided are unsafe for workers so the
// operator can see what was skipped. The pool already filtered them
// out of dispatch; we only need to log + progress-report. Quarantine
// is session-scoped per pool instance — a fresh `createWorkerPool`
// call clears it.
//
// Dedup: log full path list only for entries newly quarantined since
// the previous dispatch on the same pool. The per-chunk progress
// message still surfaces the count for UX continuity, but the
// structured `quarantinedFiles` payload is only emitted when there
// is new signal — prevents O(quarantine × chunks) log spam.
const quarantineSnapshot = workerPool.getQuarantinedPaths?.() ?? [];
const quarantineSet = new Set(quarantineSnapshot);
if (quarantineSet.size > 0) {
const quarantinedInChunk = files.filter((file) => quarantineSet.has(file.path));
if (quarantinedInChunk.length > 0) {
const seenForPool = loggedQuarantineByPool.get(workerPool) ?? new Set<string>();
const newlyQuarantined = quarantinedInChunk
.map((file) => file.path)
.filter((p) => !seenForPool.has(p));
for (const p of newlyQuarantined) seenForPool.add(p);
loggedQuarantineByPool.set(workerPool, seenForPool);
if (newlyQuarantined.length > 0) {
logger.warn(
{
newlyQuarantined,
cumulativeQuarantine: quarantineSet.size,
chunkSkipped: quarantinedInChunk.length,
},
`Worker quarantine: ${newlyQuarantined.length} new file(s) skipped this chunk ` +
`(${quarantinedInChunk.length} skipped total, ${quarantineSet.size} cumulative).`,
);
}
reportProgress?.(
lastProgress,
files.length,
`${quarantinedInChunk.length} worker-quarantined file(s) skipped`,
);
// U20.U1: Sequential gap-fill. The worker pool's Layer 3
// quarantine filtered these files out of dispatch, so the
// worker results above are missing their symbols/imports/
// calls/heritage. Without this reparse, the graph for THIS
// run is silently incomplete — exactly the silent-corruption
// class the Codex adversarial review of PR #1693 flagged.
//
// Running `processParsingSequential` on JUST the quarantined
// files mirrors the WorkerPoolDispatchError catch-block
// shape below (line 961). Worker-extracted data for the
// surviving files flows through `data` to the caller's
// deferred-extraction merge; sequential output for the
// quarantined files writes directly to the graph here.
//
// The chunk-loop caller's cache write at parse-impl.ts:500-
// 507 will NOT cache this chunk because U20.U2's guard
// detects the same quarantine intersection — so next run
// gets a cache miss and a fresh pool gives the file
// another chance.
// U20 design pivot: the worker pool's resilience layers
// (respawn budget, circuit breaker, quarantine, slot-attribution,
// cumulative timeout) are the SOLE contract for handling worker
// failures. There is no sequential-parser fallback for either
// partial quarantine or full pool failure — the operator must see
// a clear hard signal when workers can't recover, instead of a
// silently-degraded graph from a possibly-crashing main-thread
// sequential parser. A failing tree-sitter native binding that
// quarantined a worker would, under the previous design, re-trigger
// the same SIGSEGV on the main thread; we avoid that risk entirely.
//
// - Partial quarantine: the file is missing from this run's graph;
// the per-chunk warn log below surfaces it; U2's chunk-cache
// write-guard in parse-impl.ts keeps the chunk uncached so the
// next analyze gets a cache miss and a fresh pool retries.
// - Full pool failure: `WorkerPoolDispatchError` propagates from
// `processParsingWithWorkers` up through this function. The
// analyze run errors out instead of falling back to sequential.
const data = await processParsingWithWorkers(
graph,
files,
symbolTable,
astCache,
workerPool,
reportProgress,
outRawResults,
);
// Session-scoped quarantine (worker-pool resilience Layer 3): surface
// any files this pool has decided are unsafe for workers so the
// operator can see what was skipped. The pool already filtered them
// out of dispatch; we only need to log + progress-report. Quarantine
// is session-scoped per pool instance — a fresh `createWorkerPool`
// call clears it.
//
// Dedup: log full path list only for entries newly quarantined since
// the previous dispatch on the same pool. The per-chunk progress
// message still surfaces the count for UX continuity, but the
// structured `quarantinedFiles` payload is only emitted when there
// is new signal — prevents O(quarantine × chunks) log spam.
const quarantineSnapshot = workerPool.getQuarantinedPaths?.() ?? [];
const quarantineSet = new Set(quarantineSnapshot);
if (quarantineSet.size > 0) {
const quarantinedInChunk = files.filter((file) => quarantineSet.has(file.path));
if (quarantinedInChunk.length > 0) {
const seenForPool = loggedQuarantineByPool.get(workerPool) ?? new Set<string>();
const newlyQuarantined = quarantinedInChunk
.map((file) => file.path)
.filter((p) => !seenForPool.has(p));
for (const p of newlyQuarantined) seenForPool.add(p);
loggedQuarantineByPool.set(workerPool, seenForPool);
if (newlyQuarantined.length > 0) {
logger.warn(
{
reparsedPaths: quarantinedInChunk.map((f) => f.path),
count: quarantinedInChunk.length,
newlyQuarantined,
cumulativeQuarantine: quarantineSet.size,
chunkSkipped: quarantinedInChunk.length,
},
`Running sequential reparse for ${quarantinedInChunk.length} worker-quarantined ` +
`file(s) to keep this run's graph complete.`,
);
await processParsingSequential(
graph,
quarantinedInChunk,
symbolTable,
astCache,
scopeTreeCache,
reportProgress,
`Worker quarantine: ${newlyQuarantined.length} new file(s) skipped this chunk ` +
`(${quarantinedInChunk.length} skipped total, ${quarantineSet.size} cumulative).`,
);
}
}
return data;
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
let fallbackFiles = files;
if (err instanceof WorkerPoolDispatchError && err.fallbackExcludePaths.length > 0) {
const excluded = new Set(err.fallbackExcludePaths);
fallbackFiles = files.filter((file) => !excluded.has(file.path));
logger.warn(
{
skippedPaths: err.fallbackExcludePaths,
},
'Skipping worker-excluded files in sequential fallback:',
);
reportProgress?.(
lastProgress,
files.length,
`Skipping ${files.length - fallbackFiles.length} worker-excluded file(s) in sequential fallback`,
`${quarantinedInChunk.length} worker-quarantined file(s) skipped`,
);
}
logger.warn({ message }, 'Worker pool parsing stopped; continuing with sequential parser:');
reportProgress?.(
lastProgress,
files.length,
`Sequential fallback after worker issue: ${message}`,
);
await processParsingSequential(
graph,
fallbackFiles,
symbolTable,
astCache,
scopeTreeCache,
reportProgress,
);
return null;
}
return data;
}
// Fallback: sequential parsing (no pre-extracted data)
gitnexus/test/integration/parse-impl-quarantine-cache-skip.test.ts
+46 -33
View File
@@ -1,5 +1,5 @@
/**
* U20.U3 — Integration regression test for chunk-cache corruption on
* U20 — Integration regression test for chunk-cache corruption on
* worker quarantine.
*
* Pins the fix for the Codex adversarial review finding on PR #1693
@@ -27,20 +27,27 @@
* nodes give the merge step deterministic content to add to the
* graph).
*
* Assertions:
* - U1 (sequential gap-fill): the graph contains a `Function` node
* named `poison` after the run. The custom worker never emits
* anything for `poison.ts` (it crashes on it), so the only path
* for that symbol to land in the graph is `processParsing`'s
* sequential reparse of the quarantined-in-chunk file — which
* uses the real tree-sitter parser against the actual
* `export function poison() {}` source.
* U20 design pivot — no sequential fallback. The U1 sequential
* reparse for quarantined chunk files was removed: relying on the
* worker pool's resilience layers (respawn budget, circuit breaker,
* quarantine, slot-attribution, cumulative timeout) as the SOLE
* contract avoids re-triggering tree-sitter native crashes on the
* main thread and gives operators a clear hard signal when workers
* exhaust. Quarantined files are missing from this run's graph;
* they're surfaced in the per-chunk warn log; U2's cache-skip keeps
* the chunk uncached so the next analyze with a fresh pool retries.
*
* Assertions exercised here:
* - Worker-path runs and produces results for surviving files
* (good_a, good_c) via the synthesized worker output.
* - The quarantined file (poison.ts) is NOT in the graph — no
* sequential reparse fired.
* - U2 (cache-write suppression): `parseCache.entries` does NOT
* contain the chunk hash after the run. `parseCache.usedKeys`
* DOES contain it (chunk was processed; the cache write was
* specifically skipped). A cross-run scenario verifies that a
* subsequent dispatch with a fresh pool re-attempts the chunk
* (cache miss).
* (cache miss) and the cache stays empty for that chunk.
*
* Why integration over unit:
* - The fix lives at the boundary between processParsing
@@ -56,10 +63,7 @@
* reused inline here (the READY_PREAMBLE + test-worker script
* composition).
*
* Wall-clock budget: well under 5 s under normal CI conditions. The
* test bounds Worker startup at the pool's WORKER_READY_TIMEOUT_MS
* (5 s) and dispatch at subBatchIdleTimeoutMs default (30 s) — both
* generous given the worker never blocks the test on real work.
* Wall-clock budget: well under 5 s under normal CI conditions.
*/
import { describe, it, expect, afterEach, beforeEach } from 'vitest';
import { tmpdir } from 'node:os';
@@ -183,7 +187,7 @@ const FIXTURE_FILES = {
'src/good_c.ts': 'export function good_c() { return 3; }\n',
};
describe('U20.U3: parse-impl quarantine + chunk-cache integration (PR #1693 Codex finding)', () => {
describe('U20: parse-impl quarantine + chunk-cache integration (PR #1693 Codex finding)', () => {
let tempDir: string;
let repoDir: string;
let workerPath: string;
@@ -212,7 +216,7 @@ describe('U20.U3: parse-impl quarantine + chunk-cache integration (PR #1693 Code
rmSync(tempDir, { recursive: true, force: true });
});
it('runs sequential reparse for poison.ts AND suppresses cache write for the chunk', async () => {
it('worker quarantine leaves poison.ts out of the graph AND suppresses chunk-cache write', async () => {
const filePaths = Object.keys(FIXTURE_FILES);
const scanned = filePaths.map((rel) => ({
path: rel,
@@ -260,21 +264,23 @@ describe('U20.U3: parse-impl quarantine + chunk-cache integration (PR #1693 Code
},
);
// U1 assertion: poison.ts's symbols are in the graph despite the
// worker never emitting anything for that file. The only path for
// a Function node named `poison` to land here is processParsing's
// sequential reparse of the quarantined-in-chunk file (using the
// real tree-sitter parser against the actual source).
const nodes = Array.from(graph.nodes.values());
const poisonNode = nodes.find(
(n) => n.label === 'Function' && (n.properties as { name?: string }).name === 'poison',
);
expect(poisonNode).toBeDefined();
// The other two files' symbols came from the custom worker's
// synthesized output via the normal worker-path merge. Pin them
// too so a regression that drops worker results entirely doesn't
// silently pass this test.
// Quarantine contract: poison.ts is genuinely missing from the
// graph for this run. The custom worker crashed on it; no
// sequential reparse rescued it; the operator sees the per-chunk
// quarantine warn log. A future analyze with a fresh pool gets
// another chance via U2's cache-skip below.
expect(
nodes.some(
(n) => n.label === 'Function' && (n.properties as { name?: string }).name === 'poison',
),
).toBe(false);
// Surviving files' symbols come from the custom worker's
// synthesized output via the normal worker-path merge. Pinning
// them here catches a regression that would drop worker results
// entirely when quarantine fires.
expect(
nodes.some(
(n) => n.label === 'Function' && (n.properties as { name?: string }).name === 'good_a',
@@ -369,17 +375,24 @@ describe('U20.U3: parse-impl quarantine + chunk-cache integration (PR #1693 Code
},
);
// Cache stayed empty (still no entry for this chunk hash).
// Cache stayed empty (still no entry for this chunk hash) — the
// load-bearing cross-run protection.
expect(parseCache.entries.has(expectedChunkHash)).toBe(false);
expect(parseCache.usedKeys.has(expectedChunkHash)).toBe(true);
// Graph is still complete — sequential gap-fill ran again on
// poison.ts in the second pass.
// Worker path ran again; surviving files in the graph; poison
// still absent per the U20 contract (workers are the sole
// resilience layer, no sequential reparse).
const nodes2 = Array.from(graph2.nodes.values());
expect(
nodes2.some(
(n) => n.label === 'Function' && (n.properties as { name?: string }).name === 'good_a',
),
).toBe(true);
expect(
nodes2.some(
(n) => n.label === 'Function' && (n.properties as { name?: string }).name === 'poison',
),
).toBe(true);
).toBe(false);
}
});
});
gitnexus/test/unit/parsing-worker-fallback.test.ts
+112 -164
View File
@@ -1,3 +1,30 @@
/**
* processParsing — worker-pool error handling contract.
*
* U20 design pivot (PR #1693): there is NO sequential-parser fallback
* when the worker pool fails. The pool's resilience layers (respawn
* budget, circuit breaker, quarantine, slot-attribution, cumulative
* timeout) are the sole contract for handling worker failures. When
* those exhaust, `processParsing` propagates the error to the caller
* — `runChunkedParseAndResolve` and the analyze entry point above it.
*
* This file replaces the previous sequential-fallback tests (which
* asserted that processParsing caught WorkerPoolDispatchError and
* called processParsingSequential on the remaining files). The new
* contract is "errors propagate, no rescue."
*
* Why removing the fallback was the right call:
* - The fallback ran the SAME tree-sitter parser the worker just
* crashed on, but on the main thread. A native crash (SIGSEGV
* from a tree-sitter binding) in the worker would re-trigger the
* same SIGSEGV on the main thread, killing the whole analyze
* instead of just the worker.
* - It hid pool failures behind a degraded-but-completing analyze
* run, making them harder to detect and diagnose.
* - U2's chunk-cache write suppression keeps cross-run retry
* working: a quarantined file's chunk stays uncached, so the
* next analyze with a fresh pool gets another chance.
*/
import { describe, expect, it, vi } from 'vitest';
import { createASTCache } from '../../src/core/ingestion/ast-cache.js';
import { processParsing } from '../../src/core/ingestion/parsing-processor.js';
@@ -6,157 +33,9 @@ import { WorkerPoolDispatchError } from '../../src/core/ingestion/workers/worker
import { createKnowledgeGraph } from '../../src/core/graph/graph.js';
import { createSymbolTable } from '../../src/core/ingestion/model/symbol-table.js';
describe('processParsing worker fallback', () => {
it('continues sequentially with visible progress when the worker pool times out', async () => {
describe('processParsing — worker-pool error propagation (U20)', () => {
it('propagates a raw worker-pool throw to the caller without rescuing', async () => {
const graph = createKnowledgeGraph();
const progressCounts: number[] = [];
const progressDetails: string[] = [];
const workerPool: WorkerPool = {
size: 1,
dispatch: vi.fn(async (_items, onProgress?: (filesProcessed: number) => void) => {
onProgress?.(1);
throw new Error('injected worker idle timeout');
}),
terminate: vi.fn(async () => undefined),
};
const result = await processParsing(
graph,
[{ path: 'src/a.ts', content: 'export function a() { return 1; }\n' }],
createSymbolTable(),
createASTCache(),
createASTCache(),
(current, _total, detail) => {
progressCounts.push(current);
progressDetails.push(detail);
},
workerPool,
);
expect(result).toBeNull();
expect(progressDetails).toContain(
'Sequential fallback after worker issue: injected worker idle timeout',
);
expect(progressCounts).toEqual([...progressCounts].sort((a, b) => a - b));
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'a'),
).toBe(true);
});
it('skips worker-timeout singleton files during sequential fallback', async () => {
const graph = createKnowledgeGraph();
const progressDetails: string[] = [];
const workerPool: WorkerPool = {
size: 1,
dispatch: vi.fn(async () => {
throw new WorkerPoolDispatchError('injected worker idle timeout', ['src/stuck.ts']);
}),
terminate: vi.fn(async () => undefined),
};
const result = await processParsing(
graph,
[
{ path: 'src/stuck.ts', content: 'export function stuck() { return 0; }\n' },
{ path: 'src/a.ts', content: 'export function a() { return 1; }\n' },
],
createSymbolTable(),
createASTCache(),
createASTCache(),
(_current, _total, detail) => {
progressDetails.push(detail);
},
workerPool,
);
expect(result).toBeNull();
expect(progressDetails).toContain('Skipping 1 worker-excluded file(s) in sequential fallback');
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'a'),
).toBe(true);
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'stuck'),
).toBe(false);
});
it('skips worker-error in-flight file during sequential fallback', async () => {
const graph = createKnowledgeGraph();
const progressDetails: string[] = [];
const workerPool: WorkerPool = {
size: 1,
dispatch: vi.fn(async () => {
throw new WorkerPoolDispatchError('Worker 0 error: native crash', ['src/crashed.ts']);
}),
terminate: vi.fn(async () => undefined),
};
const result = await processParsing(
graph,
[
{ path: 'src/crashed.ts', content: 'export function crashed() { return 0; }\n' },
{ path: 'src/a.ts', content: 'export function a() { return 1; }\n' },
],
createSymbolTable(),
createASTCache(),
createASTCache(),
(_current, _total, detail) => {
progressDetails.push(detail);
},
workerPool,
);
expect(result).toBeNull();
expect(progressDetails).toContain('Skipping 1 worker-excluded file(s) in sequential fallback');
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'a'),
).toBe(true);
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'crashed'),
).toBe(false);
});
it('skips worker-exit in-flight file during sequential fallback', async () => {
const graph = createKnowledgeGraph();
const progressDetails: string[] = [];
const workerPool: WorkerPool = {
size: 1,
dispatch: vi.fn(async () => {
throw new WorkerPoolDispatchError(
'Worker 0 exited with code 134. Likely OOM or native addon failure (in-flight: src/oom.ts).',
['src/oom.ts'],
);
}),
terminate: vi.fn(async () => undefined),
};
const result = await processParsing(
graph,
[
{ path: 'src/oom.ts', content: 'export function oom() { return 0; }\n' },
{ path: 'src/a.ts', content: 'export function a() { return 1; }\n' },
],
createSymbolTable(),
createASTCache(),
createASTCache(),
(_current, _total, detail) => {
progressDetails.push(detail);
},
workerPool,
);
expect(result).toBeNull();
expect(progressDetails).toContain('Skipping 1 worker-excluded file(s) in sequential fallback');
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'a'),
).toBe(true);
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'oom'),
).toBe(false);
});
it('runs full sequential fallback when the worker pool throws a non-WorkerPoolDispatchError', async () => {
const graph = createKnowledgeGraph();
const progressDetails: string[] = [];
const workerPool: WorkerPool = {
size: 1,
dispatch: vi.fn(async () => {
@@ -165,10 +44,87 @@ describe('processParsing worker fallback', () => {
terminate: vi.fn(async () => undefined),
};
await expect(
processParsing(
graph,
[{ path: 'src/a.ts', content: 'export function a() { return 1; }\n' }],
createSymbolTable(),
createASTCache(),
createASTCache(),
() => {},
workerPool,
),
).rejects.toThrow('replacement worker failed');
// No sequential fallback ran, so the graph stays empty.
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'a'),
).toBe(false);
});
it('propagates WorkerPoolDispatchError with fallbackExcludePaths intact', async () => {
const graph = createKnowledgeGraph();
const workerPool: WorkerPool = {
size: 1,
dispatch: vi.fn(async () => {
throw new WorkerPoolDispatchError(
'Worker pool circuit breaker tripped: 2 consecutive failures on slot 0',
['src/poison.ts'],
);
}),
terminate: vi.fn(async () => undefined),
};
const rejection = processParsing(
graph,
[
{ path: 'src/poison.ts', content: 'export function poison() { return 0; }\n' },
{ path: 'src/a.ts', content: 'export function a() { return 1; }\n' },
],
createSymbolTable(),
createASTCache(),
createASTCache(),
() => {},
workerPool,
);
await expect(rejection).rejects.toBeInstanceOf(WorkerPoolDispatchError);
const err = await rejection.catch((e) => e as WorkerPoolDispatchError);
expect(err.fallbackExcludePaths).toEqual(['src/poison.ts']);
// No sequential fallback ran for either file. The caller (analyze
// entry point) is responsible for surfacing this as a hard
// failure.
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'a'),
).toBe(false);
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'poison'),
).toBe(false);
});
it('worker-path returns successfully when the pool reports a quarantine snapshot without throwing', async () => {
// Quarantine is a normal session-scoped signal: the pool filters
// quarantined files out of dispatch, returns the survivors'
// results, and reports the cumulative set via getQuarantinedPaths.
// processParsing's worker-path completes successfully on this
// partial-coverage signal — the quarantined file is missing from
// the graph, but no error is thrown. The chunk-loop caller uses
// the quarantine snapshot to decide whether to write the chunk
// cache (U2 in parse-impl.ts).
const graph = createKnowledgeGraph();
const workerPool: WorkerPool = {
size: 1,
dispatch: vi.fn(async () => []),
terminate: vi.fn(async () => undefined),
getQuarantinedPaths: () => ['src/poison.ts'],
};
const progressDetails: string[] = [];
const result = await processParsing(
graph,
[
{ path: 'src/keep.ts', content: 'export function keep() { return 0; }\n' },
{ path: 'src/poison.ts', content: 'export function poison() { return 0; }\n' },
{ path: 'src/a.ts', content: 'export function a() { return 1; }\n' },
],
createSymbolTable(),
@@ -180,18 +136,10 @@ describe('processParsing worker fallback', () => {
workerPool,
);
expect(result).toBeNull();
expect(progressDetails).toContain(
'Sequential fallback after worker issue: replacement worker failed',
);
expect(
progressDetails.some((d) => d.startsWith('Skipping ') && d.includes('worker-excluded file')),
).toBe(false);
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'a'),
).toBe(true);
expect(
graph.nodes.some((node) => node.label === 'Function' && node.properties.name === 'keep'),
).toBe(true);
// Worker path returned successfully (not null — null was the
// pre-U20 sentinel for "ran sequential fallback"). The progress
// log surfaces the quarantine count for operator visibility.
expect(result).not.toBeNull();
expect(progressDetails).toContain('1 worker-quarantined file(s) skipped');
});
});