Compare commits

...
Author SHA1 Message Date
Gergo Magyar e898e19714 refactor: rename FileAllScopeBindings to FileScopeBindings and update related references
- Updated the interface name from FileAllScopeBindings to FileScopeBindings to better reflect its purpose.
- Adjusted all occurrences of the renamed interface across multiple files including parsing-processor.ts, pipeline.ts, parse-worker.ts, and type-env.ts.
- Enhanced comments and documentation to clarify the narrowing of scope bindings and the rationale behind the changes.
- Improved error handling and validation in the pipeline for file-scope bindings.
- Added integration tests to ensure the correct behavior of the BindingAccumulator and its interaction with the TypeEnv flush process.
2026-04-09 19:43:59 +01:00
Gergo Magyar 448a4b22a6 fix(SM-14): address PR #743 post-fix review findings
Four items from the deep review on commit d3c25d20 — three Low
findings plus one informational note.

Low #1 — Expose `get disposed(): boolean` for API symmetry
- BindingAccumulator's `_disposed` field was set but never read or
  exposed. `_finalized` had a public getter (`get finalized()`) but
  `_disposed` did not. Added the matching `get disposed()` getter so
  debug tooling and future Phase 9 consumers can detect a disposed
  accumulator without inspecting empty state heuristically.
- JSDoc notes that disposal and finalization are orthogonal lifecycle
  dimensions — a disposed accumulator may or may not be finalized.

Low #2 — Test for Tier 0 "don't overwrite" protection
- Production enrichment loop at pipeline.ts:1104-1108 has a priority
  guard:
    if (!fileExports.has(name)) { fileExports.set(name, type); }
  preventing a worker-path binding from clobbering a higher-quality
  Tier 0 SymbolTable entry. The existing `runEnrichmentLoop` test
  helper in binding-accumulator.test.ts was missing this guard, and
  no test exercised the priority branch.
- Fixed the helper to mirror the production guard.
- Added a new test: "does not overwrite existing SymbolTable entry
  (Tier 0 priority)" — pre-populates exportedTypeMap with an
  "SymbolTableAuthoritativeType" entry, runs the enrichment loop
  against an accumulator with "WorkerInferredType" for the same name,
  asserts the authoritative type survives.

Low #3 — Move finalize() to before the enrichment loop
- Previously, finalize() was called at pipeline.ts:1715 (inside
  runPipelineFromRepo, AFTER runChunkedParseAndResolve had already
  returned). The enrichment loop at pipeline.ts:1087 (inside
  runChunkedParseAndResolve) consumed the still-mutable accumulator.
  The `finalized` state was therefore not a reliable "all reads are
  done" signal — it was a "no more writes" signal that arrived later
  than the actual last read.
- Moved finalize() to immediately before the enrichment loop at line
  1087. By that point all worker-path appends (line 934) and all
  sequential-path flushes (via processCalls at line 1051, also inside
  runChunkedParseAndResolve) have completed. Grep confirmed no further
  `bindingAccumulator.appendFile` calls exist outside runChunkedParseAndResolve.
- Lifecycle contract is now explicit:
    append phase → finalize → consume → dispose
- Replaced the old finalize() call at line 1715 with an explanatory
  comment pointing to the new seam.

Informational — parsing-processor.ts TypeEnv clarification
- parsing-processor.ts builds a FieldExtractor-only TypeEnv that is
  intentionally NOT flushed into the accumulator — the accumulator
  feed happens later in call-processor.ts via its own flush() call.
  A future reader might see `buildTypeEnv()` here and try to add a
  flush call, double-counting entries and tripping the single-use
  invariant.
- Added a multi-line comment explaining the ownership rule and
  cross-referencing PR #743 and plan 2026-04-09-005.

Verification
- `tsc --noEmit` clean
- 3110 unit tests pass (+1 new Tier 0 priority test)
- 1766 resolver integration tests pass — critically, the finalize()
  relocation did not regress any real pipeline path, proving all
  writes complete before the new finalize point
- Zero regressions

Plan: docs/plans/2026-04-09-005-fix-sm14-sequential-path-memory-regression-plan.md
Review: https://github.com/abhigyanpatwari/GitNexus/pull/743#issuecomment-4216262583
2026-04-09 18:52:51 +01:00
Gergo Magyar d3c25d2093 fix(SM-14): close sequential-path memory regression (Codex adversarial review)
Addresses the medium-severity finding from Codex's adversarial review of
commit 803631fe: the sequential path's `typeEnv.flush()` was still
writing every scope (file + function) into the BindingAccumulator, and
the accumulator stayed alive through Phase 14 and runGraphAnalysisPhases
with no reader. On fallback runs (workers disabled or unavailable),
large repos accumulated heap for nothing.

Applies BOTH Codex remediations — narrowing AND disposal:

R1 — Narrow typeEnv.flush() to file-scope only
- The sequential path now mirrors the worker-path narrowing from
  commit 803631fe. `flush()` iterates only `env.get(FILE_SCOPE)`
  instead of the nested `for (scope, scopeMap) of env` loop, writing
  entries with `scope: ''` hardcoded. Function-scope bindings never
  reach the accumulator from either execution path until a Phase 9
  consumer lands.
- The earlier rationale for keeping sequential-path full-scope data
  ("preserve a Phase 9 prototyping sample") did not survive the
  Codex challenge — Phase 9 authors use synthetic fixtures, and a
  live-repo sample from the sequential-only path isn't representative
  of production worker-dominant runs.
- Phase 9 reversion path documented inline at the flush() seam.

R2 — Add BindingAccumulator.dispose()
- New public method clears `_allByFile`, `_fileScopeByFile`, and
  explicitly resets `_totalBindings = 0` (feasibility review caught
  that clearing the maps alone would leave the `totalBindings` getter
  reporting stale counts). Idempotent and orthogonal to finalize() —
  calling dispose() doesn't change the finalized state.
- Post-dispose contract: all read methods return empty/undefined
  state matching a never-appended accumulator. Documented in class
  JSDoc with the lifecycle sequence.
- Used before finalize(): accumulator behaves like a fresh one,
  appends still succeed.
- Used after finalize(): reads return empty but appends still throw
  the existing "finalized" error.

R3 — Wire dispose() into the pipeline
- Inserted immediately after the dev telemetry log at pipeline.ts
  line ~1723, before `runCrossFileBindingPropagation` (Phase 14) and
  `runGraphAnalysisPhases`. Sequence is:
    enrichment loop → finalize → telemetry → dispose → Phase 14
  The telemetry log captures peak state before disposal, then the
  heap footprint is released for the long tail of graph analysis.
- Verified both runCrossFileBindingPropagation and
  runGraphAnalysisPhases signatures do NOT take a bindingAccumulator
  parameter — grep confirmed the last usage is at line 1720.

Tests (+6 scenarios)
- test/unit/type-env.test.ts: existing "flushes function-scoped
  bindings into accumulator" test was rewritten as a negative
  assertion ("does NOT flush function-scoped bindings, narrowed per
  PR #743 Codex review"). Plus a new "narrows mixed file-scope and
  function-scope env to file-scope only" test that builds a
  realistic TypeScript file with both scopes and asserts only the
  file-scope entry lands in the accumulator. This is the R1 red/
  green signal — both tests were written test-first and failed
  against the pre-narrowing flush() body.
- test/unit/binding-accumulator.test.ts: new `describe('dispose', ...)`
  block with 5 scenarios — empty all read methods, idempotency, pre-
  finalize behavior, post-finalize behavior, and
  `estimateMemoryBytes() === 0` guard.

Verification
- `tsc --noEmit` clean
- 3109 unit tests pass (+6 net: +3 narrowing tests — 2 new + 1
  rewritten — plus +5 dispose scenarios − 1 pre-existing test
  replaced = net +6)
- 1766 resolver integration tests pass
- Zero regressions

Plan: docs/plans/2026-04-09-005-fix-sm14-sequential-path-memory-regression-plan.md
Codex review: branch diff against main, verdict needs-attention
Previous commit: 803631fe (worker-path narrowing)
2026-04-09 18:39:37 +01:00
Gergo Magyar 803631fef7 fix(SM-14): address PR #743 BindingAccumulator review findings
Addresses the 5 findings from PR #743 review comment 4211636245.

Critical (R1) — Strip function-scope bindings from worker IPC
- parse-worker.ts previously serialized typeEnv.allScopes() over the
  worker IPC boundary on every batch, pushing ~4.9 MB of function-scope
  bindings (e.g. `handleRequest@15 → db: Database`) into the accumulator
  with zero downstream consumers. The only reader is the ExportedTypeMap
  enrichment loop in pipeline.ts, which calls fileScopeEntries() —
  the `scope = ''` subset only.
- Narrowed parse-worker.ts to use typeEnv.fileScope() and emit
  [varName, typeName] pairs. FileAllScopeBindings.bindings type narrowed
  from [string, string, string][] to [string, string][].
- pipeline.ts adapter updated to unpack the new two-element tuples and
  construct BindingEntry with scope: '' hardcoded.
- Sequential path (call-processor.ts → typeEnv.flush()) is UNCHANGED
  and still writes all scopes — preserves a working sample of higher-
  quality bindings for Phase 9 prototyping without IPC cost.
- Phase 9 reversion path documented inline on both FileAllScopeBindings
  and the pipeline adapter: change fileScope() → allScopes(), widen the
  tuple back to 3 elements, done. Field name `allScopeBindings` kept to
  keep that revert mechanically trivial.

Medium #1 (R2) — Worker-vs-sequential quality asymmetry
- BindingAccumulator class JSDoc now documents that entries are NOT
  homogeneous in resolution quality: sequential path has SymbolTable
  + importedBindings access (Tier 2 cross-file propagation); worker
  path has Tier 0 + local Tier 1 only. Phase 9 consumers that trust
  every entry equally will silently produce worse results for large
  repos (worker path dominant) than for small ones.

Medium #2 (R3) — Integration test for ExportedTypeMap enrichment
- Added a 4-scenario test suite mocking KnowledgeGraph nodes and
  running the exact enrichment loop from pipeline.ts:1082-1110 inline:
    (a) Exported Function node → enriched
    (b) Non-exported Variable → filtered out (isExported gate)
    (c) Exported Const node → enriched
    (d) No matching graph node → silently skipped via continue
- Locks in the `{Label}:{filePath}:{name}` node-ID format contract.
  If the ID format drifts for any language, this test fires.

Low #1 (R4) — Storage split for O(n_file_scope) reads
- BindingAccumulator now stores two parallel maps:
    _allByFile:       Map<string, BindingEntry[]>     — full entry list
    _fileScopeByFile: Map<string, [string, string][]> — scope='' fast path
- appendFile iterates input once, populates both maps synchronously.
  fileScopeEntries becomes O(1) map lookup + O(n_file_scope) return —
  no longer walks function-scope entries to filter.
- 4 new tests: mixed scopes correctness, only-function-scope file still
  visible via files()/fileCount, multiple appends accumulate
  consistently, 1001-entry performance guard.

Low #2 (R5) — Duplicate iteration logic
- Resolved as a side effect of R1: after the worker narrowing, the
  parse-worker loop iterates `fileScope()` (flat map) and
  typeEnv.flush() iterates `allScopes()` (nested map). Different data
  shapes — no common helper to extract.

Swift CI gap (R7)
- Acknowledged as out of scope. Not SM-14 specific — 95 Swift tests
  skipped across the broader test file.

Verification
- `tsc --noEmit` clean
- 3103 unit tests pass (+10 new scenarios in binding-accumulator.test.ts)
- 1766 resolver integration tests pass
- Zero regressions

Plan: docs/plans/2026-04-09-004-fix-sm14-binding-accumulator-review-findings-plan.md
Review: https://github.com/abhigyanpatwari/GitNexus/pull/743#issuecomment-4211636245
2026-04-09 18:05:20 +01:00
Gergo Magyar 47edb754d1 Merge remote-tracking branch 'origin/main' into sm14-binding-accumulator 2026-04-09 17:42:15 +01:00
abhigyanpatwariandClaude Opus 4.6 89feea744d refactor(sm-14): address PR review feedback
- Remove redundant typeEnvBindings worker payload — allScopeBindings is a strict
  superset (file-scope entries with scope=''). Removes duplicate IPC data and
  the dead fallback branch in pipeline.ts.
- Add single-use guard to TypeEnvironment.flush() — throws on second call to
  prevent silent duplication. Update JSDoc to clarify "copy" semantics
  (env is not actually drained — TypeEnv is per-file and discarded immediately).
- Remove redundant inner guard in parse-worker.ts allScopeBindings serialization.
- Rename allAllScopeBindings -> allScopeBindingsByFile for clarity.
- Document estimateMemoryBytes pessimistic ASCII assumption (V8 uses Latin-1
  for all-ASCII strings, so actual heap cost is ~half).
- Add test for single-use flush() guard.

Addresses PR #743 review feedback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 03:20:18 +05:30
abhigyanpatwariandClaude Opus 4.6 ec85c1e8bc style: fix prettier formatting
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 02:48:54 +05:30
abhigyanpatwariandClaude Opus 4.6 0f83912636 test(sm-14): add pipeline integration simulation for BindingAccumulator
Simulates the worker deserialization -> accumulator -> fileScopeEntries flow
to verify end-to-end correctness.

Part of #679.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 02:31:23 +05:30
abhigyanpatwariandClaude Opus 4.6 750f4336f6 feat(ingestion): wire flush() into sequential processCalls path
Pass bindingAccumulator to processCalls on the sequential code path so
TypeEnv scopes are flushed into the accumulator for Phase 9+ cross-file
type propagation, matching the worker path wired in Task 4.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 02:31:23 +05:30
abhigyanpatwariandClaude Opus 4.6 a7fdb36113 refactor(pipeline): wire BindingAccumulator into chunked parse pipeline
Replace ad-hoc workerTypeEnvBindings array with BindingAccumulator class.
Add allScopeBindings to WorkerExtractedData for multi-scope binding
collection with backward-compat fallback to old typeEnvBindings format.
Return BindingAccumulator from runChunkedParseAndResolve and finalize
before Phase 14 cross-file binding propagation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 02:30:14 +05:30
abhigyanpatwariandClaude Sonnet 4.6 2f72fcc8d1 feat(parse-worker): extend worker serialization to include all scopes
Add FileAllScopeBindings interface and allScopeBindings field to
ParseWorkerResult, serializing all TypeEnv scopes (including
function-local) for BindingAccumulator cross-file type propagation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 02:30:14 +05:30
abhigyanpatwariandClaude Sonnet 4.6 56e2e7f520 feat(type-env): add flush() method to TypeEnvironment
Adds flush(filePath, accumulator) to the TypeEnvironment interface and
buildTypeEnv return object, draining all scoped bindings into a
BindingAccumulator. Adds 4 unit tests covering file-scope, function-scope,
empty env, and multi-file accumulation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 02:30:13 +05:30
abhigyanpatwariandClaude Opus 4.6 3111f4dcd3 feat(sm-14): add BindingAccumulator class with unit tests
Read-append-only accumulator that collects (filePath, scope, varName) -> typeName
bindings from TypeEnv outputs across all files. Supports finalization, file-scope
filtering, iteration, and memory estimation.

Part of #679.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 02:30:13 +05:30
8 changed files with 1492 additions and 61 deletions
@@ -0,0 +1,368 @@
/**
* BindingAccumulator — read-append-only accumulator that collects TypeEnv
* bindings across files in the GitNexus analyzer pipeline.
*
* **Current behavior (both execution paths):** The accumulator carries only
* file-scope (`scope = ''`) entries. Function-scope bindings are stripped
* at both write sites:
*
* - **Worker path**: `parse-worker.ts` serializes only
* `typeEnv.fileScope()` entries across the IPC boundary.
* - **Sequential path**: `type-env.ts::flush()` iterates only the FILE_SCOPE
* entry of the env map and writes `BindingEntry` records with
* `scope: ''` hardcoded.
*
* The narrowing exists because function-scope bindings have zero downstream
* consumers today and were previously costing ~4.9 MB of heap + IPC on
* every pipeline run. See `type-env.ts::flush()` and the `FileScopeBindings`
* JSDoc in `parse-worker.ts` for the paired Phase 9 reversion checklist.
*
* **Historical quality asymmetry (Phase 9 consideration):** Even though
* both paths now carry only file-scope data, the two paths were built
* under different resolution capabilities, and a future Phase 9 reverter
* that widens them back to all scopes will inherit that asymmetry:
*
* - **Sequential path** had (and would regain) access to the full
* `SymbolTable` and `importedBindings`, so its bindings benefit from
* Tier 2 cross-file propagation.
* - **Worker path** runs without `SymbolTable` / `importedBindings` and
* can only produce Tier 0 (annotation-declared) and local Tier 1
* (same-file constructor inference) bindings.
*
* Phase 9 consumers that trust every entry equally will silently produce
* worse results for large repos (worker-dominant) than small ones
* (sequential-dominant). If Phase 9 needs homogeneous quality, either
* (a) tag entries with their tier at insert time so consumers can filter,
* or (b) post-process worker-path entries through a follow-up resolution
* pass after the main-thread `SymbolTable` is complete.
*
* **Lifecycle contract**: `append → finalize → consume → dispose`. See
* `finalize()` and `dispose()` for the state machine. Disposal is
* orthogonal to finalization: either order is legal.
*/
export interface BindingEntry {
readonly scope: string; // '' for file-level, 'funcName@startIndex' for function-local
readonly varName: string;
readonly typeName: string;
}
/**
* Minimal graph-node shape required by `enrichExportedTypeMap()`. Intentionally
* narrower than the full `GraphNode` type in `graph/types.ts` so tests can
* construct a minimal mock without depending on the full graph module, and
* so the enrichment logic is a pure function over this contract.
*
* Matches the shape of the real `KnowledgeGraph` node's `properties.isExported`
* access path — tests that use a different shape silently pass while
* production fails.
*/
export interface EnrichmentGraphNode {
readonly id: string;
readonly properties?: { readonly isExported?: boolean } | undefined;
}
/**
* Minimal graph lookup interface used by `enrichExportedTypeMap()`.
* Consumes only the method the enrichment loop actually calls.
*/
export interface EnrichmentGraphLookup {
getNode(id: string): EnrichmentGraphNode | undefined;
}
/**
* Merge file-scope bindings from a (finalized) `BindingAccumulator` into an
* `exportedTypeMap` for symbols whose graph nodes are marked as exported.
*
* This is the single source of truth for the worker-path ExportedTypeMap
* enrichment loop. Previously the logic lived inline in `pipeline.ts` and
* the test suite reimplemented it as a `runEnrichmentLoop` helper — a
* drift-prone pattern that meant tests could pass while production regressed.
* Extracting it here makes the production code call the same function the
* tests call.
*
* **Node ID candidate order**: `Function:{filePath}:{name}` →
* `Variable:{filePath}:{name}` → `Const:{filePath}:{name}`. First match wins.
*
* **Tier 0 priority**: if `exportedTypeMap` already has an entry for a
* `(filePath, name)` pair, the accumulator entry does NOT overwrite it —
* the SymbolTable tier-0 pass is authoritative. Without this guard, a
* worker-path binding could clobber a higher-quality type from SymbolTable.
*
* **Finalize precondition**: the accumulator should be finalized before
* calling this function. The lifecycle contract is
* `append → finalize → enrich → dispose`. Finalization is not asserted
* here (the test suite and pipeline both honor it separately), but any
* append happening concurrently with this enrichment would be a lifecycle
* bug at the caller level.
*
* @returns The number of new entries written into `exportedTypeMap`
* (0 on empty accumulator or when every candidate was filtered
* out by the export check or the Tier 0 guard).
*/
export function enrichExportedTypeMap(
bindingAccumulator: BindingAccumulator,
graph: EnrichmentGraphLookup,
exportedTypeMap: Map<string, Map<string, string>>,
): number {
if (bindingAccumulator.fileCount === 0) return 0;
let enriched = 0;
for (const filePath of bindingAccumulator.files()) {
for (const [name, type] of bindingAccumulator.fileScopeEntries(filePath)) {
// Three-candidate-ID lookup mirrors the sequential-path export check
// in `collectExportedBindings()` (call-processor.ts).
const functionNodeId = `Function:${filePath}:${name}`;
const variableNodeId = `Variable:${filePath}:${name}`;
const constNodeId = `Const:${filePath}:${name}`;
const node =
graph.getNode(functionNodeId) ??
graph.getNode(variableNodeId) ??
graph.getNode(constNodeId);
if (!node?.properties?.isExported) continue;
let fileExports = exportedTypeMap.get(filePath);
if (!fileExports) {
fileExports = new Map();
exportedTypeMap.set(filePath, fileExports);
}
// Tier 0 priority: SymbolTable-populated entries are authoritative.
if (!fileExports.has(name)) {
fileExports.set(name, type);
enriched++;
}
}
}
return enriched;
}
const ENTRY_OVERHEAD = 64; // bytes per entry (object overhead + property refs)
const MAP_ENTRY_OVERHEAD = 80; // bytes per file entry in the map
export class BindingAccumulator {
// Storage is split into two parallel maps so fileScopeEntries() is
// O(n_file_scope) instead of O(n_total).
// - _allByFile holds every BindingEntry (used by getFile, memory estimate).
// - _fileScopeByFile caches the flat [varName, typeName] view of the
// `scope === ''` subset, populated at insert time so reads are O(1) map
// lookup + O(n_file_scope) array return. Both maps carry the same key
// set modulo the `scope === ''` precondition: _allByFile has a key as
// soon as any entry is appended; _fileScopeByFile only has a key once a
// file-scope entry arrives. Code that iterates via files() uses
// _allByFile so files with only function-scope entries remain visible.
private readonly _allByFile = new Map<string, BindingEntry[]>();
private readonly _fileScopeByFile = new Map<string, [string, string][]>();
private _totalBindings = 0;
private _finalized = false;
private _disposed = false;
/**
* Append bindings for a file. Safe to call multiple times for the same file.
* Throws if the accumulator has been finalized. Skips if entries is empty.
*
* The `entries` parameter is `readonly` — this method never mutates the
* caller's array. Internally, the first `appendFile` call per filePath
* makes a defensive copy (`slice()`), and subsequent calls push into the
* accumulator's own storage.
*/
appendFile(filePath: string, entries: readonly BindingEntry[]): void {
if (this._finalized) {
throw new Error(
'[BindingAccumulator] appendFile after finalize — no further appends allowed',
);
}
if (entries.length === 0) {
return;
}
// Contract consistency: if this accumulator was previously disposed
// without being finalized, `dispose()` is documented to leave it
// "behaving like a fresh one" for subsequent appends. Clear the
// `_disposed` flag here so the `disposed` getter tracks the actual
// live state, not a stale signal from the prior lifecycle cycle.
if (this._disposed) {
this._disposed = false;
}
// Note on the file-scope-only invariant:
// The accumulator does NOT reject function-scope entries at this
// boundary. The narrowing contract is enforced by the two production
// write sites — `parse-worker.ts` (which uses `typeEnv.fileScope()`
// and hardcodes `scope: ''` in the pipeline adapter) and
// `type-env.ts::flush()` (which iterates only `env.get(FILE_SCOPE)`).
// The class JSDoc documents the invariant and the Phase 9 reversion
// path. Making `appendFile` runtime-reject non-file-scope entries
// would break the accumulator's own storage-split tests which
// legitimately exercise mixed-scope entries. If a future write path
// violates the invariant, tests should fail via missing exports in
// the enrichment loop, not via an assertion here.
// All-scope store.
const existingAll = this._allByFile.get(filePath);
if (existingAll !== undefined) {
for (const e of entries) {
existingAll.push(e);
}
} else {
this._allByFile.set(filePath, entries.slice());
}
// File-scope fast-path store. Populated lazily on first file-scope entry.
let existingFileScope = this._fileScopeByFile.get(filePath);
for (const e of entries) {
if (e.scope === '') {
if (existingFileScope === undefined) {
existingFileScope = [];
this._fileScopeByFile.set(filePath, existingFileScope);
}
existingFileScope.push([e.varName, e.typeName]);
}
}
this._totalBindings += entries.length;
}
/** Lock the accumulator — no further appends. Idempotent. */
finalize(): void {
// Dev-mode invariant: verify the parallel storage split is consistent.
// `_fileScopeByFile` must be a proper projection of `_allByFile`
// where the outer key is a subset and the inner entries are exactly
// the `scope === ''` subset of `_allByFile[key]`. A drift would
// indicate a bug in `appendFile()` where one map was updated but
// not the other.
if (process.env.NODE_ENV !== 'production' && !this._finalized) {
for (const [filePath, fileScopeTuples] of this._fileScopeByFile) {
const allEntries = this._allByFile.get(filePath);
if (allEntries === undefined) {
throw new Error(
`[BindingAccumulator] storage split drift: file ${filePath} has file-scope entries ` +
`but no _allByFile entry`,
);
}
const projectedCount = allEntries.filter((e) => e.scope === '').length;
if (projectedCount !== fileScopeTuples.length) {
throw new Error(
`[BindingAccumulator] storage split drift: file ${filePath} has ` +
`${fileScopeTuples.length} file-scope tuples but ${projectedCount} file-scope ` +
`entries in _allByFile`,
);
}
}
}
this._finalized = true;
}
/**
* Release the accumulator's heap footprint. Clears both internal storage
* maps and resets `_totalBindings` to zero. Idempotent and orthogonal to
* `finalize()` — calling `dispose()` does not change the finalized state.
*
* Post-dispose contract: all read methods return empty/undefined state
* matching a never-appended-to accumulator. Specifically:
* - `fileCount === 0`
* - `totalBindings === 0`
* - `files()` yields an empty iterator
* - `getFile(x)` returns `undefined` for all `x`
* - `fileScopeEntries(x)` returns `[]` for all `x`
* - `estimateMemoryBytes()` returns `0`
*
* If `dispose()` is called **before** `finalize()`, subsequent `appendFile`
* calls succeed — the accumulator behaves like a fresh one. If called
* **after** `finalize()`, subsequent `appendFile` calls throw the existing
* "finalized" error.
*
* Lifecycle note: the pipeline disposes the accumulator after the
* ExportedTypeMap enrichment loop consumes its file-scope entries, so
* the heap is released before Phase 14 (`runCrossFileBindingPropagation`)
* and `runGraphAnalysisPhases` begin their long-running work. When Phase 9
* wires a consumer into that stage, the dispose call should move later in
* the pipeline or be removed entirely.
*/
dispose(): void {
this._allByFile.clear();
this._fileScopeByFile.clear();
this._totalBindings = 0;
this._disposed = true;
}
/** Get all bindings for a file, or undefined if the file is unknown. */
getFile(filePath: string): readonly BindingEntry[] | undefined {
return this._allByFile.get(filePath);
}
/**
* Get only scope='' (file-level) entries as [varName, typeName] tuples.
* Backward-compatible with the old workerTypeEnvBindings pattern.
* Returns an empty array for an unknown file.
*
* O(1) map lookup + O(n_file_scope) defensive-copy construction — does
* NOT walk function-scope entries. See the `_fileScopeByFile` field
* comment for the storage split rationale.
*
* The return value is a shallow copy; mutating it does not affect
* subsequent reads or internal state. This encapsulation guard prevents
* a Phase 9 consumer from accidentally corrupting the accumulator via
* `acc.fileScopeEntries(p).push(...)` or similar.
*/
fileScopeEntries(filePath: string): readonly (readonly [string, string])[] {
const cached = this._fileScopeByFile.get(filePath);
return cached ? cached.slice() : [];
}
/** Iterate over all file paths in insertion order. */
files(): IterableIterator<string> {
return this._allByFile.keys();
}
/** Number of distinct files with at least one binding. */
get fileCount(): number {
return this._allByFile.size;
}
/** Total number of binding entries across all files. */
get totalBindings(): number {
return this._totalBindings;
}
/** Whether the accumulator has been finalized. */
get finalized(): boolean {
return this._finalized;
}
/**
* Whether the accumulator has been disposed. Exposed for symmetry with
* `finalized` so debug tooling and future Phase 9 consumers can detect a
* disposed accumulator without inspecting empty state heuristically.
*
* Disposal and finalization are orthogonal: a disposed accumulator may or
* may not be finalized, and vice versa. See `dispose()` for the full
* lifecycle contract.
*/
get disposed(): boolean {
return this._disposed;
}
/**
* Rough memory estimate in bytes (intentionally pessimistic).
* Formula: sum of (ENTRY_OVERHEAD + char bytes of scope+varName+typeName) per entry
* + MAP_ENTRY_OVERHEAD + char bytes of filePath per file.
*
* Note: V8 stores all-ASCII strings as Latin-1 (1 byte/char) and only upgrades
* to UCS-2 (2 bytes/char) for non-Latin-1 code points. Source paths and type names
* are typically all-ASCII, so actual heap cost is roughly half what this returns.
* The pessimistic factor is intentional — better to over-budget than under-budget.
*
* **⚠ Cost profile**: O(totalBindings) — iterates every entry in
* `_allByFile` and reads three string `.length` properties per entry.
* At a typical repo scale (10k files × ~20 file-scope bindings) this is
* ~200k property reads per call. Call at most once per pipeline run,
* NOT per file, per chunk, or per progress tick. The current single
* call site is the dev-mode telemetry log at the pipeline finalize
* seam. Adding a per-file-progress caller would silently make it
* quadratic in repo size.
*/
estimateMemoryBytes(): number {
let total = 0;
for (const [filePath, entries] of this._allByFile) {
total += MAP_ENTRY_OVERHEAD + filePath.length * 2;
for (const e of entries) {
total += ENTRY_OVERHEAD + (e.scope.length + e.varName.length + e.typeName.length) * 2;
}
}
return total;
}
}
+19 -1
View File
@@ -34,6 +34,7 @@ import { buildTypeEnv, isSubclassOf } from './type-env.js';
import type { ConstructorBinding, TypeEnvironment } from './type-env.js';
import type { HeritageMap } from './heritage-map.js';
import { c3Linearize } from './mro-processor.js';
import type { BindingAccumulator } from './binding-accumulator.js';
import { getTreeSitterBufferSize } from './constants.js';
import type {
ExtractedCall,
@@ -155,7 +156,15 @@ export function buildImportedRawReturnTypes(
}
/** Collect resolved type bindings for exported file-scope symbols.
* Uses graph node isExported flag — does NOT require isExported on SymbolDefinition. */
* Uses graph node isExported flag — does NOT require isExported on SymbolDefinition.
*
* **Counterpart**: the worker path populates `exportedTypeMap` via the
* accumulator enrichment loop in `pipeline.ts` (search for "Worker path
* quality enrichment"). Both sites populate the same map with subtly
* different export-check semantics — this site uses SymbolTable +
* graph lookup, the worker loop uses three-candidate-ID graph lookup.
* They must stay in sync until Phase 9 unifies them. If you edit one,
* check the other. */
function collectExportedBindings(
typeEnv: { fileScope(): ReadonlyMap<string, string> },
filePath: string,
@@ -612,6 +621,7 @@ export const processCalls = async (
/** Phase 14 E3: cross-file RAW return types for for-loop element extraction. Keyed by filePath → Map<calleeName, rawReturnType>. */
importedRawReturnTypesMap?: ReadonlyMap<string, ReadonlyMap<string, string>>,
heritageMap?: HeritageMap,
bindingAccumulator?: BindingAccumulator,
): Promise<ExtractedHeritage[]> => {
const parser = await loadParser();
const collectedHeritage: ExtractedHeritage[] = [];
@@ -733,6 +743,14 @@ export const processCalls = async (
const fileExports = collectExportedBindings(typeEnv, file.path, ctx.symbols, graph);
if (fileExports) exportedTypeMap.set(file.path, fileExports);
}
// Flush file-scope bindings into the accumulator. `flush()` is narrowed
// to iterate only FILE_SCOPE entries (type-env.ts) — function-scope
// bindings are dropped at the flush boundary until a Phase 9 consumer
// lands. See type-env.ts::flush() JSDoc for the dual-site reversion
// checklist (this sequential path + the worker path in parse-worker.ts).
if (bindingAccumulator) {
typeEnv.flush(file.path, bindingAccumulator);
}
const callRouter = provider.callRouter;
const verifiedReceivers =
@@ -42,7 +42,7 @@ import type {
ExtractedDecoratorRoute,
ExtractedToolDef,
FileConstructorBindings,
FileTypeEnvBindings,
FileScopeBindings,
ExtractedORMQuery,
} from './workers/parse-worker.js';
import { getTreeSitterBufferSize, TREE_SITTER_MAX_BUFFER } from './constants.js';
@@ -60,7 +60,7 @@ export interface WorkerExtractedData {
toolDefs: ExtractedToolDef[];
ormQueries: ExtractedORMQuery[];
constructorBindings: FileConstructorBindings[];
typeEnvBindings: FileTypeEnvBindings[];
fileScopeBindings: FileScopeBindings[];
}
// ============================================================================
@@ -94,7 +94,7 @@ const processParsingWithWorkers = async (
toolDefs: [],
ormQueries: [],
constructorBindings: [],
typeEnvBindings: [],
fileScopeBindings: [],
};
const total = files.length;
@@ -118,7 +118,7 @@ const processParsingWithWorkers = async (
const allToolDefs: ExtractedToolDef[] = [];
const allORMQueries: ExtractedORMQuery[] = [];
const allConstructorBindings: FileConstructorBindings[] = [];
const allTypeEnvBindings: FileTypeEnvBindings[] = [];
const fileScopeBindingsByFile: FileScopeBindings[] = [];
for (const result of chunkResults) {
for (const node of result.nodes) {
graph.addNode({
@@ -154,7 +154,8 @@ const processParsingWithWorkers = async (
for (const _item of result.toolDefs) allToolDefs.push(_item);
if (result.ormQueries) for (const _item of result.ormQueries) allORMQueries.push(_item);
for (const _item of result.constructorBindings) allConstructorBindings.push(_item);
for (const _item of result.typeEnvBindings) allTypeEnvBindings.push(_item);
if (result.fileScopeBindings)
for (const _item of result.fileScopeBindings) fileScopeBindingsByFile.push(_item);
}
// Merge and log skipped languages from workers
@@ -184,7 +185,7 @@ const processParsingWithWorkers = async (
toolDefs: allToolDefs,
ormQueries: allORMQueries,
constructorBindings: allConstructorBindings,
typeEnvBindings: allTypeEnvBindings,
fileScopeBindings: fileScopeBindingsByFile,
};
};
@@ -354,7 +355,14 @@ const processParsingSequential = async (
continue;
}
// Build per-file type environment for FieldExtractor context (lightweight — skipped if no fieldExtractor)
// Build per-file type environment for FieldExtractor context (lightweight — skipped if no fieldExtractor).
//
// Note: this TypeEnv is intentionally NOT flushed into the BindingAccumulator.
// The accumulator feed happens later in `call-processor.ts` via its own
// `typeEnv.flush(accumulator)` call. Flushing here would double-count
// file-scope bindings and break the single-use invariant of `flush()`.
// See the BindingAccumulator class JSDoc for the full accumulator
// lifecycle and flush-site ownership rules.
const typeEnv = provider.fieldExtractor
? buildTypeEnv(tree, language, {
enclosingFunctionFinder: provider.enclosingFunctionFinder,
+118 -36
View File
@@ -1,4 +1,9 @@
import { createKnowledgeGraph } from '../graph/graph.js';
import {
BindingAccumulator,
enrichExportedTypeMap,
type BindingEntry,
} from './binding-accumulator.js';
import { processStructure } from './structure-processor.js';
import { processMarkdown } from './markdown-processor.js';
import { processCobol, isCobolFile, isJclFile } from './cobol-processor.js';
@@ -650,6 +655,7 @@ async function runChunkedParseAndResolve(
allDecoratorRoutes: ExtractedDecoratorRoute[];
allToolDefs: ExtractedToolDef[];
allORMQueries: ExtractedORMQuery[];
bindingAccumulator: BindingAccumulator;
}> {
const symbolTable = ctx.symbols;
@@ -782,7 +788,7 @@ async function runChunkedParseAndResolve(
// Phase 14: Collect exported type bindings for cross-file propagation
const exportedTypeMap: ExportedTypeMap = new Map();
// Accumulate file-scope TypeEnv bindings from workers (closes worker/sequential quality gap)
const workerTypeEnvBindings: { filePath: string; bindings: [string, string][] }[] = [];
const bindingAccumulator = new BindingAccumulator();
// Accumulate fetch() calls from workers for Next.js route matching
const allFetchCalls: ExtractedFetchCall[] = [];
// Accumulate framework-extracted routes (Laravel, etc.) for Route node creation
@@ -916,9 +922,31 @@ async function runChunkedParseAndResolve(
});
}),
]);
// Collect TypeEnv file-scope bindings for exported type enrichment
if (chunkWorkerData.typeEnvBindings?.length) {
for (const _item of chunkWorkerData.typeEnvBindings) workerTypeEnvBindings.push(_item);
// Collect file-scope bindings into BindingAccumulator. The worker
// IPC payload carries only file-scope entries (`scope = ''`
// hardcoded here). See the FileScopeBindings JSDoc in
// parse-worker.ts for the rationale and Phase 9 reversion path.
//
// Defensive validation at the IPC boundary: silently skip entries
// with non-string varName/typeName. If a future worker regression
// (or a Phase 9 reversion mistake that emits 3-tuples into the
// 2-tuple consumer) produces malformed data, logging is better
// than silently writing `undefined` into the enrichment map.
if (chunkWorkerData.fileScopeBindings?.length) {
for (const { filePath, bindings } of chunkWorkerData.fileScopeBindings) {
if (typeof filePath !== 'string' || filePath.length === 0) continue;
if (!Array.isArray(bindings)) continue;
const entries: BindingEntry[] = [];
for (const tuple of bindings) {
if (!Array.isArray(tuple) || tuple.length !== 2) continue;
const [varName, typeName] = tuple;
if (typeof varName !== 'string' || typeof typeName !== 'string') continue;
entries.push({ scope: '', varName, typeName });
}
if (entries.length > 0) {
bindingAccumulator.appendFile(filePath, entries);
}
}
}
// Collect fetch() calls for Next.js route matching
if (chunkWorkerData.fetchCalls?.length) {
@@ -1034,6 +1062,7 @@ async function runChunkedParseAndResolve(
undefined,
undefined,
sequentialHeritageMap,
bindingAccumulator,
);
await processHeritage(graph, chunkFiles, astCache, ctx);
if (rubyHeritage.length > 0) {
@@ -1068,39 +1097,31 @@ async function runChunkedParseAndResolve(
);
}
// ── Worker path quality enrichment: merge TypeEnv file-scope bindings into ExportedTypeMap ──
// Workers return file-scope bindings from their TypeEnv fixpoint (includes inferred types
// like `const config = getConfig()` → Config). Filter by graph isExported to match
// the sequential path's collectExportedBindings behavior.
if (workerTypeEnvBindings.length > 0) {
let enriched = 0;
for (const { filePath, bindings } of workerTypeEnvBindings) {
for (const [name, type] of bindings) {
// Verify the symbol is exported via graph node
const nodeId = `Function:${filePath}:${name}`;
const varNodeId = `Variable:${filePath}:${name}`;
const constNodeId = `Const:${filePath}:${name}`;
const node =
graph.getNode(nodeId) ?? graph.getNode(varNodeId) ?? graph.getNode(constNodeId);
if (!node?.properties?.isExported) continue;
// ── Finalize the accumulator before the read phase begins. All worker-path
// appends (line ~934) and sequential-path flushes (via `processCalls` →
// `typeEnv.flush()` earlier in this function) have completed by here,
// so the finalize-write-lock is correct at this seam. Making the
// lifecycle contract explicit — `append → finalize → consume → dispose`.
// Previously `finalize()` was called much later in `runPipelineFromRepo`
// after the enrichment loop had already read the mutable accumulator.
bindingAccumulator.finalize();
let fileExports = exportedTypeMap.get(filePath);
if (!fileExports) {
fileExports = new Map();
exportedTypeMap.set(filePath, fileExports);
}
// Don't overwrite existing entries (Tier 0 from SymbolTable is authoritative)
if (!fileExports.has(name)) {
fileExports.set(name, type);
enriched++;
}
}
}
if (isDev && enriched > 0) {
console.log(
`🔗 Worker TypeEnv enrichment: ${enriched} fixpoint-inferred exports added to ExportedTypeMap`,
);
}
// ── Worker path quality enrichment: merge file-scope bindings into ExportedTypeMap ──
// Counterpart to `collectExportedBindings()` in call-processor.ts which
// handles the sequential path (main thread, full SymbolTable access).
// This call handles the worker path via the accumulator. Both sites
// populate the same `exportedTypeMap` with subtly different export-check
// semantics — sequential uses SymbolTable + graph lookup, `enrichExportedTypeMap`
// uses a three-candidate-ID graph lookup. They must stay in sync until
// Phase 9 unifies them. If you edit one, check the other.
//
// The enrichment loop itself lives in `binding-accumulator.ts` so tests
// can exercise the real production code instead of reimplementing it.
const enriched = enrichExportedTypeMap(bindingAccumulator, graph, exportedTypeMap);
if (isDev && enriched > 0) {
console.log(
`🔗 Worker TypeEnv enrichment: ${enriched} fixpoint-inferred exports added to ExportedTypeMap`,
);
}
// ── Final synthesis pass for whole-module-import languages ──
@@ -1128,6 +1149,7 @@ async function runChunkedParseAndResolve(
allDecoratorRoutes,
allToolDefs,
allORMQueries,
bindingAccumulator,
};
}
@@ -1359,6 +1381,15 @@ export const runPipelineFromRepo = async (
const ctx = createResolutionContext();
const pipelineStart = Date.now();
// Hoisted reference for error-path cleanup. The accumulator is normally
// disposed at the happy-path seam after the dev telemetry log, but if any
// step between the runChunkedParseAndResolve return and that seam throws
// (ORM processing, tool node creation, Phase 14, graph analysis), the
// catch handler disposes it here so the heap footprint does not leak
// through the rethrow. See binding-accumulator.ts dispose() JSDoc for the
// lifecycle contract.
let bindingAccumulatorForCleanup: BindingAccumulator | undefined;
try {
// Phase 1+2: Scan paths, build structure, process markdown
const { scannedFiles, allPaths, totalFiles } = await runScanAndStructure(
@@ -1375,6 +1406,7 @@ export const runPipelineFromRepo = async (
allDecoratorRoutes,
allToolDefs,
allORMQueries,
bindingAccumulator,
} = await runChunkedParseAndResolve(
graph,
ctx,
@@ -1386,6 +1418,11 @@ export const runPipelineFromRepo = async (
onProgress,
options,
);
// Track the accumulator for error-path cleanup — the happy-path dispose
// is still at the post-telemetry seam below, this reference is only
// consulted by the catch handler if any step between here and there
// throws.
bindingAccumulatorForCleanup = bindingAccumulator;
// ── Phase 3.5: Route Registry (Next.js + PHP + Laravel + decorators) ──
type RouteEntry = { filePath: string; source: string };
@@ -1697,6 +1734,45 @@ export const runPipelineFromRepo = async (
processORMQueries(graph, allORMQueries, isDev);
}
// `bindingAccumulator.finalize()` was moved inside `runChunkedParseAndResolve`
// to immediately precede the enrichment loop — see the comment there for
// the ordering rationale. By the time execution
// reaches this point, the accumulator has already been finalized, consumed
// by the enrichment loop, and is ready for dispose() below after the dev
// telemetry log captures peak state.
if (isDev) {
if (bindingAccumulator.totalBindings > 0) {
const memKB = Math.round(bindingAccumulator.estimateMemoryBytes() / 1024);
console.log(
`📦 BindingAccumulator: ${bindingAccumulator.totalBindings} bindings across ${bindingAccumulator.fileCount} files (~${memKB} KB)`,
);
} else if (totalFiles > 0) {
// Zero-binding signal: if the pipeline parsed files but the
// accumulator is empty, something upstream dropped all bindings.
// Flag it so operators can spot a regression (e.g. a worker path
// that accidentally emits empty fileScopeBindings arrays for every
// file, or a TypeEnv build failure). Dev-mode only.
console.log(
`📦 BindingAccumulator: EMPTY — 0 bindings across 0 files despite ${totalFiles} parsed files. If the codebase has typed bindings, this indicates an upstream regression.`,
);
}
}
// Release the accumulator's heap footprint now. The ExportedTypeMap
// enrichment loop above is the only current consumer, and the dev
// telemetry log just captured peak state. Phase 14 and
// runGraphAnalysisPhases do not read the accumulator today — keeping
// it alive through those long-running phases pins heap for no reason.
// When Phase 9 wires a consumer into runCrossFileBindingPropagation,
// move this dispose() call to after that consumer completes or delete
// it entirely if the consumer takes lifecycle ownership.
bindingAccumulator.dispose();
// Happy-path dispose completed — clear the cleanup ref so the catch
// handler doesn't attempt a second (harmless but noisy) dispose if a
// later phase throws.
bindingAccumulatorForCleanup = undefined;
// ── Phase 14: Cross-file binding propagation (topological level sort) ──
await runCrossFileBindingPropagation(
graph,
@@ -1741,6 +1817,12 @@ export const runPipelineFromRepo = async (
return { graph, repoPath, totalFileCount: totalFiles, communityResult, processResult };
} catch (error) {
// Error-path cleanup: dispose the accumulator if a step after the
// destructure from runChunkedParseAndResolve but before the happy-path
// dispose threw. The reference is cleared on the happy path, so this
// is a no-op when the pipeline completed successfully and then threw
// from an unrelated post-dispose step (e.g., future cleanup code).
bindingAccumulatorForCleanup?.dispose();
ctx.clear();
throw error;
}
+60 -2
View File
@@ -8,6 +8,7 @@ import { CALL_EXPRESSION_TYPES } from './utils/call-analysis.js';
import { SupportedLanguages } from 'gitnexus-shared';
import { TYPED_PARAMETER_TYPES } from './type-extractors/shared.js';
import { getProvider } from './languages/index.js';
import type { BindingAccumulator, BindingEntry } from './binding-accumulator.js';
import type {
ClassNameLookup,
ReturnTypeLookup,
@@ -42,7 +43,21 @@ type TypeEnv = Map<string, Map<string, string>>;
const FILE_SCOPE = '';
/** Shared empty map for files with no file-scope bindings. */
const EMPTY_FILE_SCOPE: ReadonlyMap<string, string> = new Map();
/**
* Create a fresh empty Map for the "no file-scope bindings" fallback.
*
* **Why not a shared sentinel**: we previously used a module-level
* `const EMPTY_FILE_SCOPE = new Map()` typed as `ReadonlyMap` and shared
* across every TypeEnv instance. That was a latent singleton-poisoning
* footgun: any caller that did `(fileScope() as Map).set(...)` — or any
* future refactor that widened the return type — would silently corrupt
* every subsequent "empty" return for the process lifetime. A Proxy
* wrapper was considered but broke Map's internal-slot methods (`.size`,
* iteration protocol). Allocating a fresh empty Map per call is a few
* bytes per file — immediately GC'd, no measurable cost even at 10k files
* — and eliminates the shared-mutation hazard entirely.
*/
const emptyFileScope = (): ReadonlyMap<string, string> => new Map();
/** Fallback for languages where class names aren't in a 'name' field (e.g. Kotlin uses type_identifier). */
const findTypeIdentifierChild = (node: SyntaxNode): SyntaxNode | null => {
@@ -73,6 +88,10 @@ export interface TypeEnvironment {
* Populated when a variable has BOTH a declared base type AND a more specific
* constructor type (e.g., `Animal a = new Dog()` → key maps to 'Dog'). */
readonly constructorTypeMap: ReadonlyMap<string, string>;
/** Copy all scoped bindings into a BindingAccumulator.
* Must be called at most once per TypeEnv instance — throws on second call.
* The source `env` is not cleared (TypeEnv is per-file and discarded immediately after). */
flush(filePath: string, accumulator: BindingAccumulator): void;
}
/**
@@ -822,6 +841,7 @@ export const buildTypeEnv = (
const parentMap = options?.parentMap;
const extractFuncNameHook = options?.extractFunctionName;
const env: TypeEnv = new Map();
let flushed = false;
const patternOverrides: PatternOverrides = new Map();
// Phase P: maps `scope\0varName` → constructor type when a declaration has BOTH
// a base type annotation AND a more specific constructor initializer.
@@ -1242,9 +1262,47 @@ export const buildTypeEnv = (
extractFuncNameHook,
),
constructorBindings: bindings,
fileScope: () => env.get(FILE_SCOPE) ?? EMPTY_FILE_SCOPE,
fileScope: () => env.get(FILE_SCOPE) ?? emptyFileScope(),
allScopes: () => env as ReadonlyMap<string, ReadonlyMap<string, string>>,
constructorTypeMap,
flush(filePath: string, accumulator: BindingAccumulator): void {
if (flushed) {
throw new Error(
`[TypeEnvironment] flush called twice for ${filePath} — flush is single-use`,
);
}
// Narrow flush() to iterate only the FILE_SCOPE entry, mirroring the
// worker-path narrowing in parse-worker.ts (commit 803631fe). Before
// this change, both execution paths had the same asymmetry bug: the
// worker path was fixed but the sequential path (this code) still
// wrote function-scope entries into long-lived accumulator storage
// that no consumer reads until Phase 9 lands.
//
// Phase 9 reversion: when a downstream consumer of function-scope
// bindings exists, restore the nested iteration:
//
// for (const [scope, scopeMap] of env) {
// for (const [varName, typeName] of scopeMap) {
// entries.push({ scope, varName, typeName });
// }
// }
//
// See BindingAccumulator class JSDoc and FileScopeBindings JSDoc in
// parse-worker.ts for the full reversion checklist.
const fileScope = env.get(FILE_SCOPE) ?? emptyFileScope();
const entries: BindingEntry[] = [];
for (const [varName, typeName] of fileScope) {
entries.push({ scope: '', varName, typeName });
}
if (entries.length > 0) {
accumulator.appendFile(filePath, entries);
}
// Mark the env as flushed AFTER the successful append. If appendFile
// throws (e.g., accumulator is already finalized due to a lifecycle
// ordering bug), the caller can catch and retry — the single-use
// guard now tracks "data was written", not "flush was attempted".
flushed = true;
},
};
};
@@ -235,10 +235,33 @@ export interface FileConstructorBindings {
bindings: ConstructorBinding[];
}
/** File-scope type bindings from TypeEnv fixpoint — used for cross-file ExportedTypeMap. */
export interface FileTypeEnvBindings {
/** All-scope type bindings from TypeEnv — includes function-local scopes.
* Used by BindingAccumulator for cross-file type propagation (Phase 9+).
*
* Carries only file-scope entries (`scope = ''`). Serializing function-scope
* bindings over IPC cost ~4.9 MB with zero downstream consumers.
* `parse-worker.ts` now iterates only `typeEnv.fileScope()` and the
* sequential path's `type-env.ts::flush()` is also narrowed to file
* scope — see the `BindingAccumulator` class JSDoc for the unified
* narrowing contract across both execution paths.
*
* **Phase 9 reversion checklist** (when a downstream consumer of
* function-scope bindings lands):
* 1. Change the loop in `runParseJob` below from `typeEnv.fileScope()`
* back to `typeEnv.allScopes()`.
* 2. Emit three-element tuples `[scope, varName, typeName]`.
* 3. Widen the `bindings` field on this interface back to
* `[string, string, string][]`.
* 4. Update the pipeline adapter in `pipeline.ts` to unpack three
* elements and populate `BindingEntry.scope` from the first tuple
* element instead of hardcoding `''`.
* 5. Also revert `type-env.ts::flush()` to iterate `env` instead of
* just `FILE_SCOPE` if the sequential path needs function-scope data too.
* 6. Consider renaming this interface back to `FileAllScopeBindings`
* along with widening. */
export interface FileScopeBindings {
filePath: string;
/** [varName, typeName] pairs from file scope (scope = '') */
/** [varName, typeName] pairs from the file scope only. */
bindings: [string, string][];
}
@@ -256,8 +279,8 @@ export interface ParseWorkerResult {
toolDefs: ExtractedToolDef[];
ormQueries: ExtractedORMQuery[];
constructorBindings: FileConstructorBindings[];
/** File-scope type bindings from TypeEnv fixpoint for exported symbol collection. */
typeEnvBindings: FileTypeEnvBindings[];
/** All-scope type bindings from TypeEnv for BindingAccumulator (includes function-local). */
fileScopeBindings: FileScopeBindings[];
skippedLanguages: Record<string, number>;
fileCount: number;
}
@@ -690,7 +713,7 @@ const processBatch = (
toolDefs: [],
ormQueries: [],
constructorBindings: [],
typeEnvBindings: [],
fileScopeBindings: [],
skippedLanguages: {},
fileCount: 0,
};
@@ -1386,14 +1409,23 @@ const processFileGroup = (
});
}
// Extract file-scope bindings for ExportedTypeMap (closes worker/sequential quality gap).
// Sequential path uses collectExportedBindings(typeEnv) directly; worker path serializes
// these bindings so the main thread can merge them into ExportedTypeMap.
// Serialize file-scope bindings for BindingAccumulator. These feed the
// ExportedTypeMap enrichment loop in pipeline.ts — the only current
// consumer of worker-path binding data.
//
// Historical note: we previously serialized all scopes
// (`typeEnv.allScopes()`), which pushed ~4.9 MB of function-scope
// bindings across the IPC boundary on every worker batch with zero
// downstream readers. Narrowing to `fileScope()` recovers that cost.
// See the `FileScopeBindings` JSDoc above for the Phase 9 reversion
// path when a function-scope consumer lands.
const fileScope = typeEnv.fileScope();
if (fileScope.size > 0) {
const bindings: [string, string][] = [];
for (const [name, type] of fileScope) bindings.push([name, type]);
result.typeEnvBindings.push({ filePath: file.path, bindings });
const scopeBindings: [string, string][] = [];
for (const [varName, typeName] of fileScope) {
scopeBindings.push([varName, typeName]);
}
result.fileScopeBindings.push({ filePath: file.path, bindings: scopeBindings });
}
// Per-file map: decorator end-line → decorator info, for associating with definitions
@@ -2113,7 +2145,7 @@ let accumulated: ParseWorkerResult = {
toolDefs: [],
ormQueries: [],
constructorBindings: [],
typeEnvBindings: [],
fileScopeBindings: [],
skippedLanguages: {},
fileCount: 0,
};
@@ -2133,7 +2165,7 @@ const mergeResult = (target: ParseWorkerResult, src: ParseWorkerResult) => {
target.toolDefs.push(...src.toolDefs);
target.ormQueries.push(...src.ormQueries);
target.constructorBindings.push(...src.constructorBindings);
target.typeEnvBindings.push(...src.typeEnvBindings);
target.fileScopeBindings.push(...src.fileScopeBindings);
for (const [lang, count] of Object.entries(src.skippedLanguages)) {
target.skippedLanguages[lang] = (target.skippedLanguages[lang] || 0) + count;
}
@@ -2184,7 +2216,7 @@ parentPort!.on('message', (msg: WorkerIncomingMessage) => {
toolDefs: [],
ormQueries: [],
constructorBindings: [],
typeEnvBindings: [],
fileScopeBindings: [],
skippedLanguages: {},
fileCount: 0,
};
@@ -0,0 +1,653 @@
import { describe, it, expect } from 'vitest';
import {
BindingAccumulator,
enrichExportedTypeMap,
type BindingEntry,
type EnrichmentGraphLookup,
type EnrichmentGraphNode,
} from '../../src/core/ingestion/binding-accumulator.js';
describe('BindingAccumulator', () => {
describe('append + read', () => {
it('returns entries for a single file', () => {
const acc = new BindingAccumulator();
const entries: BindingEntry[] = [
{ scope: '', varName: 'x', typeName: 'number' },
{ scope: 'foo@10', varName: 'y', typeName: 'string' },
];
acc.appendFile('src/a.ts', entries);
expect(acc.getFile('src/a.ts')).toEqual(entries);
});
it('returns entries for multiple files', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'a', typeName: 'number' }]);
acc.appendFile('src/b.ts', [{ scope: '', varName: 'b', typeName: 'string' }]);
expect(acc.getFile('src/a.ts')).toHaveLength(1);
expect(acc.getFile('src/b.ts')).toHaveLength(1);
expect(acc.fileCount).toBe(2);
});
it('returns undefined for unknown file', () => {
const acc = new BindingAccumulator();
expect(acc.getFile('nonexistent.ts')).toBeUndefined();
});
it('accumulates entries across multiple calls for the same file', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'number' }]);
acc.appendFile('src/a.ts', [{ scope: 'fn@5', varName: 'y', typeName: 'boolean' }]);
const entries = acc.getFile('src/a.ts');
expect(entries).toHaveLength(2);
expect(entries![0].varName).toBe('x');
expect(entries![1].varName).toBe('y');
});
it('skips append when entries is empty', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', []);
expect(acc.getFile('src/a.ts')).toBeUndefined();
expect(acc.fileCount).toBe(0);
});
it('tracks totalBindings correctly', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [
{ scope: '', varName: 'x', typeName: 'number' },
{ scope: '', varName: 'y', typeName: 'string' },
]);
acc.appendFile('src/b.ts', [{ scope: '', varName: 'z', typeName: 'boolean' }]);
expect(acc.totalBindings).toBe(3);
});
});
describe('finalize + immutability', () => {
it('finalize prevents further appends', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'number' }]);
acc.finalize();
expect(() =>
acc.appendFile('src/b.ts', [{ scope: '', varName: 'y', typeName: 'string' }]),
).toThrow(/finalize/);
});
it('finalized getter returns true after finalize', () => {
const acc = new BindingAccumulator();
expect(acc.finalized).toBe(false);
acc.finalize();
expect(acc.finalized).toBe(true);
});
it('getFile works after finalize', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'number' }]);
acc.finalize();
expect(acc.getFile('src/a.ts')).toHaveLength(1);
});
it('finalize is idempotent', () => {
const acc = new BindingAccumulator();
acc.finalize();
expect(() => acc.finalize()).not.toThrow();
});
});
describe('fileScopeEntries', () => {
it('returns only scope="" entries as [varName, typeName] tuples', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [
{ scope: '', varName: 'x', typeName: 'number' },
{ scope: 'foo@10', varName: 'y', typeName: 'string' },
{ scope: '', varName: 'z', typeName: 'boolean' },
]);
const tuples = acc.fileScopeEntries('src/a.ts');
expect(tuples).toEqual([
['x', 'number'],
['z', 'boolean'],
]);
});
it('returns empty array for unknown file', () => {
const acc = new BindingAccumulator();
expect(acc.fileScopeEntries('nonexistent.ts')).toEqual([]);
});
it('returns empty array when file has no file-scope entries', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: 'fn@1', varName: 'x', typeName: 'number' }]);
expect(acc.fileScopeEntries('src/a.ts')).toEqual([]);
});
});
describe('iteration', () => {
it('files() yields all file paths', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'number' }]);
acc.appendFile('src/b.ts', [{ scope: '', varName: 'y', typeName: 'string' }]);
acc.appendFile('src/c.ts', [{ scope: '', varName: 'z', typeName: 'boolean' }]);
const paths = [...acc.files()];
expect(paths.sort()).toEqual(['src/a.ts', 'src/b.ts', 'src/c.ts']);
});
it('files() returns empty iterator when no files added', () => {
const acc = new BindingAccumulator();
expect([...acc.files()]).toEqual([]);
});
});
describe('memory estimate', () => {
it('returns a reasonable estimate for 1000 files x 2 entries', () => {
const acc = new BindingAccumulator();
for (let i = 0; i < 1000; i++) {
acc.appendFile(`src/file${i}.ts`, [
{ scope: '', varName: `var${i}a`, typeName: 'string' },
{ scope: `fn${i}@0`, varName: `var${i}b`, typeName: 'number' },
]);
}
const bytes = acc.estimateMemoryBytes();
// Should be between 50KB and 2MB
expect(bytes).toBeGreaterThan(50 * 1024);
expect(bytes).toBeLessThan(2 * 1024 * 1024);
});
});
describe('pipeline integration (simulated)', () => {
it('deserializes allScopeBindings from worker into accumulator', () => {
const acc = new BindingAccumulator();
// Simulated worker output:
// After narrowing the worker IPC payload to file-scope only, the
// emitted tuple shape is [varName, typeName]. Function-scope entries
// are stripped at the parse-worker boundary; the sequential path's
// flush() still writes all scopes via its own code path.
const workerBindings = [
{
filePath: 'src/service.ts',
bindings: [['config', 'Config'] as [string, string]],
},
{
filePath: 'src/utils.ts',
bindings: [['logger', 'Logger'] as [string, string]],
},
];
// Pipeline deserialization logic (mirrors pipeline.ts adapter):
// two-element tuples → BindingEntry with hard-coded scope: ''.
for (const { filePath, bindings } of workerBindings) {
const entries: BindingEntry[] = bindings.map(([varName, typeName]) => ({
scope: '',
varName,
typeName,
}));
acc.appendFile(filePath, entries);
}
acc.finalize();
expect(acc.fileCount).toBe(2);
expect(acc.totalBindings).toBe(2);
// fileScopeEntries — what the ExportedTypeMap enrichment loop uses.
expect(acc.fileScopeEntries('src/service.ts')).toEqual([['config', 'Config']]);
expect(acc.fileScopeEntries('src/utils.ts')).toEqual([['logger', 'Logger']]);
// Every entry produced by the worker path has scope === '' after the
// IPC narrowing — locks the contract in place.
const serviceEntries = acc.getFile('src/service.ts');
expect(serviceEntries).toHaveLength(1);
expect(serviceEntries![0]).toEqual({
scope: '',
varName: 'config',
typeName: 'Config',
});
});
it('worker IPC payload contains ONLY file-scope entries (narrowing guard)', () => {
// Function-scope bindings were being
// serialized over worker IPC with no consumer, costing ~4.9 MB. The
// worker now uses typeEnv.fileScope() instead of typeEnv.allScopes(),
// so `handleRequest@15 → db: Database` never crosses the IPC boundary.
//
// This test simulates a TypeEnvironment that HAD both file-scope and
// function-scope bindings (as would be produced by a realistic file),
// then asserts the worker IPC payload contains only the file-scope
// ones. If a future change accidentally re-broadens the worker loop
// to `allScopes()`, this assertion fires.
const simulatedFileScope = new Map<string, string>([
['config', 'Config'],
['db', 'Database'],
]);
// Function-scope entries that must NOT appear in the worker payload.
const simulatedFunctionScope = new Map<string, string>([
['localRequest', 'Request'],
['localUser', 'User'],
]);
// Mirror the parse-worker loop (post-narrowing shape):
// const fileScope = typeEnv.fileScope();
// for (const [varName, typeName] of fileScope) {
// scopeBindings.push([varName, typeName]);
// }
const workerPayload: [string, string][] = [];
for (const [varName, typeName] of simulatedFileScope) {
workerPayload.push([varName, typeName]);
}
// Verify: the simulated function-scope variables are never pushed.
const allVarNames = workerPayload.map(([v]) => v);
expect(allVarNames).toEqual(['config', 'db']);
expect(allVarNames).not.toContain('localRequest');
expect(allVarNames).not.toContain('localUser');
// Sanity: simulatedFunctionScope exists so the test is not trivially
// vacuous — it documents what the old allScopes() path would have
// emitted and what the new fileScope() path deliberately excludes.
expect(simulatedFunctionScope.size).toBe(2);
// Round-trip through the accumulator with the pipeline adapter shape.
const acc = new BindingAccumulator();
const entries: BindingEntry[] = workerPayload.map(([varName, typeName]) => ({
scope: '',
varName,
typeName,
}));
acc.appendFile('src/service.ts', entries);
acc.finalize();
const stored = acc.getFile('src/service.ts');
expect(stored).toHaveLength(2);
// All accumulator entries from the worker path have scope === ''.
for (const entry of stored!) {
expect(entry.scope).toBe('');
}
});
});
// -------------------------------------------------------------------------
// fileScopeEntries() must be O(n_file_scope),
// not O(n_total). Storage is split into _allByFile + _fileScopeByFile so
// reads skip function-scope entries entirely.
// -------------------------------------------------------------------------
describe('storage split (fast-path fileScopeEntries)', () => {
it('mixed file-scope and function-scope input: fileScopeEntries ignores function-scope', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [
{ scope: '', varName: 'file1', typeName: 'T1' },
{ scope: 'fn@10', varName: 'local1', typeName: 'L1' },
{ scope: '', varName: 'file2', typeName: 'T2' },
{ scope: 'fn@20', varName: 'local2', typeName: 'L2' },
{ scope: 'fn@30', varName: 'local3', typeName: 'L3' },
]);
// fileScopeEntries returns exactly the two file-scope entries,
// preserving insertion order.
expect(acc.fileScopeEntries('src/a.ts')).toEqual([
['file1', 'T1'],
['file2', 'T2'],
]);
// getFile still returns all 5 entries (mixed scopes preserved).
expect(acc.getFile('src/a.ts')).toHaveLength(5);
});
it('only-function-scope file: fileScopeEntries returns [] but files() still lists it', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/only-fn.ts', [
{ scope: 'fn@5', varName: 'x', typeName: 'X' },
{ scope: 'fn@10', varName: 'y', typeName: 'Y' },
]);
expect(acc.fileScopeEntries('src/only-fn.ts')).toEqual([]);
expect(acc.getFile('src/only-fn.ts')).toHaveLength(2);
expect([...acc.files()]).toContain('src/only-fn.ts');
expect(acc.fileCount).toBe(1);
});
it('multiple appends accumulate in both maps consistently', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [
{ scope: '', varName: 'x', typeName: 'X' },
{ scope: 'fn@1', varName: 'y', typeName: 'Y' },
]);
acc.appendFile('src/a.ts', [
{ scope: '', varName: 'z', typeName: 'Z' },
{ scope: 'fn@2', varName: 'w', typeName: 'W' },
]);
expect(acc.fileScopeEntries('src/a.ts')).toEqual([
['x', 'X'],
['z', 'Z'],
]);
expect(acc.getFile('src/a.ts')).toHaveLength(4);
expect(acc.totalBindings).toBe(4);
});
it('performance guard: fileScopeEntries does not walk function-scope entries', () => {
const acc = new BindingAccumulator();
// 1 file-scope entry + 1000 function-scope entries.
const entries: BindingEntry[] = [{ scope: '', varName: 'shared', typeName: 'Shared' }];
for (let i = 0; i < 1000; i++) {
entries.push({
scope: `fn${i}@${i * 10}`,
varName: `local${i}`,
typeName: 'Local',
});
}
acc.appendFile('src/big.ts', entries);
// fileScopeEntries returns the single file-scope pair without
// iterating the 1000 function-scope entries — this is the O(1) cache
// lookup behavior guaranteed by the storage split.
const result = acc.fileScopeEntries('src/big.ts');
expect(result).toHaveLength(1);
expect(result[0]).toEqual(['shared', 'Shared']);
// Sanity: getFile still sees everything.
expect(acc.getFile('src/big.ts')).toHaveLength(1001);
});
});
// -------------------------------------------------------------------------
// Integration coverage for the sequential
// path → accumulator → ExportedTypeMap enrichment loop at pipeline.ts
// lines 1082-1110. This test mirrors that loop inline with a minimal
// KnowledgeGraph-shaped mock, locking in the node-ID format contract
// (Function:{filePath}:{name}, Variable:..., Const:...). If the ID format
// drifts for any language, this test fires.
// -------------------------------------------------------------------------
describe('ExportedTypeMap enrichment (integration)', () => {
/**
* Minimal graph backing for `enrichExportedTypeMap`. Matches the
* `EnrichmentGraphNode` shape from binding-accumulator.ts — which in
* turn matches the real `GraphNode.properties.isExported` access path
* used by the production `KnowledgeGraph`. Using this shape (rather
* than a flat `isExported` field) means a refactor of the graph's
* `properties` layout will fail this test, not silently pass.
*/
function makeGraphLookup(
nodes: Array<{ id: string; isExported: boolean }>,
): EnrichmentGraphLookup {
const byId = new Map<string, EnrichmentGraphNode>();
for (const n of nodes) {
byId.set(n.id, { id: n.id, properties: { isExported: n.isExported } });
}
return { getNode: (id) => byId.get(id) };
}
it('enriches exportedTypeMap with an exported Function node', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/utils.ts', [
{ scope: '', varName: 'helper', typeName: '(arg: string) => User' },
]);
acc.finalize();
const graph = makeGraphLookup([{ id: 'Function:src/utils.ts:helper', isExported: true }]);
const exportedTypeMap = new Map<string, Map<string, string>>();
const enriched = enrichExportedTypeMap(acc, graph, exportedTypeMap);
expect(enriched).toBe(1);
expect(exportedTypeMap.get('src/utils.ts')?.get('helper')).toBe('(arg: string) => User');
});
it('skips non-exported Variable nodes', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/app.ts', [{ scope: '', varName: 'dbClient', typeName: 'Database' }]);
acc.finalize();
const graph = makeGraphLookup([{ id: 'Variable:src/app.ts:dbClient', isExported: false }]);
const exportedTypeMap = new Map<string, Map<string, string>>();
const enriched = enrichExportedTypeMap(acc, graph, exportedTypeMap);
expect(enriched).toBe(0);
expect(exportedTypeMap.has('src/app.ts')).toBe(false);
});
it('enriches exportedTypeMap with an exported Const node', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/config.ts', [{ scope: '', varName: 'API_URL', typeName: 'string' }]);
acc.finalize();
const graph = makeGraphLookup([{ id: 'Const:src/config.ts:API_URL', isExported: true }]);
const exportedTypeMap = new Map<string, Map<string, string>>();
const enriched = enrichExportedTypeMap(acc, graph, exportedTypeMap);
expect(enriched).toBe(1);
expect(exportedTypeMap.get('src/config.ts')?.get('API_URL')).toBe('string');
});
it('silently skips accumulator entries with no matching graph node', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/missing.ts', [{ scope: '', varName: 'ghost', typeName: 'Ghost' }]);
acc.finalize();
// Empty graph — no nodes at any of the candidate IDs.
const graph = makeGraphLookup([]);
const exportedTypeMap = new Map<string, Map<string, string>>();
// Must not throw; enrichment's `continue` path fires for every
// unmatched entry.
let enriched = -1;
expect(() => {
enriched = enrichExportedTypeMap(acc, graph, exportedTypeMap);
}).not.toThrow();
expect(enriched).toBe(0);
expect(exportedTypeMap.has('src/missing.ts')).toBe(false);
});
it('does not overwrite existing SymbolTable entry (Tier 0 priority)', () => {
// When the SymbolTable's tier-0 extraction pass has already populated
// an entry for a name, the accumulator enrichment must NOT overwrite
// it with a (lower-quality) worker-path binding.
const acc = new BindingAccumulator();
acc.appendFile('src/utils.ts', [
{ scope: '', varName: 'helper', typeName: 'WorkerInferredType' },
]);
acc.finalize();
// Pre-populate exportedTypeMap to simulate what SymbolTable would
// have written in the tier-0 pass.
const exportedTypeMap = new Map<string, Map<string, string>>([
['src/utils.ts', new Map([['helper', 'SymbolTableAuthoritativeType']])],
]);
const graph = makeGraphLookup([{ id: 'Function:src/utils.ts:helper', isExported: true }]);
const enriched = enrichExportedTypeMap(acc, graph, exportedTypeMap);
// Tier 0 wins — the authoritative SymbolTable type survives.
expect(enriched).toBe(0);
expect(exportedTypeMap.get('src/utils.ts')?.get('helper')).toBe(
'SymbolTableAuthoritativeType',
);
});
it('handles nodes whose properties object is undefined (production shape)', () => {
// Regression guard: the real KnowledgeGraph stores isExported under
// `node.properties.isExported` and properties may be undefined for
// some node kinds. The enrichment guard `!node?.properties?.isExported`
// must treat an undefined properties object as non-exported.
const acc = new BindingAccumulator();
acc.appendFile('src/edge.ts', [{ scope: '', varName: 'helper', typeName: 'Helper' }]);
acc.finalize();
const graph: EnrichmentGraphLookup = {
getNode: (id) =>
id === 'Function:src/edge.ts:helper'
? ({ id, properties: undefined } satisfies EnrichmentGraphNode)
: undefined,
};
const exportedTypeMap = new Map<string, Map<string, string>>();
const enriched = enrichExportedTypeMap(acc, graph, exportedTypeMap);
expect(enriched).toBe(0);
expect(exportedTypeMap.has('src/edge.ts')).toBe(false);
});
it('returns 0 and leaves exportedTypeMap untouched when accumulator is empty', () => {
const acc = new BindingAccumulator();
acc.finalize();
const graph = makeGraphLookup([{ id: 'Function:src/utils.ts:helper', isExported: true }]);
const existingMap = new Map<string, Map<string, string>>([
['src/existing.ts', new Map([['keep', 'Type']])],
]);
const enriched = enrichExportedTypeMap(acc, graph, existingMap);
expect(enriched).toBe(0);
expect(existingMap.size).toBe(1);
expect(existingMap.get('src/existing.ts')?.get('keep')).toBe('Type');
});
});
// -------------------------------------------------------------------------
// BindingAccumulator.dispose() releases the accumulator's heap footprint
// after the enrichment loop has consumed everything it needs. Post-dispose
// reads return empty/undefined without throwing, matching "never-appended"
// state. Idempotent and orthogonal to finalize().
// -------------------------------------------------------------------------
describe('dispose', () => {
it('empties all read methods after dispose', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [
{ scope: '', varName: 'x', typeName: 'X' },
{ scope: 'fn@10', varName: 'y', typeName: 'Y' },
]);
acc.appendFile('src/b.ts', [{ scope: '', varName: 'z', typeName: 'Z' }]);
// Sanity: pre-dispose state is populated.
expect(acc.fileCount).toBe(2);
expect(acc.totalBindings).toBe(3);
acc.dispose();
// Post-dispose state: all read methods return empty/undefined.
expect(acc.fileCount).toBe(0);
expect(acc.totalBindings).toBe(0);
expect([...acc.files()]).toEqual([]);
expect(acc.getFile('src/a.ts')).toBeUndefined();
expect(acc.getFile('src/b.ts')).toBeUndefined();
expect(acc.fileScopeEntries('src/a.ts')).toEqual([]);
expect(acc.fileScopeEntries('src/b.ts')).toEqual([]);
});
it('is idempotent — calling twice is a no-op', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'X' }]);
acc.dispose();
expect(() => acc.dispose()).not.toThrow();
expect(acc.fileCount).toBe(0);
expect(acc.totalBindings).toBe(0);
});
it('works before finalize() — accumulator behaves like a fresh one after dispose', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'X' }]);
acc.dispose();
// Not finalized, so appends still work post-dispose.
expect(() =>
acc.appendFile('src/b.ts', [{ scope: '', varName: 'y', typeName: 'Y' }]),
).not.toThrow();
expect(acc.fileCount).toBe(1);
expect(acc.totalBindings).toBe(1);
expect(acc.getFile('src/b.ts')).toHaveLength(1);
expect(acc.getFile('src/a.ts')).toBeUndefined();
});
it('works after finalize() — append still throws, reads return empty', () => {
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'X' }]);
acc.finalize();
acc.dispose();
// Finalized, so appends throw even post-dispose.
expect(() =>
acc.appendFile('src/b.ts', [{ scope: '', varName: 'y', typeName: 'Y' }]),
).toThrow(/finalize/);
// But reads return empty.
expect(acc.fileCount).toBe(0);
expect(acc.totalBindings).toBe(0);
expect(acc.getFile('src/a.ts')).toBeUndefined();
});
it('estimateMemoryBytes drops to zero after dispose', () => {
const acc = new BindingAccumulator();
// Populate a large batch to give the estimate a non-trivial baseline.
for (let i = 0; i < 100; i++) {
acc.appendFile(`src/file${i}.ts`, [
{ scope: '', varName: `var${i}a`, typeName: 'string' },
{ scope: '', varName: `var${i}b`, typeName: 'number' },
]);
}
const preDisposeBytes = acc.estimateMemoryBytes();
expect(preDisposeBytes).toBeGreaterThan(0);
acc.dispose();
// After dispose, the iteration over `_allByFile` in estimateMemoryBytes
// has zero files to walk, so the returned value is exactly 0.
expect(acc.estimateMemoryBytes()).toBe(0);
});
it('disposed getter reflects dispose state', () => {
// Locks in the `get disposed()` contract for API symmetry with
// `get finalized()`. Without this test, a trivial wrong impl like
// `get disposed() { return this._finalized; }` passes everything.
const acc = new BindingAccumulator();
expect(acc.disposed).toBe(false);
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'X' }]);
expect(acc.disposed).toBe(false);
acc.dispose();
expect(acc.disposed).toBe(true);
acc.dispose(); // idempotent
expect(acc.disposed).toBe(true);
});
it('dispose then finalize: appends throw, state is consistent', () => {
// Orthogonality check: dispose() and finalize() are independent
// lifecycle dimensions. dispose → finalize → appendFile should throw
// the finalized error (because finalize was called), and the
// accumulator should report both flags as true.
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [{ scope: '', varName: 'x', typeName: 'X' }]);
acc.dispose();
acc.finalize();
expect(acc.disposed).toBe(true);
expect(acc.finalized).toBe(true);
expect(() =>
acc.appendFile('src/b.ts', [{ scope: '', varName: 'y', typeName: 'Y' }]),
).toThrow(/finalize/);
});
it('fileScopeEntries returns a defensive copy — mutation does not corrupt state', () => {
// Encapsulation guard: the cached internal array must not be exposed
// by reference. Mutating the returned array should not affect
// subsequent reads.
const acc = new BindingAccumulator();
acc.appendFile('src/a.ts', [
{ scope: '', varName: 'x', typeName: 'X' },
{ scope: '', varName: 'y', typeName: 'Y' },
]);
const firstRead = acc.fileScopeEntries('src/a.ts');
expect(firstRead).toHaveLength(2);
// Try to corrupt internal state via the returned array. The
// `readonly` return type is compile-time only; cast to mutable at
// runtime to simulate a consumer that bypasses TypeScript.
const mutableView = firstRead as unknown as [string, string][];
mutableView.push(['corrupted', 'Corrupt']);
mutableView.length = 0;
// Subsequent reads are unaffected by the mutation attempt.
const secondRead = acc.fileScopeEntries('src/a.ts');
expect(secondRead).toHaveLength(2);
expect(secondRead[0][0]).toBe('x');
expect(secondRead[1][0]).toBe('y');
});
});
});
+212
View File
@@ -1,5 +1,6 @@
import { describe, it, expect, vi } from 'vitest';
import { buildTypeEnv, type TypeEnvironment } from '../../src/core/ingestion/type-env.js';
import { BindingAccumulator } from '../../src/core/ingestion/binding-accumulator.js';
import {
createSymbolTable,
type SymbolDefinition,
@@ -5859,4 +5860,215 @@ function process() {
expect(typeEnv.lookup('c', validateCall)).toBe('Config');
});
});
describe('flush', () => {
it('flushes file-scope bindings into accumulator', () => {
const code = `const user: User = getUser();\nconst count: number = 0;`;
const tree = parse(code, TypeScript.typescript);
const typeEnv = buildTypeEnv(tree, 'typescript');
const acc = new BindingAccumulator();
typeEnv.flush('/src/test.ts', acc);
const entries = acc.getFile('/src/test.ts');
expect(entries).toBeDefined();
const userEntry = entries!.find((e) => e.varName === 'user');
expect(userEntry).toBeDefined();
expect(userEntry!.typeName).toBe('User');
expect(userEntry!.scope).toBe('');
});
// flush() is narrowed to file-scope-only, matching the worker-path
// narrowing. Function-scope entries are dropped at
// the flush seam and never reach the accumulator until a Phase 9
// consumer lands. This test was previously the positive assertion that
// function-scope entries DID land in the accumulator; it is now a
// negative assertion guarding the narrowing.
it('does NOT flush function-scoped bindings into accumulator (file-scope narrowing)', () => {
const code = `function process() {\n const result: Response = fetch();\n}`;
const tree = parse(code, TypeScript.typescript);
const typeEnv = buildTypeEnv(tree, 'typescript');
const acc = new BindingAccumulator();
typeEnv.flush('/src/test.ts', acc);
// With only a function-scope binding (`result` inside `process()`) and
// no file-scope bindings, the accumulator should have nothing for this
// file — the function-scope entry is dropped at the flush boundary.
const entries = acc.getFile('/src/test.ts');
expect(entries).toBeUndefined();
expect(acc.fileCount).toBe(0);
expect(acc.totalBindings).toBe(0);
});
it('narrows mixed file-scope and function-scope env to file-scope only', () => {
// Core narrowing assertion: a realistic file with BOTH file-scope and
// function-scope bindings flushes only the file-scope subset. This is
// the primary narrowing-contract assertion for the sequential path.
const code = `const dbClient: Database = connectDb();\nfunction handleRequest() {\n const localRequest: Request = parseRequest();\n const localUser: User = loadUser();\n}`;
const tree = parse(code, TypeScript.typescript);
const typeEnv = buildTypeEnv(tree, 'typescript');
const acc = new BindingAccumulator();
typeEnv.flush('/src/service.ts', acc);
const entries = acc.getFile('/src/service.ts');
expect(entries).toBeDefined();
// Exactly one entry: the file-scope `dbClient`. The two function-scope
// entries (`localRequest`, `localUser`) are dropped.
expect(entries).toHaveLength(1);
expect(entries![0].scope).toBe('');
expect(entries![0].varName).toBe('dbClient');
expect(entries![0].typeName).toBe('Database');
// Function-scope entries are absent from the accumulator.
expect(entries!.find((e) => e.varName === 'localRequest')).toBeUndefined();
expect(entries!.find((e) => e.varName === 'localUser')).toBeUndefined();
});
it('flushes nothing for an empty TypeEnv', () => {
const code = `// empty file`;
const tree = parse(code, TypeScript.typescript);
const typeEnv = buildTypeEnv(tree, 'typescript');
const acc = new BindingAccumulator();
typeEnv.flush('/src/empty.ts', acc);
expect(acc.getFile('/src/empty.ts')).toBeUndefined();
});
it('multiple files flush into same accumulator', () => {
const code1 = `const a: A = makeA();`;
const code2 = `const b: B = makeB();`;
const tree1 = parse(code1, TypeScript.typescript);
const tree2 = parse(code2, TypeScript.typescript);
const typeEnv1 = buildTypeEnv(tree1, 'typescript');
const typeEnv2 = buildTypeEnv(tree2, 'typescript');
const acc = new BindingAccumulator();
typeEnv1.flush('/src/a.ts', acc);
typeEnv2.flush('/src/b.ts', acc);
expect(acc.fileCount).toBe(2);
expect(acc.getFile('/src/a.ts')).toBeDefined();
expect(acc.getFile('/src/b.ts')).toBeDefined();
});
it('throws on second flush of the same TypeEnv (single-use)', () => {
const code = `const x: X = makeX();`;
const tree = parse(code, TypeScript.typescript);
const typeEnv = buildTypeEnv(tree, 'typescript');
const acc = new BindingAccumulator();
typeEnv.flush('/src/a.ts', acc);
expect(() => typeEnv.flush('/src/a.ts', acc)).toThrow(/single-use/);
});
});
// ---------------------------------------------------------------------
// End-to-end integration: drive real TypeEnv → real flush → real
// BindingAccumulator → real enrichExportedTypeMap with a realistic
// graph-node shape. Every other accumulator test is unit-level with
// mocks; this exercises the full wiring between layers that the
// accumulator's bug history has all been in. If the wiring breaks
// (e.g. a future refactor changes TypeEnv's flush output, or the
// enrichment helper's node-ID format drifts), this test fires.
// ---------------------------------------------------------------------
describe('end-to-end: real TypeEnv → flush → accumulator → enrichment', () => {
it('enriches exportedTypeMap with bindings from a real TypeScript file', async () => {
// Lazy import to keep the test co-located without hoisting binding
// accumulator imports to the top of the type-env test file.
const { enrichExportedTypeMap, type: _ignore } =
(await import('../../src/core/ingestion/binding-accumulator.js')) as typeof import('../../src/core/ingestion/binding-accumulator.js') & {
type: unknown;
};
const code = `
export const dbClient: Database = connectDb();
export const API_URL: string = 'https://api.example.com';
function internal() {
const localVar: LocalType = makeLocal();
}
`;
const tree = parse(code, TypeScript.typescript);
const typeEnv = buildTypeEnv(tree, 'typescript');
const acc = new BindingAccumulator();
// Real flush — exercises the narrowed FILE_SCOPE-only iteration.
typeEnv.flush('src/service.ts', acc);
acc.finalize();
// Verify the flush wrote only file-scope entries (no `localVar`).
const entries = acc.getFile('src/service.ts');
expect(entries).toBeDefined();
const varNames = (entries ?? []).map((e) => e.varName).sort();
expect(varNames).toEqual(['API_URL', 'dbClient']);
for (const entry of entries ?? []) {
expect(entry.scope).toBe('');
}
// Build a minimal realistic graph matching the production node-ID
// candidate order (Function → Variable → Const). The dbClient is
// exported as a Variable, API_URL is exported as a Const.
const graph = {
getNode: (id: string) => {
if (id === 'Variable:src/service.ts:dbClient') {
return { id, properties: { isExported: true } };
}
if (id === 'Const:src/service.ts:API_URL') {
return { id, properties: { isExported: true } };
}
return undefined;
},
};
const exportedTypeMap = new Map<string, Map<string, string>>();
// Real enrichment — not a reimplementation.
const enrichedCount = enrichExportedTypeMap(acc, graph, exportedTypeMap);
expect(enrichedCount).toBe(2);
expect(exportedTypeMap.get('src/service.ts')?.get('dbClient')).toBe('Database');
expect(exportedTypeMap.get('src/service.ts')?.get('API_URL')).toBe('string');
// The function-scope `localVar` is absent because flush() narrowed
// it out before it could reach the accumulator.
expect(exportedTypeMap.get('src/service.ts')?.has('localVar')).toBe(false);
// Lifecycle: dispose releases heap.
acc.dispose();
expect(acc.disposed).toBe(true);
expect(acc.fileCount).toBe(0);
});
it('respects Tier 0 priority when the SymbolTable pre-populated the export', async () => {
const { enrichExportedTypeMap } =
await import('../../src/core/ingestion/binding-accumulator.js');
const code = `export const helper: WorkerInferredType = makeHelper();`;
const tree = parse(code, TypeScript.typescript);
const typeEnv = buildTypeEnv(tree, 'typescript');
const acc = new BindingAccumulator();
typeEnv.flush('src/utils.ts', acc);
acc.finalize();
// Simulate SymbolTable pre-populating the exportedTypeMap with an
// authoritative Tier 0 type. The real enrichment loop must NOT
// overwrite it with the WorkerInferredType from the accumulator.
const exportedTypeMap = new Map<string, Map<string, string>>([
['src/utils.ts', new Map([['helper', 'SymbolTableAuthoritativeType']])],
]);
const graph = {
getNode: (id: string) =>
id === 'Const:src/utils.ts:helper' || id === 'Variable:src/utils.ts:helper'
? { id, properties: { isExported: true } }
: undefined,
};
const enriched = enrichExportedTypeMap(acc, graph, exportedTypeMap);
expect(enriched).toBe(0);
expect(exportedTypeMap.get('src/utils.ts')?.get('helper')).toBe(
'SymbolTableAuthoritativeType',
);
});
});
});