ac2012e5ed10ae93a4792f60a9cd8746dc35e5c7
431
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Gergő Magyar
|
ac2012e5ed |
feat(shared): DefIndex / ModuleScopeIndex / QualifiedNameIndex (#913, RFC #909 Ring 2 SHARED) (#958)
Three flat O(1) indexes + pure build functions over per-file artifacts. Contract-only; no runtime behavior change yet — consumers (#917 Registry lookups, #915 SCC finalize, #919 ScopeExtractor) wire in later. Each index follows the same shape: - build function: flat input list → frozen immutable index - public interface: readonly Map + get/has/size accessors - first-write-wins on id/filePath collisions (upstream bug signal) - pure, side-effect-free, safe to call repeatedly DefIndex — the global "what is this id?" lookup gitnexus-shared/src/scope-resolution/def-index.ts buildDefIndex(defs: readonly SymbolDefinition[]): DefIndex byId: ReadonlyMap<DefId, SymbolDefinition> Consumed by Registry.lookup (#917) to materialize DefId[] hits back to full SymbolDefinition records. ModuleScopeIndex — `filePath → moduleScopeId` for cross-file hops gitnexus-shared/src/scope-resolution/module-scope-index.ts buildModuleScopeIndex(entries): ModuleScopeIndex byFilePath: ReadonlyMap<string, ScopeId> Consumed by the SCC finalize link pass (#915) to resolve ImportEdge.targetFile to a concrete module scope in constant time. QualifiedNameIndex — cross-kind qualified-name fast path gitnexus-shared/src/scope-resolution/qualified-name-index.ts buildQualifiedNameIndex(defs: readonly SymbolDefinition[]): QualifiedNameIndex byQualifiedName: ReadonlyMap<string, readonly DefId[]> Returns DefId[] (not a single DefId) because partial classes, method overloads, and cross-kind collisions can legitimately share a qualifiedName. Callers filter by acceptedKinds at the lookup site. Consumed by Registry.lookup qualified fast path + resolveTypeRef dotted fallback (#916, #917). Barrel re-exports added to gitnexus-shared/src/index.ts so consumers import from 'gitnexus-shared' rather than deep paths. Tests (gitnexus/test/unit/scope-resolution/, 23 total): def-index.test.ts (6): empty, single def, multiple distinct, first-write-wins collision, missing id returns undefined, byId direct iteration module-scope-index.test.ts (6): empty, single entry, multiple files, first-write-wins on duplicate filePath, missing returns undefined, byFilePath direct iteration qualified-name-index.test.ts (11): empty, single qnamed def, partial classes accumulate, input-order preservation, qname separation, skip undefined/empty qname, pair dedup, cross-kind indexing, frozen-empty-array on miss, direct iteration Verification: - gitnexus-shared + gitnexus build clean (tsc + scripts/build.js) - test/unit/scope-resolution: 23/23 pass - model + shadow + scope-resolution combined: 129/129 pass - No runtime consumer wiring yet — indexes are standalone library functions that #915, #917, #919 will import when ready Depends on #910 (SymbolDefinition, DefId, ScopeId types — already on main). Unblocks #915 (finalize algorithm), #917 (Registry.lookup), #919 (ScopeExtractor materialization). |
||
|
Copilotandmagyargergo
|
f73389eac3 |
fix: ENOBUFS in detect_changes by setting maxBuffer on git/rg execFileSync (#957)
* Initial plan * Fix ENOBUFS in detect_changes by setting maxBuffer on git/rg execFileSync Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bb241ed0-3b39-431f-a242-b0c7ced9707b Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Gergő Magyar
|
22f0beb057 |
feat(shared): shadow-mode diff + aggregate — full implementation (#918, RFC #909 Ring 2 SHARED) (#951)
Replaces the scaffold stubs with working pure-logic implementations plus
unit-test coverage for both functions. Unblocks Ring 2 PKG #923 (shadow
harness) to consume a concrete library instead of throwing scaffolds.
gitnexus-shared/src/scope-resolution/shadow/diff.ts
`diffResolutions(callsite, legacy, newResult): ShadowDiff`
- [0] on each side is the top match
- both empty → 'both-empty', delta []
- legacy empty only → 'only-new', delta = new top evidence
- new empty only → 'only-legacy', delta = legacy top evidence
- same top nodeId → 'both-agree', delta []
- different nodeIds → 'both-disagree',
delta = symmetric difference of evidence kinds
(legacy-only first in input order, then new-only)
Evidence identity is `ResolutionEvidence.kind` — weight/note differences
for the same kind do NOT produce delta entries. Rationale: the aggregator
wants to know which *signals* explain a disagreement, not fluctuations
in calibration values.
gitnexus-shared/src/scope-resolution/shadow/aggregate.ts
`aggregateDiffs(diffs, now?): ShadowParityReport`
- buckets by `SupportedLanguages`
- tallies agreements, evidence-breakdown (divergences only — agree and
empty rows do not contribute)
- parity = bothAgree / (totalCalls - bothEmpty), yields 0 (not NaN)
when the denominator is 0
- perLanguage sorted alphabetically by enum value for stable output
- evidenceBreakdown internally sorted by kind for stable output
- overall = column-wise sum across languages
- `now` parameter makes generatedAt deterministic in tests
gitnexus-shared/src/index.ts
Re-exports the full shadow API: diffResolutions, aggregateDiffs, and all
their types (ShadowAgreement, ShadowCallsite, ShadowDiff,
LanguageParityRow, ShadowParityReport).
gitnexus/test/unit/shadow/diff.test.ts (13 tests)
- 5 agreement outcomes
- symmetric-by-kind evidence delta (disjoint, overlapping, fully-overlapping)
- weight-only differences produce no delta
- top-match only (ignores indices beyond [0])
- callsite passthrough
- delta ordering (legacy-only first, input order preserved)
gitnexus/test/unit/shadow/aggregate.test.ts (9 tests)
- empty input
- single language, all agree / mixed / all empty
- multi-language bucketing + overall sum
- alphabetical language sort
- evidence breakdown scope
- determinism via injected `now` + JSON round-trip identity
Verification:
- gitnexus-shared + gitnexus build clean (tsc + scripts/build.js)
- test/unit/shadow: 22/22 pass
- test/unit/model + test/unit/shadow combined: 106/106 pass
- No runtime behavior changes (shadow is invoked by #923, not yet wired)
Stacked on main (
|
||
|
Gergő Magyar
|
af1d278a7e |
feat(shared,ingestion): extend LanguageProvider with scope-resolution hooks (#911, RFC #909 Ring 1) (#950)
Adds the 14 optional scope-resolution hooks from RFC #909 §5.2 to
`LanguageProviderConfig` plus the supporting input/output types in
`gitnexus-shared`. Contract-only; no runtime behavior changes.
Review-driven refinements (addresses two non-blocking review comments on #950):
1. `ParsedImport` is now a 5-variant discriminated union, not a flat
record. Each variant carries only its legal fields so invalid shapes
are compile errors:
- 'named', 'alias', 'namespace', 'reexport', 'dynamic-unresolved'
'wildcard-expanded' is deliberately excluded — finalize materializes
that kind; a provider must never emit it at parse time.
'reexport' is a first-class parse-phase variant so syntactically-
detectable re-exports (TS `export { X } from './y'`, Rust
`pub use foo::bar`) keep their parse-time signal through to finalize
rather than being re-derived by the SCC pass.
`namespace` gains an `importedName` field so `import numpy as np`
can carry both `localName: 'np'` and `importedName: 'numpy'`.
`dynamic-unresolved.targetRaw` is `string | null` (was mandatory
null) so providers can emit the unresolvable expression text for
diagnostics when available.
2. `bindingScopeFor` and `importOwningScope` return type changed from
`ScopeId` to `ScopeId | null`, aligning with the X | null convention
used by the 12 sibling optional hooks (receiverBinding,
resolveScopeKind, interpretTypeBinding, …). `null` = delegate to the
central default. Enables partial overrides — a JS provider can
return a hoisted scope for `var` and `null` for `let`/`const`
without re-implementing the default lookup.
Both hooks also gain a purity JSDoc contract: same inputs yield the
same ScopeId (or null) across invocations; no closure over mutable
state. Required to keep scope-tree construction deterministic.
A richer callable-defaults pattern (typed BindingScopeDefaults /
ImportOwningDefaults helper interfaces on a `defaults` parameter)
was considered and deferred to Ring 2 PKG #919, where the concrete
ScopeExtractor will exist to inform the helper shape. Designing that
pattern before the first consumer would set cross-hook precedent
based on a single motivating example.
Supporting types added to gitnexus-shared/src/scope-resolution/types.ts:
- CaptureMatch, ParsedImport, ParsedTypeBinding
- WorkspaceIndex, ScopeTree (opaque placeholders until Ring 2)
- Callsite
14 hooks added to LanguageProviderConfig (all optional):
Parse phase: emitScopeCaptures, interpretImport, receiverBinding,
interpretTypeBinding, resolveScopeKind, shouldCreateScope,
bindingScopeFor
Finalize phase: resolveImportTarget, expandsWildcardTo,
importOwningScope, mergeBindings
Reference-extraction phase: classifyCallForm
Resolution phase: shouldShadow, arityCompatibility
Verification:
- gitnexus-shared builds clean (tsc)
- gitnexus builds clean (scripts/build.js)
- test/unit/model: 84/84 pass — no regressions
- No provider needs updating (all hooks optional)
- No BindingScopeDefaults/ImportOwningDefaults/defaults parameter
introduced (deferred to #919)
Stacked on #910 (merged as
|
||
|
Gergő Magyar
|
afc0a8b6c5 |
feat(shared): add scope-resolution types + constants (#910, RFC #909 Ring 1) (#949)
Lands the authoritative data model and constants for the pure scope-based resolution RFC (#909) as Ring 1, part 1. No runtime behavior changes — types + constants only. New in gitnexus-shared/src/scope-resolution/: - types.ts — Scope, ScopeKind, ScopeId, DefId, Range, Capture, BindingRef, ImportEdge, TypeRef, Resolution, ResolutionEvidence, Reference, ReferenceIndex, LookupParams, RegistryContributor - evidence-weights.ts — EvidenceWeights constant map + typeBindingWeightAtDepth (RFC Appendix A) - origin-priority.ts — ORIGIN_PRIORITY constant map for deterministic tie-breaks (RFC Appendix B) - language-classification.ts — LanguageClassification type + LanguageClassifications map (production × 14, experimental × 2 for vue/cobol; governs Ring 4 DAG-retirement gate) - symbol-definition.ts — SymbolDefinition moved from gitnexus/src/core/ingestion/model/symbol-table.ts so scope-resolution types can reference it from the shared package Consumer updates: - symbol-table.ts: removes local SymbolDefinition declaration; imports from gitnexus-shared - model/index.ts: drops SymbolDefinition from barrel re-export per "direct imports from gitnexus-shared" convention (see gitnexus-shared feedback in project memory) - 9 source files + 5 test files: import SymbolDefinition directly from 'gitnexus-shared' Verification: - gitnexus-shared builds clean (tsc) - gitnexus builds clean (scripts/build.js) - 131/132 unit test files pass; 3767 tests green - Zero behavior changes; SymbolDefinition shape unchanged Blocks: #911 (LanguageProvider hook interface extensions) and all of Ring 2 (#912-#925). Closes part of #909. |
||
|
Gergő Magyar
|
d9da7d6692 |
fix(test): isolate cli-e2e from shared mini-repo fixture (#954)
Deterministic fix for the Windows-flaky pipeline-graph-golden test.
Root cause
cli-e2e.test.ts wrote into the SHARED fixture directory
(test/fixtures/mini-repo/) — git init, analyze run that creates
AGENTS.md, CLAUDE.md, .claude/, .gitnexus/. When pipeline-graph-golden
ran in parallel, its `cpSync` of the source directory could capture
the mid-flight pollution before cli-e2e's afterAll cleanup fired.
macOS/Ubuntu won the race often enough that the flake presented as
Windows-only.
Fix
cli-e2e now copies mini-repo into a fresh `mkdtemp`'d parent whose
basename is `mini-repo` (preserving `--repo mini-repo` CLI lookup by
basename), runs git-init there, and rm's the whole tmpdir in afterAll.
The shared fixture source is never touched.
Fallout from the cwd change: bare `--import tsx` specifiers (2
spawnSync + 1 spawn) can't resolve `tsx` from an os.tmpdir cwd where
there is no node_modules. Switched them to the already-existing
`tsxImportUrl` (absolute file:// URL to the tsx loader), matching
the `runCliOutsideProject` pattern that was already set up for this
exact case.
Updated the "MINI_REPO is inside the project tree" comment in the
`status on non-indexed repo` test — MINI_REPO is now in os.tmpdir,
so the rationale for using a separate throwaway tmp git repo is
different (but still valid: previous tests in the suite create
MINI_REPO/.gitnexus, which findRepo() would pick up).
Also updated pipeline-graph-golden's comment explaining WHY it
copies to tmp — it's now defense-in-depth rather than a necessity,
so a future test that adds files to the source can't silently
regress the golden.
Verification
- 5x consecutive `cli-e2e + pipeline-graph-golden` runs: 20/20 pass
(deterministic)
- 3x full suite including pipeline.test: 27/27 pass
- test/fixtures/mini-repo/ post-run contents: only `src/` —
zero pollution from any test
- macOS/Ubuntu behavior unchanged (they were passing; tmpdir
isolation is purely additive)
|
||
|
azizur100389
|
131d411ae4 |
feat(mcp): rank context/impact disambiguation candidates and expose kind/file_path hints (#888)
* feat(mcp): rank context/impact disambiguation candidates and expose kind/file_path hints
The `context` MCP tool already returned `{ status: 'ambiguous', candidates }`
when a name hit multiple symbols, but the candidates were returned in
arbitrary DB order and the only hint it accepted was file_path. The
`impact` tool was worse: when its name resolver found multiple viable
matches it silently picked the first one from a priority UNION, with no
signal back to the caller that a different symbol might have been
intended.
Both failure modes were flagged in issue #470 and reconfirmed in the
comments by a second user who described impact as returning "incorrect
parsing results and meaningless tool calls" in the multi-match case.
Changes:
* Add `resolveSymbolCandidates(repo, query, hints)` private helper on
LocalBackend. Single place that:
- Short-circuits on direct uid (zero-ambiguity)
- Runs the same name-or-qualified-id match as before, with LIMIT 20
(was 10) so the ranker has headroom instead of arbitrary truncation
- Preserves the #480 Class/Constructor preference -- when the only
ambiguity is a Class and its own Constructor, the Class wins
silently
- Scores each candidate (pure TS, no extra DB round-trip): base 0.50,
+0.40 for file_path match, +0.20 for kind match, plus a small
kind-priority tiebreaker (Class > Interface > Function > Method >
Constructor) when no explicit kind hint is given
- Sorts desc by score with stable tiebreakers (shorter filePath,
then lex uid)
- Promotes to a single confident resolve when the top score is
>= 0.95 AND beats the runner-up by >= 0.10 -- lets a strong hint
cut through without forcing the caller through a disambiguation
round-trip
* Rewire `context()` to use the shared helper. Response shape is a
strict superset of today's: candidates gain a `score` field, the
existing `{ uid, name, kind, filePath, line }` keys are preserved so
every downstream consumer (rename, eval-server formatter, etc.) keeps
working. New `kind` input hint accepted.
* Rewire `impact()` to use the shared helper. Now emits the same
`{ status: 'ambiguous', candidates, impactedCount: 0, risk: 'UNKNOWN' }`
shape instead of silent first-pick. New inputs accepted:
`target_uid`, `file_path`, `kind`.
* Update tool schemas in mcp/tools.ts to advertise the new inputs and
describe ranked disambiguation.
Backward compatibility:
The #480 Class/Constructor collapse is preserved and covered by the
existing java-class-impact integration test (still green). The
ambiguous response shape is a strict superset -- `eval-formatters`
unit test that parses the old shape is unchanged and still passes.
`impact` going from silent-first-pick to structured ambiguous is a
semantic improvement that is the entire point of the issue; callers
relying on silent first-pick now get an actionable response.
Scope declined for v1:
module/community hint -- the issue lists it as one of several hints,
but kind + file_path cover the vast majority of disambiguation needs
in practice, and a community-label filter requires an extra graph
query per candidate. Natural v2 follow-up.
Tests: calltool-dispatch.test.ts gains 5 new cases covering file_path
boost, kind hint boost, impact ambiguous shape, impact target_uid
short-circuit, and score field presence on the existing ambiguous
test. Plus the extended assertions on the existing
`context tool returns disambiguation for multiple matches`.
Verification:
npx vitest run test/unit/calltool-dispatch.test.ts -> 64 pass
npx vitest run test/integration/java-class-impact.test.ts -> pass
npm run test:unit -> 3642 pass
(4 pre-existing env failures unchanged: skip-git-cli needs built
dist/, git-utils tmpdir on Windows worktree -- same on main)
npx tsc --noEmit -> clean
Closes #470
* fix(mcp): enrich labels from UNION when labels(n)[0] is empty; address review findings
CI on PR #888 caught 13 integration-test failures I did not cover locally:
my resolver refactor collected candidates via `labels(n)[0] AS type`, but
LadybugDB returns an empty string for that projection on certain node
types (most importantly Class). With an empty `type`, impact's downstream
`_runImpactBFS` no longer recognised `symType === 'Class' | 'Interface'`
and stopped seeding Constructor + File nodes into the frontier, so the
"impact(upstream) surfaces the file importer" assertion broke across 11
language fixtures plus 2 OVERRIDES filter tests.
The original impact resolver worked around this by running a prioritised
UNION across Class/Interface/Function/Method/Constructor and picking the
first hit. My refactor dropped that. Fix: keep the simple candidate MATCH
but enrich types afterward via a single scoped UNION query, so every
candidate carries an accurate label for both scoring and downstream
BFS seeding. The UID direct-lookup path is patched the same way.
Also addresses the findings from the senior reviewer on PR #888:
* MIGRATION.md: document the `impact` behavioural change (silent first-
pick → structured `{ status: 'ambiguous', candidates }`) so downstream
callers know to branch on `result.status` before reading byDepth/
summary. `context` is unchanged shape-wise (strict superset).
* New test: `context tool promotes top candidate via scoring when
multiple rows survive DB pre-filter`. The review flagged that the
existing file_path test works only because the mock ignores WHERE
parameters -- the scored-promotion path (top ≥ 0.95 AND gap > 0.09)
wasn't directly exercised. The new test uses two candidates both in
App.tsx-containing paths plus a kind hint so promotion is decided by
scoring, not DB pre-filtering. Also tightened the comment on the
earlier file_path test to describe the mock vs production divergence
honestly.
* NIT: added a paragraph explaining why `scored.length >= 2` is kept as
a defensive guard even though the `normalized.length === 1` early
return already covers the single-candidate path.
* Integration: two tests in `local-backend-calltool.test.ts` targeted
`'authenticate'`, which now correctly resolves as ambiguous (two
Method nodes: AuthService.authenticate and BaseService.authenticate).
Updated both to pass `file_path: 'src/auth.ts'` so they exercise the
new disambiguation API and still assert the METHOD_OVERRIDES filtering
they were originally about.
Edge case fix in the promotion gap check: IEEE754 makes 0.50 + 0.40 +
0.20 - 0.90 = 0.09999999999999998 instead of exactly 0.10, which would
otherwise break the "winner clearly dominates" intent for legitimate
1.00 vs 0.90 cases. Changed `>= 0.10` to `> 0.09`; same user-facing
intent, no floating-point sensitivity.
Verification (all from gitnexus/):
npx vitest run test/integration/class-impact-all-languages.test.ts
-> 52 pass (was 11 FAIL on CI before this fix)
npx vitest run test/integration/local-backend-calltool.test.ts
-> 18 pass (was 2 FAIL on CI before this fix)
npx vitest run test/integration/java-class-impact.test.ts
-> 10 pass (regression guard for #480 preserved)
npx vitest run test/unit/calltool-dispatch.test.ts
-> 65 pass (1 new test + 4 from original #470 PR)
npm run test:unit
-> 3626 pass, 4 pre-existing env failures unchanged
npx tsc --noEmit
-> clean
|
||
|
Gergő Magyar
|
b8875b9c80 |
chore(release): v1.6.2 (#952)
Bumps gitnexus to v1.6.2 and adds the matching CHANGELOG entry. Highlights since v1.6.1 (61 commits): - Docker support (#848) - Language-agnostic heritage / call / variable extractors (config+factory pattern, #877 #878 #890) - AST-aware embedding chunking (#889) - jQuery / axios HTTP consumer detection (#887) - SemanticModel wired as first-class resolution input, SM-20 (#885) - ImportSemantics split into per-strategy hooks (#886) - Python dotted-import fix (#899); worker warnings non-terminal (#900 / #261); global-install ENOTEMPTY fixes (#843 #846); embeddings staleness fix (#831) See gitnexus/CHANGELOG.md for the full list. After merge, tag `v1.6.2` triggers publish.yml which runs CI, verifies tag↔package.json match, publishes to npm with provenance, and creates the GitHub Release using the extracted CHANGELOG body. |
||
|
Kritik Bangeraandkritik.b
|
040bb7a489 |
feat: add docker support (#848)
* feat: add docker support * feat: move docker files to root * feat: add docker build and push workflow * fix: pin docker action SHAs to verified commits Made-with: Cursor * fix: remove redundant --platform=$TARGETPLATFORM from runtime stage Made-with: Cursor * fix: upgrade docker actions to Node.js 24-compatible versions Made-with: Cursor * docs: updated readme * fix: update docker references * fix(docker-server): reject null bytes in resolvePath Defensively harden the path traversal guard by returning null early when the URL contains a null byte, before normalization runs. Made-with: Cursor * fix(docker-server): handle createReadStream errors Attach an error listener before piping so mid-flight read errors (truncated file, permission change) cleanly destroy the response instead of being silently swallowed. Made-with: Cursor * fix(docker-server): replace existsSync with async stat Eliminates the TOCTOU race between the initial stat call and the subsequent existsSync check. Reuses the async stat pattern already in place and removes the now-unused existsSync import. Made-with: Cursor * test(docker-server): add integration tests; fix %00 null-byte bypass Decode the URL before the null-byte check so percent-encoded null bytes (%00) are also rejected with 400 instead of falling through to the SPA fallback. Adds 5 node:test integration tests covering valid assets, SPA fallback, path traversal, null bytes, and 404. Made-with: Cursor * style: fix prettier formatting in docker-server files Made-with: Cursor * fix(docker): wire tests into CI, fix resolvePath separator, correct image namespace - Add `node --test docker-server.test.mjs` step to ci-tests.yml so the path-traversal guard tests run in every CI pass instead of being silently skipped. - Fix resolvePath containment check: `startsWith(root)` would allow sibling directories like `/app/dist-evil/`; now guards with `root + sep` or exact match. - Update docker-compose.yaml default image from `abhigyanpatwari` namespace to `brainifii` to match what docker.yml publishes to GHCR. * fix(docker): update apt-get commands and set user permissions - Modify Dockerfile and Dockerfile.test to include options for apt-get to bypass validity checks during updates. - Set ownership of the /app directory to the 'node' user in the runtime stage for improved security and proper permission handling. * fix(docker): switch to Alpine base images for smaller footprint - Update Dockerfile to use Alpine-based Node.js images for both builder and runtime stages, reducing image size and improving performance. - Replace apt-get commands with apk for package installation in the runtime stage. * fix(docker): update Node.js version in Dockerfile - Change base image from node:20-alpine to node:22-alpine * fix(docker): update Node.js version in Dockerfile to 22-alpine for runtime --------- Co-authored-by: kritik.b <kritik.b@media.net> |
||
|
dependabot[bot]
|
725ed3fe66 | chore(deps)(deps): bump glob from 11.1.0 to 13.0.6 in /gitnexus (#867) | ||
|
dependabot[bot]
|
509185b8f9 | chore(deps)(deps): bump commander from 12.1.0 to 14.0.3 in /gitnexus (#868) | ||
|
dependabot[bot]
|
ef953beca9 | chore(deps)(deps): bump @huggingface/transformers in /gitnexus (#869) | ||
|
dependabot[bot]
|
0b5381695f | chore(deps)(deps-dev): bump @vitest/coverage-v8 in /gitnexus (#864) | ||
|
dependabot[bot]
|
30292d7179 | chore(deps)(deps): bump @modelcontextprotocol/sdk in /gitnexus (#866) | ||
|
dependabot[bot]
|
7a98a01ad5 | chore(deps)(deps): bump lru-cache from 11.2.7 to 11.3.5 in /gitnexus (#870) | ||
|
dependabot[bot]
|
94cba48b6d | chore(deps)(deps): bump mnemonist from 0.39.8 to 0.40.3 in /gitnexus (#871) | ||
|
azizur100389
|
925460ab5b | refactor(cli): trim duplicated ai-context CLAUDE.md block (#904) | ||
|
dependabot[bot]
|
08b4505197 | chore(deps)(deps): bump @ladybugdb/core in /gitnexus (#873) | ||
|
Copilot
|
dfa449ef41 | feat(ingestion): language-agnostic heritage extractor with config+factory pattern (#890) | ||
|
Yacine Hmito
|
daca8360bf | fix(python): avoid local matches for external dotted imports (#899) | ||
|
Ryanba
|
77a13113ea | fix: keep worker warnings non-terminal (#261) | ||
|
evolution
|
02739085d2 | feat(embeddings): AST-aware chunking with offset-based splitting (#889) | ||
|
|
43098784cf |
refactor(ingestion): split ImportSemantics into per-strategy hooks (Strategies 1-4) (#886)
* Initial plan
* refactor(ingestion): split ImportSemantics into per-strategy hooks
- Add ImportResolverStrategy and ImportResolutionConfig types
- Create createImportResolver factory (resolver-factory.ts)
- Add createStandardStrategy to standard.ts
- Extract per-language strategies from existing resolvers:
goPackageStrategy, javaJvmStrategy, kotlinJvmStrategy,
rustModuleStrategy, pythonImportStrategy, csharpNamespaceStrategy,
phpPsr4Strategy, swiftPackageStrategy, dartPackageStrategy,
dartRelativeStrategy, rubyRequireStrategy
- Create per-language config files in import-resolvers/configs/
- Update all 15 language providers to use createImportResolver(config)
- Add 38 unit tests for factory and strategy composition
- All 3640+ existing tests pass, tsc --noEmit passes
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/c3aec32d-2155-4808-88df-9cd6b2384174
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* chore: remove unused resolver imports from language providers
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/c3aec32d-2155-4808-88df-9cd6b2384174
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* docs: add error propagation note to createImportResolver JSDoc
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/c3aec32d-2155-4808-88df-9cd6b2384174
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* refactor: consolidate strategies into configs, remove legacy resolvers
- Move all strategies from per-language files into their config files
- Remove swift.ts and vue.ts (no shared helpers needed)
- Remove legacy monolithic resolver functions from all per-language files
- Remove unused legacy wrapper functions from standard.ts
- Per-language files now only contain shared internal helpers
- Fix lint warning in languages/php.ts (no-non-null-assertion)
- Update test imports to reference configs/ instead of per-language files
- All 3262+ tests pass, tsc --noEmit passes, zero lint errors
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/f8da6bc2-957c-4d20-87ba-402fa223c6c8
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* refactor: address review feedback — remove dart.ts shim, JSDoc language field, update ARCHITECTURE.md
- Add JSDoc to ImportResolutionConfig.language clarifying it's
documentation-only metadata not used by the factory
- Remove dart.ts legacy shim (was only kept for backward-compat tests)
- Rewrite dart-import-resolver.test.ts to test production strategies
(dartPackageStrategy/dartRelativeStrategy) directly, including full
factory composition via dartImportConfig
- Fix lint warning (no-explicit-any) by using buildSuffixIndex in makeCtx
- Update ARCHITECTURE.md to mention import-resolvers/configs/ as the
extension point for per-language import resolution
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/53f09a4f-1ff1-4a3e-a29c-fda9cdb4c4ef
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: address review comments — revert php.ts, tighten dart test assertion
- Revert php.ts: restore stack.pop()! (the while guard guarantees non-empty)
- Tighten dart relative import test to assert exact result instead of
permissive null-or-files check
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/53f09a4f-1ff1-4a3e-a29c-fda9cdb4c4ef
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* refactor(ingestion): strengthen import-resolver tests and document Vue config intent
Address non-blocking follow-ups from PR #886 review:
- Add inline comment to vueImportConfig explaining intentional
language: Vue / TypeScript-strategy mismatch (Vue SFCs are
preprocessed into TS upstream of import resolution).
- Replace 11 tautological typeof === 'function' assertions with
behavioral tests for goPackageStrategy, kotlinJvmStrategy, and
csharpNamespaceStrategy, including full-chain strategy-order
guards via createImportResolver(config).
- Apply prettier formatting to sibling configs touched during
factory introduction.
Test: 37 passed (previously 26), tsc --noEmit clean.
* test(ingestion): tighten import-resolver assertions and close coverage gaps
Apply ce-review findings on commit
|
||
|
azizur100389
|
f221f93341 |
feat(extractors): detect jQuery $.ajax/$.get/$.post and axios object-form as HTTP consumers (#887)
* feat(extractors): detect jQuery $.ajax/$.get/$.post and axios object-form as HTTP consumers
The JS/TS HTTP consumer extractor currently recognises fetch() and
axios.<verb>() but misses three patterns extremely common in Laravel
and legacy frontends:
- jQuery shorthand: $.get(url), $.post(url, data)
- jQuery ajax form: $.ajax({ url, method }) / $.ajax({ url, type })
- axios object form: axios({ method, url })
Missing them means the frontend->backend cross-link disappears from
`group sync`, breaking impact analysis for whole classes of repos.
Implementation (node.ts):
- 3 new PatternSpecs alongside the existing FETCH_/AXIOS_ specs
- NodePatternBundle extended with jqueryShorthand / jqueryAjax /
axiosObject slots, compiled for JS / TS / TSX grammars
- readStringProp() helper walks object-literal `pair` children and
resolves `url` / `method` / `type` keys independent of order,
sidestepping the positional S-expression constraint on the
query form proposed in the issue
- 3 new scan loops in scanBundle() emit HttpDetection with
framework 'jquery' (new) or 'axios' (existing), confidence 0.7
to match the existing source-scan consumers, defaulting method
to GET when absent (matches both jQuery and axios runtime)
Tests (http-route-extractor.test.ts): 4 new cases -- 3 positive
(shorthand, ajax with method:/type: and default GET, object-form
with swapped key order and default GET) plus 1 negative control
that asserts unrelated \$.fn.extend / \$.each / non-axios helper
calls with {url, method} literals produce zero consumer contracts.
Closes #828
* test(extractors): cover jQuery $.ajax with template-literal URL
Extend the existing $.ajax fixture with `url: \`/api/orders/\${id}\``
and assert the consumer is emitted as http::GET::/api/orders/{param}.
This makes jQuery + template-URL explicit rather than implicit via the
axios object test (readStringProp already accepts template_string for
both; this is coverage, not new behaviour).
Addresses the single non-blocking finding on PR #887.
|
||
|
Copilot
|
a32f5b6adb |
refactor(SM-20): wire SemanticModel as first-class resolution input (#885)
* Initial plan * refactor(SM-20): fix O(n²) BFS in gatherAncestors, complete barrel exports, consolidate imports Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1246d49e-6c67-4c79-935a-4732394b9a7a --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|
Copilotandmagyargergo
|
ed5a4220dd |
feat(ingestion): language-agnostic variable extractor with config+factory pattern (#878)
* Initial plan * feat(ingestion): add variable extraction types, factory, configs, and wire into language providers - Create variable-types.ts with VariableInfo, VariableExtractionConfig, VariableExtractor interfaces - Create variable-extractors/generic.ts with createVariableExtractor() factory - Add variableExtractor field to LanguageProvider interface - Create per-language variable extraction configs for all 16 languages - Wire variableExtractor into all language providers - Add variable metadata enrichment to parse-worker for Const/Static/Variable labels Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * feat(ingestion): add variable extraction tests and fix Python/TS config issues - Create test/unit/variable-extraction.test.ts with 29 tests covering TypeScript, JavaScript, Python, Go, Rust, C, C++, Ruby, and factory behavior - Fix isConst in generic factory to use config.isConst over node-type membership (TS let/const both use lexical_declaration) - Fix Python type extraction for annotated assignments at module scope - Fix Python dunder name visibility (e.g., __name__ is public, not protected) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address code review feedback — move imports, clarify scope comment, use shared test context Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/3cb85c68-1792-473e-9a46-ea2588da0e5e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address review comments, fix prettier formatting and lint errors - Fix prettier formatting in 5 files (c-cpp, jvm, swift configs, test file) - Remove unused SyntaxNode imports in php.ts and ruby.ts (lint errors) - Remove unused constNodeSet/variableNodeSet variables in generic.ts (warnings) - Remove semantically wrong `methodProps.isReadonly = varInfo.isConst` (review) - Remove dead `nodeLabel === 'Variable'` guard in parse-worker (review) - Fix test guard: replace `if (declNode)` with `expect(declNode).toBeDefined()` (review) - Add comment about Python expression_statement broadness (review) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/040edbbf-65b5-40e1-80c8-e98f7c4bb54a * feat(ingestion): add block-scoped variable extraction via tree-sitter queries Add @definition.const and @definition.variable tree-sitter query patterns for TypeScript, JavaScript, Python, Go, Java, C, C++, C#, PHP, Ruby, and Dart. Add parse-worker dedup logic to avoid duplicate nodes when variable captures overlap with existing function/property captures. Add 'Variable' label support in getLabelFromCaptures and DEFINITION_CAPTURE_KEYS. Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * test: add block-scoped variable extraction tests and query capture tests Add 6 tests for block-scoped variable extraction (TypeScript, Go, Rust, C, Python). Add 14 tests verifying @definition.const/@definition.variable query patterns exist in all language query strings. Import RUBY_QUERIES in test file. Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * test: add Python non-assignment expression statement rejection test Addresses code review feedback: verify that the Python variable extractor returns null for expression_statement nodes that contain function calls rather than assignments (e.g. `print("hello")`). Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9fa828c1-87b7-4482-8f26-d2079fb4c58a Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: Dart query node type, add Variable schema, update schema counts - Change `top_level_variable_declaration` → `declaration` in DART_QUERIES (the former doesn't exist in tree-sitter-dart grammar, causing all Dart integration tests to fail with TSQueryErrorNodeType) - Add VARIABLE_SCHEMA to schema.ts and register in initLbug() so that Variable-labeled nodes are persisted to LadybugDB (not silently dropped) - Add 'Variable' to MULTI_LANG_TYPES in csv-generator.ts - Update Dart variable config to remove invalid node type - Update schema test counts (30→31 node schemas, 32→33 total) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/f79931d1-207f-4fbb-91da-259d44f7fd88 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address code review comment improvements - Clarify processedDefinitionNodes tracks start indices, not nodes - Improve Python variableNodeTypes comment wording Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/f79931d1-207f-4fbb-91da-259d44f7fd88 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: add Variable to NODE_TABLES, RELATION_SCHEMA, update golden snapshot - Add 'Variable' to NODE_TABLES in gitnexus-shared so validTables.has('Variable') returns true and Variable graph edges are not silently dropped - Add FROM File TO Variable, FROM Variable TO Community, FROM Variable TO Process to RELATION_SCHEMA so KuzuDB can represent edges connecting Variable nodes - Update schema.test.ts: add Variable to multiLang list, fix count 30→31 - Regenerate pipeline-graph-golden snapshot for mini-repo fixture Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e3aad558-e7bb-40d1-b53f-0a2c0132ca96 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: isolate golden test from cli-e2e fixture pollution The pipeline-graph-golden test was non-deterministic because cli-e2e.test.ts creates AGENTS.md, CLAUDE.md, .claude/skills/, and .gitignore in the shared mini-repo fixture during analyze. These leftover files caused the golden test to find 9 files instead of 7 when tests ran in parallel. Fixes: - Golden test now copies the fixture to a temp dir before running, making it immune to concurrent test pollution - cli-e2e afterAll cleanup now removes ALL generated files (AGENTS.md, CLAUDE.md, .claude/, .gitignore) not just .git/ and .gitnexus/ - Golden snapshot regenerated from clean 7-file fixture Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bd378e73-6f37-49c6-aed6-7fabf4dc6183 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Copilotandmagyargergo
|
03821faf58 |
feat(ingestion): language-agnostic call extractor with config+factory pattern (#877)
* Initial plan * feat(ingestion): add call-types, call-extractors factory, per-language configs, and wire into providers Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/893afa77-5b34-4e6b-a1dc-03034261fb36 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * feat(ingestion): replace inline call extraction in parse-worker and call-processor, delete call-sites/ Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/893afa77-5b34-4e6b-a1dc-03034261fb36 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * test(ingestion): add unit tests for call extraction configs and factory Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/893afa77-5b34-4e6b-a1dc-03034261fb36 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * style: fix prettier formatting in call-extractor files Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/d4b06f56-03b6-4fa4-801f-7ddcc6e81f13 * fix: address review comments — doc comment, idempotency note, C# behavioral test Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e53e650b-fae6-4551-ab25-cda28e4d647f * fix: rename misleading test title, remove stale code reference in comment Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e53e650b-fae6-4551-ab25-cda28e4d647f --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Copilotandmagyargergo
|
06f18ada15 |
refactor(ingestion): move class extraction configs to configs/ subdirectory (#879)
* Initial plan * refactor(ingestion): move class extraction configs to configs/ subdirectory Extract inline ClassExtractionConfig objects from 13 language provider files into 11 config files under class-extractors/configs/, matching the pattern established by method-extractors/configs/ and field-extractors/configs/. Pure structural refactor — zero behavioral change. All class extraction tests pass unchanged. Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8ca2bf18-46ea-41c3-9c7f-9eb3f5752ade Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: resolve prettier formatting in c-cpp.ts import Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1b1e3c43-fc0e-444c-b109-cc0c56ffb470 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Gergő Magyar
|
fec06b823c |
fix: devendor tree-sitter-proto install lifecycle to prevent ENOTEMPTY on global upgrade (#846)
* fix: devendor tree-sitter-proto install lifecycle to fix ENOTEMPTY on global upgrade PR #843's preinstall cleanup hook cannot address the reported bug because it runs on the NEW package's staging tree, not the OLD install being removed. Issue #836 still reproduces on 1.6.2-rc.8. Root cause: vendor/tree-sitter-proto was declared as `file:` dep with its own `dependencies` and `install` script, so npm created `vendor/tree-sitter-proto/node_modules/node-addon-api/` at install time, which blocked npm's rmdir on global upgrade. Changes: - Strip `dependencies` and `install` script from the vendored sub-package's package.json so npm no longer creates a nested node_modules or runs a lifecycle script under vendor/. - Hoist `node-addon-api` and `node-gyp-build` into gitnexus optionalDependencies; npm resolves them at the consumer's top level. - Add scripts/build-tree-sitter-proto.cjs modeled on patch-tree-sitter-swift.cjs. Runs at gitnexus postinstall, best-effort: skips cleanly on missing toolchain or --ignore-scripts so non-proto functionality keeps working. - Remove scripts/preinstall-cleanup.cjs — dead code; cannot run against the old install being removed. - Keep .npmignore entries from PR #843 (tarball hygiene, still correct). - Add explicit .gitignore rules for gitnexus/vendor/**/build and gitnexus/vendor/**/node_modules (closes the repo-side hygiene gap). - Add .github/workflows/ci-global-upgrade.yml: matrix smoke test that installs the previously-published rc globally, upgrades to the packed current branch, and verifies no vendor install-time artifacts survive. Runs on macOS (reporter's platform), Linux, and Windows. Also includes an --ignore-scripts degraded-mode lane. Wired into ci.yml gate. Plan: docs/plans/2026-04-15-002-fix-tree-sitter-proto-vendor-deps-plan.md Phase 1 (this commit) addresses the reported `node_modules/node-addon-api` hazard. Phase 2 (follow-up) will migrate to prebuildify + prebuilt .node binaries in the tarball — the 2026 canonical shape for tree-sitter grammars, which eliminates the postinstall compile path entirely. Refs #836 * fix(ci): ci-global-upgrade should be reusable-only and use setup-gitnexus Three issues caught by CI on PR #846: 1. Concurrency linter rejected the `CIGU-` prefix (allowlist is `${{ github.workflow }}` or substring `CI-`). The literal-prefix guidance in ci.yml is specifically about disambiguating when reusable workflows run in nested contexts, and ci-global-upgrade doesn't need its own concurrency block at all — the caller (ci.yml) already governs concurrency for nested invocations. 2. `npm install` in gitnexus/ runs `prepare: node scripts/build.js`, which depends on gitnexus-shared/dist being built first. Other CI jobs handle this via the setup-gitnexus composite action. Use it here too (with build: 'false' — we only need the dep graph, then npm pack runs prepack which builds gitnexus itself). 3. Removed `pull_request` and `workflow_dispatch` triggers. The workflow is now pure `workflow_call` — invoked once from ci.yml via `uses:`. This avoids the duplicate-run problem where both the top-level pull_request trigger AND the nested workflow_call would fire on every PR. * fix(ci): relax vendor build/ guard and use bash shell on Windows Two fixes for ci-global-upgrade failures on PR #846: 1. The guard after the upgrade step was rejecting vendor/tree-sitter-proto/build/ in the global install. That was too strict. The original #836 bug was about vendor/tree-sitter-proto/node_modules/ specifically, not build/. The build/ directory appears because node-gyp-build compiles through the symlink npm creates at node_modules/gitnexus/node_modules/tree-sitter-proto, and its contents are plain .node, .obj, .lib files that rmdir handles without trouble. We know this empirically because the test got past the upgrade step in the run where the old vendor/node_modules was present. The guard now only flags nested node_modules, which is what the fix actually removes. 2. The Windows --ignore-scripts lane failed with ENOENT when npm tried to open the tarball. The path was computed in a bash step using $(pwd), which on Windows returns /d/a/... form, but npm install ran in the default cmd shell and received a mangled Windows path. Adding shell: bash to the install steps keeps path handling consistent. |
||
|
enihandSisyphus
|
eb0d9c51a0 |
fix: set env.cacheDir to user-writable location (#845)
When @huggingface/transformers is installed globally (e.g. via npm install -g), it defaults its cache directory to ./node_modules/.cache inside its own install dir, which is unwritable by non-root users. This causes EACCES errors on first use when the model is downloaded: EACCES: permission denied, mkdir '/usr/lib/node_modules/gitnexus/node_modules/@huggingface/transformers/.cache' Set env.cacheDir before pipeline() is called in both embedders (CLI and MCP). Respects HF_HOME env var if set, falls back to ~/.cache/huggingface. Co-authored-by: Sisyphus <sisyphus@gitnexus.dev> |
||
|
Copilotandmagyargergo
|
7a5ab57bd3 |
fix: add preinstall cleanup to prevent ENOTEMPTY on global upgrade (#843)
* Initial plan * fix: add preinstall cleanup for vendor/tree-sitter-proto to prevent ENOTEMPTY on upgrade When upgrading gitnexus globally, npm may fail with ENOTEMPTY because it cannot cleanly remove node_modules/ and build/ directories that a previous installation's file: dependency resolution created inside vendor/tree-sitter-proto/. Add a preinstall script that removes those leftover directories before npm resolves dependencies. Also add .npmignore entries for vendor build artifacts as a belt-and-suspenders measure. Fixes #836 Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8b7c1fdd-0c20-4cf4-a64a-9e9d1c0b20ed Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: log warnings in preinstall cleanup catch block Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8b7c1fdd-0c20-4cf4-a64a-9e9d1c0b20ed Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Copilotandmagyargergo
|
1df79c2eab |
fix: content-hash staleness detection for embeddings and vector index creation on zero-node path (#831)
* Initial plan * fix: stale vectors preserved on content edits and vector index missing after zero-node run Issue 1: Add contentHash to EMBEDDING_SCHEMA and embedding pipeline. - contentHash column persisted per CodeEmbedding row - POST /api/embed queries nodeId+contentHash, compares per-node hash - Stale rows (hash mismatch) are DELETE'd before re-embedding - Legacy DBs without contentHash treated as stale (full re-embed) - loadCachedEmbeddings and run-analyze cache restore include contentHash Issue 2: createVectorIndex called unconditionally before zero-node early return. Regression tests: - contentHashForNode determinism and content-change detection - EMBEDDING_SCHEMA includes contentHash STRING column - Pipeline exports verified Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1581c0c0-f359-4376-b47e-62d24a28fd2d Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: use parameterized query for stale embedding DELETE, revert package-lock.json Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/1581c0c0-f359-4376-b47e-62d24a28fd2d Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address review feedback — config consistency, narrow catches, extract DB logic Bug #1: Use finalConfig consistently in contentHashForNode (line 224 was using raw `config` while line 307 used `finalConfig`). Cache precomputed hashes in filter phase to avoid double computation (Perf #5). Bug #2: Narrow catch in loadCachedEmbeddings to only fall back on column/table-missing errors. Rethrow transient/connection errors. Bug #3: Log non-trivial DELETE failures instead of silently swallowing. Arch Violation #3: Extract fetchExistingEmbeddingHashes from api.ts into lbug-adapter.ts. Server layer now calls a single adapter function instead of re-implementing the DB query logic with nested try-catch. Tests: Add config consistency test, note that fetchExistingEmbeddingHashes tests require native module (run in CI). Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b8c4f6b0-4095-4507-a15d-d8469793efac Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: narrow Column error match to 'contentHash' in lbug-adapter fallback checks Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b8c4f6b0-4095-4507-a15d-d8469793efac Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address production-readiness review — eliminate competing state, use schema constants, hard-fail on stale DELETE, add incremental filter tests Gap A / Arch Violation 1: Remove duplicate vectorExtensionLoaded flag from embedding-pipeline.ts — delegate to lbug-adapter's loadVectorExtension() which owns the VECTOR extension lifecycle and resets on DB reconnect. Arch Violation 2: Replace all hardcoded 'CodeEmbedding' and 'code_embedding_idx' strings in embedding-pipeline.ts and run-analyze.ts with EMBEDDING_TABLE_NAME, EMBEDDING_INDEX_NAME, and CREATE_VECTOR_INDEX_QUERY imported from schema.ts. Add EMBEDDING_INDEX_NAME export to schema.ts. Gap B: Make DELETE failure for stale vectors a hard throw (not just a warning). Continuing after failed DELETE risks Kuzu vector-index corruption since the constraint requires DELETE-before-INSERT for vector-indexed properties. "not found" / "does not exist" errors are still safe to ignore. STALE_HASH_SENTINEL: Define a named constant in embedding types.ts for the empty-string sentinel convention. Used consistently in lbug-adapter.ts and run-analyze.ts so the invariant is self-documenting. Tests: Add comprehensive unit tests for the incremental filter logic with mocked embedder: - New node → embedded - Unchanged node (hash matches) → skipped - Stale node (hash mismatch) → DELETE + re-embed - STALE_HASH_SENTINEL → treated as stale - Zero nodes after filter → createVectorIndex still called - DELETE failure with non-trivial error → throws Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b21edee7-c9c5-4742-947b-d0def4fb26aa Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: tighten error classification — extract isMissingColumnOrTableError helper, remove broad pattern matching - Extract isMissingColumnOrTableError() helper in lbug-adapter for consistent schema-error detection (replaces duplicate inline checks) - Tighten 'contentHash' match: now requires 'property' AND 'contentHash' (Kuzu-specific pattern) instead of broad 'contentHash' substring - Tighten DELETE error check: only ignore 'does not exist' (Kuzu's actual message), not broad 'not found' which could mask connection errors - Fix test node ID/name/filePath consistency Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b21edee7-c9c5-4742-947b-d0def4fb26aa Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: CI failures and final review — move STALE_HASH_SENTINEL to schema, tighten error matching, fix test mocking, format - Move STALE_HASH_SENTINEL from embeddings/types.ts to lbug/schema.ts (fixes inverted layer dependency: lbug should not import from embeddings) - Tighten isMissingColumnOrTableError: replace broad msg.includes('not found') with /(table|column|property).*not found/i regex to avoid matching transient errors - Add vi.resetModules() in test beforeEach for explicit module isolation (fixes vi.doMock not intercepting loadVectorExtension in CI) - Skip precomputedHashes.set() on unchanged (return false) path - Run prettier on all 5 files flagged by CI format check Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e20311fd-4361-47b4-a137-9adc3e533b35 * fix: address remaining review nits — rename precomputedHashes, generalize error matcher, revert package-lock - Rename precomputedHashes → computedStaleHashes (hashes are computed on-demand during filter, only cached for stale nodes being re-embedded) - Remove contentHash-specific clause from isMissingColumnOrTableError — the regex /(table|column|property).*not found/i already covers it - Revert package-lock.json ssh→https protocol change Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/e20311fd-4361-47b4-a137-9adc3e533b35 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Copilotandmagyargergo
|
32c9ddaf32 |
fix(deps): pin tree-sitter-c-sharp to 0.23.1 (#834)
* Initial plan * fix: pin tree-sitter-c-sharp to 0.23.1 to resolve peer dependency conflict with tree-sitter@0.21.1 Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/fc416867-7239-4840-9b67-c681d00fa231 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
|
385ee037bd |
[group/sync] Fix ManifestExtractor never called — config.links always produced 0 cross-links (#827)
* fix(group/sync): wire ManifestExtractor into syncGroup pipeline ManifestExtractor was fully implemented in extractors/manifest-extractor.ts but never imported or called in sync.ts. As a result, any links declared in group.yaml were parsed and validated by config-parser.ts but silently dropped — config.links was always an empty dead-end as far as syncGroup was concerned. Changes: - Import ManifestExtractor in sync.ts - Call extractFromManifest(config.links, dbExecutors) inside the outer try block, after all repos are processed but before the finally closes the DB pools (symbol resolution via resolveSymbol requires open executors) - Collect the resulting contracts into autoContracts and the cross-links into a separate manifestCrossLinks array - Merge manifestCrossLinks into the final crossLinks alongside runExactMatch results Without this fix, users who declare explicit service dependencies in group.yaml links (the documented workaround for HTTP clients that use absolute URLs and are invisible to the auto-extractors) get 0 cross-links regardless of what they configure. * test(group/sync): cover manifest links producing cross-links Add a unit test that asserts config.links entries produce contract pairs and a manifest cross-link (matchType: 'manifest') via syncGroup. Also refactors the manifest extraction call to sit outside the else/try block so it runs regardless of extractorOverride arity — makes the code testable without mocked DB pools and ensures links work when callers supply a zero-arity override (e.g. in tests or programmatic usage). * style: prettier format sync.ts and sync.test.ts Also removes the stray empty line in the finally block (noted in review). * fix(group/sync): dedupe cross-links and warn on dangling manifest repos Addresses review feedback on PR #827: 1. Dedupe cross-links. Manifest contracts participate in runExactMatch, so a manifest-declared link also emitted a duplicate matchType:'exact' CrossLink for the same endpoint pair. Dedupe by (from, to, type, contractId) and prefer manifest (operator-declared intent). 2. Warn on dangling repos. When a manifest link references a repo not in config.repos, log a warning. Synthetic UIDs keep the cross-link deterministic, but the operator probably meant something else. 3. Tests: - Assert no duplicate 'exact' CrossLink is emitted alongside the manifest one. - Assert synthetic UID format when no DB executors are available. - New test: dangling manifest repo still produces a cross-link + logs a warning. * perf(group/manifest): parallelize and memoize symbol resolution Previous implementation ran 2N sequential Cypher round-trips per manifest (one for provider side, one for consumer, awaited in-order per link). For manifests with tens of links this dominated syncGroup latency in groups with many declared cross-repo contracts. Changes: - Resolve provider + consumer in parallel per link (Promise.all). - Resolve all links in parallel (outer Promise.all over links.map). Each repo's executor pool is independent, so cross-repo fan-out scales with the number of distinct repos in the manifest. - Memoize by (repo, type, contract). Manifests frequently declare the same contract from both directions or across sibling groups, so duplicate triples now hit the DB once instead of 2× per link. Correctness: - resolveSymbol is a pure LIMIT 1 read, so caching + concurrent invocation is safe. - Iteration order over links is preserved in the final contracts / crossLinks arrays — result shape is identical. Test: - New test asserts that two links sharing (repo, type, contract) produce exactly one DB call per distinct repo-tuple. --------- Co-authored-by: jonasvanderhaegen-xve <> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
28ddbe5d54 |
fix(lbug): wait for read stream close in splitRelCsvByLabelPair (Windows ENOTEMPTY) (#832)
* fix(lbug): wait for read stream close in splitRelCsvByLabelPair (Windows ENOTEMPTY)
The windows-latest CI job intermittently failed:
FAIL test/unit/rel-csv-split.test.ts > splitRelCsvByLabelPair > handles empty CSV (header only) without errors
Error: ENOTEMPTY: directory not empty, rmdir 'C:\Users\RUNNER~1\AppData\Local\Temp\rel-csv-test-XW5KOu'
Cause: splitRelCsvByLabelPair resolved its Promise on readline's 'close'
event, but the underlying fs.ReadStream's file descriptor is released
asynchronously after that — especially on Windows. For the empty-CSV
test the function returns so quickly that afterEach fires rmSync while
the relations.csv fd is still held, so Windows reports ENOTEMPTY on
the directory.
Fixes:
- Production: after readline 'close', wait for inputStream 'close' (or
resolve immediately if already closed/destroyed). Call inputStream
.destroy() defensively so we never hang if the fd never emits 'close'.
- Test: afterEach now retries rmSync up to 5 times on ENOTEMPTY/EBUSY/
EPERM with a brief back-off — defense-in-depth so the test doesn't
flake on slow CI runners independent of the production change.
The production fix benefits every caller, not just the test: any code
that deletes the CSV's parent directory right after the Promise
resolves previously hit the same race on Windows.
* refactor(lbug): replace custom stream state machines with stdlib primitives
Full audit of splitRelCsvByLabelPair's stream usage after the original
ENOTEMPTY fix. Replaced three hand-rolled mechanisms with their
standard-library equivalents — 147 -> 71 lines in the function, and
the caller's WriteStream closure dropped from 13 lines to 5.
- readline: 'on(line)' + pause/resume/waitingForDrain state machine
-> 'for await (const line of rl)'. Async-iterator delivery naturally
serializes line processing with our awaits, so at most one ws is in
backpressure at a time. We just 'await once(ws, "drain")' when
'write()' returns false — the custom Set, the settled flag and the
'only resume when all streams have drained' logic all go away.
- Multi-stream error coordination: hand-rolled cleanup() that had to
be entered exactly once and had to destroy the inputStream and every
pair ws -> single AbortController shared across every 'once(ws,
'drain', { signal })'. Any stream error aborts every pending wait.
- 'stream/promises.finished(inputStream)' in the 'finally' block
replaces the manual 'rl.on('close', () => inputStream.once('close',
...))' dance, and covers both the success and error paths with the
same primitive. This closes the Windows ENOTEMPTY race root cause —
we never return while the fd might still be in flight.
- Caller closure: 'new Promise((res, rej) => ws.end(cb) + remove
listener on error)' -> 'ws.end(); await finished(ws)'.
- Test 'afterEach': custom retry loop -> 'fs.rmSync(..., { maxRetries:
5, retryDelay: 50 })' (Node added these options specifically for
cross-platform tmpdir cleanup).
- Test 'destroys all streams when one errors': old code leaked
backpressure and created multiple pair streams before the first
blocked; new strict serial backpressure doesn't, so the test now
unblocks the first stream once to advance the loop and create the
second stream before triggering the error.
|
||
|
|
c100577e5e |
fix(embeddings): prevent batch errors from CodeEmbedding PK violations and vector-index SET restriction (#823)
* fix(csv-generator): deduplicate all node types, not just File nodes The pipeline can produce duplicate node IDs across all symbol types (Class, Method, Function, etc.). Only File nodes were guarded by a seenFileIds Set, leaving every other type unprotected. When the CSV was COPY'd into LadybugDB, duplicate PKs caused mass "Batch execution error: Found duplicated primary key value" warnings on gitnexus serve. Replace the per-type seenFileIds with a single seenNodeIds Set checked at the top of the iteration loop, before the switch, so every label is covered by the same O(1) deduplication guard. Fixes: #822 * fix(embeddings): use MERGE instead of CREATE for CodeEmbedding inserts CREATE fails with duplicate PK when a CodeEmbedding node already exists, which happens when: - A PostToolUse hook triggers a concurrent gitnexus analyze during an active analyze run (git commits fire the hook) - A partial prior run left some embeddings in the DB before a crash Switching to MERGE makes the insert idempotent: existing embeddings are updated in place, new ones are created, no PK violations. Fixes: #822 * fix(server): skip already-embedded nodes in POST /api/embed to avoid vector-index SET error Kuzu/LadybugDB forbids SET on a property that is part of a vector index. The /api/embed endpoint was calling runEmbeddingPipeline without skipNodeIds, causing it to attempt MERGE+SET on every node including those already embedded. Fix: query existing CodeEmbedding nodeIds before running the pipeline and pass them as skipNodeIds so only new (unembedded) nodes are processed. * fix(server): narrow catch to table-not-exist errors only in POST /api/embed Bare catch{} would silently swallow connection errors and proceed to re-embed all nodes, hiding infrastructure issues. Now only swallows errors where the CodeEmbedding table does not yet exist. * style: prettier format gitnexus/src/server/api.ts * fix(server): log skip-embedding count and table-not-found swallow path Addresses review feedback on PR #823: - Log count of already-embedded nodes when skipNodeIds is populated (aids debugging if Kuzu driver row shape changes). - Log when the 'table does not exist' swallow path fires so ops can catch it if Kuzu ever changes error wording. - Document the {} config positional argument with an inline comment referencing the runEmbeddingPipeline signature. --------- Co-authored-by: jonasvanderhaegen-xve <> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Gergő Magyar
|
baf3f9e37d |
feat(ci): add release-candidate publish pipeline (#825)
* feat(ci): add release-candidate publish pipeline
Auto-publishes gitnexus@rc on every merge to main. Version scheme is
canonical semver X.Y.Z-rc.N where the base is the current npm 'latest'
bumped by the 'bump' input (default patch) and N auto-increments by
querying existing rc versions on the registry. First rc for a new base
is rc.1; the counter resets naturally when the base advances after a
stable release.
- Reuses ci.yml via workflow_call so tests must pass before publish
- SHA-pinned actions, per-job permission scoping, provenance enabled
- Guard job dedupes duplicate dispatches against HEAD via v*-rc.* tags
- Docs-only pushes skipped via paths-ignore
- workflow_dispatch inputs: bump (patch/minor/major), force (override guard)
- Publishes under the 'rc' dist-tag so 'latest' is never moved
- Tags commits as v<rc-version> and creates GitHub prereleases
* fix(ci): address release-candidate review feedback
- Sort rc tags by creatordate (handles out-of-order pushes correctly)
- Fail fast on npm registry errors; only fall back to package.json on E404
- Drop unused pull-requests: write permission on the reused CI job
- Add secrets: inherit so any future CI secrets are available to sub-jobs
- Remove unused reltag step output
* fix(ci): address Copilot review comments
- Correct concurrency comment (runs serialize on same ref, not overlap)
- Apply E404-only fallback to 'npm view versions' query, matching the
pattern used for the 'npm view version' query
- README: clarify that docs-only merges don't trigger rc publish
- CONTRIBUTING: drop 'from main' claim for publish.yml; the tag-push
trigger does not enforce branch reachability
* fix(ci): address adversarial review — idempotency, cycle continuity, tag integrity
Codex adversarial review flagged three release-safety issues in the rc
pipeline. Fixes:
1. Cycle continuity (H). Non-patch rc trains no longer collapse back to
patch on the next push. 'bump' input accepts a new 'auto' value
(default) that infers the active rc base from the registry: if any
X.Y.Z-rc.* exists with X.Y.Z > latest, continue that base; otherwise
patch-bump. Explicit patch/minor/major still forces a cycle reset and
now also bypasses the dedup guard so an explicit dispatch on a
tagged HEAD is honored.
2. Idempotency across post-publish failures (H). The guard marker
('rc/<HEAD_SHA>' lightweight tag) and the release tag ('v<RC>'
annotated) are now pushed atomically *before* 'npm publish'. A
publish failure leaves the marker in place and the guard refuses to
re-publish. Added a defensive 'npm view <pkg>@<rc> version' check
before publish to catch registry-level races. Recovery path
documented in CONTRIBUTING.md.
3. Tag ↔ package integrity (M). 'v<RC>' now points at a detached
release commit whose tree contains the rewritten package.json, so
the tag's source archive matches the npm tarball exactly. 'main'
stays pristine; the release commit is reachable only via the tag.
* fix(ci): surface registry errors on defensive version check; drop actions: read
- npm view <pkg>@<rc> version now distinguishes E404 (safe) from network
failures (abort) via the same mktemp+grep pattern used for the other
two npm view calls
- Dropped actions: read on the ci workflow_call — no sub-workflow uses
the Actions API
|
||
|
Md. Mekayel Anik
|
b340c5d87a |
fix: prevent drain listener leak in relationship CSV streaming (#818)
* fix: add setMaxListeners(50) to relationship pair WriteStreams
Dynamically-created per-pair WriteStreams for relationship CSV splitting
default to Node.js's maxListeners limit of 10. On large repositories with
many relationship types, readline backpressure causes repeated
ws.once('drain', ...) calls that exceed this limit, flooding stderr with
MaxListenersExceededWarning messages.
This matches the existing pattern in csv-generator.ts where
BufferedCSVWriter already calls this.ws.setMaxListeners(50).
* fix: address all 3 stream bugs in relationship CSV splitting
Addresses review feedback from @magyargergo and Claude CI analysis:
Bug 1 (High): Add error handlers to per-pair WriteStreams.
Previously, if a WriteStream errored (disk full, EMFILE) while rl was
paused waiting for drain, the drain callback never fired, rl.resume()
was never called, and the outer Promise hung forever — leaking all
open file descriptors until process kill.
Now each WriteStream gets an error handler that destroys all streams,
closes the readline interface + its input ReadStream, and rejects the
Promise.
Bug 2 (Medium): Add waitingForDrain Set to prevent drain listener
accumulation. rl.pause() is not synchronous — buffered line events
continue firing after pause(), and multiple lines targeting the same
pairKey each added another ws.once('drain', ...) listener. This was the
root cause of MaxListenersExceededWarning.
Now a Set<string> tracks which streams are already waiting for drain.
Only the first backpressure event registers the listener; subsequent
lines for the same stream are silently skipped (they're already written
to the stream buffer). This eliminates listener accumulation entirely
and makes setMaxListeners(50) a safety net rather than a band-aid.
Bug 3 (Low): Close readline and destroy input ReadStream in error
handler. Previously only the WriteStreams were destroyed on error,
leaving the ReadStream FD to linger until GC.
* fix: address review feedback — remove setMaxListeners, harden cleanup
- Remove setMaxListeners(50) entirely. The waitingForDrain guard
guarantees at most 1 drain listener per stream at any time. Tested
with 200 pairs x 500 lines (100k total) — max listeners was always 1,
zero warnings. No hard-coded limit needed.
- Wrap destroy() calls in cleanup() with try/catch so already-destroyed
streams don't throw synchronously (addresses @xkonjin review point 1).
- Add ws.once('error', reject) to the ws.end() phase so flush errors
during stream close properly reject instead of hanging Promise.all
(addresses Claude CI Bug 3b finding).
* test: add 8 regression tests for relationship CSV stream fixes
Covers all bugs fixed in this PR:
- Bug 1: WriteStream error rejects Promise and destroys all streams
- Bug 2: waitingForDrain guard keeps drain listeners at max 1 per stream
- Bug 3: cleanup() handles already-destroyed streams safely
Tests use a MockWriteStream with controllable backpressure and error
injection to verify the exact patterns in loadGraphToLbug() without
needing a real LadybugDB instance.
* style: run prettier on changed files
* fix(test): use backpressure to keep promise pending during error tests
The error tests were racing — readline finished reading the tiny CSV
and resolved the Promise before setTimeout fired the error. Now the
mock streams use blocked=true to trigger backpressure, keeping the
Promise pending so the error fires while the split is still in progress.
* fix: use named error handler in ws.end() to prevent listener leak
ws.once() wraps the callback, so removeListener with the original
function reference won't match. Switch to ws.on() with a named
onError function so removeListener correctly detaches it after
successful close.
* refactor: extract splitRelCsvByLabelPair, fix multi-stream drain
1. Extract splitRelCsvByLabelPair as an exported function with optional
wsFactory parameter for dependency injection. loadGraphToLbug now
delegates to it. Tests import and call the real function instead of
a local reimplementation.
2. Fix multi-stream drain coordination: rl.resume() is now guarded by
waitingForDrain.size === 0, so readline only resumes when ALL
backpressured streams have drained. Previously, any single stream
draining would resume readline while other streams were still full,
allowing unbounded buffer growth.
3. Export WriteStreamFactory type and RelCsvSplitResult interface for
test consumption.
|
||
|
Filipe Oliveira (Redis)andGergo Magyar
|
9ad1984b17 |
fix: resolve C/C++ cross-file calls through transitive #include chains (#816)
* fix: resolve C/C++ cross-file calls through transitive #include chains In C/C++, #include is transitive: if a.c includes b.h and b.h includes c.h, then a.c can call any function declared in c.h. The wildcard import synthesis only walked direct imports (1 hop), missing symbols reachable through transitive header chains. This is the dominant pattern in large C codebases — Redis's db.c includes server.h which includes dict.h, so db.c should resolve calls to dictFind() declared in dict.h and defined in dict.c. Before this fix, those cross-file call edges were missing entirely. The fix expands the import closure transitively for C/C++ files before synthesizing wildcard bindings. A BFS walks ctx.importMap and graphImports to collect all transitively reachable headers, then passes the full closure to synthesizeForFile. Tested on Redis (github.com/redis/redis): - Before: dictFetchValue had 0 cross-file callers, processCommand had 0 - After: dictFetchValue has 9 callers, processCommand has 1, +1946 edges total Fixes #813 * refactor(ingestion): dispatch wildcard synthesis by import-semantics strategy Generalize PR #816's C/C++ transitive #include fix into a language-agnostic strategy pattern. The `wildcard-synthesis.ts` pipeline phase no longer references `SupportedLanguages.C` / `SupportedLanguages.CPlusPlus` — it dispatches on `provider.importSemantics` via an exhaustive `switch`. Also fixes a correctness bug the original BFS introduced: `queue.pop()` (LIFO/DFS) reversed the iteration order of `#include` directives, which — combined with first-seen-wins dedup in `synthesizeForFile` — silently bound overloaded symbols to the wrong header. For the `cpp-calls` fixture, `write_audit("hello")` was being resolved to `zero.h`'s arity-0 overload instead of `one.h`'s arity-1 overload, breaking arity narrowing. Switched to FIFO (`queue.shift()`) with direct imports seeded in declaration order. Taxonomy (researched across 20+ languages + stack-graphs / SCIP prior art): | Tag | Traversal | Languages | |---------------------|-----------------|------------------------------------| | named | none | TS, JS, Java, C#, Rust, PHP, Kotlin| | wildcard-transitive | BFS closure | C, C++ | | wildcard-leaf | single hop | Go, Ruby, Swift, Dart | | namespace | none at import | Python | | explicit-reexport | topological DAG | (scaffold; TS `export *` future) | Changes: - Widen `ImportSemantics` union from 3 to 5 tags with full taxonomy JSDoc - Retag 5 providers: c-cpp (x2) → wildcard-transitive; dart, go, ruby, swift → wildcard-leaf - Move BFS closure into `wildcard-synthesis.ts` as `expandTransitiveIncludeClosure` (pipeline-owned; providers stay pure declarations) - Replace `if (lang === C || CPP)` with `dispatchSynthesis` helper called by both Loop 1 (ctx.importMap) and Loop 2 (graphImports) so a future transitive language whose edges arrive via graphImports gets closure expansion consistently - `never`-assertion default arm forces compile-time exhaustiveness - `explicit-reexport` arm falls through to leaf behavior (scaffold; TODO: implement re-export DAG walk for TS `export *` / Rust `pub use`) - New unit tests covering circular includes, deep chains, diamond dedup, graphImports-only paths, and order-preservation (the regression fix) Verification: - All existing C/C++ transitive tests pass unchanged - Previously failing `cpp.test.ts > resolves run → write_audit to one.h via arity narrowing` now passes - `tsc --noEmit` clean - 225/225 tests pass across wildcard-synthesis, cross-file-binding, cpp resolver, and new closure unit tests * fix(ingestion): bound closure size, O(1) dequeue, track Strategy 4 (#816 review) Address @xkonjin's review feedback on the import-resolution strategy refactor: 1. **DoS guard**: cap transitive closures at 5,000 files via `MAX_TRANSITIVE_CLOSURE_SIZE`. Pathological codebases (boost-style headers, monoheader kernels) could previously produce closures with tens of thousands of entries per translation unit. BFS now stops early and returns a partial closure rather than risking OOM. The closest-headers-first BFS ordering means the partial closure still contains the files overload resolution cares about. 2. **Perf**: replace `Array.prototype.shift()` (O(n)) with a head-index queue (O(1) dequeue). Deep chains previously had quadratic BFS behavior; now linear in closure size. 3. **Strategy 4 tracking**: change TODO in `dispatchSynthesis` to `TODO(#821)` referencing the filed issue for TS `export *` / Rust `pub use` DAG-walk implementation, and clarify that today's leaf fallthrough preserves correctness for direct imports — only the extra re-export traversal is missing. 4. **Test**: new unit test exercising the 5,000-file cap on a 10k-file synthetic chain, verifying partial-closure invariants (starts from importer side, bounded, deep nodes excluded). Not addressed in this commit (followups): - Review point 3 (graphImports-only deep-chain *integration* fixture): unit tests already exercise the `graphImports` traversal path directly in isolation and combined with `importMap`. A fixture that stresses graphImports-only transitive resolution is valuable but requires understanding when the pipeline populates graphImports distinctly from ctx.importMap — tracking as a followup rather than blocking this PR. --------- Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Copilotandmagyargergo
|
1a597f3cc6 |
Fix npm arborist crash caused by tree-sitter-dart tarball URL format (#820)
* Initial plan * fix: change tree-sitter-dart from tarball URL to git URL to fix npm arborist crash, add error handling and troubleshooting docs Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/382c76c6-89c3-463a-8631-2a5d6510be4c Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * refine error handler patterns and troubleshooting docs for arborist crash Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/909b319b-c367-40aa-8033-32dfb6231d4e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * style: run prettier on changed files Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/50eb6b94-9300-4bf2-9b61-c2d78f637fc6 * fix: use github: shorthand for tree-sitter-dart to avoid SSH in CI Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/2ce3f4b3-4c1e-4c39-b824-c25cfe145529 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * revert: use git+https:// for tree-sitter-dart instead of github: shorthand (fixes arborist crash from PR #811) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b786b68d-6c76-4054-88eb-ad46ea9f5b81 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Gergő Magyar
|
759c983dce |
fix(extractors): resolve 3 silent contract mis-resolution bugs (#793) (#817)
* fix(extractors): resolve 3 silent contract mis-resolution bugs (#793) Addresses Codex adversarial review findings for extractor contract resolution on the new group extractor surface. F1 (manifest-extractor): resolveSymbol passed the full "METHOD::path" contract string through normalizeRoutePath, producing "/GET::/api/orders" which never matches Route.name. Adds parseHttpContract() helper that strips the METHOD:: prefix before path normalization. Contract ID construction (buildContractId) is unchanged. F2 (http-route-extractor): graph-assisted backfill used path-only detections.find(), so multi-verb same-URL files attached the wrong verb/handler to provider rows and inferred the wrong verb on FETCHES consumer edges. Now requires path+method match when method is known, and skips backfill when method is unknown and multiple detections tie on path. F3 (grpc-extractor): resolveProtoConflict seeded bestScore=-1 and only replaced on strict >, so all-zero-score ties silently selected candidates[0]. Now computes all scores, counts ties at the top score, and returns null on ambiguity (caller skips contract emission and warns with service name + candidate paths). All three fixes are test-first; 73 tests pass across the three suites. No schema changes, no new dependencies, contract ID wire format (http::METHOD::path, grpc::pkg.Service/Method, http::*::path) preserved. * fix(extractors): address PR #817 review — ambiguous symbol pick + contract id casing Copilot + Claude review on PR #817 flagged two follow-up bugs on top of the F1/F2/F3 fixes: 1. http-route-extractor: ambiguous multi-verb case left handlerName null but still ran the CONTAINS DB query. pickSymbolUid(syms, null) then silently picked pool[0] — reintroducing handler mis-attribution via a different route than the .find() bug F2 fixed. Now gates symbol enrichment on an ambiguousCandidates flag so the file-basename fallback wins instead. 2. manifest-extractor: buildContractId passed raw user casing through for the explicit-method form, so get::/api/orders and GET::/api/orders produced different contract ids even though parseHttpContract upper-cases during lookup. Now reuses parseHttpContract + normalizeRoutePath to canonicalize both method and path, so logically equivalent manifest inputs share a contract id (and share a manifestSymbolUid fallback). Adds one regression test per bug: lowercase vs uppercase manifest contract ids must match, and ambiguous multi-verb with CONTAINS rows must not silently attach a real handler or call the CONTAINS query at all. 75 tests pass across the three extractor suites. * chore: prettier formatting |
||
|
Gergő Magyar
|
3fbee2d3d2 | chore: release v1.6.1 (#815) | ||
|
|
26ff700e37 |
refactor(pipeline): DAG-based phase architecture + container-logic extraction to LanguageProvider (#809)
* Initial plan * refactor: move language-specific container node logic into LanguageProvider - Add resolveEnclosingOwner hook to LanguageProviderConfig - Add staticOwnerTypes to MethodExtractionConfig - Implement Ruby resolveEnclosingOwner (singleton_class → class/module) - Replace hardcoded STATIC_OWNER_TYPES with config.staticOwnerTypes - Move Ruby static types to rubyMethodConfig - Move Kotlin static types to kotlinMethodConfig - Remove Ruby singleton_class branch from findEnclosingClassInfo - Collapse seqFindEnclosingClassNode/seqFindRawEnclosingContainerNode into single provider-aware seqFindEnclosingOwnerNode - Update worker path to pass provider.resolveEnclosingOwner Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bc9f9d4d-f749-4872-9ff2-17fc86e08787 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * test: add regression tests for config-driven staticOwnerTypes and resolveEnclosingOwner hook Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/bc9f9d4d-f749-4872-9ff2-17fc86e08787 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * refactor: implement DAG-based pipeline architecture with phase extraction Restructure the ingestion pipeline from a ~1800-line monolithic orchestrator into a DAG (Directed Acyclic Graph) of named phases with explicit dependencies. New files under pipeline-phases/: - types.ts: PipelinePhase, PipelineContext, PhaseResult contracts - runner.ts: DAG runner with topological sort validation - scan.ts, structure.ts, markdown.ts, cobol.ts: early phases - parse.ts + parse-impl.ts: chunked parse + resolve (the core) - routes.ts, tools.ts, orm.ts: post-parse enrichment phases - cross-file.ts + cross-file-impl.ts: cross-file binding propagation - mro.ts, communities.ts, processes.ts: graph analysis phases - index.ts: barrel export pipeline.ts reduced from ~1960 lines to ~184 lines: - DAG phase array declaration - runPipelineFromRepo as thin orchestrator - topologicalLevelSort retained for backward compat Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/136bf9c3-2f4f-449b-9fff-001332c8371c * test: add DAG runner unit tests, update ARCHITECTURE.md with phase DAG docs Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/136bf9c3-2f4f-449b-9fff-001332c8371c * fix: address code review - pass resolutionContext through parse output, fix worker URL path Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/136bf9c3-2f4f-449b-9fff-001332c8371c * fix: declare transitive parse dependency explicitly in mro/communities/processes phases Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/136bf9c3-2f4f-449b-9fff-001332c8371c * refactor: improve pipeline-phases clean code and folder structure - Extract synthesizeWildcardImportBindings to wildcard-synthesis.ts - Extract extractORMQueriesInline to orm-extraction.ts - Create shared constants.ts for AST_CACHE_CAP - Fix inline type import in orm.ts (use proper top-level import) - Add comprehensive JSDoc to getPhaseOutput explaining type safety - Move isDev to module level in cross-file.ts (consistency) - Improve module-level documentation across files - Organize barrel exports in index.ts with section comments Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/2bd6d4aa-6271-4009-8dd2-332ea8ec73ab Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * address review feedback: fix circular dep, allFetchCalls mutation, progress bugs, remove DAG naming, extract isDev, fix _item naming, fix O(n²) line calc Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/6cf53c9b-d55d-4c6f-bf3d-7bfb82d512b6 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * improve JSDoc on lineNumberAtOffset binary search Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/6cf53c9b-d55d-4c6f-bf3d-7bfb82d512b6 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * address review: filter deps in runner, move totalFiles to ctx, fix cycle JSDoc, centralize isDev, remove DAG naming Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b388424f-b939-4a94-97de-3855f9465564 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix doc consistency in graph-sort.ts module-level and function-level JSDoc Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b388424f-b939-4a94-97de-3855f9465564 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix(pipeline): wrap phase errors with phase name and emit terminal error progress event Restores phase diagnostics at CLI/MCP boundary. runPipeline now wraps phase.execute() in try/catch and rethrows with 'Phase <name> failed: ...' preserving the original via { cause }. Also emits a terminal { phase: 'error' } progress event so subscribers see the failure before the rejection propagates. Handler errors during error reporting are swallowed to keep the original cause authoritative. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U1) * fix(pipeline): move bindingAccumulator dispose into crossFile try/finally; make single-use crossFile.execute() now wraps its body in try/finally so the accumulator is released on both the happy path and when runCrossFileBindingPropagation throws. Dev-mode telemetry stays inside the try block before dispose (all three counters return 0 after dispose clears internal maps). BindingAccumulator becomes single-use: appendFile after dispose now throws 'BindingAccumulator: use after dispose' instead of silently re-animating via the old _disposed auto-clear. Docs updated; the only production construction site (parse-impl) always creates a fresh instance per run, so no caller relied on the re-use contract. Residual risk documented in crossFile module JSDoc: a future phase inserted between parse and crossFile that throws would still leak the accumulator. Any such phase must manage accumulator lifetime explicitly. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U2) * docs(pipeline): explain why importCtx teardown is safe before crossFile Investigation (plan U3) confirms: `importCtx` (ImportResolutionContext) is a scratch workspace with no downstream consumer after parse. `resolutionContext` (returned to crossFile) is a distinct object that owns importMap / namedImportMap / packageMap / moduleAliasMap / model, and never closes over importCtx. cross-file-impl consumes only that ctx via processCalls. The two confusingly-similar "context" names were the root of the adversarial reviewer's concern — comment locks in the invariant so the next reader sees it. No behavioral change. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U3) * refactor(pipeline): remove ctx.totalFiles side-channel; promote to ParseOutput totalFiles was a hidden mutable field on PipelineContext written by parse and read by mro/communities/processes — five reviewers flagged this as a violation of the immutable-context invariant. Removed from PipelineContext, which is now fully readonly, and made the implicit temporal dep explicit: mro/communities/processes now declare 'parse' as a dep and read totalFiles via getPhaseOutput<ParseOutput>(...). No behavior change. Topo-sort unchanged because parse was already a transitive dep through crossFile. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U4) * feat(method-extractor): runtime staticOwnerTypes guard at factory chokepoint createMethodExtractor now rejects MethodExtractionConfigs that list companion_object / singleton_class / object_declaration in typeDeclarationNodes but omit the matching entry from staticOwnerTypes. Fails loudly at provider construction time instead of producing silent isStatic=false on the 50000th file analyzed. Opt-out convention preserved: an explicit `new Set()` (empty Set) signals intentional exclusion and passes the guard (memory obs #30588). All 13 existing language configs pass the guard; the new negative test fails without it. Test-first. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U5) * fix(pipeline): wrap sequential-fallback in try/finally so cleanup survives throws The sequential-fallback block in runChunkedParseAndResolve now runs inside a try/finally that guarantees astCache.clear(), accumulator finalize, and enrichExportedTypeMap execute even if readFileContents or processCalls throws mid-fallback. Cleanup failures are caught inside the finally so they can't mask the original error. Accumulator disposal ownership remains with crossFile (U2) — U6 only adds astCache cleanup and preserves finalize ordering on the error path. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U6) * test(pipeline): direct unit coverage for wildcard-synthesis and cross-file-impl Both modules previously had zero direct unit coverage — branches were exercised only through integration tests' happy paths. wildcard-synthesis.test.ts covers: Go graph-IMPORTS fallback, Python moduleAliasMap build, MAX_SYNTHETIC_BINDINGS_PER_FILE cap, dedup against existing namedImportMap entries, and empty-exportedSymbols early return. cross-file-impl.test.ts covers: gapRatio below threshold no-op, MAX_CROSS_FILE_REPROCESS cap, graph-only exportedTypeMap fallback, and empty namedImportMap short-circuit. Tests assert current behavior — any future regression flips them. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U7) * test(pipeline): golden-file graph-parity regression guard on mini-repo fixture Pins the current post-P1/P2 graph output (57 symbols, 92 relationships, 4 processes, deterministic edge digest) so future silent refactors cannot drift behavior unnoticed. If any count changes or any edge rewires, the test fails with a readable diff listing what changed and a copy-pasteable UPDATE_GOLDEN=1 regen command. Edge digest keyed by symbolic (label, name, filePath) triples rather than raw generateId output — stays meaningful across id-encoding refactors while still catching real semantic rewiring. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U8) * fix(pipeline): minimal cycle reporting + resolveEnclosingOwner loop safeguards U9: runner cycle detection now reports only the SCC members via DFS back-edge trace ('Cycle detected: A -> B -> C -> A') rather than everything with inDegree > 0 (which mixed cycle members with blocked dependents). Also emits the 'error' progress event for graph- validation failures, symmetric with U1's runtime-error path. U16: findEnclosingClassInfo now defends against language-provider hooks that return non-container nodes — visitedContainers Set breaks repeat-visit loops, MAX_ENCLOSING_WALK_ITERATIONS is belt-and-braces. Documented the hook contract invariant so future provider authors know the walk-continues-upward expectation. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U9, U16) * refactor(pipeline): type hygiene, dead code cleanup, shared allPathSet, graph-sort naming Bundles plan units U10, U11, U12, U14, U15: U10 — Type hygiene: readonly ParseOutput arrays (allExtractedRoutes, allDecoratorRoutes, allToolDefs, allORMQueries, allPaths); removed redundant 'as string[] | undefined' cast in routes.ts and 'as URL' in parse-impl.ts; WorkerPool is now 'import type'. Readonly contract propagated into processORMQueries (only iterates). U11 — Dead code & shims: deleted constants.ts shim (AST_CACHE_CAP inlined into its sole real consumer cross-file-impl.ts; isDev consumers now import directly from ../utils/env.js). Removed internal utility re-exports from pipeline-phases/index.ts (no external consumers). Removed topologicalLevelSort re-export from pipeline.ts; updated topological-sort.test.ts to import from the canonical utils/graph-sort.js. Stripped 'Phase 3+4:' stale JSDoc from parse-impl.ts. U12 — Perf: StructureOutput now carries allPathSet (ReadonlySet<string>) built once; cobol, markdown, and cross-file-impl consume the shared set instead of allocating their own. Parse forwards it via ParseOutput.allPathSet; processCobol/processMarkdown widened to ReadonlySet<string>. U14 — graph-sort.ts: renamed local 'inDegree' to 'pendingImportsPerFile' with expanded JSDoc explaining the reverse- graph Kahn's formulation and warning future maintainers not to 'correct' it to standard in-degree semantics. Added self-edge test. U15 — Unconditional worker-fallback logging: removed isDev guard on the worker-pool-creation-failure console.warn so operators can diagnose perf degradations in production. No behavior change. U8 golden-file test confirms pipeline output is byte-identical. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U10, U11, U12, U14, U15) * docs: fix ARCHITECTURE.md table integrity; bump AGENTS.md/CLAUDE.md to 1.3.0 U13 — documentation fixes: ARCHITECTURE.md: the prior insertion of the 'Pipeline Phase DAG' section orphaned 7 rows from the 'Where to change what' header. Moved those 7 rows back up under their header so the table reads contiguously; DAG section now follows the completed table. AGENTS.md + CLAUDE.md: bumped version 1.2.0 -> 1.3.0, updated Last reviewed to 2026-04-13, added matching Changelog row documenting the GitNexus index stats refresh after the DAG refactor. Stat bumps (symbols/relationships/execution flows) that were sitting uncommitted in the working tree are now landed under a proper changelog entry per each file's own documented schema. Plan: docs/plans/2026-04-13-001-fix-pipeline-dag-refactor-review-findings-plan.md (U13) * refactor(pipeline): drop spurious parse deps, true-readonly ParseOutput.exportedTypeMap, skip redundant wildcard synth - mro/communities/processes: switch redundant `parse` dep to `structure` — totalFiles originates in structure, so depending on parse for it was a spurious data dep that obscured the real DAG. - ParseOutput.exportedTypeMap: typed as truly ReadonlyMap<...,ReadonlyMap>>; graph→exports enrichment moved into parse-impl so the snapshot is fully populated at parse return. crossFile builds its own local mutable working copy for per-file re-resolution writes — no cast at the boundary. - parse-impl: hasSynthesized flag guards the unconditional final synthesizeWildcardImportBindings call when per-chunk/fallback synthesis already ran (graph-global + idempotent across chunks). - cross-file-impl: documented the intentional `phase: 'parsing'` progress label so telemetry bucketing stays consistent with the parse phase. - cross-file-impl test: replaced the now-moved fallback-enrichment assertion with a stronger one — crossFile must not mutate the parse-supplied map. Addresses PR #809 review pass 5 carry-overs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com> |
||
|
Copilotandmagyargergo
|
6388113e10 |
fix: prevent stack overflow and memory exhaustion on large repo analysis (#814)
* Initial plan * fix: prevent stack overflow and memory issues on large repo analysis - Convert c3Linearize from recursive to iterative (explicit work stack) to handle deep class hierarchies without stack overflow - Replace push(...arr) spread patterns with safe loops in parse-worker.ts and lbug-adapter.ts to prevent stack overflow on large arrays - Stream relationship CSV lines directly to per-pair temp files in lbug-adapter.ts instead of accumulating millions of lines in memory - Add test for deep 500-level inheritance chain Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9cb2eed2-adc7-4fa4-9216-e7ac3facb9b5 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: add stack size flag and enhanced error messages for large repos - Auto-set --stack-size=4096 alongside --max-old-space-size in analyze command to prevent stack overflow on deep class hierarchies - Add helpful error guidance for known large-repo failure modes (stack overflow, heap OOM, Map size limits) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9cb2eed2-adc7-4fa4-9216-e7ac3facb9b5 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address code review feedback - Add error handling for write stream close in lbug-adapter.ts - Handle backpressure when writing relationship CSV lines to disk - Clarify ENTER/MERGE phase transition comment in resolve.ts - Fix inconsistent stack size in error message (4096 not 8192) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/9cb2eed2-adc7-4fa4-9216-e7ac3facb9b5 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: address PR review — resource leak, redundant guard, Set, test depth - Fix write-stream resource leak on readline error by destroying all open WriteStreams before rejecting (lbug-adapter.ts) - Switch failedPairCsvPaths from array to Set for O(1) lookup - Remove redundant MERGE-phase empty-parents guard in resolve.ts (unreachable — ENTER phase already handles that case) - Increase deep inheritance test DEPTH from 500 to 2000 for reliable regression coverage across platforms Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/cf1f3e22-3864-454a-a3a5-2bded9ebfdba Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * style: fix prettier formatting in lbug-adapter.ts Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b5ca33c4-bb03-402f-a206-21ea7e1e310e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * chore: revert unintended package.json/lock changes Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/b5ca33c4-bb03-402f-a206-21ea7e1e310e Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: strip NODE_OPTIONS in skip-git-cli test child processes Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/fc59cd11-348b-4e22-b9ea-98787300de48 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: don't put --stack-size in NODE_OPTIONS (rejected by Node 24) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/fc59cd11-348b-4e22-b9ea-98787300de48 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> * fix: pass --stack-size as CLI arg only, not in NODE_OPTIONS (Node 24 compat) Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/fc59cd11-348b-4e22-b9ea-98787300de48 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Copilotandmagyargergo
|
c672697012 |
fix: replace tree-sitter-dart git URL with tarball to fix npm install crash (#811)
* Initial plan * fix: replace tree-sitter-dart git URL with tarball URL to fix npm install crash The `github:` git URL for tree-sitter-dart caused npm's arborist to create a dependency node with a null target during the rebuild phase, crashing global installs with: Cannot destructure property 'package' of 'node.target' as it is null. Using a GitHub archive tarball URL instead avoids the arborist bug while still installing from the exact same commit (80e23c0). Fixes #805 Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/c15109ae-0865-4d69-bd08-9972dcfe18f9 Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Deepak Chauhan
|
d786e692af |
[cli] Preserve Ruby singleton_class context in sequential parsing (#774)
* fix(parsing): preserve ruby singleton class context * refactor(parsing): clarify singleton class helpers |
||
|
Arkh74278andarkh
|
a6421b3b1b |
[dart] Add call patterns for await, cascade, lambda, and widget-tree contexts (#801)
* feat(dart): add call patterns for await, cascade, lambda, and widget-tree contexts * fix(dart): address review feedback — await member-chain, cascade comment, static_final comment, add to query-compilation smoke test * test(dart): add integration tests for await and widget-tree call patterns * style: apply prettier formatting to dart integration tests --------- Co-authored-by: arkh <local@localhost> |
||
|
Copilotandmagyargergo
|
9f4109a33f |
fix: remove file:../gitnexus-shared from runtime dependencies (#803)
* Initial plan * fix: remove file:../gitnexus-shared from dependencies to fix npm install outside monorepo Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/2ea9c1da-1b0c-4ab0-b370-f3970cc54ffa Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com> |
||
|
Copilotandmagyargergo
|
79e1d933fa |
fix: resolve generic TypeScript awaited function calls missing from call graph (#804)
* Initial plan
* fix: resolve generic TypeScript function callers missed by impact analysis
When a generic function call is combined with `await` (e.g. `await fn<T>(args)`),
tree-sitter-typescript parses it as a `call_expression` whose `function` field is
an `await_expression` rather than a bare `identifier`. The existing queries only
matched `call_expression { function: identifier }`, so these calls produced no
`@call.name` capture and were silently dropped from the call graph.
Fix: add two new tree-sitter query patterns to `TYPESCRIPT_QUERIES` that handle:
1. `await fn<T>(args)` — awaited generic free call
2. `await obj.fn<T>(args)` — awaited generic member call
Both patterns require the `(type_arguments)` child to be present (which is what
causes tree-sitter to parse the `function` field as an `await_expression`).
Non-generic awaited calls (`await fn(args)`) are unaffected: tree-sitter parses
them as `await_expression { call_expression { identifier } }`, which is still
captured by the existing first pattern.
Also adds a new test fixture `typescript-generic-calls` with two callers of a
generic `verifyToken<T>` function using `await` and three new integration tests.
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/4cf75290-900b-4cea-8a65-2a245ff86970
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* fix: clean up test fixture interface ordering and imports
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/4cf75290-900b-4cea-8a65-2a245ff86970
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* test: add coverage for awaited generic member-call form (await obj.fn<T>())
Address review feedback: the member-call query pattern was untested.
Adds service.ts (TokenService with generic verify<T> method) and guest.ts
(calls await svc.verify<GuestPayload>()) to the typescript-generic-calls
fixture, plus a new integration test asserting the CALLS edge resolves.
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/fcbf8d99-8dbc-40ce-b2a3-60b8d63c095a
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* revert: undo accidental ladybugdb version bump in package files
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/fcbf8d99-8dbc-40ce-b2a3-60b8d63c095a
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
* style: run prettier on changed files
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/8c7d8291-74bb-4a86-ae47-7c79e2cbb57e
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
|
||
|
ivkondandClaude
|
4d4756fe86 |
feat(group): extractor expansion + manifest extractor (2/4 of #606 split) (#796)
* feat(group): extractor expansion + manifest extractor Part 2 of 4 in the split of #606 (ticket: #792). Follows #795 (bridge.lbug storage foundation, already merged), but this PR has no code-level dependency on #795 — it only imports types and the ContractExtractor interface that existed on upstream main before either PR. It could have been reviewed in parallel with #795. ## What changed Expands the 3 existing contract extractors with substantially more language/framework coverage, and adds a new `manifest-extractor` that resolves `group.yaml`-declared cross-links against the per-repo graph via exact-name lookups. ### New file (228 LOC) - `gitnexus/src/core/group/extractors/manifest-extractor.ts` — exact graph lookup for `group.yaml`-declared cross-links. HTTP paths are canonicalized before Route.name matching; gRPC is resolved by service/method name (NO `.proto`-filename fallback); topic and lib use exact-name match. Falls back to a synthetic `manifest::<repo>::<contractId>` uid when the graph has no matching symbol, so cross-impact traversal still has a stable anchor for the contract. ### Modified extractors (+958 LOC prod) - `extractors/grpc-extractor.ts` (+522) — `.proto` parser with comment and string-literal sanitization (braces inside strings no longer truncate service bodies); package/service/method canonical IDs; server/client detection across Go (`grpc.NewServer`, `RegisterXxxServer`, `XxxGrpc.XxxImplBase`), Java (`@GrpcService`, `BlockingStub`), Python (`servicer_to_server`, `XxxStub`), and TypeScript/Node (`@GrpcMethod`, `ClientGrpc`, `loadPackageDefinition`). - `extractors/http-route-extractor.ts` (+174) — Go gin/echo/stdlib `HandleFunc`, NestJS `@Controller`+`@Get`/etc, Python FastAPI decorators, Java Spring `@RequestMapping`/`@GetMapping`, restTemplate / WebClient / OkHttp consumers. - `extractors/topic-extractor.ts` (+98) — sarama `ProducerMessage{}` struct literal detection (replaces a constructor-anchored regex that missed topics inside producer loops), kafka-go Writer/Reader, Python NATS (`await nc.subscribe`/`await nc.publish`), JetStream helpers. ### Modified and new tests (+1264 LOC) - `grpc-extractor.test.ts` (+539) — full coverage of the new proto parser (strings-with-braces regression, comments-with-braces regression), per-language server/client detection - `http-route-extractor.test.ts` (+240) — per-framework route extraction + normalization edge cases - `topic-extractor.test.ts` (+177) — the sarama in-loop regression, JetStream, Python NATS, kafka-go Writer/Reader - `manifest-extractor.test.ts` (+308 NEW) — HTTP path normalization, gRPC exact lookup with proto-fallback regression, lib and topic exact matching, synthetic-uid fallback behavior ### Self-review fixes folded in Carried forward from the #606 self-review (commit `d15b8cb`): - **HIGH #1** — `manifest-extractor.resolveSymbol` was too fuzzy. Previously used `CONTAINS` on route/name fields plus an unconditional `filePath ENDS WITH '.proto'` fallback for gRPC. Consequences: `/orders` matched `/suborders`, and any repo with any `.proto` file returned a random proto symbol for a gRPC manifest entry. Replaced with exact equality + deterministic `ORDER BY` + synthetic-uid fallback for unresolved manifests. Regression tests included. - **MED #3** — gRPC proto parser brace-depth counting now sanitizes strings and comments first (`stripProtoCommentsAndStrings`). A valid proto with `option deprecated_reason = "use NewService { instead"` used to have its service body closed early by the `"{"` inside the literal, silently dropping methods after the offending string. Regression tests for both string-with-brace and comment-with-brace cases. - **MED #4** — sarama Kafka regex changed from `sarama.NewSyncProducer[\s\S]{0,300}?Topic:` (anchored on constructor, caught only first topic in a loop) to `sarama.ProducerMessage{...Topic:}` (matches every struct literal directly). Regression test with a for-loop that constructs multiple `ProducerMessage`s. - **MED #7** — `manifest-extractor.resolveSymbol` no longer has a silent `catch { /* fall through */ }`. Errors from the graph executor are logged via `console.warn` with link type, contract name, repo key, and error message before falling through to the synthetic-uid path. ## Why Reviewer focus here is pure regex / parser correctness — no storage, no Cypher queries, no algorithmic changes to the cross-link algorithm. Separating this from the bridge foundation PR (#795) meant reviewers could stay in a single mental mode (parsing logic) instead of context-switching between DDL, Cypher, and regex. ## How to verify - `cd gitnexus && npx tsc --noEmit` - `cd gitnexus && npx vitest run test/unit/group/grpc-extractor.test.ts --pool=forks` - `cd gitnexus && npx vitest run test/unit/group/http-route-extractor.test.ts --pool=forks` - `cd gitnexus && npx vitest run test/unit/group/topic-extractor.test.ts --pool=forks` - `cd gitnexus && npx vitest run test/unit/group/manifest-extractor.test.ts --pool=forks` Local pre-push: typecheck clean, all 99 extractor unit tests pass (grpc 43, http 18, topic 30, manifest 8). ## Risk / rollback **Low.** Extractors have no user-facing surface in this PR — they produce `ExtractedContract[]` that is consumed by `sync.ts` in the next split (#793). No existing behavior changes for users who don't run a `group sync`. Rollback = `git revert` of the merge commit; the modifications to `grpc-extractor.ts` / `http-route-extractor.ts` / `topic-extractor.ts` revert to the pre-PR versions that still work (they're subsets of the new functionality). ## Scope discipline (per GUARDRAILS.md) - Only the 8 files above are touched; no drive-by refactors - No CI/release/security config changes - No secrets or machine-specific paths - Content lifted from #606 (CI 11/11 green on `d15b8cb`) ## Dependencies - **Base:** `main` (upstream already includes #795 as `1ff324c`) - **Blocks:** sync pipeline (#793) and the cross-impact feature (#794) - **Tracker issue:** #792 - **Parent PR:** #606 Co-authored-by: Claude <noreply@anthropic.com> * refactor(group): migrate topic-extractor from regex to tree-sitter queries Addresses @magyargergo's feedback on #796 that regex-based lookups should use tree-sitter nodes instead, and that the top-level extractors must NOT carry language dependencies. This is phase 1 of a multi-step migration — topic-extractor first because its patterns are the most uniform (16 "call/annotation with first-arg string literal" variants), which makes it a clean proof of the approach before grpc-extractor and http-route-extractor get the same treatment. ## Architecture: language-agnostic orchestrator + per-language plugins The top-level extractor is a thin orchestrator that never imports a tree-sitter grammar or a query string. Per-language knowledge lives in a new `topic-patterns/` folder with one file per language plus a registry that maps file extensions to compiled plugins: ``` src/core/group/extractors/ ├── tree-sitter-scanner.ts # shared, language-agnostic scanning utilities ├── topic-extractor.ts # thin orchestrator (no grammar imports) └── topic-patterns/ ├── types.ts # TopicMeta, Broker ├── index.ts # registry: extension → compiled provider ├── java.ts # tree-sitter-java + JAVA_TOPIC_PROVIDER ├── go.ts # tree-sitter-go + GO_TOPIC_PROVIDER ├── python.ts # tree-sitter-python + PYTHON_TOPIC_PROVIDER └── node.ts # tree-sitter-javascript + tree-sitter-typescript # → JAVASCRIPT_/TYPESCRIPT_/TSX_TOPIC_PROVIDER ``` **Shared scanner (`tree-sitter-scanner.ts`)** — defines `PatternSpec<TMeta>`, `LanguagePatterns<TMeta>`, `CompiledPatterns<TMeta>` and the `scanFile(parser, plugin, content)` helper. Plugins compile their queries eagerly at module load via `compilePatterns()`, so a broken pattern fails loudly at import time instead of silently at scan time. `unquoteLiteral()` handles single/double/template quotes, Python triple-quoted strings, and Go raw backtick strings. **Per-language plugins** own: - the tree-sitter grammar import (this is the ONLY place in `src/core/group/` where tree-sitter grammars are imported), - the query S-expressions, - the `TopicMeta` payload (role, broker, confidence, symbolName) that the orchestrator receives back on every match. Each plugin uses a `@value` capture name to bind the topic literal node. The JavaScript and TypeScript grammars share AST node names for every construct we query, so `node.ts` defines the pattern sources once and compiles them against `JavaScript`, `TypeScript.typescript`, and `TypeScript.tsx` — exporting three providers because `Parser.Query` objects are NOT portable across grammar instances. **Registry (`topic-patterns/index.ts`)** — maps `.java` → Java provider, `.go` → Go, `.py` → Python, `.js`/`.jsx` → JS, `.ts` → TS, `.tsx` → TSX. Also exports `TOPIC_SCAN_GLOB` so adding a new language is a single file-level edit (drop `topic-patterns/<lang>.ts`, import + register it here — zero edits required in `topic-extractor.ts`). **Orchestrator (`topic-extractor.ts`)** — ~110 lines, no grammar or query imports. Per file: `getProviderForFile(rel)` → `scanFile(parser, provider, content)` → `unquoteLiteral(valueText)` → `makeContract(...)`. Reuses one `Parser` instance across files; the scanner calls `setLanguage` per plugin. ## Why this is better than regex 1. **Comments and strings are respected for free.** The old regex would match `// kafkaTemplate.send("fake.topic")` as a real producer; tree-sitter never visits comments or string literals as code nodes, so false positives from commented-out code are eliminated. 2. **Struct/object literal patterns are structural, not textual.** `sarama.ProducerMessage{Topic: "..."}` no longer needs a 300-char lookahead (which was a known cross-match bug partly mitigated by a loop regression test in the self-review). The new query matches a specific `composite_literal` with a specific `qualified_type` and `keyed_element` — exactly one struct literal per match. 3. **No order-of-operations fragility.** Regex for `channel.publish` vs `channel.consume` was independent and file-wide; the AST scopes matches to the specific `call_expression`. 4. **Language-agnostic extension.** Adding Ruby, Rust, or C# topic detection later means dropping one file in `topic-patterns/` — no changes to shared scanner or orchestrator, and no tree-sitter imports leak into top-level code. ## Per-file fault tolerance - Malformed files that tree-sitter can't parse are silently skipped (`parser.parse` is wrapped by `scanFile`). The ingestion pipeline already logs unparseable files at index time. - A syntactically invalid query is caught at `compilePatterns` time, not scan time — broken plugins fail loudly at import. - Per-pattern `matches()` failures are swallowed so one broken query in a plugin doesn't block the rest. ## Tests All 30 existing `topic-extractor.test.ts` tests pass **without any changes to the test file** — they were written as input/output contract tests (given this source file, expect these `ExtractedContract` objects) and that contract is unchanged. Regression coverage includes: - Kafka: Java `@KafkaListener` + `kafkaTemplate.send`; Node `producer.send` + `consumer.subscribe`; Go sarama producer/consumer (sync and async); kafka-go Writer/Reader; Python `KafkaConsumer` + `producer.send/produce` - RabbitMQ: Java `@RabbitListener` + `rabbitTemplate.convertAndSend`; Node `channel.consume/publish/sendToQueue`; Python `basic_consume/ basic_publish` with keyword args - NATS: Go and Node `nc.Subscribe/Publish`; Go and Node JetStream `js.Subscribe/Publish`; Python `await nc.subscribe/publish` Including the regression test for the sarama `ProducerMessage` in-loop case — the AST-based query captures every literal in the file independently, not just the first one after `NewSyncProducer`. ## Neighbor regression check - `topic-extractor.test.ts` — 30/30 pass (rewritten extractor) - `http-route-extractor.test.ts` — 18/18 pass (untouched) - `grpc-extractor.test.ts` — 43/43 pass (untouched) - `manifest-extractor.test.ts` — 8/8 pass (untouched) - Full `npx tsc --noEmit` clean ## Scope discipline (per GUARDRAILS.md) - Only files under `src/core/group/extractors/` are touched; no changes to other extractors, tests, MCP surface, or pipeline.ts. - No CI/release/security config changes, no secrets. - New tree-sitter imports all reference grammars that are already installed as dependencies (`tree-sitter`, `tree-sitter-javascript`, `tree-sitter-typescript`, `tree-sitter-python`, `tree-sitter-java`, `tree-sitter-go` — all in `package.json` for the existing pipeline). ## Phase 2 / phase 3 plan - **Phase 2 (next commit):** rewrite `http-route-extractor.ts` Strategy B (regex fallback) on the same plugin pattern. Graph-assisted Strategy A stays as-is (already uses pipeline-built tree-sitter data via `HANDLES_ROUTE` Cypher queries). - **Phase 3 (commit after):** rewrite `grpc-extractor.ts` for Java / Go / Python / TypeScript detection. `.proto` files are the one outstanding question — there is no `tree-sitter-proto` grammar installed; the in-tree string-sanitizing parser stays as a pragmatic exception with a comment, alternative being to add `tree-sitter-proto` as a dep (open for the maintainer). Co-authored-by: Claude <noreply@anthropic.com> * refactor(group): migrate http-route-extractor Strategy B to tree-sitter plugins Phase 2 of the extractor refactor requested by @magyargergo on #796. Same architecture as the phase 1 topic-extractor rewrite: a thin, language-agnostic orchestrator plus per-language plugins that own tree-sitter grammars and query sources. The top-level extractor file no longer imports any tree-sitter grammar or query string. ## Architecture ``` src/core/group/extractors/ ├── tree-sitter-scanner.ts # shared, language-agnostic primitives ├── http-route-extractor.ts # thin orchestrator (no grammar imports) └── http-patterns/ ├── types.ts # HttpDetection, HttpLanguagePlugin, HttpRole ├── index.ts # registry: ext → plugin + HTTP_SCAN_GLOB ├── java.ts # tree-sitter-java: Spring + RestTemplate/WebClient/OkHttp ├── go.ts # tree-sitter-go: gin/echo/HandleFunc + http/resty consumers ├── python.ts # tree-sitter-python: FastAPI + requests ├── php.ts # tree-sitter-php: Laravel Route::get/... └── node.ts # tree-sitter-javascript + tree-sitter-typescript: # NestJS controllers, Express, fetch, axios ``` **Shared scanner (`tree-sitter-scanner.ts`)** — generalised from phase 1: - `ScanMatch<TMeta>.captures` is now a full `CaptureMap` (every named capture the query binds, not just a single `@value`). Topic extractor updated to read `match.captures.value` accordingly. - New `runCompiledPatterns(plugin, tree)` helper lets plugins run multiple query bundles against the same pre-parsed tree. This is needed for HTTP plugins that combine a class-prefix query with a method-route query (Spring, NestJS). - `scanFile` becomes a thin wrapper over `parser.parse + runCompiledPatterns`. **HTTP plugin shape** — unlike topic plugins, HTTP plugins expose a `scan(tree)` function rather than a flat pattern list. This reflects HTTP's more complex extraction: each detection needs method + path + handler name, and framework patterns like Spring `@RequestMapping` / NestJS `@Controller` require cross-referencing a class-level prefix with method-level annotations. Plugins internally use `compilePatterns` + `runCompiledPatterns` and walk the AST to resolve the class/method relationships. **Per-framework coverage:** - **Java (`java.ts`)** - Spring: `@RequestMapping("/api/v2")` class prefix + `@(Get|Post|Put| Delete|Patch)Mapping("/sub")` method routes, joined via the enclosing `class_declaration` node id. - `RestTemplate.getForObject/postForEntity/put/delete/patchForObject` → method derived from API name. - `WebClient.method(HttpMethod.X, "/path")` → method from `HttpMethod.X` capture. - `new Request.Builder().url("/path")` → OkHttp consumer. - **Go (`go.ts`)** - gin / echo / chi frameworks: `\w+.GET("/path", handler)` captures upper-case verb + handler identifier. - `net/http.HandleFunc("/path", handler)` → provider (default GET). - `http.Get/Post/Head` consumer, `http.NewRequest("METHOD", ...)`, resty `client.R().Get/Post/...`. - **Python (`python.ts`)** - `@app.get("/path")` FastAPI decorators. - `requests.get/post/...` and `requests.request("METHOD", "url")`. - **PHP (`php.ts`)** - Laravel `Route::get/post/.../patch('/path', ...)` via `scoped_call_expression`. Uses `PHP.php_only` to match the existing ingestion pipeline's grammar selection. - **Node (`node.ts`) — JS + TS + TSX** - Pattern sources defined once, compiled against three grammar variants (`JavaScript`, `TypeScript.typescript`, `TypeScript.tsx`) because `Parser.Query` objects are not portable across grammars. Exports three plugins sharing the same `scan` logic. - NestJS: `@Controller('prefix')` decorators are siblings of the class in `export_statement` / `program`; `@Get(':id')` decorators are siblings of the method in `class_body`. The plugin walks decorator → next named sibling to find the decorated class / method, then combines the class prefix with the method path. Only emits NestJS detections when the enclosing class has a real `@Controller` decorator — prevents false positives from generic classes that happen to use `@Get` from another library. - Express: `(router|app).<verb>('/path', ...)`. - `fetch(url)` (default GET) + `fetch(url, { method: 'X' })` (uses two queries + a SyntaxNode-id dedupe set so URL literals aren't double-emitted by the options variant). - `axios.get/post/...`. ## Orchestrator changes `http-route-extractor.ts` drops every `scanXxxProviders` / `scanXxxConsumers` regex method and replaces them with a single source-scan loop that delegates to `getPluginForFile(rel).scan(tree)`. The orchestrator still owns: - **Path normalization** (`normalizeHttpPath`, `normalizeConsumerPath`) — language-agnostic string processing shared by both strategies. - **Graph-assisted Strategy A** (`HANDLES_ROUTE` / `FETCHES` / `CONTAINS` Cypher queries) — unchanged in spirit. The only regex helpers it used (`inferMethodFromFileScan`, `pickJavaHandlerName`) are now replaced by a lookup against the plugin's detections for the same file: for each route row, find the detection whose normalized path matches, and pull the HTTP method + handler name from it. - **Per-file parse cache** — the orchestrator parses each relevant file at most once per `extract()` call. Both the graph-assisted enrichment loop and the source-scan fallback share the same `cachedDetections` map, so we never run the plugin twice for the same file. ## Why this is better than the regex version 1. **Comments and strings for free.** The old regex would match `// router.get('/fake')` as a real Express route; tree-sitter never visits string/comment nodes. 2. **Structural controller-prefix.** Spring and NestJS class-prefix joining is now scoped to the enclosing class via `class_declaration` node ids, eliminating file-wide state that broke when a file had multiple controllers. 3. **Precise NestJS disambiguation.** The plugin only emits a NestJS detection when the enclosing class has a real `@Controller` decorator — the old regex would fire on any `@Get(...)` in the file regardless of surrounding context. 4. **Language-agnostic extension.** Adding Ruby / Rust / Kotlin HTTP detection later means dropping one file in `http-patterns/` — no changes to the shared scanner, the orchestrator, or the Strategy A Cypher queries. ## Tests - `http-route-extractor.test.ts` — **18/18 pass** (tests unchanged; they're contract-style input/output tests and the contract shape is unchanged). Covers Spring class prefix, Express, gin/echo, stdlib HandleFunc, NestJS, Laravel, FastAPI for providers and fetch/axios/python-requests/rest-template/webClient/okhttp/go-stdlib/ resty for consumers, plus graph-first Strategy A for both. - `topic-extractor.test.ts` — **30/30 pass** after the `captures.value` API migration. - `grpc-extractor.test.ts` — 43/43 pass (untouched; phase 3). - `manifest-extractor.test.ts` — 8/8 pass (untouched). - `service.test.ts`, `sync.test.ts`, `storage.test.ts` — 41/41 pass. - `npx tsc -p tsconfig.json --noEmit` clean. ## Scope discipline (per GUARDRAILS.md) - Only files under `src/core/group/extractors/` are touched. - No changes to pipeline.ts, MCP surface, ingestion, or tests. - No CI / release / security / secrets changes. - Tree-sitter grammars imported by plugins (`tree-sitter-java`, `tree-sitter-go`, `tree-sitter-python`, `tree-sitter-php`, `tree-sitter-javascript`, `tree-sitter-typescript`) are all already in `package.json` for the existing ingestion pipeline. ## Phase 3 plan - **grpc-extractor** gets the same treatment: plugin-per-language under `grpc-patterns/` for Java / Go / Python / TS detection. `.proto` files remain an open question — no `tree-sitter-proto` grammar is installed, so the in-tree string-sanitizing parser from PR #796's self-review stays as a pragmatic exception unless the maintainer wants us to add `tree-sitter-proto` as a new dep. Co-authored-by: Claude <noreply@anthropic.com> * refactor(group): migrate grpc-extractor source scans to tree-sitter plugins Phase 3 (final) of the extractor refactor requested by @magyargergo on #796. Same architecture as phase 1 (topic) and phase 2 (http): thin language-agnostic orchestrator + per-language plugins that own tree-sitter grammars and query sources. With this commit the top-level extractors under `src/core/group/extractors/` import ZERO tree-sitter grammars and ZERO query strings — every grammar import lives in a `*-patterns/<lang>.ts` plugin file, and the orchestrators go through the registry indirection. ## Architecture ``` src/core/group/extractors/ ├── tree-sitter-scanner.ts # shared primitives (unchanged) ├── grpc-extractor.ts # orchestrator (only `.proto` parser left) └── grpc-patterns/ ├── types.ts # GrpcDetection, GrpcLanguagePlugin, GrpcRole ├── index.ts # registry: ext → plugin + GRPC_SCAN_GLOB ├── go.ts # tree-sitter-go: RegisterXxxServer, Unimplemented, NewXxxClient ├── java.ts # tree-sitter-java: @GrpcService + XxxImplBase + newBlockingStub ├── python.ts # tree-sitter-python: add_XxxServicer_to_server + XxxStub └── node.ts # tree-sitter-javascript + tree-sitter-typescript: # @GrpcMethod, @GrpcClient field type, # .getService<X>('Svc'), new XxxServiceClient, # loadPackageDefinition dynamic constructors ``` ## Per-language coverage **Go (`go.ts`)** - Provider: `\w+.RegisterXxxServer(...)` via `call_expression → selector_expression → field_identifier` + JS regex filter `^Register(\w+)Server$`. - Provider: `pb.UnimplementedXxxServer` embedded in a struct via `struct_type → field_declaration_list → field_declaration → qualified_type → type_identifier` + JS filter. - Consumer: `\w+.NewXxxClient(...)` via the same call_expression query + JS filter `^New(\w+)Client$`. **Java (`java.ts`)** - Provider: `class X extends YyyGrpc.YyyImplBase` — two queries handle the scoped and plain forms. `scoped_type_identifier`'s children are positional (no `scope:`/`name:` fields), so the query matches the two `type_identifier` children by position. - `#match? @inner "ImplBase$"` restricts matches at query time. - Whether the class has `@GrpcService` or not controls only the `source` metadata label — the plugin walks the class_declaration's `modifiers` child in JS to detect the marker_annotation. - Consumer: `YyyGrpc.newStub(ch)` / `newBlockingStub(ch)` via a `method_invocation` query with `#match? @method "^new(Blocking)?Stub$"`, service name extracted via `^(\w+)Grpc$` on the object identifier. **Python (`python.ts`)** - Single call-expression query covers both bare identifier and `obj.method` attribute forms: `(call function: [(identifier) @fn (attribute attribute: (identifier) @fn)])`. - Plugin filters `@fn.text` against two JS regexes: `^add_(\w+)Servicer_to_server$` (provider) and `^(\w+)Stub$` (consumer), with a reserved-names ignore list for the Stub case (Mock / Test / Fake / Stub). **Node — JavaScript + TypeScript + TSX (`node.ts`)** - Pattern sources defined once, compiled three times (one per grammar) because `Parser.Query` objects are not portable across grammars. Exports three `GrpcLanguagePlugin`s sharing the same `scan`. - `@GrpcMethod('Service', 'Method')`: decorator query captures the two string literals. Confidence is hard-coded 0.8 regardless of proto map resolution (matches the original regex version's behaviour). - `@GrpcClient(...) field: XxxServiceClient`: decorator query captures the decorator node, plugin walks up to find the enclosing `public_field_definition` (decorators on fields are CHILDREN of the field definition in tree-sitter-typescript, not siblings) and reads its first `type_annotation → type_identifier`, then runs the `^(\w+Service)Client$` JS filter. - `client.getService<X>('AuthService')`: call-expression query on `member_expression.property = "getService"` + string literal arg. - `new XxxServiceClient(...)`: `new_expression` with a bare identifier constructor, filtered by `^(\w+Service)Client$` so generic `new AuthClient(...)` (missing the `Service` infix) does NOT falsely register as a consumer. Preserves the regression test `test_extract_ts_non_service_client_constructor_is_ignored`. - `loadPackageDefinition` dynamic loader: gated on `tree.rootNode.text.includes('loadPackageDefinition')`. When set, `new foo.bar.Xxx(...)` qualified constructors with a capitalised property name register as consumers. ## Orchestrator changes `grpc-extractor.ts` loses every `scanGoProviders` / `scanJavaProviders` / ... helper and replaces them with a single source-scan loop that: 1. Parses each file with the plugin's grammar (one shared `Parser` instance across all files, `setLanguage` called per plugin). 2. Calls `plugin.scan(tree)` to get `GrpcDetection[]`. 3. Converts each detection to an `ExtractedContract` via the private `detectionToContract` helper, which: - Looks the short service name up in the proto map (filled by the `.proto` parser). - Picks confidence = `confidenceWithProto` if resolved, else `confidenceWithoutProto`. - Builds a method-level contract id (`grpc::pkg.Svc/Method`) when the detection carries a `methodName` (TS `@GrpcMethod` only), otherwise a service-level id (`grpc::pkg.Svc/*`). Everything else — the `.proto` parser, `buildProtoContext`, `buildProtoMap`, `resolveProtoConflict`, `serviceContractId`, `stripProtoCommentsAndStrings`, `extractServiceBlocks`, the dedupe function — stays exactly as before. The `.proto` parser is kept as a pragmatic exception to the "no regex in extractors" rule because no `tree-sitter-proto` grammar is installed in the repo; a comment at the top of the file explains this and flags the maintainer option of adding `tree-sitter-proto` as a dependency. ## Why this is better than the regex version 1. **Comments and strings are respected for free.** Matched node types are only code constructs, never text inside comments or string literals. 2. **No false positives on partial names.** The old `(\w+?)Grpc`-style regexes would cross-match unrelated identifiers; structural queries restrict matches to the exact AST shape (`scoped_type_identifier → type_identifier` pairs, `method_invocation → identifier` etc.). 3. **NestJS `@GrpcClient` is structural, not regex-based.** The old regex required a specific textual layout (`@GrpcClient(...) private readonly foo!: XxxServiceClient`); the plugin now walks the AST, so modifier order / optional modifiers / multi-line formatting don't break it. 4. **Language-agnostic extension.** Adding Kotlin / Rust / C# gRPC detection later is a one-file edit in `grpc-patterns/index.ts` — no touches to the shared scanner, the orchestrator, or the proto parser. ## Tests - `grpc-extractor.test.ts` — **43/43 pass** (tests unchanged; the contract shape is identical). Covers .proto parsing (including the brace-inside-string regression), Go provider/consumer, Java @GrpcService / plain ImplBase provider + newBlockingStub consumer, Python servicer + stub, TS @GrpcMethod + @GrpcClient + .getService + new XxxServiceClient + loadPackageDefinition + the `AuthClient` vs `AuthServiceClient` discrimination, dedupe across multiple patterns in one file, proto-aware confidence, and the inherited-package resolution for split proto definitions. - `topic-extractor.test.ts` — 30/30 pass. - `http-route-extractor.test.ts` — 18/18 pass. - `manifest-extractor.test.ts` — 8/8 pass. - `service.test.ts`, `sync.test.ts`, `storage.test.ts` — 41/41 pass. - `npx tsc -p tsconfig.json --noEmit` clean. ## Scope discipline (per GUARDRAILS.md) - Only files under `src/core/group/extractors/` are touched. - No pipeline.ts, MCP surface, ingestion, CI / release / security, or test changes. - New tree-sitter grammar imports (`tree-sitter-go`, `tree-sitter-java`, `tree-sitter-python`, `tree-sitter-javascript`, `tree-sitter-typescript`) are all already installed for the ingestion pipeline. ## End of phase series This commit completes the three-phase extractor refactor: - **Phase 1** (`ea06d11`): topic-extractor → `topic-patterns/` - **Phase 2** (`b6015f6`): http-route-extractor → `http-patterns/` - **Phase 3** (this commit): grpc-extractor → `grpc-patterns/` Every remaining regex-based extractor helper under the `src/core/group/ extractors/` directory is either (a) language-agnostic string processing (path normalization, dedupe keys) or (b) the `.proto` parser, which is documented as an explicit exception. Co-authored-by: Claude <noreply@anthropic.com> * feat(group): add tree-sitter-proto for .proto file parsing Addresses @magyargergo's suggestion on #796 to replace the manual string-sanitizing .proto parser with a tree-sitter grammar. - **Vendored `tree-sitter-proto`** in `vendor/tree-sitter-proto/`. Grammar source from [coder3101/tree-sitter-proto](https://github.com/coder3101/tree-sitter-proto) (latest `grammar.js`), parser.c regenerated with `tree-sitter-cli 0.24` to produce ABI version 14 — compatible with the project's `tree-sitter 0.25` runtime (which supports ABI ≤ 14). Added as `optionalDependency` with `file:./vendor/tree-sitter-proto`. - **New `grpc-patterns/proto.ts` plugin** — uses the same `compilePatterns` + `runCompiledPatterns` infrastructure as every other plugin. Two queries: - `(package (full_ident) @pkg)` — package declaration - `(service (service_name) @service_name (rpc (rpc_name) @rpc_name))` — one match per (service, rpc) pair - **Graceful fallback** — `tree-sitter-proto` is an optional dependency. If it fails to install (platform incompatibility) or fails the runtime smoke-test (`setLanguage` + `parse` on a trivial proto), `PROTO_GRPC_PLUGIN` stays `null` and the orchestrator uses the existing manual parser. The smoke-test catches the `SyntaxNode` TDZ error that occurs in vitest's fork-based test runner. - **Orchestrator updated** — when `hasProtoPlugin` is true, `.proto` files are handled by the plugin loop (they're included in `GRPC_SCAN_GLOB`), and the manual `parseProtoFile` loop is skipped. `buildProtoContext` still runs to build the proto map for cross-referencing source-file detections. 1. **No manual comment/string stripping.** The old parser needed `stripProtoCommentsAndStrings` (110 lines) to avoid counting braces inside comments and string literals. tree-sitter handles this natively. 2. **No brace-depth tracking.** `extractServiceBlocks` used a manual depth counter to find service boundaries. tree-sitter's AST gives us `service` → `service_name` + `rpc` → `rpc_name` directly. 3. **Performance.** tree-sitter's C-based parser is faster than character-by-character JS scanning + regex on large proto files. - `grpc-extractor.test.ts` — **43/43 pass** (unchanged) - All other extractor tests — 99/99 pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude <noreply@anthropic.com> * chore: add .gitignore for vendored tree-sitter-proto build artifacts https://claude.ai/code/session_01SFUCxgKMMQ8EgRHYw91xPU * fix: correct .gitignore paths for vendored tree-sitter-proto Patterns should be relative to the .gitignore file's directory. https://claude.ai/code/session_01SFUCxgKMMQ8EgRHYw91xPU * refactor(group): address Copilot review feedback on #796 Six fixes suggested by the Copilot AI review: 1. **`normalizeHttpPath` root-path edge case** — stripping trailing slashes on the input `/` produced an empty string, yielding malformed contract ids like `http::GET::`. Now preserves `/` for the root handler/fetch case. 2. **Dedupe `scanFiles` call** — `extract()` was globbing the source-scan file list twice (once for the provider fallback, once for the consumer fallback). Moved to a single lazy call that memoizes the result for the rest of the method. 3. **HTTP `scanFiles` now ignores `**/vendor/**`** — every other extractor's glob already ignored vendored sources; the HTTP one didn't. Fixed for consistency. 4. **`loadPackageDefinition` check is now structural** — was calling `tree.rootNode.text.includes('loadPackageDefinition')` which forces materialization of the entire file text from the parse tree (expensive on large files). Replaced with a dedicated compiled query on `(call_expression function: [(identifier) | (member_expression)])` so the check stays in the AST domain. 5. **`grpc-extractor.ts` header docstring updated** — still claimed ".proto parsing is not tree-sitter-based because no grammar is installed". Now describes the actual behaviour: tree-sitter when `tree-sitter-proto` is available (optionalDependency), manual fallback otherwise. 6. **Eliminated the double proto file parse on the fallback path** — `buildProtoContext` already globs + parses every `.proto` file to build `servicesByName`. On the `!hasProtoPlugin` branch the extractor was globbing + parsing again via the now-removed `parseProtoFile` helper. The fallback branch now iterates the map that `buildProtoContext` already produced to emit provider contracts directly — single pass per proto file. ## Tests - `topic-extractor.test.ts` — 30/30 pass - `http-route-extractor.test.ts` — 18/18 pass - `grpc-extractor.test.ts` — 43/43 pass - `manifest-extractor.test.ts` — 8/8 pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude <noreply@anthropic.com> * refactor(group): address Claude review feedback (bugs + dedup + hygiene) on #796 Follows up `2f28bfc` with the remaining items from the Claude AI review: ## Bugs **Bug 2 — Label-unaware Cypher queries in `resolveSymbol`.** The manifest-extractor's lookup queries were `MATCH (n) WHERE n.name = $x` with no label filter, so a topic/service/package name could silently match any node type (File, Variable, Import, Folder, …). Added label filters: - `topic` → `(n:Function|Method|Class|Interface)` (topics are best-effort symbol-name matches against listener/publisher symbols) - `grpc` method → `(n:Function|Method)` - `grpc` service → `(n:Class|Interface)` - `lib` → `(n:Package|Module)` All 8 manifest-extractor tests still pass (mock executor is label-agnostic, but the production LadybugDB graph now gets correctly scoped queries). **Bug 8 — Tautological `!handlerName` condition.** `http-route-extractor.ts:extractProvidersGraph` had `let handlerName = null; if (!method || !handlerName) { ... }` — the `!handlerName` clause was always true since there was no intervening assignment. Simplified to always run the plugin-scan lookup (we need the handler name even when `methodFromRouteReason` already resolved the method). ## Clean code / dedup **Design 7 — `readSafe` was copy-pasted in all three orchestrators.** Extracted to `extractors/fs-utils.ts` as the single source of truth for the path-traversal guard. Dropped the three local copies and the now-unused `fs`/`path` imports from topic-extractor. **Style 10 — Language-specific `_test.go` skip in the topic orchestrator.** Was `if (rel.endsWith('_test.go')) continue;` inside the language- agnostic extraction loop. Pushed into the glob's ignore list (`'**/*_test.go'`) alongside the existing `node_modules`, `vendor`, `dist`, `build` entries, with a comment explaining that other languages' test file conventions either live in separate directories (Python `tests/`, Java `src/test/`) or are already covered by the existing ignores. ## Already addressed in `2f28bfc` (mentioned again in Claude review) - Bug 3: `normalizeHttpPath('/')` returns `''` — fixed - Bug 4: double glob + double parse of `.proto` — fixed - Bug 5: `scanFiles` called twice in HTTP — fixed - Bug 6: missing `**/vendor/**` in HTTP glob — fixed - Design 9 partially: `tree.rootNode.text.includes('loadPackageDefinition')` replaced with a dedicated structural query ## Deferred - Bug 1 (`http::*::path` vs `http::GET::path` matching) — out of scope; sync.ts matching logic lands in #793, manifest extractor already emits correct synthetic uids for unresolved HTTP contracts. - Design 9 full (change plugin `scan(tree)` → `scan(tree, source)`) — the only real use case (`loadPackageDefinition` gate) is already fixed via a structural query, so the interface change would be cosmetic churn without a concrete consumer. ## Tests - `topic-extractor.test.ts` — 30/30 pass - `http-route-extractor.test.ts` — 18/18 pass - `grpc-extractor.test.ts` — 43/43 pass - `manifest-extractor.test.ts` — 8/8 pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude <noreply@anthropic.com> * docs+fix(group): address remaining Claude review items + add pipeline flow chart ## Fixes **Remaining 🔴 — HTTP contract id wildcard format.** Documented the `http::*::<path>` format as an intentional wildcard for manifest links that omit the HTTP method, alongside the explicit-method form (`GET::/path` → `http::GET::/path`). The docblock on `buildContractId` now states both forms, notes that wildcard-aware matching is the responsibility of the sync / cross-impact layer (#793), and recommends the explicit-method form whenever the author knows the method (it round-trips through exact equality without needing wildcard logic downstream). Tests unchanged — the wildcard format is what they've always asserted. **Minor 1 — stale comment at `manifest-extractor.ts:124-126`.** The comment claimed "creates a contract with an empty symbolUid/ref" but the code switched to `manifestSymbolUid(repo, contractId)` a few commits back. Updated to describe the actual synthetic-uid fallback semantics and the cross-impact path that relies on both sides of the join deriving the same uid. **Minor 2 — exhaustiveness guard on `buildContractId`.** The `switch(type)` covered all five current `ContractType` variants but silently returned `undefined` if a new variant was added. Added a `default: const _exhaustive: never = type; throw new Error(...)` clause so the build fails loudly on an unhandled variant. **Minor 3 — `tree.rootNode.text` in `grpc-patterns/node.ts`.** Already fixed in `2f28bfc` via a dedicated structural query (`LOAD_PACKAGE_DEFINITION_SPEC`). No action needed. ## New: pipeline flow chart (per @magyargergo's request) Added `src/core/group/PIPELINE.md` with four Mermaid diagrams: 1. **High-level overview** — `group.yaml` → extractors + manifest → contract matching → `bridge.lbug` → `runGroupImpact`. 2. **Per-repo extractor two-strategy shape** — graph-assisted Strategy A vs. source-scan Strategy B. 3. **Plugin architecture** — orchestrator → registry → per-language `*-patterns/<lang>.ts` → `tree-sitter-scanner.ts` → `ExtractedContract`. 4. **Manifest extraction** — label-scoped `resolveSymbol` with the synthetic-uid fallback. 5. **Cross-impact query (#606)** — local impact → bridge join → cross-repo fan-out. Each diagram is annotated with which PRs own which stage (this PR: extractors + manifest; #795: bridge storage; #606: cross-impact runtime) and points at the concrete files/functions involved. ## Tests - 99/99 extractor tests pass - `npx tsc -p tsconfig.json --noEmit` clean Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |