Compare commits

..
Author SHA1 Message Date
Gergo Magyar 9edf06bda2 chore: bump version to 1.4.10, update CHANGELOG 2026-03-27 13:52:59 +00:00
Copilot 01ddc3e756 fix: resolve tree-sitter peer dependency conflicts (#538)
Downgrade tree-sitter from ^0.25.0 to ^0.21.1 and align all parser versions to eliminate ERESOLVE peer dependency conflicts that break MCP server install via npx. Also corrects hallucinated tree-sitter-dart SHA. Fixes #537
2026-03-27 13:49:28 +00:00
Gergo Magyar f373a09ebc chore: bump version to 1.4.9, add CHANGELOG.md 2026-03-26 16:40:02 +00:00
Oleg DeezusandGergo Magyar cde3858e03 refactor: Phase 8 & 9 — Field Types and Return-Type Binding (#494)
* feat(phase8): add field type data structures and extractor interface

* feat(phase8): implement TypeScript field extractor

* feat-phase9-add-call-result-binding

* test-phase8-add-field-extraction-unit-tests

* docs: update documentation for Phase 8 and Phase 9

* feat(swift): Phase 8/9 integration tests for field-type and call-result binding

Add Swift field-type resolution and call-result binding integration tests
with fixtures, plus merge-conflict fixes for the FieldExtractor code.

**Swift integration tests:**
- `swift-field-types/` fixture (Models.swift + App.swift) — tests
  HAS_PROPERTY edges, field-chain CALLS resolution (user.address.save()
  → Address#save), and ACCESSES edges for field reads.
- `swift-call-result-binding/` fixture — tests call-result binding
  (let user = getUser(); user.save() → User#save).
- 2 new describe blocks in swift.test.ts with skipIf(!swiftAvailable).

**Swift arity fix:**
- extractMethodSignature fallback counts direct `parameter` children
  when no wrapper list node exists (Swift's tree-sitter grammar places
  parameters as direct children of function_declaration).  Without this,
  all Swift functions had parameterCount: 0 and the arity filter rejected
  valid call targets.

**FieldExtractor merge-conflict fixes:**
- field-extractor.ts: update import from removed ./utils.js to
  ./utils/ast-helpers.js; use typeEnv.fileScope() instead of .get('').
- field-extractors/typescript.ts: same import fix.
- field-types.ts: alias TypeEnvironment as TypeEnv (renamed on main).
- field-extraction.test.ts: mock TypeEnvironment interface properly.

* feat(field-extractors): generic table-driven field extractors for all 14 languages, wired into pipeline

Implements field extractors for all supported languages and integrates
them into the ingestion pipeline as the single source of truth for
Property node metadata.

**Generic field extractor factory** — `field-extractors/generic.ts`
defines a `createFieldExtractor(config)` factory that generates
FieldExtractor instances from a per-language `FieldExtractionConfig`.
Each config specifies AST node types, name/type/visibility extraction
functions, and static/readonly detection — typically 20-40 lines per
language vs 300+ for a hand-written extractor.

**Per-language configs** — `field-extractors/configs/` has 11 config
files covering 13 languages (TS/JS share, Java/Kotlin share).
TypeScript keeps its hand-written extractor for richer handling.

**LanguageProvider integration** — New optional `fieldExtractor` property
on LanguageProviderConfig, set via defineLanguage() in each language
file. Follows the same strategy pattern as typeConfig, exportChecker,
and labelOverride.  Removed the separate FieldExtractorRegistry class
and field-extractors/index.ts — extractors are accessed via
getProvider(lang).fieldExtractor.

**Pipeline wiring** — Both parse-worker.ts (worker pool) and
parsing-processor.ts (sequential fallback) now call the FieldExtractor
during Property node creation. Results are cached per class node.
Property nodes are enriched with: declaredType, visibility, isStatic,
isReadonly.

**extractPropertyDeclaredType removed** — The 100-line multi-strategy
function in type-extractors/shared.ts is replaced by the FieldExtractor.
All 14 languages register an extractor, eliminating the need for a
generic fallback.  The Python config's extractType was fixed to handle
annotation-without-value patterns (address: Address).

**Integration tests** — Each language's resolver test file gains
pipeline-based assertions verifying visibility/isStatic/isReadonly on
Property nodes via getNodesByLabelFull.  Tests run through
runPipelineFromRepo with real fixtures — no direct extractor calls.

* fix(type-env): thread enclosingFunctionFinder through scope resolution, unskip Dart ACCESSES test

The type-env's findEnclosingScopeKey had the same Dart sibling problem
as findEnclosingFunction — it walked parents but never found
function_signature because the call lives inside function_body (a
sibling).  Instead of hardcoding a function_body check, thread the
provider's enclosingFunctionFinder hook through BuildTypeEnvOptions →
lookupInEnv → findEnclosingScopeKey.  All three buildTypeEnv call sites
(call-processor, parsing-processor, parse-worker) now pass the hook.

This enables the type-env to resolve scoped parameter bindings for Dart
(e.g., `user: User` in processUser), which lets the chain-resolution
tier (Step 1c) walk `user.address` and emit ACCESSES edges.

Dart integration test unskipped — 10/10 passing including ACCESSES.
Reverted CHANGELOG.md to origin/main.

* fix: resolve all PR #494 review findings (10 items)

CRITICAL:
- parse-worker.ts: classNode: any → SyntaxNode on getFieldInfo
  and findEnclosingClassNode; removed redundant as number casts
- parsing-processor.ts: classNode: any → SyntaxNode on seqGetFieldInfo

HIGH:
- ruby.ts: attr_accessor now extracts ALL symbol arguments via
  extractNames hook in generic factory (was firstNamedChild only)
- typescript.ts: added JSDoc explaining why hand-written extractor
  coexists with config-based typescript-javascript.ts

MEDIUM:
- field-types.ts: FieldVisibility union type replaces string
  ('public'|'private'|'protected'|'internal'|'package'|'fileprivate'|'open')
  Propagated through field-extractor.ts, generic.ts, all 7 config files
- typescript.ts: extractFullType collapsed from 12 branches to 3 lines
- generic.ts: added extractNames? optional hook + buildField refactor

LOW:
- ruby.ts: extractVisibility(node) → extractVisibility(_node)
- python.ts: fixed misleading isStatic comment

TypeScript compiles cleanly.

* test: add 24 field extraction tests for generic factory + 5 languages

Generic factory (4 tests):
- createFieldExtractor with TypeScript config validates factory itself
- Body discovery for interfaces, static/readonly modifiers
- Non-type node rejection

Python (4 tests):
- Annotated class field extraction
- Underscore-based visibility: _name=protected, __name=private

Go (5 tests):
- isTypeDeclaration on type_declaration nodes
- Config functions: uppercase=public, lowercase=package visibility
- extractType, isStatic, isReadonly

C++ (5 tests):
- public/private/protected access specifier backward-sibling walk
- Default visibility: class=private, struct=public
- static/const modifier detection

Ruby (6 tests):
- attr_accessor multi-symbol: :name, :email, :age → 3 fields
- attr_reader=readonly, attr_writer=non-readonly
- Multiple attr_* calls in one class

Total: 46 tests passing

* chore: remove plan doc from PR

---------

Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
2026-03-26 16:37:34 +00:00
Gergő Magyar d2cd0b676f feat: add COBOL language support with regex extraction pipeline (#498)
* feat: add COBOL language support with regex extraction pipeline

Standalone COBOL processor following the markdown-processor.ts pattern:
- No LanguageProvider modification — COBOL uses regex, not tree-sitter
- No SupportedLanguages enum change — standalone processor pattern

New files:
- cobol-processor.ts — orchestrator (processCobol, isCobolFile, isJclFile)
- cobol/cobol-preprocessor.ts — regex state machine extraction (~888 LOC)
- cobol/cobol-copy-expander.ts — COPY statement expansion with circular detection
- cobol/jcl-parser.ts — JCL job/step/DD extraction
- cobol/jcl-processor.ts — JCL graph node creation

Extraction produces:
- Module nodes (PROGRAM-ID)
- Function nodes (paragraphs)
- Namespace nodes (sections)
- Property nodes (data items)
- CALLS edges (PERFORM intra-file, CALL cross-program)
- IMPORTS edges (COPY statements)
- CONTAINS edges (section → paragraph hierarchy)

Pipeline integration: single processCobol() call in Phase 2.6

54 new tests (33 COBOL + 21 JCL), all 3889 tests pass.

* docs: document custom processor pattern in pipeline.ts

Add comment block at the custom processor integration point
documenting the pattern for future non-tree-sitter language additions.

* feat(cobol): enrich graph with EXEC SQL/CICS, ENTRY points, MOVE data flow, PERFORM THRU

Maps the remaining 60% of CobolRegexResults to the graph:
- EXEC SQL blocks → CodeElement nodes + ACCESSES edges to DB tables
- EXEC CICS LINK/XCTL → CodeElement nodes + cross-program CALLS edges
- ENTRY points → Constructor nodes (registered for cross-program resolution)
- MOVE statements → ACCESSES edges (read/write data flow tracking)
- PERFORM THRU → expanded CALLS edges for range targets
- File declarations → Record nodes with assignment metadata
- Cross-program CALL 2nd pass: resolves unresolved targets after all programs processed

* test(cobol): add 26 integration tests with exact assertions + fix CICS resolution bug

Integration tests (test/integration/resolvers/cobol.test.ts):
- 26 tests covering full COBOL system extraction
- ALL assertions use exact toBe(N) — zero fuzzy assertions
- Fixtures: CUSTUPDT.cbl, AUDITLOG.cbl, CUSTDAT.cpy, RPTGEN.cbl, RUNJOBS.jcl

Bug fix (cobol-processor.ts):
- CICS LINK/XCTL cross-program resolution was broken — edges were
  created with "resolved" reason but pointing to <unresolved> targets
- Fix: use cics-link-unresolved / cics-xctl-unresolved suffix pattern
  matching the existing cobol-call-unresolved pattern
- Second-pass resolver now patches both CALL and CICS unresolved edges

All 3915 tests pass, 0 failures.

* test(cobol): exhaustive 57-test suite with strict exact assertions

Complete rewrite of COBOL integration tests using ground-truth approach:
dump the full graph, then assert EVERY node and EVERY edge.

57 tests across 9 sections:
- Node completeness: Module(3), Function(13), Namespace(2), Property(21),
  Record(1), CodeElement(8), Constructor(1) — exact sorted arrays
- Edge completeness: 22 tests covering every type+reason combination
  with exact source→target pairs
- Cross-program resolution: 6 tests verifying CALL, CICS LINK/XCTL, JCL
- COPY expansion: copybook data items in RPTGEN
- Section hierarchy: exact paragraph membership per section
- Data item ownership: exact per-module breakdown
- MOVE data flow: exact read/write pairs
- JCL integration: job/step/dataset containment
- Grand totals: CALLS(22), CONTAINS(48), IMPORTS(1), ACCESSES(7)

Fixture enhancements:
- CUSTUPDT.cbl: added INIT-SECTION + PROCESSING-SECTION, PERFORM THRU
- AUDITLOG.cbl: added ENTRY "AUDITLOG-BATCH"
- RPTGEN.cbl: added EXEC CICS XCTL

Zero fuzzy assertions — every expect uses toBe(N) or toEqual([...sorted]).

* fix(cobol): add removeRelationship API + single-quote CALL/COPY/ENTRY, PERFORM keyword skip

Phase 0A: Add removeRelationship(id) to KnowledgeGraph interface and
implementation (trivial Map.delete wrapper). Required for orphan edge
cleanup in next commit.

Phase 1A (from PR #500 review, modified):
- RE_CALL and RE_COPY_QUOTED now match both "double" and 'single' quotes
- parseSingleCopyStatement in copy-expander updated for single quotes
- PERFORM_KEYWORD_SKIP set prevents UNTIL/VARYING/WITH/TEST/FOREVER
  from being stored as false-positive perform targets
- Sequence number stripping uses /[^0-9 ]/ (preserves numeric seq numbers
  unlike PR #500's /\S/ which stripped them)
- Normalized || to ?? for regex group extraction in copy-expander

5 new graph unit tests, all 57 COBOL integration tests pass.

* fix(cobol): RE_ENTRY single-quote + remove orphan unresolved CALLS edges

Phase 1B: RE_ENTRY regex now supports both "double" and 'single' quoted
ENTRY targets. Uses named intermediates (entryName, usingClause) with ??
operator. USING capture group shifted from [2] to [3].

Phase 1C: Second-pass resolution now collects resolved orphan edge IDs
during iteration and removes them after the loop completes, using the new
graph.removeRelationship() API. Graph no longer contains phantom
<unresolved>: edges alongside their resolved replacements. CALLS count
drops from 22 to 18 (4 orphan edges removed).

* fix(cobol): Property ID collisions + O(1) Map lookup for MOVE edges

Phase 1D+3C (atomic): Property node IDs now use composite key
filePath:section:level:name instead of filePath:name. This prevents
duplicate data item names in different sections (e.g., STATUS in both
WORKING-STORAGE and LINKAGE) from silently colliding.

New generatePropertyId() helper ensures both node creation and MOVE
edge lookup use the identical key formula. buildDataItemMap() replaces
the O(n) findDataItemNode linear scan with O(1) Map lookup, built once
per file before MOVE processing.

* feat(cobol): MOVE multi-target extraction with OF/IN qualifier filtering

MOVE X TO A B C now produces write edges for all targets, not just the
first. extractMoveTargets() helper handles OF/IN qualified names
(WS-NAME OF WS-RECORD -> target is WS-NAME), subscript stripping
(WS-TABLE(I) -> WS-TABLE), and MOVE_SKIP filtering on targets.

Data model: CobolRegexResults.moves.to:string -> targets:string[]
MOVE CORRESPONDING stays single-target per COBOL standard.
Processor MOVE loop now iterates move.targets.

* feat(cobol): COPY IN/OF library, pseudotext REPLACING, dynamic CALL, PERFORM TIMES, CICS MAP unquoted

Phase 2B: COPY ... IN/OF library-name now captured as metadata in
CopyResolution (IN and OF are synonyms per COBOL-85 standard).

Phase 2C: COPY REPLACING ==pseudotext== support. Tokenizer handles
==...== delimiters alongside "quoted" strings. Pseudotext forces EXACT
type. Two-pass applyReplacing: first pass handles space-containing/
non-identifier pseudotext via global string replace; second pass handles
identifier-level LEADING/TRAILING/EXACT. New test file
cobol-copy-expander.test.ts with 10 tests.

Phase 2E: PERFORM WS-COUNT TIMES no longer produces a false-positive
perform target (checks for TIMES keyword after captured identifier).

Phase 2F: Dynamic CALL via data item (CALL WS-PROG-NAME without quotes)
now emits a CodeElement annotation node with description 'dynamic-call'
instead of silently ignoring. Adds isQuoted:boolean to call results.

Phase 3A: CICS MAP(WS-MAP-NAME) unquoted identifiers now captured.
Phase 3B: Normalized || to ?? in copy-expander (done in Phase 1A).

* feat(cobol): nested program support — capture multiple PROGRAM-IDs per file

Phase 2D: The state machine now captures all PROGRAM-IDs, not just the
first. The primary program name stays in programName; additional nested
programs go into nestedPrograms[]. The processor creates separate Module
nodes for each nested program, contained by the outer module, and
registers them in moduleNodeIds for cross-program CALL resolution.

Paragraphs/data items are not yet scoped per-program (attributed to the
outer module) — full per-program scoping is a future enhancement that
requires END PROGRAM boundary tracking in the state machine.

* test(cobol): expand integration tests for all new language features

New fixtures:
- NESTED.cbl — two PROGRAM-IDs (OUTER-PROG, INNER-PROG) for nested
  program support testing
- COPYLIB.cpy — copybook for pseudotext REPLACING test target

Modified fixtures:
- CUSTUPDT.cbl — single-quoted ENTRY 'ALTENTRY', multi-target MOVE
  (WS-AMT TO FIELD-A FIELD-B), dynamic CALL WS-PROG-NAME, COPY COPYLIB
  with pseudotext REPLACING, LINKAGE SECTION with LS-PARAM
- RPTGEN.cbl — PERFORM WS-COUNT TIMES (false-positive guard), unquoted
  MAP(WS-MAP-NAME), additional data items WS-COUNT WS-MAP-NAME

Integration test rewritten with 62 exact assertions covering:
- 5 Module, 17 Function, 33 Property, 9 CodeElement, 2 Constructor nodes
- Nested program containment (OUTER-PROG -> INNER-PROG)
- Dynamic CALL annotation (CodeElement with cobol-dynamic-call)
- Multi-target MOVE (UPDATE-BALANCE: 2 reads, 3 writes)
- Single-quoted ENTRY (ALTENTRY under CUSTUPDT)
- PERFORM TIMES guard (WS-COUNT not in CALLS)
- Orphan unresolved edge removal (zero -unresolved edges)
- Grand totals: 21 CALLS, 68 CONTAINS, 2 IMPORTS, 10 ACCESSES

* fix(cobol): pseudotext REPLACING now applies correctly via isPseudotext flag

Root cause: ==PREFIX-== matched /^[A-Z][A-Z0-9-]*$/i (trailing hyphens
allowed), routing it to the second-pass EXACT identifier match where
PREFIX-RECORD !== PREFIX- failed silently.

Fix: Propagate isPseudotext from parseReplacingClause to CopyReplacing
interface, then use it in applyReplacing first-pass condition to force
global string replacement for all pseudotext entries regardless of
whether the content looks like an identifier.

Result: COPY COPYLIB REPLACING ==PREFIX-== BY ==WS-==. now correctly
transforms PREFIX-RECORD → WS-RECORD, PREFIX-CODE → WS-CODE, etc.

* refactor(cobol): per-program scoping via boundary tracking + line-range grouping

State machine changes (minimal, ~30 lines):
- Add RE_END_PROGRAM regex for END PROGRAM program-name. detection
- Replace nestedPrograms[] with programs[] containing startLine/endLine/
  nestingDepth metadata for each PROGRAM-ID in the file
- Reset division/section/paragraph state on new PROGRAM-ID boundary
- EOF finalization flushes remaining stack entries (single-program files)
- Programs sorted by startLine (outer before inner)

Processor changes:
- Uses programs[] with line-range containment to find enclosing parent
  Module for nested programs (replaces hardcoded nestedParent logic)
- programModuleIds Map tracks Module node IDs per program name

Fixture: NESTED.cbl now includes END PROGRAM lines for both programs.

Integration test: PREFIX-* Property nodes now correctly appear as WS-*
after the pseudotext REPLACING fix from the previous commit.

* feat(cobol): free-format COBOL support (>>source free)

Auto-detects >>SOURCE FREE directive in the first 500 chars and switches
to free-format line processing:
- No column-position rules (cols 1-6 are program text, not sequence area)
- Comments use *> prefix instead of col 7 indicator
- No continuation line indicator
- Strip inline *> comments
- Skip >>SOURCE directive lines

preprocessCobolSource() skips col-1-6 stripping for free-format files.

Paragraph/section regexes relaxed from fixed 7-space prefix to flexible
whitespace with case-insensitivity (/^\s*([A-Z][A-Z0-9-]+)\.\s*$/i).
EXCLUDED_PARA_NAMES expanded with COBOL verbs (GOBACK, END-READ, etc.)
to prevent false-positive paragraph detection in free-format.

Also fixes: entry-point-scoring.ts crash when language is 'cobol'
(MERGED_ENTRY_POINT_PATTERNS[language] was undefined → optional chaining).

Benchmark on ACAS 3.01 (268 GnuCOBOL free-format programs, 10MB):
- Before: 407 nodes, 393 edges (near-empty, only file nodes)
- After:  4,297 nodes, 3,612 edges, 542 clusters, 11 flows

* fix(cobol): relax data item regexes for free-format (^\s+ to ^\s*)

RE_FD, RE_DATA_ITEM, RE_ANONYMOUS_REDEFINES, and RE_88_LEVEL all used
^\s+ which requires at least 1 leading space. In free-format mode, lines
are trimmed before processing, so data items like "01 WS-FIELD PIC X."
have no leading whitespace after trimming.

Changed to ^\s* (zero or more spaces) which works for both fixed-format
(indented lines still have spaces) and free-format (trimmed lines).

ACAS benchmark (268 GnuCOBOL programs):
- Before: 4,297 nodes, 3,612 edges (paragraphs only)
- After:  13,832 nodes, 8,615 edges (+ data items, FDs, 88-levels)

* feat(cobol): 100% structural feature coverage — GO TO, SCREEN, SD/RD, SORT, SEARCH, CANCEL, Level 66

New extractions: GO TO (CALLS edges), SCREEN SECTION data items,
SD/RD alongside FD (Record nodes), SORT/MERGE USING/GIVING (ACCESSES),
SEARCH (ACCESSES), CANCEL (CALLS), Level 66 RENAMES (Property),
IS EXTERNAL/IS GLOBAL (Property description enrichment).

ACAS: 13,951 nodes | 13,193 edges | 685 clusters | 150 flows
(+53% edges from new GO TO/SORT/SEARCH/CANCEL extractions)

* feat(cobol): enriched CICS extraction — file I/O, dynamic PROGRAM, queues, HANDLE ABEND

EXEC CICS blocks now extract:
- FILE/DATASET clause: captures VSAM file name (literal or data item ref)
  for READ/WRITE/REWRITE/DELETE/STARTBR/READNEXT/READPREV → ACCESSES edges
- PROGRAM clause: now handles unquoted variable references (dynamic CICS
  program transfer) → CodeElement annotation with cics-dynamic-program reason
- QUEUE clause: captures TS/TD queue names from WRITEQ/READQ → ACCESSES edges
- LABEL clause: captures HANDLE ABEND error handler targets → CALLS edges
- TRANSID: now handles unquoted variable references

CodeElement descriptions enriched with all captured fields (map, program,
transid, file, queue, label).

CardDemo benchmark: +49 nodes, +33 edges from enriched CICS extraction.

* feat(cobol): complete CICS command extraction — all 7 expert recommendations

From COBOL expert agent analysis:
1. ENDBR added to isRead file command list
2. LOAD added to PROGRAM edge commands (alongside LINK/XCTL)
3. Two-word commands expanded: WRITEQ/READQ/DELETEQ TS/TD, HANDLE
   ABEND/AID/CONDITION, START TRANSID
4. Queue reason differentiated: cics-queue-read/-write/-delete
5. RETURN/START TRANSID → CALLS edges to synthetic <transid> target
6. MAP → ACCESSES edges for screen traceability
7. INTO/FROM data fields extracted → ACCESSES edges to data items

Also: dataItemMap built before CICS block processing (was declared after),
CodeElement descriptions enriched with all captured CICS fields.

* test(cobol): strict exhaustive integration tests with exact edgeSet assertions

Every edge reason has exact sorted pair assertions via edgeSet(), not
just counts. Any change to extraction that adds, removes, or reorders
edges will produce a precise, descriptive failure.

Updated RPTGEN.cbl fixture with:
- GO TO EXIT-PARAGRAPH, SORT USING/GIVING, SEARCH table
- EXEC CICS READ FILE INTO, WRITEQ TS QUEUE FROM, SEND MAP FROM
- EXEC CICS HANDLE ABEND LABEL, RETURN TRANSID, XCTL PROGRAM(variable)
- ABEND-HANDLER and EXIT-PARAGRAPH paragraphs

46 tests covering 24 CALLS + 79 CONTAINS + 18 ACCESSES + 2 IMPORTS edges
across 15 distinct edge reason codes, all with exact sorted pair lists.

* fix(cobol): address 5 findings from second Claude review (compiler front-end perspective)

Finding #2: Numeric sequence numbers now stripped (changed /[^0-9 ]/ to
/\S/ in preprocessCobolSource). Lines like "000100 MAIN-PARAGRAPH." now
have cols 1-6 blanked so paragraph regex matches correctly.

Finding #11: JCL in-stream PROC ordering fixed — pre-register all PROCs
into moduleNames before step processing. Steps that EXEC a PROC defined
later in the same file now get CALLS edges.

Finding #A: PROCEDURE DIVISION USING no longer captures calling-convention
keywords (BY, VALUE, REFERENCE, CONTENT, ADDRESS, OF) as parameter names.

Finding #C: SORT/MERGE USING/GIVING now captures ALL file references
(multi-file), not just the first. Changed from single-match to section
extraction with split.

Finding #D: Section headers no longer set currentParagraph, preventing
PERFORM caller misattribution to Namespace instead of Function nodes.

* fix(cobol): address code review findings — ReDoS fix, perf, cleanup

P1 CRITICAL — ReDoS in SORT USING/GIVING:
Replaced nested-quantifier regex with safe indexOf+substring+split
approach. No backtracking possible on crafted input.

P2 — readCopy O(M) linear scan:
Added copybookByPath reverse Map for O(1) path-to-content lookup.

P3 — Dead code removal:
Deleted unused RE_SORT_USING and RE_SORT_GIVING constants.

P3 — EXCLUDED_PARA_NAMES simplification:
Replaced 20 END-* entries with startsWith('END-') prefix check.
Auto-covers future END-* verbs.

P3 — Misplaced JSDoc on removeRelationship:
Fixed comment that described removeNodesByFile instead.
Added missing JSDoc to removeNodesByFile.

Review agents: architecture-strategist, performance-oracle,
security-sentinel, code-simplicity-reviewer.

* refactor: add Cobol to SupportedLanguages with parseStrategy: standalone

New languages/cobol.ts — standalone regex processor provider with no-op
tree-sitter fields. Declares parseStrategy: 'standalone' to distinguish
from tree-sitter-based languages.

Added parseStrategy: 'tree-sitter' | 'standalone' to LanguageProviderConfig
for languages that use their own processor instead of tree-sitter.

Removed all 11 'cobol' as any casts — now uses SupportedLanguages.Cobol.
Added empty Cobol entries to entry-point-scoring and framework-detection.

* fix(cobol): 5 fixes from third Claude review + 3 regression tests

Fixes:
- Line numbers now 1-indexed in fixed-format (was 0-indexed, off-by-one
  in jump-to-definition links)
- Copybook content preprocessed before COPY expansion (sequence numbers
  and patch markers in copybooks no longer survive into expanded source)
- ENTRY USING filters calling-convention keywords (BY, VALUE, REFERENCE,
  CONTENT, ADDRESS, OF) — same fix as PROCEDURE DIVISION USING
- SORT/MERGE trailing period stripped from USING/GIVING file tokens
- Paragraph exclusion uses exact match for SECTION/DIVISION (was substring
  match that excluded valid names like CROSS-SECTION-ANALYSIS)

USING_KEYWORDS moved to module scope for reuse by both PROCEDURE DIVISION
USING and ENTRY USING handlers.

New unit tests:
- ENTRY USING BY VALUE filtering
- Paragraph names containing SECTION not excluded
- Numeric sequence numbers stripped enabling paragraph detection

* fix(cobol): address 6 findings from fourth Claude review + tests

Fourth review findings fixed:
- New #IV: PERFORM TIMES guard uses perfMatch.index instead of
  line.indexOf (prevents wrong match when target appears earlier in line)
- New #V: 88-level condition values now handle single-quoted literals
  ('Y' no longer stored with embedded quotes)
- New #I: CANCEL edges use two-pass resolution like CALL (no longer
  silently dropped when target indexed after source)
- New #3: Multi-line SORT/MERGE accumulation — sortAccum state variable
  accumulates lines until period, then extracts USING/GIVING from full
  statement (95% of production SORT statements span multiple lines)
- New #II: PROCEDURE DIVISION USING on split lines — pendingProcUsing
  flag defers parameter capture to next line if USING not on same line
- New #6 (prior): EXCLUDED_PARA_NAMES exact match for SECTION/DIVISION

Updated fixture: RPTGEN.cbl SORT now uses multi-line format with GIVING
on separate line (period-terminated). New sort-giving integration test.
ACCESSES total: 18 → 19 (new sort-giving edge from multi-line capture).

* fix(cobol): address 4 findings from fifth Claude review

Finding #B (5 reviews old): Section/paragraph node IDs now include
enclosing program name to prevent collision when nested programs share
section/paragraph names. New findOwningProgramName() helper uses
programs[] line ranges to find the innermost enclosing program.

Finding #α: pendingProcUsing now reset in the if(procUsingMatch) branch
(was only set in else branch, could leak across nested programs).

Finding #β: RE_CALL_DYNAMIC uses negative lookbehind (?<![A-Z0-9-]) to
prevent false-positive on compound identifiers like WS-CALL OCCURS.

Finding #γ: sortAccum flushed at EOF (parallel to flushSelect and
pendingFdName EOF cleanup). Prevents silent loss of SORT USING/GIVING
relationships in truncated files.

* fix(cobol): address findings from reviews 5+6 with full test coverage

Review 5 fixes:
- #α: pendingProcUsing reset in if(procUsingMatch) branch
- #β: RE_CALL_DYNAMIC negative lookbehind prevents WS-CALL false positive
- #γ: sortAccum flushed at EOF for truncated files
- #B: Section/paragraph IDs include owning program name

Review 6 fixes:
- #P: sectionNodeIds/paraNodeIds maps use program-scoped keys
  (PROGNAME:NAME). New scopedParaLookup/scopedCallerLookup helpers.
  findContainingSection updated with programs parameter.
- #Q: RETURNING added to USING_KEYWORDS for COBOL 2002+
- #R: RE_PERFORM matches both THRU and THROUGH via alternation

New unit tests (6):
- PERFORM THROUGH captures thruTarget
- PROCEDURE DIVISION USING RETURNING filters keyword
- RE_CALL_DYNAMIC no false-match on WS-CALL compound identifier
- Multi-line SORT captures USING/GIVING from continuation lines
- PROCEDURE DIVISION USING on split line via pendingProcUsing
- Copybook preprocessing strips sequence numbers

* fix(cobol): address findings from seventh Claude review + 3 tests

Review 7 fixes:
- #i: findContainingSection only updates best when lookup succeeds
  (prevents undefined overwriting valid parent section)
- #ii: RE_PROC_SECTION handles segment numbers (SECTION 30.)
- #III: procedureUsing now stored per-program on boundary stack
  entries, propagated to programs[] output. Inner programs no longer
  overwrite outer program's parameters.
- #δ: Dynamic CANCEL (CANCEL variable) now creates CodeElement
  annotation node, matching dynamic CALL behavior. RE_CANCEL_DYNAMIC
  with negative lookbehind. cancels[] gains isQuoted field.
- #Q: RETURNING added to USING_KEYWORDS (already in prev commit)
- #R: PERFORM THROUGH already fixed (THRU|THROUGH alternation)

New unit tests:
- Nested programs carry per-program procedureUsing
- SECTION with segment number detected
- Dynamic CANCEL via data item captured with isQuoted=false

* feat(cobol): link PROCEDURE DIVISION USING to LINKAGE data items + close 4 findings

Finding #10 FIXED: procedureUsing parameters now create ACCESSES edges
with reason 'cobol-procedure-using' from Module to matching LINKAGE
SECTION Property nodes. This exposes the program's parameter contract
in the graph (e.g., AUDITLOG → LS-CUST-ID, AUDITLOG → LS-AMOUNT).

Findings closed by expert agent consensus:
- #6 COPY IN library: WONTFIX — captured metadata, no universal
  library-to-directory mapping exists. Field costs nothing and is useful
  for library queries.
- #14 SQL DELETE: WONTFIX — DB2 requires FROM; existing FROM pattern
  handles it. Bare DELETE would risk false positives.
- #E OCCURS DEPENDING ON: WONTFIX — runtime sizing concern, not
  structural. The static occurs count is sufficient for indexing.

All 39 findings from 7 Claude reviews now resolved or closed.

* fix(cobol): resolve 48 review findings across 9 review cycles

Ninth deep review resolved all remaining COBOL parser gaps identified
by 5 specialist agents (COBOL expert, architecture strategist,
TypeScript reviewer, security sentinel, code simplicity reviewer).

Fixes (P1 — critical):
- SELECT OPTIONAL now correctly skips OPTIONAL keyword (C1)
- RETURNING params excluded from PROCEDURE DIVISION USING list (C7)
- SORT GIVING no longer captures clause keywords as file names (C5)
- Extract flushSort() helper eliminating 40-line duplication (S2)
- Flush unclosed EXEC blocks at EOF matching SORT/SELECT pattern (S3)
- Guard undefined map key in jcl-processor moduleNames (S1)
- Add MAX_TOTAL_EXPANSIONS=500 to prevent exponential COPY breadth (S4)

Fixes (P2 — important):
- Quote-aware stripInlineComment for | and *> in string literals (C2+C3)
- Fixed-format literal continuation now handles quoted strings (C6)
- PROGRAM-ID detected regardless of division state for siblings (C9)

Fixes (P3 — cleanup):
- EXEC SQL INTO restricted to INSERT INTO to avoid FETCH false-pos (C8)
- Copy expander line numbers fixed from 0-based to 1-based (C11)
- Remove dead code: inInStreamProc, fileIsLiteral, expansionDepth (S7-S10)

Also fixes 8th-review findings: nested program CONTAINS attribution,
multi-PERFORM on same line, INPUT/OUTPUT PROCEDURE IS in SORT,
GO TO DEPENDING ON multi-target, MOVE CORR abbreviation, per-program
procedureUsing ACCESSES edges.

Tests: 145 COBOL tests passing (59 integration + 86 unit)
Benchmarks: CardDemo 12,323 nodes/8,893 edges (7.4s)
            ACAS 14,016 nodes/15,452 edges (9.3s, -9% faster)

* docs(cobol): update documentation for ninth review cycle fixes

Update all 4 COBOL documentation files to reflect the 16 fixes
from the ninth review cycle:

- regex-extraction.md: quote-aware comment stripping, SELECT OPTIONAL,
  RETURNING exclusion, SORT_CLAUSE_NOISE filter, flushSort() helper,
  GO TO multi-target, PROGRAM-ID division-independent detection
- copy-expansion.md: MAX_TOTAL_EXPANSIONS=500 breadth guard, 1-based
  line numbers, removed expansionDepth/warnedCircular param
- deep-indexing.md: GO TO DEPENDING ON, INPUT/OUTPUT PROCEDURE IS,
  MOVE CORR edge reasons, INSERT INTO restriction, literal continuation
- performance.md: updated benchmarks (CardDemo 12,323n/8,893e/7.4s,
  ACAS 14,016n/15,452e/9.3s), COPY breadth guard

* fix(cobol): resolve 10th review findings — nested program edge attribution

Fix 6 findings from the 10th review (PR #498 comment #4132201110):

#A+#F: All CALL/CANCEL/CICS/ENTRY/SQL/SEARCH/file-declaration edges
now use owningModuleId() for nested program attribution instead of
the outer program's parentId. Added helper function owningModuleId()
to centralize the pattern.

#B: Added USING and GIVING to SORT_CLAUSE_NOISE set to prevent MERGE
USING + OUTPUT PROCEDURE from capturing clause keywords as file names.

#C: INPUT/OUTPUT PROCEDURE regex now captures optional THRU/THROUGH
range end paragraph, mirroring RE_PERFORM's THRU support.

#D: scopedCallerLookup fallback now uses programModuleIds.get(pgm)
instead of parentId, so PERFORM/MOVE/GOTO in nested programs with
unresolvable paragraphs fall back to the correct inner module.

#E: pendingProcUsing only set when PROCEDURE DIVISION line is NOT
period-terminated, preventing false USING expectation.

Tests: 145 passing | TypeScript clean

* fix(cobol): resolve 10th review findings — nested program edge attribution

Fix 6 findings from the 10th review (PR #498 comment #4132201110):

#A+#F: All CALL/CANCEL/CICS/ENTRY/SQL/SEARCH/file-declaration edges
now use owningModuleId() for nested program attribution instead of
the outer program's parentId. Added helper function owningModuleId()
to centralize the pattern.

#B: Added USING and GIVING to SORT_CLAUSE_NOISE set to prevent MERGE
USING + OUTPUT PROCEDURE from capturing clause keywords as file names.

#C: INPUT/OUTPUT PROCEDURE regex now captures optional THRU/THROUGH
range end paragraph, mirroring RE_PERFORM's THRU support.

#D: scopedCallerLookup fallback now uses programModuleIds.get(pgm)
instead of parentId, so PERFORM/MOVE/GOTO in nested programs with
unresolvable paragraphs fall back to the correct inner module.

#E: pendingProcUsing only set when PROCEDURE DIVISION line is NOT
period-terminated, preventing false USING expectation.

Tests: 145 passing | TypeScript clean

* fix(cobol): resolve 11th review findings — final nested program + multi-CALL gaps

#1: scopedCallerLookup(null) now uses owningModuleId(lineNum) instead
of parentId, fixing PERFORM/MOVE/GOTO before first paragraph in nested
programs.

#2+#3: CALL and CANCEL extraction now uses matchAll (global flag) to
capture multiple occurrences on the same line. Dynamic CALL/CANCEL
checked independently instead of in else branch.

#4: SORT/MERGE ACCESSES edge IDs now use owningModuleId(sort.line)
instead of parentId for nested program correctness.

#5: preprocessCobolSource free-format detection now uses first 10 lines
(consistent with extractCobolSymbolsWithRegex threshold).

#6: EXCLUDED_PARA_NAMES expanded with DISPLAY, ACCEPT, WRITE, READ,
REWRITE, DELETE, OPEN, CLOSE, RETURN, RELEASE, SORT, MERGE to prevent
false-positive paragraph detection on isolated verbs.

Also removed unused GraphNode import from cobol-processor.ts.

Tests: 145 passing | TypeScript clean

* docs(cobol): deepened full language coverage plan with research findings

3 research agents analyzed Phase 1-2 features and graph value ranking.

Key findings: cobol-call-using is #1 edge type (9.2/10); multi-line
accumulation is dominant challenge; DECLARATIVES is lowest-risk Phase 2
item; SET TO TRUE covers 80-90% of SET usage.

* feat(cobol): implement Phase 1 — high-value data flow edges

4 new extraction features that create new ACCESSES and IMPORTS edges:

1.1: EXEC SQL INCLUDE -> IMPORTS edges with reason 'sql-include'
     Handles unquoted (SQLCA), quoted ('DBRMLIB.MEMBER'), and
     underscored (CUST_TBL_DCL) member names.

1.2: CALL USING parameter extraction -> ACCESSES edges
     Extracts parameters from CALL USING clause, filtering BY/REFERENCE/
     CONTENT/VALUE/ADDRESS/OF/LENGTH/OMITTED keywords. Creates
     'cobol-call-using' ACCESSES edges (graph value: 9.2/10).

1.4: OCCURS DEPENDING ON -> ACCESSES edges with reason 'cobol-depends-on'
     Extended OCCURS regex captures DEPENDING ON field with subscript
     stripping. Creates dependency edge from table to controlling field.

1.5: VALUE clause for standard data items
     Extracts VALUE from data item clauses: quoted strings with type
     prefix (X/N/G/B), ALL literals, numerics (incl negative/decimal),
     and figurative constants. Populates Property node values.

Tests: 145 passing (+2 ACCESSES from CALL USING) | TypeScript clean

* feat(cobol): implement Phase 2 — DECLARATIVES, SET, INSPECT, EXEC DLI

4 new extraction features for error handling, data flow, and IMS/DB:

2.1: EXEC DLI (IMS/DB) -> CodeElement + ACCESSES edges
     Accumulates EXEC DLI blocks like EXEC SQL. Parses DLI verbs
     (GU, GN, ISRT, REPL, DLET, CHKP, SCHD, TERM). Extracts
     SEGMENT, PCB, INTO/FROM, PSB. Creates dli-{verb} ACCESSES
     edges to <ims>:segment Record nodes.

2.2: DECLARATIVES / USE AFTER EXCEPTION -> ACCESSES edges
     Tracks inDeclaratives state. Detects USE AFTER STANDARD
     EXCEPTION ON file-name. Creates cobol-error-handler ACCESSES
     edge from handler section to file Record.

2.3: SET statement -> ACCESSES edges
     Detects SET TO TRUE (80-90% of SET usage) and SET index
     TO/UP BY/DOWN BY. Creates cobol-set-condition / cobol-set-index
     write edges + cobol-set-read for identifier values.

2.4: INSPECT -> ACCESSES edges with multi-line accumulator
     Accumulates INSPECT until period (like SORT). Extracts inspected
     field + tally counters. Creates cobol-inspect-read/write/tally
     edges. Form detection: tallying/replacing/converting/combined.

Preprocessor: 1398 -> 1597 LOC (+199). Tests: 145 passing.

* feat(cobol): implement Phase 3 — completeness fixes

6 partial features fixed to first-class support:

3.1: CALL RETURNING -> ACCESSES write edge (cobol-call-returning)
3.2: SELECT OPTIONAL flag preserved in FileDeclaration + Record node
3.3: ALTERNATE RECORD KEY extraction (matchAll for multiple keys)
3.4: COMMON attribute on nested programs (RE_PROGRAM_ID extended)
3.5: IS EXTERNAL / IS GLOBAL as first-class boolean properties
     (removed usage string hack)
3.6: AUTHOR / DATE-WRITTEN mapped to Module node description

Tests: 145 passing | TypeScript clean

* feat(cobol): implement Phase 4 — INITIALIZE + metadata completeness

4.1: INITIALIZE statement -> ACCESSES write edge (cobol-initialize)
4.2: DATE-COMPILED and INSTALLATION paragraphs extracted and mapped
     to Module node description alongside existing AUTHOR/DATE-WRITTEN

All 4 plan phases complete. Coverage: ~95% (up from 71.9%).
Tests: 145 passing | TypeScript clean

* test(cobol): add 24 unit tests for Phase 1-4 features

Coverage for all new extraction features:

Phase 1 (8 tests):
- EXEC SQL INCLUDE (unquoted, quoted, underscored)
- CALL USING (simple, mixed modes, ADDRESS OF, OMITTED)
- CALL RETURNING
- OCCURS DEPENDING ON
- VALUE clause (string, numeric, figurative constant)

Phase 2 (10 tests):
- EXEC DLI GU/ISRT/SCHD (verb, segment, PCB, INTO, FROM, PSB)
- DECLARATIVES USE AFTER EXCEPTION (single + multiple sections)
- SET TO TRUE, SET index UP BY
- INSPECT TALLYING, INSPECT REPLACING

Phase 3-4 (6 tests):
- SELECT OPTIONAL flag
- ALTERNATE RECORD KEY
- PROGRAM-ID IS COMMON
- IS EXTERNAL / IS GLOBAL booleans
- INITIALIZE extraction
- Full programMetadata (AUTHOR, DATE-WRITTEN, DATE-COMPILED, INSTALLATION)

Total: 168 tests passing (145 + 24 - 1 removed duplicate)

* fix(cobol): use /\r?\n/ split for Windows CRLF compatibility

All 4 COBOL source files now split on /\r?\n/ instead of '\n' to
handle CRLF line endings on Windows. Previously, trailing \r in
lines caused RE_GOTO's $ anchor to fail on multi-line GO TO
DEPENDING ON statements, producing only 1 goto edge instead of 4.

Files fixed: cobol-preprocessor.ts (2 sites), cobol-processor.ts,
jcl-parser.ts, cobol-copy-expander.ts

Tests: 168 passing | TypeScript clean

* fix(cobol): resolve 12th review — dynamic CALL/CANCEL dedup + trailing anchors

#1+#2: Removed incorrect hasQuotedCall/hasQuotedCancel deduplication
guards. RE_CALL_DYNAMIC and RE_CANCEL_DYNAMIC require [A-Z] after
CALL/CANCEL, so they CANNOT match quoted targets — the guards were
both unnecessary and actively harmful, suppressing dynamic CALL/CANCEL
in ON EXCEPTION patterns.

#3+#5: Changed RE_CALL_DYNAMIC and RE_CANCEL_DYNAMIC trailing anchor
from (?:\s|\.) to (?=\s|\.|$) (lookahead). The consuming anchor
failed when the identifier was the last token on a physical line.

Tests: 168 passing | TypeScript clean

* feat(cobol): add CALL accumulator + fix SORT double-statement (#4, #6)

Finding #4: Multi-line CALL USING accumulator
Added callAccum state variable that accumulates CALL statements
spanning multiple physical lines until period or END-CALL is found.
Uses flushCallAccum() to re-extract CALL target + USING parameters
from the full accumulated statement. This fixes the silent loss of
ACCESSES parameter edges when USING appears on lines after CALL.

Finding #6: SORT double-statement on same line
After flushSort(), the code now falls through to re-check the
current line for a new SORT/MERGE start (was previously blocked
by the sortAccum === null check evaluating before flushSort ran).

Also fixed: used non-global regex for CALL detection test to avoid
the classic global regex .test() lastIndex bug.

Tests: 168 passing (+1 ACCESSES from multi-line CALL USING)

* fix(cobol): resolve 13th review — CICS LOAD, USING extraction, file scoping

#1: CICS LOAD unresolved edge no longer silently deleted in second pass.
    Changed narrow cics-link/cics-xctl check to catch-all pattern:
    rel.reason?.startsWith('cics-') && rel.reason.endsWith('-unresolved')

#2: flushCallAccum USING extraction now stops before COBOL statement
    verbs (INSPECT, SEARCH, SORT, MERGE, DISPLAY, ACCEPT, MOVE, PERFORM,
    GO TO, CALL, IF, EVALUATE). Prevents absorbing adjacent statements
    as false USING parameters in legacy pre-COBOL-85 code without END-CALL.

#3: CICS FILE Record nodes now globally-scoped (<cics-file>:FILENAME)
    instead of per-file-scoped. Enables cross-program CICS file access
    analysis, consistent with SQL table scoping (<db>:TABLE).

#4: callAccum pre-check regex now has (?<![A-Z0-9-]) lookbehind to
    prevent false activation on compound identifiers like WS-CALL-FLAG.

Tests: 168 passing | TypeScript clean

* fix(cobol): resolve 14th review — callAccum false paragraph + Area A guard

#1: callAccum continuation lines now check for COBOL statement verb
    starts (GO TO, PERFORM, MOVE, etc.) and paragraph/section headers.
    If detected, the CALL is flushed as-is and the line processed
    normally — prevents false paragraph detection and currentParagraph
    corruption from lines like "WS-ADDR." being treated as paragraphs.

#4: callAccum pre-check now guarded by currentDivision === 'procedure'
    to prevent unnecessary activations in DATA DIVISION.

#5: Fixed-format paragraph detection now rejects lines with >7 leading
    spaces (Area B indentation) as paragraph candidates. Paragraph
    names in fixed-format must start in Area A (col 8-11, max 7 spaces).
    Free-format mode is unaffected.

Tests: 168 passing | TypeScript clean

* fix(cobol): resolve 15th review — callAccum Area A + verb boundary fixes

#A: Column-position-aware paragraph detection in callAccum flush.
#B: inspectAccum early-flush on paragraph/section/verb headers.
#C: Verb boundary \b → (?:\s|$) prevents MOVE-COUNT false flush.

* test(cobol): add 17 edge-case regression tests + fix USING verb boundary

17 new tests covering all recurring review patterns:

Multi-line CALL USING (7 tests):
- Parameters on separate continuation lines (IBM mainframe style)
- No absorption of INSPECT/GO TO/paragraphs following CALL
- END-CALL scope terminator
- Hyphenated identifiers (MOVE-COUNT) not triggering false flush
- Dual quoted+dynamic CALL on same line (ON EXCEPTION)

Nested program attribution (2 tests):
- CALL in inner program within inner line range
- PERFORM before first paragraph has null caller

CRLF compatibility (1 test):
- GO TO DEPENDING ON with \r\n line endings

Area A paragraph detection (2 tests):
- Area B (>7 spaces) rejected; Area A (7 spaces) accepted

SORT/MERGE (1 test): COLLATING SEQUENCE keywords not captured
PROCEDURE USING (2 tests): RETURNING excluded, period-terminated
Comment stripping (1 test): pipe in quoted string preserved
SELECT OPTIONAL (1 test): correct file name, not OPTIONAL keyword

Bug fix: USING extraction regex verb terminators changed from
\bVERB\b to \bVERB(?=\s|$) in flushCallAccum — prevents truncation
on hyphenated identifiers like MOVE-COUNT, PERFORM-LIMIT.

Total: 185 tests passing

* test(cobol): add 32 comprehensive edge-case regression tests

13 new describe blocks covering all extraction features:

- EXEC DLI: no-SEGMENT, multi-line accumulation (2 tests)
- SET: multiple targets, DOWN BY, TO numeric (3 tests)
- INSPECT: CONVERTING, multiple counters, tallying-replacing,
  paragraph flush during accumulation (4 tests)
- DECLARATIVES: no-STANDARD keyword, I-O mode, post-END paragraphs (3)
- COPY REPLACING: pseudotext deletion ==OLD== BY ==== (1 test)
- VALUE: hex literal, negative numeric, ALL literal (3 tests)
- OCCURS: TO range, fixed-size without DEPENDING ON (2 tests)
- Dynamic CALL/CANCEL: end-of-line, multiple CANCELs (3 tests)
- EXEC SQL: INCLUDE skips tables, SELECT INTO host vars, host
  variable extraction (3 tests)
- INITIALIZE: target and caller context (1 test)
- Nested programs: sibling scoping, PROGRAM-ID without ID DIV (2)
- EXEC EOF flush: unclosed EXEC SQL flushed (1 test)
- Multi-PERFORM: IF/ELSE dual PERFORM on single line (1 test)
- IS EXTERNAL: USAGE not polluted by external flag (1 test)

Total: 215 tests passing

* fix(cobol): resolve 16th review — CANCEL in CALL block + USING boundary

#1: flushCallAccum now extracts CANCEL statements from within CALL
    ON EXCEPTION blocks. Adds RE_CANCEL + RE_CANCEL_DYNAMIC matchAll
    passes alongside existing CALL extraction.

#2: Added \bCANCEL(?=\s|$) to USING lookahead regex to prevent CANCEL
    keyword being captured as false USING parameter.

#3: Multi-line CALL start now returns immediately to prevent the CALL
    start line from simultaneously feeding sortAccum/inspectAccum.

#6: Division transitions now flush all active accumulators (callAccum,
    sortAccum, inspectAccum) to prevent state leakage across programs.

Also added CANCEL to callAccum flush trigger verb list.

Tests: 215 passing | TypeScript clean

* refactor(cobol): extract shared verb constants + resolve 17th review

Extract COBOL_STATEMENT_VERBS, RE_STATEMENT_VERB_START, and
RE_USING_PARAMS as shared constants — eliminates 4 duplicated
25-verb regex patterns.

17th review: #1 flushCallAccum before EXEC entry, #2 inspectAccum
verb parity via shared constant.

Tests: 215 passing | TypeScript clean

* test(cobol): replace all fuzzy assertions with exact toBe checks

Replaced 7 toBeGreaterThan/toBeLessThan/toBeGreaterThanOrEqual
assertions with exact toBe values:

- dataItems.length: >= 3 → toBe(3)
- calls.length: >= 1 → toBe(1)
- calls[0].line: range check → toBe(10)
- programs[].startLine/endLine: comparison → exact values
- innerA.endLine/innerB.startLine: comparison → exact values

Also added 11 new edge-case tests (accumulator flush on EXEC/division
transitions, free-format, CANCEL in CALL block, SORT THRU, verb
flush, integration).

226 tests passing — zero fuzzy assertions remain.

* fix(cobol): resolve 19th review + 15 accumulator flush tests

Fixes:
#1: END PROGRAM flushes callAccum/sortAccum/inspectAccum
#2: PROGRAM-ID sibling path flushes all accumulators
#3: Added COMPUTE/ADD/SUBTRACT/MULTIPLY/DIVIDE/STRING/UNSTRING
    to COBOL_STATEMENT_VERBS (now 32 verbs)

Tests (15 new):
- END PROGRAM flush: single + nested programs (2)
- PROGRAM-ID sibling flush (1)
- Arithmetic verb flush: COMPUTE/ADD/SUBTRACT/MULTIPLY/DIVIDE (5)
- String verb flush: STRING/UNSTRING (2)
- Arithmetic not captured as false USING params (1)
- SORT flushed at END PROGRAM (1)
- INSPECT flushed at END PROGRAM (1)
- All with exact toBe assertions (2)

Total: 239 tests passing | Zero fuzzy assertions

* fix(cobol): resolve 20th review — INITIALIZE multi-target + 2 tests

Finding 1: INITIALIZE now captures multiple targets with REPLACING
clause keyword filtering. Regex changed to lazy match stopping at
REPLACING/WITH/period boundary. Targets split on whitespace and
filtered against INITIALIZE_CLAUSE_KEYWORDS set.

Tests (2 new):
- INITIALIZE multi-target: WS-CUSTOMER WS-ORDER WS-LINE-ITEM → 3
- INITIALIZE with REPLACING: only WS-RECORD captured, not keywords

Total: 241 tests passing | TypeScript clean
2026-03-26 14:03:48 +00:00
3c896cdbcd fix: close remaining Dart language support gaps (#524)
* fix: close remaining Dart language support gaps

Four issues that were not addressed in PR #204:

1. extractFunctionName: add function_signature/method_signature handlers
   and add both to FUNCTION_NODE_TYPES. Without this, findEnclosingFunctionId
   cannot resolve Dart function scopes — all calls inside Dart functions
   have no sourceId, breaking CALLS edge attribution.

2. formal_parameter_list: add to paramListTypes in extractMethodSignature.
   Dart's tree-sitter grammar uses this node type (not formal_parameters),
   so parameter counting returns 0 for all Dart functions.

3. Write-access queries: add @assignment patterns for obj.field = value
   and this.field = value. Without these, no ACCESSES write edges are
   emitted for Dart code.

4. initialized_identifier guard in extractDartDeclaration: comma-separated
   declarations (String a, b, c) produce initialized_identifier nodes
   which are in DART_DECLARATION_NODE_TYPES but were unhandled — the type
   lives on the parent node.

Also adds Dart column to the feature matrix in type-resolution-system.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(dart): field-type resolution, call attribution, import resolution, and integration tests

Fixes five Dart language support gaps with integration tests and
architectural alignment:

**Tree-sitter queries** — Add field declaration patterns for typed and
nullable class fields (`String name = ''`, `String? name`).  Without
these, Dart class fields were invisible to the pipeline (zero Property
nodes, zero HAS_PROPERTY edges).

**Import resolution** — Dart relative imports (`import 'models.dart'`)
don't use a leading `./`.  The standard resolver only recognises paths
starting with `.` as relative; bare paths fell through to a Java-style
dot-to-slash conversion that mangled `models.dart` into `models/dart`.
Fix: prepend `./` before calling resolveStandard.

**Call attribution** — Dart's tree-sitter grammar places `function_body`
as a sibling of `function_signature`, not as a child wrapping both.  The
`findEnclosingFunction` parent-walk never found the function because the
call lives inside `function_body` which is a sibling of the signature.
Fix: add `enclosingFunctionFinder` hook to LanguageProvider interface
(following the same strategy pattern as `labelOverride`), with the
Dart-specific logic in `languages/dart.ts`.  Both `parse-worker.ts` and
`call-processor.ts` consume the hook generically — no Dart-specific code
in the generic processors.

**Receiver chain extraction** — Add `unconditional_assignable_selector`
to `MEMBER_ACCESS_NODE_TYPES` so `inferCallForm` returns `'member'` for
Dart method calls.  Add Dart-specific receiver extraction blocks in
`extractReceiverName`, `extractReceiverNode`, and a `selector` handler
in `extractMixedChain` for Dart's flat sibling-selector model (vs the
nested member-expression model used by all other languages).

**Integration tests** — New `dart.test.ts` with field-type resolution
and call-result-binding describe blocks.  Fixtures: `dart-field-types/`
(models.dart + app.dart) and `dart-call-result-binding/` (models.dart +
app.dart).  9 passing tests, 1 skipped (ACCESSES edges for field reads
depend on type-env parameter binding propagation — tracked for follow-up).

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
2026-03-26 13:45:24 +00:00
Wojciech Guziak 546128cdcb refactor: split global BUILT_IN_NAMES into per-language provider fields (#523)
* refactor: make isBuiltInOrNoise provider-aware, remove global BUILT_IN_NAMES

Add builtInNames field to LanguageProviderConfig. Rewrite noise-filter.ts
to accept a LanguageProvider and check provider.builtInNames instead of
a global Set. Update all 3 call sites to pass their existing provider.

Built-in entries will be added per-language in subsequent commits.

* refactor(js/ts): add per-language builtInNames to JS/TS providers

* refactor(python): add per-language builtInNames

* refactor(kotlin): add per-language builtInNames

* refactor(c/cpp): add per-language builtInNames

* refactor(csharp): add per-language builtInNames

* refactor(php): add per-language builtInNames

* refactor(swift): add per-language builtInNames

* refactor(rust): add per-language builtInNames

* refactor(ruby): add per-language builtInNames

* refactor(dart): add per-language builtInNames

* test: update noise-filter tests for per-language API, add isolation tests

- Update ingestion-utils.test.ts to pass provider to isBuiltInOrNoise
- Add noise-filter.test.ts with 15 cross-language isolation tests
- Fix Java heritage test: serialize() is now correctly unfiltered for Java
  (was false-positive noise from global PHP serialize entry)

* refactor: remove noise-filter.ts, add provider.isBuiltInName() method

Per review feedback: delete noise-filter.ts entirely and move the check
into LanguageProvider as isBuiltInName(name) method, generated by
defineLanguage() from the builtInNames set.

Call sites now use provider.isBuiltInName(calledName) directly.
2026-03-26 12:15:29 +00:00
Flavius MironandClaude Opus 4.6 c6ed25de8b docs: add Dart to supported languages table in README (#525)
Dart was added as the 14th supported language in PR #204 but the README
was not updated. Adds Dart row to the supported languages table and
updates the language count from 13 to 14.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 10:46:10 +00:00
ThinhKVT aec9a3f216 [CLI] Fixes a false-positive in the Cypher write-detection regex and improves the Impact tool's enrichment path by using batched chunking and entry-point grouping (#496) (#507) 2026-03-26 10:39:27 +00:00
marxo126 a047a08f54 feat: add ORM dataflow detection (Prisma + Supabase) (#511) 2026-03-26 08:48:40 +00:00
Wojciech Guziak 8864a0290c feat: add Dart language support (#204) 2026-03-26 08:42:49 +00:00
Chirag Nighut 1a0784befe feat: add more node types in filter panel (#519)
* fix(web): enable Cypher queries when connected to backend server

Route queries through HTTP API in backend mode instead of checking local WASM database.

Made-with: Cursor

* chore: add Maven/Gradle wrapper files to default ignore list

Add build wrapper scripts and directories to hardcoded ignore lists:
- Directories: .mvn, .gradle, gradle
- Files: mvnw, mvnw.cmd, gradlew, gradlew.bat

These are build infrastructure files, not source code.

Made-with: Cursor

* ci: re-trigger CI (Windows flaky timeout)

Made-with: Cursor

* feat: add more node types in filter panel

* feat: add more node types in filter panel

* revert additional changes

* test(web): add unit tests for filter panel node types

- FILTERABLE_LABELS: verify new types (Enum, Type, Decorator, Variable)
  have colors, sizes, and no duplicates
- Filter panel icons: verify every filterable label has an icon mapped
  and all icons are exported from lucide-icons
- Color legend: verify new types are included, ordered correctly, and
  are a subset of FILTERABLE_LABELS

Made-with: Cursor
2026-03-26 06:50:40 +00:00
Gergő Magyar 33225c2fce fix(ci): move shape-check-regression test to lbug-db project (#518)
The shape-check-regression test uses withTestLbugDB but was running in
the default vitest project with parallel forks, causing LadybugDB
file-lock conflicts on Windows CI. Move it to the lbug-db project
(sequential execution) and exclude from default.

Follows up on #501.
2026-03-26 06:23:02 +00:00
marxo126 5a7ac218df fix: shape_check false positives — quoted keys, DOM leaks, errorKeys (#501) 2026-03-26 05:43:37 +00:00
jelsco b959b9933b fix(python): resolve two remaining alias gaps (#417) (#505) 2026-03-26 05:23:27 +00:00
Zander Raycraft 6fabd7a2df Merge pull request #402 from adonisdoda/feat/index-cli 2026-03-25 20:22:21 -05:00
marxo126 f860653a69 feat(routes): link Next.js project-level middleware.ts to routes (#504) 2026-03-25 22:07:58 +00:00
Wojciech Guziak 77dcb06a8d chore: upgrade tree-sitter to 0.25.0 and all grammar packages (#516) 2026-03-25 21:55:14 +00:00
Zander Raycraft e7e26d6345 Merge pull request #381 from cnighut/feat/cursor-cli-wiki-provider 2026-03-25 08:20:23 -05:00
marxo126 95f97c884c feat: add Expo Router file-based route detection (#503) 2026-03-25 11:05:55 +00:00
marxo126andClaude Opus 4.6 4bc4815bd2 feat: PHP response shape extraction for json_encode patterns (#502)
* feat: add PHP response shape extraction for json_encode patterns

Adds extractPHPResponseShapes() to detect response keys from PHP
json_encode() calls with associative array literals. Supports:
- Short array syntax: json_encode(['key' => value])
- Long array syntax: json_encode(array('key' => value))
- Error classification via http_response_code() and header() status
- exit;/die; boundary detection to prevent cross-block status leaking
- Nested array filtering (only top-level keys extracted)

Pipeline integration dispatches PHP files to the new extractor.
Verified on collector project: 10 PHP routes now show responseKeys.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review — exit boundary, die; offset, CGI Status header

- Replace lastIndexOf('exit;')/lastIndexOf('die;') with regex that
  matches exit(N), exit(0), die('msg'), die($var) as boundaries
- Fixes die; off-by-one (was slicing at +5 for a 4-char keyword)
- Add header('Status: NNN') CGI/FastCGI format detection
- Add 3 regression tests for the fixed bugs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract shared helpers, remove duplicate test block

- Extract lastMatchGroup() and buildShapeResult() to eliminate repeated
  patterns in both JS/TS and PHP extractors
- Simplify detectPHPStatusCode to use ?? chaining with lastMatchGroup
- Remove duplicate 9-test PHP describe block (kept the 12-test version)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add PHP response shape integration tests

Adds a PHP fixture (api/items.php, api/submit.php) with multiple
json_encode patterns and a pipeline integration test verifying:
- Route nodes created for PHP endpoints
- responseKeys/errorKeys correctly extracted and separated
- exit(N)/die() boundaries respected
- HANDLES_ROUTE edges point to correct PHP handler files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 08:27:53 +00:00
c68d7975e6 docs: agent development framework, GitHub templates, eval refactor (#479)
* ci: E2E workflow, web typecheck job, pre-commit hook, test suite

CI:
- ci.yml consolidated to reference ci-tests.yml
- ci-quality.yml: add typecheck-web job for gitnexus-web/
- ci-e2e.yml: E2E workflow with dorny/paths-filter (web changes only)
- ci-report.yml: remove dead integration-reports references
- CI gate allows skipped E2E status
- .gitignore: playwright artifacts, eval test artifacts

Pre-commit hook:
- .githooks/pre-commit: typecheck + unit tests for both packages
- Activated via git config core.hooksPath in prepare script

Test infrastructure:
- Vitest + React Testing Library: 58 unit tests
  (graph, server-connection, mermaid, settings, constants, utils, paths)
- Playwright E2E: 5 tests + manual recording harness
- vitest.config from vitest/config, engines.node >= 20
- Playwright artifacts retain-on-failure
- wait-on in devDependencies
- vitest/coverage-v8 aligned with vitest 4.x

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update gitnexus-web package-lock.json

Reflects devDependency additions (vitest, playwright, wait-on,
@testing-library, etc.) from package.json changes in this PR.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(e2e): add missing process-list-loaded testid, increase CI timeouts

- Add data-testid="process-list-loaded" to ProcessesPanel (E2E tests
  were waiting for an element that didn't exist)
- Increase server connect timeouts from 5s to 10s for slower CI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): run gitnexus-web unit tests in CI, remove unused variable

- Add gitnexus-web npm ci + vitest run to ci-tests.yml so web unit
  tests are gated by the CI status check (were only running locally)
- Remove unused IS_PLAYWRIGHT_AUTOMATION variable from E2E spec

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(e2e): add process-row testid, wait for networkidle on page load

- Add data-testid="process-row" to ProcessItem component (E2E tests
  referenced it but it didn't exist in the source)
- Use waitUntil: 'networkidle' on page.goto to ensure Vite dev server
  is fully ready before interacting (fixes first-test timeout in CI)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(e2e): add process-view-button and process-highlight-button testids

E2E tests referenced these data-testid attributes but they didn't
exist in ProcessItem. All 6 E2E testids now have matching source
elements: status-ready, process-list-loaded, process-row,
process-view-button, process-highlight-button, server-url-input.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(e2e): remove networkidle — Vite HMR WebSocket prevents it from resolving

networkidle waits for zero network activity for 500ms, but Vite's HMR
WebSocket stays open permanently, causing page.goto to timeout at 60s
on all tests after the first. The explicit toBeVisible waits on UI
elements are sufficient and deterministic.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(e2e): wait for Server button visibility, add CI retry, all 5 tests pass locally

Root cause: test 1 clicked the Server button before React hydrated,
so the tab content never rendered and the input wasn't found.

Fixes:
- Wait for Server button toBeVisible before clicking
- Increase input wait to 15s
- Remove networkidle (Vite HMR WebSocket prevents it from resolving)
- Add retries: 1 in CI for transient cold-start flakiness

Verified locally: all 5 E2E tests pass, 198 unit tests pass, typecheck clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): tolerate LadybugDB native crash during analyze step

gitnexus analyze can crash with "double free or corruption" (known
issue #273) during the LadybugDB native addon shutdown. The index is
usually written successfully before the crash. The workflow now:
1. Allows analyze to exit non-zero with a warning
2. Verifies .gitnexus index was actually created
3. Only fails if no index exists (real failure)

All tests verified locally: 198 unit, 5 E2E pass, typecheck clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): fix shell quoting in analyze step, simplify to || true

The previous echo string had special characters that broke bash
quoting in GitHub Actions. Simplified to: analyze || true, then
check if .gitnexus exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add agent development framework, GitHub templates, eval refactor

Agent framework (layered docs for AI-assisted contributions):
- AGENTS.md: canonical instructions, impact analysis, MCP tools
- CLAUDE.md: Claude Code-specific deltas and hooks
- GUARDRAILS.md: safety boundaries, non-negotiables, escalation
- ARCHITECTURE.md: monorepo layout, data flow map
- TESTING.md: test structure, commands, categories
- RUNBOOK.md: copy-paste operations for dev/CI/MCP
- llms.txt: minimal LLM context pointer

Editor integration:
- .cursor/index.mdc + rules/100-monorepo.mdc

GitHub templates:
- PR template with areas-touched checkboxes
- Bug report + feature request issue forms

Eval harness:
- Refactored mcp_bridge, tool_registry, constants
- Error sanitization utilities
- Property-based tests via Hypothesis

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(eval): use format_exception instead of format_exc in sanitize_exception

format_exc() returns the currently handled exception traceback, which
may be unrelated if called outside an active except block. Using
format_exception(type(exc), exc, exc.__traceback__) reliably captures
the passed exception's traceback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update CONTRIBUTING.md and TESTING.md for current CI/hook setup

- CONTRIBUTING.md: add gitnexus-web typecheck command, pre-commit hook
  checklist item
- TESTING.md: add gitnexus-web typecheck command, pre-commit hook
  section (husky), update CI integration to list actual workflow files
  (ci-quality, ci-tests, ci-e2e)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update testing docs to reflect CI/E2E changes from PR #486

- AGENTS.md: update test counts (CLI ~2000 unit, ~1850 integration),
  add gitnexus-web testing section (198 unit, 5 E2E with commands)
- RUNBOOK.md: fix Node requirement to >=20, fix E2E local repro command
- TESTING.md: E2E uses data-testid selectors + real servers, not mocks
- .cursor/rules/100-monorepo.mdc: add web test/E2E commands

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: address context engineering review — deduplicate tokens, expand Cursor rules

- Remove ~100-line gitnexus:start block from CLAUDE.md (was duplicated from AGENTS.md)
- Fix gitnexus:start block inlined inside AGENTS.md Reference Docs bullet (doubled)
- Replace CLAUDE.md scope table with pointer to AGENTS.md (single source of truth)
- Expand .cursor/index.mdc with 5 non-negotiable safety rules for always-on context
- Add .cursor/rules/200-eval.mdc with Python/eval commands (glob-scoped to eval/**)
- Improve llms.txt with priority annotations and descriptions
- Bump version headers to 1.2.0, last-reviewed to 2026-03-24

Saves ~1,400 tokens/session with zero information loss.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Gergő Magyar <gergomagyar@icloud.com>
2026-03-25 06:48:41 +00:00
chirag-nighut 048347df84 fix: address PR review — TTY guard, test rename, unify debug env var
- Add process.stdin.isTTY guard before --review prompt to prevent CI hangs
- Rename misleading --verbose e2e test to reflect it checks help output
- Replace DEBUG with GITNEXUS_VERBOSE for error stack traces

Made-with: Cursor
2026-03-25 11:39:37 +05:30
chirag-nighut f17bde69ba ci: re-trigger CI (Windows flaky timeout in skills-e2e)
Made-with: Cursor
2026-03-25 11:39:37 +05:30
chirag-nighut 7e66ec3a4f test: add e2e CLI tests for wiki flags (--provider, --review, --verbose)
Spawn actual CLI process to verify:
- wiki --help surfaces all new flags
- wiki on non-git directory exits with code 1
- wiki on non-indexed repo fails with "No GitNexus index"
- --provider cursor skips API key prompt in non-TTY mode
- --verbose is accepted as valid flag

Made-with: Cursor
2026-03-25 11:39:37 +05:30
chirag-nighut 76a581b295 test: add unit tests for wiki CLI flags (--provider, --review, --verbose)
17 tests covering:
- detectCursorCLI caching (avoids repeated spawns)
- resolveCursorConfig defaults
- resolveLLMConfig provider routing (cursor vs openai)
- --verbose env var propagation
- WikiGenerator reviewOnly mode (early return with moduleTree)
- CLI config round-trip for cursor and openai providers
- invokeLLM dispatch (cursor → callCursorLLM, openai → callLLM)
- callCursorLLM error when CLI not found
- estimateTokens heuristic

Made-with: Cursor
2026-03-25 11:39:37 +05:30
chirag-nighut 1e309c1ec1 fix: address PR review — remove redundancies and add wiki help test
- Cache detectCursorCLI() result to avoid spawning `agent --version`
  on every LLM call
- Fix stale JSDoc in cursor-client.ts (no longer uses stdin or
  stream-json)
- Remove unused WikiOptions fields (model, baseUrl, apiKey) that were
  passed but never read by WikiGenerator
- Fix inconsistent progress callback phase tracking in --review
  continuation path
- Add wiki CLI help test covering --provider, --review, --verbose flags

Made-with: Cursor
2026-03-25 11:39:37 +05:30
chirag-nighut fca7815c16 feat(wiki): add Cursor CLI as LLM provider option
Add Cursor headless CLI as a 4th provider option for wiki generation,
allowing users to leverage their Cursor subscription for wiki pages.

- Add --provider cursor flag and cursor-client.ts
- Add --review flag for interactive module tree editing
- Add --verbose flag for debugging
- Improve module tree generation (flatten single-child, unique slugs)
- Prevent DB timeout during long LLM calls

Usage: gitnexus wiki --provider cursor --model claude-4.5-opus-high
Made-with: Cursor
2026-03-25 11:39:37 +05:30
Yash a191c26571 fix(#480): resolve impact/context returning empty results for Java cl… (#489) 2026-03-24 21:46:46 +00:00
Gergő Magyar 7999b6ba7b refactor: SICP-informed LanguageProvider architecture (#488)
* refactor: SICP-informed LanguageProvider architecture for ingestion pipeline

Consolidate 16 scattered dispatch surfaces into a single LanguageProvider
Strategy interface per language. Processors are now fully language-agnostic —
zero SupportedLanguages.X enum access, zero dispatch table imports.

Architecture (5-layer DAG, zero circular dependencies):
  L0: Capability modules (dispatch tables, single source of truth)
  L1: LanguageProvider interface + createLanguageProvider factory
  L2: 13 per-language provider files (Strategy objects)
  L3: Registry with satisfies Record<SL, LP> + pre-built lookup maps
  L4: Processors (language-agnostic, all behavior via provider.*)

Key changes:
- Add LanguageProvider interface with 15 properties (6 required, 9 optional)
- Create 13 provider files in languages/ + php-helpers.ts
- Migrate all processors to getProvider(language) — cached once per scope
- Replace heritage if-checks with provider.interfaceNamePattern/heritageDefaultEdge
- Replace MRO switch(language) with switch(provider.mroStrategy)
- Replace isNodeExported with provider.exportChecker
- Move PHP description extraction behind provider.descriptionExtractor
- Move Swift implicit imports behind provider.implicitImportWirer
- Move PHP route detection behind provider.isRouteFile
- Move Kotlin wildcard append behind provider.importPathPreprocessor
- Remove deprecated TypeEnvironment.env, add fileScope()/allScopes()
- De-export TypeEnv type (module-private)
- Pre-build extensionMap, WILDCARD_LANGUAGES, SYNTHESIS_LANGUAGES at load
- Remove dead entryPointPatterns/frameworkPatterns from interface
- Derive createLanguageProvider config type via Pick/Partial/Omit
- Tighten callback types from any to SyntaxNode
- Migrate 270+ test call sites from .env to TypeEnvironment API

Adding a new language: 3 files (enum + provider + registry line).
No processor file touched. Ever.

* refactor: clean architecture for LanguageProvider with O(1) AST cache

Address all PR #488 review comments and achieve pristine SICP layer separation:

Interface redesign:
- Split LanguageProvider into Config (input) + Provider (runtime with defaults)
- Rename createLanguageProvider → defineLanguage with explicit DEFAULTS constant
- Add MroStrategy, ImportSemantics named type aliases for better IDE tooltips
- Tighten labelOverride signature: string|null → NodeLabel|null (compile-time safety)
- Tighten descriptionExtractor nodeLabel: string → NodeLabel
- Un-export LanguageProviderConfig (internal to defineLanguage)

CI fixes (all 4 failures resolved):
- isNodeExported: add null guard for unknown languages
- preprocessImportPath tests: pass getProvider() instead of raw enum
- MRO tests: update expected strings to match language-agnostic prefixes

Code deduplication:
- Extract findDescendant/extractStringContent to ast-helpers.ts (single source of truth)
- Unify Kotlin method detection: remove duplicate from extractFunctionName,
  use provider.labelOverride as single source of truth via findEnclosingFunctionId
- extractFunctionName return type: string → NodeLabel

Performance (O(1) AST node access):
- Add per-file Map-based memoization in parse-worker for parent-chain walks
- Cache enclosingClassId, enclosingFunctionId, exportStatus per SyntaxNode
- Clear caches before each file parse (not after — handles parse failures)

Architecture (pristine languages/ folder):
- Move php-helpers.ts → helpers/php.ts (L0 capability, not L2 config)
- Create helpers/swift.ts from extracted Swift provider logic
- Extract cppLabelOverride AST walk → isCppInsideClassOrStruct in ast-helpers.ts
- Extract isPhpRouteFile → helpers/php.ts
- All 13 provider files are now pure configuration — zero implementation logic
- Ruby: remove no-op namedBindingExtractor assignment (undefined from dispatch table)

* refactor: eliminate LANGUAGE_QUERIES, typeConfigs, namedBindingExtractors dispatch tables

Phase 1 of L0 dispatch table elimination. Providers now import capabilities
directly instead of indexing into redundant Record<SL, T> dispatch tables:

- LANGUAGE_QUERIES: providers import named query constants directly
  (TYPESCRIPT_QUERIES, PYTHON_QUERIES, etc.). Table kept in tree-sitter-queries.ts
  for call-processor.ts dynamic lookup + test consumers.

- typeConfigs: providers import from individual type-extractor files
  (typescriptConfig from typescript.ts, javaTypeConfig from jvm.ts, etc.).
  Dispatch table fully removed from type-extractors/index.ts.

- namedBindingExtractors: providers import extractors directly from
  named-binding-extraction.ts (extractTsNamedBindings, etc.).
  Dispatch table fully removed from import-resolution.ts.

Net: -48 LOC of dispatch table indirection. L3 satisfies Record<SL, LP>
remains the single exhaustiveness check.

* refactor: eliminate exportCheckers, callRouters, importResolvers dispatch tables

Phase 2 of L0 dispatch table elimination. All 6 dispatch tables are now gone:

- exportCheckers: individual checkers exported directly (tsExportChecker,
  pythonExportChecker, etc.). isNodeExported uses a local checkersByLanguage
  map to avoid circular dependency with languages/index.ts.

- callRouters: table removed. Providers import noRouting or routeRubyCall
  directly. noRouting now exported. Dead import removed from call-processor.ts.

- importResolvers: resolver functions exported with clean names
  (resolveTypescriptImport, resolveJavaImport, etc.). Inline lambdas
  extracted to named exports. Dispatch functions renamed from *Dispatch
  suffix to clean resolve*Import pattern.

Combined with Phase 1, all 6 L0 dispatch tables have been eliminated.
L3 satisfies Record<SL, LanguageProvider> is the single exhaustiveness check.
Providers are now fully self-contained — each imports its capabilities directly.

* perf+refactor: type-env caching, sequential fallback caching, utils.ts split

Phase 3 — performance optimizations and barrel cleanup:

Type-env parent-walk caching:
- Memoize findEnclosingClassName and findEnclosingParentClassName with
  per-file Map<SyntaxNode, string|undefined> caches
- Eliminates O(n*m) repeated child scanning in extractParentClassFromNode
- Caches cleared in buildTypeEnv before each file's walk phase

Sequential fallback caching:
- Add classIdCache + exportCache Maps to parsing-processor.ts
- Mirrors the O(1) memoization pattern from parse-worker.ts
- Both paths now have identical caching for parent-chain walks

Split utils.ts barrel into focused modules:
- noise-filter.ts: BUILT_IN_NAMES + isBuiltInOrNoise (167 LOC)
- language-detection.ts: getLanguageFromFilename (58 LOC)
- utils.ts slimmed to re-exports + yieldToEventLoop + isVerboseIngestionEnabled
- Backward compatible — existing imports from utils.ts still work

* refactor: rename resolvers/ → import-resolvers/, restructure tests per-concern

Directory renames (git mv — history preserved):
- src/core/ingestion/resolvers/ → import-resolvers/ (10 files)
- test/unit/call-routing.test.ts → call-routing/ruby.test.ts
- test/unit/named-binding-extraction.test.ts → named-bindings/csharp.test.ts
- test/unit/import-resolution.test.ts → import-resolution/preprocessing.test.ts

All 11 import paths updated to reference new import-resolvers/ location.
Test imports updated for new subdirectory depth.

Note: test/integration/resolvers/ NOT renamed — those tests cover the full
ingestion pipeline per-language, not just import resolution.

* refactor: eliminate utils.ts barrel — all 33 consumers now import directly

Migrated 65 import sites across 33 files to import from the focused source
module instead of the utils.ts barrel:

- ast-helpers.js: SyntaxNode, extractFunctionName, findEnclosingClassId, etc.
- call-analysis.js: inferCallForm, extractReceiverName, countCallArguments, etc.
- noise-filter.js: BUILT_IN_NAMES, isBuiltInOrNoise
- language-detection.js: getLanguageFromFilename

utils.ts reduced to 2 original functions only:
- yieldToEventLoop
- isVerboseIngestionEnabled

Zero re-exports remain. Every import is now direct to its source module.

* refactor: create utils/ folder, move all shared utilities, delete utils.ts barrel

Final phase of module structure migration:

- git mv ast-helpers.ts, call-analysis.ts, noise-filter.ts,
  language-detection.ts → utils/ subdirectory (history preserved)
- Extract yieldToEventLoop → utils/event-loop.ts
- Extract isVerboseIngestionEnabled → utils/verbose.ts
- Delete utils.ts (zero re-exports, zero functions remain)
- Update 38 import paths across source and test files

The ingestion/ root is now clean — only processors, capability modules,
and the pipeline orchestrator live at the top level. All shared utilities
are in utils/, all language-specific helpers in helpers/, all import
resolvers in import-resolvers/.

* refactor: move findChild from import-resolvers/utils.ts to utils/ast-helpers.ts

findChild is a generic AST helper (find first named child by type) — it
belongs with the other AST traversal utilities, not in the import resolver
module. 4 consumers updated to import from utils/ast-helpers.js.

* refactor: split named-binding-extraction.ts into per-language files

Rename named-binding-extraction.ts → named-binding-processor.ts (git mv,
history preserved), keeping only walkBindingChain for re-export chain resolution.

7 per-language extractor functions moved to named-bindings/ subdirectory:
- named-bindings/typescript.ts (extractTsNamedBindings — TS + JS)
- named-bindings/python.ts (extractPythonNamedBindings)
- named-bindings/kotlin.ts (extractKotlinNamedBindings)
- named-bindings/rust.ts (extractRustNamedBindings + collectRustBindings)
- named-bindings/php.ts (extractPhpNamedBindings)
- named-bindings/csharp.ts (extractCsharpNamedBindings)
- named-bindings/java.ts (extractJavaNamedBindings)

Each provider now imports its binding extractor from the per-language file.

* refactor: eliminate import-resolution.ts — distribute to natural homes

Split per-language resolvers into import-resolvers/ per-language files and
eliminate the import-resolution.ts catch-all module entirely:

Per-language resolvers moved to import-resolvers/:
- standard.ts: resolveStandard, resolveJavascriptImport, resolveTypescriptImport,
  resolveCImport, resolveCppImport
- jvm.ts: resolveJavaImport, resolveKotlinImport
- go.ts: resolveGoImport
- csharp.ts: resolveCSharpImport (helper renamed to Internal)
- php.ts, python.ts, ruby.ts, rust.ts: same pattern
- swift.ts: new file for resolveSwiftImport

Types distributed to their concern directories:
- import-resolvers/types.ts: ImportResult, ImportConfigs, ResolveCtx, ImportResolverFn
- named-bindings/types.ts: NamedBinding, NamedBindingExtractorFn

preprocessImportPath moved to import-processor.ts (its primary consumer).

import-resolution.ts deleted — zero catch-all modules remain.

* refactor: tighten SPR — eliminate re-exports, dead code, type holes, and redundant patterns

12 review findings resolved across the ingestion layer:

Type safety:
- CallRouter callNode: any → SyntaxNode (closes type hole)
- CaptureMap type alias replaces Record<string, any>
- providersWithImplicitWiring filter now type-narrowed (removes ! assertions)
- Ruby exportChecker: unnecessary as-cast removed, named export created

Architecture:
- Circular type dependency eliminated (ImportResolutionContext moved to types.ts)
- LANGUAGE_QUERIES residual dispatch replaced with provider.treeSitterQueries
- noRouting sentinel deleted — callRouter now properly optional on 12 providers
- All 6 re-exports from import-processor/pipeline/languages eliminated

Pattern cleanup:
- Dead checkersByLanguage table + isNodeExported removed from export-detection
- 4 duplicated config interfaces consolidated to language-config.ts
- extractCsharpNamedBindings → extractCSharpNamedBindings (casing consistency)

Simplification:
- import-resolvers/index.ts barrel deleted (dead re-exports)
- helpers/ inlined into languages/ (php.ts, swift.ts) — 1 directory removed

Verified: tsc --noEmit clean, 3837 tests pass, 0 failures.

* refactor: address review — remove LANGUAGE_QUERIES table, type-extractors barrel, fix Windows timeout

Review comment fixes (github.com/abhigyanpatwari/GitNexus/pull/488#issuecomment-4117817648):

1. LANGUAGE_QUERIES dispatch table removed from tree-sitter-queries.ts
   — 5 test files migrated to getProvider(lang).treeSitterQueries
   — eliminates last parallel dispatch surface

2. type-extractors/index.ts barrel deleted
   — type-env.ts now imports TYPED_PARAMETER_TYPES from shared.js directly

3. Windows CI timeout fix: afterAll cleanup hook in test-indexed-db.ts
   now passes explicit 120s timeout to prevent KuzuDB C++ destructor
   hang from hitting vitest's default 30s testTimeout on Windows

Verified: tsc --noEmit clean, 3835 tests pass, 0 failures.

* refactor: eliminate chained getProvider property access — assign to variable first

All getProvider(lang).property calls now follow the pattern:
  const provider = getProvider(language);
  const x = provider.property;

5 source files + 4 test files updated (~35 occurrences).
This ensures consistent provider variable usage and avoids
repeated lookups in hot paths.

* refactor: remove last 4 re-exports from import-resolvers, fix stale CaptureMap comment

- Remove `export type { TsconfigPaths }` from standard.ts
- Remove `export type { GoModuleConfig }` from go.ts
- Remove `export type { ComposerConfig }` from php.ts
- Remove `export type { CSharpProjectConfig }` from csharp.ts
  All 4 types are canonically defined in language-config.ts;
  zero consumers imported via the resolver re-exports.

- Fix stale CaptureMap JSDoc: said "Uses any" but type is SyntaxNode | undefined
2026-03-24 13:42:39 +00:00
John R. Eakin f0540b33fb ci: E2E workflow, web typecheck job, pre-commit hook, test suite (#486) 2026-03-24 06:01:08 +00:00
harryphung fec47cbc32 feat(web): add GLM (Z.AI) as LLM provider (#468)
Add GLM support using OpenAI-compatible API via ChatOpenAI from LangChain.
Defaults to the Z.AI coding endpoint (https://api.z.ai/api/coding/paas/v4)
with configurable base URL. Supported models: GLM-5, GLM-5-Turbo, GLM-4.7, GLM-4.5.
2026-03-24 06:10:37 +05:30
B. Lawrence Sharma a9d43d5680 fix(ui): use actual repository name instead of top-level folder for GitHub clones (#491) 2026-03-24 05:54:38 +05:30
Zander Raycraft e6aea1c82b Merge pull request #490 from zander-raycraft/main
Updating claude mcp init for window and adding OS detection when installing
2026-03-23 17:47:03 -05:00
Zander Raycraft 3961ad01cf updating windows support 2026-03-23 17:33:32 -05:00
marxo126 c437acf6bb feat: deep flow detection — consumer access tracking, middleware chains, error shapes, api_impact tool (#482) 2026-03-23 22:13:48 +00:00
Gergő Magyar 2ede01dbff Merge pull request #477 from jreakin/perf/web-o1-lookups-memo-bundle
perf(web): O(1) lookups, memoization, bundle optimizations, React fixes
2026-03-23 17:01:09 +00:00
jreakinandClaude Opus 4.6 fd61cd2990 fix(web): allow foreignObject in DOMPurify SVG, remove leftover loop code
- Add ADD_TAGS: ['foreignObject'] to all DOMPurify.sanitize calls —
  Mermaid uses foreignObject for HTML text labels inside flowchart
  nodes. The SVG profile was stripping them, causing empty boxes.
- Remove leftover sub-batch loop lines from prepared statement hoist

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:34 -05:00
jreakinandClaude Opus 4.6 83ec256cec fix(web): address React rendering review — stale closure, ref dep, O(1) lookups
- MarkdownRenderer: wrap handleLinkClick in useCallback, add to
  markdownComponents useMemo deps (fixes stale closure)
- GraphCanvas: remove sigmaRef from useEffect deps (ref identity
  never changes), extract handleToggleAIHighlights to useCallback
- CodeReferencesPanel: add nodeById Map for O(1) focus-in-graph
  lookup (was O(N) graph.nodes.find on every click)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:34 -05:00
jreakinandClaude Opus 4.6 29795bb86e fix(web): switch lucide-icons from deep ESM paths to standard re-exports
Deep imports from lucide-react/dist/esm/icons/*.js are internal paths
that broke the Vercel production build. Replaced with standard named
re-exports from lucide-react — keeps the centralized module pattern
without relying on fragile internal paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:34 -05:00
jreakinandClaude Opus 4.6 c8cd733815 perf(web): prepare statement once per label pair, not per sub-batch
Hoists conn.prepare(cypher) outside the sub-batch loop so it's called
once per (fromLabel, toLabel) pair instead of ceil(N/4) times. The
statement is reused for all rows in the group, then closed in finally.

Yields to event loop every 500 relations instead of every sub-batch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:33 -05:00
jreakinandClaude Opus 4.6 5616f71fd2 perf(web): switch remaining components to deep lucide imports, extract ProviderConfigCard
- BackendRepoSelector, EmbeddingStatus, Header, MermaidDiagram,
  QueryFAB, RightPanel, ToolCallCard, WebGPUFallbackDialog: switch
  from lucide-react barrel imports to @/lib/lucide-icons deep imports
- MermaidDiagram: lazy-load ProcessFlowModal via React.lazy
- Extract ProviderConfigCard from SettingsPanel for cleaner separation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:33 -05:00
jreakinandClaude Opus 4.6 2bd04fe2c4 test: add positive/negative tests for O(1) lookup optimizations
8 tests verifying the data structures underlying performance changes:

nodeById Map (O(1) lookup):
  + Map.get returns correct node by ID
  + duplicate IDs: last wins
  - non-existent ID returns undefined
  - empty Map returns undefined

Set.has (O(1) highlight matching):
  + present IDs return true
  - absent IDs return false
  + handles IDs with colons, dots, slashes
  - case-sensitive matching

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:33 -05:00
jreakinandClaude Opus 4.6 4199c5e4f3 perf(web): O(1) lookups, memoization, bundle optimizations, React fixes
Performance:
- nodeById Map in GraphCanvas for O(1) click/hover lookups
- fileNodeByPath Map in useAppState for O(1) file path lookups
- Set.has() for HIGHLIGHT_NODES/IMPACT matching (was O(N²))
- useMemo for primaryLanguage in StatusBar
- useCallback on toggleLabelVisibility/toggleEdgeVisibility
- Recursive FileTreePanel search (full subtree, not 1 level)

React fixes:
- Remove stale queryResult dep from clearAICodeReferences
- Cancel RAF chains in CodeReferencesPanel on cleanup
- Clean up timeouts in SettingsPanel and MarkdownRenderer on unmount
- try/catch on localStorage in DropZone (private browsing)
- Await handleServerConnect in App.tsx auto-connect
- pendingToolCalls counter replaces allToolsDone boolean in agent.ts
- JSON.parse try/catch in agent streaming
- sessionStorage JSDoc fix in settings-service

Bundle:
- Centralized lucide icon deep imports

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:33 -05:00
jreakinandClaude Opus 4.6 0d9a2ca6b6 fix(web): allow foreignObject in DOMPurify SVG sanitization
Mermaid uses foreignObject for HTML text labels inside flowchart
nodes. The SVG profile strips them by default, causing empty boxes.
ADD_TAGS: ['foreignObject'] preserves text while still sanitizing
against XSS.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 10:16:33 -05:00
Gergő Magyar 88fa1a4e09 Merge pull request #476 from jreakin/fix/web-csv-schema-correctness
fix(web): 18 multi-language CSV tables, RFC 4180, schema sync
2026-03-23 14:12:28 +00:00
jreakinandClaude Opus 4.6 8956da61bb fix(web): add 15 missing NodeLabel colors/sizes, escape table names, schema sync
- Add NODE_COLORS and NODE_SIZES entries for all 15 multi-language labels
  (Struct, Trait, Impl, TypeAlias, Const, Static, Namespace, Union,
  Typedef, Macro, Property, Record, Delegate, Annotation, Constructor,
  Template) — fixes Record<NodeLabel, ...> completeness
- Escape table names in count/lookup queries with escapeTableName() to
  prevent silent failures for backtick-required tables
- Add HAS_PROPERTY and ACCESSES to RelationshipType union
- Update initPromise after db recreation in loadGraphToLbug so subsequent
  initLbug() calls return fresh db/conn refs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 09:03:40 -05:00
Abhigyan PatwariandClaude Opus 4.6 9ba2b22ac6 fix(web): resolve Vercel build errors from security hardening PR (#483)
- ProcessFlowModal: revert import from non-existent @/lib/lucide-icons back to lucide-react
- embedding-pipeline: remove extra argument in executeQuery call (signature only accepts 1 arg)

Both issues were introduced in PR #475 (web-security-hardening).

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 19:31:08 +05:30
Gergő Magyar db6b302fee Merge pull request #408 from marxo126/fix/swift-query-and-patch-script
feat: complete Swift support — query fix, export detection, implicit imports, constructor resolution
2026-03-23 13:53:25 +00:00
marxo126andClaude Opus 4.6 fcd2c3ff38 docs: add macro declarations to Swift ingestion gaps
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 14:21:32 +01:00
jreakinandClaude Opus 4.6 f4bb78c03a test: add negative tests for CSV generation
- empty graph produces header-only CSVs (no data rows)
- empty graph relCSV has only header
- double quotes in node names are RFC 4180 escaped
- file node without fileContents gets empty content (no crash)
- community with empty keywords array produces valid CSV
- unknown node labels are silently skipped

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 08:21:31 -05:00
jreakinandClaude Opus 4.6 47f9b6c48d fix: add multi-language NodeLabel types, close old db/conn, CSV tests
- Add 15 multi-language labels to NodeLabel union (Struct, Trait, Impl,
  TypeAlias, Const, Static, Namespace, Union, Typedef, Macro, Property,
  Record, Delegate, Annotation, Constructor, Template) — eliminates
  unsafe casts in csv-generator
- Close previous conn/db before recreating in loadGraphToLbug to prevent
  WASM resource leaks across repo switches
- Add 7 CSV generation tests: multi-language tables, column count,
  keyword comma escaping, file content, relation CSV, all NODE_TABLES

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 08:21:31 -05:00
jreakinandClaude Opus 4.6 433f403fe2 fix(web): 18 multi-language CSV tables, RFC 4180 escaping, schema sync
- Generate CSVs for Struct, Enum, Macro, Trait, Impl, TypeAlias, Const,
  Static, Property, Record, Delegate, Annotation, Constructor, Template,
  Module, Namespace, Union, Typedef (were defined in schema but silently
  dropped during CSV generation — non-JS/TS repos lost all nodes)
- Escape community keywords array in CSV to prevent comma breakage
- Add Section to NodeLabel type, NODE_COLORS, NODE_SIZES
- Add 7 missing REL_TYPES: HAS_METHOD, HAS_PROPERTY, OVERRIDES, ACCESSES,
  INHERITS, USES, DECORATES
- RFC 4180 doubled-quote regex for relation CSV parsing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 08:21:31 -05:00
marxo126andClaude Opus 4.6 fcf8fb9bdf docs: add Swift ingestion gaps tracker and update feature matrix
- Create swift-ingestion-gaps.md with prioritized gap tracker (High/Medium/Low)
- Update type-resolution-system.md feature matrix: 5 Swift entries corrected
  (for-loop→Yes, pattern binding→Partial, call-result/field/method→Yes)
- Add footnotes explaining Swift-specific semantics
- Document resolved items with commit references

Addresses @magyargergo's request to document missing Swift features.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 14:16:43 +01:00
marxo126 47fdad14ed Merge remote-tracking branch 'upstream/main' into fix/swift-query-and-patch-script
# Conflicts:
#	gitnexus/src/core/ingestion/call-processor.ts
2026-03-23 12:21:53 +01:00
marxo126andClaude Opus 4.6 1f4c4e77ab refactor: simplify Swift support code after review
- Move `pattern` node handling into shared extractVarName (like mut_pattern)
  instead of inline fallback in type-env — benefits all callers
- Remove non-null assertion (!) on firstNamedChild — defensive null check
- Avoid 100K wrapper object allocation: addSwiftImplicitImports now accepts
  string[] directly, eliminating allFileList.map(p => ({ path: p }))
- Remove duplicate comment block in for-loop test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 11:44:11 +01:00
marxo126andClaude Opus 4.6 956dfd0bb4 feat: add Swift integration tests for if-let, await/try, for-loop + fix cross-chunk imports
- Add 3 new test fixtures: swift-if-let-guard-let, swift-await-try, swift-for-loop-inference
- Add integration tests for if let/guard let binding resolution (4 assertions)
- Add integration tests for await/try expression unwrapping (3 assertions)
- Add for-loop-inference fixture (documented as known gap — type-env infrastructure
  is in place but call-processor re-parse path doesn't propagate the binding yet)
- Fix cross-chunk Swift implicit imports: standard processImports path now passes
  allFileList instead of chunk-only files to addSwiftImplicitImports, matching
  the fast-path behavior
- Add Swift type_annotation fallback in type-env declarationTypeNodes population
  (handles [User] array sugar where childForFieldName('type') returns null)
- Handle Swift 'pattern' node in extractVarName fallback (pattern wraps simple_identifier)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 11:40:24 +01:00
adonis 1bb80c3858 feat(cli): enhance index command with single path validation and add to .gitignore 2026-03-23 00:19:26 -03:00
adonis 6fe450d206 test(cli): refactor indexCommand tests for consistency and add new cases 2026-03-22 22:05:22 -03:00
marxo126andClaude Opus 4.6 884b4acf84 fix: regenerate package-lock.json for CI compatibility
npm ci was failing with "Missing: hono@4.12.8" and
"Missing: graphology-types@0.24.8" because the lock file was
out of sync after rebase. Regenerated from clean state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 18:59:55 +01:00
marxo126andClaude Opus 4.6 babf0f90d3 fix: pin tree-sitter versions and add npm overrides
Pin exact versions (no ^) to prevent surprise upgrades:
- tree-sitter: "0.22.4" (was "^0.22.4")
- tree-sitter-swift: "0.7.1" (was "^0.7.1")

Add npm overrides to suppress peer dependency warnings from grammar
packages that declare ^0.21.x but work fine with 0.22.4.

Note: tree-sitter-swift 0.6.0 fails to build on current Node (needs
node-gyp + Swift toolchain). 0.7.1 with prebuilt binaries is required
for Swift support to work at all.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:47:21 +01:00
marxo126andClaude Opus 4.6 0a3cdce00e fix: address Copilot review — private(set) export, for-loop tuple pattern
1. Export detection: exclude private(set)/fileprivate(set) from
   unexported check. Only the setter is restricted — the symbol
   itself is still readable cross-file.

2. For-loop binding: use extractVarName() instead of raw .text
   to avoid polluting scopeEnv with non-identifier keys from
   tuple destructuring patterns (e.g. `for (a, b) in ...`).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:47:09 +01:00
marxo126andClaude Opus 4.6 16b1a63134 feat: 7 Swift features — if/guard let, await/try, for-in, enum cases, self/super, optional chaining, multi-inheritance
Covers all high and medium impact gaps from the Swift feature coverage
analysis:

1. if let / guard let bindings: add if_statement and guard_statement
   to DECLARATION_NODE_TYPES, extract varName and value for Tier 2
   return-type propagation (callResult, copy, fieldAccess, methodCallResult)

2. await / try expression unwrapping: add unwrapSwiftExpression() that
   strips await_expression and try_expression wrappers before checking
   for call_expression. Applied in extractPendingAssignment,
   extractInitializer, and scanConstructorBinding.

3. for item in collection: add extractForLoopBinding for Swift with
   extractSwiftElementTypeFromTypeNode that handles [User] array sugar
   and Array<User> generic types. Registered in typeConfig.

4. Multiple inheritance specifiers: already working — tree-sitter
   queries match all inheritance_specifier occurrences automatically.
   Verified, no code changes needed.

5. Enum case extraction: add (enum_entry (simple_identifier) @name)
   @definition.property query to SWIFT_QUERIES.

6. self/super resolution: unskipped both describe.skip test suites
   (tree-sitter-swift 0.7.1 ships prebuilds, Node 22 build issue
   resolved). Both pass — 5 previously-skipped tests now running.

7. Optional chaining obj?.method(): already working — tree-sitter-swift
   parses the ? transparently. Verified, no code changes needed.

Tests: 3,603 → 3,608 (5 unskipped self/super tests)
Swift tests: 23 → 28 passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:47:09 +01:00
marxo126andClaude Opus 4.6 99f0aaaea5 test: add integration tests for Swift implicit imports, extension dedup, constructor fallback, export visibility
Addresses reviewer feedback: the new Swift behaviors (implicit imports,
constructor fallback, extension dedup, export detection) had no dedicated
integration tests. Adds 4 fixture directories and 11 new test assertions:

1. swift-implicit-imports: two files, no explicit import, cross-file
   constructor + member call resolves via addSwiftImplicitImports
2. swift-extension-dedup: extension creates duplicate Class node,
   constructor still resolves to primary definition
3. swift-constructor-fallback: ClassName() without `new` resolves as
   constructor via free→constructor retry
4. swift-export-visibility: internal symbols visible cross-file,
   public/open visible, private/fileprivate noted as Tier 3 limitation

All 3,603 tests pass (11 new, 0 regressions).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:47:09 +01:00
marxo126andClaude Opus 4.6 53b576776a feat: add extractPendingAssignment for Swift return-type inference
Swift was missing the extractPendingAssignment extractor, which meant
return-type-based variable bindings like `let user = getUser()` couldn't
propagate the return type of `getUser()` to `user`. This broke member
call resolution: `user.save()` couldn't resolve to `User.save()` when
there were competing methods (both User and Repo have save()).

Handles four Swift patterns:
- let user = getUser()        → callResult (Tier 2 propagation)
- let result = user.save()    → methodCallResult
- let name = user.name        → fieldAccess
- let copy = user             → copy

All 3,592 tests pass — including the 2 previously-failing Swift
return-type inference tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:47:09 +01:00
marxo126andClaude Opus 4.6 712598a055 fix: update Swift queries and tests for tree-sitter-swift 0.7.1
- Fix assignment query: tree-sitter-swift 0.7.1 uses named fields
  (target:/result:/suffix:) instead of positional children
- Update export detection tests: Swift `internal` (default) is now
  correctly treated as exported (module-scoped visibility)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:47:09 +01:00
marxo126andClaude Opus 4.6 90c9153ccc feat: upgrade tree-sitter to 0.22.4 and tree-sitter-swift to 0.7.1
Upgrades:
- tree-sitter: ^0.21.0 → ^0.22.4
- tree-sitter-swift: ^0.6.0 → ^0.7.1

Benefits:
- tree-sitter-swift@0.7.1 ships prebuilds (no manual node-gyp needed)
- Fixes parsing of Swift 5.9+ features: #Predicate, typed throws, ~Copyable
- All 13 language parsers verified compatible with tree-sitter@0.22.4

Tested:
- All 141 query/import/call tests pass
- All 13 parsers (C, C++, C#, Go, Java, JS, Kotlin, PHP, Python, Ruby,
  Rust, TypeScript, Swift) parse correctly with 0.22.4
- PricePal (61-file iOS 26 project) indexes fully: 3,094 nodes, 10,459 edges

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:47:09 +01:00
marxo126andClaude Opus 4.6 dfe83f333e fix: deduplicate Swift extension class nodes in call resolution
When Swift extensions create multiple Class nodes with the same name
(e.g. Product.swift + ProductMatchableConformance.swift), the call
resolver gets multiple candidates and refuses to emit a CALLS edge.

Add dedup: when all candidates share the same type (Class/Struct) and
differ only by file, prefer the primary definition (shortest filepath).

Note: This fix is partial — some constructor calls inside function
bodies may still be consumed by the type-env constructor binding
scanner before reaching resolveCallTarget. Filed as known limitation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:46:34 +01:00
marxo126andClaude Opus 4.6 561a54a154 refactor: simplify Swift fixes after code review
- Extract shared addSwiftImplicitImports() helper (DRY — was duplicated
  in processImports and processImportsFromExtracted)
- Cache importMap.get(srcFile) outside inner loop (avoids redundant
  Map lookups per iteration)
- Fix export-detection: use \bprivate\b regex instead of includes()
  to avoid substring false positives
- Fix groupSwiftFilesByTarget: check path boundary with indexOf + char
  check instead of loose includes()

All 141 relevant tests pass (queries + imports + calls).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:46:34 +01:00
marxo126andClaude Opus 4.6 65bc99c448 feat: full Swift cross-file resolution (export, imports, constructors)
Three changes that together enable cross-file call resolution for Swift:

1. export-detection.ts: Treat internal (default) Swift symbols as exported.
   Swift's default access level is `internal` (module-scoped, visible to
   all files in the same target). Only private/fileprivate are file-scoped.
   Previously all non-public/open symbols were marked unexported.

2. import-processor.ts: Add implicit import edges between all Swift files
   in the same module/target. Swift has no file-level imports — all files
   see each other automatically. Without these edges, the tiered resolver
   can't find cross-file symbols at Tier 2a (import-scoped).
   Supports SPM targets via Package.swift; falls back to single-module
   for Xcode projects without SPM.

3. call-processor.ts: Add constructor fallback for free-form calls.
   Swift constructors look like free function calls (no `new` keyword):
   `let ocr = OCRService()`. The call form is inferred as `free`, which
   filters out Class/Struct targets. Now retries with `constructor` form
   when free-form finds no callable but the name resolves to a type.

Tested on 61-file iOS 26 project (PricePal):
- Before: 0 cross-file CALLS edges
- After: full cross-file resolution (OCRService traced from ScanViewModel)
- 3,099 nodes, 10,449 edges, 246 clusters, 243 flows

Related: #406, #407

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:46:34 +01:00
marxo126andClaude Opus 4.6 0c8ec952ee fix: handle trailing commas in tree-sitter-swift binding.gyp patch
The patch script fails to parse tree-sitter-swift@0.6.0's binding.gyp
because the file contains both Python-style # comments AND trailing
commas in JSON arrays. The existing regex strips # comments but leaves
trailing commas, causing JSON.parse() to fail with:

  "Unexpected token ']'"

This silently prevents tree-sitter-swift from building, which means
Swift files are skipped entirely during analysis.

Fix: add a second regex pass to strip trailing commas before ] or }
after comment removal.

Fixes #386, #406

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 11:46:34 +01:00
adonis 3567ef5794 feat(cli): add --allow-non-git option to index command and update tests 2026-03-20 19:46:47 -03:00
adonis 680cefd00c feat(cli): refactor indexCommand for improved error handling and add unit tests 2026-03-20 19:36:07 -03:00
adonis c1996f45d8 feat(cli): add 'index' command to register existing .gitnexus/ folder 2026-03-20 19:16:09 -03:00
425 changed files with 38270 additions and 6666 deletions
+21
View File
@@ -0,0 +1,21 @@
---
alwaysApply: true
---
# GitNexus — Cursor project rules
Last reviewed: 2026-03-24
Canonical agent instructions: **[AGENTS.md](../AGENTS.md)** (GitNexus MCP rules, monorepo commands, Cursor Cloud notes). **[CLAUDE.md](../CLAUDE.md)** adds Claude Code-specific notes and points back to AGENTS.md for GitNexus.
## Non-negotiables (always apply)
- NEVER edit a function/class/method without running `gitnexus_impact` first.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename`.
- NEVER commit without running `gitnexus_detect_changes()`.
- NEVER ignore HIGH/CRITICAL risk warnings from impact analysis.
- NEVER run `npx gitnexus analyze` without `--embeddings` if `.gitnexus/meta.json` shows stored embeddings.
Full rules: **[AGENTS.md](../AGENTS.md)** (`gitnexus:start` block, Cursor Cloud section).
**Rule architecture:** Prefer this file plus optional `.cursor/rules/*.mdc` globs (YAML `globs` in frontmatter). Legacy `.cursorrules` is deprecated; content lives here.
+12
View File
@@ -0,0 +1,12 @@
---
globs:
- "gitnexus/**"
- "gitnexus-web/**"
---
# GitNexus build/test quick refs
- CLI (`gitnexus/`): `npm test`; `npm run test:integration`; `npx tsc --noEmit`.
- Web (`gitnexus-web/`): `npm test`; `npm run dev`; `npx tsc -b --noEmit`; `E2E=1 npx playwright test` (needs servers).
- `npm install` in `gitnexus/` runs `prepare` (tsc build) and `postinstall` (tree-sitter patches); needs `python3`, `make`, `g++`.
- LadybugDB locking tests may fail in containerized environments because of `/tmp` file locks (known issue, not a code bug).
+14
View File
@@ -0,0 +1,14 @@
---
globs:
- "eval/**"
---
# GitNexus eval harness (Python)
- **Run tests**: `cd eval && uv run pytest tests/`
- **Run with coverage**: `cd eval && uv run coverage run -m pytest tests/ && uv run coverage report`
- **Lint**: `cd eval && uv run ruff check .`
- **Run eval**: `cd eval && uv run python run_eval.py --config configs/<config>.yaml`
- Shared constants live in `eval/constants.py`; tool specs in `eval/tool_registry.py`.
- Error logging uses `utils/errors.py` — set `GITNEXUS_EVAL_DEBUG=1` for full tracebacks.
- Property-based tests use Hypothesis (`eval/tests/test_property_based.py`).
+3 -3
View File
@@ -1,5 +1,5 @@
# AI Agent Rules
# Deprecated for Cursor Agent Mode
Follow .gitnexus/RULES.md for all project context and coding guidelines.
Use **`.cursor/index.mdc`** (`alwaysApply: true`) for project rules. See [AGENTS.md](AGENTS.md).
This project uses GitNexus MCP for code intelligence. See .gitnexus/RULES.md for available tools and best practices.
This file is kept only as a breadcrumb for older workflows.
+86
View File
@@ -0,0 +1,86 @@
name: Bug report
description: Report unexpected behavior or a regression
labels: [bug]
body:
- type: markdown
attributes:
value: |
**Goal:** capture enough context to reproduce and fix the issue quickly.
Use **one issue per bug**; split unrelated problems.
- type: dropdown
id: area
attributes:
label: Area
description: Where does the problem show up?
options:
- gitnexus (CLI / core / indexing / MCP server)
- gitnexus-web (browser UI / WASM / workers)
- CI / GitHub Actions
- Documentation / developer experience
- Other
validations:
required: true
- type: textarea
id: summary
attributes:
label: Summary
description: One sentence — what went wrong?
validations:
required: true
- type: textarea
id: context
attributes:
label: Context
description: What were you trying to do? Any relevant links, PRs, or commits?
validations:
required: false
- type: textarea
id: expected
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual behavior
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to reproduce
description: Ordered steps, sample repo or minimal case, commands run.
placeholder: |
1. …
2. …
3. …
validations:
required: true
- type: textarea
id: environment
attributes:
label: Environment
description: OS, Node version, browser (if web), GitNexus version or commit SHA.
placeholder: |
- OS:
- Node:
- Browser (if applicable):
- Commit / version:
validations:
required: false
- type: textarea
id: logs
attributes:
label: Logs / screenshots
description: Paste errors, stack traces, or attach screenshots (redact secrets).
validations:
required: false
+1
View File
@@ -0,0 +1 @@
blank_issues_enabled: true
@@ -0,0 +1,74 @@
name: Feature request
description: Propose a new capability or improvement
labels: [enhancement]
body:
- type: markdown
attributes:
value: |
**Goal:** describe the problem and desired outcome so maintainers can size and prioritize.
Prefer **small, shippable** requests; split large ideas into phases.
- type: dropdown
id: area
attributes:
label: Area
description: Primary part of the monorepo this relates to.
options:
- gitnexus (CLI / core / indexing / MCP server)
- gitnexus-web (browser UI / WASM / workers)
- CI / release / packaging
- Documentation / developer experience
- Other
validations:
required: true
- type: textarea
id: problem
attributes:
label: Problem or opportunity
description: What pain point or gap exists today?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
description: What should happen instead? User-visible behavior, APIs, or UX.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
description: Other approaches you considered and why this one is preferred.
validations:
required: false
- type: textarea
id: acceptance
attributes:
label: Acceptance criteria
description: Testable conditions for “done” (bullets or checkboxes in prose).
placeholder: |
- When … then …
- Documentation / tests updated where appropriate
validations:
required: false
- type: textarea
id: constraints
attributes:
label: Constraints
description: Compatibility, performance, security, or “must not change” boundaries.
validations:
required: false
- type: checkboxes
id: willing
attributes:
label: Contribution
options:
- label: I am willing to open a PR for this (may need design discussion first).
required: false
+52
View File
@@ -0,0 +1,52 @@
## Summary
<!-- One or two sentences: what does this PR change? -->
## Motivation / context
<!-- Why is this change needed? Link issues, ADRs, or prior discussion. -->
## Areas touched
<!-- Check all that apply -->
- [ ] `gitnexus/` (CLI / core / MCP server)
- [ ] `gitnexus-web/` (Vite / React UI)
- [ ] `.github/` (workflows, actions)
- [ ] `eval/` or other tooling
- [ ] Docs / agent config only (`AGENTS.md`, `CLAUDE.md`, `.cursor/`, `llms.txt`, etc.)
## Scope & constraints
**In scope**
- <!-- bullets -->
**Explicitly out of scope / not done here**
- <!-- bullets — prevents reviewers assuming missing work is an oversight -->
## Implementation notes
<!-- Optional: design choices, tradeoffs, follow-ups -->
## Testing & verification
<!-- What you ran; paste commands. Omit sections that do not apply. -->
- [ ] `cd gitnexus && npm test`
- [ ] `cd gitnexus && npm run test:integration` *(if core/indexing/MCP paths changed)*
- [ ] `cd gitnexus && npx tsc --noEmit`
- [ ] `cd gitnexus-web && npm test` *(if web changed)*
- [ ] `cd gitnexus-web && npx tsc -b --noEmit` *(if web changed)*
- [ ] Manual / Playwright E2E *(note environment — see `gitnexus-web/e2e/`)*
## Risk & rollout
<!-- Breaking changes, migrations, index refresh (`npx gitnexus analyze`), release notes -->
## Checklist
- [ ] PR body meets repo minimum length (workflow may label short descriptions)
- [ ] If `AGENTS.md` / overlays changed: headers, scope block, and changelog updated per project conventions
- [ ] No secrets, tokens, or machine-specific paths committed
+91
View File
@@ -0,0 +1,91 @@
name: E2E Tests
on:
workflow_call:
jobs:
check-changes:
name: Check web module changes
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
web_changed: ${{ steps.filter.outputs.web }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
id: filter
with:
filters: |
web:
- 'gitnexus-web/**'
e2e:
name: e2e (chromium)
needs: check-changes
if: needs.check-changes.result == 'success' && needs.check-changes.outputs.web_changed == 'true'
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 20
cache: npm
cache-dependency-path: gitnexus-web/package-lock.json
- name: Install frontend dependencies
run: npm ci
working-directory: gitnexus-web
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
working-directory: gitnexus-web
- name: Install backend dependencies
run: npm ci
working-directory: gitnexus
- name: Build backend
run: npm run build
working-directory: gitnexus
- name: Analyze repository (index for backend)
run: |
node gitnexus/dist/cli/index.js analyze || true
if [ ! -d ".gitnexus" ]; then
echo "::error::No .gitnexus index created"
exit 1
fi
- name: Start backend server
run: node dist/cli/index.js serve &
working-directory: gitnexus
- name: Wait for backend readiness
run: npx wait-on http://localhost:4747/api/repos --timeout 30000
working-directory: gitnexus-web
- name: Start Vite dev server
run: npm run dev &
working-directory: gitnexus-web
- name: Wait for Vite dev server
run: npx wait-on http://localhost:5173 --timeout 30000
working-directory: gitnexus-web
- name: Run E2E tests
run: npx playwright test
working-directory: gitnexus-web
env:
E2E: '1'
- name: Upload test results
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: e2e-results
path: |
gitnexus-web/test-results/
gitnexus-web/playwright-report/
retention-days: 5
+15
View File
@@ -12,3 +12,18 @@ jobs:
- uses: ./.github/actions/setup-gitnexus
- run: npx tsc --noEmit
working-directory: gitnexus
typecheck-web:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 20
cache: npm
cache-dependency-path: gitnexus-web/package-lock.json
- run: npm ci
working-directory: gitnexus-web
- run: npx tsc -b --noEmit
working-directory: gitnexus-web
+314 -249
View File
@@ -1,132 +1,126 @@
name: CI Report
# Triggered after the CI workflow completes. Because workflow_run
# always runs code from the *default branch*, it receives a read/write
# GITHUB_TOKEN — even when the triggering PR comes from a fork.
on:
workflow_run:
workflows: ['CI']
workflows: ["CI"]
types: [completed]
permissions:
actions: read
contents: read
pull-requests: write
actions: read # needed to list/download workflow run artifacts
contents: read # needed for sparse checkout of vitest.config.ts
pull-requests: write # needed to post sticky PR comment
jobs:
pr-report:
name: PR Report
# Only run for pull-request CI runs
if: >-
github.event.workflow_run.event == 'pull_request' &&
github.event.workflow_run.conclusion != 'cancelled'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Download PR metadata
# ── Download artifacts from the CI run ────────────────────────
- name: Download artifacts
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
with:
script: |
const fs = require('fs');
const path = require('path');
const runId = context.payload.workflow_run.id;
const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
const allArtifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
run_id: runId,
});
const meta = artifacts.data.artifacts.find(a => a.name === 'pr-meta');
if (!meta) {
core.setFailed('pr-meta artifact not found — skipping report');
return;
async function downloadArtifact(name, dest) {
const match = allArtifacts.data.artifacts.find(a => a.name === name);
if (!match) {
core.warning(`Artifact "${name}" not found`);
return false;
}
const zip = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: match.id,
archive_format: 'zip',
});
fs.mkdirSync(dest, { recursive: true });
fs.writeFileSync(path.join(dest, `${name}.zip`), Buffer.from(zip.data));
return true;
}
const zip = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: meta.id,
archive_format: 'zip',
});
const temp = process.env.RUNNER_TEMP;
await downloadArtifact('pr-meta', path.join(temp, 'dl'));
await downloadArtifact('test-reports', path.join(temp, 'dl'));
const dest = path.join(process.env.RUNNER_TEMP, 'pr-meta');
fs.mkdirSync(dest, { recursive: true });
fs.writeFileSync(path.join(dest, 'pr-meta.zip'), Buffer.from(zip.data));
- name: Extract artifacts
shell: bash
run: |
cd "$RUNNER_TEMP/dl"
# Extract each artifact into its own directory to avoid filename collisions
for z in *.zip; do
[ -f "$z" ] || continue
name="${z%.zip}"
mkdir -p "$RUNNER_TEMP/artifacts/$name"
unzip -o "$z" -d "$RUNNER_TEMP/artifacts/$name"
done
- name: Extract PR metadata
- name: Read PR metadata
id: meta
shell: bash
run: |
cd "$RUNNER_TEMP/pr-meta"
unzip -o pr-meta.zip
PR_NUMBER=$(cat pr-number | tr -d '[:space:]')
if ! [[ "$PR_NUMBER" =~ ^[0-9]+$ ]]; then
echo "::error::Invalid PR number: '$PR_NUMBER'"
exit 1
DIR="$RUNNER_TEMP/artifacts/pr-meta"
if [ ! -f "$DIR/pr_number" ]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::warning::pr_number artifact missing — skipping report"
exit 0
fi
echo "pr-number=$PR_NUMBER" >> "$GITHUB_OUTPUT"
echo "quality=$(cat quality-result | tr -d '[:space:]')" >> "$GITHUB_OUTPUT"
echo "tests=$(cat tests-result | tr -d '[:space:]')" >> "$GITHUB_OUTPUT"
# Validate PR number is a positive integer (artifact comes from
# untrusted fork code, so treat contents defensively).
PR_NUM=$(cat "$DIR/pr_number" | tr -d '[:space:]')
if ! [[ "$PR_NUM" =~ ^[0-9]+$ ]]; then
echo "skip=true" >> "$GITHUB_OUTPUT"
echo "::error::Invalid PR number in artifact: '$PR_NUM'"
exit 0
fi
- name: Download test reports
id: download-test-reports
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
echo "skip=false" >> "$GITHUB_OUTPUT"
echo "pr_number=$PR_NUM" >> "$GITHUB_OUTPUT"
# Validate job-result strings against known GitHub Actions values.
# Artifact contents come from the PR workflow (potentially untrusted
# fork code), so we whitelist to prevent newline injection into
# GITHUB_OUTPUT.
validate_result() {
local val
val=$(cat "$1" | tr -d '[:space:]')
case "$val" in
success|failure|cancelled|skipped) echo "$val" ;;
*) echo "unknown" ;;
esac
}
echo "quality=$(validate_result "$DIR/quality_result")" >> "$GITHUB_OUTPUT"
echo "tests=$(validate_result "$DIR/tests_result")" >> "$GITHUB_OUTPUT"
echo "e2e=$(validate_result "$DIR/e2e_result")" >> "$GITHUB_OUTPUT"
- name: Checkout (for vitest config)
if: steps.meta.outputs.skip != 'true'
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
script: |
const fs = require('fs');
const path = require('path');
const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
});
const reports = artifacts.data.artifacts.find(a => a.name === 'test-reports');
if (!reports) {
core.warning('test-reports artifact not found');
return;
}
const zip = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: reports.id,
archive_format: 'zip',
});
const dest = path.join(process.env.RUNNER_TEMP, 'test-reports');
fs.mkdirSync(dest, { recursive: true });
fs.writeFileSync(path.join(dest, 'test-reports.zip'), Buffer.from(zip.data));
- name: Extract test reports
if: steps.download-test-reports.outcome == 'success'
shell: bash
run: |
cd "$RUNNER_TEMP/test-reports"
unzip -o test-reports.zip || true
- name: Fetch cross-platform job results
id: jobs
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
with:
script: |
const jobs = await github.rest.actions.listJobsForWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
per_page: 50,
});
const results = {};
for (const job of jobs.data.jobs) {
if (job.name.includes('ubuntu')) results.ubuntu = job.conclusion || 'pending';
else if (job.name.includes('windows')) results.windows = job.conclusion || 'pending';
else if (job.name.includes('macos')) results.macos = job.conclusion || 'pending';
}
core.setOutput('ubuntu', results.ubuntu || 'unknown');
core.setOutput('windows', results.windows || 'unknown');
core.setOutput('macos', results.macos || 'unknown');
sparse-checkout: gitnexus/vitest.config.ts
sparse-checkout-cone-mode: false
# ── Fetch base branch coverage for delta reporting ───────────
- name: Fetch base branch coverage
if: steps.meta.outputs.skip != 'true'
id: base-coverage
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
with:
@@ -134,6 +128,7 @@ jobs:
const fs = require('fs');
const path = require('path');
// Find the latest successful CI run on main
const runs = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
@@ -145,6 +140,7 @@ jobs:
if (runs.data.workflow_runs.length === 0) {
core.setOutput('found', 'false');
core.info('No successful main branch CI runs found');
return;
}
@@ -158,6 +154,7 @@ jobs:
const testReports = artifacts.data.artifacts.find(a => a.name === 'test-reports');
if (!testReports) {
core.setOutput('found', 'false');
core.info('No test-reports artifact on main branch');
return;
}
@@ -175,174 +172,242 @@ jobs:
core.setOutput('dir', dest);
- name: Extract base coverage
if: steps.base-coverage.outputs.found == 'true'
if: steps.meta.outputs.skip != 'true' && steps.base-coverage.outputs.found == 'true'
shell: bash
run: |
cd "${{ steps.base-coverage.outputs.dir }}"
mkdir -p base
unzip -o base.zip -d base
- name: Build and post report
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
- name: Build report
if: steps.meta.outputs.skip != 'true'
id: report
shell: bash
env:
PR_NUMBER: ${{ steps.meta.outputs.pr-number }}
QUALITY: ${{ steps.meta.outputs.quality }}
TESTS: ${{ steps.meta.outputs.tests }}
UBUNTU: ${{ steps.jobs.outputs.ubuntu }}
WINDOWS: ${{ steps.jobs.outputs.windows }}
MACOS: ${{ steps.jobs.outputs.macos }}
E2E: ${{ steps.meta.outputs.e2e }}
BASE_FOUND: ${{ steps.base-coverage.outputs.found }}
BASE_DIR: ${{ steps.base-coverage.outputs.dir }}
RUN_ID: ${{ github.event.workflow_run.id }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
RUN_URL: ${{ github.event.workflow_run.html_url }}
run: |
DIR="$RUNNER_TEMP/artifacts"
# ── Helper: read coverage summary into prefixed vars ──
read_cov() {
local prefix=$1 file=$2
if [ -n "$file" ] && [ -f "$file" ]; then
local val
val=$(jq -r '.total.statements.pct // "N/A"' "$file" 2>/dev/null) || val="N/A"
printf -v "${prefix}_STMTS" '%s' "$val"
val=$(jq -r '.total.branches.pct // "N/A"' "$file" 2>/dev/null) || val="N/A"
printf -v "${prefix}_BRANCH" '%s' "$val"
val=$(jq -r '.total.functions.pct // "N/A"' "$file" 2>/dev/null) || val="N/A"
printf -v "${prefix}_FUNCS" '%s' "$val"
val=$(jq -r '.total.lines.pct // "N/A"' "$file" 2>/dev/null) || val="N/A"
printf -v "${prefix}_LINES" '%s' "$val"
val=$(jq -r '"\(.total.statements.covered)/\(.total.statements.total)"' "$file" 2>/dev/null) || val=""
printf -v "${prefix}_STMTS_COV" '%s' "$val"
val=$(jq -r '"\(.total.branches.covered)/\(.total.branches.total)"' "$file" 2>/dev/null) || val=""
printf -v "${prefix}_BRANCH_COV" '%s' "$val"
val=$(jq -r '"\(.total.functions.covered)/\(.total.functions.total)"' "$file" 2>/dev/null) || val=""
printf -v "${prefix}_FUNCS_COV" '%s' "$val"
val=$(jq -r '"\(.total.lines.covered)/\(.total.lines.total)"' "$file" 2>/dev/null) || val=""
printf -v "${prefix}_LINES_COV" '%s' "$val"
return 0
else
printf -v "${prefix}_STMTS" '%s' "N/A"
printf -v "${prefix}_BRANCH" '%s' "N/A"
printf -v "${prefix}_FUNCS" '%s' "N/A"
printf -v "${prefix}_LINES" '%s' "N/A"
printf -v "${prefix}_STMTS_COV" '%s' ""
printf -v "${prefix}_BRANCH_COV" '%s' ""
printf -v "${prefix}_FUNCS_COV" '%s' ""
printf -v "${prefix}_LINES_COV" '%s' ""
return 1
fi
}
# ── Read coverage reports ──
UNIT_SUMMARY=$(find "$DIR/test-reports" -name "coverage-summary.json" -type f 2>/dev/null | head -1)
read_cov "U" "$UNIT_SUMMARY"
# ── Read base branch coverage (main) ──
BASE_SUMMARY=""
if [ "$BASE_FOUND" = "true" ] && [ -n "$BASE_DIR" ]; then
BASE_SUMMARY=$(find "$BASE_DIR/base" -name "coverage-summary.json" -type f 2>/dev/null | head -1)
fi
read_cov "B" "$BASE_SUMMARY"
# ── Locate test results ──
RESULTS_FILE=$(find "$DIR/test-reports" -name "test-results.json" -type f 2>/dev/null | head -1)
WEB_RESULTS_FILE=$(find "$DIR/test-reports" -name "web-test-results.json" -type f 2>/dev/null | head -1)
sum_results() {
local file=$1
if [ -n "$file" ] && [ -f "$file" ]; then
jq -r '"\(.numTotalTests) \(.numPassedTests) \(.numFailedTests) \(.numPendingTests) \(.numTotalTestSuites) \(((.testResults | map(.endTime) | max) - (.startTime)) / 1000 | floor)"' "$file" 2>/dev/null || echo "0 0 0 0 0 0"
else
echo "0 0 0 0 0 0"
fi
}
read CLI_T CLI_P CLI_F CLI_S CLI_SU CLI_D <<< "$(sum_results "$RESULTS_FILE")"
read WEB_T WEB_P WEB_F WEB_S WEB_SU WEB_D <<< "$(sum_results "$WEB_RESULTS_FILE")"
TOTAL=$((CLI_T + WEB_T))
PASSED=$((CLI_P + WEB_P))
FAILED=$((CLI_F + WEB_F))
SKIPPED=$((CLI_S + WEB_S))
SUITES=$((CLI_SU + WEB_SU))
DURATION=$((CLI_D > WEB_D ? CLI_D : WEB_D))
# ── Status helpers ──
status_icon() {
case "$1" in
success) echo "✅" ;;
failure) echo "❌" ;;
cancelled) echo "⏭️" ;;
*) echo "❓" ;;
esac
}
# Validate a value looks like a number (integer or decimal, optional
# leading minus). Returns 1 for anything else — guards against awk
# injection when artifact values come from untrusted fork code.
is_numeric() { [[ "$1" =~ ^-?[0-9]+(\.[0-9]+)?$ ]]; }
cov_delta() {
local pct=$1 base=$2
if [ "$pct" = "N/A" ] || [ "$base" = "N/A" ]; then echo "—"; return; fi
if ! is_numeric "$pct" || ! is_numeric "$base"; then echo "—"; return; fi
local diff
diff=$(awk -v p="$pct" -v b="$base" 'BEGIN { printf "%.1f", p - b }')
if [ "$(awk -v p="$pct" -v b="$base" 'BEGIN { print (p > b) ? 1 : 0 }')" = "1" ]; then
echo "📈 +${diff}"
elif [ "$(awk -v p="$pct" -v b="$base" 'BEGIN { print (p < b) ? 1 : 0 }')" = "1" ]; then
echo "📉 ${diff}"
else
echo "= ${diff}"
fi
}
cov_bar() {
local pct=$1 base=$2
if [ "$pct" = "N/A" ] || ! is_numeric "$pct"; then echo "—"; return; fi
local filled
filled=$(awk -v p="$pct" 'BEGIN { printf "%d", p / 5 }')
(( filled < 0 )) && filled=0
(( filled > 20 )) && filled=20
local empty=$((20 - filled))
local bar=""
for ((i=0; i<filled; i++)); do bar+="█"; done
for ((i=0; i<empty; i++)); do bar+="░"; done
# Green if >= base (or base unavailable), red if dropped
if [ "$base" = "N/A" ] || ! is_numeric "$base" || [ "$(awk -v p="$pct" -v b="$base" 'BEGIN { print (p >= b) ? 1 : 0 }')" = "1" ]; then
echo "🟢 ${bar}"
else
echo "🔴 ${bar}"
fi
}
# ── Overall status ──
if [[ "$QUALITY" == "success" && "$TESTS" == "success" && ("$E2E" == "success" || "$E2E" == "skipped") ]]; then
OVERALL="✅ **All checks passed**"
else
OVERALL="❌ **Some checks failed**"
fi
# ── Build markdown ──
{
echo "body<<GITNEXUS_CI_REPORT_EOF_7f3a"
echo "## CI Report"
echo ""
echo "${OVERALL}"
echo ""
echo "### Pipeline Status"
echo ""
echo "| Stage | Status | Details |"
echo "|-------|--------|---------|"
echo "| $(status_icon "$QUALITY") Typecheck | \`${QUALITY}\` | tsc --noEmit |"
echo "| $(status_icon "$TESTS") Tests | \`${TESTS}\` | unit tests, 3 platforms |"
echo "| $(status_icon "$E2E") E2E | \`${E2E}\` | gitnexus-web changes only |"
echo ""
if [ "$TOTAL" -gt 0 ] 2>/dev/null; then
echo "### Test Results"
echo ""
echo "| Tests | Passed | Failed | Skipped | Duration |"
echo "|-------|--------|--------|---------|----------|"
echo "| ${TOTAL} | ${PASSED} | ${FAILED} | ${SKIPPED} | ${DURATION}s |"
echo ""
if [ "$FAILED" = "0" ]; then
echo "✅ All **${PASSED}** tests passed"
else
echo "❌ **${FAILED}** failed / **${PASSED}** passed"
fi
if [ "$SKIPPED" != "0" ]; then
echo ""
echo "<details>"
echo "<summary>${SKIPPED} test(s) skipped — expand for details</summary>"
echo ""
for rf in "$RESULTS_FILE" "$WEB_RESULTS_FILE"; do
if [ -n "$rf" ] && [ -f "$rf" ]; then
jq -r '
.testResults[]
| .assertionResults[]?
| select(.status == "pending" or .status == "skipped")
| "- \(.ancestorTitles | join(" > ")) > \(.title)"
' "$rf" 2>/dev/null || true
fi
done
echo ""
echo "</details>"
fi
echo ""
fi
# ── Coverage table helper ──
cov_table() {
local label=$1 s=$2 b=$3 f=$4 l=$5 sc=$6 bc=$7 fc=$8 lc=$9
shift 9
local bs=$1 bb=$2 bf=$3 bl=$4
echo "#### ${label}"
echo ""
echo "| Metric | Coverage | Covered | Base | Delta | Status |"
echo "|--------|----------|---------|------|-------|--------|"
echo "| Statements | **${s}%** | ${sc} | ${bs}% | $(cov_delta "$s" "$bs") | $(cov_bar "$s" "$bs") |"
echo "| Branches | **${b}%** | ${bc} | ${bb}% | $(cov_delta "$b" "$bb") | $(cov_bar "$b" "$bb") |"
echo "| Functions | **${f}%** | ${fc} | ${bf}% | $(cov_delta "$f" "$bf") | $(cov_bar "$f" "$bf") |"
echo "| Lines | **${l}%** | ${lc} | ${bl}% | $(cov_delta "$l" "$bl") | $(cov_bar "$l" "$bl") |"
echo ""
}
if [ "$U_STMTS" != "N/A" ]; then
echo "### Code Coverage"
echo ""
cov_table "Tests" \
"$U_STMTS" "$U_BRANCH" "$U_FUNCS" "$U_LINES" \
"$U_STMTS_COV" "$U_BRANCH_COV" "$U_FUNCS_COV" "$U_LINES_COV" \
"$B_STMTS" "$B_BRANCH" "$B_FUNCS" "$B_LINES"
else
echo "### Code Coverage"
echo ""
echo "⚠️ Coverage data unavailable - check the [unit test job](${RUN_URL}) for details."
echo ""
fi
echo "---"
echo "<sub>📋 [View full run](${RUN_URL}) · Generated by CI</sub>"
echo "GITNEXUS_CI_REPORT_EOF_7f3a"
} >> "$GITHUB_OUTPUT"
- name: Comment on PR
if: steps.meta.outputs.skip != 'true'
uses: marocchino/sticky-pull-request-comment@773744901bac0e8cbb5a0dc842800d45e9b2b405 # v2
with:
script: |
const fs = require('fs');
const path = require('path');
const icon = (s) => ({ success: '✅', failure: '❌', cancelled: '⏭️' }[s] || '❓');
const temp = process.env.RUNNER_TEMP;
// ── Read coverage ──
function readCov(dir) {
const out = { stmts: 'N/A', branch: 'N/A', funcs: 'N/A', lines: 'N/A',
stmtsCov: '', branchCov: '', funcsCov: '', linesCov: '' };
try {
const files = require('child_process')
.execSync(`find "${dir}" -name coverage-summary.json -type f`, { encoding: 'utf8' })
.trim().split('\n').filter(Boolean);
if (!files.length) return out;
const d = JSON.parse(fs.readFileSync(files[0], 'utf8')).total;
out.stmts = d.statements.pct; out.branch = d.branches.pct;
out.funcs = d.functions.pct; out.lines = d.lines.pct;
out.stmtsCov = `${d.statements.covered}/${d.statements.total}`;
out.branchCov = `${d.branches.covered}/${d.branches.total}`;
out.funcsCov = `${d.functions.covered}/${d.functions.total}`;
out.linesCov = `${d.lines.covered}/${d.lines.total}`;
} catch {}
return out;
}
const cov = readCov(path.join(temp, 'test-reports'));
const base = process.env.BASE_FOUND === 'true'
? readCov(path.join(process.env.BASE_DIR, 'base'))
: { stmts: 'N/A', branch: 'N/A', funcs: 'N/A', lines: 'N/A' };
// ── Read test results ──
let total = 0, passed = 0, failed = 0, skipped = 0, suites = 0, duration = '0s';
let skippedTests = [];
try {
const files = require('child_process')
.execSync(`find "${path.join(temp, 'test-reports')}" -name test-results.json -type f`, { encoding: 'utf8' })
.trim().split('\n').filter(Boolean);
if (files.length) {
const r = JSON.parse(fs.readFileSync(files[0], 'utf8'));
total = r.numTotalTests || 0;
passed = r.numPassedTests || 0;
failed = r.numFailedTests || 0;
skipped = r.numPendingTests || 0;
suites = r.numTotalTestSuites || 0;
const durS = Math.floor((Math.max(...r.testResults.map(t => t.endTime)) - r.startTime) / 1000);
duration = durS >= 60 ? `${Math.floor(durS / 60)}m ${durS % 60}s` : `${durS}s`;
// Collect skipped test names
for (const suite of r.testResults) {
for (const t of (suite.assertionResults || [])) {
if (t.status === 'pending' || t.status === 'skipped') {
skippedTests.push(`- ${t.ancestorTitles.join(' > ')} > ${t.title}`);
}
}
}
}
} catch {}
// ── Coverage delta ──
function delta(pct, basePct) {
if (pct === 'N/A' || basePct === 'N/A') return '—';
const d = (pct - basePct).toFixed(1);
const dNum = parseFloat(d);
if (dNum > 0) return `📈 +${d}%`;
if (dNum < 0) return `📉 ${d}%`;
return '=';
}
// ── Build markdown ──
const { PR_NUMBER, QUALITY, TESTS, UBUNTU, WINDOWS, MACOS, RUN_ID, HEAD_SHA } = process.env;
const prNumber = parseInt(PR_NUMBER, 10);
const overall = (QUALITY === 'success' && TESTS === 'success')
? '✅ **All checks passed**' : '❌ **Some checks failed**';
const sha = HEAD_SHA.slice(0, 7);
let body = `## CI Report\n\n${overall} &ensp; \`${sha}\`\n\n`;
body += `### Pipeline\n\n`;
body += `| Stage | Status | Ubuntu | Windows | macOS |\n`;
body += `|-------|--------|--------|---------|-------|\n`;
body += `| Typecheck | ${icon(QUALITY)} \`${QUALITY}\` | — | — | — |\n`;
body += `| Tests | ${icon(TESTS)} \`${TESTS}\` | ${icon(UBUNTU)} | ${icon(WINDOWS)} | ${icon(MACOS)} |\n\n`;
if (total > 0) {
body += `### Tests\n\n`;
body += `| Metric | Value |\n|--------|-------|\n`;
body += `| Total | **${total}** |\n`;
body += `| Passed | **${passed}** |\n`;
if (failed > 0) body += `| Failed | **${failed}** |\n`;
if (skipped > 0) body += `| Skipped | ${skipped} |\n`;
body += `| Files | ${suites} |\n`;
body += `| Duration | ${duration} |\n\n`;
if (failed === 0) {
body += `✅ All **${passed}** tests passed across **${suites}** files\n`;
} else {
body += `❌ **${failed}** failed / **${passed}** passed\n`;
}
if (skippedTests.length > 0) {
body += `\n<details>\n<summary>${skipped} test(s) skipped</summary>\n\n`;
body += skippedTests.join('\n') + '\n\n</details>\n';
}
body += '\n';
}
if (cov.stmts !== 'N/A') {
body += `### Coverage\n\n`;
body += `| Metric | Coverage | Covered | Base (main) | Delta |\n`;
body += `|--------|----------|---------|-------------|-------|\n`;
body += `| Statements | **${cov.stmts}%** | ${cov.stmtsCov} | ${base.stmts}% | ${delta(cov.stmts, base.stmts)} |\n`;
body += `| Branches | **${cov.branch}%** | ${cov.branchCov} | ${base.branch}% | ${delta(cov.branch, base.branch)} |\n`;
body += `| Functions | **${cov.funcs}%** | ${cov.funcsCov} | ${base.funcs}% | ${delta(cov.funcs, base.funcs)} |\n`;
body += `| Lines | **${cov.lines}%** | ${cov.linesCov} | ${base.lines}% | ${delta(cov.lines, base.lines)} |\n\n`;
} else {
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${RUN_ID}`;
body += `### Coverage\n\n⚠️ Coverage data unavailable — check the [test job](${runUrl}) for details.\n\n`;
}
const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${RUN_ID}`;
body += `---\n<sub>📋 [Full run](${runUrl}) · Coverage from Ubuntu · Generated by CI</sub>`;
// ── Post sticky comment ──
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
per_page: 100,
direction: 'desc',
});
const marker = '<!-- ci-report -->';
const existing = comments.find(c => c.body?.includes(marker));
const fullBody = marker + '\n' + body;
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body: fullBody,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body: fullBody,
});
}
header: ci-report
number: ${{ steps.meta.outputs.pr_number }}
message: ${{ steps.report.outputs.body }}
+13
View File
@@ -28,6 +28,18 @@ jobs:
--coverage.reportOnFailure=true
working-directory: gitnexus
- name: Install gitnexus-web dependencies
run: npm ci
working-directory: gitnexus-web
- name: Run gitnexus-web unit tests
run: >-
npx vitest run
--reporter=default
--reporter=json
--outputFile=web-test-results.json
working-directory: gitnexus-web
- name: Upload test reports
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
@@ -37,6 +49,7 @@ jobs:
gitnexus/coverage/coverage-summary.json
gitnexus/coverage/coverage-final.json
gitnexus/test-results.json
gitnexus-web/web-test-results.json
retention-days: 5
cross-platform:
+62 -35
View File
@@ -16,8 +16,10 @@ concurrency:
# ── Reusable workflow orchestration ─────────────────────────────────
# Each concern lives in its own workflow file for maintainability:
# ci-quality.yml — typecheck (tsc --noEmit)
# ci-tests.yml — all tests with coverage (ubuntu) + cross-platform
# ci-report.yml — PR comment (workflow_run trigger for fork write access)
# ci-tests.yml — unit + integration tests with coverage + cross-platform
# ci-e2e.yml — E2E tests (only when gitnexus-web/ changes)
#
# Shared setup is DRY via .github/actions/setup-gitnexus composite action.
jobs:
quality:
@@ -30,11 +32,59 @@ jobs:
permissions:
contents: read
e2e:
uses: ./.github/workflows/ci-e2e.yml
permissions:
contents: read
# ── Save PR metadata for the reporting workflow ─────────────────
# The ci-report.yml workflow (triggered by workflow_run) needs the
# PR number and job results to post a comment. We save them as an
# artifact because workflow_run context doesn't reliably carry PR
# info for fork PRs.
save-pr-meta:
name: Save PR Metadata
if: always() && github.event_name == 'pull_request'
needs: [quality, tests, e2e]
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Write metadata
shell: bash
env:
PR_NUMBER: ${{ github.event.number }}
QUALITY: ${{ needs.quality.result }}
TESTS: ${{ needs.tests.result }}
E2E: ${{ needs.e2e.result }}
run: |
mkdir -p pr-meta
echo "$PR_NUMBER" > pr-meta/pr_number
echo "$QUALITY" > pr-meta/quality_result
echo "$TESTS" > pr-meta/tests_result
echo "$E2E" > pr-meta/e2e_result
# TODO(post-merge): remove backward-compat copies once ci-report.yml
# on main reads underscore names.
# Backward-compat: ci-report.yml on main still reads hyphenated
# names. workflow_run always executes from the default branch, so
# the main-branch reader won't find the underscore variants until
# this PR is merged. Write both until then.
cp pr-meta/pr_number pr-meta/pr-number
cp pr-meta/quality_result pr-meta/quality-result
cp pr-meta/tests_result pr-meta/tests-result
cp pr-meta/e2e_result pr-meta/e2e-result
- name: Upload PR metadata
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: pr-meta
path: pr-meta/
retention-days: 1
# ── Unified CI gate ──────────────────────────────────────────────
# Single required check for branch protection.
ci-status:
name: CI Gate
needs: [quality, tests]
needs: [quality, tests, e2e]
if: always()
runs-on: ubuntu-latest
timeout-minutes: 5
@@ -44,40 +94,17 @@ jobs:
env:
QUALITY: ${{ needs.quality.result }}
TESTS: ${{ needs.tests.result }}
E2E: ${{ needs.e2e.result }}
run: |
echo "Quality: $QUALITY"
echo "Tests: $TESTS"
echo "Quality: $QUALITY"
echo "Tests: $TESTS"
echo "E2E: $E2E"
if [[ "$QUALITY" != "success" ]] ||
[[ "$TESTS" != "success" ]]; then
echo "::error::One or more CI jobs failed"
echo "::error::Quality or test jobs failed"
exit 1
fi
if [[ "$E2E" != "success" && "$E2E" != "skipped" ]]; then
echo "::error::E2E job failed"
exit 1
fi
# ── PR metadata for ci-report.yml ────────────────────────────────
# Saves PR number and job results so the workflow_run-triggered
# report can post comments with a write token (works for forks).
save-pr-meta:
name: Save PR Metadata
if: always() && github.event_name == 'pull_request'
needs: [quality, tests]
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Write PR metadata
shell: bash
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
QUALITY: ${{ needs.quality.result }}
TESTS: ${{ needs.tests.result }}
run: |
mkdir -p pr-meta
echo "$PR_NUMBER" > pr-meta/pr-number
echo "$QUALITY" > pr-meta/quality-result
echo "$TESTS" > pr-meta/tests-result
- name: Upload PR metadata
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: pr-meta
path: pr-meta/
retention-days: 1
+23 -1
View File
@@ -33,6 +33,8 @@ coverage/
# Misc
*.local
HANDOFF.md
HANDOFF*.md
.vercel
@@ -57,6 +59,14 @@ assets/
# Generated files (should not be indexed)
repomix-output*
# Playwright artifacts
gitnexus-web/playwright-report/
gitnexus-web/test-results/
# Python test artifacts
eval/.coverage
eval/.hypothesis/
# Design docs (local only)
docs/plans/
@@ -71,4 +81,16 @@ GitNexus.sln
# Git worktrees
.worktrees/
/github/scripts/triage/__pycache__/
/github/scripts/triage/__pycache__/
.claude-flow/
.claude/agents/
.claude/commands/
.claude/helpers
.claude/skills/
!.claude/skills/gitnexus/
.history/
.swarm/
+104 -1
View File
@@ -1,7 +1,69 @@
<!-- version: 1.2.0 -->
<!--
Metadata: version, last reviewed, scope, model policy, reference docs, changelog.
Last updated: 2026-03-22
-->
Last reviewed: 2026-03-24
**Project:** GitNexus · **Environment:** dev · **Maintainer:** repository maintainers (see GitHub)
This file uses a standard agent header (version, scope, model policy, reference docs, changelog), adapted for this **TypeScript/JavaScript monorepo**.
## Scope
| | |
|--|--|
| **Reads** | Repository tree as needed for the task: `gitnexus/`, `gitnexus-web/`, `eval/`, plugin packages, `.github/`, `.gitnexus/` when present, and docs. |
| **Writes** | Only paths required for the requested change; keep diffs minimal. Update lockfiles when dependencies change. |
| **Executes** | `npm`, `npx`, `node` under `gitnexus/` and `gitnexus-web/`; `uv run` for Python under `eval/` when applicable; shell utilities for documented CI/dev workflows. |
| **Off-limits** | User secrets (e.g. real `.env`), production deployment credentials, unrelated repositories, destructive git history operations without explicit human confirmation. |
## Model Configuration
- **Primary:** Pin in **Cursor** (Settings → model). Use a **named** model (e.g. GPT-5.2, Claude Sonnet 4.x). Avoid relying on **Auto** when reproducibility or audit trail matters.
- **Fallback:** As configured in Cursor or your organization (do not encode `latest` or wildcards in automation configs).
- **Notes:** The open-source GitNexus CLI indexer does not call an LLM. Optional Nexus AI in the web UI uses end-user provider keys and models.
## Execution Sequence (complex tasks)
Long sessions dilute instructions. For **multi-step** work, state up front:
1. Which rules in this file and **[GUARDRAILS.md](GUARDRAILS.md)** apply (and any relevant Signs).
2. Current **Scope** boundaries (Reads / Writes / Off-limits).
3. Which **validation commands** you will run (e.g. `cd gitnexus && npm test`, `npx tsc --noEmit`).
On very long threads, the human may add *“Remember: apply all AGENTS.md rules”* to re-weight rule tokens against context dilution.
## Claude Code hooks
Hooks enforce gates that prompts cannot. In **Claude Code**, **PreToolUse** hooks can block tools such as `git_commit` until checks pass. Adapt to this repo: e.g. `cd gitnexus && npm test` before commit.
## Context budget (Cursor / standards)
Generic “core standards” playbooks are often long and stack-specific. For this monorepo, commands and gotchas live under **Cursor Cloud specific instructions** below and in **[CONTRIBUTING.md](CONTRIBUTING.md)**. If always-on rules grow, split domain rules into **`.cursor/rules/*.mdc`** (globs). **Cursor:** project-wide rules live in **`.cursor/index.mdc`** (YAML frontmatter with `alwaysApply: true`). **Claude Code:** optionally load a **`STANDARDS.md`** only when needed (e.g. *“When writing new code, read STANDARDS.md”*) to save context.
## Reference Documentation
- **This repository:** **[ARCHITECTURE.md](ARCHITECTURE.md)**, **[CONTRIBUTING.md](CONTRIBUTING.md)**, **[GUARDRAILS.md](GUARDRAILS.md)**.
- **Cursor:** `.cursor/index.mdc` (always-on rules); optional `.cursor/rules/*.mdc` (glob-scoped). Legacy `.cursorrules` is deprecated — see `.cursor/index.mdc`.
- **Optional local files:** `NOTES.md` (short vendor-neutral project snapshot). For handoffs, keep notes local (e.g., a scratch file outside the repo) rather than committing `HANDOFF.md`.
- **GitNexus:** skills under `.claude/skills/gitnexus/`; machine-oriented rules in the `gitnexus:start` … `gitnexus:end` block below.
## Changelog
| Date | Version | Change |
|------|---------|--------|
| 2026-03-24 | 1.2.0 | Fixed gitnexus:start block duplication (was inlined in Reference Docs bullet). |
| 2026-03-23 | 1.1.0 | Updated agent instructions (sections, references, Cursor layout). |
| 2026-03-22 | 1.0.0 | Added structured agent header and changelog. |
---
<!-- gitnexus:start -->
# GitNexus — Code Intelligence
This project is indexed by GitNexus as **GitNexus** (2298 symbols, 5501 relationships, 175 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
This project is indexed by GitNexus as **GitNexus**. Use the GitNexus MCP tools to understand code, assess impact, and navigate safely. For current symbol stats, run `npx gitnexus analyze` and inspect `.gitnexus/meta.json`.
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
@@ -99,3 +161,44 @@ To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.
| Index, status, clean, wiki CLI commands | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md` |
<!-- gitnexus:end -->
## Cursor Cloud specific instructions
### Repository structure
This is a monorepo with two main products and supporting config packages:
| Component | Path | Purpose |
|-----------|------|---------|
| **GitNexus CLI/Core** | `gitnexus/` | Main product — TypeScript CLI, indexing pipeline, MCP server. Published to npm. |
| **GitNexus Web UI** | `gitnexus-web/` | React/Vite browser app — graph explorer + AI chat. Runs entirely in WASM. |
| Claude Plugin | `gitnexus-claude-plugin/` | Static config for Claude marketplace (no build). |
| Cursor Integration | `gitnexus-cursor-integration/` | Static config for Cursor editor (no build). |
| SWE-bench Eval | `eval/` | Python evaluation harness (optional; needs Docker + LLM API keys). |
### Running services
- **CLI/Core**: `cd gitnexus && npm run dev` (tsx watch mode) or `npm run build && node dist/cli/index.js <command>`
- **Web UI**: `cd gitnexus-web && npm run dev` (Vite on port 5173)
- **Backend mode**: `cd <indexed-repo> && node /workspace/gitnexus/dist/cli/index.js serve` (HTTP API on port 3741 by default)
### Testing
**CLI / Core (`gitnexus/`)**
- **Unit tests**: `cd gitnexus && npm test` (vitest, ~2000 tests)
- **Integration tests**: `cd gitnexus && npm run test:integration` (vitest, ~1850 tests). Two LadybugDB file-locking tests (`lbug-core-adapter`, `search-core`) may fail in containerized environments due to `/tmp` locking limitations — this is a known environment issue, not a code bug.
- **TypeScript check**: `cd gitnexus && npx tsc --noEmit`
**Web UI (`gitnexus-web/`)**
- **Unit tests**: `cd gitnexus-web && npm test` (vitest, ~200 tests)
- **E2E tests**: `cd gitnexus-web && E2E=1 npx playwright test` (Playwright, 5 tests — requires `gitnexus serve` + `npm run dev` running)
- **TypeScript check**: `cd gitnexus-web && npx tsc -b --noEmit`
No separate lint command is configured; TypeScript strict checking serves as the primary static analysis.
### Gotchas
- `npm install` in `gitnexus/` triggers `prepare` (builds via `tsc`) and `postinstall` (patches tree-sitter-swift). Native tree-sitter bindings require `python3`, `make`, and `g++` to be present.
- `tree-sitter-kotlin` and `tree-sitter-swift` are optional dependencies — install warnings for these are expected and non-blocking.
- The Web UI uses `vite-plugin-wasm` and requires `Cross-Origin-Opener-Policy`/`Cross-Origin-Embedder-Policy` headers for `SharedArrayBuffer` (handled automatically by Vite dev server).
- There is no ESLint/Prettier configuration in this repo.
+66
View File
@@ -0,0 +1,66 @@
# Architecture — GitNexus
This repository is a **monorepo** with two main products: the **CLI / MCP package** (`gitnexus/`) and the **browser UI** (`gitnexus-web/`). Supporting folders ship editor integrations and plugins without changing the core graph engine.
## Repository layout
| Path | Role |
|------|------|
| `gitnexus/` | Published npm package `gitnexus`: CLI, MCP server (stdio), local HTTP API for bridge mode, ingestion pipeline, LadybugDB graph, embeddings (optional). |
| `gitnexus-web/` | Vite + React UI: in-browser indexing (WASM), graph visualization, optional connection to `gitnexus serve`. |
| `.claude/`, `gitnexus-claude-plugin/`, `gitnexus-cursor-integration/` | Packaged **skills** and plugin metadata so agents discover the same workflows as documented in `AGENTS.md`. |
| `eval/` | Evaluation harnesses and docs for benchmarking tool usage. |
| `.github/` | CI workflows (quality, unit, integration, E2E) and composite actions. |
## End-to-end flow: index → graph → tools
1. **Ingestion** (`gitnexus analyze`)
- Entry: `gitnexus/src/cli/analyze.ts` → `runPipelineFromRepo` in `gitnexus/src/core/ingestion/pipeline.ts`.
- Walks the git working tree, parses supported languages via **Tree-sitter**, resolves imports/calls/inheritance, detects **communities** and **processes** (execution flows), and builds an in-memory **knowledge graph** (`gitnexus/src/core/graph/`).
- Output is loaded into **LadybugDB** under **`.gitnexus/`** at the repo root (`lbug/`, `meta.json`, etc.). Optional **FTS** indexes and **embeddings** attach to the same store.
- The repo is registered in **`~/.gitnexus/registry.json`** so MCP can find it from any working directory.
2. **Persistence & metadata**
- `gitnexus/src/storage/repo-manager.ts` — paths, registry, cleanup of legacy Kuzu artifacts.
- `gitnexus/src/core/lbug/lbug-adapter.ts` — graph load, queries, embedding restore batches.
3. **Query & agents**
- **MCP (stdio):** `gitnexus/src/cli/mcp.ts` → `startMCPServer` → `LocalBackend` (`gitnexus/src/mcp/local/local-backend.ts`) opens registered repos and serves **tools** from `gitnexus/src/mcp/tools.ts` and **resources** from `gitnexus/src/mcp/resources.ts`.
- **Bridge HTTP:** `gitnexus/src/cli/serve.ts` → Express app in `gitnexus/src/server/api.ts` (CORS-limited) exposes REST + MCP-over-HTTP for the web UI.
- **CLI tools (no MCP):** `gitnexus query`, `context`, `impact`, `cypher` in `gitnexus/src/cli/tool.ts` call the same backend for scripts and CI.
4. **Staleness**
- `gitnexus/src/mcp/staleness.ts` compares indexed `lastCommit` to `HEAD` and surfaces hints when the graph is behind git.
## MCP tools (summary)
| Tool | Purpose |
|------|---------|
| `list_repos` | Discover indexed repositories when more than one is registered. |
| `query` | Natural-language / keyword search over the graph (hybrid BM25 + optional vectors). |
| `cypher` | Ad hoc **Cypher** against the schema (see resource `gitnexus://repo/{name}/schema`). |
| `context` | Callers, callees, processes for one symbol (with disambiguation). |
| `impact` | Blast radius (upstream/downstream) with depth and risk summary. |
| `detect_changes` | Map git diffs to affected symbols and processes. |
| `rename` | Graph-assisted rename with `dry_run` preview (`graph` vs `text_search` confidence). |
## Where to change what
| If you are changing… | Start in… |
|----------------------|-----------|
| CLI commands / flags | `gitnexus/src/cli/` (`index.ts`, per-command modules). |
| Parsing or graph construction | `gitnexus/src/core/ingestion/` (pipeline, processors, resolvers, type-extractors). |
| Graph schema / DB access | `gitnexus/src/core/lbug/` (`schema.ts`, `lbug-adapter.ts`), `gitnexus/src/mcp/core/lbug-adapter.ts` if MCP-specific. |
| MCP protocol, tools, resources | `gitnexus/src/mcp/server.ts`, `tools.ts`, `resources.ts`. |
| Search ranking | `gitnexus/src/core/search/` (BM25, hybrid fusion). |
| Embeddings | `gitnexus/src/core/embeddings/`, phases in `analyze.ts`. |
| Wiki generation | `gitnexus/src/core/wiki/`. |
| Web UI behavior | `gitnexus-web/src/` (components, workers, graph client). |
| CI | `.github/workflows/*.yml`, `.github/actions/setup-gitnexus/`. |
## Related docs
- [RUNBOOK.md](RUNBOOK.md) — operational commands and recovery.
- [GUARDRAILS.md](GUARDRAILS.md) — safety boundaries for humans and agents.
- [TESTING.md](TESTING.md) — how to run tests.
- `AGENTS.md` / `CLAUDE.md` — agent workflows and tool usage expectations for **this** repo when indexed by GitNexus.
+32 -81
View File
@@ -1,101 +1,52 @@
<!-- gitnexus:start -->
# GitNexus — Code Intelligence
<!-- version: 1.2.0 -->
<!--
Metadata: version, last reviewed, scope, model policy, reference docs, changelog.
Last updated: 2026-03-22
-->
This project is indexed by GitNexus as **GitNexus** (2298 symbols, 5501 relationships, 175 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
Last reviewed: 2026-03-24
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
**Project:** GitNexus · **Environment:** dev · **Maintainer:** repository maintainers (see GitHub)
## Always Do
Follow **AGENTS.md** for the canonical rules; this file adds Claude Code–specific deltas. Cursor-specific notes live only in `AGENTS.md`.
- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`.
## Scope
## When Debugging
See the **Scope** table in [AGENTS.md](AGENTS.md) for read/write/execute/off-limits boundaries. Cursor-specific workflow notes also live only in AGENTS.md.
1. `gitnexus_query({query: "<error or symptom>"})` — find execution flows related to the issue
2. `gitnexus_context({name: "<suspect function>"})` — see all callers, callees, and process participation
3. `READ gitnexus://repo/GitNexus/process/{processName}` — trace the full execution flow step by step
4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed
## Model Configuration
## When Refactoring
- **Primary:** Pin per **Claude Code** / Anthropic org policy (explicit model id). Do not rely on an unversioned `latest` alias for governed workflows.
- **Fallback:** As configured in Claude Code (organization default or user override).
- **Notes:** The GitNexus CLI analyzer does not call an LLM.
- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`.
- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code.
- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed.
## Execution Sequence (complex tasks)
## Never Do
Same discipline as [AGENTS.md](AGENTS.md): before large multi-step work, state which **AGENTS.md** / **GUARDRAILS.md** rules apply, current **Scope**, and planned validation commands (`npm test`, `tsc`, etc.). When pausing, summarize progress in the chat or a **local** scratch file (do not add `HANDOFF.md` to the repo), then `/clear` and resume with that summary.
- NEVER edit a function, class, or method without first running `gitnexus_impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph.
- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope.
## Claude Code hooks
## Tools Quick Reference
Prefer **PreToolUse** hooks for hard gates (e.g. tests before `git_commit`). Adapt hook commands to `gitnexus/` npm scripts.
| Tool | When to use | Command |
|------|-------------|---------|
| `query` | Find code by concept | `gitnexus_query({query: "auth validation"})` |
| `context` | 360-degree view of one symbol | `gitnexus_context({name: "validateUser"})` |
| `impact` | Blast radius before editing | `gitnexus_impact({target: "X", direction: "upstream"})` |
| `detect_changes` | Pre-commit scope check | `gitnexus_detect_changes({scope: "staged"})` |
| `rename` | Safe multi-file rename | `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` |
| `cypher` | Custom graph queries | `gitnexus_cypher({query: "MATCH ..."})` |
## Context budget
## Impact Risk Levels
If always-on instructions grow, load deep conventions via conditional reads (e.g. *“When writing new code, read STANDARDS.md”*) instead of pasting long blocks here. In Cursor, prefer `.cursor/index.mdc` plus optional `.cursor/rules/*.mdc` globs (see [AGENTS.md](AGENTS.md) § Context budget).
| Depth | Meaning | Action |
|-------|---------|--------|
| d=1 | WILL BREAK — direct callers/importers | MUST update these |
| d=2 | LIKELY AFFECTED — indirect deps | Should test |
| d=3 | MAY NEED TESTING — transitive | Test if critical path |
## Reference Documentation
## Resources
- **This repository:** [AGENTS.md](AGENTS.md) (Cursor + monorepo notes), [ARCHITECTURE.md](ARCHITECTURE.md), [CONTRIBUTING.md](CONTRIBUTING.md), [GUARDRAILS.md](GUARDRAILS.md).
- **GitNexus:** `.claude/skills/gitnexus/`; MCP and indexed-repo rules live only in [AGENTS.md](AGENTS.md) (`gitnexus:start` … `gitnexus:end`). See **GitNexus rules** below.
| Resource | Use for |
|----------|---------|
| `gitnexus://repo/GitNexus/context` | Codebase overview, check index freshness |
| `gitnexus://repo/GitNexus/clusters` | All functional areas |
| `gitnexus://repo/GitNexus/processes` | All execution flows |
| `gitnexus://repo/GitNexus/process/{name}` | Step-by-step execution trace |
## Changelog
## Self-Check Before Finishing
| Date | Version | Change |
|------|---------|--------|
| 2026-03-24 | 1.2.0 | Removed duplicated gitnexus:start block and scope table; replaced with pointers to AGENTS.md. |
| 2026-03-23 | 1.1.0 | Updated agent instructions to match AGENTS.md. |
| 2026-03-22 | 1.0.0 | Added structured header and changelog. |
Before completing any code modification task, verify:
1. `gitnexus_impact` was run for all modified symbols
2. No HIGH/CRITICAL risk warnings were ignored
3. `gitnexus_detect_changes()` confirms changes match expected scope
4. All d=1 (WILL BREAK) dependents were updated
---
## Keeping the Index Fresh
## GitNexus rules
After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it:
```bash
npx gitnexus analyze
```
If the index previously included embeddings, preserve them by adding `--embeddings`:
```bash
npx gitnexus analyze --embeddings
```
To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.**
> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`.
## CLI
| Task | Read this skill file |
|------|---------------------|
| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` |
| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` |
| Rename / extract / split / refactor | `.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md` |
| Tools, resources, schema reference | `.claude/skills/gitnexus/gitnexus-guide/SKILL.md` |
| Index, status, clean, wiki CLI commands | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md` |
<!-- gitnexus:end -->
GitNexus MCP rules are in the `<!-- gitnexus:start -->` … `<!-- gitnexus:end -->` block in **[AGENTS.md](AGENTS.md)** — load that section when working with MCP tools or the graph index.
+50
View File
@@ -0,0 +1,50 @@
# Contributing to GitNexus
How to propose changes, run checks locally, and open pull requests.
## License
This project uses the [PolyForm Noncommercial License 1.0.0](https://polyformproject.org/licenses/noncommercial/1.0.0/). By contributing, you agree your contributions are licensed under the same terms unless stated otherwise.
## Where to discuss
- **Issues & feature ideas:** use [GitHub Issues](https://github.com/abhigyanpatwari/GitNexus/issues) for the upstream repo, or your fork’s tracker if you work from a fork.
- **Community:** see the Discord link in the root [README.md](README.md).
## Development setup
1. Clone the repository.
2. **CLI / MCP package:** `cd gitnexus && npm install && npm run build`
3. **Web UI (if needed):** `cd gitnexus-web && npm install`
4. Run tests as described in [TESTING.md](TESTING.md).
## Branch and pull requests
- Use short-lived branches off the default branch of the repo you are targeting.
- Prefer **conventional commits** (short prefix + description), for example:
```text
feat: add graph export option
fix: correct MCP tool schema for query
test: cover cluster merge edge case
docs: clarify analyze flags
```
- **PR title:** `[area] Short description` (e.g. `[cli] Fix index refresh race`).
- **PR description:** what changed, why, how to verify (commands), and any risk or rollback notes.
## Before you open a PR
- [ ] Tests pass for the packages you touched (`gitnexus` and/or `gitnexus-web`).
- [ ] Typecheck passes: `npx tsc --noEmit` in `gitnexus/` and `npx tsc -b --noEmit` in `gitnexus-web/`.
- [ ] No secrets, tokens, or machine-specific paths committed.
- [ ] Documentation updated if behavior or public CLI/MCP contract changes.
- [ ] Pre-commit hook runs clean (`.husky/pre-commit` — typecheck + unit tests for staged packages).
## Code review
Maintainers may request changes for correctness, tests, performance, or consistency with existing patterns. Keeping diffs focused makes review faster.
## AI-assisted contributions
If you use coding agents, follow project context files (e.g. `AGENTS.md`, `CLAUDE.md`) and avoid drive-by refactors unrelated to the issue. Prefer incremental, test-backed changes.
+88
View File
@@ -0,0 +1,88 @@
# Guardrails — GitNexus (repo + agents)
Rules for **human contributors** and **AI agents** working on this codebase or publishing artifacts. These complement `AGENTS.md` / `CLAUDE.md` (which focus on GitNexus-in-GitNexus workflows).
## Scope (typical agent session)
When automating changes in this repository, treat scope as **least privilege**:
- **Read:** Source, tests, docs, public config as needed for the task.
- **Write:** Only files required for the requested fix or feature; avoid unrelated formatting or refactors.
- **Execute:** Tests, typecheck, and documented CLI commands; do not run destructive commands on user data outside the repo without explicit approval.
- **Off-limits:** Other people’s machines, production deployments you don’t own, and credentials you didn’t receive permission to use.
Adjust explicitly if the maintainer defines a different scope for a task.
---
## Non-negotiables
1. **Never commit secrets** — API keys, tokens, `.env` with real values, private URLs, or session cookies. Use `.env.example` with placeholders only.
2. **Never rename symbols with blind find-and-replace** when working in a GitNexus-indexed project — use the **`rename` MCP tool** with **`dry_run: true` first**, then review `graph` vs `text_search` edits. (There is no separate `gitnexus rename` CLI; renaming goes through MCP or editor integration.)
3. **Run impact analysis before editing shared symbols** — use **`impact`** (upstream) for functions/classes/methods others call; do not ignore **HIGH** / **CRITICAL** risk without maintainer sign-off.
4. **Prefer `detect_changes` before commit** — confirm diffs map to expected symbols/processes when the graph is available.
5. **Preserve embeddings** — if `.gitnexus/meta.json` shows embeddings, run `npx gitnexus analyze --embeddings` when refreshing the index; plain `analyze` can drop them.
---
## Signs (recurring failure patterns)
Use this format: **Trigger → Instruction → Reason**.
Append new Signs here when the same mistake repeats (e.g. CI broken twice the same way).
### Sign: Stale graph after edits
- **Trigger:** MCP or resources warn the index is behind `HEAD`, or code search doesn’t match latest commit.
- **Instruction:** Run `npx gitnexus analyze` from the repo root (plus `--embeddings` if the project used them).
- **Reason:** Tools query LadybugDB built at last analyze; git changes are invisible until re-indexed.
### Sign: Embeddings vanished after analyze
- **Trigger:** Semantic search quality drops; `stats.embeddings` in `.gitnexus/meta.json` is 0 after a refresh.
- **Instruction:** Re-run `npx gitnexus analyze --embeddings` and confirm `meta.json` reflects stored embeddings.
- **Reason:** Embedding generation is opt-in; analyze without the flag does not preserve prior vectors.
### Sign: MCP lists no repos
- **Trigger:** MCP stderr says no indexed repos.
- **Instruction:** Run `npx gitnexus analyze` in the target repository; verify `npx gitnexus list` shows it.
- **Reason:** The MCP server discovers repos via `~/.gitnexus/registry.json`, populated by analyze.
### Sign: Wrong repo in multi-repo setups
- **Trigger:** Query/impact results clearly belong to another project.
- **Instruction:** Call `list_repos`, then pass **`repo`** on subsequent tools (or use per-workspace MCP config).
- **Reason:** Default target may be ambiguous when multiple repos are registered.
### Sign: LadybugDB lock / “database busy”
- **Trigger:** Errors opening `.gitnexus/lbug` while MCP and analyze both run.
- **Instruction:** Stop overlapping processes; one writer at a time. Retry analyze or restart MCP.
- **Reason:** Embedded DB expects single-process ownership of the store.
---
## Publishing & supply chain
- **npm:** Do not publish from unreviewed automation; follow maintainer release process. Bump version intentionally; tag releases to match `package.json`.
- **Dependencies:** Prefer minimal, auditable changes to `package.json`; run tests and CI after lockfile updates.
- **License:** This project ships under **PolyForm Noncommercial 1.0.0** — do not relicense or imply a different license in docs or metadata without maintainer approval.
---
## Escalation
Stop and ask a **human maintainer** when:
- Impact analysis shows **HIGH** / **CRITICAL** risk and the task still requires the change.
- You need to alter **CI**, **release**, or **security-sensitive** config.
- Requirements conflict (e.g. “speed up analyze” vs “must keep all embeddings on huge repo”).
- You are unsure whether data loss is acceptable (`clean`, forced migrations, schema changes).
---
## Related docs
- [ARCHITECTURE.md](ARCHITECTURE.md) — components and data flow.
- [RUNBOOK.md](RUNBOOK.md) — commands for recovery.
- [CONTRIBUTING.md](CONTRIBUTING.md) — PR and commit expectations.
+14 -1
View File
@@ -59,6 +59,14 @@ https://github.com/user-attachments/assets/172685ba-8e54-4ea7-9ad1-e31a3398da72
---
## Development
- [ARCHITECTURE.md](ARCHITECTURE.md) — packages, index → graph → MCP flow, where to change code
- [RUNBOOK.md](RUNBOOK.md) — analyze, embeddings, stale index, MCP recovery, CI snippets
- [GUARDRAILS.md](GUARDRAILS.md) — safety rules and operational “Signs” for contributors and agents
- [CONTRIBUTING.md](CONTRIBUTING.md) — license, setup, commits, and pull requests
- [TESTING.md](TESTING.md) — test commands for `gitnexus` and `gitnexus-web`
## CLI + MCP (recommended)
The CLI indexes your repository and runs an MCP server that gives AI agents deep codebase awareness.
@@ -107,7 +115,11 @@ If you prefer manual configuration:
**Claude Code** (full support — MCP + skills + hooks):
```bash
# macOS / Linux
claude mcp add gitnexus -- npx -y gitnexus@latest mcp
# Windows
claude mcp add gitnexus -- cmd /c npx -y gitnexus@latest mcp
```
**Codex** (full support — MCP + skills):
@@ -363,6 +375,7 @@ GitNexus builds a complete knowledge graph of your codebase through a multi-phas
| Swift | — | — | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| C | — | — | ✓ | — | ✓ | ✓ | — | ✓ | ✓ |
| C++ | — | — | ✓ | ✓ | ✓ | ✓ | — | ✓ | ✓ |
| Dart | ✓ | — | ✓ | ✓ | ✓ | ✓ | — | ✓ | ✓ |
**Imports** — cross-file import resolution · **Named Bindings** — `import { X as Y }` / re-export tracking · **Exports** — public/exported symbol detection · **Heritage** — class inheritance, interfaces, mixins · **Type Annotations** — explicit type extraction for receiver resolution · **Constructor Inference** — infer receiver type from constructor calls (`self`/`this` resolution included for all languages) · **Config** — language toolchain config parsing (tsconfig, go.mod, etc.) · **Frameworks** — AST-based framework pattern detection · **Entry Points** — entry point scoring heuristics
@@ -527,7 +540,7 @@ The wiki generator reads the indexed graph structure, groups files into modules
- [X] Constructor-Inferred Type Resolution, `self`/`this` Receiver Mapping
- [X] Wiki Generation, Multi-File Rename, Git-Diff Impact Analysis
- [X] Process-Grouped Search, 360-Degree Context, Claude Code Hooks
- [X] Multi-Repo MCP, Zero-Config Setup, 13 Language Support
- [X] Multi-Repo MCP, Zero-Config Setup, 14 Language Support
- [X] Community Detection, Process Detection, Confidence Scoring
- [X] Hybrid Search, Vector Index
+163
View File
@@ -0,0 +1,163 @@
# Runbook — GitNexus
Short, copy-paste operations for **local development**, **MCP**, and **CI**. Commands assume a Unix shell; on Windows use Git Bash or equivalent paths.
## Prerequisites
- **Node.js** ≥ 20 (`gitnexus-web/package.json` `engines`).
- **Git** (analyze requires a git repository).
- From repo root, install and build the CLI package:
```bash
cd gitnexus
npm install
npm run build
```
Use `npx gitnexus …` from any path after global/published install, or `node dist/cli/index.js …` when developing from `gitnexus/` with a local build.
---
## Index out of date / “stale” tools
**Symptom:** MCP or resources warn the index is behind `HEAD`, or results don’t reflect recent commits.
**Fix (from the target repo root):**
```bash
npx gitnexus analyze
```
**Force full rebuild** (same commit but suspect corruption or changed ignore rules):
```bash
npx gitnexus analyze --force
```
**Check status:**
```bash
npx gitnexus status
```
**List what MCP knows about:**
```bash
npx gitnexus list
```
---
## Embeddings
**First time with vectors** (slower, more disk/RAM):
```bash
npx gitnexus analyze --embeddings
```
**Important:** If you already had embeddings, **always** pass `--embeddings` on later analyzes, or they can be dropped. See `stats.embeddings` in `.gitnexus/meta.json` (0 means none).
**Large repos:** Analyze may skip or limit embedding work when node counts are very high; watch CLI output.
---
## MCP: no repos / empty tools
**Symptom:** `GitNexus: No indexed repos yet` on stderr when starting MCP.
**Fix:** In each project you want indexed:
```bash
cd /path/to/repo
npx gitnexus analyze
```
Restart the editor MCP session if needed. The server **refreshes the registry lazily**; new analyzes are picked up without necessarily reinstalling MCP.
**Symptom:** Wrong repo when multiple are indexed — pass `repo` on tools or use `list_repos` first.
---
## Clean slate (corrupt or huge `.gitnexus`)
**Current repo only** (prompts for confirmation):
```bash
npx gitnexus clean
```
**Skip confirmation:**
```bash
npx gitnexus clean --force
```
**All registered repos:**
```bash
npx gitnexus clean --all --force
```
Then re-run `npx gitnexus analyze` (and `--embeddings` if you need vectors).
---
## Local bridge for the web UI
```bash
cd gitnexus
npx gitnexus serve
# default http://127.0.0.1:4747 — see serve --help for port/host
```
Use when the browser UI should talk to **local** indexed repos instead of WASM-only mode.
---
## CLI equivalents of MCP tools
Useful for debugging without an editor:
```bash
cd gitnexus
npx gitnexus query "authentication flow" --repo MyRepo
npx gitnexus context SomeSymbol --repo MyRepo
npx gitnexus impact SomeSymbol --direction upstream --repo MyRepo
npx gitnexus cypher "MATCH (n) RETURN count(n) LIMIT 1" --repo MyRepo
```
---
## CI failures (contributors)
Orchestrator: `.github/workflows/ci.yml`.
| Job | Typical local repro |
|-----|---------------------|
| **quality** | `cd gitnexus && npx tsc --noEmit` |
| **unit-tests** | `cd gitnexus && npx vitest run test/unit` |
| **integration** | `cd gitnexus && npx vitest run test/integration` (see workflow matrix for groups) |
| **e2e** | Triggered when `gitnexus-web/` changes; `cd gitnexus-web && E2E=1 npx playwright test` (requires `gitnexus serve` + `npm run dev`) |
**Note:** Pushes that touch only certain markdown paths may be skipped by `paths-ignore` in CI — see workflow file for exact patterns.
---
## Memory / analyze crashes
Analyze re-execs Node with a **large old-space heap** when needed (`analyze.ts`). If you still OOM on huge repos, close other processes, avoid `--embeddings` for a first pass, or analyze a smaller path if supported by your workflow.
---
## LadybugDB / lock errors
Only one process should open a repo’s `.gitnexus/lbug` store at a time. If MCP and a second `analyze` run conflict, stop one process, then retry `analyze` or restart MCP.
---
## Where to dig deeper
- Architecture overview: [ARCHITECTURE.md](ARCHITECTURE.md)
- Agent safety rules: [GUARDRAILS.md](GUARDRAILS.md)
- Tests: [TESTING.md](TESTING.md)
+95
View File
@@ -0,0 +1,95 @@
# Testing — GitNexus
How we structure tests and which commands to run locally and in CI.
## Packages
| Package | Path | Runner | Notes |
| -------------- | -------------- | -------- | ------------------------------ |
| CLI + MCP core | `gitnexus/` | Vitest | Primary test surface in CI |
| Web UI | `gitnexus-web/`| Vitest | Unit/component tests |
| Web UI E2E | `gitnexus-web/`| Playwright | Run when changing UI flows |
## Commands (local)
From repository root, unless noted:
**`gitnexus` (CLI / library)**
```bash
cd gitnexus
npm install
npm run build
npm test # unit: vitest run test/unit
npm run test:integration # integration suite
npm run test:all
npm run test:coverage
npx tsc --noEmit # typecheck (matches CI)
```
**`gitnexus-web`**
```bash
cd gitnexus-web
npm install
npm test # unit tests (vitest)
npx tsc -b --noEmit # typecheck (matches CI)
npm run test:coverage
npm run test:e2e # Playwright (requires gitnexus serve + npm run dev)
```
## Pre-commit hook
A husky pre-commit hook (`.husky/pre-commit`) runs automatically on every `git commit`:
- **`gitnexus-web/` files staged** → `tsc -b --noEmit` + `vitest run`
- **`gitnexus/` files staged** → `tsc --noEmit` + `vitest run --project default`
Skip with `git commit --no-verify` (use sparingly).
## Test categories
- **Unit** — Pure logic, parsers, graph/query helpers; fast; no network.
- **Integration** — Real combinations (filesystem, MCP wiring, larger pipelines) as already organized under `gitnexus/test/integration`.
- **Eval-style / golden sets** — For agent- or classification-style behavior, keep labeled inputs and expected outputs (JSON or table-driven tests) and run them in CI when relevant.
- **E2E (web)** — Critical user paths only; prefer `data-testid` attributes for stable selectors. Tests run against real backend (`gitnexus serve`) and Vite dev server.
## Performance metrics (targets)
Set targets to match team expectations, then tune to this repo’s CI reality:
| Metric | Target (initial) | Notes |
| ------------------- | ---------------- | ------------------------------------------ |
| Unit coverage | Align with CI | CI runs Vitest with coverage in `gitnexus` |
| Unit wall time | Fast PR feedback | Use `vitest run test/unit` for tight loop |
| Integration duration| &lt; few minutes | Guard heavy tests with env flags if needed |
## Regression testing
Re-run the full relevant suite when:
- Prompt or agent-behavior documentation changes (if tests encode behavior)
- Model or embedding-related code paths change
- Graph schema, query contracts, or MCP tool shapes change
- Dependencies with parsing or runtime impact upgrade
## CI integration
GitHub Actions (`.github/workflows/ci.yml`) orchestrate:
- **`ci-quality.yml`** — `tsc --noEmit` for `gitnexus/` + `tsc -b --noEmit` for `gitnexus-web/`
- **`ci-tests.yml`** — `vitest run` with coverage (ubuntu) + cross-platform (macOS, Windows)
- **`ci-e2e.yml`** — Playwright E2E tests, gated on `gitnexus-web/**` changes
Local checks before pushing:
```bash
cd gitnexus && npx tsc --noEmit && npm test
cd ../gitnexus-web && npx tsc -b --noEmit && npm test
```
Or rely on the pre-commit hook which runs these automatically for staged files.
## User acceptance / beta (optional)
For staged releases or UI betas: deploy to a staging environment, collect structured feedback, watch errors and latency, then iterate before a wider release.
+100
View File
@@ -0,0 +1,100 @@
# COBOL Code Indexing
GitNexus indexes COBOL codebases using a **regex-only extraction** strategy, bypassing tree-sitter entirely. This document explains why, how the pipeline works, and links to detailed sub-documents.
## Why Regex-Only?
The tree-sitter-cobol grammar (v0.0.1) has three critical limitations that make it unusable for production indexing:
| Issue | Impact | Severity |
|-------|--------|----------|
| External scanner hangs on ~5% of files | No timeout mechanism exists for the C scanner; the process blocks indefinitely | **Blocking** |
| Only ~15% of paragraph headers detected | Most procedure-division paragraphs are invisible to the grammar | High |
| Patch markers in cols 1-6 cause parse errors | Enterprise COBOL uses non-standard sequence area content (e.g., `mzADD`, `estero`, `#FIX`) | High |
Because the external scanner hang cannot be interrupted (there is no `setTimeoutMicros` equivalent for tree-sitter), using tree-sitter-cobol would hang the indexing pipeline on a non-trivial fraction of real-world files.
The regex-only approach provides:
- **Speed**: ~1ms per file average extraction time
- **Reliability**: zero hangs, zero crashes across 13,000+ files
- **Coverage**: captures all critical symbols -- program name, paragraphs, sections, CALL, PERFORM, COPY, data items (01-77, 88-level), file declarations, FD entries, EXEC SQL/CICS blocks, ENTRY points, and MOVE statements
## Architecture
```mermaid
flowchart TD
A[Repository Scan] --> B{File Detection}
B -->|Extension match| C[COBOL file]
B -->|GITNEXUS_COBOL_DIRS match| C
B -->|No match| Z[Skip]
C --> D{Copybook?}
D -->|Yes| E[Add to Copybook Map]
D -->|No| F[Source Program]
E --> G[COPY Expansion Engine]
F --> G
G -->|Inline copybook content| H[Expanded Source]
H --> I[Patch Marker Cleanup]
I --> J[Regex State Machine]
J --> K[Extracted Symbols]
K --> L[Graph Model Builder]
L --> M[Knowledge Graph]
subgraph "Per-Chunk Processing"
G
H
I
J
K
L
end
subgraph "Post-Processing"
M --> N[Community Detection]
M --> O[Process Detection]
M --> P[Contract Detection]
end
style J fill:#e8f5e9,stroke:#2e7d32
style G fill:#e3f2fd,stroke:#1565c0
```
## COBOL vs Tree-Sitter Languages
| Feature | COBOL (Regex) | Tree-Sitter Languages |
|---------|--------------|----------------------|
| Parser | Single-pass regex state machine | tree-sitter grammar + queries |
| Speed | ~1ms/file | ~5ms/file |
| AST available | No | Yes |
| COPY expansion | Yes (pre-processing step) | N/A |
| Deep indexing | Data items, SQL, CICS, FD, ENTRY | Type annotations, generics, etc. |
| Call extraction | PERFORM (intra-file) + CALL (cross-program) | AST-based call site detection |
| Import extraction | COPY statements | `import`/`require`/`use`/`#include` |
| Coverage | All critical symbols | Language-dependent query coverage |
| Failure mode | Never hangs | External scanner can hang (COBOL only) |
## Sub-Documents
| Document | Description |
|----------|-------------|
| [File Detection](./file-detection.md) | Extension mapping, `GITNEXUS_COBOL_DIRS`, copybook classification |
| [COPY Expansion](./copy-expansion.md) | Copybook inlining, REPLACING transformations, cycle detection |
| [Regex Extraction](./regex-extraction.md) | State machine, regex patterns, line processing |
| [Deep Indexing](./deep-indexing.md) | Data items, EXEC SQL/CICS, file declarations, FD, ENTRY, MOVE |
| [Graph Model](./graph-model.md) | COBOL-specific node types, edge types, full annotated example |
| [Performance](./performance.md) | Benchmarks, worker pool tuning, caps, troubleshooting |
## Key Source Files
| File | Purpose |
|------|---------|
| `gitnexus/src/core/ingestion/cobol-preprocessor.ts` | Patch marker cleanup + regex extraction engine |
| `gitnexus/src/core/ingestion/cobol-copy-expander.ts` | COPY statement expansion with REPLACING |
| `gitnexus/src/core/ingestion/utils.ts` | `getLanguageFromPath`, `getLanguageFromFilename` |
| `gitnexus/src/core/ingestion/pipeline.ts` | `isCobolCopybook`, `expandCobolCopies`, `detectCrossProgamContracts` |
| `gitnexus/src/core/ingestion/workers/parse-worker.ts` | `processCobolRegexOnly` -- graph model builder |
| `gitnexus/src/core/ingestion/workers/worker-pool.ts` | Configurable sub-batch size for COBOL |
+157
View File
@@ -0,0 +1,157 @@
# COBOL COPY Expansion
The COPY statement is COBOL's include mechanism -- analogous to `#include` in C or `import` in modern languages. GitNexus expands COPY statements **before** regex extraction so that symbols defined inside copybooks (data items, paragraphs, etc.) are visible in the program's extracted graph.
## Supported Syntax
### Basic COPY
```cobol
COPY CPSESP.
COPY "WORKGRID.CPY".
```
Inlines the content of the named copybook, replacing the COPY line(s).
### COPY with REPLACING
```cobol
COPY CPSESP REPLACING "ANAZI-KEY" BY "LK-KEY".
COPY CPSESP REPLACING LEADING "ESP-" BY "LK-ESP-"
LEADING "KPSESPL" BY "LK-KPSESPL".
COPY LINKAGE REPLACING TRAILING "-IN" BY "-OUT".
```
Three REPLACING types are supported:
| Type | Syntax | Behavior | Example |
| ------------ | ------------------------------------ | --------------------------------------- | -------------------------------- |
| **EXACT** | `REPLACING "OLD" BY "NEW"` | Replace exact identifier matches | `ANAZI-KEY` becomes `LK-KEY` |
| **LEADING** | `REPLACING LEADING "PFX-" BY "NEW-"` | Replace prefix on all COBOL identifiers | `ESP-NAME` becomes `LK-ESP-NAME` |
| **TRAILING** | `REPLACING TRAILING "-IN" BY "-OUT"` | Replace suffix on all COBOL identifiers | `DATA-IN` becomes `DATA-OUT` |
Multiple REPLACING clauses can appear in a single COPY statement. They are applied in order to each COBOL identifier in the copybook content.
### Multi-Line COPY
COPY statements can span multiple lines (standard COBOL continuation rules apply):
```cobol
COPY CPSESP REPLACING
- LEADING "ESP-" BY "LK-ESP-"
- LEADING "KPSESPL" BY "LK-KPSESPL".
```
Continuation lines (indicator `-` in column 7) are merged before COPY statement scanning.
## Expansion Flow
```mermaid
sequenceDiagram
participant Pipeline
participant Expander as COPY Expander
participant Resolver
participant Reader
Pipeline->>Pipeline: Identify all COBOL files
Pipeline->>Pipeline: Classify copybooks vs programs
Pipeline->>Reader: Read all copybook content upfront
Reader-->>Pipeline: Copybook content map (name -> content)
loop For each source file in chunk
Pipeline->>Expander: expandCopies(content, filePath, resolveFile, readFile)
Expander->>Expander: Merge continuation lines
Expander->>Expander: Detect COPY statements via regex
loop For each COPY statement (reverse order)
Expander->>Resolver: resolveFile(copyTarget)
Resolver-->>Expander: Copybook key or null
alt Resolved successfully
Expander->>Reader: readFile(resolvedKey)
Reader-->>Expander: Copybook content
Expander->>Expander: Apply REPLACING transformations
Expander->>Expander: Recurse for nested COPYs (depth + 1)
Expander->>Expander: Splice expanded content into output
else Not resolved
Expander->>Expander: Keep original COPY line
end
end
Expander-->>Pipeline: Expanded content + resolution metadata
Pipeline->>Pipeline: Replace file content with expanded content
end
```
The return type `CopyExpansionResult` contains `expandedContent` and `copyResolutions`. The `expansionDepth` field has been removed from the return type (it was unused by callers).
COPY statement line numbers in `CopyResolution` are 1-based (consistent with the preprocessor's line numbering). The splice operation that replaces COPY lines with expanded content adjusts for 0-based array indexing internally.
## Cycle Detection
Circular COPY references (e.g., copybook A includes copybook B which includes copybook A) are detected and handled:
1. Each expansion chain maintains a `visited` set of resolved copybook paths
2. If a copybook path is already in the visited set, the expansion is skipped
3. A `warnedCircular` set (internal to `expandCopies()`, not a parameter) deduplicates warning messages within a single file expansion
Known circular copybooks in PROJECT-NAME: `ANAZI`, `ANDIP`, `QDIPE` (self-referential includes).
## Max Depth
Nested COPY expansion is limited to **10 levels** (`DEFAULT_MAX_DEPTH`). If a COPY chain exceeds this depth, a warning is logged and the remaining COPY statements are left unexpanded.
## Max Total Expansions
A breadth amplification guard caps the total number of COPY expansions across all branches within a single file to **500** (`MAX_TOTAL_EXPANSIONS`). This prevents exponential blowup from diamond-shaped COPY graphs where N copybooks each include N other copybooks. Once the limit is reached, further COPY statements in that file are left unexpanded and a single warning is logged.
## REPLACING Application Detail
The REPLACING engine works by scanning all COBOL identifiers (matching `\b[A-Z][A-Z0-9-]*\b`) in the copybook content and applying each replacement rule:
```
Original copybook content:
05 ESP-NAME PIC X(30).
05 ESP-CODE PIC X(10).
05 KPSESPL-FLAG PIC X(01).
After REPLACING LEADING "ESP-" BY "LK-ESP-" LEADING "KPSESPL" BY "LK-KPSESPL":
05 LK-ESP-NAME PIC X(30).
05 LK-ESP-CODE PIC X(10).
05 LK-KPSESPL-FLAG PIC X(01).
```
For LEADING replacements, the engine checks if each identifier starts with the `from` prefix (case-insensitive) and replaces only the prefix portion, preserving the rest of the identifier.
For TRAILING replacements, the same logic applies to suffixes.
For EXACT replacements, only identifiers that match the `from` value exactly (case-insensitive) are replaced.
## Copybook Resolution
The resolver tries multiple strategies to match a COPY target name to a copybook file:
1. **Exact match**: `COPY CPSESP` resolves to copybook named `CPSESP`
2. **Strip extension**: `COPY WORKGRID.CPY` strips `.CPY` and resolves to `WORKGRID`
3. **Add extension**: `COPY CPSESP` tries `CPSESP.CPY` and `CPSESP.COPY`
If no match is found, the COPY statement is left in place (unexpanded) and a resolution record with `resolvedPath: null` is created.
## Pipeline Integration
The expansion runs **per chunk**, after file content is read but before dispatch to worker threads:
1. All copybook files are read upfront (they are typically small, collectively under 100MB)
2. Per chunk, the copybook map is merged with chunk content (in case a chunk contains copybooks)
3. Only programs (not copybooks themselves) undergo expansion
4. The expanded content replaces the original content in-place before worker dispatch
## Inline Comment Handling
The copy expander's `stripInlineComment()` helper is quote-aware: pipe characters (`|`) inside single- or double-quoted strings are preserved. This matches the same quote-aware logic used by the preprocessor.
## Source Files
- `gitnexus/src/core/ingestion/cobol-copy-expander.ts` -- `expandCopies()`, `parseReplacingClause()`, `applyReplacing()`
- `gitnexus/src/core/ingestion/pipeline.ts` -- `expandCobolCopies()`, copybook map construction, chunk integration
+312
View File
@@ -0,0 +1,312 @@
# COBOL Deep Indexing
Beyond basic symbol extraction (program name, paragraphs, CALL, PERFORM, COPY), GitNexus performs deep indexing of COBOL-specific constructs: data items, EXEC SQL/CICS blocks, file declarations, FD entries, ENTRY points, and MOVE statements.
## Data Items
### Level Numbers
| Level Range | Meaning | Graph Node Type |
|-------------|---------|-----------------|
| 01 | Record (group item) | `Record` |
| 02-49 | Elementary/group items | `Property` |
| 66 | RENAMES | `Property` |
| 77 | Independent item | `Property` |
| 88 | Condition name | `Const` |
FILLER items are skipped (no useful name for the graph).
### Clauses Parsed
The `parseDataItemClauses()` function extracts these clauses from the trailing text of a data item declaration:
| Clause | Pattern | Example |
|--------|---------|---------|
| `PIC` / `PICTURE` | `\bPIC(?:TURE)?\s+(?:IS\s+)?(\S+)` | `PIC X(30)`, `PICTURE IS 9(5)V99` |
| `USAGE` | `\bUSAGE\s+(?:IS\s+)?(COMP\|BINARY\|...)` | `USAGE IS COMP-3`, `BINARY` |
| `REDEFINES` | `\bREDEFINES\s+([A-Z][A-Z0-9-]+)` | `REDEFINES WK-DATE-NUM` |
| `OCCURS` | `\bOCCURS\s+(\d+)` | `OCCURS 12 TIMES` |
Standalone COMP variants (without the `USAGE` keyword) are also detected: `COMP`, `COMP-1` through `COMP-6`, `COMP-X`, `BINARY`, `PACKED-DECIMAL`.
### Data Hierarchy
Data items form a hierarchical structure based on level numbers. The extractor uses a **stack algorithm**:
```
Processing order:
01 WK-RECORD -> push {01, WK-RECORD} -> parent: Module
05 WK-NAME -> push {05, WK-NAME} -> parent: WK-RECORD (01 < 05)
10 WK-FIRST -> push {10, WK-FIRST} -> parent: WK-NAME (05 < 10)
10 WK-LAST -> pop WK-FIRST, push -> parent: WK-NAME (05 < 10)
05 WK-CODE -> pop WK-LAST, WK-NAME -> parent: WK-RECORD (01 < 05)
88 WK-ACTIVE -> (88 handled separately) -> parent: WK-CODE
```
The stack maintains items where each entry's level is strictly less than the next. When a new item arrives with a level <= the top of stack, items are popped until the stack top has a smaller level. A `CONTAINS` edge is created from the stack top to the new item.
For 88-level condition names, the parent is the immediately preceding non-88 data item (found by scanning backwards).
### Annotated Example
```cobol
01 WK-EMPLOYEE.
05 WK-EMP-ID PIC 9(6).
05 WK-EMP-NAME PIC X(30).
05 WK-EMP-STATUS PIC X(01).
88 WK-ACTIVE VALUE "A".
88 WK-INACTIVE VALUE "I".
05 WK-SALARY PIC 9(7)V99 COMP-3.
05 WK-DEPT PIC X(04) OCCURS 3 TIMES.
```
Produces:
- `Record` node: `WK-EMPLOYEE` (level 01, section: working-storage)
- `Property` nodes: `WK-EMP-ID`, `WK-EMP-NAME`, `WK-EMP-STATUS`, `WK-SALARY`, `WK-DEPT`
- `Const` nodes: `WK-ACTIVE` (values: `A`), `WK-INACTIVE` (values: `I`)
- `CONTAINS` edges: `WK-EMPLOYEE -> WK-EMP-ID`, `WK-EMPLOYEE -> WK-EMP-NAME`, etc.
- `CONTAINS` edges: `WK-EMP-STATUS -> WK-ACTIVE`, `WK-EMP-STATUS -> WK-INACTIVE`
### Data Item Cap
A maximum of **500 data items per file** (`MAX_DATA_ITEMS_PER_FILE`) are processed. Some COBOL programs (especially after COPY expansion) can have 10,000+ data items, which would cause graph bloat and push the V8 relationship Map past its 16.7M entry limit across thousands of files.
The cap applies after extraction: the first 500 items in source order are kept. Since 01-level records appear first, critical top-level structure is preserved.
## EXEC SQL
EXEC SQL blocks are accumulated across lines between `EXEC SQL` and `END-EXEC`, then parsed as a unit.
### Operation Classification
The first SQL keyword determines the operation:
| First Keyword | Operation |
|---------------|-----------|
| `SELECT` | SELECT |
| `INSERT` | INSERT |
| `UPDATE` | UPDATE |
| `DELETE` | DELETE |
| `DECLARE` | DECLARE |
| `OPEN` | OPEN |
| `CLOSE` | CLOSE |
| `FETCH` | FETCH |
| *(anything else)* | OTHER |
### Table Extraction
Tables are extracted from SQL clauses:
| Clause Pattern | Example |
|----------------|---------|
| `FROM <table>` | `SELECT * FROM EMPLOYEES` |
| `INSERT INTO <table>` | `INSERT INTO EMPLOYEES` |
| `UPDATE <table>` | `UPDATE EMPLOYEES SET ...` |
| `JOIN <table>` | `LEFT JOIN DEPARTMENTS ON ...` |
Note: The `INTO` pattern is restricted to `INSERT INTO` to avoid false positives from `FETCH ... INTO :host-var` and `SELECT ... INTO :host-var` statements, where `INTO` introduces host variables rather than table names.
### Cursor Detection
```cobol
EXEC SQL
DECLARE C-EMPLOYEES CURSOR FOR
SELECT EMP-ID, EMP-NAME FROM EMPLOYEES
WHERE DEPT = :WK-DEPT
END-EXEC
```
Extracts: cursor `C-EMPLOYEES`, table `EMPLOYEES`, host variable `WK-DEPT`.
### Host Variables
Host variables are COBOL variables referenced in SQL with a `:` prefix. The colon is stripped:
```sql
WHERE EMP-ID = :WK-EMP-ID AND DEPT = :WK-DEPT
```
Extracts: `WK-EMP-ID`, `WK-DEPT`.
### Graph Output
- `CodeElement` node per table, with description `sql-table op:{OP}`
- `CodeElement` node per cursor, with description `sql-cursor`
- `ACCESSES` edge from Module to each CodeElement
- Deduplication: if the same table appears in multiple SQL blocks, only one node is created
## EXEC CICS
EXEC CICS blocks are accumulated and parsed similarly to SQL blocks.
### Command Detection
Two-word commands are detected first (matched against the block start):
```
SEND MAP, RECEIVE MAP, SEND TEXT, SEND CONTROL, READ NEXT, READ PREV
```
If no two-word command matches, the first word is used (e.g., `LINK`, `XCTL`, `RETURN`, `READ`, `WRITE`).
### Extraction
| Element | Pattern | Example |
|---------|---------|---------|
| MAP name | `MAP('name')` or `MAP("name")` | `EXEC CICS SEND MAP('EMPMENU')` |
| PROGRAM name | `PROGRAM('name')` or `PROGRAM("name")` | `EXEC CICS LINK PROGRAM('BGTABUP')` |
| TRANSID | `TRANSID('name')` or `TRANSID("name")` | `EXEC CICS START TRANSID('EMP1')` |
### Graph Output
- MAP: `CodeElement` node with description `cics-map cmd:{CMD}` + `ACCESSES` edge from Module
- PROGRAM: `CALLS` edge (cross-program call via CICS LINK/XCTL)
- TRANSID: `CodeElement` node with description `cics-transid cmd:{CMD}` + `ACCESSES` edge from Module
### Annotated Example
```cobol
EXEC CICS
SEND MAP('EMPMENU')
MAPSET('EMPSET')
FROM(WK-MAP-DATA)
ERASE
END-EXEC
```
Produces:
- `CodeElement` node: `EMPMENU` (description: `cics-map cmd:SEND MAP`)
- `ACCESSES` edge: Module -> `EMPMENU`
## File Declarations
SELECT statements in the INPUT-OUTPUT SECTION are accumulated across multiple lines (until a period terminator) and parsed for:
| Clause | Pattern | Example |
|--------|---------|---------|
| SELECT | `SELECT <name>` | `SELECT MASTER-FILE` |
| ASSIGN | `ASSIGN TO <file>` | `ASSIGN TO "MASTER.DAT"` |
| ORGANIZATION | `ORGANIZATION IS <type>` | `ORGANIZATION IS INDEXED` |
| ACCESS | `ACCESS MODE IS <mode>` | `ACCESS MODE IS DYNAMIC` |
| RECORD KEY | `RECORD KEY IS <field>` | `RECORD KEY IS WK-EMP-ID` |
| FILE STATUS | `FILE STATUS IS <field>` | `FILE STATUS IS WK-FILE-STATUS` |
### Graph Output
- `CodeElement` node with description containing all parsed clauses (e.g., `select org:INDEXED access:DYNAMIC key:WK-EMP-ID status:WK-FILE-STATUS assign:MASTER.DAT`)
- `RECORD_KEY_OF` edge: from Property node to CodeElement (confidence 0.8)
- `FILE_STATUS_OF` edge: from Property node to CodeElement (confidence 0.8)
## FD Entries
FD (File Description) entries associate a file name with its record layout:
```cobol
FD MASTER-FILE.
01 MASTER-RECORD.
05 MR-EMP-ID PIC 9(6).
05 MR-EMP-NAME PIC X(30).
```
The extractor tracks `pendingFdName` state: when an `FD` line is seen, the next 01-level data item becomes its record.
### Graph Output
- `CodeElement` node with description `fd record:{recordName}`
- `CONTAINS` edge: FD CodeElement -> Record node
- `CONTAINS` edge: SELECT CodeElement -> FD CodeElement (linking file declaration to file description)
## ENTRY Points
The `ENTRY` statement defines additional entry points into a COBOL program (in addition to the main program entry):
```cobol
ENTRY "SUBPROG" USING WK-PARAM-1 WK-PARAM-2.
```
### Graph Output
- `Constructor` node with description `entry params:{param1},{param2}` (or just `entry` if no parameters)
- `CONTAINS` edge: Module -> Constructor
- Symbol table entry (so the entry point is discoverable by name)
## PROCEDURE DIVISION USING
```cobol
PROCEDURE DIVISION USING WK-INPUT-REC WK-OUTPUT-REC.
```
The USING clause identifies parameters received by the program from its caller.
### Graph Output
- `RECEIVES` edge: Module -> Property (for each parameter name, confidence 0.8)
## MOVE Statements
MOVE statements produce `ACCESSES` edges in the graph:
```cobol
MOVE WK-NAME TO OUT-NAME.
MOVE CORRESPONDING WK-INPUT TO WK-OUTPUT.
MOVE CORR WK-IN TO WK-OUT.
```
### Extraction Details
- Source and target identifiers are captured
- `CORRESPONDING` and its abbreviation `CORR` are both recognized (bulk field-by-field move)
- Figurative constants (SPACES, ZEROS, LOW-VALUES, HIGH-VALUES, QUOTES, ALL) are skipped
- The enclosing paragraph (`caller`) is tracked for context
### MOVE CORRESPONDING / CORR Edge Reasons
MOVE CORRESPONDING (and CORR) produces distinct edge reasons to differentiate from simple MOVE:
| Edge | Reason (simple MOVE) | Reason (CORRESPONDING/CORR) |
|------|---------------------|-----------------------------|
| Read (source) | `cobol-move-read` | `cobol-move-corresponding-read` |
| Write (target) | `cobol-move-write` | `cobol-move-corresponding-write` |
This distinction allows queries to find bulk field-by-field moves separately from simple variable assignments.
## GO TO DEPENDING ON
The `GO TO` statement with multiple targets and a `DEPENDING ON` clause is a computed branch:
```cobol
GO TO PARA-1 PARA-2 PARA-3
DEPENDING ON WK-SELECTOR.
```
All target paragraph names are extracted and emitted as separate `gotos` entries. Each target produces a `CALLS` edge in the graph (same semantics as PERFORM). The `DEPENDING ON` variable is not currently tracked as a data-flow dependency.
## SORT INPUT/OUTPUT PROCEDURE
SORT and MERGE statements can specify procedural entry points instead of file-based I/O:
```cobol
SORT SORT-FILE ON ASCENDING KEY SORT-KEY
INPUT PROCEDURE IS PREPARE-INPUT
OUTPUT PROCEDURE IS FORMAT-OUTPUT.
```
`INPUT PROCEDURE IS` and `OUTPUT PROCEDURE IS` targets are extracted as control-flow targets (same as PERFORM). They produce `performs` entries and corresponding `CALLS` edges in the graph.
## Fixed-Format Literal Continuation
In fixed-format COBOL, string literals can span multiple lines using the continuation indicator (`-` in column 7). When a continuation line starts with a quote character, the extractor joins it with the predecessor by removing the trailing quote from the previous line and the opening quote from the continuation:
```
Line N: MOVE "THIS IS A LONG STRI
Line N+1 (cont): - "NG VALUE" TO WK-FIELD.
Merged: MOVE "THIS IS A LONG STRING VALUE" TO WK-FIELD.
```
The trailing `"` on line N and the opening `"` on line N+1 are both removed, producing a seamless literal. If no matching quote is found on the predecessor line, the continuation is appended as-is.
## Source Files
- `gitnexus/src/core/ingestion/cobol-preprocessor.ts` -- All extraction logic, clause parsers, EXEC block parsers
- `gitnexus/src/core/ingestion/workers/parse-worker.ts` -- `processCobolRegexOnly()`, graph node/edge emission
- `gitnexus/src/core/ingestion/parsing-processor.ts` -- Sequential fallback with same `MAX_DATA_ITEMS_PER_FILE` cap
+126
View File
@@ -0,0 +1,126 @@
# COBOL File Detection
GitNexus detects COBOL files through two mechanisms: extension-based mapping and directory-based override for extensionless files. This document covers both, plus the copybook/program classification logic.
## Extension Mapping
### Program Extensions
| Extension | Type |
|-----------|------|
| `.cbl` | COBOL program |
| `.cob` | COBOL program |
| `.cobol` | COBOL program |
### Copybook Extensions
| Extension | Type | Notes |
|-----------|------|-------|
| `.cpy` | Copybook | Standard |
| `.copy` | Copybook | Standard |
| `.gnm` / `.GNM` | Copybook | Enterprise (GnuCOBOL naming) |
| `.fd` / `.FD` | Copybook | File Description fragment |
| `.wrk` / `.WRK` | Copybook | Working-Storage fragment |
| `.sel` / `.SEL` | Copybook | SELECT clause fragment |
| `.open` / `.OPEN` | Copybook | File OPEN fragment |
| `.close` / `.CLOSE` | Copybook | File CLOSE fragment |
| `.ini` / `.INI` | Copybook | Initialization fragment |
| `.def` / `.DEF` | Copybook | Definition fragment |
All extension matching is case-sensitive in `getLanguageFromFilename` (the extensions above are matched as written, including uppercase variants like `.GNM`).
## Extensionless File Detection: `GITNEXUS_COBOL_DIRS`
Many enterprise COBOL repositories use extensionless files -- the filename alone identifies the program (e.g., `s/BGTABFL` is the source for program `BGTABFL`). GitNexus handles this via the `GITNEXUS_COBOL_DIRS` environment variable.
### Configuration
Set `GITNEXUS_COBOL_DIRS` to a comma-separated list of directory names:
```bash
# Files in s/, c/, and wfproc/ directories (at any depth) are treated as COBOL
export GITNEXUS_COBOL_DIRS=s,c,wfproc
```
The matching is **case-insensitive** and checks all path segments:
- `/repo/s/BGTABFL` -- matches segment `s` -- COBOL
- `/repo/src/c/CPSESP` -- matches segment `c` -- COBOL
- `/repo/wfproc/WF001` -- matches segment `wfproc` -- COBOL
- `/repo/docs/README` -- no matching segment -- skipped
### Decision Tree
```mermaid
flowchart TD
A[getLanguageFromPath] --> B[getLanguageFromFilename]
B --> C{Known extension?}
C -->|Yes .cbl/.cob/.cobol/.cpy/...| D[Return COBOL]
C -->|Yes .ts/.py/.java/...| E[Return other language]
C -->|No match| F{Has extension?}
F -->|"Has dot in basename"| G[Return null]
F -->|"No dot = extensionless"| H{GITNEXUS_COBOL_DIRS set?}
H -->|No| G
H -->|Yes| I{Any path segment<br/>matches a configured dir?}
I -->|Yes| D
I -->|No| G
style D fill:#e8f5e9,stroke:#2e7d32
style G fill:#ffebee,stroke:#c62828
```
### Implementation Detail
The `GITNEXUS_COBOL_DIRS` value is parsed once (on first call) and cached in a `Set<string>`:
```typescript
// From gitnexus/src/core/ingestion/utils.ts
const getCobolDirs = (): Set<string> => {
if (_cobolDirs) return _cobolDirs;
const raw = process.env.GITNEXUS_COBOL_DIRS;
_cobolDirs = raw
? new Set(raw.split(',').map(d => d.trim().toLowerCase()))
: new Set();
return _cobolDirs;
};
```
The path segment check splits the full path on `/` and tests each segment against the cached set.
## Copybook vs Program Classification
After a file is identified as COBOL, it must be classified as either a **program** (to be parsed for symbols) or a **copybook** (to be loaded into the copybook map for COPY expansion).
### Classification Rules
A COBOL file is classified as a **copybook** if ANY of these conditions is true:
1. It has a recognized copybook extension (`.cpy`, `.copy`, `.gnm`, `.fd`, `.wrk`, `.sel`, `.open`, `.close`, `.ini`, `.def`)
2. It is an extensionless file whose path contains a directory segment matching one of: `c`, `copy`, `copybooks`, `copylib`, `cpy`
A file is classified as a **program** if:
1. It has a program extension (`.cbl`, `.cob`, `.cobol`), OR
2. It is extensionless and does NOT match any copybook directory pattern
### Copybook Name Resolution
Copybook names are derived from the filename:
- Strip the extension (if any)
- Convert to uppercase
Examples:
- `c/CPSESP` -- name: `CPSESP`
- `copy/workgrid.cpy` -- name: `WORKGRID`
- `c/ANAZI.GNM` -- name: `ANAZI`
This name is used to resolve `COPY CPSESP.` statements during expansion.
## Source Files
- `gitnexus/src/core/ingestion/utils.ts` -- `getLanguageFromPath()`, `getLanguageFromFilename()`, `getCobolDirs()`
- `gitnexus/src/core/ingestion/pipeline.ts` -- `isCobolCopybook()`, `getCopybookName()`, `COPYBOOK_EXTENSIONS`, `COBOL_PROGRAM_EXTENSIONS`
+193
View File
@@ -0,0 +1,193 @@
# COBOL Graph Model
This document describes the graph nodes and edges that GitNexus creates for COBOL codebases. The COBOL graph model is richer than most tree-sitter languages because it captures domain-specific constructs: file declarations, FD entries, data hierarchies, SQL tables, CICS maps, and cross-program contracts.
## Entity-Relationship Diagram
```mermaid
erDiagram
File ||--o{ Module : DEFINES
File ||--o{ Function : DEFINES
File ||--o{ Namespace : DEFINES
File ||--o{ Record : DEFINES
File ||--o{ Property : DEFINES
File ||--o{ Const : DEFINES
File ||--o{ CodeElement : DEFINES
File ||--o{ Constructor : DEFINES
File }o--o{ File : IMPORTS
Module ||--o{ Record : CONTAINS
Module ||--o{ Constructor : CONTAINS
Module }o--o{ CodeElement : ACCESSES
Module }o--o{ Module : CALLS
Module }o--o{ Module : CONTRACTS
Module }o--o{ Property : RECEIVES
Record ||--o{ Property : CONTAINS
Record ||--o{ Const : CONTAINS
Record }o--o{ Record : REDEFINES
Property ||--o{ Property : CONTAINS
Property ||--o{ Const : CONTAINS
Property }o--o{ Property : REDEFINES
Property }o--o{ CodeElement : RECORD_KEY_OF
Property }o--o{ CodeElement : FILE_STATUS_OF
CodeElement ||--o{ CodeElement : CONTAINS
CodeElement ||--o{ Record : CONTAINS
Function }o--o{ Function : CALLS
```
## Node Types
| Node Type | COBOL Concept | Created From | Example |
|-----------|--------------|--------------|---------|
| `Module` | PROGRAM-ID | `PROGRAM-ID. BGTABFL` | Name: `BGTABFL`, description may include author and date |
| `Function` | Paragraph | `PROCESS-RECORD.` at column 8 | Name: `PROCESS-RECORD` |
| `Namespace` | Procedure section | `MAIN-LOGIC SECTION.` at column 8 | Name: `MAIN-LOGIC` |
| `Record` | 01-level data item | `01 WK-EMPLOYEE.` | Description: `level:01 section:working-storage` |
| `Property` | 02-49/66/77 data item | `05 WK-NAME PIC X(30).` | Description: `level:05 pic:X(30) section:working-storage` |
| `Const` | 88-level condition | `88 WK-ACTIVE VALUE "A".` | Description: `level:88 values:A` |
| `CodeElement` | SELECT, FD, SQL table, CICS map, cursor, transid | Various | Description varies by subtype |
| `Constructor` | ENTRY point | `ENTRY "SUBPROG" USING WK-DATA` | Description: `entry params:WK-DATA` |
### CodeElement Subtypes
CodeElement is used for multiple COBOL constructs, distinguished by their description prefix:
| Subtype | ID Pattern | Description Format | Example |
|---------|-----------|-------------------|---------|
| File SELECT | `CodeElement:{path}:SELECT:{name}` | `select org:INDEXED access:DYNAMIC ...` | `SELECT MASTER-FILE` |
| FD entry | `CodeElement:{path}:FD:{name}` | `fd record:{recordName}` | `FD MASTER-FILE` |
| SQL table | `CodeElement:{path}:sql-table:{name}` | `sql-table op:SELECT` | Table `EMPLOYEES` |
| SQL cursor | `CodeElement:{path}:sql-cursor:{name}` | `sql-cursor` | Cursor `C-EMPLOYEES` |
| CICS map | `CodeElement:{path}:cics-map:{name}` | `cics-map cmd:SEND MAP` | Map `EMPMENU` |
| CICS transid | `CodeElement:{path}:cics-transid:{name}` | `cics-transid cmd:START` | Transid `EMP1` |
## Edge Types
| Edge Type | Source | Target | Created By | Confidence | Example |
|-----------|--------|--------|-----------|------------|---------|
| `DEFINES` | File | any node | File defines its symbols | 1.0 | File -> Module `BGTABFL` |
| `CALLS` | Function | Function | `PERFORM X [THRU Y]` | (via call-processor) | `PROCESS-RECORD` -> `CALC-TAX` |
| `CALLS` | Module | Module | `CALL "BGTABUP"` | (via call-processor) | `BGTABFL` -> `BGTABUP` |
| `CALLS` | Module | Module | `EXEC CICS LINK PROGRAM('X')` | (via call-processor) | `BGTABFL` -> `BGTABUP` |
| `IMPORTS` | File | File | `COPY copybook` | (via import-processor) | Source file -> Copybook file |
| `CONTAINS` | Module | Record | Data hierarchy root | 1.0 | `BGTABFL` -> `WK-EMPLOYEE` |
| `CONTAINS` | Record | Property | Data hierarchy | 1.0 | `WK-EMPLOYEE` -> `WK-NAME` |
| `CONTAINS` | Property | Property | Nested data items | 1.0 | `WK-ADDRESS` -> `WK-CITY` |
| `CONTAINS` | Record/Property | Const | 88-level parent | 1.0 | `WK-STATUS` -> `WK-ACTIVE` |
| `CONTAINS` | CodeElement (FD) | Record | FD record link | 1.0 | `FD:MASTER-FILE` -> `MASTER-RECORD` |
| `CONTAINS` | CodeElement (SELECT) | CodeElement (FD) | SELECT-FD link | 0.9 | `SELECT:MASTER-FILE` -> `FD:MASTER-FILE` |
| `CONTAINS` | Module | Constructor | ENTRY in module | 1.0 | `BGTABFL` -> `SUBPROG` |
| `REDEFINES` | Record | Record | `01 X REDEFINES Y` | 1.0 | `WK-DATE-NUM` -> `WK-DATE-ALPHA` |
| `REDEFINES` | Property | Property | `05 X REDEFINES Y` | 1.0 | `WK-CODE-NUM` -> `WK-CODE-ALPHA` |
| `RECORD_KEY_OF` | Property | CodeElement (SELECT) | `RECORD KEY IS field` | 0.8 | `WK-EMP-ID` -> `SELECT:MASTER-FILE` |
| `FILE_STATUS_OF` | Property | CodeElement (SELECT) | `FILE STATUS IS field` | 0.8 | `WK-FS` -> `SELECT:MASTER-FILE` |
| `ACCESSES` | Module | CodeElement | EXEC SQL/CICS | 0.9 | `BGTABFL` -> `sql-table:EMPLOYEES` |
| `RECEIVES` | Module | Property | `PROCEDURE USING` | 0.8 | `BGTABFL` -> `WK-INPUT-REC` |
| `CONTRACTS` | Module | Module | Shared copybook detection | 0.9 | `BGTABFL` -> `BGTABUP` (via `CPSESP`) |
## Full Annotated Example
Given this COBOL program:
```cobol
IDENTIFICATION DIVISION.
PROGRAM-ID. EMPMAINT.
AUTHOR. Development Team.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT EMP-FILE
ASSIGN TO "EMPLOYEE.DAT"
ORGANIZATION IS INDEXED
ACCESS MODE IS DYNAMIC
RECORD KEY IS EMP-ID
FILE STATUS IS WS-FILE-STATUS.
DATA DIVISION.
FILE SECTION.
FD EMP-FILE.
01 EMP-RECORD.
05 EMP-ID PIC 9(6).
05 EMP-NAME PIC X(30).
WORKING-STORAGE SECTION.
01 WS-FLAGS.
05 WS-FILE-STATUS PIC X(02).
05 WS-EOF-FLAG PIC X(01).
88 WS-EOF VALUE "Y".
LINKAGE SECTION.
01 LK-SEARCH-KEY PIC 9(6).
PROCEDURE DIVISION USING LK-SEARCH-KEY.
MAIN-LOGIC SECTION.
MAIN-START.
PERFORM OPEN-FILE
PERFORM PROCESS-RECORDS
PERFORM CLOSE-FILE
STOP RUN.
OPEN-FILE.
OPEN I-O EMP-FILE.
PROCESS-RECORDS.
MOVE LK-SEARCH-KEY TO EMP-ID
EXEC SQL
SELECT EMP_SALARY INTO :WS-SALARY
FROM EMPLOYEES
WHERE EMP_ID = :EMP-ID
END-EXEC
CALL "EMPREPORT".
CLOSE-FILE.
CLOSE EMP-FILE.
```
The graph produced contains:
**Nodes:**
- `Module`: EMPMAINT (description: `author:Development Team`)
- `Namespace`: MAIN-LOGIC
- `Function`: MAIN-START, OPEN-FILE, PROCESS-RECORDS, CLOSE-FILE
- `Record`: EMP-RECORD, WS-FLAGS, LK-SEARCH-KEY
- `Property`: EMP-ID, EMP-NAME, WS-FILE-STATUS, WS-EOF-FLAG
- `Const`: WS-EOF (values: Y)
- `CodeElement`: SELECT:EMP-FILE, FD:EMP-FILE, sql-table:EMPLOYEES
- (COPY imports, if any, would produce File IMPORTS edges)
**Edges:**
- `DEFINES`: File -> all nodes
- `CONTAINS`: EMPMAINT -> EMP-RECORD, EMPMAINT -> WS-FLAGS, EMPMAINT -> LK-SEARCH-KEY
- `CONTAINS`: EMP-RECORD -> EMP-ID, EMP-RECORD -> EMP-NAME
- `CONTAINS`: WS-FLAGS -> WS-FILE-STATUS, WS-FLAGS -> WS-EOF-FLAG
- `CONTAINS`: WS-EOF-FLAG -> WS-EOF
- `CONTAINS`: FD:EMP-FILE -> EMP-RECORD
- `CONTAINS`: SELECT:EMP-FILE -> FD:EMP-FILE
- `CALLS`: MAIN-START -> OPEN-FILE, MAIN-START -> PROCESS-RECORDS, MAIN-START -> CLOSE-FILE
- `CALLS`: EMPMAINT -> EMPREPORT (external CALL)
- `ACCESSES`: EMPMAINT -> sql-table:EMPLOYEES
- `RECEIVES`: EMPMAINT -> LK-SEARCH-KEY (PROCEDURE USING)
- `RECORD_KEY_OF`: EMP-ID -> SELECT:EMP-FILE
- `FILE_STATUS_OF`: WS-FILE-STATUS -> SELECT:EMP-FILE
## How COBOL Differs from Tree-Sitter Languages
| Aspect | COBOL | Tree-Sitter Languages |
|--------|-------|----------------------|
| Node variety | 8 types (Module, Function, Namespace, Record, Property, Const, CodeElement, Constructor) | Typically 4-6 (Function, Class, Method, Interface, Module, Const) |
| Domain edges | RECORD_KEY_OF, FILE_STATUS_OF, ACCESSES, RECEIVES, CONTRACTS, REDEFINES | Primarily CALLS, IMPORTS, EXTENDS, IMPLEMENTS |
| Data hierarchy | Deep CONTAINS chains (01 -> 05 -> 10 -> 88) | Flat class members |
| Cross-program calls | CALL "name" + CICS LINK PROGRAM | Import-based resolution |
| Contract detection | Shared COPY copybook between caller/callee | Not applicable |
| Metadata | AUTHOR, DATE-WRITTEN on Module | JSDoc/docstring (not indexed) |
## Source Files
- `gitnexus/src/core/ingestion/workers/parse-worker.ts` -- `processCobolRegexOnly()`, node/edge emission logic
- `gitnexus/src/core/ingestion/pipeline.ts` -- `detectCrossProgamContracts()` for CONTRACTS edges
- `gitnexus/src/core/ingestion/cobol-preprocessor.ts` -- `CobolRegexResults` interface (all extracted data)
+261
View File
@@ -0,0 +1,261 @@
# COBOL Performance and Tuning
This document covers real-world benchmarks, worker pool configuration, memory management, known limitations, and troubleshooting for COBOL indexing.
## PROJECT-NAME Benchmark
The PROJECT-NAME project is a large Italian payroll system written in COBOL. It serves as the primary benchmark for COBOL indexing performance.
### Input
| Metric | Value |
| --------------------------- | ---------------------------------------------------------------------------- |
| Paths scanned | 14,217 |
| Parseable files | 13,129 |
| Total source size | 224 MB |
| Chunks | 12 (at 20 MB budget) |
| Copybooks loaded | 2,976 |
| Copybooks used in expansion | 2,955 |
| Key directories | `s/` (7773 programs), `c/` (3036 copybooks), `wfproc/` (1973 workflow files) |
### Output
| Metric | Value |
| ---------------------- | ------ |
| Graph nodes | 2.79M |
| Graph edges | 5.67M |
| Clusters (communities) | 16,679 |
| Execution flows | 300 |
### Timing
| Phase | Duration |
| ------------------------------- | ----------------- |
| Total | ~251s |
| KuzuDB write | 132s |
| Full-text search indexing | 6.7s |
| Regex extraction (avg per file) | ~1ms |
| COPY expansion + deep indexing | Remainder (~112s) |
### Indexing Command
```bash
cd /path/to/PROJECT-NAME
GITNEXUS_COBOL_DIRS=s,c,wfproc GITNEXUS_VERBOSE=1 node --max-old-space-size=8192 \
/path/to/gitnexus/dist/cli/index.js analyze --force
```
## Open-Source Benchmarks
### CardDemo (AWS)
| Metric | Value |
| ------ | ----- |
| Graph nodes | 12,323 |
| Graph edges | 8,893 |
| Total time | 7.4s |
### ACAS
| Metric | Value |
| ------ | ----- |
| Graph nodes | 14,016 |
| Graph edges | 15,452 |
| Total time | 9.3s |
### Micro-Benchmark (Single-File Extraction)
| Metric | Value |
| ------ | ----- |
| Per-iteration | 0.65ms |
| Throughput | ~382K lines/sec |
## Worker Pool Tuning
### Sub-Batch Size
The worker pool splits each worker's chunk into sub-batches to bound peak memory per `postMessage` serialization. COBOL repos use a smaller sub-batch size than the default:
| Parameter | Default | COBOL Mode |
| --------------------- | ----------- | ------------------- |
| Sub-batch size | 1,500 files | 200 files |
| Per sub-batch timeout | 120s | 120s (configurable) |
**Why 200?** COBOL regex extraction + preprocessing takes ~1ms per file on average, but with COPY expansion and deep indexing the effective time is ~150ms per file. At sub-batch size 1500, that would be ~225s per sub-batch, exceeding the 120s timeout.
COBOL mode is activated automatically when `GITNEXUS_COBOL_DIRS` is set:
```typescript
// From pipeline.ts
const cobolSubBatch = process.env.GITNEXUS_COBOL_DIRS ? 200 : undefined;
workerPool = createWorkerPool(workerUrl, undefined, cobolSubBatch);
```
### Worker Count
Workers default to `min(8, cpus - 1)`. For COBOL repos, this is usually sufficient since regex extraction is CPU-bound but fast. The bottleneck is typically KuzuDB write, not extraction.
### Timeout Configuration
| Environment Variable | Default | Purpose |
| ------------------------------------ | --------------- | --------------------------------------------------- |
| `GITNEXUS_WORKER_TIMEOUT_MS` | 120,000 (2 min) | Per sub-batch processing timeout |
| `GITNEXUS_WORKER_STARTUP_TIMEOUT_MS` | 60,000 (1 min) | Worker initialization timeout (tree-sitter loading) |
For COBOL-only repos, worker startup is faster because tree-sitter native modules are loaded lazily (skipped entirely if only COBOL files are present).
## Data Item Cap
### Configuration
```typescript
const MAX_DATA_ITEMS_PER_FILE = 500;
```
This constant appears in both `parse-worker.ts` (worker path) and `parsing-processor.ts` (sequential fallback).
### Rationale
Some COBOL programs, especially after COPY expansion, can have 10,000+ data items. At that scale:
- The in-memory relationship Map (for CONTAINS, REDEFINES, etc.) approaches the V8 16.7M entry limit across thousands of files
- KuzuDB write time increases linearly with edge count
- Most deep-nested items (level 20+) are rarely queried individually
### Impact
The cap truncates data items beyond the 500th in source order. Since 01-level Records appear first in COBOL source, the cap preserves:
- All 01-level record definitions
- The most important 02-49 level items (those closest to the record root)
- 88-level conditions associated with early items
To increase the cap for specific needs, modify the `MAX_DATA_ITEMS_PER_FILE` constant in both files.
## Memory Management
### COPY Expansion Breadth Guard
A per-file `MAX_TOTAL_EXPANSIONS = 500` limit prevents exponential blowup from diamond-shaped COPY graphs (e.g., N copybooks each containing N COPY statements). Once the limit is reached, further COPY statements in that file are left unexpanded. See [copy-expansion.md](copy-expansion.md) for details.
### COPY Expansion Memory
All copybook content is loaded upfront into a Map before chunk processing begins. For PROJECT-NAME:
- 2,976 copybooks, typically under 100MB total
- The Map is shared (read-only) across chunk iterations
- Per-chunk, the copybook map is merged with chunk file content (in case a chunk contains copybooks not in the pre-loaded set)
- After all chunks are processed, the copybook map is freed (`cobolCopybookContents = undefined`)
### Chunk Budget
Source files are grouped into chunks of max 20MB (`CHUNK_BYTE_BUDGET`). Each chunk's lifecycle:
1. Read file content into memory
2. Expand COPY statements (mutates content in-place)
3. Dispatch to workers for extraction
4. Workers return serialized results
5. Merge results into graph
6. Chunk content goes out of scope (GC reclaims)
This ensures only ~20MB of source + ~200-400MB of working memory (ASTs, extracted records, serialization) is active at any time.
### Shared Warning Deduplication
The `warnedCircular` set (used by the COPY expansion engine) is shared across all files in a chunk. This prevents the same circular copybook warning (e.g., `ANAZI includes itself`) from being logged thousands of times.
## Known Limitations
| Limitation | Impact | Workaround |
| ---------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| tree-sitter-cobol hangs on ~5% of files | Cannot use tree-sitter for COBOL | Regex-only extraction (current approach) |
| Data item cap (500/file) | May miss deeply nested items in large programs | Increase `MAX_DATA_ITEMS_PER_FILE` in source |
| Circular copybooks (ANAZI, ANDIP, QDIPE) | Self-referential includes cannot be expanded | Detected and skipped with warning |
| wfproc/ files may not be pure COBOL | Workflow files may produce extraction noise | Exclude `wfproc` from `GITNEXUS_COBOL_DIRS` if problematic |
| No MOVE DATA_FLOW edges yet | Data flow between variables not in graph | Reserved for future release |
| Continuation line handling | Some complex multi-line continuations (especially in string literals spanning 3+ lines) may not merge correctly | Known edge case; affects <0.1% of lines |
| Single-line EXEC blocks | `EXEC SQL SELECT ... END-EXEC` on one line is handled, but pathological nesting is not | Extremely rare in practice |
| Extension case sensitivity | `.GNM` and `.gnm` are matched differently | Use the exact case from the codebase |
## Troubleshooting
### "COPY expansion failed"
```
[pipeline] COPY expansion failed for s/BGTABFL: Cannot read properties of null
```
**Cause:** A copybook referenced by a COPY statement cannot be found.
**Fix:**
1. Verify `GITNEXUS_COBOL_DIRS` includes the directory containing copybooks (typically `c`)
2. Check that copybook filenames match the COPY target (case-insensitive, after stripping extensions)
3. Ensure copybook files are not in `.gitignore`
### Worker sub-batch timeout
```
Worker 3 sub-batch timed out after 120s (chunk: 200 items)
```
**Cause:** A sub-batch took longer than the timeout. Typically happens when one file is extremely large (50,000+ lines after COPY expansion).
**Fix:** Increase the timeout:
```bash
GITNEXUS_WORKER_TIMEOUT_MS=300000 gitnexus analyze
```
### Memory errors (heap out of memory)
```
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
```
**Fix:** Increase Node.js heap size:
```bash
node --max-old-space-size=16384 /path/to/gitnexus/dist/cli/index.js analyze
```
For very large repos (>500MB source), consider `--max-old-space-size=32768`.
### Concurrent analyze corruption
**Rule:** Only ONE `gitnexus analyze` process should run at a time per repository. Concurrent writes to KuzuDB corrupt the database.
If corruption occurs:
```bash
# Remove the KuzuDB directory and re-index
rm -rf .gitnexus/kuzu
gitnexus analyze --force
```
### Slow KuzuDB write phase
The KuzuDB write phase (132s for PROJECT-NAME) is the bottleneck for large COBOL repos. This is proportional to the number of nodes and edges being written. Reducing `MAX_DATA_ITEMS_PER_FILE` or excluding non-essential directories from `GITNEXUS_COBOL_DIRS` can help.
### Verbose output
Enable verbose logging to see per-phase timing and statistics:
```bash
GITNEXUS_VERBOSE=1 gitnexus analyze
```
This outputs:
- Scan statistics (paths, parseable files, chunk count)
- Worker pool configuration (worker count, sub-batch size)
- COPY expansion statistics (copybooks loaded, files expanded)
- Community and process detection results
- Contract detection results
## Source Files
- `gitnexus/src/core/ingestion/workers/worker-pool.ts` -- `DEFAULT_SUB_BATCH_SIZE`, `SUB_BATCH_TIMEOUT_MS`, `WORKER_STARTUP_TIMEOUT_MS`
- `gitnexus/src/core/ingestion/pipeline.ts` -- `CHUNK_BYTE_BUDGET`, COBOL sub-batch configuration, chunk lifecycle
- `gitnexus/src/core/ingestion/workers/parse-worker.ts` -- `MAX_DATA_ITEMS_PER_FILE`, `processCobolRegexOnly()`
- `gitnexus/src/core/ingestion/parsing-processor.ts` -- Sequential fallback `MAX_DATA_ITEMS_PER_FILE`
@@ -0,0 +1,206 @@
# COBOL Regex Extraction
The `extractCobolSymbolsWithRegex()` function in `cobol-preprocessor.ts` performs single-pass, state-machine-driven extraction of all COBOL symbols. This document describes the state machine, line processing flow, and every regex pattern used.
## State Machine: Division Tracking
The extractor tracks which COBOL division is currently being processed. Division transitions are detected by the `RE_DIVISION` pattern.
```mermaid
stateDiagram-v2
[*] --> null : Start of file
null --> identification : IDENTIFICATION DIVISION
identification --> environment : ENVIRONMENT DIVISION
environment --> data : DATA DIVISION
data --> procedure : PROCEDURE DIVISION
note right of identification
Extracts: PROGRAM-ID, AUTHOR, DATE-WRITTEN
end note
note right of environment
Extracts: SELECT ... ASSIGN ... (file declarations)
end note
note right of data
Extracts: FD entries, data items (01-77, 88), COPY
end note
note right of procedure
Extracts: paragraphs, sections, PERFORM, CALL,
ENTRY, MOVE, EXEC SQL/CICS
end note
```
## State Machine: Data Section Tracking
Within the DATA DIVISION, a secondary state machine tracks the current section to tag data items with their origin.
```mermaid
stateDiagram-v2
[*] --> unknown : DATA DIVISION entered
unknown --> working_storage : WORKING-STORAGE SECTION
unknown --> linkage : LINKAGE SECTION
unknown --> file : FILE SECTION
unknown --> local_storage : LOCAL-STORAGE SECTION
working_storage --> linkage : LINKAGE SECTION
working_storage --> file : FILE SECTION
linkage --> working_storage : WORKING-STORAGE SECTION
file --> working_storage : WORKING-STORAGE SECTION
file --> linkage : LINKAGE SECTION
local_storage --> working_storage : WORKING-STORAGE SECTION
```
Within the ENVIRONMENT DIVISION, the `currentEnvSection` tracks whether we are in `INPUT-OUTPUT` or `CONFIGURATION` section. SELECT statement accumulation only occurs in `INPUT-OUTPUT`.
## Line Processing Flow
Each raw source line goes through this pipeline:
```
Raw line
|
v
Length < 7? ---------> Skip (flush pending if any)
|
v
Indicator col 7
|
+-- '*' or '/' -----> Comment: skip entirely
|
+-- '-' ------------> Continuation: append to pending line
|
+-- other ----------> Normal: flush pending, strip inline comments (|),
buffer as new pending logical line
```
After all lines are processed, the final pending line is flushed, along with any accumulated SELECT statement, SORT/MERGE accumulator, and any open EXEC block (truncated file without `END-EXEC`).
### Inline Comment Stripping
Enterprise COBOL (particularly Italian dialect) uses the pipe character `|` as an inline comment marker. The `stripInlineComment()` helper is **quote-aware**: it tracks whether the scan position is inside a single- or double-quoted string and only treats `|` as a comment marker when outside quotes. Pipe characters inside string literals are preserved.
Free-format `*>` inline comment stripping uses the same quote-aware approach: the scanner walks character by character, toggling quote state, and only recognizes `*>` as a comment marker when not inside a quoted string.
### Patch Marker Handling
The `preprocessCobolSource()` function (run before extraction in the worker) replaces non-standard content in columns 1-6. Standard COBOL expects spaces or digit sequence numbers in this area. If any letter or `#` character is found, the entire sequence area is replaced with 6 spaces:
```
Before: mzADD MOVE WK-AMT TO WK-TOTAL
After: MOVE WK-AMT TO WK-TOTAL
```
This preserves exact line count for position mapping.
## Regex Pattern Reference
All patterns are compiled once as module-level constants and reused across calls.
### Division and Section Detection
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_DIVISION` | `\b(IDENTIFICATION\|ENVIRONMENT\|DATA\|PROCEDURE)\s+DIVISION\b` | Division boundary | `PROCEDURE DIVISION` |
| `RE_SECTION` | `\b(WORKING-STORAGE\|LINKAGE\|FILE\|LOCAL-STORAGE\|INPUT-OUTPUT\|CONFIGURATION)\s+SECTION\b` | Section boundary | `WORKING-STORAGE SECTION` |
### IDENTIFICATION DIVISION
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_PROGRAM_ID` | `\bPROGRAM-ID\.\s*([A-Z][A-Z0-9-]*)` | Program name | `PROGRAM-ID. BGTABFL` |
| `RE_AUTHOR` | `^\s+AUTHOR\.\s*(.+)` | Author metadata | `AUTHOR. D. Smith` |
| `RE_DATE_WRITTEN` | `^\s+DATE-WRITTEN\.\s*(.+)` | Date metadata | `DATE-WRITTEN. 2024-01-15` |
### ENVIRONMENT DIVISION
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_SELECT_START` | `\bSELECT\s+(?:OPTIONAL\s+)?([A-Z][A-Z0-9-]+)` | File SELECT start (with optional `SELECT OPTIONAL` support) | `SELECT MASTER-FILE`, `SELECT OPTIONAL TRANS-FILE` |
SELECT statements are accumulated across multiple lines until a period terminator is found, then parsed for ASSIGN, ORGANIZATION, ACCESS, RECORD KEY, and FILE STATUS clauses.
### DATA DIVISION
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_FD` | `^\s+FD\s+([A-Z][A-Z0-9-]+)` | File description | `FD MASTER-FILE` |
| `RE_DATA_ITEM` | `^\s+(\d{1,2})\s+([A-Z][A-Z0-9-]+)\s*(.*)` | Data item (01-77) | `05 WK-NAME PIC X(30)` |
| `RE_ANONYMOUS_REDEFINES` | `^\s+(\d{1,2})\s+REDEFINES\s+([A-Z][A-Z0-9-]+)` | Anonymous REDEFINES | `01 REDEFINES WK-REC` |
| `RE_88_LEVEL` | `^\s+88\s+([A-Z][A-Z0-9-]+)\s+VALUES?\s+(?:ARE\s+)?(.+)` | Condition name | `88 WK-ACTIVE VALUE "Y"` |
The trailing clauses of `RE_DATA_ITEM` are parsed by `parseDataItemClauses()` for PIC, USAGE, OCCURS, and REDEFINES.
### PROCEDURE DIVISION
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_PROC_SECTION` | `^ ([A-Z][A-Z0-9-]+)\s+SECTION\.\s*$` | Procedure section header | ` MAIN-LOGIC SECTION.` |
| `RE_PROC_PARAGRAPH` | `^ ([A-Z][A-Z0-9-]+)\.\s*$` | Paragraph header | ` PROCESS-RECORD.` |
| `RE_PERFORM` | `\bPERFORM\s+([A-Z][A-Z0-9-]+)(?:\s+THRU\s+([A-Z][A-Z0-9-]+))?` | PERFORM call | `PERFORM CALC-TAX THRU CALC-TAX-EXIT` |
| `RE_PROC_USING` | `\bPROCEDURE\s+DIVISION\s+USING\s+([\s\S]*?)(?:\.\|$)` | USING parameters | `PROCEDURE DIVISION USING WK-PARAM` |
| `RE_ENTRY` | `\bENTRY\s+"([^"]+)"(?:\s+USING\s+([\s\S]*?))?(?:\.\|$)` | ENTRY point | `ENTRY "SUBPROG" USING WK-DATA` |
| `RE_MOVE` | `\bMOVE\s+((?:CORRESPONDING\|CORR)\s+)?([A-Z][A-Z0-9-]+)\s+TO\s+(.+)` | MOVE statement (supports CORR abbreviation and multi-target) | `MOVE WK-NAME TO OUT-NAME`, `MOVE CORR WK-IN TO WK-OUT` |
The USING parameter list (`RE_PROC_USING`) is split on `\bRETURNING\b` before tokenization -- any RETURNING clause and everything after it is excluded from the parameter list (`.split(/\bRETURNING\b/i)[0]`).
Note: `RE_PROC_SECTION` and `RE_PROC_PARAGRAPH` require exactly 7 spaces of leading indentation (COBOL area A starting at column 8). This is the standard COBOL paragraph indentation.
### All-Division Patterns
These patterns are checked regardless of current division:
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_CALL` | `\bCALL\s+"([^"]+)"` | External program call | `CALL "BGTABUP"` |
| `RE_COPY_UNQUOTED` | `\bCOPY\s+([A-Z][A-Z0-9-]+)(?:\s\|\.)` | COPY (unquoted) | `COPY CPSESP.` |
| `RE_COPY_QUOTED` | `\bCOPY\s+"([^"]+)"(?:\s\|\.)` | COPY (quoted) | `COPY "WORKGRID.CPY".` |
### SORT/MERGE Support
| Constant | Purpose |
|----------|---------|
| `SORT_CLAUSE_NOISE` | Set of SORT/MERGE clause keywords filtered from USING/GIVING file lists: `ON`, `ASCENDING`, `DESCENDING`, `KEY`, `WITH`, `DUPLICATES`, `IN`, `ORDER`, `COLLATING`, `SEQUENCE`, `IS`, `THROUGH`, `THRU`, `INPUT`, `OUTPUT`, `PROCEDURE` |
SORT and MERGE statements are accumulated across multiple lines (like SELECT) until a period terminator is found, then parsed for USING/GIVING file lists and INPUT/OUTPUT PROCEDURE targets. The `flushSort()` helper encapsulates the flush-and-parse logic, mirroring the existing `flushSelect()` pattern. Both helpers are called at EOF to handle truncated files.
### GO TO Multi-Target
`RE_GOTO` captures all paragraph names in a `GO TO` statement, including the multi-target form `GO TO p1 p2 p3 DEPENDING ON x`. The captured group contains all target names (space-separated), which are split into individual targets. Each target produces a separate `gotos` entry.
### PROGRAM-ID Detection
PROGRAM-ID is detected regardless of the current division state. This handles sibling programs that appear after `END PROGRAM` and omit the `IDENTIFICATION DIVISION` header -- the extractor will still capture the PROGRAM-ID and push a new program boundary.
### EXEC Block Patterns
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_EXEC_SQL_START` | `\bEXEC\s+SQL\b` | Start of EXEC SQL block | `EXEC SQL` |
| `RE_EXEC_CICS_START` | `\bEXEC\s+CICS\b` | Start of EXEC CICS block | `EXEC CICS` |
| `RE_END_EXEC` | `\bEND-EXEC\b` | End of EXEC block | `END-EXEC` |
EXEC blocks accumulate all lines between `EXEC SQL/CICS` and `END-EXEC`, then delegate to `parseExecSqlBlock()` or `parseExecCicsBlock()` for detailed extraction.
## Excluded Paragraph Names
The following names are excluded from paragraph detection to avoid false positives from division/section headers:
```
DECLARATIVES, END, PROCEDURE, IDENTIFICATION,
ENVIRONMENT, DATA, WORKING-STORAGE, LINKAGE,
FILE, LOCAL-STORAGE, COMMUNICATION, REPORT,
SCREEN, INPUT-OUTPUT, CONFIGURATION
```
Additionally, paragraph candidates containing `DIVISION` or `SECTION` as substrings are excluded.
## MOVE Skip List (Figurative Constants)
MOVE statements where the source is a figurative constant are skipped:
```
SPACES, ZEROS, ZEROES, LOW-VALUES, LOW-VALUE,
HIGH-VALUES, HIGH-VALUE, QUOTES, QUOTE, ALL
```
## Source Files
- `gitnexus/src/core/ingestion/cobol-preprocessor.ts` -- `preprocessCobolSource()`, `extractCobolSymbolsWithRegex()`, all regex constants
@@ -0,0 +1,326 @@
---
title: "feat: Complete COBOL language feature coverage for maximum knowledge graph value"
type: feat
status: active
date: 2026-03-26
origin: Feature audit from v3-integration-architect agent (session 8642401e)
---
## Enhancement Summary
**Deepened on:** 2026-03-26
**Research agents used:** COBOL expert (Phase 1+2), graph value analyst, codebase explorer
**Sections enhanced:** Phase 1 (5 features), Phase 2 (4 features), graph value ranking
### Key Improvements from Research
1. **CALL USING** is the #1 highest-value edge type (9.2/10) — fixes ~40% of missing caller references
2. **EXEC DLI** requires dual-interface support (EXEC DLI + CBLTDLI CALL) for full IMS coverage
3. **DECLARATIVES** is lowest-risk Phase 2 item — existing section/paragraph detection already captures structure
4. **SET TO TRUE** accounts for 80-90% of all SET statements — prioritize this form
5. **INSPECT** needs multi-line accumulator (like SORT) — can span 5+ continuation lines
6. **Graph value ranking**: cobol-call-using (9.2) > cobol-error-handler (9.0) > dli-gu (8.2) > cobol-string (6.2)
### New Edge Cases Discovered
- CALL USING supports mixed modes: `USING BY REFERENCE WS-A BY CONTENT WS-B BY VALUE WS-C`
- CALL USING `ADDRESS OF` and `OMITTED` must be filtered from parameter lists
- EXEC DLI can have multiple SEGMENT levels in hierarchical retrieval (use matchAll)
- DECLARATIVES can have multiple USE sections (one per file + catch-all for INPUT/OUTPUT/I-O/EXTEND)
- INSPECT TALLYING can have multiple counters in a single statement
- STRING/UNSTRING can span multiple lines (need accumulator pattern)
---
# Complete COBOL Language Feature Coverage
## Overview
Implement the remaining 25 unhandled COBOL language features and fix 10 partial features to achieve ~95% coverage (up from 71.9%). The goal is to build the richest possible knowledge graph from COBOL codebases, enabling a future `modernize` MCP command (out of scope for this plan) that would use the graph to assist with COBOL-to-modern-language migration.
## Problem Statement
The COBOL processor currently handles 54 of 89 applicable language features (71.9%). The 25 unhandled features represent real data loss in the knowledge graph:
- **Cross-program data flow** is invisible (CALL ... USING parameters not extracted)
- **IMS/DB programs** produce empty graphs (EXEC DLI not recognized)
- **String transformation logic** is invisible (STRING/UNSTRING/INSPECT not tracked)
- **SQL copybook dependencies** are missing (EXEC SQL INCLUDE not mapped)
- **Error handling flows** are lost (DECLARATIVES/USE AFTER not captured)
## Proposed Solution
Implement features in 4 phases, ordered by graph value density (edges created per LOC of implementation). Each phase is independently shippable and testable.
## Technical Approach
### Phase 1: High-Value Data Flow Edges (~150 LOC, ~8 new edge types)
The highest-ROI features: they create new ACCESSES and IMPORTS edges that directly improve impact analysis.
**Critical research finding**: Multi-line statement accumulation is the dominant challenge. CALL USING, STRING/UNSTRING, and multi-line data item clauses all span multiple lines in production COBOL. The free-format path processes each line independently — these features need statement accumulators (like SORT/SELECT) or the free-format path needs multi-line awareness. Estimated LOC increased from 110 to 150 to account for accumulator infrastructure.
#### 1.1 EXEC SQL INCLUDE -> IMPORTS edges
- **File:** `cobol-preprocessor.ts` (parseExecSqlBlock)
- **What:** Detect `INCLUDE` as the operation, extract member name, emit as a `copies[]` entry
- **Graph:** IMPORTS edge from File to included copybook/SQLCA with reason `sql-include`
- **Tests:** Unit test for `EXEC SQL INCLUDE SQLCA END-EXEC` and `EXEC SQL INCLUDE CUSTCOPY END-EXEC`
**Research insights (EXEC SQL INCLUDE):**
- DB2 member names can contain underscores: `EXEC SQL INCLUDE CUST_TBL_DCL END-EXEC` — regex must use `[A-Z][A-Z0-9_-]+`
- Quoted literal form: `EXEC SQL INCLUDE 'DBRMLIB.MEMBER' END-EXEC` (z/OS PDS qualified name)
- SQLCA/SQLDA are DB2 builtins — won't resolve to repo files. Emit unresolved IMPORTS edge (still valuable)
- No REPLACING support on EXEC SQL INCLUDE (unlike COPY)
- Add `INCLUDE` to `OP_MAP` in `parseExecSqlBlock`; extract member via `RE_SQL_INCLUDE = /^INCLUDE\s+(?:'([^']+)'|"([^"]+)"|([A-Z][A-Z0-9_-]+))/i`
#### 1.2 CALL ... USING parameter extraction -> ACCESSES edges (Graph value: 9.2/10)
- **File:** `cobol-preprocessor.ts` (processLogicalLine CALL section)
- **What:** After capturing CALL target, scan for USING clause. Extract parameter names (reuse USING_KEYWORDS filter). Store as `calls[].parameters: string[]`
- **Interface:** Add `parameters?: string[]` to calls array type in CobolRegexResults
- **File:** `cobol-processor.ts` (CALL edge block)
- **Graph:** For each USING parameter, create ACCESSES edge from caller to data item Property node with reason `cobol-call-using`
- **Tests:** `CALL 'AUDITLOG' USING CUST-ID WS-AMOUNT` -> 2 ACCESSES edges
**Research insights (CALL USING forms):**
- Mixed modes: `CALL 'PGM' USING BY REFERENCE WS-A BY CONTENT WS-B BY VALUE WS-C`
- Pointer passing: `CALL 'PGM' USING ADDRESS OF WS-A`
- Placeholder: `CALL 'PGM' USING OMITTED WS-B`
- Filter keywords: add `ADDRESS`, `OMITTED`, `LENGTH` to USING_KEYWORDS (already has BY/VALUE/REFERENCE/CONTENT)
- **Impact tool enhancement:** CALL-USING edges enable BFS traversal through parameter data flow — single most impactful edge type for COBOL impact analysis
#### 1.3 STRING/UNSTRING data flow -> ACCESSES edges
- **File:** `cobol-preprocessor.ts` (new section in extractProcedure)
- **What:** Accumulate multi-line STRING/UNSTRING until period or END-STRING/END-UNSTRING. Extract sources and INTO targets.
- **Interface:** Add `strings: Array<{ sources: string[]; target: string; type: 'string' | 'unstring'; line: number; caller: string | null }>` to CobolRegexResults
- **Graph:** read-ACCESSES on sources, write-ACCESSES on INTO target with reason `cobol-string-read` / `cobol-string-write`
- **Tests:** 2 unit tests + integration test assertions
**Research insights (STRING/UNSTRING):**
- **Needs statement accumulator** — STRING/UNSTRING always span multiple lines in production
- Terminate accumulation at: period, END-STRING/END-UNSTRING, or start of next COBOL verb
- STRING sources: identifiers before each `DELIMITED BY`. Filter: STRING, DELIMITED, BY, SIZE, ALL, INTO, WITH, POINTER, ON, OVERFLOW, NOT, END-STRING
- UNSTRING: source is first identifier after UNSTRING; INTO targets are identifiers after INTO. Filter: DELIMITER, IN, COUNT, TALLYING, OR
- WITH POINTER field is both read AND written (starting position updated)
- TALLYING IN / COUNT IN fields are write targets
- Literal sources (`'text'`) must be filtered — quote-aware tokenization needed
- **Edge case**: STRING terminated by next verb, not period — existing fixture has `STRING ... DISPLAY` without period between them
#### 1.4 OCCURS DEPENDING ON -> ACCESSES edge
- **File:** `cobol-preprocessor.ts` (parseDataItemClauses)
- **What:** Extend OCCURS regex to capture DEPENDING ON field, KEY fields, and INDEXED BY names
- **Interface:** Add `dependingOn?: string`, `occursMax?: number`, `occursKeys?: Array<{direction: string; fields: string[]}>`, `indexedBy?: string[]` to data items
- **Graph:** ACCESSES edge from table item to controlling field with reason `cobol-depends-on`
- **Tests:** `05 WS-TABLE OCCURS 100 DEPENDING ON WS-COUNT` -> edge
**Research insights (OCCURS):**
- IBM allows `OCCURS 0 TO n DEPENDING ON` (zero minimum) and `OCCURS UNBOUNDED DEPENDING ON` (V6.4)
- Subscripted controlling fields: `DEPENDING ON WS-COUNT(WS-IDX)` — strip subscripts before storing
- **Pre-existing gap**: Multi-line data item clauses without continuation indicator are NOT captured. `05 WS-TABLE\n OCCURS 100\n DEPENDING ON WS-COUNT.` — the current RE_DATA_ITEM only gets the first line, `rest` is empty. Fixing properly requires a data item accumulator (like SELECT). **Defer full fix to Phase 3; implement same-line capture now.**
- KEY IS fields: `ASCENDING KEY IS WS-KEY-1 WS-KEY-2` — capture for SEARCH ALL resolution
- INDEXED BY: `INDEXED BY IDX-1 IDX-2` — capture for SET/SEARCH context
#### 1.5 VALUE clause for standard data items
- **File:** `cobol-preprocessor.ts` (parseDataItemClauses)
- **What:** Extract VALUE using a pragmatic function that handles quoted strings, numerics, figurative constants, hex/national literals
- **Interface:** Already exists as `values?: string[]` on data items (currently only populated for 88-level)
- **Graph:** Stored in Property node description (no new edges)
- **Tests:** `01 WS-STATUS PIC X VALUE 'A'` -> values: ['A']
**Research insights (VALUE forms):**
- Hex literals: `VALUE X'F1F2F3F4'`, National: `VALUE N'text'`, DBCS: `VALUE G'text'`
- Figurative constants: SPACES, ZEROS, ZEROES, LOW-VALUES, HIGH-VALUES, QUOTES, NULL, NULLS
- ALL literal: `VALUE ALL '*'`
- Numeric with sign/decimal: `VALUE -123.45`, `VALUE +1`
- `VALUE IS` optional — both `VALUE 'A'` and `VALUE IS 'A'` valid
- **Decimal vs period ambiguity**: `VALUE 100.` — is `.` decimal or terminator? `parseDataItemClauses` already strips trailing period, so this is handled
- IBM V6.4: floating-point `VALUE 1.0E5` — extend numeric regex if needed
- Implementation: use a pragmatic `extractValue(rest)` function, not a single complex regex
### Phase 2: EXEC DLI + DECLARATIVES (~90 LOC, ~4 new edge types)
IMS/DB support and error handling flows.
#### 2.1 EXEC DLI (IMS/DB) -> ACCESSES edges (Graph value: 8.2/10)
- **File:** `cobol-preprocessor.ts` (processLogicalLine — add RE_EXEC_DLI_START check alongside SQL/CICS)
- **What:** Accumulate EXEC DLI blocks like EXEC SQL. Parse DLI verbs (GU, GN, GNP, GHU, GHN, GHNP, ISRT, DLET, REPL, CHKP, SCHD, TERM). Extract segment name, PCB number, INTO/FROM areas, WHERE fields, PSB name.
- **Interface:** Add `execDliBlocks: Array<{ line: number; verb: string; pcbNumber?: number; segmentName?: string; intoField?: string; fromField?: string; whereField?: string; psbName?: string }>` to CobolRegexResults
- **Graph:** CodeElement node + ACCESSES edge to `<ims>:<segmentName>` Record node with reason `dli-{verb}`; ACCESSES edges to INTO/FROM data areas; PSB ACCESSES for SCHD
- **Tests:** `EXEC DLI GU USING PCB(1) SEGMENT(CUSTOMER) INTO(WS-CUST) END-EXEC`
**Research insights (dual IMS interface):**
- **EXEC DLI**: Embedded command interface for CICS-DL/I programs only
- **CBLTDLI CALL**: Batch interface via `CALL 'CBLTDLI' USING function-code PCB io-area SSA1..SSA15`
- CBLTDLI is already captured as a CALL to 'CBLTDLI' — enrich with USING parameter semantics later
- Multiple SEGMENT levels in hierarchical retrieval — use `matchAll` on segment regex
- DLI verbs: GU (most common), GN, GNP, GHU, GHN, GHNP, ISRT, REPL, DLET, CHKP, SCHD, TERM, ROLL, ROLB
- **Edge case**: DLET/REPL have no SEGMENT clause (operate on current position)
- **Recommended order**: Implement AFTER DECLARATIVES and SET (lower risk, higher frequency)
#### 2.2 DECLARATIVES / USE AFTER STANDARD EXCEPTION (Graph value: 9.0/10)
- **File:** `cobol-preprocessor.ts` (processLogicalLine — detect DECLARATIVES keyword, track USE AFTER blocks)
- **What:** When `DECLARATIVES.` is encountered, switch to declaratives mode. Extract USE statements binding sections to files/modes.
- **Interface:** Add `declaratives: Array<{ sectionName: string; useType: 'error' | 'debug' | 'label' | 'reporting'; target: string; line: number }>` to CobolRegexResults
- **Graph:** ACCESSES edge from declarative Namespace to file Record with reason `cobol-declarative-error-handler`
- **Tests:** Unit test with DECLARATIVES section, integration test for error flow
**Research insights (DECLARATIVES syntax):**
- `USE AFTER STANDARD {EXCEPTION|ERROR} ON {file-name|INPUT|OUTPUT|I-O|EXTEND}`
- EXCEPTION and ERROR are synonymous; STANDARD is optional in IBM dialects
- Multiple USE sections allowed (one per file + catch-all for I/O modes)
- `END DECLARATIVES.` must NOT reset PROCEDURE DIVISION state
- `DECLARATIVES` is already in EXCLUDED_PARA_NAMES — no false paragraph risk
- Existing section/paragraph detection already captures structural elements — just need USE binding
- **Lowest risk Phase 2 item** — implement first
#### 2.3 SET statement -> ACCESSES edges
- **File:** `cobol-preprocessor.ts` (extractProcedure — new RE_SET regex)
- **Interface:** Add `sets: Array<{ targets: string[]; form: 'to-true'|'to-value'|'up-by'|'down-by'|'address-of'|'to-null'|'to-entry'; value?: string; entryTarget?: string; entryIsLiteral?: boolean; line: number; caller: string | null }>` to CobolRegexResults
- **Graph:** ACCESSES write edge with reason `cobol-set-condition` (TO TRUE), `cobol-set-index` (TO/UP/DOWN), `cobol-set-address` (ADDRESS OF). SET ENTRY with literal -> CALLS edge.
- **Tests:** `SET WS-EOF TO TRUE`, `SET IDX-1 TO 5`, `SET IDX-1 UP BY 1`
**Research insights (SET forms by frequency):**
- `SET condition TO TRUE` — 80-90% of all SET usage. Multiple targets: `SET COND-A COND-B TO TRUE`
- `SET index TO/UP BY/DOWN BY` — ~8%. Multiple indices: `SET IDX-1 IDX-2 UP BY 1`
- `SET pointer TO ADDRESS OF data-item` / `SET ADDRESS OF data-item TO pointer` — ~2%
- `SET proc-ptr TO ENTRY "PROGNAME"` — rare but creates CALLS edge (like dynamic CALL)
- Filter OF/IN qualifiers: `SET COND-A OF WS-RECORD TO TRUE` (strip OF WS-RECORD)
- **Prioritize**: SET TO TRUE alone covers 80-90% — implement this form first
#### 2.4 INSPECT -> ACCESSES edges
- **File:** `cobol-preprocessor.ts` (extractProcedure — new `inspectAccum` accumulator like SORT)
- **What:** Accumulate multi-line INSPECT until period. Extract inspected field + tally counters.
- **Interface:** Add `inspects: Array<{ inspectedField: string; counters: string[]; form: 'tallying'|'replacing'|'converting'|'tallying-replacing'; line: number; caller: string | null }>` to CobolRegexResults
- **Graph:** ACCESSES read on inspected field always; write if REPLACING/CONVERTING. Write edges for tally counters. Reason: `cobol-inspect-read`/`cobol-inspect-write`/`cobol-inspect-tally`
- **Tests:** `INSPECT WS-FIELD TALLYING WS-COUNT FOR ALL 'A'` -> read on WS-FIELD, write on WS-COUNT
**Research insights (INSPECT forms by frequency):**
- REPLACING (~60%): `INSPECT WS-STR REPLACING ALL 'A' BY 'B'`
- TALLYING (~25%): `INSPECT WS-STR TALLYING WS-CNT FOR ALL 'A'` — multiple counters possible
- CONVERTING (~10%): `INSPECT WS-STR CONVERTING 'abc' TO 'ABC'`
- Combined (~5%): TALLYING + REPLACING in single statement
- **Needs multi-line accumulator** — INSPECT frequently spans 3-5 lines in production
- Extract tally counters with `([A-Z][A-Z0-9-]+)\s+FOR\b` matchAll pattern
- Filter figurative constants (SPACES, ZEROS) using existing MOVE_SKIP set
### Phase 3: Completeness Fixes (~60 LOC)
Fix the 10 partial features and small gaps.
#### 3.1 CALL ... RETURNING extraction
- Extend RE_CALL processing to capture RETURNING target after the USING clause
- Store as `calls[].returning?: string`
- Graph: ACCESSES write edge with reason `cobol-call-returning`
#### 3.2 SELECT OPTIONAL flag preservation
- Store `isOptional: boolean` in FileDeclaration interface
- Include in Record node description
#### 3.3 ALTERNATE RECORD KEY extraction
- Add regex in parseSelectStatement: `/\bALTERNATE\s+RECORD\s+KEY\s+(?:IS\s+)?([A-Z][A-Z0-9-]+)/i`
- Store as `alternateKeys?: string[]`
#### 3.4 COMMON attribute on nested programs
- Extend RE_PROGRAM_ID: `/\bPROGRAM-ID\.\s*([A-Z][A-Z0-9-]+)(?:\s+IS\s+COMMON)?/i`
- Store `isCommon: boolean` on Module node
- Affects cross-program CALL resolution scope
#### 3.5 IS EXTERNAL / IS GLOBAL as first-class properties
- Change from usage string hack to proper boolean fields on data items
- Add `isExternal?: boolean`, `isGlobal?: boolean` to data item interface
#### 3.6 AUTHOR / DATE-WRITTEN mapped to Module node
- Already extracted as programMetadata — map to Module node properties
- `graph.addNode({ ..., properties: { ..., author, dateWritten } })`
#### 3.7 REPLACE statement
- Track REPLACE / REPLACE OFF state in preprocessor
- Apply text substitutions during preprocessing (before regex extraction)
- Complex: requires careful scoping rules
### Phase 4: Niche Features (~30 LOC)
Low-priority but nice for completeness.
#### 4.1 INITIALIZE statement -> write ACCESSES
- `/\bINITIALIZE\s+([A-Z][A-Z0-9-]+)/i`
- ACCESSES write edge with reason `cobol-initialize`
#### 4.2 Remaining IDENTIFICATION DIVISION paragraphs
- DATE-COMPILED, INSTALLATION, SECURITY, REMARKS
- Map to Module node description properties
#### 4.3 EXEC SQL INCLUDE -> IMPORTS edge (expansion)
- For EXEC SQL INCLUDE inside EXEC blocks that reference copybooks containing SQL
- Create IMPORTS edge similar to COPY
## Acceptance Criteria
### Functional Requirements
- [ ] Phase 1: All 5 features implemented with unit + integration tests
- [ ] Phase 2: All 4 features implemented with unit + integration tests
- [ ] Phase 3: All 7 partial features fixed
- [ ] Phase 4: At least 2 of 3 niche features implemented
- [ ] All existing 145 tests continue to pass
- [ ] TypeScript compiles cleanly
### Non-Functional Requirements
- [ ] No performance regression: CardDemo benchmark stays under 8s
- [ ] No file exceeds 1500 LOC (preprocessor currently 1326)
- [ ] ACAS benchmark shows increased node/edge counts (more data extracted)
- [ ] CardDemo benchmark shows increased edge counts (CALL USING, STRING, etc.)
### Quality Gates
- [ ] Each phase has its own commit
- [ ] Integration test assertions updated with exact counts per phase
- [ ] Benchmark run after each phase to track graph growth
## Dependencies & Risks
### Dependencies
- None. All changes are additive to existing COBOL processor code.
- No LanguageProvider changes needed.
- No graph schema changes needed (all new constructs map to existing node labels + edge types).
### Risks
- **preprocessor.ts size**: Currently 1326 LOC. Phase 1+2 adds ~200 LOC -> 1526 LOC. May need to extract helpers into a separate `cobol-data-flow.ts` module if it exceeds 1500.
- **REPLACE statement** (Phase 3.7) is the most complex feature — requires tracking text substitution state across logical lines. Consider deferring to a separate PR if it takes >100 LOC.
- **EXEC DLI** (Phase 2.1) is only testable against IMS codebases. Need fixture data or synthetic test cases.
## Graph Value Ranking by MCP Tool Impact
Research agent analyzed all 5 MCP tools (query, context, impact, detect_changes, rename) against planned edge types:
| Edge Type | QUERY | CONTEXT | IMPACT | DETECT | RENAME | **Overall** |
|-----------|-------|---------|--------|--------|--------|-------------|
| `cobol-call-using` | 4/5 | 5/5 | 5/5 | 4/5 | 4/5 | **9.2/10** |
| `cobol-error-handler` | 5/5 | 4/5 | 5/5 | 5/5 | 2/5 | **9.0/10** |
| `dli-*` (IMS verbs) | 4/5 | 4/5 | 5/5 | 4/5 | 2/5 | **8.2/10** |
| `cobol-string-*` | 4/5 | 3/5 | 3/5 | 3/5 | 2/5 | **6.2/10** |
**Key finding**: `cobol-call-using` alone would fix ~40% of missing caller references in COBOL graphs.
## Future Considerations
This plan provides the graph data foundation for a future `modernize` MCP command (out of scope) that would:
- Use CALL USING edges to map data contracts between programs
- Use STRING/UNSTRING edges to identify data transformation logic
- Use EXEC SQL/DLI edges to map database access patterns
- Use DECLARATIVES to understand error handling architecture
- Use the complete knowledge graph to generate migration plans
**MCP tool enhancements needed** (after this plan ships):
- Add `cobol-call-using`, `cobol-error-handler`, `dli-*` to IMPACT tool's default `relationTypes` for COBOL repos
- Add confidence floors for new edge types in `IMPACT_RELATION_CONFIDENCE`
- Register new edge types in `VALID_RELATION_TYPES` set (`local-backend.ts:52`)
## Sources & References
### Internal References
- Feature audit: session 8642401e (COBOL expert agent, 123 features audited)
- Prior plans: `docs/plans/2026-03-25-feat-cobol-100-percent-feature-coverage-plan.md`
- Architecture: `docs/code-indexing/cobol/` (7 documentation files)
### External References
- COBOL features reference: mainframestechhelp.com/tutorials/cobol/features.htm
- COBOL-85 standard: ISO/IEC 1989:1985
- IBM Enterprise COBOL reference
+4
View File
@@ -50,6 +50,10 @@ All models are routed through **OpenRouter** by default, so a single `OPENROUTER
docker pull swebench/sweb.eval.x86_64.django_1776_django-16527:latest
```
### Debug logging
Set `GITNEXUS_EVAL_DEBUG=1` to include full Python tracebacks in run summaries and logs. By default, errors are sanitized to avoid leaking host paths or stack traces.
## Quick Start
### Debug a single instance
+8 -18
View File
@@ -22,8 +22,10 @@ import time
from enum import Enum
from pathlib import Path
from constants import AUGMENT_TIMEOUT_SECONDS
from minisweagent import Environment, Model
from minisweagent.agents.default import AgentConfig, DefaultAgent
from tool_registry import BINARIES_BY_KEY, TOOL_METRIC_KEYS
logger = logging.getLogger("gitnexus_agent")
@@ -40,7 +42,7 @@ class GitNexusMode(str, Enum):
class GitNexusAgentConfig(AgentConfig):
"""Extended config for GitNexus evaluation agent."""
gitnexus_mode: GitNexusMode = GitNexusMode.BASELINE
augment_timeout: float = 5.0
augment_timeout: float = AUGMENT_TIMEOUT_SECONDS
augment_min_pattern_length: int = 3
track_gitnexus_usage: bool = True
@@ -152,16 +154,10 @@ class GitNexusAgent(DefaultAgent):
"""Track which GitNexus tools the agent uses."""
for action in message.get("extra", {}).get("actions", []):
command = action.get("command", "")
if "gitnexus-query" in command:
self.gitnexus_metrics.tool_calls["query"] += 1
elif "gitnexus-context" in command:
self.gitnexus_metrics.tool_calls["context"] += 1
elif "gitnexus-impact" in command:
self.gitnexus_metrics.tool_calls["impact"] += 1
elif "gitnexus-cypher" in command:
self.gitnexus_metrics.tool_calls["cypher"] += 1
elif "gitnexus-overview" in command:
self.gitnexus_metrics.tool_calls["overview"] += 1
for key, binary in BINARIES_BY_KEY.items():
if binary in command and key in self.gitnexus_metrics.tool_calls:
self.gitnexus_metrics.tool_calls[key] += 1
break
def serialize(self, *extra_dicts) -> dict:
"""Serialize with GitNexus-specific metrics."""
@@ -180,13 +176,7 @@ class GitNexusMetrics:
"""Tracks GitNexus-specific metrics during evaluation."""
def __init__(self):
self.tool_calls: dict[str, int] = {
"query": 0,
"context": 0,
"impact": 0,
"cypher": 0,
"overview": 0,
}
self.tool_calls: dict[str, int] = {key: 0 for key in TOOL_METRIC_KEYS}
self.augmentation_calls: int = 0
self.augmentation_hits: int = 0
self.augmentation_errors: int = 0
+31 -22
View File
@@ -27,6 +27,8 @@ import typer
from rich.console import Console
from rich.table import Table
from tool_registry import TOOL_METRIC_KEYS
logger = logging.getLogger("analyze_results")
console = Console()
app = typer.Typer(rich_markup_mode="rich", add_completion=False)
@@ -77,13 +79,20 @@ def load_run_results(results_dir: Path) -> dict[str, dict]:
def parse_run_id(run_id: str) -> tuple[str, str]:
"""Parse 'model_mode' into (model, mode)."""
# Handle multi-word model names like 'minimax-2.5'
# Modes are: baseline, mcp, augment, full
known_modes = {"baseline", "mcp", "augment", "full"}
parts = run_id.rsplit("_", 1)
if len(parts) == 2 and parts[1] in known_modes:
return parts[0], parts[1]
"""Parse 'model_mode' into (model, mode) using known suffixes."""
# Match the longest known suffix first to avoid hyphen collisions in model names.
known_modes = [
"native_augment",
"native",
"baseline",
"mcp",
"augment",
"full",
]
for mode in known_modes:
suffix = f"_{mode}"
if run_id.endswith(suffix):
return run_id[: -len(suffix)], mode
return run_id, "unknown"
@@ -266,12 +275,17 @@ def compare_modes(
_, mode = parse_run_id(run_id)
metrics[mode] = compute_metrics(run_data)
mode_order = [
mode
for mode in ["baseline", "native", "native_augment", "mcp", "augment", "full"]
if mode in metrics
] or sorted(metrics.keys())
# Print comparison table
table = Table(title=f"Mode Comparison: {model}")
table.add_column("Metric", style="bold")
for mode in ["baseline", "mcp", "augment", "full"]:
if mode in metrics:
table.add_column(mode, justify="right")
for mode in mode_order:
table.add_column(mode, justify="right")
rows = [
("Instances", "n_instances", "d"),
@@ -288,7 +302,7 @@ def compare_modes(
for label, key, fmt in rows:
values = []
for mode in ["baseline", "mcp", "augment", "full"]:
for mode in mode_order:
if mode in metrics:
v = metrics[mode].get(key, 0)
if fmt == ".1%":
@@ -307,8 +321,8 @@ def compare_modes(
baseline_calls = metrics["baseline"]["avg_api_calls"]
table.add_section()
for mode in ["mcp", "augment", "full"]:
if mode not in metrics:
for mode in mode_order:
if mode == "baseline":
continue
mode_cost = metrics[mode]["avg_cost"]
mode_calls = metrics[mode]["avg_api_calls"]
@@ -340,10 +354,8 @@ def gitnexus_usage(
table = Table(title="Tool Usage by Run")
table.add_column("Run", style="bold")
table.add_column("query", justify="right")
table.add_column("context", justify="right")
table.add_column("impact", justify="right")
table.add_column("cypher", justify="right")
for key in TOOL_METRIC_KEYS:
table.add_column(key, justify="right")
table.add_column("Total", justify="right")
table.add_column("Augment Hits", justify="right")
@@ -353,7 +365,7 @@ def gitnexus_usage(
continue
# Aggregate tool calls across trajectories
tool_totals: dict[str, int] = {"query": 0, "context": 0, "impact": 0, "cypher": 0, "overview": 0}
tool_totals: dict[str, int] = {key: 0 for key in TOOL_METRIC_KEYS}
augment_hits = 0
for traj in run_data.get("trajectories", {}).values():
@@ -373,10 +385,7 @@ def gitnexus_usage(
if total > 0 or augment_hits > 0:
table.add_row(
run_id,
str(tool_totals.get("query", 0)),
str(tool_totals.get("context", 0)),
str(tool_totals.get("impact", 0)),
str(tool_totals.get("cypher", 0)),
*[str(tool_totals.get(key, 0)) for key in TOOL_METRIC_KEYS],
str(total),
str(augment_hits),
)
+77 -35
View File
@@ -17,6 +17,14 @@ import time
from pathlib import Path
from typing import Any
from constants import (
MCP_FIND_GITNEXUS_FALLBACK_TIMEOUT_SECONDS,
MCP_FIND_GITNEXUS_TIMEOUT_SECONDS,
MCP_READ_TIMEOUT_SECONDS,
MCP_STOP_WAIT_SECONDS,
)
from utils.errors import is_debug_enabled, log_safe_exception
logger = logging.getLogger("mcp_bridge")
@@ -78,7 +86,7 @@ class MCPBridge:
return True
except Exception as e:
logger.error(f"Failed to start MCP bridge: {e}")
log_safe_exception(logger, "Failed to start MCP bridge", e, include_debug=is_debug_enabled())
self.stop()
return False
@@ -86,9 +94,14 @@ class MCPBridge:
"""Stop the MCP server subprocess."""
if self.process:
try:
self.process.stdin.close()
if self.process.stdin:
self.process.stdin.close()
if self.process.stdout:
self.process.stdout.close()
if self.process.stderr:
self.process.stderr.close()
self.process.terminate()
self.process.wait(timeout=5)
self.process.wait(timeout=MCP_STOP_WAIT_SECONDS)
except Exception:
try:
self.process.kill()
@@ -146,7 +159,9 @@ class MCPBridge:
try:
result = subprocess.run(
[cmd, "gitnexus", "--version"],
capture_output=True, text=True, timeout=15,
capture_output=True,
text=True,
timeout=MCP_FIND_GITNEXUS_TIMEOUT_SECONDS,
cwd=self.repo_path,
)
if result.returncode == 0:
@@ -158,7 +173,9 @@ class MCPBridge:
try:
result = subprocess.run(
["gitnexus", "--version"],
capture_output=True, text=True, timeout=10,
capture_output=True,
text=True,
timeout=MCP_FIND_GITNEXUS_FALLBACK_TIMEOUT_SECONDS,
)
if result.returncode == 0:
return "gitnexus"
@@ -194,7 +211,7 @@ class MCPBridge:
self.process.stdin.flush()
# Read response
response = self._read_response(timeout=30)
response = self._read_response(timeout=MCP_READ_TIMEOUT_SECONDS)
if response and response.get("id") == request_id:
if "error" in response:
logger.error(f"MCP error: {response['error']}")
@@ -203,7 +220,7 @@ class MCPBridge:
return None
except Exception as e:
logger.error(f"MCP request failed: {e}")
log_safe_exception(logger, "MCP request failed", e, include_debug=is_debug_enabled())
return None
def _send_notification(self, method: str, params: dict):
@@ -224,42 +241,67 @@ class MCPBridge:
self.process.stdin.write(message.encode("utf-8"))
self.process.stdin.flush()
except Exception as e:
logger.error(f"MCP notification failed: {e}")
log_safe_exception(logger, "MCP notification failed", e, include_debug=is_debug_enabled())
def _read_response(self, timeout: float = 30) -> dict | None:
def _read_content_length(self, deadline: float) -> int | None:
"""Read Content-Length header, returning the byte length or None."""
if not self.process or not self.process.stdout:
return None
header_line = b""
while time.time() < deadline:
byte = self.process.stdout.read(1)
if not byte:
return None
header_line += byte
if header_line.endswith(b"\r\n\r\n") or header_line.endswith(b"\n\n"):
break
if not header_line:
return None
header_str = header_line.decode("utf-8").strip()
for line in header_str.split("\r\n"):
if line.lower().startswith("content-length:"):
try:
return int(line.split(":", 1)[1].strip())
except (ValueError, IndexError):
return None
return None
def _read_body(self, content_length: int, deadline: float) -> bytes | None:
"""Read a response body of the expected length before deadline."""
if not self.process or not self.process.stdout:
return None
remaining = content_length
chunks: list[bytes] = []
while remaining > 0 and time.time() < deadline:
chunk = self.process.stdout.read(remaining)
if not chunk:
return None
chunks.append(chunk)
remaining -= len(chunk)
if remaining > 0:
return None
return b"".join(chunks)
def _read_response(self, timeout: float = MCP_READ_TIMEOUT_SECONDS) -> dict | None:
"""Read a JSON-RPC response from the MCP server."""
if not self.process or not self.process.stdout:
return None
start = time.time()
try:
while time.time() - start < timeout:
# Read Content-Length header
header_line = b""
while True:
byte = self.process.stdout.read(1)
if not byte:
return None
header_line += byte
if header_line.endswith(b"\r\n\r\n"):
break
if header_line.endswith(b"\n\n"):
break
# Parse content length
header_str = header_line.decode("utf-8").strip()
content_length = None
for line in header_str.split("\r\n"):
if line.lower().startswith("content-length:"):
content_length = int(line.split(":")[1].strip())
break
deadline = time.time() + timeout
while time.time() < deadline:
content_length = self._read_content_length(deadline)
if content_length is None:
continue
# Read body
body = self.process.stdout.read(content_length)
body = self._read_body(content_length, deadline)
if not body:
return None
@@ -272,7 +314,7 @@ class MCPBridge:
return None
except Exception as e:
logger.error(f"Error reading MCP response: {e}")
log_safe_exception(logger, "Error reading MCP response", e, include_debug=is_debug_enabled())
return None
+15
View File
@@ -0,0 +1,15 @@
DEBUG_ENV_VAR = "GITNEXUS_EVAL_DEBUG"
# GitNexus eval-server health checks
EVAL_SERVER_HEALTH_RETRIES = 30
EVAL_SERVER_HEALTH_INTERVAL_SECONDS = 0.5
EVAL_SERVER_HEALTH_TIMEOUT_SECONDS = 3
# MCP bridge timeouts
MCP_FIND_GITNEXUS_TIMEOUT_SECONDS = 15
MCP_FIND_GITNEXUS_FALLBACK_TIMEOUT_SECONDS = 10
MCP_READ_TIMEOUT_SECONDS = 30
MCP_STOP_WAIT_SECONDS = 5
# Agent defaults
AUGMENT_TIMEOUT_SECONDS = 5.0
+75 -81
View File
@@ -26,72 +26,20 @@ import shutil
import time
from pathlib import Path
from constants import (
EVAL_SERVER_HEALTH_INTERVAL_SECONDS,
EVAL_SERVER_HEALTH_RETRIES,
EVAL_SERVER_HEALTH_TIMEOUT_SECONDS,
)
from minisweagent.environments.docker import DockerEnvironment
from tool_registry import TOOL_SPECS, ToolScriptSpec
from utils.errors import is_debug_enabled, log_safe_exception
logger = logging.getLogger("gitnexus_docker")
DEFAULT_CACHE_DIR = Path.home() / ".gitnexus-eval-cache"
EVAL_SERVER_PORT = 4848
# Standalone tool scripts installed into /usr/local/bin/ inside the container.
# Each script calls the eval-server via curl, with a CLI fallback.
# These are standalone — no sourcing, no env inheritance needed.
TOOL_SCRIPT_QUERY = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
query="$1"; task_ctx="${2:-}"; goal="${3:-}"
[ -z "$query" ] && echo "Usage: gitnexus-query <query> [task_context] [goal]" && exit 1
args="{\"query\": \"$query\""
[ -n "$task_ctx" ] && args="$args, \"task_context\": \"$task_ctx\""
[ -n "$goal" ] && args="$args, \"goal\": \"$goal\""
args="$args}"
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/query" -H "Content-Type: application/json" -d "$args" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus query "$query" 2>&1
'''
TOOL_SCRIPT_CONTEXT = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
name="$1"; file_path="${2:-}"
[ -z "$name" ] && echo "Usage: gitnexus-context <symbol_name> [file_path]" && exit 1
args="{\"name\": \"$name\""
[ -n "$file_path" ] && args="$args, \"file_path\": \"$file_path\""
args="$args}"
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/context" -H "Content-Type: application/json" -d "$args" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus context "$name" 2>&1
'''
TOOL_SCRIPT_IMPACT = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
target="$1"; direction="${2:-upstream}"
[ -z "$target" ] && echo "Usage: gitnexus-impact <symbol_name> [upstream|downstream]" && exit 1
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/impact" -H "Content-Type: application/json" -d "{\"target\": \"$target\", \"direction\": \"$direction\"}" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus impact "$target" --direction "$direction" 2>&1
'''
TOOL_SCRIPT_CYPHER = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
query="$1"
[ -z "$query" ] && echo "Usage: gitnexus-cypher <cypher_query>" && exit 1
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/cypher" -H "Content-Type: application/json" -d "{\"query\": \"$query\"}" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus cypher "$query" 2>&1
'''
TOOL_SCRIPT_AUGMENT = r'''#!/bin/bash
cd /testbed && npx gitnexus augment "$1" 2>&1 || true
'''
TOOL_SCRIPT_OVERVIEW = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
echo "=== Code Knowledge Graph Overview ==="
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/list_repos" -H "Content-Type: application/json" -d "{}" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus list 2>&1
'''
class GitNexusDockerEnvironment(DockerEnvironment):
"""
@@ -133,7 +81,13 @@ class GitNexusDockerEnvironment(DockerEnvironment):
try:
self._setup_gitnexus()
except Exception as e:
logger.warning(f"GitNexus setup failed, continuing without it: {e}")
log_safe_exception(
logger,
"GitNexus setup failed, continuing without it",
e,
include_debug=is_debug_enabled(),
level="warning",
)
self._gitnexus_ready = False
return result
@@ -222,27 +176,59 @@ class GitNexusDockerEnvironment(DockerEnvironment):
"timeout": 5,
})
# Wait for the server to be ready (up to 15s for KuzuDB init)
for i in range(30):
time.sleep(0.5)
# Wait for the server to be ready (up to ~15s for KuzuDB init)
for i in range(EVAL_SERVER_HEALTH_RETRIES):
time.sleep(EVAL_SERVER_HEALTH_INTERVAL_SECONDS)
health = self.execute({
"command": f"curl -sf http://127.0.0.1:{self.eval_server_port}/health 2>/dev/null || echo 'NOT_READY'",
"timeout": 3,
"timeout": EVAL_SERVER_HEALTH_TIMEOUT_SECONDS,
})
output = health.get("output", "").strip()
if "NOT_READY" not in output and "ok" in output:
logger.info(f"Eval-server ready after {(i + 1) * 0.5:.1f}s")
logger.info(
f"Eval-server ready after {(i + 1) * EVAL_SERVER_HEALTH_INTERVAL_SECONDS:.1f}s"
)
return
log_output = self.execute({
"command": "cat /tmp/gitnexus-eval-server.log 2>/dev/null | tail -20",
})
logger.warning(
f"Eval-server didn't become ready in 15s. "
f"Eval-server didn't become ready in "
f"{EVAL_SERVER_HEALTH_RETRIES * EVAL_SERVER_HEALTH_INTERVAL_SECONDS:.1f}s. "
f"Tools will fall back to direct CLI.\n"
f"Server log: {log_output.get('output', 'N/A')}"
)
@staticmethod
def _render_tool_script(spec: ToolScriptSpec, port: str) -> str:
"""
Render a standalone bash script for a GitNexus tool.
Scripts call the eval-server fast path when an endpoint is present,
and fall back to the CLI otherwise.
"""
lines = ["#!/bin/bash"]
if spec.endpoint:
lines.append(f'PORT="${{GITNEXUS_EVAL_PORT:-{port}}}"')
if spec.header:
lines.append(spec.header.strip())
if spec.payload_builder:
lines.append(spec.payload_builder.strip())
if spec.endpoint:
lines.append(
f'result=$(curl -sf -X POST "http://127.0.0.1:${{PORT}}{spec.endpoint}" '
'-H "Content-Type: application/json" -d "$payload" 2>/dev/null)'
)
lines.append('if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi')
lines.append(spec.fallback.strip())
return "\n".join(lines)
def _install_tools(self):
"""
Install standalone GitNexus tool scripts in /usr/local/bin/.
@@ -259,24 +245,20 @@ class GitNexusDockerEnvironment(DockerEnvironment):
"""
port = str(self.eval_server_port)
tools = {
"gitnexus-query": TOOL_SCRIPT_QUERY,
"gitnexus-context": TOOL_SCRIPT_CONTEXT,
"gitnexus-impact": TOOL_SCRIPT_IMPACT,
"gitnexus-cypher": TOOL_SCRIPT_CYPHER,
"gitnexus-augment": TOOL_SCRIPT_AUGMENT,
"gitnexus-overview": TOOL_SCRIPT_OVERVIEW,
}
for name, script in tools.items():
script_content = script.replace("__PORT__", port).strip()
for spec in TOOL_SPECS.values():
script_content = self._render_tool_script(spec, port).strip()
# Use heredoc with quoted delimiter — prevents all variable expansion and quoting issues
self.execute({
"command": f"cat << 'GITNEXUS_SCRIPT_EOF' > /usr/local/bin/{name}\n{script_content}\nGITNEXUS_SCRIPT_EOF\nchmod +x /usr/local/bin/{name}",
"command": (
f"cat << 'GITNEXUS_SCRIPT_EOF' > /usr/local/bin/{spec.bin_name}\n"
f"{script_content}\n"
"GITNEXUS_SCRIPT_EOF\n"
f"chmod +x /usr/local/bin/{spec.bin_name}"
),
"timeout": 5,
})
logger.info(f"Installed {len(tools)} GitNexus tool scripts in /usr/local/bin/")
logger.info(f"Installed {len(TOOL_SPECS)} GitNexus tool scripts in /usr/local/bin/")
def _get_repo_info(self) -> dict:
"""Get repository identity info from the container."""
@@ -325,7 +307,13 @@ class GitNexusDockerEnvironment(DockerEnvironment):
logger.info(f"Cached GitNexus index: {cache_path}")
except Exception as e:
logger.warning(f"Failed to cache GitNexus index: {e}")
log_safe_exception(
logger,
"Failed to cache GitNexus index",
e,
include_debug=is_debug_enabled(),
level="warning",
)
if cache_path.exists():
shutil.rmtree(cache_path, ignore_errors=True)
@@ -361,7 +349,13 @@ class GitNexusDockerEnvironment(DockerEnvironment):
logger.info("GitNexus index restored from cache")
except Exception as e:
logger.warning(f"Failed to restore cache, re-indexing: {e}")
log_safe_exception(
logger,
"Failed to restore cache, re-indexing",
e,
include_debug=is_debug_enabled(),
level="warning",
)
self._index_repository()
def stop(self) -> dict:
+4 -2
View File
@@ -20,6 +20,8 @@ dependencies = [
dev = [
"pytest>=8.0.0",
"ruff>=0.5.0",
"hypothesis>=6.88.0",
"coverage>=7.6.0",
]
[project.scripts]
@@ -31,8 +33,8 @@ requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["agents", "environments", "analysis", "bridge"]
extra-files = ["run_eval.py"]
packages = ["agents", "environments", "analysis", "bridge", "utils"]
extra-files = ["run_eval.py", "tool_registry.py", "constants.py"]
[tool.ruff]
line-length = 120
+73 -37
View File
@@ -25,7 +25,6 @@ import logging
import os
import threading
import time
import traceback
from itertools import product
from pathlib import Path
from typing import Any
@@ -36,6 +35,8 @@ from rich.console import Console
from rich.live import Live
from rich.table import Table
from utils.errors import is_debug_enabled, log_safe_exception
# Load .env file from eval/ directory
_env_file = Path(__file__).parent / ".env"
if _env_file.exists():
@@ -138,6 +139,65 @@ def get_swebench_docker_image(instance: dict) -> str:
return image_name
def _build_model(config: dict):
"""Construct the model from config."""
from minisweagent.models import get_model
return get_model(config=config.get("model", {}))
def _build_environment(config: dict, instance: dict):
"""Construct the environment for the instance."""
env_config = dict(config.get("environment", {}))
env_class_name = env_config.pop("environment_class", "docker")
if env_class_name == "eval.environments.gitnexus_docker.GitNexusDockerEnvironment":
from environments.gitnexus_docker import GitNexusDockerEnvironment
env_config["image"] = get_swebench_docker_image(instance)
return GitNexusDockerEnvironment(**env_config)
from minisweagent.environments.docker import DockerEnvironment
return DockerEnvironment(image=get_swebench_docker_image(instance), **env_config)
def _build_agent(config: dict, model, env, instance_dir: Path, instance_id: str):
"""Construct the GitNexus agent with trajectory output configured."""
from agents.gitnexus_agent import GitNexusAgent
agent_config = dict(config.get("agent", {}))
agent_config.pop("agent_class", "eval.agents.gitnexus_agent.GitNexusAgent")
traj_path = instance_dir / f"{instance_id}.traj.json"
agent_config["output_path"] = traj_path
return GitNexusAgent(model, env, **agent_config)
def _extract_submission(env, info: dict, run_id: str) -> str:
"""Pull the git diff patch from the container, falling back to the agent submission."""
try:
patch_output = env.execute({"command": "cd /testbed && git diff"})
return patch_output.get("output", "").strip()
except Exception as patch_err:
logger.warning(f"[{run_id}] Failed to extract patch: {patch_err}")
return info.get("submission", "")
def _record_failure(run_id: str, instance_id: str, result: dict, error: Exception):
sanitized = log_safe_exception(
logger,
f"[{run_id}] Error on {instance_id}",
error,
include_debug=is_debug_enabled(),
)
result["exit_status"] = sanitized["error_type"]
result["error_type"] = sanitized["error_type"]
result["error_message"] = sanitized["error_message"]
result["error"] = sanitized["error_message"]
if "error_detail_debug" in sanitized:
result["error_detail_debug"] = sanitized["error_detail_debug"]
def process_instance(
instance: dict,
config: dict,
@@ -149,8 +209,6 @@ def process_instance(
Process a single SWE-bench instance with the given config.
Returns result dict with instance_id, exit_status, submission, metrics.
"""
from minisweagent.models import get_model
instance_id = instance["instance_id"]
run_id = f"{model_name}_{mode_name}"
instance_dir = output_dir / run_id / instance_id
@@ -168,31 +226,12 @@ def process_instance(
}
agent = None
env = None
try:
# Build model
model = get_model(config=config.get("model", {}))
# Build environment
env_config = dict(config.get("environment", {}))
env_class_name = env_config.pop("environment_class", "docker")
if env_class_name == "eval.environments.gitnexus_docker.GitNexusDockerEnvironment":
from environments.gitnexus_docker import GitNexusDockerEnvironment
env_config["image"] = get_swebench_docker_image(instance)
env = GitNexusDockerEnvironment(**env_config)
else:
from minisweagent.environments.docker import DockerEnvironment
env = DockerEnvironment(image=get_swebench_docker_image(instance), **env_config)
# Build agent
agent_config = dict(config.get("agent", {}))
agent_class_name = agent_config.pop("agent_class", "eval.agents.gitnexus_agent.GitNexusAgent")
from agents.gitnexus_agent import GitNexusAgent
traj_path = instance_dir / f"{instance_id}.traj.json"
agent_config["output_path"] = traj_path
agent = GitNexusAgent(model, env, **agent_config)
model = _build_model(config)
env = _build_environment(config, instance)
agent = _build_agent(config, model, env, instance_dir, instance_id)
# Run
logger.info(f"[{run_id}] Starting {instance_id}")
@@ -204,18 +243,10 @@ def process_instance(
result["gitnexus_metrics"] = agent.gitnexus_metrics.to_dict()
# Extract git diff patch from the container (SWE-bench needs the model_patch)
try:
patch_output = env.execute({"command": "cd /testbed && git diff"})
result["submission"] = patch_output.get("output", "").strip()
except Exception as patch_err:
logger.warning(f"[{run_id}] Failed to extract patch: {patch_err}")
result["submission"] = info.get("submission", "")
result["submission"] = _extract_submission(env, info, run_id)
except Exception as e:
logger.error(f"[{run_id}] Error on {instance_id}: {e}")
result["exit_status"] = type(e).__name__
result["error"] = str(e)
result["traceback"] = traceback.format_exc()
_record_failure(run_id, instance_id, result, e)
finally:
if agent:
@@ -287,7 +318,12 @@ def run_configuration(
results.append(future.result())
except Exception as e:
iid = futures[future]
logger.error(f"[{run_id}] Uncaught error for {iid}: {e}")
log_safe_exception(
logger,
f"[{run_id}] Uncaught error for {iid}",
e,
include_debug=is_debug_enabled(),
)
# Save run summary
summary = {
+1
View File
@@ -0,0 +1 @@
"""Tests for the GitNexus eval harness."""
+6
View File
@@ -0,0 +1,6 @@
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
+29
View File
@@ -0,0 +1,29 @@
from utils.errors import sanitize_exception
def _raise_value_error():
raise ValueError("boom")
def test_sanitize_exception_without_debug(monkeypatch):
monkeypatch.delenv("GITNEXUS_EVAL_DEBUG", raising=False)
try:
_raise_value_error()
except Exception as exc: # noqa: BLE001
data = sanitize_exception(exc, include_debug=False)
assert data["error_type"] == "ValueError"
assert data["error_message"] == "boom"
assert "error_detail_debug" not in data
def test_sanitize_exception_with_debug(monkeypatch):
monkeypatch.setenv("GITNEXUS_EVAL_DEBUG", "1")
try:
_raise_value_error()
except Exception as exc: # noqa: BLE001
data = sanitize_exception(exc)
assert data["error_type"] == "ValueError"
assert "error_detail_debug" in data
assert "ValueError" in data["error_detail_debug"]
+20
View File
@@ -0,0 +1,20 @@
import pytest
analysis_module = pytest.importorskip("analysis.analyze_results")
parse_run_id = analysis_module.parse_run_id
def test_parse_run_id_native_augment():
model, mode = parse_run_id("claude-sonnet_native_augment")
assert model == "claude-sonnet"
assert mode == "native_augment"
def test_parse_run_id_hyphenated_model():
model, mode = parse_run_id("glm-4.7_native")
assert model == "glm-4.7"
assert mode == "native"
def test_parse_run_id_unknown():
assert parse_run_id("custom_model") == ("custom_model", "unknown")
+63
View File
@@ -0,0 +1,63 @@
from __future__ import annotations
from hypothesis import given
from hypothesis import strategies as st
from analysis.analyze_results import parse_run_id
from environments.gitnexus_docker import GitNexusDockerEnvironment
from tool_registry import TOOL_SPECS
from utils.errors import sanitize_exception
KNOWN_MODES = [
"native_augment",
"native",
"baseline",
"mcp",
"augment",
"full",
]
def _model_strategy():
base_chars = st.characters(
blacklist_categories=("Cs",),
blacklist_characters={" ", "\n", "\t"},
)
text = st.text(alphabet=base_chars, min_size=1)
return text.filter(lambda s: not any(s.endswith(f"_{m}") for m in KNOWN_MODES))
@given(_model_strategy(), st.sampled_from(KNOWN_MODES))
def test_parse_run_id_round_trip(model: str, mode: str) -> None:
run_id = f"{model}_{mode}"
parsed_model, parsed_mode = parse_run_id(run_id)
assert parsed_model == model
assert parsed_mode == mode
@given(st.text())
def test_sanitize_exception_respects_debug_flag(message: str) -> None:
exc = ValueError(message)
data = sanitize_exception(exc, include_debug=False)
assert data["error_type"] == "ValueError"
assert data["error_message"] == (message or "ValueError")
assert "error_detail_debug" not in data
data_debug = sanitize_exception(exc, include_debug=True)
assert data_debug["error_type"] == "ValueError"
assert "error_detail_debug" in data_debug
assert data_debug["error_detail_debug"]
@given(st.sampled_from(list(TOOL_SPECS.values())), st.integers(min_value=1, max_value=99999))
def test_render_tool_script_contains_expected_paths(spec, port: int) -> None:
script = GitNexusDockerEnvironment._render_tool_script(spec, str(port))
assert spec.fallback.strip() in script
if spec.endpoint:
assert spec.endpoint in script
assert f"${{GITNEXUS_EVAL_PORT:-{port}}}" in script
assert "curl" in script
else:
assert "curl" not in script
+21
View File
@@ -0,0 +1,21 @@
import pytest
GitNexusDockerEnvironment = pytest.importorskip(
"environments.gitnexus_docker"
).GitNexusDockerEnvironment
tool_registry = pytest.importorskip("tool_registry")
TOOL_SPECS = tool_registry.TOOL_SPECS
def test_render_query_script_uses_endpoint_and_fallback():
script = GitNexusDockerEnvironment._render_tool_script(TOOL_SPECS["query"], "4848")
assert "/tool/query" in script
assert "gitnexus query" in script
assert "GITNEXUS_EVAL_PORT" in script
def test_render_augment_script_skips_curl():
script = GitNexusDockerEnvironment._render_tool_script(TOOL_SPECS["augment"], "4848")
assert "/tool/" not in script
assert "curl" not in script
assert "gitnexus augment" in script
+79
View File
@@ -0,0 +1,79 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Dict, Tuple
@dataclass(frozen=True)
class ToolScriptSpec:
key: str
bin_name: str
endpoint: str | None
payload_builder: str
fallback: str
header: str | None = None
TOOL_METRIC_KEYS: Tuple[str, ...] = ("query", "context", "impact", "cypher", "overview")
TOOL_SPECS: Dict[str, ToolScriptSpec] = {
"query": ToolScriptSpec(
key="query",
bin_name="gitnexus-query",
endpoint="/tool/query",
payload_builder=r'''query="$1"; task_ctx="${2:-}"; goal="${3:-}"
[ -z "$query" ] && echo "Usage: gitnexus-query <query> [task_context] [goal]" && exit 1
payload="{\"query\": \"$query\""
[ -n "$task_ctx" ] && payload="$payload, \"task_context\": \"$task_ctx\""
[ -n "$goal" ] && payload="$payload, \"goal\": \"$goal\""
payload="$payload}"''',
fallback='cd /testbed && npx gitnexus query "$query" 2>&1',
),
"context": ToolScriptSpec(
key="context",
bin_name="gitnexus-context",
endpoint="/tool/context",
payload_builder=r'''name="$1"; file_path="${2:-}"
[ -z "$name" ] && echo "Usage: gitnexus-context <symbol_name> [file_path]" && exit 1
payload="{\"name\": \"$name\""
[ -n "$file_path" ] && payload="$payload, \"file_path\": \"$file_path\""
payload="$payload}"''',
fallback='cd /testbed && npx gitnexus context "$name" 2>&1',
),
"impact": ToolScriptSpec(
key="impact",
bin_name="gitnexus-impact",
endpoint="/tool/impact",
payload_builder=r'''target="$1"; direction="${2:-upstream}"
[ -z "$target" ] && echo "Usage: gitnexus-impact <symbol_name> [upstream|downstream]" && exit 1
payload="{\"target\": \"$target\", \"direction\": \"$direction\"}"''',
fallback='cd /testbed && npx gitnexus impact "$target" --direction "$direction" 2>&1',
),
"cypher": ToolScriptSpec(
key="cypher",
bin_name="gitnexus-cypher",
endpoint="/tool/cypher",
payload_builder=r'''query="$1"
[ -z "$query" ] && echo "Usage: gitnexus-cypher <cypher_query>" && exit 1
payload="{\"query\": \"$query\"}"''',
fallback='cd /testbed && npx gitnexus cypher "$query" 2>&1',
),
"overview": ToolScriptSpec(
key="overview",
bin_name="gitnexus-overview",
endpoint="/tool/list_repos",
header='echo "=== Code Knowledge Graph Overview ==="',
payload_builder='payload="{}"',
fallback='cd /testbed && npx gitnexus list 2>&1',
),
"augment": ToolScriptSpec(
key="augment",
bin_name="gitnexus-augment",
endpoint=None,
payload_builder="",
fallback='cd /testbed && npx gitnexus augment "$1" 2>&1 || true',
),
}
BINARIES_BY_KEY: Dict[str, str] = {spec.key: spec.bin_name for spec in TOOL_SPECS.values()}
ENDPOINTS_BY_KEY: Dict[str, str | None] = {spec.key: spec.endpoint for spec in TOOL_SPECS.values()}
+1
View File
@@ -0,0 +1 @@
"""Utility package for the eval harness."""
+60
View File
@@ -0,0 +1,60 @@
from __future__ import annotations
import os
import traceback
from typing import Any, Callable
from constants import DEBUG_ENV_VAR
def is_debug_enabled() -> bool:
"""Return True when debug output (full tracebacks) should be emitted."""
return os.getenv(DEBUG_ENV_VAR, "").strip().lower() in {"1", "true", "yes", "on"}
def sanitize_exception(exc: BaseException, *, include_debug: bool | None = None) -> dict[str, str]:
"""
Produce a log-safe, JSON-friendly view of an exception.
- Always returns error_type and error_message.
- Only includes error_detail_debug (full traceback) when debug is enabled.
"""
debug = is_debug_enabled() if include_debug is None else include_debug
error_type = type(exc).__name__
message = str(exc) or error_type
data: dict[str, str] = {
"error_type": error_type,
"error_message": message,
}
if debug:
tb = "".join(traceback.format_exception(type(exc), exc, exc.__traceback__))
if tb:
data["error_detail_debug"] = tb
return data
def log_safe_exception(
logger: Any,
prefix: str,
exc: BaseException,
*,
include_debug: bool | None = None,
level: str = "error",
) -> dict[str, str]:
"""
Log an exception without leaking stack traces unless debug is enabled.
Returns the sanitized dict so callers can persist it.
"""
data = sanitize_exception(exc, include_debug=include_debug)
debug = "error_detail_debug" in data
log_fn: Callable[..., None] = getattr(logger, level, logger.error)
message = f"{prefix}: {data['error_type']}: {data['error_message']}"
log_kwargs = {"exc_info": True} if debug else {}
log_fn(message, **log_kwargs)
return data
Generated
+2529
View File
File diff suppressed because it is too large Load Diff
+127
View File
@@ -0,0 +1,127 @@
import { test, expect, type TestInfo } from '@playwright/test';
/**
* Debug harnesses for investigating specific UI issues.
* Excluded from `npm run test:e2e` via testIgnore in playwright.config.ts.
* Run directly: DEBUG_E2E=1 npx playwright test e2e/debug-issues.spec.ts
*/
const BACKEND_URL = process.env.BACKEND_URL ?? 'http://localhost:4747';
const debugTest = process.env.DEBUG_E2E ? test : test.skip;
async function connectToServer(page: import('@playwright/test').Page) {
page.on('console', msg => {
if (msg.type() === 'error') console.log(`[error] ${msg.text()}`);
});
await page.goto('/');
await page.getByText('Server').click();
const serverInput = page.locator('input[name="server-url-input"]');
await serverInput.fill(BACKEND_URL);
await page.getByRole('button', { name: /Connect/ }).click();
await expect(page.locator('[data-testid="status-ready"]')).toBeVisible({ timeout: 30_000 });
// Wait for LadybugDB to finish loading — poll isDatabaseReady via process list visibility
await expect(page.locator('[data-testid="status-ready"]')).toBeVisible({ timeout: 30_000 });
}
debugTest('debug: process view Reset View button', async ({ page }, testInfo) => {
await connectToServer(page);
// Open Processes tab
await page.getByRole('button', { name: 'Nexus AI' }).click();
await page.getByText('Processes').click();
await expect(page.getByText(/\d+ processes detected/)).toBeVisible({ timeout: 10_000 });
// Click View on the first Cross-Community process
// The View button has opacity-0 by default, use JS click to bypass
const viewButtons = page.locator('button:has-text("View")');
const count = await viewButtons.count();
console.log(`Found ${count} View buttons`);
// Use evaluate to click the first one regardless of visibility
await page.evaluate(() => {
const btns = document.querySelectorAll('button');
for (const btn of btns) {
if (btn.textContent?.trim() === 'View') {
btn.click();
return;
}
}
});
// Wait for modal to appear
const modal = page.locator('[data-testid="process-modal"]');
await expect(modal).toBeVisible({ timeout: 5_000 });
// Screenshot: modal should be open with flowchart
await page.screenshot({ path: testInfo.outputPath('debug-modal-open.png'), fullPage: true });
// Get the diagram's current transform
const diagramDiv = modal.locator('[style*="transform"]');
const transformBefore = await diagramDiv.getAttribute('style');
console.log('Transform BEFORE zoom:', transformBefore);
// Zoom in using the + button
const zoomInBtn = modal.getByRole('button', { name: /Zoom in/ });
await zoomInBtn.click();
await zoomInBtn.click();
await zoomInBtn.click();
// Wait for zoom animation to settle
await expect(async () => {
const t = await diagramDiv.getAttribute('style');
expect(t).not.toBe(transformBefore);
}).toPass({ timeout: 2_000 });
const transformAfterZoom = await diagramDiv.getAttribute('style');
console.log('Transform AFTER zoom:', transformAfterZoom);
await page.screenshot({ path: testInfo.outputPath('debug-modal-zoomed.png'), fullPage: true });
// Click Reset View
const resetBtn = modal.getByRole('button', { name: 'Reset View' });
await resetBtn.click();
// Wait for reset animation to settle
await expect(async () => {
const t = await diagramDiv.getAttribute('style');
expect(t).toBe(transformBefore);
}).toPass({ timeout: 2_000 });
const transformAfterReset = await diagramDiv.getAttribute('style');
console.log('Transform AFTER reset:', transformAfterReset);
await page.screenshot({ path: testInfo.outputPath('debug-modal-after-reset.png'), fullPage: true });
// Verify transform actually changed back
expect(transformAfterZoom).not.toBe(transformBefore);
expect(transformAfterReset).toBe(transformBefore);
});
debugTest('debug: lightbulb clears node selection dimming', async ({ page }, testInfo) => {
await connectToServer(page);
// Wait for graph canvas to render
await expect(page.locator('canvas').first()).toBeVisible({ timeout: 10_000 });
await page.screenshot({ path: testInfo.outputPath('debug-before-select.png'), fullPage: true });
// Click a file in the tree to select a node (causes dimming)
const fileItem = page.getByText('start.sh');
await fileItem.click();
await page.waitForTimeout(500);
await page.screenshot({ path: testInfo.outputPath('debug-node-selected.png'), fullPage: true });
// Check the lightbulb button state
const lightbulbBtn = page.locator('button[title*="Turn off"], button[title*="Turn on"]');
const title = await lightbulbBtn.getAttribute('title');
console.log('Lightbulb title before click:', title);
// Click the lightbulb
await lightbulbBtn.click();
await page.waitForTimeout(500);
const titleAfter = await lightbulbBtn.getAttribute('title');
console.log('Lightbulb title after click:', titleAfter);
await page.screenshot({ path: testInfo.outputPath('debug-after-lightbulb.png'), fullPage: true });
// Click it again to toggle back on
await lightbulbBtn.click();
await page.waitForTimeout(500);
await page.screenshot({ path: testInfo.outputPath('debug-after-lightbulb-toggle-back.png'), fullPage: true });
});
+28
View File
@@ -0,0 +1,28 @@
import { test } from '@playwright/test';
/**
* Manual recording session for interactive debugging.
* Opens the app and pauses so you can interact with the UI.
* Trace, video, and screenshots are saved automatically on close.
*
* Run with: npx playwright test e2e/manual-record.spec.ts --headed --timeout=0
*
* Excluded from `npm run test:e2e` via testIgnore in playwright.config.ts.
* Also skipped when PWDEBUG is not set or in CI, as a safety net.
*/
test.skip(
!!process.env.CI || process.env.PWDEBUG !== '1',
'Manual recording requires --headed and PWDEBUG=1. Run: PWDEBUG=1 npx playwright test e2e/manual-record.spec.ts --headed --timeout=0'
);
test('manual recording session', async ({ page }) => {
page.on('console', msg => {
if (msg.type() === 'error' || msg.type() === 'warning') {
console.log(`[${msg.type()}] ${msg.text()}`);
}
});
page.on('pageerror', err => console.log(`[crash] ${err.message}`));
await page.goto('http://localhost:5173');
await page.pause();
});
+165
View File
@@ -0,0 +1,165 @@
import { test, expect, type TestInfo } from '@playwright/test';
/**
* E2E tests for the GitNexus web UI.
* Requires:
* - gitnexus serve running on localhost:4747
* - gitnexus-web dev server running on localhost:5173
*
* Skipped when servers aren't available (CI without services, etc.).
* Set E2E=1 to force-run even without the availability check.
*/
const BACKEND_URL = process.env.BACKEND_URL ?? 'http://localhost:4747';
const FRONTEND_URL = process.env.FRONTEND_URL ?? 'http://localhost:5173';
// Skip all tests if the gitnexus server or Vite dev server isn't reachable
test.beforeAll(async () => {
if (process.env.E2E) return; // force-run
try {
const [backendRes, frontendRes] = await Promise.allSettled([
fetch(`${BACKEND_URL}/api/repos`),
fetch(FRONTEND_URL),
]);
if (backendRes.status === 'rejected' || (backendRes.status === 'fulfilled' && !backendRes.value.ok)) {
test.skip(true, 'gitnexus serve not available on :4747');
return;
}
if (frontendRes.status === 'rejected' || (frontendRes.status === 'fulfilled' && !frontendRes.value.ok)) {
test.skip(true, 'Vite dev server not available on :5173');
return;
}
} catch {
test.skip(true, 'servers not available');
}
});
/** Shared helper: connect to the local server and wait for the graph to load */
async function connectAndWaitForGraph(page: import('@playwright/test').Page, testInfo: TestInfo) {
// Signal to the app that we are running under Playwright (used to skip heavy Ladybug loads).
await page.addInitScript(() => {
(window as unknown as { __PLAYWRIGHT_TEST__?: boolean }).__PLAYWRIGHT_TEST__ = true;
});
await page.goto('/');
// Wait for the app to fully render before interacting
const serverTab = page.getByRole('button', { name: 'Server' });
await expect(serverTab).toBeVisible({ timeout: 15_000 });
await page.screenshot({ path: testInfo.outputPath('step-1-landing.png') });
// Click "Server" tab and wait for the input to appear
await serverTab.click();
const serverInput = page.locator('input[name="server-url-input"]');
await expect(serverInput).toBeVisible({ timeout: 15_000 });
await page.screenshot({ path: testInfo.outputPath('step-2-server-tab.png') });
await serverInput.fill(BACKEND_URL);
await page.screenshot({ path: testInfo.outputPath('step-3-url-filled.png') });
await page.getByRole('button', { name: /Connect/ }).click();
// Wait for graph to load — status bar shows "Ready"
await expect(page.locator('[data-testid="status-ready"]')).toBeVisible({ timeout: 30_000 });
await expect(page.getByText(/\d+ nodes/).first()).toBeVisible();
await page.screenshot({ path: testInfo.outputPath('step-4-graph-loaded.png') });
}
test.describe('Server Connection & Graph Loading', () => {
test('connects to server and loads graph', async ({ page }, testInfo) => {
await connectAndWaitForGraph(page, testInfo);
await page.screenshot({ path: testInfo.outputPath('graph-loaded.png'), fullPage: true });
});
});
test.describe('Nexus AI', () => {
test('panel opens and agent initializes without error', async ({ page }, testInfo) => {
await connectAndWaitForGraph(page, testInfo);
// Click Nexus AI button to open the panel
await page.getByRole('button', { name: 'Nexus AI' }).click();
// Should see the Nexus AI tab content
await expect(page.getByText('Ask me anything')).toBeVisible({ timeout: 15_000 });
await page.screenshot({ path: testInfo.outputPath('nexus-ai-panel.png'), fullPage: true });
// "Database not ready" should NOT be visible
const errorBanner = page.getByText('Database not ready');
expect(await errorBanner.isVisible().catch(() => false)).toBe(false);
});
});
test.describe('Processes Panel', () => {
test('shows process list and View button works', async ({ page }, testInfo) => {
await connectAndWaitForGraph(page, testInfo);
// Open Nexus AI panel, switch to Processes tab
await page.getByRole('button', { name: 'Nexus AI' }).click();
await page.getByText('Processes').click();
// Should show process count — wait for data-testid instead of fixed timeout
await expect(page.locator('[data-testid="process-list-loaded"]')).toBeVisible({ timeout: 15_000 });
await page.screenshot({ path: testInfo.outputPath('processes-panel.png'), fullPage: true });
// Hover first process item to reveal View button, then click it
const processRow = page.locator('[data-testid="process-row"]').first();
await expect(processRow).toBeVisible({ timeout: 10_000 });
await processRow.hover();
const viewBtn = processRow.locator('[data-testid="process-view-button"]');
await viewBtn.waitFor({ state: 'visible', timeout: 5_000 });
await viewBtn.click();
// Wait for modal to appear
await expect(page.locator('[data-testid="process-modal"]')).toBeVisible({ timeout: 5_000 });
await page.screenshot({ path: testInfo.outputPath('process-view-clicked.png'), fullPage: true });
});
test('lightbulb highlights nodes in graph', async ({ page }, testInfo) => {
await connectAndWaitForGraph(page, testInfo);
await page.getByRole('button', { name: 'Nexus AI' }).click();
await page.getByText('Processes').click();
await expect(page.locator('[data-testid="process-list-loaded"]')).toBeVisible({ timeout: 15_000 });
await page.screenshot({ path: testInfo.outputPath('before-highlight.png'), fullPage: true });
// Hover first process to reveal lightbulb
const processRow = page.locator('[data-testid="process-row"]').first();
await expect(processRow).toBeVisible({ timeout: 10_000 });
await processRow.hover();
const lightbulb = processRow.locator('[data-testid="process-highlight-button"]');
await lightbulb.waitFor({ state: 'visible', timeout: 5_000 });
await lightbulb.click();
// Wait for highlight to apply — the process row gets amber styling when focused
await expect(processRow).toHaveClass(/bg-amber-950/, { timeout: 5_000 });
await page.screenshot({ path: testInfo.outputPath('after-highlight.png'), fullPage: true });
});
});
test.describe('Turn Off All Highlights', () => {
test('selecting a node dims others, button clears it', async ({ page }, testInfo) => {
await connectAndWaitForGraph(page, testInfo);
// Wait for graph to fully render by checking for canvas element
await expect(page.locator('canvas').first()).toBeVisible({ timeout: 10_000 });
await page.screenshot({ path: testInfo.outputPath('before-select.png'), fullPage: true });
// Click a file in the file tree to select a node
const fileItem = page.getByText('package.json').first();
await expect(fileItem).toBeVisible({ timeout: 10_000 });
await fileItem.click();
// Wait for highlight toggle to show "Turn off" (indicates highlights are active)
const highlightToggle = page.locator('[data-testid="ai-highlights-toggle"]');
await expect(highlightToggle).toHaveAttribute('title', 'Turn off all highlights', { timeout: 5_000 });
await page.screenshot({ path: testInfo.outputPath('node-selected.png'), fullPage: true });
// Click the toggle to clear all highlights
await highlightToggle.click();
// Verify highlights are now off — button title changes to "Turn on"
await expect(highlightToggle).toHaveAttribute('title', 'Turn on AI highlights', { timeout: 5_000 });
await page.screenshot({ path: testInfo.outputPath('highlights-cleared.png'), fullPage: true });
});
});
+1882 -838
View File
File diff suppressed because it is too large Load Diff
+23 -6
View File
@@ -2,16 +2,25 @@
"name": "gitnexus",
"private": true,
"version": "0.0.0",
"engines": {
"node": ">=20.0.0"
},
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc -b && vite build",
"preview": "vite preview",
"test": "vitest run"
"test": "vitest run",
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
"test:e2e": "playwright test",
"test:e2e:ui": "playwright test --ui",
"test:e2e:report": "playwright show-report"
},
"dependencies": {
"@huggingface/transformers": "^3.0.0",
"@isomorphic-git/lightning-fs": "^4.6.2",
"@ladybugdb/wasm-core": "^0.15.1",
"@langchain/anthropic": "^1.3.10",
"@langchain/core": "^1.1.15",
"@langchain/google-genai": "^2.1.10",
@@ -24,22 +33,22 @@
"buffer": "^6.0.3",
"comlink": "^4.4.2",
"d3": "^7.9.0",
"dompurify": "^3.3.3",
"graphology": "^0.26.0",
"graphology-indices": "^0.17.0",
"graphology-utils": "^2.3.0",
"mnemonist": "^0.39.0",
"pandemonium": "^2.4.0",
"graphology-layout-force": "^0.2.4",
"graphology-layout-forceatlas2": "^0.10.1",
"graphology-layout-noverlap": "^0.4.2",
"graphology-utils": "^2.3.0",
"isomorphic-git": "^1.36.1",
"jszip": "^3.10.1",
"@ladybugdb/wasm-core": "^0.15.2",
"langchain": "^1.2.10",
"lru-cache": "^11.2.4",
"lucide-react": "^0.562.0",
"mermaid": "^11.12.2",
"minisearch": "^7.2.0",
"mnemonist": "^0.39.0",
"pandemonium": "^2.4.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-markdown": "^10.1.0",
@@ -56,6 +65,11 @@
},
"devDependencies": {
"@babel/types": "^7.28.5",
"@playwright/test": "^1.58.2",
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/react": "^16.3.2",
"@testing-library/user-event": "^14.6.1",
"@types/dompurify": "^3.0.5",
"@types/jszip": "^3.4.0",
"@types/node": "^24.10.1",
"@types/react": "^18.3.5",
@@ -63,10 +77,13 @@
"@types/react-syntax-highlighter": "^15.5.13",
"@vercel/node": "^5.5.16",
"@vitejs/plugin-react": "^5.1.0",
"@vitest/coverage-v8": "^3.2.4",
"jsdom": "^29.0.0",
"tree-sitter-wasms": "^0.1.13",
"typescript": "^5.4.5",
"vite": "^5.2.0",
"vite-plugin-static-copy": "^3.1.4",
"vitest": "^4.0.18"
"vitest": "^3.2.4",
"wait-on": "^8.0.5"
}
}
+48
View File
@@ -0,0 +1,48 @@
import { defineConfig } from '@playwright/test';
// Enable insecure browser config (disabled security + CSP bypass) only when explicitly requested.
// Example: PLAYWRIGHT_INSECURE=1 npx playwright test
const insecureE2E = process.env.PLAYWRIGHT_INSECURE === '1';
// Base launch args: always enable software WebGL for sigma.js graph rendering in headless mode.
const launchArgs = [
'--use-gl=angle',
'--use-angle=swiftshader',
'--enable-webgl',
'--enable-unsafe-swiftshader',
];
if (insecureE2E) {
// Allow cross-origin requests to gitnexus serve on a different port when explicitly enabled.
launchArgs.unshift('--disable-web-security', '--disable-site-isolation-trials');
}
export default defineConfig({
testDir: './e2e',
testIgnore: ['**/manual-record.spec.ts', '**/debug-issues.spec.ts'],
timeout: 60_000,
retries: process.env.CI ? 1 : 0,
use: {
baseURL: 'http://localhost:5173',
trace: 'retain-on-failure',
screenshot: 'retain-on-failure',
video: 'retain-on-failure',
launchOptions: {
args: launchArgs,
},
// Vite dev server sets COEP require-corp for SharedArrayBuffer (LadybugDB WASM).
// Only bypass CSP when explicitly running in insecure E2E mode.
bypassCSP: insecureE2E,
},
projects: [
{
name: 'chromium',
use: { browserName: 'chromium' },
},
],
reporter: [
['list'],
['html', { open: 'never', outputFolder: 'playwright-report' }],
],
outputDir: 'test-results',
});
+19 -43
View File
@@ -13,7 +13,7 @@ import { FileEntry } from './services/zip';
import { getActiveProviderConfig } from './core/llm/settings-service';
import { createKnowledgeGraph } from './core/graph/graph';
import { connectToServer, fetchRepos, normalizeServerUrl, type ConnectToServerResult } from './services/server-connection';
import { HelpPanel } from './components/HelpPanel';
import { ERROR_RESET_DELAY_MS } from './config/ui-constants';
const AppContent = () => {
const {
@@ -29,11 +29,9 @@ const AppContent = () => {
runPipelineFromFiles,
isSettingsPanelOpen,
setSettingsPanelOpen,
isHelpDialogBoxOpen,
setHelpDialogBoxOpen,
refreshLLMSettings,
initializeAgent,
startEmbeddings,
startEmbeddingsWithFallback,
embeddingStatus,
codeReferences,
selectedNode,
@@ -44,7 +42,6 @@ const AppContent = () => {
setAvailableRepos,
switchRepo,
loadServerGraph,
graph
} = useAppState();
const graphCanvasRef = useRef<GraphCanvasHandle>(null);
@@ -72,13 +69,7 @@ const AppContent = () => {
// Auto-start embeddings pipeline in background
// Uses WebGPU if available, falls back to WASM
startEmbeddings().catch((err) => {
if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) {
startEmbeddings('wasm').catch(console.warn);
} else {
console.warn('Embeddings auto-start failed:', err);
}
});
startEmbeddingsWithFallback();
} catch (error) {
console.error('Pipeline error:', error);
setProgress({
@@ -90,13 +81,16 @@ const AppContent = () => {
setTimeout(() => {
setViewMode('onboarding');
setProgress(null);
}, 3000);
}, ERROR_RESET_DELAY_MS);
}
}, [setViewMode, setGraph, setFileContents, setProgress, setProjectName, runPipeline, startEmbeddings, initializeAgent]);
}, [setViewMode, setGraph, setFileContents, setProgress, setProjectName, runPipeline, startEmbeddingsWithFallback, initializeAgent]);
const handleGitClone = useCallback(async (files: FileEntry[]) => {
const firstPath = files[0]?.path || 'repository';
const projectName = firstPath.split('/')[0].replace(/-\d+$/, '') || 'repository';
const handleGitClone = useCallback(async (files: FileEntry[], repoName?: string) => {
let projectName = repoName;
if (!projectName) {
const firstPath = files[0]?.path || 'repository';
projectName = firstPath.split('/')[0].replace(/-\d+$/, '') || 'repository';
}
setProjectName(projectName);
setProgress({ phase: 'extracting', percent: 0, message: 'Starting...', detail: 'Preparing to process files' });
@@ -115,13 +109,7 @@ const AppContent = () => {
initializeAgent(projectName);
}
startEmbeddings().catch((err) => {
if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) {
startEmbeddings('wasm').catch(console.warn);
} else {
console.warn('Embeddings auto-start failed:', err);
}
});
startEmbeddingsWithFallback();
} catch (error) {
console.error('Pipeline error:', error);
setProgress({
@@ -133,14 +121,15 @@ const AppContent = () => {
setTimeout(() => {
setViewMode('onboarding');
setProgress(null);
}, 3000);
}, ERROR_RESET_DELAY_MS);
}
}, [setViewMode, setGraph, setFileContents, setProgress, setProjectName, runPipelineFromFiles, startEmbeddings, initializeAgent]);
}, [setViewMode, setGraph, setFileContents, setProgress, setProjectName, runPipelineFromFiles, startEmbeddingsWithFallback, initializeAgent]);
const handleServerConnect = useCallback((result: ConnectToServerResult): Promise<void> => {
// Extract project name from repoPath
const repoPath = result.repoInfo.repoPath;
const projectName = repoPath.split('/').pop() || 'server-project';
const parts = repoPath.split('/').filter(p => p && !p.startsWith('.'));
const projectName = parts[parts.length - 1] || parts[0] || 'server-project';
setProjectName(projectName);
// Build KnowledgeGraph from server data for visualization
@@ -172,13 +161,7 @@ const AppContent = () => {
}
})
.then(() => {
startEmbeddings().catch((err) => {
if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) {
startEmbeddings('wasm').catch(console.warn);
} else {
console.warn('Embeddings auto-start failed:', err);
}
});
startEmbeddingsWithFallback();
})
.catch((err) => {
console.warn('Failed to load graph into LadybugDB:', err);
@@ -186,7 +169,7 @@ const AppContent = () => {
});
return loadGraphPromise;
}, [setViewMode, setGraph, setFileContents, setProjectName, loadServerGraph, initializeAgent, startEmbeddings]);
}, [setViewMode, setGraph, setFileContents, setProjectName, loadServerGraph, initializeAgent, startEmbeddingsWithFallback]);
// Auto-connect when ?server query param is present (bookmarkable shortcut)
const autoConnectRan = useRef(false);
@@ -235,7 +218,7 @@ const AppContent = () => {
setTimeout(() => {
setViewMode('onboarding');
setProgress(null);
}, 3000);
}, ERROR_RESET_DELAY_MS);
});
}, [handleServerConnect, setProgress, setViewMode, setServerBaseUrl, setAvailableRepos]);
@@ -309,13 +292,6 @@ const AppContent = () => {
onSettingsSaved={handleSettingsSaved}
/>
<HelpPanel
isOpen={isHelpDialogBoxOpen}
onClose={() => setHelpDialogBoxOpen(false)}
nodeCount={graph!.nodes.length}
edgeCount={graph!.relationships.length}
/>
</div>
);
};
@@ -1,4 +1,4 @@
import { Server, ArrowRight } from 'lucide-react';
import { Server, ArrowRight } from '@/lib/lucide-icons';
import { BackendRepo } from '../services/backend';
interface BackendRepoSelectorProps {
@@ -1,8 +1,9 @@
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { Code, PanelLeftClose, PanelLeft, Trash2, X, Target, FileCode, Sparkles, MousePointerClick } from 'lucide-react';
import { Code, PanelLeftClose, PanelLeft, Trash2, X, Target, FileCode, Sparkles, MousePointerClick } from '@/lib/lucide-icons';
import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
import { vscDarkPlus } from 'react-syntax-highlighter/dist/esm/styles/prism';
import { useAppState } from '../hooks/useAppState';
import type { GraphNode } from '../core/graph/types';
import { NODE_COLORS } from '../lib/constants';
/** Map file extension to Prism syntax highlighter language identifier */
@@ -75,6 +76,11 @@ export const CodeReferencesPanel = ({ onFocusNode }: CodeReferencesPanelProps) =
codeReferenceFocus,
} = useAppState();
const nodeById = useMemo(() => {
if (!graph) return new Map<string, GraphNode>();
return new Map(graph.nodes.map(n => [n.id, n]));
}, [graph]);
const [isCollapsed, setIsCollapsed] = useState(false);
const [glowRefId, setGlowRefId] = useState<string | null>(null);
const panelRef = useRef<HTMLElement | null>(null);
@@ -161,8 +167,9 @@ export const CodeReferencesPanel = ({ onFocusNode }: CodeReferencesPanelProps) =
if (!target) return;
// Double rAF: wait for collapse state + list DOM to render.
requestAnimationFrame(() => {
requestAnimationFrame(() => {
const rafIds: number[] = [];
const outerRafId = requestAnimationFrame(() => {
const innerRafId = requestAnimationFrame(() => {
const el = refCardEls.current.get(target.id);
if (!el) return;
@@ -177,7 +184,13 @@ export const CodeReferencesPanel = ({ onFocusNode }: CodeReferencesPanelProps) =
glowTimerRef.current = null;
}, 1200);
});
rafIds.push(innerRafId);
});
rafIds.push(outerRafId);
return () => {
rafIds.forEach(id => cancelAnimationFrame(id));
};
}, [codeReferenceFocus?.ts, aiReferences]);
const refsWithSnippets = useMemo(() => {
@@ -414,7 +427,7 @@ export const CodeReferencesPanel = ({ onFocusNode }: CodeReferencesPanelProps) =
const nodeId = ref.nodeId!;
// Sync selection + focus graph
if (graph) {
const node = graph.nodes.find((n) => n.id === nodeId);
const node = nodeById.get(nodeId);
if (node) setSelectedNode(node);
}
onFocusNode(nodeId);
+15 -7
View File
@@ -1,12 +1,12 @@
import { useState, useCallback, useRef, DragEvent } from 'react';
import { Upload, FileArchive, Github, Loader2, ArrowRight, Key, Eye, EyeOff, Globe, X } from 'lucide-react';
import { Upload, FileArchive, Github, Loader2, ArrowRight, Key, Eye, EyeOff, Globe, X } from '@/lib/lucide-icons';
import { cloneRepository, parseGitHubUrl } from '../services/git-clone';
import { connectToServer, type ConnectToServerResult } from '../services/server-connection';
import { FileEntry } from '../services/zip';
interface DropZoneProps {
onFileSelect: (file: File) => void;
onGitClone?: (files: FileEntry[]) => void;
onGitClone?: (files: FileEntry[], repoName?: string) => void;
onServerConnect?: (result: ConnectToServerResult, serverUrl?: string) => void;
}
@@ -27,9 +27,13 @@ export const DropZone = ({ onFileSelect, onGitClone, onServerConnect }: DropZone
const [error, setError] = useState<string | null>(null);
// Server tab state
const [serverUrl, setServerUrl] = useState(() =>
localStorage.getItem('gitnexus-server-url') || ''
);
const [serverUrl, setServerUrl] = useState(() => {
try {
return localStorage.getItem('gitnexus-server-url') || '';
} catch {
return '';
}
});
const [isConnecting, setIsConnecting] = useState(false);
const [serverProgress, setServerProgress] = useState<{
phase: string;
@@ -104,7 +108,7 @@ export const DropZone = ({ onFileSelect, onGitClone, onServerConnect }: DropZone
setGithubToken('');
if (onGitClone) {
onGitClone(files);
onGitClone(files, parsed.repo);
}
} catch (err) {
console.error('Clone failed:', err);
@@ -133,7 +137,11 @@ export const DropZone = ({ onFileSelect, onGitClone, onServerConnect }: DropZone
}
// Persist URL to localStorage
localStorage.setItem('gitnexus-server-url', serverUrl);
try {
localStorage.setItem('gitnexus-server-url', serverUrl);
} catch {
// localStorage may be unavailable (e.g. private browsing, quota exceeded)
}
setError(null);
setIsConnecting(true);
@@ -1,4 +1,4 @@
import { Brain, Loader2, Check, AlertCircle, Zap, FlaskConical } from 'lucide-react';
import { Brain, Loader2, Check, AlertCircle, Zap, FlaskConical } from '@/lib/lucide-icons';
import { useAppState } from '../hooks/useAppState';
import { useState } from 'react';
import { WebGPUFallbackDialog } from './WebGPUFallbackDialog';
+16 -7
View File
@@ -14,7 +14,10 @@ import {
Variable,
Hash,
Target,
} from 'lucide-react';
List,
AtSign,
Type,
} from '@/lib/lucide-icons';
import { useAppState } from '../hooks/useAppState';
import { FILTERABLE_LABELS, NODE_COLORS, ALL_EDGE_TYPES, EDGE_INFO, type EdgeType } from '../lib/constants';
import { GraphNode, NodeLabel } from '../core/graph/types';
@@ -98,13 +101,15 @@ const TreeItem = ({
const isSelected = selectedPath === node.path;
const hasChildren = node.children.length > 0;
// Filter children based on search
// Filter children based on search (recursive)
const filteredChildren = useMemo(() => {
if (!searchQuery) return node.children;
return node.children.filter(child =>
child.name.toLowerCase().includes(searchQuery.toLowerCase()) ||
child.children.some(c => c.name.toLowerCase().includes(searchQuery.toLowerCase()))
);
const searchLower = searchQuery.toLowerCase();
const matchesSearch = (node: TreeNode, query: string): boolean => {
if (node.name.toLowerCase().includes(query)) return true;
return node.children?.some(child => matchesSearch(child, query)) ?? false;
};
return node.children.filter(child => matchesSearch(child, searchLower));
}, [node.children, searchQuery]);
// Check if this node matches search
@@ -185,7 +190,11 @@ const getNodeTypeIcon = (label: NodeLabel) => {
case 'Function': return Braces;
case 'Method': return Braces;
case 'Interface': return Hash;
case 'Enum': return List;
case 'Type': return Type;
case 'Decorator': return AtSign;
case 'Import': return FileCode;
case 'Variable': return Variable;
default: return Variable;
}
};
@@ -499,7 +508,7 @@ export const FileTreePanel = ({ onFocusNode }: FileTreePanelProps) => {
Color Legend
</h3>
<div className="grid grid-cols-2 gap-2">
{(['Folder', 'File', 'Class', 'Function', 'Interface', 'Method'] as NodeLabel[]).map(label => (
{(['Folder', 'File', 'Class', 'Interface', 'Enum', 'Type', 'Function', 'Method', 'Variable', 'Decorator'] as NodeLabel[]).map(label => (
<div key={label} className="flex items-center gap-1.5">
<div
className="w-2.5 h-2.5 rounded-full"
+34 -26
View File
@@ -1,8 +1,9 @@
import { useEffect, useCallback, useMemo, useState, forwardRef, useImperativeHandle } from 'react';
import { ZoomIn, ZoomOut, Maximize2, Focus, RotateCcw, Play, Pause, Lightbulb, LightbulbOff } from 'lucide-react';
import { ZoomIn, ZoomOut, Maximize2, Focus, RotateCcw, Play, Pause, Lightbulb, LightbulbOff } from '@/lib/lucide-icons';
import { useSigma } from '../hooks/useSigma';
import { useAppState } from '../hooks/useAppState';
import { knowledgeGraphToGraphology, filterGraphByDepth, SigmaNodeAttributes, SigmaEdgeAttributes } from '../lib/graph-adapter';
import type { GraphNode } from '../core/graph/types';
import { QueryFAB } from './QueryFAB';
import Graph from 'graphology';
@@ -54,30 +55,44 @@ export const GraphCanvas = forwardRef<GraphCanvasHandle>((_, ref) => {
return animatedNodes;
}, [animatedNodes, isAIHighlightsEnabled]);
const nodeById = useMemo(() => {
if (!graph) return new Map<string, GraphNode>();
return new Map(graph.nodes.map(n => [n.id, n]));
}, [graph]);
const handleNodeClick = useCallback((nodeId: string) => {
if (!graph) return;
const node = graph.nodes.find(n => n.id === nodeId);
const node = nodeById.get(nodeId);
if (node) {
setSelectedNode(node);
openCodePanel();
}
}, [graph, setSelectedNode, openCodePanel]);
}, [graph, nodeById, setSelectedNode, openCodePanel]);
const handleNodeHover = useCallback((nodeId: string | null) => {
if (!nodeId || !graph) {
setHoveredNodeName(null);
return;
}
const node = graph.nodes.find(n => n.id === nodeId);
if (node) {
setHoveredNodeName(node.properties.name);
}
}, [graph]);
const node = nodeById.get(nodeId);
setHoveredNodeName(node ? node.properties.name : null);
}, [graph, nodeById]);
const handleStageClick = useCallback(() => {
setSelectedNode(null);
}, [setSelectedNode]);
const handleToggleAIHighlights = useCallback(() => {
if (isAIHighlightsEnabled) {
clearAIToolHighlights();
clearAICitationHighlights();
clearBlastRadius();
setSelectedNode(null);
setSigmaSelectedNode(null);
}
toggleAIHighlights();
}, [isAIHighlightsEnabled, clearAIToolHighlights, clearAICitationHighlights, clearBlastRadius, setSelectedNode, toggleAIHighlights]);
const {
containerRef,
sigmaRef,
@@ -106,7 +121,7 @@ export const GraphCanvas = forwardRef<GraphCanvasHandle>((_, ref) => {
focusNode: (nodeId: string) => {
// Also update app state so the selection syncs properly
if (graph) {
const node = graph.nodes.find(n => n.id === nodeId);
const node = nodeById.get(nodeId);
if (node) {
setSelectedNode(node);
openCodePanel();
@@ -114,7 +129,7 @@ export const GraphCanvas = forwardRef<GraphCanvasHandle>((_, ref) => {
}
focusNode(nodeId);
}
}), [focusNode, graph, setSelectedNode, openCodePanel]);
}), [focusNode, graph, nodeById, setSelectedNode, openCodePanel]);
// Update Sigma graph when KnowledgeGraph changes
useEffect(() => {
@@ -126,10 +141,11 @@ export const GraphCanvas = forwardRef<GraphCanvasHandle>((_, ref) => {
graph.relationships.forEach(rel => {
if (rel.type === 'MEMBER_OF') {
// Find the community node to get its index
const communityNode = graph.nodes.find(n => n.id === rel.targetId && n.label === 'Community');
if (communityNode) {
const communityNode = nodeById.get(rel.targetId);
if (communityNode && communityNode.label === 'Community') {
// Extract community index from id (e.g., "comm_5" -> 5)
const communityIdx = parseInt(rel.targetId.replace('comm_', ''), 10) || 0;
const numericPart = rel.targetId.replace('comm_', '');
const communityIdx = /^\d+$/.test(numericPart) ? parseInt(numericPart, 10) : 0;
communityMemberships.set(rel.sourceId, communityIdx);
}
}
@@ -137,7 +153,7 @@ export const GraphCanvas = forwardRef<GraphCanvasHandle>((_, ref) => {
const sigmaGraph = knowledgeGraphToGraphology(graph, communityMemberships);
setSigmaGraph(sigmaGraph);
}, [graph, setSigmaGraph]);
}, [graph, nodeById, setSigmaGraph]);
// Update node visibility when filters change
useEffect(() => {
@@ -149,7 +165,8 @@ export const GraphCanvas = forwardRef<GraphCanvasHandle>((_, ref) => {
filterGraphByDepth(sigmaGraph, appSelectedNode?.id || null, depthFilter, visibleLabels);
sigma.refresh();
}, [visibleLabels, depthFilter, appSelectedNode, sigmaRef]);
// eslint-disable-next-line react-hooks/exhaustive-deps -- sigmaRef identity never changes
}, [visibleLabels, depthFilter, appSelectedNode]);
// Sync app selected node with sigma
useEffect(() => {
@@ -307,23 +324,14 @@ export const GraphCanvas = forwardRef<GraphCanvasHandle>((_, ref) => {
{/* AI Highlights toggle - Top Right */}
<div className="absolute top-4 right-4 z-20">
<button
onClick={() => {
if (isAIHighlightsEnabled) {
// Turning off — clear AI highlights and selection (preserve user query highlights)
clearAIToolHighlights();
clearAICitationHighlights();
clearBlastRadius();
setSelectedNode(null);
setSigmaSelectedNode(null);
}
toggleAIHighlights();
}}
onClick={handleToggleAIHighlights}
className={
isAIHighlightsEnabled
? 'w-10 h-10 flex items-center justify-center bg-cyan-500/15 border border-cyan-400/40 rounded-lg text-cyan-200 hover:bg-cyan-500/20 hover:border-cyan-300/60 transition-colors'
: 'w-10 h-10 flex items-center justify-center bg-elevated border border-border-subtle rounded-lg text-text-muted hover:bg-hover hover:text-text-primary transition-colors'
}
title={isAIHighlightsEnabled ? 'Turn off all highlights' : 'Turn on AI highlights'}
data-testid="ai-highlights-toggle"
>
{isAIHighlightsEnabled ? <Lightbulb className="w-4 h-4" /> : <LightbulbOff className="w-4 h-4" />}
</button>
+1 -1
View File
@@ -1,4 +1,4 @@
import { Search, Settings, HelpCircle, Sparkles, Github, Star, ChevronDown } from 'lucide-react';
import { Search, Settings, HelpCircle, Sparkles, Github, Star, ChevronDown } from '@/lib/lucide-icons';
import { useAppState } from '../hooks/useAppState';
import type { RepoSummary } from '../services/server-connection';
import { useState, useMemo, useRef, useEffect, useCallback } from 'react';
@@ -1,11 +1,11 @@
import React, { useState } from 'react';
import React, { useState, useRef, useEffect } from 'react';
import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm';
import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
import { vscDarkPlus } from 'react-syntax-highlighter/dist/esm/styles/prism';
import { MermaidDiagram } from './MermaidDiagram';
import { ToolCallCard } from './ToolCallCard';
import { Copy, Check } from 'lucide-react';
import { Copy, Check } from '@/lib/lucide-icons';
// Custom syntax theme
const customTheme = {
@@ -39,12 +39,24 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = ({
showCopyButton = false
}) => {
const [copied, setCopied] = useState(false);
const copyTimerRef = useRef<ReturnType<typeof setTimeout>>(undefined);
useEffect(() => {
return () => {
if (copyTimerRef.current) {
clearTimeout(copyTimerRef.current);
}
};
}, []);
const handleCopy = async () => {
try {
await navigator.clipboard.writeText(content);
setCopied(true);
setTimeout(() => setCopied(false), 2000);
if (copyTimerRef.current) {
clearTimeout(copyTimerRef.current);
}
copyTimerRef.current = setTimeout(() => setCopied(false), 2000);
} catch (err) {
console.error('Failed to copy:', err);
}
@@ -78,13 +90,13 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = ({
return parts.join('```');
};
const handleLinkClick = (e: React.MouseEvent<HTMLAnchorElement>, href: string) => {
const handleLinkClick = React.useCallback((e: React.MouseEvent<HTMLAnchorElement>, href: string) => {
if (href.startsWith('code-ref:') || href.startsWith('node-ref:')) {
e.preventDefault();
onLinkClick?.(href);
}
// External links open in new tab (default behavior)
};
}, [onLinkClick]);
const formattedContent = React.useMemo(() => formatMarkdownForDisplay(content), [content]);
@@ -164,7 +176,7 @@ export const MarkdownRenderer: React.FC<MarkdownRendererProps> = ({
);
},
pre: ({ children }: any) => <>{children}</>,
}), [onLinkClick]); // Removed handleLinkClick dependency as it is defined inside component but depends on onLinkClick
}), [handleLinkClick]);
return (
<div className="text-text-primary text-sm">
+19 -9
View File
@@ -1,10 +1,13 @@
import { useEffect, useRef, useState } from 'react';
import { Suspense, useEffect, useRef, useState, lazy } from 'react';
import mermaid from 'mermaid';
import DOMPurify from 'dompurify';
import { AlertTriangle, Maximize2 } from 'lucide-react';
import { ProcessFlowModal } from './ProcessFlowModal';
import { AlertTriangle, Maximize2 } from '@/lib/lucide-icons';
import type { ProcessData } from '../lib/mermaid-generator';
const ProcessFlowModal = lazy(() =>
import('./ProcessFlowModal').then((m) => ({ default: m.ProcessFlowModal })),
);
// Initialize mermaid with cyan theme matching ProcessFlowModal
mermaid.initialize({
startOnLoad: false,
@@ -68,7 +71,8 @@ export const MermaidDiagram = ({ code }: MermaidDiagramProps) => {
// Render the diagram
const { svg: renderedSvg } = await mermaid.render(id, code.trim());
setSvg(renderedSvg);
const sanitizedSvg = DOMPurify.sanitize(renderedSvg, { USE_PROFILES: { svg: true, svgFilters: true }, ADD_TAGS: ['foreignObject'] });
setSvg(sanitizedSvg);
setError(null);
} catch (err) {
// Silent catch for streaming:
@@ -141,17 +145,23 @@ export const MermaidDiagram = ({ code }: MermaidDiagramProps) => {
<div
ref={containerRef}
className="flex items-center justify-center p-4 overflow-auto max-h-[400px]"
dangerouslySetInnerHTML={{ __html: DOMPurify.sanitize(svg, { USE_PROFILES: { svg: true, svgFilters: true } }) }}
dangerouslySetInnerHTML={{ __html: DOMPurify.sanitize(svg, { USE_PROFILES: { svg: true, svgFilters: true }, ADD_TAGS: ['foreignObject'] }) }}
/>
</div>
</div>
{/* Use ProcessFlowModal for expansion */}
{showModal && processData && (
<ProcessFlowModal
process={processData}
onClose={() => setShowModal(false)}
/>
<Suspense
fallback={
<div className="p-4 text-sm text-text-muted">Loading diagram…</div>
}
>
<ProcessFlowModal
process={processData}
onClose={() => setShowModal(false)}
/>
</Suspense>
)}
</>
);
@@ -148,7 +148,7 @@ export const ProcessFlowModal = ({ process, onClose, onFocusInGraph, isFullScree
const { svg } = await mermaid.render(id, mermaidCode);
if (!diagramRef.current) return;
diagramRef.current!.innerHTML = DOMPurify.sanitize(svg, { USE_PROFILES: { svg: true, svgFilters: true } });
diagramRef.current!.innerHTML = DOMPurify.sanitize(svg, { USE_PROFILES: { svg: true, svgFilters: true }, ADD_TAGS: ['foreignObject'] });
} catch (error) {
console.error('Mermaid render error:', error);
const errorMessage = error instanceof Error ? error.message : String(error);
@@ -211,6 +211,7 @@ export const ProcessFlowModal = ({ process, onClose, onFocusInGraph, isFullScree
ref={containerRef}
className="fixed inset-0 z-50 flex items-center justify-center bg-black/20 animate-fade-in"
onClick={handleBackdropClick}
data-testid="process-modal"
>
{/* Glassmorphism Modal */}
<div className={`bg-slate-900/60 backdrop-blur-2xl border border-white/10 rounded-3xl shadow-2xl shadow-cyan-500/10 flex flex-col animate-scale-in overflow-hidden relative ${isFullScreen
@@ -326,7 +326,7 @@ export const ProcessesPanel = () => {
/>
</div>
</div>
<div className="flex items-center gap-2 text-xs text-text-muted">
<div className="flex items-center gap-2 text-xs text-text-muted" data-testid="process-list-loaded">
<span>{totalCount} processes detected</span>
</div>
</div>
@@ -462,7 +462,7 @@ const ProcessItem = ({ process, isLoading, isSelected, isFocused, onView, onTogg
: '';
return (
<div className={`flex items-center gap-2 px-4 py-2 mx-2 rounded-lg hover:bg-hover group transition-all ${rowClass}`}>
<div data-testid="process-row" className={`flex items-center gap-2 px-4 py-2 mx-2 rounded-lg hover:bg-hover group transition-all ${rowClass}`}>
<GitBranch className="w-4 h-4 text-text-muted flex-shrink-0" />
<div className="flex-1 min-w-0">
<div className="text-sm text-text-primary truncate">{process.label}</div>
@@ -484,12 +484,14 @@ const ProcessItem = ({ process, isLoading, isSelected, isFocused, onView, onTogg
: 'text-text-muted hover:text-cyan-400 bg-white/5 hover:bg-cyan-500/20 border border-white/10 hover:border-cyan-400/40 opacity-0 group-hover:opacity-100'
}`}
title={isFocused ? 'Click to remove highlight from graph' : 'Click to highlight in graph'}
data-testid="process-highlight-button"
>
<Lightbulb className="w-4 h-4" />
</button>
<button
onClick={onView}
disabled={isLoading}
data-testid="process-view-button"
className={`flex items-center gap-1.5 px-2.5 py-1.5 text-xs font-medium rounded-md transition-all disabled:opacity-50 shadow-sm ${isSelected
? 'text-cyan-300 bg-cyan-900/60 border border-cyan-400/60 opacity-100'
: 'text-cyan-400 hover:text-cyan-300 bg-cyan-950/30 hover:bg-cyan-900/50 border border-cyan-500/30 hover:border-cyan-400/50 opacity-0 group-hover:opacity-100 shadow-cyan-900/20'
+1 -1
View File
@@ -1,5 +1,5 @@
import { useState, useRef, useEffect, useCallback } from 'react';
import { Terminal, Play, X, ChevronDown, ChevronUp, Loader2, Sparkles, Table } from 'lucide-react';
import { Terminal, Play, X, ChevronDown, ChevronUp, Loader2, Sparkles, Table } from '@/lib/lucide-icons';
import { useAppState } from '../hooks/useAppState';
const EXAMPLE_QUERIES = [
+1 -1
View File
@@ -2,7 +2,7 @@ import { useState, useRef, useEffect, useCallback } from 'react';
import {
Send, Square, Sparkles, User,
PanelRightClose, Loader2, AlertTriangle, GitBranch
} from 'lucide-react';
} from '@/lib/lucide-icons';
import { useAppState } from '../hooks/useAppState';
import { ToolCallCard } from './ToolCallCard';
import { isProviderConfigured } from '../core/llm/settings-service';
+178 -220
View File
@@ -1,12 +1,15 @@
import { useState, useEffect, useCallback, useRef, useMemo } from 'react';
import { X, Key, Server, Brain, Check, AlertCircle, Eye, EyeOff, RefreshCw, ChevronDown, Loader2, Search } from 'lucide-react';
import { X, Key, Server, Brain, Check, AlertCircle, Eye, EyeOff, RefreshCw, ChevronDown, Loader2, Search } from '@/lib/lucide-icons';
import {
loadSettings,
saveSettings,
getProviderDisplayName,
getAvailableModels,
fetchOpenRouterModels,
} from '../core/llm/settings-service';
import type { LLMSettings, LLMProvider } from '../core/llm/types';
import { DEFAULT_OLLAMA_BASE_URL } from '../config/ui-constants';
import { ProviderConfigCard } from './settings/ProviderConfigCard';
interface SettingsPanelProps {
isOpen: boolean;
@@ -216,6 +219,7 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
const [settings, setSettings] = useState<LLMSettings>(loadSettings);
const [showApiKey, setShowApiKey] = useState<Record<string, boolean>>({});
const [saveStatus, setSaveStatus] = useState<'idle' | 'saved' | 'error'>('idle');
const saveTimerRef = useRef<ReturnType<typeof setTimeout>>(undefined);
// Ollama connection state
const [ollamaError, setOllamaError] = useState<string | null>(null);
const [isCheckingOllama, setIsCheckingOllama] = useState(false);
@@ -223,6 +227,15 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
const [openRouterModels, setOpenRouterModels] = useState<Array<{ id: string; name: string }>>([]);
const [isLoadingModels, setIsLoadingModels] = useState(false);
// Clean up save timer on unmount
useEffect(() => {
return () => {
if (saveTimerRef.current) {
clearTimeout(saveTimerRef.current);
}
};
}, []);
// Load settings when panel opens
useEffect(() => {
if (isOpen) {
@@ -252,7 +265,7 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
useEffect(() => {
if (settings.activeProvider === 'ollama') {
const baseUrl = settings.ollama?.baseUrl ?? 'http://localhost:11434';
const baseUrl = settings.ollama?.baseUrl ?? DEFAULT_OLLAMA_BASE_URL;
const timer = setTimeout(() => {
checkOllamaConnection(baseUrl);
}, 300);
@@ -269,7 +282,10 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
saveSettings(settings);
setSaveStatus('saved');
onSettingsSaved?.();
setTimeout(() => setSaveStatus('idle'), 2000);
if (saveTimerRef.current) {
clearTimeout(saveTimerRef.current);
}
saveTimerRef.current = setTimeout(() => setSaveStatus('idle'), 2000);
} catch {
setSaveStatus('error');
}
@@ -281,7 +297,7 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
if (!isOpen) return null;
const providers: LLMProvider[] = ['openai', 'gemini', 'anthropic', 'azure-openai', 'ollama', 'openrouter', 'minimax'];
const providers: LLMProvider[] = ['openai', 'gemini', 'anthropic', 'azure-openai', 'ollama', 'openrouter', 'minimax', 'glm'];
return (
@@ -366,7 +382,7 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
w-8 h-8 rounded-lg flex items-center justify-center text-lg
${settings.activeProvider === provider ? 'bg-accent/20' : 'bg-surface'}
`}>
{provider === 'openai' ? '🤖' : provider === 'gemini' ? '💎' : provider === 'anthropic' ? '🧠' : provider === 'ollama' ? '🦙' : provider === 'openrouter' ? '🌐' : provider === 'minimax' ? '⚡' : '☁️'}
{provider === 'openai' ? '🤖' : provider === 'gemini' ? '💎' : provider === 'anthropic' ? '🧠' : provider === 'ollama' ? '🦙' : provider === 'openrouter' ? '🌐' : provider === 'minimax' ? '⚡' : provider === 'glm' ? '🔮' : '☁️'}
</div>
<span className="font-medium">{getProviderDisplayName(provider)}</span>
</button>
@@ -374,60 +390,36 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
</div>
</div>
<div className="p-3 bg-amber-500/10 border border-amber-500/30 rounded-xl text-xs text-amber-200">
API keys are stored in session storage and will be cleared when you close this tab.
</div>
{/* OpenAI Settings */}
{settings.activeProvider === 'openai' && (
<div className="space-y-4 animate-fade-in">
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
<Key className="w-4 h-4" />
API Key
</label>
<div className="relative">
<input
type={showApiKey['openai'] ? 'text' : 'password'}
value={settings.openai?.apiKey ?? ''}
onChange={e => setSettings(prev => ({
...prev,
openai: { ...prev.openai!, apiKey: e.target.value }
}))}
placeholder="Enter your OpenAI API key"
className="w-full px-4 py-3 pr-12 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all"
/>
<button
type="button"
onClick={() => toggleApiKeyVisibility('openai')}
className="absolute right-3 top-1/2 -translate-y-1/2 p-1 text-text-muted hover:text-text-primary transition-colors"
>
{showApiKey['openai'] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
<p className="text-xs text-text-muted">
Get your API key from{' '}
<a
href="https://platform.openai.com/api-keys"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
OpenAI Platform
</a>
</p>
</div>
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">Model</label>
<input
type="text"
value={settings.openai?.model ?? 'gpt-5.2-chat'}
onChange={e => setSettings(prev => ({
...prev,
openai: { ...prev.openai!, model: e.target.value }
}))}
placeholder="e.g., gpt-4o, gpt-4-turbo, gpt-3.5-turbo"
className="w-full px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
/>
</div>
<ProviderConfigCard
title="OpenAI"
apiKey={{
value: settings.openai?.apiKey ?? '',
placeholder: 'Enter your OpenAI API key',
helperText: 'Get your API key from',
helperLink: 'https://platform.openai.com/api-keys',
helperLinkLabel: 'OpenAI Platform',
isVisible: !!showApiKey['openai'],
onChange: (value) => setSettings(prev => ({
...prev,
openai: { ...prev.openai!, apiKey: value }
})),
onToggleVisibility: () => toggleApiKeyVisibility('openai'),
}}
model={{
value: settings.openai?.model ?? 'gpt-5.2-chat',
placeholder: 'e.g., gpt-4o, gpt-4-turbo, gpt-3.5-turbo',
onChange: (value) => setSettings(prev => ({
...prev,
openai: { ...prev.openai!, model: value }
})),
}}
>
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
<Server className="w-4 h-4" />
@@ -447,119 +439,63 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
Leave empty to use the default OpenAI API. Set a custom URL for proxies or compatible APIs.
</p>
</div>
</div>
</ProviderConfigCard>
)}
{/* Gemini Settings */}
{settings.activeProvider === 'gemini' && (
<div className="space-y-4 animate-fade-in">
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
<Key className="w-4 h-4" />
API Key
</label>
<div className="relative">
<input
type={showApiKey['gemini'] ? 'text' : 'password'}
value={settings.gemini?.apiKey ?? ''}
onChange={e => setSettings(prev => ({
...prev,
gemini: { ...prev.gemini!, apiKey: e.target.value }
}))}
placeholder="Enter your Google AI API key"
className="w-full px-4 py-3 pr-12 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all"
/>
<button
type="button"
onClick={() => toggleApiKeyVisibility('gemini')}
className="absolute right-3 top-1/2 -translate-y-1/2 p-1 text-text-muted hover:text-text-primary transition-colors"
>
{showApiKey['gemini'] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
<p className="text-xs text-text-muted">
Get your API key from{' '}
<a
href="https://aistudio.google.com/app/apikey"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
Google AI Studio
</a>
</p>
</div>
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">Model</label>
<input
type="text"
value={settings.gemini?.model ?? 'gemini-2.0-flash'}
onChange={e => setSettings(prev => ({
...prev,
gemini: { ...prev.gemini!, model: e.target.value }
}))}
placeholder="e.g., gemini-2.0-flash, gemini-1.5-pro"
className="w-full px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
/>
</div>
</div>
<ProviderConfigCard
title="Google Gemini"
apiKey={{
value: settings.gemini?.apiKey ?? '',
placeholder: 'Enter your Google AI API key',
helperText: 'Get your API key from',
helperLink: 'https://aistudio.google.com/app/apikey',
helperLinkLabel: 'Google AI Studio',
isVisible: !!showApiKey['gemini'],
onChange: (value) => setSettings(prev => ({
...prev,
gemini: { ...prev.gemini!, apiKey: value }
})),
onToggleVisibility: () => toggleApiKeyVisibility('gemini'),
}}
model={{
value: settings.gemini?.model ?? 'gemini-2.0-flash',
placeholder: 'e.g., gemini-2.0-flash, gemini-1.5-pro',
onChange: (value) => setSettings(prev => ({
...prev,
gemini: { ...prev.gemini!, model: value }
})),
}}
/>
)}
{/* Anthropic Settings */}
{settings.activeProvider === 'anthropic' && (
<div className="space-y-4 animate-fade-in">
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
<Key className="w-4 h-4" />
API Key
</label>
<div className="relative">
<input
type={showApiKey['anthropic'] ? 'text' : 'password'}
value={settings.anthropic?.apiKey ?? ''}
onChange={e => setSettings(prev => ({
...prev,
anthropic: { ...prev.anthropic!, apiKey: e.target.value }
}))}
placeholder="Enter your Anthropic API key"
className="w-full px-4 py-3 pr-12 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all"
/>
<button
type="button"
onClick={() => toggleApiKeyVisibility('anthropic')}
className="absolute right-3 top-1/2 -translate-y-1/2 p-1 text-text-muted hover:text-text-primary transition-colors"
>
{showApiKey['anthropic'] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
<p className="text-xs text-text-muted">
Get your API key from{' '}
<a
href="https://console.anthropic.com/settings/keys"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
Anthropic Console
</a>
</p>
</div>
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">Model</label>
<input
type="text"
value={settings.anthropic?.model ?? 'claude-sonnet-4-20250514'}
onChange={e => setSettings(prev => ({
...prev,
anthropic: { ...prev.anthropic!, model: e.target.value }
}))}
placeholder="e.g., claude-sonnet-4-20250514, claude-3-opus"
className="w-full px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
/>
</div>
</div>
<ProviderConfigCard
title="Anthropic"
apiKey={{
value: settings.anthropic?.apiKey ?? '',
placeholder: 'Enter your Anthropic API key',
helperText: 'Get your API key from',
helperLink: 'https://console.anthropic.com/settings/keys',
helperLinkLabel: 'Anthropic Console',
isVisible: !!showApiKey['anthropic'],
onChange: (value) => setSettings(prev => ({
...prev,
anthropic: { ...prev.anthropic!, apiKey: value }
})),
onToggleVisibility: () => toggleApiKeyVisibility('anthropic'),
}}
model={{
value: settings.anthropic?.model ?? 'claude-sonnet-4-20250514',
placeholder: 'e.g., claude-sonnet-4-20250514, claude-3-opus',
onChange: (value) => setSettings(prev => ({
...prev,
anthropic: { ...prev.anthropic!, model: value }
})),
}}
/>
)}
{/* Azure OpenAI Settings */}
@@ -695,17 +631,17 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
<div className="flex gap-2">
<input
type="url"
value={settings.ollama?.baseUrl ?? 'http://localhost:11434'}
value={settings.ollama?.baseUrl ?? DEFAULT_OLLAMA_BASE_URL}
onChange={e => setSettings(prev => ({
...prev,
ollama: { ...prev.ollama!, baseUrl: e.target.value }
}))}
placeholder="http://localhost:11434"
placeholder={DEFAULT_OLLAMA_BASE_URL}
className="flex-1 px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
/>
<button
type="button"
onClick={() => checkOllamaConnection(settings.ollama?.baseUrl ?? 'http://localhost:11434')}
onClick={() => checkOllamaConnection(settings.ollama?.baseUrl ?? DEFAULT_OLLAMA_BASE_URL)}
disabled={isCheckingOllama}
className="px-3 py-3 bg-elevated border border-border-subtle rounded-xl text-text-secondary hover:text-text-primary hover:border-accent/50 transition-colors disabled:opacity-50"
title="Check connection"
@@ -749,44 +685,22 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
{/* OpenRouter Settings */}
{settings.activeProvider === 'openrouter' && (
<div className="space-y-4 animate-fade-in">
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
<Key className="w-4 h-4" />
API Key
</label>
<div className="relative">
<input
type={showApiKey['openrouter'] ? 'text' : 'password'}
value={settings.openrouter?.apiKey ?? ''}
onChange={e => setSettings(prev => ({
...prev,
openrouter: { ...prev.openrouter!, apiKey: e.target.value }
}))}
placeholder="Enter your OpenRouter API key"
className="w-full px-4 py-3 pr-12 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all"
/>
<button
type="button"
onClick={() => toggleApiKeyVisibility('openrouter')}
className="absolute right-3 top-1/2 -translate-y-1/2 p-1 text-text-muted hover:text-text-primary transition-colors"
>
{showApiKey['openrouter'] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
<p className="text-xs text-text-muted">
Get your API key from{' '}
<a
href="https://openrouter.ai/keys"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
OpenRouter Keys
</a>
</p>
</div>
<ProviderConfigCard
title="OpenRouter"
apiKey={{
value: settings.openrouter?.apiKey ?? '',
placeholder: 'Enter your OpenRouter API key',
helperText: 'Get your API key from',
helperLink: 'https://openrouter.ai/keys',
helperLinkLabel: 'OpenRouter Keys',
isVisible: !!showApiKey['openrouter'],
onChange: (value) => setSettings(prev => ({
...prev,
openrouter: { ...prev.openrouter!, apiKey: value }
})),
onToggleVisibility: () => toggleApiKeyVisibility('openrouter'),
}}
>
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">Model</label>
<OpenRouterModelCombobox
@@ -811,11 +725,40 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
</a>
</p>
</div>
</div>
</ProviderConfigCard>
)}
{/* MiniMax Settings */}
{settings.activeProvider === 'minimax' && (
<ProviderConfigCard
title="MiniMax"
apiKey={{
value: settings.minimax?.apiKey ?? '',
placeholder: 'Enter your MiniMax API key',
helperText: 'Get your API key from',
helperLink: 'https://platform.minimax.io',
helperLinkLabel: 'MiniMax Platform',
isVisible: !!showApiKey['minimax'],
onChange: (value) => setSettings(prev => ({
...prev,
minimax: { ...prev.minimax!, apiKey: value }
})),
onToggleVisibility: () => toggleApiKeyVisibility('minimax'),
}}
model={{
value: settings.minimax?.model ?? 'MiniMax-M2.5',
placeholder: 'e.g., MiniMax-M2.5, MiniMax-M2.5-highspeed',
onChange: (value) => setSettings(prev => ({
...prev,
minimax: { ...prev.minimax!, model: value }
})),
helperText: 'Available: MiniMax-M2.5 (default), MiniMax-M2.5-highspeed (faster)',
}}
/>
)}
{/* GLM Settings */}
{settings.activeProvider === 'glm' && (
<div className="space-y-4 animate-fade-in">
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
@@ -824,50 +767,66 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
</label>
<div className="relative">
<input
type={showApiKey['minimax'] ? 'text' : 'password'}
value={settings.minimax?.apiKey ?? ''}
type={showApiKey['glm'] ? 'text' : 'password'}
value={settings.glm?.apiKey ?? ''}
onChange={e => setSettings(prev => ({
...prev,
minimax: { ...prev.minimax!, apiKey: e.target.value }
glm: { ...prev.glm!, apiKey: e.target.value }
}))}
placeholder="Enter your MiniMax API key"
placeholder="Enter your Z.AI API key"
className="w-full px-4 py-3 pr-12 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all"
/>
<button
type="button"
onClick={() => toggleApiKeyVisibility('minimax')}
onClick={() => toggleApiKeyVisibility('glm')}
className="absolute right-3 top-1/2 -translate-y-1/2 p-1 text-text-muted hover:text-text-primary transition-colors"
>
{showApiKey['minimax'] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
{showApiKey['glm'] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
<p className="text-xs text-text-muted">
Get your API key from{' '}
<a
href="https://platform.minimax.io"
href="https://docs.z.ai"
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
MiniMax Platform
Z.AI Platform
</a>
</p>
</div>
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">Model</label>
<input
type="text"
value={settings.minimax?.model ?? 'MiniMax-M2.5'}
<select
value={settings.glm?.model ?? 'GLM-5'}
onChange={e => setSettings(prev => ({
...prev,
minimax: { ...prev.minimax!, model: e.target.value }
glm: { ...prev.glm!, model: e.target.value }
}))}
placeholder="e.g., MiniMax-M2.5, MiniMax-M2.5-highspeed"
className="w-full px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
>
{getAvailableModels('glm').map(model => (
<option key={model} value={model}>{model}</option>
))}
</select>
</div>
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">Base URL</label>
<input
type="text"
value={settings.glm?.baseUrl ?? 'https://api.z.ai/api/coding/paas/v4'}
onChange={e => setSettings(prev => ({
...prev,
glm: { ...prev.glm!, baseUrl: e.target.value }
}))}
placeholder="https://api.z.ai/api/coding/paas/v4"
className="w-full px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
/>
<p className="text-xs text-text-muted">
Available models: MiniMax-M2.5 (default), MiniMax-M2.5-highspeed (faster)
Coding API (default). Use https://api.z.ai/api/paas/v4 for the general API.
</p>
</div>
</div>
@@ -880,8 +839,7 @@ export const SettingsPanel = ({ isOpen, onClose, onSettingsSaved, backendUrl, is
🔒
</div>
<div className="text-xs text-text-muted leading-relaxed">
<span className="text-text-secondary font-medium">Privacy:</span> Your API keys are stored only in your browser's local storage.
They're sent directly to the LLM provider when you chat. Your code never leaves your machine.
<span className="text-text-secondary font-medium">Privacy:</span> Your API keys are stored only in your browser's session storage and are cleared when the tab closes. They're sent directly to the LLM provider when you chat. Your code never leaves your machine.
</div>
</div>
</div>
+5 -4
View File
@@ -1,4 +1,5 @@
import { Heart } from 'lucide-react';
import { useMemo } from 'react';
import { Heart } from '@/lib/lucide-icons';
import { useAppState } from '../hooks/useAppState';
export const StatusBar = () => {
@@ -8,7 +9,7 @@ export const StatusBar = () => {
const edgeCount = graph?.relationships.length ?? 0;
// Detect primary language
const primaryLanguage = (() => {
const primaryLanguage = useMemo(() => {
if (!graph) return null;
const languages = graph.nodes
.map(n => n.properties.language)
@@ -21,7 +22,7 @@ export const StatusBar = () => {
}, {} as Record<string, number>);
return Object.entries(counts).sort((a, b) => b[1] - a[1])[0]?.[0];
})();
}, [graph]);
return (
<footer className="flex items-center justify-between px-5 py-2 bg-deep border-t border-dashed border-border-subtle text-[11px] text-text-muted">
@@ -38,7 +39,7 @@ export const StatusBar = () => {
<span>{progress.message}</span>
</>
) : (
<div className="flex items-center gap-1.5">
<div className="flex items-center gap-1.5" data-testid="status-ready">
<span className="w-1.5 h-1.5 bg-node-function rounded-full" />
<span>Ready</span>
</div>
+1 -1
View File
@@ -6,7 +6,7 @@
*/
import { useState } from 'react';
import { ChevronDown, ChevronRight, Sparkles, Check, Loader2, AlertCircle } from 'lucide-react';
import { ChevronDown, ChevronRight, Sparkles, Check, Loader2, AlertCircle } from '@/lib/lucide-icons';
import type { ToolCallInfo } from '../core/llm/types';
interface ToolCallCardProps {
@@ -1,5 +1,5 @@
import { useState, useEffect } from 'react';
import { X, Snail, Rocket, SkipForward } from 'lucide-react';
import { X, Snail, Rocket, SkipForward } from '@/lib/lucide-icons';
interface WebGPUFallbackDialogProps {
isOpen: boolean;
@@ -0,0 +1,110 @@
import { ReactNode } from 'react';
import { Eye, EyeOff, Key } from '@/lib/lucide-icons';
type ApiKeyField = {
value: string;
placeholder: string;
helperText?: string;
helperLink?: string;
helperLinkLabel?: string;
isVisible: boolean;
onChange: (value: string) => void;
onToggleVisibility: () => void;
};
type ModelField = {
value: string;
placeholder: string;
label?: string;
helperText?: string;
onChange: (value: string) => void;
};
interface ProviderConfigCardProps {
title: string;
description?: string;
apiKey?: ApiKeyField;
model?: ModelField;
children?: ReactNode;
}
export const ProviderConfigCard = ({
title,
description,
apiKey,
model,
children,
}: ProviderConfigCardProps) => {
return (
<div className="space-y-4 animate-fade-in">
<div className="flex items-center justify-between">
<div>
<h3 className="text-sm font-semibold text-text-primary">{title}</h3>
{description ? (
<p className="text-xs text-text-muted">{description}</p>
) : null}
</div>
</div>
{apiKey && (
<div className="space-y-2">
<label className="flex items-center gap-2 text-sm font-medium text-text-secondary">
<Key className="w-4 h-4" />
API Key
</label>
<div className="relative">
<input
type={apiKey.isVisible ? 'text' : 'password'}
value={apiKey.value}
onChange={e => apiKey.onChange(e.target.value)}
placeholder={apiKey.placeholder}
className="w-full px-4 py-3 pr-12 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all"
/>
<button
type="button"
onClick={apiKey.onToggleVisibility}
className="absolute right-3 top-1/2 -translate-y-1/2 p-1 text-text-muted hover:text-text-primary transition-colors"
>
{apiKey.isVisible ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
{apiKey.helperText && (
<p className="text-xs text-text-muted">
{apiKey.helperText}{' '}
{apiKey.helperLink ? (
<a
href={apiKey.helperLink}
target="_blank"
rel="noopener noreferrer"
className="text-accent hover:underline"
>
{apiKey.helperLinkLabel ?? 'Learn more'}
</a>
) : null}
</p>
)}
</div>
)}
{model && (
<div className="space-y-2">
<label className="text-sm font-medium text-text-secondary">
{model.label ?? 'Model'}
</label>
<input
type="text"
value={model.value}
onChange={e => model.onChange(e.target.value)}
placeholder={model.placeholder}
className="w-full px-4 py-3 bg-elevated border border-border-subtle rounded-xl text-text-primary placeholder:text-text-muted focus:border-accent focus:ring-2 focus:ring-accent/20 outline-none transition-all font-mono text-sm"
/>
{model.helperText ? (
<p className="text-xs text-text-muted">{model.helperText}</p>
) : null}
</div>
)}
{children}
</div>
);
};
@@ -12,4 +12,5 @@ export enum SupportedLanguages {
Ruby = 'ruby',
Kotlin = 'kotlin',
Swift = 'swift',
}
Dart = 'dart',
}
+7
View File
@@ -0,0 +1,7 @@
// Centralized UI and provider defaults to reduce magic numbers and duplicated URLs.
export const ERROR_RESET_DELAY_MS = 3000;
export const BACKEND_URL_DEBOUNCE_MS = 500;
export const DEFAULT_BACKEND_URL = 'http://localhost:4747';
export const DEFAULT_OLLAMA_BASE_URL = 'http://localhost:11434';
export const DEFAULT_OPENROUTER_BASE_URL = 'https://openrouter.ai/api/v1';
+20 -1
View File
@@ -15,7 +15,24 @@ export type NodeLabel =
| 'Type'
| 'CodeElement'
| 'Community'
| 'Process';
| 'Process'
| 'Section'
| 'Struct'
| 'Trait'
| 'Impl'
| 'TypeAlias'
| 'Const'
| 'Static'
| 'Namespace'
| 'Union'
| 'Typedef'
| 'Macro'
| 'Property'
| 'Record'
| 'Delegate'
| 'Annotation'
| 'Constructor'
| 'Template';
export type NodeProperties = {
@@ -55,6 +72,8 @@ export type RelationshipType =
| 'IMPLEMENTS'
| 'EXTENDS'
| 'HAS_METHOD'
| 'HAS_PROPERTY'
| 'ACCESSES'
| 'MEMBER_OF'
| 'STEP_IN_PROCESS'
@@ -41,6 +41,7 @@ export const callRouters = {
[SupportedLanguages.C]: noRouting,
[SupportedLanguages.Ruby]: routeRubyCall,
[SupportedLanguages.Kotlin]: noRouting,
[SupportedLanguages.Dart]: noRouting,
} satisfies Record<SupportedLanguages, CallRouter>;
// ── Result types ────────────────────────────────────────────────────────────
@@ -315,6 +315,33 @@ export function detectFrameworkFromPath(filePath: string): FrameworkHint | null
return { framework: 'ios-router', entryPointMultiplier: 2.0, reason: 'ios-router' };
}
// ========== DART / FLUTTER ==========
// Flutter main entry point
if (p.endsWith('/main.dart')) {
return { framework: 'flutter', entryPointMultiplier: 3.0, reason: 'flutter-main' };
}
// Flutter screens/pages (high priority - route entry points)
if ((p.includes('/lib/screens/') || p.includes('/lib/pages/')) && p.endsWith('.dart')) {
return { framework: 'flutter', entryPointMultiplier: 2.5, reason: 'flutter-screen' };
}
// Flutter BLoC / controllers (state management entry points)
if ((p.includes('/lib/bloc/') || p.includes('/lib/controllers/') || p.includes('/lib/cubit/')) && p.endsWith('.dart')) {
return { framework: 'flutter', entryPointMultiplier: 2.0, reason: 'flutter-state-management' };
}
// Flutter services
if (p.includes('/lib/services/') && p.endsWith('.dart')) {
return { framework: 'flutter', entryPointMultiplier: 1.8, reason: 'flutter-service' };
}
// Flutter widgets (reusable components)
if (p.includes('/lib/widgets/') && p.endsWith('.dart')) {
return { framework: 'flutter', entryPointMultiplier: 1.5, reason: 'flutter-widget' };
}
// ========== GENERIC PATTERNS ==========
// Any language: index files in API folders
@@ -366,6 +393,10 @@ export const FRAMEWORK_AST_PATTERNS = {
'axum': ['Router::new'],
'rocket': ['#[get', '#[post'],
// Dart/Flutter
'flutter': ['StatelessWidget', 'StatefulWidget', 'BuildContext', 'Widget build',
'ChangeNotifier', 'GetxController', 'Cubit<', 'Bloc<'],
// Swift/iOS
'uikit': ['viewDidLoad', 'viewWillAppear', 'viewDidAppear', 'UIViewController'],
'swiftui': ['@main', 'WindowGroup', 'ContentView', '@StateObject', '@ObservedObject'],
@@ -109,6 +109,10 @@ const isNodeExported = (node: any, name: string, language: string): boolean => {
case 'ruby':
return true;
// Dart: Public if no leading underscore (same convention as Python)
case 'dart':
return !name.startsWith('_');
default:
return false;
}
@@ -483,6 +483,102 @@ export const SWIFT_QUERIES = `
(inheritance_specifier inherits_from: (user_type (type_identifier) @heritage.extends))) @heritage
`;
// Dart queries - works with tree-sitter-dart
export const DART_QUERIES = `
; ── Classes ──────────────────────────────────────────────────────────────────
(class_definition
name: (identifier) @name) @definition.class
; ── Mixins ───────────────────────────────────────────────────────────────────
(mixin_declaration
(identifier) @name) @definition.trait
; ── Extensions ───────────────────────────────────────────────────────────────
(extension_declaration
name: (identifier) @name) @definition.class
; ── Enums ────────────────────────────────────────────────────────────────────
(enum_declaration
name: (identifier) @name) @definition.enum
; ── Type aliases ─────────────────────────────────────────────────────────────
; Anchor "=" after the name to avoid capturing the RHS type
(type_alias
(type_identifier) @name
"=") @definition.type
; ── Top-level functions (parent is program, not method_signature) ────────────
(program
(function_signature
name: (identifier) @name) @definition.function)
; ── Abstract method declarations (function_signature inside class body declaration) ──
(declaration
(function_signature
name: (identifier) @name)) @definition.method
; ── Methods (inside class/mixin/extension bodies) ────────────────────────────
(method_signature
(function_signature
name: (identifier) @name)) @definition.method
; ── Constructors ─────────────────────────────────────────────────────────────
(constructor_signature
name: (identifier) @name) @definition.constructor
; ── Factory constructors ─────────────────────────────────────────────────────
(method_signature
(factory_constructor_signature
(identifier) @name)) @definition.constructor
; ── Getters ──────────────────────────────────────────────────────────────────
(method_signature
(getter_signature
name: (identifier) @name)) @definition.property
; ── Setters ──────────────────────────────────────────────────────────────────
(method_signature
(setter_signature
name: (identifier) @name)) @definition.property
; ── Imports ──────────────────────────────────────────────────────────────────
(import_or_export
(library_import
(import_specification
(configurable_uri) @import.source))) @import
; ── Calls: direct function/constructor calls (identifier immediately before argument_part) ──
(expression_statement
(identifier) @call.name
.
(selector (argument_part))) @call
; ── Calls: method calls (obj.method()) ───────────────────────────────────────
(expression_statement
(selector
(unconditional_assignable_selector
(identifier) @call.name))) @call
; ── Heritage: extends ────────────────────────────────────────────────────────
(class_definition
name: (identifier) @heritage.class
superclass: (superclass
(type_identifier) @heritage.extends)) @heritage
; ── Heritage: implements ─────────────────────────────────────────────────────
(class_definition
name: (identifier) @heritage.class
interfaces: (interfaces
(type_identifier) @heritage.implements)) @heritage.impl
; ── Heritage: with (mixins) ──────────────────────────────────────────────────
(class_definition
name: (identifier) @heritage.class
superclass: (superclass
(mixins
(type_identifier) @heritage.trait))) @heritage
`;
export const LANGUAGE_QUERIES: Record<SupportedLanguages, string> = {
[SupportedLanguages.TypeScript]: TYPESCRIPT_QUERIES,
[SupportedLanguages.JavaScript]: JAVASCRIPT_QUERIES,
@@ -497,5 +593,6 @@ export const LANGUAGE_QUERIES: Record<SupportedLanguages, string> = {
[SupportedLanguages.Ruby]: RUBY_QUERIES,
[SupportedLanguages.Kotlin]: '', // Kotlin WASM parser not yet available for web
[SupportedLanguages.Swift]: SWIFT_QUERIES,
[SupportedLanguages.Dart]: DART_QUERIES,
};
+1
View File
@@ -45,6 +45,7 @@ export const getLanguageFromFilename = (filename: string): SupportedLanguages |
}
// Swift
if (filename.endsWith('.swift')) return SupportedLanguages.Swift;
if (filename.endsWith('.dart')) return SupportedLanguages.Dart;
return null;
};
+41 -2
View File
@@ -202,6 +202,35 @@ const generateCodeElementCSV = (
return rows.join('\n');
};
/**
* Generate CSV for multi-language code element nodes (Struct, Enum, Macro, etc.)
* These do NOT have isExported column.
* Headers: id,name,filePath,startLine,endLine,content
*/
const generateMultiLangCSV = (
nodes: GraphNode[],
label: NodeLabel,
fileContents: Map<string, string>
): string => {
const headers = ['id', 'name', 'filePath', 'startLine', 'endLine', 'content'];
const rows: string[] = [headers.join(',')];
for (const node of nodes) {
if (node.label !== label) continue;
const content = extractContent(node, fileContents);
rows.push([
escapeCSVField(node.id),
escapeCSVField(node.properties.name || ''),
escapeCSVField(node.properties.filePath || ''),
escapeCSVNumber(node.properties.startLine, -1),
escapeCSVNumber(node.properties.endLine, -1),
escapeCSVField(content),
].join(','));
}
return rows.join('\n');
};
/**
* Generate CSV for Community nodes (from Leiden algorithm)
* Headers: id,label,heuristicLabel,keywords,description,enrichedBy,cohesion,symbolCount
@@ -221,7 +250,7 @@ const generateCommunityCSV = (nodes: GraphNode[]): string => {
escapeCSVField(node.id),
escapeCSVField(node.properties.name || ''), // label is stored in name
escapeCSVField(node.properties.heuristicLabel || ''),
keywordsStr, // Array format for LadybugDB
escapeCSVField(keywordsStr), // Array format for LadybugDB, needs CSV escaping for commas
escapeCSVField((node.properties as any).description || ''),
escapeCSVField((node.properties as any).enrichedBy || 'heuristic'),
escapeCSVNumber(node.properties.cohesion, 0),
@@ -312,7 +341,17 @@ export const generateAllCSVs = (
nodeCSVs.set('CodeElement', generateCodeElementCSV(nodes, 'CodeElement', fileContents));
nodeCSVs.set('Community', generateCommunityCSV(nodes));
nodeCSVs.set('Process', generateProcessCSV(nodes));
// Generate CSVs for remaining multi-language tables (no isExported column)
const handledTables = new Set<NodeTableName>([
'File', 'Folder', 'Function', 'Class', 'Interface', 'Method', 'CodeElement', 'Community', 'Process',
]);
for (const table of NODE_TABLES) {
if (!handledTables.has(table)) {
nodeCSVs.set(table, generateMultiLangCSV(nodes, table as NodeLabel, fileContents));
}
}
// Generate single relation CSV
const relCSV = generateRelationCSV(graph);
+43 -32
View File
@@ -106,11 +106,24 @@ export const loadGraphToLbug = async (
}
const { lbug: lbugModule } = await initLbug();
// Close previous connection/database to avoid leaking WASM resources across repo switches
if (conn) {
try { await conn.close(); } catch {}
conn = null;
}
if (db) {
try { await db.close(); } catch {}
db = null;
}
// Recreate a fresh in-memory DB each load to avoid cleanup/quoting issues with reserved names
const BUFFER_POOL_SIZE = 512 * 1024 * 1024; // 512MB (mirror init)
db = new lbugModule.Database(':memory:', BUFFER_POOL_SIZE);
conn = new lbugModule.Connection(db);
// Update initPromise so subsequent initLbug() calls return the fresh db/conn
initPromise = Promise.resolve({ db, conn, lbug: lbugModule });
// Re-run schema creation
for (let i = 0; i < SCHEMA_QUERIES.length; i++) {
try {
@@ -213,7 +226,7 @@ export const loadGraphToLbug = async (
}
// Execute batched prepared statements per label pair
const SUB_BATCH_SIZE = 4;
// Prepare once per (fromLabel, toLabel) pair and reuse across all rows
for (const [key, rels] of relsByLabelPair) {
const [fromLabel, toLabel] = key.split(':');
const cypher = `
@@ -222,39 +235,37 @@ export const loadGraphToLbug = async (
CREATE (a)-[:${REL_TABLE_NAME} {type: $relType, confidence: $confidence, reason: $reason, step: $step}]->(b)
`;
for (let i = 0; i < rels.length; i += SUB_BATCH_SIZE) {
const subBatch = rels.slice(i, i + SUB_BATCH_SIZE);
const stmt = await conn.prepare(cypher);
if (!stmt.isSuccess()) {
const errMsg = await stmt.getErrorMessage();
if (import.meta.env.DEV) console.warn(`Prepare failed for ${key}: ${errMsg}`);
skippedRels += subBatch.length;
await stmt.close();
continue;
}
const stmt = await conn.prepare(cypher);
if (!stmt.isSuccess()) {
const errMsg = await stmt.getErrorMessage();
if (import.meta.env.DEV) console.warn(`Prepare failed for ${key}: ${errMsg}`);
skippedRels += rels.length;
await stmt.close();
continue;
}
try {
for (const r of subBatch) {
try {
await conn.execute(stmt, r);
insertedRels++;
} catch (err) {
skippedRels++;
const statKey = `${r.relType}:${fromLabel}->${toLabel}`;
skippedRelStats.set(statKey, (skippedRelStats.get(statKey) || 0) + 1);
if (import.meta.env.DEV) {
console.warn(`⚠️ Skipped: ${statKey} | "${r.fromId}" → "${r.toId}" | ${err instanceof Error ? err.message : String(err)}`);
}
try {
for (let i = 0; i < rels.length; i++) {
try {
await conn.execute(stmt, rels[i]);
insertedRels++;
} catch (err) {
skippedRels++;
const r = rels[i];
const statKey = `${r.relType}:${fromLabel}->${toLabel}`;
skippedRelStats.set(statKey, (skippedRelStats.get(statKey) || 0) + 1);
if (import.meta.env.DEV) {
console.warn(`⚠️ Skipped: ${statKey} | "${r.fromId}" → "${r.toId}" | ${err instanceof Error ? err.message : String(err)}`);
}
}
} finally {
await stmt.close();
}
// Yield to event loop between sub-batches
if (i + SUB_BATCH_SIZE < rels.length) {
await new Promise(r => setTimeout(r, 0));
// Yield to event loop every 500 relations
if (i > 0 && i % 500 === 0) {
await new Promise(r => setTimeout(r, 0));
}
}
} finally {
await stmt.close();
}
}
@@ -272,7 +283,7 @@ export const loadGraphToLbug = async (
let totalNodes = 0;
for (const tableName of NODE_TABLES) {
try {
const countRes = await conn.query(`MATCH (n:${tableName}) RETURN count(n) AS cnt`);
const countRes = await conn.query(`MATCH (n:${escapeTableName(tableName)}) RETURN count(n) AS cnt`);
const countRows = await countRes.getAllRows();
const countRow = countRows[0];
const count = countRow ? (countRow.cnt ?? countRow[0] ?? 0) : 0;
@@ -430,7 +441,7 @@ export const getLbugStats = async (): Promise<{ nodes: number; edges: number }>
let totalNodes = 0;
for (const tableName of NODE_TABLES) {
try {
const nodeResult = await conn.query(`MATCH (n:${tableName}) RETURN count(n) AS cnt`);
const nodeResult = await conn.query(`MATCH (n:${escapeTableName(tableName)}) RETURN count(n) AS cnt`);
const nodeRows = await nodeResult.getAllRows();
const nodeRow = nodeRows[0];
totalNodes += Number(nodeRow?.cnt ?? nodeRow?.[0] ?? 0);
@@ -573,7 +584,7 @@ export const testArrayParams = async (): Promise<{ success: boolean; error?: str
let testNodeId: string | null = null;
for (const tableName of NODE_TABLES) {
try {
const nodeResult = await conn.query(`MATCH (n:${tableName}) RETURN n.id AS id LIMIT 1`);
const nodeResult = await conn.query(`MATCH (n:${escapeTableName(tableName)}) RETURN n.id AS id LIMIT 1`);
const nodeRows = await nodeResult.getAllRows();
const nodeRow = nodeRows[0];
if (nodeRow) {
+1 -1
View File
@@ -26,7 +26,7 @@ export type NodeTableName = typeof NODE_TABLES[number];
export const REL_TABLE_NAME = 'CodeRelation';
// Valid relation types
export const REL_TYPES = ['CONTAINS', 'DEFINES', 'IMPORTS', 'CALLS', 'EXTENDS', 'IMPLEMENTS', 'MEMBER_OF', 'STEP_IN_PROCESS'] as const;
export const REL_TYPES = ['CONTAINS', 'DEFINES', 'IMPORTS', 'CALLS', 'EXTENDS', 'IMPLEMENTS', 'MEMBER_OF', 'STEP_IN_PROCESS', 'HAS_METHOD', 'HAS_PROPERTY', 'OVERRIDES', 'ACCESSES', 'INHERITS', 'USES', 'DECORATES'] as const;
export type RelType = typeof REL_TYPES[number];
// ============================================================================
+39 -12
View File
@@ -22,12 +22,14 @@ import type {
OllamaConfig,
OpenRouterConfig,
MiniMaxConfig,
GLMConfig,
AgentStreamChunk,
} from './types';
import {
type CodebaseContext,
buildDynamicSystemPrompt,
} from './context-builder';
import { DEFAULT_OLLAMA_BASE_URL, DEFAULT_OPENROUTER_BASE_URL } from '../../config/ui-constants';
/**
* System prompt for the Graph RAG agent
@@ -184,7 +186,7 @@ export const createChatModel = (config: ProviderConfig): BaseChatModel => {
case 'ollama': {
const ollamaConfig = config as OllamaConfig;
return new ChatOllama({
baseUrl: ollamaConfig.baseUrl ?? 'http://localhost:11434',
baseUrl: ollamaConfig.baseUrl ?? DEFAULT_OLLAMA_BASE_URL,
model: ollamaConfig.model,
temperature: ollamaConfig.temperature ?? 0.1,
streaming: true,
@@ -203,7 +205,6 @@ export const createChatModel = (config: ProviderConfig): BaseChatModel => {
if (import.meta.env.DEV) {
console.log('🌐 OpenRouter config:', {
hasApiKey: !!openRouterConfig.apiKey,
apiKeyLength: openRouterConfig.apiKey?.length || 0,
model: openRouterConfig.model,
baseUrl: openRouterConfig.baseUrl,
});
@@ -221,7 +222,7 @@ export const createChatModel = (config: ProviderConfig): BaseChatModel => {
maxTokens: openRouterConfig.maxTokens,
configuration: {
apiKey: openRouterConfig.apiKey, // Ensure client receives it
baseURL: openRouterConfig.baseUrl ?? 'https://openrouter.ai/api/v1',
baseURL: openRouterConfig.baseUrl ?? DEFAULT_OPENROUTER_BASE_URL,
},
streaming: true,
});
@@ -246,6 +247,26 @@ export const createChatModel = (config: ProviderConfig): BaseChatModel => {
});
}
case 'glm': {
const glmConfig = config as GLMConfig;
if (!glmConfig.apiKey || glmConfig.apiKey.trim() === '') {
throw new Error('GLM API key is required but was not provided');
}
return new ChatOpenAI({
apiKey: glmConfig.apiKey,
modelName: glmConfig.model,
temperature: glmConfig.temperature ?? 0.1,
maxTokens: glmConfig.maxTokens,
configuration: {
apiKey: glmConfig.apiKey,
baseURL: glmConfig.baseUrl ?? 'https://api.z.ai/api/coding/paas/v4',
},
streaming: true,
});
}
default:
throw new Error(`Unsupported provider: ${(config as any).provider}`);
}
@@ -355,8 +376,8 @@ export async function* streamAgentResponse(
const yieldedToolCalls = new Set<string>();
const yieldedToolResults = new Set<string>();
let lastProcessedMsgCount = formattedMessages.length;
// Track if all tools are done (for distinguishing reasoning vs final content)
let allToolsDone = true;
// Track pending tool calls (for distinguishing reasoning vs final content)
let pendingToolCalls = 0;
// Track if we've seen any tool calls in this response turn.
// Anything before the first tool call should be treated as "reasoning/narration"
// so the UI can show the Cursor-like loop: plan → tool → update → tool → answer.
@@ -420,7 +441,7 @@ export async function* streamAgentResponse(
const isReasoning =
!hasSeenToolCallThisTurn ||
toolCalls.length > 0 ||
!allToolsDone;
pendingToolCalls > 0;
yield {
type: isReasoning ? 'reasoning' : 'content',
[isReasoning ? 'reasoning' : 'content']: content,
@@ -430,17 +451,23 @@ export async function* streamAgentResponse(
// Track tool calls from message chunks
if (toolCalls.length > 0) {
hasSeenToolCallThisTurn = true;
allToolsDone = false;
pendingToolCalls += toolCalls.length;
for (const tc of toolCalls) {
const toolId = tc.id || `tool-${Date.now()}-${Math.random().toString(36).slice(2)}`;
if (!yieldedToolCalls.has(toolId)) {
yieldedToolCalls.add(toolId);
let parsedArgs: Record<string, any>;
try {
parsedArgs = tc.function?.arguments ? JSON.parse(tc.function.arguments) : {};
} catch {
parsedArgs = {};
}
yield {
type: 'tool_call',
toolCall: {
id: toolId,
name: tc.name || tc.function?.name || 'unknown',
args: tc.args || (tc.function?.arguments ? JSON.parse(tc.function.arguments) : {}),
args: tc.args || parsedArgs,
status: 'running',
},
};
@@ -465,8 +492,8 @@ export async function* streamAgentResponse(
status: 'completed',
},
};
// After tool result, next AI content could be reasoning or final
allToolsDone = true;
// After tool result, decrement pending count
pendingToolCalls = Math.max(0, pendingToolCalls - 1);
}
}
}
@@ -486,7 +513,7 @@ export async function* streamAgentResponse(
for (const tc of toolCalls) {
const toolId = tc.id || `tool-${Date.now()}`;
if (!yieldedToolCalls.has(toolId)) {
allToolsDone = false;
pendingToolCalls++;
yieldedToolCalls.add(toolId);
yield {
type: 'tool_call',
@@ -517,7 +544,7 @@ export async function* streamAgentResponse(
status: 'completed',
},
};
allToolsDone = true;
pendingToolCalls = Math.max(0, pendingToolCalls - 1);
}
}
}
+164 -113
View File
@@ -16,56 +16,92 @@ import {
OllamaConfig,
OpenRouterConfig,
MiniMaxConfig,
GLMConfig,
ProviderConfig,
} from './types';
import { DEFAULT_OPENROUTER_BASE_URL, DEFAULT_OLLAMA_BASE_URL } from '../../config/ui-constants';
const STORAGE_KEY = 'gitnexus-llm-settings';
const mergeWithDefaults = (parsed?: Partial<LLMSettings> | null): LLMSettings => ({
...DEFAULT_LLM_SETTINGS,
...parsed,
openai: {
...DEFAULT_LLM_SETTINGS.openai,
...parsed?.openai,
},
azureOpenAI: {
...DEFAULT_LLM_SETTINGS.azureOpenAI,
...parsed?.azureOpenAI,
},
gemini: {
...DEFAULT_LLM_SETTINGS.gemini,
...parsed?.gemini,
},
anthropic: {
...DEFAULT_LLM_SETTINGS.anthropic,
...parsed?.anthropic,
},
ollama: {
...DEFAULT_LLM_SETTINGS.ollama,
...parsed?.ollama,
},
openrouter: {
...DEFAULT_LLM_SETTINGS.openrouter,
...parsed?.openrouter,
},
minimax: {
...DEFAULT_LLM_SETTINGS.minimax,
...parsed?.minimax,
},
glm: {
...DEFAULT_LLM_SETTINGS.glm,
...parsed?.glm,
},
});
const readSettings = (storage: Storage): Partial<LLMSettings> | null => {
const raw = storage.getItem(STORAGE_KEY);
if (!raw) return null;
try {
return JSON.parse(raw) as Partial<LLMSettings>;
} catch (error) {
console.warn('Failed to parse LLM settings:', error);
return null;
}
};
const writeSettings = (storage: Storage, settings: LLMSettings): void => {
storage.setItem(STORAGE_KEY, JSON.stringify(settings));
};
/**
* Load settings from localStorage
* Load settings from sessionStorage (migrates legacy localStorage once).
*/
export const loadSettings = (): LLMSettings => {
try {
const stored = localStorage.getItem(STORAGE_KEY);
if (!stored) {
return DEFAULT_LLM_SETTINGS;
const sessionData = typeof sessionStorage !== 'undefined' ? readSettings(sessionStorage) : null;
if (sessionData) {
return mergeWithDefaults(sessionData);
}
const parsed = JSON.parse(stored) as Partial<LLMSettings>;
// Merge with defaults to handle new fields
return {
...DEFAULT_LLM_SETTINGS,
...parsed,
openai: {
...DEFAULT_LLM_SETTINGS.openai,
...parsed.openai,
},
azureOpenAI: {
...DEFAULT_LLM_SETTINGS.azureOpenAI,
...parsed.azureOpenAI,
},
gemini: {
...DEFAULT_LLM_SETTINGS.gemini,
...parsed.gemini,
},
anthropic: {
...DEFAULT_LLM_SETTINGS.anthropic,
...parsed.anthropic,
},
ollama: {
...DEFAULT_LLM_SETTINGS.ollama,
...parsed.ollama,
},
openrouter: {
...DEFAULT_LLM_SETTINGS.openrouter,
...parsed.openrouter,
},
minimax: {
...DEFAULT_LLM_SETTINGS.minimax,
...parsed.minimax,
},
};
const legacyData = typeof localStorage !== 'undefined' ? readSettings(localStorage) : null;
if (legacyData) {
const merged = mergeWithDefaults(legacyData);
try {
if (typeof sessionStorage !== 'undefined') {
writeSettings(sessionStorage, merged);
}
if (typeof localStorage !== 'undefined') {
localStorage.removeItem(STORAGE_KEY);
}
} catch (error) {
console.warn('Failed to migrate legacy LLM settings to sessionStorage:', error);
}
return merged;
}
return DEFAULT_LLM_SETTINGS;
} catch (error) {
console.warn('Failed to load LLM settings:', error);
return DEFAULT_LLM_SETTINGS;
@@ -73,11 +109,13 @@ export const loadSettings = (): LLMSettings => {
};
/**
* Save settings to localStorage
* Save settings to sessionStorage
*/
export const saveSettings = (settings: LLMSettings): void => {
try {
localStorage.setItem(STORAGE_KEY, JSON.stringify(settings));
if (typeof sessionStorage !== 'undefined') {
writeSettings(sessionStorage, settings);
}
} catch (error) {
console.error('Failed to save LLM settings:', error);
}
@@ -94,7 +132,9 @@ export const updateProviderSettings = <T extends LLMProvider>(
T extends 'gemini' ? Partial<Omit<GeminiConfig, 'provider'>> :
T extends 'anthropic' ? Partial<Omit<AnthropicConfig, 'provider'>> :
T extends 'ollama' ? Partial<Omit<OllamaConfig, 'provider'>> :
T extends 'openrouter' ? Partial<Omit<OpenRouterConfig, 'provider'>> :
T extends 'minimax' ? Partial<Omit<MiniMaxConfig, 'provider'>> :
T extends 'glm' ? Partial<Omit<GLMConfig, 'provider'>> :
never
>
): LLMSettings => {
@@ -179,6 +219,17 @@ export const updateProviderSettings = <T extends LLMProvider>(
saveSettings(updated);
return updated;
}
case 'glm': {
const updated: LLMSettings = {
...current,
glm: {
...(current.glm ?? {}),
...(updates as Partial<Omit<GLMConfig, 'provider'>>),
},
};
saveSettings(updated);
return updated;
}
default: {
// Should be unreachable due to T extends LLMProvider, but keep a safe fallback
const updated: LLMSettings = { ...current };
@@ -204,77 +255,64 @@ export const setActiveProvider = (provider: LLMProvider): LLMSettings => {
/**
* Get the current provider configuration
*/
type ProviderBuilder = (settings: LLMSettings) => ProviderConfig | null;
const providerBuilders: Record<LLMProvider, ProviderBuilder> = {
openai: (settings) => {
if (!settings.openai?.apiKey) return null;
return { provider: 'openai', ...settings.openai } as OpenAIConfig;
},
'azure-openai': (settings) => {
if (!settings.azureOpenAI?.apiKey || !settings.azureOpenAI?.endpoint) return null;
return { provider: 'azure-openai', ...settings.azureOpenAI } as AzureOpenAIConfig;
},
gemini: (settings) => {
if (!settings.gemini?.apiKey) return null;
return { provider: 'gemini', ...settings.gemini } as GeminiConfig;
},
anthropic: (settings) => {
if (!settings.anthropic?.apiKey) return null;
return { provider: 'anthropic', ...settings.anthropic } as AnthropicConfig;
},
ollama: (settings) => {
return {
provider: 'ollama',
...settings.ollama,
baseUrl: settings.ollama?.baseUrl ?? DEFAULT_OLLAMA_BASE_URL,
} as OllamaConfig;
},
openrouter: (settings) => {
if (!settings.openrouter?.apiKey || settings.openrouter.apiKey.trim() === '') return null;
return {
provider: 'openrouter',
apiKey: settings.openrouter.apiKey,
model: settings.openrouter.model || '',
baseUrl: settings.openrouter.baseUrl || DEFAULT_OPENROUTER_BASE_URL,
temperature: settings.openrouter.temperature,
maxTokens: settings.openrouter.maxTokens,
} as OpenRouterConfig;
},
minimax: (settings) => {
if (!settings.minimax?.apiKey) return null;
return { provider: 'minimax', ...settings.minimax } as MiniMaxConfig;
},
glm: (settings) => {
if (!settings.glm?.apiKey) return null;
return {
provider: 'glm',
apiKey: settings.glm.apiKey,
model: settings.glm.model || 'GLM-5',
baseUrl: settings.glm.baseUrl || 'https://api.z.ai/api/coding/paas/v4',
temperature: settings.glm.temperature,
maxTokens: settings.glm.maxTokens,
} as GLMConfig;
},
};
export const getActiveProviderConfig = (): ProviderConfig | null => {
const settings = loadSettings();
switch (settings.activeProvider) {
case 'openai':
if (!settings.openai?.apiKey) {
return null;
}
return {
provider: 'openai',
...settings.openai,
} as OpenAIConfig;
case 'azure-openai':
if (!settings.azureOpenAI?.apiKey || !settings.azureOpenAI?.endpoint) {
return null;
}
return {
provider: 'azure-openai',
...settings.azureOpenAI,
} as AzureOpenAIConfig;
case 'gemini':
if (!settings.gemini?.apiKey) {
return null;
}
return {
provider: 'gemini',
...settings.gemini,
} as GeminiConfig;
case 'anthropic':
if (!settings.anthropic?.apiKey) {
return null;
}
return {
provider: 'anthropic',
...settings.anthropic,
} as AnthropicConfig;
case 'ollama':
return {
provider: 'ollama',
...settings.ollama,
} as OllamaConfig;
case 'openrouter':
if (!settings.openrouter?.apiKey || settings.openrouter.apiKey.trim() === '') {
return null;
}
return {
provider: 'openrouter',
apiKey: settings.openrouter.apiKey,
model: settings.openrouter.model || '',
baseUrl: settings.openrouter.baseUrl || 'https://openrouter.ai/api/v1',
temperature: settings.openrouter.temperature,
maxTokens: settings.openrouter.maxTokens,
} as OpenRouterConfig;
case 'minimax':
if (!settings.minimax?.apiKey) {
return null;
}
return {
provider: 'minimax',
...settings.minimax,
} as MiniMaxConfig;
default:
return null;
}
const builder = providerBuilders[settings.activeProvider];
return builder ? builder(settings) : null;
};
/**
@@ -288,7 +326,16 @@ export const isProviderConfigured = (): boolean => {
* Clear all settings (reset to defaults)
*/
export const clearSettings = (): void => {
localStorage.removeItem(STORAGE_KEY);
try {
if (typeof sessionStorage !== 'undefined') {
sessionStorage.removeItem(STORAGE_KEY);
}
if (typeof localStorage !== 'undefined') {
localStorage.removeItem(STORAGE_KEY);
}
} catch (error) {
console.warn('Failed to clear LLM settings:', error);
}
};
/**
@@ -310,6 +357,8 @@ export const getProviderDisplayName = (provider: LLMProvider): string => {
return 'OpenRouter';
case 'minimax':
return 'MiniMax';
case 'glm':
return 'GLM (Z.AI)';
default:
return provider;
}
@@ -333,6 +382,8 @@ export const getAvailableModels = (provider: LLMProvider): string[] => {
return ['llama3.2', 'llama3.1', 'mistral', 'codellama', 'deepseek-coder'];
case 'minimax':
return ['MiniMax-M2.5', 'MiniMax-M2.5-highspeed'];
case 'glm':
return ['GLM-5', 'GLM-5-Turbo', 'GLM-4.7', 'GLM-4.5'];
default:
return [];
}
@@ -343,7 +394,7 @@ export const getAvailableModels = (provider: LLMProvider): string[] => {
*/
export const fetchOpenRouterModels = async (): Promise<Array<{ id: string; name: string }>> => {
try {
const response = await fetch('https://openrouter.ai/api/v1/models');
const response = await fetch(`${DEFAULT_OPENROUTER_BASE_URL}/models`);
if (!response.ok) throw new Error('Failed to fetch models');
const data = await response.json();
return data.data.map((model: any) => ({
+23 -5
View File
@@ -2,13 +2,14 @@
* LLM Provider Types
*
* Type definitions for multi-provider LLM support.
* Supports Azure OpenAI and Google Gemini (with extensibility for others).
* Supports OpenAI, Azure OpenAI, Gemini, Anthropic, Ollama, OpenRouter, MiniMax, and GLM5.
*/
/**
* Supported LLM providers
*/
export type LLMProvider = 'openai' | 'azure-openai' | 'gemini' | 'anthropic' | 'ollama' | 'openrouter' | 'minimax';
import { DEFAULT_OLLAMA_BASE_URL, DEFAULT_OPENROUTER_BASE_URL } from '../../config/ui-constants';
export type LLMProvider = 'openai' | 'azure-openai' | 'gemini' | 'anthropic' | 'ollama' | 'openrouter' | 'minimax' | 'glm';
/**
* Base configuration shared by all providers
@@ -87,10 +88,20 @@ export interface MiniMaxConfig extends BaseProviderConfig {
model: string; // e.g., 'MiniMax-M2.5', 'MiniMax-M2.5-highspeed'
}
/**
* GLM (Z.AI) configuration — OpenAI-compatible API
*/
export interface GLMConfig extends BaseProviderConfig {
provider: 'glm';
apiKey: string;
model: string; // e.g., 'GLM-4.7', 'GLM-4.5', 'GLM-4.5-Air', 'GLM-5'
baseUrl?: string; // defaults to https://api.z.ai/api/coding/paas/v4
}
/**
* Union type for all provider configurations
*/
export type ProviderConfig = OpenAIConfig | AzureOpenAIConfig | GeminiConfig | AnthropicConfig | OllamaConfig | OpenRouterConfig | MiniMaxConfig;
export type ProviderConfig = OpenAIConfig | AzureOpenAIConfig | GeminiConfig | AnthropicConfig | OllamaConfig | OpenRouterConfig | MiniMaxConfig | GLMConfig;
/**
* Stored settings (what goes to localStorage)
@@ -108,6 +119,7 @@ export interface LLMSettings {
ollama?: Partial<Omit<OllamaConfig, 'provider'>>;
openrouter?: Partial<Omit<OpenRouterConfig, 'provider'>>;
minimax?: Partial<Omit<MiniMaxConfig, 'provider'>>;
glm?: Partial<Omit<GLMConfig, 'provider'>>;
// Intelligent Clustering Settings
intelligentClustering: boolean;
@@ -148,14 +160,14 @@ export const DEFAULT_LLM_SETTINGS: LLMSettings = {
temperature: 0.1,
},
ollama: {
baseUrl: 'http://localhost:11434',
baseUrl: DEFAULT_OLLAMA_BASE_URL,
model: 'llama3.2',
temperature: 0.1,
},
openrouter: {
apiKey: '',
model: '',
baseUrl: 'https://openrouter.ai/api/v1',
baseUrl: DEFAULT_OPENROUTER_BASE_URL,
temperature: 0.1,
},
minimax: {
@@ -163,6 +175,12 @@ export const DEFAULT_LLM_SETTINGS: LLMSettings = {
model: 'MiniMax-M2.5',
temperature: 0.1,
},
glm: {
apiKey: '',
model: 'GLM-5',
baseUrl: 'https://api.z.ai/api/coding/paas/v4',
temperature: 0.1,
},
};
/**
@@ -43,6 +43,7 @@ const getWasmPath = (language: SupportedLanguages, filePath?: string): string =>
[SupportedLanguages.Ruby]: '/wasm/ruby/tree-sitter-ruby.wasm',
[SupportedLanguages.Kotlin]: '', // Kotlin WASM parser not yet available for web
[SupportedLanguages.Swift]: '/wasm/swift/tree-sitter-swift.wasm',
[SupportedLanguages.Dart]: '/wasm/dart/tree-sitter-dart.wasm',
};
return languageFileMap[language];
@@ -0,0 +1,75 @@
import { createContext, useContext, useCallback, useMemo, useState, ReactNode } from 'react';
import type { KnowledgeGraph, GraphNode, NodeLabel } from '../../core/graph/types';
import { DEFAULT_VISIBLE_LABELS, DEFAULT_VISIBLE_EDGES, type EdgeType } from '../../lib/constants';
interface GraphStateContextValue {
graph: KnowledgeGraph | null;
setGraph: (graph: KnowledgeGraph | null) => void;
fileContents: Map<string, string>;
setFileContents: (contents: Map<string, string>) => void;
selectedNode: GraphNode | null;
setSelectedNode: (node: GraphNode | null) => void;
visibleLabels: NodeLabel[];
toggleLabelVisibility: (label: NodeLabel) => void;
visibleEdgeTypes: EdgeType[];
toggleEdgeVisibility: (edgeType: EdgeType) => void;
depthFilter: number | null;
setDepthFilter: (depth: number | null) => void;
highlightedNodeIds: Set<string>;
setHighlightedNodeIds: (ids: Set<string>) => void;
}
const GraphStateContext = createContext<GraphStateContextValue | null>(null);
export const GraphStateProvider = ({ children }: { children: ReactNode }) => {
const [graph, setGraph] = useState<KnowledgeGraph | null>(null);
const [fileContents, setFileContents] = useState<Map<string, string>>(new Map());
const [selectedNode, setSelectedNode] = useState<GraphNode | null>(null);
const [visibleLabels, setVisibleLabels] = useState<NodeLabel[]>(DEFAULT_VISIBLE_LABELS);
const [visibleEdgeTypes, setVisibleEdgeTypes] = useState<EdgeType[]>(DEFAULT_VISIBLE_EDGES);
const [depthFilter, setDepthFilter] = useState<number | null>(null);
const [highlightedNodeIds, setHighlightedNodeIds] = useState<Set<string>>(new Set());
const toggleLabelVisibility = useCallback((label: NodeLabel) => {
setVisibleLabels(prev =>
prev.includes(label) ? prev.filter(l => l !== label) : [...prev, label]
);
}, []);
const toggleEdgeVisibility = useCallback((edgeType: EdgeType) => {
setVisibleEdgeTypes(prev =>
prev.includes(edgeType) ? prev.filter(e => e !== edgeType) : [...prev, edgeType]
);
}, []);
const value = useMemo<GraphStateContextValue>(() => ({
graph,
setGraph,
fileContents,
setFileContents,
selectedNode,
setSelectedNode,
visibleLabels,
toggleLabelVisibility,
visibleEdgeTypes,
toggleEdgeVisibility,
depthFilter,
setDepthFilter,
highlightedNodeIds,
setHighlightedNodeIds,
}), [graph, fileContents, selectedNode, visibleLabels, visibleEdgeTypes, depthFilter, highlightedNodeIds]);
return (
<GraphStateContext.Provider value={value}>
{children}
</GraphStateContext.Provider>
);
};
export const useGraphState = (): GraphStateContextValue => {
const ctx = useContext(GraphStateContext);
if (!ctx) {
throw new Error('useGraphState must be used within a GraphStateProvider');
}
return ctx;
};
+98 -114
View File
@@ -1,18 +1,21 @@
import { createContext, useContext, useState, useCallback, useRef, useEffect, ReactNode } from 'react';
import { createContext, useContext, useState, useCallback, useRef, useEffect, useMemo, ReactNode } from 'react';
import * as Comlink from 'comlink';
import { KnowledgeGraph, GraphNode, GraphRelationship, NodeLabel } from '../core/graph/types';
import { PipelineProgress, PipelineResult, deserializePipelineResult } from '../types/pipeline';
import { createKnowledgeGraph } from '../core/graph/graph';
import { DEFAULT_VISIBLE_LABELS } from '../lib/constants';
import type { IngestionWorkerApi } from '../workers/ingestion.worker';
import type { FileEntry } from '../services/zip';
import type { EmbeddingProgress, SemanticSearchResult } from '../core/embeddings/types';
import type { LLMSettings, ProviderConfig, AgentStreamChunk, ChatMessage, ToolCallInfo, MessageStep } from '../core/llm/types';
import { loadSettings, getActiveProviderConfig, saveSettings } from '../core/llm/settings-service';
import type { AgentMessage } from '../core/llm/agent';
import { DEFAULT_VISIBLE_EDGES, type EdgeType } from '../lib/constants';
import { type EdgeType } from '../lib/constants';
import type { RepoSummary, ConnectToServerResult } from '../services/server-connection';
import { fetchRepos, connectToServer } from '../services/server-connection';
import { ERROR_RESET_DELAY_MS } from '../config/ui-constants';
import { normalizePath, resolveFilePath as resolvePathFromContents } from '../lib/path-resolution';
import { FILE_REF_REGEX, NODE_REF_REGEX } from '../lib/grounding-patterns';
import { GraphStateProvider, useGraphState } from './app-state/graph';
export type ViewMode = 'onboarding' | 'loading' | 'exploring';
export type RightPanelTab = 'code' | 'chat';
@@ -74,6 +77,8 @@ interface AppState {
setRightPanelTab: (tab: RightPanelTab) => void;
openCodePanel: () => void;
openChatPanel: () => void;
helpDialogBoxOpen: boolean;
setHelpDialogBoxOpen: (open: boolean) => void;
// Filters
visibleLabels: NodeLabel[];
@@ -134,6 +139,7 @@ interface AppState {
// Embedding methods
startEmbeddings: (forceDevice?: 'webgpu' | 'wasm') => Promise<void>;
startEmbeddingsWithFallback: () => void;
semanticSearch: (query: string, k?: number) => Promise<SemanticSearchResult[]>;
semanticSearchWithContext: (query: string, k?: number, hops?: number) => Promise<any[]>;
isEmbeddingReady: boolean;
@@ -145,9 +151,7 @@ interface AppState {
llmSettings: LLMSettings;
updateLLMSettings: (updates: Partial<LLMSettings>) => void;
isSettingsPanelOpen: boolean;
isHelpDialogBoxOpen: boolean;
setSettingsPanelOpen: (open: boolean) => void;
setHelpDialogBoxOpen: (open: boolean) => void;
isAgentReady: boolean;
isAgentInitializing: boolean;
agentError: string | null;
@@ -177,20 +181,37 @@ interface AppState {
const AppStateContext = createContext<AppState | null>(null);
export const AppStateProvider = ({ children }: { children: ReactNode }) => {
export const AppStateProvider = ({ children }: { children: ReactNode }) => (
<GraphStateProvider>
<AppStateProviderInner>{children}</AppStateProviderInner>
</GraphStateProvider>
);
const AppStateProviderInner = ({ children }: { children: ReactNode }) => {
// View state
const [viewMode, setViewMode] = useState<ViewMode>('onboarding');
// Graph data
const [graph, setGraph] = useState<KnowledgeGraph | null>(null);
const [fileContents, setFileContents] = useState<Map<string, string>>(new Map());
// Selection
const [selectedNode, setSelectedNode] = useState<GraphNode | null>(null);
const {
graph,
setGraph,
fileContents,
setFileContents,
selectedNode,
setSelectedNode,
visibleLabels,
toggleLabelVisibility,
visibleEdgeTypes,
toggleEdgeVisibility,
depthFilter,
setDepthFilter,
highlightedNodeIds,
setHighlightedNodeIds,
} = useGraphState();
// Right Panel
const [isRightPanelOpen, setRightPanelOpen] = useState(false);
const [rightPanelTab, setRightPanelTab] = useState<RightPanelTab>('code');
const [helpDialogBoxOpen, setHelpDialogBoxOpen] = useState(false);
const openCodePanel = useCallback(() => {
// Legacy API: used by graph/tree selection.
@@ -204,15 +225,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
setRightPanelTab('chat');
}, []);
// Filters
const [visibleLabels, setVisibleLabels] = useState<NodeLabel[]>(DEFAULT_VISIBLE_LABELS);
const [visibleEdgeTypes, setVisibleEdgeTypes] = useState<EdgeType[]>(DEFAULT_VISIBLE_EDGES);
// Depth filter
const [depthFilter, setDepthFilter] = useState<number | null>(null);
// Query state
const [highlightedNodeIds, setHighlightedNodeIds] = useState<Set<string>>(new Set());
const [queryResult, setQueryResult] = useState<QueryResult | null>(null);
// AI highlights (separate from user/query highlights)
@@ -298,7 +311,6 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
// LLM/Agent state
const [llmSettings, setLLMSettings] = useState<LLMSettings>(loadSettings);
const [isSettingsPanelOpen, setSettingsPanelOpen] = useState(false);
const [isHelpDialogBoxOpen, setHelpDialogBoxOpen] = useState(false);
const [isAgentReady, setIsAgentReady] = useState(false);
const [isAgentInitializing, setIsAgentInitializing] = useState(false);
const [agentError, setAgentError] = useState<string | null>(null);
@@ -313,54 +325,24 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
const [isCodePanelOpen, setCodePanelOpen] = useState(false);
const [codeReferenceFocus, setCodeReferenceFocus] = useState<CodeReferenceFocus | null>(null);
const normalizePath = useCallback((p: string) => {
return p.replace(/\\/g, '/').replace(/^\.?\//, '');
}, []);
const resolveFilePath = useCallback((requestedPath: string): string | null => {
const req = normalizePath(requestedPath).toLowerCase();
if (!req) return null;
return resolvePathFromContents(fileContents, requestedPath);
}, [fileContents]);
// Exact match first
for (const key of fileContents.keys()) {
if (normalizePath(key).toLowerCase() === req) return key;
}
// Ends-with match (best for partial paths like "src/foo.ts")
let best: { path: string; score: number } | null = null;
for (const key of fileContents.keys()) {
const norm = normalizePath(key).toLowerCase();
if (norm.endsWith(req)) {
const score = 1000 - norm.length; // shorter is better
if (!best || score > best.score) best = { path: key, score };
const fileNodeByPath = useMemo(() => {
if (!graph) return new Map<string, string>();
const map = new Map<string, string>();
for (const n of graph.nodes) {
if (n.label === 'File') {
map.set(normalizePath(n.properties.filePath), n.id);
}
}
if (best) return best.path;
// Segment match fallback
const segs = req.split('/').filter(Boolean);
for (const key of fileContents.keys()) {
const normSegs = normalizePath(key).toLowerCase().split('/').filter(Boolean);
let idx = 0;
for (const s of segs) {
const found = normSegs.findIndex((x, i) => i >= idx && x.includes(s));
if (found === -1) { idx = -1; break; }
idx = found + 1;
}
if (idx !== -1) return key;
}
return null;
}, [fileContents, normalizePath]);
return map;
}, [graph]);
const findFileNodeId = useCallback((filePath: string): string | undefined => {
if (!graph) return undefined;
const target = normalizePath(filePath);
const fileNode = graph.nodes.find(
(n) => n.label === 'File' && normalizePath(n.properties.filePath) === target
);
return fileNode?.id;
}, [graph, normalizePath]);
return fileNodeByPath.get(normalizePath(filePath));
}, [fileNodeByPath]);
// Code References methods
const addCodeReference = useCallback((ref: Omit<CodeReference, 'id'>) => {
@@ -419,7 +401,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
}
return kept;
});
}, [queryResult, selectedNode]);
}, [selectedNode]);
// Auto-add a code reference when the user selects a node in the graph/tree
useEffect(() => {
@@ -546,6 +528,25 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
}
}, []);
const startEmbeddingsWithFallback = useCallback(() => {
// Skip auto-start in automated/headless runs to avoid WebGPU errors and long downloads.
const isPlaywright =
(typeof navigator !== 'undefined' && navigator.webdriver) ||
(typeof import.meta !== 'undefined' && typeof import.meta.env !== 'undefined' && import.meta.env.VITE_PLAYWRIGHT_TEST) ||
(typeof process !== 'undefined' && process.env.PLAYWRIGHT_TEST);
if (isPlaywright) {
setEmbeddingStatus('idle');
return;
}
startEmbeddings().catch((err) => {
if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) {
startEmbeddings('wasm').catch(console.warn);
} else {
console.warn('Embeddings auto-start failed:', err);
}
});
}, [startEmbeddings]);
const semanticSearch = useCallback(async (
query: string,
k: number = 10
@@ -709,6 +710,15 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
}
});
};
let pendingUpdate = false;
const scheduleMessageUpdate = () => {
if (pendingUpdate) return;
pendingUpdate = true;
requestAnimationFrame(() => {
pendingUpdate = false;
updateMessage();
});
};
try {
const onChunk = Comlink.proxy((chunk: AgentStreamChunk) => {
@@ -731,7 +741,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
content: chunk.reasoning,
});
}
updateMessage();
scheduleMessageUpdate();
}
break;
@@ -754,7 +764,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
content: chunk.content,
});
}
updateMessage();
scheduleMessageUpdate();
// Parse inline grounding references and add them to the Code References panel.
// Supports: [[file.ts:10-25]] (file refs) and [[Class:View]] (node refs)
@@ -765,7 +775,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
// Pattern 1: File refs - [[path/file.ext]] or [[path/file.ext:line]] or [[path/file.ext:line-line]]
// Line numbers are optional
const fileRefRegex = /\[\[([a-zA-Z0-9_\-./\\]+\.[a-zA-Z0-9]+)(?::(\d+)(?:[-–](\d+))?)?\]\]/g;
const fileRefRegex = new RegExp(FILE_REF_REGEX.source, FILE_REF_REGEX.flags);
let fileMatch: RegExpExecArray | null;
while ((fileMatch = fileRefRegex.exec(fullText)) !== null) {
const rawPath = fileMatch[1].trim();
@@ -791,7 +801,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
}
// Pattern 2: Node refs - [[Type:Name]] or [[graph:Type:Name]]
const nodeRefRegex = /\[\[(?:graph:)?(Class|Function|Method|Interface|File|Folder|Variable|Enum|Type|CodeElement):([^\]]+)\]\]/g;
const nodeRefRegex = new RegExp(NODE_REF_REGEX.source, NODE_REF_REGEX.flags);
let nodeMatch: RegExpExecArray | null;
while ((nodeMatch = nodeRefRegex.exec(fullText)) !== null) {
const nodeType = nodeMatch[1];
@@ -832,7 +842,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
toolCall: tc,
});
setCurrentToolCalls(prev => [...prev, tc]);
updateMessage();
scheduleMessageUpdate();
}
break;
@@ -891,7 +901,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
return prev;
});
updateMessage();
scheduleMessageUpdate();
// Parse highlight marker from tool results
if (tc.result) {
@@ -900,15 +910,15 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
const rawIds = highlightMatch[1].split(',').map((id: string) => id.trim()).filter(Boolean);
if (rawIds.length > 0 && graph) {
const matchedIds = new Set<string>();
const graphNodeIds = graph.nodes.map(n => n.id);
const graphNodeIdSet = new Set(graph.nodes.map(n => n.id));
for (const rawId of rawIds) {
if (graphNodeIds.includes(rawId)) {
if (graphNodeIdSet.has(rawId)) {
matchedIds.add(rawId);
} else {
const found = graphNodeIds.find(gid =>
gid.endsWith(rawId) || gid.endsWith(':' + rawId)
);
const found = graph.nodes.find(n =>
n.id.endsWith(rawId) || n.id.endsWith(':' + rawId)
)?.id;
if (found) {
matchedIds.add(found);
}
@@ -929,15 +939,15 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
const rawIds = impactMatch[1].split(',').map((id: string) => id.trim()).filter(Boolean);
if (rawIds.length > 0 && graph) {
const matchedIds = new Set<string>();
const graphNodeIds = graph.nodes.map(n => n.id);
const graphNodeIdSet = new Set(graph.nodes.map(n => n.id));
for (const rawId of rawIds) {
if (graphNodeIds.includes(rawId)) {
if (graphNodeIdSet.has(rawId)) {
matchedIds.add(rawId);
} else {
const found = graphNodeIds.find(gid =>
gid.endsWith(rawId) || gid.endsWith(':' + rawId)
);
const found = graph.nodes.find(n =>
n.id.endsWith(rawId) || n.id.endsWith(':' + rawId)
)?.id;
if (found) {
matchedIds.add(found);
}
@@ -961,7 +971,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
case 'done':
// Finalize the assistant message - just call updateMessage one more time
updateMessage();
scheduleMessageUpdate();
break;
}
});
@@ -997,7 +1007,6 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
setProgress({ phase: 'extracting', percent: 0, message: 'Switching repository...', detail: `Loading ${repoName}` });
setViewMode('loading');
setIsAgentReady(false);
// Clear stale graph state from previous repo (highlights, selections, blast radius)
@@ -1027,7 +1036,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
// Reuse the same handleServerConnect logic inline
const repoPath = result.repoInfo.repoPath;
const pName = result.repoInfo.name || repoPath.split('/').pop() || 'server-project';
const pName = repoName || result.repoInfo.name || repoPath.split('/').pop() || 'server-project';
setProjectName(pName);
const graph = createKnowledgeGraph();
@@ -1046,13 +1055,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
await initializeAgent(pName);
}
setViewMode('exploring');
startEmbeddings().catch((err) => {
if (err?.name === 'WebGPUNotAvailableError' || err?.message?.includes('WebGPU')) {
startEmbeddings('wasm').catch(console.warn);
} else {
console.warn('Embeddings auto-start failed:', err);
}
});
startEmbeddingsWithFallback();
setProgress(null);
} catch (err) {
console.warn('Failed to load graph into LadybugDB:', err);
@@ -1071,9 +1074,9 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
});
setIsAgentReady(false);
await apiRef.current?.disposeAgent();
setTimeout(() => { setViewMode('exploring'); setProgress(null); }, 3000);
setTimeout(() => { setViewMode('exploring'); setProgress(null); }, ERROR_RESET_DELAY_MS);
}
}, [serverBaseUrl, setProgress, setViewMode, setProjectName, setGraph, setFileContents, loadServerGraph, initializeAgent, startEmbeddings, setHighlightedNodeIds, clearAIToolHighlights, clearAICitationHighlights, clearBlastRadius, setSelectedNode, setQueryResult, setCodeReferences, setCodePanelOpen, setCodeReferenceFocus]);
}, [serverBaseUrl, setProgress, setViewMode, setProjectName, setGraph, setFileContents, loadServerGraph, initializeAgent, startEmbeddingsWithFallback, setHighlightedNodeIds, clearAIToolHighlights, clearAICitationHighlights, clearBlastRadius, setSelectedNode, setQueryResult, setCodeReferences, setCodePanelOpen, setCodeReferenceFocus]);
const removeCodeReference = useCallback((id: string) => {
setCodeReferences(prev => {
@@ -1107,26 +1110,6 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
setCodeReferenceFocus(null);
}, []);
const toggleLabelVisibility = useCallback((label: NodeLabel) => {
setVisibleLabels(prev => {
if (prev.includes(label)) {
return prev.filter(l => l !== label);
} else {
return [...prev, label];
}
});
}, []);
const toggleEdgeVisibility = useCallback((edgeType: EdgeType) => {
setVisibleEdgeTypes(prev => {
if (prev.includes(edgeType)) {
return prev.filter(t => t !== edgeType);
} else {
return [...prev, edgeType];
}
});
}, []);
const value: AppState = {
viewMode,
setViewMode,
@@ -1142,6 +1125,8 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
setRightPanelTab,
openCodePanel,
openChatPanel,
helpDialogBoxOpen,
setHelpDialogBoxOpen,
visibleLabels,
toggleLabelVisibility,
visibleEdgeTypes,
@@ -1184,6 +1169,7 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
embeddingStatus,
embeddingProgress,
startEmbeddings,
startEmbeddingsWithFallback,
semanticSearch,
semanticSearchWithContext,
isEmbeddingReady: embeddingStatus === 'ready',
@@ -1194,8 +1180,6 @@ export const AppStateProvider = ({ children }: { children: ReactNode }) => {
updateLLMSettings,
isSettingsPanelOpen,
setSettingsPanelOpen,
isHelpDialogBoxOpen,
setHelpDialogBoxOpen,
isAgentReady,
isAgentInitializing,
agentError,
+7 -7
View File
@@ -6,16 +6,13 @@ import {
getBackendUrl,
type BackendRepo,
} from '../services/backend';
import { BACKEND_URL_DEBOUNCE_MS, DEFAULT_BACKEND_URL } from '../config/ui-constants';
// ── localStorage keys ────────────────────────────────────────────────────────
const LS_URL_KEY = 'gitnexus-backend-url';
const LS_REPO_KEY = 'gitnexus-backend-repo';
const DEFAULT_URL = 'http://localhost:4747';
// ── Debounce delay ───────────────────────────────────────────────────────────
const DEBOUNCE_MS = 500;
const DEFAULT_URL = DEFAULT_BACKEND_URL;
// ── Public interface ─────────────────────────────────────────────────────────
@@ -90,7 +87,10 @@ export function useBackend(): UseBackendResult {
// Re-check: still the latest probe?
if (id !== probeIdRef.current) return false;
setRepos(repoList);
} catch {
} catch (err) {
if (import.meta.env.DEV) {
console.warn('Failed to fetch repos:', err);
}
if (id === probeIdRef.current) {
setRepos([]);
}
@@ -133,7 +133,7 @@ export function useBackend(): UseBackendResult {
debounceRef.current = setTimeout(() => {
debounceRef.current = null;
void probe();
}, DEBOUNCE_MS);
}, BACKEND_URL_DEBOUNCE_MS);
},
[probe],
);
+39 -2
View File
@@ -19,6 +19,23 @@ export const NODE_COLORS: Record<NodeLabel, string> = {
CodeElement: '#64748b', // Slate - muted
Community: '#818cf8', // Indigo light - cluster indicator
Process: '#f43f5e', // Rose - execution flow indicator
Section: '#60a5fa', // Blue light - structural section
Struct: '#f59e0b', // Amber - like Class
Trait: '#ec4899', // Pink - like Interface
Impl: '#14b8a6', // Teal - like Method
TypeAlias: '#a78bfa', // Violet light - like Type
Const: '#64748b', // Slate - like Variable
Static: '#64748b', // Slate - like Variable
Namespace: '#7c3aed', // Violet - like Module
Union: '#f97316', // Orange - like Enum
Typedef: '#a78bfa', // Violet light - like Type
Macro: '#eab308', // Yellow - like Decorator
Property: '#64748b', // Slate - like Variable
Record: '#f59e0b', // Amber - like Class
Delegate: '#14b8a6', // Teal - like Method
Annotation: '#eab308', // Yellow - like Decorator
Constructor: '#10b981', // Emerald - like Function
Template: '#a78bfa', // Violet light - like Type
};
// Node sizes by type - clear visual hierarchy with dramatic size differences
@@ -41,6 +58,23 @@ export const NODE_SIZES: Record<NodeLabel, number> = {
CodeElement: 2, // Generic small
Community: 0, // Hidden by default - metadata node
Process: 0, // Hidden by default - metadata node
Section: 8, // Structural section - similar to Folder
Struct: 8, // Like Class
Trait: 7, // Like Interface
Impl: 3, // Like Method
TypeAlias: 3, // Like Type
Const: 2, // Like Variable
Static: 2, // Like Variable
Namespace: 13, // Like Module
Union: 5, // Like Enum
Typedef: 3, // Like Type
Macro: 2, // Like Decorator
Property: 2, // Like Variable
Record: 8, // Like Class
Delegate: 3, // Like Method
Annotation: 2, // Like Decorator
Constructor: 4, // Like Function
Template: 3, // Like Type
};
// Community color palette for cluster-based coloring
@@ -78,15 +112,18 @@ export const DEFAULT_VISIBLE_LABELS: NodeLabel[] = [
'Type',
];
// All filterable labels
// All filterable labels (in display order)
export const FILTERABLE_LABELS: NodeLabel[] = [
'Folder',
'File',
'Class',
'Interface',
'Enum',
'Type',
'Function',
'Method',
'Variable',
'Interface',
'Decorator',
'Import',
];
@@ -0,0 +1,7 @@
// Shared regex patterns for grounding references in chat/markdown.
// Pattern 1: File refs - [[path/file.ext]] or [[path/file.ext:line]] or [[path/file.ext:line-line]]
// Line numbers are optional.
export const FILE_REF_REGEX = /\[\[([a-zA-Z0-9_\-./\\]+\.[a-zA-Z0-9]+)(?::(\d+)(?:[-–](\d+))?)?\]\]/g;
// Pattern 2: Node refs - [[Type:Name]] or [[graph:Type:Name]]
export const NODE_REF_REGEX = /\[\[(?:graph:)?(Class|Function|Method|Interface|File|Folder|Variable|Enum|Type|CodeElement):([^\]]+)\]\]/g;
+76
View File
@@ -0,0 +1,76 @@
/**
* Centralized icon re-exports from lucide-react.
*
* All components import icons from this module (@/lib/lucide-icons) rather
* than directly from lucide-react. This provides a single place to manage
* which icons are used and allows future optimization (e.g., tree-shaking
* configuration, icon subset bundling) without touching every component.
*/
export {
AlertCircle,
AlertTriangle,
ArrowRight,
AtSign,
Brain,
Box,
Braces,
Check,
ChevronDown,
ChevronRight,
ChevronUp,
Code,
Copy,
Eye,
EyeOff,
FileArchive,
FileCode,
Filter,
FlaskConical,
Focus,
Folder,
FolderOpen,
GitBranch,
Github,
Globe,
Hash,
Heart,
HelpCircle,
Home,
Key,
Layers,
Lightbulb,
LightbulbOff,
List,
Loader2,
Maximize2,
MousePointerClick,
PanelLeft,
PanelLeftClose,
PanelRightClose,
Pause,
Play,
RefreshCw,
Rocket,
RotateCcw,
Search,
Send,
Server,
Settings,
SkipForward,
Snail,
Sparkles,
Square,
Star,
Table,
Target,
Terminal,
Trash2,
Type,
Upload,
User,
Variable,
X,
Zap,
ZoomIn,
ZoomOut,
} from 'lucide-react';
+45
View File
@@ -0,0 +1,45 @@
// Utilities for normalizing and resolving file paths referenced in chat and code panels.
export const normalizePath = (p: string): string => {
return p.replace(/\\/g, '/').replace(/^\.?\//, '');
};
/**
* Resolve a user-supplied path (which may be partial) to an exact file path in the repo.
* Follows the same heuristics previously embedded in useAppState:
* 1) exact match, 2) ends-with match (prefers shorter paths), 3) segment containment.
*/
export const resolveFilePath = (fileContents: Map<string, string>, requestedPath: string): string | null => {
const req = normalizePath(requestedPath).toLowerCase();
if (!req) return null;
// Exact match first
for (const key of fileContents.keys()) {
if (normalizePath(key).toLowerCase() === req) return key;
}
// Ends-with match (best for partial paths like "src/foo.ts")
let best: { path: string; score: number } | null = null;
for (const key of fileContents.keys()) {
const norm = normalizePath(key).toLowerCase();
if (norm.endsWith(req)) {
const score = 1000 - norm.length; // shorter is better
if (!best || score > best.score) best = { path: key, score };
}
}
if (best) return best.path;
// Segment match fallback
const segs = req.split('/').filter(Boolean);
for (const key of fileContents.keys()) {
const normSegs = normalizePath(key).toLowerCase().split('/').filter(Boolean);
let idx = 0;
for (const s of segs) {
const found = normSegs.findIndex((x, i) => i >= idx && x.includes(s));
if (found === -1) { idx = -1; break; }
idx = found + 1;
}
if (idx !== -1) return key;
}
return null;
};
+2 -3
View File
@@ -12,9 +12,8 @@ declare module '@ladybugdb/wasm-core' {
close(): Promise<void>;
}
export interface QueryResult {
getAll?(): Promise<any[]>;
getAllRows?(): Promise<any[]>;
getAllObjects?(): Promise<any[]>;
getAll(): Promise<any[]>;
getAllRows(): Promise<any[]>;
hasNext(): Promise<boolean>;
getNext(): Promise<any>;
}
+66
View File
@@ -0,0 +1,66 @@
/**
* Shared test data factories for graph structures.
* No test code — pure data exports.
*/
import type { GraphNode, GraphRelationship } from '../../src/core/graph/types';
export function createFileNode(name: string, filePath?: string): GraphNode {
return {
id: `File:${filePath ?? name}`,
label: 'File',
properties: { name, filePath: filePath ?? name },
};
}
export function createFunctionNode(name: string, filePath: string, line = 1): GraphNode {
return {
id: `Function:${filePath}:${name}:${line}`,
label: 'Function',
properties: { name, filePath, startLine: line, endLine: line + 10 },
};
}
export function createClassNode(name: string, filePath: string): GraphNode {
return {
id: `Class:${filePath}:${name}`,
label: 'Class',
properties: { name, filePath },
};
}
export function createProcessNode(id: string, label: string, type: 'cross_community' | 'intra_community' = 'cross_community'): GraphNode {
return {
id,
label: 'Process',
properties: {
name: label,
heuristicLabel: label,
processType: type,
stepCount: 3,
communities: ['cluster-a', 'cluster-b'],
} as any,
};
}
export function createCallsRelationship(sourceId: string, targetId: string): GraphRelationship {
return {
id: `${sourceId}_CALLS_${targetId}`,
sourceId,
targetId,
type: 'CALLS',
confidence: 0.9,
reason: 'same-file',
};
}
export function createContainsRelationship(sourceId: string, targetId: string): GraphRelationship {
return {
id: `${sourceId}_CONTAINS_${targetId}`,
sourceId,
targetId,
type: 'CONTAINS',
confidence: 1.0,
reason: '',
};
}
+4 -2
View File
@@ -1,6 +1,8 @@
import { beforeEach } from 'vitest';
import '@testing-library/jest-dom/vitest';
// Reset storage between tests
beforeEach(() => {
sessionStorage.clear();
localStorage.clear();
sessionStorage.removeItem('gitnexus-llm-settings');
localStorage.removeItem('gitnexus-llm-settings'); // legacy key (migration)
});
+116
View File
@@ -0,0 +1,116 @@
import { describe, expect, it } from 'vitest';
import {
NODE_COLORS,
NODE_SIZES,
COMMUNITY_COLORS,
getCommunityColor,
DEFAULT_VISIBLE_LABELS,
FILTERABLE_LABELS,
ALL_EDGE_TYPES,
DEFAULT_VISIBLE_EDGES,
EDGE_INFO,
} from '../../src/lib/constants';
describe('NODE_COLORS', () => {
it('has a color for every node label used in NODE_SIZES', () => {
for (const label of Object.keys(NODE_SIZES)) {
expect(NODE_COLORS).toHaveProperty(label);
expect(NODE_COLORS[label as keyof typeof NODE_COLORS]).toMatch(/^#[0-9a-f]{6}$/i);
}
});
});
describe('NODE_SIZES', () => {
it('gives Project the largest size', () => {
const maxLabel = Object.entries(NODE_SIZES).reduce((a, b) => a[1] > b[1] ? a : b);
expect(maxLabel[0]).toBe('Project');
});
it('gives structural nodes larger sizes than code nodes', () => {
expect(NODE_SIZES.Folder).toBeGreaterThan(NODE_SIZES.Function);
expect(NODE_SIZES.File).toBeGreaterThan(NODE_SIZES.Variable);
});
});
describe('getCommunityColor', () => {
it('returns valid hex colors', () => {
for (let i = 0; i < 20; i++) {
expect(getCommunityColor(i)).toMatch(/^#[0-9a-f]{6}$/i);
}
});
it('wraps around the palette', () => {
const paletteSize = COMMUNITY_COLORS.length;
expect(getCommunityColor(0)).toBe(getCommunityColor(paletteSize));
expect(getCommunityColor(1)).toBe(getCommunityColor(paletteSize + 1));
});
});
describe('DEFAULT_VISIBLE_LABELS', () => {
it('includes common structural and code labels', () => {
expect(DEFAULT_VISIBLE_LABELS).toContain('File');
expect(DEFAULT_VISIBLE_LABELS).toContain('Function');
expect(DEFAULT_VISIBLE_LABELS).toContain('Class');
});
it('excludes noisy labels by default', () => {
expect(DEFAULT_VISIBLE_LABELS).not.toContain('Variable');
expect(DEFAULT_VISIBLE_LABELS).not.toContain('Import');
});
});
describe('FILTERABLE_LABELS', () => {
it('includes all newly added node types', () => {
expect(FILTERABLE_LABELS).toContain('Enum');
expect(FILTERABLE_LABELS).toContain('Type');
expect(FILTERABLE_LABELS).toContain('Decorator');
expect(FILTERABLE_LABELS).toContain('Variable');
});
it('every filterable label has a defined color in NODE_COLORS', () => {
for (const label of FILTERABLE_LABELS) {
expect(NODE_COLORS).toHaveProperty(label);
expect(NODE_COLORS[label as keyof typeof NODE_COLORS]).toMatch(/^#[0-9a-f]{6}$/i);
}
});
it('every filterable label has a defined size in NODE_SIZES', () => {
for (const label of FILTERABLE_LABELS) {
expect(NODE_SIZES).toHaveProperty(label);
expect(NODE_SIZES[label as keyof typeof NODE_SIZES]).toBeGreaterThan(0);
}
});
it('has no duplicate entries', () => {
const unique = new Set(FILTERABLE_LABELS);
expect(unique.size).toBe(FILTERABLE_LABELS.length);
});
it('is a subset of DEFAULT_VISIBLE_LABELS plus togglable labels', () => {
const allKnown = new Set(Object.keys(NODE_COLORS));
for (const label of FILTERABLE_LABELS) {
expect(allKnown.has(label)).toBe(true);
}
});
});
describe('edge types', () => {
it('ALL_EDGE_TYPES contains all EDGE_INFO keys', () => {
const edgeInfoKeys = Object.keys(EDGE_INFO).sort();
const allEdgeTypes = [...ALL_EDGE_TYPES].sort();
expect(edgeInfoKeys).toEqual(allEdgeTypes);
});
it('DEFAULT_VISIBLE_EDGES is a subset of ALL_EDGE_TYPES', () => {
for (const type of DEFAULT_VISIBLE_EDGES) {
expect(ALL_EDGE_TYPES).toContain(type);
}
});
it('EDGE_INFO entries have color and label', () => {
for (const info of Object.values(EDGE_INFO)) {
expect(info.color).toMatch(/^#[0-9a-f]{6}$/i);
expect(info.label.length).toBeGreaterThan(0);
}
});
});
@@ -0,0 +1,201 @@
import { describe, expect, it } from 'vitest';
import { generateAllCSVs } from '../../src/core/lbug/csv-generator';
import { createKnowledgeGraph } from '../../src/core/graph/graph';
import { NODE_TABLES } from '../../src/core/lbug/schema';
describe('generateAllCSVs', () => {
it('generates CSV for all NODE_TABLES present in the graph', () => {
const graph = createKnowledgeGraph();
// Add one node per multi-language table type
const testTables = ['Struct', 'Enum', 'Trait', 'Impl', 'Macro', 'TypeAlias'] as const;
for (const label of testTables) {
graph.addNode({
id: `${label}:test.rs:MyItem`,
label,
properties: { name: 'MyItem', filePath: 'test.rs', startLine: 1, endLine: 10 },
});
}
const csvData = generateAllCSVs(graph, new Map());
for (const label of testTables) {
const csv = csvData.nodes.get(label);
expect(csv, `CSV for ${label} should be generated`).toBeDefined();
expect(csv!.split('\n').length, `CSV for ${label} should have header + 1 row`).toBeGreaterThanOrEqual(2);
}
});
it('multi-language table CSVs have 6 columns (no isExported)', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'Struct:lib.rs:Point',
label: 'Struct',
properties: { name: 'Point', filePath: 'lib.rs', startLine: 5, endLine: 15 },
});
const csvData = generateAllCSVs(graph, new Map());
const csv = csvData.nodes.get('Struct')!;
const header = csv.split('\n')[0];
const columns = header.split(',').length;
expect(columns).toBe(6); // id, name, filePath, startLine, endLine, content
});
it('community keywords with commas are properly CSV-escaped', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'comm_0',
label: 'Community',
properties: {
name: 'TestCluster',
heuristicLabel: 'test',
keywords: ['auth, login', 'user management'],
description: 'test community',
cohesion: 0.8,
symbolCount: 5,
},
});
const csvData = generateAllCSVs(graph, new Map());
const csv = csvData.nodes.get('Community')!;
const rows = csv.split('\n');
expect(rows.length).toBeGreaterThanOrEqual(2);
// The keywords field should be quoted (RFC 4180) since it contains commas
const dataRow = rows[1];
expect(dataRow).toContain('auth');
expect(dataRow).toContain('login');
});
it('generates File CSV with content from fileContents map', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'File:src/index.ts',
label: 'File',
properties: { name: 'index.ts', filePath: 'src/index.ts' },
});
const fileContents = new Map([['src/index.ts', 'console.log("hello")']]);
const csvData = generateAllCSVs(graph, fileContents);
const csv = csvData.nodes.get('File')!;
expect(csv).toContain('hello');
});
it('returns a relCSV string', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'Function:a.ts:foo',
label: 'Function',
properties: { name: 'foo', filePath: 'a.ts', startLine: 1, endLine: 5 },
});
graph.addNode({
id: 'Function:a.ts:bar',
label: 'Function',
properties: { name: 'bar', filePath: 'a.ts', startLine: 10, endLine: 15 },
});
graph.addRelationship({
sourceId: 'Function:a.ts:foo',
targetId: 'Function:a.ts:bar',
type: 'CALLS',
properties: {},
});
const csvData = generateAllCSVs(graph, new Map());
expect(csvData.relCSV).toContain('CALLS');
expect(csvData.relCSV).toContain('Function:a.ts:foo');
});
it('handles all NODE_TABLES without crashing', () => {
const graph = createKnowledgeGraph();
for (const table of NODE_TABLES) {
if (table === 'File' || table === 'Folder' || table === 'Community' || table === 'Process') continue;
graph.addNode({
id: `${table}:test:item`,
label: table,
properties: { name: 'item', filePath: 'test', startLine: 1, endLine: 2 },
});
}
expect(() => generateAllCSVs(graph, new Map())).not.toThrow();
const csvData = generateAllCSVs(graph, new Map());
expect(csvData.nodes.size).toBeGreaterThan(0);
});
// ── Negative tests ──────────────────────────────────────────────
it('empty graph produces no node CSVs with data rows', () => {
const graph = createKnowledgeGraph();
const csvData = generateAllCSVs(graph, new Map());
// Nodes map may have header-only entries or be empty
for (const [, csv] of csvData.nodes.entries()) {
const rows = csv.split('\n').filter(r => r.trim());
expect(rows.length).toBeLessThanOrEqual(1); // header only, no data
}
});
it('empty graph produces relCSV with only a header', () => {
const graph = createKnowledgeGraph();
const csvData = generateAllCSVs(graph, new Map());
const lines = csvData.relCSV.split('\n').filter(r => r.trim());
expect(lines.length).toBeLessThanOrEqual(1);
});
it('node with double quotes in name is properly escaped', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'Function:a.ts:say"hello"',
label: 'Function',
properties: { name: 'say"hello"', filePath: 'a.ts', startLine: 1, endLine: 5 },
});
const csvData = generateAllCSVs(graph, new Map());
const csv = csvData.nodes.get('Function')!;
// RFC 4180: double quotes inside fields are doubled
expect(csv).toContain('""');
});
it('file node without matching fileContents gets empty content', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'File:missing.ts',
label: 'File',
properties: { name: 'missing.ts', filePath: 'missing.ts' },
});
const csvData = generateAllCSVs(graph, new Map()); // no file contents
const csv = csvData.nodes.get('File')!;
const rows = csv.split('\n');
expect(rows.length).toBeGreaterThanOrEqual(2);
// Content field should be empty (not undefined or crash)
});
it('community with empty keywords array produces valid CSV', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'comm_1',
label: 'Community',
properties: {
name: 'EmptyCluster',
heuristicLabel: 'empty',
keywords: [],
description: '',
cohesion: 0,
symbolCount: 0,
},
});
expect(() => generateAllCSVs(graph, new Map())).not.toThrow();
const csvData = generateAllCSVs(graph, new Map());
expect(csvData.nodes.get('Community')).toBeDefined();
});
it('node labels not in NODE_TABLES are silently skipped', () => {
const graph = createKnowledgeGraph();
graph.addNode({
id: 'FakeLabel:test:item',
label: 'FakeLabel' as any,
properties: { name: 'item', filePath: 'test' },
});
// Should not crash — unknown labels are just not in any CSV
expect(() => generateAllCSVs(graph, new Map())).not.toThrow();
});
});

Some files were not shown because too many files have changed in this diff Show More