Compare commits

..
Author SHA1 Message Date
copilot-swe-agent[bot]andmagyargergo 1d7782e4e1 fix(cobol): single-quote CALL/COPY, sequence number stripping, PERFORM keyword filtering
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/afed41bd-98b4-48e8-a9e5-ebf0b97deaa8
2026-03-24 17:10:16 +00:00
copilot-swe-agent[bot]andmagyargergo 7ce1371ad8 Initial plan: review COBOL processor completeness
Co-authored-by: magyargergo <11230420+magyargergo@users.noreply.github.com>
Agent-Logs-Url: https://github.com/abhigyanpatwari/GitNexus/sessions/afed41bd-98b4-48e8-a9e5-ebf0b97deaa8
2026-03-24 16:59:25 +00:00
copilot-swe-agent[bot] ae437dc30e Initial plan 2026-03-24 16:48:12 +00:00
Gergo Magyar 832789f288 test(cobol): exhaustive 57-test suite with strict exact assertions
Complete rewrite of COBOL integration tests using ground-truth approach:
dump the full graph, then assert EVERY node and EVERY edge.

57 tests across 9 sections:
- Node completeness: Module(3), Function(13), Namespace(2), Property(21),
  Record(1), CodeElement(8), Constructor(1) — exact sorted arrays
- Edge completeness: 22 tests covering every type+reason combination
  with exact source→target pairs
- Cross-program resolution: 6 tests verifying CALL, CICS LINK/XCTL, JCL
- COPY expansion: copybook data items in RPTGEN
- Section hierarchy: exact paragraph membership per section
- Data item ownership: exact per-module breakdown
- MOVE data flow: exact read/write pairs
- JCL integration: job/step/dataset containment
- Grand totals: CALLS(22), CONTAINS(48), IMPORTS(1), ACCESSES(7)

Fixture enhancements:
- CUSTUPDT.cbl: added INIT-SECTION + PROCESSING-SECTION, PERFORM THRU
- AUDITLOG.cbl: added ENTRY "AUDITLOG-BATCH"
- RPTGEN.cbl: added EXEC CICS XCTL

Zero fuzzy assertions — every expect uses toBe(N) or toEqual([...sorted]).
2026-03-24 16:10:29 +00:00
Gergo Magyar 41b0d8dfad test(cobol): add 26 integration tests with exact assertions + fix CICS resolution bug
Integration tests (test/integration/resolvers/cobol.test.ts):
- 26 tests covering full COBOL system extraction
- ALL assertions use exact toBe(N) — zero fuzzy assertions
- Fixtures: CUSTUPDT.cbl, AUDITLOG.cbl, CUSTDAT.cpy, RPTGEN.cbl, RUNJOBS.jcl

Bug fix (cobol-processor.ts):
- CICS LINK/XCTL cross-program resolution was broken — edges were
  created with "resolved" reason but pointing to <unresolved> targets
- Fix: use cics-link-unresolved / cics-xctl-unresolved suffix pattern
  matching the existing cobol-call-unresolved pattern
- Second-pass resolver now patches both CALL and CICS unresolved edges

All 3915 tests pass, 0 failures.
2026-03-24 15:41:00 +00:00
Gergo Magyar 9760f966cb feat(cobol): enrich graph with EXEC SQL/CICS, ENTRY points, MOVE data flow, PERFORM THRU
Maps the remaining 60% of CobolRegexResults to the graph:
- EXEC SQL blocks → CodeElement nodes + ACCESSES edges to DB tables
- EXEC CICS LINK/XCTL → CodeElement nodes + cross-program CALLS edges
- ENTRY points → Constructor nodes (registered for cross-program resolution)
- MOVE statements → ACCESSES edges (read/write data flow tracking)
- PERFORM THRU → expanded CALLS edges for range targets
- File declarations → Record nodes with assignment metadata
- Cross-program CALL 2nd pass: resolves unresolved targets after all programs processed
2026-03-24 15:20:26 +00:00
Gergo Magyar 88c89c42e6 docs: document custom processor pattern in pipeline.ts
Add comment block at the custom processor integration point
documenting the pattern for future non-tree-sitter language additions.
2026-03-24 14:59:09 +00:00
Gergo Magyar 4af677e637 feat: add COBOL language support with regex extraction pipeline
Standalone COBOL processor following the markdown-processor.ts pattern:
- No LanguageProvider modification — COBOL uses regex, not tree-sitter
- No SupportedLanguages enum change — standalone processor pattern

New files:
- cobol-processor.ts — orchestrator (processCobol, isCobolFile, isJclFile)
- cobol/cobol-preprocessor.ts — regex state machine extraction (~888 LOC)
- cobol/cobol-copy-expander.ts — COPY statement expansion with circular detection
- cobol/jcl-parser.ts — JCL job/step/DD extraction
- cobol/jcl-processor.ts — JCL graph node creation

Extraction produces:
- Module nodes (PROGRAM-ID)
- Function nodes (paragraphs)
- Namespace nodes (sections)
- Property nodes (data items)
- CALLS edges (PERFORM intra-file, CALL cross-program)
- IMPORTS edges (COPY statements)
- CONTAINS edges (section → paragraph hierarchy)

Pipeline integration: single processCobol() call in Phase 2.6

54 new tests (33 COBOL + 21 JCL), all 3889 tests pass.
2026-03-24 14:39:25 +00:00
1166 changed files with 37587 additions and 153048 deletions
-1
View File
@@ -1 +0,0 @@
plans/
-21
View File
@@ -1,21 +0,0 @@
---
alwaysApply: true
---
# GitNexus — Cursor project rules
Last reviewed: 2026-03-24
Canonical agent instructions: **[AGENTS.md](../AGENTS.md)** (GitNexus MCP rules, monorepo commands, Cursor Cloud notes). **[CLAUDE.md](../CLAUDE.md)** adds Claude Code-specific notes and points back to AGENTS.md for GitNexus.
## Non-negotiables (always apply)
- NEVER edit a function/class/method without running `gitnexus_impact` first.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename`.
- NEVER commit without running `gitnexus_detect_changes()`.
- NEVER ignore HIGH/CRITICAL risk warnings from impact analysis.
- NEVER run `npx gitnexus analyze` without `--embeddings` if `.gitnexus/meta.json` shows stored embeddings.
Full rules: **[AGENTS.md](../AGENTS.md)** (`gitnexus:start` block, Cursor Cloud section).
**Rule architecture:** Prefer this file plus optional `.cursor/rules/*.mdc` globs (YAML `globs` in frontmatter). Legacy `.cursorrules` is deprecated; content lives here.
-12
View File
@@ -1,12 +0,0 @@
---
globs:
- "gitnexus/**"
- "gitnexus-web/**"
---
# GitNexus build/test quick refs
- CLI (`gitnexus/`): `npm test`; `npm run test:integration`; `npx tsc --noEmit`.
- Web (`gitnexus-web/`): `npm test`; `npm run dev`; `npx tsc -b --noEmit`; `E2E=1 npx playwright test` (needs servers).
- `npm install` in `gitnexus/` runs `prepare` (tsc build) and `postinstall` (tree-sitter patches); needs `python3`, `make`, `g++`.
- LadybugDB locking tests may fail in containerized environments because of `/tmp` file locks (known issue, not a code bug).
-14
View File
@@ -1,14 +0,0 @@
---
globs:
- "eval/**"
---
# GitNexus eval harness (Python)
- **Run tests**: `cd eval && uv run pytest tests/`
- **Run with coverage**: `cd eval && uv run coverage run -m pytest tests/ && uv run coverage report`
- **Lint**: `cd eval && uv run ruff check .`
- **Run eval**: `cd eval && uv run python run_eval.py --config configs/<config>.yaml`
- Shared constants live in `eval/constants.py`; tool specs in `eval/tool_registry.py`.
- Error logging uses `utils/errors.py` — set `GITNEXUS_EVAL_DEBUG=1` for full tracebacks.
- Property-based tests use Hypothesis (`eval/tests/test_property_based.py`).
+3 -3
View File
@@ -1,5 +1,5 @@
# Deprecated for Cursor Agent Mode
# AI Agent Rules
Use **`.cursor/index.mdc`** (`alwaysApply: true`) for project rules. See [AGENTS.md](AGENTS.md).
Follow .gitnexus/RULES.md for all project context and coding guidelines.
This file is kept only as a breadcrumb for older workflows.
This project uses GitNexus MCP for code intelligence. See .gitnexus/RULES.md for available tools and best practices.
-19
View File
@@ -1,19 +0,0 @@
.git
.gitignore
.DS_Store
node_modules
**/node_modules
dist
**/dist
coverage
**/coverage
.env
.env.local
.env.*.local
.gitnexus
gitnexus-web/playwright-report
gitnexus-web/test-results
-15
View File
@@ -1,15 +0,0 @@
# Images (signed Cosign keyless on every push from main / vX.Y.Z tags)
SERVER_IMAGE=ghcr.io/abhigyanpatwari/gitnexus:latest
WEB_IMAGE=ghcr.io/abhigyanpatwari/gitnexus-web:latest
# Container names
SERVER_CONTAINER_NAME=gitnexus-server
WEB_CONTAINER_NAME=gitnexus-web
# Host ports — the web UI expects the server on http://localhost:4747 by default.
SERVER_HOST_PORT=4747
WEB_HOST_PORT=4173
# Optional read-only mount, exposed to the server as /workspace.
# Override with the directory that contains the repos you want to index.
WORKSPACE_DIR=./
-5
View File
@@ -1,5 +0,0 @@
# Prettier initial formatting (2026-03-28)
afcc3d1523f99c77ff67c4fd1af12334660113f6
# ESLint unused import removal (2026-03-28)
1491826bc8da5436d3b1eb092d9274f2c9f028a7
-2
View File
@@ -1,2 +0,0 @@
* text=auto eol=lf
.husky/* text eol=lf
-86
View File
@@ -1,86 +0,0 @@
name: Bug report
description: Report unexpected behavior or a regression
labels: [bug]
body:
- type: markdown
attributes:
value: |
**Goal:** capture enough context to reproduce and fix the issue quickly.
Use **one issue per bug**; split unrelated problems.
- type: dropdown
id: area
attributes:
label: Area
description: Where does the problem show up?
options:
- gitnexus (CLI / core / indexing / MCP server)
- gitnexus-web (browser UI / WASM / workers)
- CI / GitHub Actions
- Documentation / developer experience
- Other
validations:
required: true
- type: textarea
id: summary
attributes:
label: Summary
description: One sentence — what went wrong?
validations:
required: true
- type: textarea
id: context
attributes:
label: Context
description: What were you trying to do? Any relevant links, PRs, or commits?
validations:
required: false
- type: textarea
id: expected
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual behavior
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to reproduce
description: Ordered steps, sample repo or minimal case, commands run.
placeholder: |
1. …
2. …
3. …
validations:
required: true
- type: textarea
id: environment
attributes:
label: Environment
description: OS, Node version, browser (if web), GitNexus version or commit SHA.
placeholder: |
- OS:
- Node:
- Browser (if applicable):
- Commit / version:
validations:
required: false
- type: textarea
id: logs
attributes:
label: Logs / screenshots
description: Paste errors, stack traces, or attach screenshots (redact secrets).
validations:
required: false
-1
View File
@@ -1 +0,0 @@
blank_issues_enabled: true
@@ -1,74 +0,0 @@
name: Feature request
description: Propose a new capability or improvement
labels: [enhancement]
body:
- type: markdown
attributes:
value: |
**Goal:** describe the problem and desired outcome so maintainers can size and prioritize.
Prefer **small, shippable** requests; split large ideas into phases.
- type: dropdown
id: area
attributes:
label: Area
description: Primary part of the monorepo this relates to.
options:
- gitnexus (CLI / core / indexing / MCP server)
- gitnexus-web (browser UI / WASM / workers)
- CI / release / packaging
- Documentation / developer experience
- Other
validations:
required: true
- type: textarea
id: problem
attributes:
label: Problem or opportunity
description: What pain point or gap exists today?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
description: What should happen instead? User-visible behavior, APIs, or UX.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
description: Other approaches you considered and why this one is preferred.
validations:
required: false
- type: textarea
id: acceptance
attributes:
label: Acceptance criteria
description: Testable conditions for “done” (bullets or checkboxes in prose).
placeholder: |
- When … then …
- Documentation / tests updated where appropriate
validations:
required: false
- type: textarea
id: constraints
attributes:
label: Constraints
description: Compatibility, performance, security, or “must not change” boundaries.
validations:
required: false
- type: checkboxes
id: willing
attributes:
label: Contribution
options:
- label: I am willing to open a PR for this (may need design discussion first).
required: false
-52
View File
@@ -1,52 +0,0 @@
## Summary
<!-- One or two sentences: what does this PR change? -->
## Motivation / context
<!-- Why is this change needed? Link issues, ADRs, or prior discussion. -->
## Areas touched
<!-- Check all that apply -->
- [ ] `gitnexus/` (CLI / core / MCP server)
- [ ] `gitnexus-web/` (Vite / React UI)
- [ ] `.github/` (workflows, actions)
- [ ] `eval/` or other tooling
- [ ] Docs / agent config only (`AGENTS.md`, `CLAUDE.md`, `.cursor/`, `llms.txt`, etc.)
## Scope & constraints
**In scope**
- <!-- bullets -->
**Explicitly out of scope / not done here**
- <!-- bullets — prevents reviewers assuming missing work is an oversight -->
## Implementation notes
<!-- Optional: design choices, tradeoffs, follow-ups -->
## Testing & verification
<!-- What you ran; paste commands. Omit sections that do not apply. -->
- [ ] `cd gitnexus && npm test`
- [ ] `cd gitnexus && npm run test:integration` *(if core/indexing/MCP paths changed)*
- [ ] `cd gitnexus && npx tsc --noEmit`
- [ ] `cd gitnexus-web && npm test` *(if web changed)*
- [ ] `cd gitnexus-web && npx tsc -b --noEmit` *(if web changed)*
- [ ] Manual / Playwright E2E *(note environment — see `gitnexus-web/e2e/`)*
## Risk & rollout
<!-- Breaking changes, migrations, index refresh (`npx gitnexus analyze`), release notes -->
## Checklist
- [ ] PR body meets repo minimum length (workflow may label short descriptions)
- [ ] If `AGENTS.md` / overlays changed: headers, scope block, and changelog updated per project conventions
- [ ] No secrets, tokens, or machine-specific paths committed
@@ -1,21 +0,0 @@
name: Setup GitNexus Web
description: Setup Node.js 20, build gitnexus-shared, install web dependencies
runs:
using: composite
steps:
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 20
cache: npm
cache-dependency-path: gitnexus-web/package-lock.json
- name: Build gitnexus-shared
run: npm install && npm run build
shell: bash
working-directory: gitnexus-shared
- name: Install web dependencies
run: npm ci
shell: bash
working-directory: gitnexus-web
@@ -16,11 +16,6 @@ runs:
cache: npm
cache-dependency-path: gitnexus/package-lock.json
- name: Build gitnexus-shared
run: npm install && npm run build
shell: bash
working-directory: gitnexus-shared
- name: Install dependencies
run: npm ci
shell: bash
-75
View File
@@ -1,75 +0,0 @@
version: 2
updates:
# Keep third-party Actions SHA pins current. See CONTRIBUTING.md — when
# reviewing these bumps, verify the SHA corresponds to the claimed tag by
# running `gh api repos/<owner>/<action>/git/refs/tags/<tag>` before merge.
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
open-pull-requests-limit: 5
commit-message:
prefix: chore
include: scope
labels:
- dependencies
- ci
# Gitnexus npm deps — tree-sitter grammars checked daily so we catch
# new releases that unblock the tree-sitter 0.25 upgrade ASAP. Grammars
# are grouped so lockstep bumps produce a single PR. The tree-sitter
# RUNTIME is pinned — upgrade deliberately via the drift check workflow.
# See .github/scripts/check-tree-sitter-upgrade-readiness.py for
# the upgrade readiness tracker.
- package-ecosystem: npm
directory: /gitnexus
schedule:
interval: daily
open-pull-requests-limit: 10
commit-message:
prefix: chore(deps)
include: scope
labels:
- dependencies
groups:
tree-sitter-grammars:
patterns:
- tree-sitter-*
exclude-patterns:
- tree-sitter
- tree-sitter-cli
ignore:
# Pin the tree-sitter runtime at 0.21.x until the drift check
# reports all grammars are peer-dep compatible with 0.25.
- dependency-name: tree-sitter
update-types:
- version-update:semver-major
- version-update:semver-minor
# tree-sitter-cli follows the runtime's version cadence. Bump when
# regenerating vendor/tree-sitter-proto/src/parser.c, not on a schedule.
- dependency-name: tree-sitter-cli
# gitnexus-web (thin frontend client).
- package-ecosystem: npm
directory: /gitnexus-web
schedule:
interval: weekly
open-pull-requests-limit: 5
commit-message:
prefix: chore(deps)
include: scope
labels:
- dependencies
- frontend
# Shared types package.
- package-ecosystem: npm
directory: /gitnexus-shared
schedule:
interval: weekly
open-pull-requests-limit: 5
commit-message:
prefix: chore(deps)
include: scope
labels:
- dependencies
-53
View File
@@ -1,53 +0,0 @@
# release-drafter config — used only for PR autolabeling by
# `.github/workflows/pr-labeler.yml` (the workflow passes `disable-releaser: true`,
# so the draft-release side of release-drafter never runs).
#
# The labels applied here are the same ones `.github/release.yml` maps to
# categorized release-notes sections.
#
# `sync-labels: true` removes managed autolabels that no longer match the PR —
# critical for the breaking-change case: if a PR title drops the `!` or the body
# drops `BREAKING CHANGE:`, the `breaking` label is pulled off automatically.
# Required by release-drafter; not used because releaser is disabled.
name-template: 'unused'
tag-template: 'unused'
template: |
$CHANGES
sync-labels: true
autolabeler:
- label: enhancement
title:
- '/^feat(\([^)]+\))?!?:/i'
- label: bug
title:
- '/^fix(\([^)]+\))?!?:/i'
- label: performance
title:
- '/^perf(\([^)]+\))?!?:/i'
- label: refactor
title:
- '/^refactor(\([^)]+\))?!?:/i'
- label: documentation
title:
- '/^docs(\([^)]+\))?!?:/i'
- label: test
title:
- '/^test(\([^)]+\))?!?:/i'
- label: ci
title:
- '/^ci(\([^)]+\))?!?:/i'
- label: dependencies
title:
- '/^(build|deps)(\([^)]+\))?!?:/i'
- label: chore
title:
- '/^(chore|revert)(\([^)]+\))?!?:/i'
# Breaking-change marker: either `!` in the type prefix or `BREAKING CHANGE:` in body.
- label: breaking
title:
- '/^[a-z]+(\([^)]+\))?!:/i'
body:
- '/BREAKING[ -]CHANGE:/i'
+1 -1
View File
@@ -35,7 +35,7 @@ changelog:
- dependencies
- title: "\U0001F4DD Other Changes"
labels:
- '*'
- "*"
exclude:
labels:
- dependencies
@@ -1,358 +0,0 @@
#!/usr/bin/env python3
"""Monitor tree-sitter 0.25 upgrade readiness.
Tracks two things Dependabot cannot see:
1. Peer-dep compatibility. Each tree-sitter-* grammar declares a peer
dependency on the tree-sitter runtime. We want to know when every
grammar's *latest npm release* satisfies tree-sitter@0.25.0 so we
can upgrade without --legacy-peer-deps.
2. Vendored upstream drift. vendor/tree-sitter-proto/ is a snapshot of
coder3101/tree-sitter-proto's parser.c. When upstream moves, we want
to know whether we can pick it up.
Invoked from .github/workflows/tree-sitter-upgrade-readiness.yml daily.
Runs locally too:
python3 .github/scripts/check-tree-sitter-upgrade-readiness.py
Outputs Markdown to stdout. Exit 0 when every grammar is upgrade-ready
and the vendored proto is in sync. Exit 1 when blockers remain (the
workflow uses this to open or update a tracking issue).
No external deps -- stdlib only, so it runs on any vanilla runner.
"""
from __future__ import annotations
import json
import os
import pathlib
import re
import sys
import urllib.error
import urllib.request
REPO_ROOT = pathlib.Path(__file__).resolve().parents[2]
GITNEXUS_DIR = REPO_ROOT / "gitnexus"
VENDOR_PROTO_DIR = GITNEXUS_DIR / "vendor" / "tree-sitter-proto"
# ── Upgrade target ──────────────────────────────────────────────────────
# The runtime version we want to upgrade TO. Update this when the goal
# changes (e.g. once 0.25 lands and we target 0.26).
TARGET_RUNTIME = "0.25.0"
TARGET_RUNTIME_MAJOR_MINOR = ".".join(TARGET_RUNTIME.split(".")[:2])
# Tree-sitter runtime -> (min_abi, max_abi) it can load. Only the current
# and target entries matter; extend when changing TARGET_RUNTIME.
RUNTIME_ABI_RANGES: dict[str, tuple[int, int]] = {
"0.21": (13, 14),
"0.25": (13, 15),
}
assert TARGET_RUNTIME_MAJOR_MINOR in RUNTIME_ABI_RANGES, (
f"RUNTIME_ABI_RANGES has no entry for {TARGET_RUNTIME_MAJOR_MINOR!r}. "
f"Add the ABI range after auditing the upstream release notes."
)
# Grammars we use. Values are the upstream GitHub repos to check for
# unreleased ABI bumps (owner/repo, branch, parser.c path).
GRAMMARS: dict[str, tuple[str, str, str]] = {
"tree-sitter-c": ("tree-sitter/tree-sitter-c", "master", "src/parser.c"),
"tree-sitter-c-sharp": ("tree-sitter/tree-sitter-c-sharp", "master", "src/parser.c"),
"tree-sitter-cpp": ("tree-sitter/tree-sitter-cpp", "master", "src/parser.c"),
"tree-sitter-dart": ("UserNobody14/tree-sitter-dart", "master", "src/parser.c"),
"tree-sitter-go": ("tree-sitter/tree-sitter-go", "master", "src/parser.c"),
"tree-sitter-java": ("tree-sitter/tree-sitter-java", "master", "src/parser.c"),
"tree-sitter-javascript": ("tree-sitter/tree-sitter-javascript", "master", "src/parser.c"),
"tree-sitter-kotlin": ("fwcd/tree-sitter-kotlin", "main", "src/parser.c"),
"tree-sitter-php": ("tree-sitter/tree-sitter-php", "master", "php/src/parser.c"),
"tree-sitter-python": ("tree-sitter/tree-sitter-python", "master", "src/parser.c"),
"tree-sitter-ruby": ("tree-sitter/tree-sitter-ruby", "master", "src/parser.c"),
"tree-sitter-rust": ("tree-sitter/tree-sitter-rust", "master", "src/parser.c"),
"tree-sitter-swift": ("alex-pinkus/tree-sitter-swift", "main", "src/parser.c"),
"tree-sitter-typescript": ("tree-sitter/tree-sitter-typescript", "master", "typescript/src/parser.c"),
}
UPSTREAM_PROTO_OWNER = "coder3101"
UPSTREAM_PROTO_REPO = "tree-sitter-proto"
UPSTREAM_PROTO_BRANCH = "main"
# ── Helpers ─────────────────────────────────────────────────────────────
def read_current_runtime() -> str:
"""Return the tree-sitter runtime version pinned in package.json (e.g. '0.21')."""
pkg = json.loads((GITNEXUS_DIR / "package.json").read_text())
raw = pkg["dependencies"]["tree-sitter"]
match = re.search(r"(\d+)\.(\d+)", raw)
if not match:
raise SystemExit(f"could not parse tree-sitter version: {raw!r}")
return f"{match.group(1)}.{match.group(2)}"
def npm_view_json(pkg: str) -> dict | None:
"""Fetch package metadata from the npm registry via HTTPS.
Uses the registry API directly so we don't depend on the npm CLI
being available (it's a batch file on Windows which complicates
subprocess calls).
"""
url = f"https://registry.npmjs.org/{pkg}/latest"
try:
req = urllib.request.Request(url, headers={"Accept": "application/json"})
with urllib.request.urlopen(req, timeout=8) as resp:
return json.loads(resp.read().decode("utf-8"))
except (urllib.error.URLError, urllib.error.HTTPError, json.JSONDecodeError):
return None
def satisfies_target(peer_range: str | None, target: str) -> bool:
"""Check if a semver range like '^0.22.4' or '^0.25.0' satisfies the target.
Simple heuristic: extract the minimum version from the range and check
if target >= min. For caret ranges (^X.Y.Z), the upper bound is the
next major (for X>0) or next minor (for X==0). We check both bounds.
"""
if peer_range is None:
# No peer dep declared = no constraint = compatible.
return True
match = re.search(r"(\d+)\.(\d+)\.(\d+)", peer_range)
if not match:
return False
min_major, min_minor, min_patch = int(match.group(1)), int(match.group(2)), int(match.group(3))
t_match = re.search(r"(\d+)\.(\d+)\.(\d+)", target)
if not t_match:
return False
t_major, t_minor, t_patch = int(t_match.group(1)), int(t_match.group(2)), int(t_match.group(3))
# Target must be >= minimum.
target_tuple = (t_major, t_minor, t_patch)
min_tuple = (min_major, min_minor, min_patch)
if target_tuple < min_tuple:
return False
# For caret ranges with major 0: ^0.X.Y allows [0.X.Y, 0.(X+1).0).
if peer_range.startswith("^") and min_major == 0:
if t_major != 0 or t_minor >= min_minor + 1:
return False
# For caret ranges with major >0: ^X.Y.Z allows [X.Y.Z, (X+1).0.0).
elif peer_range.startswith("^") and min_major > 0:
if t_major >= min_major + 1:
return False
return True
_GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN")
def fetch_text(url: str, timeout: int = 8) -> str | None:
"""Fetch a URL and return its text, or None on failure.
Adds an Authorization header for github.com URLs when GITHUB_TOKEN is
set (raises the rate limit from 60 to 5 000 requests/hour).
"""
headers: dict[str, str] = {}
if _GITHUB_TOKEN and ("github.com" in url or "githubusercontent.com" in url):
headers["Authorization"] = f"Bearer {_GITHUB_TOKEN}"
try:
req = urllib.request.Request(url, headers=headers)
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.read().decode("utf-8", errors="ignore")
except (urllib.error.URLError, urllib.error.HTTPError):
return None
def extract_abi_from_text(text: str) -> int | None:
"""Extract LANGUAGE_VERSION from parser.c text."""
match = re.search(r"#define\s+LANGUAGE_VERSION\s+(\d+)", text[:4096])
return int(match.group(1)) if match else None
def extract_language_version(parser_c: pathlib.Path) -> int | None:
"""Return the LANGUAGE_VERSION defined in a parser.c, or None if absent."""
if not parser_c.is_file():
return None
with parser_c.open("r", encoding="utf-8", errors="ignore") as fh:
head = fh.read(4096)
return extract_abi_from_text(head)
def md_h(text: str, level: int = 2) -> str:
return f"{'#' * level} {text}\n"
# ── Main ────────────────────────────────────────────────────────────────
def main() -> int:
blockers: dict[str, str] = {}
lines: list[str] = []
lines.append(md_h("Tree-sitter 0.25 upgrade readiness", 1))
lines.append("")
current_runtime = read_current_runtime()
current_abi_range = RUNTIME_ABI_RANGES.get(current_runtime, (0, 0))
target_abi_range = RUNTIME_ABI_RANGES.get(TARGET_RUNTIME_MAJOR_MINOR, (0, 0))
lines.append(f"- Current runtime: `tree-sitter@{current_runtime}.x` (ABI {current_abi_range[0]}..{current_abi_range[1]})")
lines.append(f"- Target runtime: `tree-sitter@{TARGET_RUNTIME}` (ABI {target_abi_range[0]}..{target_abi_range[1]})")
lines.append("")
# ── Grammar peer-dep compatibility ───────────────────────────────
lines.append(md_h("Grammar compatibility", 2))
lines.append("| Grammar | npm latest | Peer dep | Satisfies 0.25? | ABI | Upstream ABI | Status |")
lines.append("|---|---|---|---|---|---|---|")
ready_count = 0
total_count = len(GRAMMARS)
for name, (upstream_repo, upstream_branch, parser_path) in sorted(GRAMMARS.items()):
# Fetch latest npm metadata.
info = npm_view_json(name)
fetch_failed = info is None
npm_version = "?"
peer_range = None
peer_optional = True
if info:
npm_version = info.get("version", "?")
peers = info.get("peerDependencies") or {}
peer_range = peers.get("tree-sitter")
meta = info.get("peerDependenciesMeta") or {}
ts_meta = meta.get("tree-sitter") or {}
peer_optional = ts_meta.get("optional", False) if peer_range else True
if fetch_failed:
peer_display = "? (fetch failed)"
compatible = False
else:
peer_display = peer_range or "none"
if peer_range and not peer_optional:
peer_display += " (required)"
compatible = satisfies_target(peer_range, TARGET_RUNTIME)
# Check installed ABI using the same parser_path from GRAMMARS.
installed_parser = GITNEXUS_DIR / "node_modules" / name / parser_path
if not installed_parser.is_file():
# Fallback to default location.
installed_parser = GITNEXUS_DIR / "node_modules" / name / "src" / "parser.c"
installed_abi = extract_language_version(installed_parser)
abi_display = str(installed_abi) if installed_abi else "?"
# Check upstream (main/master branch) ABI for unreleased work.
upstream_url = (
f"https://raw.githubusercontent.com/{upstream_repo}/"
f"{upstream_branch}/{parser_path}"
)
upstream_text = fetch_text(upstream_url)
upstream_abi = extract_abi_from_text(upstream_text) if upstream_text else None
upstream_abi_display = str(upstream_abi) if upstream_abi else "?"
# Determine status.
if fetch_failed:
status = "Unknown (fetch failed)"
blockers[name] = f"`{name}`: npm registry fetch failed — could not verify peer dep"
elif compatible:
status = "Ready"
ready_count += 1
elif upstream_abi and upstream_abi >= 15:
status = "Unreleased (ABI 15 on main)"
blockers[name] = f"`{name}`: ABI 15 on `{upstream_repo}` main but not published to npm"
else:
status = "Blocking"
blockers[name] = f"`{name}@{npm_version}`: peer `{peer_display}` incompatible with 0.25"
# Also check upstream package.json for relaxed peer dep.
if not compatible and not fetch_failed:
upstream_pkg_url = (
f"https://raw.githubusercontent.com/{upstream_repo}/"
f"{upstream_branch}/package.json"
)
upstream_pkg_text = fetch_text(upstream_pkg_url)
if upstream_pkg_text:
try:
upstream_pkg = json.loads(upstream_pkg_text)
upstream_peer = (upstream_pkg.get("peerDependencies") or {}).get("tree-sitter")
if upstream_peer and satisfies_target(upstream_peer, TARGET_RUNTIME):
status = "Unreleased (peer relaxed on main)"
blockers[name] = f"`{name}`: peer dep relaxed on `{upstream_repo}` main but not published to npm"
except json.JSONDecodeError:
pass
compat_icon = "Yes" if compatible else "**No**"
lines.append(
f"| `{name}` | {npm_version} | {peer_display} | {compat_icon} | {abi_display} | {upstream_abi_display} | {status} |"
)
lines.append("")
lines.append(f"**{ready_count}/{total_count}** grammars ready for `tree-sitter@{TARGET_RUNTIME}`.")
lines.append("")
# ── Vendored proto drift ─────────────────────────────────────────
lines.append(md_h("Vendored tree-sitter-proto", 2))
vendored_abi = extract_language_version(VENDOR_PROTO_DIR / "src" / "parser.c")
upstream_proto_url = (
f"https://raw.githubusercontent.com/{UPSTREAM_PROTO_OWNER}/"
f"{UPSTREAM_PROTO_REPO}/{UPSTREAM_PROTO_BRANCH}/src/parser.c"
)
upstream_proto_text = fetch_text(upstream_proto_url)
upstream_proto_abi = extract_abi_from_text(upstream_proto_text) if upstream_proto_text else None
sha_url = (
f"https://api.github.com/repos/{UPSTREAM_PROTO_OWNER}/"
f"{UPSTREAM_PROTO_REPO}/commits/{UPSTREAM_PROTO_BRANCH}"
)
sha_text = fetch_text(sha_url)
upstream_sha = "?"
if sha_text:
try:
upstream_sha = json.loads(sha_text).get("sha", "?")[:12]
except json.JSONDecodeError:
pass
local_proto_path = VENDOR_PROTO_DIR / "src" / "parser.c"
local_proto_text = local_proto_path.read_text(encoding="utf-8", errors="ignore") if local_proto_path.is_file() else ""
in_sync = bool(
upstream_proto_text
and local_proto_text.replace("\r\n", "\n")
== upstream_proto_text.replace("\r\n", "\n")
)
lines.append(f"- Upstream: `{UPSTREAM_PROTO_OWNER}/{UPSTREAM_PROTO_REPO}@{UPSTREAM_PROTO_BRANCH}` (HEAD `{upstream_sha}`)")
lines.append(f"- Upstream ABI: **{upstream_proto_abi}**")
lines.append(f"- Vendored ABI: **{vendored_abi}**")
lines.append(f"- In sync: {'yes' if in_sync else 'no — upstream has diverged'}")
if upstream_proto_abi and vendored_abi and upstream_proto_abi > vendored_abi:
can_upgrade = upstream_proto_abi <= target_abi_range[1]
lines.append(f"- Upstream ABI {upstream_proto_abi} {'is' if can_upgrade else 'is NOT'} within target runtime range ({target_abi_range[0]}..{target_abi_range[1]})")
if can_upgrade:
lines.append(f"- **Action:** after upgrading to tree-sitter@{TARGET_RUNTIME}, regenerate vendored parser.c from upstream `{upstream_sha}`")
else:
lines.append(f"- **Action:** wait for runtime upgrade beyond {TARGET_RUNTIME} that supports ABI {upstream_proto_abi}")
blockers["vendored-proto-abi"] = f"vendored tree-sitter-proto: upstream ABI {upstream_proto_abi} outside target range"
elif not in_sync:
lines.append("- **Action:** review upstream changes; vendored copy may need updating")
blockers["vendored-proto-sync"] = "vendored tree-sitter-proto: out of sync with upstream"
# ── Summary ──────────────────────────────────────────────────────
lines.append("")
lines.append(md_h("Summary", 2))
if blockers:
lines.append(f"**{len(blockers)} blocker(s) remaining:**\n")
for b in blockers.values():
lines.append(f"- {b}")
lines.append("")
lines.append("Upgrade to `tree-sitter@0.25` is **blocked**.")
else:
lines.append("All grammars are compatible. Upgrade to `tree-sitter@0.25` is **ready**.")
print("\n".join(lines))
return 1 if blockers else 0
if __name__ == "__main__":
sys.exit(main())
@@ -1,179 +0,0 @@
#!/usr/bin/env python3
"""Enforce the GitHub Actions concurrency convention.
See CONTRIBUTING.md -> "GitHub Actions — Concurrency Convention" for the rules.
Invoked from .github/workflows/ci-quality.yml. Runs locally too:
python3 .github/scripts/check-workflow-concurrency.py .github/workflows
Rules:
1. Every entry-point (non-reusable) workflow declares a top-level
`concurrency:` block.
2. Reusable workflows (on: workflow_call ONLY) do NOT declare one.
3. The `concurrency.group` expression MUST reference either
`${{ github.workflow }}` or one of the approved hardcoded literal prefixes
for workflows that are simultaneously entry-points AND reusable (on: push/
workflow_call). Two such exceptions are currently approved:
- `CI-` for ci.yml (the original canonical form)
- `docker-build-push-` for docker.yml
This is checked by substring containment rather than prefix match because
the group value is a conditional expression that resolves to a `CI-…` or
`docker-build-push-…` literal at runtime.
We deliberately do not use a YAML library — keeps the script dependency-free
on any vanilla runner. `on:` block parsing is line-based and handles both the
flat (`on: workflow_call`) and mapping (`on:\n workflow_call:`) forms.
"""
from __future__ import annotations
import pathlib
import re
import sys
REQUIRED_TOKENS = ("${{ github.workflow }}", "CI-", "docker-build-push-")
def is_reusable(lines: list[str]) -> bool:
"""Return True iff the workflow's `on:` block names only `workflow_call`."""
in_on = False
on_indent: int | None = None
keys: list[str] = []
for raw in lines:
# Skip blank lines and comments
stripped = raw.strip()
if not stripped or stripped.startswith("#"):
continue
indent = len(raw) - len(raw.lstrip(" "))
if not in_on:
if raw.startswith("on:"):
remainder = raw[len("on:"):].strip()
if not remainder:
# `on:` followed by indented mapping on next lines
in_on = True
on_indent = indent
continue
if remainder.startswith("[") and remainder.endswith("]"):
# Flow-style list: on: [workflow_call]
items = [
item.strip() for item in remainder.strip("[]").split(",")
]
return items == ["workflow_call"]
# Scalar form: on: workflow_call (or a single other event)
return remainder == "workflow_call"
continue
# Inside the `on:` block; stop when indentation returns to <= on_indent
if on_indent is not None and indent <= on_indent:
break
# Only consider keys at on_indent + indentation step (anything deeper
# is nested config like `types:`)
if ":" not in stripped:
continue
# Heuristic: first-level event keys are those with indent == on_indent + 2
# (the canonical step for a 2-space YAML doc). We collect all first-level
# keys by tracking the smallest indent seen inside the block.
keys.append((indent, stripped.split(":", 1)[0].strip()))
if not keys:
return False
# Take only the outermost-indented keys as the event list
min_indent = min(i for i, _ in keys)
events = [name for i, name in keys if i == min_indent]
return events == ["workflow_call"]
CONCURRENCY_RE = re.compile(r"^concurrency:\s*$")
GROUP_RE = re.compile(r"^\s+group:\s*(.+?)\s*$")
def extract_group_key(lines: list[str]) -> str | None:
"""Return the `group:` value of the top-level `concurrency:` block, or None."""
for idx, raw in enumerate(lines):
if CONCURRENCY_RE.match(raw):
# Scan forward until we leave the concurrency block (next top-level key
# is at column 0 and ends with `:`).
for follow in lines[idx + 1:]:
if follow and not follow.startswith(" ") and follow.rstrip().endswith(":"):
break
m = GROUP_RE.match(follow)
if m:
return m.group(1).strip().strip("'").strip('"')
break
return None
def has_top_level_concurrency(lines: list[str]) -> bool:
return any(CONCURRENCY_RE.match(raw) for raw in lines)
def check(workflows_dir: pathlib.Path) -> int:
fail = 0
files = sorted(
list(workflows_dir.glob("*.yml")) + list(workflows_dir.glob("*.yaml"))
)
for path in files:
lines = path.read_text(encoding="utf-8").splitlines()
reusable = is_reusable(lines)
has_conc = has_top_level_concurrency(lines)
if reusable:
if has_conc:
print(
f"::error file={path}::Reusable workflow (on: workflow_call) "
"must NOT declare its own concurrency block — it inherits "
"from the caller. See CONTRIBUTING.md -> GitHub Actions — "
"Concurrency Convention."
)
fail = 1
continue
if not has_conc:
print(
f"::error file={path}::Missing top-level concurrency block. "
"See CONTRIBUTING.md -> GitHub Actions — Concurrency Convention."
)
fail = 1
continue
group = extract_group_key(lines)
if group is None:
print(
f"::error file={path}::concurrency block is missing a "
"`group:` key."
)
fail = 1
continue
if not any(token in group for token in REQUIRED_TOKENS):
print(
f"::error file={path}::concurrency.group `{group}` must "
f"reference one of {REQUIRED_TOKENS} (use ${{{{ github.workflow }}}} "
"for normal entry-point workflows; use an approved literal prefix "
"only for workflows that are both entry-points AND reusable — "
"see CONTRIBUTING.md -> GitHub Actions — Concurrency Convention)."
)
fail = 1
return fail
def main(argv: list[str]) -> int:
if len(argv) != 2:
print(f"usage: {argv[0]} <workflows-dir>", file=sys.stderr)
return 2
workflows_dir = pathlib.Path(argv[1])
if not workflows_dir.is_dir():
print(f"not a directory: {workflows_dir}", file=sys.stderr)
return 2
return check(workflows_dir)
if __name__ == "__main__":
sys.exit(main(sys.argv))
+13 -5
View File
@@ -11,8 +11,8 @@ jobs:
outputs:
web_changed: ${{ steps.filter.outputs.web }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v3
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
id: filter
with:
filters: |
@@ -26,9 +26,17 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: ./.github/actions/setup-gitnexus-web
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 20
cache: npm
cache-dependency-path: gitnexus-web/package-lock.json
- name: Install frontend dependencies
run: npm ci
working-directory: gitnexus-web
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
@@ -74,7 +82,7 @@ jobs:
- name: Upload test results
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: e2e-results
path: |
+9 -52
View File
@@ -4,37 +4,11 @@ on:
workflow_call:
jobs:
format:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 20
cache: npm
cache-dependency-path: package-lock.json
- run: npm ci
- run: npx prettier --check .
lint:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 20
cache: npm
cache-dependency-path: package-lock.json
- run: npm ci
- run: npx eslint .
typecheck:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: ./.github/actions/setup-gitnexus
- run: npx tsc --noEmit
working-directory: gitnexus
@@ -43,30 +17,13 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: ./.github/actions/setup-gitnexus-web
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 20
cache: npm
cache-dependency-path: gitnexus-web/package-lock.json
- run: npm ci
working-directory: gitnexus-web
- run: npx tsc -b --noEmit
working-directory: gitnexus-web
# Enforces the convention documented in CONTRIBUTING.md → "GitHub Actions —
# Concurrency Convention":
# 1. Every entry-point (non-reusable) workflow declares a top-level
# `concurrency:` block.
# 2. Reusable workflows (`on: workflow_call` only) do NOT declare one —
# they inherit concurrency from the caller.
# 3. The concurrency group key starts with `${{ github.workflow }}` or
# the literal `CI-` prefix (the documented ci.yml exception for
# reusable-workflow-safe grouping).
# Reusability is detected by parsing each workflow's `on:` block, not an
# allowlist, so new reusable workflows never produce false positives.
workflow-convention:
name: Workflow concurrency convention
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Validate workflow concurrency convention
shell: bash
run: |
set -euo pipefail
python3 .github/scripts/check-workflow-concurrency.py .github/workflows
+7 -17
View File
@@ -6,24 +6,14 @@ name: CI Report
on:
workflow_run:
workflows: ['CI']
workflows: ["CI"]
types: [completed]
permissions:
actions: read # needed to list/download workflow run artifacts
contents: read # needed for sparse checkout of vitest.config.ts
actions: read # needed to list/download workflow run artifacts
contents: read # needed for sparse checkout of vitest.config.ts
pull-requests: write # needed to post sticky PR comment
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Serialize sticky-comment writes per PR so two rapid CI completions don't race.
# Internal PRs surface in `pull_requests[0].number`. Fork PRs leave that array empty,
# so we fall back to `<head-repo-full-name>/<head-branch>`, which is stable across
# reruns and subsequent pushes for the same fork PR (unlike `workflow_run.id` which
# is unique per run and therefore does not serialize anything).
concurrency:
group: ${{ github.workflow }}-${{ github.event.workflow_run.pull_requests[0].number || format('{0}/{1}', github.event.workflow_run.head_repository.full_name, github.event.workflow_run.head_branch) }}
cancel-in-progress: false
jobs:
pr-report:
name: PR Report
@@ -36,7 +26,7 @@ jobs:
steps:
# ── Download artifacts from the CI run ────────────────────────
- name: Download artifacts
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v7
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
with:
script: |
const fs = require('fs');
@@ -123,7 +113,7 @@ jobs:
- name: Checkout (for vitest config)
if: steps.meta.outputs.skip != 'true'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
sparse-checkout: gitnexus/vitest.config.ts
sparse-checkout-cone-mode: false
@@ -132,7 +122,7 @@ jobs:
- name: Fetch base branch coverage
if: steps.meta.outputs.skip != 'true'
id: base-coverage
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v7
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
with:
script: |
const fs = require('fs');
@@ -416,7 +406,7 @@ jobs:
- name: Comment on PR
if: steps.meta.outputs.skip != 'true'
uses: marocchino/sticky-pull-request-comment@0ea0beb66eb9baf113663a64ec522f60e49231c0 # v2
uses: marocchino/sticky-pull-request-comment@773744901bac0e8cbb5a0dc842800d45e9b2b405 # v2
with:
header: ci-report
number: ${{ steps.meta.outputs.pr_number }}
+3 -7
View File
@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: ./.github/actions/setup-gitnexus
with:
build: 'true'
@@ -28,7 +28,6 @@ jobs:
--coverage.reportOnFailure=true
working-directory: gitnexus
# gitnexus-shared already built by setup-gitnexus action above
- name: Install gitnexus-web dependencies
run: npm ci
working-directory: gitnexus-web
@@ -41,12 +40,9 @@ jobs:
--outputFile=web-test-results.json
working-directory: gitnexus-web
- name: Run docker-server integration tests
run: node --test docker-server.test.mjs
- name: Upload test reports
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: test-reports
path: |
@@ -66,7 +62,7 @@ jobs:
runs-on: ${{ matrix.os }}
timeout-minutes: 25
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: ./.github/actions/setup-gitnexus
with:
build: 'true'
+6 -17
View File
@@ -9,20 +9,9 @@ on:
paths-ignore: ['**.md', 'docs/**', 'LICENSE']
workflow_call:
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Hardcoded `CI-` prefix (not `${{ github.workflow }}`) because this workflow is
# invoked as a reusable workflow from publish.yml and release-candidate.yml. In
# called-workflow context `github.workflow` evaluation is ambiguous across GitHub
# Actions versions, and a prefix that could resolve to the caller's name would
# share a concurrency group with the caller → deadlock. A literal prefix is
# immune. Direct `push`/`pull_request` invocations use `CI-<ref>`; invocations
# from a reusable-workflow caller fall into a per-run-unique group that never
# serializes with the caller.
# cancel-in-progress is event-aware: cancel superseded PR runs, queue every other
# event (push to main, workflow_call from publish.yml, etc.).
concurrency:
group: ${{ (github.event_name == 'pull_request' || github.event_name == 'push') && format('CI-{0}', github.ref) || format('CI-nested-{0}', github.run_id) }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
group: ci-${{ github.ref }}
cancel-in-progress: true
# ── Reusable workflow orchestration ─────────────────────────────────
# Each concern lives in its own workflow file for maintainability:
@@ -85,7 +74,7 @@ jobs:
cp pr-meta/e2e_result pr-meta/e2e-result
- name: Upload PR metadata
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: pr-meta
path: pr-meta/
@@ -107,9 +96,9 @@ jobs:
TESTS: ${{ needs.tests.result }}
E2E: ${{ needs.e2e.result }}
run: |
echo "Quality: $QUALITY"
echo "Tests: $TESTS"
echo "E2E: $E2E"
echo "Quality: $QUALITY"
echo "Tests: $TESTS"
echo "E2E: $E2E"
if [[ "$QUALITY" != "success" ]] ||
[[ "$TESTS" != "success" ]]; then
echo "::error::Quality or test jobs failed"
+3 -4
View File
@@ -16,10 +16,9 @@ on:
issue_comment:
types: [created]
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Serialize per-PR to avoid racing review comments.
concurrency:
group: ${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number }}
group: claude-review-${{ github.event.issue.number || github.event.pull_request.number }}
cancel-in-progress: false
jobs:
@@ -57,7 +56,7 @@ jobs:
# For issue_comment triggers, resolve the PR number, head SHA, and fork repo
- name: Resolve PR context
id: pr
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v7
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
with:
script: |
let pr;
@@ -77,7 +76,7 @@ jobs:
core.setOutput('branch', pr.head.ref);
- name: Checkout PR head
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
repository: ${{ steps.pr.outputs.repo }}
ref: ${{ steps.pr.outputs.sha }}
+4 -5
View File
@@ -10,10 +10,9 @@ on:
pull_request_review:
types: [submitted]
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Serialize per-PR/issue to avoid racing comments.
concurrency:
group: ${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number || github.event.issue.id }}
group: claude-code-${{ github.event.issue.number || github.event.pull_request.number || github.event.issue.id }}
cancel-in-progress: false
jobs:
@@ -54,12 +53,12 @@ jobs:
pull-requests: write
issues: write
id-token: write
actions: read # required for Claude to read CI results on PRs
actions: read # required for Claude to read CI results on PRs
steps:
# For PR-related triggers, resolve the fork repo so we can checkout correctly.
- name: Resolve PR context
id: pr
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v7
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
with:
script: |
// Determine if this event is PR-related
@@ -91,7 +90,7 @@ jobs:
core.setOutput('branch', pr.head.ref);
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
repository: ${{ steps.pr.outputs.is_pr == 'true' && steps.pr.outputs.repo || github.repository }}
ref: ${{ steps.pr.outputs.is_pr == 'true' && steps.pr.outputs.sha || '' }}
-202
View File
@@ -1,202 +0,0 @@
name: Docker Build & Push
on:
push:
tags:
- 'v*'
# No workflow_dispatch: publishing is exclusively tag-driven so that every
# signed image corresponds 1:1 to a published `gitnexus@X.Y.Z` on npm. A
# manual run from a branch ref would fail the version check below anyway.
workflow_call:
inputs:
tag:
description: >-
The full v-prefixed tag to build (e.g. v1.2.3-rc.1).
The tag must already exist in the repo and its tree must contain
a gitnexus/package.json whose version matches the tag.
required: true
type: string
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Tag refs are unique per release, so distinct tags run in parallel.
# Re-pushes of the same tag serialize. cancel-in-progress: false — never cancel a publish mid-flight.
# Hardcoded `docker-build-push-` prefix (not `${{ github.workflow }}`) when invoked as a reusable
# workflow: in called-workflow context `github.workflow` is ambiguous and could resolve to the
# caller's name, sharing a concurrency group with the caller → deadlock.
# Direct tag-push invocations use `docker-build-push-<ref>`; workflow_call invocations get a
# per-run-unique group (they are already serialized by the caller's own concurrency group).
concurrency:
group: ${{ (github.event_name == 'push') && format('docker-build-push-{0}', github.ref) || format('docker-build-push-nested-{0}', github.run_id) }}
cancel-in-progress: false
jobs:
build-push:
name: Build & Push ${{ matrix.image.name }}
runs-on: ubuntu-latest
timeout-minutes: 60
permissions:
contents: read
packages: write
# Required for Cosign keyless signing via the OIDC token exchange,
# and for build provenance / SBOM attestations.
id-token: write
attestations: write
strategy:
fail-fast: false
matrix:
image:
# Static UI bundle. Small, fast image. Drop-in replacement for the
# legacy single-image setup at the same `gitnexus` repository slug
# is intentionally avoided — the UI now lives at `gitnexus-web` and
# the CLI/server takes the canonical `gitnexus` slug below.
- name: gitnexus-web
dockerfile: Dockerfile.web
slug: gitnexus-web
# CLI / `gitnexus serve` backend. Heavy native deps (tree-sitter,
# onnxruntime-node) live only in this image.
- name: gitnexus
dockerfile: Dockerfile.cli
slug: gitnexus
steps:
- name: Validate tag input
if: github.event_name == 'workflow_call'
shell: bash
env:
TAG_INPUT: ${{ inputs.tag }}
run: |
if [ -z "${TAG_INPUT}" ]; then
echo "::error::No tag provided to docker.yml — refusing to build/push."
exit 1
fi
# When triggered by workflow_call the caller passes the RC tag as an input;
# we check out that tag so the Dockerfile and package.json match the built image.
# For tag-push events github.ref is already the tag ref — no override needed.
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.tag || github.ref }}
# ── Lock the docker image version to the npm package version ──────────
# Mirrors the check in publish.yml: refuse to build unless the git tag
# exactly matches `gitnexus/package.json`'s version. This guarantees
# `ghcr.io/<owner>/gitnexus:X.Y.Z` always corresponds to the same
# `gitnexus@X.Y.Z` published to npm — no drift, no surprises.
- name: Verify tag matches gitnexus/package.json version
id: version
shell: bash
env:
# For workflow_call the tag comes from the caller input; for push events
# it is derived from GITHUB_REF (set to empty so the else-branch fires).
INPUT_TAG: ${{ inputs.tag }}
run: |
if [ -n "$INPUT_TAG" ]; then
TAG_VERSION="${INPUT_TAG#v}"
else
TAG_VERSION="${GITHUB_REF#refs/tags/v}"
fi
if ! [[ "$TAG_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9.]+)?$ ]]; then
echo "::error::Tag does not follow semver: v$TAG_VERSION"
exit 1
fi
PKG_VERSION=$(node -p "require('./gitnexus/package.json').version")
if [ "$TAG_VERSION" != "$PKG_VERSION" ]; then
echo "::error::Tag version (v$TAG_VERSION) does not match gitnexus/package.json version ($PKG_VERSION)"
exit 1
fi
echo "version=$PKG_VERSION" >> "$GITHUB_OUTPUT"
echo "Version verified: $PKG_VERSION"
# Required for multi-platform (linux/arm64) emulation.
- name: Set up QEMU
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Install Cosign
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
- name: Log in to GitHub Container Registry
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Computes image tags and labels from the verified semver tag:
# v1.2.3 → :1.2.3, :1.2, :1, :latest (auto, only for non-prerelease)
# v1.2.3-rc.1 → :1.2.3-rc.1 only (prereleases never become :latest)
# `:latest` is only emitted for tag pushes thanks to `flavor: latest=auto`,
# ensuring it always points at a real npm-published version.
#
# For workflow_call invocations github.ref is the caller's branch ref, so
# the type=semver patterns would not match. In that case we add an explicit
# type=raw tag using the version already verified above, so the same
# image-naming rules apply regardless of how the workflow was triggered.
# NOTE: We check `inputs.tag` rather than `github.event_name` because in a
# reusable workflow the github context is inherited from the caller —
# `github.event_name` would still be "push", not "workflow_call".
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
with:
images: ghcr.io/${{ github.repository_owner }}/${{ matrix.image.slug }}
flavor: latest=auto
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=raw,value=${{ steps.version.outputs.version }},enable=${{ inputs.tag != '' }}
- name: Build and push
id: build
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
file: ${{ matrix.image.dockerfile }}
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha,scope=${{ matrix.image.slug }}
cache-to: type=gha,mode=max,scope=${{ matrix.image.slug }}
provenance: mode=max
sbom: true
# Cosign keyless signing. Each pushed tag is signed by the workflow's
# OIDC identity, so consumers can verify the image with the strict,
# fully-anchored identity regex (kept in sync with README.md and
# deploy/kubernetes/cluster-image-policy.yaml — update all three together).
# NOTE: `${...}` expression syntax is NOT evaluated inside YAML comments, so
# the example below uses literal `<owner>/<repo>` placeholders that consumers
# substitute themselves; the canonical, fully-rendered command lives in README.md.
# cosign verify ghcr.io/<owner>/<slug>:<tag> \
# --certificate-identity-regexp '^https://github\.com/<owner>/<repo>/\.github/workflows/docker\.yml@refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9.]+)?$' \
# --certificate-oidc-issuer https://token.actions.githubusercontent.com
# Do NOT relax to `@.*` — that accepts signatures from any ref, including
# unprotected branches and PRs, and defeats the supply-chain guarantee.
- name: Sign image with Cosign (keyless)
env:
# Cosign v2 (installed by sigstore/cosign-installer above) makes
# keyless the default. COSIGN_EXPERIMENTAL is a v1-only opt-in flag
# that is now deprecated/no-op, so it is intentionally omitted.
DIGEST: ${{ steps.build.outputs.digest }}
TAGS: ${{ steps.meta.outputs.tags }}
run: |
# Sign every tag at the same digest so consumers can verify by tag or by digest.
# Use `while read` instead of `for $TAGS` to be robust against tags that
# could ever contain whitespace (the metadata-action output is newline-
# separated, not space-separated).
while IFS= read -r tag; do
[[ -n "$tag" ]] && cosign sign --yes "${tag}@${DIGEST}"
done <<< "$TAGS"
# Attach the SBOM produced by buildx as a verifiable attestation on the digest.
- name: Generate build provenance attestation
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
with:
subject-name: ghcr.io/${{ github.repository_owner }}/${{ matrix.image.slug }}
subject-digest: ${{ steps.build.outputs.digest }}
push-to-registry: true
+2 -3
View File
@@ -8,9 +8,8 @@ on:
permissions:
pull-requests: write
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
group: pr-desc-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
@@ -19,7 +18,7 @@ jobs:
timeout-minutes: 5
steps:
- name: Check PR description quality
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const MIN_BODY_LENGTH = 50;
-113
View File
@@ -1,113 +0,0 @@
name: PR Conventional Labeler
# Two workflows in one file with different triggers, matched to the minimum
# privilege each needs:
#
# validate-title (on: pull_request)
# Fork-safe. Runs with the PR-head's read-only GITHUB_TOKEN. Uses
# `amannn/action-semantic-pull-request` to fail the check when the PR
# title doesn't follow the conventional-commit format. Because the
# action only reads the event payload, no fork-controlled code runs.
#
# autolabel (on: pull_request_target)
# Needs `pull-requests: write` to apply labels, so must be
# pull_request_target. Uses `release-drafter/release-drafter` with
# `dry-run: true` to only run the autolabeler against the
# `.github/release-drafter.yml` config from the BASE ref (release-
# drafter reads the config from the repository's default branch, NOT
# the PR head — verify with `gh api repos/release-drafter/release-drafter/contents/...`
# or a fork-test PR before merging if the repo is high-value).
# `sync-labels: true` in the config removes managed autolabels that no
# longer match (e.g. when `!` or `BREAKING CHANGE:` is dropped).
#
# Title format: <type>[(scope)][!]: <subject>
# Allowed types: feat, fix, perf, refactor, docs, test, ci, build, chore, revert, deps
# Trailing `!` on the type marks a breaking change.
# See CONTRIBUTING.md → "Pull request titles".
on:
pull_request:
# Title-only changes fire `edited`. `opened` and `reopened` cover creation.
# `synchronize` (push to the PR branch) is intentionally excluded — titles
# don't change on push, so it only wastes CI minutes and broadens the
# privileged-token exposure window on the autolabel job.
types: [opened, edited, reopened]
pull_request_target:
types: [opened, edited, reopened]
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Include `github.event_name` so `pull_request` (validate-title) and
# `pull_request_target` (autolabel) runs for the same PR do NOT share a slot
# and therefore cannot cancel each other — a cancelled required-check would
# permanently block merge until the next title edit.
# Within each trigger the latest title edit still supersedes the prior run.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
validate-title:
# Fork-safe job — only runs on `pull_request` (not `pull_request_target`).
# Token is read-only; writes a commit status that branch protection can
# require before merge.
name: Validate PR title
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
pull-requests: read
steps:
# Pinned to v6.1.1. Verify SHA via:
# gh api repos/amannn/action-semantic-pull-request/git/refs/tags/v6.1.1
- uses: amannn/action-semantic-pull-request@48f256284bd46cdaab1048c3721360e808335d50 # v6.1.1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
types: |
feat
fix
perf
refactor
docs
test
ci
build
chore
revert
deps
requireScope: false
# Subject must be non-empty. We DO allow capitalized proper nouns
# (MCP, GitHub, API, etc.) — the old `^(?![A-Z]).+$` pattern
# rejected legitimate titles like `fix: MCP tool schema`.
subjectPattern: ^\S.{2,}$
subjectPatternError: |
The subject "{subject}" in PR title "{title}" is invalid.
Subjects must be at least 3 characters and must not start with whitespace.
wip: false
autolabel:
# Privileged job — runs only on `pull_request_target` so it can write labels.
# Never checks out fork code, never executes fork-controlled input; only
# reads the PR metadata (title, body, labels) and calls the GitHub API.
name: Apply conventional label
if: github.event_name == 'pull_request_target'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
# `contents: read` is required — release-drafter's context.config() reads
# `.github/release-drafter.yml` from the repo's default branch via the
# repo-contents API. Without it the job silently 403s and no labels are
# applied. Job-level permissions nullify all unlisted scopes, so an
# explicit grant is necessary here.
contents: read
pull-requests: write
steps:
# Pinned to v7.2.0. Verify SHA via:
# gh api repos/release-drafter/release-drafter/git/refs/tags/v7.2.0
# v7 removed `disable-releaser`; use `dry-run: true` to only autolabel.
- uses: release-drafter/release-drafter@5de93583980a40bd78603b6dfdcda5b4df377b32 # v7.2.0
with:
config-name: release-drafter.yml
dry-run: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+5 -33
View File
@@ -7,22 +7,13 @@ on:
# No workflow-level permissions — scoped per job below.
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Tag refs are unique per release, so distinct tags run in parallel. Re-pushes of the
# same tag serialize. cancel-in-progress: false — never cancel a publish mid-flight.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
jobs:
ci:
uses: ./.github/workflows/ci.yml
permissions:
contents: read
actions: read
# No pull-requests:write — `ci.yml`'s save-pr-meta job is gated on
# `github.event_name == 'pull_request'`, so it never runs during a
# tag-triggered publish. Least-privilege for release-critical paths.
pull-requests: write
publish:
needs: ci
@@ -32,17 +23,13 @@ jobs:
contents: write
id-token: write
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: 20
registry-url: https://registry.npmjs.org
cache: npm
cache-dependency-path: gitnexus/package-lock.json
- name: Build gitnexus-shared
run: npm install && npm run build
working-directory: gitnexus-shared
- run: npm ci
working-directory: gitnexus
@@ -76,22 +63,7 @@ jobs:
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Extract release notes from CHANGELOG
id: changelog
shell: bash
run: |
VERSION="${GITHUB_REF#refs/tags/v}"
NOTES=$(awk "/^## \\[$VERSION\\]/{found=1; next} /^## \\[/{if(found) exit} found" gitnexus/CHANGELOG.md)
if [ -z "$NOTES" ]; then
echo "::warning::No CHANGELOG entry found for v$VERSION, falling back to auto-generated notes"
echo "fallback=true" >> "$GITHUB_OUTPUT"
else
echo "$NOTES" > /tmp/release-notes.md
echo "fallback=false" >> "$GITHUB_OUTPUT"
fi
- name: Create GitHub Release
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v2
uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2
with:
body_path: ${{ steps.changelog.outputs.fallback == 'false' && '/tmp/release-notes.md' || '' }}
generate_release_notes: ${{ steps.changelog.outputs.fallback == 'true' }}
generate_release_notes: true
-388
View File
@@ -1,388 +0,0 @@
name: Release Candidate
on:
# Publish a release-candidate build whenever a merge/commit lands on main.
# Docs/README-only changes are filtered out so prose updates don't
# cut a release.
push:
branches: [main]
paths-ignore:
- '**.md'
- 'docs/**'
- 'LICENSE'
workflow_dispatch:
inputs:
bump:
description: >-
Cycle policy. 'auto' (default) continues the active rc cycle on
this branch if there is one, otherwise bumps patch from latest.
Choose 'patch' / 'minor' / 'major' to explicitly start or reset
an rc cycle.
required: false
default: 'auto'
type: choice
options:
- auto
- patch
- minor
- major
force:
description: 'Publish even when HEAD already has an rc marker'
required: false
default: 'false'
type: choice
options:
- 'false'
- 'true'
# No workflow-level permissions — scoped per job below.
permissions: {}
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Serialize all runs on the same ref (push + workflow_dispatch) to prevent two publishes
# racing on the rc counter. cancel-in-progress: false — the earlier merge publishes first.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
jobs:
# ── Skip when HEAD already has an rc marker (retry / duplicate dispatch) ──
# The marker is a lightweight tag `rc/<HEAD_SHA>` pushed *before* `npm
# publish`, so a failed publish leaves the marker in place and the guard
# refuses to re-publish. Recovery path after a partial failure:
# git push --delete origin rc/<HEAD_SHA> v<RC_VERSION>
# then redispatch with force=true.
guard:
name: Check if release candidate should run
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
should_run: ${{ steps.decide.outputs.should_run }}
head_sha: ${{ steps.decide.outputs.head_sha }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
fetch-tags: true
- name: Decide
id: decide
shell: bash
env:
FORCE: ${{ inputs.force }}
BUMP_INPUT: ${{ inputs.bump }}
EVENT_NAME: ${{ github.event_name }}
run: |
set -euo pipefail
HEAD_SHA=$(git rev-parse HEAD)
echo "head_sha=$HEAD_SHA" >> "$GITHUB_OUTPUT"
if [ "$FORCE" = "true" ]; then
echo "Force flag set — running regardless of marker tag."
echo "should_run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# An explicit cycle reset on dispatch (bump != auto) also bypasses
# the dedup guard — the maintainer is deliberately asking for a
# new rc from the same commit.
if [ "$EVENT_NAME" = "workflow_dispatch" ] \
&& [ -n "${BUMP_INPUT:-}" ] \
&& [ "${BUMP_INPUT:-auto}" != "auto" ]; then
echo "Explicit bump=$BUMP_INPUT — bypassing marker dedup."
echo "should_run=true" >> "$GITHUB_OUTPUT"
exit 0
fi
# Dedup: is there already an rc/<HEAD_SHA> marker pointing at HEAD?
MARKER="rc/${HEAD_SHA}"
if git rev-parse "refs/tags/$MARKER" >/dev/null 2>&1; then
echo "HEAD already has marker $MARKER — skipping."
echo "should_run=false" >> "$GITHUB_OUTPUT"
else
echo "No marker on HEAD — proceeding."
echo "should_run=true" >> "$GITHUB_OUTPUT"
fi
# ── Reuse the stable CI workflow ─────────────────────────────────────
ci:
needs: guard
if: needs.guard.outputs.should_run == 'true'
uses: ./.github/workflows/ci.yml
permissions:
contents: read
secrets: inherit
# ── Publish the rc build to npm + create GitHub prerelease ───────────
publish:
name: Publish release candidate to npm
needs: [guard, ci]
if: needs.guard.outputs.should_run == 'true'
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: write # push rc tag + marker
id-token: write # npm provenance
outputs:
vtag: ${{ steps.reltag.outputs.vtag }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
fetch-tags: true
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 20
registry-url: https://registry.npmjs.org
cache: npm
cache-dependency-path: gitnexus/package-lock.json
- name: Build gitnexus-shared
run: npm install && npm run build
working-directory: gitnexus-shared
- name: Install gitnexus dependencies
run: npm ci
working-directory: gitnexus
- name: Resolve rc version
id: version
shell: bash
working-directory: gitnexus
env:
BUMP_INPUT: ${{ inputs.bump }}
EVENT_NAME: ${{ github.event_name }}
PKG_NAME: gitnexus
run: |
set -euo pipefail
# 1. Current published `latest` — the floor for any new rc base.
# Only E404 ("never published") falls back to package.json; any
# other error (network, auth, malformed response) fails fast.
NPM_STDERR_LATEST="$(mktemp)"
if CURRENT_LATEST="$(npm view "$PKG_NAME" version 2>"$NPM_STDERR_LATEST")"; then
:
else
if grep -q 'E404' "$NPM_STDERR_LATEST"; then
CURRENT_LATEST="$(node -p "require('./package.json').version")"
echo "Package not on registry (E404) — seeding from package.json: $CURRENT_LATEST"
else
echo "::error::npm registry unreachable for 'view version':" >&2
cat "$NPM_STDERR_LATEST" >&2
rm -f "$NPM_STDERR_LATEST"
exit 1
fi
fi
rm -f "$NPM_STDERR_LATEST"
CURRENT_LATEST_CLEAN="${CURRENT_LATEST%%-*}"
# 2. Full version list — needed for the counter and for active-cycle
# inference. Same E404-only fallback.
NPM_STDERR_VERSIONS="$(mktemp)"
if VERSIONS_JSON="$(npm view "$PKG_NAME" versions --json 2>"$NPM_STDERR_VERSIONS")"; then
:
else
if grep -q 'E404' "$NPM_STDERR_VERSIONS"; then
VERSIONS_JSON='[]'
echo "No published versions for $PKG_NAME yet (E404)."
else
echo "::error::npm registry unreachable for 'view versions':" >&2
cat "$NPM_STDERR_VERSIONS" >&2
rm -f "$NPM_STDERR_VERSIONS"
exit 1
fi
fi
rm -f "$NPM_STDERR_VERSIONS"
# 3. Base selection.
# - workflow_dispatch + bump ∈ {patch,minor,major} → explicit cycle
# reset from latest.
# - Everything else (push, or dispatch with bump=auto) → continue
# the highest active rc base > latest if one exists; else
# default to patch from latest.
if [ "$EVENT_NAME" = "workflow_dispatch" ] \
&& [ -n "${BUMP_INPUT:-}" ] \
&& [ "${BUMP_INPUT:-auto}" != "auto" ]; then
BASE="$(npx --yes -p semver@7 semver -i "$BUMP_INPUT" "$CURRENT_LATEST_CLEAN")"
echo "Explicit bump=$BUMP_INPUT → BASE=$BASE"
else
cat > /tmp/active_base.mjs <<'NODESCRIPT'
const latest = process.env.LATEST;
let v;
try { v = JSON.parse(process.env.VERSIONS_JSON); } catch { v = []; }
if (!Array.isArray(v)) v = [v];
const parse = s => s.split(".").map(n => parseInt(n, 10));
const gt = (a, b) => {
const [A, B] = [parse(a), parse(b)];
for (let i = 0; i < 3; i++) if (A[i] !== B[i]) return A[i] > B[i];
return false;
};
const bases = new Set();
for (const s of v) {
const m = /^(\d+\.\d+\.\d+)-rc\.\d+$/.exec(s);
if (m && gt(m[1], latest)) bases.add(m[1]);
}
if (!bases.size) { process.stdout.write(""); process.exit(0); }
const sorted = [...bases].sort((a, b) => gt(a, b) ? 1 : -1);
process.stdout.write(sorted[sorted.length - 1]);
NODESCRIPT
ACTIVE_BASE="$(LATEST="$CURRENT_LATEST_CLEAN" VERSIONS_JSON="$VERSIONS_JSON" node /tmp/active_base.mjs)"
if [ -n "$ACTIVE_BASE" ]; then
BASE="$ACTIVE_BASE"
echo "Continuing active rc cycle → BASE=$BASE"
else
BASE="$(npx --yes -p semver@7 semver -i patch "$CURRENT_LATEST_CLEAN")"
echo "No active rc cycle → patch bump from latest → BASE=$BASE"
fi
fi
# 4. Counter: 1 + max existing N for `${BASE}-rc.*`, else 1.
cat > /tmp/next_rc.mjs <<'NODESCRIPT'
const base = process.env.BASE;
const prefix = base + "-rc.";
let v;
try { v = JSON.parse(process.env.VERSIONS_JSON); } catch { v = []; }
if (!Array.isArray(v)) v = [v];
const ns = v
.filter(s => typeof s === "string" && s.startsWith(prefix))
.map(s => parseInt(s.slice(prefix.length), 10))
.filter(n => Number.isInteger(n) && n >= 0);
process.stdout.write(String(ns.length ? Math.max(...ns) + 1 : 1));
NODESCRIPT
NEXT_N="$(BASE="$BASE" VERSIONS_JSON="$VERSIONS_JSON" node /tmp/next_rc.mjs)"
RC_VERSION="${BASE}-rc.${NEXT_N}"
echo "Computed rc: $RC_VERSION"
# 5. Defensive: if the exact version already exists on the registry
# (e.g., race with another run), abort before re-publishing.
# Same E404-only pattern used above — a transient network
# failure must fail loudly, not pretend the version is missing.
NPM_STDERR_EXISTS="$(mktemp)"
if npm view "$PKG_NAME@$RC_VERSION" version 2>"$NPM_STDERR_EXISTS" >/dev/null; then
rm -f "$NPM_STDERR_EXISTS"
echo "::error::Version $RC_VERSION already exists on npm — aborting."
exit 1
else
if grep -qiE 'E404|not found' "$NPM_STDERR_EXISTS"; then
rm -f "$NPM_STDERR_EXISTS"
# Version doesn't exist — safe to proceed.
else
echo "::error::npm registry unreachable for existence check:" >&2
cat "$NPM_STDERR_EXISTS" >&2
rm -f "$NPM_STDERR_EXISTS"
exit 1
fi
fi
echo "base=$BASE" >> "$GITHUB_OUTPUT"
echo "rc_n=$NEXT_N" >> "$GITHUB_OUTPUT"
echo "rc_version=$RC_VERSION" >> "$GITHUB_OUTPUT"
- name: Apply rc version in-CI
shell: bash
working-directory: gitnexus
run: |
set -euo pipefail
npm version "${{ steps.version.outputs.rc_version }}" \
--no-git-tag-version --allow-same-version
- name: Build gitnexus
run: npm run build
working-directory: gitnexus
- name: Dry-run publish
run: npm publish --dry-run --tag rc
working-directory: gitnexus
# ── Acquire the "rc lock" BEFORE publishing (fixes idempotency) ─────
# We create two tags and push them atomically:
# v<RC_VERSION> → annotated tag on a detached release commit
# whose tree contains the rewritten package.json
# (so the tag's source matches the npm tarball)
# rc/<HEAD_SHA> → lightweight tag on HEAD; the guard's dedup key
# If this push fails, nothing is published — safe.
# If this push succeeds but npm publish fails, the marker stays on
# the remote and blocks retries until an operator manually cleans up.
- name: Create and push rc tags
id: reltag
shell: bash
working-directory: gitnexus
env:
RC_VERSION: ${{ steps.version.outputs.rc_version }}
HEAD_SHA: ${{ needs.guard.outputs.head_sha }}
run: |
set -euo pipefail
VTAG="v${RC_VERSION}"
MARKER="rc/${HEAD_SHA}"
git config user.name 'github-actions[bot]'
git config user.email '41898282+github-actions[bot]@users.noreply.github.com'
# Detached release commit with the version bump — keeps `main`
# pristine but gives the v-tag a tree that matches the published
# package contents exactly (fixes release-integrity gap).
git add package.json package-lock.json 2>/dev/null || git add package.json
git commit -m "release: ${VTAG}" --allow-empty
RELEASE_SHA="$(git rev-parse HEAD)"
echo "Detached release commit: $RELEASE_SHA"
# Annotated release tag on the release commit.
git tag -a "$VTAG" "$RELEASE_SHA" -m "$VTAG"
# Lightweight marker on the user-visible HEAD for the guard.
git tag "$MARKER" "$HEAD_SHA"
# Atomic push of both refs. If either would clobber an existing
# remote ref, the push fails and we stop before npm publish.
git push --atomic origin "refs/tags/$VTAG" "refs/tags/$MARKER"
echo "vtag=$VTAG" >> "$GITHUB_OUTPUT"
echo "marker=$MARKER" >> "$GITHUB_OUTPUT"
echo "release_sha=$RELEASE_SHA" >> "$GITHUB_OUTPUT"
- name: Publish to npm (rc dist-tag)
run: npm publish --provenance --access public --tag rc
working-directory: gitnexus
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
- name: Create GitHub prerelease
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v2
with:
tag_name: ${{ steps.reltag.outputs.vtag }}
name: Release Candidate ${{ steps.reltag.outputs.vtag }}
prerelease: true
make_latest: 'false'
generate_release_notes: true
body: |
Automated release candidate build from `main`.
**npm:** `npm install gitnexus@rc`
**Version:** `${{ steps.version.outputs.rc_version }}`
**Target base:** `${{ steps.version.outputs.base }}` (rc #${{ steps.version.outputs.rc_n }})
**Source commit (main):** ${{ needs.guard.outputs.head_sha }}
**Release commit (versioned tree):** ${{ steps.reltag.outputs.release_sha }}
Release candidates are pre-stable builds intended for early testing.
Stable releases remain on the `latest` dist-tag.
# ── Build & push RC Docker images ────────────────────────────────────
# Calls docker.yml as a reusable workflow so that the build, signing, and
# attestation logic stays in one place. The publish job exposes `vtag`
# (e.g. `v1.2.3-rc.1`) as an output so we can pass it as the tag input.
# RC images are signed with Cosign keyless signing; the OIDC identity
# will be `docker.yml@refs/heads/main` (the caller's ref) rather than a
# tag ref — see README.md § Docker for the correct verify command for RCs.
docker:
name: Build & Push RC Docker images
needs: [guard, publish]
if: needs.guard.outputs.should_run == 'true' && needs.publish.outputs.vtag != ''
uses: ./.github/workflows/docker.yml
permissions:
contents: read
packages: write
id-token: write
attestations: write
with:
tag: ${{ needs.publish.outputs.vtag }}
@@ -1,185 +0,0 @@
name: Tree-sitter Upgrade Readiness
# Monitors readiness for upgrading tree-sitter to 0.25.x. Tracks:
# 1. Peer-dep compatibility — can each grammar install cleanly with
# tree-sitter@0.25.0 without --legacy-peer-deps?
# 2. Vendored proto drift — has coder3101/tree-sitter-proto moved
# ahead of our vendored snapshot?
# See .github/scripts/check-tree-sitter-upgrade-readiness.py for the logic.
#
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
on:
schedule:
# Daily at 09:00 UTC. Matches Dependabot's daily cadence so drift
# and dep PRs surface together.
- cron: '0 9 * * *'
workflow_dispatch:
pull_request:
paths:
- '.github/scripts/check-tree-sitter-upgrade-readiness.py'
- '.github/workflows/tree-sitter-upgrade-readiness.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
jobs:
readiness:
name: Check upgrade readiness
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
# Needed to open/update the tracking issue on scheduled runs.
issues: write
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: ./.github/actions/setup-gitnexus
with:
build: 'false'
- name: Run upgrade readiness check
id: readiness
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set +e
python3 .github/scripts/check-tree-sitter-upgrade-readiness.py > drift-report.md
code=$?
set -e
echo "exit_code=$code" >> "$GITHUB_OUTPUT"
{
echo 'report<<DRIFT_EOF'
cat drift-report.md
echo 'DRIFT_EOF'
} >> "$GITHUB_OUTPUT"
echo "=== Report ==="
cat drift-report.md
# On PR runs, the script validates that it runs correctly. Blockers
# are informational — the scheduled run opens a tracking issue.
- name: Annotate PR with readiness status
if: github.event_name == 'pull_request' && steps.readiness.outputs.exit_code != '0'
run: |
echo "::warning::Tree-sitter 0.25 upgrade has blockers. See job output for the full readiness report."
- name: Upsert tracking issue on scheduled runs
if: >
github.event_name == 'schedule' &&
steps.readiness.outputs.exit_code != '0'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
REPORT: ${{ steps.readiness.outputs.report }}
with:
script: |
const title = 'Tree-sitter 0.25 upgrade readiness';
const report = process.env.REPORT;
const body = report + '\n\n' +
'<sub>Generated daily by `.github/workflows/tree-sitter-upgrade-readiness.yml`. ' +
'Closes automatically when all blockers are resolved.</sub>';
const { data: open } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
labels: 'tree-sitter-drift',
per_page: 10,
});
const existing = open.find(i => i.title === title);
if (existing) {
// Extract ready/total count for the changelog comment.
const readyMatch = report.match(/\*\*(\d+)\/(\d+)\*\* grammars ready/);
const blockerMatch = report.match(/\*\*(\d+) blocker/);
const ready = readyMatch ? readyMatch[1] : '?';
const total = readyMatch ? readyMatch[2] : '?';
const blockers = blockerMatch ? blockerMatch[1] : '?';
// Find grammars whose status changed by diffing the old and
// new table rows. Each row looks like:
// | `tree-sitter-foo` | ... | Ready |
// | `tree-sitter-foo` | ... | Blocking |
const parseRows = (md) => {
const map = {};
for (const m of md.matchAll(/\| `(tree-sitter-[^`]+)` \|.*?\| (\S+(?:\s\S+)*?) \|$/gm)) {
map[m[1]] = m[2].trim();
}
return map;
};
const oldRows = parseRows(existing.body || '');
const newRows = parseRows(report);
const changes = [];
for (const [name, newStatus] of Object.entries(newRows)) {
const oldStatus = oldRows[name];
if (oldStatus && oldStatus !== newStatus) {
changes.push(`\`${name}\`: ${oldStatus} → ${newStatus}`);
}
}
const today = new Date().toISOString().slice(0, 10);
let comment = `**${today}:** ${ready}/${total} ready. ${blockers} blocker(s) remaining.`;
if (changes.length > 0) {
comment += '\n\nChanges:\n' + changes.map(c => `- ${c}`).join('\n');
} else {
comment += ' No changes from previous run.';
}
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body: comment,
});
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body,
});
core.info(`Updated existing issue #${existing.number}`);
} else {
const { data: created } = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title,
body,
labels: ['tree-sitter-drift', 'dependencies'],
});
core.info(`Opened issue #${created.number}`);
}
- name: Close tracking issue on clean scheduled runs
if: >
github.event_name == 'schedule' &&
steps.readiness.outputs.exit_code == '0'
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const title = 'Tree-sitter 0.25 upgrade readiness';
const { data: open } = await github.rest.issues.listForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
state: 'open',
labels: 'tree-sitter-drift',
per_page: 10,
});
const existing = open.find(i => i.title === title);
if (existing) {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
body: 'All grammars are now compatible with tree-sitter@0.25. Upgrade is ready! Closing automatically.',
});
await github.rest.issues.update({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: existing.number,
state: 'closed',
});
core.info(`Closed issue #${existing.number}`);
}
+2 -4
View File
@@ -47,10 +47,8 @@ permissions:
issues: write
pull-requests: write
# Concurrency convention: see CONTRIBUTING.md → "GitHub Actions — Concurrency Convention".
# Single global slot — newest manual dispatch supersedes any in-flight run.
concurrency:
group: ${{ github.workflow }}
group: triage-sweep
cancel-in-progress: true
jobs:
@@ -76,7 +74,7 @@ jobs:
run: pip install -r .github/scripts/triage/requirements.txt
- name: Cache FastEmbed model weights
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
with:
path: ${{ github.workspace }}/.fastembed_cache
key: fastembed-bge-small-en-v1.5
+1 -13
View File
@@ -23,7 +23,6 @@ Thumbs.db
.env
.env.local
.env.*.local
docker/.env
# Logs
*.log
@@ -82,11 +81,6 @@ GitNexus.sln
# Git worktrees
.worktrees/
# Vendored tree-sitter grammar build artifacts (created at install time,
# never committed). See docs/plans/2026-04-15-002-fix-tree-sitter-proto-vendor-deps-plan.md
gitnexus/vendor/**/build/
gitnexus/vendor/**/node_modules/
/github/scripts/triage/__pycache__/
.claude-flow/
@@ -99,10 +93,4 @@ gitnexus/vendor/**/node_modules/
.history/
.swarm/
local_docs/
# Local agent scratch / review prompts (never commit)
.tmp/
.agents/
.swarm/
+16 -9
View File
@@ -1,26 +1,33 @@
#!/usr/bin/env bash
# Pre-commit hook: format staged files + typecheck.
# Tests run in CI (ci-tests.yml), not here.
# Pre-commit hook (husky): typecheck + unit tests for both packages.
# Mirrors CI checks from ci-quality.yml and ci-tests.yml.
# Skip with: git commit --no-verify
#
# CI coverage:
# quality / typecheck → tsc --noEmit in gitnexus/
# quality / typecheck-web → tsc -b --noEmit in gitnexus-web/
# tests / ubuntu+coverage → vitest run in gitnexus/ (all projects)
# e2e / chromium → playwright (requires servers — skipped)
ROOT="$(git rev-parse --show-toplevel)"
# 1. Format staged files with prettier via lint-staged
echo "pre-commit: formatting staged files..."
"$ROOT/node_modules/.bin/lint-staged" || exit 1
# 2. Typecheck changed packages
WEB_CHANGED=$(git diff --cached --name-only -- 'gitnexus-web/' | head -1)
CLI_CHANGED=$(git diff --cached --name-only -- 'gitnexus/' | head -1)
if [ -n "$WEB_CHANGED" ]; then
echo "pre-commit: typechecking gitnexus-web (tsc -b)..."
cd "$ROOT/gitnexus-web" && ./node_modules/.bin/tsc -b --noEmit || exit 1
cd "$ROOT/gitnexus-web" && npx tsc -b --noEmit
echo "pre-commit: running gitnexus-web unit tests..."
npx vitest run --reporter=dot
fi
if [ -n "$CLI_CHANGED" ]; then
echo "pre-commit: typechecking gitnexus..."
cd "$ROOT/gitnexus" && ./node_modules/.bin/tsc --noEmit || exit 1
cd "$ROOT/gitnexus" && npx tsc --noEmit
echo "pre-commit: running gitnexus unit tests (default project)..."
npx vitest run --project default --reporter=dot
fi
echo "pre-commit: all checks passed"
-16
View File
@@ -1,16 +0,0 @@
dist/
coverage/
gitnexus/vendor/
gitnexus/test/fixtures/
gitnexus-web/playwright-report/
gitnexus-web/test-results/
*.d.ts
*.snap
*.wasm
*.md
.gitnexus/
.vercel/
.claude-flow/
.swarm/
assets/
repomix-output*
-10
View File
@@ -1,10 +0,0 @@
{
"semi": true,
"singleQuote": true,
"trailingComma": "all",
"printWidth": 100,
"tabWidth": 2,
"endOfLine": "lf",
"plugins": ["prettier-plugin-tailwindcss"],
"tailwindStylesheet": "./gitnexus-web/src/index.css"
}
+45 -152
View File
@@ -1,125 +1,54 @@
<!-- version: 1.4.0 -->
<!-- Last updated: 2026-04-16 -->
Last reviewed: 2026-04-16
**Project:** GitNexus · **Environment:** dev · **Maintainer:** repository maintainers (see GitHub)
## Scope
| Boundary | Rule |
|----------|------|
| **Reads** | `gitnexus/`, `gitnexus-web/`, `eval/`, plugin packages, `.github/`, `.gitnexus/`, docs. |
| **Writes** | Only paths required for the change; keep diffs minimal. Update lockfiles when deps change. |
| **Executes** | `npm`, `npx`, `node` under `gitnexus/` and `gitnexus-web/`; `uv run` for Python under `eval/`; documented CI/dev workflows. |
| **Off-limits** | Real `.env` / secrets, production credentials, unrelated repos, destructive git ops without confirmation. |
## Model Configuration
- **Primary:** Use a named model (e.g. Claude Sonnet 4.x). Avoid `Auto` or unversioned `latest` when reproducibility matters.
- **Notes:** The GitNexus CLI indexer does not call an LLM.
## Execution Sequence (complex tasks)
For multi-step work, state up front:
1. Which rules in this file and **[GUARDRAILS.md](GUARDRAILS.md)** apply (and any relevant Signs).
2. Current **Scope** boundaries.
3. Which **validation commands** you will run (`cd gitnexus && npm test`, `npx tsc --noEmit`).
On long threads, *"Remember: apply all AGENTS.md rules"* re-weights these instructions against context dilution.
## Claude Code hooks
**PreToolUse** hooks can block tools (e.g. `git_commit`) until checks pass. Adapt to this repo: `cd gitnexus && npm test` before commit.
## Context budget
Commands and gotchas live under **Repo reference** below and in **[CONTRIBUTING.md](CONTRIBUTING.md)**. If always-on rules grow, split into **`.cursor/rules/*.mdc`** (globs). **Cursor:** project-wide rules in `.cursor/index.mdc`. **Claude Code:** load `STANDARDS.md` only when needed.
## Reference docs
- **[ARCHITECTURE.md](ARCHITECTURE.md)**, **[CONTRIBUTING.md](CONTRIBUTING.md)**, **[GUARDRAILS.md](GUARDRAILS.md)**
- **Call-resolution DAG:** See ARCHITECTURE.md § Call-Resolution DAG. Typed 6-stage DAG inside the `parse` phase; language-specific behavior behind `inferImplicitReceiver` / `selectDispatch` hooks on `LanguageProvider`. Shared code in `gitnexus/src/core/ingestion/` must not name languages. Types: `gitnexus/src/core/ingestion/call-types.ts`.
- **Cursor:** `.cursor/index.mdc` (always-on); `.cursor/rules/*.mdc` (glob-scoped). Legacy `.cursorrules` deprecated.
- **GitNexus:** skills in `.claude/skills/gitnexus/`; MCP rules in `gitnexus:start` block below.
## Changelog
| Date | Version | Change |
|------|---------|--------|
| 2026-04-19 | 1.5.0 | Cross-repo impact (#794): `impact`/`query`/`context` accept `repo: "@<group>"` + `service`. Removed `group_query`/`group_contracts`/`group_status` MCP tools; added `gitnexus://group/{name}/contracts` and `gitnexus://group/{name}/status` resources. |
| 2026-04-16 | 1.4.0 | Fixed: web UI description, pre-commit behavior, MCP tools (7->16), added gitnexus-shared, removed stale vite-plugin-wasm gotcha. |
| 2026-04-13 | 1.3.0 | Updated GitNexus index stats after DAG refactor. |
| 2026-03-24 | 1.2.0 | Fixed gitnexus:start block duplication. |
| 2026-03-23 | 1.1.0 | Updated agent instructions, references, Cursor layout. |
| 2026-03-22 | 1.0.0 | Initial structured header and changelog. |
---
<!-- gitnexus:start -->
# GitNexus — Code Intelligence
Indexed as **GitNexus** (4325 symbols, 10556 relationships, 300 execution flows). Use MCP tools to understand code, assess impact, and navigate safely.
This project is indexed by GitNexus as **GitNexus** (2487 symbols, 6056 relationships, 188 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
> If any tool warns the index is stale, run `npx gitnexus analyze` first.
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
## Always Do
- **MUST run impact analysis before editing any symbol.** `gitnexus_impact({target: "symbolName", direction: "upstream"})` — report blast radius to the user.
- **MUST run `gitnexus_detect_changes()` before committing** — verify only expected symbols and flows are affected.
- **MUST warn the user** if impact returns HIGH or CRITICAL risk.
- Explore unfamiliar code with `gitnexus_query({query: "concept"})` (process-grouped, ranked) instead of grepping.
- Full context on a symbol: `gitnexus_context({name: "symbolName"})`.
- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`.
## When Debugging
1. `gitnexus_query({query: "<error or symptom>"})` — find related execution flows
2. `gitnexus_context({name: "<suspect function>"})` — callers, callees, process participation
3. `READ gitnexus://repo/GitNexus/process/{processName}` — trace flow step by step
4. Regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})`
1. `gitnexus_query({query: "<error or symptom>"})` — find execution flows related to the issue
2. `gitnexus_context({name: "<suspect function>"})` — see all callers, callees, and process participation
3. `READ gitnexus://repo/GitNexus/process/{processName}` — trace the full execution flow step by step
4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed
## When Refactoring
- **Rename:** `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Graph edits are safe; text_search edits need manual review.
- **Extract/Split:** `gitnexus_context` (incoming/outgoing refs) then `gitnexus_impact` (upstream callers) before moving code.
- **After any refactor:** `gitnexus_detect_changes({scope: "all"})` to verify scope.
- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`.
- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code.
- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed.
## Never Do
- Edit a symbol without running `gitnexus_impact` first.
- Ignore HIGH/CRITICAL risk warnings.
- Rename with find-and-replace — use `gitnexus_rename`.
- Commit without `gitnexus_detect_changes()`.
- Add language-specific behavior to shared ingestion code (`gitnexus/src/core/ingestion/`) — use a `LanguageProvider` hook. Seeing `provider.mroStrategy === 'xxx'` or an import from `languages/xxx.ts` in shared code means stop and add a hook.
- NEVER edit a function, class, or method without first running `gitnexus_impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph.
- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope.
## Tools Quick Reference
| Tool | When to use | Example |
| Tool | When to use | Command |
|------|-------------|---------|
| `list_repos` | Discover indexed repos | `gitnexus_list_repos({})` |
| `query` | Find code by concept | `gitnexus_query({query: "auth validation"})` |
| `context` | 360-degree view of one symbol | `gitnexus_context({name: "validateUser"})` |
| `impact` | Blast radius before editing | `gitnexus_impact({target: "X", direction: "upstream"})` |
| `detect_changes` | Pre-commit scope check | `gitnexus_detect_changes({scope: "staged"})` |
| `rename` | Safe multi-file rename | `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` |
| `cypher` | Custom graph queries | `gitnexus_cypher({query: "MATCH ..."})` |
| `api_impact` | Pre-change API route impact | `gitnexus_api_impact({route: "/api/users", method: "GET"})` |
| `route_map` | Route → handler → consumer map | `gitnexus_route_map({})` |
| `tool_map` | MCP/RPC tool definitions | `gitnexus_tool_map({})` |
| `shape_check` | Response shape vs consumer access | `gitnexus_shape_check({route: "/api/users"})` |
| `group_list` | List repo groups | `gitnexus_group_list({})` |
| `group_sync` | Rebuild group Contract Registry | `gitnexus_group_sync({name: "myGroup"})` |
| `query` (group mode) | Cross-repo search in a group (RRF-merged) | `gitnexus_query({repo: "@myGroup", query: "auth"})` |
| `context` (group mode) | 360° view across all member repos | `gitnexus_context({repo: "@myGroup", name: "validateUser"})` |
| `impact` (group mode) | Cross-repo blast radius via Contract Bridge | `gitnexus_impact({repo: "@myGroup", target: "X", direction: "upstream"})` |
> Group mode: pass `repo: "@<groupName>"` to fan out across all member repos, or `repo: "@<groupName>/<memberPath>"` to target a single member (path keys from `group.yaml`). Optional `service: "<monorepo/path>"` filters by service root. Group-level state (contracts, staleness) lives in the resources table below — there are **no** `group_query` / `group_context` / `group_impact` / `group_contracts` / `group_status` MCP tools.
## Impact Risk Levels
| Depth | Meaning | Action |
|-------|---------|--------|
| d=1 | WILL BREAK — direct callers/importers | MUST update |
| d=1 | WILL BREAK — direct callers/importers | MUST update these |
| d=2 | LIKELY AFFECTED — indirect deps | Should test |
| d=3 | MAY NEED TESTING — transitive | Test if critical path |
@@ -127,82 +56,46 @@ Indexed as **GitNexus** (4325 symbols, 10556 relationships, 300 execution flows)
| Resource | Use for |
|----------|---------|
| `gitnexus://repo/GitNexus/context` | Codebase overview, index freshness |
| `gitnexus://repo/GitNexus/context` | Codebase overview, check index freshness |
| `gitnexus://repo/GitNexus/clusters` | All functional areas |
| `gitnexus://repo/GitNexus/processes` | All execution flows |
| `gitnexus://repo/GitNexus/process/{name}` | Step-by-step execution trace |
| `gitnexus://group/{name}/contracts` | Group Contract Registry (provider/consumer rows + cross-links) |
| `gitnexus://group/{name}/status` | Per-member index + Contract Registry staleness report |
## Self-Check Before Finishing
Before completing any code modification task, verify:
1. `gitnexus_impact` was run for all modified symbols
2. No HIGH/CRITICAL warnings were ignored
3. `gitnexus_detect_changes()` confirms expected scope
4. All d=1 dependents were updated
2. No HIGH/CRITICAL risk warnings were ignored
3. `gitnexus_detect_changes()` confirms changes match expected scope
4. All d=1 (WILL BREAK) dependents were updated
## Keeping the Index Fresh
After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it:
```bash
npx gitnexus analyze # basic refresh
npx gitnexus analyze --embeddings # preserve embeddings
npx gitnexus analyze
```
Check `.gitnexus/meta.json` `stats.embeddings` (0 = none). Running without `--embeddings` deletes existing vectors.
If the index previously included embeddings, preserve them by adding `--embeddings`:
> Claude Code: PostToolUse hook handles this after `git commit` and `git merge`.
```bash
npx gitnexus analyze --embeddings
```
## CLI Skills
To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.**
| Task | Skill file |
|------|-----------|
| Architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` |
| Blast radius / "What breaks?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Debugging / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` |
| Refactoring | `.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md` |
| Tools/resources/schema reference | `.claude/skills/gitnexus/gitnexus-guide/SKILL.md` |
| CLI commands (index, status, clean, wiki) | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md` |
> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`.
## CLI
| Task | Read this skill file |
|------|---------------------|
| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` |
| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` |
| Rename / extract / split / refactor | `.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md` |
| Tools, resources, schema reference | `.claude/skills/gitnexus/gitnexus-guide/SKILL.md` |
| Index, status, clean, wiki CLI commands | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md` |
<!-- gitnexus:end -->
## Repo reference
### Packages
| Package | Path | Purpose |
|---------|------|---------|
| **CLI/Core** | `gitnexus/` | TypeScript CLI, indexing pipeline, MCP server. Published to npm. |
| **Web UI** | `gitnexus-web/` | React/Vite thin client. All queries via `gitnexus serve` HTTP API. |
| **Shared** | `gitnexus-shared/` | Shared TypeScript types and constants. |
| Claude Plugin | `gitnexus-claude-plugin/` | Static config for Claude marketplace. |
| Cursor Integration | `gitnexus-cursor-integration/` | Static config for Cursor editor. |
| Eval | `eval/` | Python evaluation harness (Docker + LLM API keys). |
### Running services
```bash
cd gitnexus && npm run dev # CLI: tsx watch mode
cd gitnexus-web && npm run dev # Web UI: Vite on port 5173
npx gitnexus serve # HTTP API on port 4747 (from any indexed repo)
```
### Testing
**CLI / Core (`gitnexus/`)**
- `npm test` — full vitest suite (~2000 tests)
- `npm run test:unit` — unit tests only
- `npm run test:integration` — integration (~1850 tests). LadybugDB file-locking tests may fail in containers (known env issue).
- `npx tsc --noEmit` — typecheck
**Web UI (`gitnexus-web/`)**
- `npm test` — vitest (~200 tests)
- `npm run test:e2e` — Playwright (7 spec files; requires `gitnexus serve` + `npm run dev`)
- `npx tsc -b --noEmit` — typecheck
**Pre-commit hook** (`.husky/pre-commit`): formatting (prettier via lint-staged) + typecheck for staged packages. Tests do **not** run in pre-commit — CI only.
### Gotchas
- `npm install` in `gitnexus/` triggers `prepare` (builds via `tsc`) and `postinstall` (patches tree-sitter-swift, builds tree-sitter-proto). Native bindings need `python3`, `make`, `g++`.
- `tree-sitter-kotlin` and `tree-sitter-swift` are optional — install warnings expected.
- ESLint configured via `eslint.config.mjs` (TS, React Hooks, unused-imports). No `npm run lint` script; use `npx eslint .`. Prettier runs via lint-staged. CI checks both in `ci-quality.yml`.
-370
View File
@@ -1,370 +0,0 @@
# Architecture — GitNexus
Monorepo: **CLI/MCP** (`gitnexus/`) + **browser UI** (`gitnexus-web/`).
## Repository layout
| Path | Role |
|------|------|
| `gitnexus/` | npm package `gitnexus`: CLI, MCP server (stdio), HTTP API, ingestion pipeline, LadybugDB graph, embeddings. |
| `gitnexus-web/` | Vite + React thin client: graph explorer + AI chat. All queries via `gitnexus serve` HTTP API. |
| `gitnexus-shared/` | Shared TypeScript types and constants (consumed by CLI and Web). |
| `.claude/`, `gitnexus-claude-plugin/`, `gitnexus-cursor-integration/` | Agent skills and plugin metadata. |
| `eval/` | Evaluation harnesses for benchmarking tool usage. |
| `.github/` | CI workflows + composite actions (`setup-gitnexus/`, `setup-gitnexus-web/`). |
## End-to-end flow: index → graph → tools
1. **Ingestion** — `analyze.ts` → `runFullAnalysis` (`run-analyze.ts`) → `runPipelineFromRepo` (`pipeline.ts`). DAG of 12 phases builds a `KnowledgeGraph` in memory, then loads into LadybugDB under `.gitnexus/`. Repo registered in `~/.gitnexus/registry.json` for MCP discovery.
2. **Persistence** — `repo-manager.ts` (paths, registry, KuzuDB cleanup). `lbug-adapter.ts` (graph load, queries, embedding batches).
3. **Query layer** — three interfaces to the same backend:
- **MCP (stdio):** `mcp.ts` → `LocalBackend` → tools (`tools.ts`) + resources (`resources.ts`)
- **HTTP bridge:** `serve.ts` → Express (`api.ts`, `mcp-http.ts`) for web UI
- **CLI direct:** `gitnexus query|context|impact|cypher` in `tool.ts`
4. **Staleness** — `staleness.ts` compares indexed `lastCommit` to `HEAD`, surfaces hints.
## MCP tools
| Tool | Purpose |
|------|---------|
| `list_repos` | Discover indexed repos |
| `query` | Hybrid BM25 + vector search over the graph |
| `cypher` | Ad hoc Cypher against the schema |
| `context` | Callers, callees, processes for one symbol |
| `impact` | Blast radius (upstream/downstream) with risk summary |
| `detect_changes` | Map git diffs to affected symbols and processes |
| `rename` | Graph-assisted multi-file rename with `dry_run` preview |
| `api_impact` | Pre-change impact report for an API route handler |
| `route_map` | API route → handler → consumer mappings |
| `tool_map` | MCP/RPC tool definitions and handlers |
| `shape_check` | Response shape vs consumer property access mismatches |
| `group_list` | List repo groups or details for one group |
| `group_sync` | Rebuild group Contract Registry (`contracts.json`) and bridge graph |
`query`, `context`, and `impact` are group-aware: pass `repo: "@<groupName>"` (or `"@<groupName>/<memberPath>"` to scope to one member) plus optional `service: "<monorepo/path>"`. Group-mode `query` merges per-repo results via Reciprocal Rank Fusion; group-mode `impact` runs the local walk in the chosen member and fans out across boundaries via the Contract Bridge (`gitnexus/src/core/group/cross-impact.ts`). The previously-planned `group_query`, `group_context`, `group_impact`, `group_contracts`, `group_status` MCP tools are intentionally not introduced — group-level state is exposed via resources instead:
| Resource URI | Purpose |
|--------------|---------|
| `gitnexus://group/{name}/contracts` | Contract Registry (provider/consumer rows + cross-links) |
| `gitnexus://group/{name}/status` | Per-member index + Contract Registry staleness |
## Where to change what
| Concern | Start in |
|---------|----------|
| CLI commands/flags | `src/cli/` (`index.ts`, per-command modules) |
| Parsing/graph construction | `src/core/ingestion/pipeline-phases/` + `pipeline.ts` |
| Graph schema/DB | `src/core/lbug/` (`schema.ts`, `lbug-adapter.ts`) |
| MCP tools/resources | `src/mcp/server.ts`, `tools.ts`, `resources.ts` |
| Cross-repo groups (sync, contracts, `@<group>` routing) | `src/core/group/` (`service.ts`, `cross-impact.ts`, `sync.ts`, `bridge-db.ts`) |
| Search ranking | `src/core/search/` (BM25, hybrid fusion) |
| Embeddings | `src/core/embeddings/` + `src/core/run-analyze.ts` |
| Wiki generation | `src/core/wiki/` |
| Language support | `src/core/ingestion/languages/` + `tree-sitter-queries.ts` + `gitnexus-shared/src/languages.ts` |
| Import resolution | `src/core/ingestion/import-processor.ts` + `import-resolvers/configs/` + `model/resolution-context.ts` |
| Call resolution/MRO | `src/core/ingestion/call-processor.ts` + `model/resolve.ts` |
| Type extraction | `src/core/ingestion/type-extractors/` |
| Worker pool | `src/core/ingestion/workers/` |
| Web UI | `gitnexus-web/src/` |
| CI | `.github/workflows/*.yml`, `.github/actions/` |
> Paths above are relative to `gitnexus/` unless they start with `gitnexus-web/` or `.github/`.
---
## Pipeline Phase DAG
12 phases defined in `gitnexus/src/core/ingestion/pipeline-phases/`, each with explicit `deps` and typed output.
```
scan → structure → [markdown, cobol] → parse → [routes, tools, orm]
→ crossFile → mro → communities → processes
```
| Phase | File | Deps | Output |
|-------|------|------|--------|
| `scan` | `scan.ts` | (root) | File paths + sizes |
| `structure` | `structure.ts` | `scan` | File/Folder nodes, CONTAINS edges, `allPathSet` |
| `markdown` | `markdown.ts` | `structure` | Section nodes, cross-link edges from .md/.mdx |
| `cobol` | `cobol.ts` | `structure` | COBOL program/paragraph/section nodes (regex, no tree-sitter) |
| `parse` | `parse.ts` + `parse-impl.ts` | `structure`, `markdown`, `cobol` | Symbol nodes, IMPORTS/CALLS/EXTENDS edges, extracted routes/tools/ORM queries |
| `routes` | `routes.ts` | `parse` | Route nodes + HANDLES_ROUTE edges (Next.js, Expo, PHP, decorators) |
| `tools` | `tools.ts` | `parse` | Tool nodes + HANDLES_TOOL edges |
| `orm` | `orm.ts` | `parse` | QUERIES edges (Prisma, Supabase) |
| `crossFile` | `cross-file.ts` + `cross-file-impl.ts` | `parse`, `routes`, `tools`, `orm` | Cross-file type propagation in topological import order |
| `mro` | `mro.ts` | `crossFile`, `structure` | METHOD_OVERRIDES + METHOD_IMPLEMENTS edges |
| `communities` | `communities.ts` | `mro`, `structure` | Community nodes + MEMBER_OF edges (Leiden algorithm) |
| `processes` | `processes.ts` | `communities`, `routes`, `tools`, `structure` | Process nodes + STEP_IN_PROCESS edges |
**Non-phase files in the same directory:** `parse-impl.ts`, `cross-file-impl.ts` (implementation), `wildcard-synthesis.ts` (whole-module import expansion), `orm-extraction.ts` (sequential ORM fallback), `types.ts`, `runner.ts`, `index.ts`.
### DAG runner
`runner.ts` — static phase graph, no plugins, compile-time type safety.
1. **Validation** — Kahn's topological sort. Rejects on: duplicate names, missing deps, cycles (DFS traces the concrete cycle path, e.g., `A -> B -> C -> A`, plus count of transitively blocked dependents).
2. **Execution** — sequential in topological order. Each phase receives:
- `ctx: PipelineContext` — shared mutable `KnowledgeGraph`, `repoPath`, progress callback, options
- `deps: ReadonlyMap<string, PhaseResult>` — **declared deps only** (runner filters the results map to prevent hidden coupling)
3. **Error handling** — wraps phase errors with the phase name, emits terminal `error` progress event, swallows progress handler errors to preserve the original cause.
4. **Timing** — per-phase `durationMs` in `PhaseResult`, dev-mode console logging.
**Design patterns:**
- **Single graph accumulator** — all phases mutate the same `KnowledgeGraph` in `ctx`; the graph is the primary output.
- **Typed phase access** — `getPhaseOutput<T>(deps, 'name')` for type-safe upstream results.
- **Binding accumulator lifecycle** — created in `parse`, disposed by `crossFile` (in `finally`). No other phase should take ownership.
- **Skippable phases** — `skipGraphPhases` omits MRO/communities/processes (faster tests). `skipWorkers` forces sequential parsing.
### How to add a new phase
1. Create `pipeline-phases/my-phase.ts` with a `PipelinePhase<MyOutput>` (name, deps, execute)
2. Export from `pipeline-phases/index.ts`
3. Add to `buildPhaseList()` in `pipeline.ts`
```typescript
import type { PipelinePhase, PhaseResult } from './types.js';
import { getPhaseOutput } from './types.js';
import type { ParseOutput } from './parse.js';
export interface MyPhaseOutput { /* ... */ }
export const myPhase: PipelinePhase<MyPhaseOutput> = {
name: 'myPhase',
deps: ['parse'],
async execute(ctx, deps) {
const { allPaths } = getPhaseOutput<ParseOutput>(deps, 'parse');
// ... write to ctx.graph ...
return { /* typed output */ };
},
};
```
---
## Call-Resolution DAG
Typed 6-stage pipeline in `call-processor.ts` (inside the `parse` phase) that resolves method/function calls and emits CALLS edges. Language behavior plugs in at two `LanguageProvider` hook points (stages 3–4); shared code names no languages. Scope: call resolution only — import resolution, type extraction, heritage, and symbol-table population live in other phases.
### Stages
```
extract-call ──▶ classify-form ──▶ infer-receiver ──▶ select-dispatch ──▶ resolve-target ──▶ emit-edge
(1) (2) (3) [hook] (4) [hook] (5) (6)
```
| Stage | Produces | Location |
|-------|----------|----------|
| **extract-call** | `ExtractedCallSite` (name, form, receiver, argCount) | `call-extractors/` (per-language); runs in worker |
| **classify-form** | callForm (`free`/`member`/`constructor`) + arity | `call-analysis.ts` → `inferCallForm`; shared, runs in worker |
| **infer-receiver** | `ReceiverEnriched` (receiver type finalized) | `call-processor.ts`; shared default chain, then `inferImplicitReceiver` hook |
| **select-dispatch** | `DispatchDecision` (primary, fallback, ancestryView) | `selectDispatch` hook, falls back to shared default |
| **resolve-target** | `TieredCandidates` | `model/resolve.ts` → `lookupMethodByOwnerWithMRO` (MRO walk) |
| **emit-edge** | CALLS edge in graph | `call-processor.ts`; writes edge with confidence tier |
### Provider hooks
Both hooks are optional on `LanguageProvider`. Ruby is the only current implementer.
**`inferImplicitReceiver`** — called after shared infer-receiver defaults. Returns `ImplicitReceiverOverride | null`.
| | |
|---|---|
| Inputs | `calledName`, `callForm`, `receiverName`, `receiverTypeName`, `callNode` (AST), `filePath` |
| Non-null fields | `callForm`, `receiverName`, `receiverTypeName` (required); `receiverSource: 'implicit-self'` (fixed); `hint?` (opaque, passed to `selectDispatch`) |
| Null | Keep existing `ReceiverEnriched` state |
**`selectDispatch`** — called after infer-receiver (including hook). Returns `DispatchDecision | null`; null uses shared default (constructor → `primary:'constructor'`; typed receiver → `primary:'owner-scoped'`; else → `primary:'free'`).
| | |
|---|---|
| Inputs | `calledName`, `callForm`, `receiverName`, `receiverTypeName`, `receiverSource`, `hint` |
| Non-null fields | `primary: 'owner-scoped' \| 'free' \| 'constructor'`; `fallback?: 'free-arity-narrowed'`; `ancestryView?: 'instance' \| 'singleton'`; `hint?` |
**`DispatchDecision` field semantics:**
- `primary: 'owner-scoped'` — MRO walk from receiver's type; used when receiver type is known.
- `fallback: 'free-arity-narrowed'` — after owner-scoped miss, search free-call candidates by arity only (Ruby uses this for implicit-self calls that miss their owner's MRO).
- `ancestryView: 'singleton'` — walk singleton/class ancestry instead of instance ancestry (Ruby `def self.foo` bodies, so `extend`-ed methods are found).
### Adding language behavior
1. **Implicit receivers** — implement `inferImplicitReceiver`: return null if call already has a receiver; otherwise use `findEnclosingClassInfo` (`ast-helpers.ts`) to find the enclosing context, return `ImplicitReceiverOverride` with `receiverSource: 'implicit-self'`, and optionally set `hint` for `selectDispatch`.
2. **Custom dispatch** — implement `selectDispatch`: inspect `receiverSource` and `hint`, return `DispatchDecision` with `primary`, optional `fallback`, optional `ancestryView`; return null to keep shared defaults.
3. **MRO strategy** — confirm `mroStrategy` is `'first-wins'`, `'c3'`, `'ruby-mixin'`, or `'none'`; consumed by `lookupMethodByOwnerWithMRO`.
**Ruby example** (`languages/ruby.ts` + `utils/ruby-self-call.ts`): `inferImplicitReceiver` rewrites bare-identifier calls to `self.method` and sets `hint` to `'instance'`/`'singleton'`; `selectDispatch` uses hint for `ancestryView` and adds `fallback: 'free-arity-narrowed'` for implicit-self calls.
### Code references
| Module | Purpose |
|--------|---------|
| `core/ingestion/call-types.ts` | DAG types: `ReceiverEnriched`, `DispatchDecision`, `ImplicitReceiverOverride` |
| `core/ingestion/language-provider.ts` | Hook signatures: `inferImplicitReceiver`, `selectDispatch` |
| `core/ingestion/call-processor.ts` | `processCalls`: stages 3–6 |
| `core/ingestion/model/resolve.ts` | `lookupMethodByOwnerWithMRO`: stage 5 MRO walk |
| `core/ingestion/languages/ruby.ts` | Both hooks + `mroStrategy: 'ruby-mixin'` |
| `core/ingestion/utils/ruby-self-call.ts` | Bare-call rewrite for `inferImplicitReceiver` |
---
## Language-agnostic graph feeding
16 languages → single unified graph. Four abstraction layers:
```
Unified Graph Schema (44 node types, 21 relationship types)
↑
Unified Resolution (3-tier name lookup + MRO walk)
↑
Language Providers (import semantics, type config, export checker, MRO strategy)
↑
Tree-Sitter Queries (per-language S-expressions, unified capture tags)
```
### Language providers
Each language implements `LanguageProvider` (`language-provider.ts`). Key fields:
| Field | Purpose |
|-------|---------|
| `id`, `extensions` | Language identity and file matching |
| `treeSitterQueries` | S-expression queries for AST extraction |
| `importSemantics` | `named` / `wildcard-leaf` / `wildcard-transitive` / `namespace` |
| `importResolver` | Language-specific path → file resolution |
| `exportChecker` | Public/exported symbol detection |
| `typeConfig` | Type annotation extraction rules |
| `mroStrategy` | `first-wins` / `c3` / `none` |
16 providers in `languages/index.ts` via `satisfies Record<SupportedLanguages, LanguageProvider>` — missing a language is a compile error.
### Unified capture tags
Per-language tree-sitter queries use different AST node names but produce the **same semantic capture tags**: `@definition.class`, `@definition.function`, `@call.name`, `@import.source`, `@heritage.extends`. Downstream extraction needs no language branching. Defined in `tree-sitter-queries.ts`.
### Import resolution
Per-language import resolution uses the **configs + factory** pattern (like call/method/class extractors). Each language declares an `ImportResolutionConfig` in `import-resolvers/configs/`, listing an ordered chain of `ImportResolverStrategy` functions. `createImportResolver()` (in `resolver-factory.ts`) composes them: first non-null result wins. Low-level helpers shared across strategies live alongside the configs in `import-resolvers/` (e.g. `go.ts`, `rust.ts`, `python.ts`).
Unified 3-tier algorithm (`model/resolution-context.ts`), per-language `importSemantics` controls which tier activates:
| Tier | Confidence | Mechanism |
|------|-----------|-----------|
| 1 — same-file | 0.95 | Symbol table for caller's file |
| 2 — import-scoped | 0.9 | `NamedImportMap` chains (named) or all files in `importMap` (wildcard) |
| 3 — global | 0.5 | O(1) index lookups: class, impl, callable. Fallback only |
| Import strategy | Languages | Behavior |
|----------------|-----------|----------|
| `named` | TS, JS, Java, C#, Rust, PHP, Kotlin | Only explicitly imported names visible |
| `wildcard-leaf` | Go, Ruby, Swift, Dart | Whole-package import, no transitive re-exports |
| `wildcard-transitive` | C, C++ | `#include` closure chains through re-exports |
| `namespace` | Python | Module aliases resolved at call site |
### Chunked parse-and-resolve
`parse` processes files in ~20 MB byte-budget chunks to bound memory. Per chunk:
1. Worker pool dispatches files (or sequential fallback via `skipWorkers`)
2. Each worker: detect language → load grammar → run queries → return unified `ParseWorkerResult`
3. Synthesize wildcard bindings (`wildcard-synthesis.ts`)
4. Resolve imports and heritage
5. Collect `BindingAccumulator` entries for cross-file propagation
Workers: `workers/worker-pool.ts`, `workers/parse-worker.ts`.
### Heritage and MRO
All languages emit unified `ExtractedHeritage` (child, parent, `EXTENDS`/`IMPLEMENTS`). MRO phase walks the heritage graph using per-language strategy:
- **`first-wins`** — Java, C#, C++, TS, Ruby, Go
- **`c3`** — Python (C3 linearization)
- **`none`** — single-inheritance languages
Unified walk: `lookupMethodByOwnerWithMRO()` in `model/resolve.ts`.
---
## Full analysis flow
`runFullAnalysis` in `run-analyze.ts` orchestrates everything around the pipeline:
```
CLI (analyze.ts) → runFullAnalysis(repoPath, options, callbacks)
1. Early exit if lastCommit == HEAD (unless --force) [0%]
2. Cache existing embeddings from prior index [0%]
3. runPipelineFromRepo() → KnowledgeGraph [0-60%]
4. Clean up legacy KuzuDB files [60%]
5. initLbug() → loadGraphToLbug() via CSV streaming [60-85%]
6. Create FTS indexes (File, Function, Class, Method...) [85-90%]
7. Restore cached embeddings (batch insert) [88%]
8. Generate new embeddings if --embeddings [90-98%]
9. Save metadata + register repo + update .gitignore [98-100%]
10. Generate AI context files (AGENTS.md, CLAUDE.md) [100%]
```
**Options:** `--force` (rebuild regardless), `--embeddings` (opt-in, skipped if >50k nodes), `--skipGit`, `--noStats`.
## Storage
```
<repo>/.gitnexus/
├── lbug # LadybugDB database
├── lbug.wal # Write-ahead log
├── lbug.lock # Single-writer lock
└── meta.json # lastCommit, indexedAt, stats
~/.gitnexus/
└── registry.json # Global repo registry (MCP discovery)
```
Managed by `repo-manager.ts`.
## LadybugDB schema
Defined in `lbug/schema.ts`. Separate node tables per type, single `CodeRelation` table.
**Node tables:** File, Folder, Function, Class, Interface, Method, Constructor, CodeElement, Struct, Enum, Macro, Typedef, Union, Namespace, Trait, Impl, TypeAlias, Const, Static, Property, Record, Delegate, Annotation, Template, Module, Community, Process, Route, Tool, Section, Embedding.
**Relation types** (`CodeRelation.type`): CONTAINS, DEFINES, CALLS, IMPORTS, EXTENDS, IMPLEMENTS, HAS_METHOD, HAS_PROPERTY, ACCESSES, METHOD_OVERRIDES, METHOD_IMPLEMENTS, MEMBER_OF, STEP_IN_PROCESS, HANDLES_ROUTE, FETCHES, HANDLES_TOOL, ENTRY_POINT_OF.
## Embeddings and search
**Embeddings** (`src/core/embeddings/`): Snowflake arctic-embed-xs (384D). Embeddable: File, Function, Class, Method, Interface. Incremental via SHA1 content hash. Separate `Embedding` table.
**Search** (`src/core/search/`): Hybrid BM25 + semantic vector, merged via Reciprocal Rank Fusion (K=60).
## Known limitations
### Overloaded method resolution
Node IDs use arity suffix (`#<paramCount>`): `Method:file:Class.method#1` vs `#2`.
**Same-arity disambiguation:** type-hash suffix `~type1,type2` when collision detected and type annotations present. Languages without types (Python, Ruby, JS) use arity-only. TS/JS overload signatures excluded (collapse to implementation body). See #651.
**C++ const-qualified:** `$const` suffix after type-hash when non-const collision exists: `Method:file:Container.begin#0$const`.
**Generic/template types:** type-hash uses `rawType` (full AST text including generics): `~vector<int>` vs `~vector<std::string>`.
**ID stability:** collision-only tags mean IDs change when overloads are added. `save#1` becomes `save#1~int` when `save(String)` is added.
**Variadic matching:** confidence 0.7 when one side is variadic and the other has fixed count.
**METHOD_IMPLEMENTS confidence tiering:**
| Match quality | Confidence |
|---|---|
| Exact parameter types match | 1.0 |
| Arity match, types unavailable | 1.0 |
| Variadic vs fixed | 0.7 |
| Insufficient info | 0.7 |
## Related docs
- [MIGRATION.md](MIGRATION.md) — breaking changes and migration guidance
- [RUNBOOK.md](RUNBOOK.md) — operational commands and recovery
- [GUARDRAILS.md](GUARDRAILS.md) — safety boundaries for humans and agents
- [TESTING.md](TESTING.md) — how to run tests
- `AGENTS.md` / `CLAUDE.md` — agent workflows and tool usage
-13
View File
@@ -10,19 +10,6 @@ All notable changes to GitNexus will be documented in this file.
- Added automatic cleanup of stale KuzuDB index files
- LadybugDB v0.15 requires explicit VECTOR extension loading for semantic search
## [1.5.3] - 2026-04-01
### Added
- **TypeScript/JavaScript MethodExtractor config** — shared extraction config covering abstract methods, visibility modifiers, async/override keywords, decorators, rest/optional/destructured parameters, and return types (#588) — @compound-ai
### Fixed
- **Azure OpenAI compatibility** — use `max_completion_tokens` instead of deprecated `max_tokens` (newer models reject `max_tokens`); skip `temperature` for Azure provider (some models reject non-default values) (#618)
- **Simplified Azure interactive setup** — 3 prompts (endpoint, deployment, key) instead of 7 (#618)
- **Wiki HTML viewer script injection** — escape `</script>` in embedded JSON so LLM-generated markdown no longer breaks the viewer (#618)
- Ensure import rewrites survive npm publish lifecycle
## [1.4.0] - 2026-03-13
### Added
+81 -34
View File
@@ -1,54 +1,101 @@
<!-- version: 1.3.0 -->
<!--
Metadata: version, last reviewed, scope, model policy, reference docs, changelog.
Last updated: 2026-03-22
-->
<!-- gitnexus:start -->
# GitNexus — Code Intelligence
Last reviewed: 2026-04-13
This project is indexed by GitNexus as **GitNexus** (2487 symbols, 6056 relationships, 188 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
**Project:** GitNexus · **Environment:** dev · **Maintainer:** repository maintainers (see GitHub)
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
Follow **AGENTS.md** for the canonical rules; this file adds Claude Code–specific deltas. Cursor-specific notes live only in `AGENTS.md`.
## Always Do
## Scope
- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`.
See the **Scope** table in [AGENTS.md](AGENTS.md) for read/write/execute/off-limits boundaries. Cursor-specific workflow notes also live only in AGENTS.md.
## When Debugging
## Model Configuration
1. `gitnexus_query({query: "<error or symptom>"})` — find execution flows related to the issue
2. `gitnexus_context({name: "<suspect function>"})` — see all callers, callees, and process participation
3. `READ gitnexus://repo/GitNexus/process/{processName}` — trace the full execution flow step by step
4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed
- **Primary:** Pin per **Claude Code** / Anthropic org policy (explicit model id). Do not rely on an unversioned `latest` alias for governed workflows.
- **Fallback:** As configured in Claude Code (organization default or user override).
- **Notes:** The GitNexus CLI analyzer does not call an LLM.
## When Refactoring
## Execution Sequence (complex tasks)
- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`.
- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code.
- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed.
Same discipline as [AGENTS.md](AGENTS.md): before large multi-step work, state which **AGENTS.md** / **GUARDRAILS.md** rules apply, current **Scope**, and planned validation commands (`npm test`, `tsc`, etc.). When pausing, summarize progress in the chat or a **local** scratch file (do not add `HANDOFF.md` to the repo), then `/clear` and resume with that summary.
## Never Do
## Claude Code hooks
- NEVER edit a function, class, or method without first running `gitnexus_impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph.
- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope.
Prefer **PreToolUse** hooks for hard gates (e.g. tests before `git_commit`). Adapt hook commands to `gitnexus/` npm scripts.
## Tools Quick Reference
## Context budget
| Tool | When to use | Command |
|------|-------------|---------|
| `query` | Find code by concept | `gitnexus_query({query: "auth validation"})` |
| `context` | 360-degree view of one symbol | `gitnexus_context({name: "validateUser"})` |
| `impact` | Blast radius before editing | `gitnexus_impact({target: "X", direction: "upstream"})` |
| `detect_changes` | Pre-commit scope check | `gitnexus_detect_changes({scope: "staged"})` |
| `rename` | Safe multi-file rename | `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` |
| `cypher` | Custom graph queries | `gitnexus_cypher({query: "MATCH ..."})` |
If always-on instructions grow, load deep conventions via conditional reads (e.g. *“When writing new code, read STANDARDS.md”*) instead of pasting long blocks here. In Cursor, prefer `.cursor/index.mdc` plus optional `.cursor/rules/*.mdc` globs (see [AGENTS.md](AGENTS.md) § Context budget).
## Impact Risk Levels
## Reference Documentation
| Depth | Meaning | Action |
|-------|---------|--------|
| d=1 | WILL BREAK — direct callers/importers | MUST update these |
| d=2 | LIKELY AFFECTED — indirect deps | Should test |
| d=3 | MAY NEED TESTING — transitive | Test if critical path |
- **This repository:** [AGENTS.md](AGENTS.md) (Cursor + monorepo notes), [ARCHITECTURE.md](ARCHITECTURE.md), [CONTRIBUTING.md](CONTRIBUTING.md), [GUARDRAILS.md](GUARDRAILS.md).
- **Call-resolution DAG:** See ARCHITECTURE.md § Call-Resolution DAG. Shared pipeline code in `gitnexus/src/core/ingestion/` must not name languages — use `LanguageProvider` hooks instead (see AGENTS.md).
- **GitNexus:** `.claude/skills/gitnexus/`; MCP and indexed-repo rules live only in [AGENTS.md](AGENTS.md) (`gitnexus:start` … `gitnexus:end`). See **GitNexus rules** below.
## Resources
## Changelog
| Resource | Use for |
|----------|---------|
| `gitnexus://repo/GitNexus/context` | Codebase overview, check index freshness |
| `gitnexus://repo/GitNexus/clusters` | All functional areas |
| `gitnexus://repo/GitNexus/processes` | All execution flows |
| `gitnexus://repo/GitNexus/process/{name}` | Step-by-step execution trace |
| Date | Version | Change |
|------|---------|--------|
| 2026-04-13 | 1.3.0 | Updated GitNexus index stats after DAG refactor. |
| 2026-03-24 | 1.2.0 | Removed duplicated gitnexus:start block and scope table; replaced with pointers to AGENTS.md. |
| 2026-03-23 | 1.1.0 | Updated agent instructions to match AGENTS.md. |
| 2026-03-22 | 1.0.0 | Added structured header and changelog. |
## Self-Check Before Finishing
---
Before completing any code modification task, verify:
1. `gitnexus_impact` was run for all modified symbols
2. No HIGH/CRITICAL risk warnings were ignored
3. `gitnexus_detect_changes()` confirms changes match expected scope
4. All d=1 (WILL BREAK) dependents were updated
## GitNexus rules
## Keeping the Index Fresh
See the `<!-- gitnexus:start --> … <!-- gitnexus:end -->` block in **[AGENTS.md](AGENTS.md)** for the canonical MCP tools, impact analysis rules, and index instructions.
After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it:
```bash
npx gitnexus analyze
```
If the index previously included embeddings, preserve them by adding `--embeddings`:
```bash
npx gitnexus analyze --embeddings
```
To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.**
> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`.
## CLI
| Task | Read this skill file |
|------|---------------------|
| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` |
| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` |
| Rename / extract / split / refactor | `.claude/skills/gitnexus/gitnexus-refactoring/SKILL.md` |
| Tools, resources, schema reference | `.claude/skills/gitnexus/gitnexus-guide/SKILL.md` |
| Index, status, clean, wiki CLI commands | `.claude/skills/gitnexus/gitnexus-cli/SKILL.md` |
<!-- gitnexus:end -->
-173
View File
@@ -1,173 +0,0 @@
# Contributing to GitNexus
How to propose changes, run checks locally, and open pull requests.
## License
This project uses the [PolyForm Noncommercial License 1.0.0](https://polyformproject.org/licenses/noncommercial/1.0.0/). By contributing, you agree your contributions are licensed under the same terms unless stated otherwise.
## Where to discuss
- **Issues & feature ideas:** use [GitHub Issues](https://github.com/abhigyanpatwari/GitNexus/issues) for the upstream repo, or your fork’s tracker if you work from a fork.
- **Community:** see the Discord link in the root [README.md](README.md).
## Development setup
1. Clone the repository.
2. **CLI / MCP package:** `cd gitnexus && npm install && npm run build`
3. **Web UI (if needed):** `cd gitnexus-web && npm install`
4. Run tests as described in [TESTING.md](TESTING.md).
## Branch and pull requests
- Use short-lived branches off the default branch of the repo you are targeting.
- **PR titles MUST follow the conventional-commit format** — `pr-labeler.yml` enforces this on every PR and auto-applies the matching label so release notes group the change correctly.
- **PR description:** what changed, why, how to verify (commands), and any risk or rollback notes.
### Pull request titles
Format: `<type>[(scope)][!]: <subject>`
Allowed types and the release-notes section each one lands in (defined in `.github/release.yml`):
| Type | Label applied | Release-notes section |
|------|---------------|-----------------------|
| `feat` | `enhancement` | 🚀 Features |
| `fix` | `bug` | 🐛 Bug Fixes |
| `perf` | `performance` | 🏎️ Performance |
| `refactor` | `refactor` | 🔄 Refactoring |
| `test` | `test` | 🧪 Tests |
| `ci` | `ci` | 👷 CI/CD |
| `build` / `deps` | `dependencies` | 📦 Dependencies |
| `docs` | `documentation` | (grouped under Other Changes unless a Docs section is added) |
| `chore` / `revert` | `chore` | (excluded from release notes) |
Append `!` to the type (e.g. `feat(api)!: drop /v1 endpoint`) or include `BREAKING CHANGE:` in the PR body to flag a breaking change — the labeler then adds the `breaking` label and the 💥 Breaking Changes section is rendered first.
Examples:
```text
feat(web): add smart chat scroll
fix(extractors): resolve silent contract mis-resolution
perf: avoid O(n²) traversal in heritage walker
chore(deps): bump vitest to 3.0.0
ci: standardize workflow concurrency
```
Commits within a PR may use any style — only the **merged PR title** shows up in release notes, so that's the one the convention applies to.
## Before you open a PR
- [ ] Tests pass for the packages you touched (`gitnexus` and/or `gitnexus-web`).
- [ ] Typecheck passes: `npx tsc --noEmit` in `gitnexus/` and `npx tsc -b --noEmit` in `gitnexus-web/`.
- [ ] No secrets, tokens, or machine-specific paths committed.
- [ ] Documentation updated if behavior or public CLI/MCP contract changes.
- [ ] Pre-commit hook runs clean (`.husky/pre-commit` — formatting via lint-staged + typecheck for staged packages; tests run in CI only).
## Code review
Maintainers may request changes for correctness, tests, performance, or consistency with existing patterns. Keeping diffs focused makes review faster.
## GitHub Actions — Concurrency Convention
Every workflow under `.github/workflows/` MUST declare a top-level `concurrency:` block using this convention:
- **Group key** starts with `${{ github.workflow }}` so no two workflows can collide on the same group name. The discriminator that follows is chosen per event shape:
- Branch/tag scope: `${{ github.workflow }}-${{ github.ref }}`
- Per-PR scope (for `issue_comment`, `pull_request_review*`, `pull_request` meta events): `${{ github.workflow }}-${{ github.event.pull_request.number || github.event.issue.number }}`
- `workflow_run` scope (e.g. `ci-report.yml`): `${{ github.workflow }}-${{ github.event.workflow_run.pull_requests[0].number || format('{0}/{1}', github.event.workflow_run.head_repository.full_name, github.event.workflow_run.head_branch) }}` — the fork fallback must be stable across reruns (never `workflow_run.id`, which is per-run-unique and defeats serialization).
- Global single-slot (manual dispatch utilities): `${{ github.workflow }}`
- **Reusable workflows invoked via `workflow_call`:** do NOT use `${{ github.workflow }}` in the group key — in called-workflow context its evaluation is ambiguous and can resolve to the caller's name, which would deadlock against the caller's own group. Use a hardcoded literal prefix and a `github.event_name`-aware expression that falls through to `github.run_id` for reusable invocations (see `ci.yml` for the canonical form). Approved literal prefixes: `CI-` (`ci.yml`) and `docker-build-push-` (`docker.yml`). The `check-workflow-concurrency.py` validation script must be updated whenever a new approved literal prefix is added.
- **Merge queue (`merge_group`)**: when this event is added, use `${{ github.workflow }}-${{ github.event.merge_group.head_ref }}` with `cancel-in-progress: false` (every queue entry is a distinct ref; never cancel).
- **`cancel-in-progress` policy:**
| Event | `cancel-in-progress` | Why |
|-------|----------------------|-----|
| `pull_request` CI run | `true` | New push supersedes old run |
| `push` to `main` | `false` | Every main commit gets validated |
| Tag push (`v*` publish) | `false` | Never cancel mid-publish |
| `push` to `main` for release-candidate | `false` | Never cancel mid-RC publish |
| `workflow_dispatch` (release/publish) | `false` | Manual runs are intentional |
| `workflow_run` (sticky-comment reports) | `false` | Serialize, don't race |
| Per-PR bot workflows (`@claude`, review) | `false` | Serialize comments per PR |
| PR-meta re-checks (pr-description-check) | `true` | Cheap, latest wins |
| Single-slot utilities (triage sweep) | `true` | Latest dispatch supersedes |
- For workflows that serve multiple events at once (e.g. `ci.yml` handles `pull_request`, `push`, and `workflow_call`), make `cancel-in-progress` event-aware:
```yaml
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
```
- When adding a new workflow, copy the concurrency block from an existing workflow of the same event shape.
## AI-assisted contributions
If you use coding agents, follow project context files (e.g. `AGENTS.md`, `CLAUDE.md`) and avoid drive-by refactors unrelated to the issue. Prefer incremental, test-backed changes.
## Releases
Two publish workflows ship `gitnexus` to npm:
- **Stable** (`.github/workflows/publish.yml`) — triggered by pushing any `v*`
tag. Publishes to the `latest` dist-tag with a changelog-backed GitHub
release. Maintainers are expected to tag from `main` as a convention; the
workflow itself does not enforce branch reachability.
- **Release Candidate** (`.github/workflows/release-candidate.yml`) — runs on
every push to `main` (typically a merged PR) plus manual dispatch. Docs-only
changes are skipped via `paths-ignore`. Publishes to the `rc` dist-tag with
version `X.Y.Z-rc.N` and a GitHub prerelease, where:
- `X.Y.Z` is selected automatically. On push (and on dispatch with
`bump: auto`, the default) the workflow **continues the active rc cycle**:
if the registry already has `X.Y.Z-rc.*` versions with `X.Y.Z` > current
`latest`, it reuses the highest such base; otherwise it patch-bumps
from `latest`. Dispatching with `bump: patch|minor|major` **resets**
the cycle from `latest`.
- `N` is auto-incremented against existing `X.Y.Z-rc.*` entries on the
registry. First rc for a given base is `rc.1`.
- After the npm publish succeeds, the workflow calls `docker.yml` as a
reusable workflow to build and push the corresponding RC Docker images
(e.g. `ghcr.io/abhigyanpatwari/gitnexus:1.7.0-rc.1`). The images are
signed with Cosign; the OIDC identity is `docker.yml@refs/heads/main`
(the caller's ref — see README.md § Docker for the verify command).
Idempotency: the workflow pushes an `rc/<HEAD_SHA>` marker tag and a
`v<RC>` release tag **atomically, before** calling `npm publish`. The guard
refuses to re-run once the marker exists, so a post-publish failure will
not mint a duplicate rc for the same commit. The `v<RC>` tag points at a
detached release commit whose `package.json` matches the npm tarball
exactly (traceable releases). Recovery after a partial failure:
```bash
git push --delete origin rc/<HEAD_SHA> v<RC>
# then redispatch the workflow with force: true
```
**Docker-only partial failure:** if `publish` succeeds (npm tarball + tags
are live) but the `docker` job subsequently fails (e.g. GHCR flakiness),
the npm RC is already published and the `rc/<HEAD_SHA>` marker is in place.
Re-running `release-candidate.yml` with `force: true` will abort at the
"Version already exists on npm" guard. To recover without cutting a new RC:
```bash
# 1. Manually trigger only the docker workflow, passing the existing RC tag:
gh workflow run docker.yml --ref main -f tag=v<RC_VERSION>
# (requires a workflow_dispatch trigger on docker.yml — see note below)
```
Because `docker.yml` intentionally has no `workflow_dispatch` (images are
tag-driven by design), the practical recovery options are:
- Wait for the next commit on `main`, which will cut a new RC that includes
the Docker build.
- Manually run `docker build` + `docker push` locally and sign with Cosign
against the same digest.
- Delete `rc/<HEAD_SHA>` and `v<RC>` tags, then redispatch with `force:
true` to re-run the full RC pipeline (cuts a new RC number).
The rc workflow never moves `latest`. To verify after a change, inspect dist-tags:
```bash
npm view gitnexus dist-tags
```
-57
View File
@@ -1,57 +0,0 @@
ARG BUILDPLATFORM
ARG TARGETPLATFORM
# ── Builder ────────────────────────────────────────────────────────────
# Native modules (tree-sitter-*, onnxruntime-node, node-gyp builds for
# tree-sitter-proto / tree-sitter-swift) require python3 + a C/C++ toolchain.
FROM node:22-alpine AS builder
WORKDIR /app
# Toolchain for node-gyp / native builds.
RUN apk add --no-cache python3 make g++ git
# Build gitnexus-shared first — gitnexus depends on it as a workspace.
COPY gitnexus-shared/package.json gitnexus-shared/package-lock.json ./gitnexus-shared/
RUN npm ci --prefix gitnexus-shared
COPY gitnexus-shared ./gitnexus-shared
RUN npm run build --prefix gitnexus-shared
# Copy the full gitnexus package before installing — `npm ci` triggers
# `postinstall` (patches tree-sitter-swift, builds the vendored
# tree-sitter-proto) and `prepare` (compiles TypeScript via scripts/build.js),
# both of which need the source tree.
COPY gitnexus ./gitnexus
RUN npm ci --prefix gitnexus
# Drop dev dependencies for a smaller runtime layer.
RUN npm prune --omit=dev --prefix gitnexus
# ── Runtime ────────────────────────────────────────────────────────────
FROM node:22-alpine AS runtime
# curl for the healthcheck; git so `gitnexus` can clone repos at runtime.
RUN apk add --no-cache curl git
WORKDIR /app
# Pre-create the data directory and hand it to the unprivileged `node` user
# so the bind-mounted volume is writable without root.
RUN mkdir -p /data/gitnexus && chown -R node:node /data
COPY --from=builder --chown=node:node /app/gitnexus/dist ./gitnexus/dist
COPY --from=builder --chown=node:node /app/gitnexus/node_modules ./gitnexus/node_modules
COPY --from=builder --chown=node:node /app/gitnexus/package.json ./gitnexus/package.json
COPY --from=builder --chown=node:node /app/gitnexus/vendor ./gitnexus/vendor
USER node
# The web UI defaults to http://localhost:4747 — keep that contract.
ENV GITNEXUS_HOME=/data/gitnexus \
NODE_ENV=production \
PORT=4747
EXPOSE 4747
# Bind to 0.0.0.0 so the server is reachable from the host's mapped port.
CMD ["node", "gitnexus/dist/cli/index.js", "serve", "--host", "0.0.0.0", "--port", "4747"]
-37
View File
@@ -1,37 +0,0 @@
ARG BUILDPLATFORM
ARG TARGETPLATFORM
FROM --platform=$BUILDPLATFORM node:22-alpine AS builder
WORKDIR /app
COPY gitnexus-shared/package.json gitnexus-shared/package-lock.json ./gitnexus-shared/
RUN npm ci --prefix gitnexus-shared
COPY gitnexus-shared ./gitnexus-shared
RUN npm run build --prefix gitnexus-shared
COPY gitnexus/package.json ./gitnexus/
COPY gitnexus-web/package.json gitnexus-web/package-lock.json ./gitnexus-web/
RUN npm ci --prefix gitnexus-web
COPY gitnexus-web ./gitnexus-web
RUN npm run build --prefix gitnexus-web
FROM node:22-alpine AS runtime
RUN apk add --no-cache curl
WORKDIR /app
COPY --from=builder /app/gitnexus-web/dist ./dist
COPY docker-server.mjs ./docker-server.mjs
RUN chown -R node:node /app
USER node
EXPOSE 4173
CMD ["node", "docker-server.mjs"]
-85
View File
@@ -1,85 +0,0 @@
# Guardrails — GitNexus
Rules for **human contributors** and **AI agents**. Complements `AGENTS.md` (workflows) and `CONTRIBUTING.md` (PR process).
## Scope (least privilege)
- **Read:** Source, tests, docs, public config as needed.
- **Write:** Only files required for the fix or feature; no unrelated formatting or refactors.
- **Execute:** Tests, typecheck, documented CLI commands. No destructive commands on user data without approval.
- **Off-limits:** Other people's machines, production deployments you don't own, credentials you lack permission to use.
Maintainer may widen scope per task.
---
## Non-negotiables
1. **Never commit secrets** — API keys, tokens, real `.env` values, private URLs, session cookies. Use `.env.example` with placeholders.
2. **Never rename with find-and-replace** in GitNexus-indexed projects — use `rename` MCP tool with `dry_run: true` first, review `graph` vs `text_search` edits. No separate `gitnexus rename` CLI exists.
3. **Run impact analysis before editing shared symbols** — `impact` (upstream) for functions/classes/methods others call. Do not ignore HIGH/CRITICAL without maintainer sign-off.
4. **Run `detect_changes` before commit** — confirm diffs map to expected symbols/processes when the graph is available.
5. **Preserve embeddings** — if `.gitnexus/meta.json` shows embeddings, use `npx gitnexus analyze --embeddings`; plain `analyze` drops them.
---
## Signs (recurring failure patterns)
Format: **Trigger → Instruction → Reason**. Append new Signs when the same mistake repeats.
### Stale graph after edits
- **Trigger:** MCP warns index is behind `HEAD`, or search doesn't match latest commit.
- **Do:** `npx gitnexus analyze` (plus `--embeddings` if used).
- **Why:** Tools query LadybugDB from last analyze; git changes are invisible until re-indexed.
### Embeddings vanished after analyze
- **Trigger:** Semantic search quality drops; `stats.embeddings` in `meta.json` is 0 after refresh.
- **Do:** `npx gitnexus analyze --embeddings`, confirm `meta.json` reflects stored embeddings.
- **Why:** Embedding generation is opt-in; analyze without the flag does not preserve prior vectors.
### MCP lists no repos
- **Trigger:** MCP stderr says no indexed repos.
- **Do:** `npx gitnexus analyze` in the target repo; verify `npx gitnexus list` shows it.
- **Why:** MCP discovers repos via `~/.gitnexus/registry.json`, populated by analyze.
### Wrong repo in multi-repo setups
- **Trigger:** Query/impact results belong to another project.
- **Do:** Call `list_repos`, then pass `repo` on subsequent tools.
- **Why:** Default target is ambiguous when multiple repos are registered.
### LadybugDB lock / "database busy"
- **Trigger:** Errors opening `.gitnexus/lbug` while MCP and analyze both run.
- **Do:** Stop overlapping processes (one writer at a time). Retry analyze or restart MCP.
- **Why:** Embedded DB expects single-process ownership.
---
## Publishing & supply chain
- **npm:** Do not publish from unreviewed automation. Bump version intentionally; tag releases to match `package.json`.
- **Dependencies:** Minimal, auditable `package.json` changes; run tests and CI after lockfile updates.
- **License:** PolyForm Noncommercial 1.0.0 — do not relicense without maintainer approval.
---
## Escalation
Stop and ask a **human maintainer** when:
- Impact analysis shows HIGH/CRITICAL risk and the task still requires the change.
- You need to alter CI, release, or security-sensitive config.
- Requirements conflict (e.g. "speed up analyze" vs "must keep all embeddings on huge repo").
- You are unsure whether data loss is acceptable (`clean`, forced migrations, schema changes).
---
## Related docs
- [ARCHITECTURE.md](ARCHITECTURE.md) — components and data flow
- [RUNBOOK.md](RUNBOOK.md) — commands for recovery
- [CONTRIBUTING.md](CONTRIBUTING.md) — PR and commit expectations
-71
View File
@@ -1,71 +0,0 @@
# Migration Guide
## `impact` tool may now return `{ status: 'ambiguous' }` (PR #888, issue #470)
Before this change the `impact` MCP tool silently picked the first match
when the `target` name hit multiple symbols (Class → Interface → Function
→ Method → Constructor priority UNION). This often produced analysis for
the wrong symbol with no signal back to the caller.
After this change, when the resolver finds more than one viable match
and the caller supplied none of `target_uid` / `file_path` / `kind`,
`impact` returns a disambiguation response shaped like:
```json
{
"status": "ambiguous",
"message": "Found N symbols matching '<target>'. Use target_uid, file_path, or kind to disambiguate.",
"target": { "name": "<target>" },
"direction": "upstream",
"impactedCount": 0,
"risk": "UNKNOWN",
"candidates": [
{ "uid": "...", "name": "...", "kind": "Function", "filePath": "...", "line": 42, "score": 0.76 }
]
}
```
### Do I need to migrate?
**Probably not, but check for assumptions.** Callers that unconditionally
read `result.byDepth` / `result.summary` / `result.affected_processes`
without first checking `result.status` will now see `undefined` in the
ambiguous case. The fix is to branch on `result.status === 'ambiguous'`
first and follow up with `target_uid` (preferred) or `file_path` / `kind`.
The `context` tool's ambiguous response is a strict superset of the
existing shape — every candidate gains a `score` field, no existing field
has changed. No migration required for `context` callers.
### What happens on re-index?
Nothing — this is an MCP-surface change only. The graph schema, indexer,
and stored data are untouched.
---
## OVERRIDES → METHOD_OVERRIDES (PR #642)
The `OVERRIDES` relationship type has been renamed to `METHOD_OVERRIDES` for
consistency with the new `METHOD_IMPLEMENTS` edge type.
### Do I need to migrate?
**No.** Backward compatibility is handled automatically at runtime:
- `local-backend.ts` dual-reads both `OVERRIDES` and `METHOD_OVERRIDES` in all
impact-analysis and context queries. Existing stored graphs with `OVERRIDES`
edges continue to return correct results without any manual intervention.
- The `REL_TYPES` array in `schema-constants.ts` includes both names so Cypher
queries that reference either will work.
### What happens on re-index?
Running `npx gitnexus analyze` on a repository produces `METHOD_OVERRIDES`
edges going forward. The old `OVERRIDES` edges are replaced as part of the
normal full re-index.
### When will the legacy alias be removed?
The `OVERRIDES` compat alias will remain until a future major version. Removal
will be announced in this file and in the changelog before it happens.
+15 -213
View File
@@ -9,7 +9,7 @@
<h2>Join the official Discord to discuss ideas, issues etc!</h2>
<a href="https://discord.gg/MgJrmsqr62">
<a href="https://discord.gg/AAsRVT6fGb">
<img src="https://img.shields.io/discord/1477255801545429032?color=5865F2&logo=discord&logoColor=white" alt="Discord"/>
</a>
<a href="https://www.npmjs.com/package/gitnexus">
@@ -19,8 +19,6 @@
<img src="https://img.shields.io/badge/License-PolyForm%20Noncommercial-blue.svg" alt="License: PolyForm Noncommercial"/>
</a>
<p><strong>Enterprise (SaaS & Self-hosted)</strong> - <a href="https://akonlabs.com">akonlabs.com</a></p>
</div>
**Building nervous system for agent context.**
@@ -52,7 +50,7 @@ https://github.com/user-attachments/assets/172685ba-8e54-4ea7-9ad1-e31a3398da72
| **What** | Index repos locally, connect AI agents via MCP | Visual graph explorer + AI chat in browser |
| **For** | Daily development with Cursor, Claude Code, Codex, Windsurf, OpenCode | Quick exploration, demos, one-off analysis |
| **Scale** | Full repos, any size | Limited by browser memory (~5k files), or unlimited via backend mode |
| **Install** | `npm install -g gitnexus` | No install — [gitnexus.vercel.app](https://gitnexus.vercel.app) |
| **Install** | `npm install -g gitnexus` | No install —[gitnexus.vercel.app](https://gitnexus.vercel.app) |
| **Storage** | LadybugDB native (fast, persistent) | LadybugDB WASM (in-memory, per session) |
| **Parsing** | Tree-sitter native bindings | Tree-sitter WASM |
| **Privacy** | Everything local, no network | Everything in-browser, no server |
@@ -61,36 +59,6 @@ https://github.com/user-attachments/assets/172685ba-8e54-4ea7-9ad1-e31a3398da72
---
## Enterprise
GitNexus is available as an **enterprise offering** - either as a fully managed **SaaS** or a **self-hosted** deployment. Also available for **commercial use** of the OSS version with proper licensing.
Enterprise includes:
- **PR Review** - automated blast radius analysis on pull requests
- **Auto-updating Code Wiki** - always up-to-date documentation (Code Wiki is also available in OSS)
- **Auto-reindexing** - knowledge graph stays fresh automatically
- **Multi-repo support** - unified graph across repositories
- **OCaml support** - additional language coverage
- **Priority feature/language support** - request new languages or features
**Upcoming:**
- Auto regression forensics
- End-to-end test generation
👉 Learn more at [akonlabs.com](https://akonlabs.com)
💬 For commercial licensing or enterprise inquiries, ping us on [Discord](https://discord.gg/AAsRVT6fGb) or drop an email at founders@akonlabs.com
---
## Development
- [ARCHITECTURE.md](ARCHITECTURE.md) — packages, index → graph → MCP flow, where to change code
- [RUNBOOK.md](RUNBOOK.md) — analyze, embeddings, stale index, MCP recovery, CI snippets
- [GUARDRAILS.md](GUARDRAILS.md) — safety rules and operational “Signs” for contributors and agents
- [CONTRIBUTING.md](CONTRIBUTING.md) — license, setup, commits, and pull requests
- [TESTING.md](TESTING.md) — test commands for `gitnexus` and `gitnexus-web`
## CLI + MCP (recommended)
The CLI indexes your repository and runs an MCP server that gives AI agents deep codebase awareness.
@@ -119,6 +87,7 @@ To configure MCP for your editor, run `npx gitnexus setup` once — or set it up
| **Codex** | Yes | Yes | — | MCP + Skills |
| **Windsurf** | Yes | — | — | MCP |
| **OpenCode** | Yes | Yes | — | MCP + Skills |
| **Codex** | Yes | — | — | MCP |
> **Claude Code** gets the deepest integration: MCP tools + agent skills + PreToolUse hooks that enrich searches with graph context + PostToolUse hooks that auto-reindex after commits.
@@ -170,8 +139,8 @@ codex mcp add gitnexus -- npx -y gitnexus@latest mcp
{
"mcp": {
"gitnexus": {
"type": "local",
"command": ["gitnexus", "mcp"]
"command": "npx",
"args": ["-y", "gitnexus@latest", "mcp"]
}
}
}
@@ -188,15 +157,13 @@ args = ["-y", "gitnexus@latest", "mcp"]
### CLI Commands
```bash
gitnexus setup # Configure MCP for your editors (one-time)
gitnexus analyze [path] # Index a repository (or update stale index)
gitnexus analyze --force # Force full re-index
gitnexus analyze --skills # Generate repo-specific skill files from detected communities
gitnexus setup # Configure MCP for your editors (one-time)
gitnexus analyze [path] # Index a repository (or update stale index)
gitnexus analyze --force # Force full re-index
gitnexus analyze --skills # Generate repo-specific skill files from detected communities
gitnexus analyze --skip-embeddings # Skip embedding generation (faster)
gitnexus analyze --skip-agents-md # Preserve custom AGENTS.md/CLAUDE.md gitnexus section edits
gitnexus analyze --skip-git # Index folders that are not Git repositories
gitnexus analyze --embeddings # Enable embedding generation (slower, better search)
gitnexus analyze --verbose # Log skipped files when parsers are unavailable
gitnexus analyze --embeddings # Enable embedding generation (slower, better search)
gitnexus analyze --verbose # Log skipped files when parsers are unavailable
gitnexus mcp # Start MCP server (stdio) — serves all indexed repos
gitnexus serve # Start local HTTP server (multi-repo) for web UI connection
gitnexus list # List all indexed repositories
@@ -206,21 +173,11 @@ gitnexus clean --all --force # Delete all indexes
gitnexus wiki [path] # Generate repository wiki from knowledge graph
gitnexus wiki --model <model> # Wiki with custom LLM model (default: gpt-4o-mini)
gitnexus wiki --base-url <url> # Wiki with custom LLM API base URL
# Repository groups (multi-repo / monorepo service tracking)
gitnexus group create <name> # Create a repository group
gitnexus group add <name> <repo> # Add a repo to a group
gitnexus group remove <name> <repo> # Remove a repo from a group
gitnexus group list [name] # List groups, or show one group's config
gitnexus group sync <name> # Extract contracts and match across repos/services
gitnexus group contracts <name> # Inspect extracted contracts and cross-links
gitnexus group query <name> <q> # Search execution flows across all repos in a group
gitnexus group status <name> # Check staleness of repos in a group
```
### What Your AI Agent Gets
**16 tools** exposed via MCP (11 per-repo + 5 group):
**7 tools** exposed via MCP:
| Tool | What It Does | `repo` Param |
| ------------------ | ----------------------------------------------------------------- | -------------- |
@@ -231,11 +188,6 @@ gitnexus group status <name> # Check staleness of repos in a group
| `detect_changes` | Git-diff impact — maps changed lines to affected processes | Optional |
| `rename` | Multi-file coordinated rename with graph + text search | Optional |
| `cypher` | Raw Cypher graph queries | Optional |
| `group_list` | List configured repository groups | — |
| `group_sync` | Extract contracts and match across repos/services | — |
| `group_contracts`| Inspect extracted contracts and cross-links | — |
| `group_query` | Search execution flows across all repos in a group | — |
| `group_status` | Check staleness of repos in a group | — |
> When only one repo is indexed, the `repo` parameter is optional. With multiple repos, specify which one: `query({query: "auth", repo: "my-app"})`.
@@ -331,160 +283,11 @@ Or run locally:
```bash
git clone https://github.com/abhigyanpatwari/gitnexus.git
cd gitnexus/gitnexus-shared && npm install && npm run build
cd ../gitnexus-web && npm install
cd gitnexus/gitnexus-web
npm install
npm run dev
```
## Docker
The official Docker setup ships **two signed images** orchestrated by `docker-compose.yaml`:
| Image | Purpose |
| -------------------------------------------------- | ---------------------------------------------------------------------- |
| `ghcr.io/abhigyanpatwari/gitnexus:latest` | CLI / `gitnexus serve` backend (HTTP API on port `4747`, MCP, indexer) |
| `ghcr.io/abhigyanpatwari/gitnexus-web:latest` | Static web UI (port `4173`) |
> **Heads-up — image rename.** Earlier releases published the web UI under
> `ghcr.io/abhigyanpatwari/gitnexus`. Starting with the introduction of the
> bundled backend, that slug now hosts the CLI/server image and the UI moved
> to `ghcr.io/abhigyanpatwari/gitnexus-web`. The previous tags remain
> available for pulling, but new versions are only published under the new
> slugs. Update your `docker run` / compose files accordingly (or just adopt
> the bundled compose).
### One-command setup
```bash
docker compose up -d
```
This starts the server on `http://localhost:4747` and the web UI on
`http://localhost:4173`. The UI auto-detects the server because the browser
runs on the host and reaches the container via the mapped port.
A named volume (`gitnexus-data`) persists the global registry, indexes, and
cloned repos at `/data/gitnexus` inside the server container. To make repos on
your host machine indexable, set `WORKSPACE_DIR` before bringing the stack up:
```bash
WORKSPACE_DIR=$HOME/code docker compose up -d
# Inside the server container the directory is mounted read-only at /workspace.
docker compose exec gitnexus-server gitnexus index /workspace/my-repo
```
### Direct `docker run`
```bash
# Server
docker run --rm -d \
--name gitnexus-server \
-p 4747:4747 \
-v gitnexus-data:/data/gitnexus \
ghcr.io/abhigyanpatwari/gitnexus:latest
# Web UI
docker run --rm -d \
--name gitnexus-web \
-p 4173:4173 \
ghcr.io/abhigyanpatwari/gitnexus-web:latest
```
Optional env file (override image tags, container names, ports, workspace dir):
```bash
cp .env.example .env
docker compose --env-file .env up -d
```
### Versioning & supply-chain protection
The Docker images are version-locked to the npm package:
- Stable images are **only published from `vX.Y.Z` git tags** (via `docker.yml`
triggered directly by the tag push), and the workflow refuses to build unless
the tag exactly matches `gitnexus/package.json`'s version. So
`ghcr.io/abhigyanpatwari/gitnexus:1.6.2` is byte-for-byte the same release
as `npm install gitnexus@1.6.2` — no drift, no floating builds from `main`.
- Release-candidate images (e.g. `:1.7.0-rc.1`) are published alongside each
RC npm release. They are built by `release-candidate.yml` calling `docker.yml`
as a reusable workflow after the RC tag is created and pushed.
- `:latest` is auto-promoted only from non-prerelease tags by the Docker
metadata action, so it always points at a real, npm-published version.
Both images are signed with [Cosign keyless signing][cosign-keyless] using the
workflow's GitHub OIDC identity, and shipped with build provenance and SBOM
attestations. **This is your protection against supply-chain attacks**: even if
an attacker republishes a same-named image elsewhere (or somehow pushes to a
typo-squatted registry), they cannot forge a Cosign signature tied to
`abhigyanpatwari/GitNexus`'s `docker.yml`. Always verify before pulling into
sensitive environments:
**Stable releases** — signed from the `v*` tag ref:
```bash
cosign verify ghcr.io/abhigyanpatwari/gitnexus:1.6.2 \
--certificate-identity-regexp '^https://github\.com/abhigyanpatwari/GitNexus/\.github/workflows/docker\.yml@refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9.]+)?$' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
```
The regex pins the certificate identity to this repo's `docker.yml` workflow
**run from a `v*` tag** — rejecting unsigned images, images signed by other
workflows, and images signed from unprotected refs.
**Release candidates** — signed from `refs/heads/main` (the caller's ref when
`release-candidate.yml` invokes `docker.yml` as a reusable workflow):
```bash
cosign verify ghcr.io/abhigyanpatwari/gitnexus:1.7.0-rc.1 \
--certificate-identity 'https://github.com/abhigyanpatwari/GitNexus/.github/workflows/docker.yml@refs/heads/main' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
```
You can also inspect the build provenance and SBOM:
```bash
cosign download attestation ghcr.io/abhigyanpatwari/gitnexus:1.6.2 \
--predicate-type https://slsa.dev/provenance/v1
```
#### Kubernetes: enforce signatures at admission
For Kubernetes deployments, ship the bundled
[`ClusterImagePolicy`](deploy/kubernetes/cluster-image-policy.yaml) so the
[Sigstore policy-controller][policy-controller] rejects any GitNexus pod whose
image is not signed by this repo's `docker.yml` running from a `vX.Y.Z` tag —
the same identity the `cosign verify` snippet above pins.
```bash
# 1. Install the controller (one-time, cluster-wide)
helm repo add sigstore https://sigstore.github.io/helm-charts && helm repo update
helm install policy-controller -n cosign-system --create-namespace \
sigstore/policy-controller
# 2. Opt your namespace in
kubectl label namespace <your-ns> policy.sigstore.dev/include=true
# 3. Apply the policy
kubectl apply -f deploy/kubernetes/cluster-image-policy.yaml
```
After this, attempting to deploy an unsigned image — or one signed by anything
other than `abhigyanpatwari/GitNexus`'s `docker.yml` at a `v*` tag — fails the
admission webhook before a pod is ever created. This turns the verifiable
signature into an enforced policy, which is the supply-chain control most
clusters actually need.
[cosign-keyless]: https://docs.sigstore.dev/cosign/signing/overview/
[policy-controller]: https://docs.sigstore.dev/policy-controller/overview/
### Files
- [Dockerfile.web](Dockerfile.web) — builds `gitnexus-shared` and `gitnexus-web`, then serves the production frontend.
- [Dockerfile.cli](Dockerfile.cli) — builds the CLI/server (with its native deps) and runs `gitnexus serve --host 0.0.0.0`.
- [docker-compose.yaml](docker-compose.yaml) — starts both signed images side by side.
- [.env.example](.env.example) — overrides for image names, container names, ports, and the workspace mount.
The web UI uses the same indexing pipeline as the CLI but runs entirely in WebAssembly (Tree-sitter WASM, LadybugDB WASM, in-browser embeddings). It's great for quick exploration but limited by browser memory for larger repos.
**Local Backend Mode:** Run `gitnexus serve` and open the web UI locally — it auto-detects the server and shows all your indexed repos, with full AI chat support. No need to re-upload or re-index. The agent's tools (Cypher queries, search, code navigation) route through the backend HTTP API automatically.
@@ -564,7 +367,6 @@ GitNexus builds a complete knowledge graph of your codebase through a multi-phas
| Swift | — | — | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| C | — | — | ✓ | — | ✓ | ✓ | — | ✓ | ✓ |
| C++ | — | — | ✓ | ✓ | ✓ | ✓ | — | ✓ | ✓ |
| Dart | ✓ | — | ✓ | ✓ | ✓ | ✓ | — | ✓ | ✓ |
**Imports** — cross-file import resolution · **Named Bindings** — `import { X as Y }` / re-export tracking · **Exports** — public/exported symbol detection · **Heritage** — class inheritance, interfaces, mixins · **Type Annotations** — explicit type extraction for receiver resolution · **Constructor Inference** — infer receiver type from constructor calls (`self`/`this` resolution included for all languages) · **Config** — language toolchain config parsing (tsconfig, go.mod, etc.) · **Frameworks** — AST-based framework pattern detection · **Entry Points** — entry point scoring heuristics
@@ -729,7 +531,7 @@ The wiki generator reads the indexed graph structure, groups files into modules
- [X] Constructor-Inferred Type Resolution, `self`/`this` Receiver Mapping
- [X] Wiki Generation, Multi-File Rename, Git-Diff Impact Analysis
- [X] Process-Grouped Search, 360-Degree Context, Claude Code Hooks
- [X] Multi-Repo MCP, Zero-Config Setup, 14 Language Support
- [X] Multi-Repo MCP, Zero-Config Setup, 13 Language Support
- [X] Community Detection, Process Detection, Confidence Scoring
- [X] Hybrid Search, Vector Index
-163
View File
@@ -1,163 +0,0 @@
# Runbook — GitNexus
Short, copy-paste operations for **local development**, **MCP**, and **CI**. Commands assume a Unix shell; on Windows use Git Bash or equivalent paths.
## Prerequisites
- **Node.js** ≥ 20 (`gitnexus-web/package.json` `engines`).
- **Git** (analyze requires a git repository).
- From repo root, install and build the CLI package:
```bash
cd gitnexus
npm install
npm run build
```
Use `npx gitnexus …` from any path after global/published install, or `node dist/cli/index.js …` when developing from `gitnexus/` with a local build.
---
## Index out of date / “stale” tools
**Symptom:** MCP or resources warn the index is behind `HEAD`, or results don’t reflect recent commits.
**Fix (from the target repo root):**
```bash
npx gitnexus analyze
```
**Force full rebuild** (same commit but suspect corruption or changed ignore rules):
```bash
npx gitnexus analyze --force
```
**Check status:**
```bash
npx gitnexus status
```
**List what MCP knows about:**
```bash
npx gitnexus list
```
---
## Embeddings
**First time with vectors** (slower, more disk/RAM):
```bash
npx gitnexus analyze --embeddings
```
**Important:** If you already had embeddings, **always** pass `--embeddings` on later analyzes, or they can be dropped. See `stats.embeddings` in `.gitnexus/meta.json` (0 means none).
**Large repos:** Analyze may skip or limit embedding work when node counts are very high; watch CLI output.
---
## MCP: no repos / empty tools
**Symptom:** `GitNexus: No indexed repos yet` on stderr when starting MCP.
**Fix:** In each project you want indexed:
```bash
cd /path/to/repo
npx gitnexus analyze
```
Restart the editor MCP session if needed. The server **refreshes the registry lazily**; new analyzes are picked up without necessarily reinstalling MCP.
**Symptom:** Wrong repo when multiple are indexed — pass `repo` on tools or use `list_repos` first.
---
## Clean slate (corrupt or huge `.gitnexus`)
**Current repo only** (prompts for confirmation):
```bash
npx gitnexus clean
```
**Skip confirmation:**
```bash
npx gitnexus clean --force
```
**All registered repos:**
```bash
npx gitnexus clean --all --force
```
Then re-run `npx gitnexus analyze` (and `--embeddings` if you need vectors).
---
## Local bridge for the web UI
```bash
cd gitnexus
npx gitnexus serve
# default http://127.0.0.1:4747 — see serve --help for port/host
```
Use when the browser UI should talk to **local** indexed repos instead of WASM-only mode.
---
## CLI equivalents of MCP tools
Useful for debugging without an editor:
```bash
cd gitnexus
npx gitnexus query "authentication flow" --repo MyRepo
npx gitnexus context SomeSymbol --repo MyRepo
npx gitnexus impact SomeSymbol --direction upstream --repo MyRepo
npx gitnexus cypher "MATCH (n) RETURN count(n) LIMIT 1" --repo MyRepo
```
---
## CI failures (contributors)
Orchestrator: `.github/workflows/ci.yml`.
| Job | Typical local repro |
|-----|---------------------|
| **quality** | `cd gitnexus && npx tsc --noEmit` |
| **unit-tests** | `cd gitnexus && npx vitest run test/unit` |
| **integration** | `cd gitnexus && npx vitest run test/integration` (see workflow matrix for groups) |
| **e2e** | Triggered when `gitnexus-web/` changes; `cd gitnexus-web && E2E=1 npx playwright test` (requires `gitnexus serve` + `npm run dev`) |
**Note:** Pushes that touch only certain markdown paths may be skipped by `paths-ignore` in CI — see workflow file for exact patterns.
---
## Memory / analyze crashes
Analyze re-execs Node with a **large old-space heap** when needed (`analyze.ts`). If you still OOM on huge repos, close other processes, avoid `--embeddings` for a first pass, or analyze a smaller path if supported by your workflow.
---
## LadybugDB / lock errors
Only one process should open a repo’s `.gitnexus/lbug` store at a time. If MCP and a second `analyze` run conflict, stop one process, then retry `analyze` or restart MCP.
---
## Where to dig deeper
- Architecture overview: [ARCHITECTURE.md](ARCHITECTURE.md)
- Agent safety rules: [GUARDRAILS.md](GUARDRAILS.md)
- Tests: [TESTING.md](TESTING.md)
-98
View File
@@ -1,98 +0,0 @@
# Testing — GitNexus
How we structure tests and which commands to run locally and in CI.
## Packages
| Package | Path | Runner | Notes |
| -------------- | -------------- | -------- | ------------------------------ |
| CLI + MCP core | `gitnexus/` | Vitest | Primary test surface in CI |
| Web UI | `gitnexus-web/`| Vitest | Unit/component tests |
| Web UI E2E | `gitnexus-web/`| Playwright | Run when changing UI flows |
## Commands (local)
From repository root, unless noted:
**`gitnexus` (CLI / library)**
```bash
cd gitnexus
npm install
npm run build
npm test # full suite: vitest run
npm run test:unit # unit only: vitest run test/unit
npm run test:integration # integration suite
npm run test:coverage
npx tsc --noEmit # typecheck (matches CI)
```
**`gitnexus-web`**
```bash
cd gitnexus-web
npm install
npm test # unit tests (vitest)
npx tsc -b --noEmit # typecheck (matches CI)
npm run test:coverage
npm run test:e2e # Playwright (requires gitnexus serve + npm run dev)
```
## Pre-commit hook
A husky pre-commit hook (`.husky/pre-commit`) runs automatically on every `git commit`:
1. **Formatting** — `lint-staged` runs prettier on staged files
2. **`gitnexus-web/` files staged** → `tsc -b --noEmit`
3. **`gitnexus/` files staged** → `tsc --noEmit`
Tests do **not** run in the pre-commit hook — they run in CI (`ci-tests.yml`) only.
Skip with `git commit --no-verify` (use sparingly).
## Test categories
- **Unit** — Pure logic, parsers, graph/query helpers; fast; no network.
- **Integration** — Real combinations (filesystem, MCP wiring, larger pipelines) as already organized under `gitnexus/test/integration`.
- **Eval-style / golden sets** — For agent- or classification-style behavior, keep labeled inputs and expected outputs (JSON or table-driven tests) and run them in CI when relevant.
- **E2E (web)** — Critical user paths only; prefer `data-testid` attributes for stable selectors. Tests run against real backend (`gitnexus serve`) and Vite dev server.
## Performance metrics (targets)
Set targets to match team expectations, then tune to this repo’s CI reality:
| Metric | Target (initial) | Notes |
| ------------------- | ---------------- | ------------------------------------------ |
| Unit coverage | Align with CI | CI runs Vitest with coverage in `gitnexus` |
| Unit wall time | Fast PR feedback | Use `vitest run test/unit` for tight loop |
| Integration duration| &lt; few minutes | Guard heavy tests with env flags if needed |
## Regression testing
Re-run the full relevant suite when:
- Prompt or agent-behavior documentation changes (if tests encode behavior)
- Model or embedding-related code paths change
- Graph schema, query contracts, or MCP tool shapes change
- Dependencies with parsing or runtime impact upgrade
## CI integration
GitHub Actions (`.github/workflows/ci.yml`) orchestrate:
- **`ci-quality.yml`** — prettier format check, eslint lint, `tsc --noEmit` for `gitnexus/`, `tsc -b --noEmit` for `gitnexus-web/`
- **`ci-tests.yml`** — `vitest run` with coverage (ubuntu) + cross-platform (macOS, Windows)
- **`ci-e2e.yml`** — Playwright E2E tests, gated on `gitnexus-web/**` changes
Local checks before pushing:
```bash
cd gitnexus && npx tsc --noEmit && npm test
cd ../gitnexus-web && npx tsc -b --noEmit && npm test
```
Or rely on the pre-commit hook which runs these automatically for staged files.
## User acceptance / beta (optional)
For staged releases or UI betas: deploy to a staging environment, collect structured feedback, watch errors and latency, then iterate before a wider release.
@@ -1,64 +0,0 @@
# Sigstore policy-controller ClusterImagePolicy for GitNexus container images.
#
# This enforces — at admission time — that every Pod pulling a
# `ghcr.io/abhigyanpatwari/gitnexus` or `gitnexus-web` image is using a build
# that was Cosign-keyless-signed by this repository's `docker.yml` workflow
# running from a `vX.Y.Z` git tag. Unsigned images, images signed by other
# workflows, and images signed from unprotected refs (e.g. `main`, PR branches)
# are rejected.
#
# Prerequisites
# -------------
# 1. Install the Sigstore policy-controller in your cluster (Helm):
#
# helm repo add sigstore https://sigstore.github.io/helm-charts
# helm repo update
# helm install policy-controller -n cosign-system --create-namespace \
# sigstore/policy-controller
#
# 2. Opt namespaces in to verification:
#
# kubectl label namespace <your-ns> policy.sigstore.dev/include=true
#
# 3. Apply this policy:
#
# kubectl apply -f deploy/kubernetes/cluster-image-policy.yaml
#
# After this, `kubectl run --image=ghcr.io/abhigyanpatwari/gitnexus:<tag>` in
# any opted-in namespace will only succeed if the image carries a valid
# Sigstore signature with the pinned identity.
#
# References
# - https://docs.sigstore.dev/policy-controller/overview/
# - https://github.com/sigstore/policy-controller
apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
name: gitnexus-signed-images
spec:
# Apply to both published GitNexus images on GHCR. Image references always
# carry a tag or digest at admission time, so these two globs cover every
# `gitnexus:<tag>`, `gitnexus@sha256:...`, `gitnexus-web:<tag>`, and
# `gitnexus-web@sha256:...` reference.
images:
- glob: 'ghcr.io/abhigyanpatwari/gitnexus*'
authorities:
- name: gitnexus-cosign-keyless
keyless:
# Public-good Sigstore Fulcio root.
url: https://fulcio.sigstore.dev
identities:
# Pin both the OIDC issuer (GitHub Actions) AND the exact workflow
# path running from a `vX.Y.Z` (or `vX.Y.Z-prerelease`) tag. Same
# regex the README's `cosign verify` example uses; it rejects:
# * unsigned images
# * signatures from any other repo / workflow
# * signatures from non-tag refs (main, PRs, release branches)
# * signatures from arbitrary non-semver tags
- issuer: https://token.actions.githubusercontent.com
subjectRegExp: ^https://github\.com/abhigyanpatwari/GitNexus/\.github/workflows/docker\.yml@refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9.]+)?$
# Cross-check the signature against the public Rekor transparency log,
# so an attacker who briefly compromised Fulcio cannot retroactively
# mint a signature without leaving a public, append-only audit record.
ctlog:
url: https://rekor.sigstore.dev
-45
View File
@@ -1,45 +0,0 @@
services:
gitnexus-server:
image: ${SERVER_IMAGE:-ghcr.io/abhigyanpatwari/gitnexus:latest}
container_name: ${SERVER_CONTAINER_NAME:-gitnexus-server}
# Map the server to the same host port the web UI expects by default
# (http://localhost:4747). The browser runs on the host, so the UI's
# built-in default works without any reconfiguration.
ports:
- '${SERVER_HOST_PORT:-4747}:4747'
volumes:
# Persist the global registry, indexes, and cloned repos across runs.
- gitnexus-data:/data/gitnexus
# Optional: mount a host workspace so `gitnexus index <path>` can see
# repos you already have on disk. The default points at an empty
# `./workspace/` sibling that compose will create on first start —
# it intentionally does NOT bind-mount the repo root, which would
# expose `.git`, `.env`, and CI secrets to the container.
# Override with `WORKSPACE_DIR=/abs/path/to/your/repos`.
- ${WORKSPACE_DIR:-./workspace}:/workspace:ro
restart: unless-stopped
healthcheck:
test: ['CMD', 'curl', '-fsS', 'http://localhost:4747/api/heartbeat']
interval: 30s
timeout: 5s
retries: 3
start_period: 15s
gitnexus-web:
image: ${WEB_IMAGE:-ghcr.io/abhigyanpatwari/gitnexus-web:latest}
container_name: ${WEB_CONTAINER_NAME:-gitnexus-web}
ports:
- '${WEB_HOST_PORT:-4173}:4173'
depends_on:
gitnexus-server:
condition: service_healthy
restart: unless-stopped
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:4173/']
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
volumes:
gitnexus-data:
-81
View File
@@ -1,81 +0,0 @@
import { createReadStream } from 'node:fs';
import { stat } from 'node:fs/promises';
import { createServer } from 'node:http';
import { extname, join, normalize, sep } from 'node:path';
const host = '0.0.0.0';
const port = Number(process.env.PORT || '4173');
const root = join(process.cwd(), 'dist');
const contentTypes = {
'.css': 'text/css; charset=utf-8',
'.html': 'text/html; charset=utf-8',
'.js': 'text/javascript; charset=utf-8',
'.json': 'application/json; charset=utf-8',
'.map': 'application/json; charset=utf-8',
'.png': 'image/png',
'.svg': 'image/svg+xml',
'.txt': 'text/plain; charset=utf-8',
'.woff': 'font/woff',
'.woff2': 'font/woff2',
};
function resolvePath(urlPath) {
let decoded;
try {
decoded = decodeURIComponent(urlPath);
} catch {
return null;
}
if (decoded.includes('\0')) return null;
const cleanPath = normalize(decoded.replace(/^\/+/, ''));
const candidate = join(root, cleanPath);
if (candidate !== root && !candidate.startsWith(root + sep)) return null;
return candidate;
}
const server = createServer(async (req, res) => {
const requestPath = req.url?.split('?')[0] || '/';
let filePath = resolvePath(requestPath);
if (!filePath) {
res.writeHead(400);
res.end('Bad request');
return;
}
try {
const fileStat = await stat(filePath).catch(() => null);
if (fileStat?.isDirectory()) {
filePath = join(filePath, 'index.html');
} else if (!fileStat?.isFile()) {
filePath = join(root, 'index.html');
}
const finalStat = await stat(filePath).catch(() => null);
if (!finalStat?.isFile()) {
res.writeHead(404);
res.end('Not found');
return;
}
res.writeHead(200, {
'Cache-Control': filePath.includes('/assets/')
? 'public, max-age=31536000, immutable'
: 'no-cache',
'Content-Type': contentTypes[extname(filePath)] || 'application/octet-stream',
'Cross-Origin-Opener-Policy': 'same-origin',
'Cross-Origin-Embedder-Policy': 'require-corp',
});
const stream = createReadStream(filePath);
stream.on('error', () => res.destroy());
stream.pipe(res);
} catch (error) {
res.writeHead(500);
res.end(error instanceof Error ? error.message : 'Internal server error');
}
});
server.listen(port, host, () => {
console.log(`gitnexus-web listening on http://${host}:${port}`);
});
-107
View File
@@ -1,107 +0,0 @@
import { mkdir, mkdtemp, rm, unlink, writeFile } from 'node:fs/promises';
import http, { createServer } from 'node:http';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { spawn } from 'node:child_process';
import { fileURLToPath } from 'node:url';
import { after, before, it } from 'node:test';
import assert from 'node:assert/strict';
const __dirname = dirname(fileURLToPath(import.meta.url));
const serverScript = join(__dirname, 'docker-server.mjs');
function getFreePort() {
return new Promise((resolve) => {
const s = createServer();
s.listen(0, '127.0.0.1', () => {
const { port } = s.address();
s.close(() => resolve(port));
});
});
}
function rawGet(port, path) {
return new Promise((resolve, reject) => {
const req = http.request({ host: '127.0.0.1', port, path }, (res) => {
let body = '';
res.setEncoding('utf8');
res.on('data', (chunk) => {
body += chunk;
});
res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, body }));
});
req.on('error', reject);
req.end();
});
}
async function waitForServer(port, retries = 30) {
for (let i = 0; i < retries; i++) {
try {
await rawGet(port, '/');
return;
} catch {
await new Promise((r) => setTimeout(r, 100));
}
}
throw new Error('Server did not start in time');
}
let tmpDir, serverPort, child;
before(async () => {
tmpDir = await mkdtemp(join(tmpdir(), 'gitnexus-docker-test-'));
const distDir = join(tmpDir, 'dist');
const assetsDir = join(distDir, 'assets');
await mkdir(assetsDir, { recursive: true });
await writeFile(join(distDir, 'index.html'), '<html><body>spa</body></html>');
await writeFile(join(assetsDir, 'app.abc123.js'), 'console.log("app")');
serverPort = await getFreePort();
child = spawn(process.execPath, [serverScript], {
cwd: tmpDir,
env: { ...process.env, PORT: String(serverPort) },
stdio: 'pipe',
});
child.on('error', (err) => {
throw err;
});
await waitForServer(serverPort);
});
after(async () => {
child?.kill();
if (tmpDir) await rm(tmpDir, { recursive: true, force: true });
});
it('serves a valid asset with immutable cache header', async () => {
const res = await rawGet(serverPort, '/assets/app.abc123.js');
assert.equal(res.status, 200);
assert.match(res.headers['cache-control'], /immutable/);
assert.equal(res.headers['cross-origin-opener-policy'], 'same-origin');
assert.equal(res.headers['cross-origin-embedder-policy'], 'require-corp');
});
it('serves SPA fallback for unknown routes', async () => {
const res = await rawGet(serverPort, '/some/unknown/route');
assert.equal(res.status, 200);
assert.match(res.body, /spa/);
assert.match(res.headers['cache-control'], /no-cache/);
});
it('rejects path traversal with 400', async () => {
const res = await rawGet(serverPort, '/../../../etc/passwd');
assert.equal(res.status, 400);
});
it('rejects percent-encoded null bytes with 400', async () => {
const res = await rawGet(serverPort, '/foo%00bar');
assert.equal(res.status, 400);
});
it('returns 404 when dist/index.html is missing', async () => {
await unlink(join(tmpDir, 'dist', 'index.html'));
const res = await rawGet(serverPort, '/nonexistent-page');
assert.equal(res.status, 404);
});
+1 -13
View File
@@ -84,17 +84,13 @@ sequenceDiagram
end
```
The return type `CopyExpansionResult` contains `expandedContent` and `copyResolutions`. The `expansionDepth` field has been removed from the return type (it was unused by callers).
COPY statement line numbers in `CopyResolution` are 1-based (consistent with the preprocessor's line numbering). The splice operation that replaces COPY lines with expanded content adjusts for 0-based array indexing internally.
## Cycle Detection
Circular COPY references (e.g., copybook A includes copybook B which includes copybook A) are detected and handled:
1. Each expansion chain maintains a `visited` set of resolved copybook paths
2. If a copybook path is already in the visited set, the expansion is skipped
3. A `warnedCircular` set (internal to `expandCopies()`, not a parameter) deduplicates warning messages within a single file expansion
3. A `warnedCircular` set (shared across all files in a chunk) deduplicates warning messages
Known circular copybooks in PROJECT-NAME: `ANAZI`, `ANDIP`, `QDIPE` (self-referential includes).
@@ -102,10 +98,6 @@ Known circular copybooks in PROJECT-NAME: `ANAZI`, `ANDIP`, `QDIPE` (self-refere
Nested COPY expansion is limited to **10 levels** (`DEFAULT_MAX_DEPTH`). If a COPY chain exceeds this depth, a warning is logged and the remaining COPY statements are left unexpanded.
## Max Total Expansions
A breadth amplification guard caps the total number of COPY expansions across all branches within a single file to **500** (`MAX_TOTAL_EXPANSIONS`). This prevents exponential blowup from diamond-shaped COPY graphs where N copybooks each include N other copybooks. Once the limit is reached, further COPY statements in that file are left unexpanded and a single warning is logged.
## REPLACING Application Detail
The REPLACING engine works by scanning all COBOL identifiers (matching `\b[A-Z][A-Z0-9-]*\b`) in the copybook content and applying each replacement rule:
@@ -147,10 +139,6 @@ The expansion runs **per chunk**, after file content is read but before dispatch
3. Only programs (not copybooks themselves) undergo expansion
4. The expanded content replaces the original content in-place before worker dispatch
## Inline Comment Handling
The copy expander's `stripInlineComment()` helper is quote-aware: pipe characters (`|`) inside single- or double-quoted strings are preserved. This matches the same quote-aware logic used by the preprocessor.
## Source Files
- `gitnexus/src/core/ingestion/cobol-copy-expander.ts` -- `expandCopies()`, `parseReplacingClause()`, `applyReplacing()`
+4 -51
View File
@@ -100,12 +100,10 @@ Tables are extracted from SQL clauses:
| Clause Pattern | Example |
|----------------|---------|
| `FROM <table>` | `SELECT * FROM EMPLOYEES` |
| `INSERT INTO <table>` | `INSERT INTO EMPLOYEES` |
| `INTO <table>` | `INSERT INTO EMPLOYEES` |
| `UPDATE <table>` | `UPDATE EMPLOYEES SET ...` |
| `JOIN <table>` | `LEFT JOIN DEPARTMENTS ON ...` |
Note: The `INTO` pattern is restricted to `INSERT INTO` to avoid false positives from `FETCH ... INTO :host-var` and `SELECT ... INTO :host-var` statements, where `INTO` introduces host variables rather than table names.
### Cursor Detection
```cobol
@@ -244,66 +242,21 @@ The USING clause identifies parameters received by the program from its caller.
## MOVE Statements
MOVE statements produce `ACCESSES` edges in the graph:
MOVE statements are extracted but currently only stored in the regex results (not emitted as graph edges):
```cobol
MOVE WK-NAME TO OUT-NAME.
MOVE CORRESPONDING WK-INPUT TO WK-OUTPUT.
MOVE CORR WK-IN TO WK-OUT.
```
### Extraction Details
- Source and target identifiers are captured
- `CORRESPONDING` and its abbreviation `CORR` are both recognized (bulk field-by-field move)
- `CORRESPONDING` keyword is tracked (bulk field-by-field move)
- Figurative constants (SPACES, ZEROS, LOW-VALUES, HIGH-VALUES, QUOTES, ALL) are skipped
- The enclosing paragraph (`caller`) is tracked for context
### MOVE CORRESPONDING / CORR Edge Reasons
MOVE CORRESPONDING (and CORR) produces distinct edge reasons to differentiate from simple MOVE:
| Edge | Reason (simple MOVE) | Reason (CORRESPONDING/CORR) |
|------|---------------------|-----------------------------|
| Read (source) | `cobol-move-read` | `cobol-move-corresponding-read` |
| Write (target) | `cobol-move-write` | `cobol-move-corresponding-write` |
This distinction allows queries to find bulk field-by-field moves separately from simple variable assignments.
## GO TO DEPENDING ON
The `GO TO` statement with multiple targets and a `DEPENDING ON` clause is a computed branch:
```cobol
GO TO PARA-1 PARA-2 PARA-3
DEPENDING ON WK-SELECTOR.
```
All target paragraph names are extracted and emitted as separate `gotos` entries. Each target produces a `CALLS` edge in the graph (same semantics as PERFORM). The `DEPENDING ON` variable is not currently tracked as a data-flow dependency.
## SORT INPUT/OUTPUT PROCEDURE
SORT and MERGE statements can specify procedural entry points instead of file-based I/O:
```cobol
SORT SORT-FILE ON ASCENDING KEY SORT-KEY
INPUT PROCEDURE IS PREPARE-INPUT
OUTPUT PROCEDURE IS FORMAT-OUTPUT.
```
`INPUT PROCEDURE IS` and `OUTPUT PROCEDURE IS` targets are extracted as control-flow targets (same as PERFORM). They produce `performs` entries and corresponding `CALLS` edges in the graph.
## Fixed-Format Literal Continuation
In fixed-format COBOL, string literals can span multiple lines using the continuation indicator (`-` in column 7). When a continuation line starts with a quote character, the extractor joins it with the predecessor by removing the trailing quote from the previous line and the opening quote from the continuation:
```
Line N: MOVE "THIS IS A LONG STRI
Line N+1 (cont): - "NG VALUE" TO WK-FIELD.
Merged: MOVE "THIS IS A LONG STRING VALUE" TO WK-FIELD.
```
The trailing `"` on line N and the opening `"` on line N+1 are both removed, producing a seamless literal. If no matching quote is found on the predecessor line, the continuation is appended as-is.
DATA_FLOW edges from MOVE statements are reserved for a future release.
## Source Files
-29
View File
@@ -45,31 +45,6 @@ GITNEXUS_COBOL_DIRS=s,c,wfproc GITNEXUS_VERBOSE=1 node --max-old-space-size=8192
/path/to/gitnexus/dist/cli/index.js analyze --force
```
## Open-Source Benchmarks
### CardDemo (AWS)
| Metric | Value |
| ------ | ----- |
| Graph nodes | 12,323 |
| Graph edges | 8,893 |
| Total time | 7.4s |
### ACAS
| Metric | Value |
| ------ | ----- |
| Graph nodes | 14,016 |
| Graph edges | 15,452 |
| Total time | 9.3s |
### Micro-Benchmark (Single-File Extraction)
| Metric | Value |
| ------ | ----- |
| Per-iteration | 0.65ms |
| Throughput | ~382K lines/sec |
## Worker Pool Tuning
### Sub-Batch Size
@@ -134,10 +109,6 @@ To increase the cap for specific needs, modify the `MAX_DATA_ITEMS_PER_FILE` con
## Memory Management
### COPY Expansion Breadth Guard
A per-file `MAX_TOTAL_EXPANSIONS = 500` limit prevents exponential blowup from diamond-shaped COPY graphs (e.g., N copybooks each containing N COPY statements). Once the limit is reached, further COPY statements in that file are left unexpanded. See [copy-expansion.md](copy-expansion.md) for details.
### COPY Expansion Memory
All copybook content is loaded upfront into a Map before chunk processing begins. For PROJECT-NAME:
+4 -24
View File
@@ -71,13 +71,11 @@ Indicator col 7
buffer as new pending logical line
```
After all lines are processed, the final pending line is flushed, along with any accumulated SELECT statement, SORT/MERGE accumulator, and any open EXEC block (truncated file without `END-EXEC`).
After all lines are processed, the final pending line is flushed, along with any accumulated SELECT statement.
### Inline Comment Stripping
Enterprise COBOL (particularly Italian dialect) uses the pipe character `|` as an inline comment marker. The `stripInlineComment()` helper is **quote-aware**: it tracks whether the scan position is inside a single- or double-quoted string and only treats `|` as a comment marker when outside quotes. Pipe characters inside string literals are preserved.
Free-format `*>` inline comment stripping uses the same quote-aware approach: the scanner walks character by character, toggling quote state, and only recognizes `*>` as a comment marker when not inside a quoted string.
Enterprise COBOL (particularly Italian dialect) uses the pipe character `|` as an inline comment marker. Everything from `|` to end of line is stripped before processing.
### Patch Marker Handling
@@ -113,7 +111,7 @@ All patterns are compiled once as module-level constants and reused across calls
| Constant | Pattern | Purpose | Example Match |
|----------|---------|---------|---------------|
| `RE_SELECT_START` | `\bSELECT\s+(?:OPTIONAL\s+)?([A-Z][A-Z0-9-]+)` | File SELECT start (with optional `SELECT OPTIONAL` support) | `SELECT MASTER-FILE`, `SELECT OPTIONAL TRANS-FILE` |
| `RE_SELECT_START` | `\bSELECT\s+([A-Z][A-Z0-9-]+)` | File SELECT start | `SELECT MASTER-FILE` |
SELECT statements are accumulated across multiple lines until a period terminator is found, then parsed for ASSIGN, ORGANIZATION, ACCESS, RECORD KEY, and FILE STATUS clauses.
@@ -137,9 +135,7 @@ The trailing clauses of `RE_DATA_ITEM` are parsed by `parseDataItemClauses()` fo
| `RE_PERFORM` | `\bPERFORM\s+([A-Z][A-Z0-9-]+)(?:\s+THRU\s+([A-Z][A-Z0-9-]+))?` | PERFORM call | `PERFORM CALC-TAX THRU CALC-TAX-EXIT` |
| `RE_PROC_USING` | `\bPROCEDURE\s+DIVISION\s+USING\s+([\s\S]*?)(?:\.\|$)` | USING parameters | `PROCEDURE DIVISION USING WK-PARAM` |
| `RE_ENTRY` | `\bENTRY\s+"([^"]+)"(?:\s+USING\s+([\s\S]*?))?(?:\.\|$)` | ENTRY point | `ENTRY "SUBPROG" USING WK-DATA` |
| `RE_MOVE` | `\bMOVE\s+((?:CORRESPONDING\|CORR)\s+)?([A-Z][A-Z0-9-]+)\s+TO\s+(.+)` | MOVE statement (supports CORR abbreviation and multi-target) | `MOVE WK-NAME TO OUT-NAME`, `MOVE CORR WK-IN TO WK-OUT` |
The USING parameter list (`RE_PROC_USING`) is split on `\bRETURNING\b` before tokenization -- any RETURNING clause and everything after it is excluded from the parameter list (`.split(/\bRETURNING\b/i)[0]`).
| `RE_MOVE` | `\bMOVE\s+(CORRESPONDING\s+)?([A-Z][A-Z0-9-]+)\s+TO\s+([A-Z][A-Z0-9-]+)` | MOVE statement | `MOVE WK-NAME TO OUT-NAME` |
Note: `RE_PROC_SECTION` and `RE_PROC_PARAGRAPH` require exactly 7 spaces of leading indentation (COBOL area A starting at column 8). This is the standard COBOL paragraph indentation.
@@ -153,22 +149,6 @@ These patterns are checked regardless of current division:
| `RE_COPY_UNQUOTED` | `\bCOPY\s+([A-Z][A-Z0-9-]+)(?:\s\|\.)` | COPY (unquoted) | `COPY CPSESP.` |
| `RE_COPY_QUOTED` | `\bCOPY\s+"([^"]+)"(?:\s\|\.)` | COPY (quoted) | `COPY "WORKGRID.CPY".` |
### SORT/MERGE Support
| Constant | Purpose |
|----------|---------|
| `SORT_CLAUSE_NOISE` | Set of SORT/MERGE clause keywords filtered from USING/GIVING file lists: `ON`, `ASCENDING`, `DESCENDING`, `KEY`, `WITH`, `DUPLICATES`, `IN`, `ORDER`, `COLLATING`, `SEQUENCE`, `IS`, `THROUGH`, `THRU`, `INPUT`, `OUTPUT`, `PROCEDURE` |
SORT and MERGE statements are accumulated across multiple lines (like SELECT) until a period terminator is found, then parsed for USING/GIVING file lists and INPUT/OUTPUT PROCEDURE targets. The `flushSort()` helper encapsulates the flush-and-parse logic, mirroring the existing `flushSelect()` pattern. Both helpers are called at EOF to handle truncated files.
### GO TO Multi-Target
`RE_GOTO` captures all paragraph names in a `GO TO` statement, including the multi-target form `GO TO p1 p2 p3 DEPENDING ON x`. The captured group contains all target names (space-separated), which are split into individual targets. Each target produces a separate `gotos` entry.
### PROGRAM-ID Detection
PROGRAM-ID is detected regardless of the current division state. This handles sibling programs that appear after `END PROGRAM` and omit the `IDENTIFICATION DIVISION` header -- the extractor will still capture the PROGRAM-ID and push a new program boundary.
### EXEC Block Patterns
| Constant | Pattern | Purpose | Example Match |
@@ -1,326 +0,0 @@
---
title: "feat: Complete COBOL language feature coverage for maximum knowledge graph value"
type: feat
status: active
date: 2026-03-26
origin: Feature audit from v3-integration-architect agent (session 8642401e)
---
## Enhancement Summary
**Deepened on:** 2026-03-26
**Research agents used:** COBOL expert (Phase 1+2), graph value analyst, codebase explorer
**Sections enhanced:** Phase 1 (5 features), Phase 2 (4 features), graph value ranking
### Key Improvements from Research
1. **CALL USING** is the #1 highest-value edge type (9.2/10) — fixes ~40% of missing caller references
2. **EXEC DLI** requires dual-interface support (EXEC DLI + CBLTDLI CALL) for full IMS coverage
3. **DECLARATIVES** is lowest-risk Phase 2 item — existing section/paragraph detection already captures structure
4. **SET TO TRUE** accounts for 80-90% of all SET statements — prioritize this form
5. **INSPECT** needs multi-line accumulator (like SORT) — can span 5+ continuation lines
6. **Graph value ranking**: cobol-call-using (9.2) > cobol-error-handler (9.0) > dli-gu (8.2) > cobol-string (6.2)
### New Edge Cases Discovered
- CALL USING supports mixed modes: `USING BY REFERENCE WS-A BY CONTENT WS-B BY VALUE WS-C`
- CALL USING `ADDRESS OF` and `OMITTED` must be filtered from parameter lists
- EXEC DLI can have multiple SEGMENT levels in hierarchical retrieval (use matchAll)
- DECLARATIVES can have multiple USE sections (one per file + catch-all for INPUT/OUTPUT/I-O/EXTEND)
- INSPECT TALLYING can have multiple counters in a single statement
- STRING/UNSTRING can span multiple lines (need accumulator pattern)
---
# Complete COBOL Language Feature Coverage
## Overview
Implement the remaining 25 unhandled COBOL language features and fix 10 partial features to achieve ~95% coverage (up from 71.9%). The goal is to build the richest possible knowledge graph from COBOL codebases, enabling a future `modernize` MCP command (out of scope for this plan) that would use the graph to assist with COBOL-to-modern-language migration.
## Problem Statement
The COBOL processor currently handles 54 of 89 applicable language features (71.9%). The 25 unhandled features represent real data loss in the knowledge graph:
- **Cross-program data flow** is invisible (CALL ... USING parameters not extracted)
- **IMS/DB programs** produce empty graphs (EXEC DLI not recognized)
- **String transformation logic** is invisible (STRING/UNSTRING/INSPECT not tracked)
- **SQL copybook dependencies** are missing (EXEC SQL INCLUDE not mapped)
- **Error handling flows** are lost (DECLARATIVES/USE AFTER not captured)
## Proposed Solution
Implement features in 4 phases, ordered by graph value density (edges created per LOC of implementation). Each phase is independently shippable and testable.
## Technical Approach
### Phase 1: High-Value Data Flow Edges (~150 LOC, ~8 new edge types)
The highest-ROI features: they create new ACCESSES and IMPORTS edges that directly improve impact analysis.
**Critical research finding**: Multi-line statement accumulation is the dominant challenge. CALL USING, STRING/UNSTRING, and multi-line data item clauses all span multiple lines in production COBOL. The free-format path processes each line independently — these features need statement accumulators (like SORT/SELECT) or the free-format path needs multi-line awareness. Estimated LOC increased from 110 to 150 to account for accumulator infrastructure.
#### 1.1 EXEC SQL INCLUDE -> IMPORTS edges
- **File:** `cobol-preprocessor.ts` (parseExecSqlBlock)
- **What:** Detect `INCLUDE` as the operation, extract member name, emit as a `copies[]` entry
- **Graph:** IMPORTS edge from File to included copybook/SQLCA with reason `sql-include`
- **Tests:** Unit test for `EXEC SQL INCLUDE SQLCA END-EXEC` and `EXEC SQL INCLUDE CUSTCOPY END-EXEC`
**Research insights (EXEC SQL INCLUDE):**
- DB2 member names can contain underscores: `EXEC SQL INCLUDE CUST_TBL_DCL END-EXEC` — regex must use `[A-Z][A-Z0-9_-]+`
- Quoted literal form: `EXEC SQL INCLUDE 'DBRMLIB.MEMBER' END-EXEC` (z/OS PDS qualified name)
- SQLCA/SQLDA are DB2 builtins — won't resolve to repo files. Emit unresolved IMPORTS edge (still valuable)
- No REPLACING support on EXEC SQL INCLUDE (unlike COPY)
- Add `INCLUDE` to `OP_MAP` in `parseExecSqlBlock`; extract member via `RE_SQL_INCLUDE = /^INCLUDE\s+(?:'([^']+)'|"([^"]+)"|([A-Z][A-Z0-9_-]+))/i`
#### 1.2 CALL ... USING parameter extraction -> ACCESSES edges (Graph value: 9.2/10)
- **File:** `cobol-preprocessor.ts` (processLogicalLine CALL section)
- **What:** After capturing CALL target, scan for USING clause. Extract parameter names (reuse USING_KEYWORDS filter). Store as `calls[].parameters: string[]`
- **Interface:** Add `parameters?: string[]` to calls array type in CobolRegexResults
- **File:** `cobol-processor.ts` (CALL edge block)
- **Graph:** For each USING parameter, create ACCESSES edge from caller to data item Property node with reason `cobol-call-using`
- **Tests:** `CALL 'AUDITLOG' USING CUST-ID WS-AMOUNT` -> 2 ACCESSES edges
**Research insights (CALL USING forms):**
- Mixed modes: `CALL 'PGM' USING BY REFERENCE WS-A BY CONTENT WS-B BY VALUE WS-C`
- Pointer passing: `CALL 'PGM' USING ADDRESS OF WS-A`
- Placeholder: `CALL 'PGM' USING OMITTED WS-B`
- Filter keywords: add `ADDRESS`, `OMITTED`, `LENGTH` to USING_KEYWORDS (already has BY/VALUE/REFERENCE/CONTENT)
- **Impact tool enhancement:** CALL-USING edges enable BFS traversal through parameter data flow — single most impactful edge type for COBOL impact analysis
#### 1.3 STRING/UNSTRING data flow -> ACCESSES edges
- **File:** `cobol-preprocessor.ts` (new section in extractProcedure)
- **What:** Accumulate multi-line STRING/UNSTRING until period or END-STRING/END-UNSTRING. Extract sources and INTO targets.
- **Interface:** Add `strings: Array<{ sources: string[]; target: string; type: 'string' | 'unstring'; line: number; caller: string | null }>` to CobolRegexResults
- **Graph:** read-ACCESSES on sources, write-ACCESSES on INTO target with reason `cobol-string-read` / `cobol-string-write`
- **Tests:** 2 unit tests + integration test assertions
**Research insights (STRING/UNSTRING):**
- **Needs statement accumulator** — STRING/UNSTRING always span multiple lines in production
- Terminate accumulation at: period, END-STRING/END-UNSTRING, or start of next COBOL verb
- STRING sources: identifiers before each `DELIMITED BY`. Filter: STRING, DELIMITED, BY, SIZE, ALL, INTO, WITH, POINTER, ON, OVERFLOW, NOT, END-STRING
- UNSTRING: source is first identifier after UNSTRING; INTO targets are identifiers after INTO. Filter: DELIMITER, IN, COUNT, TALLYING, OR
- WITH POINTER field is both read AND written (starting position updated)
- TALLYING IN / COUNT IN fields are write targets
- Literal sources (`'text'`) must be filtered — quote-aware tokenization needed
- **Edge case**: STRING terminated by next verb, not period — existing fixture has `STRING ... DISPLAY` without period between them
#### 1.4 OCCURS DEPENDING ON -> ACCESSES edge
- **File:** `cobol-preprocessor.ts` (parseDataItemClauses)
- **What:** Extend OCCURS regex to capture DEPENDING ON field, KEY fields, and INDEXED BY names
- **Interface:** Add `dependingOn?: string`, `occursMax?: number`, `occursKeys?: Array<{direction: string; fields: string[]}>`, `indexedBy?: string[]` to data items
- **Graph:** ACCESSES edge from table item to controlling field with reason `cobol-depends-on`
- **Tests:** `05 WS-TABLE OCCURS 100 DEPENDING ON WS-COUNT` -> edge
**Research insights (OCCURS):**
- IBM allows `OCCURS 0 TO n DEPENDING ON` (zero minimum) and `OCCURS UNBOUNDED DEPENDING ON` (V6.4)
- Subscripted controlling fields: `DEPENDING ON WS-COUNT(WS-IDX)` — strip subscripts before storing
- **Pre-existing gap**: Multi-line data item clauses without continuation indicator are NOT captured. `05 WS-TABLE\n OCCURS 100\n DEPENDING ON WS-COUNT.` — the current RE_DATA_ITEM only gets the first line, `rest` is empty. Fixing properly requires a data item accumulator (like SELECT). **Defer full fix to Phase 3; implement same-line capture now.**
- KEY IS fields: `ASCENDING KEY IS WS-KEY-1 WS-KEY-2` — capture for SEARCH ALL resolution
- INDEXED BY: `INDEXED BY IDX-1 IDX-2` — capture for SET/SEARCH context
#### 1.5 VALUE clause for standard data items
- **File:** `cobol-preprocessor.ts` (parseDataItemClauses)
- **What:** Extract VALUE using a pragmatic function that handles quoted strings, numerics, figurative constants, hex/national literals
- **Interface:** Already exists as `values?: string[]` on data items (currently only populated for 88-level)
- **Graph:** Stored in Property node description (no new edges)
- **Tests:** `01 WS-STATUS PIC X VALUE 'A'` -> values: ['A']
**Research insights (VALUE forms):**
- Hex literals: `VALUE X'F1F2F3F4'`, National: `VALUE N'text'`, DBCS: `VALUE G'text'`
- Figurative constants: SPACES, ZEROS, ZEROES, LOW-VALUES, HIGH-VALUES, QUOTES, NULL, NULLS
- ALL literal: `VALUE ALL '*'`
- Numeric with sign/decimal: `VALUE -123.45`, `VALUE +1`
- `VALUE IS` optional — both `VALUE 'A'` and `VALUE IS 'A'` valid
- **Decimal vs period ambiguity**: `VALUE 100.` — is `.` decimal or terminator? `parseDataItemClauses` already strips trailing period, so this is handled
- IBM V6.4: floating-point `VALUE 1.0E5` — extend numeric regex if needed
- Implementation: use a pragmatic `extractValue(rest)` function, not a single complex regex
### Phase 2: EXEC DLI + DECLARATIVES (~90 LOC, ~4 new edge types)
IMS/DB support and error handling flows.
#### 2.1 EXEC DLI (IMS/DB) -> ACCESSES edges (Graph value: 8.2/10)
- **File:** `cobol-preprocessor.ts` (processLogicalLine — add RE_EXEC_DLI_START check alongside SQL/CICS)
- **What:** Accumulate EXEC DLI blocks like EXEC SQL. Parse DLI verbs (GU, GN, GNP, GHU, GHN, GHNP, ISRT, DLET, REPL, CHKP, SCHD, TERM). Extract segment name, PCB number, INTO/FROM areas, WHERE fields, PSB name.
- **Interface:** Add `execDliBlocks: Array<{ line: number; verb: string; pcbNumber?: number; segmentName?: string; intoField?: string; fromField?: string; whereField?: string; psbName?: string }>` to CobolRegexResults
- **Graph:** CodeElement node + ACCESSES edge to `<ims>:<segmentName>` Record node with reason `dli-{verb}`; ACCESSES edges to INTO/FROM data areas; PSB ACCESSES for SCHD
- **Tests:** `EXEC DLI GU USING PCB(1) SEGMENT(CUSTOMER) INTO(WS-CUST) END-EXEC`
**Research insights (dual IMS interface):**
- **EXEC DLI**: Embedded command interface for CICS-DL/I programs only
- **CBLTDLI CALL**: Batch interface via `CALL 'CBLTDLI' USING function-code PCB io-area SSA1..SSA15`
- CBLTDLI is already captured as a CALL to 'CBLTDLI' — enrich with USING parameter semantics later
- Multiple SEGMENT levels in hierarchical retrieval — use `matchAll` on segment regex
- DLI verbs: GU (most common), GN, GNP, GHU, GHN, GHNP, ISRT, REPL, DLET, CHKP, SCHD, TERM, ROLL, ROLB
- **Edge case**: DLET/REPL have no SEGMENT clause (operate on current position)
- **Recommended order**: Implement AFTER DECLARATIVES and SET (lower risk, higher frequency)
#### 2.2 DECLARATIVES / USE AFTER STANDARD EXCEPTION (Graph value: 9.0/10)
- **File:** `cobol-preprocessor.ts` (processLogicalLine — detect DECLARATIVES keyword, track USE AFTER blocks)
- **What:** When `DECLARATIVES.` is encountered, switch to declaratives mode. Extract USE statements binding sections to files/modes.
- **Interface:** Add `declaratives: Array<{ sectionName: string; useType: 'error' | 'debug' | 'label' | 'reporting'; target: string; line: number }>` to CobolRegexResults
- **Graph:** ACCESSES edge from declarative Namespace to file Record with reason `cobol-declarative-error-handler`
- **Tests:** Unit test with DECLARATIVES section, integration test for error flow
**Research insights (DECLARATIVES syntax):**
- `USE AFTER STANDARD {EXCEPTION|ERROR} ON {file-name|INPUT|OUTPUT|I-O|EXTEND}`
- EXCEPTION and ERROR are synonymous; STANDARD is optional in IBM dialects
- Multiple USE sections allowed (one per file + catch-all for I/O modes)
- `END DECLARATIVES.` must NOT reset PROCEDURE DIVISION state
- `DECLARATIVES` is already in EXCLUDED_PARA_NAMES — no false paragraph risk
- Existing section/paragraph detection already captures structural elements — just need USE binding
- **Lowest risk Phase 2 item** — implement first
#### 2.3 SET statement -> ACCESSES edges
- **File:** `cobol-preprocessor.ts` (extractProcedure — new RE_SET regex)
- **Interface:** Add `sets: Array<{ targets: string[]; form: 'to-true'|'to-value'|'up-by'|'down-by'|'address-of'|'to-null'|'to-entry'; value?: string; entryTarget?: string; entryIsLiteral?: boolean; line: number; caller: string | null }>` to CobolRegexResults
- **Graph:** ACCESSES write edge with reason `cobol-set-condition` (TO TRUE), `cobol-set-index` (TO/UP/DOWN), `cobol-set-address` (ADDRESS OF). SET ENTRY with literal -> CALLS edge.
- **Tests:** `SET WS-EOF TO TRUE`, `SET IDX-1 TO 5`, `SET IDX-1 UP BY 1`
**Research insights (SET forms by frequency):**
- `SET condition TO TRUE` — 80-90% of all SET usage. Multiple targets: `SET COND-A COND-B TO TRUE`
- `SET index TO/UP BY/DOWN BY` — ~8%. Multiple indices: `SET IDX-1 IDX-2 UP BY 1`
- `SET pointer TO ADDRESS OF data-item` / `SET ADDRESS OF data-item TO pointer` — ~2%
- `SET proc-ptr TO ENTRY "PROGNAME"` — rare but creates CALLS edge (like dynamic CALL)
- Filter OF/IN qualifiers: `SET COND-A OF WS-RECORD TO TRUE` (strip OF WS-RECORD)
- **Prioritize**: SET TO TRUE alone covers 80-90% — implement this form first
#### 2.4 INSPECT -> ACCESSES edges
- **File:** `cobol-preprocessor.ts` (extractProcedure — new `inspectAccum` accumulator like SORT)
- **What:** Accumulate multi-line INSPECT until period. Extract inspected field + tally counters.
- **Interface:** Add `inspects: Array<{ inspectedField: string; counters: string[]; form: 'tallying'|'replacing'|'converting'|'tallying-replacing'; line: number; caller: string | null }>` to CobolRegexResults
- **Graph:** ACCESSES read on inspected field always; write if REPLACING/CONVERTING. Write edges for tally counters. Reason: `cobol-inspect-read`/`cobol-inspect-write`/`cobol-inspect-tally`
- **Tests:** `INSPECT WS-FIELD TALLYING WS-COUNT FOR ALL 'A'` -> read on WS-FIELD, write on WS-COUNT
**Research insights (INSPECT forms by frequency):**
- REPLACING (~60%): `INSPECT WS-STR REPLACING ALL 'A' BY 'B'`
- TALLYING (~25%): `INSPECT WS-STR TALLYING WS-CNT FOR ALL 'A'` — multiple counters possible
- CONVERTING (~10%): `INSPECT WS-STR CONVERTING 'abc' TO 'ABC'`
- Combined (~5%): TALLYING + REPLACING in single statement
- **Needs multi-line accumulator** — INSPECT frequently spans 3-5 lines in production
- Extract tally counters with `([A-Z][A-Z0-9-]+)\s+FOR\b` matchAll pattern
- Filter figurative constants (SPACES, ZEROS) using existing MOVE_SKIP set
### Phase 3: Completeness Fixes (~60 LOC)
Fix the 10 partial features and small gaps.
#### 3.1 CALL ... RETURNING extraction
- Extend RE_CALL processing to capture RETURNING target after the USING clause
- Store as `calls[].returning?: string`
- Graph: ACCESSES write edge with reason `cobol-call-returning`
#### 3.2 SELECT OPTIONAL flag preservation
- Store `isOptional: boolean` in FileDeclaration interface
- Include in Record node description
#### 3.3 ALTERNATE RECORD KEY extraction
- Add regex in parseSelectStatement: `/\bALTERNATE\s+RECORD\s+KEY\s+(?:IS\s+)?([A-Z][A-Z0-9-]+)/i`
- Store as `alternateKeys?: string[]`
#### 3.4 COMMON attribute on nested programs
- Extend RE_PROGRAM_ID: `/\bPROGRAM-ID\.\s*([A-Z][A-Z0-9-]+)(?:\s+IS\s+COMMON)?/i`
- Store `isCommon: boolean` on Module node
- Affects cross-program CALL resolution scope
#### 3.5 IS EXTERNAL / IS GLOBAL as first-class properties
- Change from usage string hack to proper boolean fields on data items
- Add `isExternal?: boolean`, `isGlobal?: boolean` to data item interface
#### 3.6 AUTHOR / DATE-WRITTEN mapped to Module node
- Already extracted as programMetadata — map to Module node properties
- `graph.addNode({ ..., properties: { ..., author, dateWritten } })`
#### 3.7 REPLACE statement
- Track REPLACE / REPLACE OFF state in preprocessor
- Apply text substitutions during preprocessing (before regex extraction)
- Complex: requires careful scoping rules
### Phase 4: Niche Features (~30 LOC)
Low-priority but nice for completeness.
#### 4.1 INITIALIZE statement -> write ACCESSES
- `/\bINITIALIZE\s+([A-Z][A-Z0-9-]+)/i`
- ACCESSES write edge with reason `cobol-initialize`
#### 4.2 Remaining IDENTIFICATION DIVISION paragraphs
- DATE-COMPILED, INSTALLATION, SECURITY, REMARKS
- Map to Module node description properties
#### 4.3 EXEC SQL INCLUDE -> IMPORTS edge (expansion)
- For EXEC SQL INCLUDE inside EXEC blocks that reference copybooks containing SQL
- Create IMPORTS edge similar to COPY
## Acceptance Criteria
### Functional Requirements
- [ ] Phase 1: All 5 features implemented with unit + integration tests
- [ ] Phase 2: All 4 features implemented with unit + integration tests
- [ ] Phase 3: All 7 partial features fixed
- [ ] Phase 4: At least 2 of 3 niche features implemented
- [ ] All existing 145 tests continue to pass
- [ ] TypeScript compiles cleanly
### Non-Functional Requirements
- [ ] No performance regression: CardDemo benchmark stays under 8s
- [ ] No file exceeds 1500 LOC (preprocessor currently 1326)
- [ ] ACAS benchmark shows increased node/edge counts (more data extracted)
- [ ] CardDemo benchmark shows increased edge counts (CALL USING, STRING, etc.)
### Quality Gates
- [ ] Each phase has its own commit
- [ ] Integration test assertions updated with exact counts per phase
- [ ] Benchmark run after each phase to track graph growth
## Dependencies & Risks
### Dependencies
- None. All changes are additive to existing COBOL processor code.
- No LanguageProvider changes needed.
- No graph schema changes needed (all new constructs map to existing node labels + edge types).
### Risks
- **preprocessor.ts size**: Currently 1326 LOC. Phase 1+2 adds ~200 LOC -> 1526 LOC. May need to extract helpers into a separate `cobol-data-flow.ts` module if it exceeds 1500.
- **REPLACE statement** (Phase 3.7) is the most complex feature — requires tracking text substitution state across logical lines. Consider deferring to a separate PR if it takes >100 LOC.
- **EXEC DLI** (Phase 2.1) is only testable against IMS codebases. Need fixture data or synthetic test cases.
## Graph Value Ranking by MCP Tool Impact
Research agent analyzed all 5 MCP tools (query, context, impact, detect_changes, rename) against planned edge types:
| Edge Type | QUERY | CONTEXT | IMPACT | DETECT | RENAME | **Overall** |
|-----------|-------|---------|--------|--------|--------|-------------|
| `cobol-call-using` | 4/5 | 5/5 | 5/5 | 4/5 | 4/5 | **9.2/10** |
| `cobol-error-handler` | 5/5 | 4/5 | 5/5 | 5/5 | 2/5 | **9.0/10** |
| `dli-*` (IMS verbs) | 4/5 | 4/5 | 5/5 | 4/5 | 2/5 | **8.2/10** |
| `cobol-string-*` | 4/5 | 3/5 | 3/5 | 3/5 | 2/5 | **6.2/10** |
**Key finding**: `cobol-call-using` alone would fix ~40% of missing caller references in COBOL graphs.
## Future Considerations
This plan provides the graph data foundation for a future `modernize` MCP command (out of scope) that would:
- Use CALL USING edges to map data contracts between programs
- Use STRING/UNSTRING edges to identify data transformation logic
- Use EXEC SQL/DLI edges to map database access patterns
- Use DECLARATIVES to understand error handling architecture
- Use the complete knowledge graph to generate migration plans
**MCP tool enhancements needed** (after this plan ships):
- Add `cobol-call-using`, `cobol-error-handler`, `dli-*` to IMPACT tool's default `relationTypes` for COBOL repos
- Add confidence floors for new edge types in `IMPACT_RELATION_CONFIDENCE`
- Register new edge types in `VALID_RELATION_TYPES` set (`local-backend.ts:52`)
## Sources & References
### Internal References
- Feature audit: session 8642401e (COBOL expert agent, 123 features audited)
- Prior plans: `docs/plans/2026-03-25-feat-cobol-100-percent-feature-coverage-plan.md`
- Architecture: `docs/code-indexing/cobol/` (7 documentation files)
### External References
- COBOL features reference: mainframestechhelp.com/tutorials/cobol/features.htm
- COBOL-85 standard: ISO/IEC 1989:1985
- IBM Enterprise COBOL reference
@@ -1,725 +0,0 @@
# PR #626 HIGH-Priority Fixes Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Fix 4 HIGH-priority issues from PR #626 code review before merge.
**Architecture:** Minimal targeted fixes — each task is independent. TDD: tests first, then implementation. No refactoring beyond what's needed.
**Tech Stack:** TypeScript, Vitest, Node.js fs/path APIs
**Spec:** `docs/superpowers/specs/2026-04-02-pr626-high-fixes-design.md`
**Paths:** All file paths are relative to the monorepo root (`GitNexus/`). Git commands run from the root. The `gitnexus/` prefix is a package subdirectory, not a separate repo.
---
### Task 1: Path Traversal — Validate Group Name
**Files:**
- Modify: `gitnexus/src/core/group/storage.ts:17-19` (getGroupDir) and `:63-68` (createGroupDir)
- Test: `gitnexus/test/unit/group/storage.test.ts`
- [ ] **Step 1: Write failing tests for validateGroupName**
In `gitnexus/test/unit/group/storage.test.ts`, add `createGroupDir` and `validateGroupName` to the existing import from `'../../../src/core/group/storage.js'` (line 6-11). Then add these describe blocks at the end of the outer `describe('Group storage', ...)`:
```typescript
describe('validateGroupName', () => {
it('test_validateGroupName_traversal_path_throws', () => {
expect(() => validateGroupName('../../evil')).toThrow(/Invalid group name/);
});
it('test_validateGroupName_slash_in_name_throws', () => {
expect(() => validateGroupName('foo/bar')).toThrow(/Invalid group name/);
});
it('test_validateGroupName_empty_string_throws', () => {
expect(() => validateGroupName('')).toThrow(/Invalid group name/);
});
it('test_validateGroupName_starts_with_dash_throws', () => {
expect(() => validateGroupName('-leading-dash')).toThrow(/Invalid group name/);
});
it('test_validateGroupName_starts_with_underscore_throws', () => {
expect(() => validateGroupName('_leading')).toThrow(/Invalid group name/);
});
it('test_validateGroupName_dots_throws', () => {
expect(() => validateGroupName('com.example')).toThrow(/Invalid group name/);
});
it('test_validateGroupName_valid_alphanumeric_passes', () => {
expect(() => validateGroupName('my-group_01')).not.toThrow();
});
it('test_validateGroupName_single_char_passes', () => {
expect(() => validateGroupName('A')).not.toThrow();
});
it('test_validateGroupName_all_digits_passes', () => {
expect(() => validateGroupName('123')).not.toThrow();
});
});
describe('getGroupDir rejects invalid names', () => {
it('test_getGroupDir_traversal_throws', () => {
expect(() => getGroupDir(tmpDir, '../../etc')).toThrow(/Invalid group name/);
});
it('test_getGroupDir_valid_name_returns_path', () => {
const dir = getGroupDir(tmpDir, 'company');
expect(dir).toBe(path.join(tmpDir, 'groups', 'company'));
});
});
describe('createGroupDir rejects invalid names', () => {
it('test_createGroupDir_traversal_throws', async () => {
await expect(createGroupDir(tmpDir, '../evil')).rejects.toThrow(/Invalid group name/);
});
});
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd gitnexus && npx vitest run test/unit/group/storage.test.ts`
Expected: FAIL — `validateGroupName` is not exported, `getGroupDir` does not throw.
- [ ] **Step 3: Implement validateGroupName and wire into getGroupDir and createGroupDir**
In `gitnexus/src/core/group/storage.ts`, add the validation function before `getGroupDir` and call it:
```typescript
const GROUP_NAME_RE = /^[a-zA-Z0-9][a-zA-Z0-9_-]*$/;
export function validateGroupName(name: string): void {
if (!GROUP_NAME_RE.test(name)) {
throw new Error(
`Invalid group name "${name}". Names must start with a letter or digit and contain only [a-zA-Z0-9_-].`,
);
}
}
export function getGroupDir(gitnexusDir: string, groupName: string): string {
validateGroupName(groupName);
return path.join(gitnexusDir, 'groups', groupName);
}
```
`createGroupDir` already calls `getGroupDir` at line 68, so it inherits validation automatically. No change needed in `createGroupDir`.
- [ ] **Step 4: Run tests to verify they pass**
Run: `cd gitnexus && npx vitest run test/unit/group/storage.test.ts`
Expected: ALL PASS
- [ ] **Step 5: Commit**
```bash
cd gitnexus && git add src/core/group/storage.ts test/unit/group/storage.test.ts
git commit -m "fix(group): validate group name to prevent path traversal
Add validateGroupName() with regex [a-zA-Z0-9][a-zA-Z0-9_-]*.
Called in getGroupDir (defense in depth) which covers all CLI entry
points: create, add, remove, status, sync.
Addresses PR #626 review item 1 (HIGH).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>"
```
---
### Task 2: Directory Exclusions in Service Boundary Detector
**Files:**
- Modify: `gitnexus/src/core/group/service-boundary-detector.ts:24-51` (add constant), `:78` (walkForBoundaries), `:130` (hasSourceFilesInSubdirs)
- Test: `gitnexus/test/unit/group/service-boundary-detector.test.ts`
- [ ] **Step 1: Write failing tests for excluded directories**
Add this describe block inside the existing `detectServiceBoundaries` describe in `gitnexus/test/unit/group/service-boundary-detector.test.ts`:
```typescript
it('test_detect_skips_vendor_directory', async () => {
writeFile('services/auth/package.json', '{}');
writeFile('services/auth/src/index.ts', '');
// vendor should be skipped — its contents should not create a boundary
writeFile('vendor/some-dep/package.json', '{}');
writeFile('vendor/some-dep/src/lib.go', '');
const boundaries = await detectServiceBoundaries(tmpDir);
const paths = boundaries.map((b) => b.servicePath);
expect(paths).toContain('services/auth');
expect(paths).not.toContain('vendor/some-dep');
});
it('test_detect_skips_target_directory', async () => {
writeFile('services/api/go.mod', 'module api');
writeFile('services/api/main.go', '');
writeFile('target/classes/Main.java', '');
writeFile('target/pom.xml', '<project/>');
const boundaries = await detectServiceBoundaries(tmpDir);
const paths = boundaries.map((b) => b.servicePath);
expect(paths).toContain('services/api');
expect(paths).not.toContain('target');
});
it('test_detect_skips_pycache_directory', async () => {
writeFile('services/ml/pyproject.toml', '[project]');
writeFile('services/ml/model.py', '');
// __pycache__ with a marker + source files — would be detected as
// a boundary if not excluded, since it has package.json + .py file
writeFile('__pycache__/package.json', '{}');
writeFile('__pycache__/cached.py', '');
const boundaries = await detectServiceBoundaries(tmpDir);
const paths = boundaries.map((b) => b.servicePath);
expect(paths).toContain('services/ml');
expect(paths.every((p) => !p.includes('__pycache__'))).toBe(true);
});
it('test_detect_skips_dotfile_directories_regression', async () => {
writeFile('services/api/package.json', '{}');
writeFile('services/api/src/index.ts', '');
writeFile('.hidden/package.json', '{}');
writeFile('.hidden/src/index.ts', '');
const boundaries = await detectServiceBoundaries(tmpDir);
const paths = boundaries.map((b) => b.servicePath);
expect(paths).toContain('services/api');
expect(paths).not.toContain('.hidden');
});
it('test_detect_does_not_skip_regular_source_directories', async () => {
writeFile('services/api/package.json', '{}');
writeFile('services/api/src/index.ts', '');
const boundaries = await detectServiceBoundaries(tmpDir);
expect(boundaries).toHaveLength(1);
expect(boundaries[0].serviceName).toBe('api');
});
```
- [ ] **Step 2: Run tests to verify `vendor` and `target` tests fail**
Run: `cd gitnexus && npx vitest run test/unit/group/service-boundary-detector.test.ts`
Expected: `test_detect_skips_vendor_directory` and `test_detect_skips_target_directory` FAIL (vendor/target not excluded). Other new tests may pass since dotfile exclusion already exists.
- [ ] **Step 3: Add EXCLUDED_DIRS constant and update both walking functions**
In `gitnexus/src/core/group/service-boundary-detector.ts`:
After `SOURCE_EXTENSIONS` (after line 51), add:
```typescript
const EXCLUDED_DIRS = new Set([
'node_modules',
'vendor',
'target',
'build',
'dist',
'__pycache__',
'.venv',
'venv',
'.tox',
'.mypy_cache',
'.gradle',
'.mvn',
'out',
'bin',
]);
```
In `walkForBoundaries`, replace line 78:
```typescript
if (entry.name.startsWith('.') || entry.name === 'node_modules') continue;
```
with:
```typescript
if (entry.name.startsWith('.') || EXCLUDED_DIRS.has(entry.name)) continue;
```
In `hasSourceFilesInSubdirs`, replace line 130:
```typescript
if (entry.isDirectory() && !entry.name.startsWith('.') && entry.name !== 'node_modules') {
```
with:
```typescript
if (entry.isDirectory() && !entry.name.startsWith('.') && !EXCLUDED_DIRS.has(entry.name)) {
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `cd gitnexus && npx vitest run test/unit/group/service-boundary-detector.test.ts`
Expected: ALL PASS
- [ ] **Step 5: Commit**
```bash
cd gitnexus && git add src/core/group/service-boundary-detector.ts test/unit/group/service-boundary-detector.test.ts
git commit -m "fix(group): add directory exclusions to service boundary detector
Add EXCLUDED_DIRS set: vendor, target, build, dist, __pycache__,
.venv, venv, .tox, .mypy_cache, .gradle, .mvn, out, bin.
Applied in walkForBoundaries and hasSourceFilesInSubdirs.
Replaces inline node_modules check.
Addresses PR #626 review item 3 (HIGH).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>"
```
---
### Task 3: Remove Double-Close of LadybugDB Pools
**Files:**
- Modify: `gitnexus/src/cli/group.ts:160` (remove import), `:187-189` (remove finally block body)
- Test: `gitnexus/test/unit/group/sync.test.ts` (add pool cleanup test)
- Test: `gitnexus/test/integration/group/group-cli.test.ts` (verify no blanket close in source)
- [ ] **Step 1: Write unit tests for per-id pool cleanup in sync.ts**
Add to `gitnexus/test/unit/group/sync.test.ts`, inside the existing `describe('syncGroup', ...)`:
```typescript
it('test_syncGroup_closes_only_opened_pools', async () => {
const config = makeConfig({
'app/backend': 'backend-repo',
'app/frontend': 'frontend-repo',
});
const closedIds: string[] = [];
// Mock initLbug/closeLbug via per-repo override that tracks pool lifecycle
const { vi } = await import('vitest');
const poolAdapter = await import('../../../src/core/lbug/pool-adapter.js');
const initSpy = vi.spyOn(poolAdapter, 'initLbug').mockResolvedValue(undefined);
const closeSpy = vi.spyOn(poolAdapter, 'closeLbug').mockImplementation(async (id?: string) => {
if (id) closedIds.push(id);
});
try {
await syncGroup(config, {
resolveRepoHandle: async (_name, groupPath) => ({
id: groupPath.replace(/\//g, '-'),
path: groupPath,
repoPath: '/tmp/' + groupPath,
storagePath: '/tmp/' + groupPath + '/.gitnexus',
}),
skipWrite: true,
}).catch(() => {});
// Regardless of extraction errors, closeLbug should be called per id
// closeLbug should only receive specific pool ids, never undefined/empty
for (const id of closedIds) {
expect(id).toBeTruthy();
expect(typeof id).toBe('string');
}
// No blanket close (no-arg call)
const blanketCalls = closeSpy.mock.calls.filter((args) => args.length === 0 || !args[0]);
expect(blanketCalls).toHaveLength(0);
} finally {
initSpy.mockRestore();
closeSpy.mockRestore();
}
});
```
- [ ] **Step 2: Run sync unit test to verify it passes (sync.ts already does per-id cleanup)**
Run: `cd gitnexus && npx vitest run test/unit/group/sync.test.ts`
Expected: PASS — sync.ts already cleans up correctly. This test locks the behavior.
- [ ] **Step 3: Write test verifying CLI source has no blanket closeLbug()**
Add to `gitnexus/test/integration/group/group-cli.test.ts`:
```typescript
it('test_sync_command_source_does_not_call_blanket_closeLbug', () => {
const cliGroupPath = path.join(repoRoot, 'src', 'cli', 'group.ts');
const source = fs.readFileSync(cliGroupPath, 'utf-8');
// closeLbug() without arguments (blanket close) must not appear.
// closeLbug(id) with argument is fine (that's in sync.ts, not here).
// Match closeLbug() but not closeLbug(someArg)
const blanketClosePattern = /closeLbug\s*\(\s*\)/;
expect(source).not.toMatch(blanketClosePattern);
});
```
- [ ] **Step 4: Run test to verify it fails**
Run: `cd gitnexus && npx vitest run test/integration/group/group-cli.test.ts`
Expected: FAIL — `closeLbug()` (no args) exists at line 188.
- [ ] **Step 5: Remove blanket closeLbug() from cli/group.ts**
In `gitnexus/src/cli/group.ts`:
Remove the `closeLbug` import at line 160:
```typescript
const { closeLbug } = await import('../core/lbug/pool-adapter.js');
```
Replace the try/finally wrapper (lines 162-189):
```typescript
try {
const groupDir = getGroupDir(getDefaultGitnexusDir(), name);
const config = await loadGroupConfig(groupDir);
console.log(`Syncing group "${name}" (${Object.keys(config.repos).length} repos)...\n`);
const result = await syncGroup(config, {
groupDir,
allowStale: Boolean(opts.allowStale),
verbose: Boolean(opts.verbose),
skipEmbeddings: Boolean(opts.skipEmbeddings),
exactOnly: Boolean(opts.exactOnly),
});
if (opts.json) {
console.log(JSON.stringify(result, null, 2));
} else {
console.log(`\nMatching cascade:`);
const exactLinks = result.crossLinks.filter((l) => l.matchType === 'exact');
console.log(` exact: ${exactLinks.length} cross-links (confidence 1.0)`);
console.log(` unmatched: ${result.unmatched.length} contracts`);
console.log(
`\nWrote contracts.json (${result.contracts.length} contracts, ${result.crossLinks.length} cross-links)`,
);
}
} finally {
await closeLbug().catch(() => {});
}
```
Becomes (remove try/finally entirely, since sync.ts handles its own cleanup):
```typescript
const groupDir = getGroupDir(getDefaultGitnexusDir(), name);
const config = await loadGroupConfig(groupDir);
console.log(`Syncing group "${name}" (${Object.keys(config.repos).length} repos)...\n`);
const result = await syncGroup(config, {
groupDir,
allowStale: Boolean(opts.allowStale),
verbose: Boolean(opts.verbose),
skipEmbeddings: Boolean(opts.skipEmbeddings),
exactOnly: Boolean(opts.exactOnly),
});
if (opts.json) {
console.log(JSON.stringify(result, null, 2));
} else {
console.log(`\nMatching cascade:`);
const exactLinks = result.crossLinks.filter((l) => l.matchType === 'exact');
console.log(` exact: ${exactLinks.length} cross-links (confidence 1.0)`);
console.log(` unmatched: ${result.unmatched.length} contracts`);
console.log(
`\nWrote contracts.json (${result.contracts.length} contracts, ${result.crossLinks.length} cross-links)`,
);
}
```
- [ ] **Step 6: Run tests to verify they pass**
Run: `cd gitnexus && npx vitest run test/integration/group/group-cli.test.ts test/unit/group/sync.test.ts`
Expected: ALL PASS
- [ ] **Step 7: Commit**
```bash
cd gitnexus && git add src/cli/group.ts test/integration/group/group-cli.test.ts test/unit/group/sync.test.ts
git commit -m "fix(group): remove blanket closeLbug() from CLI sync command
sync.ts already closes pools per-id in its finally block.
The blanket closeLbug() in cli/group.ts tears down ALL active pools
including unrelated ones in MCP server context.
Addresses PR #626 review item 4 (HIGH).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>"
```
---
### Task 4: gRPC Proto Regex — Brace-Depth Counter
**Files:**
- Modify: `gitnexus/src/core/group/extractors/grpc-extractor.ts:101-130` (parseProtoFile)
- Test: `gitnexus/test/unit/group/grpc-extractor.test.ts`
- [ ] **Step 1: Write failing tests for nested braces in proto services**
Add this describe block inside the existing `proto file parsing` describe in `gitnexus/test/unit/group/grpc-extractor.test.ts`:
```typescript
it('test_extract_proto_with_google_api_http_nested_braces', async () => {
writeFile(
'api/gateway.proto',
`syntax = "proto3";
package gateway.v1;
import "google/api/annotations.proto";
service GatewayService {
rpc GetUser (GetUserRequest) returns (UserResponse) {
option (google.api.http) = {
get: "/v1/users/{user_id}"
};
}
rpc CreateUser (CreateUserRequest) returns (UserResponse) {
option (google.api.http) = {
post: "/v1/users"
body: "*"
};
}
}`,
);
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
const providers = contracts.filter(
(c) => c.role === 'provider' && c.symbolRef.filePath === 'api/gateway.proto',
);
expect(providers).toHaveLength(2);
const ids = providers.map((c) => c.contractId).sort();
expect(ids).toEqual([
'grpc::gateway.v1.GatewayService/CreateUser',
'grpc::gateway.v1.GatewayService/GetUser',
]);
});
it('test_extract_proto_with_multiple_services', async () => {
writeFile(
'api/multi.proto',
`syntax = "proto3";
package multi;
service ServiceA {
rpc MethodA (Req) returns (Res);
}
service ServiceB {
rpc MethodB1 (Req) returns (Res);
rpc MethodB2 (Req) returns (Res);
}`,
);
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
const providers = contracts.filter(
(c) => c.role === 'provider' && c.symbolRef.filePath === 'api/multi.proto',
);
expect(providers).toHaveLength(3);
const ids = providers.map((c) => c.contractId).sort();
expect(ids).toEqual([
'grpc::multi.ServiceA/MethodA',
'grpc::multi.ServiceB/MethodB1',
'grpc::multi.ServiceB/MethodB2',
]);
});
it('test_extract_proto_with_nested_option_blocks_in_rpc', async () => {
writeFile(
'api/nested.proto',
`syntax = "proto3";
package nested;
service DeepService {
rpc DeepMethod (Req) returns (Res) {
option (google.api.http) = {
post: "/v1/deep"
body: "*"
additional_bindings {
get: "/v1/deep/{id}"
}
};
}
}`,
);
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
const providers = contracts.filter(
(c) => c.role === 'provider' && c.symbolRef.filePath === 'api/nested.proto',
);
expect(providers).toHaveLength(1);
expect(providers[0].contractId).toBe('grpc::nested.DeepService/DeepMethod');
});
it('test_extract_proto_malformed_unclosed_brace_skips_service', async () => {
writeFile(
'api/broken.proto',
`syntax = "proto3";
package broken;
service IncompleteService {
rpc SomeMethod (Req) returns (Res);
// Missing closing brace — EOF before depth returns to 0
`,
);
// Should not throw; incomplete service is silently skipped
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
const providers = contracts.filter(
(c) => c.role === 'provider' && c.symbolRef.filePath === 'api/broken.proto',
);
// The old regex would find partial match; the new parser should skip it
expect(providers).toHaveLength(0);
});
```
- [ ] **Step 2: Run tests to verify the nested brace test fails**
Run: `cd gitnexus && npx vitest run test/unit/group/grpc-extractor.test.ts`
Expected: `test_extract_proto_with_google_api_http_nested_braces` FAIL — regex stops at first `}` inside the `option` block.
- [ ] **Step 3: Replace serviceRe regex with extractServiceBlocks function**
In `gitnexus/src/core/group/extractors/grpc-extractor.ts`, replace the `parseProtoFile` method (lines 101-130):
```typescript
private parseProtoFile(content: string, filePath: string): ExtractedContract[] {
const out: ExtractedContract[] = [];
const pkgMatch = content.match(/^package\s+([\w.]+)\s*;/m);
const pkg = pkgMatch ? pkgMatch[1] : '';
for (const { name: serviceName, body } of extractServiceBlocks(content)) {
const rpcRe = /rpc\s+(\w+)\s*\(/g;
let rpcMatch: RegExpExecArray | null;
while ((rpcMatch = rpcRe.exec(body)) !== null) {
const methodName = rpcMatch[1];
const cid = contractId(pkg, serviceName, methodName);
out.push(
makeContract(cid, 'provider', filePath, `${serviceName}.${methodName}`, 0.85, {
package: pkg,
service: serviceName,
method: methodName,
source: 'proto',
}),
);
}
}
return out;
}
```
Add this function before the class (e.g. after `serviceOnlyContractId`, around line 26):
```typescript
function extractServiceBlocks(content: string): Array<{ name: string; body: string }> {
const results: Array<{ name: string; body: string }> = [];
const headerRe = /service\s+(\w+)\s*\{/g;
let headerMatch: RegExpExecArray | null;
while ((headerMatch = headerRe.exec(content)) !== null) {
const serviceName = headerMatch[1];
const bodyStart = headerMatch.index + headerMatch[0].length;
let depth = 1;
let pos = bodyStart;
while (pos < content.length && depth > 0) {
const ch = content[pos];
if (ch === '{') depth++;
else if (ch === '}') depth--;
pos++;
}
// If EOF before depth returns to 0, skip incomplete service
if (depth !== 0) continue;
// body is between opening { (consumed by regex) and closing } (pos is one past it)
const body = content.slice(bodyStart, pos - 1);
results.push({ name: serviceName, body });
}
return results;
}
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `cd gitnexus && npx vitest run test/unit/group/grpc-extractor.test.ts`
Expected: ALL PASS (including existing regression tests)
- [ ] **Step 5: Commit**
```bash
cd gitnexus && git add src/core/group/extractors/grpc-extractor.ts test/unit/group/grpc-extractor.test.ts
git commit -m "fix(group): replace gRPC proto regex with brace-depth counter
The serviceRe regex used [^}]* which stopped at the first '}'.
Proto services with google.api.http annotations contain nested {}
blocks, causing methods to be missed.
New extractServiceBlocks() uses a brace-depth counter (init depth=1
after opening {, scan char-by-char). Malformed protos with unclosed
braces are silently skipped.
Addresses PR #626 review item 2 (HIGH).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>"
```
---
### Task 5: Run Full Test Suite
- [ ] **Step 1: Run all group-related tests**
Run: `cd gitnexus && npx vitest run test/unit/group/ test/integration/group/`
Expected: ALL PASS
- [ ] **Step 2: Run full test suite to catch regressions**
Run: `cd gitnexus && npx vitest run`
Expected: ALL PASS, 0 failures
- [ ] **Step 3: Run typecheck**
Run: `cd gitnexus && npx tsc --noEmit`
Expected: No errors
---
### Task 6: CLI Integration Smoke Test
- [ ] **Step 1: Add CLI smoke test for path traversal**
Add to `gitnexus/test/integration/group/group-cli.test.ts` inside the existing `group CLI` describe:
```typescript
it('test_create_with_invalid_name_fails', () => {
const result = runGroup(['create', '../../evil']);
expect(result.status).not.toBe(0);
expect(result.stderr).toContain('Invalid group name');
});
```
- [ ] **Step 2: Run test**
Run: `cd gitnexus && npx vitest run test/integration/group/group-cli.test.ts`
Expected: ALL PASS
- [ ] **Step 3: Commit**
```bash
cd gitnexus && git add test/integration/group/group-cli.test.ts
git commit -m "test(group): add CLI smoke test for path traversal rejection
Verifies that 'group create ../../evil' fails with Invalid group name.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>"
```
@@ -1,175 +0,0 @@
# PR #626 HIGH-Priority Fixes Design
**Date:** 2026-04-02
**PR:** abhigyanpatwari/GitNexus#626 — Intra-repo service communication tracking
**Scope:** 4 HIGH-priority issues identified by abhigyanpatwari and xkonjin
**Approach:** Minimal targeted fixes (option A) — no refactoring, no scope creep
---
## Fix 1: Path Traversal via Group Name
**File:** `gitnexus/src/core/group/storage.ts`
**Risk:** A group name like `../../etc` creates directories outside the intended path.
### Solution
Add `validateGroupName(name: string): void` that enforces `/^[a-zA-Z0-9][a-zA-Z0-9_-]*$/`.
- Call in `createGroupDir` (primary entry point)
- Call in `getGroupDir` (defense in depth)
- Throw descriptive error on invalid names
**Legacy:** Groups already on disk with names outside this pattern are not auto-renamed; only new `create` / resolved paths are validated.
### Why regex over path.resolve + startsWith
- abhigyanpatwari explicitly requested `[a-zA-Z0-9_-]`
- Stricter: disallows spaces, dots, Unicode edge cases
- Simpler to reason about
### Tests
- `../../evil` throws
- `foo/bar` throws
- Empty string throws
- `my-group_01` passes
- `A` (single char) passes
- CLI smoke: one integration test that hits `getGroupDir` / `createGroupDir` (e.g. `group create` or `group add`) with an invalid name proves wiring for every subcommand that resolves a group through storage
### CLI/API entry points accepting groupName
All paths flow through `getGroupDir` (which validates), so coverage is implicit. For reference:
| Command | Entry | Calls |
|---------|-------|-------|
| `group create` | `cli/group.ts` action | `createGroupDir` -> `getGroupDir` |
| `group add` | `cli/group.ts` action | `getGroupDir` |
| `group remove` | `cli/group.ts` action | `getGroupDir` |
| `group list` | `cli/group.ts` action | reads `groups/` dir directly — no traversal risk (reads, not writes) |
| `group status` | `cli/group.ts` action | `getGroupDir` |
| `group sync` | `cli/group.ts` action | `getGroupDir` |
**`listGroups`:** Reads directory names from disk without validation. Not a write path, so no traversal risk. May surface manually-created directories with non-conforming names — accepted as-is, not in scope.
---
## Fix 2: gRPC Proto Regex -> Brace-Depth Counter
**File:** `gitnexus/src/core/group/extractors/grpc-extractor.ts`
**Risk:** `serviceRe = /service\s+(\w+)\s*\{([^}]*)}/gs` stops at first `}`. Proto services with `google.api.http` annotations inside RPCs contain nested `{ }` blocks.
### Solution
Replace `serviceRe` regex with `extractServiceBlocks(content: string): Array<{ name: string; body: string }>`:
1. Use regex only to find `service <Name> {` start positions (regex consumes the opening `{`)
2. Initialise depth to 1 immediately after the opening `{`
3. Scan forward char by char: `{` -> depth++, `}` -> depth--; collect into body
4. Stop when depth reaches 0 (the matching closing `}`)
5. Return name + body pairs
Inner `rpcRe` regex remains unchanged — it operates on the already-extracted body.
**Malformed input:** If EOF is reached before `depth` returns to 0, skip the incomplete service (do not add to results). Lock this in the test.
**Scope limitation (v1):** Brace-depth only — no lexer for string literals or comments containing `{`/`}`. Sufficient for `google.api.http` annotations. Known false positive: braces inside `//` comments or quoted strings within proto options. Accepted for v1; a proper proto lexer is out of scope.
### Tests
- Proto with single service, no nesting (regression)
- Proto with `google.api.http` nested braces inside RPC options
- Proto with multiple services
- Proto with nested `option` blocks inside RPC (e.g. `google.api.http`)
- Malformed proto with unclosed brace (graceful handling)
---
## Fix 3: Directory Exclusions in Service Boundary Detector
**File:** `gitnexus/src/core/group/service-boundary-detector.ts`
**Risk:** Walks entire repo tree, only skipping dotfiles and `node_modules`. Extremely slow on repos with `vendor/`, `target/`, `__pycache__/`, `.venv/`.
### Solution
Create `EXCLUDED_DIRS` as a `Set<string>` (alongside existing `SERVICE_MARKERS`, `SOURCE_EXTENSIONS`), for example:
```text
node_modules, vendor, target, build, dist,
__pycache__, .venv, venv, .tox, .mypy_cache,
.gradle, .mvn, out, bin
```
(Implement as `new Set([...])` — the list above is the membership, not a string literal.)
Apply in both:
- `walkForBoundaries` (line 77-78) — replace current inline `=== 'node_modules'` check with `EXCLUDED_DIRS.has(entry.name)`
- `hasSourceFilesInSubdirs` (line 130) — replace `entry.name !== 'node_modules'` with `!EXCLUDED_DIRS.has(entry.name)`
Note: remove the old `=== 'node_modules'` literal from both locations — it is covered by `EXCLUDED_DIRS`.
Dotfile exclusion (`.` prefix) remains as a separate check since it's a pattern, not a name.
Exclusions apply only to `isDirectory()` entries — file names are never checked against `EXCLUDED_DIRS`.
**Tradeoff:** Rare layouts that keep source under names like `out/` or `bin/` will be skipped; accepted for performance on typical monorepos.
**Case sensitivity:** `Set.has` is case-sensitive (matches current `=== 'node_modules'` behavior). Windows case-insensitive FS not handled — accepted as-is, consistent with existing code.
### Tests
- Directory named `vendor/` is skipped
- Directory named `target/` is skipped
- Directory named `__pycache__/` is skipped
- Regular source directories are NOT skipped
- Dotfile directories still skipped (regression)
---
## Fix 4: Double-Close of LadybugDB Pools
**Files:**
- `gitnexus/src/core/group/sync.ts` (lines 155-157) — per-id cleanup (KEEP)
- `gitnexus/src/cli/group.ts` (line 188) — blanket `closeLbug()` (REMOVE)
**Risk:** In MCP server context, `closeLbug()` without arguments tears down ALL active pools, including ones from unrelated operations.
### Solution
Remove the `closeLbug()` call (no arguments) from `cli/group.ts` finally block. The per-id cleanup in `sync.ts` is sufficient:
```typescript
// sync.ts — KEEP: cleans up only pools opened by this sync
finally {
for (const id of [...new Set(openPoolIds)]) {
await closeLbug(id).catch(() => {});
}
}
```
```typescript
// cli/group.ts — REMOVE: blanket close that kills all pools
finally {
await closeLbug().catch(() => {}); // DELETE THIS
}
```
Remove the `closeLbug` import from `cli/group.ts` — after removing the `finally` call it has no remaining usages.
### Tests (unit level — mock pool adapter)
- `syncGroup` closes only the pools it opened (mock `closeLbug`, assert called with specific ids)
- Two-pool scenario: sync opens pools A and B, both closed in finally; pool C (opened elsewhere) not touched
- CLI `sync` command does not call blanket `closeLbug()` (verify no zero-arg call in source — static check or grep-based test)
---
## Out of Scope
- JSON -> LadybugDB migration (tracked in #606)
- MEDIUM/LOW issues (items 5-10 from review summary)
- Test gap coverage beyond what's needed for these 4 fixes
- Any refactoring or architectural changes
## Execution Order
Fixes are independent — can be implemented in parallel or any order.
Recommended order for review clarity: 1 -> 3 -> 4 -> 2 (simplest to most complex).
-83
View File
@@ -1,83 +0,0 @@
import tsPlugin from '@typescript-eslint/eslint-plugin';
import tsParser from '@typescript-eslint/parser';
import unusedImports from 'eslint-plugin-unused-imports';
import reactHooks from 'eslint-plugin-react-hooks';
import prettierConfig from 'eslint-config-prettier';
export default [
// Global ignores
{
ignores: [
'**/dist/**',
'**/node_modules/**',
'**/coverage/**',
'gitnexus/vendor/**',
'gitnexus-web/src/vendor/**',
'gitnexus/test/fixtures/**',
'gitnexus-web/playwright-report/**',
'gitnexus-web/test-results/**',
'**/*.d.ts',
'.claude/**',
'.history/**',
],
},
// Base TypeScript config for all packages
{
files: ['**/*.{ts,tsx}'],
languageOptions: {
parser: tsParser,
parserOptions: {
ecmaVersion: 2022,
sourceType: 'module',
},
},
plugins: {
'@typescript-eslint': tsPlugin,
'unused-imports': unusedImports,
},
rules: {
// Unused imports — auto-fixable
'unused-imports/no-unused-imports': 'error',
'unused-imports/no-unused-vars': [
'warn',
{ vars: 'all', varsIgnorePattern: '^_', args: 'after-used', argsIgnorePattern: '^_' },
],
// TypeScript quality
'@typescript-eslint/no-unused-vars': 'off', // handled by unused-imports plugin
'no-unused-vars': 'off', // handled by unused-imports plugin
'@typescript-eslint/no-explicit-any': 'warn',
'@typescript-eslint/no-non-null-assertion': 'warn',
// General quality
'no-debugger': 'error',
'prefer-const': 'error',
'no-var': 'error',
eqeqeq: ['error', 'always', { null: 'ignore' }],
},
},
// CLI package — allow console.log (it's a CLI tool)
{
files: ['gitnexus/src/cli/**/*.ts', 'gitnexus/src/server/**/*.ts'],
rules: {
'no-console': 'off',
},
},
// React-specific rules for gitnexus-web
{
files: ['gitnexus-web/src/**/*.{ts,tsx}'],
plugins: {
'react-hooks': reactHooks,
},
rules: {
'react-hooks/rules-of-hooks': 'error',
'react-hooks/exhaustive-deps': 'warn',
},
},
// Disable formatting rules (prettier handles those)
prettierConfig,
];
-4
View File
@@ -50,10 +50,6 @@ All models are routed through **OpenRouter** by default, so a single `OPENROUTER
docker pull swebench/sweb.eval.x86_64.django_1776_django-16527:latest
```
### Debug logging
Set `GITNEXUS_EVAL_DEBUG=1` to include full Python tracebacks in run summaries and logs. By default, errors are sanitized to avoid leaking host paths or stack traces.
## Quick Start
### Debug a single instance
+18 -8
View File
@@ -22,10 +22,8 @@ import time
from enum import Enum
from pathlib import Path
from constants import AUGMENT_TIMEOUT_SECONDS
from minisweagent import Environment, Model
from minisweagent.agents.default import AgentConfig, DefaultAgent
from tool_registry import BINARIES_BY_KEY, TOOL_METRIC_KEYS
logger = logging.getLogger("gitnexus_agent")
@@ -42,7 +40,7 @@ class GitNexusMode(str, Enum):
class GitNexusAgentConfig(AgentConfig):
"""Extended config for GitNexus evaluation agent."""
gitnexus_mode: GitNexusMode = GitNexusMode.BASELINE
augment_timeout: float = AUGMENT_TIMEOUT_SECONDS
augment_timeout: float = 5.0
augment_min_pattern_length: int = 3
track_gitnexus_usage: bool = True
@@ -154,10 +152,16 @@ class GitNexusAgent(DefaultAgent):
"""Track which GitNexus tools the agent uses."""
for action in message.get("extra", {}).get("actions", []):
command = action.get("command", "")
for key, binary in BINARIES_BY_KEY.items():
if binary in command and key in self.gitnexus_metrics.tool_calls:
self.gitnexus_metrics.tool_calls[key] += 1
break
if "gitnexus-query" in command:
self.gitnexus_metrics.tool_calls["query"] += 1
elif "gitnexus-context" in command:
self.gitnexus_metrics.tool_calls["context"] += 1
elif "gitnexus-impact" in command:
self.gitnexus_metrics.tool_calls["impact"] += 1
elif "gitnexus-cypher" in command:
self.gitnexus_metrics.tool_calls["cypher"] += 1
elif "gitnexus-overview" in command:
self.gitnexus_metrics.tool_calls["overview"] += 1
def serialize(self, *extra_dicts) -> dict:
"""Serialize with GitNexus-specific metrics."""
@@ -176,7 +180,13 @@ class GitNexusMetrics:
"""Tracks GitNexus-specific metrics during evaluation."""
def __init__(self):
self.tool_calls: dict[str, int] = {key: 0 for key in TOOL_METRIC_KEYS}
self.tool_calls: dict[str, int] = {
"query": 0,
"context": 0,
"impact": 0,
"cypher": 0,
"overview": 0,
}
self.augmentation_calls: int = 0
self.augmentation_hits: int = 0
self.augmentation_errors: int = 0
+22 -31
View File
@@ -27,8 +27,6 @@ import typer
from rich.console import Console
from rich.table import Table
from tool_registry import TOOL_METRIC_KEYS
logger = logging.getLogger("analyze_results")
console = Console()
app = typer.Typer(rich_markup_mode="rich", add_completion=False)
@@ -79,20 +77,13 @@ def load_run_results(results_dir: Path) -> dict[str, dict]:
def parse_run_id(run_id: str) -> tuple[str, str]:
"""Parse 'model_mode' into (model, mode) using known suffixes."""
# Match the longest known suffix first to avoid hyphen collisions in model names.
known_modes = [
"native_augment",
"native",
"baseline",
"mcp",
"augment",
"full",
]
for mode in known_modes:
suffix = f"_{mode}"
if run_id.endswith(suffix):
return run_id[: -len(suffix)], mode
"""Parse 'model_mode' into (model, mode)."""
# Handle multi-word model names like 'minimax-2.5'
# Modes are: baseline, mcp, augment, full
known_modes = {"baseline", "mcp", "augment", "full"}
parts = run_id.rsplit("_", 1)
if len(parts) == 2 and parts[1] in known_modes:
return parts[0], parts[1]
return run_id, "unknown"
@@ -275,17 +266,12 @@ def compare_modes(
_, mode = parse_run_id(run_id)
metrics[mode] = compute_metrics(run_data)
mode_order = [
mode
for mode in ["baseline", "native", "native_augment", "mcp", "augment", "full"]
if mode in metrics
] or sorted(metrics.keys())
# Print comparison table
table = Table(title=f"Mode Comparison: {model}")
table.add_column("Metric", style="bold")
for mode in mode_order:
table.add_column(mode, justify="right")
for mode in ["baseline", "mcp", "augment", "full"]:
if mode in metrics:
table.add_column(mode, justify="right")
rows = [
("Instances", "n_instances", "d"),
@@ -302,7 +288,7 @@ def compare_modes(
for label, key, fmt in rows:
values = []
for mode in mode_order:
for mode in ["baseline", "mcp", "augment", "full"]:
if mode in metrics:
v = metrics[mode].get(key, 0)
if fmt == ".1%":
@@ -321,8 +307,8 @@ def compare_modes(
baseline_calls = metrics["baseline"]["avg_api_calls"]
table.add_section()
for mode in mode_order:
if mode == "baseline":
for mode in ["mcp", "augment", "full"]:
if mode not in metrics:
continue
mode_cost = metrics[mode]["avg_cost"]
mode_calls = metrics[mode]["avg_api_calls"]
@@ -354,8 +340,10 @@ def gitnexus_usage(
table = Table(title="Tool Usage by Run")
table.add_column("Run", style="bold")
for key in TOOL_METRIC_KEYS:
table.add_column(key, justify="right")
table.add_column("query", justify="right")
table.add_column("context", justify="right")
table.add_column("impact", justify="right")
table.add_column("cypher", justify="right")
table.add_column("Total", justify="right")
table.add_column("Augment Hits", justify="right")
@@ -365,7 +353,7 @@ def gitnexus_usage(
continue
# Aggregate tool calls across trajectories
tool_totals: dict[str, int] = {key: 0 for key in TOOL_METRIC_KEYS}
tool_totals: dict[str, int] = {"query": 0, "context": 0, "impact": 0, "cypher": 0, "overview": 0}
augment_hits = 0
for traj in run_data.get("trajectories", {}).values():
@@ -385,7 +373,10 @@ def gitnexus_usage(
if total > 0 or augment_hits > 0:
table.add_row(
run_id,
*[str(tool_totals.get(key, 0)) for key in TOOL_METRIC_KEYS],
str(tool_totals.get("query", 0)),
str(tool_totals.get("context", 0)),
str(tool_totals.get("impact", 0)),
str(tool_totals.get("cypher", 0)),
str(total),
str(augment_hits),
)
+35 -77
View File
@@ -17,14 +17,6 @@ import time
from pathlib import Path
from typing import Any
from constants import (
MCP_FIND_GITNEXUS_FALLBACK_TIMEOUT_SECONDS,
MCP_FIND_GITNEXUS_TIMEOUT_SECONDS,
MCP_READ_TIMEOUT_SECONDS,
MCP_STOP_WAIT_SECONDS,
)
from utils.errors import is_debug_enabled, log_safe_exception
logger = logging.getLogger("mcp_bridge")
@@ -86,7 +78,7 @@ class MCPBridge:
return True
except Exception as e:
log_safe_exception(logger, "Failed to start MCP bridge", e, include_debug=is_debug_enabled())
logger.error(f"Failed to start MCP bridge: {e}")
self.stop()
return False
@@ -94,14 +86,9 @@ class MCPBridge:
"""Stop the MCP server subprocess."""
if self.process:
try:
if self.process.stdin:
self.process.stdin.close()
if self.process.stdout:
self.process.stdout.close()
if self.process.stderr:
self.process.stderr.close()
self.process.stdin.close()
self.process.terminate()
self.process.wait(timeout=MCP_STOP_WAIT_SECONDS)
self.process.wait(timeout=5)
except Exception:
try:
self.process.kill()
@@ -159,9 +146,7 @@ class MCPBridge:
try:
result = subprocess.run(
[cmd, "gitnexus", "--version"],
capture_output=True,
text=True,
timeout=MCP_FIND_GITNEXUS_TIMEOUT_SECONDS,
capture_output=True, text=True, timeout=15,
cwd=self.repo_path,
)
if result.returncode == 0:
@@ -173,9 +158,7 @@ class MCPBridge:
try:
result = subprocess.run(
["gitnexus", "--version"],
capture_output=True,
text=True,
timeout=MCP_FIND_GITNEXUS_FALLBACK_TIMEOUT_SECONDS,
capture_output=True, text=True, timeout=10,
)
if result.returncode == 0:
return "gitnexus"
@@ -211,7 +194,7 @@ class MCPBridge:
self.process.stdin.flush()
# Read response
response = self._read_response(timeout=MCP_READ_TIMEOUT_SECONDS)
response = self._read_response(timeout=30)
if response and response.get("id") == request_id:
if "error" in response:
logger.error(f"MCP error: {response['error']}")
@@ -220,7 +203,7 @@ class MCPBridge:
return None
except Exception as e:
log_safe_exception(logger, "MCP request failed", e, include_debug=is_debug_enabled())
logger.error(f"MCP request failed: {e}")
return None
def _send_notification(self, method: str, params: dict):
@@ -241,67 +224,42 @@ class MCPBridge:
self.process.stdin.write(message.encode("utf-8"))
self.process.stdin.flush()
except Exception as e:
log_safe_exception(logger, "MCP notification failed", e, include_debug=is_debug_enabled())
logger.error(f"MCP notification failed: {e}")
def _read_content_length(self, deadline: float) -> int | None:
"""Read Content-Length header, returning the byte length or None."""
if not self.process or not self.process.stdout:
return None
header_line = b""
while time.time() < deadline:
byte = self.process.stdout.read(1)
if not byte:
return None
header_line += byte
if header_line.endswith(b"\r\n\r\n") or header_line.endswith(b"\n\n"):
break
if not header_line:
return None
header_str = header_line.decode("utf-8").strip()
for line in header_str.split("\r\n"):
if line.lower().startswith("content-length:"):
try:
return int(line.split(":", 1)[1].strip())
except (ValueError, IndexError):
return None
return None
def _read_body(self, content_length: int, deadline: float) -> bytes | None:
"""Read a response body of the expected length before deadline."""
if not self.process or not self.process.stdout:
return None
remaining = content_length
chunks: list[bytes] = []
while remaining > 0 and time.time() < deadline:
chunk = self.process.stdout.read(remaining)
if not chunk:
return None
chunks.append(chunk)
remaining -= len(chunk)
if remaining > 0:
return None
return b"".join(chunks)
def _read_response(self, timeout: float = MCP_READ_TIMEOUT_SECONDS) -> dict | None:
def _read_response(self, timeout: float = 30) -> dict | None:
"""Read a JSON-RPC response from the MCP server."""
if not self.process or not self.process.stdout:
return None
start = time.time()
try:
deadline = time.time() + timeout
while time.time() < deadline:
content_length = self._read_content_length(deadline)
while time.time() - start < timeout:
# Read Content-Length header
header_line = b""
while True:
byte = self.process.stdout.read(1)
if not byte:
return None
header_line += byte
if header_line.endswith(b"\r\n\r\n"):
break
if header_line.endswith(b"\n\n"):
break
# Parse content length
header_str = header_line.decode("utf-8").strip()
content_length = None
for line in header_str.split("\r\n"):
if line.lower().startswith("content-length:"):
content_length = int(line.split(":")[1].strip())
break
if content_length is None:
continue
body = self._read_body(content_length, deadline)
# Read body
body = self.process.stdout.read(content_length)
if not body:
return None
@@ -314,7 +272,7 @@ class MCPBridge:
return None
except Exception as e:
log_safe_exception(logger, "Error reading MCP response", e, include_debug=is_debug_enabled())
logger.error(f"Error reading MCP response: {e}")
return None
+2 -2
View File
@@ -1,8 +1,8 @@
# Claude Haiku 4.5 — fast, cheap, good baseline
# Via OpenRouter (set OPENROUTER_API_KEY in .env)
model:
model_name: 'openrouter/anthropic/claude-haiku-4.5'
cost_tracking: 'ignore_errors'
model_name: "openrouter/anthropic/claude-haiku-4.5"
cost_tracking: "ignore_errors"
model_kwargs:
max_tokens: 8192
temperature: 0
+2 -2
View File
@@ -2,8 +2,8 @@
# Via OpenRouter (set OPENROUTER_API_KEY in .env)
# To use Anthropic directly, change to: anthropic/claude-opus-4-20250514
model:
model_name: 'openrouter/anthropic/claude-opus-4'
cost_tracking: 'ignore_errors'
model_name: "openrouter/anthropic/claude-opus-4"
cost_tracking: "ignore_errors"
model_kwargs:
max_tokens: 16384
temperature: 0
+2 -2
View File
@@ -2,8 +2,8 @@
# Via OpenRouter (set OPENROUTER_API_KEY in .env)
# To use Anthropic directly, change to: anthropic/claude-sonnet-4-20250514
model:
model_name: 'openrouter/anthropic/claude-sonnet-4'
cost_tracking: 'ignore_errors'
model_name: "openrouter/anthropic/claude-sonnet-4"
cost_tracking: "ignore_errors"
model_kwargs:
max_tokens: 16384
temperature: 0
+3 -3
View File
@@ -1,9 +1,9 @@
model: deepseek-ai/deepseek-chat
provider: openrouter
cost:
input: 0.14 # per 1M tokens
output: 0.28 # per 1M tokens
input: 0.14 # per 1M tokens
output: 0.28 # per 1M tokens
# Native DeepSeek API (direct)
api_key: null
base_url: null
+3 -3
View File
@@ -1,9 +1,9 @@
model: deepseek-ai/DeepSeek-V3
provider: openrouter
cost:
input: 0.27 # per 1M tokens
output: 1.10 # per 1M tokens
input: 0.27 # per 1M tokens
output: 1.10 # per 1M tokens
# Native DeepSeek API (direct)
# Get your API key at: https://platform.deepseek.com/
# Or use OpenRouter with: OPENROUTER_API_KEY
+2 -2
View File
@@ -1,7 +1,7 @@
# GLM 4.7 — via OpenRouter (set OPENROUTER_API_KEY in .env)
model:
model_name: 'openrouter/zhipuai/glm-4.7'
cost_tracking: 'ignore_errors'
model_name: "openrouter/zhipuai/glm-4.7"
cost_tracking: "ignore_errors"
model_kwargs:
max_tokens: 8192
temperature: 0
+2 -2
View File
@@ -1,7 +1,7 @@
# GLM 5 — via OpenRouter (set OPENROUTER_API_KEY in .env)
model:
model_name: 'openrouter/zhipuai/glm-5'
cost_tracking: 'ignore_errors'
model_name: "openrouter/zhipuai/glm-5"
cost_tracking: "ignore_errors"
model_kwargs:
max_tokens: 8192
temperature: 0
+2 -2
View File
@@ -1,7 +1,7 @@
# MiniMax M1 2.5 — via OpenRouter (set OPENROUTER_API_KEY in .env)
model:
model_name: 'openrouter/minimax/minimax-m1-2.5'
cost_tracking: 'ignore_errors'
model_name: "openrouter/minimax/minimax-m1-2.5"
cost_tracking: "ignore_errors"
model_kwargs:
max_tokens: 8192
temperature: 0
+2 -2
View File
@@ -3,9 +3,9 @@
# The action_regex tells mini-swe-agent to parse ```bash blocks from responses.
model:
model_class: litellm_textbased
model_name: 'openrouter/minimax/minimax-m2.5'
model_name: "openrouter/minimax/minimax-m2.5"
action_regex: "```(?:bash|mswea_bash_command)\\s*\\n(.*?)\\n```"
cost_tracking: 'ignore_errors'
cost_tracking: "ignore_errors"
model_kwargs:
max_tokens: 8192
temperature: 0
+3 -3
View File
@@ -1,9 +1,9 @@
# Baseline mode — no GitNexus, pure mini-swe-agent (control group)
agent:
agent_class: 'eval.agents.gitnexus_agent.GitNexusAgent'
gitnexus_mode: 'baseline'
agent_class: "eval.agents.gitnexus_agent.GitNexusAgent"
gitnexus_mode: "baseline"
step_limit: 30
cost_limit: 3.0
environment:
environment_class: 'docker'
environment_class: "docker"
+3 -3
View File
@@ -5,14 +5,14 @@
#
# Use this mode to isolate the value of explicit tools without grep augmentation.
agent:
agent_class: 'eval.agents.gitnexus_agent.GitNexusAgent'
gitnexus_mode: 'native'
agent_class: "eval.agents.gitnexus_agent.GitNexusAgent"
gitnexus_mode: "native"
step_limit: 30
cost_limit: 3.0
track_gitnexus_usage: true
environment:
environment_class: 'eval.environments.gitnexus_docker.GitNexusDockerEnvironment'
environment_class: "eval.environments.gitnexus_docker.GitNexusDockerEnvironment"
enable_gitnexus: true
skip_embeddings: true
gitnexus_timeout: 120
+3 -3
View File
@@ -8,8 +8,8 @@
#
# The agent decides when to use explicit tools vs rely on enriched grep results.
agent:
agent_class: 'eval.agents.gitnexus_agent.GitNexusAgent'
gitnexus_mode: 'native_augment'
agent_class: "eval.agents.gitnexus_agent.GitNexusAgent"
gitnexus_mode: "native_augment"
step_limit: 30
cost_limit: 3.0
augment_timeout: 5.0
@@ -17,7 +17,7 @@ agent:
track_gitnexus_usage: true
environment:
environment_class: 'eval.environments.gitnexus_docker.GitNexusDockerEnvironment'
environment_class: "eval.environments.gitnexus_docker.GitNexusDockerEnvironment"
enable_gitnexus: true
skip_embeddings: true
gitnexus_timeout: 120
-15
View File
@@ -1,15 +0,0 @@
DEBUG_ENV_VAR = "GITNEXUS_EVAL_DEBUG"
# GitNexus eval-server health checks
EVAL_SERVER_HEALTH_RETRIES = 30
EVAL_SERVER_HEALTH_INTERVAL_SECONDS = 0.5
EVAL_SERVER_HEALTH_TIMEOUT_SECONDS = 3
# MCP bridge timeouts
MCP_FIND_GITNEXUS_TIMEOUT_SECONDS = 15
MCP_FIND_GITNEXUS_FALLBACK_TIMEOUT_SECONDS = 10
MCP_READ_TIMEOUT_SECONDS = 30
MCP_STOP_WAIT_SECONDS = 5
# Agent defaults
AUGMENT_TIMEOUT_SECONDS = 5.0
+81 -75
View File
@@ -26,20 +26,72 @@ import shutil
import time
from pathlib import Path
from constants import (
EVAL_SERVER_HEALTH_INTERVAL_SECONDS,
EVAL_SERVER_HEALTH_RETRIES,
EVAL_SERVER_HEALTH_TIMEOUT_SECONDS,
)
from minisweagent.environments.docker import DockerEnvironment
from tool_registry import TOOL_SPECS, ToolScriptSpec
from utils.errors import is_debug_enabled, log_safe_exception
logger = logging.getLogger("gitnexus_docker")
DEFAULT_CACHE_DIR = Path.home() / ".gitnexus-eval-cache"
EVAL_SERVER_PORT = 4848
# Standalone tool scripts installed into /usr/local/bin/ inside the container.
# Each script calls the eval-server via curl, with a CLI fallback.
# These are standalone — no sourcing, no env inheritance needed.
TOOL_SCRIPT_QUERY = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
query="$1"; task_ctx="${2:-}"; goal="${3:-}"
[ -z "$query" ] && echo "Usage: gitnexus-query <query> [task_context] [goal]" && exit 1
args="{\"query\": \"$query\""
[ -n "$task_ctx" ] && args="$args, \"task_context\": \"$task_ctx\""
[ -n "$goal" ] && args="$args, \"goal\": \"$goal\""
args="$args}"
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/query" -H "Content-Type: application/json" -d "$args" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus query "$query" 2>&1
'''
TOOL_SCRIPT_CONTEXT = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
name="$1"; file_path="${2:-}"
[ -z "$name" ] && echo "Usage: gitnexus-context <symbol_name> [file_path]" && exit 1
args="{\"name\": \"$name\""
[ -n "$file_path" ] && args="$args, \"file_path\": \"$file_path\""
args="$args}"
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/context" -H "Content-Type: application/json" -d "$args" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus context "$name" 2>&1
'''
TOOL_SCRIPT_IMPACT = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
target="$1"; direction="${2:-upstream}"
[ -z "$target" ] && echo "Usage: gitnexus-impact <symbol_name> [upstream|downstream]" && exit 1
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/impact" -H "Content-Type: application/json" -d "{\"target\": \"$target\", \"direction\": \"$direction\"}" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus impact "$target" --direction "$direction" 2>&1
'''
TOOL_SCRIPT_CYPHER = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
query="$1"
[ -z "$query" ] && echo "Usage: gitnexus-cypher <cypher_query>" && exit 1
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/cypher" -H "Content-Type: application/json" -d "{\"query\": \"$query\"}" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus cypher "$query" 2>&1
'''
TOOL_SCRIPT_AUGMENT = r'''#!/bin/bash
cd /testbed && npx gitnexus augment "$1" 2>&1 || true
'''
TOOL_SCRIPT_OVERVIEW = r'''#!/bin/bash
PORT="${GITNEXUS_EVAL_PORT:-__PORT__}"
echo "=== Code Knowledge Graph Overview ==="
result=$(curl -sf -X POST "http://127.0.0.1:${PORT}/tool/list_repos" -H "Content-Type: application/json" -d "{}" 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi
cd /testbed && npx gitnexus list 2>&1
'''
class GitNexusDockerEnvironment(DockerEnvironment):
"""
@@ -81,13 +133,7 @@ class GitNexusDockerEnvironment(DockerEnvironment):
try:
self._setup_gitnexus()
except Exception as e:
log_safe_exception(
logger,
"GitNexus setup failed, continuing without it",
e,
include_debug=is_debug_enabled(),
level="warning",
)
logger.warning(f"GitNexus setup failed, continuing without it: {e}")
self._gitnexus_ready = False
return result
@@ -176,59 +222,27 @@ class GitNexusDockerEnvironment(DockerEnvironment):
"timeout": 5,
})
# Wait for the server to be ready (up to ~15s for KuzuDB init)
for i in range(EVAL_SERVER_HEALTH_RETRIES):
time.sleep(EVAL_SERVER_HEALTH_INTERVAL_SECONDS)
# Wait for the server to be ready (up to 15s for KuzuDB init)
for i in range(30):
time.sleep(0.5)
health = self.execute({
"command": f"curl -sf http://127.0.0.1:{self.eval_server_port}/health 2>/dev/null || echo 'NOT_READY'",
"timeout": EVAL_SERVER_HEALTH_TIMEOUT_SECONDS,
"timeout": 3,
})
output = health.get("output", "").strip()
if "NOT_READY" not in output and "ok" in output:
logger.info(
f"Eval-server ready after {(i + 1) * EVAL_SERVER_HEALTH_INTERVAL_SECONDS:.1f}s"
)
logger.info(f"Eval-server ready after {(i + 1) * 0.5:.1f}s")
return
log_output = self.execute({
"command": "cat /tmp/gitnexus-eval-server.log 2>/dev/null | tail -20",
})
logger.warning(
f"Eval-server didn't become ready in "
f"{EVAL_SERVER_HEALTH_RETRIES * EVAL_SERVER_HEALTH_INTERVAL_SECONDS:.1f}s. "
f"Eval-server didn't become ready in 15s. "
f"Tools will fall back to direct CLI.\n"
f"Server log: {log_output.get('output', 'N/A')}"
)
@staticmethod
def _render_tool_script(spec: ToolScriptSpec, port: str) -> str:
"""
Render a standalone bash script for a GitNexus tool.
Scripts call the eval-server fast path when an endpoint is present,
and fall back to the CLI otherwise.
"""
lines = ["#!/bin/bash"]
if spec.endpoint:
lines.append(f'PORT="${{GITNEXUS_EVAL_PORT:-{port}}}"')
if spec.header:
lines.append(spec.header.strip())
if spec.payload_builder:
lines.append(spec.payload_builder.strip())
if spec.endpoint:
lines.append(
f'result=$(curl -sf -X POST "http://127.0.0.1:${{PORT}}{spec.endpoint}" '
'-H "Content-Type: application/json" -d "$payload" 2>/dev/null)'
)
lines.append('if [ $? -eq 0 ] && [ -n "$result" ]; then echo "$result"; exit 0; fi')
lines.append(spec.fallback.strip())
return "\n".join(lines)
def _install_tools(self):
"""
Install standalone GitNexus tool scripts in /usr/local/bin/.
@@ -245,20 +259,24 @@ class GitNexusDockerEnvironment(DockerEnvironment):
"""
port = str(self.eval_server_port)
for spec in TOOL_SPECS.values():
script_content = self._render_tool_script(spec, port).strip()
tools = {
"gitnexus-query": TOOL_SCRIPT_QUERY,
"gitnexus-context": TOOL_SCRIPT_CONTEXT,
"gitnexus-impact": TOOL_SCRIPT_IMPACT,
"gitnexus-cypher": TOOL_SCRIPT_CYPHER,
"gitnexus-augment": TOOL_SCRIPT_AUGMENT,
"gitnexus-overview": TOOL_SCRIPT_OVERVIEW,
}
for name, script in tools.items():
script_content = script.replace("__PORT__", port).strip()
# Use heredoc with quoted delimiter — prevents all variable expansion and quoting issues
self.execute({
"command": (
f"cat << 'GITNEXUS_SCRIPT_EOF' > /usr/local/bin/{spec.bin_name}\n"
f"{script_content}\n"
"GITNEXUS_SCRIPT_EOF\n"
f"chmod +x /usr/local/bin/{spec.bin_name}"
),
"command": f"cat << 'GITNEXUS_SCRIPT_EOF' > /usr/local/bin/{name}\n{script_content}\nGITNEXUS_SCRIPT_EOF\nchmod +x /usr/local/bin/{name}",
"timeout": 5,
})
logger.info(f"Installed {len(TOOL_SPECS)} GitNexus tool scripts in /usr/local/bin/")
logger.info(f"Installed {len(tools)} GitNexus tool scripts in /usr/local/bin/")
def _get_repo_info(self) -> dict:
"""Get repository identity info from the container."""
@@ -307,13 +325,7 @@ class GitNexusDockerEnvironment(DockerEnvironment):
logger.info(f"Cached GitNexus index: {cache_path}")
except Exception as e:
log_safe_exception(
logger,
"Failed to cache GitNexus index",
e,
include_debug=is_debug_enabled(),
level="warning",
)
logger.warning(f"Failed to cache GitNexus index: {e}")
if cache_path.exists():
shutil.rmtree(cache_path, ignore_errors=True)
@@ -349,13 +361,7 @@ class GitNexusDockerEnvironment(DockerEnvironment):
logger.info("GitNexus index restored from cache")
except Exception as e:
log_safe_exception(
logger,
"Failed to restore cache, re-indexing",
e,
include_debug=is_debug_enabled(),
level="warning",
)
logger.warning(f"Failed to restore cache, re-indexing: {e}")
self._index_repository()
def stop(self) -> dict:
+3 -5
View File
@@ -6,7 +6,7 @@ readme = "README.md"
requires-python = ">=3.11"
dependencies = [
"mini-swe-agent>=2.0.0",
"litellm>=1.50.0,!=1.82.7,!=1.82.8",
"litellm>=1.50.0",
"datasets>=3.0.0",
"typer>=0.12.0",
"rich>=13.0.0",
@@ -20,8 +20,6 @@ dependencies = [
dev = [
"pytest>=8.0.0",
"ruff>=0.5.0",
"hypothesis>=6.88.0",
"coverage>=7.6.0",
]
[project.scripts]
@@ -33,8 +31,8 @@ requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["agents", "environments", "analysis", "bridge", "utils"]
extra-files = ["run_eval.py", "tool_registry.py", "constants.py"]
packages = ["agents", "environments", "analysis", "bridge"]
extra-files = ["run_eval.py"]
[tool.ruff]
line-length = 120
+37 -73
View File
@@ -25,6 +25,7 @@ import logging
import os
import threading
import time
import traceback
from itertools import product
from pathlib import Path
from typing import Any
@@ -35,8 +36,6 @@ from rich.console import Console
from rich.live import Live
from rich.table import Table
from utils.errors import is_debug_enabled, log_safe_exception
# Load .env file from eval/ directory
_env_file = Path(__file__).parent / ".env"
if _env_file.exists():
@@ -139,65 +138,6 @@ def get_swebench_docker_image(instance: dict) -> str:
return image_name
def _build_model(config: dict):
"""Construct the model from config."""
from minisweagent.models import get_model
return get_model(config=config.get("model", {}))
def _build_environment(config: dict, instance: dict):
"""Construct the environment for the instance."""
env_config = dict(config.get("environment", {}))
env_class_name = env_config.pop("environment_class", "docker")
if env_class_name == "eval.environments.gitnexus_docker.GitNexusDockerEnvironment":
from environments.gitnexus_docker import GitNexusDockerEnvironment
env_config["image"] = get_swebench_docker_image(instance)
return GitNexusDockerEnvironment(**env_config)
from minisweagent.environments.docker import DockerEnvironment
return DockerEnvironment(image=get_swebench_docker_image(instance), **env_config)
def _build_agent(config: dict, model, env, instance_dir: Path, instance_id: str):
"""Construct the GitNexus agent with trajectory output configured."""
from agents.gitnexus_agent import GitNexusAgent
agent_config = dict(config.get("agent", {}))
agent_config.pop("agent_class", "eval.agents.gitnexus_agent.GitNexusAgent")
traj_path = instance_dir / f"{instance_id}.traj.json"
agent_config["output_path"] = traj_path
return GitNexusAgent(model, env, **agent_config)
def _extract_submission(env, info: dict, run_id: str) -> str:
"""Pull the git diff patch from the container, falling back to the agent submission."""
try:
patch_output = env.execute({"command": "cd /testbed && git diff"})
return patch_output.get("output", "").strip()
except Exception as patch_err:
logger.warning(f"[{run_id}] Failed to extract patch: {patch_err}")
return info.get("submission", "")
def _record_failure(run_id: str, instance_id: str, result: dict, error: Exception):
sanitized = log_safe_exception(
logger,
f"[{run_id}] Error on {instance_id}",
error,
include_debug=is_debug_enabled(),
)
result["exit_status"] = sanitized["error_type"]
result["error_type"] = sanitized["error_type"]
result["error_message"] = sanitized["error_message"]
result["error"] = sanitized["error_message"]
if "error_detail_debug" in sanitized:
result["error_detail_debug"] = sanitized["error_detail_debug"]
def process_instance(
instance: dict,
config: dict,
@@ -209,6 +149,8 @@ def process_instance(
Process a single SWE-bench instance with the given config.
Returns result dict with instance_id, exit_status, submission, metrics.
"""
from minisweagent.models import get_model
instance_id = instance["instance_id"]
run_id = f"{model_name}_{mode_name}"
instance_dir = output_dir / run_id / instance_id
@@ -226,12 +168,31 @@ def process_instance(
}
agent = None
env = None
try:
model = _build_model(config)
env = _build_environment(config, instance)
agent = _build_agent(config, model, env, instance_dir, instance_id)
# Build model
model = get_model(config=config.get("model", {}))
# Build environment
env_config = dict(config.get("environment", {}))
env_class_name = env_config.pop("environment_class", "docker")
if env_class_name == "eval.environments.gitnexus_docker.GitNexusDockerEnvironment":
from environments.gitnexus_docker import GitNexusDockerEnvironment
env_config["image"] = get_swebench_docker_image(instance)
env = GitNexusDockerEnvironment(**env_config)
else:
from minisweagent.environments.docker import DockerEnvironment
env = DockerEnvironment(image=get_swebench_docker_image(instance), **env_config)
# Build agent
agent_config = dict(config.get("agent", {}))
agent_class_name = agent_config.pop("agent_class", "eval.agents.gitnexus_agent.GitNexusAgent")
from agents.gitnexus_agent import GitNexusAgent
traj_path = instance_dir / f"{instance_id}.traj.json"
agent_config["output_path"] = traj_path
agent = GitNexusAgent(model, env, **agent_config)
# Run
logger.info(f"[{run_id}] Starting {instance_id}")
@@ -243,10 +204,18 @@ def process_instance(
result["gitnexus_metrics"] = agent.gitnexus_metrics.to_dict()
# Extract git diff patch from the container (SWE-bench needs the model_patch)
result["submission"] = _extract_submission(env, info, run_id)
try:
patch_output = env.execute({"command": "cd /testbed && git diff"})
result["submission"] = patch_output.get("output", "").strip()
except Exception as patch_err:
logger.warning(f"[{run_id}] Failed to extract patch: {patch_err}")
result["submission"] = info.get("submission", "")
except Exception as e:
_record_failure(run_id, instance_id, result, e)
logger.error(f"[{run_id}] Error on {instance_id}: {e}")
result["exit_status"] = type(e).__name__
result["error"] = str(e)
result["traceback"] = traceback.format_exc()
finally:
if agent:
@@ -318,12 +287,7 @@ def run_configuration(
results.append(future.result())
except Exception as e:
iid = futures[future]
log_safe_exception(
logger,
f"[{run_id}] Uncaught error for {iid}",
e,
include_debug=is_debug_enabled(),
)
logger.error(f"[{run_id}] Uncaught error for {iid}: {e}")
# Save run summary
summary = {
-1
View File
@@ -1 +0,0 @@
"""Tests for the GitNexus eval harness."""
-6
View File
@@ -1,6 +0,0 @@
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
-29
View File
@@ -1,29 +0,0 @@
from utils.errors import sanitize_exception
def _raise_value_error():
raise ValueError("boom")
def test_sanitize_exception_without_debug(monkeypatch):
monkeypatch.delenv("GITNEXUS_EVAL_DEBUG", raising=False)
try:
_raise_value_error()
except Exception as exc: # noqa: BLE001
data = sanitize_exception(exc, include_debug=False)
assert data["error_type"] == "ValueError"
assert data["error_message"] == "boom"
assert "error_detail_debug" not in data
def test_sanitize_exception_with_debug(monkeypatch):
monkeypatch.setenv("GITNEXUS_EVAL_DEBUG", "1")
try:
_raise_value_error()
except Exception as exc: # noqa: BLE001
data = sanitize_exception(exc)
assert data["error_type"] == "ValueError"
assert "error_detail_debug" in data
assert "ValueError" in data["error_detail_debug"]
-20
View File
@@ -1,20 +0,0 @@
import pytest
analysis_module = pytest.importorskip("analysis.analyze_results")
parse_run_id = analysis_module.parse_run_id
def test_parse_run_id_native_augment():
model, mode = parse_run_id("claude-sonnet_native_augment")
assert model == "claude-sonnet"
assert mode == "native_augment"
def test_parse_run_id_hyphenated_model():
model, mode = parse_run_id("glm-4.7_native")
assert model == "glm-4.7"
assert mode == "native"
def test_parse_run_id_unknown():
assert parse_run_id("custom_model") == ("custom_model", "unknown")
-63
View File
@@ -1,63 +0,0 @@
from __future__ import annotations
from hypothesis import given
from hypothesis import strategies as st
from analysis.analyze_results import parse_run_id
from environments.gitnexus_docker import GitNexusDockerEnvironment
from tool_registry import TOOL_SPECS
from utils.errors import sanitize_exception
KNOWN_MODES = [
"native_augment",
"native",
"baseline",
"mcp",
"augment",
"full",
]
def _model_strategy():
base_chars = st.characters(
blacklist_categories=("Cs",),
blacklist_characters={" ", "\n", "\t"},
)
text = st.text(alphabet=base_chars, min_size=1)
return text.filter(lambda s: not any(s.endswith(f"_{m}") for m in KNOWN_MODES))
@given(_model_strategy(), st.sampled_from(KNOWN_MODES))
def test_parse_run_id_round_trip(model: str, mode: str) -> None:
run_id = f"{model}_{mode}"
parsed_model, parsed_mode = parse_run_id(run_id)
assert parsed_model == model
assert parsed_mode == mode
@given(st.text())
def test_sanitize_exception_respects_debug_flag(message: str) -> None:
exc = ValueError(message)
data = sanitize_exception(exc, include_debug=False)
assert data["error_type"] == "ValueError"
assert data["error_message"] == (message or "ValueError")
assert "error_detail_debug" not in data
data_debug = sanitize_exception(exc, include_debug=True)
assert data_debug["error_type"] == "ValueError"
assert "error_detail_debug" in data_debug
assert data_debug["error_detail_debug"]
@given(st.sampled_from(list(TOOL_SPECS.values())), st.integers(min_value=1, max_value=99999))
def test_render_tool_script_contains_expected_paths(spec, port: int) -> None:
script = GitNexusDockerEnvironment._render_tool_script(spec, str(port))
assert spec.fallback.strip() in script
if spec.endpoint:
assert spec.endpoint in script
assert f"${{GITNEXUS_EVAL_PORT:-{port}}}" in script
assert "curl" in script
else:
assert "curl" not in script
-21
View File
@@ -1,21 +0,0 @@
import pytest
GitNexusDockerEnvironment = pytest.importorskip(
"environments.gitnexus_docker"
).GitNexusDockerEnvironment
tool_registry = pytest.importorskip("tool_registry")
TOOL_SPECS = tool_registry.TOOL_SPECS
def test_render_query_script_uses_endpoint_and_fallback():
script = GitNexusDockerEnvironment._render_tool_script(TOOL_SPECS["query"], "4848")
assert "/tool/query" in script
assert "gitnexus query" in script
assert "GITNEXUS_EVAL_PORT" in script
def test_render_augment_script_skips_curl():
script = GitNexusDockerEnvironment._render_tool_script(TOOL_SPECS["augment"], "4848")
assert "/tool/" not in script
assert "curl" not in script
assert "gitnexus augment" in script
-79
View File
@@ -1,79 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Dict, Tuple
@dataclass(frozen=True)
class ToolScriptSpec:
key: str
bin_name: str
endpoint: str | None
payload_builder: str
fallback: str
header: str | None = None
TOOL_METRIC_KEYS: Tuple[str, ...] = ("query", "context", "impact", "cypher", "overview")
TOOL_SPECS: Dict[str, ToolScriptSpec] = {
"query": ToolScriptSpec(
key="query",
bin_name="gitnexus-query",
endpoint="/tool/query",
payload_builder=r'''query="$1"; task_ctx="${2:-}"; goal="${3:-}"
[ -z "$query" ] && echo "Usage: gitnexus-query <query> [task_context] [goal]" && exit 1
payload="{\"query\": \"$query\""
[ -n "$task_ctx" ] && payload="$payload, \"task_context\": \"$task_ctx\""
[ -n "$goal" ] && payload="$payload, \"goal\": \"$goal\""
payload="$payload}"''',
fallback='cd /testbed && npx gitnexus query "$query" 2>&1',
),
"context": ToolScriptSpec(
key="context",
bin_name="gitnexus-context",
endpoint="/tool/context",
payload_builder=r'''name="$1"; file_path="${2:-}"
[ -z "$name" ] && echo "Usage: gitnexus-context <symbol_name> [file_path]" && exit 1
payload="{\"name\": \"$name\""
[ -n "$file_path" ] && payload="$payload, \"file_path\": \"$file_path\""
payload="$payload}"''',
fallback='cd /testbed && npx gitnexus context "$name" 2>&1',
),
"impact": ToolScriptSpec(
key="impact",
bin_name="gitnexus-impact",
endpoint="/tool/impact",
payload_builder=r'''target="$1"; direction="${2:-upstream}"
[ -z "$target" ] && echo "Usage: gitnexus-impact <symbol_name> [upstream|downstream]" && exit 1
payload="{\"target\": \"$target\", \"direction\": \"$direction\"}"''',
fallback='cd /testbed && npx gitnexus impact "$target" --direction "$direction" 2>&1',
),
"cypher": ToolScriptSpec(
key="cypher",
bin_name="gitnexus-cypher",
endpoint="/tool/cypher",
payload_builder=r'''query="$1"
[ -z "$query" ] && echo "Usage: gitnexus-cypher <cypher_query>" && exit 1
payload="{\"query\": \"$query\"}"''',
fallback='cd /testbed && npx gitnexus cypher "$query" 2>&1',
),
"overview": ToolScriptSpec(
key="overview",
bin_name="gitnexus-overview",
endpoint="/tool/list_repos",
header='echo "=== Code Knowledge Graph Overview ==="',
payload_builder='payload="{}"',
fallback='cd /testbed && npx gitnexus list 2>&1',
),
"augment": ToolScriptSpec(
key="augment",
bin_name="gitnexus-augment",
endpoint=None,
payload_builder="",
fallback='cd /testbed && npx gitnexus augment "$1" 2>&1 || true',
),
}
BINARIES_BY_KEY: Dict[str, str] = {spec.key: spec.bin_name for spec in TOOL_SPECS.values()}
ENDPOINTS_BY_KEY: Dict[str, str | None] = {spec.key: spec.endpoint for spec in TOOL_SPECS.values()}
-1
View File
@@ -1 +0,0 @@
"""Utility package for the eval harness."""
-60
View File
@@ -1,60 +0,0 @@
from __future__ import annotations
import os
import traceback
from typing import Any, Callable
from constants import DEBUG_ENV_VAR
def is_debug_enabled() -> bool:
"""Return True when debug output (full tracebacks) should be emitted."""
return os.getenv(DEBUG_ENV_VAR, "").strip().lower() in {"1", "true", "yes", "on"}
def sanitize_exception(exc: BaseException, *, include_debug: bool | None = None) -> dict[str, str]:
"""
Produce a log-safe, JSON-friendly view of an exception.
- Always returns error_type and error_message.
- Only includes error_detail_debug (full traceback) when debug is enabled.
"""
debug = is_debug_enabled() if include_debug is None else include_debug
error_type = type(exc).__name__
message = str(exc) or error_type
data: dict[str, str] = {
"error_type": error_type,
"error_message": message,
}
if debug:
tb = "".join(traceback.format_exception(type(exc), exc, exc.__traceback__))
if tb:
data["error_detail_debug"] = tb
return data
def log_safe_exception(
logger: Any,
prefix: str,
exc: BaseException,
*,
include_debug: bool | None = None,
level: str = "error",
) -> dict[str, str]:
"""
Log an exception without leaking stack traces unless debug is enabled.
Returns the sanitized dict so callers can persist it.
"""
data = sanitize_exception(exc, include_debug=include_debug)
debug = "error_detail_debug" in data
log_fn: Callable[..., None] = getattr(logger, level, logger.error)
message = f"{prefix}: {data['error_type']}: {data['error_message']}"
log_kwargs = {"exc_info": True} if debug else {}
log_fn(message, **log_kwargs)
return data
Generated
-2529
View File
File diff suppressed because it is too large Load Diff
+25 -60
View File
@@ -64,26 +64,10 @@ function extractPattern(toolName, toolInput) {
const tokens = cmd.split(/\s+/);
let foundCmd = false;
let skipNext = false;
const flagsWithValues = new Set([
'-e',
'-f',
'-m',
'-A',
'-B',
'-C',
'-g',
'--glob',
'-t',
'--type',
'--include',
'--exclude',
]);
const flagsWithValues = new Set(['-e', '-f', '-m', '-A', '-B', '-C', '-g', '--glob', '-t', '--type', '--include', '--exclude']);
for (const token of tokens) {
if (skipNext) {
skipNext = false;
continue;
}
if (skipNext) { skipNext = false; continue; }
if (!foundCmd) {
if (/\brg$|\bgrep$/.test(token)) foundCmd = true;
continue;
@@ -114,42 +98,33 @@ function runGitNexusCli(args, cwd, timeout) {
// Detect whether 'gitnexus' is on PATH (cheap check, no execution)
let useDirectBinary = false;
try {
const which = spawnSync(isWin ? 'where' : 'which', ['gitnexus'], {
encoding: 'utf-8',
timeout: 3000,
stdio: ['pipe', 'pipe', 'pipe'],
});
const which = spawnSync(
isWin ? 'where' : 'which', ['gitnexus'],
{ encoding: 'utf-8', timeout: 3000, stdio: ['pipe', 'pipe', 'pipe'] }
);
useDirectBinary = which.status === 0;
} catch {
/* not on PATH */
}
} catch { /* not on PATH */ }
if (useDirectBinary) {
return spawnSync(isWin ? 'gitnexus.cmd' : 'gitnexus', args, {
encoding: 'utf-8',
timeout,
cwd,
stdio: ['pipe', 'pipe', 'pipe'],
});
return spawnSync(
isWin ? 'gitnexus.cmd' : 'gitnexus', args,
{ encoding: 'utf-8', timeout, cwd, stdio: ['pipe', 'pipe', 'pipe'] }
);
}
// npx fallback needs shell on Windows since npx is a .cmd script
return spawnSync(isWin ? 'npx.cmd' : 'npx', ['-y', 'gitnexus', ...args], {
encoding: 'utf-8',
timeout: timeout + 5000,
cwd,
stdio: ['pipe', 'pipe', 'pipe'],
});
return spawnSync(
isWin ? 'npx.cmd' : 'npx', ['-y', 'gitnexus', ...args],
{ encoding: 'utf-8', timeout: timeout + 5000, cwd, stdio: ['pipe', 'pipe', 'pipe'] }
);
}
/**
* Emit a hook response with additional context for the agent.
*/
function sendHookResponse(hookEventName, message) {
console.log(
JSON.stringify({
hookSpecificOutput: { hookEventName, additionalContext: message },
}),
);
console.log(JSON.stringify({
hookSpecificOutput: { hookEventName, additionalContext: message }
}));
}
/**
@@ -174,9 +149,7 @@ function handlePreToolUse(input) {
if (!child.error && child.status === 0) {
result = child.stderr || '';
}
} catch {
/* graceful failure */
}
} catch { /* graceful failure */ }
if (result && result.trim()) {
sendHookResponse('PreToolUse', result.trim());
@@ -212,15 +185,10 @@ function handlePostToolUse(input) {
let currentHead = '';
try {
const headResult = spawnSync('git', ['rev-parse', 'HEAD'], {
encoding: 'utf-8',
timeout: 3000,
cwd,
stdio: ['pipe', 'pipe', 'pipe'],
encoding: 'utf-8', timeout: 3000, cwd, stdio: ['pipe', 'pipe', 'pipe'],
});
currentHead = (headResult.stdout || '').trim();
} catch {
return;
}
} catch { return; }
if (!currentHead) return;
@@ -229,19 +197,16 @@ function handlePostToolUse(input) {
try {
const meta = JSON.parse(fs.readFileSync(path.join(gitNexusDir, 'meta.json'), 'utf-8'));
lastCommit = meta.lastCommit || '';
hadEmbeddings = meta.stats && meta.stats.embeddings > 0;
} catch {
/* no meta — treat as stale */
}
hadEmbeddings = (meta.stats && meta.stats.embeddings > 0);
} catch { /* no meta — treat as stale */ }
// If HEAD matches last indexed commit, no reindex needed
if (currentHead && currentHead === lastCommit) return;
const analyzeCmd = `npx gitnexus analyze${hadEmbeddings ? ' --embeddings' : ''}`;
sendHookResponse(
'PostToolUse',
sendHookResponse('PostToolUse',
`GitNexus index is stale (last indexed: ${lastCommit ? lastCommit.slice(0, 7) : 'never'}). ` +
`Run \`${analyzeCmd}\` to update the knowledge graph.`,
`Run \`${analyzeCmd}\` to update the knowledge graph.`
);
}
-29
View File
@@ -1,29 +0,0 @@
{
"name": "gitnexus-shared",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "gitnexus-shared",
"version": "1.0.0",
"devDependencies": {
"typescript": "^6.0.2"
}
},
"node_modules/typescript": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.2.tgz",
"integrity": "sha512-bGdAIrZ0wiGDo5l8c++HWtbaNCWTS4UTv7RaTH/ThVIgjkveJt83m74bBHMJkuCbslY8ixgLBVZJIOiQlQTjfQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
}
}
}
-25
View File
@@ -1,25 +0,0 @@
{
"name": "gitnexus-shared",
"version": "1.0.0",
"private": true,
"description": "Shared type definitions for GitNexus CLI and web",
"type": "module",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"exports": {
".": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
}
},
"scripts": {
"build": "tsc"
},
"files": [
"dist",
"src"
],
"devDependencies": {
"typescript": "^6.0.2"
}
}
-150
View File
@@ -1,150 +0,0 @@
/**
* Graph type definitions — single source of truth.
*
* Both gitnexus (CLI) and gitnexus-web import from this package.
* Do NOT add Node.js-specific or browser-specific imports here.
*/
import { SupportedLanguages } from '../languages.js';
export type NodeLabel =
| 'Project'
| 'Package'
| 'Module'
| 'Folder'
| 'File'
| 'Class'
| 'Function'
| 'Method'
| 'Variable'
| 'Interface'
| 'Enum'
| 'Decorator'
| 'Import'
| 'Type'
| 'CodeElement'
| 'Community'
| 'Process'
// Multi-language node types
| 'Struct'
| 'Macro'
| 'Typedef'
| 'Union'
| 'Namespace'
| 'Trait'
| 'Impl'
| 'TypeAlias'
| 'Const'
| 'Static'
| 'Property'
| 'Record'
| 'Delegate'
| 'Annotation'
| 'Constructor'
| 'Template'
| 'Section'
| 'Route'
| 'Tool';
export type NodeProperties = {
name: string;
filePath: string;
startLine?: number;
endLine?: number;
language?: SupportedLanguages | string;
isExported?: boolean;
astFrameworkMultiplier?: number;
astFrameworkReason?: string;
// Community
heuristicLabel?: string;
cohesion?: number;
symbolCount?: number;
keywords?: string[];
description?: string;
enrichedBy?: 'heuristic' | 'llm';
// Process
processType?: 'intra_community' | 'cross_community';
stepCount?: number;
communities?: string[];
entryPointId?: string;
terminalId?: string;
entryPointScore?: number;
entryPointReason?: string;
// Method/property
parameterCount?: number;
level?: number;
returnType?: string;
declaredType?: string;
visibility?: string;
isStatic?: boolean;
isReadonly?: boolean;
isAbstract?: boolean;
isFinal?: boolean;
isVirtual?: boolean;
isOverride?: boolean;
isAsync?: boolean;
isPartial?: boolean;
annotations?: string[];
// Route/response
responseKeys?: string[];
errorKeys?: string[];
middleware?: string[];
// Extensible
[key: string]: unknown;
};
export type RelationshipType =
| 'CONTAINS'
| 'CALLS'
| 'INHERITS'
| 'METHOD_OVERRIDES'
| 'METHOD_IMPLEMENTS'
| 'IMPORTS'
| 'USES'
| 'DEFINES'
| 'DECORATES'
| 'IMPLEMENTS'
| 'EXTENDS'
| 'HAS_METHOD'
| 'HAS_PROPERTY'
| 'ACCESSES'
| 'MEMBER_OF'
| 'STEP_IN_PROCESS'
| 'HANDLES_ROUTE'
| 'FETCHES'
| 'HANDLES_TOOL'
| 'ENTRY_POINT_OF'
| 'WRAPS'
| 'QUERIES';
export interface GraphNode {
id: string;
label: NodeLabel;
properties: NodeProperties;
}
export interface GraphRelationship {
id: string;
sourceId: string;
targetId: string;
type: RelationshipType;
confidence: number;
reason: string;
step?: number;
/**
* Per-signal evidence trace for edges emitted by the scope-based
* resolution pipeline (RFC #909 Ring 2 PKG #925). Populated by
* `emit-references.ts` when draining `ReferenceIndex` into the graph
* so downstream query / audit tools can inspect *why* a given edge
* was emitted with its confidence value.
*
* Optional and additive — every existing edge emitter ignores this
* field, and every existing query continues to work whether or not
* an edge carries it.
*/
evidence?: readonly {
readonly kind: string;
readonly weight: number;
readonly note?: string;
}[];
}
-150
View File
@@ -1,150 +0,0 @@
// Graph types
export type {
NodeLabel,
NodeProperties,
RelationshipType,
GraphNode,
GraphRelationship,
} from './graph/types.js';
// Schema constants
export {
NODE_TABLES,
REL_TABLE_NAME,
REL_TYPES,
EMBEDDING_TABLE_NAME,
} from './lbug/schema-constants.js';
export type { NodeTableName, RelType } from './lbug/schema-constants.js';
// Language support
export { SupportedLanguages } from './languages.js';
export { getLanguageFromFilename, getSyntaxLanguageFromFilename } from './language-detection.js';
export type { MroStrategy } from './mro-strategy.js';
// Pipeline progress
export type { PipelinePhase, PipelineProgress } from './pipeline.js';
// ─── Scope-based resolution — RFC #909 (Ring 1 #910) ────────────────────────
// Data model (RFC §2)
export type { SymbolDefinition } from './scope-resolution/symbol-definition.js';
export type {
ScopeId,
DefId,
ScopeKind,
Range,
Capture,
CaptureMatch,
BindingRef,
ImportEdge,
TypeRef,
Scope,
ResolutionEvidence,
Resolution,
Reference,
ReferenceIndex,
LookupParams,
RegistryContributor,
ParsedImport,
ParsedTypeBinding,
WorkspaceIndex,
Callsite,
ScopeLookup,
} from './scope-resolution/types.js';
// Evidence + tie-break constants (RFC Appendix A, Appendix B)
export { EvidenceWeights, typeBindingWeightAtDepth } from './scope-resolution/evidence-weights.js';
export { ORIGIN_PRIORITY } from './scope-resolution/origin-priority.js';
export type { OriginForTieBreak } from './scope-resolution/origin-priority.js';
// Language classification (RFC §6.1 Ring 3/4 governance)
export {
LanguageClassifications,
isProductionLanguage,
} from './scope-resolution/language-classification.js';
export type { LanguageClassification } from './scope-resolution/language-classification.js';
// Core indexes over per-file artifacts (RFC §3.1; Ring 2 SHARED #913)
export { buildDefIndex } from './scope-resolution/def-index.js';
export type { DefIndex } from './scope-resolution/def-index.js';
export { buildModuleScopeIndex } from './scope-resolution/module-scope-index.js';
export type { ModuleScopeIndex, ModuleScopeEntry } from './scope-resolution/module-scope-index.js';
export { buildQualifiedNameIndex } from './scope-resolution/qualified-name-index.js';
export type { QualifiedNameIndex } from './scope-resolution/qualified-name-index.js';
// Strict type-reference resolver (RFC §4.6; Ring 2 SHARED #916)
// `ScopeLookup` is defined in `./scope-resolution/types.js` and exported
// from the type-export block above — not from this module.
export { resolveTypeRef } from './scope-resolution/resolve-type-ref.js';
export type { ResolveTypeRefContext } from './scope-resolution/resolve-type-ref.js';
// ScopeExtractor output contracts (RFC §3.2 Phase 1; Ring 2 PKG #919)
export type { ParsedFile } from './scope-resolution/parsed-file.js';
export type { ReferenceSite, ReferenceKind, CallForm } from './scope-resolution/reference-site.js';
// Method-dispatch materialized view over HeritageMap (RFC §3.1; Ring 2 SHARED #914)
export { buildMethodDispatchIndex } from './scope-resolution/method-dispatch-index.js';
export type {
MethodDispatchIndex,
MethodDispatchInput,
} from './scope-resolution/method-dispatch-index.js';
// SCC-aware cross-file finalize (RFC §3.2 Phase 2; Ring 2 SHARED #915)
export { finalize } from './scope-resolution/finalize-algorithm.js';
export type {
FinalizeInput,
FinalizeFile,
FinalizeHooks,
FinalizeOutput,
FinalizedScc,
FinalizeStats,
} from './scope-resolution/finalize-algorithm.js';
// Scope-aware registries + 7-step lookup (RFC §4; Ring 2 SHARED #917)
export { buildClassRegistry } from './scope-resolution/registries/class-registry.js';
export type { ClassRegistry } from './scope-resolution/registries/class-registry.js';
export { buildMethodRegistry } from './scope-resolution/registries/method-registry.js';
export type {
MethodRegistry,
MethodLookupOptions,
} from './scope-resolution/registries/method-registry.js';
export { buildFieldRegistry } from './scope-resolution/registries/field-registry.js';
export type {
FieldRegistry,
FieldLookupOptions,
} from './scope-resolution/registries/field-registry.js';
export { lookupCore } from './scope-resolution/registries/lookup-core.js';
export type { CoreLookupParams } from './scope-resolution/registries/lookup-core.js';
export { lookupQualified } from './scope-resolution/registries/lookup-qualified.js';
export type { LookupQualifiedParams } from './scope-resolution/registries/lookup-qualified.js';
export { composeEvidence, confidenceFromEvidence } from './scope-resolution/registries/evidence.js';
export type { RawSignals } from './scope-resolution/registries/evidence.js';
export {
compareByConfidenceWithTiebreaks,
CONFIDENCE_EPSILON,
} from './scope-resolution/registries/tie-breaks.js';
export type { TieBreakKey } from './scope-resolution/registries/tie-breaks.js';
export { CLASS_KINDS, METHOD_KINDS, FIELD_KINDS } from './scope-resolution/registries/context.js';
export type {
RegistryContext,
RegistryProviders,
OwnerScopedContributor,
ArityVerdict,
} from './scope-resolution/registries/context.js';
// Scope tree spine + position lookup (RFC §2.2 + §3.1; Ring 2 SHARED #912)
export { makeScopeId, clearScopeIdInternPool } from './scope-resolution/scope-id.js';
export type { ScopeIdInput } from './scope-resolution/scope-id.js';
export { buildScopeTree, ScopeTreeInvariantError } from './scope-resolution/scope-tree.js';
export type { ScopeTree } from './scope-resolution/scope-tree.js';
export { buildPositionIndex } from './scope-resolution/position-index.js';
export type { PositionIndex } from './scope-resolution/position-index.js';
// Shadow-mode diff + aggregation (RFC §6.3; Ring 2 SHARED #918)
export { diffResolutions } from './scope-resolution/shadow/diff.js';
export type {
ShadowAgreement,
ShadowCallsite,
ShadowDiff,
} from './scope-resolution/shadow/diff.js';
export { aggregateDiffs } from './scope-resolution/shadow/aggregate.js';
export type { LanguageParityRow, ShadowParityReport } from './scope-resolution/shadow/aggregate.js';
-148
View File
@@ -1,148 +0,0 @@
/**
* Language Detection — maps file paths to SupportedLanguages enum values.
*
* Shared between CLI (ingestion pipeline) and web (syntax highlighting).
*
* ADDING A NEW LANGUAGE:
* 1. Add enum member to SupportedLanguages in languages.ts
* 2. Add file extensions to EXTENSION_MAP below
* 3. TypeScript will error if you miss either step (exhaustive Record)
*/
import { SupportedLanguages } from './languages.js';
/** Ruby extensionless filenames recognised as Ruby source */
const RUBY_EXTENSIONLESS_FILES = new Set([
'Rakefile',
'Gemfile',
'Guardfile',
'Vagrantfile',
'Brewfile',
]);
/**
* Exhaustive map: every SupportedLanguages member → its file extensions.
*
* If a new language is added to the enum without adding an entry here,
* TypeScript emits a compile error: "Property 'NewLang' is missing in type..."
*/
const EXTENSION_MAP: Record<SupportedLanguages, readonly string[]> = {
[SupportedLanguages.JavaScript]: ['.js', '.jsx', '.mjs', '.cjs'],
[SupportedLanguages.TypeScript]: ['.ts', '.tsx', '.mts', '.cts'],
[SupportedLanguages.Python]: ['.py'],
[SupportedLanguages.Java]: ['.java'],
[SupportedLanguages.C]: ['.c'],
[SupportedLanguages.CPlusPlus]: ['.cpp', '.cc', '.cxx', '.h', '.hpp', '.hxx', '.hh'],
[SupportedLanguages.CSharp]: ['.cs'],
[SupportedLanguages.Go]: ['.go'],
[SupportedLanguages.Ruby]: ['.rb', '.rake', '.gemspec'],
[SupportedLanguages.Rust]: ['.rs'],
[SupportedLanguages.PHP]: ['.php', '.phtml', '.php3', '.php4', '.php5', '.php8'],
[SupportedLanguages.Kotlin]: ['.kt', '.kts'],
[SupportedLanguages.Swift]: ['.swift'],
[SupportedLanguages.Dart]: ['.dart'],
[SupportedLanguages.Vue]: ['.vue'],
[SupportedLanguages.Cobol]: ['.cbl', '.cob', '.cpy', '.cobol'],
} satisfies Record<SupportedLanguages, readonly string[]>; // Ensure exhaustiveness
/** Pre-built reverse lookup: extension → language (built once at module load). */
const extToLang = new Map<string, SupportedLanguages>();
for (const [lang, exts] of Object.entries(EXTENSION_MAP) as [
SupportedLanguages,
readonly string[],
][]) {
for (const ext of exts) {
extToLang.set(ext, lang);
}
}
/**
* Map file extension to SupportedLanguage enum.
* Returns null if the file extension is not recognized.
*/
export const getLanguageFromFilename = (filename: string): SupportedLanguages | null => {
// Fast path: check the extension map
const lastDot = filename.lastIndexOf('.');
if (lastDot >= 0) {
const ext = filename.slice(lastDot).toLowerCase();
const lang = extToLang.get(ext);
if (lang !== undefined) return lang;
}
// Ruby extensionless files (Rakefile, Gemfile, etc.)
const basename = filename.split('/').pop() || filename;
if (RUBY_EXTENSIONLESS_FILES.has(basename)) {
return SupportedLanguages.Ruby;
}
return null;
};
/**
* Exhaustive map: every SupportedLanguages member → Prism syntax identifier.
*
* If a new language is added to the enum without adding an entry here,
* TypeScript emits a compile error.
*/
const SYNTAX_MAP: Record<SupportedLanguages, string> = {
[SupportedLanguages.JavaScript]: 'javascript',
[SupportedLanguages.TypeScript]: 'typescript',
[SupportedLanguages.Python]: 'python',
[SupportedLanguages.Java]: 'java',
[SupportedLanguages.C]: 'c',
[SupportedLanguages.CPlusPlus]: 'cpp',
[SupportedLanguages.CSharp]: 'csharp',
[SupportedLanguages.Go]: 'go',
[SupportedLanguages.Ruby]: 'ruby',
[SupportedLanguages.Rust]: 'rust',
[SupportedLanguages.PHP]: 'php',
[SupportedLanguages.Kotlin]: 'kotlin',
[SupportedLanguages.Swift]: 'swift',
[SupportedLanguages.Dart]: 'dart',
[SupportedLanguages.Vue]: 'typescript',
[SupportedLanguages.Cobol]: 'cobol',
} satisfies Record<SupportedLanguages, string>; // Ensure exhaustiveness
/** Non-code file extensions → Prism-compatible syntax identifiers */
const AUXILIARY_SYNTAX_MAP: Record<string, string> = {
json: 'json',
yaml: 'yaml',
yml: 'yaml',
md: 'markdown',
mdx: 'markdown',
html: 'markup',
htm: 'markup',
erb: 'markup',
xml: 'markup',
css: 'css',
scss: 'css',
sass: 'css',
sh: 'bash',
bash: 'bash',
zsh: 'bash',
sql: 'sql',
toml: 'toml',
ini: 'ini',
dockerfile: 'docker',
};
/** Extensionless filenames → Prism-compatible syntax identifiers */
const AUXILIARY_BASENAME_MAP: Record<string, string> = {
Makefile: 'makefile',
Dockerfile: 'docker',
};
/**
* Map file path to a Prism-compatible syntax highlight language string.
* Covers all SupportedLanguages (code files) plus common non-code formats.
* Returns 'text' for unrecognised files.
*/
export const getSyntaxLanguageFromFilename = (filePath: string): string => {
const lang = getLanguageFromFilename(filePath);
if (lang) return SYNTAX_MAP[lang];
const ext = filePath.split('.').pop()?.toLowerCase();
if (ext && ext in AUXILIARY_SYNTAX_MAP) return AUXILIARY_SYNTAX_MAP[ext];
const basename = filePath.split('/').pop() || '';
if (basename in AUXILIARY_BASENAME_MAP) return AUXILIARY_BASENAME_MAP[basename];
return 'text';
};
-25
View File
@@ -1,25 +0,0 @@
/**
* Supported language enum — single source of truth.
*
* Both CLI and web use this to identify which language a file/node belongs to.
* The CLI uses it throughout the ingestion pipeline; the web uses it for display.
*/
export enum SupportedLanguages {
JavaScript = 'javascript',
TypeScript = 'typescript',
Python = 'python',
Java = 'java',
C = 'c',
CPlusPlus = 'cpp',
CSharp = 'csharp',
Go = 'go',
Ruby = 'ruby',
Rust = 'rust',
PHP = 'php',
Kotlin = 'kotlin',
Swift = 'swift',
Dart = 'dart',
Vue = 'vue',
/** Standalone regex processor — no tree-sitter, no LanguageProvider. */
Cobol = 'cobol',
}
@@ -1,74 +0,0 @@
/**
* LadybugDB schema constants — single source of truth.
*
* NODE_TABLES and REL_TYPES define what the knowledge graph can contain.
* Both CLI and web must agree on these for data compatibility.
*
* Full DDL schemas remain in each package's own schema.ts because
* the CLI uses native LadybugDB and the web uses WASM.
*/
export const NODE_TABLES = [
'File',
'Folder',
'Function',
'Class',
'Interface',
'Method',
'CodeElement',
'Community',
'Process',
'Section',
'Struct',
'Enum',
'Macro',
'Typedef',
'Union',
'Namespace',
'Trait',
'Impl',
'TypeAlias',
'Const',
'Static',
'Variable',
'Property',
'Record',
'Delegate',
'Annotation',
'Constructor',
'Template',
'Module',
'Route',
'Tool',
] as const;
export type NodeTableName = (typeof NODE_TABLES)[number];
export const REL_TABLE_NAME = 'CodeRelation';
export const REL_TYPES = [
'CONTAINS',
'DEFINES',
'IMPORTS',
'CALLS',
'EXTENDS',
'IMPLEMENTS',
'HAS_METHOD',
'HAS_PROPERTY',
'ACCESSES',
'METHOD_OVERRIDES',
'OVERRIDES', // Legacy compat alias — kept until all stored indexes are migrated
'METHOD_IMPLEMENTS',
'MEMBER_OF',
'STEP_IN_PROCESS',
'HANDLES_ROUTE',
'FETCHES',
'HANDLES_TOOL',
'ENTRY_POINT_OF',
'WRAPS',
'QUERIES',
] as const;
export type RelType = (typeof REL_TYPES)[number];
export const EMBEDDING_TABLE_NAME = 'CodeEmbedding';

Some files were not shown because too many files have changed in this diff Show More