Compare commits

...
Author SHA1 Message Date
Gergő Magyar 95b0176413 Merge branch 'main' into feat/ci-cascading-change-detection 2026-05-14 17:39:11 +01:00
Gergo Magyar 1688ef2ab3 fix(review): address all residual ce-code-review findings
Five reviewers (maintainability, reliability, testing, agent-native,
adversarial) flagged that ci-report.yml's sticky PR comment ignores
scope-parity entirely. Add scope_parity_result to the artifact read
step, render it in the Pipeline Status table, and include it in the
OVERALL pass/fail condition. Skipped is now visualised distinctly from
cancelled (skipped → ⏭, cancelled → 🛑).

Add `.github/scripts/check-language-filters.py` and wire it into
ci-quality.yml's workflow-convention job. The script parses
MIGRATED_LANGUAGES from registry-primary-flag.ts and asserts:

  1. ci.yml has a per-language filter block for every migrated slug
  2. ci.yml's FULL_LANG_LIST env var matches
  3. ci-scope-parity.yml's changed-languages default matches

This catches the most common drift mode: adding a language to
MIGRATED_LANGUAGES without updating the two YAML hardcoded lists,
which would silently make scope-parity skip the new language. Negative-
tested with a synthetic flag.ts that adds Kotlin without updating the
filter blocks — script correctly fails with all three location-
specific errors.

Move the resolver test-file existence check from the parity matrix
job into the discover job, iterating the full MIGRATED_LANGUAGES set
rather than just the filtered matrix. A missing test file now fails
the gate even on docs-only PRs that prune the matrix to empty,
holding the invariant regardless of which slugs were filtered.

Dismissed (verified out-of-scope): the performance reviewer's
suggestion to replace setup-gitnexus with a lighter setup in the
discover job. gitnexus-shared/package.json exports only ./dist/, so
`tsx` cannot resolve `import 'gitnexus-shared'` without the build
step. The optimisation would require modifying gitnexus-shared's
exports field to expose ./src/, which is a separate refactor.
2026-05-14 17:34:52 +01:00
Gergo Magyar 8674b2b453 ci: cascade quality gate, centralize change detection, prune scope-parity matrix
Add a `paths-changes` job to ci.yml that runs `dorny/paths-filter@v3`
ONCE per CI run and exposes three outputs to the four reusable
sub-workflows via `with:`:

  - web (e2e gate)
  - shared-resolution (forces full parity matrix)
  - changed-languages (JSON array of language slugs to run parity for)

Tier the four sub-workflows so cheap quality checks gate the expensive
tier. `quality` failure now auto-skips `tests`, `e2e`, and
`scope-parity` via `needs: quality` + `if: needs.quality.result ==
'success'`. The `ci-status` aggregator surfaces the originating failure
first so cascade-skipped downstream results don't produce misleading
secondary errors. The aggregator now also includes `paths-changes` in
its `needs:` so a transient filter API failure surfaces with the
correct error, not a fake "Quality jobs failed".

`shared-resolution` is the OR of six independent signals — any change
that could regress any language's parity forces the full matrix:

  - shared-ingestion   (gitnexus/src/core/ingestion/** minus languages/)
  - shared-types       (gitnexus-shared/**)
  - shared-parser-infra (gitnexus/src/core/tree-sitter/**)
  - shared-test-harness (resolver helpers + gitnexus/test/helpers/**)
  - shared-pipeline    (ci.yml, ci-quality.yml, ci-scope-parity.yml,
                        setup-gitnexus action, discovery script)
  - shared-config      (package.json, package-lock.json, vitest.config,
                        tsconfig)

`predicate-quantifier: 'every'` is required for the shared-ingestion
negation to actually exclude (dorny/paths-filter#184); the other
filters use default OR semantics. Both filter steps carry
`continue-on-error: true` so a transient REST API blip falls back to
the conservative full-battery posture rather than blocking the PR.

In `ci-scope-parity.yml`, the discover job intersects MIGRATED_LANGUAGES
with `changed-languages` using strict equality (`IN(.slug; $changed[])`,
not `inside()` which would substring-match `c` against `cpp`). When
`shared-resolution-changed=true` the full matrix runs. The discover
step validates both its own script output and the caller-supplied
`changed-languages` input as JSON arrays before passing them through
`jq --argjson`, so malformed input fails loud at the discover step
instead of producing `fromJSON('')` matrix crashes downstream. An
empty filtered matrix is handled by `if: needs.discover.outputs.languages != '[]'`
on the parity job; the outer workflow conclusion stays `success`.

`ci-e2e.yml` drops its in-workflow `check-changes` job and consumes the
centralized `web-changed` input.

`save-pr-meta` and `ci-status` switch from `if: always()` to
`if: ${{ !cancelled() }}` so user-initiated workflow cancellation
actually cancels them (actions/runner#491).

Dead backward-compat hyphenated artifact copies in `save-pr-meta`
removed — verified the `ci-report.yml` on main reads only
underscore-named files.

Conservative `'true'` defaults on every input for non-PR callers
(release-candidate.yml, publish.yml via workflow_call): the full
battery still runs on every push to main and every release.

Adds `GITHUB_STEP_SUMMARY` lines in both `paths-changes` and the
scope-parity `discover` step so reviewers and agents can see why a
tier was pruned vs. ran without scraping logs.
2026-05-14 17:27:16 +01:00
6 changed files with 755 additions and 74 deletions
+237
View File
@@ -0,0 +1,237 @@
#!/usr/bin/env python3
"""Enforce per-language CI filter consistency.
`gitnexus/src/core/ingestion/registry-primary-flag.ts` is the single source
of truth for which languages run through the RFC #909 Ring 3 scope-parity
gate. CI references that list in three places that must stay synchronised:
1. `.github/workflows/ci.yml` — per-language filter blocks under the
`paths-changes` job's `Detect web, shared, and per-language changes`
step. Each migrated language MUST have a `<slug>:` filter block,
otherwise PRs touching that language won't trigger its parity entry.
2. `.github/workflows/ci.yml` — the `FULL_LANG_LIST` env var inside the
`Compute outputs` step. Drives the conservative full-matrix default
for non-PR callers (release-candidate, publish via workflow_call)
AND the fallback when `dorny/paths-filter` itself fails.
3. `.github/workflows/ci-scope-parity.yml` — the `changed-languages`
input's `default:` value. Drives the conservative full matrix when
a caller invokes scope-parity without passing the input.
When a language is added to or removed from `MIGRATED_LANGUAGES`, all
three locations must change in the same commit, or the parity gate
silently under-runs (new language not validated) or over-runs (dropped
language still consumes runner time). This script asserts the three
locations agree with the TypeScript source of truth.
Invoked from `.github/workflows/ci-quality.yml`. Runs locally too:
python3 .github/scripts/check-language-filters.py [repo-root]
The script is dependency-free regex parsing — no Node.js setup required
on the CI runner. The grammar of the TS source-of-truth file is stable
enough that a regex is preferable to spawning `tsx` for a five-second
check.
"""
from __future__ import annotations
import pathlib
import re
import sys
# Parses `Python = 'python',` or `Python = "python",` inside the
# `SupportedLanguages` enum. Captures (TypeScript identifier, string slug).
ENUM_MEMBER_RE = re.compile(r"^\s*(\w+)\s*=\s*['\"]([^'\"]+)['\"]")
# Parses `SupportedLanguages.Python,` inside the `MIGRATED_LANGUAGES` set
# body. Captures the TypeScript identifier.
MIGRATED_REF_RE = re.compile(r"SupportedLanguages\.(\w+)")
# Pull the bracketed JSON-array string out of a `FULL_LANG_LIST: '[...]'`
# or `default: '[...]'` line. Captures the inner array including brackets.
ARRAY_LITERAL_RE = re.compile(r"\[[^\]]*\]")
def read_enum_map(languages_ts: pathlib.Path) -> dict[str, str]:
"""Return TypeScript-identifier -> slug for every SupportedLanguages entry."""
out: dict[str, str] = {}
inside_enum = False
for raw in languages_ts.read_text(encoding="utf-8").splitlines():
if not inside_enum:
if "export enum SupportedLanguages" in raw:
inside_enum = True
continue
if "}" in raw:
break
m = ENUM_MEMBER_RE.match(raw)
if m:
out[m.group(1)] = m.group(2)
if not out:
print(f"::error file={languages_ts}::could not parse SupportedLanguages enum")
sys.exit(2)
return out
def read_migrated_slugs(flag_ts: pathlib.Path, enum_map: dict[str, str]) -> list[str]:
"""Return the slug list referenced by MIGRATED_LANGUAGES, in source order."""
text = flag_ts.read_text(encoding="utf-8")
# Restrict the search to the body of the MIGRATED_LANGUAGES const so we
# don't accidentally pick up unrelated SupportedLanguages.X references
# elsewhere in the file (e.g. inside doc comments or other helpers).
start = text.find("MIGRATED_LANGUAGES")
if start == -1:
print(f"::error file={flag_ts}::could not locate MIGRATED_LANGUAGES export")
sys.exit(2)
open_bracket = text.find("[", start)
close_bracket = text.find("]", open_bracket)
if open_bracket == -1 or close_bracket == -1:
print(f"::error file={flag_ts}::MIGRATED_LANGUAGES body not bracketed by [ ... ]")
sys.exit(2)
body = text[open_bracket : close_bracket + 1]
slugs: list[str] = []
for m in MIGRATED_REF_RE.finditer(body):
ident = m.group(1)
slug = enum_map.get(ident)
if slug is None:
print(
f"::error file={flag_ts}::MIGRATED_LANGUAGES references "
f"SupportedLanguages.{ident}, which is not declared in the enum"
)
sys.exit(2)
slugs.append(slug)
if not slugs:
print(f"::error file={flag_ts}::MIGRATED_LANGUAGES is empty — parity gate will not run")
sys.exit(2)
return slugs
def read_filter_blocks(ci_yml: pathlib.Path, slugs: list[str]) -> set[str]:
"""Return the subset of `slugs` that have a filter block in ci.yml.
The filter block grammar is a YAML key at consistent indentation:
python:
- 'gitnexus/src/core/ingestion/languages/python/**'
We match each candidate slug as `<slug>:` at some indent level. Anchoring
to start-of-line + whitespace would also match other YAML keys (e.g. a
field named `python:` elsewhere), so we restrict the search to a
window after the `Detect web, shared, and per-language changes` step
marker to keep false positives out.
"""
text = ci_yml.read_text(encoding="utf-8")
marker = "Detect web, shared, and per-language changes"
start = text.find(marker)
if start == -1:
print(
f"::error file={ci_yml}::could not locate "
f"'{marker}' step — language-filter consistency cannot be verified"
)
sys.exit(2)
# Window is from the marker to the next top-level `- name:` step or the
# next job declaration (anything starting at column 6 or fewer).
rest = text[start:]
found: set[str] = set()
for slug in slugs:
# `<slug>:` flush with the filter indent level (12 spaces in the
# canonical layout). The pattern intentionally matches any
# whitespace indent to tolerate minor reflow.
pat = re.compile(rf"^\s+{re.escape(slug)}:\s*$", re.MULTILINE)
if pat.search(rest):
found.add(slug)
return found
def read_array_literal(workflow_yml: pathlib.Path, line_prefix: str) -> list[str]:
"""Locate the FIRST line containing `line_prefix` and parse its JSON-array literal."""
text = workflow_yml.read_text(encoding="utf-8")
for raw in text.splitlines():
if line_prefix in raw:
m = ARRAY_LITERAL_RE.search(raw)
if m is None:
print(
f"::error file={workflow_yml}::found '{line_prefix}' line "
f"but could not extract JSON-array literal"
)
sys.exit(2)
inner = m.group(0).strip("[]").strip()
if not inner:
return []
parts = [p.strip().strip('"').strip("'") for p in inner.split(",")]
return [p for p in parts if p]
print(f"::error file={workflow_yml}::could not locate '{line_prefix}' line")
sys.exit(2)
def check(repo_root: pathlib.Path) -> int:
languages_ts = repo_root / "gitnexus-shared" / "src" / "languages.ts"
flag_ts = repo_root / "gitnexus" / "src" / "core" / "ingestion" / "registry-primary-flag.ts"
ci_yml = repo_root / ".github" / "workflows" / "ci.yml"
scope_parity_yml = repo_root / ".github" / "workflows" / "ci-scope-parity.yml"
for path in (languages_ts, flag_ts, ci_yml, scope_parity_yml):
if not path.is_file():
print(f"::error file={path}::required file missing")
return 2
enum_map = read_enum_map(languages_ts)
expected = read_migrated_slugs(flag_ts, enum_map)
expected_set = set(expected)
fail = 0
# 1. Each migrated slug has a filter block in ci.yml.
found = read_filter_blocks(ci_yml, expected)
missing = expected_set - found
if missing:
for slug in sorted(missing):
print(
f"::error file={ci_yml}::no filter block found for migrated "
f"language '{slug}' under the paths-changes step. Add a "
f"`{slug}:` filter block with the language's ingestion + "
f"resolver-test path globs, or remove the language from "
f"MIGRATED_LANGUAGES."
)
fail = 1
# 2. ci.yml's FULL_LANG_LIST env var matches the source-of-truth slug set.
actual_ci_full = read_array_literal(ci_yml, "FULL_LANG_LIST:")
if set(actual_ci_full) != expected_set:
print(
f"::error file={ci_yml}::FULL_LANG_LIST in the paths-changes "
f"compute step does not match MIGRATED_LANGUAGES.\n"
f" expected: {sorted(expected_set)}\n"
f" actual: {sorted(actual_ci_full)}"
)
fail = 1
# 3. ci-scope-parity.yml's changed-languages default matches.
actual_sp_default = read_array_literal(scope_parity_yml, "default: '[")
if set(actual_sp_default) != expected_set:
print(
f"::error file={scope_parity_yml}::`changed-languages` input "
f"default does not match MIGRATED_LANGUAGES.\n"
f" expected: {sorted(expected_set)}\n"
f" actual: {sorted(actual_sp_default)}"
)
fail = 1
if fail == 0:
print(f"Per-language filter consistency: OK ({len(expected)} languages)")
print(f" MIGRATED_LANGUAGES: {expected}")
return fail
def main(argv: list[str]) -> int:
if len(argv) > 2:
print(f"usage: {argv[0]} [repo-root]", file=sys.stderr)
return 2
repo_root = pathlib.Path(argv[1]) if len(argv) == 2 else pathlib.Path.cwd()
if not repo_root.is_dir():
print(f"not a directory: {repo_root}", file=sys.stderr)
return 2
return check(repo_root)
if __name__ == "__main__":
sys.exit(main(sys.argv))
+17 -17
View File
@@ -2,30 +2,30 @@ name: E2E Tests
on:
workflow_call:
inputs:
web-changed:
description: >-
Whether files under `gitnexus-web/**` (or shared types in
`gitnexus-shared/**`) changed on this PR. Computed by the
centralised `paths-changes` job in `ci.yml` and passed in
via `with:`. When `'false'`, the e2e job skips. For non-PR
callers (release-candidate.yml) `paths-changes` emits `'true'`
conservatively so the full battery still runs.
required: false
type: string
default: 'true'
permissions:
contents: read
jobs:
check-changes:
name: Check web module changes
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
web_changed: ${{ steps.filter.outputs.web }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v3
id: filter
with:
filters: |
web:
- 'gitnexus-web/**'
e2e:
name: e2e (chromium)
needs: check-changes
if: needs.check-changes.result == 'success' && needs.check-changes.outputs.web_changed == 'true'
# Centralised change detection: ci.yml computes `web` ONCE and passes
# it here via `inputs.web-changed`. The legacy in-workflow `check-changes`
# job (which ran `dorny/paths-filter` on its own) is removed — the same
# detection now feeds every sub-workflow from a single source.
if: inputs.web-changed == 'true'
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
+13 -1
View File
@@ -63,7 +63,7 @@ jobs:
# Reusability is detected by parsing each workflow's `on:` block, not an
# allowlist, so new reusable workflows never produce false positives.
workflow-convention:
name: Workflow concurrency convention
name: Workflow conventions
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
@@ -73,3 +73,15 @@ jobs:
run: |
set -euo pipefail
python3 .github/scripts/check-workflow-concurrency.py .github/workflows
# Asserts ci.yml's per-language filter blocks AND the FULL_LANG_LIST
# / changed-languages defaults agree with MIGRATED_LANGUAGES in
# registry-primary-flag.ts. Catches the most common scope-parity
# drift mode: a contributor adds a language to MIGRATED_LANGUAGES
# but forgets to add the corresponding `<slug>:` filter block in
# ci.yml, which would silently make path-aware pruning skip the
# new language's parity entry.
- name: Validate per-language filter consistency
shell: bash
run: |
set -euo pipefail
python3 .github/scripts/check-language-filters.py
+13 -3
View File
@@ -121,6 +121,7 @@ jobs:
echo "quality=$(validate_result "$DIR/quality_result")"
echo "tests=$(validate_result "$DIR/tests_result")"
echo "e2e=$(validate_result "$DIR/e2e_result")"
echo "scope_parity=$(validate_result "$DIR/scope_parity_result")"
} >> "$GITHUB_OUTPUT"
- name: Checkout (for vitest config)
@@ -215,6 +216,7 @@ jobs:
QUALITY: ${{ steps.meta.outputs.quality }}
TESTS: ${{ steps.meta.outputs.tests }}
E2E: ${{ steps.meta.outputs.e2e }}
SCOPE_PARITY: ${{ steps.meta.outputs.scope_parity }}
BASE_FOUND: ${{ steps.base-coverage.outputs.found }}
BASE_DIR: ${{ steps.base-coverage.outputs.dir }}
RUN_URL: ${{ github.event.workflow_run.html_url }}
@@ -299,7 +301,8 @@ jobs:
case "$1" in
success) echo "✅" ;;
failure) echo "❌" ;;
cancelled) echo "⏭️" ;;
skipped) echo "⏭️" ;;
cancelled) echo "🛑" ;;
*) echo "❓" ;;
esac
}
@@ -344,7 +347,13 @@ jobs:
}
# ── Overall status ──
if [[ "$QUALITY" == "success" && "$TESTS" == "success" && ("$E2E" == "success" || "$E2E" == "skipped") ]]; then
# scope-parity may also report `skipped` cleanly when the parity
# matrix was pruned to empty (no migrated-language source AND no
# shared-resolution paths changed); we accept that as a pass.
if [[ "$QUALITY" == "success" \
&& "$TESTS" == "success" \
&& ( "$E2E" == "success" || "$E2E" == "skipped" ) \
&& ( "$SCOPE_PARITY" == "success" || "$SCOPE_PARITY" == "skipped" || "$SCOPE_PARITY" == "unknown" ) ]]; then
OVERALL="✅ **All checks passed**"
else
OVERALL="❌ **Some checks failed**"
@@ -361,9 +370,10 @@ jobs:
echo ""
echo "| Stage | Status | Details |"
echo "|-------|--------|---------|"
echo "| $(status_icon "$QUALITY") Typecheck | \`${QUALITY}\` | tsc --noEmit |"
echo "| $(status_icon "$QUALITY") Quality | \`${QUALITY}\` | format, lint, typecheck, convention |"
echo "| $(status_icon "$TESTS") Tests | \`${TESTS}\` | unit tests, 3 platforms |"
echo "| $(status_icon "$E2E") E2E | \`${E2E}\` | gitnexus-web changes only |"
echo "| $(status_icon "$SCOPE_PARITY") Scope parity | \`${SCOPE_PARITY}\` | RFC #909 Ring 3, per-language pruning |"
echo ""
if [ "$TOTAL" -gt 0 ] 2>/dev/null; then
+120 -27
View File
@@ -21,12 +21,45 @@ name: Scope Resolution Parity
# discovers it, runs parity, and the language's default production path
# flips to registry-primary in the same change.
#
# When the set is empty (e.g. mid-Ring-3 for every language), the parity
# matrix is skipped and the workflow reports success — no-op until a
# language is explicitly claimed migrated.
# ── Path-aware matrix pruning ─────────────────────────────────────────
# `ci.yml` computes change detection ONCE per run and passes:
# - `shared-resolution-changed` — when 'true', shared ingestion code
# changed and ALL migrated languages must run (any could regress).
# - `changed-languages` — JSON array of language slugs whose source or
# resolver-test files changed (e.g. `["python","go"]`).
#
# The discover job intersects MIGRATED_LANGUAGES with `changed-languages`
# unless `shared-resolution-changed=true`, in which case the full matrix
# runs. On non-PR callers (release-candidate.yml) both inputs default to
# the full set, so the full matrix runs there too.
#
# When the filtered matrix is empty, the matrix job is skipped via the
# `if: needs.discover.outputs.languages != '[]'` guard. The outer workflow
# conclusion stays `success` so the `ci-status` aggregator in ci.yml
# passes the scope-parity gate.
on:
workflow_call:
inputs:
shared-resolution-changed:
description: >-
When `'true'`, shared ingestion/resolution code changed and the
full migrated-languages matrix must run. Conservatively defaults
to `'true'` so any caller that omits the input gets the full
battery.
required: false
type: string
default: 'true'
changed-languages:
description: >-
JSON array of migrated language slugs whose source or resolver
test files changed on this PR (e.g. `["python","go"]`). Used to
prune the parity matrix when `shared-resolution-changed` is
`'false'`. Defaults to the full set so omitting the input runs
everything.
required: false
type: string
default: '["python","csharp","typescript","go","c","cpp","php"]'
permissions:
contents: read
@@ -38,32 +71,100 @@ jobs:
timeout-minutes: 5
outputs:
languages: ${{ steps.read.outputs.languages }}
has-any: ${{ steps.read.outputs.has-any }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: ./.github/actions/setup-gitnexus
- name: Extract MIGRATED_LANGUAGES from registry-primary-flag.ts
- name: Extract MIGRATED_LANGUAGES and filter by changed paths
id: read
shell: bash
working-directory: gitnexus
env:
SHARED_CHANGED: ${{ inputs.shared-resolution-changed }}
CHANGED_LANGS: ${{ inputs.changed-languages }}
run: |
set -euo pipefail
# `tsx` evaluates the TS source directly (no build step), imports
# the exported `Set`, and emits a GH-Actions-friendly JSON matrix.
LANGS=$(npx tsx scripts/ci-list-migrated-languages.ts)
COUNT=$(printf '%s' "$LANGS" | jq 'length')
HAS_ANY="false"
if [[ "$COUNT" -gt 0 ]]; then HAS_ANY="true"; fi
# the exported `Set`, and emits a GH-Actions-friendly JSON matrix
# array: [{slug, envvar}, ...].
ALL_LANGS=$(npx tsx scripts/ci-list-migrated-languages.ts)
# Validate the discovery script's output is a well-formed JSON
# array. A crash or schema regression here would otherwise
# propagate to `fromJSON('')` at matrix expansion time, where
# the failure surface is much harder to read.
if ! printf '%s' "$ALL_LANGS" | jq -e 'type == "array"' >/dev/null; then
echo "::error::ci-list-migrated-languages.ts produced non-array output: $ALL_LANGS"
exit 1
fi
ALL_COUNT=$(printf '%s' "$ALL_LANGS" | jq 'length')
echo "MIGRATED_LANGUAGES: $ALL_LANGS ($ALL_COUNT entries)"
# Shared-resolution change forces the full matrix — any change
# to shared ingestion code could regress any language's parity.
if [ "${SHARED_CHANGED:-true}" = "true" ]; then
LANGS="$ALL_LANGS"
REASON="shared-resolution changed → running full matrix"
else
# Validate the caller-supplied CHANGED_LANGS is a JSON array
# of strings. Malformed input would otherwise crash `jq
# --argjson` with a hard-to-diagnose parse error inside the
# filter pipeline.
if ! printf '%s' "${CHANGED_LANGS:-[]}" | jq -e 'type == "array" and all(type == "string")' >/dev/null; then
echo "::error::changed-languages input is not a JSON array of strings: ${CHANGED_LANGS:-<unset>}"
exit 1
fi
# Intersect MIGRATED_LANGUAGES with the changed-languages list.
# `IN(.slug; $changed[])` is strict equality membership — using
# `inside()` here would substring-match (`"c"` would match
# `"cpp"` because the `c` string is a prefix), which would
# over-trigger the matrix. The empty-list case is handled by
# the parity job's `if: != '[]'` guard.
LANGS=$(jq -c --argjson changed "${CHANGED_LANGS:-[]}" \
'[.[] | select(IN(.slug; $changed[]))]' <<< "$ALL_LANGS")
REASON="pruned by changed-languages: $CHANGED_LANGS"
fi
echo "languages=$LANGS" >> "$GITHUB_OUTPUT"
echo "has-any=$HAS_ANY" >> "$GITHUB_OUTPUT"
echo "Discovered $COUNT migrated language(s): $LANGS"
echo "Parity matrix will run: $HAS_ANY"
COUNT=$(printf '%s' "$LANGS" | jq 'length')
echo "Parity matrix entries: $COUNT ($REASON)"
# Resolver test-file existence is verified against the FULL
# MIGRATED_LANGUAGES set (ALL_LANGS), not just the filtered
# matrix. This catches a missing test file even on a docs-only
# PR that prunes the matrix to empty — otherwise the invariant
# could only fail on a PR that happens to touch the affected
# language. The check runs from repo root because the test
# paths are relative to `gitnexus/`.
while IFS= read -r slug; do
TEST_FILE="test/integration/resolvers/${slug}.test.ts"
if [[ ! -f "$TEST_FILE" ]]; then
echo "::error title=Missing resolver test::Expected gitnexus/$TEST_FILE for migrated language '$slug'. Either fix the slug, add the test file, or remove the language from MIGRATED_LANGUAGES."
exit 1
fi
done < <(printf '%s' "$ALL_LANGS" | jq -r '.[].slug')
# Job-summary line so reviewers and agents can see which
# languages actually ran without scraping logs.
{
echo "### Scope-parity matrix"
echo ""
echo "- **Entries:** $COUNT of $ALL_COUNT migrated languages"
echo "- **Reason:** $REASON"
if [ "$COUNT" -gt 0 ]; then
SLUGS=$(printf '%s' "$LANGS" | jq -r 'map(.slug) | join(", ")')
echo "- **Slugs:** $SLUGS"
fi
} >> "$GITHUB_STEP_SUMMARY"
parity:
name: ${{ matrix.lang.slug }} parity
needs: discover
if: needs.discover.outputs.has-any == 'true'
# Empty-matrix guard. When the filter produced no entries, the matrix
# job MUST skip — `fromJSON('[]')` in `strategy.matrix` would otherwise
# crash with "Matrix vector does not contain any values"
# (community#27096). Comparing the JSON string against `'[]'` is the
# idiomatic, documented pattern for this guard.
if: needs.discover.outputs.languages != '[]'
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
@@ -79,19 +180,11 @@ jobs:
with:
build: 'true'
- name: Verify resolver test file exists
shell: bash
working-directory: gitnexus
run: |
set -euo pipefail
TEST_FILE="test/integration/resolvers/${{ matrix.lang.slug }}.test.ts"
if [[ ! -f "$TEST_FILE" ]]; then
echo "::error title=Missing resolver test::\
Expected $TEST_FILE for '${{ matrix.lang.slug }}' (listed in \
MIGRATED_LANGUAGES). Either fix the slug or add the test file \
before listing this language as migrated."
exit 1
fi
# Resolver test-file existence is now validated up front in the
# `discover` job against the full MIGRATED_LANGUAGES set, so a
# missing file fails before any matrix runner spins up — and the
# invariant holds even on docs-only PRs that prune the matrix to
# empty.
- name: Resolver tests — legacy DAG (REGISTRY_PRIMARY_${{ matrix.lang.envvar }}=0)
shell: bash
+355 -26
View File
@@ -25,43 +25,350 @@ concurrency:
# ── Reusable workflow orchestration ─────────────────────────────────
# Each concern lives in its own workflow file for maintainability:
# ci-quality.yml — typecheck (tsc --noEmit)
# ci-quality.yml — typecheck (tsc --noEmit), lint, format, workflow-convention
# ci-tests.yml — unit + integration tests with coverage + cross-platform
# ci-e2e.yml — E2E tests (only when gitnexus-web/ changes)
# ci-scope-parity.yml — RFC #909 Ring 3 parity gate: legacy DAG + registry-primary
# both pass, per migrated language in the JSON registry
# both pass, per migrated language. Matrix is filtered to
# only the languages whose source files changed.
#
# Shared setup is DRY via .github/actions/setup-gitnexus composite action.
#
# ── Cascading execution model ───────────────────────────────────────
# Tier 1 (fast fail-fast): paths-changes → quality
# Tier 2 (gated on quality): tests, e2e, scope-parity
# Quality failure short-circuits the expensive tier via `needs: quality` —
# the downstream jobs auto-skip with `result == 'skipped'`. The `ci-status`
# aggregator reports the quality failure first so the cascade reason is
# unambiguous.
#
# ── Path-based change detection ─────────────────────────────────────
# The `paths-changes` job runs `dorny/paths-filter` ONCE per CI run and
# exposes a stable output set the reusable sub-workflows consume via
# `with:`. Two filter steps run because `predicate-quantifier: every` is
# required to make the `shared-resolution` negation work, but the other
# filters need the default `some` (OR) semantics.
#
# For non-`pull_request` triggers (workflow_call from release-candidate),
# there is no PR base to diff against. The job emits conservative `'true'`
# defaults for every output so the release-candidate run keeps exercising
# the full battery.
jobs:
# ── Centralised change detection ────────────────────────────────
# outputs.web — gitnexus-web/ or shared types changed
# outputs.shared-resolution — shared ingestion code (NOT under languages/) changed
# outputs.changed-languages — JSON array of migrated language slugs whose
# source or resolver-test files changed
paths-changes:
name: Detect changed paths
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
# dorny/paths-filter reads PR file lists via the REST API on
# pull_request events. Required even though the default GITHUB_TOKEN
# gets this scope on fork PRs automatically — explicit grant is
# robust to future job-level permission tightening.
pull-requests: read
outputs:
web: ${{ steps.compute.outputs.web }}
shared-resolution: ${{ steps.compute.outputs.shared-resolution }}
changed-languages: ${{ steps.compute.outputs.changed-languages }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# Step 1 — shared-ingestion detection (negation-only).
# `predicate-quantifier: 'every'` is required for the negation
# (`!gitnexus/src/core/ingestion/languages/**`) to actually exclude
# — the default `'some'` quantifier silently ignores negation
# entries (dorny/paths-filter#184). This step is intentionally
# narrow: it is the ONLY filter that uses negation. Every other
# shared signal goes through Step 2's default-OR semantics where
# we can stack many independent patterns cleanly.
#
# `continue-on-error: true` ensures a transient GitHub REST API blip
# doesn't take down the whole CI run with a misleading "Quality jobs
# failed" message. The compute step below detects empty outputs and
# falls back to conservative `'true'` defaults (full battery) — the
# same posture as non-PR callers like release-candidate.yml.
- name: Detect shared-ingestion changes (negation step)
if: github.event_name == 'pull_request'
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v3
id: shared-filter
continue-on-error: true
with:
predicate-quantifier: 'every'
filters: |
shared-ingestion:
- 'gitnexus/src/core/ingestion/**'
- '!gitnexus/src/core/ingestion/languages/**'
# Step 2 — web, per-language, and the rest of the shared surface.
# Default `some` (OR) quantifier — a file matching ANY pattern in
# the filter triggers the filter.
#
# The per-language filters include both ingestion source AND
# resolver integration test files so a test-only edit (e.g.,
# python.test.ts) still triggers Python parity. The action's
# built-in `changes` output is the JSON array of matched filter
# names — we strip the non-language filter names (web, shared-*)
# in the compute step to produce the language matrix input.
#
# The shared-* filters are OR'd together in the compute step to
# derive the `shared-resolution` output. Anything that can regress
# the parity matrix for ANY language — shared ingestion code,
# shared types, parser infrastructure, the test harness, the CI
# pipeline itself, or dependency/config — forces the full matrix.
# Per-language pruning only applies when every shared-* filter is
# false AND only per-language files changed.
- name: Detect web, shared, and per-language changes
if: github.event_name == 'pull_request'
uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v3
id: filter
continue-on-error: true
with:
filters: |
web:
- 'gitnexus-web/**'
- 'gitnexus-shared/**'
shared-types:
- 'gitnexus-shared/**'
shared-parser-infra:
- 'gitnexus/src/core/tree-sitter/**'
shared-test-harness:
- 'gitnexus/test/integration/resolvers/helpers.ts'
- 'gitnexus/test/helpers/**'
shared-pipeline:
- '.github/workflows/ci.yml'
- '.github/workflows/ci-quality.yml'
- '.github/workflows/ci-scope-parity.yml'
- '.github/actions/setup-gitnexus/**'
- 'gitnexus/scripts/ci-list-migrated-languages.ts'
shared-config:
- 'gitnexus/package.json'
- 'gitnexus/package-lock.json'
- 'gitnexus/vitest.config.ts'
- 'gitnexus/tsconfig.json'
- 'gitnexus-shared/package.json'
- 'gitnexus-shared/package-lock.json'
python:
- 'gitnexus/src/core/ingestion/languages/python/**'
- 'gitnexus/src/core/ingestion/languages/python.ts'
- 'gitnexus/test/integration/resolvers/python.test.ts'
csharp:
- 'gitnexus/src/core/ingestion/languages/csharp/**'
- 'gitnexus/src/core/ingestion/languages/csharp.ts'
- 'gitnexus/test/integration/resolvers/csharp.test.ts'
typescript:
- 'gitnexus/src/core/ingestion/languages/typescript/**'
- 'gitnexus/src/core/ingestion/languages/typescript.ts'
- 'gitnexus/test/integration/resolvers/typescript*.test.ts'
go:
- 'gitnexus/src/core/ingestion/languages/go/**'
- 'gitnexus/src/core/ingestion/languages/go.ts'
- 'gitnexus/test/integration/resolvers/go.test.ts'
c:
- 'gitnexus/src/core/ingestion/languages/c/**'
- 'gitnexus/src/core/ingestion/languages/c-cpp.ts'
- 'gitnexus/test/integration/resolvers/c.test.ts'
cpp:
- 'gitnexus/src/core/ingestion/languages/cpp/**'
- 'gitnexus/src/core/ingestion/languages/c-cpp.ts'
- 'gitnexus/test/integration/resolvers/cpp.test.ts'
php:
- 'gitnexus/src/core/ingestion/languages/php/**'
- 'gitnexus/src/core/ingestion/languages/php.ts'
- 'gitnexus/test/integration/resolvers/php.test.ts'
# Step 3 — derive job outputs from the two filter steps OR (for
# non-PR events) emit conservative defaults. release-candidate.yml
# calls ci.yml via `workflow_call`; without PR base/head context
# dorny/paths-filter has no commit range, so we short-circuit it
# for any non-PR event and run the full validation battery.
#
# `shared-resolution` is the OR of every "any-language-could-
# regress" signal:
# - shared-ingestion (Step 1, predicate-quantifier=every)
# - shared-types (gitnexus-shared/**)
# - shared-parser-infra (tree-sitter loader/safe-parse)
# - shared-test-harness (resolver helpers + test/helpers/**)
# - shared-pipeline (ci.yml, ci-quality.yml, ci-scope-parity.yml,
# setup-gitnexus, discovery script)
# - shared-config (gitnexus package manifests, vitest config,
# tsconfig, gitnexus-shared manifests)
# When any of these fire, the full migrated-languages matrix runs.
# Per-language pruning only applies when EVERY shared-* filter is
# false and only per-language files changed.
#
# Conservative-default fallback: when a filter step's `continue-on-
# error` fires (transient GitHub REST API blip), its outputs are
# unset. We detect step.outcome != 'success' and short-circuit to
# the full-battery posture — same as a non-PR caller. The
# alternative (`:-false` per-var fallback) would mask the filter
# failure as "nothing changed" and silently skip every downstream
# gate. Better to over-run on a flake than under-run.
- name: Compute outputs
id: compute
shell: bash
env:
IS_PR: ${{ github.event_name == 'pull_request' }}
WEB: ${{ steps.filter.outputs.web }}
# Shared signals OR'd into the final `shared-resolution` output.
SHARED_INGESTION: ${{ steps.shared-filter.outputs.shared-ingestion }}
SHARED_TYPES: ${{ steps.filter.outputs.shared-types }}
SHARED_PARSER_INFRA: ${{ steps.filter.outputs.shared-parser-infra }}
SHARED_TEST_HARNESS: ${{ steps.filter.outputs.shared-test-harness }}
SHARED_PIPELINE: ${{ steps.filter.outputs.shared-pipeline }}
SHARED_CONFIG: ${{ steps.filter.outputs.shared-config }}
# `changes` is dorny/paths-filter's built-in output: a JSON
# string-array listing every filter name that matched. We strip
# non-language filter names to get the per-language matrix input.
CHANGES: ${{ steps.filter.outputs.changes }}
# Step outcomes feed the fallback: if a filter step failed,
# treat shared-resolution as `true` and run the full matrix.
SHARED_FILTER_OUTCOME: ${{ steps.shared-filter.outcome }}
FILTER_OUTCOME: ${{ steps.filter.outcome }}
FULL_LANG_LIST: '["python","csharp","typescript","go","c","cpp","php"]'
# Non-language filter names — anything in this list is stripped
# from `changes` when computing `changed-languages`. Keep this
# set in sync with the filter definitions in Step 2 above.
NON_LANG_FILTERS: '["web","shared-types","shared-parser-infra","shared-test-harness","shared-pipeline","shared-config"]'
run: |
set -euo pipefail
if [ "$IS_PR" = "true" ]; then
# Filter-step failure → conservative full-battery posture.
if [ "$SHARED_FILTER_OUTCOME" != "success" ] || [ "$FILTER_OUTCOME" != "success" ]; then
echo "::warning::dorny/paths-filter failed (shared=$SHARED_FILTER_OUTCOME, filter=$FILTER_OUTCOME) — falling back to full battery"
{
echo "web=true"
echo "shared-resolution=true"
echo "changed-languages=$FULL_LANG_LIST"
} >> "$GITHUB_OUTPUT"
{
echo "### paths-changes — filter-step failure"
echo ""
echo "shared=\`$SHARED_FILTER_OUTCOME\`, filter=\`$FILTER_OUTCOME\` → running full battery"
} >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
# OR the six shared signals together into the final
# `shared-resolution` flag. Any 'true' wins.
SHARED="false"
for v in "$SHARED_INGESTION" "$SHARED_TYPES" "$SHARED_PARSER_INFRA" \
"$SHARED_TEST_HARNESS" "$SHARED_PIPELINE" "$SHARED_CONFIG"; do
if [ "$v" = "true" ]; then SHARED="true"; fi
done
# Filter the `changes` array down to language slugs only by
# removing non-language filter names. `jq -c` keeps the output
# compact so it round-trips cleanly through $GITHUB_OUTPUT and
# downstream `fromJSON()`.
CHANGED_LANGS=$(jq -c --argjson nonlang "$NON_LANG_FILTERS" \
'[.[] | select(IN(.; $nonlang[]) | not)]' <<< "${CHANGES:-[]}")
{
echo "web=${WEB:-true}"
echo "shared-resolution=$SHARED"
echo "changed-languages=$CHANGED_LANGS"
} >> "$GITHUB_OUTPUT"
{
echo "### paths-changes (pull_request)"
echo ""
echo "| Output | Value |"
echo "|---|---|"
echo "| web | ${WEB:-true} |"
echo "| **shared-resolution** | **$SHARED** |"
echo "| changed-languages | \`$CHANGED_LANGS\` |"
echo ""
echo "Shared signal sources:"
echo ""
echo "| Signal | Matched? |"
echo "|---|---|"
echo "| shared-ingestion (ingestion/ minus languages/) | ${SHARED_INGESTION:-false} |"
echo "| shared-types (gitnexus-shared/) | ${SHARED_TYPES:-false} |"
echo "| shared-parser-infra (tree-sitter/) | ${SHARED_PARSER_INFRA:-false} |"
echo "| shared-test-harness (test helpers) | ${SHARED_TEST_HARNESS:-false} |"
echo "| shared-pipeline (ci.yml + setup-gitnexus + discovery script) | ${SHARED_PIPELINE:-false} |"
echo "| shared-config (package.json, vitest/tsconfig) | ${SHARED_CONFIG:-false} |"
} >> "$GITHUB_STEP_SUMMARY"
else
# Conservative defaults for workflow_call / push / dispatch:
# run the full battery. release-candidate validates the
# entire codebase on every push to main regardless of diff.
{
echo "web=true"
echo "shared-resolution=true"
echo "changed-languages=$FULL_LANG_LIST"
} >> "$GITHUB_OUTPUT"
{
echo "### paths-changes (non-PR — full matrix)"
echo "Event: \`$GITHUB_EVENT_NAME\`"
} >> "$GITHUB_STEP_SUMMARY"
fi
# ── Tier 1: cheap quality gates ─────────────────────────────────
# Format, lint, typecheck (gitnexus + web), and workflow-convention.
# Median runtime under 5 minutes — fails fast on the most common
# contributor mistakes (missing format, lint errors, type errors).
quality:
needs: paths-changes
uses: ./.github/workflows/ci-quality.yml
permissions:
contents: read
# ── Tier 2: expensive validation, gated on quality success ──────
# `needs: quality` makes these auto-skip when quality fails — saves
# ~25 minutes of Vitest matrix + ~5-15 minutes of scope-parity matrix
# for the most common failure mode (a contributor PR that misses
# format/lint/typecheck locally).
tests:
needs: [paths-changes, quality]
if: needs.quality.result == 'success'
uses: ./.github/workflows/ci-tests.yml
permissions:
contents: read
# E2E is conditionally gated on `gitnexus-web/**` changes via the
# `web` input. When unchanged, the inner workflow's `e2e` job skips
# and the outer reusable-workflow conclusion is `success` (a skipped
# matrix with no parity entries returns success). The `ci-status`
# aggregator accepts `success` OR `skipped` for the e2e gate.
e2e:
needs: [paths-changes, quality]
if: needs.quality.result == 'success'
uses: ./.github/workflows/ci-e2e.yml
permissions:
contents: read
with:
web-changed: ${{ needs.paths-changes.outputs.web }}
# Scope-parity consumes shared-resolution + changed-languages. The
# discovery job inside the workflow uses changed-languages to filter
# MIGRATED_LANGUAGES; when shared-resolution=true the full matrix runs.
scope-parity:
needs: [paths-changes, quality]
if: needs.quality.result == 'success'
uses: ./.github/workflows/ci-scope-parity.yml
permissions:
contents: read
with:
shared-resolution-changed: ${{ needs.paths-changes.outputs.shared-resolution }}
changed-languages: ${{ needs.paths-changes.outputs.changed-languages }}
# ── Save PR metadata for the reporting workflow ─────────────────
# The ci-report.yml workflow (triggered by workflow_run) needs the
# PR number and job results to post a comment. We save them as an
# artifact because workflow_run context doesn't reliably carry PR
# info for fork PRs.
#
# `!cancelled()` instead of `always()` so a user-initiated workflow
# cancellation actually cancels this job too. `always()` is documented
# to make a job uncancellable (actions/runner#491), which masks
# cancel-button presses.
save-pr-meta:
name: Save PR Metadata
if: always() && github.event_name == 'pull_request'
if: ${{ !cancelled() && github.event_name == 'pull_request' }}
needs: [quality, tests, e2e, scope-parity]
runs-on: ubuntu-latest
timeout-minutes: 5
@@ -81,16 +388,6 @@ jobs:
echo "$TESTS" > pr-meta/tests_result
echo "$E2E" > pr-meta/e2e_result
echo "$SCOPE_PARITY" > pr-meta/scope_parity_result
# TODO(post-merge): remove backward-compat copies once ci-report.yml
# on main reads underscore names.
# Backward-compat: ci-report.yml on main still reads hyphenated
# names. workflow_run always executes from the default branch, so
# the main-branch reader won't find the underscore variants until
# this PR is merged. Write both until then.
cp pr-meta/pr_number pr-meta/pr-number
cp pr-meta/quality_result pr-meta/quality-result
cp pr-meta/tests_result pr-meta/tests-result
cp pr-meta/e2e_result pr-meta/e2e-result
- name: Upload PR metadata
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
@@ -100,42 +397,74 @@ jobs:
retention-days: 1
# ── Unified CI gate ──────────────────────────────────────────────
# Single required check for branch protection.
# Single required check for branch protection. This is THE check the
# rest of the workflow rolls up into — see emmer.dev's "Skippable
# GitHub Status Checks Aren't Really Required" for the rationale.
# The originating failure is surfaced FIRST so cascade-skipped
# downstream results don't produce a misleading secondary error.
#
# `!cancelled()` instead of `always()` for the same reason as
# save-pr-meta above — a cancel button press should cancel this too.
ci-status:
name: CI Gate
needs: [quality, tests, e2e, scope-parity]
if: always()
needs: [paths-changes, quality, tests, e2e, scope-parity]
if: ${{ !cancelled() }}
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check all jobs passed
shell: bash
env:
PATHS_CHANGES: ${{ needs.paths-changes.result }}
QUALITY: ${{ needs.quality.result }}
TESTS: ${{ needs.tests.result }}
E2E: ${{ needs.e2e.result }}
SCOPE_PARITY: ${{ needs.scope-parity.result }}
run: |
echo "Paths-changes: $PATHS_CHANGES"
echo "Quality: $QUALITY"
echo "Tests: $TESTS"
echo "E2E: $E2E"
echo "Scope parity: $SCOPE_PARITY"
if [[ "$QUALITY" != "success" ]] ||
[[ "$TESTS" != "success" ]]; then
echo "::error::Quality or test jobs failed"
# Walk the cascade top-down so the originating failure is the
# one we report. Filter steps have `continue-on-error: true`,
# so a transient REST API failure inside `paths-changes` does
# NOT take down the job — the compute step falls back to a
# full-battery posture and `PATHS_CHANGES=success`. The only
# way we see `PATHS_CHANGES != 'success'` is a genuine job-
# level failure (runner crash, timeout, dispatch failure).
if [[ "$PATHS_CHANGES" != "success" ]]; then
echo "::error::Path-change detection failed — downstream tiers were skipped via cascading needs"
exit 1
fi
# Quality is the first cheap gate. Its failure auto-skips
# tests/e2e/scope-parity via `needs: quality`. Those `skipped`
# results are an expected cascade, not a separate bug, so we
# check quality before the dependent jobs.
if [[ "$QUALITY" != "success" ]]; then
echo "::error::Quality jobs failed — downstream tiers were skipped via cascading needs"
exit 1
fi
# Tests should succeed when quality succeeded. `skipped` here
# only happens when an upstream `needs` failed, which the two
# checks above already handled — so any non-success at this
# point is a real test failure.
if [[ "$TESTS" != "success" ]]; then
echo "::error::Test jobs failed"
exit 1
fi
if [[ "$E2E" != "success" && "$E2E" != "skipped" ]]; then
echo "::error::E2E job failed"
exit 1
fi
# scope-parity is a reusable workflow. With an empty migrated-
# languages list, its parity matrix is skipped and the outer
# workflow still reports `success`. If any entry's legacy-DAG or
# registry-primary run fails, the workflow reports `failure`.
# Accept only `success`; `skipped` would mean the entire
# discover job was skipped too (upstream failure), which should
# still block.
# scope-parity is a reusable workflow. When changed-languages
# is empty AND shared-resolution didn't change, the parity
# matrix is pruned to zero but `discover` itself runs and
# succeeds, so the outer workflow conclusion is `success` —
# which this gate accepts. The only ways we'd see `skipped`
# here are upstream `needs` failures already caught above.
# If any matrix entry's legacy-DAG or registry-primary run
# fails, the outer workflow conclusion is `failure`.
if [[ "$SCOPE_PARITY" != "success" ]]; then
echo "::error::Scope-resolution parity gate failed (RFC #909 Ring 3)"
exit 1