Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4ceb6e1568 | ||
|
|
96578aa4a8 | ||
|
|
a418c47e29 | ||
|
|
05ca80ea30 | ||
|
|
e55256ba53 | ||
|
|
9d91530f94 | ||
|
|
46e4c979c4 | ||
|
|
8fc32e6af9 | ||
|
|
e60e62f193 | ||
|
|
816ae5e66e | ||
|
|
4048f53e35 | ||
|
|
027340292f | ||
|
|
cbe5dac8b7 | ||
|
|
bf11269260 | ||
|
|
f10135649e | ||
|
|
3732fa1e21 | ||
|
|
0add072f25 |
+1
-1
@@ -19,7 +19,7 @@ services:
|
||||
- ${WORKSPACE_DIR:-./workspace}:/workspace:ro
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ['CMD', 'curl', '-fsSI', 'http://localhost:4747/api/heartbeat']
|
||||
test: ['CMD', 'curl', '-f', 'http://localhost:4747/api/health']
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
# Using GitNexus across Apache Thrift microservices
|
||||
|
||||
## When to use this guide
|
||||
|
||||
Use this guide when several repositories communicate through Apache Thrift and you want GitNexus to trace impact across provider and consumer boundaries. The walkthrough assumes each service is indexed on its own, then joined through a GitNexus group.
|
||||
|
||||
This is not a framework integration guide. GitNexus reads portable Thrift IDL and common Java generated-code shapes. Framework-specific wiring, service discovery, deployment metadata, and private annotations belong outside the open-source core.
|
||||
|
||||
## Mental model
|
||||
|
||||
- `.thrift` files define the canonical service contract. A method in an IDL service becomes a stable contract id in the form `thrift::<namespace>.<Service>/<Method>`.
|
||||
- Service wildcard ids in the form `thrift::<namespace>.<Service>/*` are supported as manifest and matching fallback forms when a service-level link is needed.
|
||||
- Java generated-code usage points GitNexus toward implementation and call sites. Providers commonly implement generated `Service.Iface`; consumers commonly hold or construct generated service interfaces or clients.
|
||||
- Group sync matches provider and consumer contracts with the same id, then cross-repo impact can hop through those links.
|
||||
- Framework-specific wiring should be modeled by extractor plugins, manifest links, or downstream integrations rather than hard-coded into core Thrift support.
|
||||
|
||||
## Fictional IDL
|
||||
|
||||
```thrift
|
||||
namespace java billing.v1
|
||||
|
||||
struct PlaceOrderRequest {
|
||||
1: string orderId
|
||||
2: double amount
|
||||
}
|
||||
|
||||
struct PlaceOrderResponse {
|
||||
1: bool accepted
|
||||
}
|
||||
|
||||
struct GetOrderRequest {
|
||||
1: string orderId
|
||||
}
|
||||
|
||||
struct GetOrderResponse {
|
||||
1: string orderId
|
||||
2: string status
|
||||
}
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
GetOrderResponse GetOrder(1: GetOrderRequest request)
|
||||
}
|
||||
```
|
||||
|
||||
The service methods above produce canonical ids:
|
||||
|
||||
- `thrift::billing.v1.OrderService/PlaceOrder`
|
||||
- `thrift::billing.v1.OrderService/GetOrder`
|
||||
- `thrift::billing.v1.OrderService/*` as a service-level manifest or matching fallback form
|
||||
|
||||
## Java provider example
|
||||
|
||||
Generated Java code usually exposes an `Iface` interface for the service. A provider implementation can be detected when it implements that generated interface.
|
||||
|
||||
```java
|
||||
package example.billing;
|
||||
|
||||
import billing.v1.GetOrderRequest;
|
||||
import billing.v1.GetOrderResponse;
|
||||
import billing.v1.OrderService;
|
||||
import billing.v1.PlaceOrderRequest;
|
||||
import billing.v1.PlaceOrderResponse;
|
||||
|
||||
public final class OrderServiceHandler implements OrderService.Iface {
|
||||
@Override
|
||||
public PlaceOrderResponse PlaceOrder(PlaceOrderRequest request) {
|
||||
return new PlaceOrderResponse(true);
|
||||
}
|
||||
|
||||
@Override
|
||||
public GetOrderResponse GetOrder(GetOrderRequest request) {
|
||||
return new GetOrderResponse(request.getOrderId(), "CREATED");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
With the IDL available, GitNexus can connect the implementation to `thrift::billing.v1.OrderService/PlaceOrder` and `thrift::billing.v1.OrderService/GetOrder`.
|
||||
|
||||
## Java consumer examples
|
||||
|
||||
Consumers are strongest when Java usage can be tied back to the IDL namespace and service.
|
||||
|
||||
```java
|
||||
package example.checkout;
|
||||
|
||||
import billing.v1.OrderService;
|
||||
import billing.v1.PlaceOrderRequest;
|
||||
|
||||
public final class CheckoutWorkflow {
|
||||
private final OrderService.Iface orders;
|
||||
|
||||
public CheckoutWorkflow(OrderService.Iface orders) {
|
||||
this.orders = orders;
|
||||
}
|
||||
|
||||
public void submit(String orderId) throws Exception {
|
||||
orders.PlaceOrder(new PlaceOrderRequest(orderId, 42.0));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Some generated-code styles use the generated service type directly while keeping enough IDL context through imports and method calls.
|
||||
|
||||
```java
|
||||
package example.reporting;
|
||||
|
||||
import billing.v1.GetOrderRequest;
|
||||
import billing.v1.OrderService;
|
||||
|
||||
public final class OrderLookup {
|
||||
private final OrderService.Client client;
|
||||
|
||||
public OrderLookup(OrderService.Client client) {
|
||||
this.client = client;
|
||||
}
|
||||
|
||||
public String status(String orderId) throws Exception {
|
||||
return client.GetOrder(new GetOrderRequest(orderId)).getStatus();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When IDL context is missing, GitNexus may still emit a weaker consumer signal for generated `Iface` or `Client` shapes, but confidence is lower.
|
||||
|
||||
## Group configuration
|
||||
|
||||
New group configs enable Thrift contract detection by default. Keep `detect.thrift: true`
|
||||
when a group should scan for Thrift contracts, or set it to `false` to skip Thrift
|
||||
extraction for that group.
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
name: billing-platform
|
||||
description: Fictional services connected by Apache Thrift
|
||||
|
||||
repos:
|
||||
checkout: checkout-service
|
||||
billing: billing-service
|
||||
|
||||
links: []
|
||||
|
||||
detect:
|
||||
http: true
|
||||
grpc: false
|
||||
thrift: true
|
||||
topics: false
|
||||
shared_libs: true
|
||||
```
|
||||
|
||||
To disable Thrift extraction explicitly:
|
||||
|
||||
```yaml
|
||||
detect:
|
||||
thrift: false
|
||||
```
|
||||
|
||||
After indexing each member repository, run group sync to extract contracts and write cross-repo links:
|
||||
|
||||
```bash
|
||||
npx gitnexus group sync billing-platform
|
||||
```
|
||||
|
||||
## Manifest escape hatch
|
||||
|
||||
Use manifest links when automatic extraction cannot see a provider or consumer, or when generated code is wrapped behind an abstraction. Write the contract without the `thrift::` prefix; GitNexus canonicalizes it to the full Thrift contract id.
|
||||
|
||||
```yaml
|
||||
links:
|
||||
- from: checkout
|
||||
to: billing
|
||||
type: thrift
|
||||
contract: billing.v1.OrderService/PlaceOrder
|
||||
role: consumer
|
||||
```
|
||||
|
||||
GitNexus canonicalizes that manifest entry to `thrift::billing.v1.OrderService/PlaceOrder` and uses it to connect the two repositories.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- Java detection currently targets v1 generated-code patterns.
|
||||
- Maven and POM dependency coordinates are not used for inference.
|
||||
- Framework-specific annotations and service discovery metadata are ignored by open-source Thrift extraction.
|
||||
- Ambiguous same-name services are skipped instead of guessed.
|
||||
- Java consumers without IDL context are lower confidence and limited to generated `Iface` and `Client` shapes.
|
||||
Generated
+5
-5
@@ -16,7 +16,7 @@
|
||||
"@langchain/openai": "^1.4.5",
|
||||
"@sigma/edge-curve": "^3.1.0",
|
||||
"@tailwindcss/vite": "^4.2.4",
|
||||
"axios": "^1.13.2",
|
||||
"axios": "^1.16.0",
|
||||
"d3": "^7.9.0",
|
||||
"dompurify": "^3.4.2",
|
||||
"gitnexus-shared": "file:../gitnexus-shared",
|
||||
@@ -3401,12 +3401,12 @@
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/axios": {
|
||||
"version": "1.15.0",
|
||||
"resolved": "https://registry.npmjs.org/axios/-/axios-1.15.0.tgz",
|
||||
"integrity": "sha512-wWyJDlAatxk30ZJer+GeCWS209sA42X+N5jU2jy6oHTp7ufw8uzUTVFBX9+wTfAlhiJXGS0Bq7X6efruWjuK9Q==",
|
||||
"version": "1.16.0",
|
||||
"resolved": "https://registry.npmjs.org/axios/-/axios-1.16.0.tgz",
|
||||
"integrity": "sha512-6hp5CwvTPlN2A31g5dxnwAX0orzM7pmCRDLnZSX772mv8WDqICwFjowHuPs04Mc8deIld1+ejhtaMn5vp6b+1w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"follow-redirects": "^1.15.11",
|
||||
"follow-redirects": "^1.16.0",
|
||||
"form-data": "^4.0.5",
|
||||
"proxy-from-env": "^2.1.0"
|
||||
}
|
||||
|
||||
@@ -27,7 +27,7 @@
|
||||
"@langchain/openai": "^1.4.5",
|
||||
"@sigma/edge-curve": "^3.1.0",
|
||||
"@tailwindcss/vite": "^4.2.4",
|
||||
"axios": "^1.13.2",
|
||||
"axios": "^1.16.0",
|
||||
"d3": "^7.9.0",
|
||||
"dompurify": "^3.4.2",
|
||||
"graphology": "^0.26.0",
|
||||
|
||||
@@ -72,7 +72,19 @@ export class BackendError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public readonly status: number,
|
||||
public readonly code: 'network' | 'server' | 'client' | 'not_found' | 'timeout',
|
||||
public readonly code:
|
||||
| 'network'
|
||||
| 'server'
|
||||
| 'client'
|
||||
| 'not_found'
|
||||
| 'timeout'
|
||||
| 'rate_limited',
|
||||
/**
|
||||
* Milliseconds until the caller should retry. Populated for rate-limited
|
||||
* responses (HTTP 429) from the server's `Retry-After` header. `undefined`
|
||||
* for every other code, including `client` errors that aren't 429.
|
||||
*/
|
||||
public readonly retryAfterMs?: number,
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'BackendError';
|
||||
@@ -279,10 +291,32 @@ const assertOk = async (response: Response): Promise<void> => {
|
||||
const code =
|
||||
response.status === 404
|
||||
? 'not_found'
|
||||
: response.status >= 400 && response.status < 500
|
||||
? 'client'
|
||||
: 'server';
|
||||
throw new BackendError(message, response.status, code);
|
||||
: response.status === 429
|
||||
? 'rate_limited'
|
||||
: response.status >= 400 && response.status < 500
|
||||
? 'client'
|
||||
: 'server';
|
||||
|
||||
// Retry-After is the standard HTTP signal for when the client may try again.
|
||||
// express-rate-limit emits it on 429 with seconds (integer) or HTTP-date.
|
||||
// We accept both shapes; an unparseable header yields undefined retryAfterMs.
|
||||
let retryAfterMs: number | undefined;
|
||||
if (response.status === 429) {
|
||||
const header = response.headers.get('retry-after');
|
||||
if (header) {
|
||||
const seconds = Number(header);
|
||||
if (Number.isFinite(seconds) && seconds >= 0) {
|
||||
retryAfterMs = seconds * 1000;
|
||||
} else {
|
||||
const dateMs = Date.parse(header);
|
||||
if (Number.isFinite(dateMs)) {
|
||||
retryAfterMs = Math.max(0, dateMs - Date.now());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
throw new BackendError(message, response.status, code, retryAfterMs);
|
||||
};
|
||||
|
||||
const repoParam = (repo?: string): string => (repo ? `repo=${encodeURIComponent(repo)}` : '');
|
||||
|
||||
Generated
+9
-8
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "gitnexus",
|
||||
"version": "1.6.3",
|
||||
"version": "1.6.4-rc.75",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "gitnexus",
|
||||
"version": "1.6.3",
|
||||
"version": "1.6.4-rc.75",
|
||||
"hasInstallScript": true,
|
||||
"license": "PolyForm-Noncommercial-1.0.0",
|
||||
"dependencies": {
|
||||
@@ -18,6 +18,7 @@
|
||||
"commander": "^14.0.3",
|
||||
"cors": "^2.8.5",
|
||||
"express": "^4.19.2",
|
||||
"express-rate-limit": "^8.4.1",
|
||||
"glob": "^13.0.6",
|
||||
"graphology": "^0.26.0",
|
||||
"graphology-indices": "^0.17.0",
|
||||
@@ -3017,9 +3018,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/express-rate-limit": {
|
||||
"version": "8.3.1",
|
||||
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-8.3.1.tgz",
|
||||
"integrity": "sha512-D1dKN+cmyPWuvB+G2SREQDzPY1agpBIcTa9sJxOPMCNeH3gwzhqJRDWCXW3gg0y//+LQ/8j52JbMROWyrKdMdw==",
|
||||
"version": "8.5.0",
|
||||
"resolved": "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-8.5.0.tgz",
|
||||
"integrity": "sha512-XKhFohWaSBdVJNTi5TaHziqnPkv04I9UQV6q1Wy7Ui6GGQZVW12ojDFwqer14EvCXxjvPG0CyWXx7cAXpALB4Q==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"ip-address": "10.1.0"
|
||||
@@ -3891,9 +3892,9 @@
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/lru-cache": {
|
||||
"version": "11.3.5",
|
||||
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.3.5.tgz",
|
||||
"integrity": "sha512-NxVFwLAnrd9i7KUBxC4DrUhmgjzOs+1Qm50D3oF1/oL+r1NpZ4gA7xvG0/zJ8evR7zIKn4vLf7qTNduWFtCrRw==",
|
||||
"version": "11.3.6",
|
||||
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.3.6.tgz",
|
||||
"integrity": "sha512-Gf/KoL3C/MlI7Bt0PGI9I+TeTC/I6r/csU58N4BSNc4lppLBeKsOdFYkK+dX0ABDUMJNfCHTyPpzwwO21Awd3A==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": "20 || >=22"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gitnexus",
|
||||
"version": "1.6.3",
|
||||
"version": "1.6.4-rc.75",
|
||||
"description": "Graph-powered code intelligence for AI agents. Index any codebase, query via MCP or CLI.",
|
||||
"author": "Abhigyan Patwari",
|
||||
"license": "PolyForm-Noncommercial-1.0.0",
|
||||
@@ -60,6 +60,7 @@
|
||||
"commander": "^14.0.3",
|
||||
"cors": "^2.8.5",
|
||||
"express": "^4.19.2",
|
||||
"express-rate-limit": "^8.4.1",
|
||||
"glob": "^13.0.6",
|
||||
"graphology": "^0.26.0",
|
||||
"graphology-indices": "^0.17.0",
|
||||
|
||||
@@ -95,7 +95,14 @@ function ensureHeap(): boolean {
|
||||
|
||||
export interface AnalyzeOptions {
|
||||
force?: boolean;
|
||||
embeddings?: boolean;
|
||||
/**
|
||||
* Embedding generation toggle. Commander parses `--embeddings [limit]` as:
|
||||
* - `undefined` when the flag is omitted
|
||||
* - `true` when passed without an argument (use default 50K node cap)
|
||||
* - a string when passed with an argument (`--embeddings 0` disables the
|
||||
* cap, `--embeddings <n>` uses `<n>` as the cap)
|
||||
*/
|
||||
embeddings?: boolean | string;
|
||||
/**
|
||||
* Explicitly drop existing embeddings on rebuild instead of preserving
|
||||
* them. Without this flag, a routine `analyze` keeps any embeddings
|
||||
@@ -167,6 +174,25 @@ export const analyzeCommand = async (inputPath?: string, options?: AnalyzeOption
|
||||
);
|
||||
}
|
||||
|
||||
// Parse `--embeddings [limit]`: `true` → default cap, string → numeric cap
|
||||
// (0 disables the cap entirely). Validated up here so failures match the
|
||||
// sibling-validation pattern (exit before bar.start() — otherwise
|
||||
// process.exit() leaves the progress bar's hidden cursor uncleared).
|
||||
let embeddingsNodeLimit: number | undefined;
|
||||
if (typeof options?.embeddings === 'string') {
|
||||
const parsed = Number(options.embeddings);
|
||||
if (!Number.isInteger(parsed) || parsed < 0) {
|
||||
console.error(
|
||||
` --embeddings expects a non-negative integer (got "${options.embeddings}"). ` +
|
||||
`Pass 0 to disable the safety cap, or omit the value to keep the default.\n`,
|
||||
);
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
embeddingsNodeLimit = parsed;
|
||||
}
|
||||
const embeddingsEnabled = !!options?.embeddings;
|
||||
|
||||
const setPositiveEnv = (
|
||||
optionName: string,
|
||||
envName: string,
|
||||
@@ -338,7 +364,8 @@ export const analyzeCommand = async (inputPath?: string, options?: AnalyzeOption
|
||||
// needs a fresh pipelineResult. Has no bearing on the registry
|
||||
// collision guard (see allowDuplicateName below).
|
||||
force: options?.force || options?.skills,
|
||||
embeddings: options?.embeddings,
|
||||
embeddings: embeddingsEnabled,
|
||||
embeddingsNodeLimit,
|
||||
dropEmbeddings: options?.dropEmbeddings,
|
||||
skipGit: options?.skipGit,
|
||||
skipAgentsMd: options?.skipAgentsMd,
|
||||
|
||||
@@ -23,7 +23,11 @@ program
|
||||
.command('analyze [path]')
|
||||
.description('Index a repository (full analysis)')
|
||||
.option('-f, --force', 'Force full re-index even if up to date')
|
||||
.option('--embeddings', 'Enable embedding generation for semantic search (off by default)')
|
||||
.option(
|
||||
'--embeddings [limit]',
|
||||
'Enable embedding generation for semantic search (off by default). ' +
|
||||
'Optional [limit] overrides the 50,000-node safety cap; pass 0 to disable the cap entirely.',
|
||||
)
|
||||
.option(
|
||||
'--drop-embeddings',
|
||||
'Drop existing embeddings on rebuild. By default, an `analyze` without `--embeddings` ' +
|
||||
|
||||
@@ -30,6 +30,38 @@ export interface EmbeddingMode {
|
||||
shouldLoadCache: boolean;
|
||||
}
|
||||
|
||||
/** Default safety cap on graph node count for embedding generation. */
|
||||
export const DEFAULT_EMBEDDING_NODE_LIMIT = 50_000;
|
||||
|
||||
export interface EmbeddingCapDecision {
|
||||
/** True when the node-count cap blocks generation for this graph. */
|
||||
skipForCap: boolean;
|
||||
/** True when the user explicitly disabled the cap (`--embeddings 0`). */
|
||||
capDisabled: boolean;
|
||||
/** Effective node limit applied (`0` means disabled). */
|
||||
nodeLimit: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether the node-count safety cap blocks embedding generation.
|
||||
*
|
||||
* - `embeddingsNodeLimit === undefined` → use {@link DEFAULT_EMBEDDING_NODE_LIMIT}
|
||||
* - `embeddingsNodeLimit === 0` → cap disabled, generation always proceeds
|
||||
* - any positive integer → custom cap (skip if `nodeCount > limit`)
|
||||
*
|
||||
* Lives in `embedding-mode.ts` (not `run-analyze.ts`) so the branching
|
||||
* contract is unit-testable without spinning up LadybugDB or the pipeline.
|
||||
*/
|
||||
export function deriveEmbeddingCap(
|
||||
nodeCount: number,
|
||||
embeddingsNodeLimit: number | undefined,
|
||||
): EmbeddingCapDecision {
|
||||
const nodeLimit = embeddingsNodeLimit ?? DEFAULT_EMBEDDING_NODE_LIMIT;
|
||||
const capDisabled = nodeLimit === 0;
|
||||
const skipForCap = !capDisabled && nodeCount > nodeLimit;
|
||||
return { skipForCap, capDisabled, nodeLimit };
|
||||
}
|
||||
|
||||
export function deriveEmbeddingMode(
|
||||
options: EmbeddingModeInput,
|
||||
existingEmbeddingCount: number,
|
||||
|
||||
@@ -4,12 +4,13 @@ import type { GroupConfig, GroupManifestLink, ContractType, ContractRole } from
|
||||
const _require = createRequire(import.meta.url);
|
||||
const yaml = _require('js-yaml') as typeof import('js-yaml');
|
||||
|
||||
const VALID_CONTRACT_TYPES: ContractType[] = ['http', 'grpc', 'topic', 'lib', 'custom'];
|
||||
const VALID_CONTRACT_TYPES: ContractType[] = ['http', 'grpc', 'thrift', 'topic', 'lib', 'custom'];
|
||||
const VALID_ROLES: ContractRole[] = ['provider', 'consumer'];
|
||||
|
||||
const DEFAULT_DETECT = {
|
||||
http: true,
|
||||
grpc: true,
|
||||
thrift: true,
|
||||
topics: true,
|
||||
shared_libs: true,
|
||||
embedding_fallback: true,
|
||||
|
||||
@@ -177,7 +177,7 @@ export class ManifestExtractor {
|
||||
|
||||
// NOTE: All lookups use EXACT equality on the relevant name field and
|
||||
// deterministic ORDER BY before LIMIT 1. Previous versions used CONTAINS
|
||||
// for fuzzy matching (plus an unconditional ".proto" fallback for gRPC)
|
||||
// for fuzzy matching (plus an unconditional IDL file fallback for gRPC)
|
||||
// which produced silent false positives: e.g. manifest "/orders" would
|
||||
// match "/suborders", and a gRPC manifest entry in a repo with any
|
||||
// .proto file would attach to a random proto symbol.
|
||||
@@ -225,16 +225,21 @@ export class ManifestExtractor {
|
||||
LIMIT 1`,
|
||||
{ contract: link.contract },
|
||||
);
|
||||
} else if (link.type === 'grpc') {
|
||||
} else if (link.type === 'grpc' || link.type === 'thrift') {
|
||||
// Contract is "Service/Method" or just "Service" (or package.Service
|
||||
// variants). Prefer matching by method name when present, otherwise
|
||||
// by service name. NO .proto path fallback — that's guaranteed to
|
||||
// return a wrong symbol in any repo with more than one proto file.
|
||||
// by service name. Thrift generated Java classes often use
|
||||
// package.Service in manifests while graph Class/Interface names are
|
||||
// stored as bare Service, so strip the package prefix for thrift
|
||||
// service-name lookups. NO IDL path fallback — that's guaranteed to
|
||||
// return a wrong symbol in any repo with more than one IDL file.
|
||||
// Label filters scope lookups: methods → Function|Method, services
|
||||
// → Class|Interface (no label match = no silent wrong hits on
|
||||
// File/Variable nodes that happen to share the name).
|
||||
const parts = link.contract.split('/');
|
||||
const serviceName = parts[0]?.trim() ?? '';
|
||||
const rawServiceName = parts[0]?.trim() ?? '';
|
||||
const serviceName =
|
||||
link.type === 'thrift' ? (rawServiceName.split('.').pop() ?? '') : rawServiceName;
|
||||
const methodName = parts[1]?.trim() ?? '';
|
||||
if (methodName) {
|
||||
rows = await executor(
|
||||
@@ -344,6 +349,8 @@ export class ManifestExtractor {
|
||||
}
|
||||
case 'grpc':
|
||||
return `grpc::${contract}`;
|
||||
case 'thrift':
|
||||
return `thrift::${contract}`;
|
||||
case 'topic':
|
||||
return `topic::${contract}`;
|
||||
case 'lib':
|
||||
|
||||
@@ -0,0 +1,379 @@
|
||||
import { glob } from 'glob';
|
||||
import Parser from 'tree-sitter';
|
||||
import type { ContractExtractor, CypherExecutor } from '../contract-extractor.js';
|
||||
import type { ExtractedContract, RepoHandle } from '../types.js';
|
||||
import { readSafe } from './fs-utils.js';
|
||||
import {
|
||||
getPluginForFile,
|
||||
THRIFT_SCAN_GLOB,
|
||||
type ThriftDetection,
|
||||
} from './thrift-patterns/index.js';
|
||||
|
||||
export interface ThriftServiceInfo {
|
||||
namespace: string;
|
||||
serviceName: string;
|
||||
methods: string[];
|
||||
thriftPath: string;
|
||||
}
|
||||
|
||||
export interface ThriftContext {
|
||||
namespacesByThrift: Map<string, string>;
|
||||
servicesByName: Map<string, ThriftServiceInfo[]>;
|
||||
}
|
||||
|
||||
function normalizeThriftPath(rel: string): string {
|
||||
return rel.replace(/\\/g, '/');
|
||||
}
|
||||
|
||||
export function thriftMethodContractId(
|
||||
namespace: string,
|
||||
serviceName: string,
|
||||
methodName: string,
|
||||
): string {
|
||||
const prefix = namespace ? `${namespace}.${serviceName}` : serviceName;
|
||||
return `thrift::${prefix}/${methodName}`;
|
||||
}
|
||||
|
||||
export function thriftServiceContractId(namespace: string, serviceName: string): string {
|
||||
const prefix = namespace ? `${namespace}.${serviceName}` : serviceName;
|
||||
return `thrift::${prefix}/*`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace Thrift comments and string literals with spaces while preserving
|
||||
* newlines and character offsets. Service block scanning can then count braces
|
||||
* without being confused by examples or comments inside the IDL.
|
||||
*/
|
||||
function stripThriftCommentsAndStrings(content: string): string {
|
||||
const out = new Array<string>(content.length);
|
||||
let i = 0;
|
||||
|
||||
while (i < content.length) {
|
||||
const ch = content[i];
|
||||
const next = content[i + 1];
|
||||
|
||||
if (ch === '/' && next === '/') {
|
||||
out[i] = ' ';
|
||||
out[i + 1] = ' ';
|
||||
i += 2;
|
||||
while (i < content.length && content[i] !== '\n') {
|
||||
out[i] = content[i] === '\r' ? '\r' : ' ';
|
||||
i++;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === '#') {
|
||||
out[i] = ' ';
|
||||
i++;
|
||||
while (i < content.length && content[i] !== '\n') {
|
||||
out[i] = content[i] === '\r' ? '\r' : ' ';
|
||||
i++;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === '/' && next === '*') {
|
||||
out[i] = ' ';
|
||||
out[i + 1] = ' ';
|
||||
i += 2;
|
||||
while (i < content.length) {
|
||||
if (content[i] === '*' && content[i + 1] === '/') {
|
||||
out[i] = ' ';
|
||||
out[i + 1] = ' ';
|
||||
i += 2;
|
||||
break;
|
||||
}
|
||||
out[i] = content[i] === '\n' || content[i] === '\r' ? content[i] : ' ';
|
||||
i++;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (ch === '"' || ch === "'") {
|
||||
const quote = ch;
|
||||
out[i] = ' ';
|
||||
i++;
|
||||
while (i < content.length) {
|
||||
const c = content[i];
|
||||
if (c === '\\' && i + 1 < content.length) {
|
||||
out[i] = ' ';
|
||||
out[i + 1] = ' ';
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
if (c === quote) {
|
||||
out[i] = ' ';
|
||||
i++;
|
||||
break;
|
||||
}
|
||||
out[i] = c === '\n' || c === '\r' ? c : ' ';
|
||||
i++;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
out[i] = ch;
|
||||
i++;
|
||||
}
|
||||
|
||||
return out.join('');
|
||||
}
|
||||
|
||||
function extractNamespace(sanitizedContent: string): string {
|
||||
const namespaces: Array<{ language: string; namespace: string }> = [];
|
||||
const namespaceRe = /^\s*namespace\s+([A-Za-z_*][\w.*-]*)\s+([A-Za-z_][\w.]*)\s*$/gm;
|
||||
let match: RegExpExecArray | null;
|
||||
|
||||
while ((match = namespaceRe.exec(sanitizedContent)) !== null) {
|
||||
namespaces.push({ language: match[1], namespace: match[2] });
|
||||
}
|
||||
|
||||
return (
|
||||
namespaces.find((entry) => entry.language === 'java')?.namespace ??
|
||||
namespaces[0]?.namespace ??
|
||||
''
|
||||
);
|
||||
}
|
||||
|
||||
function extractServiceBlocks(sanitizedContent: string): Array<{ name: string; body: string }> {
|
||||
const results: Array<{ name: string; body: string }> = [];
|
||||
const headerRe = /service\s+([A-Za-z_]\w*)\s*(?:extends\s+[A-Za-z_][\w.]*)?\s*\{/g;
|
||||
let headerMatch: RegExpExecArray | null;
|
||||
|
||||
while ((headerMatch = headerRe.exec(sanitizedContent)) !== null) {
|
||||
const serviceName = headerMatch[1];
|
||||
const bodyStart = headerMatch.index + headerMatch[0].length;
|
||||
let depth = 1;
|
||||
let pos = bodyStart;
|
||||
|
||||
while (pos < sanitizedContent.length && depth > 0) {
|
||||
const ch = sanitizedContent[pos];
|
||||
if (ch === '{') depth++;
|
||||
else if (ch === '}') depth--;
|
||||
pos++;
|
||||
}
|
||||
|
||||
if (depth !== 0) continue;
|
||||
|
||||
results.push({
|
||||
name: serviceName,
|
||||
body: sanitizedContent.slice(bodyStart, pos - 1),
|
||||
});
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
function extractMethods(sanitizedServiceBody: string): string[] {
|
||||
const methods: string[] = [];
|
||||
const methodRe =
|
||||
/(?:^|[;,\n\r])\s*(?:oneway\s+)?[A-Za-z_][\w.]*(?:\s*<[^(){};]*>)?\s+([A-Za-z_]\w*)\s*\(/g;
|
||||
let match: RegExpExecArray | null;
|
||||
|
||||
while ((match = methodRe.exec(sanitizedServiceBody)) !== null) {
|
||||
methods.push(match[1]);
|
||||
}
|
||||
|
||||
return methods;
|
||||
}
|
||||
|
||||
function thriftSourceScanSymbolUid(
|
||||
contractId: string,
|
||||
role: 'provider' | 'consumer',
|
||||
filePath: string,
|
||||
symbolName: string,
|
||||
): string {
|
||||
const contractKey = contractId.startsWith('thrift::')
|
||||
? contractId.slice('thrift::'.length)
|
||||
: contractId;
|
||||
return ['source-scan::thrift', role, contractKey, normalizeThriftPath(filePath), symbolName].join(
|
||||
'::',
|
||||
);
|
||||
}
|
||||
|
||||
function makeContract(
|
||||
cid: string,
|
||||
role: 'provider' | 'consumer',
|
||||
filePath: string,
|
||||
symbolName: string,
|
||||
confidence: number,
|
||||
meta: Record<string, unknown>,
|
||||
): ExtractedContract {
|
||||
return {
|
||||
contractId: cid,
|
||||
type: 'thrift',
|
||||
role,
|
||||
symbolUid: thriftSourceScanSymbolUid(cid, role, filePath, symbolName),
|
||||
symbolRef: { filePath: normalizeThriftPath(filePath), name: symbolName },
|
||||
symbolName,
|
||||
confidence,
|
||||
meta: { ...meta, extractionStrategy: 'source_scan' },
|
||||
};
|
||||
}
|
||||
|
||||
export async function buildThriftContext(repoPath: string): Promise<ThriftContext> {
|
||||
const thriftFiles = await glob('**/*.thrift', {
|
||||
cwd: repoPath,
|
||||
absolute: false,
|
||||
nodir: true,
|
||||
ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**', '**/dist/**', '**/build/**'],
|
||||
});
|
||||
const namespacesByThrift = new Map<string, string>();
|
||||
const servicesByName = new Map<string, ThriftServiceInfo[]>();
|
||||
|
||||
for (const rel of thriftFiles) {
|
||||
const thriftPath = normalizeThriftPath(rel);
|
||||
const content = readSafe(repoPath, rel);
|
||||
if (!content) continue;
|
||||
|
||||
const sanitized = stripThriftCommentsAndStrings(content);
|
||||
const namespace = extractNamespace(sanitized);
|
||||
namespacesByThrift.set(thriftPath, namespace);
|
||||
|
||||
for (const block of extractServiceBlocks(sanitized)) {
|
||||
const methods = extractMethods(block.body);
|
||||
const info: ThriftServiceInfo = {
|
||||
namespace,
|
||||
serviceName: block.name,
|
||||
methods,
|
||||
thriftPath,
|
||||
};
|
||||
const existing = servicesByName.get(block.name) ?? [];
|
||||
existing.push(info);
|
||||
servicesByName.set(block.name, existing);
|
||||
}
|
||||
}
|
||||
|
||||
return { namespacesByThrift, servicesByName };
|
||||
}
|
||||
|
||||
export class ThriftExtractor implements ContractExtractor {
|
||||
type = 'thrift' as const;
|
||||
|
||||
async canExtract(_repo: RepoHandle): Promise<boolean> {
|
||||
return true;
|
||||
}
|
||||
|
||||
async extract(
|
||||
_dbExecutor: CypherExecutor | null,
|
||||
repoPath: string,
|
||||
_repo: RepoHandle,
|
||||
): Promise<ExtractedContract[]> {
|
||||
const out: ExtractedContract[] = [];
|
||||
const context = await buildThriftContext(repoPath);
|
||||
|
||||
for (const infos of context.servicesByName.values()) {
|
||||
for (const info of infos) {
|
||||
for (const methodName of info.methods) {
|
||||
const symbolName = `${info.serviceName}.${methodName}`;
|
||||
out.push(
|
||||
makeContract(
|
||||
thriftMethodContractId(info.namespace, info.serviceName, methodName),
|
||||
'provider',
|
||||
info.thriftPath,
|
||||
symbolName,
|
||||
0.85,
|
||||
{
|
||||
namespace: info.namespace,
|
||||
service: info.serviceName,
|
||||
method: methodName,
|
||||
source: 'thrift_idl',
|
||||
},
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const sourceFiles = await glob(THRIFT_SCAN_GLOB, {
|
||||
cwd: repoPath,
|
||||
absolute: false,
|
||||
nodir: true,
|
||||
ignore: ['**/node_modules/**', '**/.git/**', '**/vendor/**', '**/dist/**', '**/build/**'],
|
||||
});
|
||||
|
||||
const parser = new Parser();
|
||||
for (const rel of sourceFiles) {
|
||||
const plugin = getPluginForFile(rel);
|
||||
if (!plugin) continue;
|
||||
const content = readSafe(repoPath, rel);
|
||||
if (!content) continue;
|
||||
|
||||
let detections: ThriftDetection[] = [];
|
||||
try {
|
||||
parser.setLanguage(plugin.language);
|
||||
const tree = parser.parse(content);
|
||||
detections = plugin.scan(tree);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
for (const detection of detections) {
|
||||
const contract = this.detectionToContract(detection, rel, context);
|
||||
if (contract) out.push(contract);
|
||||
}
|
||||
}
|
||||
|
||||
return this.dedupe(out);
|
||||
}
|
||||
|
||||
private detectionToContract(
|
||||
detection: ThriftDetection,
|
||||
filePath: string,
|
||||
context: ThriftContext,
|
||||
): ExtractedContract | null {
|
||||
const candidates = context.servicesByName.get(detection.serviceName) ?? [];
|
||||
if (candidates.length > 1) return null;
|
||||
|
||||
const info = candidates[0];
|
||||
if (info) {
|
||||
if (!info.methods.includes(detection.methodName)) return null;
|
||||
return makeContract(
|
||||
thriftMethodContractId(info.namespace, info.serviceName, detection.methodName),
|
||||
detection.role,
|
||||
filePath,
|
||||
detection.symbolName,
|
||||
detection.confidenceWithIdl,
|
||||
{
|
||||
namespace: info.namespace,
|
||||
service: info.serviceName,
|
||||
method: detection.methodName,
|
||||
source: detection.source,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
if (
|
||||
detection.role !== 'consumer' ||
|
||||
!detection.methodName ||
|
||||
!detection.usesGeneratedServiceMember
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return makeContract(
|
||||
thriftMethodContractId('', detection.serviceName, detection.methodName),
|
||||
detection.role,
|
||||
filePath,
|
||||
detection.symbolName,
|
||||
detection.confidenceWithoutIdl,
|
||||
{
|
||||
service: detection.serviceName,
|
||||
method: detection.methodName,
|
||||
source: 'java_thrift_consumer_weak',
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
private dedupe(items: ExtractedContract[]): ExtractedContract[] {
|
||||
const byKey = new Map<string, ExtractedContract>();
|
||||
for (const c of items) {
|
||||
const key = `${c.contractId}|${c.role}|${c.symbolRef.filePath}|${c.symbolName}`;
|
||||
const existing = byKey.get(key);
|
||||
if (!existing || c.confidence > existing.confidence) {
|
||||
byKey.set(key, c);
|
||||
}
|
||||
}
|
||||
return Array.from(byKey.values());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import * as path from 'node:path';
|
||||
import type { ThriftLanguagePlugin } from './types.js';
|
||||
import { JAVA_THRIFT_PLUGIN } from './java.js';
|
||||
|
||||
export type { ThriftDetection, ThriftLanguagePlugin, ThriftRole } from './types.js';
|
||||
|
||||
const REGISTRY: Record<string, ThriftLanguagePlugin> = {
|
||||
'.java': JAVA_THRIFT_PLUGIN,
|
||||
};
|
||||
|
||||
export const THRIFT_SCAN_GLOB = '**/*.java';
|
||||
|
||||
export function getPluginForFile(rel: string): ThriftLanguagePlugin | undefined {
|
||||
const ext = path.extname(rel).toLowerCase();
|
||||
return REGISTRY[ext];
|
||||
}
|
||||
@@ -0,0 +1,258 @@
|
||||
import Parser from 'tree-sitter';
|
||||
import Java from 'tree-sitter-java';
|
||||
import {
|
||||
compilePatterns,
|
||||
runCompiledPatterns,
|
||||
type LanguagePatterns,
|
||||
} from '../tree-sitter-scanner.js';
|
||||
import type { ThriftDetection, ThriftLanguagePlugin } from './types.js';
|
||||
|
||||
const GENERATED_MEMBER_TYPES = new Set(['Iface', 'Client']);
|
||||
const SERVICE_TYPE_RE = /^[A-Z][A-Za-z0-9]*(?:Service|Management)$/;
|
||||
|
||||
interface VariableBinding {
|
||||
name: string;
|
||||
serviceName: string;
|
||||
usesGeneratedServiceMember: boolean;
|
||||
scopeStart: number;
|
||||
scopeEnd: number;
|
||||
declarationEnd: number;
|
||||
scopeSize: number;
|
||||
}
|
||||
|
||||
interface ServiceTypeMatch {
|
||||
serviceName: string;
|
||||
usesGeneratedServiceMember: boolean;
|
||||
}
|
||||
|
||||
const VARIABLE_PATTERNS = compilePatterns({
|
||||
name: 'java-thrift-variables',
|
||||
language: Java,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(field_declaration
|
||||
type: (_) @type
|
||||
declarator: (variable_declarator
|
||||
name: (identifier) @var))
|
||||
`,
|
||||
},
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(local_variable_declaration
|
||||
type: (_) @type
|
||||
declarator: (variable_declarator
|
||||
name: (identifier) @var))
|
||||
`,
|
||||
},
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(formal_parameter
|
||||
type: (_) @type
|
||||
name: (identifier) @var)
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
const CALL_PATTERNS = compilePatterns({
|
||||
name: 'java-thrift-method-calls',
|
||||
language: Java,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(method_invocation
|
||||
object: (identifier) @receiver
|
||||
name: (identifier) @method)
|
||||
`,
|
||||
},
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(method_invocation
|
||||
object: (field_access
|
||||
object: (this)
|
||||
field: (identifier) @receiver)
|
||||
name: (identifier) @method)
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
const PROVIDER_PATTERNS = compilePatterns({
|
||||
name: 'java-thrift-providers',
|
||||
language: Java,
|
||||
patterns: [
|
||||
{
|
||||
meta: {},
|
||||
query: `
|
||||
(class_declaration
|
||||
name: (identifier) @class_name
|
||||
(super_interfaces
|
||||
(type_list
|
||||
(_) @type))
|
||||
body: (class_body) @body) @class
|
||||
`,
|
||||
},
|
||||
],
|
||||
} satisfies LanguagePatterns<Record<string, never>>);
|
||||
|
||||
function serviceFromType(typeText: string): ServiceTypeMatch | null {
|
||||
const segments = typeText.split('.').filter((segment) => segment.length > 0);
|
||||
const last = segments.at(-1);
|
||||
const service = segments.at(-2);
|
||||
if (last && service && GENERATED_MEMBER_TYPES.has(last)) {
|
||||
return { serviceName: service, usesGeneratedServiceMember: true };
|
||||
}
|
||||
return last && SERVICE_TYPE_RE.test(last)
|
||||
? { serviceName: last, usesGeneratedServiceMember: false }
|
||||
: null;
|
||||
}
|
||||
|
||||
function methodNamesInClassBody(body: Parser.SyntaxNode): string[] {
|
||||
const names: string[] = [];
|
||||
for (let i = 0; i < body.namedChildCount; i++) {
|
||||
const child = body.namedChild(i);
|
||||
if (!child || child.type !== 'method_declaration') continue;
|
||||
const name = child.childForFieldName('name');
|
||||
if (name?.text) names.push(name.text);
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
function nearestAncestor(node: Parser.SyntaxNode, types: Set<string>): Parser.SyntaxNode | null {
|
||||
let current: Parser.SyntaxNode | null = node;
|
||||
while (current) {
|
||||
if (types.has(current.type)) return current;
|
||||
current = current.parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function bindingScope(varNode: Parser.SyntaxNode): {
|
||||
scope: Parser.SyntaxNode;
|
||||
declarationEnd: number;
|
||||
} | null {
|
||||
const declaration = nearestAncestor(
|
||||
varNode,
|
||||
new Set(['field_declaration', 'local_variable_declaration', 'formal_parameter']),
|
||||
);
|
||||
if (!declaration) return null;
|
||||
|
||||
if (declaration.type === 'field_declaration') {
|
||||
const classBody = nearestAncestor(declaration, new Set(['class_body']));
|
||||
if (!classBody) return null;
|
||||
return { scope: classBody, declarationEnd: 0 };
|
||||
}
|
||||
|
||||
if (declaration.type === 'formal_parameter') {
|
||||
const callable = nearestAncestor(
|
||||
declaration,
|
||||
new Set(['method_declaration', 'constructor_declaration']),
|
||||
);
|
||||
if (!callable) return null;
|
||||
return { scope: callable, declarationEnd: 0 };
|
||||
}
|
||||
|
||||
const block = nearestAncestor(declaration, new Set(['block']));
|
||||
if (!block) return null;
|
||||
return { scope: block, declarationEnd: declaration.endIndex };
|
||||
}
|
||||
|
||||
function resolveServiceForReceiver(
|
||||
bindings: VariableBinding[],
|
||||
receiver: string,
|
||||
callNode: Parser.SyntaxNode,
|
||||
): VariableBinding | null {
|
||||
const callStart = callNode.startIndex;
|
||||
const candidates = bindings.filter(
|
||||
(binding) =>
|
||||
binding.name === receiver &&
|
||||
binding.scopeStart <= callStart &&
|
||||
callStart <= binding.scopeEnd &&
|
||||
binding.declarationEnd <= callStart,
|
||||
);
|
||||
candidates.sort((a, b) => {
|
||||
if (a.scopeSize !== b.scopeSize) return a.scopeSize - b.scopeSize;
|
||||
return b.declarationEnd - a.declarationEnd;
|
||||
});
|
||||
return candidates[0] ?? null;
|
||||
}
|
||||
|
||||
export const JAVA_THRIFT_PLUGIN: ThriftLanguagePlugin = {
|
||||
name: 'java-thrift',
|
||||
language: Java,
|
||||
scan(tree) {
|
||||
const out: ThriftDetection[] = [];
|
||||
const bindings: VariableBinding[] = [];
|
||||
|
||||
for (const match of runCompiledPatterns(VARIABLE_PATTERNS, tree)) {
|
||||
const typeNode = match.captures.type;
|
||||
const varNode = match.captures.var;
|
||||
if (!typeNode || !varNode) continue;
|
||||
const service = serviceFromType(typeNode.text);
|
||||
if (!service) continue;
|
||||
const scope = bindingScope(varNode);
|
||||
if (!scope) continue;
|
||||
bindings.push({
|
||||
name: varNode.text,
|
||||
serviceName: service.serviceName,
|
||||
usesGeneratedServiceMember: service.usesGeneratedServiceMember,
|
||||
scopeStart: scope.scope.startIndex,
|
||||
scopeEnd: scope.scope.endIndex,
|
||||
declarationEnd: scope.declarationEnd,
|
||||
scopeSize: scope.scope.endIndex - scope.scope.startIndex,
|
||||
});
|
||||
}
|
||||
|
||||
for (const match of runCompiledPatterns(CALL_PATTERNS, tree)) {
|
||||
const receiver = match.captures.receiver?.text;
|
||||
const methodName = match.captures.method?.text;
|
||||
const callNode = match.captures.receiver?.parent;
|
||||
if (!receiver || !methodName) continue;
|
||||
if (!callNode) continue;
|
||||
const binding = resolveServiceForReceiver(bindings, receiver, callNode);
|
||||
if (!binding) continue;
|
||||
out.push({
|
||||
role: 'consumer',
|
||||
serviceName: binding.serviceName,
|
||||
methodName,
|
||||
symbolName: `${receiver}.${methodName}`,
|
||||
source: 'java_thrift_consumer',
|
||||
confidenceWithIdl: 0.75,
|
||||
confidenceWithoutIdl: 0.45,
|
||||
usesGeneratedServiceMember: binding.usesGeneratedServiceMember,
|
||||
});
|
||||
}
|
||||
|
||||
const emittedProviders = new Set<string>();
|
||||
for (const match of runCompiledPatterns(PROVIDER_PATTERNS, tree)) {
|
||||
const typeNode = match.captures.type;
|
||||
const bodyNode = match.captures.body;
|
||||
if (!typeNode || !bodyNode) continue;
|
||||
const service = serviceFromType(typeNode.text);
|
||||
if (!service) continue;
|
||||
|
||||
for (const methodName of methodNamesInClassBody(bodyNode)) {
|
||||
const key = `${service.serviceName}.${methodName}`;
|
||||
if (emittedProviders.has(key)) continue;
|
||||
emittedProviders.add(key);
|
||||
out.push({
|
||||
role: 'provider',
|
||||
serviceName: service.serviceName,
|
||||
methodName,
|
||||
symbolName: `${service.serviceName}.${methodName}`,
|
||||
source: 'java_thrift_provider',
|
||||
confidenceWithIdl: 0.8,
|
||||
confidenceWithoutIdl: 0,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return out;
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,20 @@
|
||||
import type Parser from 'tree-sitter';
|
||||
|
||||
export type ThriftRole = 'provider' | 'consumer';
|
||||
|
||||
export interface ThriftDetection {
|
||||
role: ThriftRole;
|
||||
serviceName: string;
|
||||
methodName: string;
|
||||
symbolName: string;
|
||||
source: string;
|
||||
confidenceWithIdl: number;
|
||||
confidenceWithoutIdl: number;
|
||||
usesGeneratedServiceMember?: boolean;
|
||||
}
|
||||
|
||||
export interface ThriftLanguagePlugin {
|
||||
name: string;
|
||||
language: unknown;
|
||||
scan(tree: Parser.Tree): ThriftDetection[];
|
||||
}
|
||||
@@ -10,8 +10,8 @@ export interface WildcardMatchResult {
|
||||
remaining: StoredContract[];
|
||||
}
|
||||
|
||||
function isGrpcWildcard(cid: string): boolean {
|
||||
return cid.startsWith('grpc::') && cid.endsWith('/*');
|
||||
function isServiceWildcard(cid: string): boolean {
|
||||
return (cid.startsWith('grpc::') || cid.startsWith('thrift::')) && cid.endsWith('/*');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -69,8 +69,9 @@ export function normalizeContractId(id: string): string {
|
||||
}
|
||||
return id;
|
||||
}
|
||||
case 'grpc': {
|
||||
// Canonical form: `grpc::<lowercased-package-or-service>[/<method>]`.
|
||||
case 'grpc':
|
||||
case 'thrift': {
|
||||
// Canonical form: `<type>::<lowercased-package-or-service>[/<method>]`.
|
||||
//
|
||||
// The package/service segment is lowercased because gRPC package
|
||||
// names are effectively case-insensitive across language bindings
|
||||
@@ -84,22 +85,23 @@ export function normalizeContractId(id: string): string {
|
||||
// as DISTINCT canonical forms: `grpc::userservice` does not match
|
||||
// `grpc::userservice/Login`. That's by design — callers that want
|
||||
// service-level manifest matching against method-level providers
|
||||
// should use the gRPC wildcard form `grpc::UserService/*` which is
|
||||
// should use the service wildcard form `grpc::UserService/*` or
|
||||
// `thrift::UserService/*` which is
|
||||
// handled by runWildcardMatch below.
|
||||
const slashIdx = rest.indexOf('/');
|
||||
if (slashIdx > 0) {
|
||||
const pkg = rest.substring(0, slashIdx).toLowerCase();
|
||||
const method = rest.substring(slashIdx);
|
||||
return `grpc::${pkg}${method}`;
|
||||
return `${type}::${pkg}${method}`;
|
||||
}
|
||||
if (slashIdx === 0) {
|
||||
// Malformed "/method" with leading slash — keep as-is so two
|
||||
// equally malformed ids can still match each other.
|
||||
return `grpc::${rest}`;
|
||||
return `${type}::${rest}`;
|
||||
}
|
||||
// No slash: package/service only. Lowercase to match the package
|
||||
// segment produced by the pkg/method branch above.
|
||||
return `grpc::${rest.toLowerCase()}`;
|
||||
return `${type}::${rest.toLowerCase()}`;
|
||||
}
|
||||
case 'topic':
|
||||
return `topic::${rest.trim().toLowerCase()}`;
|
||||
@@ -125,6 +127,32 @@ function findMatchingKeys(contractId: string, index: Map<string, StoredContract[
|
||||
return matches;
|
||||
}
|
||||
|
||||
if (normalized.startsWith('thrift::')) {
|
||||
const rest = normalized.substring('thrift::'.length);
|
||||
const slashIdx = rest.indexOf('/');
|
||||
if (slashIdx > 0) {
|
||||
const service = rest.substring(0, slashIdx);
|
||||
const method = rest.substring(slashIdx + 1);
|
||||
if (!service.includes('.') && method && method !== '*') {
|
||||
const matches: string[] = [];
|
||||
for (const key of index.keys()) {
|
||||
if (!key.startsWith('thrift::') || key.endsWith('/*')) continue;
|
||||
const providerRest = key.substring('thrift::'.length);
|
||||
const providerSlashIdx = providerRest.indexOf('/');
|
||||
if (providerSlashIdx < 0) continue;
|
||||
const providerService = providerRest.substring(0, providerSlashIdx);
|
||||
const providerMethod = providerRest.substring(providerSlashIdx + 1);
|
||||
if (providerMethod !== method) continue;
|
||||
if (providerService === service || providerService.endsWith('.' + service)) {
|
||||
matches.push(key);
|
||||
}
|
||||
}
|
||||
matches.sort();
|
||||
return matches.length === 1 ? matches : [];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return [];
|
||||
}
|
||||
|
||||
@@ -152,8 +180,9 @@ export function runExactMatch(
|
||||
const isNoisy = buildNoisyContractFilter(matchingConfig);
|
||||
const index = providerIndex ?? buildProviderIndex(contracts, matchingConfig);
|
||||
|
||||
// Skip service wildcard consumers — they go to wildcard pass only
|
||||
const consumers = contracts.filter(
|
||||
(c) => c.role === 'consumer' && !isGrpcWildcard(c.contractId) && !isNoisy(c.contractId),
|
||||
(c) => c.role === 'consumer' && !isServiceWildcard(c.contractId) && !isNoisy(c.contractId),
|
||||
);
|
||||
|
||||
const matched: CrossLink[] = [];
|
||||
@@ -198,15 +227,15 @@ export function runExactMatch(
|
||||
|
||||
// normalUnmatched: contracts that weren't matched in exact pass
|
||||
const normalUnmatched = contracts.filter((c) => {
|
||||
if (isGrpcWildcard(c.contractId)) return false; // excluded from exact, handled separately
|
||||
if (isServiceWildcard(c.contractId)) return false; // excluded from exact, handled separately
|
||||
if (isNoisy(c.contractId)) return false; // excluded from matching — don't surface as unmatched
|
||||
const id = `${c.repo}::${c.contractId}`;
|
||||
return c.role === 'provider' ? !matchedProviderIds.has(id) : !matchedConsumerIds.has(id);
|
||||
});
|
||||
|
||||
// Re-add gRPC wildcard contracts — they were never in exact matching
|
||||
const grpcWildcards = contracts.filter((c) => isGrpcWildcard(c.contractId));
|
||||
const unmatched = [...normalUnmatched, ...grpcWildcards];
|
||||
// Re-add service wildcard contracts — they were never in exact matching
|
||||
const serviceWildcards = contracts.filter((c) => isServiceWildcard(c.contractId));
|
||||
const unmatched = [...normalUnmatched, ...serviceWildcards];
|
||||
|
||||
return { matched, unmatched };
|
||||
}
|
||||
@@ -216,21 +245,28 @@ export function runWildcardMatch(
|
||||
providerIndex: Map<string, StoredContract[]>,
|
||||
): WildcardMatchResult {
|
||||
const wildcardConsumers = unmatched.filter(
|
||||
(c) => c.role === 'consumer' && isGrpcWildcard(c.contractId),
|
||||
(c) => c.role === 'consumer' && isServiceWildcard(c.contractId),
|
||||
);
|
||||
const matched: CrossLink[] = [];
|
||||
const matchedConsumerIds = new Set<string>();
|
||||
|
||||
for (const consumer of wildcardConsumers) {
|
||||
const normalized = normalizeContractId(consumer.contractId);
|
||||
const typeEnd = normalized.indexOf('::');
|
||||
const consumerType = normalized.slice(0, typeEnd);
|
||||
// "grpc::com.example.userservice/*" → "com.example.userservice"
|
||||
// "grpc::userservice/*" → "userservice"
|
||||
const fqService = normalized.slice(normalized.indexOf('::') + 2, -2); // strip "grpc::" and "/*"
|
||||
// "thrift::userservice/*" → "userservice"
|
||||
const fqService = normalized.slice(typeEnd + 2, -2); // strip "<type>::" and "/*"
|
||||
const candidateProviders: StoredContract[] = [];
|
||||
const matchedProviderServices = new Set<string>();
|
||||
|
||||
for (const [key, providers] of providerIndex) {
|
||||
// Only match against non-wildcard gRPC providers (method-level IDs)
|
||||
if (!key.startsWith('grpc::') || key.endsWith('/*')) continue;
|
||||
const afterPrefix = key.slice(6); // strip "grpc::"
|
||||
// Only match against non-wildcard same-type providers (method-level IDs).
|
||||
const keyTypeEnd = key.indexOf('::');
|
||||
if (keyTypeEnd < 0 || key.endsWith('/*')) continue;
|
||||
const providerType = key.slice(0, keyTypeEnd);
|
||||
if (providerType !== consumerType) continue;
|
||||
const afterPrefix = key.slice(keyTypeEnd + 2); // strip "<type>::"
|
||||
const slashIdx = afterPrefix.indexOf('/');
|
||||
if (slashIdx < 0) continue;
|
||||
const providerFqService = afterPrefix.slice(0, slashIdx);
|
||||
@@ -242,39 +278,46 @@ export function runWildcardMatch(
|
||||
|
||||
if (!isMatch) continue;
|
||||
|
||||
for (const provider of providers) {
|
||||
// Skip same-repo same-service (same logic as runExactMatch)
|
||||
if (provider.repo === consumer.repo) {
|
||||
if (!provider.service || !consumer.service || provider.service === consumer.service) {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
matchedProviderServices.add(providerFqService);
|
||||
candidateProviders.push(...providers);
|
||||
}
|
||||
|
||||
matched.push({
|
||||
from: {
|
||||
repo: consumer.repo,
|
||||
service: consumer.service,
|
||||
symbolUid: consumer.symbolUid,
|
||||
symbolRef: consumer.symbolRef,
|
||||
},
|
||||
to: {
|
||||
repo: provider.repo,
|
||||
service: provider.service,
|
||||
symbolUid: provider.symbolUid,
|
||||
symbolRef: provider.symbolRef,
|
||||
},
|
||||
type: consumer.type,
|
||||
contractId: consumer.contractId, // consumer's wildcard ID
|
||||
matchType: 'wildcard',
|
||||
confidence: Math.min(provider.confidence, consumer.confidence),
|
||||
});
|
||||
matchedConsumerIds.add(`${consumer.repo}::${consumer.contractId}`);
|
||||
if (consumerType === 'thrift' && !fqService.includes('.') && matchedProviderServices.size > 1) {
|
||||
continue;
|
||||
}
|
||||
|
||||
for (const provider of candidateProviders) {
|
||||
// Skip same-repo same-service (same logic as runExactMatch)
|
||||
if (provider.repo === consumer.repo) {
|
||||
if (!provider.service || !consumer.service || provider.service === consumer.service) {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
matched.push({
|
||||
from: {
|
||||
repo: consumer.repo,
|
||||
service: consumer.service,
|
||||
symbolUid: consumer.symbolUid,
|
||||
symbolRef: consumer.symbolRef,
|
||||
},
|
||||
to: {
|
||||
repo: provider.repo,
|
||||
service: provider.service,
|
||||
symbolUid: provider.symbolUid,
|
||||
symbolRef: provider.symbolRef,
|
||||
},
|
||||
type: consumer.type,
|
||||
contractId: consumer.contractId, // consumer's wildcard ID
|
||||
matchType: 'wildcard',
|
||||
confidence: Math.min(provider.confidence, consumer.confidence),
|
||||
});
|
||||
matchedConsumerIds.add(`${consumer.repo}::${consumer.contractId}`);
|
||||
}
|
||||
}
|
||||
|
||||
const remaining = unmatched.filter((c) => {
|
||||
if (c.role !== 'consumer' || !isGrpcWildcard(c.contractId)) return true;
|
||||
if (c.role !== 'consumer' || !isServiceWildcard(c.contractId)) return true;
|
||||
return !matchedConsumerIds.has(`${c.repo}::${c.contractId}`);
|
||||
});
|
||||
|
||||
|
||||
@@ -6,10 +6,11 @@ import { readRegistry, type RegistryEntry } from '../../storage/repo-manager.js'
|
||||
import type { GroupConfig, RepoHandle, RepoSnapshot, StoredContract, CrossLink } from './types.js';
|
||||
import { HttpRouteExtractor } from './extractors/http-route-extractor.js';
|
||||
import { GrpcExtractor } from './extractors/grpc-extractor.js';
|
||||
import { ThriftExtractor } from './extractors/thrift-extractor.js';
|
||||
import { TopicExtractor } from './extractors/topic-extractor.js';
|
||||
import { ManifestExtractor } from './extractors/manifest-extractor.js';
|
||||
import { discoverWorkspaceLinks } from './extractors/workspace-extractor.js';
|
||||
import { runExactMatch } from './matching.js';
|
||||
import { buildProviderIndex, runExactMatch, runWildcardMatch } from './matching.js';
|
||||
import { detectServiceBoundaries, assignService } from './service-boundary-detector.js';
|
||||
import type { CypherExecutor } from './contract-extractor.js';
|
||||
import { writeContractRegistry } from './storage.js';
|
||||
@@ -96,6 +97,7 @@ export async function syncGroup(config: GroupConfig, opts?: SyncOptions): Promis
|
||||
const resolve = opts?.resolveRepoHandle ?? defaultResolveHandle(entries);
|
||||
const httpEx = new HttpRouteExtractor();
|
||||
const grpcEx = new GrpcExtractor();
|
||||
const thriftEx = new ThriftExtractor();
|
||||
const topicEx = new TopicExtractor();
|
||||
dbExecutors = new Map<string, CypherExecutor>();
|
||||
const openPoolIds: string[] = [];
|
||||
@@ -143,6 +145,17 @@ export async function syncGroup(config: GroupConfig, opts?: SyncOptions): Promis
|
||||
}
|
||||
}
|
||||
|
||||
if (config.detect.thrift) {
|
||||
const extracted = await thriftEx.extract(executor, handle.repoPath, handle);
|
||||
for (const c of extracted) {
|
||||
autoContracts.push({
|
||||
...c,
|
||||
repo: groupPath,
|
||||
service: assignService(c.symbolRef.filePath, boundaries),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (config.detect.topics) {
|
||||
const extracted = await topicEx.extract(executor, handle.repoPath, handle);
|
||||
for (const c of extracted) {
|
||||
@@ -234,13 +247,15 @@ export async function syncGroup(config: GroupConfig, opts?: SyncOptions): Promis
|
||||
}
|
||||
}
|
||||
|
||||
const { matched, unmatched } = runExactMatch(autoContracts, undefined, config.matching);
|
||||
const providerIndex = buildProviderIndex(autoContracts, config.matching);
|
||||
const { matched, unmatched } = runExactMatch(autoContracts, providerIndex, config.matching);
|
||||
const wildcard = runWildcardMatch(unmatched, providerIndex);
|
||||
|
||||
// Dedupe cross-links. Manifest contracts participate in runExactMatch, so a
|
||||
// manifest-declared link can also emit a matchType:'exact' CrossLink with the
|
||||
// same endpoints. Prefer the manifest version — it reflects operator intent
|
||||
// and carries matchType:'manifest' which downstream consumers may rely on.
|
||||
const crossLinks = dedupeCrossLinks([...manifestCrossLinks, ...matched]);
|
||||
const crossLinks = dedupeCrossLinks([...manifestCrossLinks, ...matched, ...wildcard.matched]);
|
||||
const allContracts: StoredContract[] = autoContracts;
|
||||
|
||||
const registry: ContractRegistry = {
|
||||
@@ -259,7 +274,7 @@ export async function syncGroup(config: GroupConfig, opts?: SyncOptions): Promis
|
||||
return {
|
||||
contracts: allContracts,
|
||||
crossLinks,
|
||||
unmatched,
|
||||
unmatched: wildcard.remaining,
|
||||
missingRepos,
|
||||
repoSnapshots,
|
||||
};
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
export type ContractType = 'http' | 'grpc' | 'topic' | 'lib' | 'custom';
|
||||
export type ContractType = 'http' | 'grpc' | 'thrift' | 'topic' | 'lib' | 'custom';
|
||||
export type MatchType = 'exact' | 'manifest' | 'wildcard' | 'bm25' | 'embedding';
|
||||
export type ContractRole = 'provider' | 'consumer';
|
||||
|
||||
@@ -24,6 +24,7 @@ export interface GroupManifestLink {
|
||||
export interface DetectConfig {
|
||||
http: boolean;
|
||||
grpc: boolean;
|
||||
thrift: boolean;
|
||||
topics: boolean;
|
||||
shared_libs: boolean;
|
||||
embedding_fallback: boolean;
|
||||
|
||||
@@ -94,6 +94,31 @@ export function resolveWorkerPoolOptions(
|
||||
};
|
||||
}
|
||||
|
||||
function waitForWorkerOnline(worker: Worker): Promise<void> {
|
||||
return new Promise<void>((resolve, reject) => {
|
||||
const cleanup = () => {
|
||||
worker.removeListener('online', onOnline);
|
||||
worker.removeListener('error', onError);
|
||||
worker.removeListener('exit', onExit);
|
||||
};
|
||||
const onOnline = () => {
|
||||
cleanup();
|
||||
resolve();
|
||||
};
|
||||
const onError = (err: Error) => {
|
||||
cleanup();
|
||||
reject(err);
|
||||
};
|
||||
const onExit = (code: number) => {
|
||||
cleanup();
|
||||
reject(new Error(`Replacement worker exited with code ${code} before coming online`));
|
||||
};
|
||||
worker.once('online', onOnline);
|
||||
worker.once('error', onError);
|
||||
worker.once('exit', onExit);
|
||||
});
|
||||
}
|
||||
|
||||
function estimateItemBytes(item: unknown): number {
|
||||
if (typeof item !== 'object' || item === null) return 0;
|
||||
const content = (item as { content?: unknown }).content;
|
||||
@@ -209,7 +234,21 @@ export const createWorkerPool = (
|
||||
const replaceWorker = async (workerIndex: number) => {
|
||||
const worker = workers[workerIndex];
|
||||
await worker?.terminate().catch(() => undefined);
|
||||
if (!stopped) workers[workerIndex] = new Worker(workerUrl);
|
||||
if (stopped) return;
|
||||
const replacement = new Worker(workerUrl);
|
||||
try {
|
||||
await waitForWorkerOnline(replacement);
|
||||
} catch (err) {
|
||||
await replacement.terminate().catch(() => undefined);
|
||||
throw new Error(
|
||||
`Replacement worker ${workerIndex} failed to start: ${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
}
|
||||
if (stopped) {
|
||||
await replacement.terminate().catch(() => undefined);
|
||||
return;
|
||||
}
|
||||
workers[workerIndex] = replacement;
|
||||
};
|
||||
|
||||
const fail = async (err: Error) => {
|
||||
@@ -341,9 +380,7 @@ export const createWorkerPool = (
|
||||
try {
|
||||
await replaceWorker(workerIndex);
|
||||
} catch (err) {
|
||||
void fail(
|
||||
err instanceof Error ? err : new Error(`Worker replacement failed: ${err}`),
|
||||
);
|
||||
void fail(err instanceof Error ? err : new Error(String(err)));
|
||||
return;
|
||||
} finally {
|
||||
activeWorkers--;
|
||||
|
||||
@@ -257,6 +257,14 @@ export const withLbugDb = async <T>(dbPath: string, operation: () => Promise<T>)
|
||||
// Close stale connection inside the session lock to prevent race conditions
|
||||
// with concurrent operations that might acquire the lock between cleanup steps
|
||||
await runWithSessionLock(async () => {
|
||||
// CHECKPOINT before close to flush WAL contents (same rationale as closeLbug)
|
||||
if (conn) {
|
||||
try {
|
||||
await conn.query('CHECKPOINT');
|
||||
} catch {
|
||||
/* best-effort */
|
||||
}
|
||||
}
|
||||
try {
|
||||
if (conn) await conn.close();
|
||||
} catch {
|
||||
@@ -294,6 +302,14 @@ const ensureLbugInitialized = async (dbPath: string) => {
|
||||
const doInitLbug = async (dbPath: string) => {
|
||||
// Different database requested — close the old one first
|
||||
if (conn || db) {
|
||||
// CHECKPOINT before close to flush WAL contents (same rationale as closeLbug)
|
||||
if (conn) {
|
||||
try {
|
||||
await conn.query('CHECKPOINT');
|
||||
} catch {
|
||||
/* ignore — older LadybugDB or schemaless DB may not accept it */
|
||||
}
|
||||
}
|
||||
try {
|
||||
if (conn) await conn.close();
|
||||
} catch {}
|
||||
@@ -1048,6 +1064,21 @@ export const fetchExistingEmbeddingHashes = async (
|
||||
};
|
||||
|
||||
export const closeLbug = async (): Promise<void> => {
|
||||
// CHECKPOINT before close so the WAL/.shadow contents are flushed into
|
||||
// the main database file. Without this, LadybugDB 0.16.0's non-blocking
|
||||
// checkpoint thread can outlive the close call and leave sidecar pages
|
||||
// pending on disk, which makes a subsequent read-side open either race
|
||||
// with the WAL replay or trip the database-id check on the sidecars.
|
||||
// This is especially critical after embedding writes, which generate
|
||||
// large amounts of WAL data. CHECKPOINT is a no-op when there's nothing
|
||||
// pending, so it's cheap on the happy path.
|
||||
if (conn) {
|
||||
try {
|
||||
await conn.query('CHECKPOINT');
|
||||
} catch {
|
||||
/* ignore — older LadybugDB or schemaless DB may not accept it */
|
||||
}
|
||||
}
|
||||
if (conn) {
|
||||
try {
|
||||
await conn.close();
|
||||
|
||||
@@ -30,7 +30,13 @@ import {
|
||||
registerRepo,
|
||||
cleanupOldKuzuFiles,
|
||||
} from '../storage/repo-manager.js';
|
||||
import { getCurrentCommit, getRemoteUrl, hasGitDir, getInferredRepoName } from '../storage/git.js';
|
||||
import {
|
||||
getCurrentCommit,
|
||||
getRemoteUrl,
|
||||
hasGitDir,
|
||||
getInferredRepoName,
|
||||
resolveRepoIdentityRoot,
|
||||
} from '../storage/git.js';
|
||||
import type { CachedEmbedding } from './embeddings/types.js';
|
||||
import { generateAIContextFiles } from '../cli/ai-context.js';
|
||||
import { EMBEDDING_TABLE_NAME } from './lbug/schema.js';
|
||||
@@ -54,6 +60,13 @@ export interface AnalyzeOptions {
|
||||
*/
|
||||
force?: boolean;
|
||||
embeddings?: boolean;
|
||||
/**
|
||||
* Override the auto-skip node-count cap for embedding generation.
|
||||
* `undefined` (default) keeps the built-in 50,000-node safety limit;
|
||||
* `0` disables the cap entirely; any positive integer sets a custom cap.
|
||||
* Mapped from the CLI's `--embeddings [limit]` argument.
|
||||
*/
|
||||
embeddingsNodeLimit?: number;
|
||||
/**
|
||||
* Explicitly drop any embeddings present in the existing index instead of
|
||||
* preserving them. Only meaningful when `embeddings` is false/undefined:
|
||||
@@ -101,14 +114,15 @@ export interface AnalyzeResult {
|
||||
pipelineResult?: any;
|
||||
}
|
||||
|
||||
/** Threshold: auto-skip embeddings for repos with more nodes than this */
|
||||
const EMBEDDING_NODE_LIMIT = 50_000;
|
||||
|
||||
// Re-export the pure flag-derivation helper so external callers (and tests)
|
||||
// keep importing from this module's stable surface.
|
||||
export { deriveEmbeddingMode } from './embedding-mode.js';
|
||||
export { deriveEmbeddingMode, DEFAULT_EMBEDDING_NODE_LIMIT } from './embedding-mode.js';
|
||||
export type { EmbeddingMode } from './embedding-mode.js';
|
||||
import { deriveEmbeddingMode as _deriveEmbeddingMode } from './embedding-mode.js';
|
||||
import {
|
||||
deriveEmbeddingMode as _deriveEmbeddingMode,
|
||||
deriveEmbeddingCap,
|
||||
DEFAULT_EMBEDDING_NODE_LIMIT,
|
||||
} from './embedding-mode.js';
|
||||
|
||||
export const PHASE_LABELS: Record<string, string> = {
|
||||
extracting: 'Scanning files',
|
||||
@@ -168,7 +182,13 @@ export async function runFullAnalysis(
|
||||
if (currentCommit !== '') {
|
||||
await ensureGitNexusIgnored(repoPath);
|
||||
return {
|
||||
repoName: options.registryName ?? getInferredRepoName(repoPath) ?? path.basename(repoPath),
|
||||
// `resolveRepoIdentityRoot` collapses worktree roots to the
|
||||
// canonical repo basename (#1259) but leaves arbitrary subdirs
|
||||
// and `--skip-git` paths unchanged (#1232/#1233 intent preserved).
|
||||
repoName:
|
||||
options.registryName ??
|
||||
getInferredRepoName(repoPath) ??
|
||||
path.basename(resolveRepoIdentityRoot(repoPath)),
|
||||
repoPath,
|
||||
stats: existingMeta.stats ?? {},
|
||||
alreadyUpToDate: true,
|
||||
@@ -321,8 +341,27 @@ export async function runFullAnalysis(
|
||||
let semanticMode: 'vector-index' | 'exact-scan' | undefined;
|
||||
|
||||
if (shouldGenerateEmbeddings) {
|
||||
if (stats.nodes <= EMBEDDING_NODE_LIMIT) {
|
||||
const { skipForCap, capDisabled, nodeLimit } = deriveEmbeddingCap(
|
||||
stats.nodes,
|
||||
options.embeddingsNodeLimit,
|
||||
);
|
||||
if (!skipForCap) {
|
||||
embeddingSkipped = false;
|
||||
if (capDisabled && stats.nodes > DEFAULT_EMBEDDING_NODE_LIMIT) {
|
||||
log(
|
||||
`Embedding node-count cap disabled — generating embeddings for ` +
|
||||
`${stats.nodes.toLocaleString()} nodes. Ensure sufficient memory; ` +
|
||||
`the default ${DEFAULT_EMBEDDING_NODE_LIMIT.toLocaleString()}-node ` +
|
||||
`cap exists to prevent OOM.`,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
log(
|
||||
`Embeddings skipped: ${stats.nodes.toLocaleString()} nodes exceeds ` +
|
||||
`the ${nodeLimit.toLocaleString()}-node safety cap. ` +
|
||||
`Override with \`--embeddings 0\` to disable the cap, or ` +
|
||||
`\`--embeddings <n>\` to set a custom cap.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -345,7 +384,19 @@ export async function runFullAnalysis(
|
||||
}
|
||||
|
||||
const { readServerMapping } = await import('./embeddings/server-mapping.js');
|
||||
const projectName = path.basename(repoPath);
|
||||
// Mirror the registry's name-resolution chain so the server-mapping
|
||||
// lookup key stays aligned with the final registry name (#1259):
|
||||
// --name → remote-derived → canonical-root basename
|
||||
// (preserved-alias is intentionally NOT consulted here — server
|
||||
// mappings are addressed by the operationally-meaningful name the
|
||||
// user configures, not by a sticky registry-only alias they may not
|
||||
// know about. The previous canonical-only logic ignored both --name
|
||||
// and remote-derived names, silently breaking server-mapping for
|
||||
// anyone with a `--name` alias or remote-named repo.)
|
||||
const projectName =
|
||||
options.registryName ??
|
||||
getInferredRepoName(repoPath) ??
|
||||
path.basename(resolveRepoIdentityRoot(repoPath));
|
||||
const serverName = await readServerMapping(projectName);
|
||||
const embeddingResult = await runEmbeddingPipeline(
|
||||
executeQuery,
|
||||
|
||||
@@ -33,7 +33,7 @@ import { mountMCPEndpoints } from './mcp-http.js';
|
||||
import { fork } from 'child_process';
|
||||
import { fileURLToPath, pathToFileURL } from 'url';
|
||||
import { JobManager } from './analyze-job.js';
|
||||
import { assertString, escapeRegExp, BadRequestError } from './validation.js';
|
||||
import { assertString, escapeRegExp, BadRequestError, createRouteLimiter } from './validation.js';
|
||||
import { extractRepoName, getCloneDir, cloneOrPull } from './git-clone.js';
|
||||
|
||||
const _require = createRequire(import.meta.url);
|
||||
@@ -183,6 +183,7 @@ a.ext:hover{text-decoration:underline}
|
||||
<div class="section-title">Endpoints</div>
|
||||
<p class="endpoint"><a href="/api/info">/api/info</a> <span style="color:#5a5a70">— Server version & context</span></p>
|
||||
<p class="endpoint"><a href="/api/repos">/api/repos</a> <span style="color:#5a5a70">— Indexed repositories</span></p>
|
||||
<p class="endpoint"><code>/api/health</code> <span style="color:#5a5a70">— Docker/orchestrator healthcheck</span></p>
|
||||
<p class="endpoint"><code>/api/heartbeat</code> <span style="color:#5a5a70">— SSE heartbeat</span></p>
|
||||
<p class="endpoint"><code>/api/graph</code> <code>/api/query</code> <code>/api/search</code> <span style="color:#5a5a70">— Data</span></p>
|
||||
<p class="endpoint"><code>/api/mcp</code> <span style="color:#5a5a70">— MCP over StreamableHTTP</span></p>
|
||||
@@ -217,7 +218,19 @@ export const registerWebUI = (app: express.Express, staticDir: string | null): v
|
||||
// The regex excludes /api paths AND paths with file extensions (.js, .css, etc.)
|
||||
// so missing assets get real 404s instead of the SPA HTML.
|
||||
// Adding routes below this will be unreachable for non-API, non-asset paths.
|
||||
app.get(SPA_FALLBACK_REGEX, (_req, res) => {
|
||||
// Rate-limited (CodeQL js/missing-rate-limiting): the SPA fallback
|
||||
// serves a constant index.html, but the FS access from a route handler
|
||||
// is enough to trip the analyzer. The limit is generous (300 rpm/IP =
|
||||
// 5 req/s sustained) so that multi-tab browser navigation, prefetch,
|
||||
// and service-worker revalidation do not produce 429s for legitimate
|
||||
// SPA users. At this rate, real browser navigation is extremely
|
||||
// unlikely to hit the limit in practice, so the cosmetic issue of
|
||||
// JSON-on-429 to a browser is a low-likelihood path. Content
|
||||
// negotiation on the 429 (returning the SPA shell to HTML clients
|
||||
// instead of `{ error: '...' }`) would require swapping
|
||||
// express-rate-limit's `message` for a `handler` function and is
|
||||
// deferred to keep this PR focused on closing the CodeQL alert.
|
||||
app.get(SPA_FALLBACK_REGEX, createRouteLimiter({ limit: 300 }), (_req, res) => {
|
||||
res.sendFile(path.join(staticDir, 'index.html'));
|
||||
});
|
||||
} else {
|
||||
@@ -612,6 +625,27 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
const app = express();
|
||||
app.disable('x-powered-by');
|
||||
|
||||
// Trust X-Forwarded-* headers only when the connection comes from the
|
||||
// local loopback or RFC1918 private/link-local addresses — exactly the
|
||||
// origins the CORS allowlist accepts. Without this, every request behind
|
||||
// any reverse proxy / Docker bridge counts as the same `req.ip` and a
|
||||
// single user can trip the per-IP rate limiter for everyone.
|
||||
//
|
||||
// SCOPE: this setting is process-wide. Every middleware and route in this
|
||||
// Express app sees req.ip resolved from X-Forwarded-For when the upstream
|
||||
// hop is in the trusted set above — not just the rate-limited routes.
|
||||
// Future IP-based middleware (audit logging, IP-bound authz) inherits this
|
||||
// behavior.
|
||||
//
|
||||
// CLOUD-DEPLOY CAVEAT: a public cloud LB (AWS ALB, Cloudflare, Fly.io
|
||||
// edge, CGNAT 100.64/10) is NOT in the trusted set. In those topologies
|
||||
// req.ip will collapse to the LB hop IP for every request and the per-IP
|
||||
// rate limiter degrades to per-server. Add an explicit env-var override
|
||||
// and document the cloud-deploy story before binding to a non-loopback
|
||||
// host in those topologies (tracked as a follow-up; not blocking for the
|
||||
// local-bound default).
|
||||
app.set('trust proxy', 'loopback, linklocal, uniquelocal');
|
||||
|
||||
// CORS: allow localhost, private/LAN networks, and the deployed site.
|
||||
// Non-browser requests (curl, server-to-server) have no origin and are allowed.
|
||||
// Disallowed origins get the response without Access-Control-Allow-Origin,
|
||||
@@ -744,6 +778,13 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
return found;
|
||||
};
|
||||
|
||||
// Lightweight healthcheck for Docker/orchestrator probes (#1147).
|
||||
// Returns immediately so container managers do not confuse a long-lived
|
||||
// SSE stream with an unhealthy server.
|
||||
app.get('/api/health', (_req, res) => {
|
||||
res.json({ status: 'ok' });
|
||||
});
|
||||
|
||||
// SSE heartbeat — clients connect to detect server liveness instantly.
|
||||
// When the server shuts down, the TCP connection drops and the client's
|
||||
// EventSource fires onerror immediately (no polling delay).
|
||||
@@ -829,7 +870,10 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
});
|
||||
|
||||
// Delete a repo — removes index, clone dir (if any), and unregisters it
|
||||
app.delete('/api/repo', async (req, res) => {
|
||||
// Rate-limited (CodeQL js/missing-rate-limiting): destructive operation
|
||||
// doing fs.rm of clone + storage dirs. Default 60 rpm/IP is generous for
|
||||
// delete; tighten if abuse is observed.
|
||||
app.delete('/api/repo', createRouteLimiter(), async (req, res) => {
|
||||
try {
|
||||
const repoName = requestedRepo(req);
|
||||
if (!repoName) {
|
||||
@@ -1142,7 +1186,8 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
});
|
||||
|
||||
// Read file — with path traversal guard
|
||||
app.get('/api/file', async (req, res) => {
|
||||
// Rate-limited (CodeQL js/missing-rate-limiting): per-request fs.readFile.
|
||||
app.get('/api/file', createRouteLimiter(), async (req, res) => {
|
||||
const entry = await resolveRepo(requestedRepo(req));
|
||||
if (!entry) {
|
||||
res.status(404).json({ error: 'Repository not found' });
|
||||
@@ -1153,7 +1198,10 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
|
||||
// Grep — regex search across file contents in the indexed repo
|
||||
// Uses filesystem-based search for memory efficiency (never loads all files into memory)
|
||||
app.get('/api/grep', async (req, res) => {
|
||||
// Rate-limited (CodeQL js/missing-rate-limiting): scans every file in
|
||||
// the indexed repo per request — heaviest I/O endpoint. Same default 60
|
||||
// rpm/IP for now; consider tightening if real-world load shows abuse.
|
||||
app.get('/api/grep', createRouteLimiter(), async (req, res) => {
|
||||
try {
|
||||
const entry = await resolveRepo(requestedRepo(req));
|
||||
if (!entry) {
|
||||
@@ -1308,7 +1356,7 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
// ── Analyze API ──────────────────────────────────────────────────────
|
||||
|
||||
// POST /api/analyze — start a new analysis job
|
||||
app.post('/api/analyze', async (req, res) => {
|
||||
app.post('/api/analyze', createRouteLimiter({ limit: 10 }), async (req, res) => {
|
||||
try {
|
||||
const { url: repoUrl, path: repoLocalPath, force, embeddings, dropEmbeddings } = req.body;
|
||||
|
||||
@@ -1575,7 +1623,7 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
const embedJobManager = new JobManager();
|
||||
|
||||
// POST /api/embed — trigger server-side embedding generation
|
||||
app.post('/api/embed', async (req, res) => {
|
||||
app.post('/api/embed', createRouteLimiter({ limit: 20 }), async (req, res) => {
|
||||
try {
|
||||
const entry = await resolveRepo(requestedRepo(req));
|
||||
if (!entry) {
|
||||
@@ -1654,6 +1702,16 @@ export const createServer = async (port: number, host: string = '127.0.0.1') =>
|
||||
undefined, // context
|
||||
existingEmbeddings,
|
||||
);
|
||||
|
||||
// Flush WAL so subsequent /api/search requests see the new
|
||||
// embeddings immediately (#1149). In the CLI path closeLbug()
|
||||
// handles this during process exit, but the server keeps the
|
||||
// connection open for other routes -- a CHECKPOINT is enough.
|
||||
try {
|
||||
await executeQuery('CHECKPOINT');
|
||||
} catch {
|
||||
/* best-effort -- older LadybugDB may not support it */
|
||||
}
|
||||
});
|
||||
|
||||
clearTimeout(embedTimeout);
|
||||
|
||||
@@ -19,6 +19,8 @@
|
||||
*/
|
||||
|
||||
import path from 'node:path';
|
||||
import rateLimit, { type RateLimitRequestHandler, ipKeyGenerator } from 'express-rate-limit';
|
||||
import type { Request } from 'express';
|
||||
|
||||
/**
|
||||
* Thrown by validation helpers when user input is rejected.
|
||||
@@ -95,3 +97,68 @@ export function assertSafePath(rawPath: string, root: string): string {
|
||||
export function escapeRegExp(input: string): string {
|
||||
return input.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
}
|
||||
|
||||
/**
|
||||
* Default rate-limit policy for FS-touching API routes (CodeQL
|
||||
* js/missing-rate-limiting). Tuned for the local-bound HTTP server's expected
|
||||
* traffic — interactive web UI use stays well under the limit; abusive loops
|
||||
* trip 429.
|
||||
*
|
||||
* Module-internal — not exported. Tests assert the observable behavior
|
||||
* (61st request returns 429), not the literal value, so callers don't grow
|
||||
* a coupling on this number.
|
||||
*/
|
||||
const DEFAULT_RATE_LIMIT_RPM = 60;
|
||||
|
||||
/**
|
||||
* Project-specific subset of express-rate-limit options that callers may
|
||||
* override. Intentionally narrow — `Partial<RateLimitOptions>` would let a
|
||||
* caller pass `{ skip: () => true }` and silently disable limiting on a
|
||||
* route. The two knobs below are sufficient for tests and any future
|
||||
* legitimate per-route tuning.
|
||||
*/
|
||||
export interface RouteLimiterOverrides {
|
||||
windowMs?: number;
|
||||
/** Canonical name in express-rate-limit v8+. `max` is the deprecated alias. */
|
||||
limit?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a per-route rate-limit middleware with project-uniform defaults.
|
||||
*
|
||||
* Each call returns a NEW limiter instance — independent counters per route,
|
||||
* so /api/file traffic doesn't push /api/grep into 429.
|
||||
*
|
||||
* Defaults:
|
||||
* - 60 requests per IP per minute
|
||||
* - draft-7 RateLimit-* response headers (no legacy X-RateLimit-* headers)
|
||||
* - 429 with a JSON body matching the project's `{ error: '...' }` shape
|
||||
* - passOnStoreError: store failures let the request through rather than
|
||||
* producing an HTML 500 from Express's default error handler
|
||||
* - keyGenerator: req.ip with a socket.remoteAddress fallback so abruptly
|
||||
* closed connections do not trigger ERR_ERL_UNDEFINED_IP_ADDRESS
|
||||
* (which would 500 the request via Express's default error handler).
|
||||
* The IP is passed through `ipKeyGenerator` so IPv6 addresses are
|
||||
* normalised to their /56 subnet — without this, each IPv6 address
|
||||
* gets its own counter and the limit is trivially bypassed (#1360).
|
||||
* Caller must wire `app.set('trust proxy', ...)` correctly — see
|
||||
* createServer in api.ts.
|
||||
*
|
||||
* Tests pass `{ windowMs: 100, limit: 3 }` to keep limiter tests fast and
|
||||
* deterministic.
|
||||
*/
|
||||
export function createRouteLimiter(opts?: RouteLimiterOverrides): RateLimitRequestHandler {
|
||||
return rateLimit({
|
||||
windowMs: 60 * 1000,
|
||||
limit: DEFAULT_RATE_LIMIT_RPM,
|
||||
standardHeaders: 'draft-7',
|
||||
legacyHeaders: false,
|
||||
passOnStoreError: true,
|
||||
keyGenerator: (req: Request) => {
|
||||
const ip = req.ip ?? req.socket?.remoteAddress;
|
||||
return ip ? ipKeyGenerator(ip) : 'unknown';
|
||||
},
|
||||
message: { error: 'Too many requests, please try again later.' },
|
||||
...opts,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -15,7 +15,17 @@ export const isGitRepo = (repoPath: string): boolean => {
|
||||
|
||||
export const getCurrentCommit = (repoPath: string): string => {
|
||||
try {
|
||||
return execSync('git rev-parse HEAD', { cwd: repoPath }).toString().trim();
|
||||
return execSync('git rev-parse HEAD', {
|
||||
cwd: repoPath,
|
||||
// Suppress stderr -- without an explicit stdio option, Node's execSync
|
||||
// forwards the child's stderr to the parent process (documented behaviour).
|
||||
// When repoPath is not inside a git worktree, git prints
|
||||
// "fatal: not a git repository" to stderr, which leaks to the user's
|
||||
// terminal even though the error is caught here (#1172).
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
})
|
||||
.toString()
|
||||
.trim();
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
@@ -86,7 +96,13 @@ export const getRemoteUrl = (repoPath: string): string | undefined => {
|
||||
*/
|
||||
export const getGitRoot = (fromPath: string): string | null => {
|
||||
try {
|
||||
const raw = execSync('git rev-parse --show-toplevel', { cwd: fromPath }).toString().trim();
|
||||
const raw = execSync('git rev-parse --show-toplevel', {
|
||||
cwd: fromPath,
|
||||
// Suppress stderr -- see getCurrentCommit comment and #1172.
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
})
|
||||
.toString()
|
||||
.trim();
|
||||
// On Windows, git returns /d/Projects/Foo — path.resolve normalizes to D:\Projects\Foo
|
||||
return path.resolve(raw);
|
||||
} catch {
|
||||
@@ -94,6 +110,80 @@ export const getGitRoot = (fromPath: string): string | null => {
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Get the *canonical* repository root, dereferencing git worktrees.
|
||||
*
|
||||
* Unlike `getGitRoot` (which uses `git rev-parse --show-toplevel` and
|
||||
* returns the WORKTREE's root when called inside a linked worktree),
|
||||
* this uses `git rev-parse --git-common-dir` — the shared `.git`
|
||||
* directory, identical for the main checkout and every linked
|
||||
* worktree — and returns its parent.
|
||||
*
|
||||
* Why it matters (#1259): when `gitnexus analyze` runs inside a
|
||||
* worktree (e.g. `/repo/wt-feature/`), deriving `repoName` from
|
||||
* `path.basename(getGitRoot(cwd))` registers the project under the
|
||||
* worktree's directory slug (`wt-feature`) instead of the canonical
|
||||
* repo's basename (`repo`). Each worktree then re-registers as a
|
||||
* "different" project, AGENTS.md is rewritten with the wrong MCP URI,
|
||||
* and Claude-Code-style worktree workflows silently accumulate
|
||||
* duplicate registry entries.
|
||||
*
|
||||
* Returns `null` when the path is not inside a git repository or
|
||||
* `git` is not available, so callers can chain safely:
|
||||
* `getCanonicalRepoRoot(p) ?? getGitRoot(p) ?? p`.
|
||||
*
|
||||
* `--path-format=absolute` is required because `--git-common-dir`
|
||||
* returns a path *relative to cwd* by default (e.g. `../.git` when
|
||||
* called from a worktree), which would resolve to the wrong absolute
|
||||
* path if the caller later resolved it from a different directory.
|
||||
*/
|
||||
export const getCanonicalRepoRoot = (fromPath: string): string | null => {
|
||||
try {
|
||||
const commonDir = execSync('git rev-parse --path-format=absolute --git-common-dir', {
|
||||
cwd: fromPath,
|
||||
stdio: ['ignore', 'pipe', 'ignore'],
|
||||
})
|
||||
.toString()
|
||||
.trim();
|
||||
if (!commonDir) return null;
|
||||
// Common dir is `<repo>/.git` for both the main checkout and all
|
||||
// linked worktrees. Its parent is the canonical repo root.
|
||||
return path.dirname(path.resolve(commonDir));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve `fromPath` to the directory whose basename should drive the
|
||||
* registry name (#1259) — the *identity root*. Three outcomes:
|
||||
*
|
||||
* 1. `fromPath` IS the canonical checkout root → returns it unchanged.
|
||||
* 2. `fromPath` is a linked-worktree root (has its own `.git` entry, but
|
||||
* `git rev-parse --git-common-dir` points at a different `.git`) →
|
||||
* returns the canonical repo root.
|
||||
* 3. `fromPath` is anything else — an arbitrary subdir under a git repo,
|
||||
* a non-git folder, a `--skip-git` subdir of an unrelated parent
|
||||
* checkout — returns `fromPath` unchanged.
|
||||
*
|
||||
* Why not just use `getCanonicalRepoRoot` directly? Because `git rev-parse
|
||||
* --git-common-dir` resolves the same canonical root for ANY path inside
|
||||
* a git repo, including unrelated subdirs. Using it for registry-name
|
||||
* derivation would silently re-key a `--skip-git` subdir analyze under
|
||||
* the parent git's basename, defeating the user's `--skip-git` intent
|
||||
* (regressing the #1232/#1233 fix). The "is this path a tree root"
|
||||
* gate confines the canonical-root collapse to exactly the cases where
|
||||
* #1259 matters: main checkouts and linked worktrees.
|
||||
*/
|
||||
export const resolveRepoIdentityRoot = (fromPath: string): string => {
|
||||
const resolved = path.resolve(fromPath);
|
||||
const canonical = getCanonicalRepoRoot(resolved);
|
||||
if (!canonical) return resolved; // non-git → use as-is
|
||||
if (canonical === resolved) return canonical; // canonical checkout
|
||||
if (hasGitDir(resolved)) return canonical; // linked worktree (has .git file)
|
||||
return resolved; // arbitrary subdir under a git repo → preserve as-is
|
||||
};
|
||||
|
||||
/**
|
||||
* Find a git root by checking only `.git` entries on the ancestor chain.
|
||||
*
|
||||
|
||||
@@ -10,7 +10,7 @@ import fs from 'fs/promises';
|
||||
import { realpathSync } from 'fs';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
import { getInferredRepoName } from './git.js';
|
||||
import { getInferredRepoName, resolveRepoIdentityRoot } from './git.js';
|
||||
|
||||
/**
|
||||
* Normalise a repo path for registry comparison across platforms
|
||||
@@ -389,6 +389,17 @@ export class RegistryNameCollisionError extends Error {
|
||||
const hasCustomAlias = (entry: RegistryEntry, inferredName: string | null): boolean => {
|
||||
const resolved = path.resolve(entry.path);
|
||||
if (entry.name === path.basename(resolved)) return false;
|
||||
// Canonical-root-derived names are not user aliases either (#1259):
|
||||
// a worktree registered under the canonical repo's basename
|
||||
// (e.g. `{name: 'repo', path: '/repo/wt-feature'}`) must re-register
|
||||
// cleanly without firing the duplicate-name collision guard. Without
|
||||
// this check `entry.name = 'repo'` !== `path.basename('/repo/wt-feature') = 'wt-feature'`,
|
||||
// so the prior check returns true → `isPreservedAlias = true` → guard
|
||||
// throws `RegistryNameCollisionError` against the also-registered
|
||||
// canonical checkout entry. The Claude-Code per-task worktree workflow
|
||||
// — analyze canonical, then analyze worktree, then re-analyze worktree
|
||||
// — would break on the third call.
|
||||
if (entry.name === path.basename(resolveRepoIdentityRoot(resolved))) return false;
|
||||
if (inferredName && entry.name === inferredName) return false;
|
||||
return true;
|
||||
};
|
||||
@@ -470,7 +481,13 @@ export const registerRepo = async (
|
||||
name = existing.name;
|
||||
isPreservedAlias = true;
|
||||
} else {
|
||||
name = inferred ?? path.basename(resolved);
|
||||
// Canonical-root fallback: when `resolved` is a worktree root,
|
||||
// derive the registry name from the canonical repo's basename, not
|
||||
// the worktree slug — see #1259. `resolveRepoIdentityRoot` confines
|
||||
// the collapse to canonical checkouts and linked worktree roots only,
|
||||
// so `--skip-git` subdirs of unrelated parent git repos keep using
|
||||
// their own basename (preserves the #1232/#1233 fix's intent).
|
||||
name = inferred ?? path.basename(resolveRepoIdentityRoot(resolved));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -300,7 +300,7 @@ describe('worker pool integration', () => {
|
||||
|
||||
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => undefined);
|
||||
pool = createWorkerPool(pathToFileURL(workerPath) as URL, 1, {
|
||||
subBatchIdleTimeoutMs: 150,
|
||||
subBatchIdleTimeoutMs: 500,
|
||||
maxTimeoutRetries: 1,
|
||||
timeoutBackoffFactor: 4,
|
||||
});
|
||||
@@ -308,7 +308,46 @@ describe('worker pool integration', () => {
|
||||
try {
|
||||
const results = await pool.dispatch<any, any>([{ path: 'retry.ts', content: '' }]);
|
||||
expect(results).toEqual([{ fileCount: 1, recovered: true }]);
|
||||
expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('Retrying with 0.6s timeout'));
|
||||
expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('Retrying with 2s timeout'));
|
||||
} finally {
|
||||
warnSpy.mockRestore();
|
||||
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects dispatch when replacement worker crashes during startup', async () => {
|
||||
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-worker-replace-fail-'));
|
||||
const markerPath = path.join(tempDir, 'first-attempt.txt');
|
||||
const workerPath = path.join(tempDir, 'worker.js');
|
||||
fs.writeFileSync(
|
||||
workerPath,
|
||||
`
|
||||
const fs = require('node:fs');
|
||||
const { parentPort } = require('node:worker_threads');
|
||||
const markerPath = ${JSON.stringify(markerPath)};
|
||||
if (fs.existsSync(markerPath)) {
|
||||
throw new Error('simulated startup crash');
|
||||
}
|
||||
parentPort.on('message', (msg) => {
|
||||
if (msg && msg.type === 'sub-batch') {
|
||||
fs.writeFileSync(markerPath, 'stalled');
|
||||
return;
|
||||
}
|
||||
});
|
||||
`,
|
||||
);
|
||||
|
||||
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => undefined);
|
||||
pool = createWorkerPool(pathToFileURL(workerPath) as URL, 1, {
|
||||
subBatchIdleTimeoutMs: 150,
|
||||
maxTimeoutRetries: 1,
|
||||
timeoutBackoffFactor: 4,
|
||||
});
|
||||
|
||||
try {
|
||||
await expect(pool.dispatch<any, any>([{ path: 'crash.ts', content: '' }])).rejects.toThrow(
|
||||
/simulated startup crash|exited with code/,
|
||||
);
|
||||
} finally {
|
||||
warnSpy.mockRestore();
|
||||
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
const runFullAnalysisMock = vi.fn();
|
||||
|
||||
vi.mock('../../src/core/run-analyze.js', () => ({
|
||||
runFullAnalysis: runFullAnalysisMock,
|
||||
}));
|
||||
|
||||
vi.mock('../../src/core/lbug/lbug-adapter.js', () => ({
|
||||
closeLbug: vi.fn(async () => undefined),
|
||||
}));
|
||||
|
||||
vi.mock('../../src/storage/repo-manager.js', () => ({
|
||||
getStoragePaths: vi.fn(() => ({ storagePath: '.gitnexus', lbugPath: '.gitnexus/lbug' })),
|
||||
getGlobalRegistryPath: vi.fn(() => 'registry.json'),
|
||||
RegistryNameCollisionError: class RegistryNameCollisionError extends Error {},
|
||||
AnalysisNotFinalizedError: class AnalysisNotFinalizedError extends Error {},
|
||||
assertAnalysisFinalized: vi.fn(async () => undefined),
|
||||
}));
|
||||
|
||||
vi.mock('../../src/storage/git.js', () => ({
|
||||
getGitRoot: vi.fn(() => '/repo'),
|
||||
hasGitDir: vi.fn(() => true),
|
||||
}));
|
||||
|
||||
vi.mock('../../src/core/ingestion/utils/max-file-size.js', () => ({
|
||||
getMaxFileSizeBannerMessage: vi.fn(() => null),
|
||||
}));
|
||||
|
||||
describe('analyzeCommand --embeddings [limit] parsing', () => {
|
||||
beforeEach(() => {
|
||||
vi.resetModules();
|
||||
runFullAnalysisMock.mockReset();
|
||||
runFullAnalysisMock.mockResolvedValue({
|
||||
repoName: 'repo',
|
||||
repoPath: '/repo',
|
||||
stats: {},
|
||||
alreadyUpToDate: true,
|
||||
});
|
||||
process.exitCode = undefined;
|
||||
process.env.NODE_OPTIONS = `${process.env.NODE_OPTIONS ?? ''} --max-old-space-size=8192`.trim();
|
||||
});
|
||||
|
||||
it.each(['abc', '-1', '1.5', 'NaN', 'Infinity'])(
|
||||
'rejects invalid --embeddings value %s before analysis starts',
|
||||
async (embeddings) => {
|
||||
const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => undefined);
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
|
||||
await analyzeCommand(undefined, { embeddings });
|
||||
|
||||
expect(process.exitCode).toBe(1);
|
||||
expect(runFullAnalysisMock).not.toHaveBeenCalled();
|
||||
const msg = errorSpy.mock.calls[0]?.[0] ?? '';
|
||||
expect(msg).toContain('--embeddings expects a non-negative integer');
|
||||
expect(msg).toContain(`got "${embeddings}"`);
|
||||
errorSpy.mockRestore();
|
||||
},
|
||||
);
|
||||
|
||||
it('bare --embeddings forwards undefined limit (default cap honored downstream)', async () => {
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
|
||||
await analyzeCommand(undefined, { embeddings: true });
|
||||
|
||||
expect(runFullAnalysisMock).toHaveBeenCalledTimes(1);
|
||||
const opts = runFullAnalysisMock.mock.calls[0][1];
|
||||
expect(opts.embeddings).toBe(true);
|
||||
expect(opts.embeddingsNodeLimit).toBeUndefined();
|
||||
});
|
||||
|
||||
it('--embeddings 0 forwards 0 (cap disabled downstream)', async () => {
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
|
||||
await analyzeCommand(undefined, { embeddings: '0' });
|
||||
|
||||
const opts = runFullAnalysisMock.mock.calls[0][1];
|
||||
expect(opts.embeddings).toBe(true);
|
||||
expect(opts.embeddingsNodeLimit).toBe(0);
|
||||
});
|
||||
|
||||
it('--embeddings <n> forwards a positive custom cap', async () => {
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
|
||||
await analyzeCommand(undefined, { embeddings: '100000' });
|
||||
|
||||
const opts = runFullAnalysisMock.mock.calls[0][1];
|
||||
expect(opts.embeddings).toBe(true);
|
||||
expect(opts.embeddingsNodeLimit).toBe(100_000);
|
||||
});
|
||||
|
||||
it('omitted --embeddings keeps embeddings off (boolean false, no limit)', async () => {
|
||||
const { analyzeCommand } = await import('../../src/cli/analyze.js');
|
||||
|
||||
await analyzeCommand(undefined, {});
|
||||
|
||||
const opts = runFullAnalysisMock.mock.calls[0][1];
|
||||
expect(opts.embeddings).toBe(false);
|
||||
expect(opts.embeddingsNodeLimit).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -4,7 +4,7 @@
|
||||
* Tests isGitRepo, getCurrentCommit, getGitRoot, and the newly added
|
||||
* hasGitDir helper introduced for issue #384 (indexing non-git folders).
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { describe, it, expect, vi } from 'vitest';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
import fs from 'fs';
|
||||
@@ -97,6 +97,26 @@ describe('getCurrentCommit', () => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// Regression: #1172 — without explicit stdio on execSync, Node forwards
|
||||
// the child's stderr to the parent process, printing "fatal: not a git
|
||||
// repository" to the user's terminal even though the error is caught.
|
||||
it('does not leak git stderr to process.stderr (#1172)', async () => {
|
||||
const { getCurrentCommit } = await import('../../src/storage/git.js');
|
||||
// git-init a dir without commits so `git rev-parse HEAD` fails with a
|
||||
// "fatal:" message — the exact class of error that leaked before the fix.
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-test-'));
|
||||
execSync('git init -q', { cwd: tmpDir, stdio: 'ignore' });
|
||||
const spy = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
|
||||
try {
|
||||
expect(getCurrentCommit(tmpDir)).toBe('');
|
||||
const stderrOutput = spy.mock.calls.map((c) => String(c[0])).join('');
|
||||
expect(stderrOutput).not.toContain('fatal');
|
||||
} finally {
|
||||
spy.mockRestore();
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── getGitRoot ───────────────────────────────────────────────────────────
|
||||
@@ -111,6 +131,21 @@ describe('getGitRoot', () => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
// Regression: #1172 -- mirrors the getCurrentCommit stderr test above.
|
||||
it('does not leak git stderr to process.stderr (#1172)', async () => {
|
||||
const { getGitRoot } = await import('../../src/storage/git.js');
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-test-'));
|
||||
const spy = vi.spyOn(process.stderr, 'write').mockImplementation(() => true);
|
||||
try {
|
||||
getGitRoot(tmpDir);
|
||||
const stderrOutput = spy.mock.calls.map((c) => String(c[0])).join('');
|
||||
expect(stderrOutput).not.toContain('fatal');
|
||||
} finally {
|
||||
spy.mockRestore();
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── getRemoteUrl ─────────────────────────────────────────────────────────
|
||||
@@ -180,3 +215,90 @@ describe('getRemoteUrl', () => {
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ─── getCanonicalRepoRoot (#1259) ────────────────────────────────────────
|
||||
//
|
||||
// Critical for the worktree-naming bug: when `gitnexus analyze` runs from a
|
||||
// linked worktree, deriving `repoName` from `path.basename(getGitRoot(cwd))`
|
||||
// uses the worktree's directory slug instead of the canonical repo's
|
||||
// basename. `getCanonicalRepoRoot` exists specifically to dereference
|
||||
// worktrees via `git rev-parse --git-common-dir`.
|
||||
|
||||
describe('getCanonicalRepoRoot', () => {
|
||||
it('returns null for a plain temp directory (not a git repo)', async () => {
|
||||
const { getCanonicalRepoRoot } = await import('../../src/storage/git.js');
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-canonical-'));
|
||||
try {
|
||||
expect(getCanonicalRepoRoot(tmpDir)).toBeNull();
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('returns null for a non-existent path', async () => {
|
||||
const { getCanonicalRepoRoot } = await import('../../src/storage/git.js');
|
||||
expect(getCanonicalRepoRoot('/tmp/__gitnexus_canonical_nonexistent__')).toBeNull();
|
||||
});
|
||||
|
||||
it('returns the repo root when called from a regular (non-worktree) checkout', async () => {
|
||||
const { getCanonicalRepoRoot } = await import('../../src/storage/git.js');
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-canonical-main-'));
|
||||
try {
|
||||
execSync('git init -q', { cwd: tmpDir });
|
||||
// Compare via `path.basename` instead of full-path string equality so
|
||||
// the test is robust to platform path-format quirks (Windows 8.3 short
|
||||
// names like `C:\Users\RUNNER~1\…` vs long form `C:\Users\runneradmin\…`,
|
||||
// macOS `/var/folders/… ↔ /private/var/folders/…`). The basename is the
|
||||
// only part that registry name derivation actually uses (#1259).
|
||||
const result = getCanonicalRepoRoot(tmpDir);
|
||||
expect(result).not.toBeNull();
|
||||
expect(path.basename(result!)).toBe(path.basename(tmpDir));
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('returns the CANONICAL repo root when called from inside a linked worktree (#1259)', async () => {
|
||||
const { getCanonicalRepoRoot, getGitRoot } = await import('../../src/storage/git.js');
|
||||
const repoDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-canonical-wt-'));
|
||||
try {
|
||||
execSync('git init -q', { cwd: repoDir });
|
||||
// `git worktree add` requires at least one commit on a real branch.
|
||||
execSync('git config user.email "test@example.com"', { cwd: repoDir });
|
||||
execSync('git config user.name "Test"', { cwd: repoDir });
|
||||
execSync('git commit --allow-empty -q -m "initial"', { cwd: repoDir });
|
||||
// Create a linked worktree on a new branch outside the main checkout.
|
||||
const worktreeDir = path.join(repoDir, 'wt-feature');
|
||||
execSync(`git worktree add -q -b feature "${worktreeDir}"`, { cwd: repoDir });
|
||||
|
||||
// Both calls go through the same git executable, so their path-format
|
||||
// output is guaranteed consistent — equality between them is the
|
||||
// stable cross-platform assertion. (Comparing against `realpathSync`
|
||||
// breaks on Windows where 8.3 short names and long names diverge.)
|
||||
const fromMain = getCanonicalRepoRoot(repoDir);
|
||||
const fromWorktree = getCanonicalRepoRoot(worktreeDir);
|
||||
|
||||
expect(fromMain).not.toBeNull();
|
||||
// From inside the worktree: canonical points BACK to the main repo's
|
||||
// shared `.git`. This is the regression-guard for #1259 — the
|
||||
// registry name derivation collapses across worktrees.
|
||||
expect(fromWorktree).toBe(fromMain);
|
||||
// Basename matches the canonical repo dir (NOT the worktree slug).
|
||||
expect(path.basename(fromWorktree!)).toBe(path.basename(repoDir));
|
||||
expect(path.basename(fromWorktree!)).not.toBe('wt-feature');
|
||||
// Sanity: getGitRoot returns the worktree-local root (existing
|
||||
// behavior unchanged). Compare basenames for the same path-format
|
||||
// reason as above.
|
||||
expect(path.basename(getGitRoot(worktreeDir)!)).toBe('wt-feature');
|
||||
} finally {
|
||||
// Best-effort cleanup; worktree teardown can leak open handles on
|
||||
// Windows so use force.
|
||||
try {
|
||||
execSync('git worktree remove -f wt-feature', { cwd: repoDir });
|
||||
} catch {
|
||||
// ignore — fall through to recursive rm
|
||||
}
|
||||
fs.rmSync(repoDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -64,6 +64,36 @@ repos:
|
||||
expect(config.matching.exclude_links_param_only_paths).toBe(false);
|
||||
});
|
||||
|
||||
it('defaults thrift detection to true', () => {
|
||||
const minimal = `
|
||||
version: 1
|
||||
name: test
|
||||
repos:
|
||||
app: my-app
|
||||
`;
|
||||
const config = parseGroupConfig(minimal);
|
||||
expect(config.detect.thrift).toBe(true);
|
||||
});
|
||||
|
||||
it('parses thrift manifest links', () => {
|
||||
const yaml = `
|
||||
version: 1
|
||||
name: test
|
||||
repos:
|
||||
gateway: gateway-repo
|
||||
orders: orders-repo
|
||||
links:
|
||||
- from: gateway
|
||||
to: orders
|
||||
type: thrift
|
||||
contract: billing.v1.OrderService/PlaceOrder
|
||||
role: consumer
|
||||
`;
|
||||
const config = parseGroupConfig(yaml);
|
||||
expect(config.links[0].type).toBe('thrift');
|
||||
expect(config.links[0].contract).toBe('billing.v1.OrderService/PlaceOrder');
|
||||
});
|
||||
|
||||
it('throws on missing required fields', () => {
|
||||
expect(() => parseGroupConfig('version: 1')).toThrow(/name.*required/i);
|
||||
expect(() => parseGroupConfig('name: test')).toThrow(/version.*required/i);
|
||||
|
||||
@@ -169,6 +169,90 @@ describe('ManifestExtractor', () => {
|
||||
expect(provider?.symbolUid).toBe('uid-correct-login');
|
||||
});
|
||||
|
||||
it('resolves grpc package-qualified service-only manifest by full service name', async () => {
|
||||
const links: GroupManifestLink[] = [
|
||||
{
|
||||
from: 'platform/orders',
|
||||
to: 'platform/auth',
|
||||
type: 'grpc',
|
||||
contract: 'auth.AuthService',
|
||||
role: 'consumer',
|
||||
},
|
||||
];
|
||||
|
||||
let seenServiceName: string | undefined;
|
||||
const dbExecutors = new Map<
|
||||
string,
|
||||
(cypher: string, params?: Record<string, unknown>) => Promise<Record<string, unknown>[]>
|
||||
>([
|
||||
[
|
||||
'platform/auth',
|
||||
async (_cypher, params) => {
|
||||
seenServiceName = params?.serviceName as string;
|
||||
if (params?.serviceName === 'auth.AuthService') {
|
||||
return [
|
||||
{
|
||||
uid: 'uid-auth-service',
|
||||
name: 'auth.AuthService',
|
||||
filePath: 'src/auth.proto',
|
||||
},
|
||||
];
|
||||
}
|
||||
return [];
|
||||
},
|
||||
],
|
||||
['platform/orders', async () => []],
|
||||
]);
|
||||
|
||||
const result = await extractor.extractFromManifest(links, dbExecutors);
|
||||
|
||||
expect(seenServiceName).toBe('auth.AuthService');
|
||||
const provider = result.contracts.find((c) => c.role === 'provider');
|
||||
expect(provider?.symbolUid).toBe('uid-auth-service');
|
||||
});
|
||||
|
||||
it('resolves thrift package-qualified service-only manifest by simple service name', async () => {
|
||||
const links: GroupManifestLink[] = [
|
||||
{
|
||||
from: 'gateway',
|
||||
to: 'orders',
|
||||
type: 'thrift',
|
||||
contract: 'billing.v1.OrderService',
|
||||
role: 'consumer',
|
||||
},
|
||||
];
|
||||
|
||||
let seenServiceName: string | undefined;
|
||||
const dbExecutors = new Map<
|
||||
string,
|
||||
(cypher: string, params?: Record<string, unknown>) => Promise<Record<string, unknown>[]>
|
||||
>([
|
||||
[
|
||||
'orders',
|
||||
async (_cypher, params) => {
|
||||
seenServiceName = params?.serviceName as string;
|
||||
if (params?.serviceName === 'OrderService') {
|
||||
return [
|
||||
{
|
||||
uid: 'uid-order-service',
|
||||
name: 'OrderService',
|
||||
filePath: 'idl/order.thrift',
|
||||
},
|
||||
];
|
||||
}
|
||||
return [];
|
||||
},
|
||||
],
|
||||
['gateway', async () => []],
|
||||
]);
|
||||
|
||||
const result = await extractor.extractFromManifest(links, dbExecutors);
|
||||
|
||||
expect(seenServiceName).toBe('OrderService');
|
||||
const provider = result.contracts.find((c) => c.role === 'provider');
|
||||
expect(provider?.symbolUid).toBe('uid-order-service');
|
||||
});
|
||||
|
||||
it('resolves lib manifest links by exact name only', async () => {
|
||||
const links: GroupManifestLink[] = [
|
||||
{
|
||||
@@ -578,6 +662,33 @@ describe('ManifestExtractor', () => {
|
||||
expect(lowerContractId).toBe(upperContractId);
|
||||
});
|
||||
|
||||
it('builds thrift manifest contracts with synthetic uids when unresolved', async () => {
|
||||
const extractor = new ManifestExtractor();
|
||||
const result = await extractor.extractFromManifest([
|
||||
{
|
||||
from: 'gateway',
|
||||
to: 'orders',
|
||||
type: 'thrift',
|
||||
contract: 'billing.v1.OrderService/PlaceOrder',
|
||||
role: 'consumer',
|
||||
},
|
||||
]);
|
||||
|
||||
expect(result.contracts).toHaveLength(2);
|
||||
expect(result.contracts.map((c) => c.contractId)).toEqual([
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
]);
|
||||
expect(result.crossLinks).toHaveLength(1);
|
||||
expect(result.crossLinks[0].type).toBe('thrift');
|
||||
expect(result.crossLinks[0].from.symbolUid).toBe(
|
||||
'manifest::gateway::thrift::billing.v1.OrderService/PlaceOrder',
|
||||
);
|
||||
expect(result.crossLinks[0].to.symbolUid).toBe(
|
||||
'manifest::orders::thrift::billing.v1.OrderService/PlaceOrder',
|
||||
);
|
||||
});
|
||||
|
||||
it('resolves custom manifest links by exact symbol name', async () => {
|
||||
const links: GroupManifestLink[] = [
|
||||
{
|
||||
|
||||
@@ -22,6 +22,16 @@ describe('normalizeContractId', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('lowercases thrift package and service while preserving method case', () => {
|
||||
expect(normalizeContractId('thrift::Billing.V1.OrderService/PlaceOrder')).toBe(
|
||||
'thrift::billing.v1.orderservice/PlaceOrder',
|
||||
);
|
||||
});
|
||||
|
||||
it('preserves case for malformed thrift id with leading slash', () => {
|
||||
expect(normalizeContractId('thrift::/PlaceOrder')).toBe('thrift::/PlaceOrder');
|
||||
});
|
||||
|
||||
it('preserves case for malformed gRPC id with leading slash (no full-string lowercasing)', () => {
|
||||
expect(normalizeContractId('grpc::/MyPkg/DoThing')).toBe('grpc::/MyPkg/DoThing');
|
||||
});
|
||||
@@ -219,6 +229,26 @@ function makeGrpcContract(
|
||||
};
|
||||
}
|
||||
|
||||
function makeThriftContract(
|
||||
id: string,
|
||||
role: 'provider' | 'consumer',
|
||||
repo: string,
|
||||
overrides: Partial<StoredContract> = {},
|
||||
): StoredContract {
|
||||
return {
|
||||
contractId: id,
|
||||
type: 'thrift',
|
||||
role,
|
||||
symbolUid: `uid-${repo}-${id}`,
|
||||
symbolRef: { filePath: `src/${repo}.ts`, name: `fn-${id}` },
|
||||
symbolName: `fn-${id}`,
|
||||
confidence: 0.9,
|
||||
meta: {},
|
||||
repo,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// buildProviderIndex
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -258,6 +288,18 @@ describe('runExactMatch — gRPC wildcard handling', () => {
|
||||
expect(unmatched).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('test_runExactMatch_skips_thrift_wildcard_contracts', () => {
|
||||
const contracts: StoredContract[] = [
|
||||
makeThriftContract('thrift::billing.v1.OrderService/*', 'consumer', 'frontend'),
|
||||
makeThriftContract('thrift::billing.v1.OrderService/*', 'provider', 'backend'),
|
||||
];
|
||||
|
||||
const { matched, unmatched } = runExactMatch(contracts);
|
||||
|
||||
expect(matched).toHaveLength(0);
|
||||
expect(unmatched).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('test_runExactMatch_does_not_skip_http_wildcards', () => {
|
||||
const contracts: StoredContract[] = [
|
||||
{
|
||||
@@ -402,6 +444,161 @@ describe('runWildcardMatch', () => {
|
||||
expect(matched).toHaveLength(1);
|
||||
expect(matched[0].contractId).toBe('grpc::com.example.UserService/*');
|
||||
});
|
||||
|
||||
it('matches thrift fully-qualified service wildcard to a thrift provider method', () => {
|
||||
const consumer = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/*',
|
||||
'consumer',
|
||||
'frontend',
|
||||
);
|
||||
const provider = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'backend',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([provider]);
|
||||
const { matched, remaining } = runWildcardMatch([consumer], providerIndex);
|
||||
|
||||
expect(matched).toHaveLength(1);
|
||||
expect(matched[0].type).toBe('thrift');
|
||||
expect(matched[0].from.repo).toBe('frontend');
|
||||
expect(matched[0].to.repo).toBe('backend');
|
||||
expect(remaining).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('matches bare thrift service wildcard to a package-qualified thrift provider', () => {
|
||||
const consumer = makeThriftContract('thrift::OrderService/*', 'consumer', 'frontend');
|
||||
const provider = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'backend',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([provider]);
|
||||
const { matched } = runWildcardMatch([consumer], providerIndex);
|
||||
|
||||
expect(matched).toHaveLength(1);
|
||||
expect(matched[0].contractId).toBe('thrift::OrderService/*');
|
||||
});
|
||||
|
||||
it('does not match bare thrift service wildcard when multiple package-qualified services match', () => {
|
||||
const consumer = makeThriftContract('thrift::OrderService/*', 'consumer', 'frontend');
|
||||
const billingProvider = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'billing',
|
||||
);
|
||||
const salesProvider = makeThriftContract(
|
||||
'thrift::sales.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'sales',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([billingProvider, salesProvider]);
|
||||
const { matched, remaining } = runWildcardMatch([consumer], providerIndex);
|
||||
|
||||
expect(matched).toHaveLength(0);
|
||||
expect(remaining).toEqual([consumer]);
|
||||
});
|
||||
|
||||
it('keeps fully-qualified thrift service wildcard matching when same bare service appears elsewhere', () => {
|
||||
const consumer = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/*',
|
||||
'consumer',
|
||||
'frontend',
|
||||
);
|
||||
const billingProvider = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'billing',
|
||||
);
|
||||
const salesProvider = makeThriftContract(
|
||||
'thrift::sales.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'sales',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([billingProvider, salesProvider]);
|
||||
const { matched, remaining } = runWildcardMatch([consumer], providerIndex);
|
||||
|
||||
expect(matched).toHaveLength(1);
|
||||
expect(matched[0].to.repo).toBe('billing');
|
||||
expect(remaining).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('matches bare thrift service method to a package-qualified thrift provider method', () => {
|
||||
const consumer = makeThriftContract('thrift::OrderService/PlaceOrder', 'consumer', 'frontend');
|
||||
const provider = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'backend',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([provider]);
|
||||
const { matched, unmatched } = runExactMatch([consumer, provider], providerIndex);
|
||||
|
||||
expect(matched).toHaveLength(1);
|
||||
expect(matched[0].type).toBe('thrift');
|
||||
expect(matched[0].matchType).toBe('exact');
|
||||
expect(matched[0].contractId).toBe('thrift::OrderService/PlaceOrder');
|
||||
expect(matched[0].from.repo).toBe('frontend');
|
||||
expect(matched[0].to.repo).toBe('backend');
|
||||
expect(unmatched).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('does not match bare thrift service method to a different provider method', () => {
|
||||
const consumer = makeThriftContract('thrift::OrderService/PlaceOrder', 'consumer', 'frontend');
|
||||
const provider = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/GetOrderStatus',
|
||||
'provider',
|
||||
'backend',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([provider]);
|
||||
const { matched, unmatched } = runExactMatch([consumer, provider], providerIndex);
|
||||
|
||||
expect(matched).toHaveLength(0);
|
||||
expect(unmatched).toEqual([consumer, provider]);
|
||||
});
|
||||
|
||||
it('does not match bare thrift service method when multiple package-qualified providers match', () => {
|
||||
const consumer = makeThriftContract('thrift::OrderService/PlaceOrder', 'consumer', 'frontend');
|
||||
const billingProvider = makeThriftContract(
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'billing',
|
||||
);
|
||||
const salesProvider = makeThriftContract(
|
||||
'thrift::sales.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'sales',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([salesProvider, billingProvider]);
|
||||
const { matched, unmatched } = runExactMatch(
|
||||
[consumer, salesProvider, billingProvider],
|
||||
providerIndex,
|
||||
);
|
||||
|
||||
expect(matched).toHaveLength(0);
|
||||
expect(unmatched).toEqual([consumer, salesProvider, billingProvider]);
|
||||
});
|
||||
|
||||
it('does not match a thrift wildcard to a gRPC provider', () => {
|
||||
const consumer = makeThriftContract('thrift::OrderService/*', 'consumer', 'frontend');
|
||||
const provider = makeGrpcContract(
|
||||
'grpc::billing.v1.OrderService/PlaceOrder',
|
||||
'provider',
|
||||
'backend',
|
||||
);
|
||||
|
||||
const providerIndex = buildProviderIndex([provider]);
|
||||
const { matched, remaining } = runWildcardMatch([consumer], providerIndex);
|
||||
|
||||
expect(matched).toHaveLength(0);
|
||||
expect(remaining).toEqual([consumer]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildNoisyContractFilter (via runExactMatch)', () => {
|
||||
|
||||
@@ -22,6 +22,7 @@ describe('syncGroup', () => {
|
||||
detect: {
|
||||
http: true,
|
||||
grpc: false,
|
||||
thrift: false,
|
||||
topics: false,
|
||||
shared_libs: false,
|
||||
embedding_fallback: false,
|
||||
@@ -229,9 +230,11 @@ describe('syncGroup', () => {
|
||||
detect: {
|
||||
http: true,
|
||||
grpc: false,
|
||||
thrift: false,
|
||||
topics: false,
|
||||
shared_libs: false,
|
||||
embedding_fallback: false,
|
||||
workspace_deps: false,
|
||||
},
|
||||
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
|
||||
};
|
||||
@@ -264,6 +267,359 @@ describe('syncGroup', () => {
|
||||
expect(result.crossLinks).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('runs thrift wildcard matching after exact matching and returns wildcard remaining', async () => {
|
||||
const config = makeConfig({ 'app/provider': 'provider-repo', 'app/consumer': 'consumer-repo' });
|
||||
const provider: StoredContract = {
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
symbolUid: 'uid-provider-place-order',
|
||||
symbolRef: { filePath: 'src/provider.ts', name: 'OrderService.PlaceOrder' },
|
||||
symbolName: 'OrderService.PlaceOrder',
|
||||
confidence: 0.9,
|
||||
meta: {},
|
||||
repo: 'app/provider',
|
||||
};
|
||||
const consumer: StoredContract = {
|
||||
contractId: 'thrift::OrderService/*',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
symbolUid: 'uid-consumer-order-service',
|
||||
symbolRef: { filePath: 'src/consumer.ts', name: 'OrderClient' },
|
||||
symbolName: 'OrderClient',
|
||||
confidence: 0.8,
|
||||
meta: {},
|
||||
repo: 'app/consumer',
|
||||
};
|
||||
|
||||
const result = await syncGroup(config, {
|
||||
extractorOverride: async () => [provider, consumer],
|
||||
skipWrite: true,
|
||||
});
|
||||
|
||||
expect(result.crossLinks).toHaveLength(1);
|
||||
expect(result.crossLinks[0].matchType).toBe('wildcard');
|
||||
expect(result.crossLinks[0].contractId).toBe('thrift::OrderService/*');
|
||||
expect(result.crossLinks[0].from.repo).toBe('app/consumer');
|
||||
expect(result.crossLinks[0].to.repo).toBe('app/provider');
|
||||
expect(result.unmatched).toEqual([provider]);
|
||||
});
|
||||
|
||||
it('keeps wildcard thrift links to multiple extracted IDL provider methods', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-wildcard-'));
|
||||
fs.mkdirSync(path.join(tmpDir, 'idl'), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, 'idl', 'order.thrift'),
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
OrderResponse GetOrder(1: string orderId)
|
||||
}`,
|
||||
);
|
||||
|
||||
try {
|
||||
const { ThriftExtractor } =
|
||||
await import('../../../src/core/group/extractors/thrift-extractor.js');
|
||||
const extractedProviders = (
|
||||
await new ThriftExtractor().extract(null, tmpDir, {
|
||||
id: 'provider-repo',
|
||||
path: 'app/provider',
|
||||
repoPath: tmpDir,
|
||||
storagePath: path.join(tmpDir, '.gitnexus'),
|
||||
})
|
||||
)
|
||||
.filter((c) => c.role === 'provider')
|
||||
.map(
|
||||
(c): StoredContract => ({
|
||||
...c,
|
||||
repo: 'app/provider',
|
||||
}),
|
||||
);
|
||||
|
||||
const consumer: StoredContract = {
|
||||
contractId: 'thrift::OrderService/*',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
symbolUid: 'manifest::app/consumer::thrift::OrderService/*',
|
||||
symbolRef: { filePath: 'group.yaml', name: 'OrderService' },
|
||||
symbolName: 'OrderService',
|
||||
confidence: 1,
|
||||
meta: {},
|
||||
repo: 'app/consumer',
|
||||
};
|
||||
|
||||
const result = await syncGroup(makeConfig({}), {
|
||||
extractorOverride: async () => [...extractedProviders, consumer],
|
||||
skipWrite: true,
|
||||
});
|
||||
|
||||
expect(result.crossLinks).toHaveLength(2);
|
||||
expect(result.crossLinks.map((cl) => cl.to.symbolRef.name).sort()).toEqual([
|
||||
'OrderService.GetOrder',
|
||||
'OrderService.PlaceOrder',
|
||||
]);
|
||||
expect(new Set(result.crossLinks.map((cl) => cl.to.symbolUid)).size).toBe(2);
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('matches weak thrift method consumers to namespace-qualified providers during sync', async () => {
|
||||
const config = makeConfig({ 'app/provider': 'provider-repo', 'app/consumer': 'consumer-repo' });
|
||||
const provider: StoredContract = {
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
symbolUid: 'uid-provider-place-order',
|
||||
symbolRef: { filePath: 'idl/order.thrift', name: 'OrderService.PlaceOrder' },
|
||||
symbolName: 'OrderService.PlaceOrder',
|
||||
confidence: 0.85,
|
||||
meta: {},
|
||||
repo: 'app/provider',
|
||||
};
|
||||
const consumer: StoredContract = {
|
||||
contractId: 'thrift::OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
symbolUid: 'uid-consumer-place-order',
|
||||
symbolRef: { filePath: 'src/BillingWorkflow.java', name: 'orderService.PlaceOrder' },
|
||||
symbolName: 'orderService.PlaceOrder',
|
||||
confidence: 0.45,
|
||||
meta: {},
|
||||
repo: 'app/consumer',
|
||||
};
|
||||
|
||||
const result = await syncGroup(config, {
|
||||
extractorOverride: async () => [provider, consumer],
|
||||
skipWrite: true,
|
||||
});
|
||||
|
||||
expect(result.crossLinks).toHaveLength(1);
|
||||
expect(result.crossLinks[0].matchType).toBe('exact');
|
||||
expect(result.crossLinks[0].contractId).toBe('thrift::OrderService/PlaceOrder');
|
||||
expect(result.crossLinks[0].from.repo).toBe('app/consumer');
|
||||
expect(result.crossLinks[0].to.repo).toBe('app/provider');
|
||||
expect(result.unmatched).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('keeps exact thrift links to extracted IDL and Java providers for same method', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-exact-'));
|
||||
fs.mkdirSync(path.join(tmpDir, 'idl'), { recursive: true });
|
||||
fs.mkdirSync(path.join(tmpDir, 'src', 'main', 'java', 'example'), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, 'idl', 'order.thrift'),
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, 'src', 'main', 'java', 'example', 'IfaceOrderHandler.java'),
|
||||
`package example;
|
||||
|
||||
class IfaceOrderHandler implements OrderService.Iface {
|
||||
public PlaceOrderResponse PlaceOrder(PlaceOrderRequest request) {
|
||||
return new PlaceOrderResponse();
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
try {
|
||||
const { ThriftExtractor } =
|
||||
await import('../../../src/core/group/extractors/thrift-extractor.js');
|
||||
const extractedProviders = (
|
||||
await new ThriftExtractor().extract(null, tmpDir, {
|
||||
id: 'provider-repo',
|
||||
path: 'app/provider',
|
||||
repoPath: tmpDir,
|
||||
storagePath: path.join(tmpDir, '.gitnexus'),
|
||||
})
|
||||
)
|
||||
.filter((c) => c.role === 'provider')
|
||||
.map(
|
||||
(c): StoredContract => ({
|
||||
...c,
|
||||
repo: 'app/provider',
|
||||
}),
|
||||
);
|
||||
|
||||
const consumer: StoredContract = {
|
||||
contractId: 'thrift::OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
symbolUid: [
|
||||
'source-scan::thrift',
|
||||
'consumer',
|
||||
'OrderService/PlaceOrder',
|
||||
'src/BillingWorkflow.java',
|
||||
'orderService.PlaceOrder',
|
||||
].join('::'),
|
||||
symbolRef: { filePath: 'src/BillingWorkflow.java', name: 'orderService.PlaceOrder' },
|
||||
symbolName: 'orderService.PlaceOrder',
|
||||
confidence: 0.45,
|
||||
meta: {},
|
||||
repo: 'app/consumer',
|
||||
};
|
||||
|
||||
const result = await syncGroup(makeConfig({}), {
|
||||
extractorOverride: async () => [...extractedProviders, consumer],
|
||||
skipWrite: true,
|
||||
});
|
||||
|
||||
expect(result.crossLinks).toHaveLength(2);
|
||||
expect(result.crossLinks.map((cl) => cl.to.symbolRef.filePath).sort()).toEqual([
|
||||
'idl/order.thrift',
|
||||
'src/main/java/example/IfaceOrderHandler.java',
|
||||
]);
|
||||
expect(new Set(result.crossLinks.map((cl) => cl.to.symbolUid)).size).toBe(2);
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('extracts thrift contracts during real sync when thrift detection is enabled', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-'));
|
||||
const storageDir = path.join(tmpDir, '.gitnexus');
|
||||
fs.mkdirSync(path.join(tmpDir, 'services', 'billing', 'idl'), { recursive: true });
|
||||
fs.mkdirSync(path.join(tmpDir, 'services', 'billing', 'src'), { recursive: true });
|
||||
fs.mkdirSync(storageDir, { recursive: true });
|
||||
fs.writeFileSync(path.join(tmpDir, 'services', 'billing', 'package.json'), '{}');
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, 'services', 'billing', 'src', 'BillingWorkflow.java'),
|
||||
'package example; class BillingWorkflow {}',
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, 'services', 'billing', 'idl', 'order.thrift'),
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
|
||||
const config = makeConfig({ 'services/billing': 'billing-repo' });
|
||||
config.detect.http = false;
|
||||
config.detect.thrift = true;
|
||||
|
||||
const poolAdapter = await import('../../../src/core/lbug/pool-adapter.js');
|
||||
const initSpy = vi.spyOn(poolAdapter, 'initLbug').mockResolvedValue(undefined);
|
||||
const closeSpy = vi.spyOn(poolAdapter, 'closeLbug').mockResolvedValue(undefined);
|
||||
|
||||
try {
|
||||
const result = await syncGroup(config, {
|
||||
resolveRepoHandle: async (_name, groupPath) => ({
|
||||
id: 'billing-repo',
|
||||
path: groupPath,
|
||||
repoPath: tmpDir,
|
||||
storagePath: storageDir,
|
||||
}),
|
||||
skipWrite: true,
|
||||
});
|
||||
|
||||
expect(result.missingRepos).toHaveLength(0);
|
||||
expect(result.contracts).toHaveLength(1);
|
||||
expect(result.contracts[0]).toMatchObject({
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
repo: 'services/billing',
|
||||
service: 'services/billing',
|
||||
symbolRef: {
|
||||
filePath: 'services/billing/idl/order.thrift',
|
||||
name: 'OrderService.PlaceOrder',
|
||||
},
|
||||
});
|
||||
expect(initSpy).toHaveBeenCalledWith('billing-repo', path.join(storageDir, 'lbug'));
|
||||
expect(closeSpy).toHaveBeenCalledWith('billing-repo');
|
||||
} finally {
|
||||
initSpy.mockRestore();
|
||||
closeSpy.mockRestore();
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('does not extract thrift contracts during real sync when thrift detection is disabled', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gitnexus-sync-thrift-off-'));
|
||||
const storageDir = path.join(tmpDir, '.gitnexus');
|
||||
fs.mkdirSync(path.join(tmpDir, 'services', 'billing', 'idl'), { recursive: true });
|
||||
fs.mkdirSync(storageDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, 'services', 'billing', 'idl', 'order.thrift'),
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
|
||||
const config = makeConfig({ 'services/billing': 'billing-repo' });
|
||||
config.detect.http = false;
|
||||
config.detect.thrift = false;
|
||||
|
||||
const poolAdapter = await import('../../../src/core/lbug/pool-adapter.js');
|
||||
const initSpy = vi.spyOn(poolAdapter, 'initLbug').mockResolvedValue(undefined);
|
||||
const closeSpy = vi.spyOn(poolAdapter, 'closeLbug').mockResolvedValue(undefined);
|
||||
|
||||
try {
|
||||
const result = await syncGroup(config, {
|
||||
resolveRepoHandle: async (_name, groupPath) => ({
|
||||
id: 'billing-repo',
|
||||
path: groupPath,
|
||||
repoPath: tmpDir,
|
||||
storagePath: storageDir,
|
||||
}),
|
||||
skipWrite: true,
|
||||
});
|
||||
|
||||
expect(result.missingRepos).toHaveLength(0);
|
||||
expect(result.contracts).toHaveLength(0);
|
||||
} finally {
|
||||
initSpy.mockRestore();
|
||||
closeSpy.mockRestore();
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('dedupes duplicate wildcard cross-links during sync', async () => {
|
||||
const config = makeConfig({ 'app/provider': 'provider-repo', 'app/consumer': 'consumer-repo' });
|
||||
const provider: StoredContract = {
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
symbolUid: 'uid-provider-place-order',
|
||||
symbolRef: { filePath: 'src/provider.ts', name: 'OrderService.PlaceOrder' },
|
||||
symbolName: 'OrderService.PlaceOrder',
|
||||
confidence: 0.9,
|
||||
meta: {},
|
||||
repo: 'app/provider',
|
||||
};
|
||||
const duplicateProvider: StoredContract = {
|
||||
...provider,
|
||||
confidence: 0.7,
|
||||
};
|
||||
const consumer: StoredContract = {
|
||||
contractId: 'thrift::OrderService/*',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
symbolUid: 'uid-consumer-order-service',
|
||||
symbolRef: { filePath: 'src/consumer.ts', name: 'OrderClient' },
|
||||
symbolName: 'OrderClient',
|
||||
confidence: 0.8,
|
||||
meta: {},
|
||||
repo: 'app/consumer',
|
||||
};
|
||||
|
||||
const result = await syncGroup(config, {
|
||||
extractorOverride: async () => [provider, duplicateProvider, consumer],
|
||||
skipWrite: true,
|
||||
});
|
||||
|
||||
expect(result.crossLinks).toHaveLength(1);
|
||||
expect(result.crossLinks[0].matchType).toBe('wildcard');
|
||||
});
|
||||
|
||||
it('manifest links referencing unknown repos still produce cross-links via synthetic UIDs', async () => {
|
||||
const links: GroupManifestLink[] = [
|
||||
{
|
||||
@@ -285,9 +641,11 @@ describe('syncGroup', () => {
|
||||
detect: {
|
||||
http: true,
|
||||
grpc: false,
|
||||
thrift: false,
|
||||
topics: false,
|
||||
shared_libs: false,
|
||||
embedding_fallback: false,
|
||||
workspace_deps: false,
|
||||
},
|
||||
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
|
||||
};
|
||||
@@ -350,6 +708,7 @@ describe('syncGroup', () => {
|
||||
detect: {
|
||||
http: false,
|
||||
grpc: false,
|
||||
thrift: false,
|
||||
topics: false,
|
||||
shared_libs: false,
|
||||
embedding_fallback: false,
|
||||
@@ -506,6 +865,7 @@ describe('syncGroup', () => {
|
||||
detect: {
|
||||
http: false,
|
||||
grpc: false,
|
||||
thrift: false,
|
||||
topics: false,
|
||||
shared_libs: false,
|
||||
embedding_fallback: false,
|
||||
|
||||
@@ -0,0 +1,651 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
||||
import * as fs from 'node:fs';
|
||||
import fsp from 'node:fs/promises';
|
||||
import * as path from 'node:path';
|
||||
import * as os from 'node:os';
|
||||
import {
|
||||
ThriftExtractor,
|
||||
buildThriftContext,
|
||||
thriftMethodContractId,
|
||||
thriftServiceContractId,
|
||||
} from '../../../src/core/group/extractors/thrift-extractor.js';
|
||||
import type { RepoHandle } from '../../../src/core/group/types.js';
|
||||
|
||||
describe('ThriftExtractor', () => {
|
||||
let tmpDir: string;
|
||||
let extractor: ThriftExtractor;
|
||||
|
||||
beforeEach(async () => {
|
||||
tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), 'gitnexus-thrift-'));
|
||||
extractor = new ThriftExtractor();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fsp.rm(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function writeFile(relPath: string, content: string): void {
|
||||
const full = path.join(tmpDir, relPath);
|
||||
fs.mkdirSync(path.dirname(full), { recursive: true });
|
||||
fs.writeFileSync(full, content);
|
||||
}
|
||||
|
||||
const makeRepo = (repoPath: string): RepoHandle => ({
|
||||
id: 'test-repo',
|
||||
path: 'test/app',
|
||||
repoPath,
|
||||
storagePath: path.join(repoPath, '.gitnexus'),
|
||||
});
|
||||
|
||||
it('test_extract_thrift_single_method_returns_idl_provider', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts).toHaveLength(1);
|
||||
expect(contracts[0]).toMatchObject({
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
symbolName: 'OrderService.PlaceOrder',
|
||||
confidence: 0.85,
|
||||
meta: {
|
||||
namespace: 'billing.v1',
|
||||
service: 'OrderService',
|
||||
method: 'PlaceOrder',
|
||||
source: 'thrift_idl',
|
||||
},
|
||||
});
|
||||
expect(contracts[0].symbolRef).toEqual({
|
||||
filePath: 'idl/order.thrift',
|
||||
name: 'OrderService.PlaceOrder',
|
||||
});
|
||||
});
|
||||
|
||||
it('test_extract_thrift_multiple_services_and_methods_returns_all', async () => {
|
||||
writeFile(
|
||||
'contracts/orders.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
OrderStatus GetOrderStatus(1: string orderId)
|
||||
}
|
||||
|
||||
service InvoiceService {
|
||||
Invoice CreateInvoice(1: string orderId)
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts.map((c) => c.contractId).sort()).toEqual([
|
||||
'thrift::billing.v1.InvoiceService/CreateInvoice',
|
||||
'thrift::billing.v1.OrderService/GetOrderStatus',
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
]);
|
||||
});
|
||||
|
||||
it('test_extract_thrift_prefers_java_namespace_over_other_namespaces', async () => {
|
||||
writeFile(
|
||||
'order.thrift',
|
||||
`namespace py billing_python.v1
|
||||
namespace java billing.v1
|
||||
namespace go billinggo
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts[0].contractId).toBe('thrift::billing.v1.OrderService/PlaceOrder');
|
||||
expect(contracts[0].meta.namespace).toBe('billing.v1');
|
||||
});
|
||||
|
||||
it('test_extract_thrift_uses_first_non_java_namespace_when_java_missing', async () => {
|
||||
writeFile(
|
||||
'order.thrift',
|
||||
`namespace py billing_python.v1
|
||||
namespace go billinggo
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts[0].contractId).toBe('thrift::billing_python.v1.OrderService/PlaceOrder');
|
||||
expect(contracts[0].meta.namespace).toBe('billing_python.v1');
|
||||
});
|
||||
|
||||
it('test_extract_thrift_without_namespace_uses_service_only', async () => {
|
||||
writeFile(
|
||||
'order.thrift',
|
||||
`service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts[0].contractId).toBe('thrift::OrderService/PlaceOrder');
|
||||
expect(contracts[0].meta.namespace).toBe('');
|
||||
});
|
||||
|
||||
it('test_extract_thrift_ignores_braces_inside_comments_and_strings', async () => {
|
||||
writeFile(
|
||||
'idl/tricky.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
// A comment with } should not close the service.
|
||||
/* A block comment with { and } should not affect depth. */
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
const string NOTE = "literal with } and { braces"
|
||||
OrderStatus GetOrderStatus(1: string orderId)
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts.map((c) => c.symbolName).sort()).toEqual([
|
||||
'OrderService.GetOrderStatus',
|
||||
'OrderService.PlaceOrder',
|
||||
]);
|
||||
});
|
||||
|
||||
it('test_extract_thrift_malformed_unclosed_service_is_skipped', async () => {
|
||||
writeFile(
|
||||
'idl/broken.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
`,
|
||||
);
|
||||
|
||||
await expect(extractor.extract(null, tmpDir, makeRepo(tmpDir))).resolves.toEqual([]);
|
||||
});
|
||||
|
||||
it('test_extract_repo_without_thrift_returns_empty', async () => {
|
||||
writeFile('src/index.ts', 'console.log("hello")');
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts).toEqual([]);
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_consumers_from_iface_client_and_service_fields', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/BillingWorkflow.java',
|
||||
`package example;
|
||||
|
||||
class BillingWorkflow {
|
||||
private OrderService.Iface orderService;
|
||||
private OrderService.Client orderClient;
|
||||
private OrderService generatedOrderService;
|
||||
|
||||
void submit(PlaceOrderRequest request) throws Exception {
|
||||
orderService.PlaceOrder(request);
|
||||
orderClient.PlaceOrder(request);
|
||||
generatedOrderService.PlaceOrder(request);
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const consumers = contracts
|
||||
.filter((c) => c.role === 'consumer')
|
||||
.sort((a, b) => a.symbolName.localeCompare(b.symbolName));
|
||||
|
||||
expect(consumers).toHaveLength(3);
|
||||
expect(consumers.map((c) => c.symbolName)).toEqual([
|
||||
'generatedOrderService.PlaceOrder',
|
||||
'orderClient.PlaceOrder',
|
||||
'orderService.PlaceOrder',
|
||||
]);
|
||||
for (const contract of consumers) {
|
||||
expect(contract).toMatchObject({
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
confidence: 0.75,
|
||||
meta: {
|
||||
namespace: 'billing.v1',
|
||||
service: 'OrderService',
|
||||
method: 'PlaceOrder',
|
||||
source: 'java_thrift_consumer',
|
||||
},
|
||||
});
|
||||
expect(contract.symbolRef.filePath).toBe('src/main/java/example/BillingWorkflow.java');
|
||||
}
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_consumers_from_this_field_access', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/BillingWorkflow.java',
|
||||
`package example;
|
||||
|
||||
class BillingWorkflow {
|
||||
private OrderService.Client orderClient;
|
||||
|
||||
void submit(PlaceOrderRequest request) throws Exception {
|
||||
this.orderClient.PlaceOrder(request);
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const consumers = contracts.filter((c) => c.role === 'consumer');
|
||||
|
||||
expect(consumers).toHaveLength(1);
|
||||
expect(consumers[0]).toMatchObject({
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
symbolName: 'orderClient.PlaceOrder',
|
||||
confidence: 0.75,
|
||||
meta: {
|
||||
namespace: 'billing.v1',
|
||||
service: 'OrderService',
|
||||
method: 'PlaceOrder',
|
||||
source: 'java_thrift_consumer',
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_consumers_from_fully_qualified_generated_types', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/BillingWorkflow.java',
|
||||
`package example;
|
||||
|
||||
class BillingWorkflow {
|
||||
private billing.v1.OrderService.Iface orderService;
|
||||
private billing.v1.OrderService.Client orderClient;
|
||||
|
||||
void submit(PlaceOrderRequest request) throws Exception {
|
||||
orderService.PlaceOrder(request);
|
||||
orderClient.PlaceOrder(request);
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const consumers = contracts
|
||||
.filter((c) => c.role === 'consumer')
|
||||
.sort((a, b) => a.symbolName.localeCompare(b.symbolName));
|
||||
|
||||
expect(consumers).toHaveLength(2);
|
||||
expect(consumers.map((c) => c.symbolName)).toEqual([
|
||||
'orderClient.PlaceOrder',
|
||||
'orderService.PlaceOrder',
|
||||
]);
|
||||
expect(new Set(consumers.map((c) => c.contractId))).toEqual(
|
||||
new Set(['thrift::billing.v1.OrderService/PlaceOrder']),
|
||||
);
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_consumers_from_local_variables', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/BillingWorker.java',
|
||||
`package example;
|
||||
|
||||
class BillingWorker {
|
||||
void submit(OrderService.Iface iface, OrderService.Client client, OrderService service) throws Exception {
|
||||
OrderService.Iface orderService = iface;
|
||||
OrderService.Client orderClient = client;
|
||||
OrderService generatedOrderService = service;
|
||||
|
||||
orderService.PlaceOrder(new PlaceOrderRequest());
|
||||
orderClient.PlaceOrder(new PlaceOrderRequest());
|
||||
generatedOrderService.PlaceOrder(new PlaceOrderRequest());
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const consumers = contracts.filter((c) => c.role === 'consumer');
|
||||
|
||||
expect(consumers.map((c) => c.symbolName).sort()).toEqual([
|
||||
'generatedOrderService.PlaceOrder',
|
||||
'orderClient.PlaceOrder',
|
||||
'orderService.PlaceOrder',
|
||||
]);
|
||||
expect(new Set(consumers.map((c) => c.contractId))).toEqual(
|
||||
new Set(['thrift::billing.v1.OrderService/PlaceOrder']),
|
||||
);
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_consumers_resolve_receiver_by_nearest_scope', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}
|
||||
|
||||
service InvoiceService {
|
||||
Invoice CreateInvoice(1: string orderId)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/BillingWorker.java',
|
||||
`package example;
|
||||
|
||||
class BillingWorker {
|
||||
void submitOrder(OrderService.Iface client, PlaceOrderRequest request) throws Exception {
|
||||
client.PlaceOrder(request);
|
||||
}
|
||||
|
||||
void submitInvoice() throws Exception {
|
||||
InvoiceService.Client client = new InvoiceService.Client(null);
|
||||
client.CreateInvoice("order-1");
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const consumers = contracts
|
||||
.filter((c) => c.role === 'consumer')
|
||||
.sort((a, b) => a.contractId.localeCompare(b.contractId));
|
||||
|
||||
expect(consumers.map((c) => c.contractId)).toEqual([
|
||||
'thrift::billing.v1.InvoiceService/CreateInvoice',
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
]);
|
||||
expect(consumers.map((c) => c.symbolName).sort()).toEqual([
|
||||
'client.CreateInvoice',
|
||||
'client.PlaceOrder',
|
||||
]);
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_providers_from_iface_and_service_implements', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/IfaceOrderHandler.java',
|
||||
`package example;
|
||||
|
||||
class IfaceOrderHandler implements OrderService.Iface {
|
||||
public PlaceOrderResponse PlaceOrder(PlaceOrderRequest request) {
|
||||
return new PlaceOrderResponse();
|
||||
}
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/GeneratedOrderHandler.java',
|
||||
`package example;
|
||||
|
||||
class GeneratedOrderHandler implements OrderService {
|
||||
public PlaceOrderResponse PlaceOrder(PlaceOrderRequest request) {
|
||||
return new PlaceOrderResponse();
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const providers = contracts
|
||||
.filter((c) => c.meta.source === 'java_thrift_provider')
|
||||
.sort((a, b) => a.symbolRef.filePath.localeCompare(b.symbolRef.filePath));
|
||||
|
||||
expect(providers).toHaveLength(2);
|
||||
for (const contract of providers) {
|
||||
expect(contract).toMatchObject({
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
symbolName: 'OrderService.PlaceOrder',
|
||||
confidence: 0.8,
|
||||
meta: {
|
||||
namespace: 'billing.v1',
|
||||
service: 'OrderService',
|
||||
method: 'PlaceOrder',
|
||||
source: 'java_thrift_provider',
|
||||
},
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('test_extract_thrift_source_scan_contracts_have_stable_distinct_symbol_uids', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/IfaceOrderHandler.java',
|
||||
`package example;
|
||||
|
||||
class IfaceOrderHandler implements OrderService.Iface {
|
||||
public PlaceOrderResponse PlaceOrder(PlaceOrderRequest request) {
|
||||
return new PlaceOrderResponse();
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const first = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const second = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const providers = first
|
||||
.filter((c) => c.role === 'provider')
|
||||
.sort((a, b) => a.symbolRef.filePath.localeCompare(b.symbolRef.filePath));
|
||||
const repeatedProviders = second
|
||||
.filter((c) => c.role === 'provider')
|
||||
.sort((a, b) => a.symbolRef.filePath.localeCompare(b.symbolRef.filePath));
|
||||
|
||||
expect(providers).toHaveLength(2);
|
||||
expect(providers.map((c) => c.symbolUid)).toEqual(repeatedProviders.map((c) => c.symbolUid));
|
||||
expect(providers.every((c) => c.symbolUid.length > 0)).toBe(true);
|
||||
expect(new Set(providers.map((c) => c.symbolUid)).size).toBe(2);
|
||||
expect(providers.every((c) => !c.symbolUid.includes('::thrift::billing.v1'))).toBe(true);
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_providers_from_fully_qualified_generated_iface', async () => {
|
||||
writeFile(
|
||||
'idl/order.thrift',
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
}`,
|
||||
);
|
||||
writeFile(
|
||||
'src/main/java/example/IfaceOrderHandler.java',
|
||||
`package example;
|
||||
|
||||
class IfaceOrderHandler implements billing.v1.OrderService.Iface {
|
||||
public PlaceOrderResponse PlaceOrder(PlaceOrderRequest request) {
|
||||
return new PlaceOrderResponse();
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
const providers = contracts.filter((c) => c.meta.source === 'java_thrift_provider');
|
||||
|
||||
expect(providers).toHaveLength(1);
|
||||
expect(providers[0]).toMatchObject({
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
symbolName: 'OrderService.PlaceOrder',
|
||||
confidence: 0.8,
|
||||
meta: {
|
||||
namespace: 'billing.v1',
|
||||
service: 'OrderService',
|
||||
method: 'PlaceOrder',
|
||||
source: 'java_thrift_provider',
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_consumer_without_idl_emits_weak_method_contract', async () => {
|
||||
writeFile(
|
||||
'src/main/java/example/BillingWorkflow.java',
|
||||
`package example;
|
||||
|
||||
class BillingWorkflow {
|
||||
private OrderService.Iface orderService;
|
||||
|
||||
void submit(PlaceOrderRequest request) throws Exception {
|
||||
orderService.PlaceOrder(request);
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts).toHaveLength(1);
|
||||
expect(contracts[0]).toMatchObject({
|
||||
contractId: 'thrift::OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'consumer',
|
||||
symbolName: 'orderService.PlaceOrder',
|
||||
confidence: 0.45,
|
||||
meta: {
|
||||
service: 'OrderService',
|
||||
method: 'PlaceOrder',
|
||||
source: 'java_thrift_consumer_weak',
|
||||
},
|
||||
});
|
||||
expect(contracts[0].symbolRef.filePath).toBe('src/main/java/example/BillingWorkflow.java');
|
||||
});
|
||||
|
||||
it('test_extract_java_thrift_direct_service_consumer_without_idl_returns_empty', async () => {
|
||||
writeFile(
|
||||
'src/main/java/example/PaymentWorkflow.java',
|
||||
`package example;
|
||||
|
||||
class PaymentWorkflow {
|
||||
private PaymentService paymentService;
|
||||
|
||||
void submit() {
|
||||
paymentService.charge();
|
||||
}
|
||||
}`,
|
||||
);
|
||||
|
||||
const contracts = await extractor.extract(null, tmpDir, makeRepo(tmpDir));
|
||||
|
||||
expect(contracts).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildThriftContext', () => {
|
||||
let tmpDir: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), 'gitnexus-thrift-context-'));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await fsp.rm(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('test_buildThriftContext_parses_namespace_service_methods_and_path', async () => {
|
||||
await fsp.mkdir(path.join(tmpDir, 'idl'), { recursive: true });
|
||||
await fsp.writeFile(
|
||||
path.join(tmpDir, 'idl', 'order.thrift'),
|
||||
`namespace java billing.v1
|
||||
|
||||
service OrderService {
|
||||
PlaceOrderResponse PlaceOrder(1: PlaceOrderRequest request)
|
||||
OrderStatus GetOrderStatus(1: string orderId)
|
||||
}`,
|
||||
);
|
||||
|
||||
const context = await buildThriftContext(tmpDir);
|
||||
|
||||
expect(context.namespacesByThrift.get('idl/order.thrift')).toBe('billing.v1');
|
||||
expect(context.servicesByName.get('OrderService')).toEqual([
|
||||
{
|
||||
namespace: 'billing.v1',
|
||||
serviceName: 'OrderService',
|
||||
methods: ['PlaceOrder', 'GetOrderStatus'],
|
||||
thriftPath: 'idl/order.thrift',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('test_buildThriftContext_without_files_returns_empty_maps', async () => {
|
||||
const context = await buildThriftContext(tmpDir);
|
||||
|
||||
expect(context.namespacesByThrift.size).toBe(0);
|
||||
expect(context.servicesByName.size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Thrift contract id helpers', () => {
|
||||
it('test_thriftMethodContractId_with_namespace', () => {
|
||||
expect(thriftMethodContractId('billing.v1', 'OrderService', 'PlaceOrder')).toBe(
|
||||
'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
);
|
||||
});
|
||||
|
||||
it('test_thriftMethodContractId_without_namespace', () => {
|
||||
expect(thriftMethodContractId('', 'OrderService', 'PlaceOrder')).toBe(
|
||||
'thrift::OrderService/PlaceOrder',
|
||||
);
|
||||
});
|
||||
|
||||
it('test_thriftServiceContractId_with_namespace', () => {
|
||||
expect(thriftServiceContractId('billing.v1', 'OrderService')).toBe(
|
||||
'thrift::billing.v1.OrderService/*',
|
||||
);
|
||||
});
|
||||
|
||||
it('test_thriftServiceContractId_without_namespace', () => {
|
||||
expect(thriftServiceContractId('', 'OrderService')).toBe('thrift::OrderService/*');
|
||||
});
|
||||
});
|
||||
@@ -21,6 +21,7 @@ describe('Group types', () => {
|
||||
detect: {
|
||||
http: true,
|
||||
grpc: true,
|
||||
thrift: true,
|
||||
topics: true,
|
||||
shared_libs: true,
|
||||
embedding_fallback: true,
|
||||
@@ -63,6 +64,41 @@ describe('Group types', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('ExtractedContract accepts thrift contract type', () => {
|
||||
const contract: ExtractedContract = {
|
||||
contractId: 'thrift::billing.v1.OrderService/PlaceOrder',
|
||||
type: 'thrift',
|
||||
role: 'provider',
|
||||
symbolUid: 'uid-thrift',
|
||||
symbolRef: { filePath: 'idl/order.thrift', name: 'OrderService.PlaceOrder' },
|
||||
symbolName: 'OrderService.PlaceOrder',
|
||||
confidence: 0.9,
|
||||
meta: {},
|
||||
};
|
||||
expect(contract.type).toBe('thrift');
|
||||
});
|
||||
|
||||
it('DetectConfig includes thrift toggle', () => {
|
||||
const config: GroupConfig = {
|
||||
version: 1,
|
||||
name: 'company',
|
||||
description: 'All company microservices',
|
||||
repos: { orders: 'orders-repo' },
|
||||
links: [],
|
||||
packages: {},
|
||||
detect: {
|
||||
http: true,
|
||||
grpc: true,
|
||||
thrift: true,
|
||||
topics: true,
|
||||
shared_libs: true,
|
||||
embedding_fallback: true,
|
||||
},
|
||||
matching: { bm25_threshold: 0.7, embedding_threshold: 0.65, max_candidates_per_step: 3 },
|
||||
};
|
||||
expect(config.detect.thrift).toBe(true);
|
||||
});
|
||||
|
||||
it('CrossLink stores match metadata', () => {
|
||||
const link: CrossLink = {
|
||||
from: {
|
||||
|
||||
@@ -0,0 +1,289 @@
|
||||
/**
|
||||
* Tests for createRouteLimiter and the integration shape used by api.ts.
|
||||
*
|
||||
* Closes the U4 test gap (CodeQL js/missing-rate-limiting). Without these,
|
||||
* a refactor that drops the limiter middleware from any route would silently
|
||||
* regress and CodeQL would re-fire — but no test would fail before reaching
|
||||
* CI.
|
||||
*
|
||||
* Two layers of coverage:
|
||||
* 1. Helper unit tests — createRouteLimiter returns distinct middleware
|
||||
* per call, has the right signature, exposes the right error shape.
|
||||
* 2. Integration tests — mount the same factory on a tiny isolated express
|
||||
* app that does fs.readFile (the exact CodeQL sink class) and prove the
|
||||
* 429 fires after the configured limit. windowMs (2 000 ms) is generous
|
||||
* enough that 4 sequential requests fit inside one window even on slow
|
||||
* Windows CI runners; each test uses a fresh limiter so counter state
|
||||
* never carries between tests.
|
||||
*/
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it } from 'vitest';
|
||||
import express, { type Express } from 'express';
|
||||
import http from 'node:http';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import { createRouteLimiter } from '../../src/server/validation.js';
|
||||
|
||||
let tmpFile: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Real fs.readFile target so the route does the same kind of FS work
|
||||
// the production routes do — keeps the test honest about what it covers.
|
||||
tmpFile = path.join(
|
||||
await fs.mkdtemp(path.join(os.tmpdir(), 'gitnexus-ratelimit-')),
|
||||
'fixture.txt',
|
||||
);
|
||||
await fs.writeFile(tmpFile, 'hello\n', 'utf-8');
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await fs.rm(path.dirname(tmpFile), { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// Build a fresh app + server per test so counter state never carries between
|
||||
// tests. windowMs = 2 000 ms gives ample headroom for Windows CI where
|
||||
// sequential loopback HTTP requests can take 50–80 ms each.
|
||||
const buildApp = (limit: number, windowMs = 2000): Express => {
|
||||
const app = express();
|
||||
app.set('trust proxy', 'loopback, linklocal, uniquelocal');
|
||||
app.get('/test/file', createRouteLimiter({ windowMs, limit }), async (_req, res) => {
|
||||
const content = await fs.readFile(tmpFile, 'utf-8');
|
||||
res.json({ content });
|
||||
});
|
||||
return app;
|
||||
};
|
||||
|
||||
const startServer = (app: Express): Promise<{ server: http.Server; baseUrl: string }> =>
|
||||
new Promise((resolve) => {
|
||||
const server = app.listen(0, '127.0.0.1', () => {
|
||||
const addr = server.address();
|
||||
const baseUrl = typeof addr === 'object' && addr ? `http://127.0.0.1:${addr.port}` : '';
|
||||
resolve({ server, baseUrl });
|
||||
});
|
||||
});
|
||||
|
||||
const stopServer = (server: http.Server): Promise<void> =>
|
||||
new Promise((resolve) => server.close(() => resolve()));
|
||||
|
||||
describe('createRouteLimiter — defaults', () => {
|
||||
it('returns a different middleware instance per call (independent counters)', () => {
|
||||
const a = createRouteLimiter();
|
||||
const b = createRouteLimiter();
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
|
||||
it('produces a callable express RequestHandler', () => {
|
||||
const limiter = createRouteLimiter();
|
||||
expect(typeof limiter).toBe('function');
|
||||
// express middleware signature is (req, res, next) — 3 args.
|
||||
expect(limiter.length).toBe(3);
|
||||
});
|
||||
|
||||
// Regression guard for #1360 — createRouteLimiter must not throw
|
||||
// ERR_ERL_KEY_GEN_IPV6. The validation fires at construction time
|
||||
// (inside `rateLimit()`), so a simple `createRouteLimiter()` call is
|
||||
// the canary: if the keyGenerator references `req.ip` without using
|
||||
// `ipKeyGenerator`, the `rateLimit()` constructor throws before the
|
||||
// middleware is ever invoked.
|
||||
it('does not throw ERR_ERL_KEY_GEN_IPV6 on construction (#1360)', () => {
|
||||
expect(() => createRouteLimiter()).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('createRouteLimiter — integration with a real route', () => {
|
||||
let server: http.Server;
|
||||
let baseUrl: string;
|
||||
|
||||
beforeEach(async () => {
|
||||
({ server, baseUrl } = await startServer(buildApp(3)));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await stopServer(server);
|
||||
});
|
||||
|
||||
// The exact regression guard CodeQL would re-fire if a maintainer
|
||||
// dropped createRouteLimiter from any of the 4 protected routes:
|
||||
// without the limiter, max+1 requests all return 200.
|
||||
it('lets max requests through and rejects the next one with 429', async () => {
|
||||
for (let i = 1; i <= 3; i++) {
|
||||
const res = await fetch(`${baseUrl}/test/file`);
|
||||
expect(res.status).toBe(200);
|
||||
}
|
||||
const res = await fetch(`${baseUrl}/test/file`);
|
||||
expect(res.status).toBe(429);
|
||||
const body = await res.json();
|
||||
expect(body.error).toContain('Too many');
|
||||
});
|
||||
|
||||
it('emits draft-7 RateLimit response header (combined form), not legacy X-RateLimit-*', async () => {
|
||||
const res = await fetch(`${baseUrl}/test/file`);
|
||||
expect(res.status).toBe(200);
|
||||
// draft-7: single combined `RateLimit` header in `limit=N, remaining=N, reset=N` shape,
|
||||
// NO individual `X-RateLimit-*` legacy keys.
|
||||
const rateLimitHeader = res.headers.get('ratelimit');
|
||||
expect(rateLimitHeader).toMatch(/limit=\d+/);
|
||||
expect(rateLimitHeader).toMatch(/remaining=\d+/);
|
||||
expect(rateLimitHeader).toMatch(/reset=\d+/);
|
||||
expect(res.headers.get('x-ratelimit-limit')).toBeNull();
|
||||
});
|
||||
|
||||
it('429 response body uses the project { error } JSON shape', async () => {
|
||||
// Trip the limiter.
|
||||
for (let i = 1; i <= 3; i++) await fetch(`${baseUrl}/test/file`);
|
||||
const res = await fetch(`${baseUrl}/test/file`);
|
||||
expect(res.status).toBe(429);
|
||||
const body = await res.json();
|
||||
expect(body).toEqual({ error: expect.stringContaining('Too many') });
|
||||
});
|
||||
|
||||
it('429 response includes a Retry-After header so clients can back off', async () => {
|
||||
for (let i = 1; i <= 3; i++) await fetch(`${baseUrl}/test/file`);
|
||||
const res = await fetch(`${baseUrl}/test/file`);
|
||||
expect(res.status).toBe(429);
|
||||
const retryAfter = res.headers.get('retry-after');
|
||||
expect(retryAfter).toBeTruthy();
|
||||
// express-rate-limit v8 emits Retry-After in integer-seconds form. The
|
||||
// RFC also allows HTTP-date, but ERL does not use that shape; if a
|
||||
// future version switches, this assertion needs an HTTP-date branch.
|
||||
const seconds = Number(retryAfter);
|
||||
expect(Number.isFinite(seconds) && seconds >= 0).toBe(true);
|
||||
});
|
||||
|
||||
it('window resets after windowMs — counter does not carry across windows', async () => {
|
||||
// Trip the limiter.
|
||||
for (let i = 1; i <= 3; i++) await fetch(`${baseUrl}/test/file`);
|
||||
const tripped = await fetch(`${baseUrl}/test/file`);
|
||||
expect(tripped.status).toBe(429);
|
||||
// Wait for the window to roll over (2 000 ms window + 200 ms margin).
|
||||
await new Promise((r) => setTimeout(r, 2200));
|
||||
const reset = await fetch(`${baseUrl}/test/file`);
|
||||
expect(reset.status).toBe(200);
|
||||
});
|
||||
});
|
||||
|
||||
// Behavioral pin replacing the prior `expect(DEFAULT_RATE_LIMIT_RPM).toBe(60)`
|
||||
// constant assertion — that test pinned the magic number, this test pins the
|
||||
// observable contract that the production default does not 429 at typical
|
||||
// interactive load.
|
||||
describe('createRouteLimiter — production default', () => {
|
||||
it('default policy permits 60 requests in a minute (no opts override)', async () => {
|
||||
// Build an app that uses the production-default limiter (no opts override).
|
||||
// 60 requests is well under the default 60 rpm/IP, so all should pass.
|
||||
// Going to 61 would 429 but takes the full window to test deterministically;
|
||||
// the contract we want pinned here is "default does not throttle interactive
|
||||
// use" — the 429 path is already covered by the integration tests above.
|
||||
const { server, baseUrl } = await startServer(
|
||||
(() => {
|
||||
const app = express();
|
||||
app.set('trust proxy', 'loopback, linklocal, uniquelocal');
|
||||
app.get('/test/file', createRouteLimiter(), async (_req, res) => {
|
||||
const content = await fs.readFile(tmpFile, 'utf-8');
|
||||
res.json({ content });
|
||||
});
|
||||
return app;
|
||||
})(),
|
||||
);
|
||||
try {
|
||||
// Send 60 requests — all should succeed under the default policy.
|
||||
for (let i = 1; i <= 60; i++) {
|
||||
const res = await fetch(`${baseUrl}/test/file`);
|
||||
if (res.status !== 200) {
|
||||
throw new Error(`request ${i}/60 returned ${res.status} under default policy`);
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
await stopServer(server);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// Production-wiring assertions — proves each of the 4 protected routes in
|
||||
// api.ts actually has rate-limit middleware. Closes the gap reviewers flagged
|
||||
// where a maintainer could drop createRouteLimiter from a route and no test
|
||||
// would fail (only CodeQL would re-fire next scan).
|
||||
//
|
||||
// Walks the express router stack on a real createServer-built app, finds
|
||||
// each protected route by method+path, and asserts the middleware chain
|
||||
// includes the express-rate-limit handler. This is intentionally a
|
||||
// structural check (not behavioral) — the behavioral guarantees are
|
||||
// covered by the integration tests above.
|
||||
describe('production routes — rate-limit middleware wiring', () => {
|
||||
// Small structural check that does not require booting the full server
|
||||
// (which depends on LadybugDB, MCP transport, fork(), etc.). We grep the
|
||||
// api.ts source for the createRouteLimiter call adjacent to each route
|
||||
// registration. If a future refactor drops the call, the regex no longer
|
||||
// matches and the test fails.
|
||||
//
|
||||
// This is admittedly a light-weight check, but it is enough to catch the
|
||||
// single most likely regression (someone removes the middleware while
|
||||
// editing the route handler) without dragging in the full server boot.
|
||||
|
||||
let apiSource: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
apiSource = await fs.readFile(
|
||||
path.join(__dirname, '..', '..', 'src', 'server', 'api.ts'),
|
||||
'utf-8',
|
||||
);
|
||||
});
|
||||
|
||||
it('GET /api/file is wired with createRouteLimiter', () => {
|
||||
expect(apiSource).toMatch(/app\.get\('\/api\/file',\s*createRouteLimiter\(/);
|
||||
});
|
||||
|
||||
it('GET /api/grep is wired with createRouteLimiter', () => {
|
||||
expect(apiSource).toMatch(/app\.get\('\/api\/grep',\s*createRouteLimiter\(/);
|
||||
});
|
||||
|
||||
it('DELETE /api/repo is wired with createRouteLimiter', () => {
|
||||
expect(apiSource).toMatch(/app\.delete\('\/api\/repo',\s*createRouteLimiter\(/);
|
||||
});
|
||||
|
||||
it('POST /api/analyze is wired with createRouteLimiter', () => {
|
||||
expect(apiSource).toMatch(/app\.post\('\/api\/analyze',\s*createRouteLimiter\(/);
|
||||
});
|
||||
|
||||
it('POST /api/embed is wired with createRouteLimiter', () => {
|
||||
expect(apiSource).toMatch(/app\.post\('\/api\/embed',\s*createRouteLimiter\(/);
|
||||
});
|
||||
|
||||
it('SPA fallback is wired with createRouteLimiter', () => {
|
||||
expect(apiSource).toMatch(/app\.get\(SPA_FALLBACK_REGEX,\s*createRouteLimiter\(/);
|
||||
});
|
||||
|
||||
it('GET /api/health is registered (Docker healthcheck, #1147)', () => {
|
||||
expect(apiSource).toMatch(/app\.get\('\/api\/health',\s*\(_req,\s*res\)\s*=>/);
|
||||
});
|
||||
|
||||
it('createServer wires trust proxy to loopback/linklocal/uniquelocal', () => {
|
||||
expect(apiSource).toMatch(
|
||||
/app\.set\(\s*'trust proxy'\s*,\s*'loopback,\s*linklocal,\s*uniquelocal'\s*\)/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// Structural guard for #1360 — validates that the validation module uses
|
||||
// `ipKeyGenerator` so IPv6 addresses are normalised to their /56 subnet.
|
||||
// Without this, each IPv6 address gets an independent counter and the
|
||||
// rate-limit is trivially bypassed. The construction-time test above
|
||||
// catches the same regression behaviourally; this source-grep test catches
|
||||
// it structurally so the failure message is immediately obvious.
|
||||
describe('validation.ts — IPv6 key normalisation (#1360)', () => {
|
||||
let validationSource: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
validationSource = await fs.readFile(
|
||||
path.join(__dirname, '..', '..', 'src', 'server', 'validation.ts'),
|
||||
'utf-8',
|
||||
);
|
||||
});
|
||||
|
||||
it('imports ipKeyGenerator from express-rate-limit', () => {
|
||||
expect(validationSource).toMatch(/import.*ipKeyGenerator.*from\s+'express-rate-limit'/);
|
||||
});
|
||||
|
||||
it('keyGenerator body calls ipKeyGenerator', () => {
|
||||
expect(validationSource).toMatch(/ipKeyGenerator\(ip\)/);
|
||||
});
|
||||
});
|
||||
@@ -870,3 +870,169 @@ describe('assertSafeStoragePath (#1003)', () => {
|
||||
expect(() => assertSafeStoragePath(entry)).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
// ─── Worktree-aware registry-name fallback (#1259) ─────────────────────
|
||||
//
|
||||
// The first @claude review on PR #1296 caught a critical gap: my initial
|
||||
// fix only patched the early-return path in `runFullAnalysis`, leaving
|
||||
// the full-analysis path (which calls `registerRepo` directly) still
|
||||
// using the worktree-slug basename when no `--name` and no remote are
|
||||
// configured. This block proves `registerRepo`'s OWN basename fallback
|
||||
// now uses the canonical repo root via `getCanonicalRepoRoot` — the
|
||||
// regression-guard for the wiring at the registry layer, complementing
|
||||
// the helper-level coverage in `git-utils.test.ts`.
|
||||
|
||||
describe('registerRepo worktree-aware basename fallback (#1259)', () => {
|
||||
let tmpHome: Awaited<ReturnType<typeof createTempDir>>;
|
||||
let tmpRepo: Awaited<ReturnType<typeof createTempDir>>;
|
||||
let savedGitnexusHome: string | undefined;
|
||||
|
||||
const meta: RepoMeta = {
|
||||
repoPath: '',
|
||||
lastCommit: 'abc1234',
|
||||
indexedAt: '2026-05-03T00:00:00.000Z',
|
||||
stats: { files: 1, nodes: 1 },
|
||||
};
|
||||
|
||||
beforeEach(async () => {
|
||||
tmpHome = await createTempDir('gitnexus-registry-home-');
|
||||
tmpRepo = await createTempDir('gitnexus-canonical-repo-');
|
||||
savedGitnexusHome = process.env.GITNEXUS_HOME;
|
||||
process.env.GITNEXUS_HOME = tmpHome.dbPath;
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
if (savedGitnexusHome === undefined) delete process.env.GITNEXUS_HOME;
|
||||
else process.env.GITNEXUS_HOME = savedGitnexusHome;
|
||||
await tmpHome.cleanup();
|
||||
await tmpRepo.cleanup();
|
||||
});
|
||||
|
||||
it('registerRepo from a linked worktree uses canonical repo basename, not worktree slug', async () => {
|
||||
// Set up a real git repo with at least one commit (worktree add requires
|
||||
// a non-empty branch). No remote is configured — that's the trigger for
|
||||
// the basename fallback this test guards.
|
||||
execSync('git init -q', { cwd: tmpRepo.dbPath });
|
||||
execSync('git config user.email "test@example.com"', { cwd: tmpRepo.dbPath });
|
||||
execSync('git config user.name "Test"', { cwd: tmpRepo.dbPath });
|
||||
execSync('git commit --allow-empty -q -m "initial"', { cwd: tmpRepo.dbPath });
|
||||
|
||||
const worktreeDir = path.join(tmpRepo.dbPath, 'wt-feature');
|
||||
execSync(`git worktree add -q -b feature "${worktreeDir}"`, { cwd: tmpRepo.dbPath });
|
||||
|
||||
try {
|
||||
// Call registerRepo with the WORKTREE path and NO --name. Pre-fix this
|
||||
// would register under the worktree's basename ("wt-feature"). The
|
||||
// canonical-root fallback in registerRepo now resolves it to the
|
||||
// canonical repo's basename (whatever `tmpRepo`'s temp-dir basename
|
||||
// happens to be).
|
||||
await registerRepo(worktreeDir, meta);
|
||||
|
||||
const entries = await listRegisteredRepos();
|
||||
expect(entries).toHaveLength(1);
|
||||
// The registered name MUST NOT be the worktree slug.
|
||||
expect(entries[0].name).not.toBe('wt-feature');
|
||||
// It MUST match the canonical repo dir's basename. We compare via
|
||||
// basename (not full-path equality) for the same Windows 8.3
|
||||
// short-name reason as the `getCanonicalRepoRoot` helper tests:
|
||||
// git and `fs.realpathSync` may resolve to different long/short
|
||||
// forms of the same path on Windows runners, but both have the
|
||||
// same `basename`.
|
||||
expect(entries[0].name).toBe(path.basename(tmpRepo.dbPath));
|
||||
} finally {
|
||||
// Best-effort worktree teardown before the temp-dir cleanup runs.
|
||||
try {
|
||||
execSync(`git worktree remove -f "${worktreeDir}"`, { cwd: tmpRepo.dbPath });
|
||||
} catch {
|
||||
// Falls through to recursive rm in afterEach.
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Pinned by the second @claude review on PR #1296: the FIRST review-fix
|
||||
// commit (`7ceb839b`) introduced a regression in `hasCustomAlias`. Once
|
||||
// a worktree is registered with the canonical basename
|
||||
// (`{name: 'repo', path: '/repo/wt-feature'}`), `hasCustomAlias` saw
|
||||
// `'repo' !== path.basename('/repo/wt-feature') = 'wt-feature'` and
|
||||
// wrongly classified the canonical-root name as a sticky user alias.
|
||||
// On re-analyze the duplicate-name guard then fired against the
|
||||
// canonical checkout's entry → `RegistryNameCollisionError` blocking
|
||||
// the primary "per-task worktree, repeated re-analyze" workflow this
|
||||
// PR is supposed to FIX. This test exercises the full sequence:
|
||||
// canonical → worktree → re-worktree, with both paths registered.
|
||||
it('canonical → worktree → re-worktree re-register does not throw collision (#1259 hasCustomAlias regression)', async () => {
|
||||
execSync('git init -q', { cwd: tmpRepo.dbPath });
|
||||
execSync('git config user.email "test@example.com"', { cwd: tmpRepo.dbPath });
|
||||
execSync('git config user.name "Test"', { cwd: tmpRepo.dbPath });
|
||||
execSync('git commit --allow-empty -q -m "initial"', { cwd: tmpRepo.dbPath });
|
||||
|
||||
const worktreeDir = path.join(tmpRepo.dbPath, 'wt-feature');
|
||||
execSync(`git worktree add -q -b feature "${worktreeDir}"`, { cwd: tmpRepo.dbPath });
|
||||
|
||||
try {
|
||||
// 1. Register the canonical checkout — gets the canonical basename.
|
||||
await registerRepo(tmpRepo.dbPath, meta);
|
||||
// 2. Register the worktree — gets the SAME canonical basename
|
||||
// (because of the `resolveRepoIdentityRoot` fix). Two entries
|
||||
// coexist with the same name but different paths; this is the
|
||||
// documented "silent basename collision" behavior, not an error.
|
||||
await registerRepo(worktreeDir, meta);
|
||||
// 3. Re-register the worktree. Pre-`hasCustomAlias`-fix this threw
|
||||
// `RegistryNameCollisionError` because the existing worktree
|
||||
// entry (`{name: 'repo', path: worktreeDir}`) was misclassified
|
||||
// as a custom alias by `hasCustomAlias`, fired the guard
|
||||
// against the canonical entry. With the fix it must complete
|
||||
// without throwing.
|
||||
await expect(registerRepo(worktreeDir, meta)).resolves.toBeDefined();
|
||||
|
||||
// Both registry entries should still be present and named
|
||||
// canonically.
|
||||
const entries = await listRegisteredRepos();
|
||||
expect(entries).toHaveLength(2);
|
||||
const canonicalBasename = path.basename(tmpRepo.dbPath);
|
||||
for (const entry of entries) {
|
||||
expect(entry.name).toBe(canonicalBasename);
|
||||
}
|
||||
} finally {
|
||||
try {
|
||||
execSync(`git worktree remove -f "${worktreeDir}"`, { cwd: tmpRepo.dbPath });
|
||||
} catch {
|
||||
// Falls through to recursive rm in afterEach.
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Pinned by the third @claude review on PR #1296 (MEDIUM #2): the
|
||||
// `hasGitDir` gate inside `resolveRepoIdentityRoot` is the safeguard
|
||||
// that keeps the #1232/#1233 `--skip-git` behaviour working — an
|
||||
// arbitrary subdir under a parent git repo (no `.git` of its own)
|
||||
// must NOT collapse to the parent's canonical root, otherwise users
|
||||
// who analyze a subdir get the parent repo's basename in the
|
||||
// registry. The COOLIO `--skip-git` integration test in
|
||||
// `skip-git-cli.test.ts` already proves this end-to-end, but no
|
||||
// direct test sat at the `registerRepo` layer to guard the gate
|
||||
// against future refactors. This is that direct test.
|
||||
it('registerRepo on an arbitrary subdir under a git repo preserves the subdir basename (#1232 / #1233 gate)', async () => {
|
||||
execSync('git init -q', { cwd: tmpRepo.dbPath });
|
||||
execSync('git config user.email "test@example.com"', { cwd: tmpRepo.dbPath });
|
||||
execSync('git config user.name "Test"', { cwd: tmpRepo.dbPath });
|
||||
execSync('git commit --allow-empty -q -m "initial"', { cwd: tmpRepo.dbPath });
|
||||
|
||||
// A subdir of the canonical checkout, NOT a worktree (no `.git` file
|
||||
// here — `mkdirSync` only). `resolveRepoIdentityRoot` must keep
|
||||
// returning this exact path (basename used for the registry name)
|
||||
// rather than collapsing to the parent repo's canonical root.
|
||||
const subdir = path.join(tmpRepo.dbPath, 'arbitrary-subdir');
|
||||
await fs.mkdir(subdir, { recursive: true });
|
||||
|
||||
await registerRepo(subdir, meta);
|
||||
|
||||
const entries = await listRegisteredRepos();
|
||||
expect(entries).toHaveLength(1);
|
||||
// Registered name must be the SUBDIR's basename, NOT the parent
|
||||
// canonical repo's basename — the inverse of what worktree
|
||||
// collapse does.
|
||||
expect(entries[0].name).toBe('arbitrary-subdir');
|
||||
expect(entries[0].name).not.toBe(path.basename(tmpRepo.dbPath));
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2,7 +2,11 @@ import { execSync } from 'child_process';
|
||||
import fs from 'fs/promises';
|
||||
import path from 'path';
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { deriveEmbeddingMode } from '../../src/core/embedding-mode.js';
|
||||
import {
|
||||
deriveEmbeddingMode,
|
||||
deriveEmbeddingCap,
|
||||
DEFAULT_EMBEDDING_NODE_LIMIT,
|
||||
} from '../../src/core/embedding-mode.js';
|
||||
import { getStoragePaths, saveMeta, type RepoMeta } from '../../src/storage/repo-manager.js';
|
||||
import { createTempDir } from '../helpers/test-db.js';
|
||||
|
||||
@@ -136,3 +140,39 @@ describe('deriveEmbeddingMode', () => {
|
||||
expect(m.preserveExistingEmbeddings).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('deriveEmbeddingCap', () => {
|
||||
it('uses the default 50K cap when limit is undefined', () => {
|
||||
const d = deriveEmbeddingCap(10_000, undefined);
|
||||
expect(d.nodeLimit).toBe(DEFAULT_EMBEDDING_NODE_LIMIT);
|
||||
expect(d.capDisabled).toBe(false);
|
||||
expect(d.skipForCap).toBe(false);
|
||||
});
|
||||
|
||||
it('skips when node count exceeds the default cap', () => {
|
||||
const d = deriveEmbeddingCap(75_000, undefined);
|
||||
expect(d.skipForCap).toBe(true);
|
||||
expect(d.capDisabled).toBe(false);
|
||||
});
|
||||
|
||||
it('does not skip when node count equals the default cap (boundary)', () => {
|
||||
const d = deriveEmbeddingCap(DEFAULT_EMBEDDING_NODE_LIMIT, undefined);
|
||||
expect(d.skipForCap).toBe(false);
|
||||
});
|
||||
|
||||
it('limit=0 disables the cap regardless of node count', () => {
|
||||
const d = deriveEmbeddingCap(1_000_000, 0);
|
||||
expect(d.capDisabled).toBe(true);
|
||||
expect(d.skipForCap).toBe(false);
|
||||
expect(d.nodeLimit).toBe(0);
|
||||
});
|
||||
|
||||
it('honors a custom positive cap', () => {
|
||||
expect(deriveEmbeddingCap(99_999, 100_000).skipForCap).toBe(false);
|
||||
expect(deriveEmbeddingCap(100_001, 100_000).skipForCap).toBe(true);
|
||||
});
|
||||
|
||||
it('custom cap below default still applies', () => {
|
||||
expect(deriveEmbeddingCap(15_000, 10_000).skipForCap).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user