Compare commits

...
Author SHA1 Message Date
Gergo Magyar 14cebdb80e fix(windows): replace 32767-char truncation with chunked-input parsing
The tree-sitter 0.21.x Node binding crashes (SIGSEGV) on Windows when
parser.parse(string, ...) is handed a JS string longer than 32 767 chars.
The crash is in the bindings V8 string-to-buffer conversion and cannot
be intercepted from JS. Previous mitigation truncated source at the last
newline before that boundary, silently losing the file tail and producing
ERROR-root trees from mid-class cuts.

Switch to the callback (Parser.Input) overload via a new parseSourceSafe
helper. tree-sitter pulls source in 16 KiB chunks via repeated callback
invocations, bypassing the broken conversion path. Files are parsed in
full, no data loss, no platform-specific code path.

Removes the now-unnecessary ERROR-root short-circuit in csharp/captures.ts
and the empty-captures shim in scope-extractor-bridge.ts; both existed only
to swallow truncation-induced parse failures.
2026-05-10 13:15:06 +01:00
Gergo Magyar 5398880542 fix: address codeql findings on PR #1433
The four `lastIndexOf('\n', ...)` calls were committed with a literal
newline inside the single-quoted string instead of the `\n` escape, so
the files do not parse — `tsc` and CodeQL both flagged them. Replace
the embedded newline with `'\n'`.

Also remove the two helpers that were superseded during review and
became dead code: `rangeNonStrictlyContains` in scope-tree.ts (the
equal-range carve-out is handled by `rangeStrictlyContains` +
`rangesEqual` in `canParentScope`) and `moduleAwareContains` in
scope-extractor.ts (`pass1BuildScopes` calls `canParentScope` directly).
2026-05-10 09:39:46 +01:00
Gergő Magyar 66a17785c0 Merge branch 'main' into fix/windows-crashes-32767-chars-and-vector-extension 2026-05-10 09:35:11 +01:00
Gergő Magyar 29085304b8 Merge branch 'main' into fix/windows-crashes-32767-chars-and-vector-extension 2026-05-09 18:10:44 +01:00
Gergő Magyar 80d21f9a1c Merge branch 'main' into fix/windows-crashes-32767-chars-and-vector-extension 2026-05-09 09:11:21 +01:00
Gergő Magyar 980cbd6435 Merge branch 'main' into fix/windows-crashes-32767-chars-and-vector-extension 2026-05-09 06:55:09 +01:00
Gergő Magyar cc1e90bb52 Merge branch 'main' into fix/windows-crashes-32767-chars-and-vector-extension 2026-05-08 17:18:17 +01:00
Hector Prats f61f61d94c fix(windows): skip FTS extension load in pool-adapter on Windows to prevent SIGSEGV
LOAD EXTENSION fts crashes the process with SIGSEGV on Windows when the
FTS extension binary is not installed locally. This is an @ladybugdb/core
native bug — the extension loader hits an unhandled error path that raises
a native signal instead of a JS exception, so try/catch cannot protect here.

Add a process.platform === 'win32' guard in both doInitLbug and
initLbugWithDb. When skipped, bm25-index.js catches the resulting
Kuzu catalog errors (CREATE_FTS_INDEX not defined) and returns empty
BM25 results gracefully. All graph queries (cypher, context, impact)
are unaffected.

This is patch 9 of the Windows fix series for gitnexus on Windows:
patch 8 (same PR) already fixed INSTALL VECTOR SIGSEGV in lbug-adapter.ts.
pool-adapter.ts is the separate MCP-server code path that was not covered.
2026-05-08 18:16:01 +02:00
Hector Prats 93a65e5c13 fix(windows): 32767-char tree-sitter crash + VECTOR extension SIGSEGV
tree-sitter 0.21.x on Windows crashes with SIGSEGV when parsing source
strings longer than 32 767 chars (signed 16-bit integer overflow in the
native binding). Five call sites passed raw file content without any
length guard:

  - captures.ts (C# scope extraction)
  - namespace-siblings.ts (extractFileStructure)
  - parse-worker.ts (worker thread parse path)
  - parsing-processor.ts (sequential parse fallback)

Fix: truncate at the last newline before the limit so the fragment stays
syntactically coherent. Files truncated mid-class produce ERROR roots;
captures.ts returns [] for any ERROR-root tree so the legacy DAG handles
the file silently without orphaned scope errors.

Additional C# scope fixes:
  - scope-tree.ts: Module scopes may share the same range as a top-level
    namespace_declaration (files with no leading `using` directives). The
    rangeStrictlyContains check rejects equal ranges. Added
    rangeNonStrictlyContains for Module parents.
  - scope-extractor.ts: pass1BuildScopes stack-pop used strict containment;
    same Module == Namespace range case caused orphaned scopes. Added
    moduleAwareContains helper.
  - scope-extractor-bridge.ts: empty captures from ERROR-root files still
    called extractScope -> "no Module scope found" warning. Added early
    return for empty/non-array captures.
  - namespace-siblings.ts: three sites pushed onto binding arrays frozen by
    finalize-algorithm. Fixed with spread-copy before mutation.

lbug-adapter.ts: INSTALL VECTOR in loadVectorExtension calls the KuzuDB
native extension installer, which crashes with SIGSEGV on Windows via an
unhandled error path in native code. JS try/catch cannot intercept native
signals. Skip extension loading on win32 — vector/embedding search is
unavailable on Windows but all graph index queries work correctly.

Verified on: Windows 11, Node.js 24, gitnexus 1.6.3, pcf8-game codebase
(61 757 nodes / 111 796 edges / 300 flows after fix).
2026-05-08 17:01:46 +02:00
7 changed files with 75 additions and 9 deletions
@@ -24,6 +24,7 @@ import { synthesizeCsharpReceiverBinding } from './receiver-binding.js';
import { getCsharpParser, getCsharpScopeQuery } from './query.js';
import { recordCacheHit, recordCacheMiss } from './cache-stats.js';
import { getTreeSitterBufferSize } from '../../constants.js';
import { parseSourceSafe } from '../../utils/safe-parse.js';
/** Declaration anchors that carry function-like arity metadata. */
const FUNCTION_DECL_TAGS = [
@@ -86,7 +87,7 @@ export function emitCsharpScopeCaptures(
// the LanguageProvider contract layer; cast here at the use site.
let tree = cachedTree as ReturnType<ReturnType<typeof getCsharpParser>['parse']> | undefined;
if (tree === undefined) {
tree = getCsharpParser().parse(sourceText, undefined, {
tree = parseSourceSafe(getCsharpParser(), sourceText, undefined, {
bufferSize: getTreeSitterBufferSize(sourceText),
});
recordCacheMiss();
@@ -36,6 +36,7 @@ import type { BindingRef, ParsedFile, Scope, ScopeId, SymbolDefinition } from 'g
import type { ScopeResolutionIndexes } from '../../model/scope-resolution-indexes.js';
import { getCsharpParser } from './query.js';
import { getTreeSitterBufferSize } from '../../constants.js';
import { parseSourceSafe } from '../../utils/safe-parse.js';
interface CsharpFileStructure {
/** Declared namespace names in file source order. Empty array means
@@ -56,7 +57,7 @@ function extractFileStructure(content: string, cachedTree: unknown): CsharpFileS
type CsharpTree = ReturnType<ReturnType<typeof getCsharpParser>['parse']>;
const tree =
(cachedTree as CsharpTree | undefined) ??
getCsharpParser().parse(content, undefined, {
parseSourceSafe(getCsharpParser(), content, undefined, {
bufferSize: getTreeSitterBufferSize(content),
});
const namespaces: string[] = [];
@@ -359,7 +360,7 @@ export function populateCsharpNamespaceSiblings(
const q = def.qualifiedName ?? '';
const key = q.includes('.') ? q.slice(q.lastIndexOf('.') + 1) : q;
if (key === '') continue;
const arr = defsByName.get(key) ?? [];
const arr = [...(defsByName.get(key) ?? [])];
arr.push(def);
defsByName.set(key, arr);
}
@@ -5,12 +5,11 @@ import { loadParser, loadLanguage, isLanguageAvailable } from '../tree-sitter/pa
import { getProvider } from './languages/index.js';
import { generateId } from '../../lib/utils.js';
import type { SymbolTableReader, SymbolTableWriter, ExtractedHeritage } from './model/index.js';
// SymbolTableReader is used for the FieldExtractorContext stub; the
// parsing functions themselves need Writer because they call .add().
import { ASTCache } from './ast-cache.js';
import { getLanguageFromFilename, SupportedLanguages } from 'gitnexus-shared';
import { extractVueScript, isVueSetupTopLevel } from './vue-sfc-extractor.js';
import { yieldToEventLoop } from './utils/event-loop.js';
import { parseSourceSafe } from './utils/safe-parse.js';
import { isVerboseIngestionEnabled } from './utils/verbose.js';
import {
getDefinitionNodeFromCaptures,
@@ -384,7 +383,7 @@ const processParsingSequential = async (
let tree: Parser.Tree;
try {
tree = parser.parse(parseContent, undefined, {
tree = parseSourceSafe(parser, parseContent, undefined, {
bufferSize: getTreeSitterBufferSize(parseContent),
});
} catch (parseError) {
@@ -0,0 +1,40 @@
import type Parser from 'tree-sitter';
/**
* tree-sitter 0.21.x's Node native binding crashes (SIGSEGV) on Windows when
* `parser.parse(string, …)` is handed a JS string longer than 32 767 chars.
* The crash happens inside the binding's V8 string-to-buffer conversion and
* cannot be intercepted from JavaScript. The callback (`Parser.Input`) overload
* pulls source in fixed-size chunks via repeated callback invocations and
* bypasses that conversion path entirely.
*
* Chunk size is comfortably below the boundary; any value < 32 767 works.
*/
const SAFE_PARSE_CHUNK_CHARS = 16 * 1024;
/**
* Files at or below this length skip the callback machinery and use the
* direct string overload — the bug only manifests above the int16 boundary,
* so small inputs save the cost of N callback invocations per parse.
*/
const DIRECT_PARSE_LIMIT_CHARS = 16 * 1024;
/**
* Parse `sourceText` safely on every platform. See {@link SAFE_PARSE_CHUNK_CHARS}
* for the underlying tree-sitter binding bug this works around.
*/
export function parseSourceSafe(
parser: Parser,
sourceText: string,
oldTree?: Parser.Tree,
options?: Parser.Options,
): Parser.Tree {
if (sourceText.length <= DIRECT_PARSE_LIMIT_CHARS) {
return parser.parse(sourceText, oldTree, options);
}
const input: Parser.Input = (index) => {
if (index >= sourceText.length) return null;
return sourceText.slice(index, index + SAFE_PARSE_CHUNK_CHARS);
};
return parser.parse(input, oldTree, options);
}
@@ -20,6 +20,7 @@ import {
getTreeSitterContentByteLength,
TREE_SITTER_MAX_BUFFER,
} from '../constants.js';
import { parseSourceSafe } from '../utils/safe-parse.js';
import type { SymbolTableReader } from '../model/symbol-table.js';
import type { ExtractedHeritage } from '../model/heritage-map.js';
@@ -1416,7 +1417,7 @@ const processFileGroup = (
let tree;
try {
tree = parser.parse(parseContent, undefined, {
tree = parseSourceSafe(parser, parseContent, undefined, {
bufferSize: getTreeSitterBufferSize(parseContent),
});
} catch (err) {
+9
View File
@@ -1248,6 +1248,15 @@ export const loadVectorExtension = async (
): Promise<boolean> => {
const useModuleState = targetConn === undefined;
if (useModuleState && vectorExtensionLoaded) return true;
// INSTALL VECTOR crashes with SIGSEGV on Windows: the KuzuDB native extension
// installer has an unhandled error path on Windows that raises a fatal signal
// that JS try/catch cannot intercept. Skip loading — vector/embedding search
// is unavailable but all graph index queries still work.
// See: https://github.com/abhigyanpatwari/GitNexus/issues/...
if (process.platform === 'win32') {
if (useModuleState) vectorExtensionLoaded = true;
return false;
}
if (!isVectorExtensionSupportedByPlatform()) return false;
const c: lbug.Connection | null = targetConn ?? conn;
+17 -2
View File
@@ -420,7 +420,17 @@ async function doInitLbug(repoId: string, dbPath: string): Promise<void> {
// install; analyze owns extension installation. If LOAD fails, search
// features degrade gracefully and the user-facing query path proceeds.
if (!shared.ftsLoaded) {
shared.ftsLoaded = await loadFTSExtension(available[0], { policy: 'load-only' });
// Windows guard: LOAD EXTENSION fts crashes with SIGSEGV on Windows when
// the FTS extension binary is not installed locally (@ladybugdb/core native
// bug — the extension loader hits an unhandled error path that signals SIGSEGV
// rather than throwing a JS exception, so try/catch cannot protect here).
// Skip the load on Windows; bm25-index.js catches the resulting Kuzu catalog
// errors and returns empty BM25 results gracefully. Graph queries are unaffected.
if (process.platform === 'win32') {
shared.ftsLoaded = true;
} else {
shared.ftsLoaded = await loadFTSExtension(available[0], { policy: 'load-only' });
}
}
// Register pool entry only after all connections are pre-warmed and FTS is
@@ -484,8 +494,13 @@ export async function initLbugWithDb(
// Load FTS extension if not already loaded on this Database.
// policy: 'load-only' — same contract as initLbug above; the read pool
// must not block on a network install during query execution.
// Windows guard: same SIGSEGV risk as doInitLbug above — skip on Windows.
if (!shared.ftsLoaded) {
shared.ftsLoaded = await loadFTSExtension(available[0], { policy: 'load-only' });
if (process.platform === 'win32') {
shared.ftsLoaded = true;
} else {
shared.ftsLoaded = await loadFTSExtension(available[0], { policy: 'load-only' });
}
}
pool.set(repoId, {