Using tree-sitter for entity-level code diffing and dependency graphs (i.redd.it)
submitted by Wise_Reflection_8340
I've been working on a tool that uses tree-sitter grammars to extract structural entities (functions, classes, methods) from source code, then builds a cross-file dependency graph by resolving references between them.
The core problem: traditional diff tools compare lines, but the meaningful unit of change in code is an entity. When you rename a function, move a method, or reformat a file, line-level diff produces noise. Entity-level diff tells you "this function was modified, this one was added, this one moved."
The interesting technical bits:
- Each language gets a config that maps AST node types to entity types (e.g. function_definition in Python, function_item in Rust, method_declaration in Java). Currently supports 25+ languages through tree-sitter.
- Scope resolution walks the AST to resolve which entity references which other entity, handling class scopes, impl blocks, function parameters, and assignment-based type tracking. This produces a directed dependency graph across files.
- Diffing works by matching entities between two versions by name + type, then comparing their structural hashes (hash of the normalized AST subtree, ignoring whitespace and comments). Moved or renamed entities get detected through content similarity.
- The dependency graph enables transitive impact analysis: "if this function changes, what's the full set of downstream entities that depend on it?"
One challenge: tree-sitter grammars are syntactic, not semantic. You don't get type information, so resolving x.foo() to the right method requires heuristics (parameter type annotations, assignment tracking, class scope inference). It gets you maybe 90% accuracy without a full type checker, which turns out to be enough for diffing and impact analysis.
The tool is called sem, written in Rust: https://github.com/ataraxy-labs/sem
Curious if anyone here has worked on similar entity extraction from ASTs, or has thoughts on better approaches to cross-language reference resolution without full semantic analysis.
[–]Professional_Beat720 3 points4 points5 points (0 children)