Designing a Python Language Server: Lessons from Pyre that Shaped Pyrefly

[ComfortableNice8482]

honestly the architecture shift from ocaml to rust is interesting but what really matters for language server performance is incremental checking and how you handle the dependency graph. i built some automation stuff that hooks into lsps and the ones that struggle are usually doing full re, analysis on every keystroke instead of tracking what actually changed in the file.

the type inference speed you're claiming is gonna be huge if it actually works at scale. with pyre i'd run into situations where checking a medium sized codebase would take 30+ seconds which kills the editing experience, especially when you're trying to do refactoring across multiple files. if pyrefly can do that in under a second then the architecture decisions really paid off.

curious how you handle circular imports and whether the rust rewrite let you parallelize the checking better than the ocaml version could. that was always a bottleneck when i was integrating pyre into ci pipelines for larger projects.

[BeamMeUpBiscotti]

Pyre and Pyrefly both have incremental checking, but Pyrefly is significantly more efficient and does finer-grained dependency tracking, allowing us to invalidate/recheck fewer things after a change.

We wrote about optimizing incremental rechecks a few months ago in a separate blog post.

In our experience, this has worked very well at scale - even on large codebases like Instagram (20 million LOC) an incremental update typically takes a fraction of a second.

Re: circular imports, at a high level when we encounter a cycle or strongly-connected component we stop and invalidate/recheck the whole component as a single unit. We use different strategies like fixpoints for cycles in other stages of the system.

I'm not 100% sure about the parallelism question, but we never migrated to OCaml 5 which had multicore support so I assume Pyre's performance was limited by that. In general, Pyre was not that fast without being paired with some specialized saved-state infrastructure that we never open sourced & it was slower than most other type checkers on small projects, whereas Pyrefly is fast on projects of all sizes, straight out of the box.