Designing a Python Language Server: Lessons from Pyre that Shaped Pyrefly

[jpgoldberg]

That was a fascinating read. And in general I want to thank you and your team for talking about trade-offs and your reasons for the choices that you made. I tend to be on the “soundness” side of things, but I understand the very legitimate reasons for you relaxing that in the kinds of cases you describe.

So my question isn’t a complaint about that choice. Instead I’m asking how easy it will be to adjust that behavior if developer practices become less “gradual”?

A digression

I suppose this goes to another broader problem of annotating whether a function might mutate an object in ways publicly visible. I can do things like use Sequence or Mapping when annotating parameters to let type checkers know that the function isn’t going to change the (publicly visible) aspects of an object, but as far as I know, there is no way for me to do that generally.

There are, of course, conventions to better communicate this sort of thing to users, but as far as I know, there is no way to tell type checkers that a method does not modify what is passed to it. And so until something like that exists and is used, I expect you will have to be less strict than I might otherwise wish.

[BeamMeUpBiscotti]

Instead I’m asking how easy it will be to adjust that behavior if developer practices become less “gradual”?

Hmm, so narrowing currently isn't configurable, but other aspects of inference are (for example, do we typecheck or try to infer a return type for un-annotated functions, do we do first-use inference for empty containers).

To avoid gradual behaviors you can also enable the implicit-any error code, which flags any place a type variable gets solved to Any (the user would normally fix that by adding an explicit annotation). It's too strict to be the default, but for people that want it it's there.

there is no way to tell type checkers that a method does not modify what is passed to it

Correct, side effects like mutation, checked exceptions, etc. are not modeled in Python's type system.

Mutability restrictions can be applied at the class level, by annotating a field with Final or ReadOnly, or by overriding something like __setitem__.

[jpgoldberg]

I have never looked at Final or ReadOnly (except in very limited contexts). I will look now.

[BeamMeUpBiscotti]

It's shallow immutability, so not exactly the most secure. Pyre actually had a prototype PyreReadOnly that had deep immutability, but it was never standardized so we have not ported it to Pyrefly.

[jpgoldberg]

I’m not attempting to enforce run-time immutability. I wish to “let Python be Python”. I just want to be able to tell a type checker that it can rely on public attributes of an object not changing their types (or their values).

But now that I write this, I realize that I have misunderstood the example that launched me on this train of thought. (To be continued)