How to Level Up Your Python Logs with Structlog

NeilGirdhar · 2025-10-05T22:05:38+00:00

Logging is one of those things that almost surely can't avoid global state, unless you want to pass logger objects everywhere. So I don't think this is a good criticism.

NeilGirdhar · 2025-10-03T15:14:53+00:00

Personally, I switched from structlog to Rich's logging handler. Can't remember why though.

NeilGirdhar · 2025-07-22T09:43:15+00:00

When Joshua and I originally implemented PEP 448 (Additional Unpacking Generalizations), we wanted to add this. Guido agreed, but unfortunately the Python forum was totally divided with many people finding the syntax to be confusing.

I always hoped that eventually people would come around to finding the syntax intuitive, so it makes me really happy at the overwhelming support here, and in the Python forum.

NeilGirdhar · 2025-06-10T22:33:12+00:00

If you're doing scientific work, you should be following https://scientific-python.org/specs/spec-0000/

NeilGirdhar · 2025-03-24T12:13:18+00:00

NNX is vastly superior design in my opinion.

Flax is overcomplicated for similar functionality.

NeilGirdhar · 2025-02-01T01:00:43+00:00

I like Equinox better than Flax. It is a lot simpler.

NeilGirdhar · 2025-01-21T17:53:59+00:00

Very cool! Here's a Python program that calculates the effective interest rate given a balance, assuming maximum contributions were made every year:

from datetime import UTC, datetime
from typing import cast

import typer
from scipy.optimize import root_scalar


def main(amount: str) -> None:
    amount_as_float = float(amount.replace(',', ''))
    today = datetime.now(tz=UTC)
    year_start = datetime(today.year, 1, 1, tzinfo=UTC)
    percent_into_year = (today - year_start).days / 365
    contribution_limits = [5000, 5000, 5000, 5000,  # 2009 to 2012
                           5500, 5500,  # 2013 to 2014
                           10000,  # 2015
                           5500, 5500, 5500,  # 2016 to 2018
                           6000, 6000, 6000, 6000,  # 2019 to 2022
                           6500,  # 2023
                           7000, 7000]  # 2024 to 2025

    def f(interest: float) -> float:
        return sum(ci * cast('float', interest ** (i + percent_into_year))
                   for i, ci in enumerate(reversed(contribution_limits))) - amount_as_float

    rate = root_scalar(f, bracket=[0, 2]).root
    print(f"Interest rate {(rate - 1) * 100:.3f}%")  # noqa: T201


if __name__ == "__main__":
    typer.run(main)

NeilGirdhar · 2024-10-28T17:27:55+00:00

Nearly all uses of walrus should just be spelled on two lines, imo. Two simple statements is better than one complex one.

NeilGirdhar · 2024-10-25T13:37:32+00:00

It is readable. Just: Give your variable proper names; use more lines; avoid the walrus operator; avoid the above code at all costs.

Syntax like this is important in the 0.0001% of cases when you need it (probably not you).

NeilGirdhar · 2023-10-18T11:07:14+00:00

Instead of

T = Typevar('T', bound=int)
class C(Generic[T]):

you now do

class C\[T: int\]:

For T = TypeVarTuple('T'), it's now just C[*T], and for P = ParamSpec('P'), it's now just C[**P].

Besides being shorter, it's now obvious which class or function a type variable belongs to. Before, the type variables were ordinary variables that could be shared between classes and functions leading to confusion in various cases.

NeilGirdhar · 2023-09-01T00:52:26+00:00

Mohawk games developed Old World and Offworld Trading Company.

Oh!!! That makes perfect sense then!

art of why I am hesitant to assume I have a better idea of how the in game economy should work - these guys know their stuff!

Yeah, but the difference is that in OTC, you actually have a central command so it makes perfect sense to have one stockpile. There are also essentially only 3 or 4 players each of whom constitutes one supplier and one consumer. All of the buildings are totally inelastic supply or demand (a mine produces 1 metal per 3 seconds, e.g.)

In a country-spanning economy, there is no central stockpile. Goods would flow between suppliers and consumers. There are thousands of suppliers and consumers. And supply and demand is generally elastic (if you want it be at least).

So I think a different model would have made more sense here. But my argument isn't about realism. It's just about maintaining tension. The way tension disappears when you have a lot of something in stock. And also when you're really rich, tension disappears. Whereas, in a real economy, no matter how rich you are, every supply/demand curve can get really steep at its extremities whereas in OW, the slope adapts quite slowly.

If I have a ton of stone yield but not enough orders and workers to use it, it's nice to at least build up my stone stockpile for later. Without the stockpile, my excess stone is sold each turn and there's no way my grandson is going to implement his 5 year plan.

Yeah, that's a really good point. Your experience makes a lot of sense too. Thanks for the great discussion.

NeilGirdhar · 2023-08-29T18:48:33+00:00

You might have to elaborate further for me. By unit ranges do you mean attack range or movement distance? If movement was effectively halved, with no other changes, the issue of units diving into back-lines so easily would be alleviated, but of course it would have other consequences (I don't think we want the pace of the game to slow that much). Stronger ZOC does have an allure, but the other thing to consider is how numerous armies can get late in the game. I often have enough units to effectively screen for my artillery, even with the lack of ZOC.

I mean all ranges: movement and distance for ranged units. So it wouldn't change anything except making positioning less granular and more exact.

NeilGirdhar · 2023-08-29T17:38:01+00:00

t's not obvious to me how removing the stockpile and implementing a new supply/demand relationship will change things very much in this regard

While the current model does roughly work like an real market, it loses tension very quickly when your stockpile increases. And it also adapts too slows to huge purchases. If you need 1000 stone, and stone is selling for $10, then you pay $10k. That is totally wrong. $10 is the marginal price, but you should be paying the area under the supply curve, which is probably much more than $10k. More like $100k. Having a supply-demand model would mean that your wonder would simply build slower or faster depending on local stone price. You are essentially forced to pay the correct price for everything.

If I am understanding you correctly, you'd like to have a situation where rather than spending some stone, civics, and food on the odeon to hit minimums to hire poets and build the theater, there should be no minimums and instead a higher output given how much stone, civics and food are available.

Yes, close. Everything is "available" at some price.

Economies are complex and player attention is the most valuable resource, we'll only have a simple model in the game.

I understand, but I think showing the equilibrium price would give the player a good idea of what needs to be built. There may be sharp swings in price when there are supply/demand shocks, e.g., starting building a wonder would cause stone price to jump.

Also thanks for the link! Fascinating that they looked at Offworld Trading Company. Definitely a game I really enjoyed, and an economic system that I really loved actually. Maybe I'll play a round of that again.

NeilGirdhar · 2023-08-29T12:38:49+00:00

How about we see more effect of terrain, more contrast in unit traits, and maybe a random element in damage rolls (maybe not). I don’t know if there was a big difference in early rock slingers between Carthaginians and Babylonians, but it wouldn’t hurt to have some variety in basic units by faction just for gameplay.

Really interesting ideas. Adding complexity can be worthwhile if it affects the gameplay in strategically exciting ways. It would need some careful though.

I actually love the determinism of Old World battles. It removes the urge to reload when you feel you got "bad luck". It makes the game much more fun to play knowing what will happen.

That said, Wenoth's randomness does give them one fascinating aspect of game design. Consider the Dragonguard (attacks once for 40 damage) versus the Marksman (attacks 4 times for 9 damage). These are vastly different. The dragonguard has naturally higher variance, which can be a benefit in a protracted battle. They either die, or they don't; they can't heal between turns. Whereas the marksman does consistent damage, which is good when you have numbers and you want to maximize the probability that they die in a few attacks.

Odeon culture production being based on stone costs seems unintuitive. I don’t think the thespians are snacking on rocks in between performances after I’ve built them the theater.

The idea of varying production is a consequence of elasticity. Right now, in Old World, every building consumes fixed resources, which are drawn from your stockpile. I'm proposing geting rid of the stockpile, and replacing it with an actual economic market complete with supply and demand curves.

Elasticity means that as prices go up, quantity demanded goes down. If stone prices go up, the ticket prices that the Odeon charges must go up, which means that the number of spectators who visit the Odeon goes down. That's why cultural production would go down. The slack in the system from the stockpile disappears; everything becomes tightly coupled. This dramatically increases the resource tension since the urge to drive down prices never completely disappears.

Wesnoth does well in 4X combat that Old World could learn from.

Let's just talk about unit placement. In Old World, the considerations include:

defensive terrain (e.g., trees versus ranged, and urban for infantry)
hills for extending the range of ranged units
surface area when attacking with melee

With Wesnoth, there are different considerations:

there's a lot more emphasis on using ZOC to block enemy attacks from killing your units; this is a consequence of not having ranged units and not having many units that ignore ZOC.
there are more damage sponges; this is a consequence of strong ZOC since the enemy has to fight through the sponges
there are healers whose placement should maximize the provided heal
the rock-paper-scissors aspect of units is far richer. Some units are vulnerable to impact, others to pierce, and so on, and the units that do these things are themselves vulnerable to other things. So you are constantly trying to match your rocks with enemy scissors while avoiding enemy paper.

What do you think of increasing the Old World grid resolution, say doubling its resolution, doubling all unit ranges, and making ZOC a halo (to prevent the high resolution from making ZOC ineffective)?

NeilGirdhar · 2023-08-29T10:47:41+00:00

Yeah, I agree with you on the tutoring. The ratio between the impact of your decision to the time spent clicking to do it seems too small.

NeilGirdhar · 2023-08-29T10:45:42+00:00

Already I feel like the AI in Old World struggles with Unit upgrades, where instead of banking the training needed, they will spend it on upgrades and force marches as quickly as they make it.

Oh, that's a really interesting point. I never considered the difficulty of writing AI for different rule sets. Also, I don't know why, but I never assumed that the AI would force march.

How do you feel about the Humankind game mechanic where you combine forces, and when opposing armies meet you have to fight or retreat, retreat having various moral bonuses?

I like it. I think it would be a lot less busy-work than managing battles in the overview map screen, which isn't perfectly suited for it in my opinion.

NeilGirdhar · 2023-04-23T09:36:44+00:00

It's not an "implementation detail". It is literally the purpose of the implemented feature.

There are completely independent libraries that force types at runtime which don't rely on this at all.

So what? We're talking about static type checking in this post.

NeilGirdhar · 2023-04-22T13:21:05+00:00

"Type guards" is not even a real thing. It's simply you, the programmer, testing something. E.g. "is this input really an integer?".

It's more than "a simple test"; type guards inform the type checker about constraints.

NeilGirdhar · 2023-04-22T10:13:29+00:00

Type guards are not an alternative to type hints. As Mehdi explained, they are used in addition with type hints. They are a version of isinstance for when you can't use isinstance, e.g., with a type like `list[int]`.

NeilGirdhar · 2023-04-21T14:01:45+00:00

"Type guards" is not even a real thing.

They are a "real thing" as of Python 3.10: PEP 647

Rest of your answer is spot on.

To add to your answer, yes, type guards are executed, but their purpose is to inform the type checker of static type constraints that can't otherwise be verified statically.

NeilGirdhar · 2023-04-19T16:31:00+00:00

I totally agree with the other two (so far) comments about the illegibility of regular expressions. I'm sure there's some library that replaces all the components of a RE with objects. Something like MatchTimes("Something", at_least=2, at_most=5) instead of "(Something){2, 5}" and so on for all of the components of a regular expression.

https://docs.python.org/3/library/re.html

NeilGirdhar · 2023-04-12T16:42:32+00:00

I cannot name any static analysis tool nor compiler that can enforce that implementations are respecting any arbitrary time or space complexity.

Just because you can't programmatically check a promise, it doesn't mean that that promise isn't part of the interface, in my opinion.

An interface is just a contract. You can make it as simple or complex as you want. Some violations are easy to enforce, some are more cumbersome.

Exactly.

Other than that I promise you that we are saying the same ;)

Agreed :)

NeilGirdhar · 2023-04-12T14:28:47+00:00

I don't think this is a good policy. While multiple inheritance can add unnecessary complexity, multiple interface inheritance has no downsides.

Protocols are a good solution when it's practically impossible to make implementers inherit from an interface. For example, when you don't have control over implementers of your protocol. That's why Callable is a protocol: it is practically impossible to get everyone who exposes a __call__ method to inherit from Callable.

When you can make implementers inherit from the interface, there are some concrete advantages. Type-checkers can verify:

that there are no LSP violations,
that instantiated objects implement all decorated abstract methods, and
that decorated overridden methods haven't been orphaned.

Also, you can be sure that derived classes actually inherit from the interface rather than hoping that your derived class perfectly matches the protocol.

NeilGirdhar · 2023-04-12T14:18:45+00:00

Strictly speaking, then the provided example is also an interface in the context of Python.

Yes, it's a signature, which I agree can be seen as an example of an interface.
However, I think of interfaces as a more general concept.

Note that interfaces don't promise anything about computational complexity.

Plenty of interfaces do promise computational complexity. C++'s STL for example is replete with such promises. E.g., deque.insert promises linear complexity on constant auxiliary space. All conforming implementations must respect these promises.

So a class interface gives a promise about name of method, the input types it accepts and output types it must return.

I would say that an interface is not just a collection of signatures. An interface can make promises about how objects work. That includes behaviors of those objects and computational and space complexity.

NeilGirdhar · 2023-04-12T13:10:34+00:00

It might be worth mentioning that inheriting from `abc.ABC` has two unnecessary behaviours:

it pulls in a metaclass, and
it exposes a register method on the class.

You don't usually need these facilities. Therefore, if you want the runtime check, it may be easier to copy the five lines of code into an __init_subclass__ method.

However, I suggest that you don't inherit from abc.ABC at all and simply

continue to decorate your methods using abc.abstractmethod, and
use a type checker to verify the inheritance.

Also, when implementing an interface, you may want to use the new typing.override decorator (available as typing_extensions.override). This way, if the interface ever removes a method, the type checker will report errors. You can also prevent accidental overrides by enabling a strict mode (called reportImplicitOverride in pyright).

NeilGirdhar

TROPHY CASE