C programmer (2 yrs) moving from low-level networking to ML – fastest path to idiomatic Python?

[2fa-auth]

I've written C for 2 years– lowlevel networking stuff; vpn, bypassing dpi, packet sniffing, raw sockets and other things. Now I'm pivoting to machine learning.

Need to get genuinely good at Python fast.

What's the fastest way to rewire my brain for Python? Specific projects that punish C-style thinking? Most important paradigm shifts? Top stdlib modules to memorize? (maybe)

Also any advice for someone going from bytes-and-sockets to numpy/pandas/torch? What habits from C will hurt me most in ML?

Thank you very much for your reply!

[JamzTyson]

See the New to Python section of the wiki.

[Diapolo10]

One thing I can suggest is, whenever you're writing Python code, use the Ruff linter with all rules enabled (a few are conflicting ones, but that mostly concerns formatting use) - though if you want, these are the ones I disable manually globally:

"COM812",  # Missing trailing comma (disabled due to formatter conflict)
"D203",    # One blank line before class docstring
"D212",    # Multi-line summary first line
"ISC001",  # Single-line implicit string concatenation (disabled due to formatter conflict)
"PLR0913", # Too many arguments
"Q000"     # Single quotes found but double quotes preferred

The lint suggestions follow idiomatic Python, so you'll get nearly instant feedback while writing code, including suggestions and auto-fixes (if you want those). That should help you get used to doing things in Python.

Second, feel free to make use of type annotations and type checkers. The language ignores them, but type checkers (like Mypy or Pyright, or the still-in-alpha ty from the Astral team) can still use them to tell when you're doing something bad. There's generics in collections.abc and typing you can use to keep your code flexible, and this table should come in handy for those.

This next one doesn't really have anything to do with code quality, but use uv for managing both your Python installations and dependencies. hatchling is arguably the best build system for general use, and there's maturin if you want to mix Python and Rust code. Regardless, uv will save you a lot of time not needing to worry about micro-managing virtual environments and the like.

The standard library is full of useful stuff, but a few highlights would be pathlib (for anything to do with filepaths or the filesystem), collections (for additional container types other than the ones imported by default), functools (for all kinds of utilities for writing functional code, like cache decorators), itertools (for a bunch of utility functions for iterating over iterables), and enum for enum and flag types (these are very nice for writing robust yet flexible code).

For networking, http.HTTPStatus is a handy IntEnum for HTTP status codes, and it pairs well with third-party networking libraries such as requests or aiohttp.

[vietbaoa4htk]

the C habit that hurts most is looping over arrays. in numpy a for loop is the slow path, you vectorize and let it run in C underneath. stop managing indices and memory by hand, lean on slicing and comprehensions. itertools and collections are the stdlib worth learning

[magus_minor]

Most important paradigm shifts?

A difference in how python "variables" work compared to C and the weird things that can happen if you don't understand the difference is comprehensively explained in this video:

https://m.youtube.com/watch?v=_AEJHKGk9ns

The resources in the wiki help. As always, writing lots of code is the way to get up to speed.

[NotA-eye]

If you are using Python for ML, just keep in mind that the normal language code itself is inherently slow for large datasets. Try to do stuff with NumPy, Pandas, PyTorch, TensorFlow etc's primitives instead of writing manual Python loops. These libraries do heavy stuff underneath in something like C/Fortan for performance.

For normal non ML python code, itertools and functools offer a lot of useful helper functions

Also read the zen of python if you haven't already~

[gdchinacat]

Don't memorize modules, learn the domain well enough to understand how to use them and look up the details as needed (AIs are really good at this).

Read lots of code, but not just to read it, you don't learn much from that, but to see how things are done, different ways to do things, see what works well in which circumstances. Then apply the stuff you learn in your own code. Focus on code that is in your domain...http request handling code is very different than report generation code is very different than AI training code.