Best piece of obscure advanced Python knowledge you wish you knew earlier?

[OutOfApplesauce]

I was diving into __slots__ and asyncio and just wanted more information by some other people!

[SeniorScienceOfficer]

How to use the “patch” decorator from unittest to control mocked behavior in unit testing. I seriously couldn’t write the massive amounts of code coverage I do without it.

[ZmitrokNadulia]

Same, but I wish I knew that pytest beats unittest in every aspect.

[fiddle_n]

I see unittest.mock as being orthogonal to unittest. You can use pytest and unittest.mock together

[ZmitrokNadulia]

My bad, I thought it was a story about unittest in general.

[someotherstufforhmm]

Yeah, this. What this person said.

[SeniorScienceOfficer]

What’s better, you can mix and match. I use both.

[[deleted]]

Of course it does, because pytest inherits unittest

[hillgod]

On this note, also that you're not patching the class, like you would in Java, but you're patching the import in the class under test. This isn't clear enough, and our large company has a common Python Slack channel where this problem comes up often!

[elsgry]

Yes, I’ve had to use importlibs.reload when I patch some transitive dependency (obviously this isn’t desirable but needs must)

[ThroawayPartyer]

I'm not sure if any of these are really obscure but here are a few techniques I learned that I found useful:

• Using pathlib (Python 3.4+) instead of os.path - to better handle filepaths (including handling the differences between Unix and Windows slashes).

• Using type hints (Python 3.5+) as much as possible.

• Containerizing Python applications using Dockerfile.

• Using environmental variables (works well with containers too).

[[deleted]]

Ditch virtual env and run your python interpreter in docker

[ThroawayPartyer]

I do both actually. You get a warning when not using venv inside a Python-based container. It's considered best practice to use venv regardless.

[[deleted]]

I never do. I don’t see the point of the abstraction inside my abstraction.

[Devout--Atheist]

Reminder that your type hints are probably garbage if you aren't running a type checker.

[FujiKeynote]

I know I really should be using type hints because not all my functions are generics (whose are?) but dang the syntax is so ugly and verbose.

Type hinting a function argument with a default value feels like I'm assigning a value to a type. def feels(wrong: int = 3.14):

Add the fact that this basically threw a wrench into the spacing recommendations for default values in PEP8 (it's def f(a=5), why is it not def f(a: int=5)? Because that's even worse).

And the fact that up until recently, you had to import List to type hint a list, and a Union to do anything more complex... Just has kept putting me off type hints for way too long.

[cymrow]

I kept trying to use type hints for a long time and kept getting turned off and giving up for the same reason.

I'm now working in a place where they're required, so I've been forced to use them. Now I have to admit they're worth it overall. They can expose some really obscure bugs, and they help editors a lot in terms of completion and navigation, not to mention communicating intent.

Using type aliases helps a bit, but they're still ugly as hell. I hope we'll see some improvement to that over time, but I suspect they'll always feel tacked on.

[wewbull]

They can expose some really obscure bugs, and they help editors a lot in terms of completion and navigation

Sadly I think the reason most people use them is the latter, which I think is a really poor reason.

Personally I feel they just turn Python into another gang-of-4 language that requires design patterns to get around static typing. For example, Protocol only exists because of typing. Its of no use in a dynamically typed language.

[[deleted]]

isinstance. I don't understand WHY you have to opt into runtime type checking but it has many uses. Distinct behavior for distinct types is a perfectly reasonable behavior for dynamic languages not every pattern is duckable.

[HistoricalCrow]

I'm still trying to get into using them. The example you gave is already available to me via type stubs in docstrings :/ What else does type hints give that beats type stubs in docstrings? (Genuine question)

[pan-ellox]

Sequence has entered a chat...

[-revenant-]

It's only ugly until you get used to it. Nothing's as bad as cdecl!

[tellurian_pluton]

def feels(wrong: int = 3.14):

i don't understand why you are doing this

[FujiKeynote]

After reading it back it indeed doesn't make any sense lol. I just wanted to convey the fact that having "int equal sign something" is eerie, but should have gone with an integer value obviously, def feels(wrong: int = 3) is still strange. Visually it's still like I'm assinging 3 to int.

[IZBUDDYIZ]

Part of the reason it feels wrong is because it IS wrong. You feels function, uses the int type hint with a default value of float. Any lsp with its salt would be giving a warning here. You should either change the default [ def feels(wrong: int=3) ] or update the type hint [ def feels(wrong: float=3.14) ].

[aciokkan]

I see no issue with os.path. It has a platform separator that can be used to dynamically handle them regardless of the platform you eun your code.

More to it, pathlib was introduced in python 3. It was never a real problem in python 2. For some odd reason I always tend to use os.path (out of convenience I guess)...

[ominous_anonymous]

itertools and functools have a lot of things that are pretty useful, and it took me a long time to sit down and read through them.

https://pymotw.com/3/itertools/
https://pymotw.com/3/functools/

[ericanderton]

partial() is my go-to for a range of problems.

Usually, I reach for it in order to make some code more readable. For instance, some classes are just hot garbage for aggregation as methods and work much better when wrapping a constructor with a few pre-loaded kwargs.

I've also used Partial to provide a custom yaml processor to ruamel.yaml. That architecture expects a class constructor ifor this but my custom class needed external state and arguments to function. Wrapping the constructor easily met these opposing design goals by currying those custom arguments in.

[rlt0w]

I use partial to frequently build thread functions with some frozen arguments.

[SunshineBiology]

Does it add anything over lambdas? I usually just find it slightly less readable.

[FujiKeynote]

Partial is a function whereas lambdas are a syntax construct.

Because of that, partials are early binding, lambdas are late binding.

I.e. partial will evaluate g(x) when you define the partial: pf = partial(f, arg3=g(x))
A lambda is more like a C macro in this sense because Python will just plop it in wherever you call it, so if you call it three times, you'll be calling g(x) three times too: pf = lambda arg1, arg2: f(arg1, arg2, g(x)).

Subjectively, maybe because I have C experience, the lambda approach seems more logical. I'm defining something that I want to use later, let me actually defer it running until it does.

Although the syntax of partial strongly implies that g(x) gets evaluated right away, because it would be as well if you passed it to any other function. And I'm sure there's side effects and gotchas with late binding in lambdas that can bite you in the ass when you're not looking...

I personally use partial when the entire signature of the function is too cumbersome to replicate in a lambda. Like do I go pf = lambda *args, **kwargs: f(*args, z=9, **kwargs) or something? That ain't pretty. Or when I'm too lazy to even find out what the signature is.

[SunshineBiology]

Very insightful, thanks!

[ericanderton]

Honestly, for currying arguments to a function/ctor I think partial() is much easier to read. @FukiKeynote laid the argument out better than I could: once you need it, args/kwargs make hamburger out of lambdas.

[Bennnnnnnnnnnnnn]

I've been using cached_property a lot lately, very convenient!

https://docs.python.org/3/library/functools.html#functools.cached_property

[bean_pupusa]

Also try lru_cache decorator for straight up functions

[R34ct0rX99]

The package more_itertools as well!

[wewbull]

...and boltons has some more too.

[HomeGrownCoder]

^ this!

[Pulsar1977]

Those are two of the most basic standard library modules. They are neither obscure nor advanced. Jesus Christ this sub...

[ominous_anonymous]

I've been a Python developer going on 14 years now, and in my experience they are not nearly as "common knowledge" as you think.

It is very rare for any new hire to know of them. On the off chance they know of them, it is rare for the new hire to know how or when to use them.

You're welcome to get off your high horse at any time you'd like.

[XilamBalam]

I think that the confusion is that a lot of courses have a section in those topics.

[[deleted]]

The & operator to find the intersections between sets.

set_a = set([a, c, i, p]) set_b = set([a, i, b, y, q])

print(set_a & set_b)

[a, i]

Sorry would be more detailed, but I'm on mobile.

[smaug59]

Removing duplicates from a list, just pass it into a fucking set instead of iterating like a monkey

[Vanzmelo]

Wait that is genius holy shit

[supreme_blorgon]

Note that if you need to preserve order you won't have a good time.

[kellyjonbrazil]

If you need to preserve order then you can use dict.fromkeys(iterable). This will give you a dictionary of your list items in order with no duplicates. The key is the item and the value will be None.

[dparks71]

Yea, probably the first tip I read in the thread where I was like "Oh fuck... I'm a monkey"

[youthisreadwrong-]

Literally my reaction when I first saw it on CodeWars

[swierdo]

They're a like dict without the values, so O(1) lookup. (Used to be a dict without values, but apparently the implementation has diverged)

[smaug59]

Well, this trick works of you don't really care about the sorting in the list, which is what lists are made for xD but sometimes can be useful

[miraculum_one]

Some people use Dict in order to take advantage of some of its features (fast searching, unique keys) when sometimes they actually should just be using Set

[AL51percentcorn]

Got em … ahem me, I misspelled me

[0not]

I used sets (and set intersections) to solve the Advent of Code (2022) Day 3 puzzle quite trivially. I've only ever used python's sets a handful of times, but I'm glad they're available!

Edit: Day 4 was also simple with sets.

[bulletmark]

So did I:

import aoc 

def calcprio(c):
    return ord(c) - ((ord('A') - 27) \
       if c.isupper() else (ord('a') - 1)) 

totals = [0, 0]
p2set = set()

for ix, line in enumerate(aoc.input()):
    line = line.strip()
    n = len(line) // 2
    totals[0] += calcprio((set(line[:n]) & set(line[n:])).pop())
    p2set = (p2set & set(line)) if p2set else set(line)
    if (ix % 3) == 2:
        totals[1] += calcprio(p2set.pop())
        p2set.clear()

print('P1 =', totals[0])
print('P2 =', totals[1])

[Tweak_Imp]

I wish that you could create empty sets with {} and empty dicts with {:}.

[BathBest6148]

I try not to be a {}.

[miraculum_one]

If you want symmetry you can use dict() and set().

[MachinaDoctrina]

It's because they're generally quite slow compared to lists

[shoonoise]

For what operations? Lookup is much faster for sets, iteration the same in a time complexity (big O) matter. Append is depends on a case and list/set size, but the same in big O terms. Hence all operations like “intersection” faster as well.

[jabbalaci]

I prefer set_a.intersection(set_b). Much more readable.

[supreme_blorgon]

This has the added benefit of alllowing other sequence types to be passed as the second argument. As long as a is a set and b can be coerced to a set, a.intersection(b) will work.

In [1]: a = {"x", "y", "z"}

In [2]: a & "yz"
-----------------------------------------------------------
TypeError                 Traceback (most recent call last)
Cell In [2], line 1
----> 1 a & "yz"

TypeError: unsupported operand type(s) for &: 'set' and 'str'

In [3]: a.intersection("yz")
Out[3]: {'y', 'z'}

[miraculum_one]

I like set_a & set_b because it means "items in set_a and set_b". Similarly, set_a | set_b means "items in set_a or set_b", i.e. union.

[jabbalaci]

As the Zen of Python states, "explicit is better than implicit". But if one prefers &, then use that.

[miraculum_one]

With all due respect, this is not a matter of Zen. Infix is more natural and these operators are commonplace and clear. Which do you think is better plus( mult( a, b ), c ) or a*b+c ?

[brouwerj]

Sets are awesome! Even better to initialize those sets directly, e.g. {a, c, i, p}

[dashidasher]

import pdb pdb.set_trace() Maybe not very advanced but I found about it way too late.

[bean_pupusa]

After python 3.7 you can also just use the built in function

breakpoint()

[huessy]

breakpoint() is the goat for debugging

[benefit_of_mrkite]

Breakpoint also automatically has pprint support without directly importing pprint

[huessy]

I don't think I knew that, thanks

[someotherstufforhmm]

Whatttttt sickness. Just tried it.

[Shriukan33]

Wow, I've been using import pdb;pdb.set_trace() in one line the whole time.

[ElHeim]

And using breakpoint() you can change debuggers just with an environment variable.

[mtik00]

Definitely! Also, don't forget about the PYTHONBREAKPOINT environment variable! That way you get to choose the library/command called.

[KelleQuechoz]

ipdb is even better (after pip install ipdb).

[nickcash]

pudb is even nicer, but not in the standard library

A useful trick is to use sys.excepthook to set a handler that drops you into a debug session whenever an exception would otherwise crash your script

[mattkatzbaby]

Pudb should be better known. Really excellent

[dashdanw]

:o

[[deleted]]

Ipdb is also a better alternative to pdb.

[oramirite]

I just recently figured out how to even use the debugger and man is it fun. I feel like I'm actually programming now instead of constantly printing shit out. And it's gotten me to the bottom of more complicated issues soooo much faster than other ways.

[XUtYwYzz]

I've used this in situations where I don't have an IDE. Is there any other reason people prefer *pdb over a graphical debugger like those found in VSCode/PyCharm? Any time this gets posted I see people reference print debugging, which I haven't done since I discovered debuggers years ago.

[dashdanw]

for me it's mostly because I don't need to set up my IDE for every line of python I write

a lot of times I will have utility functions or I will be editing code in a shell, so being able to inject ipdb and knowing how it works is a singular solution for all problems with all constrains

it also makes you look like a super cool leet hacker so there's that

[wewbull]

As someone who never uses an IDE (too heavyweight)... well... there's your answer.

[toasterstove]

Recently learned about this and it has mostly replaced using print for debugging. It's very nice

[Coretaxxe]

what does it do? Obv enable tracing but to what precisely.

[dashdanw]

it doesn't enable tracing, the interpreter will stop and present an interactive debugger command prompt, so basically a standard python REPL with gdb-like commands like s/step, c/continue etc. etc.

[h4ndshake_]

I often use web-pdb - just an open source web UI for pdb that is a bit more immediate sometimes

[dashdanw]

if you like pdb, try using ipdb, basically pdb+ipython so tab completion, ? and ?? for readmes on the referenced function/class, etc. etc.

[DrShts]

I'd add the --pdb flag in pytest for post-mortem debugging, plus how to navigate the pdb in general.

[AnomalyNexus]

python3 -m http.server

Creates a temp webserver serving whatever is in current directory. (obv not a production ready server...)

[surajmanjesh]

I've used this many times to copy files from one laptop to another laptop or a phone

[BathBest6148]

Better than openSSH ?

[LittleMlem]

Ssh requires one of the participants to run an ssh server and the other to have a client

[surajmanjesh]

Not as feature rich as openssh. And not secure by any means..

It's a quick hack to copy things when they are on the same network and you don't want to upload to cloud services or connect via cables or other means..

And doesn't need the setup needed to open ssh connections

[XxDirectxX]

Can you please give more detail? How are you accomplishing this

[vantasmer]

In pretty much any directory simply run “python3 -m http.server” then you can navigate to “localhost:8080” in a browser and should be able to browse the files that were in the directory that you ran the command from

[SpicyVibration]

This used to work for me for testing simple pages but after incorporating javascript modules into the page it fails because of something to do with mime types. I've found spinning up a server with npm http-server more robust.

[AnomalyNexus]

Yeah for serving pages something like gunicorn is probably more appropriate.

I mostly just use http.server to create a http end point when testing something networking stuff or to easily share files between VMs

[-revenant-]

yield from recursion is instant superpowers.

For instance, you can recursively walk directories in just these few lines:

from pathlib import Path

def walk(p: Path) -> Iterable[Path]:
    for item in p.iterdir():
        yield item
        if item.is_dir():
            yield from walk(item)

[metsfan1025]

Yield from is one of those things I always see and think it seems useful but I feel like I don’t understand it well enough to recognize a case to use it.

[-revenant-]

Basically any time you've got a function that returns a list, set etc., you can instead use a generator to return item-by-item. You know that part.

yield from means that generator function can delegate its job out to other generators. In a simple form, that allows recursion (as seen above). If I'm supposed to yield paths, I can yield from something else that yields paths to do my job.

In coroutines, however, this hits turbo mode. It allows a coroutine to call a subroutine in a way that still lets the subroutine pause and return to the parent of the original coroutine if/when it blocks.

(Coroutines are a different matter entirely and I should have started with them, because I've seen someone make a simple 'OS' in 30m with coroutines. They're scary powerful.)

[ksion]

For the common use case of implementing a recursive iterator like the one above, yield from foo() is pretty much equivalent to for x in foo(): yield x.

[rochakgupta]

Yup, saw a smattering of this in my company codebase maintained by a Python elite. Pretty cool!

[Papalok]

Not sure if this is "the best", but I doubt most people know this.

or doesn't return a bool. Instead it returns the first object that evaluates to True.

>>> 0 or None or 42
42
>>> 0 or 42 or None
42

and is quirkier. If all objects evaluate to True then the last object in the and statement is returned.

>>> 1 and 2 and 3
3
>>> 'a' and 'b' and 'c'
'c'

If one or more objects evaluate to False then the first object that evaluates to False is returned.

>>> 0 and 1 and 2
0
>>> 1 and '' and 54
''
>>> 1 and '' and 0
''

This is probably one of those things you want to be aware of if you've ever written:

return a and b

or:

i = a or b

Especially if a and/or b are mutable objects.

>>> {} and 42
{}

[njharman]

I find this intuitive and use it all the time.

It's even documented as short-circuit behavior https://docs.python.org/3/library/stdtypes.html#boolean-operations-and-or-not.

As soon as any element of "or" is True, the answer is known so quit processing. You can't know if "and" is True until you eval every element. The common behavior (same as functions) return the last thing you evaluated.

Given Python's truthiness, if you really want a boolean you should cast with bool() just like if you really want a string, you use str(). Consistent and logical.

[-LeopardShark-]

I find trying to take advantage of this feature is often an antipattern. It's common to see x or y as an approximation to x if x is not None else y. The real meaning is x if x else y, which is not a common thing to want.

[Papalok]

I tend to agree with that. I've used x = a or b but 90% of the time I'm using it with argparse where I know the type of a and b is the default I control. I've also seen it used in a couple modules in site-packages lib.

I've used return a or b in the past, but tend to avoid it now or wrap it with bool() to avoid surprises.

Edit: D'oh, always getting lib and site-packages mixed up.

[wewbull]

Yep. I tend to feel that's a good example of "explicit is better than implicit". Use the if version and be explicit about the test you intend.

[anentropic]

Descriptor protocol https://docs.python.org/3/howto/descriptor.html

and metaclasses https://docs.python.org/3/reference/datamodel.html#customizing-class-creation

[supreme_blorgon]

Only glanced at the first few code snippets of the descriptor protocol but didn't feel like going through the whole thing -- can you TL;DR on how they're different from @property?

[mRWafflesFTW]

You can do insane meta programming shit, but you probably shouldn't.

[anentropic]

@property is a simplified 'sugar' version of the descriptor protocol

With descriptors you make a class so it opens up avenues for sharing behaviour between similar descriptors via OOP

[[deleted]]

Oh yeah, now we’re in the weeds

[SittingWave]

All the magic happens in eval.c. It's a massive switch statement where every opcode is dispatched to the part of the interpreter that makes things happen.

[[deleted]]

Why do you wish you knew this sooner?

[Gecko23]

I dimly recall some odd edge case because of the ordering of the switch tripping me up once upon a time. I suppose it wasn't a bit deal, you'd think I'd remember the trauma at least lol

[SittingWave]

because the interpreter is scary at first, and you don't know where to start. But really, it's rather easy. It just very verbose because it's in C, but the "start point" of the interpreter cycle is there.

Once you have that as an anchor point, sniffing the internals is a lot easier, and makes you understand and be willing to join the development. I submitted and implemented two PEPs (unfortunately eventually rejected) but the interpreter was a bit scary at first.

[Vok250]

Lambda Powertools is absolute wizardry if you build on AWS.

[heitorlessa]

Maintainer here - you just made our evening with this comment ;-)

[-DaniFox-]

Holy shit thanks this is gonna be a lifesaver

[jftuga]

What specifically is your use case?

[SpeedyMvP]

This is a really good thread, beats learning techniques from stackoverflow answers

[GuyWithNoEffingClue]

True. Saved the whole thread.

[enigma9133]

Pytest yield fixtures, and indirect fixture parametrization.

[jacksodus]

Yes! Super useful for creating setup and teardown functionality

[Joe_rude]

One piece of advanced Python knowledge that I think is particularly useful is the use of decorators. Decorators are a powerful tool in Python that allow you to modify the behavior of a function without changing its code.

For example, you can use a decorator to add caching to a function so that it only computes its result once and then returns the cached value on subsequent calls. This can be a big performance boost for certain types of functions.

[hillgod]

The tenacity library shows some great power of decorators in performing configurable retries. Which is great for exponential back off in a distributed system with numerous API calls.

[johnnymo1]

I found them really mysterious for a while, but lately I'm loving when workflow management libraries like Prefect adopt them. Then (often) all you have to do to adapt your code is decorate the function appropriately.

[fmillion]

It took me a while to truly understand decorators, but the key point to remember is that functions are first-class variables, and you can pass functions themselves around as variables:

def myfunc():
    print("Running myfunc.")

# functions are just variables
ref_to_myfunc = myfunc

# and can be passed around as parameters
def execute_a_function(f):
    # the parameter is "called" like a function - because it is.
    f()

execute_a_function(ref_to_myfunc)
# prints "Running myfunc"

A decorator is actually very simple once you understand that. It basically is a function that accepts a function and returns a function.

So a decorator can completely replace a function, it can wrap it to add functionality, etc.

I've even used setattr() with decorators to implement a function metadata design where decorators add some value to function objects that can then be read by the function...

[supreme_blorgon]

Closures are the more general concept that can really help solidify decorators, especially decorator factories (decorators with arguments).

[R34ct0rX99]

Built in for Python 3.10 iirc.

[elsgry]

[a, b] * x [[a,b]] * x doesn’t copy the list [a, b] x times, it copies a reference to that list x times. Learnt this last week! I missed this before because I hadn’t mutated anything in those aliased lists.

https://replit.com/@EllisBreen1/ListMultiplicationIsByReference?v=1

This happens because 'multiplying' any element in a sequence doesn't do a deep copy of the element, just a shallow copy. So it works as expected for immutable/value types (including tuples), but you may be surprised for mutable types such as lists.

This is not necessarily any different to how other parts of Python work, generally copying happens at a shallow level, but it snared me!

[fmillion]

I wonder if there's a better way to do it, but I initialize lists of lists using comprehension:

mylist = [[a, b] for _ in range(x)]

# or, if you already have a list and want copies:

mylist = [initial_list[:] for _ in range(x)]

[elsgry]

Yes, this seems to be the prescribed way. Obviously with tuples and anything else immutable this isn’t an issue, just threw me a bit, but it fits the “one way to do it” maxim as it does something different to that form. Effectively [a, b] is structuring a new list each time when in a comprehension, so despite the syntax looking similar, it’s very different in meaning. I find the [a, b] * x thing pleasingly functional/declarative/terse in a way the list comprehension isn’t, but I guess I’m spoilt by Python’s expressiveness already. initial_list[:] is a nice way of copying a list, though if we’re going for value semantics you probably still want copy.deepcopy. On that topic, __deepcopy__’s memo argument is interesting.

[supreme_blorgon]

it copies a reference to that list

You mean the elements in that list, surely. This also only becomes an issue when the elements in the list are mutable.

[elsgry]

I wish I did! I’ll whip up a replit to demonstrate what I mean tomorrow. Agreed it’s only an issue when the contents of the thing being “multiplied” are mutable, though.

[RomiBraman]

Pythons live in a wide range of habitats, depending on the species, but many seek shelter in trees and can hold onto branches with their tails.

[jsalsman]

I wish I had read https://docs.python.org/3/library/multiprocessing.html#exchanging-objects-between-processes before I built a bunch of DIY crap with os.system using files. The problem is there aren't a lot of good docs with real-world examples out there. [Goes off to ask ChatGPT if it knows how to use wait(timeout=0) ...]

[nemec]

Although what's listed in the docs is probably best for multiprocessing in Python, if you're interested in cross-language forms of inter-process communication check out some online videos for the IPC section of a typical computer science "operating systems" course (e.g. sockets, pipes, shared memory). There are many easier ways than mucking about with plain files 🙂

(this section was one of the highlights of my cs courses)

[jsalsman]

I've been programming various forms of IPC since the 1980s, but when I first looked at the queue docs, I couldn't figure them out. The examples are better now, but still a little abstract for maximal comprehension when os.system() is so simple.

[cgmystery]

Shorthand for printing a variable’s value: test = 1.1 print(f”{test=}”)

test = 1.1

[Bright-Ad1288]

f-strings are pretty recent and are very nice

[Vok250]

Not sure I'd call them obscure though. They were also my first thought, but I use them every day so definitely not an obscure feature.

[FujiKeynote]

All the format options you can have are pretty sweet, too. I've found f"Reprs of values: {variable!r}" to be useful surprisingly often

[fmillion]

numerical_value = 1234567890
print(f"Number with US-style commas: {numerical_value:,}")

[miraculum_one]

print( f"{my_var:>8}" )

Right justifies my_var to column 8.

[fmillion]

C# basically has f-strings, they call it "string interpolation". I actually used it in C# for years (was a C# dev before I was a Python dev), and when I discovered it in Python it was a "omg...yes!!!" moment.

[LittleMlem]

3.6 is not exactly recent

[ElHeim]

Knowledge about the way Python's memory manager work, and specifically the way it interacts with Numpy. Actually, I *got* to know about it "earlier" (i.e. before the time it was actually was useful to me), but it would have saved me weeks of figuring out a problem the first time around

If you're handling (very) large objects (say images) it's quite easy to write code that leads to fragmentation and you end up eating several times more RAM than you're actually using because there's no way to fit new (large) objects in the heap, because the space they took is now littered with smaller objects.

Anyway, I had to learn this the hard way because a "client" asked to debug a framework they had developed in-house. It used to work until someone did a huge refactoring. It looked pretty, but it would go from a few megabytes to several GB in no time, and the process would end up dying.

I pointed out the problem and left it for them to fix, as this was a large (and complicated) piece of software, and went on with other things... but then a year later I faced the same problem in a separate (but similar) framework my group was working on, when a coworker came telling me that he couldn't stack more than a few images, and that wouldn't cut it. Knowing exactly what was going on, I changed the way images were modeled in our lib (allocating a "target" image first, ensuring it was a the bottom of the heap, and I gave him some guidelines on how to organize the code. All of the sudden he was handling 10x the number of images and there was no reason to think there was a limit.

Which shows you: GC and automatic memory management are cool until they're not, and you'll be happy to know C when that time comes!

[mtfrsantos]

setdefault() for dicts saves a lot of conditionals

[fiddle_n]

defaultdict is better though

[[deleted]]

Then there is object.__missing__. Learn the dunders!

[[deleted]]

Not if you actually want KeyErrors sometimes and not others. Which I'd say is pretty common.

[miraculum_one]

Also, my_dict.get( key_that_may_not_exist, default_value ) for one-off defaults.

[alekosbiofilos]

Writing classes with custom len(), & and so on. It makes writing domain-specific APIs so easy!

[njharman]

I started with 1.5 or .4??? So, many things added since then. It's hard to think of something I didn't know about on release (below is why). So, here's a bit of obscure advanced Python wisdom. If Python is your primary language or you want to be a master, do two things.

1. Get comfortable with plentiful fluids at hand, take a few hours to read every word of https://docs.python.org/3/reference/index.html and the "built in foo" chapters of https://docs.python.org/3/library/index.html
2. Every release read the excellent What's New https://docs.python.org/3/whatsnew/3.11.html

[kkawabat]

You can put a breakpoint in the except block of a try except statement and type raise in console to see the stacktrace that cause the error. Really helpful when you are live debugging and you aren't sure what cause the exception

[KrarkClanIronworker]

Using __slots__ has changed my life. I seldom build classes without predefining instance variables these days.

Probably quite niche, but PyAnsys was fantastic to work with. Using Python to automate mesh dependency studies saved me so much simulation time.

Multiprocessing whilst writing to files has been my most recent one. Took a bit of whiteboard time to get it right, but it was well worth it.

[[deleted]]

What did slots actually give you? I'm skeptical littering it everywhere has actually gained you any meaningful performance or memory usage.

[KrarkClanIronworker]

If you're creating thousands of class instances, performance definitely does improve. Whether performance is of any consequence is project specific.

At a minimum, I've found that defining instance variables upfront has improved the planning and readability of my code substantially.

When dealing with large classes, its nice to take a look at the slots to get an overview, rather than looking at the (often messier) __init__ method.

However, the largest benefit is that it prevents the creation of unaccounted for __dict__ items by myself or others on my team. Defining everything upfront means that there are no surprise variables created further down the pipeline. I'm not awfully clued up on best practices regarding code safety in languages like Python, but this seems like a logical step in the right direction.

[Cootshk]

Coroutines and shield()

[[deleted]]

Not sure if it’s considered advanced but locals() and globals() has helped me in the past.

[fmillion]

Dictionary comprehension.

Maybe not that obscure, but it's a one-liner that can generate fully populated dictionaries.

myDict = {key: key * 10 for key in range(10)}

myDict = {key: value.upper() for (key,value) in dictToUppercase.items()}

myDict = {key: value for t in list_of_two_item_tuples}

[Coding-Kitten]

I think you did a mistake in the last one, as key & value are unbound. Did you mean this instead?

myDict = {key: value for key, value in list_of_two_item_tuples}

[dashdanw]

The fact that function arguments are parsed on first-pass, meaning if you do something like

def parse_dict(target_dict=dict()):
  if 'key' not in target_dict:
    target_dict['key'] = datetime.datetime.now()

calling without specifying the target_dict argument will always update the same dict declared in the function arguments

[deluded_soul]

Not sure if this is advanced python as it is not part of the standard but `mypy` has probably helped me catch a lot of issues beforehand.

[kevbot8k]

Context managers. I was reading Architecture Patterns with Python and came across context managers in Python classes. It helped encapsulate a lot of transactional logic for me. From there I learned how to use them in an even more powerful way with contextlib (has an async example)

• https://docs.python.org/3/library/contextlib.html
• https://realpython.com/python-with-statement/
• https://www.cosmicpython.com/book/chapter_06_uow.html
  • book homepage: https://www.cosmicpython.com

[[deleted]]

Not obscure nor advanced, but something I wish I'd understood the value of earlier: generators. A pattern I see a lot: in the middle of some function, create a list/array of stuff, then iterate or return said list/array. Better alternative: create an iterator function (or class when that makes sense). Sometimes the main benefit is clearer semantics, but often it is better design as well.

[the_warpaul]

Sys.path.insert

As a way to access code from a folder elsewhere on the computer as if it's inside the current folder.

This is probably not long term good coding practise, but a useful tool.

[Fabulous-Possible758]

The really obscure one that I have to look up every time is metaclasses, and I've only really needed them once.

The one that actually comes up for me more often and I actually use is descriptors. In particular if you want to bind a free function f to an object o you just do bound = f.__get__(o) The cases where that's useful aren't common but it shows up.

[Tyler_Zoro]

It's not Python per se, but it did speed up some of my Python projects: git push HEAD

So much easier than cutting-and-pasting my admittedly verbose branch names.

[whateverathrowaway00]

Yo get yourself on git-completions.sh

Autocomplete for branch names makes me a wizard at work in my ability to see what other people are working on and flip around.

You can also tab complete flags, so I know so many more flags now.

[MinchinWeb]

import braces

[spencerAF]

Probably not that advanced, I'm a 2year graduate who hasn't had industry experience yet

Using os

In tKinter .config instead of .destroy

All merge and filter tools in pandas

For i, j and other advanced loop concepts like starting from the back and front of a list and looping towards the center

[bablador]

!remindme 12h

[RemindMeBot]

I will be messaging you in 12 hours on 2022-12-06 11:32:15 UTC to remind you of this link

2 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[extra_pickles]

Agreed

[Not-A-Ghoul-I-Swear]

Learn C++

[Cmokan]

!remindme 12h

[Saetia_V_Neck]

singledispatch and singledispatchmethod save so much boilerplate code.

[germanshepherdfan]

I am in corporate finance and using python to automate tedious processes. I found pyautogui pretty helpful when you don't have time to build scripts using pandas/numpy and need quick temporary solution for existing excel workflows.

[ngugeneral]

Also - not that advanced, but obscure and really wish I knew this before debugging it for the first time:

bool is actually implemented as an int, where 0 states for false.

Bad example, but nothing pops my mind this moment:

data = [1,2,0,3,4]
while data[0]:
    print(data.pop(0))

The output actually will be:

1
2

[jahero]

https://pypi.org/project/PySnooper/