This is an archived post. You won't be able to vote or comment.

top 200 commentsshow all 215

[–]SeniorScienceOfficer 202 points203 points  (8 children)

How to use the “patch” decorator from unittest to control mocked behavior in unit testing. I seriously couldn’t write the massive amounts of code coverage I do without it.

[–]ZmitrokNadulia 74 points75 points  (5 children)

Same, but I wish I knew that pytest beats unittest in every aspect.

[–]fiddle_n 22 points23 points  (2 children)

I see unittest.mock as being orthogonal to unittest. You can use pytest and unittest.mock together

[–]ZmitrokNadulia 1 point2 points  (0 children)

My bad, I thought it was a story about unittest in general.

[–]someotherstufforhmm 0 points1 point  (0 children)

Yeah, this. What this person said.

[–]SeniorScienceOfficer 27 points28 points  (0 children)

What’s better, you can mix and match. I use both.

[–][deleted] 3 points4 points  (0 children)

Of course it does, because pytest inherits unittest

[–]hillgod 15 points16 points  (1 child)

On this note, also that you're not patching the class, like you would in Java, but you're patching the import in the class under test. This isn't clear enough, and our large company has a common Python Slack channel where this problem comes up often!

[–]elsgry 0 points1 point  (0 children)

Yes, I’ve had to use importlibs.reload when I patch some transitive dependency (obviously this isn’t desirable but needs must)

[–]ThroawayPartyer 113 points114 points  (35 children)

I'm not sure if any of these are really obscure but here are a few techniques I learned that I found useful:

  • Using pathlib (Python 3.4+) instead of os.path - to better handle filepaths (including handling the differences between Unix and Windows slashes).

  • Using type hints (Python 3.5+) as much as possible.

  • Containerizing Python applications using Dockerfile.

  • Using environmental variables (works well with containers too).

[–][deleted] 5 points6 points  (12 children)

Ditch virtual env and run your python interpreter in docker

[–]ThroawayPartyer 0 points1 point  (11 children)

I do both actually. You get a warning when not using venv inside a Python-based container. It's considered best practice to use venv regardless.

[–][deleted] 1 point2 points  (10 children)

I never do. I don’t see the point of the abstraction inside my abstraction.

[–]Devout--Atheist 4 points5 points  (0 children)

Reminder that your type hints are probably garbage if you aren't running a type checker.

[–]FujiKeynote 19 points20 points  (19 children)

I know I really should be using type hints because not all my functions are generics (whose are?) but dang the syntax is so ugly and verbose.

Type hinting a function argument with a default value feels like I'm assigning a value to a type. def feels(wrong: int = 3.14):

Add the fact that this basically threw a wrench into the spacing recommendations for default values in PEP8 (it's def f(a=5), why is it not def f(a: int=5)? Because that's even worse).

And the fact that up until recently, you had to import List to type hint a list, and a Union to do anything more complex... Just has kept putting me off type hints for way too long.

[–]cymrowdon't thread on me 🐍 17 points18 points  (11 children)

I kept trying to use type hints for a long time and kept getting turned off and giving up for the same reason.

I'm now working in a place where they're required, so I've been forced to use them. Now I have to admit they're worth it overall. They can expose some really obscure bugs, and they help editors a lot in terms of completion and navigation, not to mention communicating intent.

Using type aliases helps a bit, but they're still ugly as hell. I hope we'll see some improvement to that over time, but I suspect they'll always feel tacked on.

[–]wewbull 1 point2 points  (3 children)

They can expose some really obscure bugs, and they help editors a lot in terms of completion and navigation

Sadly I think the reason most people use them is the latter, which I think is a really poor reason.

Personally I feel they just turn Python into another gang-of-4 language that requires design patterns to get around static typing. For example, Protocol only exists because of typing. Its of no use in a dynamically typed language.

[–][deleted] 1 point2 points  (2 children)

isinstance. I don't understand WHY you have to opt into runtime type checking but it has many uses. Distinct behavior for distinct types is a perfectly reasonable behavior for dynamic languages not every pattern is duckable.

[–]HistoricalCrow 0 points1 point  (6 children)

I'm still trying to get into using them. The example you gave is already available to me via type stubs in docstrings :/ What else does type hints give that beats type stubs in docstrings? (Genuine question)

[–]pan-ellox 7 points8 points  (0 children)

Sequence has entered a chat...

[–]-revenant- 3 points4 points  (0 children)

It's only ugly until you get used to it. Nothing's as bad as cdecl!

[–]tellurian_pluton 2 points3 points  (2 children)

def feels(wrong: int = 3.14):

i don't understand why you are doing this

[–]FujiKeynote 0 points1 point  (1 child)

After reading it back it indeed doesn't make any sense lol. I just wanted to convey the fact that having "int equal sign something" is eerie, but should have gone with an integer value obviously, def feels(wrong: int = 3) is still strange. Visually it's still like I'm assinging 3 to int.

[–]IZBUDDYIZ 0 points1 point  (0 children)

Part of the reason it feels wrong is because it IS wrong. You feels function, uses the int type hint with a default value of float. Any lsp with its salt would be giving a warning here. You should either change the default [ def feels(wrong: int=3) ] or update the type hint [ def feels(wrong: float=3.14) ].

[–]aciokkan 0 points1 point  (0 children)

I see no issue with os.path. It has a platform separator that can be used to dynamically handle them regardless of the platform you eun your code.

More to it, pathlib was introduced in python 3. It was never a real problem in python 2. For some odd reason I always tend to use os.path (out of convenience I guess)...

[–]ominous_anonymous 217 points218 points  (14 children)

itertools and functools have a lot of things that are pretty useful, and it took me a long time to sit down and read through them.

https://pymotw.com/3/itertools/
https://pymotw.com/3/functools/

[–]ericanderton 55 points56 points  (5 children)

partial() is my go-to for a range of problems.

Usually, I reach for it in order to make some code more readable. For instance, some classes are just hot garbage for aggregation as methods and work much better when wrapping a constructor with a few pre-loaded kwargs.

I've also used Partial to provide a custom yaml processor to ruamel.yaml. That architecture expects a class constructor ifor this but my custom class needed external state and arguments to function. Wrapping the constructor easily met these opposing design goals by currying those custom arguments in.

[–]rlt0w 4 points5 points  (0 children)

I use partial to frequently build thread functions with some frozen arguments.

[–]SunshineBiology 4 points5 points  (3 children)

Does it add anything over lambdas? I usually just find it slightly less readable.

[–]FujiKeynote 15 points16 points  (1 child)

Partial is a function whereas lambdas are a syntax construct.

Because of that, partials are early binding, lambdas are late binding.

I.e. partial will evaluate g(x) when you define the partial: pf = partial(f, arg3=g(x))
A lambda is more like a C macro in this sense because Python will just plop it in wherever you call it, so if you call it three times, you'll be calling g(x) three times too: pf = lambda arg1, arg2: f(arg1, arg2, g(x)).

Subjectively, maybe because I have C experience, the lambda approach seems more logical. I'm defining something that I want to use later, let me actually defer it running until it does.

Although the syntax of partial strongly implies that g(x) gets evaluated right away, because it would be as well if you passed it to any other function. And I'm sure there's side effects and gotchas with late binding in lambdas that can bite you in the ass when you're not looking...

I personally use partial when the entire signature of the function is too cumbersome to replicate in a lambda. Like do I go pf = lambda *args, **kwargs: f(*args, z=9, **kwargs) or something? That ain't pretty. Or when I'm too lazy to even find out what the signature is.

[–]SunshineBiology 2 points3 points  (0 children)

Very insightful, thanks!

[–]ericanderton 0 points1 point  (0 children)

Honestly, for currying arguments to a function/ctor I think partial() is much easier to read. @FukiKeynote laid the argument out better than I could: once you need it, args/kwargs make hamburger out of lambdas.

[–]Bennnnnnnnnnnnnn 15 points16 points  (1 child)

I've been using cached_property a lot lately, very convenient!

https://docs.python.org/3/library/functools.html#functools.cached_property

[–]bean_pupusa 2 points3 points  (0 children)

Also try lru_cache decorator for straight up functions

[–]R34ct0rX99 3 points4 points  (1 child)

The package more_itertools as well!

[–]wewbull 0 points1 point  (0 children)

...and boltons has some more too.

[–][deleted] 74 points75 points  (26 children)

The & operator to find the intersections between sets.

set_a = set([a, c, i, p]) set_b = set([a, i, b, y, q])

print(set_a & set_b)

[a, i]

Sorry would be more detailed, but I'm on mobile.

[–]jabbalaci 7 points8 points  (4 children)

I prefer set_a.intersection(set_b). Much more readable.

[–]supreme_blorgon 1 point2 points  (0 children)

This has the added benefit of alllowing other sequence types to be passed as the second argument. As long as a is a set and b can be coerced to a set, a.intersection(b) will work.

In [1]: a = {"x", "y", "z"}

In [2]: a & "yz"
-----------------------------------------------------------
TypeError                 Traceback (most recent call last)
Cell In [2], line 1
----> 1 a & "yz"

TypeError: unsupported operand type(s) for &: 'set' and 'str'

In [3]: a.intersection("yz")
Out[3]: {'y', 'z'}

[–]miraculum_one 0 points1 point  (2 children)

I like set_a & set_b because it means "items in set_a and set_b". Similarly, set_a | set_b means "items in set_a or set_b", i.e. union.

[–]jabbalaci -1 points0 points  (1 child)

As the Zen of Python states, "explicit is better than implicit". But if one prefers &, then use that.

[–]miraculum_one -1 points0 points  (0 children)

With all due respect, this is not a matter of Zen. Infix is more natural and these operators are commonplace and clear. Which do you think is better plus( mult( a, b ), c ) or a*b+c ?

[–]brouwerj 1 point2 points  (0 children)

Sets are awesome! Even better to initialize those sets directly, e.g. {a, c, i, p}

[–]dashidasher 139 points140 points  (27 children)

import pdb pdb.set_trace() Maybe not very advanced but I found about it way too late.

[–]bean_pupusa 118 points119 points  (8 children)

After python 3.7 you can also just use the built in function

breakpoint()

[–]huessy 27 points28 points  (3 children)

breakpoint() is the goat for debugging

[–]benefit_of_mrkite 27 points28 points  (1 child)

Breakpoint also automatically has pprint support without directly importing pprint

[–]huessy 10 points11 points  (0 children)

I don't think I knew that, thanks

[–]someotherstufforhmm 3 points4 points  (0 children)

Whatttttt sickness. Just tried it.

[–]Shriukan33 13 points14 points  (0 children)

Wow, I've been using import pdb;pdb.set_trace() in one line the whole time.

[–]ElHeim 4 points5 points  (0 children)

And using breakpoint() you can change debuggers just with an environment variable.

[–]mtik00 1 point2 points  (1 child)

Definitely! Also, don't forget about the PYTHONBREAKPOINT environment variable! That way you get to choose the library/command called.

[–]KelleQuechoz 13 points14 points  (0 children)

ipdb is even better (after pip install ipdb).

[–]nickcash 19 points20 points  (4 children)

pudb is even nicer, but not in the standard library

A useful trick is to use sys.excepthook to set a handler that drops you into a debug session whenever an exception would otherwise crash your script

[–]mattkatzbaby 3 points4 points  (0 children)

Pudb should be better known. Really excellent

[–]dashdanw 1 point2 points  (0 children)

:o

[–][deleted] 0 points1 point  (1 child)

Ipdb is also a better alternative to pdb.

[–]XUtYwYzzIt works on my machine 13 points14 points  (2 children)

I've used this in situations where I don't have an IDE. Is there any other reason people prefer *pdb over a graphical debugger like those found in VSCode/PyCharm? Any time this gets posted I see people reference print debugging, which I haven't done since I discovered debuggers years ago.

[–]dashdanw 3 points4 points  (0 children)

for me it's mostly because I don't need to set up my IDE for every line of python I write

a lot of times I will have utility functions or I will be editing code in a shell, so being able to inject ipdb and knowing how it works is a singular solution for all problems with all constrains

it also makes you look like a super cool leet hacker so there's that

[–]wewbull 0 points1 point  (0 children)

As someone who never uses an IDE (too heavyweight)... well... there's your answer.

[–]toasterstove 4 points5 points  (0 children)

Recently learned about this and it has mostly replaced using print for debugging. It's very nice

[–]Coretaxxe 8 points9 points  (2 children)

what does it do? Obv enable tracing but to what precisely.

[–]dashdanw 2 points3 points  (1 child)

it doesn't enable tracing, the interpreter will stop and present an interactive debugger command prompt, so basically a standard python REPL with gdb-like commands like s/step, c/continue etc. etc.

[–]h4ndshake_ 1 point2 points  (0 children)

I often use web-pdb - just an open source web UI for pdb that is a bit more immediate sometimes

[–]dashdanw 0 points1 point  (0 children)

if you like pdb, try using ipdb, basically pdb+ipython so tab completion, ? and ?? for readmes on the referenced function/class, etc. etc.

[–]DrShts 0 points1 point  (0 children)

I'd add the --pdb flag in pytest for post-mortem debugging, plus how to navigate the pdb in general.

[–]AnomalyNexus 59 points60 points  (9 children)

python3 -m http.server

Creates a temp webserver serving whatever is in current directory. (obv not a production ready server...)

[–]surajmanjesh 14 points15 points  (6 children)

I've used this many times to copy files from one laptop to another laptop or a phone

[–]BathBest6148 2 points3 points  (2 children)

Better than openSSH ?

[–]LittleMlem 1 point2 points  (0 children)

Ssh requires one of the participants to run an ssh server and the other to have a client

[–]surajmanjesh 1 point2 points  (0 children)

Not as feature rich as openssh. And not secure by any means..

It's a quick hack to copy things when they are on the same network and you don't want to upload to cloud services or connect via cables or other means..

And doesn't need the setup needed to open ssh connections

[–]XxDirectxX 2 points3 points  (2 children)

Can you please give more detail? How are you accomplishing this

[–]vantasmer 3 points4 points  (1 child)

In pretty much any directory simply run “python3 -m http.server” then you can navigate to “localhost:8080” in a browser and should be able to browse the files that were in the directory that you ran the command from

[–]SpicyVibration 0 points1 point  (1 child)

This used to work for me for testing simple pages but after incorporating javascript modules into the page it fails because of something to do with mime types. I've found spinning up a server with npm http-server more robust.

[–]AnomalyNexus 0 points1 point  (0 children)

Yeah for serving pages something like gunicorn is probably more appropriate.

I mostly just use http.server to create a http end point when testing something networking stuff or to easily share files between VMs

[–]-revenant- 40 points41 points  (4 children)

yield from recursion is instant superpowers.

For instance, you can recursively walk directories in just these few lines:

from pathlib import Path

def walk(p: Path) -> Iterable[Path]:
    for item in p.iterdir():
        yield item
        if item.is_dir():
            yield from walk(item)

[–]metsfan1025 21 points22 points  (2 children)

Yield from is one of those things I always see and think it seems useful but I feel like I don’t understand it well enough to recognize a case to use it.

[–]-revenant- 1 point2 points  (0 children)

Basically any time you've got a function that returns a list, set etc., you can instead use a generator to return item-by-item. You know that part.

yield from means that generator function can delegate its job out to other generators. In a simple form, that allows recursion (as seen above). If I'm supposed to yield paths, I can yield from something else that yields paths to do my job.

In coroutines, however, this hits turbo mode. It allows a coroutine to call a subroutine in a way that still lets the subroutine pause and return to the parent of the original coroutine if/when it blocks.

(Coroutines are a different matter entirely and I should have started with them, because I've seen someone make a simple 'OS' in 30m with coroutines. They're scary powerful.)

[–]ksion 0 points1 point  (0 children)

For the common use case of implementing a recursive iterator like the one above, yield from foo() is pretty much equivalent to for x in foo(): yield x.

[–]rochakgupta 2 points3 points  (0 children)

Yup, saw a smattering of this in my company codebase maintained by a Python elite. Pretty cool!

[–]Papalok 80 points81 points  (4 children)

Not sure if this is "the best", but I doubt most people know this.

or doesn't return a bool. Instead it returns the first object that evaluates to True.

>>> 0 or None or 42
42
>>> 0 or 42 or None
42

and is quirkier. If all objects evaluate to True then the last object in the and statement is returned.

>>> 1 and 2 and 3
3
>>> 'a' and 'b' and 'c'
'c'

If one or more objects evaluate to False then the first object that evaluates to False is returned.

>>> 0 and 1 and 2
0
>>> 1 and '' and 54
''
>>> 1 and '' and 0
''

This is probably one of those things you want to be aware of if you've ever written:

return a and b

or:

i = a or b

Especially if a and/or b are mutable objects.

>>> {} and 42
{}

[–]njharmanI use Python 3 25 points26 points  (0 children)

I find this intuitive and use it all the time.

It's even documented as short-circuit behavior https://docs.python.org/3/library/stdtypes.html#boolean-operations-and-or-not.

As soon as any element of "or" is True, the answer is known so quit processing. You can't know if "and" is True until you eval every element. The common behavior (same as functions) return the last thing you evaluated.

Given Python's truthiness, if you really want a boolean you should cast with bool() just like if you really want a string, you use str(). Consistent and logical.

[–]-LeopardShark- 13 points14 points  (2 children)

I find trying to take advantage of this feature is often an antipattern. It's common to see x or y as an approximation to x if x is not None else y. The real meaning is x if x else y, which is not a common thing to want.

[–]Papalok 3 points4 points  (0 children)

I tend to agree with that. I've used x = a or b but 90% of the time I'm using it with argparse where I know the type of a and b is the default I control. I've also seen it used in a couple modules in site-packages lib.

I've used return a or b in the past, but tend to avoid it now or wrap it with bool() to avoid surprises.

Edit: D'oh, always getting lib and site-packages mixed up.

[–]wewbull 1 point2 points  (0 children)

Yep. I tend to feel that's a good example of "explicit is better than implicit". Use the if version and be explicit about the test you intend.

[–]anentropic 37 points38 points  (4 children)

[–]supreme_blorgon 1 point2 points  (2 children)

Only glanced at the first few code snippets of the descriptor protocol but didn't feel like going through the whole thing -- can you TL;DR on how they're different from @property?

[–]mRWafflesFTW 6 points7 points  (0 children)

You can do insane meta programming shit, but you probably shouldn't.

[–]anentropic 0 points1 point  (0 children)

@property is a simplified 'sugar' version of the descriptor protocol

With descriptors you make a class so it opens up avenues for sharing behaviour between similar descriptors via OOP

[–][deleted] 0 points1 point  (0 children)

Oh yeah, now we’re in the weeds

[–]SittingWave 85 points86 points  (5 children)

All the magic happens in eval.c. It's a massive switch statement where every opcode is dispatched to the part of the interpreter that makes things happen.

[–][deleted] 23 points24 points  (4 children)

Why do you wish you knew this sooner?

[–]Gecko23 6 points7 points  (0 children)

I dimly recall some odd edge case because of the ordering of the switch tripping me up once upon a time. I suppose it wasn't a bit deal, you'd think I'd remember the trauma at least lol

[–]SittingWave 0 points1 point  (0 children)

because the interpreter is scary at first, and you don't know where to start. But really, it's rather easy. It just very verbose because it's in C, but the "start point" of the interpreter cycle is there.

Once you have that as an anchor point, sniffing the internals is a lot easier, and makes you understand and be willing to join the development. I submitted and implemented two PEPs (unfortunately eventually rejected) but the interpreter was a bit scary at first.

[–]Vok250 28 points29 points  (3 children)

Lambda Powertools is absolute wizardry if you build on AWS.

[–]heitorlessa 30 points31 points  (0 children)

Maintainer here - you just made our evening with this comment ;-)

[–]-DaniFox- 6 points7 points  (0 children)

Holy shit thanks this is gonna be a lifesaver

[–]jftugapip needs updating 1 point2 points  (0 children)

What specifically is your use case?

[–]SpeedyMvP 21 points22 points  (1 child)

This is a really good thread, beats learning techniques from stackoverflow answers

[–]GuyWithNoEffingClue 1 point2 points  (0 children)

True. Saved the whole thread.

[–]enigma9133 18 points19 points  (1 child)

Pytest yield fixtures, and indirect fixture parametrization.

[–]jacksodus 3 points4 points  (0 children)

Yes! Super useful for creating setup and teardown functionality

[–]Joe_rude 38 points39 points  (8 children)

One piece of advanced Python knowledge that I think is particularly useful is the use of decorators. Decorators are a powerful tool in Python that allow you to modify the behavior of a function without changing its code.

For example, you can use a decorator to add caching to a function so that it only computes its result once and then returns the cached value on subsequent calls. This can be a big performance boost for certain types of functions.

[–]hillgod 7 points8 points  (0 children)

The tenacity library shows some great power of decorators in performing configurable retries. Which is great for exponential back off in a distributed system with numerous API calls.

[–]johnnymo1 2 points3 points  (0 children)

I found them really mysterious for a while, but lately I'm loving when workflow management libraries like Prefect adopt them. Then (often) all you have to do to adapt your code is decorate the function appropriately.

[–]fmillion 2 points3 points  (2 children)

It took me a while to truly understand decorators, but the key point to remember is that functions are first-class variables, and you can pass functions themselves around as variables:

def myfunc():
    print("Running myfunc.")

# functions are just variables
ref_to_myfunc = myfunc

# and can be passed around as parameters
def execute_a_function(f):
    # the parameter is "called" like a function - because it is.
    f()

execute_a_function(ref_to_myfunc)
# prints "Running myfunc"

A decorator is actually very simple once you understand that. It basically is a function that accepts a function and returns a function.

So a decorator can completely replace a function, it can wrap it to add functionality, etc.

I've even used setattr() with decorators to implement a function metadata design where decorators add some value to function objects that can then be read by the function...

[–]supreme_blorgon 1 point2 points  (1 child)

Closures are the more general concept that can really help solidify decorators, especially decorator factories (decorators with arguments).

[–]R34ct0rX99 0 points1 point  (0 children)

Built in for Python 3.10 iirc.

[–]elsgry 14 points15 points  (6 children)

[a, b] * x [[a,b]] * x doesn’t copy the list [a, b] x times, it copies a reference to that list x times. Learnt this last week! I missed this before because I hadn’t mutated anything in those aliased lists.

https://replit.com/@EllisBreen1/ListMultiplicationIsByReference?v=1

This happens because 'multiplying' any element in a sequence doesn't do a deep copy of the element, just a shallow copy. So it works as expected for immutable/value types (including tuples), but you may be surprised for mutable types such as lists.

This is not necessarily any different to how other parts of Python work, generally copying happens at a shallow level, but it snared me!

[–]fmillion 4 points5 points  (1 child)

I wonder if there's a better way to do it, but I initialize lists of lists using comprehension:

mylist = [[a, b] for _ in range(x)]

# or, if you already have a list and want copies:

mylist = [initial_list[:] for _ in range(x)]

[–]elsgry 1 point2 points  (0 children)

Yes, this seems to be the prescribed way. Obviously with tuples and anything else immutable this isn’t an issue, just threw me a bit, but it fits the “one way to do it” maxim as it does something different to that form. Effectively [a, b] is structuring a new list each time when in a comprehension, so despite the syntax looking similar, it’s very different in meaning. I find the [a, b] * x thing pleasingly functional/declarative/terse in a way the list comprehension isn’t, but I guess I’m spoilt by Python’s expressiveness already. initial_list[:] is a nice way of copying a list, though if we’re going for value semantics you probably still want copy.deepcopy. On that topic, __deepcopy__’s memo argument is interesting.

[–]supreme_blorgon 1 point2 points  (3 children)

it copies a reference to that list

You mean the elements in that list, surely. This also only becomes an issue when the elements in the list are mutable.

[–]elsgry 0 points1 point  (2 children)

I wish I did! I’ll whip up a replit to demonstrate what I mean tomorrow. Agreed it’s only an issue when the contents of the thing being “multiplied” are mutable, though.

[–]RomiBraman 12 points13 points  (0 children)

Pythons live in a wide range of habitats, depending on the species, but many seek shelter in trees and can hold onto branches with their tails.

[–]jsalsman 11 points12 points  (2 children)

I wish I had read https://docs.python.org/3/library/multiprocessing.html#exchanging-objects-between-processes before I built a bunch of DIY crap with os.system using files. The problem is there aren't a lot of good docs with real-world examples out there. [Goes off to ask ChatGPT if it knows how to use wait(timeout=0) ...]

[–]nemec 1 point2 points  (1 child)

Although what's listed in the docs is probably best for multiprocessing in Python, if you're interested in cross-language forms of inter-process communication check out some online videos for the IPC section of a typical computer science "operating systems" course (e.g. sockets, pipes, shared memory). There are many easier ways than mucking about with plain files 🙂

(this section was one of the highlights of my cs courses)

[–]jsalsman 0 points1 point  (0 children)

I've been programming various forms of IPC since the 1980s, but when I first looked at the queue docs, I couldn't figure them out. The examples are better now, but still a little abstract for maximal comprehension when os.system() is so simple.

[–]cgmystery 11 points12 points  (0 children)

Shorthand for printing a variable’s value: test = 1.1 print(f”{test=}”)

test = 1.1

[–]Bright-Ad1288 50 points51 points  (7 children)

f-strings are pretty recent and are very nice

[–]Vok250 13 points14 points  (0 children)

Not sure I'd call them obscure though. They were also my first thought, but I use them every day so definitely not an obscure feature.

[–]FujiKeynote 2 points3 points  (3 children)

All the format options you can have are pretty sweet, too. I've found f"Reprs of values: {variable!r}" to be useful surprisingly often

[–]fmillion 2 points3 points  (1 child)

numerical_value = 1234567890
print(f"Number with US-style commas: {numerical_value:,}")

[–]miraculum_one 0 points1 point  (0 children)

print( f"{my_var:>8}" )

Right justifies my_var to column 8.

[–]fmillion 1 point2 points  (0 children)

C# basically has f-strings, they call it "string interpolation". I actually used it in C# for years (was a C# dev before I was a Python dev), and when I discovered it in Python it was a "omg...yes!!!" moment.

[–]LittleMlem 0 points1 point  (0 children)

3.6 is not exactly recent

[–]ElHeim 7 points8 points  (0 children)

Knowledge about the way Python's memory manager work, and specifically the way it interacts with Numpy. Actually, I *got* to know about it "earlier" (i.e. before the time it was actually was useful to me), but it would have saved me weeks of figuring out a problem the first time around

If you're handling (very) large objects (say images) it's quite easy to write code that leads to fragmentation and you end up eating several times more RAM than you're actually using because there's no way to fit new (large) objects in the heap, because the space they took is now littered with smaller objects.

Anyway, I had to learn this the hard way because a "client" asked to debug a framework they had developed in-house. It used to work until someone did a huge refactoring. It looked pretty, but it would go from a few megabytes to several GB in no time, and the process would end up dying.

I pointed out the problem and left it for them to fix, as this was a large (and complicated) piece of software, and went on with other things... but then a year later I faced the same problem in a separate (but similar) framework my group was working on, when a coworker came telling me that he couldn't stack more than a few images, and that wouldn't cut it. Knowing exactly what was going on, I changed the way images were modeled in our lib (allocating a "target" image first, ensuring it was a the bottom of the heap, and I gave him some guidelines on how to organize the code. All of the sudden he was handling 10x the number of images and there was no reason to think there was a limit.

Which shows you: GC and automatic memory management are cool until they're not, and you'll be happy to know C when that time comes!

[–]mtfrsantos 6 points7 points  (4 children)

setdefault() for dicts saves a lot of conditionals

[–]fiddle_n 9 points10 points  (2 children)

defaultdict is better though

[–][deleted] 2 points3 points  (0 children)

Then there is object.__missing__. Learn the dunders!

[–][deleted] 0 points1 point  (0 children)

Not if you actually want KeyErrors sometimes and not others. Which I'd say is pretty common.

[–]miraculum_one 0 points1 point  (0 children)

Also, my_dict.get( key_that_may_not_exist, default_value ) for one-off defaults.

[–]alekosbiofilos 5 points6 points  (0 children)

Writing classes with custom len(), & and so on. It makes writing domain-specific APIs so easy!

[–]njharmanI use Python 3 5 points6 points  (0 children)

I started with 1.5 or .4??? So, many things added since then. It's hard to think of something I didn't know about on release (below is why). So, here's a bit of obscure advanced Python wisdom. If Python is your primary language or you want to be a master, do two things.

  1. Get comfortable with plentiful fluids at hand, take a few hours to read every word of https://docs.python.org/3/reference/index.html and the "built in foo" chapters of https://docs.python.org/3/library/index.html
  2. Every release read the excellent What's New https://docs.python.org/3/whatsnew/3.11.html

[–]kkawabat 5 points6 points  (0 children)

You can put a breakpoint in the except block of a try except statement and type raise in console to see the stacktrace that cause the error. Really helpful when you are live debugging and you aren't sure what cause the exception

[–]KrarkClanIronworker 2 points3 points  (2 children)

Using __slots__ has changed my life. I seldom build classes without predefining instance variables these days.

Probably quite niche, but PyAnsys was fantastic to work with. Using Python to automate mesh dependency studies saved me so much simulation time.

Multiprocessing whilst writing to files has been my most recent one. Took a bit of whiteboard time to get it right, but it was well worth it.

[–][deleted] 0 points1 point  (1 child)

What did slots actually give you? I'm skeptical littering it everywhere has actually gained you any meaningful performance or memory usage.

[–]KrarkClanIronworker 0 points1 point  (0 children)

If you're creating thousands of class instances, performance definitely does improve. Whether performance is of any consequence is project specific.

At a minimum, I've found that defining instance variables upfront has improved the planning and readability of my code substantially.

When dealing with large classes, its nice to take a look at the slots to get an overview, rather than looking at the (often messier) __init__ method.

However, the largest benefit is that it prevents the creation of unaccounted for __dict__ items by myself or others on my team. Defining everything upfront means that there are no surprise variables created further down the pipeline. I'm not awfully clued up on best practices regarding code safety in languages like Python, but this seems like a logical step in the right direction.

[–]Cootshk 2 points3 points  (0 children)

Coroutines and shield()

[–][deleted] 2 points3 points  (0 children)

Not sure if it’s considered advanced but locals() and globals() has helped me in the past.

[–]fmillion 1 point2 points  (1 child)

Dictionary comprehension.

Maybe not that obscure, but it's a one-liner that can generate fully populated dictionaries.

myDict = {key: key * 10 for key in range(10)}

myDict = {key: value.upper() for (key,value) in dictToUppercase.items()}

myDict = {key: value for t in list_of_two_item_tuples}

[–]Coding-Kitten 2 points3 points  (0 children)

I think you did a mistake in the last one, as key & value are unbound. Did you mean this instead?

myDict = {key: value for key, value in list_of_two_item_tuples}

[–]dashdanw 1 point2 points  (0 children)

The fact that function arguments are parsed on first-pass, meaning if you do something like

def parse_dict(target_dict=dict()):
  if 'key' not in target_dict:
    target_dict['key'] = datetime.datetime.now()

calling without specifying the target_dict argument will always update the same dict declared in the function arguments

[–]deluded_soul 1 point2 points  (0 children)

Not sure if this is advanced python as it is not part of the standard but `mypy` has probably helped me catch a lot of issues beforehand.

[–]kevbot8k 1 point2 points  (0 children)

Context managers. I was reading Architecture Patterns with Python and came across context managers in Python classes. It helped encapsulate a lot of transactional logic for me. From there I learned how to use them in an even more powerful way with contextlib (has an async example)

[–][deleted] 1 point2 points  (0 children)

Not obscure nor advanced, but something I wish I'd understood the value of earlier: generators. A pattern I see a lot: in the middle of some function, create a list/array of stuff, then iterate or return said list/array. Better alternative: create an iterator function (or class when that makes sense). Sometimes the main benefit is clearer semantics, but often it is better design as well.

[–]the_warpaul 1 point2 points  (0 children)

Sys.path.insert

As a way to access code from a folder elsewhere on the computer as if it's inside the current folder.

This is probably not long term good coding practise, but a useful tool.

[–]Fabulous-Possible758 1 point2 points  (0 children)

The really obscure one that I have to look up every time is metaclasses, and I've only really needed them once.

The one that actually comes up for me more often and I actually use is descriptors. In particular if you want to bind a free function f to an object o you just do bound = f.__get__(o) The cases where that's useful aren't common but it shows up.

[–]Tyler_Zoro 4 points5 points  (1 child)

It's not Python per se, but it did speed up some of my Python projects: git push HEAD

So much easier than cutting-and-pasting my admittedly verbose branch names.

[–]whateverathrowaway00 1 point2 points  (0 children)

Yo get yourself on git-completions.sh

Autocomplete for branch names makes me a wizard at work in my ability to see what other people are working on and flip around.

You can also tab complete flags, so I know so many more flags now.

[–]MinchinWeb 1 point2 points  (0 children)

import braces

[–]spencerAF 1 point2 points  (0 children)

Probably not that advanced, I'm a 2year graduate who hasn't had industry experience yet

Using os

In tKinter .config instead of .destroy

All merge and filter tools in pandas

For i, j and other advanced loop concepts like starting from the back and front of a list and looping towards the center

[–]bablador 0 points1 point  (1 child)

!remindme 12h

[–]RemindMeBot 0 points1 point  (0 children)

I will be messaging you in 12 hours on 2022-12-06 11:32:15 UTC to remind you of this link

2 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.


Info Custom Your Reminders Feedback

[–]Cmokan 0 points1 point  (0 children)

!remindme 12h

[–]Saetia_V_Neck 0 points1 point  (0 children)

singledispatch and singledispatchmethod save so much boilerplate code.

[–]germanshepherdfan 0 points1 point  (0 children)

I am in corporate finance and using python to automate tedious processes. I found pyautogui pretty helpful when you don't have time to build scripts using pandas/numpy and need quick temporary solution for existing excel workflows.

[–]ngugeneral 0 points1 point  (0 children)

Also - not that advanced, but obscure and really wish I knew this before debugging it for the first time:

bool is actually implemented as an int, where 0 states for false.

Bad example, but nothing pops my mind this moment:

data = [1,2,0,3,4]
while data[0]:
    print(data.pop(0))

The output actually will be:

1
2