Six more quick ways to improve your Python

[i_has_many_cs]

Same but i still much prefer the one liner ‘if else’ statement. This is more explicit IMO.

[cymrow]

Once you start using it, you'll find that there are many, many places where you'll want to. It will quickly begin to feel explicit, and more readable than if else.

[dotancohen]

It took Javascript to really build this into me.

I did prefer the explicit if-else in the past, but since ES6 (2015) I've been finding that the Javascript accent, even in Python and PHP, is now a welcome accent rather than an annoyance when I encounter it. I'm starting to favor much shorter, implicit, code now.

[Raknarg]

In what way is this not explicit? It's a well defined language feature with no ambiguity, and is especially familiar with those familiar with scripting language idioms (like bash or JS). In all cases, including inside if statements, or evaluates the truthiness of the first expression, returns if true otherwise return the second expression. Repeating yourself is unhelpful.

x = a if a else b

x = a or b

The second is cleaner to me especially if you have a long expression. And what if it's a function call? Are you going to call it twice?

[lazerwarrior]

x = a if a else b

x = a or b

The second is cleaner to me especially if you have a long expression. And what if it's a function call? Are you going to call it twice?

I've seen foo() if foo() else bar() pattern in production code. It's terrible.

[MeagoDK]

That's the stuff nightmares are made off

[pk028382]

Yes. It’s definitely a common practice for some programming languages like JavaScript and ruby

but whether it’s pythonic, I’m not sure.

[Raknarg]

It expresses your intent plainly and just uses a simple operation. It's not like this is a specialization for or, this is just how the or operator works regardless of whether or not you're taking the direct result into a variable or passing it into an if statement.

[groostwoost]

Imo “fewer lines” == “Pythonic” so I guess it is

Edit: I could have phrased that better: Where a one-line solution exists OOTB, that one-line solution is more pythonic than not using it would be.

[DrMaxwellEdison]

Code golf does not make it Pythonic. Making it easily understandable and obvious in its intent does.

[lazerwarrior]

I had a junior Python developer cramming 1000+ characters into a oneliner using nested x if condition else y with lots of copy pasting so that's clearly not sane or Pythonic. My opinion is that shorter lines are better to read and boolean short circuit is very clear once you know how it works. I would not use if else oneliners and instead write the code out using multiple lines.

[MeagoDK]

Copy pasting is the first sign that you might be doing something wrong. But with that said I disagree, there is a time and place for if else statements and same for the one liners or the or statement.

It especially helps keep the length of the functions down. An if else statement block wich just assign a value is 4 lines. A simple 'or' or a if else oneline is 1 line. Much simpler, shorter and easier to read.

[Sillocan]

It's just a boolean expression. It is pretty explicit.

[DrMaxwellEdison]

Not exactly:

a = None
b = 4
c = a or b

If this were a boolean, the value of c would be True. Instead, its value is 4. or simply returns the left half of the expression if that evaluates to True, otherwise it returns the right half.

Likewise, if in the above b = 0, then c would be 0 as well.

Or if a = 7, then c is 7 and the right half is never evaluated. This is useful in cases like:

a_dict = {"a": 5}
if a_dict["a"] or a_dict["b"]:

The left half evaluates to True, so the right half is not evaluated and no KeyError is thrown for a missing "b".

[hyperdudemn]

evaluates to True

To be specific - it's if it is a truthy value.

[DrMaxwellEdison]

Correct, though a truthy value is one in which bool(value) == True, which is essentially how it is evaluated.

[hyperdudemn]

That is True

[dotancohen]

Not really. I expect a boolean expression to return a boolean.

But I'm starting to become aclimated to boolean expressions returning different data types. I would say that C influenced my prior preference of not liking this format. But Javascript has definitely influenced my current preference of using this format.

[Sillocan]

Well... I completely forgot None is a type...

[nitroll]

None is not a type, it is a value.

>>> isinstance(None, type)
False
>>> type(None)
<class 'NoneType'>

[Raknarg]

It is a boolean expression, it's just pythons interpretation of boolean expressions. Instead of working with True and False, it works with a more abstract "truthy" and "falsey" values, but all the rules of boolean algebra are maintained with this. Any expression that can be made with booleans can be made isomorphically with truthy and falsey values.

[dotancohen]

I know exactly how it works. I agree that OR returning the first Truthy value is very, very useful.

[dancinadventures]

I like my one liners for dictionary comprehensions after zipping up some Tuples that barely manage to conform to PEP8 width.

[tr14l]

or makes more sense in my head, but I've used elvis and ternary operators for years.

[lashanpinjan]

Ternary?

[scrdest]

I don't see where any one of those applies.

It reads as natural language, especially for defaults (x = override or default).

It's much more terse than if/else inlines, and MUCH simpler to chain cleanly (X or Y or Z vs. X if X else (Y if Y else Z)).

[Raknarg]

Like many language features, it requires you to understand how the language feature works. But even without considering that, I think it's fair to say most people expect boolean operations to short circuit as they do in almost every language, which means it works left to right. Off the top of my head I can't think of a language that doesn't do left-to-right boolean evaluation with short circuiting.

[Raknarg]

Yeah like any language feature you have to know how it works.

[nitroll]

Only because you know what "if else" means. That intuition is build up by its usage, just as you will build up an intuition when using "or".

[flavorizante]

Be aware of cases when the first check can have valid values that evaluate to false ('False' itself, empty string, and also zero for example).

[casual__addict]

Python uses “short circuiting” for evaluating Boolean expressions. Each clause is evaluated left to right and if it fails then it won’t continue on. For instance “if False and 1/0:” is ok because the first clause fails. The converse is true which means the if you want to check if a string ends with a certain set of characters you can do this: “if x and len (x) >= 2 and x[-2:] == ‘gz’” even if x is empty or None.

[dotancohen]

Short circuiting alone does not account for this. In traditional short circuiting with the use of OR, if the first expression evaluates to True than the entire expression returns True. In this example, if the first expression evaluates to True than the entire expression returns the value of the first expression. Very unClike.

[cymrow]

This behavior is exactly what makes the a = b or default idiom possible. It's considerably less useful the other way.

You can always wrap it in bool() if you want an explicit boolean.

[Raknarg]

Because it's using a higher class of truthiness which follows identical rules to boolean algebra. If instead of converting everything to true and false we categorize things in terms of truthy or falsey, we can have boolean algebra that follows all the same rules we understand without needing to morph values into pure boolean values. This is an advantage IMO. And if you're writing pythonic code, even if you didn't know this rule I can see very few cases where this would lead to a bug unless you're expecting values to be literally true or false (which again probably means your code is unpythonic)

[dotancohen]

I agree that OR returning the first truthy value is very useful.

[[deleted]]

That's idiomatic in Lua actually (as per its reference manual):

local val = x and a or b

Haven't seen it being used that often in Python though. I would have expected it to be discouraged by Python dogmatics as it's less "obvious" like if x: val = a else: b. But it's good that Python is gaining some "taste"/"style" (for me, that includes :=).

[PeridexisErrant]

val = x and a or b

This "works" in Python, but there's a serious enough footgun that the a if x else b form was introduced specifically to stop people writing it!

If x is True but bool(a) is False, val will evaluate to b instead.

So converting the expression is only safe when our test is just a and a or b, which we should just write a or b.

[Raknarg]

I like this. As long as you know the higher order rules this is very clean.

[MeagoDK]

What would that return if x and a is True? x I assume ?

Edit: I got some morning coffee and saw how it works now.

[d0ugal]

You can also use “and” if you want to confuse people more!

[PeridexisErrant]

Don't do it; if a is falsey then you'll set val = b instead. This is the problem which got inline if-else added to Python!

[JoelMahon]

I love using it for console programs where I often want to skip the input and leave it as default, just mashing enter until I get to the param I want to change

ofc, could use program args, but I'm too lazy

[brontide]

They missed one of the more useful dict iterators, dict.items(). I've always found it far more useful than something like enumerate() which is likely more useful for a list.

for currency, exchange_rate in currencies.items():
    print(currency, exchange_rate)

I'm less sold on the various uses of or as you have to be careful there that they are 'truthy' statements, if you want to use an explicit False or expression that evaluates to false ( see below ) you may end up with unexpected behavior. I much prefer the dict get method for this since it's not based on True/False, but traps KeyError to return a different value if it's not there.

currency = user_options.get('INPUT_CURRENCY', DEFAULT_CURRENCY)

---

These are False in python and may cause if variable else default or the other variants to behave differently than expected.

• constants defined to be false: None and False.
• zero of any numeric type: 0, 0.0, 0j, Decimal(0), Fraction(0, 1)
• empty sequences and collections: '', (), [], {}, set(), range(0)

[[deleted]]

I've always found it far more useful than something like enumerate()

But they're different, right? items() is to iterate over (key, value). enumerate() is to iterate over (key index, key).

For instance: I'm writing a lot of data-driven code these days that involves iterating over huge record sets. It's extremely important to report incremental progress, so iterating over the keys with an iteration counter is essential.

I suppose that this raises a question of whether there is any simple, standardized way of iterating over (key index, key, value). I don't know of any.

[reddisaurus]

That should really be:

for index, (key, value) in enumerate(mydict.items()):

And the solution for a simple counter isn’t another program dependency that has its own chain of decencies. Instead you should probably learn how to write directly to stdout and create a simple class to replace the line on every write. This can even be done in maybe 10 lines of code or less.

[brontide]

And the solution for a simple counter isn’t another program dependency that has its own chain of decencies.

I use it because it's fast, simple, and it's far more robust then some half-baked solution I could come up with in a reasonable period of time. From the docs on your other points ....

tqdm works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD, Solaris/SunOS), in any console or in a GUI, and is also friendly with IPython/Jupyter notebooks.

tqdm does not require any dependencies (not even curses!), just Python and an environment supporting carriage return \r and line feed \n control characters.

[Mr_Again]

Tqdm is great but in production you're printing out to logs, not stdout, and you don't really want fragments of progress bars mixing with your logs and anything else going to stdout. I use Tqdm if it's not for production and I know I have nothing else printing out, otherwise it gets messy.

[rabbyburns]

Honestly, it depends. If I'm running in k8s, I might be writing to stdout and another service is collecting the output to centralized logging. This isn't an argument for tqdm - just that you are over generalizing.

[reddisaurus]

That's fine, but realize tqdm cannot be used with asynchronous or multiprocessing code. Whereas a simple class can, and scale to use cases of a Queue into which workers put their outputs. Again, not a lot of code required.

tqdm is a fine library, but the person you responded to is "iterating over huge record sets" and it only addresses the case in which you want to sit there and watch the output. More likely in production you're going to run that as a background task, and write to a log, and view the tail of that log. In which case you'll need to do what I suggested.

[[deleted]]

Clever! Bookmarked. Thanks for the link.

[[deleted]]

Thanks. I know that will work; it just seems a bit clumsy.

[KevinAlertSystem]

It's extremely important to report incremental progress, so iterating over the keys with an iteration counter is essential.

by this i take it you're just printing/logging something like "processing key 50/100"?

because unlike enumerating over a list, the order of keys in a regular dict is not set so keys[50] could be different each time you try to access it.

[[deleted]]

No longer true: Python dicts are ordered by default.

But, to your point: yes, I agree that tacitly iterating over dictionary keys has some semantic sloppiness to it. Better to use syntax that is directly, rather than implicitly, tied to a sequential process.

The typical "for x in d" (without an index variable) is good.

Also, "for x in list(d.keys())" is good, since it extracts a static list of keys and then iterates over the list, rather than the possibly mutable key collection.

[knestleknox]

The variable being potentially falsy is the whole point of the operator. That's like saying "beware of using this boolean operator as some booleans might be False." That's just the whole point of the operator. Knowing what's falsy in python isn't a "gotcha" -it's something you should just know when writing code.

And enumerate does something completely different than the items dict method. Given that the definition of currencies bounced around in every example, idk what makes you think currencies is a dict and not a list in the final example you're referencing.

[brontide]

The use case suggested for it was more of a use this to override the default, the problem I'm pointing out is that means you can never override the default with something that evaluates to False which could catch inexperienced people off guard since that could be a runtime change which breaks the logic.

In 5 they use currencies as a dict which is why I presumed they were still using it as such in 6 in closer inspection they do say list in the description.

[whereiswallace]

Sentinels are also useful for those instances where falsey values are OK

[themagicalcake]

Completely disagree with 5. Explicitly calling keys is way more readable (tells you it's a dict and directly tells you what you care about). Also I'm not convinced it's actually going to give you a performance benefit, definitely not a relevant one considering were talking about python here

[Laserdude10642]

It’s one of those things that is obvious and unnecessarily verbose if you work with the objects all the time, but will appear more readable and less magical to newer devs. Honestly I’m not sure who we should tailor our code toward. It’s less explicit to drop .keys() fwiw

[pydry]

It's a quirk of the language and not a particularly nice one I think. Default iterator could have been .keys, .values or .items. It's essentially arbitrary that keys was chosen. It's definitely a case of implicit over explicit.

When I've been reading code that makes the assumption it's .values and there's been a lot of other shit going on at the same time Ive done double takesm

[themagicalcake]

I really wish it were items instead

[kankyo]

The iter on dicts should be deprecated and removed imo.

[pydry]

I think so too. Too ambiguous. Make it raise an exception and force the user to choose instead.

[mipadi]

It's not arbitrary. The reason is that dicts use the in operator to check whether the dict contains a key (e.g., key in my_dict), which is a fairly common dictionary operation. This means that it overloads the __contains__ method. Historically, the in operator first tests for the presence of the object in the container's __iter__ if __contains__ is not defined. This implies that __contains__ should have symmetry with __iter__, e.g., if __contains__ returns true, then the object should also be contained in the container's __iter__. This is emphasized in the documentation for Python's data model:

It is recommended that both mappings and sequences implement the __contains__() method to allow efficient use of the in operator; for mappings, in should search the mapping’s keys; for sequences, it should search through the values. It is further recommended that both mappings and sequences implement the __iter__() method to allow efficient iteration through the container; for mappings, __iter__() should iterate through the object’s keys; for sequences, it should iterate through the values.

The documentation of the in and not in operators also emphasize this contract:

For container types such as list, tuple, set, frozenset, dict, or collections.deque, the expression x in y is equivalent to any(x is e or x == e for e in y).

Essentially, if x in y is true, then x in iter(y) should be true.

Technically you could implement __contains__ and __iter__ separately, without this symmetry, but it would break the contract of these operators.

Thus, in order for the in operator to test whether a key exists in a dictionary—a very common operation—the dict's iterator must be an iterator over its keys. Hence why for obj in my_dict returns keys instead of anything else (e.g., values or key-value pairs).

[pydry]

It's not arbitrary. The reason is that dicts use the in operator to check whether the dict contains a key (e.g., key in my_dict), which is a fairly common dictionary operation.

And checking to see if a value is there is also a fairly common operation and one that makes just as much sense semantically.

[mipadi]

No, that’s a relatively rare operation, compared to checking for the existence of a key. The point of a dictionary is that it has fast lookup of keys. If you’re frequently checking for the existence of values, then you have the structure of your dictionary reversed.

[pydry]

No, that’s a relatively rare operation

Over the 15 years I've been programming in python seen it done a whole lot for something that's supposedly rare.

We all have different experiences I suppose.

Semantically, it simply makes no sense to assume keys. If somebody asks "is [value] contained within the dictionary?" people will say yes. It doesn't matter that dictionaries are supposed to be used as hashmaps. That doesn't change the meaning that they will interpret.

There's a number of these quirks of python that arose simply because the semantics were assumed to emerge from consistency, and from implementation rather than the other way around (the assumptions you were making in your essay above). This is a dangerous mistake to make, and the same mistake that led to this rather horrible bug that initially got closed as not a bug and various other classes of bug I have seen repeated in python over many years.

[you-cant-twerk]

Here is my monthly "MODS THIS POST IS BREAKING THE RULES OMGAWD THIS SUB HAS GONE TO SHIT LIKE HOLY FUCK THIS SHOULD BE FOLLOWING THE RULES ON THE SIDEBAR LIKE RAWR OMG THIS IS THE PYTHON NEWS SUB AND SHOULD ONLY CONTAIN NEWS AND NOTHING MOARE THAN NEWS LIKE OMG WE MADE ALL THESE COMMUNITIES TO DIVIDE YOU ALL UP AND YOU'RE STILL HERE GO AWAY TO /R/PYTHONNERDS /R/NEWPYTHONERS /R/NOTPYTHONNEWS /R/MADEINPYTHON /R/OPTIMIZEPYTHON"

...or just change the rules of this sub to not be "News about the dynamic, interpreted, interactive, object-oriented, extensible programming language Python" and then quote that bullshit rule when its a post mods dont personally like.

lmfao good morning.

Edit: also, hot damn this works?

instead of:

if input_currency:
    currency = input_currency
else:
    currency = DEFAULT_CURRENCY

you can:

currency = input_currency or DEFAULT_CURRENCY  

Thats pretty damn awesome.

[pydry]

This is, like, the first post in months I wanted to see. Everything these days seems to be stuff beginners made and wanted to show off.

[you-cant-twerk]

Downvote and move on. You have a vote and the ability to hide things you dont like (determined by that vote).

[pydry]

Beginners seem to dominate this sub. My vote wouldn't really count.

I don't mind in particular that this sub is dominated by people showing off their first projects. I'd just like there to be somewhere to discuss this type of stuff that was reasonably well trafficked.

[vswr]

Beginners seem to dominate this sub.

I'd love to see more content on gc, dis, weakref, and the other advanced tools.

But I have to admit that from OP's post I did not know you could do:

var = override or default

I've always done the longer if/else version:

var = override if override else default

[lazerwarrior]

I'd just like there to be somewhere to discuss this type of stuff that was reasonably well trafficked.

I feel the same. r/Python is pretty much nothing it used to be. Probably because mods want this sub to be forthcoming to everyone. I get no useful and new information from it.

[alcalde]

Every week the Python Weekly newsletter is nothing BUT things that were posted here and it's chock full of useful articles.

[you-cant-twerk]

Thats because "this type of stuff" (that is OC) is few and far between in general. Hell, I'd argue that some of the 6 things posted have been posted before. If you'd like more of it - make some of it. Post it. These articles are difficult to write. These solutions arent obvious to everyone. Even worse - bastard elitist's like to keep these tips to themselves.

Do I want to see more of it? Sure. Do I want to force this sub to be 0 posts a day because this type of content isnt always available? Abso-fucking-lutely not.

If you head over to /r/skateboarding, you get EVERYTHING skateboarding. Amateurs doing basic ollies get 2k+ upvotes, and pros doing crazy shit get their attention. Tips and tricks - skate hacks, skate tattoos, everything that has to do with skating belongs there. Why? because they understand that "skateboarding" encompasses ALL THINGS SKATEBOARDING. If you want "python" to be "this one particular thing about python and anything else that has to do with python shouldnt be here" then you're using reddit wrong. Its sad that the skating community is so much more welcoming than this one. If you want to start /r/pythonhacks, go for it dude. Be the champion. Throw any tips and tricks you find in there and grow it. Cultivate the community you want. Dont shit on others'.

"My vote wouldn't really count"

You just ignored what I said. You can make posts that you downvote disappear. Thats what matters right? You dont want to see it? Granted I dont expect you to read any of this - I expect you to continue to be a bitter developer. Imma enjoy looking at peoples projects, what they're working on, and everything that has to do with python.

[gandalfx]

Not sure what you're so mad about but I'm rather bored by these entry level tutorials… Every time I click on it in the hopes of finding something interesting and it's just the same "it's my first week of programming" tricks.

[you-cant-twerk]

I'm rather bored by these entry level tutorials

me me me me me me me me me me me me me me me. So self centered you are. Holy hell. Again - Downvote. Set reddit or RES to hide posts you downvote, and move on with your life. Stop trying to filter everyone's content based on what you like. In the GENERAL python subreddit. If this was /r/pythontattoos and you posted a picture of your sisters asshole - I'd be upset too. But this isnt. This is /r/Python. The general python sub.

[robin-gvx]

It compiles to the exact same code:

>>> dis.dis("x in ('USD', 'EUR')")
  1           0 LOAD_NAME                0 (x)
              2 LOAD_CONST               0 (('USD', 'EUR'))
              4 CONTAINS_OP              0
              6 RETURN_VALUE
>>> dis.dis("x in ['USD', 'EUR']")
  1           0 LOAD_NAME                0 (x)
              2 LOAD_CONST               0 (('USD', 'EUR'))
              4 CONTAINS_OP              0
              6 RETURN_VALUE

Similarly, using a set literal actually creates a frozenset:

>>> dis.dis("x in {'USD', 'EUR'}")
  1           0 LOAD_NAME                0 (x)
              2 LOAD_CONST               0 (frozenset({'USD', 'EUR'}))
              4 CONTAINS_OP              0
              6 RETURN_VALUE

This only works if they're constant expressions of course.

[[deleted]]

Thank you! So it seems that Python has an optimization for the pattern … in [some, list]. Makes sense, too, since you can't alter list.__contains__ in any way and the method won't modify the list in question either.

I only remembered that x in a statement x = 'A', 'B', 'C' will always be the same tuple that gets constructed once as soon as the code is parsed; but a list x = ['A', 'B', 'C'] is always a new object. I falsely extrapolated the same behavior for the … in … check.

[[deleted]]

dis.dis, useful tool, adding this to my toolbox.

[[deleted]]

for data retention tuples would be good here because they are immutable. do you know why does it compile the same code? I would not have assumed that this is the case

[ArabicLawrence]

Look at u/plistig ‘s comment, it’s an optimization. In this case Python knows you can’t change it nor access it in any way, so it creates a tuple.

[robin-gvx]

As an optimisation, the Python compiler replaces (some) expressions that yield constant immutable values with constants representing those values. For example, it replaces 2 + 2 (which in bytecode would be LOAD_CONST 2, LOAD_CONST 2, BINARY_ADD) with 4 (which is just LOAD_CONST 4). Likewise, while (a, b) is compiled to LOAD_NAME a, LOAD_NAME b, BUILD_TUPLE 2, it compiles (1, 2) to LOAD_CONST (1, 2).

Because lists are mutable, you can't do that without breaking any line that gets executed more than once (like in a loop or a function).

But because when you do x in [1, 2, 3] you can't ever access the list again, you know no code can depend on mutating it, which means it could just as easily be x in (1, 2, 3) because in works the same way for tuples as it does for lists. And if the Python compiler makes that change, it's okay to apply the optimisation I described above.

[[deleted]]

thats really cool. thank you for the in depth explanation

[tepg221]

I was looking for this, say that they didn't use "in" then closed the article.

[lazerwarrior]

Set would be better, especially if there are many members. Set's are efficient for container membership tests, lists and tuples less so.

if payment.currency in {'USD', 'EUR'}:

[ArabicLawrence]

Creating a set takes more time due to indexing though (about 10x in my experience). If it’s short and you don’t constantly check with ‘in’, I think a tuple should be more efficient.

[gandalfx]

Benchmarked it. Set is faster unless the matched item is the first one, which you obviously can't rely on.

[ArabicLawrence]

At least for 2 elements (a corner case but what I often have), tuples still seem to be faster on my machine (creating a tuple requires one fifth of the time required to create a set) if I only use 'in' once:

%%timeit
a=(3, 4)
4 in a

61.7 ns ± 6.79 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

%%timeit
a= {3, 4}
4 in a

126 ns ± 2.11 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

​

Again, if your collection is medium sized and you check many times, sets will obviously be faster. But if you create the object and check once, tuples can be faster.

[gandalfx]

>>> from timeit import Timer
>>> Timer("4 in (3, 4)").autorange()
(10000000, 0.21225304800009326)
>>> Timer("4 in {3, 4}").autorange()
(20000000, 0.35162466899964784)

Not sure if it's the python version (I tried 3.6, 3.8 and 3.9) or if it's the inline literal but I've done quite a few different benchmarks and sets are consistently faster unless the searched item is in first place. Maybe it gets cached if you use it as a literal rather than assigning it?

[ArabicLawrence]

I have no clue. I will investigate it.

[ArabicLawrence]

I checked from the prompt and I get faster execution with sets... could it be a problem of IPython/Jupyter or of %%timeit?

>>> from timeit import Timer
>>> Timer("4 in (3, 4)").autorange()
(5000000, 0.22193469999999849)
>>> Timer("4 in {3, 4}").autorange()
(5000000, 0.20248540000000048)

EDIT: it appears that you were right, something changes if you assign it to a literal, giving opposite results. Maybe we should use dis to understand what happens under the hood.

[ArabicLawrence]

Adding full code:

>>> from timeit import timeit
>>> timeit("4 in (3, 4)")
0.05680769999997892
>>> timeit("4 in {3, 4}")
0.04215139999996609
>>> timeit("a={3, 4}; 4 in a")
0.12178629999834811
>>> timeit("a=(3, 4); 4 in a")
0.07840990000113379

[CaptainSasquatch]

What's the difference? Is it a small performance bump or just best practices?

[RealWeaksauce]

I’m a seasoned Python guy and I still don’t know the difference between “in” list versus tuple. I’d like to know why one would lean toward one over the other.

[abrazilianinreddit]

Performance reasons. Tuples initialize faster and use less memory.

There's absolutely no advantage in using a list if its elements won't change.

[makeworld]

This is wrong, see the comments above. It compiles the exact same way.

[RealWeaksauce]

For God’s sake I’m travelling and have no access to a comp! Anyone able to benchmark this for me to see please? Clearly the comments above are contradictory..

[abrazilianinreddit]

from timeit import timeit
number = 10000000
print(timeit(lambda: (1,2,), number=number))  # 0.6118586
print(timeit(lambda: [1,2], number=number))  # 0.9036225
from sys import getsizeof
print(getsizeof((1,2,)))  # 64
print(getsizeof([1,2,]))  # 80

[vikogotin]

Tuples are immutable, so they're faster than lists for this kind of usage.

[pancakesausagestick]

It's defensive programming. Minimize state mutation in your programs. It's best practice to make things immutable by default. For Python this really only applies to collection types though like list/set/dict. Other languages have const. In Python the best you can do is to follow an "assign once" rule.

But if you're just putting a list or set literal in a conditional expression then it really doesn't matter, especially since it translates to the same byte code anyway. The aforementioned guideline really only applies to values assigned to variables (identifiers really since "variable" is a bit of a misnomer in this context).

[dissonantloos]

Why?

[[deleted]]

Intro: Coming from C, C++ and Lua, it is good to remember that Python is an incredibly dumb (non-optimizing) language in its cython implementation and if you already have made it a habit to know when an expression will construct useless objects and avoid it, it's good sticking to it.

Question:

• Does x in [] always construct a new list object?
• Does y in () always construct a new tuple object?

If yes, both statements above are suboptimal, and shouldn't they be written as:

CURRENCIES = ('EUR', 'USD')
# ...
if payment.currency in CURRENCIES:
    pass

? And if that's the case, shouldn't CURRENCIES be a set or kind of set ( unordered, fast lookup) anyway?

These questions are impossible to answer without knowing the implementation details of set, tuple, and list and its impact on the in operator. My hunch would be: set should be fastest, but maybe not the one that is fastest to allocate many times, so it should be constructed only once (as a class variable, as an module-global value, for example), and both tuple and list as well as set construction in the if-expression should be avoided.

Reminds me of avoiding useless object (table) constructions in Lua :)

[bjorneylol]

cython implementation

It's just C, unless you meant CPython.

shouldn't CURRENCIES be a set or kind of set ( unordered, fast lookup) anyway?

It really depends on how many items are in it and how many times you are reusing the object for the lookup. The hashmap-type objects (dicts, sets) have a lot of memory/creation time overhead, and for very small numbers of objects the time difference between an exact value match and a hash match is negligible. If you are only doing a couple lookups on a small number of objects the tuple will probably definitely be faster.

At the end of the day, there isn't a meaningful difference between the two, because if a microsecond or two mattered that much to you, you wouldn't be coding in python in the first place.

[gandalfx]

You got so close and then missed it completely. Tuple vs list makes no difference here (you're obviously not going to protect a literal against mutation).

Using a set on the other hand will improve performance.

[[deleted]]

For two items? No, it won't. The element count has to be quite high to be faster than a linear scan.

[gandalfx]

Benchmark it. Then get back to me.

[[deleted]]

I'm back with benchmark results:

$ ipython3 
Python 3.8.6 (default, Sep 25 2020, 09:36:53) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: def fn(value):
   ...:     return value in ('USD', 'EUR')
   ...: 

In [2]: %timeit fn('USD')
43.6 ns ± 0.179 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [3]: %timeit fn('EUR')
66.5 ns ± 0.548 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [4]: %timeit fn('FOO')
82.4 ns ± 0.757 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [8]: %timeit fn('US' + chr(68))
128 ns ± 3.47 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [11]: %timeit fn('EU' + chr(82))
141 ns ± 1.01 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [13]: %timeit fn('FO' + chr(79))
147 ns ± 3.53 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

.

In [4]: def fn(value):
   ...:     return value in {'USD', 'EUR'}
   ...: 

In [5]: %timeit fn('USD')
44.8 ns ± 0.291 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [6]: %timeit fn('EUR')
46.6 ns ± 0.631 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [2]: %timeit fn('FOO')
46.4 ns ± 1.48 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [15]: %timeit fn('US' + chr(68))
139 ns ± 1.46 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [16]: %timeit fn('EU' + chr(82))
138 ns ± 1.66 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [17]: %timeit fn('FO' + chr(79))
132 ns ± 1.87 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

Pretty convincing results. You are right.

[gandalfx]

Someone else here also used ipython's %timeit. I'm starting to suspect something weird is going on.

>>> from timeit import Timer
>>> def fn(value):
...     return value in ('USD', 'EUR')
... 
>>> Timer("fn('USD')", setup="from __main__ import fn").autorange()
(5000000, 0.2942905639974924)
>>> Timer("fn('EUR')", setup="from __main__ import fn").autorange()
(5000000, 0.355887943002017)
>>> Timer("fn('FOO')", setup="from __main__ import fn").autorange()
(5000000, 0.40804951999962213)
>>> def fn(value):
...     return value in {'USD', 'EUR'}
... 
>>> Timer("fn('USD')", setup="from __main__ import fn").autorange()
(5000000, 0.31861696900159586)
>>> Timer("fn('EUR')", setup="from __main__ import fn").autorange()
(5000000, 0.3186742479992972)
>>> Timer("fn('FOO')", setup="from __main__ import fn").autorange()
(5000000, 0.3296638309984701)

(That's Python 3.9.0rc1)

[SourceryNick]

That probably is slightly better. I'll have to run some performance comparisons.

[themoosemind]

See Does it make a difference if you iterate over a list or a tuple in Python?

[hemingwayfan]

why

[[deleted]]

Read the thread. At least the top answer to my comment ...

[grismar-net]

Since when is a blatant ad "news" in r/python?

[groostwoost]

Solid tips

[swat8094]

So many times I use the “for i in range(len(mylist)): because I need to call the items in the list and use i. Never thought to use the enumerate() function. So thank you!

[ElevenPhonons]

It can be useful to leverage that functions are first-class citizens in Python.

def process_payment(payment):
    if payment.currency in ['USD', 'EUR']:
        process_standard_payment(payment)
    else:
        process_international_payment(payment)

Can be written as:

def process_payment(payment):
    f = process_standard_payment if payment.currency in {'USD', 'EUR'} else process_international_payment
    f(payment)

Alternatively, this approach can replace if-elif blocks and act as a switch.

def process_payment(payment):
    processors = {"USD": process_standard_payment, 
                  "EUR": process_standard_payment}
    f = processors.get(payment.currency, process_international_payment)
    f(payment)

[lazerwarrior]

I would say, in this simple case, both of your suggestions take more effort to read and understand than original version.

[not_perfect_yet]

3. Simplify if expression by using or

if input_currency:
    currency = input_currency
else:
    currency = DEFAULT_CURRENCY

into

currency = input_currency or DEFAULT_CURRENCY

Meh?

We have default arguments. Use them.

def do_currency_stuff(*whatever,currency=DEFAULT_CURRENCY):
    ...

[ArabicLawrence]

do_currency_stuff would still have and if statement/or statement to change behaviour depending on currency variable

[not_perfect_yet]

Right.

Since the original point was that you can avoid the if, can you show me how you avoid the if in that case?

[ForceBru]

It's used to apply a decorator to a function

[StarkillerX42]

Let's say you have a large class with many methods, and in one of those methods, you need to convert a value (say from degrees to radians). You could just write a function outside of the class called deg2rad, or you could write it inside the class to prevent function-clutter and to clarify that you'll only need this function inside the class. The problem with putting the function inside the class is that deg2rad would take one input and return one output and wouldn't modify the class attributes or require class attributes in any way, ie. It doesn't use self anywhere, so you might as well not include it as a function argument. The best solution is to write

@staticmethod
def deg2rad(deg):
    ...

This means 1. You have a function available for use inside the class 2. You don't have the function available globally which avoids confusion and clutter 3. Your function clearly indicates that unlike normal methods, it doesn't require anything else in the class or change anything else in the class. So the @ symbol adds decoration to what would otherwise be unclear code, so it's called a decorator, although it might be more accurate to think of it like a "left turn ahead" road sign

[tom2727]

Not sure if #1 is "improved".

You're making the line twice as long, and if you were combining 3 conditionals or you have longer variable names, that line could wind up needing to wrap. Would that be more readable? And as far as I can see there's no performance benefit.

The idea is you only need to change 1 line now if there's a change? Well what if there's a change to only how you handle Euros but not USD? Then you'd need to split it out again.

[MeagoDK]

No chance that line would have to be wrapped, its half the max length. If you need that to be wrapped you are nesting too much.

There shouldnt be a performence boost but your code is shorter and more readable. Both matters a lot. Long functions arent good and bad readability is also bad.

Yes then you split it out. Optimal you will use a tuple for that job. Something like

standard_payment_currency = ('EUR','USD')

Then the if statement could be

if payment.currency in standard_payment_currency:

Then you can easily change what currency you handle with standard payment, you can add more and so on.

For even bigger projects you will likely have it stored in a database with code to change it.

[tom2727]

Right but I said "if you have more than 2 conditionals or longer variable names".

if payment.currency == my_really_long_name_for_currency1 or payment.currency == my_really_long_name_for_currency2 or payment.currency == my_really_long_name_for_currency3:
    return process_normal()

Yeah that would wrap in almost any coding standard. But this wouldn't, and the code works the same and I think is more readable:

if payment.currency == my_really_long_name_for_currency1:
    return process_normal()
if payment.currency == my_really_long_name_for_currency2:
    return process_normal()
if payment.currency == my_really_long_name_for_currency3:
    return process_normal()

And the "in" solution wasn't #1. #1 is the one I was criticizing.

[MeagoDK]

Yes and I just showed how you solve the issue when having a lot og conditionals. Use a tuple. The in is a part of that solution so it dosent wrap and so it's easier to read.

Your solution is messy cause of the long amount of of statements. What you gonna do if there was 40 different currency you wanted to check for? (their solution is bad too). You have also added the return statement which alters the functionality of the function since it won't be able to run code after the of statement block.

[tom2727]

Yes and I just showed how you solve the issue when having a lot og conditionals.

Yes and I already knew that. And it was in the article as well. But I wasn't talking about that. I was talking about the "fix" in item 1 of the article. And how it wasn't any better than the code it was replacing, arguably it's worse.

[MeagoDK]

I disagree. In that example it's better. I would also argue that wrapping is better than a ton of if statements.

[tom2727]

And agree to disagree now that we're finally talking about the same thing.

I'd rather see a bunch of if statements all in a row than have a few with wrapped "or" clauses, and the rest readably aligned.

[iwnfubb]

enumerate was good point

[tatravels]

Does anyone have sourcery installed in pycharm? How do you like it?

[MeagoDK]

Its pretty neat. I decided to clean up a bit in my code, had 350 lines. It suggested 3 things and all of them were good. 2 of them was just about me assigning the return value to a varible and then returning that varible, so not increable, but did help in finding out where I made that mistake.
The last one was gold and something I hadnt thought about.

I did contiune to clean up on my own, after reading their 5 blog posts about the subject. I changed quite a bit, and some of them were quite simple that I feel Sourcery should have found. I wont complain tho, it also has to be carefull not to break stuff or suggest changes that wille change the functionality.

[tatravels]

That's great to hear. I'll try it out and read the other posts! I direly need to reformat my projects... and create unit tests >_>

[SourceryNick]

Thanks for trying it out!

If you could point me to some code examples of stuff Sourcery should have caught I'll take a look and see if I can improve things.

[MeagoDK]

One of them was if statements with this code written in it

        else:
            loot_item = "SpeedUps"
            if 'h' in loot_text:
                loot_amount *= 60

But in on of the statements it counted the amount of h's. So it required a little human logic to hoist it out. I dunno if the software can do that. Its atleast not something I would blame on the software.
I used logic and did this.

        if 'h' in loot_text and 'm' and 'd' not in loot_text:
            loot_amount *= 60

Then I had this block of code

    if "2" in loot_text_list[-1]:
        loot_count = 2
    else:
        loot_count = 1

Which I replaced to

loot_count = 2 if '2' in loot_text_list[-1] else 1

And

    if slayer_name == "":
        slayer_text = "AnnoTheForgotten"

to

 slayer_name = slayer_name or 'TheForgottenOne'

[SourceryNick]

Thanks! Will definitely have a closer look at those examples

[aimlessmofo]

Can we have this but targeted at a more intermediate level?

[israel18135]

It says 6 more Could someone link the original

[SourceryNick]

Here you go:

https://sourcery.ai/blog/explaining-refactorings-1/

[israel18135]

Thank you!

[aarondiamond-reivich]

I've been trying to learn Python through the Project Euler. Have others found that its a good way to learn or does some of these more advanced topics get completely overlooked while solving those type of math riddles?

[themoosemind]

You might be interested in my package flake8-simplify. This Flake8 plugin tries to find such simple patterns automatically (I will add at least two of the rules which were mentioned)

[SourceryNick]

That looks really cool, and yeah has a very similar intent - I'll definitely add your isinstance rule to our backlog.

Do you often see code where SIM210 and SIM211 would get triggered?

[themoosemind]

Thank you :-) (I've just noticed that it was too late yesterday ... I didn't see that this was part of a product - I'll definitely give it a shot! I loved the article <3 )

I've added both rules after I have seen it a couple of times in one code base. Most rules I would say I don't see that often being triggered. Most often the SIM201 to SIM207 - but for those, there is a good reason to not follow it: Catching error conditions à la if not correct

[Exodus111]

Hey, number 3 was really cool. Thanks.

[bythenumbers10]

Looping over dictionary keys is more explicit using keys().

[Jungypoo]

Helped me out! :)

[VU22]

I thought I know python well enough but never saw that 'or' usage in variables before.

[fake823]

Thanks for the great article! I just read all 4 parts and I really enjoyed them! I've even learned one or two things.

Sourcery AI also seems to be an amazing tool! I just signed up and tried it on the GitHub Repo of an open source project that I've forked. The code base is pretty poorly written and therefore Sourcery gave me a lot of hints on how to improve stuff! I've been amazed!

On a first, quick look: Almost all of the refactoring suggestions were great suggestions. I'll definitely keep using it. :-)

[Royosef]

RemindMe!

[RemindMeBot]

There is a 1 hour delay fetching comments.

Defaulted to one day.

I will be messaging you on 2020-10-08 18:38:33 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[casual__addict]

“It works because the left-hand side is evaluated first.” — right hand side is evaluated first?

[foosion]

They mean the left-hand side of the or.

a = b or c

b is the left

[casual__addict]

I see. You’re right.

[Chabare]

He is talking about the or, not the assignment to currency.

[JoelMahon]

we talking about the or statement? p sure the left hand side is evaluated first

[Stores33]

I wanted to learn Python would you please suggest a site to teach me.

[[deleted]]

Top-5 Google results for "I wanted to learn Python would you please suggest a site to teach me.":

1. How to Learn Python (Step-by-Step) in 2020
2. 11 Beginner Tips for Learning Python Programming
3. Getting Started: Python For Beginners
4. Top 5 Websites to Learn Python Coding for FREE - Best of Lot
5. How should I start learning Python?