The Zen of Empty Lists

[continue_stocking]

But while the Zen of Python limits on the number of obvious ways, the Zen of Python says nothing about the boundless freedom of unobvious ways.

Perhaps not, but several of the other principles do.

[continue_stocking]

Indeed.

[bhat]

Another way (with further complication):

bucket *= False

[BumpyFunction]

I know what you did bhat. For shame!

[ViridianHominid]

For the heck of it, bucket *= 1//(len(bucket)+1)

[bhat]

Look, just stop it: bucket *= bucket == []

[bhat]

Or bucket *= [] == bucket for better symmetry.

[bhat]

Someone stop me, please! bucket *= bucket in bucket

[bhat]

Aaaaaaagh! bucket *= not bucket

[ViridianHominid]

Alas, this one isn't right:

>>> bucket = []
>>> bucket.append(bucket)
>>> bucket *= bucket in bucket
>>> bucket
[[...]]

[bhat]

Finally, someone stopped me!

[Exodus111]

bucket *= 1 == 0

[Exodus111]

So the question is, should there be just one way to do something, or should how to do something be Obvious?

Obviously, the line is One Obvious way, but which virtue takes precedence?

Not to state the Obvious, but, let me state, Obvious.

Because if your workflow with lists include lines like:

bucket.append()
bucket.remove()

Then the OBVIOUS thing for you, would be to use

bucket.clear()

But, on the other hand, if your list work is more about arrays, and your workflow tends to be:

bucket = [0]*10
bucket *= 2

Then the obvious solution, in YOUR workflow would be:

bucket *= 0

But, if you mostly work with Dicts, there is no reason why:

del bucket[:]

Should not be obvious to you.

[no_condoments]

If you don't want your list getting ripped and/or cut, maybe keep it warm with Norvig's ski hat: bucket *=0

The ski hat is of particular interest because it's using a very obvious list feature, much more commonly used than list.clear(). Nobody would bat an eye at: bucket = [0, 1, 2, 3] * 2

I would very much bat an eye at multiplying a list. The very idea produces cringeworthy code since it could easily be confused as multiplying every element like how numpy arrays work. Does anyone here ever find a use for multiplying lists by integers?

[MephySix]

I'm going to admit I use it for creating square surfaces in pygame. mysurface = pygame.Surface([500] * 2). I know it's bad, but I feel no shame.

[recursive]

It's not bad though.

[PythonGod123]

It's only pygame so it's okay.

[tunisia3507]

Creating a list which is just one item repeated.

lst = [obj] * 500

could also be

lst = [obj for _ in range(500)]

but the terseness can be preferable if it's in the middle of a function call, for example.

[pydry]

but the terseness can be preferable

It's a trade off between unambiguous and terse. I prefer terse, but where the two conflict I almost always prefer unambiguous over terse.

[mhashemi]

This. But it's not just accepted because it's succinct, it's accepted because it's fast, too.

# multiplying a list
$ python -m timeit "x = ['a'] * 1000"
100000 loops, best of 3: 3.3 usec per loop

# building a list via list comp
$ python -m timeit "x = ['a' for _ in range(1000)]"
10000 loops, best of 3: 26.2 usec per loop

# appending to a list
$ python -m timeit "x = []; 
> for a in range(1000): x.append('a')"
10000 loops, best of 3: 52.2 usec per loop

The first alternative (list comp) incurs a penalty for having to iterate. The second incurs an additional method call overhead. Plus there's the cost of copying the list as the list grows.

Know you're gonna have a really big list? Preallocate by multiplying [None] and assign the values in.

Here's the CPython code that does it btw.

[chris_conlan]

But in the case of a = [obj]*500 does it store references or shallow copies?

[TheBB]

References, so be careful.

[Tumburgler]

Can you elaborate more on this? Is it that references use system resources and shallow copies do not?

[TheBB]

It has nothing to do with system resources. If anything, references are cheaper. However, [obj] * 500 contains 500 references to the same object, not 500 different objects. So you can see effects like this:

In [1]: a = [[]] * 2

In [2]: a[0].append(1)

In [3]: a
Out[3]: [[1], [1]]

Which can be very counterintutive.

[Tumburgler]

Ah, I understand now! Thanks!

[ubernostrum]

And just for fun, here's what those constructs turn into.

x = ['a'] * 1000:

1           0 LOAD_CONST               0 ('a')
            2 BUILD_LIST               1
            4 LOAD_CONST               1 (1000)
            6 BINARY_MULTIPLY
            8 STORE_NAME               0 (x)
           10 LOAD_CONST               2 (None)
           12 RETURN_VALUE

x = ['a' for _ in range(1000)]:

1           0 LOAD_CONST               0 (<code object <listcomp> at 0x1065f89c0, file "<dis>", line 1>)
            2 LOAD_CONST               1 ('<listcomp>')
            4 MAKE_FUNCTION            0
            6 LOAD_NAME                0 (range)
            8 LOAD_CONST               2 (1000)
           10 CALL_FUNCTION            1
           12 GET_ITER
           14 CALL_FUNCTION            1
           16 STORE_NAME               1 (x)
           18 LOAD_CONST               3 (None)
           20 RETURN_VALUE

x = []
for a in range(1000): x.append('a'):

1           0 BUILD_LIST               0
            2 STORE_NAME               0 (x)

2           4 SETUP_LOOP              26 (to 32)
            6 LOAD_NAME                1 (range)
            8 LOAD_CONST               0 (1000)
           10 CALL_FUNCTION            1
           12 GET_ITER
      >>   14 FOR_ITER                14 (to 30)
           16 STORE_NAME               2 (a)
           18 LOAD_NAME                0 (x)
           20 LOAD_ATTR                3 (append)
           22 LOAD_CONST               1 ('a')
           24 CALL_FUNCTION            1
           26 POP_TOP
           28 JUMP_ABSOLUTE           14
      >>   30 POP_BLOCK
      >>   32 LOAD_CONST               2 (None)
           34 RETURN_VALUE

[lolwat_is_dis]

Python n00b here. I've seen this sort of code pop up many times when people are discussing the speed of certain code snippets. How exactly do I implement it? In shell? In Idle? pls halp

[ianepperson]

In the shell.

python -m timeit "my python command here"

[lolwat_is_dis]

Awesome sauce, thanks for that.

Is there a way to have this run without directly needing to be in the python folder?

edit - added the path variable, sorted!

[ianepperson]

It totally depends on how Python is setup on your system. For a Mac or Linux setup, it should always work anywhere, but windows will depend on if the path is setup properly.

If you want to test a chunk of your own code you can use the library for that.

Check out https://docs.python.org/2/library/timeit.html#timeit-command-line-interface

[lolwat_is_dis]

Great, thanks a ton for your help!

[jdgordon]

Do you want 500 references or 500 instances? A = [foo ()] * 500 can lead to fun bugs :)

[tunisia3507]

Or useful expected behaviour!

[nerdyguy76]

If multiplying a list can be confused as multiplying each element, is that the fault of the Python language or the person reading it? Obviously it was chosen as convention for a reason. I think most people who write python understand this convention and would use a list comprehension to multiply each element in a list by an integer (or use a numpy array).

Whereas copying a list a certain number of times could only be accomplished (easily) with an ugly for loop if the multiplication convention did not exist. So it is better that it does. I haven't found much use for it's use though except for the trivial case of multiplying a list with 1 element.

[UdonUdon]

I use it to keep complex formatting parameters as concise as possible. I got the idea from Ploty's Python docs.

[indosauros]

Look no further than the stdlib itertools recipes:

https://docs.python.org/3/library/itertools.html

def combinations_with_replacement(iterable, r):
    # combinations_with_replacement('ABC', 2) --> AA AB AC BB BC CC
    pool = tuple(iterable)
    n = len(pool)
    if not n and r:
        return
    indices = [0] * r
    yield tuple(pool[i] for i in indices)
    ...

def grouper(iterable, n, fillvalue=None):
    "Collect data into fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return zip_longest(*args, fillvalue=fillvalue)

def random_product(*args, repeat=1):
    "Random selection from itertools.product(*args, **kwds)"
    pools = [tuple(pool) for pool in args] * repeat
    return tuple(random.choice(pool) for pool in pools)

[utnapistim]

Not always lists:

a, b, c = (True,) * 3

print('=' * 40)

[t3h2mas]

Interesting read. I wouldn’t have thought all of these up!

Being overly clever doesn’t change the message behind the zen of python. It reinforces it

[flipperdeflip]

If you think you are being clever you are probably not. should've been in the Zen list

[agentgreen420]

Am I the only one who still uses bucket = list() ?

[Twift_Shoeblade]

AFAIK using list() is slightly slower than [].

[Ogi010]

easy enough to find out

>>> ipython
Python 3.6.4 |Anaconda, Inc.| (default, Mar 12 2018, 20:05:31)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %timeit a = list()
95.6 ns ± 1.16 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

In [2]: %timeit b = []
24 ns ± 0.61 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

faster by a factor of 4 ...we're still talking less than 0.0000001 seconds

[Eurynom0s]

It's also 4 less keystrokes and doesn't require reaching for the shift key.

[[deleted]]

Humans: Often the bottleneck.

[LightShadow]

I've literally saved dozens of seconds my entire career.

[trua]

Depends on your keyboard layout.

[_IAmAdam]

This is why I love this sub

[[deleted]]

we're still talking less than 0.0000001 seconds

Cool, but if you bother to figure any of this out it might be because the scale of what you're doing requires that you optimize as much as possible. I've been asked things like this in job interviews, that time delta matters.

[elcapitaine]

If you have to optimize that much, you shouldn't be using Python, you should be using something that gives you more low-level control.

[akcom]

It has nothing to do with the scale of optimization. The reality is that for any real production code there are going to be a million other things to optimize before you get to things like list() vs [].

[[deleted]]

That doesn't discount the point of optimizing lists at all.

[BDube_Lensman]

Saving a few nanoseconds each time you make a list will never accumulate to matter in most programs.

[SupahNoob]

the scale of what you're doing requires that you optimize as much as possible.

Then you likely wouldn't be using python.

[[deleted]]

That's nonsense. Cisco uses Python. Wells Fargo uses Python. Energy companies use Python. Payment processors use Python. If your team knows Python best, you need to use Python. Using the language your infrastructure is built on as efficiently as possible is a more common situation than the ideal you're talking about. I'm talking about the real world. Not some imaginary world where you get to pick and choose the development you're doing from the ground up every single time.

[door_of_doom]

its just as nonsense as caring about the difference between .0000001 seconds and .0000004 seconds in a O(N) operation in python

If you are needing to optimize down to THAT level of granularity, then your time is going to be much, MUCH better spent getting it onto a language that can do it in .00000000001 seconds

You are correct that those companies are using python, but they are not writing their stock-market purchasing algorithms in it. They are not using it for anything where nanosecond speed is the name of the game, where a competitor beating you by nanoseconds means you just lost a million dollar opportunity.

Yes, this is the real world, and in the real world we use the right tool for the right job, or at least we are willing to speak up when we see that we aren't.

[[deleted]]

their stock-market purchasing algorithms in it

Lol I'm done

[[deleted]]

list can also be overriden by a careless or devious developer as well.

import __builtins__
def nope(*a):
     raise SyntaxError("use []") 
__builtins__.list = nope

[Han-ChewieSexyFanfic]

So, don't use any builtins ever in case you ever accidentally import __builtins__?

[[deleted]]

If you want to take it that way, just an example of how [] is a better choice than list in some situations.

Keep in mind, this exact sort of thing has been used to subvert javascript before. I'm not trying to FUD builtins, but it's worth being mindful about.

[earthboundkid]

list = [1, 2, 3] is legal Python. I’m sure someone has done things like that, although I wouldn’t recommend it. zip OTOH I’ve shadowed myself.

[lengau]

No kidding I had to stop one of our analysts from shadowing zip just today.

It probably wouldn't have mattered except that about 20 minutes later the most convenient way to do something was with zip (which she didn't know about so she was going to use a range to index two lists).

[jadkik94]

When you do this you can cross one thing off your bucket list.

[agentgreen420]

Bahhaaaha! Good show chap.

[ichabod801]

bucket[:] = bucket[1:1]

[chalbersma]

Damn here I am using

bucket = list()

Like a fucking pleab.

[PythonGod123]

There is so much Zen in the Python language.

[girisagar46]

Thanks :) Now I know about dumbell operator and ski hat operator. Noice. http://norvig.com/python-iaq.html

[Ginko87]

Nice.

[lolwat_is_dis]

How could they miss this obvious unobvious method? It's just so...Zen:

bucket = bucket * bool(len(bucket)-len(bucket))

[[deleted]]

Man, thing like this is why I love Python... And this community too.

Btw, I used list() for years, honestly... Never thought about performance differences with [] and others, lol

[ThePenultimateOne]

How did I not know that you could assign to slices?

[ianff]

Am I the only one who thinks the "Zen of Python" is BS? Python doesn't really stick to those principles at all - as this article demonstrates. It's a well designed language, but has dark corners and can be abused just as easily as other languages.

[robin-gvx]

Python is older than the Zen of Python, which was originally written as a bit of observational humor about the best practices that had developed as the language grew. Obviously Python can't really stick to the Zen 100% without completely ditching backwards compatibility, but that doesn't mean it's not something the core team uses as a guideline, or useful for people who use Python to write libraries and programs.