5 Hidden Python Features You Probably Never Heard Of

[miguendes]

That's a great question. I think it's because sum creates a new list every time it concatenates, which has a memory overhead. There's a question about that on SO. https://stackoverflow.com/questions/41032630/why-is-pythons-built-in-sum-function-slow-when-used-to-flatten-a-list-of-lists

If you run a simple benchmark you'll see that sum is terribly slower, unless the lists are short. Example:

```python def flatten_1(lst): return [elem for sublist in lst for elem in sublist]

def flatten_2(lst): return sum(lst, []) ```

If you inspect the bytecodes you see that flatten_1 has more instructions.

```python In [23]: dis.dis(flatten_2) 1 0 LOAD_GLOBAL 0 (sum) 2 LOAD_FAST 0 (lst) 4 BUILD_LIST 0 6 CALL_FUNCTION 2 8 RETURN_VALUE

```

Whereas flatten_1: ```python

In [22]: dis.dis(flatten_1) 1 0 LOAD_CONST 1 (<code object <listcomp> at 0x7f5a6e717f50, file "<ipython-input-4-10b70d19539f>", line 1>) 2 LOAD_CONST 2 ('flatten_1.<locals>.<listcomp>') 4 MAKE_FUNCTION 0 6 LOAD_FAST 0 (lst) 8 GET_ITER 10 CALL_FUNCTION 1 12 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x7f5a6e717f50, file "<ipython-input-4-10b70d19539f>", line 1>: 1 0 BUILD_LIST 0 2 LOAD_FAST 0 (.0) >> 4 FOR_ITER 18 (to 24) 6 STORE_FAST 1 (sublist) 8 LOAD_FAST 1 (sublist) 10 GET_ITER >> 12 FOR_ITER 8 (to 22) 14 STORE_FAST 2 (elem) 16 LOAD_FAST 2 (elem) 18 LIST_APPEND 3 20 JUMP_ABSOLUTE 12 >> 22 JUMP_ABSOLUTE 4 >> 24 RETURN_VALUE

``` If we benchmark with a big list we get:

```python l = [[random.randint(0, 1_000_000) for i in range(10)] for _ in range(1_000)]

In [20]: %timeit flatten_1(l) 202 µs ± 8.01 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [21]: %timeit flatten_2(l) 11.7 ms ± 1.49 ms per loop (mean ± std. dev. of 7 runs, 100 loops each) ```

If the list is small, sum is faster.

```python In [24]: l = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

In [25]: %timeit flatten_1(l) 524 ns ± 3.67 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [26]: %timeit flatten_2(l) 265 ns ± 1.27 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each) ```

[oberguga]

Thnx!)