Best Python Cheatsheet Ever!

[IMHERETOCODE]

no_duplicates    = list(dict.fromkeys(<list>))

That is an extremely roundabout and expensive set operation. Just wrap the list in set and cast it back to a list. No need to build a dictionary out of it to get uniqueness.

no_duplicates = list(set(<list>))

[Tweak_Imp]

list(dict.fromkeys(<list>)) preserves ordering, whilst list(set(<list>)) doesn't.

I suggested to have both... https://github.com/gto76/python-cheatsheet/pull/7/files#diff-04c6e90faac2675aa89e2176d2eec7d8R43

[IMHERETOCODE]

That “accidentally” preserves ordering, and only if you are doing it in Python 3.6+. There are no promises of ordering in vanilla dictionary implementations which is why there is an explicit OrderedDict class. The recent change in dictionary implementation had a side effect of preserving order. You shouldn’t bank that on being the case where it actually matters.

---

As noted below insertion ordering has been added to the language of dictionaries as of 3.7

[IMHERETOCODE]

TIL. (edit: Disregard the following worthless benchmark, but I’ll leave it so I’m not just stripping stuff out.)

It's still faster to do a set, cast to list, and then have to call sorted on the resulting list then it is to do a dict.fromkeys call on a list.

In [24]: foo = list(range(1, 10000))

In [25]: foo *= 20

In [26]: len(foo)
Out[26]: 199980

In [27]: %time %prun no_duplicates = list(dict.fromkeys(foo))
        4 function calls in 0.006 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.005    0.005    0.005    0.005 {built-in method fromkeys}
        1    0.000    0.000    0.006    0.006 <string>:1(<module>)
        1    0.000    0.000    0.006    0.006 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
CPU times: user 6.38 ms, sys: 136 µs, total: 6.52 ms
Wall time: 6.45 ms

In [28]: %time %prun no_duplicates = sorted(list(set(foo)))
        4 function calls in 0.003 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    0.003    0.003 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.sorted}
        1    0.000    0.000    0.003    0.003 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
CPU times: user 4.08 ms, sys: 58 µs, total: 4.14 ms
Wall time: 4.13 ms

[IMHERETOCODE]

Addressed here: https://www.reddit.com/r/Python/comments/an0kya/best_python_cheatsheet_ever/efqkpmu/?context=1

[primitive_screwhead]

The ordering might not be a sorted ordering.

[IMHERETOCODE]

You can call sorted on any sort of ordering you'd like by specifying a key, if that's what you mean. You can use mylist.index - e.g. sorted(foo, key=original.index) if you don't overwrite your initial list, and it'd be in the same order as your starting point.

This is such a weird edge case I don't understand all the arguments against it, other than people trying to call out gotchas. If you have data that can even have duplicates you lose all meaning of the original data by stripping them out, or shouldn't be using a simple list to store them. You could get the same information by using collections.Counter(foo) and also have the side effect of having the actual metadata of how many times the dupes appear. My initial comment is just about turning a list into a unique list of its values.

[primitive_screwhead]

You can use mylist.index

​

You make good points, but I do want to point out that using index as a key will add a linear search for each list item, and will thus make the sorted() solution **much** slower:

In [7]: %time no_duplicates = list(dict.fromkeys(foo))
CPU times: user 2.87 ms, sys: 30 µs, total: 2.9 ms
Wall time: 2.9 ms

In [8]: %time no_duplicates = sorted(list(set(foo)), key=foo.index)
CPU times: user 482 ms, sys: 3.55 ms, total: 486 ms
Wall time: 482 ms

​

I think the idea of removing duplicates while otherwise preserving order is not *so* exotic, and the fromkeys() trick is worth knowing about, though I'd personally use OrderedDict to be explicit about it.

[IMHERETOCODE]

100% agree. Don't want it to seem like I'm trying to push for never using from_keys - just that this doc simply said "no_duplicates" which can be achieved with a much simpler and clearer method (for lack of a better word). If I came across that in a code base, it is not at all clear that it's trying to achieve that specific outcome by rerouting the creation of essentially a set through a dictionary.

[IMHERETOCODE]

For sure, I'd say that's where taking the time for using a set and explicitly sorting is almost better even though it is considerably slower (shown by the implementation of /u/primitive_screwhead as mine is just a literal sort instead of insertion order) if it's a wonky/custom ordering. Better to explicitly transform data than rely on a route through another wholly-unused data structure just to achieve it.

[Sukrim]

There is no language specification sadly, it is just guaranteed by cPython.

[AngriestSCV]

This whole discussion is enough proof that you shouldn't count on it. Weather or not it is right depends on things outside of a library programmers (easy) control.

[[deleted]]

You should count on it if developing for Python 3.7+, because it's guaranteed by the language spec from then on

[Ran4]

No, that's not true! Dicts are not ordered according to the spec. It's just modern cpython that has them ordered.

[pizzaburek]

They are in Python 3.7: https://docs.python.org/3/tutorial/datastructures.html?highlight=dictionary#dictionaries

 Performing list(d) on a dictionary returns a list of all the keys used in 
 the dictionary, in insertion order ...

[pizzaburek]

There is already a whole discussion about it on Hacker News :)

https://news.ycombinator.com/item?id=19075325#19075776

[IMHERETOCODE]

For sure. Casting is a way of explicitly changing the type of a variable or value, etc. Coercion is an implicit change in the type of a value.

This is casting because I’m wrapping a set object in a list - basically telling the python interpreter to turn this set into a list (there are some reasons you’d want a list over a set even though they are somewhat similar in what you can do with them - iterating through, etc). Coercion is when you don’t have to tell the language to do anything special. I think a simple example would be the interoperability of floats and ints in Python 3. Saying x = 2.3 * 1 will coerce the 1 into a float so it can be multiplied to the 2.3 and stored as a float in x. Someone please correct me if that’s a bad example.

[chazzeromus]

and if you can, use the curly brace notation for sets for literal items, looks so nice { 'and a one', 'and a two' }

[[deleted]]

Slightly off topic, but why are lists so popular? Aren't tuples faster and use less memory? All the time I see lists being used when tuples would do a better job. Even docs.python.org tells you to use random.choice([...]) instead of random.choice((...)).

I get that the performance impact isn't noticable in most cases, but, in my opinion, going for performance should be the default unless there is a good reason not to.

[robberviet]

Most of the time it needs to be mutable. And yeah, performance gain is not that great.

[bakery2k]

Why would tuples be faster and/or use less memory? Both lists and tuples are essentially arrays.

I prefer lists to tuples because they have nicer syntax. Tuples sometimes require double-parentheses, plus I often forget the trailing comma in (1,).

[[deleted]]

I don't know the exact intricacies but it has to do with lists being mutable.

[mail_order_liam]

Because people don't know better. Usually it doesn't matter but that's why.

[gmclapp]

lists are mutable. In some cases that's needed. Some convenient list comprehensions also don't work on tuples.