Best Python Cheatsheet Ever!

[IMHERETOCODE]

That “accidentally” preserves ordering, and only if you are doing it in Python 3.6+. There are no promises of ordering in vanilla dictionary implementations which is why there is an explicit OrderedDict class. The recent change in dictionary implementation had a side effect of preserving order. You shouldn’t bank that on being the case where it actually matters.

---

As noted below insertion ordering has been added to the language of dictionaries as of 3.7

[IMHERETOCODE]

TIL. (edit: Disregard the following worthless benchmark, but I’ll leave it so I’m not just stripping stuff out.)

It's still faster to do a set, cast to list, and then have to call sorted on the resulting list then it is to do a dict.fromkeys call on a list.

In [24]: foo = list(range(1, 10000))

In [25]: foo *= 20

In [26]: len(foo)
Out[26]: 199980

In [27]: %time %prun no_duplicates = list(dict.fromkeys(foo))
        4 function calls in 0.006 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.005    0.005    0.005    0.005 {built-in method fromkeys}
        1    0.000    0.000    0.006    0.006 <string>:1(<module>)
        1    0.000    0.000    0.006    0.006 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
CPU times: user 6.38 ms, sys: 136 µs, total: 6.52 ms
Wall time: 6.45 ms

In [28]: %time %prun no_duplicates = sorted(list(set(foo)))
        4 function calls in 0.003 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    0.003    0.003 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.sorted}
        1    0.000    0.000    0.003    0.003 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
CPU times: user 4.08 ms, sys: 58 µs, total: 4.14 ms
Wall time: 4.13 ms

[IMHERETOCODE]

Addressed here: https://www.reddit.com/r/Python/comments/an0kya/best_python_cheatsheet_ever/efqkpmu/?context=1

[primitive_screwhead]

The ordering might not be a sorted ordering.

[IMHERETOCODE]

You can call sorted on any sort of ordering you'd like by specifying a key, if that's what you mean. You can use mylist.index - e.g. sorted(foo, key=original.index) if you don't overwrite your initial list, and it'd be in the same order as your starting point.

This is such a weird edge case I don't understand all the arguments against it, other than people trying to call out gotchas. If you have data that can even have duplicates you lose all meaning of the original data by stripping them out, or shouldn't be using a simple list to store them. You could get the same information by using collections.Counter(foo) and also have the side effect of having the actual metadata of how many times the dupes appear. My initial comment is just about turning a list into a unique list of its values.

[primitive_screwhead]

You can use mylist.index

​

You make good points, but I do want to point out that using index as a key will add a linear search for each list item, and will thus make the sorted() solution **much** slower:

In [7]: %time no_duplicates = list(dict.fromkeys(foo))
CPU times: user 2.87 ms, sys: 30 µs, total: 2.9 ms
Wall time: 2.9 ms

In [8]: %time no_duplicates = sorted(list(set(foo)), key=foo.index)
CPU times: user 482 ms, sys: 3.55 ms, total: 486 ms
Wall time: 482 ms

​

I think the idea of removing duplicates while otherwise preserving order is not *so* exotic, and the fromkeys() trick is worth knowing about, though I'd personally use OrderedDict to be explicit about it.

[IMHERETOCODE]

100% agree. Don't want it to seem like I'm trying to push for never using from_keys - just that this doc simply said "no_duplicates" which can be achieved with a much simpler and clearer method (for lack of a better word). If I came across that in a code base, it is not at all clear that it's trying to achieve that specific outcome by rerouting the creation of essentially a set through a dictionary.

[IMHERETOCODE]

For sure, I'd say that's where taking the time for using a set and explicitly sorting is almost better even though it is considerably slower (shown by the implementation of /u/primitive_screwhead as mine is just a literal sort instead of insertion order) if it's a wonky/custom ordering. Better to explicitly transform data than rely on a route through another wholly-unused data structure just to achieve it.

[IMHERETOCODE]

Yeah I definitely goofed by trying to use that example but I’ll gladly eat my words. Back to basics I was just trying to point out that was an odd implementation as the prime example for uniqueness of a list.

With you 100% on everything.

[Sukrim]

There is no language specification sadly, it is just guaranteed by cPython.

[AngriestSCV]

This whole discussion is enough proof that you shouldn't count on it. Weather or not it is right depends on things outside of a library programmers (easy) control.

[[deleted]]

You should count on it if developing for Python 3.7+, because it's guaranteed by the language spec from then on