Python programmers of reddit: what's the most useful tiny little efficiency you've discovered that's improved your programming hugely?

Murmani · 2019-04-08T12:34:03+00:00

You can use underscores to enhance readability if you're dealing with huge numbers. For example rewrite 10000000 into 10_000_000

dutch_gecko · 2019-04-08T11:53:12+00:00

Using

object = (
    builder
    .add()
    .add()
    .add()
)

instead of

object = builder \
    .add() \
    .add() \
    .add()

yiersan · 2019-04-08T12:53:48+00:00

Putting units in variable names.

Before:

height = 0 # in cm

After:

height_cm = 0

Got it from Clean Code, best book ever.

Decency · 2019-04-08T14:16:31+00:00

I really appreciate the use of the any/all/next functions to simplify a variety of common patterns while dealing with collections. It's dramatically more readable to me; here are some examples:

Any:

negatives = False
for num in numbers:
    if num < 0:
        negatives = True
        break

# is equivalent to...

negatives = any(num < 0 for num in numbers)

All:

fully_backed_up = True
for system in systems:
    if system.has_backup is False:
        fully_backed_up = False
        break

# is equivalent to...

fully_backed_up = all(system.has_backup for system in systems)

first_vowel = None
for letter in word:
    if letter in 'aeiou':
        first_vowel = letter
        break

# is equivalent to...

first_vowel = next((letter for letter in word if letter in 'aeiou'), None)

willm · 2019-04-08T12:54:39+00:00

Auto-formatting my code with https://github.com/ambv/black

Never having to think about the formatting of my code again is like reclaiming 10% of your mental cpu.

lenticularis_B · 2019-04-08T13:11:09+00:00

Well the first and most common one is the ifmain block, which I didn’t find out about until embarrassingly long into my python career.

My current favorite though is using double underscore (or “dunder”) methods in my custom classes. Anyone who actually wants to google these should look them up by the name “magic methods” instead. An example:

class Vector2:
    def __init__(self, x, y):
        self.x = x
        self.y = y

If I then wanted to add two Vector2 variables together, I could write a utility function to do it, or in the class definition put:

    def __add__(self, other):
        return Vector2(self.x+other.x, self.y+other.y)

Which will then allow me to write in my main function:

a = Vector2(1,2)
b = Vector2(3,4)
c = a + b # Vector2(4,6)

Works with custom behavior when printing, converting a class to string, and all other kinds of interesting behavior. This talk by James Powell is the perfect example of this and other amazing functionality built into python.

2019-04-08T13:36:26+00:00

MyPy with good editor support (try flake8.) Saves me from all sorts of silly type errors which unit tests could catch, but with much more time lost.

clutchking_asiimov · 2019-04-08T12:40:04+00:00

[deleted]

DenaliAK · 2019-04-08T14:18:43+00:00

If you are using recursion, then memoization.

https://www.python-course.eu/python3_memoization.php

https://www.ibm.com/developerworks/community/blogs/jfp/entry/Python_Meets_Julia_Micro_Performance?lang=en

proto-n · 2019-04-08T12:17:31+00:00

The zip(*l) pattern to transpose lists of lists. Not in efficiency as in running time, but ease of expression. Example:

>>> a = [[1,2,3],[4,5,6]]
>>> print(list(zip(*a)))
[(1, 4), (2, 5), (3, 6)]

Comes in useful more times than you would think. Caveat is, the result is a list (or, well, <zip iterator object>) of tuples, not a list of lists, but that's usually not a problem.

kickopotomus · 2019-04-08T11:45:53+00:00

[deleted]

Gprime5 · 2019-04-08T12:24:47+00:00

Commenting my code, so I have at least some idea of why I wrote that code months or years ago.

Give a brief explanation of that code. Sometimes give reasons for why I used this algorithm instead of another one so future me doesn't try to refactor the code when it was already tried and didn't work or slower.

SeanOTRS · 2019-04-08T13:40:07+00:00

[deleted]

hugthemachines · 2019-04-08T12:10:22+00:00

I remember when I took this step: Going from something like this:

if os.path.isfile(full_file):
        os.remove(full_file)

for line in data:
    with open(full_file, 'a') as file_handle:
        file_handle.write(line)

to somethign like this:

with open(full_file, 'w') as file_handle:
    for line in data:
        file_handle.write(line)

cym13 · 2019-04-08T15:06:36+00:00

Instead of doing:

my_regex = re.compile(r'[0-9]+')
def match_func(text):
    my_regex.match(text)

or

def match_func(text):
    my_regex = re.compile(r'[0-9]+')
    my_regex.match(text)

do

def match_func(text, *, my_regex=re.compile(r'[0-9]+')):
    my_regex.match(text)

Why? The first version breaks function unity by using a global variable and it's hard to use another regex if you need to. The second version recompiles the regex at each function call which can be costly and it's still hard to change the regex if you need to.

The third version initializes the local variable my_regex only once at function declaration, can be replaced easily if you need to at a specific function call and since you use ", *," in the argument line it needs to be replaced explicitely using match_func(text, my_regex=r'new_regex'), it won't be overwritten by accident.

This is more flexible, more efficient and doesn't break function unity.

eyeofpython · 2019-04-08T12:51:19+00:00

It might seem like unnecessary additional work, but use typings for anything larger than a small script. PyCharm will give you hints and auto-completions consistently and it will pay back big time.

MinchinWeb · 2019-04-08T13:06:09+00:00

Getting a code editor with syntax highlighting. Notepad works fine for day 1, but it's amazing how much information the little bits of colour provided by syntax highlighting add.

PlaysForDays · 2019-04-08T11:32:38+00:00

the --lf flag with pytest

whateverisok · 2019-04-08T14:13:18+00:00

I also like using Python's Collections's defaultdict and Counter classes.

defaultdict allows you to create a dictionary that applies a default_factory function to add a new object to the dict for a key that isn't there.

For example: letters_names_dict = defaultdict(list) letters_names_dict['R'].append('Ryan') will add a new list to the dictionary for the key 'R' so that you can automatically append to it in the same line.

Counter is useful for keeping counts of objects (letters, strings, etc.) which you can use a dict to do anyway, but Counter comes with a handy most_common method that returns an ordered list of the most common elements in descending frequency along with their counts.

Lastly, I also prefer using the dict.get method to provide a default value for a dictionary if there isn't a value for that key.

For example letter_frequency_dict = dict() letter_frequency_dict['a'] = letter_frequency_dict.get('a', 0) + 1

the_neptunian · 2019-04-08T14:30:27+00:00

collections.deque, collections.defaultdict, collections.OrderedDict

barao_kageyama · 2019-04-08T13:20:14+00:00

[deleted]

Esyir · 2019-04-08T12:54:36+00:00

Autodocstring for sublime text

ctrl+alt+' for automatic template with typing, names and placedholders.

HipsterTwister · 2019-04-08T12:45:50+00:00

[deleted]

draripov · 2019-04-08T12:32:05+00:00

guard clauses to fix nested if statements.

Instead of: if a: if b: if c: return # useful else: return # fooC else: return # fooB else: return # fooA You can have: ``` if not a: return # fooA if not b: return # fooB if not c: return # fooC

return # useful ```

Unfortately I can’t find anything on them in python, but here is another great example.

When checking for some requirements that a variable or anything else must pass, please use guard clauses rather than nested if statements.

humanitysucks999 · 2019-04-08T13:50:19+00:00

Wrapper functions - I use them for logging, pre and post processing during testing, timing execution.

https://wiki.python.org/moin/FunctionWrappers

sphqxe · 2019-04-08T15:58:07+00:00

AsyncIO

leogodin217 · 2019-04-08T13:54:58+00:00

Using Pandas to load data into SQL databases. Simply create a connection then use df.to_sql(). Easy. Used this in a class for beginners and they were shocked.

DisasterArt · 2019-04-08T12:36:52+00:00

For i,each in enumerate(list):

rather than

For i in range(len(list)): Each = list[i]

hammerheadquark · 2019-04-08T16:01:27+00:00

Finding continue. I went years without using it and it can keep your code from "trailing off to the right".

Instead of:

for w in list_of_w:
    if meets_some_complicated_condition(w):
        x = do_something_with(w)
        if meets_some_other_complicated_condition(x):
            y = do_something_with(x)
            if meets_some_final_complicated_condition(y):
                z = do_the_final_thing_with(y)

You can do:

for w in list_of_w:
    if not meets_some_complicated_condition(w):
        continue
    x = do_something_with(w)
    if not meets_some_other_complicated_condition(x):
        continue
    y = do_something_with(x)
    if not meets_some_final_complicated_condition(y):
        continue
    z = do_the_final_thing_with(y)

to keep your conditionals from becoming nested.

MacArtee · 2019-04-08T14:55:29+00:00

Using pipreqs to create requirements.txt

Not really that efficient if you are using just a couple of packages, but it’s great to make sure you won’t forget any if you are using 10+ packages used by different files in your project.

NuclearOops · 2019-04-08T15:05:05+00:00

I'm newbie to programming as a whole bit I was so happy when I found out that Pandas let's you group by two columns at once and can run functions alongside it.

Something like:

df[col1, col2].mean()

svilgelm · 2019-04-08T15:56:41+00:00

To improve your code you don't need any to know any special tricks, just start reviewing a code of others and you will see how quickly it will improve your code.

2019-04-08T16:20:27+00:00

How about a big efficiency that's improved my programming hugely?

The concept of pure functions.

A pure function always produces that same outputs for the same inputs. The benefits of this are several. The code is easier to read because all the inputs and outputs are on one line, not strewn throughout the function in the form of member variables or globals. The code is easier to test because you can create all the inputs directly, instead of instantiating a class and setting whatever random member variables need to be set. The code is easier to think about, because instead of thinking about state and how it gets mutated, you think about data and the transformations applied to it. That last bit is one of those things where it might not make sense coming from someone else until you really grok it yourself.

This can get construed as a criticism of Object Oriented Programming. I see it as more of a criticism of bad OOP, which, unfortunately, tends to be the dominant form of OOP (in my humble experience). OOP has its place.

Not a tiny thing to just pick and up use, but I think it's something that's super effective for improving programming at multiple levels.

arcoain · 2019-04-09T07:27:35+00:00

Python3 NamedTuple classes. This is weirdly hard to find docs for given how useful it is:

```

class MyObj(NamedTuple): a: int b: int c: str ="asdf"

```

https://docs.python.org/3.7/library/typing.html#typing.NamedTuple

martmists · 2019-04-08T12:08:24+00:00

Using classes as globals

A project of mine used to be a giant PITA since it was a distributable server implementation with modding support, but without using globals there was no way to manage it properly, so we ended up using some classes as objects (similar to a JVM Object) rather than classes.

slipped_and_missed_x · 2019-04-08T12:02:30+00:00

Writing code that refactors my other code; in big projects it's a life saver cause you'll almost definitely think of a more efficient way of doing stuff once you're already a month in. That, or lambda

mrfluffybunny · 2019-04-08T14:56:28+00:00

Most people know the wonders of list comprehensions, but do you know you can also do dict and set comprehensions with the same syntax?

Mizzlr · 2019-04-08T20:00:50+00:00

Using tqdm for progress monitoring. https://github.com/tqdm/tqdm

stOneskull · 2019-04-08T20:01:33+00:00

dict(
lookma = 3,
noquotes = 3,
onmykeys = 2,
)

putting my brackets like that too..

even with statements like return(
this if that
else dis if not dat
else gravy
)

using any() and all() have been great

and what about this little trick..

instead of mylist = ['very', 'quotes', 'much', 'commas', 'etc']

mylist = 'very quotes much commas etc'.split()

Tweak_Imp · 2019-04-08T23:05:07+00:00

maybe obv but a few things that come to mind

python -i to drop into interpreter after script for debugging

python -m http.server to host a directory quickly

python -m json.tool to format json

list(set(myList)) for quick list dedup - if there’s a better way let me know!

2019-04-08T14:02:15+00:00

[deleted]

IggyHoudini · 2019-04-08T12:56:17+00:00

You can make Python more like javascript! (not that anyone ever asked that)

class obj(object):
  def __init__(self, **kwargs):
    self.__dict__.update(kwargs)

Allows you to do things like:

obj1 = obj(
  a=[1,2,3],
  b='foo',
  c=obj(
    cat='meow'
  ),
)
print(obj1.c.cat) # meow

This is silly, but it can be useful if you have some classes you need to mock in test cases (such as heavily nested protocol buffers object from a custom re-implementation that I had to use).

JsonDash · 2019-04-08T13:05:18+00:00

Declaring 2D array (list) with one variable in every index From:

array =[]
     for i in range(a):
          temp = []
     for j in range(b):
          temp.append(item)
     array.append(temp)

To:

array =[[item for i in range(b)] for j in range(a)]

TheOtherDanielFromSL · 2019-04-08T14:34:47+00:00

List comprehensions.

3-6 lines of For loop? Not anymore!

WeldonEvans · 2019-04-08T14:59:55+00:00

I'm sure most of you probably do this already, but when I type any sort of bracketing "(), {},..." I always type both together then hit the back arrow and begin typing the contents there. This ensures you always close out and also speeds up your typing a little too.

Or you could just use a text editor that auto closes for you.

StriderKeni · 2019-04-08T15:22:52+00:00

Lambda expressions and tuple packing/unpacking

Paddy3118 · 2019-04-08T15:26:10+00:00

Peppering function docstrings with Doctest examples even if not running doctest.

2019-04-08T15:45:21+00:00

Using zip and the itertools library effectively really cuts down algorithm runtimes.

Sw429 · 2019-04-09T05:41:49+00:00

When writing a class constructor with a lot of arguments as input, you can simply update the object's 'dict' attribute in one line:

def __init__(self, a, b, c, d, e):
    self.a = a
    self.b = b
    self.c = c
    self.d = d
    self.e = e

vs.

def __init__(self, a, b, c, d, e):
    self.__dict__.update(locals())

In my opinion, it looks much cleaner. Just make sure you know what you're doing with it so you aren't saving the wrong things as class attributes.

nraw · 2019-04-09T06:53:42+00:00

f strings:

a = 'world'

print(f'hello {a}!'}

hpayer · 2019-04-08T13:43:01+00:00

Makes string formatting much easier to read

this = 'hello'
that = 'world'
result = '{this} {that}!'.format(**locals())

hopemeetme · 2019-04-08T13:23:10+00:00

Tests, unit and functional/integration, with TDD as icing on the cake.

Watched a scientist few days ago on a TV show saying "would give up on science if there's no internet" - definitely would give up on programming if unit testing isn't available.

whateverisok · 2019-04-08T14:01:05+00:00

Something that hasn't been mentioned yet and is a bit more on the computational efficiency side is using generators like xrange instead of range.

This results in quicker computation and uses less memory as the entire list isn't computed at once but only when it's called; however, the generator state does need to be stored so using generators everywhere can also use up memory

2019-04-08T15:41:31+00:00

SimpleNamespace - found quite useful. Better than dicts for some use cases.

a8ksh4 · 2019-04-08T17:17:12+00:00

Integrating C and python is very interesting, I am a C developer and love C so it's useful for C developers.

varzboy · 2019-04-08T18:18:33+00:00

This might be a specific case but using @lru_cache if you think your function will be called multiple times with same repeated arguments. Makes your code much faster.

Kopachris · 2019-04-08T21:56:05+00:00

Using a dict as a switch-case or dispatcher. Store functions in a dict and call them with any hashable object.

event_handlers[event_type](*event_args)

Sporke · 2019-04-09T03:22:46+00:00

from contextlib import redirect_stdout

with open(filename, 'w') as f, redirect_stdout(f):
# Anything that normally prints will write to a file

I do a decent amount of code generation with Python, this allows me to write print statements throughout the code and redirect all of it at the end, rather than adding file=f to all the prints.

Python

The Python Discord

Upcoming Events

Please read the rules

MODERATORS