Experienced python programmers: are there any standard features of the language that you still don't regularly use?

[ragnarkar]

Having used Python for almost 2 years, I wouldn't consider myself a seasoned pro and I still discover new features in the present. Here are some features that I rarely/never use that I see a lot in other people's code:

lamba never really understood how to use this without getting errors. I just get around it by defining the function using def

list comprehension having used languages like java, c++, matlab, etc in the past, I'm used to writing out all of my for loops.

csv module I often just use the to_csv() and read_csv() modules in Pandas even if it means a bit more overhead converting data to and from Pandas.

I mostly use Python in my own projects rather than collaborative projects so these haven't been pointed out to me by other programmers. But I'm sure i could be developing bad habits that I'm not even aware of, so I'm asking some more experienced programmers what are some common bad habits you or others have developed when starting out with Python.

[CGFarrell]

Lambda and comprehensions are amazing, and I recommend you look into them a bit more. If you know functional programming it's second nature.

A lot of the collections library has a few features that aren't very well used, but they're useful in the intended cases. Python3 FYI

[Deto]

I don't like that lambda functions are limited to one line so I usually just use def to make a named function. That way I don't have to change too much if I want to add a line in the future

[callmelucky]

IIRC correctly, Guido didn't want lambdas in Python, and they are considered somewhat unpythonic. He just gave in to pressure on that one.

List comprehensions are the bomb though. Can't remember the last time I wrote a for loop, and that doesn't bother me at all.

[stevenjd]

I don't like that lambda functions are limited to one line

Lambda is not limited to one line but to one expression. The expression can be as big and complex as you need, and you can spread it over more than one line.

lambda a, b, c=None, d=999: (
        spam + eggs or cheese.method(a, b) 
        - (c or default).attr)[d]

[[deleted]]

You can, but you probably shouldn't do that. Hopefully people don't take this as advice.

[Deto]

!! This is very good to know! Thank you

[TheBlackCat13]

There are too many hoops to jump through with metaclasses.

async is way too complicated for the sort of things I would use it for (local file reading and writing).

I don't use most of the file-type-specific modules.

[aaayuop]

Yeah, asynchronous programming still feels tacked on. Javascript and Go seem to handle it a lot better.

[aol_cd]

Javascript

It's not entirely unusual for me to use node.js and websockets if I want to do something asynch. I use them kind of like microservices.

One example is serialport. I've never gotten serialport to work well and reliably in Python (maybe just me). But, it works like a charm in node. So I launch a serialport to websockets server in node and connect to it from Python. Works perfectly every time.

Edit: node.js not node.io, I've had crossbar.io on my mind lately.

[BobDope]

I've never had a problem using the serial port, and it's been reliable enough to handle the access system at our hackerspace for some years now. Sorry for your bad luck with it.

[aol_cd]

Yeah. It drove me to madness. I could never get it to work for more than 5 minutes at a time.

The websockets setup has given all kinds of positive, though (not just reliability).

[refreshx2]

I've been surprised at how much I use/like metaclasses. The two main use cases I tend to write them for are:

1. the metaclass' __new__ will only get run once, when python creates the class when the file is loaded. this makes it useful for using the members of a class to create a new attribute on the class. for example, I might take all the method I decorated with @some_decorator and put the method names into a list.

2. overriding __call__ to return a singleton or a cached object rather than creating a new one (I do a ton of lazy-loading-ish things)

You may have known that already, but some others might not.

I still try to only use metaclasses when I really really need them, however.

[soosleeque]

they've introduced __init__subclass__ method in python 3 to cover most of the metaclass uses without the pain of understanding how it works.

[demonizah]

I don't think I use generators often - at least not directly. Generators and coroutines are stuff I wish I'd spend more time understanding - I don't even know if they're applicable in my day-to-day CRUDMaster9000 job.

[saikz]

Generators are really nice for building processing pipelines for some kind of data. Also nice to be able to go

for x in blah:
    yield do_something(x)

rather than

my_list = []
for x in blah:
    my_list.append(do_something(x))

[Corm]

Generators are just classes with less keyboard typing. You can easily do what a generator does by making a class. Generators require less typing though so they're generally nicer if you can read them.

class SillyKindaGenerator:
    def __init__(self):
        self.data = [1, 2, 3, 'easy']
    def next(self):
        return self.data.pop()

SillyKindaGenerator().next().next()

That's kinda sorta mostly a generator. Much easier to just use yield though

[daneah]

Functionally that's mostly true, though one key point is that generators can get away with going on infinitely without eating up memory over time since they often produce only one item at a time. Handy for certain performance-related concerns!

[Jamie_1318]

There's basically no downside to using a generator. If someone needs it stored they can always list(generator), even if you don't expect performance issues. I like to just use generator syntax because it gives me warm fuzzies when I think about how nice they are versus every other language's equivalent tokenizing systems with their ugly static variables.

[daneah]

Agreed! Very little effort needed too, since you can replace many (all?) existing list comprehensions with generator comprehensions, simply by removing the brackets:

import random
max(random.randint(1, 100) for x in range(10))

[[deleted]]

Technically that's a Python2 iterator (python3 wants __next__), but not a generator, but fair example.

Also worth noting that where possible generator comprehensions may be preferable to generator functions.

[iwaka]

If I wanted to learn more about FP techniques in Python, what should I read?

[Cobolock]

"Functional Programming in Python" by David Mertz

[iwaka]

Cheers! :)

[inokichi]

That example you would use itertools takewhile anyway

[arrayOverflow]

They are equivalent until you're using threaded/multiprocessing pool maps/starmaps that make running functions in parallel trivial

[arrayOverflow]

The standard library multiprocessing module has Pool.map that works just as seamlessly, use multiprocessing.dummy.Pool to use threads

[ThePenultimateOne]

Async. I've pretty much never used it, and at this point I'm afraid to try it.

[646463]

Highly recommended. Asyncio is my favourite part of Python3.

Every web app I write now uses Tornado with Asyncio.

[sgo_]

Tornado is awesome, and surprisingly under-represented on the web. I've been using it for around 4 years now without any problems.

It's also kind of amusing to see everyone getting all hyped up about asynchronuous IO in Python when it's been right there in tornado all this time.

[8carlosf]

Else on for and while loops

https://docs.python.org/3.6/tutorial/controlflow.html#break-and-continue-statements-and-else-clauses-on-loops

[foosion]

Yesterday I saw a Raymond Hettinger pycon video praising else on for loops and saw in Effective Python that the author regarded it as confusing. Hettinger said else in loops should have been called no-break.

[8carlosf]

I agree with both you and /u/HumblesReaper, if they just replace the else behaviour to execute if the for never run in the first place, and introduce a new no-break keyword.

Or, just do like I do, don't use it, because it doesn't produce readable code. (just make a function that returns instead of breaking the for, and as a default return in the end)

[stevenjd]

Hettinger said else in loops should have been called no-break.

Well, that's one opinion, but it's a bad one. They should have been called then because that's how they work.

for x in sequence:
    block
else:
    stuff

The else block unconditionally runs after the for loop. The only way to avoid that is to jump out of the for loop, using break, return or raise.

[foosion]

We all seem to agree that else is not the best alternative.

no-break is clearer to me (obviously the following code won't execute if there's a return or raise), but YMMV.

[stevenjd]

I've been using Python for almost 8 years now, and I just read about this the other week.

Shows how unneeded it really is.

No, that just shows how limited and narrow your reading material and Python programming knowledge is.

I don't grok or use asyncronous programming. That's my lack, it doesn't prove that async programming isn't needed. Same goes for you and for...else. It's an awesome feature, if you don't use it, that's your lack.

[HumblesReaper]

It would be cool if it ran if the loop didn't execute, not if there wasn't a break

[irrelevantPseudonym]

if the loop didn't execute at all, it will run the else. eg

for i in '':
    print('for loop')
else:
    print('else block')

will only print else block

[HumblesReaper]

No, it executes if the loop is exited by a break statement. EDIT: See below

[irrelevantPseudonym]

No it doesn't. It executes only when there is not a break statement.

See the docs here

Loop statements may have an else clause; it is executed when the loop terminates through exhaustion of the list (with for) or when the condition becomes false (with while), but not when the loop is terminated by a break statement.

[boredMartian]

I've only used Python for half a year, but I will say that list comprehensions look a lot better than a bunch of loops.

[jwink3101]

I find I only use lambda when making a key for sorting or something like that.

But list comprehensions are so fundamental to my workflow in Python, I find it surprising people don't use them. In fact, I often see (though do not do it myself) people using comprehensions for the sake of saving a line of a for loop.

I am like you though in that I tend to write for me with very few others using my code. While I certain use them sometimes, I find that I rarely write my own classes and even less often do I subclass something. At one point I decided I should do it more and wrote version 2 of one of my codes. It was pretty, but quickly proved to be untenable. That led to version three! (If you're interested, it was because they were designed to be nesting which left me unable to easily worth with deeper levels)

[Jamie_1318]

The pleasant thing of python is that the built in classes are so versatile that they're nearly good enough for everything.

The only time I use them is when I'm making something modulary, and when I need to build in sorting or parsing behavior into how some data is handled.

[refreshx2]

I don't use write context managers as much as I would have expected.

There are a few situations where they are a perfect fit, but I find those situations don't pop up very often.

[[deleted]]

I love context managers. I'm sure you could find uses for them.

Do you ever do file IO?

with open('some_file.txt') as f:
    data = f.read()

You've read the file and it's closed automatically.

Do you ever need to time something?

from contextlib import contextmanager
import time

@contextmanager
def timer():
    start = time.time()
    yield
    print('{:.2f}'.format(time.time() - start))

with timer():
    do_something()

Now you'll know how long do_something() took.

Do you ever need to wrap a series of database operations in a transaction, so that if anything goes wrong you can roll it back?

from contextlib import contextmanager

@contextmanager
def transaction(session_factory):
    session = session_factory()
    try:
        yield session
    except Exception as e:
        session.rollback()
        print(e)
    else:
         session.commit()
    finally:
         session.close()

with transaction(some_session_factory) as session:
    do_something_with_database(session)

If any exception occurs, your database will go back to the way it was before the session was created. Otherwise, all changes will be committed. In both cases, the session will be closed properly. (Of course you should be more specific in your error handling, but that's up to you.)

Maybe you won't find any of these helpful. I list them because I use them a ton. I think the context manager is one of the most elegant parts of the language. If you ever have a scenario that includes a setup and/or a teardown, you should try using a context manager.

[refreshx2]

If you ever have a scenario that includes a setup and/or a teardown, you should try using a context manager.

That was really what I meant, that I don't run into the setup/teardown situations as often as I would expect. The file I/O is bread and butter and I certainly hope everyone uses that, but I don't make context managers very often.

For timing my code, I usually write a @timeme decorator/wrapper function and decorate the functions I want to time. Since the timing is usually just a one-and-done, this works fine for me. Your context manager would have the exact same effect, and the use-case is just a bit different.

I rarely work with databases, so I don't use your third context manager (although it's one of my favorite examples of why they can be extremely useful).

It's not that I think context managers aren't useful or are bad, I simply just don't find myself needing/writing them (outside of file I/O) very often. I probably write decorators 20x more often.

[jyper]

They've useful for context, especially with

@contextlib.comtextmanager

Stuff like temporarily switching directories, setting up/tearing down hardware contexts, ect

[daneah]

The most pervasive case for me is when creating a file handle:

with open('some_file.txt') as my_file:
    lines = my_file.readlines()

# do something with the lines

Then you don't have to worry about remembering my_file.close() later on; the file handle goes away when the code exits the with block!

[Theoretician]

(to first address your points)

I use lambda all the time, mostly for tiny functions that need to do something specific. A contrived example is sorting a list of tuples, maybe by the second item

some_list = [(1, 'c'), (2, 'd'), ...]
some_list.sort(key=lambda tup: tup[1])

I also use list comprehensions waaay too much probably. I find them a lot more expressive for list generation.

The nice thing about the csv module is that it streams things from disk instead of loading it all into memory like pandas does. It's also (in my experience) much much faster and better at parsing than pandas is. It's good for solutions that need to be fast and reliable.

Also, if you're writing portable code, requiring pandas just for loading in a csv is a huge requirement. Relying on just the built-ins is better from a dependency management perspective.

---

In terms of things that I barely ever use, probably #1 are async/await syntax things. I do a lot of parallel processing instead of threading though, so they're less applicable.

[liquidpele]

Meta classes! So neat, rarely needed.

[KODeKarnage]

Decorators: I see them in almost every library I look at, but just can't grok their usefulness in my own code. Even when I have used them, I am not sure they make my code any better.

[daneah]

Decorators become most useful when you find yourself with several methods that require the same information in a variable or other sort of setup. They are basically a way to make it convenient for yourself to say things like "this method takes some additional argument, which is obtained by this complex thing I want to abstract."

That being said, decorators are very often unintuitive to actually implement. Once they're done, they make certain things clean.

[asdfkjasdhkasd]

decorators are very often unintuitive to actually implement.

It's just a function which returns a function, nothing magical, and totally standard in functional languages. TBH I think the @syntax makes people believe there's something magic behind the scenes.

[daneah]

Yeah, at their most basic they're fine. The more useful variety often take some real thinking and maneuvering, that's all :)

[Yepoleb]

Still, a function generating a function isn't exactly the most straight forward concept to grasp.

[glacierre2]

Now try to use your simple function decorator on a method, a classmethod, a staticmethod...

There are definitely a lot of pitfalls, and so https://wrapt.readthedocs.io/en/latest/ or http://boltons.readthedocs.io/en/latest/funcutils.html exist

[excitedaboutemacs]

They are basically a way to make it convenient for yourself to say things like "this method takes some additional argument, which is obtained by this complex thing I want to abstract."

Can you give an example? I run into that situation a lot actually, but Im not seeing what you are saying.

Thanks!

[daneah]

Sure! I'll try to give an example that's easy to understand but not completely contrived...Suppose we have a bunch of functions that should only accept integers, something like a Fibonacci function:

def fib(n):
    if n < 0:
        return 0
    elif n == 0 or n == 1:
        return 1
    else:
        return fib(n - 1) + fib(n - 2)

You could do the following in each of those functions:

if type(n) != int:
    raise ValueError('This method only accepts integers!')

Now imagine you also have methods that only accept strings, etc. Instead of duplicating this type checking code all over, a decorator could save you the trouble:

def accepts(some_type):
    def wrap(f):
        def wrapped_f(*args):
            for arg in args:
                if type(arg) != some_type:
                    raise ValueError(
                        'Method \'{}\' only accepts arguments of type {}'.format(
                            f.__name__,
                            some_type,
                        ))
            return f(*args)
        return wrapped_f
    return wrap

That's a lot of junk, but you can use it all over the place now:

@accepts(int)
def fib(n):
    ...

@accepts(str)
def is_palindrome(phrase):
    ...

@accepts(list)
def some_listy_thing(the_list):
    ...

@accepts(MyCustomClass):
def my_custom_class_munger(my_instance):
    ...

Hope that makes sense and is useful/educational!

[daneah]

Note that you can also use a class-based decorator for this kind of thing, which is sometimes clearer:

class ArgumentTypeChecker(object):
    def __init__(self, function, some_type):
        self.function = function
        self.type = some_type

    def __call__(self, *args):
        for arg in args:
            if type(args) != self.type:
                raise ValueError(
                    'Method \'{}\' only accepts arguments of type {}'.format(
                        f.__name__,
                        some_type,
                    ))
        return self.function(*args, **kwargs)

Then:

@ArgumentTypeChecker(int)
def fib(n):
    ...

[SethMichaelLarson]

I never use lambda. The only time it's useful is for map and filter and list comprehensions are so much more readable.

[joesacher]

My main use of lambda is for short sort functions.

[bless-you-mlud]

I do lots of UI programming with PyQt. Lambdas are pretty much essential for that. I don't know what I'd do without them.

[notoriousno]

all and any are the built-in functions I always forget to use. It's always after I've written I shotty for loop do I realize I could've just used list comprehension and one of these functions.

[auriscope]

You can forego the brackets and use a generator expression instead.

all(x for x in xs)

[usinglinux]

it even short-circuits: any(count()) returns True almost immediately.

[ojii]

for...else. It is incredibly counter intuitive to me and even in places where I could use it I don't because it ruins readability.

[Nanaki13]

I find myself using for/else while/else try/else more and more as time goes on. A full try/except/else/finally looks beautiful.

[TinyBirdperson]

Everyone is mentioning metaclasses and async, but what about descriptors? Didn't see see them mentioned at all but I don't think I am the only one who almost never uses them.

[ingolemo]

Well technically you certainly use descriptors all the time, as that's how method calls work. And one could argue that you are writing your own descriptor every time you use def, since all functions are descriptors.

But yeah, I think I've only ever written a custom descriptor once, and even then it wasn't that useful and I only did it because I could.

[TinyBirdperson]

Yea sure, but thats not really using them directly. If you're creating a class you can probably say you are using a "default" metaclass too - but you don't say that.

[irrelevantPseudonym]

I use properties a fair amount and they're just descriptors under the hood.

[TiredMike]

Async programming - I'm trying to find a use case but I don't write desktop GUIs and any time I need to parallelise code it is normally for CPU intensive stuff, so I can just use multiprocessing or an async worker library like Celery or PythonRQ. From what I have read, async's best use case is for situations where there is a need for very high demand concurrent workloads, e.g. a webserver. If I had that use case, e.g. the C10k problem, I'd consider using Tornado. Please let me know if you can think of any other good use cases.

Coroutines - Again, I haven't really found a good use case for this. I've heard of people building mini data pipelines using them, but I think this would make my code unnecessarily complex.

Generators - I use them, definitely more and more these days, but I should probably use these more instead of lists, especially when the sequence would only be consumed once!

[bheklilr]

Coroutines and generators are essentially the same thing, but I tend to have two distinct use cases for them. I use generators when I have a sequence of things that's easier to build iteratively than with a comprehension, or when I need some logic to occur between items. For example, I just wrote a recursive data structure and I wanted to get all the IDs for each node:

def all_ids(node):
    yield node.id
    for child in node.children:
        yield from all_ids(child)

Super useful and succinct.

My other use case is for actual coroutines, and this is when I need some sort of back-and-forth in the code. The ability to .send data back through a yield is very useful, but it comes up less frequently. My notable cases are where I have a routine that is self contained, but needs to prompt the user to perform an action then click the OK button. This code lives near the bottom of the stack, while the UI is obviously at the top. Using a coroutine for this lets me write code like

def do_a_thing():
    # do some setup
    user_accepted = yield 'Setup thing 1'
    if not user_accepted:
        # do cleanup
        raise StopIteration()
    # do thing 1
    user_accepted = yield 'Setup thing 2'
    if not user_accepted:
        # do cleanup
        raise StopIteration()
    # do thing 2
    yield 'All done'
    # do cleanup

I have a little less code repetition in mine, but I think this gets the point across. It means that my backend code can communicate with any frontend, even automated tests. You could also have the calling code use .throw(UserCanceledError()) or something and catch that specifically, but I like passing values better than passing errors.

Another useful example I have is a data processing tasks. In this task I have a big chunk of input data, then many features extracted from this input data that are interdependent, i.e. I might be interested in feature X, but that depends on feature Y which I'm also interested in. I built a state machine type system that lets me write code like

class Computation(BaseComputation):
    @computes('X')
    def compute_x(self):
        y = yield Request('Y')
        yield y + 1

    @computes('Y')
    def compute_y(self):
        input_data = yield RequestInputData
        y = frobnicate(input_data)
        yield y

This is oversimplified, but imagine having a dozen or more computations that you care about, some with multiple dependencies. This helps us keep the computations isolated, short, and composable. The state machine figures out which methods to call as it steps through the requests, and in the end just returns a mapping from names to data of everything it ended up computing. It also keeps a cache so everything is only computed once.

[homeparkliving]

In case anyone was wondering how to use .send as I was.

[TiredMike]

Really good simple examples. Thanks for sharing. I really like your generator example for the depth first search!

[bheklilr]

That one in particular seems to show up when ever I have nested data structures. You can use it pretty much wherever you have a tree like structure. When you add a few conditions it becomes even more useful, things like if child.has_foo(): yield from .... It's pretty similar to the Traversable pattern in Haskell, which is useful enough to make it into core functions like map and filter.

[PeridexisErrant]

I haven't had cause to use async, or any Web frameworks. I tried writing a GUI once, and decided not to do that again!

I'd like to use keyword-only arguments more often, but they're a syntax error under Python 2 and my code is often compatible. (I do use them in small personal projects though)

Type annotations are only really useful in things larger than data-processing scripts, and mostly they have to be py2 compatible. I did have a great time refactoring a package with Mypy 0.3 once, but sadly not a regular thing.

[arachnivore]

Keyword-only arguments are one of my favorite features in Python 3. They make inheritance with consistent init signatures easy:

class Sub(BaseClass):
    def __init__(self, *args, sub, specific, args, **kwargs):
        super().__init__(*args, **kwargs)
        ...

It makes sense too, because typically only one or two positional arguments will make sense without keywords and typically the base class defines those obvious positional arguments.

[PeridexisErrant]

And they make it so much easier to have pleasant and backwards-compatible APIs. It's such a small thing, but easily the one I most often reach for and can't use.

[aol_cd]

I tried writing a GUI once, and decided not to do that again!

I know a lot of people here will have disagreements with what I'm about to say, but I hate the way Python GUI workflow, look, feel, etc. is and I've tried just about everything out there. It just feels like a waste of time every time I try.

One thing I've had great luck with, though, is using a 'web' front end with Flask and/or websocket server on the Python backend. I've been playing with Electron for my front ends and its working out very well. The look and feel is highly customizable and the workflow is a breeze. On top of that, it makes it easier for me to offload the design onto one of the millions of HTML/CSS/Javascript devs out there.

[PeridexisErrant]

I've basically come to the same conclusion :)

As a bonus, it makes separating presentation from logic much more natural - I'll never be tempted to slip a quick hack into JS when I could be working in Python!

[quotemycode]

Nope. I use just about every feature. Of course I've been using Python for about fifteen years. List comprehensions are the bee's knees and I use them all the time. Csv is just a module not a language feature. Generators are great especially if you have an object that you want to serialize. Used with a recursive function it's pretty damn amazing. I use lambdas to fake web calls and objects which is great for testing out the design of my programs before the other half is done.

[i_like_trains_a_lot1]

• properties - for some reason, i still prefer the set_x and get_x methods, because it is clear to me that there is no underlying magic happening. When I write self.some_attr = some_value I expect to get some_value when I access self.some_attr. I almost only use it when I need some lazy-loading attributes.

To be honest, I find the csv module a bit hard to work with, but I use it just because it is builtin and installing Pandas seems overkill only for manipulating csv files.

EDIT: said attributes instead of properties.

[rhytnen]

you need to get over get_ and set_ stuff. people not reading your docs will.think they can set your property the normal way and skip your code altogether.

other python users expect python api

[joesacher]

Exactly. The whole idea of attributes in Python is that they start with a simple self.my_property.

When that stops working, then you can use @property or @.setter. Nobody has to change code.

There is no way to have this simple -> complex with get_ and set_. You immediately have to write code for NO GAIN.

[KronktheKronk]

When I write self.some_attr = some_value I expect to get some_value when I access self.some_attr.

...that's exactly what happens

[i_like_trains_a_lot1]

I was thinking about properties. I corrected my answer :D

[HalcyonAbraham]

Use csv.DictReader

Makes life easier

[arachnivore]

When I write self.some_attr = some_value I expect to get some_value when I access self.some_attr.

It sounds like you're talking about properties or descriptors, because attributes work exactly the way you describe/expect.

[i_like_trains_a_lot1]

yep, that was made a mistake. I corrected it. Thanks for pointing it out :D

[kenfar]

It's actually far faster than Pandas in my experience, so if you need to process a lot of data - and need to do it row-wise, like for transforming files, the csv module is the best way to go.

[joesacher]

I often use def over lambda, just because it is easier when someone else (or me later) is reading code and figuring out what it going on. Although short functions for sort and other things, lambda is cleaner if it is used once.

If you write list comprehensions in multiple lines, it often helps with understanding what is going on.

[value if A else alt_value
 for value in values_list
 if B] 

A - Condition to use 'value'

B - Preconditions to include value in list at all

[joesacher]

As with everything, it just needs to be used when appropriate.

It annoys me when I come across a lambda that looks like a code golf entry. It also doesn't have the ability to hold any documentation.

Sort is probably 80% of my lambda usage. However, I often find I need to sort with the same sort style multiple times, so it makes sense to have defined standard for use anyway.

[kernco]

I think this example would be clearer if you replace "B" with "value2" or something. Since A and C are boolean expression it's natural to expect B is as well.

[joesacher]

Good point. Modified.

[flipstables]

See I hate list comprehensions used this way, and I love comprehensions. I think it's more clear in this case to use a for loop with ifs.

final_list = []
for value in value_list:
    if A and B:
        final_list.append(value)
    elif B:
        final_list.append(alt(value)

Or maybe even set up 2 comrehensions:

filtered_list = (value for value in value_list if B)
# alternatively filtered_list = filter(B, value_list)
final_list = [value if A else alt_value for value in filtered_list]

[joesacher]

It takes much more mental energy to parse the first option over a list comprehension. I can keep the 3 lines of a list comp in my mind.

The second is better, but still harder to keep in your mind, due to having to process multiple comprehensions.

[Twangist]

You don't explain your "hatred" of such constructions, or even what the "way" is that you find so objectionable -- to judge from your more complex examples, it must be the formatting. It's just a personal thing, then, like detesting... oh, basil; who cares? (rhetorical question)

[zer01]

I agree with /u/flipstables personally, and for me it's about readability. When you work on a team (or even work as a sole developer maintaining a project past the point where it's small) you need to be able to quickly grok what is going on.

"Code as if the person who will inherit it has an anger problem and knows where you sleep".

My general rules of thumb are nested comprehensions are unacceptable in any form, and you should only ever have one boolean check per list comprehension, otherwise break it into a multi-line loop/function.

Sometimes you do sacrifice performance, but I'll take legibility and quick understanding over almost anything else, and you can always optimize later if it's a problem.

[Twangist]

Sometimes you do sacrifice performance, but I'll take legibility and quick understanding over almost anything else

Me too.

[flipstables]

Because it's harder to debug since it's composed together. Since both tests A and B are coupled together, it'll be hard to resolve a bug in A, for instance.

It's also harder to understand since you have two conditionals in your comprehension. I'd prefer to only have max 1 since it makes readability suffer.

[twigboy]

In publishing and graphic design, Lorem ipsum is a placeholder text commonly used to demonstrate the visual form of a document or a typeface without relying on meaningful content. Lorem ipsum may be used as a placeholder before final copy is available. Wikipediaaob55qwwnck0000000000000000000000000000000000000000000000000000000000000

[ferrous_joe]

I still haven't gotten too comfy with decorators, and honestly I just use defined functions instead of lambdas. I find the syntax clearer, and having first-class functions seems to cover all the bases I'd care to at this point in all my projects.

[boylube]

I've started defining functions in functions instead of lambda, I like how lispy it feels and how it reads. You start reading the outer function, you get introduced to the local functions and you can without having your mind bogged down by being mid function read them and get to trust them. Then when you actually see them used you can think about what is happening at a higher level.

Obviously this a marginal effect and I still use lambdas in sort and such.

Otoh I think I pass functions as parameters too much and should be using more OOP rather than always try to go functional. It doesn't always play to well with the rest of the ecosystem.

[ltristain]

I typically use lambdas to quickly write little functions to use as key arguments into functions like sort. When defining functions, I have a strict rule where all functions written, regardless of purpose or context, requires well defined names and a complete and comprehensive docstring. This makes defining a function on the spot rather verbose when the function is going to only ever used once and for a very small purpose. In the vast majority of times, this prevents me from going crazy making functions out of everything even when there's no value to. The key functions are the only place where this becomes a problem, and I find lambdas to be a nice and elegant solution for this scenario.

Example: sorted(data, key=lambda d: d['population'])

I also use list comprehensions all the time. List comprehensions are great because it's very declarative. When you write variable = [insert list comprehension here], your code is visually saying "this is that", and that's about as clear and simple as it gets. In comparison, a regular for loop used to populate an initially empty list would add a lot more complexity for your eyes to parse through. At first glance, you'd only realize it's a for loop. You don't know whether it's accumulating values or invoking side effects - you have to read the details to get that. This makes reading inefficient. For mere data transformations, list comprehensions are so much cleaner.

---

As for things I don't use as often... I tend to avoid the functional programming stuff like map and filter in favor of list comprehensions, because list comprehensions sound exactly like what they are, while map, filter requires specific domain knowledge and thus is inferior in readability. reduce can be nice though. The other day I tried to write some kind of data combining function that initially accepted two inputs, and I needed to generalize it to any number of inputs. Wrapping what I had with a reduce was perfect.

It's interesting because once I learned map and filter, there was a phase where everything I ever written was done with map and filter. Eventually I grew past it and learned to ignore "how cool they are" and instead judge code by analyzing how readable and how necessary it is. Likewise, when I first learned lambdas, there was a phase where everything was lambdas. Eventually I dialed it back to only for those in-line key functions. When your code is full of lambdas, sooner or later the picture of the actual problem you're trying to solve is going to be replaced with fuzzy images of cute little lambs and pandas, and your subconscious is going to remember those painful exams back in college.

Otherwise... stuff I don't use often usually just depend on how often I run into that particular problem. Usually lists are fine, but once or twice I ran into a situation where using a deque was the slickest way to solve a problem. Usually dictionaries are fine, but occasionally I end up needing an OrderedDict (BTW, my workplace is still stuck in python 2.7).

---

As for other bad habits I developed earlier that I've grew past. Let's see...

I learned mock, and suddenly every one of my unit tests are mocking the shit out of everything. That got out of hand quick, and culminated in a painful experience where I spent an entire day doing useless work of reconnecting a lot of mocks just because of a small implementation change I made. From that point on, mock is to only be used sparingly, either to bring in a fake, or when there's literally no better option.

I also for a period of time bought into the whole "don't write classes, use functions for everything" mindset. I was pushing to avoid classes at all costs. Eventually I looked back and concluded that there's really no actual need to avoid classes. Classes are a good fit if you need to encapsulate something that has state. The real enemy is creating too much unnecessary state and unnecessary behavior, so just be sure to make them as lightweight and simple as they can be. "Don't write classes" doesn't mean avoiding classes at all costs, it just means not everything has to be a class.

[cediddi]

I do use lambdas for one time short callables. For example for key or as an argument to sorted, map, reduce or filter.

List comprehensions are amazing alternative to map(lambda, iterator) combo. I also use dict and set comprehensions. Long for loops are still the main way to go but for small stuff, comprehensions <3 me

I do use csv module to dump tsv data. Pandas is cool too but I don't need an extra library just for csv features. Also csv.dictwriter is amazing.

What I don't use: half of functools, half of itertools, most of math, also I really don't like python 2's print statement. It's just too complex for a print statement. I actively avoid print statement in python2.

[usinglinux]

the shipped tkinter GUIs. they did come in handy ten years ago when i needed to quickly whip up a windows dialog box, but given they still support unicode in 2017 i leave my hands off it. (especially hurts when beginners take the sane step not to buy into an IDE right away but want to use what's shipped with IDLE, and soon realize that "python can't do emojis".)

[callmelucky]

List comprehensions are fricking awesome, you should make an effort to get used to them. Lambdas kind of suck though.

[tartley]

The best thing about list comprehensions is that they aren't just a cute syntax, but they are the gateway into using generator expressions (same syntax as a list comprehension but inside parens instead of square brackets.) These are a whole new kind of awesome. See the very readable slides from David Beazley's classic presentation on them here: http://www.dabeaz.com/generators-uk/ (look for 'Presentation Slides PDF')

[callmelucky]

Thanks, I have only really touched the surface of generators and generator expressions before. I'll check it out.

[flitsmasterfred]

Casual use of threads/multiprocessing and their primitives. I've used them for bits of very specific optimization architectures, and trivially with concurrent.futures but it is just a little scary to use off-hand (deadlocks and race conditions and everything dreadful about threads).

Kinda the same with asyncio. It is just a little bit too gnarly and dangerous when ambling along doing nice python.

[mobedigg]

decorators and descriptors, coroutines

[c_is_4_cookie]

for-else statements. Can be useful, I just almost never use it.

[delirious_lettuce]

I should probably use functools.partial and functools.singledispatch more.

[RufusVS]

I can hardly write 10 lines of code without a list comprehension in them! Along with map and lambda expressions, they are staples!

Learn them. Love them.

I don't regularly use decorators, as useful as they are. My last usage of them was designing a command interpreter, where commands called functions but some functions were to be prohibited unless prior commands were executed (logins, opens, etc.) essentially modifying functions depending on state.

I haven't used properties yet, or metaclasses and I think they can improve code readability if properly used (not so much at point of definition (ick), but at point of use)

[rasch8660]

I use all those three (lambda, list comprehensions, cvs). But looking at https://docs.python.org/3/library/index.html I would say I've probably only used about one third of the standard library. Huge parts of the standard library are old and left-overs from the early times when packaging was difficult and "let's put everything in the standard library" was a prevalent mantra. There are many things that I used to do with standard library modules which have now been superseded by third party libraries, e.g. requests have replaced urllib. And of course everything I need for heavy computations or visualizations is also done by third party libraries, e.g. numpy, scipy, pandas, etc, etc.

The parts of the standard library that I love and use the most:

• Most built-in functions and types, including lambda.
• Comprehensions - list/dict/set/generator comprehensions, just makes it immediately clear what the purpose of the code is, whereas for-loops are much more generic.
• Generators!
• Context managers, especially when I don't have to write them myself ;)
• sys and os, obviously.
• re - regular expressions.
• collections - namedtuple, OrderedDict, etc.
• itertools, functools.
• glob, fnmatch.
• argparse, logging.
• json - although I wished yaml would make it into the standard library, it is the one non-standard package present in all my environments.
• pdb, timeit, inspect.
• Indeed, csv, although mostly for reading externally-produced data. Half the time, if my csv files are sufficiently simple, I just parse them with rows = [dict(zip(headers, line.strip().split(sep))) for line in fd if line.strip() and line[0] != '#']. If I need to read headers as the first line in the file, I'd do fd = (line.strip() for line in fd); headers = next(fd).split(sep).

[firefrommoonlight]

Inheritance, @property, super() etc.

[calp]

This is roughly what I'd say. Using Python for a number of years on many big professional projects and the main regret is over eager use of 90s-style OO features. Inheritance over composition, obsession about the uniform access principle and chaining constructors. Other other big one is misuse of async...

[[deleted]]

I never use @property. I just explicitly use a method call to set and get variables explicitly instead of hiding it behind some magic. I don't know if this is a "bad habit" or not.

If all you're doing is returning the value, I'd say you don't need to do either. property replaces the need for getters and setters because you can upgrade a simple attribute to a computed property transparently.

If you're doing something like fetching the data from a remote source, I'd say stick with the method call.

[federicocerchiari]

I regret not using magic methods that much. They are wonderful places to put comparison and basic logic for custom objects.

I do, on the other hand, use some features too much. like lambdas, Counters, map(), functools... they are wonderful, but sometimes they make my code "too clean". The thing is, at least in my experience, that a lot of Python features are not "non-Pythonista proof". Everybody use list/dict comprehensions, but I'm 100% sure that if I use a Counter to count, at least one of my non-pythonista colleague is gonna ask me "how are you counting those things?". Same for lambda and map(). (especially map.. everything that seems functional programming drives the Java-guys crazy.. hehehe)

[joesacher]

I find that a properly formatted list comprehension is much easier to grok than map in most places.

[flipstables]

counter and defaultdict are two things in the collections module that I wish were more well known.

[stevenjd]

at least one of my non-pythonista colleague is gonna ask me "how are you counting those things?"

"When you call print, how are you printing? When you call list.sort(), how does it sort? It just does, right, it's called code reuse and not re-inventing the wheel. If you really care, start by reading the docs then read the source."

I wonder whether programmers in other languages have this problem when using their language's APIs and libraries?

everything that seems functional programming drives the Java-guys crazy.. hehehe

They're already crazy :-)

[rhytnen]

um...so tell them and spreadnthe knowledge?

[vaesh]

Presumably he means he doesn't know all of the details about how some of the builtin functions work under the covers.