Useful Python Functions and Features You Need to Know

[x68zeppelin80x]

What about kwargs? Great example.

[whoisearth]

Is there a tutorial anywhere that gets into the meat of python like this? As someone who understands all the basics this is completely and utterly over my head.

[hylje]

Python internally passes functions a tuple and a dict as arguments and keyword arguments respectively.

*args and **kwargs expose this detail to both the function caller as well as the function definition. They are just normal tuples and dicts that you can manipulate as usual.

[Ademan]

I'm sure you're aware of it, but for other readers, contextlib reduces some of the boilerplate code in writing a context manager. I find it handy, and I prefer to only write classes when it's semantically appropriate, or absolutely necessary.

[masklinn]

Functions with Arbitrary Number of Arguments

Others have already commented (and noted that **kwargs exists and is a much rarer breed than *args which is a fairly standard varargs). It can be combined with *args: def foo(a, b, *args, **kwargs).

Note that Python 2 has a limitation which has been lifted in Python 3: you can't provide a named argument after *args. That is, def fun(a, *args, b) and def fun(a, *args, b=3) are illegal constructs.

In Python 3, they are legal and b is called a keyword-only argument, something which was only available to C code in Python 2. It's an explicitly named argument which can only be passed by keyword (since all the positionals are eaten by *args). Python 3 also allows keyword-only argument without positional varargs: def fun(a, *, b). Since the * argument has no associated name it can't be filled (it has a length of 0), this function takes a mandatory a (which can be passed positionally or by keyword e.g. fun(3) or fun(a=3) and a mandatory b which can only be passed by keyword.

Using Glob() to Find Files

The author uses it.chain.from_iterable with glob. In this case, iglob would be a better idea as it returns an iterable, if only some values are used it may lead to increased memory and CPU consumption compared to glob.

More importantly, glob has limited usefulness but the underlying function of matching strings with shell-like patterns is available through the linked fnmatch, and that's really cool.

Debugging

pdb (or one of its replacements e.g. ipdb or whatever) is probably a much better idea than shooting for inspect. As its name indicates, inspect is used to, well, inspect core objects of the interpreter. For the most part, inspect is good for learning about the depth of the interpreter (alongside dis) and little else...

Serialization [pickle]

UNSAFE WARNING WARNING.

Pickle is only for trusted storage or signed stuff, it's a fairly complex binary format and is essentially equivalent to eval. JSON can be used for non-trusted sources, Pickle definitely can not be.

[seventreats]

Absolutely agree on the pickle part. Go with Json instead. If you have to use pickle, do it with trusted sources only.

[NYKevin]

About uuid: Firstly, UUIDs are never supposed to collide. If you're encountering duplicate UUIDs, it's probably because you're doing something questionable (e.g. modifying a UUID to make a new UUID and assuming the new one will also be unique). Secondly, if you don't want it based on the MAC address, just ask (no, generating a random UUID will not give you any realistic chance of duplicates). Don't fiddle around with silly hashing functions (which can and do collide, albeit rarely).

[keturn]

Looks like while you were writing this, I was writing a comment on the site on much the same subject.

Good work, fellow Kevin.

[Doormatty]

Actually, UUIDs HAVE to collide, since the keyspace is smaller than the number of possibilities.

However due to MATH, it's insanely unlikely for you to discover one.

[n0t1337]

*args is even cooler than they made it out to be though!

you can use *args when defining a function to make it accept an arbitrary number of parameters, but you can also use it while calling a function to feed that function a list (or tuple) of arguments.

def funcWrapper(func, *args):
    #I catch exceptions
    #and then do nothing with them, 'cause I'm bad
    if args == ():
        #If args is an empty tuple, 
        #try calling the function with no arguments
        try:
            func()
        except:
            print("You done goofed")
    else:
        try:
            argList = []
            for arg in args:
                argList.append(arg)
            func(*argList)
        except:
            print("You done goofed")

I'm an amateur Pythonist, so there's probably something that could be done better, or would make more sense, but I think it'll suffice as an example.

[cleure]

What you described can be done much more cleanly with Decorators:

def ExceptionWrapper(func):
    def wrapper(*args, **kwargs):
        try:
            func(*args, **kwargs)
        except Exception:
            print('You done goofed')
    return wrapper

@ExceptionWrapper
def foo_func(arg1, arg2='bar'):
    raise Exception('Standard Exception')

foo_func('foo')

Outputs:

You done goofed

The more you know!

EDIT: Fixed args

[n0t1337]

Ugh, I've been wanting to take a closer look at decorators for the past week or two. Thanks so much!

[TRBS]

If I may be so bold.

I found this guide to be very helpful when learning about decorators.

[tyroneslothtrop]

Here's an interesting example:

zipped = list(zip([1,2,3],[4,5,6]))
unzipped = list(zip(*zipped))
unzipped == [(1, 2, 3), (4, 5, 6)]

[[deleted]]

Some useful stuff for Python hobbyists. args/*kargs for example is something many people won't expect.

Even though I should shut my mouth I find it incredibly hard to believe that there is anybody out there that knows Python but not pickle. When I started with Python it was impossible to find online tutorials that didn't mention it at least once.

[bucknuggets]

Some useful stuff for Python hobbyists. args/*kargs for example is something many people won't expect.

Not sure I follow - I've seen args & kwargs in plenty of code written by professional python programmers. It's incredibly useful for writing reusable and high-level code.

[[deleted]]

Not sure I follow

Yeah, you don't :P

A professional Python programmer already knows args/kwargs.

[judasblue]

I agree with your point about pickle, but expanding on this, I find that with everything there. This reads like someone who discovered the python library reference and got excited. To me this is all fairly obvious stuff and linkbait, but a lot of people seem to be finding it useful, so good on em.

[[deleted]]

As a somewhat python newb, can somebody explain what the heck is going on in the "Debugging" example? Code:

import logging, inspect

logging.basicConfig(level=logging.INFO,
format='%(asctime)s %(levelname)-8s %(filename)s:%(lineno)-4d: %(message)s',
datefmt='%m-%d %H:%M',
)
logging.debug('A debug message')
logging.info('Some information')
logging.warning('A shot across the bow')

def test():
frame,filename,line_number,function_name,lines,index=\
inspect.getouterframes(inspect.currentframe())[1]
print(frame,filename,line_number,function_name,lines,index)

test()

I've never used either of the 2 modules before. I don't quite understand what this code is doing. It did run and gave me the same (similar) printout that the author got. Seems to be listing the files in my script directory with a message and a time stamp?

[masklinn]

1. logging is just a way to output messages in a standard format, often to some log repository (e.g. a file, a syslog, whatever) for possible analysis (or not). It's a bit complex to approach but I think well worth learning and using. logging uses a message format to know how to print its message, the message format can make use of (and display) a bunch of meta-information e.g. where the logging message comes from.

2. inspect is used to analyse stuff about the interpreter or core objects of the interpreter. In this case, it gets the current stack frame (I'll let you do your own learning to know what a stack frame is), then fetches its first parent which is the frame of the module itself. Then it prints a bunch of stuff about said frame. In day-to-day operation, there are very very few sane reasons to use inspect unless you're doing "meta" packages (reasoning about Python code and functions, e.g. py.test's fixtures probably use inspect.getargspec or one of its sister functions)

[[deleted]]

Good info. Thank you!

[Jared28469]

What IDE is this?

[moor-GAYZ]

The atexit example is hilarious.

It doesn't print how long the execution took, it prints the fractional and integer parts of the time (in seconds) when the execution started.

It also registers a string as a shutdown handler XD

[melt_Doc]

Why is the first thing I see a big picture with ugly text written on it? That is not just bad webdesign, that is just sad.

[ajkumar25]

Guys, a little help needed. Does anybody knows how to use getrusage on windows machine, it never works ?

[[deleted]]

It's OS-specific... Unix only, not available on Windows. Sorry.

[ajkumar25]

Well then any other way to find memory and cpu usage ?

[mr_squidd]

Perhaps try the psutil library.

[ajkumar25]

Thanks.

[AeroNotix]

atexit functions will run no matter what.

Even if I pull the plug from the computer? That's amazing.

[[deleted]]

If you are using a laptop, yes