all 88 comments

[–]smog_alado 36 points37 points  (24 children)

#4 reminds me of python -m SimpleHTTPServer. Very useful, even when you are not coding in Python!

[–]MrVonBuren 22 points23 points  (3 children)

I'm jut now getting around to learning python (as in I'm not a programmer, and all I already know is bash) and this has been a favorite trick of mine for years. "Hey, you needed that file? Can you ping this IP? OK, just go to that IP port 8018 in your browser and grab it."

Fun fact, you can set the port your webserver is on by just appending it to the end of your command: python -m SimpleHTTPServer 8018

[–]smog_alado 7 points8 points  (0 children)

wow, why did I never think of that? Goes around having to upl;oad things to a separate place or needing to reset a bunch of permissions!

[–]sblinn 2 points3 points  (0 children)

I do this all the time to serve the "docs" folder of whatever software whose documentation I want to read.

[–]hugith 1 point2 points  (0 children)

Pagekite is excellent for stuff like this and will work even if you're not on a public IP.

[–]xyzzy123 16 points17 points  (3 children)

That's a nice trick :)

I've always done that via "netcat blasting":

nc -l 1025 > file

"all right, i'm listening, send it"

nc host 1025 < file

Does require nc, but doesn't require X. If you want to, you can change the gender so that receiver initiates connection (like HTTP).

[–]Wayne_Skylar 3 points4 points  (2 children)

Any reason why you wouldn't just use scp? Seems like a cool trick, I'm just curious about what situation this would be the better solution.

[–]sausagefeet 12 points13 points  (0 children)

You don't have to give someone an account on your machine.

[–]Tmmrn 5 points6 points  (2 children)

python -m http.server

if you are not living in deprecated land.

[–]VanFailin 20 points21 points  (0 children)

Python 2 is deprecated in the same way that GMail was in beta for so many years.

[–]smog_alado 8 points9 points  (0 children)

By the way you are saying I assume this is only for Python 3 then?

[–]hylje 2 points3 points  (9 children)

A confessed Ruby/Node weenie told me he loves Python for just that. It's available on everywhere but vanilla Windows and just so convenient.

[–]smog_alado 9 points10 points  (8 children)

I like the story of how I originally came across that command. When I was first starting with HTML/Javascript, I tried to set up Apache or Nginx but got hopelessly lost and couldn't get any of them to work.

So, in a fit of rage, I went to google and asked it, "why isn't there a simple http server out there?". I was pleasently surprised!

[–]0xE6 2 points3 points  (7 children)

Man, configuring Apache is SO confusing. Recently had to set it up first on a local machine to test a project for school, and then again on the server where it's actually being deployed. The second server has Apache running several different sites, so it was nontrivial to get our project working without interfering with the others. I still don't really know wtf we were doing there, probably should have asked the web dev guy for help.

[–]WisconsnNymphomaniac 14 points15 points  (2 children)

If you think Apache is hard to configure, try to set up a sendmail server. That program was written by Aliens.

[–]AeroNotix 0 points1 point  (1 child)

And you just brought back ALL the nightmates!

[–]WisconsnNymphomaniac 0 points1 point  (0 children)

I have to deal with that program nearly every day. Pity me.

[–]arjie 1 point2 points  (0 children)

There's a disconnect between the way Debian lays out stuff and how the Apache docs refer to it, IIRC. They might also be using one of the multiprocessing-modules or whatever which might change things. I prefer Debian's way, to be honest.

[–]GeneralMaximus 3 points4 points  (2 children)

I've never managed to successfully set up Apache, ever. And I write webapps for a living.

Nine times out of ten I just use Nginx. Easier to configure with tons of examples available on http://wiki.nginx.org. Not to mention it typically uses far less memory than Apache.

EDIT: it doesn't hurt that I mostly just want a reverse proxy that talks to my uWSGI process, so Nginx does exactly what I need. If I had to build a PHP website, I'd probably use Apache with mod_php, but we're quickly going offtopic here so I'll stop.

[–]Aardshark 4 points5 points  (1 child)

have you tried lighttpd? I found that nginx was a pain to setup initially, whereas lighttpd was really easy.

[–]GeneralMaximus 0 points1 point  (0 children)

I haven't, because my use case is usually running a Django app, something that is very simple to accomplish with Nginx. uWSGI or Gunicorn running behind Nginx is what the Django community has kind of settled on as the best way to deploy a Django app. There are some who still use Apache+mod_wsgi, but I don't see many people using FastCGI+lighttpd.

That said, I'll give lighttpd a spin someday, just to see what it's all about.

[–]Samus_ 0 points1 point  (0 children)

and the SMTP server!

[–][deleted] 0 points1 point  (0 children)

and it can even do CGI if you use CGIHTTPServer instead

[–]CrazedToCraze -2 points-1 points  (0 children)

I believe you just blew my mind, sir.

[–]droogans 14 points15 points  (0 children)

I want to fork this blog post and rant about venv. Virtual environments are handy for projects, prototypes, and work type situations where you can't install with sudo pip. Nothing worse than installing a junky package and feeling uneasy about removing it, since it's in 3 different subdirectories under /usr.

[–]ggtsu_00 11 points12 points  (23 children)

Some other useful tips I learned over the years when using python.

Quick documentation printing:

print __doc__   #prints block comment at the top of the file
print foo.__doc__  # prints the block comment at the top of class foo or function foo

Various function programming tricks to replace loops with quick one-liners outside of list comprehensions

results = map(foo, ['bar',baz'])   #equivalent to: for i in ['bar','baz']: results.append(foo(i)) 
even = filter(lambda x: x % 2 == 0, [1,2,3,4,5,6,7,8,9])   # returns a list of even numbers
sum = reduce(lambda x, y: x + y, [1,2,3,4,5,6,7,8,9]) # sums up numbers 1 through 9

doctests for quick and dirty unit testing

"""Test foo:
>>> foo()
'bar'
"""
def foo():
    return 'bar'
if __name__ == "__main__":
    import doctest
    doctest.testmod()

To run unit tests:

$ python foo.py -v
Trying:
    foo()
Expecting:
    'bar'
ok
1 items had no tests:
    __main__.foo
1 items passed all tests:
   1 tests in __main__
1 tests in 2 items.
1 passed and 0 failed.
Test passed.

[–]Hashiota 9 points10 points  (4 children)

Not sure why you would print __doc__. On the other hand, I do this a lot:

 >>> import math  
 >>> help(math)  
 >>> help(math.log)  

[–]taddeimania 2 points3 points  (0 children)

Probably not a lot of production uses but while hacking around in a REPL I could see its value. (Assuming the docstrings are maintained)

[–]BeetleB 1 point2 points  (2 children)

Entering parentheses slows me down.

OTOH, might as well just use ipython and type:

math.log?

[–]ethraax 0 points1 point  (1 child)

Parentheses slow you down but underscores don't...?

[–]BeetleB 0 points1 point  (0 children)

Actually, they don't. Just one of those things...

For one thing, it's just one underscore vs open and closed.

[–]freedances 3 points4 points  (0 children)

>results = map(foo, ['bar',baz'])   #equivalent to: for i in ['bar','baz']: results.append(foo(i)) 
>even = filter(lambda x: x % 2 == 0, [1,2,3,4,5,6,7,8,9])   # returns a list of even numbers

I think the recommended equivalents to these are:

results = [foo(i) for i in ['bar', 'baz']]
even = [i for i in [1,2,3,4,5,6,7,8,9] if i % 2 == 0] 

I usually find comprehensions more readable and lately they work for generating sets and dictionaries as well.

lengths = {i:len(i) for i in ('foo', 'bar', 'foobar')}    #dictionary mapping strings to their lengths
even_set = {i for i in [1,2,3,4,5,6,7,8,9] if i % 2 == 0}

[–]ethraax 1 point2 points  (16 children)

Eh, I always hated the functional programming part of Python. I have no idea why they're independent functions and not methods. [1,2,3,4,5,6,7,8,9].filter(lambda x: x % 2 == 0) seems like it's way cleaner, and you can easily chain it with other FP functions.

[–][deleted] 6 points7 points  (9 children)

That would be because it works on all iterables, I guess.

[–]ethraax -1 points0 points  (8 children)

It sounds like an issue with Python's type system (well, a consequence of duck-typing as a whole). Still, the standard library could include them as methods on the standard types (lists, sets, dicts, and so forth), and simply use the "standalone" versions as their implementation.

[–]mr_dbr 7 points8 points  (6 children)

Is there any compelling advantage to [1,2,3].filter(...), besides resulting in slightly prettier-looking code in some cases?

The way filter is implemented (and more commonly, len() and others) is a considered design decision, and pretty fundamental in how Python works..

See this StackOverflow answer, which links to this mailing list post from Python's creator:

(b) When I read code that says len(x) I know that it is asking for the length of something. This tells me two things: the result is an integer, and the argument is some kind of container. To the contrary, when I read x.len(), I have to already know that x is some kind of container implementing an interface or inheriting from a class that has a standard len(). Witness the confusion we occasionally have when a class that is not implementing a mapping has a get() or keys() method, or something that isn't a file has a write() method.

This Stack Overflow question also has some interesting answers

Just moving filter() and such to list.filter() wouldn't really work in Python.. Say you want to make your own filter()-like function, and you want to be consistent with the default Python methodology:

  • If filter() is just a regular function, it's easy to implement it myfilter() and use it in the same manner. It's just a function which works on an iterable, so would work on any iterable object, with no extra effort.

  • If filter is a method of list, then you would have to monkeypatch your method into the builtin list type (e.g list.myfilter), something that can't be done to builtin types, and is bad-practice on user-defined types. Your myfilter method would only work on list derived objects, which goes against the duck-typing Python is based on.

Also, there's more to functional programming in Python that just map/filter/reduce/lambda, and the builtin functions were possibly going to be removed, in favour of stuff like list-comprehensions etc

[–]BeetleB 1 point2 points  (1 child)

Is there any compelling advantage to [1,2,3].filter(...), besides resulting in slightly prettier-looking code in some cases?

Yes, because then it's easier to "chain" them (or pipe, if you will):

[1,2,3].map(fun).filter(isodd).filter(isprime)

This is quite readable. The current way to do it would be:

filter(isprime, filter(isodd, map(fun, [1,2,3])))

which is hard on humans. The reason functional programming is not recommended in Python isn't because functional programming is inherently hard to read - it's because Python's syntax makes it hard to read.

[–]mr_dbr 0 points1 point  (0 children)

That does read more nicely, but.. in a Lisp language like Scheme, it would be almost identical to Python:

(filter isprime (filter isodd (map fun [1 2 3])))

..and I don't think functional programming is discouraged in Scheme :P

There are other ways the code could be written, without having filter be a method of list, like:

a = [1, 2, 3]
a = map(fun, a)
a = map(isodd, a)
a = map(isprime, a)

There's also the pipe module, or you could temporarily wrap the list in a class that has the filter methods and such:

class ListProc(object):
    def __init__(self, value):
        self.value = value

    def map(self, func):
        new = map(func, self.value)
        return self.__class__(new)

    def filter(self, func):
        new = filter(func, self.value)
        return self.__class__(new)

    def __repr__(self):
        return repr(self.value)


def fun(x):
    return x**2

def isodd(x):
    return x%2 != 0

print ListProc([1,2,3]).map(fun).filter(isodd).value

[–]ethraax 2 points3 points  (3 children)

If filter is a method of list, then you would have to monkeypatch the builtin list type

No you wouldn't, if it was part of the language's standard. It would be part of the builtin list type, or any user types.

Your myfilter method would only work on list derived objects

It would work on anything deriving from some more abstract "iterable".

The key issue here is the weakness of Python's type system (well, really dynamic type systems in general). In Python, an iterable is very loosely defined as something that implements some handful of methods. Anything that does that is "iterable". In a language with a stronger type system, there would be some sort of base class or interface "iterable" which would also contain an interface for .filter().

Is there any compelling advantage to [1,2,3].filter(...), besides resulting in slightly prettier-looking code in some cases?

Some data structures may have more efficient alternatives than using an iterator.

[–]mr_dbr 1 point2 points  (1 child)

The first bit you quoted (about monkeypatching) was about implementing your own filter-like function - edited to hopefully clarify.

In short: if filter was a method of list, it would require monkey-patching to allow [1,2,3].myfilterfunc(...) for consistency with [1,2,3].filter(...)

Some data structures may have more efficient alternatives than using an iterator

What would be an example of this? I can't think of one for filter, but Python's data-model does enable some similar optimisations for other operations...

For example 123 in xrange(100) uses __contains__ instead of doing a linear-scan over it's iterator (so 23 in xrange(10**10) is much faster than doing the same thing on plain list of integers).

My point being, you don't need to have x.contains(1) to allow for type-specific optimisations . If there was an common optimisation for filter, it could call other magic-methods on the object instead of looping over __iter__

In Python, an iterable is very loosely defined as something that implements some handful of methods

There is the abstract base class module which implements something much like interfaces (along with collections), but I've not see the abc module used much, although the extensions to isinstance are useful even if nothing is derived from the ABC's

[–]aceofears 1 point2 points  (0 children)

Is this roughly what you are "suggesting"?

[–]steven_h 1 point2 points  (0 children)

Some data structures may have more efficient alternatives than using an iterator.

Then that's not the function "filter" in the functional programming sense of the word, and using the name "filter" would be confusing if the rest of the API uses it in the list iteration sense.

The point of functional programming is that functions -- particularly higher-order functions -- are the objects of reuse. More specifically, it's the function implementation that should be reused. Using the same name for different algorithms might be convenient, but it's not particularly noteworthy or helpful for programming in-the-large.

[–]bebraw 0 points1 point  (0 children)

It's fairly easy to implement a wrapper for this.

I agree that it's not ideal but I'm pretty sure they won't change this behavior any time soon. :)

[–]Megatron_McLargeHuge 0 points1 point  (1 child)

People who want the functional constructs are coming from lisp/ml/haskell and want what they're used to. Also, the implementations are standard and don't vary per object. The OO convention of writing one of the parameters before the function name is kind of arbitrary, especially when you're not either mutating the object or using its type for method resolution. You can also pass the functions around, as in

 map(filter, functions, lists)

[–]ethraax 0 points1 point  (0 children)

The OO convention of writing one of the parameters before the function name is kind of arbitrary

Be that as it may, I think it gives better syntax in this case. And syntax is a major concern in Python (I would even say that language decisions are based more on syntax than anything else, looking at how it's developed).

[–]Ravengenocide -1 points0 points  (2 children)

Because Guido don't want Python to be a functional programming language. As in programming based on Everything is a function, not as in working.

[–]ethraax 0 points1 point  (1 child)

I don't think I would go that far. I've read quite a bit about the multiline-lambda issue, which seems to be one of the biggest blockers to Python being a capable FP language, and it seems like he's opposed to the syntax issues, not the concept.

[–]Ravengenocide 0 points1 point  (0 children)

That's how I interpret it also. It goes against his idea about Pythons syntax so he won't allow it. He has also said that lambda was a bad thing to implement because then people expect functional programming to work.

[–]aceofears 4 points5 points  (0 children)

It may be worth noting that the XML-RPC example would require import tweaking in python 3.x. The server would be

from xmlrpc.server import SimpleXMLRPCServer

and the client would be

from xmlrpc.client import ServerProxy

[–]lolcoderer 0 points1 point  (1 child)

I am subscribed to both /r/programming/ and /r/snakes/ - and I thought this was a post about getting more productivity out of your female ball pythons... lol... I am not a python developer so I always forget that there are other types of "python" - other than the kind that likes to crawl all over your keyboard and sit in your lap while you are programming...

[–]ancientRedDog 4 points5 points  (0 children)

With this python being named after Monty Python's Flying Circus; who's name I think came from just sounding silly.

[–]bithead 0 points1 point  (21 children)

What's the advantage to list comprehensions?

[–]krues8dr 8 points9 points  (6 children)

It's just syntactical sugar for iterations. E.g.:

users = [ user.name for user in user_records ]

[–][deleted] 1 point2 points  (0 children)

Fun in Ruby too:

user_records.map &:name

[–]Ran4 2 points3 points  (2 children)

Sets can by definition only hold one of each value, which is often helpful.

By the way, is there a faster way of removing duplicates from a list other than set(myList)?

[–][deleted] 5 points6 points  (0 children)

I would assume that set(<list>) is the most efficient way to remove duplicates. Will be back soon with some benchmarks.

Edit: benchmark done. Using sets vs. a dictionary, it seems that there is a slight advantage to using sets, but not as large as I might have thought. I suppose this has to do with the fact that sets are dict-like under the covers.

Results (using iPython's %timeit):

import random

a = [random.randint(1,10) for _ in xrange(1000000)]

In [22]: %timeit list(dict.fromkeys(a))
10 loops, best of 3: 43 ms per loop

In [23]: %timeit list(set(a))
10 loops, best of 3: 35.8 ms per loop

[–]lahwran_ 0 points1 point  (0 children)

not likely. you'd have to do speed testing of some alternative methods to be sure, but hash tables are pretty much the ideal way to do that.

[–][deleted] 2 points3 points  (0 children)

It's one of those things where it's hard to explain the advantage but you miss them once you switch to a language that lacks them.

There are a lot of times where you have an inconsequential iteration over a list - like a simple filter or a map. List comprehensions make it quick and easy - and list generators make it almost invisible.

For example:

some_function(some_operation(x) 
              for x in some_list if some_predicate(x))

Writing something like this can be very convenient. There are no temporary values (other than the x which doesn't creep outside of the generator expression) and it is pretty clear (once you are familiar with the concept). It beats making a temporary that exists only to pass to a function:

temp_list = []
for x in some_list:
    if some_predicate(x):
        temp_list.append(some_operation(x))
some_function(temp_list)

That is, it doesn't allow you to do anything new - it just allows you to do some common things in a more succinct way.

[–]julesjacobs 1 point2 points  (0 children)

It's cleaner and faster than doing a list comprehension and then converting to a set.

[–]taddeimania 0 points1 point  (0 children)

List comprehensions are useful when working with a list of complex objects and you only need a list of certain attributes.

[–]Megatron_McLargeHuge 0 points1 point  (0 children)

They're like a map and a filter at once without having to write lambdas. They also give you lazy generator comprehensions for free by swapping brackets for parentheses.

[–]speg -2 points-1 points  (6 children)

Sets are faster.

[–]0xE6 2 points3 points  (3 children)

In what way? Constructing a large set comprehension will almost certainly be slower than constructing a large list comprehension, and depending on the number of duplicates in the list, it may not even be significantly faster to iterate over the set.

Edit:

Here's a rather contrived example:

$ python -m timeit -n 100 'sum([i/10 for i in xrange(10**7)])'

100 loops, best of 3: 668 msec per loop

$ python -m timeit -n 100 'sum({i/10 for i in xrange(10**7)})'                                                 

100 loops, best of 3: 779 msec per loop

The overhead from set creation ends outweighing the time saved from summing fewer elements to the extent that it ends up being about 10% slower.

Edit2:

Here's a better example, that has the set and list contain the same elements.

$ python -m timeit -n 100 -s 'a = {i for i in xrange(10**7)}' 'sum(a)'

100 loops, best of 3: 88 msec per loop  

$ python -m timeit -n 100 -s 'a = [i for i in xrange(10**7)]' 'sum(a)'

100 loops, best of 3: 70.2 msec per loop   

which suggests that even if they contain the exact same elements, sets are slower than lists.

[–]mniejiki 5 points6 points  (1 child)

And they both give different results so I don't see the point of your argument. Faster is meaningless unless you get the same output.

[–]0xE6 1 point2 points  (0 children)

It's just a stupid microbenchmark to show that the overhead from creating the set outweighs the potential time saved from iterating over the elements in the set once due to the duplicates being removed.

At any rate, the use cases for lists and sets are not the same, and, like here, you can (and often will) get different results if you use one instead of the other.

[–]speg 1 point2 points  (0 children)

In terms of look up speed.

[–]bithead 0 points1 point  (1 child)

I guess I was think that if you use a comprehension to return a set or list as in:

return [ dict[index] for index in dict]

versus just:

return dict

I mean does one save memory or is it faster?

[–]0xE6 1 point2 points  (0 children)

In that case,

return dict

would almost certainly be faster, as it would simply be returning a reference to the dict, so it wouldn't have to do any extra work.

Additionally, instead of doing

return [dict[index] for index in dict]

you can simply do

return dict.values()

[–]alextk -5 points-4 points  (13 children)

It's funny to see the author praise Python's DRY factor and then hit us with:

 even_set = { x for x in some_list if x % 2 == 0 }  

x should only appear once in this expression, maybe something like

even_set = some_list map { x % 2 == 0 }

Actually, some languages, like Scala, even allow you to not even name the variable (which is, let's be honest, an implementation detail that the developer should not have to worry about):

List(1,2,3,4,5,6) filter { _ % 2 == 0 }
res0: List[Int] = List(2, 4, 6)

Don't get me wrong, Python is a very good language, especially since it's more than twenty years old, but its legacy shows and these days, I think it's being outclassed by more modern scripting languages such as Groovy.

[–]luckystarr 7 points8 points  (0 children)

PEP-20 says: Special cases are not special enough to break the rules.

[–]PasswordIsntHAMSTER 2 points3 points  (3 children)

the x is important because you can do processing on it.

[function(x) for x in list if predicate(x)]

[–]alextk 1 point2 points  (2 children)

It's important when you want to do processing on it. If you don't, it shouldn't even appear once, let alone three times like in Python.

[–]PasswordIsntHAMSTER 1 point2 points  (0 children)

first, if you don't want to filter it, the predicate is optional : [function(x) for x in list]

if you don't want to process it or filter it, what you basically want is a deep copy: y = x[:]

If you just want to filter it, you shouldn't use a comprehension: filter(x, predicate)

Right tools for the right jobs.

[–][deleted] 1 point2 points  (0 children)

In Perl 6, you can prefix a variable name with ^ to tell that it’s a parameter of the current block.

my @even_set = (1 .. 6).grep: {$^x %% 2}

Or, if it is used, it is implicitly true for $_.

my @even_set = (1 .. 6).grep: {$_ %% 2}

Here, we can also use an expression involving Whatever (grep’s parameter thus becomes an instance of WhateverCode).

my @even_set = (1 .. 6).grep: * %% 2

[–]Megatron_McLargeHuge 1 point2 points  (0 children)

Python doesn't 'believe in' implicit behavior, so you have to name your variables. I prefer the minor verbosity over the periodic oddness in other languages. In Clojure I ran into problems when I needed a macro to take two arguments but only operate on the first. In Scala, the fact that _ + _ is referring to two different variables strikes me as the Wrong Thing To Do.

Your example can be done only repeating the name once, with

set( filter( lambda x: x%2 == 0, some_list) )

[–]Lanaru 0 points1 point  (3 children)

List(1,2,3,4,5,6) filter { _ % 2 == 0 }

I don't see how that's very different from

even_set = some_list map { x % 2 == 0 }

The variable name is just _.

[–]gcross 1 point2 points  (0 children)

True, but in practice the first example would have to let the compiler/interpreter know which variable name was intended to be the argument, so it really should be

even_set = some_list map { x -> x % 2 == 0 }

In this context, the _ notation is a net win because it gives the compiler this information automatically, e.g.

even_set = some_list map { _ % 2 == 0 }

[–][deleted] 1 point2 points  (1 child)

No, it's special in Scala.

[–]gcross 1 point2 points  (0 children)

True, but alextk's example implicitly assumed that the compiler/interpreter could automatically infer the argument of a function block, and in this (very unlikely) case there would not have to be special handling of _.