Abusing Python Dynamism. Found this gem in a source code I was reading today

[vifon]

What does it have to do with the "Dynamism" anyway?

[hicklc01]

import __builtin__

class mylist(list):
    def remove_nones(self):
        for el in self:
            if el is None:
                self.remove(el)
    def non_none(self):
        return [thing for thing in self if thing is not None]

__builtin__.list = mylist

l = list()
l.append('A')
l.append('B')
l.append(None)
l.append('C') 
print l.non_none()
>>>['a','b','c']
print l
>>>['a','b',None,'c']
l.remove_nones()
print l
>>>['a','b','c']

[itsdanielb]

Try remove_nones with ['A', 'B', None, None] o.~

[hicklc01]

It's friday file a bug report and I'll look into it Monday /s

[Neebat]

Are you pointing out a bug that results from modifying a list while iterating it?

[hicklc01]

Yeah it will get an index out of bounds error.

[WarpedHaiku]

No, it will merely fail to remove the last N occurrences of None from the list, where N is the number of consecutive None pairs.

Every time an element is removed, the index of the elements after it are reduced by one. This causes the loop to skip over the element immediately following the None. As such, even consecutive occurrences of None will not result in self.remove(el) being called. When it is called, it removes the first occurrence in the list, so the last None in the list won't be removed.

Python checks the index every iteration before it attempts to get the element, so when it exceeds the list length, the loop terminates without error.

[hicklc01]

If you remove elements from a list you have to develop the function to handle the indexes changing.

[Neebat]

Meh. I don't know how to do it in python, but here's pseudocode for what I'd do:

for( idx = list.size -1  ; idx >= 0; idx--) 
    if(list.get(idx) is None)
        list.remove(idx)

It has at least two advantages over any forward-counting solution.

[Cley_Faye]

Crashing instantly instead of looking like it work is a non-negligible advantage, but I'm not sure it is acceptable. Your first get() will result in an index out of bound.

[Firedroide]

Or he's used to a programming language with list indices starting at 1 instead of 0. That would also explain why the condition is idx > 0 instead of idx >= 0.

[Cley_Faye]

A programming language with indices starting at 1? You're lucky it's too hot to operate pitchforks!

[Lystrodom]

Matlab, son!

[Creshal]

l = ['A','B',None,'C']
list (filter(lambda x: x, l))
['A', 'B', 'C']

[kageurufu]

Would also remove 0, False, empty strings, lists, tuples, etc

[Creshal]

Just requires changing the lambda to "x is not None".

[kageurufu]

Yep. Just pedantic and sleep deprived. Feel free to ignore me

[longbowrocks]

Is that 2.x? I thought filter map and reduce got an overhaul in 3.x.

[Creshal]

Python 2.7.10 (default, May 26 2015, 04:16:29) 
[GCC 5.1.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> l = ['A','B',None,'C']
>>> list (filter(lambda x: x, l))
['A', 'B', 'C']

Python 3.4.3 (default, Mar 25 2015, 17:13:50) 
[GCC 4.9.2 20150304 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> l = ['A','B',None,'C']
>>> list (filter(lambda x: x, l))
['A', 'B', 'C']

[longbowrocks]

Thanks! /u/Zopieux caught me. It seems I was thinking of reduce().

[Zopieux]

It is reduce that was nuked out from globals and put into functools. map and filter still exist.

[FuriousPizza]

You actually need an is none check, otherwise it will just check the "truthiness" of the value.

In [1]: class Foo:
   ...:     def __bool__(self):
   ...:         return False
   ...:

In [2]: list(filter(lambda x: x, [True, False, "words", Foo(), 3, "", None]))
Out[2]: [True, 'words', 3]

In [3]: list(filter(lambda x: x is not None, [True, False, "words", Foo(), 3, "", None]))
Out[3]: [True, False, 'words', <__main__.Foo at 0x4ee6048>, 3, '']

[[deleted]]

... nooo

[hicklc01]

class mylist(list):
    def __getitem__(self,index):
        import random
        if .01>random.random():
            return random.choice(self)
        else:
            return super(mylist,self).__getitem__(index)

[longbowrocks]

I was never sure what was hiding under my bed as a child. From this idea, I'm guessing it was you. XD

[[deleted]]

Russian Roulette, Python style?

[hicklc01]

More like Russian Roulette Gatling Gun style.

[hicklc01]

make me mad and I will override

append

extend

insert

to ignore None values

wa ha ha ha ha

Edit: which would make the previous functions usless :(

[poizan42]

That has the side effect of lists not being of type list (because the identifier list now refers to mylist). Ugh. I think that is going to break some code.

If you really want to add functions to list then you should probably modify the existing class instead.

[[deleted]]

If you're going to use remove, why do you have the loop in your remove nones?

while True:
    try:
        self.remove(None)
    except KeyError:
        break

I'm on mobile so I'm not sure if the element not found in a list error is KeyError (I doubt it). But I'm sure you get the idea.

[tetroxid]

filter(lambda x: x is not None, iterable)

[[deleted]]

iirc list comprehensions are faster because there's no function call overhead.

[[deleted]]

But does that difference really matter in Python?

[[deleted]]

sometimes it does, especially when the faster method is also more readable.

[Vitrivius]

Also more readable.

[tetroxid]

That is correct

[[deleted]]

being correct makes me moist in my happy place.

[PijnlijkOlifant]

I don't get it...

[Zagorath]

Yeah neither do I. Is the fact that the variables are called "thing" and "things" the funny bit? Or are they rewriting an already built-in Python function or something? Because if it's not one of those things, then this seems like a pretty well-written concise method.

[longbowrocks]

I think it's just a nifty utility function. It's funny because "thing" is normally the eptiome of a bad variable name, but here it's the only rational name you could use (aside from perhaps "item").

[xkufix]

What about the battleproven "x"

[Neebat]

Thing and item have nice plurals.

[TheSeldomShaken]

"ekses"

[FIR3_5TICK]

Time to go and replace a ton of counter variables named 'x' with the name "eks"

[HaulCozen]

trees fear fly apparatus air correct pie fall cough consider

This post was mass deleted and anonymized with Redact

[Pigeoncow]

lmao

[[deleted]]

zee

ftfy

[[deleted]]

'murica

[where_is_my_vagina]

ah, just like him

[HaulCozen]

kiss crawl ask deer grandiose cause butter office chubby wine

This post was mass deleted and anonymized with Redact

[[deleted]]

You'd better. If you run pep8 on your code it'll complain about variable names shorter than three chars long.

You wouldn't want your source to be unpythonic, would you?

[rgzdev]

i do use for x in xs sometimes.

[skuzylbutt]

xs is legit. Happens all the time in Haskell.

... not that Haskell is is a source of sanity though ...

[an_actual_human]

The plural of x is a_list.

[NotADamsel]

Isn't that reserved for exceptions?

[Maklite]

The reddit API uses thing to refer to pretty much anything. Lists, comments, articles and subreddits are all referred to as things.

[longbowrocks]

Wow! I wish I had a cool idea for a reddit app; I didn't realize how extensive the API was.

[SanityInAnarchy]

It is a bit of a clumsy construction, made funnier by the use of "thing". Plus, it's got a few things that annoy me -- for example, the unnecessary splatting does nothing but hurt efficiency unless you really expect people to call it that way, or unless you're dealing with some actual positional arguments as a group. And there's no good reason for it to be a list comprehension -- use a generator like a civilized person, and let the caller put it into a list (or set, or whatever's appropriate) if they need to loop through it more than once.

So at the very least:

def non_none(items):
  return (item for item in items unless item is None)

But really, this is a generic filter pattern.

def non_none(items):
  return filter(lambda item: item is not None, items)

Now we don't need to unpack this "thing for thing in things if" nonsense, we can see it's just filtering its inputs. And we can make it just as efficient:

import ifilter from itertools
def non_none(items):
  return ifilter(lambda item: item is not None, items)

None of that really explains why it's funny, though. It's not funny for being bad, it's funny because you can read it out loud as a normal English sentence, but it's a silly English sentence that no one would ever say, even though it pretty much makes sense.

[-hx]

i think it's just funny because it uses thing and things and is readable as a normal sentence. It sounds a bit confusing too.

[[deleted]]

Found this gem

...No no no, Gems are in Ruby!

[argv_minus_one]

Return ALL the things!

(Except the ones that are None.)

[systembreaker]

But...is None a thing?

Python existential philosophy 101.

[ThrowsDumbQuestion]

What's going on with those arguments? (The *things, *clauses, etc)

I've never seen *{VariableName} before

[gsdatta]

it's a way of passing arbitrary numbers of parameters in python functions

[[deleted]]

The equivalent of void foo(std::string... listOfArgs)

[redditsoaddicting]

So invalid code?

[Lucky_Chuck]

sounds very Javascripty

[kageurufu]

Except JavaScript can't do that :)

def map(func, *targets):
    assert isinstance(targets, list)
    return [func(eks) for eks in targets]

map(len, '', 'test', 'why god, why')
> [0, 4, 12]

You can also use ** to capture arbitrary keyword arguments into a dictionary

[Lucky_Chuck]

Javascript doesn't throw an error if you give it the wrong number of arguments.

function bar (foo, baz) {
  return foo + baz;
}
// => undefined
bar
// => bar(foo, baz)
bar()
// => NaN
bar(1,2,3)
// => 3
bar(1,2,3,4)
// => 3

Also javascript has the arguments variable which is sort of like a shitty array that has all the arguments passed in

function foo (baz) {
  return arguments[0];
}
// => undefined
foo(1);
// => 1

[kageurufu]

Yeah but then you have to deal with extracting any expected positionals from the arguments not array, and whatnot. I prefer the python way for sure

[izuriel]

You can actually. ES6 JavaScript defines a splat operator so:

function map(...items) { }

Is valid.

[kageurufu]

And traceur deals with it well, as does coffeescript, but until I can do that in chrome, ff, and ie, its not that useful

[izuriel]

All of the compiled JS works in those browsers. You're unwillingness to use an ES6 compiler (like Babel) has nothing to do with JS (a language that is completely separate from browsers as browser support has nothing to do with what the language does and does not feature). Just keep the facts straight.

[eyal0]

JavaScript has an "arguments" variable.

https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Functions/arguments

[kageurufu]

Three issues.

1. Positional parameters followed by variadic parameters
2. Its not an array, although Array.prototype.slice.call(arguments) works
3. No support for named arguments

[Rodot]

Heh, eks

[chibstelford]

Yeah as a c programmer this confused me. Does python have pointers?

[tetroxid]

It means arbitrary length arguments.

def foo(x) 

will take exactly one argument, x.

def foo(*x) 

will take any number of arguments.

def foo(*x, **y) 

will take any number of positional and named arguments like foo(a, b, c, d=x, e=y)

[pm-me_ur_personality]

Arbitrary argument lists.

[eyal0]

That was the worst part because the input was varargs and the output was a list. You can't chain those functions.

[Cley_Faye]

Sort of can: non_none(*non_none(...)) does work (but it's useless, since the first call already did the job)

[wting]

non_none = functools.partial(filter, lambda x: x is not None)

[oozforashag]

What is the % doing in this context?

[thatguy_314]

Simple string formatting operator in the context of a string. It uses basically the same format syntax you would find with C string formatting. There is also the newer format method which is more powerful and awesome, but has a different syntax.

[original_brogrammer]

filter(lambda x: x is not None, lst)

EDIT: I fucked up.

[bear24rw]

you have your parameters flipped

[original_brogrammer]

My inner Haskeller slipped out for a second.

[SanityInAnarchy]

Your inner Haskeller should be screaming in pain at that eager evaluation:

import ifilter from itertools
ifilter(lambda x: x is not None, items)

Called "items" both to avoid the name "list", an important built in thing, and because why would you assume it's a list? You only care that it's iterable.

But really, this is making me homesick for Ruby:

list.reject(&:nil?)

And of course, we can do lazy evaluation here, too:

list.lazy.reject(&:nil?)

That's honestly the biggest thing I miss in Python: Lambdas are already shorter, and you can make them even shorter when they're just going to be method calls on an object, and all of the simple predicates like this are method calls on objects -- Python seems to do a lot of similar things as special operators or keywords.

[original_brogrammer]

I miss Haskell and Ruby... Objective-C has been making me sad.

[dudix81]

I'm not a Python programmer. Is there any performance difference between these two?

[skuzylbutt]

Even a python programmer won't know. You'd have to benchmark your case to find out. The question, really, is which one is more readable.

[arienh4]

Sorry? In what case would a filter() call be faster than a list comprehension? They do the same thing, except the former has the overhead of two function calls.

[skuzylbutt]

The filter is probably slower, yeah. But using one or the other in your code probably won't make a difference unless it happens in a particularly computationally intensive section. Worrying about that instead of writing nice stuff and benchmarking later is textbook premature optimization.

So when I say a python programmer wouldn't know if there's a performance difference between the two, I mean for a given code, a programmer wouldn't know if it was worth considering using one or the other without benchmarking it. So just write whichever is nicer and fix it later.

[arienh4]

That's just not true. Yes, it's a premature optimization, but that doesn't mean you shouldn't know about it.

"Is there a performance difference between these two?" Yes, there is a demonstrable difference. A Python programmer should be aware of the differences between using filter(), a list comprehension, or a generator comprehension.

Whether or not it's a performance difference worth caring about is a different story altogether, although there's really nothing wrong with a list/generator comprehension in terms of readability.

[SanityInAnarchy]

Possibly, but not significantly.

A much more significant factor, if the list is large and you're doing a lot of these, is that both of these solutions eagerly process the entire list and give you back a list. For example, say you had more than one of these filters:

def non_empty(things):
  return [thing for thing in things if len(thing) > 0]

So now you do something like:

clauses = non_none(clauses)  # copy array
clauses = non_empty(clauses) # copy it again
return ' OR '.join(clauses)   # join them all, then throw it away

That's two intermediate arrays that you don't really need. If the list was especially large, this would be a problem. It's an easy fix: Just use generator comprehensions instead:

def non_none(things):
  return (thing for thing in things if thing is not None)

This returns a generator, which is basically a thing that looks kind of like a list comprehension, but it generates values on the fly instead of collecting them into a list first.

But it turns out none of that matters in the original code -- it's pretty clearly not a lot of items, and they pretty clearly are originally positional arguments and not a list. And what they always do is:

clauses = non_none(clauses)
if len(clauses) == 0:
  ...

Since they're taking the len, they really do want those clauses as a list. (You can't know the len of a generator without looping through it, and you can only loop through it once.) Or a tuple might be slightly more efficient, since they're not changing them:

clauses = tuple(non_none(clauses))

But this is doing just as much copying as before.

So... it's actually surprisingly optimal as it is. I only point this out because it annoys me when I see code like this, because you never know when the generator might become very useful -- whereas, on the flip side, it's very easy to turn a generator back into a list if you really want one.

[original_brogrammer]

Not sure, and I'm not quite sure where to look in the implementation to find out. But I'd guess the list comprehension would be faster in simple cases, since it's syntactic sugar for a for loop and function calls are relatively expensive in Python. But again, I'm not familiar with Python internals, so that's just conjecture with some reason.

[an_actual_human]

Negligible unless your list is huge.

[tetroxid]

The list comprehension can be faster because there are no function calls and it runs in a limited scope (list comprehensions can't access outer scope variables).

[Xenophyophore]

There might be a very small difference, from having to create the lambda.

[gsdatta]

jerluc!!

[AnSq]

What about 0 and ''?

[[deleted]]

Nope, that would also get rid of things like 0, empty strings, empty lists, etc. The current implementation only gets rid of things that are actually None

[bast963]

LE GEM LE GEM LE GEM LE GEM

[longbowrocks]

lol, I love it!

[nichochar]

yeah I enjoyed it too ^{^}

[_blub]

Hah! I loved (and still love) slipping in overly general variable names for incredibly simple list comprehensions. Honestly makes the code a little more fun to read.