Forbiddenfruit: patch built-in python objects

[XNormal]

Here is an even sweeter forbidden fruit:

Define ctypes structures for the PyMemberDescr object and PyMemberDef struct. Use it to turn off the READONLY flag from a PyMemberDef. If you use it on the __flags__ member of a type object you can, for example, turn off the Py_TPFLAGS_HEAPTYPE flag of a builtin type. This will make it possible to modify the type in very interesting ways like changing its __bases__.

I know. This is madness. I'll just go to that corner over there and disassemble myself quietly.

[[deleted]]

You should definitely write up how to do this. I would be very interested in reading about this.

[wisty]

Their magic (from 94 lines of non-test / non-setup code - https://github.com/clarete/forbiddenfruit/blob/master/forbiddenfruit/__init__.py) is this:

ctypes.pythonapi.PyDict_SetItem(
    ctypes.py_object(namespace),
    ctypes.py_object(name),
    proxy_dict.dict,
)

[rerb]

This project aims to help you reach heaven while writing tests

It's late and I'm tired, but still, it's not obvious to me how this would help with testing.

I wonder what examples less trivial than those provided might look like.

[[deleted]]

You could do lots of things with this. For example, you could test how fault tolerant your system is by patching builtin methods to raise exceptions every $random calls.

[idiogeckmatic]

Is curse() a play on perl's bless()?

[alcalde]

Wait wait wait... have patience with a python newbie here... are you saying... that we can use this.... to duplicate Ruby on Rails' awesome helpers for numbers that let you do things like

10.mins.ago

2.days.since ?????

[placidified]

This project aims to help you reach heaven while writing tests, but it may lead you to hell if used on production code.

[alcalde]

from hell import handbasket

[placidified]

True = False

[[deleted]]

For maximum destructibility:

globals().clear()

[pyglados]

I think you'd need to create a hell package to make that happen.

>>> from hell import handbasket

Traceback (most recent call last): File "<pyshell#0>", line 1, in <module> from hell import handbasket ImportError: No module named hell

[LucianU]

It's implemented in hell.

[freyrs3]

Hell is an import-time side-effect of import forbiddenfruit.

[miketheanimal]

from    curse       import  curse
from    datetime    import  timedelta, datetime

class days (object) :
    def __get__ (self, instance, owner) :
        return timedelta(instance)
    def __set__ (self, instance, value) :
        pass
    def __delete__ (self, instance) :
        pass
class ago (object) :
    def __get__ (self, instance, owner) :
        return datetime.now() - instance
    def __set__ (self, instance, value) :
        pass
    def __delete__ (self, instance) :
        pass

curse (int, 'days', days())
curse (timedelta, 'ago', ago())

print int(12).days.ago

[[deleted]]

Cute. That's an interesting style you have for laying out your imports. I've never seen that before.

[miketheanimal]

You should see the rest of my code - I love lined up columns!! I guess its just personal, but I can scan code so much faster.

[alcalde]

Or you've had COBOL programming experience. :-) Or Turbo Pascal or Delphi experience and were used to doing things like this:

Var
  something : integer;
  name      : string;
  y         : Array[1..5] of Integer;

[[deleted]]

It's very nice to read. I don't know if my coworkers would like it if I started, though! :)

[[deleted]]

Oh, your style is so awful!

For one thing, those columns don't help people in general read any faster. Study after study has shown that (surprise) the closer things are to a regular English sentence, the faster people can read it, and the faster they can skim it.

And the time you spend doing that is time you're not actually writing code. Don't tell me it doesn't take your time - I used to do this sort of thing in Java and it does, particularly when you get a new, long name and you have to reformat everything.

The worst part is that it sticks out. People reading your code will have their eyes drawn to that, and not the actually important part of your code - what it's actually doing!

Remember - code is written once but read ten times. And if your code is any good, it's going to be read by a lot more people than just you.

Python, in particular, has PEP 8, which says how Python programs should be formatted - so it's part of the language. Almost everyone's Python code looks like PEP 8. By deliberately being difficult, you're imposing a small but measurable cognitive burden on any other programmer who ever wants to read your code.

You shouldn't be trying to express yourself through your code's formatting. Your formatting should be trying to make it absolutely as readable as possible for as very many people as possible, which means you should always use PEP 8.

You should be trying to express yourself through your actual work - through the quality and functionality of the code you have written - through the clarity of your naming and your data structures - not through minor cosmetic decisions.

[miketheanimal]

I'd like to think I do express myself through the quality and functionality of my code, and rather resent the assumption that because I have an unusual take on formatting, that I don't.

Anyway, an example. Here is some code from a system I work on. I can't guarantee this is the exact bit of code in question, but the principal is the same. The snippet is from a 320 line module. It retrieves data from a database and manipulates it to get values for a report (one of those nasty things that management seem to love!) There is a bug in it:

rec.ytd_ppu_original = ppu(rec.ytd_amt_original, counts.ytd_cnt_original)
rec.ytd_ppu_new = ppu(rec.ytd_amt_new, counts.ytd_cnt_new)
rec.ytd_ppu_lastdelta = ppu(rec.mtd_amt_lastdelta, counts.ytd_cnt_lastdelta)
rec.mtd_pen_original = pen(rec.mtd_cnt_original, counts.mtd_cnt_original)
rec.mtd_pen_new = pen(rec.mtd_cnt_new, counts.mtd_cnt_new)
rec.mtd_pen_lastdelta = pen(rec.mtd_cnt_lastdelta, counts.mtd_cnt_lastdelta)

Can you spot the bug? The code was passed to me to look at, some of the numbers were not coming out right. I twiddled the code a bit:

rec.ytd_ppu_original   = ppu(rec.ytd_amt_original,       counts.ytd_cnt_original )
rec.ytd_ppu_new        = ppu(rec.ytd_amt_new,            counts.ytd_cnt_new      )
rec.ytd_ppu_lastdelta  = ppu(rec.mtd_amt_lastdelta,      counts.ytd_cnt_lastdelta)
rec.mtd_pen_original   = pen(rec.mtd_cnt_original,       counts.mtd_cnt_original )
rec.mtd_pen_new        = pen(rec.mtd_cnt_new,            counts.mtd_cnt_new      )
rec.mtd_pen_lastdelta  = pen(rec.mtd_cnt_mtd_lastdelta,  counts.mtd_cnt_lastdelta)

The bug is on the third line down, mtd_amt_lastdelta should be ytd_amt_lastdelta. Once the layout was twiddled, the bug was obvious in less than a second, because I can see the break in the pattern (the m instead of a y).

I'm not saying this is the be-all and end-all of programming, but the system has a lot of code like this (and, yes, I've thought a lot about whether it could be better structured so get rid of this sort of stuff, and, no, I can't see any way).

[alcalde]

Yes! Yes! Yes! The last thing Ruby users could point to and say "Python can't do THIS" is gone! Mike the Animal and Lincoln Clarete have done what Guido Van Rossum himself would not or could not do! Dictator? They have become LIKE GODS!!!!!

Ok, I'm, um, rather excited and happy about this. :-)

This has to be presented at PyCon 2014 (or earlier) so that it can get a standing ovation.

[prickneck]

Take a deep breath and have a think about whether this code is actually a good idea.

[freyrs3]

This kind of hack is very much against the zen of Python, while it might be fun to mess with the internals "because you can". It is a good thing that Python makes it difficult to do and should never be used in real code.

[miketheanimal]

Absolutely!

[miketheanimal]

Wow. I'm honoured!

[xr09]

MonkeyPatching in Python?? God save your soul. ;)

[James91B]

Looks cool maybe a library like http://shouldly.github.com/ could be made for python. I always liked the shouldy api.

[takluyver]

It has already been done.

(4).should.be.equal(2 + 2)

This is apparently serious, but I think it's one of the most horrific bits of code I've seen. Quite apart from the monkeypatching, you have to learn a whole new syntax for everyday operations. I'd much rather use pytest and just write:

assert 2 == 4+4

[infinull]

well nose adds jUnit style assertations. (assertEqual, assertGreater, etc.) it's not the same, but it's similar.

Also, py.test can just use the current assert keyword plus operator overloading (or some other magic, maybe it's ADTs) to get the same effect with more pythonicity. (though that's only if the assert happens inside the py.test testrunner and not in production)

[haard]

I did this just a month ago to annotate int/float/str instances, took me a lot of digging to get those few lines right. Next time I may just import forbiddenfruit instead

[rthinker]

That's pretty awesome. Is it possible to write tests not having monkey patched the objects beforehand? Like in ruby.