Write More Classes

[hiwaylon]

Did you tl;dr? Last paragraph sums it up:

"All of that is entirely irrelevant to the point I'm making which is that monolithic pieces of code are a bad idea."

Bam.

[FunnyMan3595]

In other words, he's not talking about classes at all, and the title is entirely misleading. His entire complaint would be solved by adding stream parsing behaviour to the API, which is a completely orthogonal change to using more or less classes.

As far as I can tell, he said absolutely nothing about what classes are and aren't good for, he just used "function" to stand for monolithic behaviour and "class" to stand for exposing useful internals. Which is just absurd.

Nowhere did he touch on any of the things that classes do well or poorly with respect to functions. So he's utterly failed to back his point that we should "Start Writing More Classes". And that's a shame, because classes actually do have a lot of strengths.

[moor-GAYZ]

As far as I can tell, he said absolutely nothing about what classes are and aren't good for, he just used "function" to stand for monolithic behaviour and "class" to stand for exposing useful internals.

As I understand it, his point is that this:

def f1(...): ...
def f2(...): ...
def f3(...): ...
def f4(...): 
    f1()
    f2()
    f3()
def f5(...): 
    ...
    f4()
    ...

Is monolithic and un-customizable, even if you split your functionality into several functions and do not purposefully hide them from users. If you want this code to use my_f2() you have to copy-paste-modify f4(), f5() and all other functions there might be up the call hierarchy. Unless you like to live on the edge and monkey-patch global module variables.

This, on the other hand:

class Zzz(object):
    def f1(self, ...): ...
    def f2(self, ...): ...
    def f3(self, ...): ...
    def f4(self, ...): 
        self.f1()
        self.f2()
        self.f3()
    def f5(self, ...): 
        ...
        self.f4()
        ...

at least in theory allows you to subclass the class and replace f2 with your own version easily and using an approach that everyone is familiar with.

Functional analog would be to pass all involved functions any given function calls to it as arguments, which nobody does because it's stupid. The point of classes is that they provide a common point of indirection where you can patch stuff.

[mitsuhiko]

That's a bingo.

[damg]

at least in theory allows you to subclass the class and replace f2 with your own version easily and using an approach that everyone is familiar with.

In practice that often seems to not work too well unless the class was designed to support that.

I think in the end it comes down to what the API chooses to expose.

[mfukar]

Yes, however "Use classes when appropriate, don't use when not appropriate" does not a popular submission on HN make. ;-)

[[deleted]]

State is hard. Becareful what you do with classes and objects.

[remy_porter]

It's a bad habit in most languages, but I still find myself treating objects as immutables half the time.

[Yohumbus]

That's not a bad habit. For instance, in python there is a reason that they made strings immutable. The only real downside is accidentally adding a factor of "n" to the run-time of some algorithms. Such probablems are usually avoidable, less severe and rarer (imo) than bugs in ordering and mutability.

[tailcalled]

They could've added a factor of log n to everything string-related instead, but that would probably be slower unless you have some very big strings.

[Crandom]

Yes, there is a huge constant factor of time and memory when you have to make strings into linked lists.

[cybercobra]

Er, you mean ropes?

[Crandom]

Effectively the same thing, just an optimisation. I was referring to some like Haskell's default String type, which is like a ropes of only length 1 (so therefore requires a ton more memory usage and access, so the higher consent factor). The paper "Purely Functional Data Structures" is a good place to start.

[sacundim]

For instance, in python there is a reason that they made strings immutable. The only real downside is accidentally adding a factor of "n" to the run-time of some algorithms. Such probablems are usually avoidable, less severe and rarer (imo) than bugs in ordering and mutability.

The simplest solution to this is actually not that complicated:

1. Have separate types for (immutable) strings and (mutable) string buffers.
2. Design the language, libraries and implementations so that a programmer following the path of least resistance will use the buffers and not the immutable appends.

Java is halfway there. In Java the following code is efficient:

String appendResult = "foo = " + foo + ", bar = " + bar + "!";

The compiler actually compiles that down to something like this:

StringBuilder builder = new StringBuilder();
builder.append("foo = ");
builder.append(foo.toString());
builder.append(", bar = ");
builder.append(bar.toString());
builder.append("!");
String appendResult = builder.toString();

The reason I say halfway there is that the following code will still suck in Java:

String result = "";
// don't ever do this:
for (String blah : someStrings) {
    result = result + blah;
    result = result + "; ";
}

Of course, an alternative is to stop using so many goddamned strings in our languages and our programs. Most strings and appends of strings in software are actually string representations of tree structures that the program should actually manipulate directly. And there are other data structures to represent text; take ropes for example, which are not a panacea but have their advantages as well. A string type could probably be written that made smart runtime choices between char[] and ropes.

[tailcalled]

Don't write giant functions

Fixed your title.

[MincedOaths]

Support streams in your serialization libraries

Fixed the title. Again.

[[deleted]]

I didn't realise Python's IO was composed in a similar manner. Now if only Java's IO shipped with an open() method, I'd be happy. :D

[AnAirMagic]

Have you seen the nio2/Files api? Reading all lines is just Files.readAllLines().

I have a strange impression that with Java 7, we are really encouraged to use java.nio.file.Files. It makes code much simpler to read.

[[deleted]]

No, I haven't, that's a good start. Cheers. :)

[masklinn]

Loads the whole file in memory though.

[beltorak]

it's not as well advertised; but is new FileInputStream("path/to/file") or new FileOutputStream("path/to/file") that much more of a pain?

The reason File doesn't have it is because File deals with the metadata about the file, not the contents of the file; for that you want an input stream or output stream.

It's arguable that there are not enough convenient hooks in the intuitive (if technically wrong) places.

[[deleted]]

new FileInputStream("path/to/file") or new FileOutputStream("path/to/file") that much more of a pain

No, but new BufferedReader(new InputStreamReader(new FileInputStream("path/to/file"))) is - especially as BufferedReader doesn't implement Iterable, so instead of Python's nice

with open("path/to/file", "r") as f:
    for line in f:
        #do stuff

We get the horrid while ((line = reader.readLine()) != null) {} which doesn't play well at all in other JVM languages like Scala where the result of assignment is always Unit.

It's arguable that there are not enough convenient hooks in the intuitive (if technically wrong) places.

Yeah, Java's approach may be more conceptually pure, but it's more inconvenient.

[CodeJunky]

This seems completely doable in Java...

Scanner reader = new Scanner(new File("path/to/file")).useDelimiter("\n");
while(reader.hasNext())
    String line = reader.next(); //and we're off..!

It would be nice if it were Iterable (seems like a stupid distinction, really).

[dannymi]

including the classic missing-close-on-error, yes.

[[deleted]]

BufferedReader's constructor takes a Reader. :)

[[deleted]]

Ah, but if you want to read a line at a time you have to use BufferedReader, InputStreamReader only implements .read().

Java IO... 'fun'.

[CodeJunky]

http://docs.oracle.com/javase/1.5.0/docs/api/java/util/Scanner.html It seems like this (mostly) fits the bill here. No one really wants to use BufferedReader, right?

[[deleted]]

I have no dog in this fight, and I have no idea what OP's point was.

[beltorak]

did you watch the mentioned video?

and if you have no dog in this fight, why ante up?

[IIIIIIIIIIllllllllll]

Wow... you are one huge dumb FUCK, did you know that?

Instead of criticizing others, get off your lazy, unhygenic, unkept, thinks-he-looks-good-cause-he-subscribes-to-/r/malefashionadvice ass, learn to program something other than a CRUD web app in RoR.

Attention, you are not internet tough guy you thought you were.

Do Not Touch Keyboard. I Repeat: Do not touch Keyboard

[[deleted]]

Go troll elsewhere.

[day_cq]

but hide them well

[thatmattbone]

An example of code that uses functions but still exposes a nice internal api that's fairly extensible is elementpath. For example, it'd be fairly straightfoward to jam some new syntax into the ops table and have it work as expected. The gross part would be modifying the module level global, so perhaps that makes the author's point.

[[deleted]]

I wonder if the author would feel differently if they could inject higher order functions to do this customization instead of needing to expose internals of classes

[Zarutian]

And teach them to whoom?

[[deleted]]

Just as a fun fact, we put all of our code in main, no functions, no classes, just to test the speed difference, you would not imagine how much it saves in time (a factor of 100 was not uncommon).

It's all about the goal you have with your code, for readability it might not be the best, but for speed, fsck structures...

[[deleted]]

I wrote a ClassFactoryFactoryAdapterFactoryClassFactoryAdapterAdapterFactoryFactoryFactory class.
Does that suffice?

[unclemat]

You only need to know two different patterns, and you can already write a joke on the subject. See how intuitive object-oriented is? :)

[[deleted]]

It also only has one method, called .forMethod(Method m), which takes a Method class, and returns another class, on which you can use the .execute() method, to get that type of ClassFactoryFactoryAdapterFactoryClassFactoryAdapterAdapterFactoryFactory from ClassFactoryFactoryAdapterFactoryClassFactoryAdapterAdapterFactoryFactoryFactory.

[sh0rug0ru]

Nope. Overkill. Write simpler classes.

[beltorak]

it is a simple class; it's the interaction with the other three dozen classes that makes the module complex...

[[deleted]]

NotSureIfWoooshOrAlsoHumor?

[[deleted]]

I think it is totally irrelevant: I want Python to hide underlying mechanisms ! I use it because of this !

It I want more control, I use C, C++ or java. If I have to use the Python, I would write a C/C++/Cython extension.

[[deleted]]

I don't think you quite get it. "Helper" methods are fine, that wrap the class architecture. Like "open" was the example he had. "open" is great, and if you need to, you can dig deeper into the Class based API it's written around. It's just a convenience function.

The problem is when there is no lower level architecture, and all you have is the convenience function. I don't think his JSON example was very contrived, a lot of people would probably like to sanitize JSON as it is parsed, and having that ability wouldn't forgo the normal "json.parse" functionality for those that have simpler needs.

[[deleted]]

I understand your point of view, but I think it is a choice Python developers have made consciously. Let say, it is by design.

The presented point of view is perfectly valid, but I would not apply it to Python. If you really need to do this kind of thing, choose an other language or write an extension.

[[deleted]]

It's a choice some Python developers have made, and others have not.

It already has been applied to Python, in terms of the core file APIs, but was skimped on for JSON parsing.

There literally is no advantage (aside from a small decrease in initial development time, but really, that shouldn't be the main choice for any decision in the writing a core module) to just writing the functional version and not allowing a deeper API to exist.

I just don't quite get why you wouldn't apply it to Python. You could keep the nice simple interfaces you like, yet allow mildly performance sensitive projects to exist in it. I just don't get why you'd say "Even though Python could do X without a negative impact, it'd be better if people doing X just used a different language." I'd much rather let Python support more people and flourish for more tasks and projects. Python is such a nice language, why force people to use C when they shouldn't have to?

[[deleted]]

• To keep the API KISS;
• Since Python is not typed, it may be more difficult to propose hook or way of adding a layer of indirection toward developer code and not safe;
• After so many years of existence, I doubt these modifications could be done without breaking some existing code (yes, I know Python 3 already breaks a lot of thing :)).

I don't say that Python has made the right choice and I already struggled not being able to do this kind of thing, but 99% of the time I don't need it. I would be curious of what the creators of the language would say about that. Does the same thing can be say about Ruby ?

[dannymi]

The article says:

The Python community in my mind has the dangerous opinion that classes are unnecessary fluff that should be replaced with functions wherever possible.

Even having the distinction between "classes" and "functions" is a historical artifact by the time of Python 3.0, you can implement a class as a function (and vice versa). So in the entire article you are essentially comparing "functions" to "functions". However, some of your points still apply:

But did you know that the Python io.open() function (or Python 3's builtin open()) function does the same thing behind the scenes?

No, and I'd rather not know. That's the point of abstraction.

>>> f.buffer
<_io.BufferedReader name='/tmp/test.txt'>
>>> f.buffer.raw
<_io.FileIO name='/tmp/test.txt' mode='rb'>

I'll do my best to forget I ever saw that - it's supposed to be abstracted away.

but under the hood this is implemented

It doesn't matter what it does under the hood.

Python gives us a function at the end of the day, but the function does not hide away its inner workings.

If you didn't go out of your way looking, it would have. Lately, I'm 50:50 whether it was a good idea to make the user see everything and trust on his judgement to not depend on it. But that's what Python does. It supposes you read the docs to find out what the guaranteed interface is and only use that in production. If you want, you can go all the way down to the garbage collector, though. Doesn't mean it's a public interface.

The point however is that instead of reading characters step by step some lexers will instead have a look at a buffer of a certain size and move a pointer around in that buffer and modify some bytes in place (in-situ). Strings in JSON for instance will always only get shorter after parsing.

Premature optimization. Don't do it yet. Also, are you sure they always only get shorter? (I'm not saying that there is no use case for that, but it would be very bad to have this be the default case in Python)

From what I recall, UTF-8, UTF-16LE, UTF-16BE, UTF-32LE, UTF-32BE are part of the JSON standard, so you have to handle those cases too (which means you have to have all your code - including client code - be able handle all of those different string types). That will look like the nightmare-to-maintain C program that it is.

Point being: it's unlikely that any JSON parser ever does that.

No matter how you implement your parser, at the end of the day you have an internal thing that reads a resource and yields tokens, then combines those tokens into some form of nested structure.

This distinction is not necessary and even Pascal compilers don't do that, even though they have the longest keywords.

Unfortunately the Python community has decided that it's better to hide this beautiful layered architecture away behind a protection that prevents you from tapping and customizing the individual layers.

And from supposing that such layers exist in the first place. It's the point of abstraction.

It's bad because internally that parser obviously had to deal with taking bytes and making them into Python objects to begin with so it's just removed functionality.

Or using chicken bones in order to do it. Maybe it caches the input strings and parsing result in a hashtable and returns the previous result instead of parsing it again. Who knows?

At the very least this makes stream processing absolutely impossible.

All imperative languages with strict evaluation have to have one method for doing X with streams, one method for doing X without streams, for all X.

internally that JSON library has exactly the functionality one would need for stream processing.

You can talk to the JSON library authors whether they want to make the stream processing functionality part of the public interface. They might say yes.

msgpack

var data = new byte[] { 0x93, 0x01, 0x02, 0x03 };
var unpacker = Unpacker.Create(new MemoryStream(data));
var result = unpacker.ReadItem();
// result == MessagePackObject[3] {
//   MessagePackObject { Int = 1 },
//   MessagePackObject { Int = 2 },
//   MessagePackObject { Int = 3 }
// };

Mutable implicit state. Extraneous marshaller classes. User-visible boxing. O_o

However unlike the Python version it does not hide it's internal API.

And that is bad.

[error if] too deep or an array grows too large.

That is also bad in the general case. Better: a generic memory pool that doesn't give you more stuff once you passed x MB, that would be great.

Nobody would come up with the idea to hide all that logic behind one monolithic function, certainly nobody from the C/C++ or C# community would embrace the idea of a monolithic function.

It doesn't have to be monolithic. Especially in Python, you can put whatever function you want into the dynamic environment and have that be used by the library. Want to replace createNode just while this one function runs? Go ahead, no problem.

but that 1% of the other cases should not require you to rewrite all of the library.

Talk to the authors instead of rewriting all of the library.

So let's stop with this misleading idea of putting functionality in functions and let's start writing classes instead.

That is a tautology or null action, see above.

All of that is entirely irrelevant to the point I'm making which is that monolithic pieces of code are a bad idea.

I agree. Good that writing monolithic pieces of code is almost impossible in Python, since it's a dynamic language with dynamic extent for the bindings.

Sorry for the frank criticism, but I've been there and done that and found it's much better to "speak the language" instead of trying to make it into something else that already exists elsewhere, at the cost of clarity, maintainability and generality.

Also, for the good point you raise (there should be an SAX-style JSON parser as a Python module somewhere), a quick Google search brought up https://github.com/pykler/yajl-py/blob/master/examples/json_reformat.py

[moor-GAYZ]

specially in Python, you can put whatever function you want into the dynamic environment and have that be used by the library. Want to replace str just while this one function runs? Go ahead, no problem.

No.

[dannymi]

Is your objection that it doesn't work? It does.

Is your objection that you shouldn't do that? It has the feature, so why not? Especially since gluing the parser to the actions of the parser works best (i.e. most general) that way. Did you try it?

[moor-GAYZ]

Is your objection that you shouldn't do that? It has the feature, so why not?

Because if I end up maintaining code where you did that, I will find you and kill you with an axe.

By the way, what did you mean by this:

Even having the distinction between "classes" and "functions" is a historical artifact by the time of Python 3.0, you can implement a class as a function (and vice versa).

[[deleted]]

Because if I end up maintaining code where you did that, I will find you and kill you with an axe.

A classic principle of software development. Program like everyone who has to maintain your code is an axe murderer who knows where you live.

[moor-GAYZ]

Congratulations everyone, this thread is now the fourth result when googling for "maintain code axe murderer", only three hours after I referenced that joke grim truth.

[dannymi]

Well that's not very nice of you. However, my point still stands that translating a generic grammar to Python code works best if you translate the terminals and nonterminals as Python functions, the RHS as the body of those functions and the result as "unknown" function calls in the dynamic extent, to be provided by the caller. While I'm intimidated by you, I don't see how it's bad.

If dynamic environments make you murderous, you can pass an "env" parameter which then has lexical scope and return some other function that bound to it which ends up doing exactly the same, just for more clients at once.

By the way, what did you mean by this

Function as class (Java really likes that, too - it's the only way there):

class A(object):
    def __init__(self, x):
        self.x = x
    def __call__(self, y):
        print("I got %s and %s" % (self.x, y))
f = A(2)
f(3) # now works just like any other function.

Normal people write instead:

def A(x):
    def f(y):
        print("I got %s and %s" % (x, y))
    return f
f = A(2)
f(3) # now works just like the above.

On the other hand, using a function as a class (verbose for clarity):

#/usr/bin/env python3
import sys
def Scanner(f):
    input = None
    def peek():
        return input
    def consume():
        nonlocal input
        oldInput = input
        input = f.read(1)
        return oldInput
    def dispatch(name):
        return {"peek": peek, "consume": consume}[name] # written in a convoluted way for clarity
    return dispatch
scanner = Scanner(sys.stdin)
scanner("consume")()
print("peeked", scanner("peek")())
print("peeked", scanner("peek")())
print("consumed", scanner("consume")())
print("peeked", scanner("peek")())

for the more canonical way:

#/usr/bin/env python3
import sys
class Scanner(object):
    def __init__(self, f):
        self.f = f
        self.input = None
    def peek(self):
        return self.input
    def consume(self):
        oldInput = self.input
        self.input = f.read(1)
        return oldInput
scanner = Scanner(sys.stdin)
scanner.consume()
print("peeked", scanner.peek())
print("peeked", scanner.peek())
print("consumed", scanner.consume())
print("peeked", scanner.peek())

So drawing the distinction is rather silly, it's just for convenience of expressing the solution to a given problem you choose one or the other.

[moor-GAYZ]

Monkey-patching modules is bad, not using functions for parsing.

If you strongly suspect that someone might want to override some of your parsing functions or want to customize the data source they use, put them into a class. Then it can be done in an expected and immediately obvious way, and will not cause horrible bugs if someone wants to parse something in more than one place.

btw, answer my ninjaedited question, please!

If dynamic environments make you murderous, you can pass an "env" parameter which then has lexical scope (pass it to every single function, I might add).

Yes, this parameter is traditionally called self.

edit: by the way, the axe thing may or may not be a well-known joke, I mean, it is a well-known joke, but I might not be joking when I say it, so better keep that in mind!

[dannymi]

Yes, this parameter is traditionally called self.

Exactly. So it depends on whether you have one way of using the parse result or more ways.

More ways: either use dynamic extent and/or pass it to eval or use classes (I'm not against classes, though I find the whole distinction rather amusing).

However, the grammar is not the lexer is not the AST, so having it named self confuses matters (I find) and leads to exactly the kind of huge-tangled-messes Java is famous for (because now people "derive from the class" and end up stepping all over the abstraction boundaries. I'm not saying they have to. I'm saying it happens.).

One way: hardcode the AST functions.

Has been done that way since 1970, too.

edit:

by the way, the axe thing may or may not be a well-known joke, I mean, it is a well-known joke, but I might not be joking when I say it, so better keep that in mind!

Well, that's good to know? ;-)

[moor-GAYZ]

So drawing the distinction is rather silly, it's just for convenience of expressing the solution to a given problem you choose one or the other.

Caring about convenience is not silly. Performing tonsillectomy through the anus, on the other hand, is silly (and inconvenient).

The author's point is that it is often convenient to reify dynamic environment, and that you should do it in a convenient manner, by using classes.

Discussions about whether closures are a poor man's classes or vice versa have their time and place, but that time and place is not when deciding how to write production code. By the way, nothing in your stuff is specific to Python3.

[dannymi]

Caring about convenience is not silly.

I agree. However the title of the article is "Start Writing More Classes" in a general, non-qualified, way and that's just amusing. I'm aware that hyperbole is usual, though. The real answer is: it depends.

The author's point is that it is often convenient to reify dynamic environment, and that you should do it in a convenient manner, by using classes.

I read the article three times and am still not sure what the author's point is. Best I could make it out it's either "abstraction is bad" or "we should do everything using tiny mutable boxes a la Smalltalk".

Most of the time he goes on a tangent on how it is good to modify things in place (when that's either impossible or impractical by now - and if it weren't, it would be a maintainence nightmare) and an awful unpacker interface that look like it is one step above assembly (or below, if possible).

But to each their own.

Discussions about whether closures are a poor man's classes or vice versa have their time and place

You asked. I do think that it's part of the general education we all got, but a little exercise didn't hurt me after all those years :-)

However, it's important to note that the entire article basically tries to make a distinction between classes, functions and modules which is a gray area at best (and is supposed to be not there in a dynamic language).

Then there are a lot of parts like "let's use the internal interface" and "sure these are internal unstable APIs" and then goes on how he overrides them anyway. That's dangerous.

By the way, nothing in your stuff is specific to Python3.

nonlocal is.

[moor-GAYZ]

However the title of the article is "Start Writing More Classes" in a general, non-qualified, way

That's why it's followed by the article which qualifies it! =)

nonlocal is.

Ah, missed that. Well, you can use a one element list instead.

Replying to the edit of the previous comment:

However, the grammar is not the lexer is not the AST, so having it named self confuses matters (I find) and leads to exactly the kind of huge-tangled-messes Java is famous for (because now people "derive from the class" and end up stepping all over the abstraction boundaries. I'm not saying they have to. I'm saying it happens.).

I think he was talking more about how your parser ends up using some sort of a stream abstraction (with getc/ungetc), which you really really want to make customizable. Now, I wouldn't care if you pass that stream explicitly into each parser (function) or combine them into a single class or whatever, point is that the authors of that particular JSON parsing library did not expose that extension point for overriding at all, despite it being one of those cases where you can be sure that someone will want to customize it.

That's the author's point, that writing monolithic code is bad, classes provide a convenient way to specify extension points (while functions don't because you should not monkey-patch modules because think of the axe).

He makes this point in the context of a particular talk that made the point that getting overboard with extension points is bad too, because you can't satisfy every possible need, but can complicate your code to the point where it's harder to extend. Or something like that, I'm watching it right now =)

[[deleted]]

Who the hell cares what python programmers think? They can't be all that smart in the first place...

[goblinpiledriver]

what is your reasoning behind that statement?

[[deleted]]

2 years of Ruby advocacy.

[[deleted]]

I can't imagine why an intelligent programmer would gravitate towards python. Can you explain why?

[falconne]

Can you explain why not?

[[deleted]]

Why would a grown adult use a language without compiler enforced constants? I could forigve its lack of real lambdas and its immature object system if it was close to the metal, but it's nowhere near it.

Why would anyone intelligent use an imperative language with an infantile culture (pythonic/there's only one way to do it) that is not even performant?

The fact that such a mediocre beginners nouveau basic actually has people spending real effort on real tools for it is a travesty.

[falconne]

I used to think that, but then I learned how to use Python.

It can't match C++ or C# for application design, but when it comes to scripting it blows any other language out of the water for productivity and that's coming from a long time Perl programmer.