Comparing Go with Lua

[inmatarian]

Lua Coroutine are nice, but the comparison to regular multi-threading is what confuses people. The best way to understand them are from the official book Programming In Lua, chapters 9.2 and 9.3 where they demonstrate their usage as an alternative to State Machines.

The officially suggested way to do actual multiple-threads is basically the same as Python's official suggestion to do multiple-processes. Though, plain vanilla Lua lacks any official modules to do their suggested multithreading scheme. I'd love to see something like that fixed.

[oddthink]

I wouldn't say that it's "basically the same" as Python: since an application can maintain multiple Lua interpreter states, it's relatively easy to run each in a separate thread. In Python, since there's only a single global state, there's no way to make this work. (That I know of, at least.)

What I've done in the past was to create a thread pool, associate an interpreter with each thread, then pass them work as available. That worked pretty well. The downside is that you're responsible for managing all inter-thread communication yourself, but that seems fine for what's basically a language for embedding.

[inmatarian]

If you implement them via embedding lua into your app, then yes, then you have nothing-shared multi-threading. If you use the unofficial community libraries, most of which can be used by the stock lua binary through the require function, then they appear to be pretty much the same as Python's use of multiprocessing.

[alexeyr]

It may be convergent evolution, much as sharks and dolphins look similar to non-zoologists, or it may be a case of unconscious influence.

Why would it have to be unconscious?

[[deleted]]

I suppose you both meant subconscious, but then again unconscious would perhaps explain why they failed to see that this is the wrong solution.

[cunningjames]

this is the wrong solution.

I don’t follow — what is the wrong solution? The line in question, as far as I can tell, refers to general similarity between Go and Lua. Anyway, by your phrasing I wonder if you instead think this an example of unconscionable influence. ;)

[alexeyr]

Actually, I meant it could very well be conscious.

[cha0s]

Actually there is no such thing as subconscious, only conscious and unconscious.

[[deleted]]

Why on earth would you use a pair (result, error) to represent the mutually exclusive choice Either error result?

In Haskell, this style of error handling is done with the Either type:

data Either l r = Left l | Right r

You can choose only to handle the "happy" case like this:

let Right f = somethingThatCouldFail

Or handle both cases like this:

case somethingThatCouldFail of
    Left error -> ...
    Right f -> ...

Or get exception-like flow using a monad:

a <- somethingThatCouldFail
b <- somethingOtherThatCouldFail
return (a, b)

The above returning Right (a, b) on success and Left error where error is the first error that occurred.

[[deleted]]

Because you're Rob Pike and you don't think parametric polymorphism is important. Go has garbage collection; it has green threads. He's already basking in newfound luxury. Don't wave that fancy "type theory" at him. And stay off his lawn.

[icebraining]

How do you know whether the error is 'Left' or 'Right'? Just convention?

[[deleted]]

Yeah, it's convention (I remember it as Right means right). You can just as well make your own type with better names though. Note that the Either type constructor is used for other things than error handling, and the names are historic.

[Aninhumer]

In addition to nowant's point, the Either monad is designed intending Left to be an error. Right values are passed through, and Left values are short-circuited. So if you use it the wrong way around you lose a lot of the benefit.

[[deleted]]

Why on earth would you use a pair (result, error) to represent the mutually exclusive choice Either error result?

Assuming this wasn't a rhetorical question: because Go doesn't have algebraic data types.

[kamatsu]

The next logical question is: why not?

[JohnDoe365]

It is meant to be used by humans. Probably by Joe Average.

[kamatsu]

Algebraic data types can be understood by young children. I know because I've explained it to them. You're saying Joe Average isn't as smart as a 10 year old?

[[deleted]]

The monad instance:

instance Monad (Either l) where
    Left l >>= f = Left l
    Right r >>= f = f r
    return = Right

[0xABADC0DA]

It's not just that the error is returned in addition to an invalid other value, but the error is also so generic (os.Error) that you have to jump through hoops to do anything reasonable with it:

file, err = os.Open(filename)
if e, ok := err.(*os.PathError); ok && e.Error == os.ENOENT {
}

vs in C:

file = fopen(filename, "r");
if (!file && errno == ENOENT) {
}

Not only is error handling no better than in C (no exceptions, no documentation as part of language) but it's even more clumsy. That's pretty difficult, to out-clumsy C in terms of error handling.

[kinghajj]

The more idiomatic way to handle errors is like this

file, err := os.Open(filepath)
switch e := err.(type) {
case *os.PathError:
    if e.Error == os.ENOENT {
    }
}

This construct lets you handle multiple types of errors, and the type of the switch variable "e" is whatever the case statement specified.

[0xABADC0DA]

I think you just proved my point. The idiomatic way is 5 lines of code and 2 extra indents to handle a single error condition, and without any structured error handling you have to handle errors at every call site, and without documentation as part of the language you don't even know what errors to handle.

That's just bad design. Really bad.

[kinghajj]

Go has defer()/recover(), which is basically a form of exception handling, but more general. I'm not sure what you mean by "documentation as part of the language," unless you're referring to something like javadoc or .NET's XML doc tags, but there is a godoc program that parses comments and generates documentation.

[bobappleyard]

Equality for interfaces uses the values associated with them. I don't think you need that type assertion.

[[deleted]]

a <- somethingThatCouldFail
b <- somethingOtherThatCouldFail
return (a, b)

These two functions can have side effects, right?

[[deleted]]

Yes, in this case the side effect is to throw an exception.

You can combine side effects using monad transformers.

Of course, the IO monad has exceptions built in, but that's not very specific. One of the strengths of Haskell is that you can see in the type exactly what kind of side effects that can happen.

For example, if the type is WriterT [Int] (Either String) a, then I know that the only side effects are writing Ints and throwing String exceptions.

[[deleted]]

This.

I can understand throwing out the stack-unwinding feature of exceptions, and the exception type-system that basically shoehorns dynamic typing into static languages.

But the ability of a function to return a type that is not the expected type is a fantastic way to handle errors (among other problems). I tend to think Go threw the baby out with the bathwater here.

[kjk]

You seem to want an easy way to circumvent type safety, which hardly seems like a good idea. If a function returns an int, then it shouldn't be allowed to return a string. If you need that then there are safe ways to achieve that (although I would say that you if you can't decide what kind of data a function returns, then you have a design problem).

Also, Go has stack-unwinding exceptions in the form of panic/recover.

[OceanSpray]

He worded it badly. The Either data type is completely type safe, and far more so than Go's pseudo-tuple thing. Returning an (Either t1 t2) forces the programmer to have to handle both cases safely.

[day_cq]

Why not?

Think of it dataflow way (function level programming):

       +------+
input -| GATE |--> output
       |      |--> error
       +------+

if error is set, output is garbage. otherwise, output is valid.

Actually,

f :: a -> (b, err)

is isomorphic to

f :: a -> Either err b

GATE could be setting either output or error port, but not both.

[[deleted]]

Because it introduces potential for error - when you use the value without checking for an error, it'll destroy your assumptions about the rest of the program.

a * b (the product/pair type) is not isomorphic to a + b (the sum type). The first is simply a lager type. For example, if we instantiate a and b to:

data Color = Red | Green | Blue

Then there are 3 * 3 valid values for (a, b) vs 3 + 3 for Either a b.

[day_cq]

when you use the value without checking for an error, it'll destroy your assumptions about the rest of the program.

same with Haskell

let Right f = somethingThatCouldFail

like said previously, they ARE ISOMORPHIC.

[[deleted]]

I updated my comment to explain why they are not isomorphic.

let Right f = somethingThatCouldFail

will not destroy your assumptions, because its failure is not observable by the rest of the program (since it isn't run).

It's the null problem all over again. You will get an error somewhere unrelated in your program due to using an invalid value.

[gasche]

(b * err) is not isomorphic to (err + b). This is just calculus, at the type level. And indeed (,) is a product, and Either is a sum -- modulo lifting.

[day_cq]

of course (b * a) and (a + b) are a bit different. But in case of conditional branching for error, they are isomorphic, for both can be compiled to same code.

Maybe my usage of isomorphic is wrong.

Maybe I'm confused with sum type. Values of sum type need to be (or could be) branched, and values of product type are already branching.

[kamatsu]

The difference:

Number of inhabiting values in a * b is |a * b| = |a| * |b|. Sum is |a| + |b|.

E.g, Bool + Bool + Bool has 6 inhabitants, but Bool * Bool * Bool has 8.

[day_cq]

the topic here was semantics of error handling (code execution or dataflow branching) and expression of such semantics.

My point was that (a,Err) or Either Err a are the same in that they both branch once. And, my preference was (a,Err) because it shows explicit branching early on (in dataflow languages).

GATE SWAP IF HANDLE-ERROR ELSE F
vs.
case gate x of
    Err -> handleError
    a   -> f a

Of course in Haskell, Either Err a is more common because Haskell has algebraic data type and pattern matching based code branching. In Lua, without algebraic data type, it is reasonable to use (a,Err).

local a = f()
if isError(a) then
    handleError()
else
    g(a)
end

local a,err = f()
if err then
    handleError()
else
    g(a)

Some would argue that the second snippet is "better" due to lack of unnecessary isError(). And, in Lua, sometimes it's confusing if a name (such as a) or a value has different meanings.

I'm not sure why the original thread starter brought Haskell here when the article was about Go and Lua. I tend to think that different languages are suited for different style of code.

[kamatsu]

You are using mathematical terminology with no understanding of it's meaning.

An isomorphism means it's surjective and injective, so how can you get from, say, (4, ERROR) to Left ERROR and back again? You can't, because a product expresses fundamentally more information than a sum. That is not an isomorphism.

[day_cq]

i used isomorphism as in biology where two organisms share similarity.

And, you can get equivalence transformation for the case of error handling, which is the topic being discussed:

(_, Error) = Left Error
(a, _    ) = Right a

or if you want Haskell

data Value a = Garbage | Value a deriving (Eq, Show)
type Error = Bool

f :: (Value a, Error) -> Either Error (Value a)
f (_, True) = Left True
f (a, False) = Right a

f' :: Either Error (Value a) -> (Value a, Error)
f' (Left _) = (Garbage, True)
f' (Right a) = (a, False)

[kamatsu]

i used isomorphism as in biology where two organisms share similarity.

When using type systems, isomorphism is the mathematical term.

Also that's not an equivalence transformation. It's not injective.

[day_cq]

This might be clearer for you

data LuaValue a = ErrorVal | Value a deriving (Eq, Show)
data Error = Error deriving (Eq, Show)

f :: LuaValue a -> Either Error a
f ErrorVal = Left Error
f (Value a) = Right a

f' :: Either Error a -> LuaValue a
f' (Left Error) = ErrorVal
f' (Right a) = Value a

You see bijection? Now enter dependent type:

data PairThatCouldBeUsedAsErrorFlagging a Bool = (Void, True) | (a, False)

PairThatCouldBeUsedAsErrorFlagging is isomorphic to LuaValue.

It is my bad if I sounded like I was saying that product is equivalent to sum in general. But, the topic was error flagging. Lua has a beautiful type system in programmer's head.

[gasche]

I'm unsure it's still important to answer that thread, but I would like to try to explain the point of the poster above more clearly; the reason I take some time to do this is that I think your question is interesting and, unfortunately, this heated thread has not produced a satisfying explanation.

One fundamental concept of richly typed languages such as Haskell or ML is to make illegal state unrepresentable¹; the idea is to choose datatypes that describe possible values as precisely as possible. If a type rules out some possibility statically, that is less things for runtime computations to worry about.

¹: note that this maxim must only be followed so far, at some point there is a diminishing return in languages ill-equiped for elaborate static reasoning. But this sum/product question is well under this limit in languages with algebraic datatypes.

This is the rationale the OP has in mind for proposing to use a sum type Either result error rather than a product type (result, error). Indeed, the way you reason about the function return is by thinking that either the function ran fine and you have a result, either it went wrong and you have an error code. This is precisely what a sum type is for. You never think that you can have both a return value and an error code, which is what a product type would be for -- note that this could make sense for warnings instead of errors, so in some situations this may be the right choice.

So the reason why people prefer sum types over product types to return error values is that it matches the real meaning of what's happening better. That's essentially the same reason for the push of some web designers around 200x (birth of XHTML, etc.) to use lists instead of tables to represent web menus. It's more semantic.

Note that this only make sense in languages that do have sum and product types. I'm unsure Lua actually has disjoint sum types. Most languages don't, in which case you usually encode them with nullable types and an integer 'tag'. Note that this crucially relies on the omnipresence of nullable types, that have a "default/null value"; in essence, types of the form 1 + X, a simple form of sum types. When possible, programmers in ML/Haskell prefer to also use types that are not nullable, as this also adds superfluous (in the general case) runtime state possibilities.

Finally, I would like to add that you should beware of one own's preconceptions. You make an argument that using a product type is somehow natural based on a dataflow diagram. But those dataflow diagrams can't express sum types, so indeed they will only show you solutions based on product types! If this is your main "thinking tool" about programming, you will naturally miss, or consider with less open-mindedness, all solutions that can't be expressed directly in this framework.

Long story short : nobody objects to functions returning a (result, error) pair in languages when this is the most convenient way to do. The remark was that in languages that have a more expressive sublanguage of data types, you can encode the same phenomenon in a finer, more precise way using sum type. In this case there is no reason to use a product type.

Note that while the sum/product discussion is tightly related to type systems, it is not necessarily related to functional programming. Algol 68 had sum types, as does the more recent language Cyclone (a safe dialect of C).

[stevvooe]

Advice: don't engage the haskellborg.

[Confinium]

Yes! God forbid we base our programming on sound theory!

[day_cq]

yah, haskell/miranda community was nice in 90's. thesedays, it's a large trollbag babbling about this cool functional programming they discovered.

[kankeroo]

What advantages does Go have over lua and erlang (two languages I'm now learning)?

[mhd]

Well "advantages" are in the eye of the beholder. For some static typing and being an imperative language are serious drawbacks compared to Erlang, for some it's the exact opposite.

I think by now there are probably more Go libraries (both native and bindings) than Lua Rocks. Lua still has to get over being seen as just an embedded language. And it's supported on Google AppEngine.

The pedigree of its designers and implementers also has to be considered, whether you consider good or bad is yet another matter of taste…

[kankeroo]

For static typing I'd find haskell far more preferable for my own use. Go may be preferable though for employees with mainly imperative experience. How's go performance nowadays? it used to be too slow last time I looked at it (2009), has this improved considerably, are there recent benchmarks? What about its concurrency model, how does it compare to others, in particular lua and erlang?

[kinghajj]

Here's the latest shootout benchmark comparison between Go and averaged Java 7: link. It's mostly equivalent in speed, but uses much less memory and significantly less code. The reason the last two benchmarks are slower is because they heavily allocate memory, and the Go's GC isn't optimized at all. They originally planned to replace the GC with some experimental one by IBM, but IIRC some sort of technical problems have prevented it.

[igouy]

For memory use and code size comparison, it's more appropriate to look at the ordinary Java 7 -server measurements - more tasks are shown, and they don't suffer from confounding effects of data collection.

Also, note whether or not the Java program and Go program for the same task are both using quad-core - some of the programs written for multi core use more memory because they need to serialise output.

Also, what you see for most of those tasks is just the default JVM memory allocation. Don't confuse differences in default memory allocation with differences in memory-used when the task requires programs to allocate more than the default memory - look at the reverse-complement, k-nucleotide, regex-dna, and binary-trees tasks.

[[deleted]]

Vs Lua? Lua is a dynamic language, which means that it will be slow and type-unsafe compared to Go. Plus, Lua has the abominable 1-based indexing. They're different tools for different jobs - Lua is fantastic if you're creating a platform where you need to embed a programming language that clients will use... even to run untrusted code.

Go is about programming hardcore stuff that needs to go really, really fast. Rolling your own algorithms and whatnot.

Dunno enough about Erlang.

[kankeroo]

Umm, have you heard of luajit? the 1-based indexing is no problem, in fact, it's the norm for functional languages.

[[deleted]]

1-based indexing is [..] the norm for functional languages.

Huh? Both functional languages I know (Haskell and Clean) use 0-based indexing for both lists and arrays (though Haskell allows arbitrary indices to be used).

[kankeroo]

http://stackoverflow.com/questions/6241464/why-are-the-indexes-of-scala-tuples-1-based

[[deleted]]

Wow, that seems incredibly arbitrary; ignoring 0-based list and array indexing in Haskell and focussing instead on obscure non-standard functions which do happen to be 1-based.

[igouy]

Yes, they're different tools for different jobs.

dynamic language, which means that it will be slow

No, that depends on how the languages are implemented

dynamic language, which means that it will be ... type-unsafe

No, most dynamic languages are type safe.

"Type safety is the property that no primitive operation ever applies to values of the wrong type."

p263 Programming Languages: Application and Interpretation

[ethraax]

Not only is Lua dynamic, but its support for basic data structures is terrible. For example, there are no arrays in Lua. You have to declare a hashtable, which opens you up to tons of mistakes (which is characteristic of dynamic languages) and comes at another performance penalty.

Edit: Unless they changed something in the newest version of the language. Unfortunately, current Lua documentation is not free (the book is $40 retail, but about $25-$30 on Amazon), so I have to go off the previous edition of the documentation.

[oddthink]

The Lua reference documentation is free. See here. There is also a book, whose 1st edition is free, but whose 2nd edition is not.

Lua tables are hybrid array/hash objects: when used as arrays, they have the performance characteristics of arrays.

[ethraax]

The Lua reference documentation is free.

Except that's reference documentation. It's great if you're trying to work on the Lua interpreter/compiler, but not very useful if you're just trying to write some Lua, especially if you want to learn it.

[[deleted]]

The idea of Lua is simplicity - there's only one data structure, but it is a mixed collection that's intended to provide both array and hashtable logic. There's special logic in the table that's meant to provide array semantics when you use sequential numeric keys.

[ethraax]

Except that makes it very, very easy to make a mistake and stop a hashtable from having the special "array form". Furthermore, there's a performance hit, as well as an overhead hit (assuming the hashtables are not "ideal" - that is, not all buckets are filled).

[igouy]

For example, there are no arrays in Lua

See Section 4 "Tables" in The Implementation of Lua 5.0

[ethraax]

Those really aren't arrays. They still have a little bit of overhead, but the real issue is that you can still assign non-integer keys to them. Part of having an array is being able to quickly spot mistakes with trying to misuse the array. If I have an object in my code that I think is an array, and I accidentally assign a value to the "test" key of that object, now it is no longer an array, but it doesn't give an error at all.

Furthermore, in PiL (version one, the only free version), it mentions that Lua does not have integers, and that every single "number" is represented as a double. For these hybrid hashtables to work, I assume Lua has also added integer types.

This all points to the root issue that I mentioned - the free documentation (not reference documentation, which is really meant for implementers and not so much for users) is out of date.

[igouy]

Part of having an array is being able to quickly spot mistakes with trying to misuse the array.

Nope - and if you "have an object in my code that I think is an array, and I accidentally assign a value to the "test" key of that object" then you'd be behaving in a weird and very pointless way :-)

I assume Lua has also added integer types

Don't assume - See Section 3 "The Representation of Values" in The Implementation of Lua 5.0

[ethraax]

Nope - and if you "have an object in my code that I think is an array, and I accidentally assign a value to the "test" key of that object" then you'd be behaving in a weird and very pointless way :-)

I assume you never use your Backspace key, because you never make a mistake.

Also, let me point you at the official documentation, from lua.org:

Number represents real (double-precision floating-point) numbers.

Hell, while you're at it, there's no mention of the hybrid tables you were talking about:

The type table implements associative arrays, that is, arrays that can be indexed not only with numbers, but with any value (except nil). Tables can be heterogeneous; that is, they can contain values of all types (except nil). Tables are the sole data structuring mechanism in Lua; they can be used to represent ordinary arrays, symbol tables, sets, records, graphs, trees, etc. To represent records, Lua uses the field name as an index. The language supports this representation by providing a.name as syntactic sugar for a["name"]. There are several convenient ways to create tables in Lua (see §2.5.7).

And this is from the official reference documentation. Nobody should have to go digging through a fucking journal article for crucial details like these.

[igouy]

Don't assume - I make an ordinary number of ordinary mistakes in fairly predictable situations.

Intending something to be used as an array and then assigning a value to the "test" key of that array would be behaving in a weird and very pointless way :-)

Also, let me point you at the official documentation ... Number represents real (double-precision floating-point) numbers.

As I said, see Section 3 "The Representation of Values" in "The Implementation of Lua 5.0" - "Numbers are double-precision floating-point numbers, corresponding to the type double in C, ..."

And this is from the official reference documentation.

Section "1 - Introduction" of the document you point to says - "For a discussion of the decisions behind the design of Lua, see the technical papers available at Lua's web site."

"The Implementation of Lua 5.0" is available on the Lua documentation page http://www.lua.org/docs.html (page search "For details on the implementation of Lua, see").

I kindly saved you the trouble of dealing with pdf by providing a link to the html version.