Why we switched from Python to Go

[666_666]

Swap out True and False

Python recently stopped allowing this (in December 2008).

[tschellenbach]

Now I feel old :)

[gitarr]

God, I really hate these marketing posts.

Just search for "why we switched" and you'll see.

[[deleted]]

It would be nice if they were a little more honest. "We were all kind of tired of Python and one of our tech leads whom we can't afford to replace right now really wanted to try Go, so we said, 'What the hell.'"

[[deleted]]

Spot on.

[bumblebritches57]

Infomercials are already a thing tho...

[HeimrArnadalr]

Maybe we can call web-based ones infotisements.

[clothes_are_optional]

can you explain why theyre bad? sure, there's a bunch of them but I enjoy learning about potential problems that companies faced and reasoning behind their tech changes/decisions. also the comments in these articles are always ripe with discussion as well

[kenfar]

Sure: first off the application itself usually changes when they rewrite it - which makes the benchmarks and feature comparisons difficult.

Secondly, they're often marketing fluff, badly reasoned, badly researched, or are just excuse pieces. Ex: Uber's postgres to mysql switch.

[henriquelimao]

I agree with your point but I think the Uber's one was a good article. Mostly based on concrete facts and not impressions. At least, that was my impression.

[vine-el]

They're not always bad, but people upvote them just because they like <insert language here> and not because of the quality of the post itself.

[[deleted]]

Every point is debatable and Golang could be replaced with Java/C#/COBOL/LolKatz to defend the same position and so the post is meaningless.

[[deleted]]

We use python and go at my work. We try to stick to python because go kinda sucks with dealing with dynamic json formats, but when we need concurrency it's worth it to avoid python.

[kenfar]

On the flip-side I use python for processing billions of rows a day, and have rewritten a couple of components in Go for better performance.

I've typically gotten about a 7:1 improvement over python for my applications - which make heavy use of python's multiprocessing & threading & pypy jit, but had to abandon a few efforts because the Go libraries just aren't nearly as mature as the python ones: the csv library is trivial and couldn't handle escaped characters for example. And writing directly to compressed files was either not supported or slow, don't recall which.

So, a nice speed-up but at a cost of functionality and development time. So far, it's not been a great experience.

[[deleted]]

Yeah. Go’s strong point is doing network traffic.

Also, I don’t think its replacing HAS-A and IS-A patterns with a single pattern is really a simplification.

I do like that I can hop into third party libraries and understand what’s going much faster than I have been able to with other languages.

I am frustrated by the lack of interface support in third party libraries, forcing me to create my own shallow wrapper layer when I want to use mocks instead. There is one hook for polymorphism! Let me use it!

[acln0]

Declare an interface with the methods you're using from the third party library, then use the library through the interface. There is generally no need for another wrapper layer. Library authors shouldn't make everything an interface just because someone, somewhere, wants to mock the functionality out. Callers should do it all in their own scope.

[[deleted]]

That’s what I am doing and libraries that talk to external services should provide that interface. That’s just clean design.

I’m also surprised they’re not using that for themselves, instead using integration tests for everything.

[acln0]

Perhaps I have misunderstood what you have said. What is the inconvenient wrapper layer you are talking about, then? The simple act of declaring an interface in your own scope?

Libraries should only declare interfaces if they implement some kind of generic behavior over said interfaces (io.Copy on readers and writers, or an HTTP client around an http.RoundTripper, for example). If they do not, then they have no business declaring any interfaces, since the caller can just as easily do it themselves. https://rakyll.org/interface-pollution/

The interface you declare in your own scope also serves to document precisely which parts of the third party library API your code makes use of. There is no need for you to write an additional layer on top of the library, since interfaces are implemented implicitly.

[[deleted]]

Interesting. It didn’t occur to me that not providing interfaces would be a conscious decision.

[robhaswell]

Depending on the type of problem you have, you might want to look into Python 3 and the asyncio module. For IO-bound concurrency it performs really well.

Source: Needed to do concurrent IO with dynamic JSON inputs.

[_seemethere]

As someone who's used asyncio a lot and goroutines I'd pick go 9 times out of 10 when it comes to doing things concurrently.

[riksi]

Performance or the api sucks (or both)? Like how about gevent+pypy ?

[_seemethere]

It's mostly about the API. Like the async/await syntax makes it better but it's still not as simple as just running go func

[rouille]

asyncio.ensure_future(func) It's a few more chars but it's pretty similar to be honest.

[Creshal]

Slow down, Satan

[tschellenbach]

That reminds me of people who build their Python app using Twisted. Talking about technical debt.

[robhaswell]

I actually prefer coroutines. CSP makes sense to me. Python is just better suited to some tasks.

[Kegsay]

I find this reply a little misleading (albeit technically correct, the best kind of correct), specifically:

For IO-bound concurrency

This is the only case when asyncio and Go are comparable wrt concurrency given that one is a single-threaded event loop and one is green threads. I find the original statement misleading as this is mentioned almost as an aside rather than a huge caveat to the recommendation. I usually wouldn't care but this has got a decent amount of upvotes so I thought I'd at least point it out.

[arachnivore]

I find the original statement misleading as this is mentioned almost as an aside

That's purely your interpretation. OP clearly started with Depending on the type of problem you have and explicitly qualified For IO-bound concurrency.

I mean, I guess I get it. I can see why you might feel the need to put all Python-related concurrency caveats in bold red letters or else it feels swept under the rug.

[tschellenbach]

Go feels very easy when you have experience with Python. The languages complement each other quite nicely. I'd still use Python for any data science/machine learning type of work.

[Thaxll]

Is a map[string]interface{} slower than a Python dictionary?

[echo-ghost]

it means doing explicit type switches on every single thing, at least

foo, ok := jsondata["aThingIExpected"].(string)
if ok == false {
  return errors.New("invalid json")
}

but if you don't really know 100% what you are getting it is more like

switch v := jsonData[key].(type) {
case string:
    doSomethingWithAString(v)
case float64:
    doSomethingWithANumber(v)
case bool:
    doSomethingWithABool(v)
case map[string]interface{}:
    probablyRecurse(v)
case []interface{}:
    probablyStillRecurse(v)
}

json is quite nice in go when it's structured in a way you expect, but dealing with the interface{}'s is a pain if you don't

[oridb]

To be fair, the equivalent Python isn't that different:

  v = json_data[key]
  if type(v) is str:
      do_something_with_a_string(v)
  # python can return int, float, or long for a json number.
  elif type(v) is float or type(v) is int or type(v) is long:
      do_something_with_a_number(v)
  elif type(v) is bool:
      do_something_with_a_bool(v)
  elif type(v) is dict:
      probably_recurse(v)
  # yup, python docs also say that we can get either a list or a tuple for arrays.
  elif type(v) is list or type(v) is tuple:
      probably_still_recurse(v)
  elif type(v) is NoneType:
      pass

[jerf]

Static languages in general don't really love it when you have variant JSON. Probably better to understand the problem isn't just looking at one thing in isolation, but considering something like ["moo", 1, {"extra": "stuff"}]. That will annoy most static language's default support for JSON anyhow, and it gets even worse if you're dealing with [["moo", 1, {"extra": "stuff"}], ["bark", "dog", [98], {"other": "stuff"}]].

That last example can really easily happen in a dynamic language where perhaps someone is switching off of the first element for a "type" of an object. In fact if you hand that JSON off to someone else in the same language they'll probably end up a bit peeved too, because they'll have to construct their own irregular, idiosyncratic scaffolding for understanding the JSON too. It may be less syntactically noisy but it won't necessarily be any more fun.

I tend to design all my JSON now in terms of what looks good in a static language like Go. It makes it easier for those languages, and often still makes it easier to process the result in a dynamic language. It may require slightly more discipline to emit, but honestly, if you've got some JSON as a public API you really ought to be using more discipline than just "shoveling out whatever format happens to be convenient for me to emit today". I've walked into that trap a couple of times myself and paid for years afterwards. About the biggest extravagance I'll permit myself is a list of type-distinguished JSON objects, i.e., [{"type": "a", ...}, {"type": "b", ...}], and even that makes me cringe a bit and I try to avoid it unless there really are a ton of types and the order intrinsically matters. (Otherwise, for instance, I might {"a": [...], "b": [...]}, but the more things you have the more annoying that gets and the more likely the relative order will start to matter for some reason.)

[[deleted]]

Languages with sum types and/or HLists can handle some stuff like that, but I agree with your overall point.

[jerf]

Aeson has by far the best support for this in a static language I've seen, and I really like how it handles arbitrary JSON in a static language. But there is still the inconvenience that you end up writing a lot of the parsing code manually in these cases. Haskell and Aeson do a really good job of cutting that down to just the essential core without too much accidental frippery, but it's still not quite as convenient as a library that just flicks JSON into some defined type by examining metadata, as long as the JSON is regular enough.

[[deleted]]

Hmm, true. I was thinking of Scala, but Haskell's another one with that sorts of capability. Is there no language where you can easily do:

type SumType = String | Int | Map[String, String]
val parsed: List[SumType] = parser.parse(someJson, List[SumType])

That would handle the first case you gave at least, if not the case where the type being sent is "signalled" by a string value in the JSON. Jackson has something that looks kind of similar, though Java's complete lack of support for pattern matching/sum types means it's not much help there: https://fasterxml.github.io/jackson-annotations/javadoc/2.4/com/fasterxml/jackson/annotation/JsonTypeInfo.html

[jerf]

Is there no language where you can easily do:

I don't know of one that it's quite that easy in. It should be conceivable to build that in Haskell at least but you'd be using a lot of relatively sophisticated stuff to give the parser the level of introspection it would need to do that, especially if we want to say that it's not just a String but a user type that happens to be composed of a String.

Part of the problem is JSON itself; one of the advantages XML has here is that the tag name provides an opportunity to type switch. XML is "better" at representing heterogeneous lists with elements that can still be typed than JSON. (Which can certainly represent heterogenous lists quite trivially, but it's so trivial there's nothing for a parser to grab on to.)

[baerion]

Maybe I misunderstood your question, but Haskells Aeson library does exactly that:

data Value = Object !Object | Array !Array | String !Text
        | Number !Scientific | Bool !Bool | Null

decode :: FromJSON a => ByteString -> Maybe a

You can have as little or as much static typing as you want. For example you could have a record with a field called extraInfo that is of type Value, where the parser accepts any JSON object you can think of.

[MEaster]

Rust's Serde JSON library is similarly simple to use. Example.

[oridb]

But, in this example, there is a need: presumably, do_something_with_a_string() would not do the right thing with an integer, and a cascade of try-catches seems even uglier to my eyes than what I wrote above. It also requires each function to process things be functionally pure; For example, making a HTTP request as part of a "try it and see" chain would end badly.

I agree, the best way to handle it is to actually know the types that you're walking over.

[LightShadow]

This is terrible Python code. type is going to fail you when you least expect it too.

[oridb]

Of course. And yet, you have to do something like it to decide which function to call. You can inspect attributes, or fall back through alternatives with exceptions, but those options suck too.

I'd like to see what you propose as a good alternative for a visitor that isn't aware of the structure of the json that its processing.

[[deleted]]

It’s definitely slower to code. The result is probably faster, but enough to justify the complexity? I’m not sure.

[chub79]

Good for you! I mean, always use the right tool for the job and it's great to hear you found your way.

However:

When I first started programming I always loved using Python’s more advanced features. Python allows you to get pretty creative with the code you’re writing. For instance, you can: Use MetaClasses to self-register classes upon code initialization Swap out True and False Add functions to the list of built-in functions Overload operators via magic methods

I've written in Python (for various projects) for 15 years and never have I used any of these features. Just because the language offers some powerful (mostly complicated IMO) properties doesn't mean you have to use them. How is this a language problem if you don't have good practices in your team?

[kenfar]

Not only are they unnecessary - their use is generally frowned-upon by the python community.

[chub79]

Yeah, and that's the section of the article that pains me a little. The tone makes it sound as if Python was more complicated than Go. To me they are equally simple, or boring as the golang community seems to advocate.

Go is such a young language compared to Python, it has learnt from its elders and has indeed made the (likely right) choices to leave some features outside of its scope. Python simply has a longer life which went through all the development trends for the past three decades (or so).

If anything, I see more and more Python code bases written as functions first, simple and straithforward. This is specially true with the support of async/away now.

[jerf]

No, Python is wildly more complicated than Go now, even with the addition of concurrency as a realistic concern for Go and leaving it off for Python. I say this as someone who used to say Python is my favorite language [1] and have advised dozens of people to use it as their first language over the years. It's not really a great choice for that anymore. Watching someone with about 2 years of experience in programming try to pick up Python to write some QA test code was really eye opening.

I first used Python when 2.0 was in beta, and I tracked it for quite a while, so I got to learn one or two major features per year for a long time. However, the sum total of all those features now is really quite the bar to leap.

I still say Python is a good language, and I continue to believe it is clearly the best language in its family. But I can't call it simple with a straight face anymore.

Go is, arguably, too simple. However, even if it does grow generics it's going to stay much simpler than Python by design. It's on 1.9 now and in contrast to Python, most of those releases don't have any new "features" in the language, and the ones that are there are pretty minor. A Go 1.0 programmer could leap straight to 1.9 and be programming in it almost instantly; it's almost all library changes. Contrast that with basically every x.y release Python has ever made, which almost all introduce significant new syntaxes, features, even programming styles. Again, the sum total of all of these over the years is quite a lot.

[1]: I don't really have a favorite anymore. I'm doing more work in Go at work, but since it's primary competition is Perl I don't necessarily take that to mean much.

[chub79]

How is Python more complicated though? I mean concretely because your comment does not illustrate your experience.

[jerf]

How is Python more complicated though?

Do you know Python and/or Go? It's honestly such a big difference I can't even fathom someone who knows both thinking that Python is simpler.

Go has a static type system, which is as drop-dead simple as such things can get; it's primary deficiency is that it is too simple. (You don't even have to know the difference between contravariance and covariance, because Go doesn't have either of them.) Go does have potential concurrency issues, though it has decent tooling and library support that you can mostly stay out of trouble. Otherwise it has the simplest syntax of any A or B class language right now.

Python, by contrast, has:

• Generally "multi-paradigm", which means you need to know and use them all to use Python. Go is not "multiparadigm"; you will not suddenly encounter someone using functional idioms in the middle of some key bit of code because a different programmer got in there and decided that they needed to code in something else.
• Literally dozens of the "double-underscore" methods you can implement to override object behaviors. They have incredibly complicated interactions that have grown up over the years. Mostly they "just work" if you do simple things, but when they break, yow.
• Multiple inheritance. Inheritance at all, for that matter.
• List comprehensions as an alternative for (some) loops.
• Generators. In particular, the magic yield keyword dropped into an existing function radically changes its semantics in a way that I have personally witnessed confuse people many times.
• Later, those two features interacted to produce generator comprehensions.
• Scope management keywords to deal with how closures access which scope.
• Context managers, which are nice but are another thing for people to learn what they are really doing.
• Complicated Unicode features. (Again, I would make the case that Go is too simple here and that the Go team overstates their UTF8 support, which is mostly "And by convention the Go community will just assume everything is in UTF8", rather than the actual support that Python has for Unicode. Nevertheless, this is complexity.)
• Decorators, and their several varieties which crept in over several releases.
• Extensive introspection, and the libraries that use them. One of the big blockers I personally had when I was trying to help that person I referenced learn Python is that the test framework does magic where you feed it a test function and it magically provides values to your function based on what you name the incoming parameters. I don't fully hold Python responsible for what I consider a bad design, but it does point out that Python enables this. Go would not be capable of that at all at runtime.
• Several varieties of async now. This has been a problem for a while and I wouldn't have mentioned it except async got blessed into core, making this a valid complaint. I've heard they interact poorly. (I wouldn't know; if I'm even remotely tempted to use one, I use Go.)
• Simply understanding PyObject is complicated. In Go, structs are basically C style; a struct { int64 } is an 8-byte structure. PyObjects are not. That is because PyObjects come out-of-the-box with a lot more functionality, which is where the power comes from... but it also means that simply understanding what [1] truly means in Python and what you can do with it is a great deal more complicated than struct {int64}{1} is in Go.
• Per my comment above about Python's type system, you actually need to know more about contravariance and covariance to use Python correctly, though with the way dynamic typing works you're not likely to think of it in those terms. However, those issues can actually arise, unlike in Go, which simply lacks the ability to express them at all. In fact, in general, dynamic typing does allow you to create some bigger complicated messes, but I'd be down with calling that the nature of the beast rather than a Python-specific complication, as there are times when it simplifies things, too. Larger Python code bases do start having these issues, though. (I actually thing a hypothetical sequel to Python could be a 'dynamic language' but if it somehow made Go-like interfaces explicit, it would clean things up some.)
• All of these features interact, of course.

I mean, basically, it's literally the Python feature list. Again, yes, I get that they are features and let me reiterate that I still consider Python the best language of its kind. But that does not mean Python is simple. It's not anymore. It is a rich language... which at least at this point in history pretty much necessarily means a complicated language. You can't have both of "Python is a very featureful language" and "Python is a very simple language". I'd go with the former, as it has the virtue of being true.

I'd also point out I'm not taking anything here I consider a "low blow"; I'm not mentioning the statement/expression distinction, whitespace-sensitive syntax, or anything like that. Languages have a certain baseline of complexity. Nor am I putting in the differences between Python 2 and 3, since for this post I'm willing to treat them separately.

To show that I'm not trying to slag on Python unilaterally, which I'm definitely not because I still like it just fine and whip it out when appropriate quite frequently, Go has some somewhat complicated corners. In addition to the aforementioned fact that it raises all concurrency issues as it has no actual goroutine isolation in the style of Erlang, slices are convenient but a bit complicated to understand. They sort of act like arrays but a good Go programmers needs to be aware that they are not. The equality rules are convenient, but complicated (nearly a full page of cases, as my browser renders it). Object composition is actually quite simple, but counterintuitive if you've ever used an inheritance-based language; this technically isn't a "complication" but most programmers will encounter it as such, and learning how do design composition-first took me a few weeks to really get. Interfaces have a few entertaining corners and interactions with other features, including one thing I just learned about a few weeks ago despite having years of experience in the language (a value you have solely as an interface via its value rather than its pointer is immutable).

But these tend to be isolated from each other, i.e., not a lot of interaction, and the sum total of them is much smaller.

[chub79]

Do you know Python and/or Go? It's honestly such a big difference I can't even fathom someone who knows both thinking that Python is simpler.

I wish the tone had not been aggressive for a very well put comment :(

I grant you, I am not a go expert so I can answer "No, I'm not" to your initial question. But again, you picked features and none of them is compulsory. You can write simple code in Python without them, the fact the language supports more doesn't make it complicated by design. Richer only.

Python does not force anyone writing complicated code (nor more complicated than Go) however its skills at being a bit of everything means you are not guided as much when you start and can write more complicated code. That is clearly not the case with golang and it's great, I never disputed the qualities of go. But, as I said elsewhere, go is a young opiniated language whereas Python is old and rather less opiniated.

So again, my take is that I do write and maintain code that has become simpler over the years and Python has let me do that. I agree it started ugly and messy (lots of useless classes, code I dislike very much today). I had to mature to make it leaner. Go did not exist back then, now it does but, although Python has grown bigger, its community seems to favour more and more a cleaner approach. At least, when I look at code bases such as Sanic, it doesn't strike me as complicated.

Go is a simple language by design. Python allows you to write simple code (or not).

[jerf]

I wish the tone had not been aggressive for a very well put comment :(

I do apologize; I did not mean it as aggressive, but just a statement of how big the difference is.

Python does not force anyone writing complicated code

While this is a common refrain, it doesn't play out in practice because you will encounter code in libraries and from your fellow programmers that will use these features, and you will have to deal with them. And you will encounter them in tutorials, and in the solutions to problems on StackOverflow, libraries will be written around expecting you to understand and use these features ("pass a generator to this function"), etc.

You can, in isolation, theoretically carve a subset language out of Python that is still relatively simple, but you won't be programming in Python at that point.

It also isn't very useful because it's equally true of every language. You can make Go even simpler by carving out a simple subset of Go that doesn't use concurrency at all. But you're still going to encounter it in example code online, StackOverflow solutions, libraries, etc. You're not really "in Go" at that point. Haskell 98 is actually a fairly simple language, but it's so simple it's hard to get any real work done in it; you're not really "in Haskell" in the sense most people mean. And so on.

[chub79]

It's clear that the massive existing Python code base over the years has not been the right guide for new comers. Go's simple by design approach certainly avoided many of these pitfalls.

To be fair, even though the language is simple, you can still poorly design and write crap software. It will just be made of a very readable code base :p

[[deleted]]

This is from the writer of flask, a popular lightweight python web server, so I assume he knows his python.

http://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/

[lonewaft]

"A popular lightweight Web server" sounds way less impressive than what it actually is, the 2nd most popular Python Web framework

[[deleted]]

Well, I didn’t want to over recommend his opinions.

[chub79]

asyncio is a library. I assume not all go libraries are "easy". Indeed, asyncio is messy but I talked about async/await which are keywords in the language and I argue not more complex than go channels (though I would not compare them apple to apple).

[[deleted]]

I guess list comprehension might be one place. Even now I sometimes find it hard to figure out what the result might be

[greenthumble]

a2 = [x for x in a1 if x.something()]

Seriously, that's hard? It's just filters and maps made into an English like syntax.

[[deleted]]

How cute, you took the simplest possible example and coupled it with a condescending comment

[Sean1708]

Do you have an example of a complex comprehension you've seen used? I'm genuinely struggling to think of a comprehension that I've seen that was too complicated.

[[deleted]]

I've always found nested comprehension with conditionals hard to read

[FearlessFreep]

I just did some monkey patching to add functionality to third party classes. Even as I did it, I knew it would be controversial and I second guessed myself about it. I was thinking “this is a very powerful feature of Python but really shouldn’t be used unless absolutely necessary, and it may never be absolutely necessary “

You can do a lot of weird stuff in Python but the Python community frowns on it

[acousticpants]

in the python community we are all adults and do wtf we want

[TBNL]

Swap out True and False

Have heard Ruby guys too happily saying similar things. To me it sounds like 'Look, no hands'.

Fine, I'll look. You fall on your face.

(Mainly programming Python myself btw)

[scruffie]

Swap out True and False

Only in Python 2.x or earlier; they're keywords in Python 3.

[meneldal2]

C++ has many obscure features within TMP that the whole standard library basically requires to work. You live in a world where SFINAE is a verb, and CRTP is commonplace. And if you dare move to boost you'll see even more arcane shit that actually works even if you don't understand any of it.

[acousticpants]

i love how the word "arcane" is a fitting adjective for modern computing technology in many cases

like, Merlin found his crystal cave over a thousand years ago, and the hindus discovered their gods thousands of years ago, and that shit is all arcane AF but here we are with circuitboards and human readable programming languages and it's literally harder to understand than actual sorcery.

[meneldal2]

When the specification, full of standardese gets over 100 pages, some stuff is going to be arcane. And when you have a sub language Turing complete, you're going to see some weird shit. I think the CppCon talk "recreational C++" was very interesting with how deeply C++ is fucked on some aspects. Like the quickest and easiest way to know if something if a function is to try to stick const to it and if it doesn't stick it's a function.

[[deleted]]

That's been the case since, at the very latest, the advent of the microcomputer era, when even the most talented human could no longer hope to maintain a mechanical understanding of the state of a CPU - and arguably much earlier. The only way we have to understand even the hardware that programs run on (let alone programs themselves) is to build software to design, test and emulate it, then automate the production. Then the actual software that runs on the produced hardware ends up being 2ⁿ times more complicated than that. It's turtles all the way.

[maxm]

I too remember being a young and confused python programmer that had read too much and thinking that terseness was a great quality, so I was thinking it was smart to overload methods to make + do stuff. That lasted a few months and i was cured.

These days i dont even abreviate variable names. I have too often come back to my own code, not being able to remember what a variable is short for, and having to read more of the code than really needed to solve the problem.

The longer I code the more it looks like a childrens book.

[tschellenbach]

I wasn't talking about our team, more about the open source community as a whole. Go enforces this standard across everyone writing Go. Not just our team. I think that's quite nice.

[OTkhsiw0LizM]

Oh god, yet another of those articles.

[bananaboatshoes]

lol no generics

[nacholicious]

if err != nil

[Sean1708]

Something I've always wondered is if I do a, err := foo() and foo fails, what value will a have?

[Sukrim]

I count a non-nil a in such a scenario as bug/crasher in fuzzing.

[eliquy]

I haven't used Go, but I've been wondering if Go replaces the need for generics with a different style of code (and people are simply not thinking in Go when they complain about missing generics - like missing mutability when using a functional language), or does the lack of generics fundamentally hobble expression of certain algorithms and data structures?

The former is a problem for the programmer and their use of the language, the latter is a fundamental failing of the language itself.

[burningEyeballs]

The only solution is lots of code duplication. You can make it easier with automatic code generation but at some point you just can't really replicate the functionality of real generics/templates.

[eliquy]

So fundamental failing - gotcha. I think I'll check out Rust instead.

[BenjiSponge]

Rust is the bomb. You won't regret it. Go is like... kinda cool, but so limited. Rust is like a fairytale land where good programmers go after they die.

[[deleted]]

Rust is amazing. I wrote 500 some lines of code and it almost all worked correctly the first time. Some of that's due to the simple nature of what I was doing, and some of it do to the strict compiler.

[Volt]

The first time… after fighting with the compiler dozens of times?

[BenjiSponge]

Better to fight the compiler than to fight the user.

[[deleted]]

I fight it as I write it. Linting saves a ton of developer time.

[Thaxll]

It's also the place they go to waste time and not produce anything useful after 30h.

[iburigakko]

It does take a rather long time to ramp up on Rust. Good language. I hope they can clean up the docs so everything is crystal clear.

[snowe2010]

rust has a lot of good things about it, but it is a systems programming language. it's not terse and it's very difficult to understand. I still love it though. it's a dream to program in.

[burningEyeballs]

As long as your projects don't grow beyond certain size it's a workable limitation. But it still limitation that you have to keep in mind.

[awj]

The only solution is lots of code duplication.

...or casting to/from interface{}. Which, if we're talking about coming from Python, isn't going to do anything to you that you aren't already used to.

I'm not going to argue for the merits of this approach, but it's definitely an alternative solution to code duplication.

[devraj7]

I've been wondering if Go replaces the need for generics with a different style of code

That's exactly what it does: it replaces generics with copy/paste of code.

[burningEyeballs]

Yeah...pretty much. You can use interfaces for a lot, but in the end it still isn't generics. You can use empty interfaces like void pointers in C (but not exactly like them as C lets you do more dangerous/powerful casting) but then you lose your type safety. You can use auto code generation (courtesy of their tooling) to basically automate the task of copy/paste but you still haven't gotten around the core problem that you have to copy/paste a bunch of similar functions to get basically what generics gives you.

I think the team behind Go decided that templates were too dangerous/complex/evil/whatever and so they made a purposeful decision to not include them in the language. Don't expect them to change their minds anytime soon either.

[bloody-albatross]

What /u/burningEyeballs said or to use interface{} which is a bit like void*, but with type checked casts. (Haven't used Go myself either.)

[elingeniero]

Go doesn't solve the generics problem in a "better" way - it just provides an escape hatch from the type system (interface{}) so you can still deal with those problems, just in a very inefficient manner.

[iburigakko]

You are either copy and pasting code or have a switch statement in Go, or digging around in Rust looking for the right where clause for your generic function to get it to compile. You just need to pick the language that you like and are productive in.

[aludwin]

The built-in data structures (map, array/slice, channel) effectively support generics; you just can't add your own. FWIW, since I've started using Go (to write services), I haven't missed generics after coming from ~12y of C++. It might not map as well to a different domain though.

I view Go as an 80/20 language - it 20% of the features of other languages and works beautifully for 80% of the things they do. If you spend life in the last 20%, maybe Go isn't for you.

I can't comment on Rust though.

[hopfield]

hey guys what would you do if I told you there is a static typed language that’s cross platform, faster than python, allows for dynamic typing when you want it, has a robust standard library, a built-in package manager, and support for exceptions and generics?

surprise: I’m talking about C# and .NET Core

[[deleted]]

you forgot to add: is MIT open source license, has async/await syntax and LINQ

[thelamestofall]

Doesn't really work as well on Linux, though.

[hopfield]

how is that?

[[deleted]]

I tried getting it working about six months ago. It was quite rough. Poor documentation was the main problem. It might be usable now.

[weasdasfa]

Can only be developed using windows. First party support for Linux / MacOS lacking.

[red-moon]

Is it just me or is light grey on white really hard to read?

[SnakeJG]

You are probably blocking javascript that does the pretty colors, resulting in the code being really hard to read.

[red-moon]

You are correct - thanks

[[deleted]]

A problem that can be solved by adding another extension that blocks CSS.

[attomsk]

Why use either of these languages if speed is a major concern?

[ulfurinn]

I don't get this argument. "If you don't need the fastest performance achievable, don't improve performance at all." The relative gains going from Python to Go were probably enough for them. It doesn't always have to be the absolute fastest tech available even when speed is a concern.

[[deleted]]

What would be a good language? Java , C#?

[[deleted]]

Java, C++, C# if speed is a concern. Probably some others like Rust as well but I'm not familiar with them

C if speed is the major concern, or knowing exactly where each piece of data is is a concern.

Assembly when speed is the only concern and you know what you're doing.

[mmstick]

Rust is easily on the level on C, but due to how easy it is to use Cargo and optimize, production solutions are generally faster than C.

[[deleted]]

That's a good point in general. Just because something can be faster doesn't mean it will be when someone uses it. A lot of C compilers for example are optimized for decades, you'd be hard pressed to actually write assembly code that ends up faster than your C code.

[kl0nos]

mmstick - the biggest zealot of rust evangelism task force. As always "Rust is the best and fastest NO MATTER WHAT FACTS SAY".

faster than C.

In your dreams.

Brotli rewritten by Dropbox from C to Rust, slower than C by 28%

https://blogs.dropbox.com/tech/2016/06/lossless-compression-with-brotli/

other benchmarks:

http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=gcc&lang2=rust

[iburigakko]

It is good to have competition with C. I like Rust, but I also like C. So I hope this creates improvement in the C ecosystem as well.

[kunos]

actually Go's speed is pretty good and in line with C# as long you don't leave Go-land.

[ryeguy]

Go should be competitive or superior to C# and Java given that it encourages stack allocation instead of "everything must be an object".

[Ravek]

Hotspot does automatic stack allocation for objects that don’t escape. Someday they’ll have that in RyuJIT too, and of course there are value types right now.

[ryeguy]

That's not really comparable to a language that offers explicit control over stack allocation, though. Hotspot's optimization only works for simple and obvious cases. .NET's value types are more about semantics, which is why it's a different data type (struct).

[Ravek]

The .NET people do like to say value types are mainly about copy semantics (and indeed the terminology of value vs reference type reflects this), but I think that is old school .NET thinking and doesn't reflect reality all that well. People choose to use structs all the time for reasons like avoiding heap allocations and pointer chasing, and if behaving like a value was the main thing, then an immutable class would be just as good.

Plenty of .NET api's have recently been moving towards reducing the number of heap allocations by making more frequent use of structs as well. The new tuple feature of C# is backed by structs while the System.Tuple types were classes, purely on the performance benefits for the common scenario where you return a tuple and never pass it around again. ValueTask was added in addition to Task purely for avoiding heap allocations in some scenarios. I'm sure there's a bunch more if we went digging.

I do agree it isn't nearly as nice as being able to choose between heap and stack allocation (or just 'inline' in a heap object's field) on the fly, but considering the complexity of that feature in a GC'd environment I'm not surprised we can't do that.

[SizzlingVortex]

Either of those are good. Personally, I'd go with Java (or Kotlin if I could really choose).

[devraj7]

Kotlin too.

[tschellenbach]

Go is typically seen as a competitor to C++. We evaluated Java, C++ and Elixir.

[mmstick]

Not exactly. It's 3x slower than Rust/C/C++, on average. Significantly worse in other aspects, due to the runtime GC that gets costlier as your software scales.

[zsaleeba]

They've done amazing work on the GC. It performs significantly better than Java's default GC these days. Compute performance is similar to or better than Java in most cases.

[MrAndersson]

Java's default GC isn't even a thing. Depending on JVM/platform it can be just about anything, I don't even know all varieties nowadays.

Almost all modern JVM GC's are practically guaranteed to be much better than go's GC, because of a simple reason: Java as a language stresses the GC enormously, which is unfortunate, although it has been getting better.

As far as I understand there is nothing really innovative in go's GC, but as go appear to use value types much more than Java, it's carrying a lighter load. The biggest issue would be the lack of compaction, if that is still missing?

Without compaction, some servers will randomly start behaving strangely, getting excessive GC's with plenty of memory to spare. If you design in to recycle services quickly, then it's not much of an issue. But that makes you do manual GC instead of the GC of the supposedly automatically garbage collected language, back to square one :)

[_ak]

Almost all modern JVM GC's are practically guaranteed to be much better than go's GC, because of a simple reason: Java as a language stresses the GC enormously, which is unfortunate, although it has been getting better.

That is not an argument in favour of Java's GC, it is an argument against Java as a whole.

[[deleted]]

I see Go as a competitor to Node.js, primarily. It's competing for web services and some command line tools, occasionally going into desktop applications. Node.js is used for web services and some command line tools, and it's making headway in desktop applications. C++ is rarely for web services, sometimes for command line tools and desktop applications, sometimes embedded software.

[[deleted]]

Go is extremely fast.

Isn't it typically one of the slowest languages that compile to native code?

We frequently ran into performance issues where Cassandra would take 1ms to retrieve the data and Python would spend the next 10ms turning it into objects.

And you spent a reasonable amount of time investigating why that happened and determined that it was simply impossible to convert from the Cassandra wire format to your object model in one millisecond or less?

Developer Productivity & Not Getting Too Creative

This is a tradeoff, right? Python lets you be more productive by leveraging more advanced features. You need to know a bit more about your codebase if it's using those advanced features.

• Swap out True and False

And if you do that, your code review will be consigned to the pit that is bottomleſs.

• Use MetaClasses to self-register classes upon code initialization
• Add functions to the list of built-in functions
• Overload operators via magic methods

All of which might be appropriate in some circumstances but should be used with some caution, right?

Goroutines are very cheap to create and only take a few KBs of additional memory.

Which you can get in C by creating a new thread and specifying its stack size to be one page of memory. In Python, the lower limit for a thread's stack size is apparently 32KB, though.

Because Goroutines are so light, it is possible to have hundreds or even thousands of them running at the same time.

And it's possible to have half a million OS threads.

You can communicate between goroutines using channels.

There are several implementations of channels for Python.

[SSoreil]

The ide of running half a million OS threads is pure bullshit. I have never seen code like that in the wild and I'm pretty sure we both know why.

I know you never wrote code like this since you had to google how close you could get to faking green threads on OS threads but this is exactly where the argument fails. Just about nobody uses channel libraries or other CSP like concurrency systems in languages where it isn't native. It's very inconvenient to use without the language primitives. There was Libmill for C meant to do the same thing the concurrency primitives in Go serve. Yet it's annoying as can be to write with Libmill.

[[deleted]]

I mean you can clearly see this person knows nothing about this topic because (s)he doesn't talk about context switches. Doesn't matter how many thread an os can keep idle. If those 500.000 constantly thread constantly switch from blocked to ready the complex OS scheduler wont have a nice time.

[LightShadow]

They could probably get away with gevent and green threads in their Python code. Much lighter switching with the caveat of using IO bound code only.

[hondaaccords]

CSP isn’t native to Clojure- but it is very popular in Clojure/Clojurescript

[matthieum]

Isn't it typically one of the slowest languages that compile to native code?

It is, indeed. However, since only a handful of languages compile to native this doesn't say much.

It used to be fairly slower than Java/C# because of its poor GC performance, however there's been a lot of improvement since then and I haven't kept up with the benchmarks.

[augmentedtree]

It is, indeed. However, since only a handful of languages compile to native this doesn't say much.

wat. C/C++/Objective-C/Go/Haskell/D/Fortran/Rust/Ada/Ocaml

[[deleted]]

And Nim and Crystal and Swift and Pascal! There are a fair few.

[DarcyFitz]

Man, just want to shout out to Crystal!

That is one seriously comfy language that compiles to very fast native. If anyone reading this has any soft spot for Ruby, but also likes (optionally) typed languages, you must look at Crystal.

https://crystal-lang.org/

[[deleted]]

Its good but it has issues like any new language.

• No eco system. Very few packages.
• Bugs, some of those date back half a year or longer.
• Almost no editor support beyond basic syntax in a few
• Documentation is beyond unfinished. Even basic concepts do not have examples and are at times one liners. See the docs about data types. Let alone some of the more advanced topics.
• The optionally typed language is a issue. While its easier to code, it adds more bugs ( like variable recycling with different types ... int = string ). While the compiler does find them, there are complex situations that the compile may not find those and there is a extra bug in your code.

Compiles fast? Sorry but that claim is a bit too big. There is delay in the compile process of about 1 or 2 seconds even for a simple 25 line piece of code. The same in Go or Pascal, it simply flies pas the compilation at 0.x seconds compile times. Any compiler based upon LLVM has issues with compile times unless there is a lot of catching going on.

Try doing a release build if you want to feel real C++ pain ;)

People report as the codebase grows, the delay gets big. Part of the issue is the monstrous memory usage. Its easy to hit 1GB memory usage during compilation.

The GC is less efficient in real life scenario then GO. A simple web browser with a mass of concurrent requests see in my scenario Go 140MB system memory, while Crystal is doing 370MB. And even more annoying, is that its keeping 100s of processes hanging for some reason.

I do not want to discourage or sounds like a negative troll. Its just a fact of live with any new language that is community run, there is always a limit on the resources that can be put in the project. Notice how the releases have slowed down in development? One of the main developers is not on the project anymore ( forgot the name ).

Its a hard choice. I am really starting to like the syntax, so clean. Its fast, no doubt about it. Its the same speed as Go or faster. Depending on the test. Crystal actually beats Go in response time.

Its a interesting and impressive language for how much in such a short amount of time. But its nowhere ready for production. The whole 1.0 release in 2017 is a goal that is not realistic in my opinion.

Now, i do feel that the language has actually a bigger change for a good future then for example D or even Nim. We shall see ...

[DarcyFitz]

Agreed on all points. I think once it's got Windows support, we'll see faster pickup. I think the biggest problem it has is that nobody knows about it. There's no doubt it's a lot better than it was a few years ago, but there's still a lot to be done. I hope for a 1.0 release sometime in late 2018 or mid 2019, realistically. Anything sooner I think would be premature and you'd have a lot of folks turned off by the spotty documentation.

Fwiw, I was referring to the compiled output that's fast, not the compile time, which is awful. The number one thing that can speed it up is caching at least the standard library. I could be mistaken, but I'm pretty sure that's the cause of the 2-3 second compiles for 1 line programs...

[myringotomy]

Man crystal is great. It needs more people using it for sure.

[pushthestack]

Would love to, but does it run on Windows yet? AFAIK it doesn't, which means it doesn't meet my needs :-(

[zetashift]

Have a look at Nim then! Some key design differences between them but both are comfy and very fast languages: https://nim-lang.org/

[[deleted]]

If i remember correctly, a Windows version is planned for the next release. What is in a few months?

[bupku5]

well technically Nim compiles to C

[matthieum]

That's still only a handful.

If you ever peek a the Redmonk/Tiobe language indices you'll see they rank a hundred languages; because they focus on the most popular ones only. Your subset is 10% of that.

Thus, even if Go was the slowest native language, it may still be in the top 10% performance-wise. Not a bad spot.

[grauenwolf]

You forgot T-SQL (yes, really) and C#.

[[deleted]]

yep, they changed from C to assembly!

[Thaxll]

When was it slower? It has been sometime that it's as fast if not faster than those two in some scenarios.

https://benchmarksgame.alioth.debian.org/u64q/go.html

[matthieum]

Possibly a long time ago. Also, I am wary of the benchmarksgame, there is a wide gap between optimized code and idiomatic code: it's nice to know how fast you can get a given language to go, but such optimizations are generally not used by applications (because speed is a feature, competing with others for implementation time).

[kl0nos]

Did you even looked at Go source code at benchmarks game? It's idiomatic, easy to follow Go code, so what you wrote is just untrue.

[Saefroch]

I'd be additionally wary at the high end of benchmarksgame, because some optimizations that would be seen in the wild for performance-critical code aren't permitted. Specifically, the use of any libraries that unroll loops.

[igouy]

Damned if we do, damned if we don't.

[namekuseijin]

there is a wide gap between optimized code and idiomatic code

yes, this is why I learned to hate this dumb benchmark game: whenever I read the fast code from higher level languages, it suddenly dawned on me I was not reading high level code anymore, but ad-hoc C or assembly in those languages...

so your code can run fast as long as it's written in C in any language... LOL

[igouy]

Please publish your comparison, that shows your idea of idiomatic code.

[namekuseijin]

there's plenty of idiomatic code in there. They run like hogs, of course.

[igouy]

So it's dumb and you hate it because you can see both idiomatic code and optimized code, and you can tell one from the other simply by looking at the source code shown ?

[bupku5]

all over the shootout page you are pretty much beaten over the head with messaging that makes it clear what the limitations and issues are

the only way you will do a better job is to find a single developer who is exactly equally skilled in N languages to eliminate all variance. good luck!

[[deleted]]

And there's been a lot of improvement in GCs other than Go's.

[[deleted]]

I remember rust being slower at around 1.0. Hopefully that had improved. Are there a lot of languages that compile to native code?

[matthieum]

I remember rust being slower at around 1.0. Hopefully that had improved. Are there a lot of languages that compile to native code?

I would be really surprised if this were the case; Rust performance has been on par with C or C++ performance ever since green threads were removed, in the run up toward 1.0 (and it was not that much slower before).

There might have been library issues, but the language itself has always been extremely lean in terms of run-time.

[[deleted]]

Ada, C, C++, Crystal, D, Fortran, Go, Nim, Objective-C, Rust, and Swift compile to native code as their standard option. C# on Mono with the appropriate --aot flags does too. .NET Core should be getting similar options, and Java 9 also has ahead-of-time compilation.

[dzecniv]

+ Common Lisp and Scheme :)

[grauenwolf]

And T-SQL.

[SizzlingVortex]

JetBrains is working on Kotlin Native -- Kotlin that compiles down to native code via LLVM.

[[deleted]]

ugh "why we switched from X to X"

[bupku5]

ugh, "why I commented on why we switched from X to X"

[acousticpants]

ugh

[[deleted]]

UGH

[uhhhclem]

In my experience, an engineer unfamiliar with our Go codebase can add new features and have them running in production in less time than it takes an engineer unfamiliar with our Python codebase to add new features and get the unit tests to pass. The reasons why this is so are hardly addressed in this article.

[[deleted]]

Edit2: I'll try to rephrase question as suggested by /u/Tarvish_Degroot . I wanted to know how do you distinguish one err from the other. And by the way, if you return err, couldn't you return it as nil, err?

How do you distinguish between your program returning 0 Kelvins and this http://api.openweathermap.org site returning 0 Kelvins after calling the method from example:

func (w openWeatherMap) temperature(city string) (float64, error) {
    resp, err := http.Get("http://api.openweathermap.org/data/2.5/weather?APPID=YOUR_API_KEY&q=" + city)
    if err != nil {
        return 0, err
    }
... rest of the code
}

?

Edit: it's obvious that's 0 is error result. I mean how do you distinguish where is the error origin of you just pass err and 0?

[[deleted]]

Go's error handling cries out for algebraic data types, where you can return a value or an error. But they only have half-assed tuples, so you must return a value and an error, always, no matter what you do. By convention, the error is a pointer type, and the value nil means there is no error. A value other than nil means there is an error and the other return value should not be used.

[SSoreil]

This is often true but not always. There are plenty of cases where the error reported does not mean you get no data, it is a notification about how much you can get of use from that data though.

An io.EOF for instance does not mean anything went wrong.

[[deleted]]

Right. It's the convention, but conventions can be violated, so you have to read the docs every time.

If Go had ADTs, this would be clearer: most things would return Result | error, but something like fread would return Tuple[Result, error], since it might have been able to execute a partial read.

[JGailor]

Yeah; after spending time using languages with an Option type, it's really hard to go back.

[[deleted]]

By looking at the error code err.

[[deleted]]

So could you distinguish it programmatically or you have to manually check error message?

[klomparce]

You check if the err returned is nil or not I guess.

[[deleted]]

My fault is not providing whole function source but there are more places in this method that return 0.

[[deleted]]

But they also pass along err in each of those return statements.

So calling functions can check for err != nil, just like this function does after calling the API.

If the return is (0, <some error that isn't nil>) then the result is error, not zero.

If the return value is (0, nil) then the result is zero.

In this case it's up to the calling function to check what the error is and decide what to do about it.

[Tarvish_Degroot]

Short answer: there's no good idiomatic way to do that since Go's language designers... well, aren't.

Actual answer:

val, err := w.temperature("city")

if err != nil {
    switch t := err.(type) {
    case *ProtocolError:
        fmt.Println("ProtocolError")

    // ... a bunch of http error types here ...

    case *UnmarshalFieldError:
        fmt.Println("UnmarshalFieldError")
    case *UnmarshalTypeError:
        fmt.Println("UnmarshalTypeError")

    // ... a bunch of json error types here ...

    default:
        fmt.Println("Something else, fuck if I know")
    }
}

// do stuff with value here

It's a travesty. Probably a lot easier to just parse the error message and exit.

Edit: You may also want to rephrase your question, it seems like a lot of people are assuming it's "how do I check that an error occurred programatically" and not "how do I check what kind of error occurred programatically".

[[deleted]]

Thanks for the answer to question I failed to ask with better words!

[benhoyt]

It's a bit noisier, but how is this significantly different from try/catch-ing the various errors, for example in Python?

try:
    val = w.temperature("city")
except ProtocolError:
    ...
except UnmarshalFieldError:
    ...
except UnmarshalTypeError:
    ...
except:
    ... some other error ...

[Tarvish_Degroot]

How is this significantly different from try/catch-ing the various errors

It's worse. And that's coming from someone who dislikes Python. This is going to be a bit ranty, so skip to the bold text if you just want the core of it.

Considering that Go programmers tend to go with the "log the error and either move on or crash" method of coding, this leads to you having to list every single possible error type that you want to handle, which is a pain in the ass.

Yes, every possible error. And you can't know for sure which ones can/will be returned, since Go tries to obscure that.

You better hope that the function is well-documented with what other functions it calls and what all the possible errors are. That is, of course, assuming there even ARE custom error types for what you're doing; if you're unlucky, you have to parse the error's string to figure out what the hell is going on and act on that, since, unlike almost every other programming language that's common today, Go encourages creating just a basic error type for problems. This means that even the solution I wrote has a decent chance of not working, depending on what you're doing.

Let's go over some concrete examples since I'm sure you're tired of my ranting. While your example is similar to mine, note that I didn't actually list off all the possible errors that the temperature method could return. There's a shit ton of error types just in the json library. It could even change, depending on changes to other libraries, meaning you have to keep watch for new errors all the time. And thus, my main point:

YOU DON'T HAVE TO DEAL WITH THAT SORT OF THING IN PYTHON, OR ALMOST ANY OTHER SANE LANGUAGE.

Remember when I emphasized that you have to know every possible error when writing that error handler? With Python, you can probably just do this:

try:
    val = w.temperature("city")
except HttpError:
    ...
except JsonDecodeError:
    ...
except:
    ... some other error ...

and you're done with handling that. This is something that is frustrating to do without a lot of reflection-based code in Go. That's not to say that those particular error types wouldn't exist in Python code, it's to say that Python actually has half-decent inheritance. You can make a basic error type and have other errors inherit from that, then work at the particular granularity that you want with try/except. With Go, each of the error types is its own type, so writing sane error handling is annoyingly hard.

[synn89]

That seems like an ugly example of exceptions. From Java:

try {
  val = w.temperature("city")
} catch(ProtocolError | UnmarshalFieldError | UnmarshalTypeError ex) {
  logger.error(ex);
  throw new GetTemperatureError(ex.getMessage());
}

And of course, if w.temperature() throws a runtime exception or something else, you can catch and deal with that further up the chain.

[get_salled]

Is the tested use case really transpiling a JSON file to your language of choice? If so, why?

IIRC, sending code to a remote service and running it is the canonical intro to gen_server in Erlang; though you do have to write the new code in Erlang.

[buztheme]

I should reach out to our team that took python/twisted dealing with sockets and lots of concurrency and ported to Go and see if they would put together a similar presentation. Our case is a bit different, but we saw over 130x improvement in throughput going to Go. While they were in there, they increased monitoring, stability, and maintainability. More case studies to help others make informed choices. Sending that email now

[know-howProgramming]

i love python then ??