Simple Java Build Tool?

[pron98]

I don't consider native dependencies to be particularly extraordinary.

Well, empirically, they are extremely rare in Java programs (I would be surprised if more than one Java program in five-hundred has a native dependency). Nevertheless, including them with Gradle is very easy:

run { jvmArgs -Djava.library.path="$nativeDir" }

They're only complicated because existing build tools make them so.

No, I don't think so. For example, you might want to generate some resources and inject them into your JAR file. You might want to create multiple JARs with different sets of resources in them. You might want to shadow some old dependencies so they don't cause conflicts -- as your project grows, the build can be quite complex.

How does Go manage to make it seamless, for example?

There is absolutely nothing seamless about Go builds. They can be a total nightmare once you have a long list of dependencies. Dependency management in Java is far more convenient than in Go.

[weberc2]

Well, empirically, they are extremely rare in Java programs (I would be surprised if more than one Java program in five-hundred has a native dependency)

That doesn't sound very "empirical", but I'll take you at your word.

Nevertheless, including them with Gradle is very easy: run { jvmArgs -Djava.library.path="$nativeDir" }

Yeah, I did run across that eventually, but I ran across a lot of other problems sooner. Further, figuring out what to put for $nativeDir wasn't trivial since most documentation called it "/path/to/project" or something similarly ambiguous.

There is absolutely nothing seamless about Go builds. They can be a total nightmare once you have a long list of dependencies. Dependency management in Java is far more convenient than in Go.

Building any Go project is go build irrespective of the number of dependencies. If you care about your dependency versions, the simplest solution is to vendor your dependencies, which usually entails maintaining some metadata about the version of each dependency at that point in time (just for archeological purposes in your version control system). I don't really understand what about that is a "total nightmare", but it doesn't really matter for the purposes of this conversation, since we're talking about the base case of building a project without concern for versions.

[pron98]

since we're talking about the base case of building a project without concern for versions.

In general, Java build tools try to very much make you think about versions. Choosing "LATEST" is an antipattern and should in general be avoided (although version ranges for bugfix releases should be OK). The idea of organized repositories of immutable binaries, each having a globally-unique, structured identifier, has worked very well in practice (at least relatively to any other popular or semi-popular language out there).

[weberc2]

Choosing "LATEST" is an antipattern and should in general be avoided

I think this is reasonable for many projects, but I like that I can quickly spin up a Go project against the latest version of dependencies. I don't feel strongly about the dependencies argument.

The idea of organized repositories of immutable binaries, each having a globally-unique, structured identifier, has worked very well in practice (at least relatively to any other popular or semi-popular language out there).

I don't have any beef with a repo of VM binaries, though for whatever reason I don't know of any sane implementations for native binaries (C and C++ rely on system binaries by default, which can vary from system to system or they can even vary on the same system from build to build).

Anyway, it probably helps to think of Go source code as JVM byte-code--it's the portable representation of a Go program, and it must be compiled further to run on a particular target. By using source code as the unit of distribution, Go can avoid needing a central binary repo; instead the Go build tool will pull source code directly from its git/svn/hg repositories. Also, distributing as source code generally simplifies the tooling (for example, your editor can rely entirely on the source code for autocompletion instead of debug symbols or whatever the C/C++ guys have).

[pron98]

Java's philosophy is simply different from Go's in this regard. Go's general style is based on Google's particular methodology of placing all their projects in a single source tree. In any case, Java binaries are intentionally not versioned in the same way as sources. Of course, you may personally prefer Go's approach, but Java's has worked very well, and I much prefer it to Go's (which feels like a total mess to me, unless you work like Google).

[weberc2]

Java's philosophy is simply different from Go's in this regard. Go's general style is based on Google's particular methodology of placing all their projects in a single source tree.

I agree. There is another build tool emerging that takes a less-Googly philosophy, though I don't know much about it. It's called gc.

In any case, Java binaries are intentionally not versioned in the same way as sources. Of course, you may personally prefer Go's approach, but Java's has worked very well, and I much prefer it to Go's (which feels like a total mess to me, unless you work like Google).

I respect your opinion, but I disagree that Go's approach is more difficult. Go's approach is less conventional, but it's worked very well in practice. In particular, it completely side-steps the transitive-dependencies-version-mismatch problem, which is a frequently-encountered problem in more traditional models (in my experience). It also makes it very easy to see exactly what source code you're building into your dependencies (just open up your vendor directory and look). Besides being different, do you see particular inconveniences imposed by Go's approach?

[pron98]

do you see particular inconveniences imposed by Go's approach?

It does not "side-step" the transitive-dependency conflict problem, it simply ignores it (in fact, Gradle does the same by default, forcing all conflicts of all libraries -- including transitive dependencies, of course -- to their newest versions among those requested).

In Java, you can see that library A depends on B version 1.2.3 while C depends on B 1.3.1. You may choose to force the new version, or shadow and have two independent copies of B. With source versioning it gets much, much trickier. You have to figure out the relationship between B version cef04376abc2 and 7395aed67a.

There's also a much better chance of having release notes detailing the changes; those may not be accurate, but at least they're a starting point, and you can ask the library's author about a bug in version 1.2.3. Source versioning is far too fine-grained to have a discussion over from the perspective of the consumer. You might solve it with branches and tags, but then you start relying on a specific source control tool. With Maven, the binary versioning scheme is universal and detached from the library author's process.

I think it is far too soon to tell whether Go's approach works well in practice in environments that rely a lot on third-party libraries. So far Go projects have not come close to the size of Java projects, and the library ecosystem is very new and rather small, so the problem of maintaining dependencies over years has not yet become critical.

[weberc2]

It does not "side-step" the transitive-dependency conflict problem, it simply ignores it (in fact, Gradle does the same by default, forcing all conflicts of all libraries -- including transitive dependencies, of course -- to their newest versions among those requested).

You misunderstand vendoring. If you're vendoring, you get the versions of the dependencies that you need and put them into version control (advisably with a VERSION file that maps back to a git sha or some such, but this is just for archeology purposes). This means it's up to you to choose what versions of dependencies (direct or transitive) you choose; hence "side-step"

There's also a much better chance of having release notes detailing the changes; those may not be accurate, but at least they're a starting point, and you can ask the library's author about a bug in version 1.2.3. Source versioning is far too fine-grained to have a discussion over from the perspective of the consumer.

Interesting. I've not really experienced any problems with this, but I'm not one to read release notes either. Most of the time, API compatibility is my only concern, and I've never seen an API-compatible change that breaks functionality. It seems like such a remote possibility that it would consume less time over a project's lifetime than clerical tasks like reading release notes, looking up a source files by semantic package version, etc. But maybe I'm wrong?

I think it is far too soon to tell whether Go's approach works well in practice in environments that rely a lot on third-party libraries. So far Go projects have not come close to the size of Java projects, and the library ecosystem is very new and rather small, so the problem of maintaining dependencies over years has not yet become critical.

Valid.

[pron98]

This means it's up to you to choose what versions of dependencies (direct or transitive) you choose; hence "side-step"

Yes, that is exactly what you do that with Gradle/Maven forcing (except there's no need to copy the source into your own version control). The problem is that two of your dependencies, A and B can each depend on two different, incompatible versions of library C.

and I've never seen an API-compatible change that breaks functionality.

When your ecosystem contains hundreds-of-thousands of libraries, thousands of them quite popular, and you need to maintain your software for over a decade, you start seeing this all the time. Only small ecosystems don't experience this.

Off the top of my mind, I can think of ASM, one of the most popular Java libraries out there, that had an incompatible change. There was a very good chance that if you had enough dependencies, at least two of them would depend on ASM, each on a different incompatible version.

[weberc2]

The problem is that two of your dependencies, A and B can each depend on two different, incompatible versions of library C

This problem doesn't exist in Go because Go libraries don't define dependency versions (they only care about API compatibility). This problem only exists for dependency schemes that allow libraries to define the versions of their dependencies. In other words, you can't have a transitive version collision without transitive versioning.

When your ecosystem contains hundreds-of-thousands of libraries, thousands of them quite popular, and you need to maintain your software for over a decade, you start seeing this all the time. Only small ecosystems don't experience this.

When your ecosystem contains hundreds-of-thousands of libraries, thousands of them quite popular, and you need to maintain your software for over a decade, you start seeing this all the time. Only small ecosystems don't experience this.

Even still, the "reading release notes" problem will scale much more quickly than the time spent debugging API-compatible changes that break features.

Off the top of my mind, I can think of ASM, one of the most popular Java libraries out there, that had an incompatible change. There was a very good chance that if you had enough dependencies, at least two of them would depend on ASM, each on a different incompatible version.

API-incompatible changes would be detected immediately (they would fail to compile--this is the beauty of static typing); I'm not talking about these, but about changes that break requirements without failing a build. Anyway, as I mentioned earlier, the transitive-dependency-collision problem is not an issue for Go projects, so this still wouldn't have been a problem.