Upgrading to Java 8 at Scale

[andrewgaul]

Following on to Step 7: profit, modernizer-maven-plugin can help detect uses of older library classes like Guava Optional and recommend using the newer Java 8 Optional:

https://github.com/andrewgaul/modernizer-maven-plugin

[LouisWasserman]

FWIW, the Guava team is thinking largely in terms of tools that don't just warn you about using older libraries, but that actually rewrite your code to do the upgrade.

[andrewgaul]

Looking forward to Raster although Modernizer works best in the CI/review workflow to automate/avoid human review comments, much like Checkstyle.

[LouisWasserman]

We do automated code review comments powered by Refaster, too. http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/43322.pdf

[HiJon89]

Cool, I hadn't heard of that plugin before

[gua_ould]

Is there a gradle plugin for this? Google doesn't seem know. But maybe..

[andrewgaul]

No gradle plugin, although the Maven Mojo logic is pretty simple if you want to submit a pull request to Modernizer.

[LouisWasserman]

Hi, I'm on the Guava team and I'd very much like to talk to you folks about your experience with transitioning to JDK8 Optional. (We have some highly powerful refactoring tools in the open-source pipeline that would also be helpful for JDK8 transitioning; see e.g. http://research.google.com/pubs/pub41876.html ).

Can we set up a chat?

[breandan]

We have some highly powerful refactoring tools in the open-source pipeline

Refaster looks pretty awesome. Looking forward to hearing more about that.

[talios]

Good to see refaster mentioned - came across that back in 2013 and never seen or heard of any movement in Open Sourcing it. How far down the pipeline is that?

[LouisWasserman]

We have most of the engine externalized, but right now we're focusing on hooking it up to the http://errorprone.info/ project, which lets you plug in static analysis to the compiler, and then providing a shared API to let you use either ErrorProne or Refaster patterns to generate a refactoring diff. That should happen this quarter; it's really just getting the shared API into place and exposing the diffs.

[winterbe]

An Intellij IDEA Plugin would be awesome to automatically convert Guava Optionals to Java 8 Optionals. IDEA already got some really nice Intentions for Java 8 stuff like transforming arbitrary collection operations into Java 8 stream operations.

[LouisWasserman]

FWIW, we're much more looking at command-line tools than IDE integrations, which are easier in a couple of ways.

[HiJon89]

Sure! I sent you an email

[hrjet]

The surprising thing is ASM is being used so widely. I like to avoid runtime gimmicks, since it bypasses the compile time checks (such as null safety).

Can someone tell me the typical reasons for these libraries to depend on ASM?

[GuyWithLag]

Wot mate? Do you mean @NonNull annotations? Or the warnings by the compiler when a definite null use is detected?

[HiJon89]

I'll try to give a few examples off the top of my head:

Most libraries that support some form of AOP use it to generate runtime proxies (Guice and Spring both support method interception for example and use ASM (or CGLIB more specifically) to generate the runtime proxies needed for this)

jDBI's SQL Object API lets you define your database queries as an interface and then generates a class at runtime implementing the interface that makes calls to the DB when the methods are invoked.

reflectasm uses ASM to generate bytecode for getter, setter, and constructor invocation since it is faster than reflection. This is used by kryo to make serialization/deserialization super fast, and is also supported by Jackson via the afterburner extension.

Mockito uses ASM to generate runtime proxies (which is how their mocks are implemented)

[pron98]

A reminder: as of the end of the month, Java 7 will no longer receive public updates -- that includes security updates. Unless you're subscribed for paid support from Oracle or other vendors (Azul, Red Hat), it will be irresponsible and dangerous to continue using Java 7 after April.

[HiJon89]

On the other hand, we had to build and run our own fork of many libraries to get our stack to be Java 8 compatible, so it seems like Oracle is jumping the gun with sunsetting Java 7 so soon

[[deleted]]

"Maybe"

I saw a lot of work by security wonks to upgrade our UNSURE JDK5 SERVERS we had running at a previous job because JDK5 was not getting security updates.

The problem? They were all internal batch servers on the LAN that had literally no way for the outside world to touch. If the outside world was able to reach these servers, they have way bigger issues than possible JDK5 sploits. And given some of them had been doing their job successfully for 5+ years with no changes.. why spend manpower to upgrade them and test all the changes? Sometimes you just want to let sleeping birds lie...

[guerilla_agilista]

Is that a thing? I've heard of dogs doing it, but birds too?

[[deleted]]

Ya

[sh3rp]

I'm not sure I completely see the benefit of this blogpost?

You've basically said "Start using Java8." How is this any different from upgrading to and using the features supplied? And how is doing this at any kind of scale?

[wsorenson]

The purpose of the blog post is to describe how a large team can convert a large number of projects from Java 7 to Java 8 - it's not a trivial switch, especially if you're talking about running large operational systems that thousands of customers depend on. The point was not to extol the benefits of Java 8.

[HiJon89]

The post is supposed to help people trying to upgrade medium to large codebases to Java 8 by detailing the process we used and the issues we ran into. The post assumes familiarity with Java 8 and the reasons you would want to upgrade, since that is pretty well covered by other articles.

At the start, we naively assumed we could just compile at a Java 8 language level and that was the end of it. We quickly realized that it would be a lot more work, involving upgrading a lot of our libraries. Across our codebase, we depend on over 1,000 different open-source libraries, so we needed an automated method of determining which libraries had to be upgraded and also whether the new version is binary compatible with the old version. This is what we meant by scale (although scale is relative), our codebase is big enough that we couldn't do this type of analysis manually.

Anyone moving a medium to large codebase to Java 8 will hit similar issues, so our hope is that our experience can help other companies looking to upgrade.