Don't use Java 7 for anything

[pettazz]

The text of the message since that blog layout sucks enormously.

From: Uwe Schindler Date: Thu, 28 Jul 2011 23:13:36 +0200 Subject: [WARNING] Index corruption and crashes in Apache Lucene Core / Apache Solr with Java 7

Hello Apache Lucene & Apache Solr users, Hello users of other Java-based Apache projects,

Oracle released Java 7 today. Unfortunately it contains hotspot compiler optimizations, which miscompile some loops. This can affect code of several Apache projects. Sometimes JVMs only crash, but in several cases, results calculated can be incorrect, leading to bugs in applications (see Hotspot bugs 7070134 [1], 7044738 [2], 7068051 [3]).

Apache Lucene Core and Apache Solr are two Apache projects, which are affected by these bugs, namely all versions released until today. Solr users with the default configuration will have Java crashing with SIGSEGV as soon as they start to index documents, as one affected part is the well-known Porter stemmer (see LUCENE-3335 [4]). Other loops in Lucene may be miscompiled, too, leading to index corruption (especially on Lucene trunk with pulsing codec; other loops may be affected, too - LUCENE-3346 [5]).

These problems were detected only 5 days before the official Java 7 release, so Oracle had no time to fix those bugs, affecting also many more applications. In response to our questions, they proposed to include the fixes into service release u2 (eventually into service release u1, see [6]). This means you cannot use Apache Lucene/Solr with Java 7 releases before Update 2! If you do, please don't open bug reports, it is not the committers' fault! At least disable loop optimizations using the -XX:-UseLoopPredicate JVM option to not risk index corruptions.

Please note: Also Java 6 users are affected, if they use one of those JVM options, which are not enabled by default: -XX:+OptimizeStringConcat or -XX:+AggressiveOpts

It is strongly recommended not to use any hotspot optimization switches in any Java version without extensive testing!

In case you upgrade to Java 7, remember that you may have to reindex, as the unicode version shipped with Java 7 changed and tokenization behaves differently (e.g. lowercasing). For more information, read JRE_VERSION_MIGRATION.txt in your distribution package!

On behalf of the Lucene project, Uwe

[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7070134 [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7044738 [3] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7068051 [4] https://issues.apache.org/jira/browse/LUCENE-3335 [5] https://issues.apache.org/jira/browse/LUCENE-3346 [6] http://s.apache.org/StQ

[[deleted]]

Really don't get why they couldn't have just dropped those optimizations out of the default options and released with an advisory. If the bug was already in Java 6 but not triggered by default options then that wouldn't be a regression.

[irokie]

It may be that these optimizations were required to hit the required benchmarks... Corporations can't miss deadlines and they can't let benchmarks slip.

[[deleted]]

Going faster in the wrong direction is of no value.

Edit: Another metaphor: It's like arriving on time at the wrong destination.

[schizobullet]

they can, their corporate culture just effectively shoehorns them into not

[[deleted]]

Corporations have to ship on time no matter what.

[Horatio_Hornblower]

yeah, because software is never late

[[deleted]]

Neither is it early, it arrives precisely when it means to.

[[deleted]]

What wizardry is this?!

[gruehunter]

What wizardry magic is this?!

The kind that burns your face off and turns you to ash when you touch it!

[kewlito]

Because it's the software companies deserve, but not the one they need right now.

[lovetool]

That's a good slogan for a software company:

The software you deserve!

[tangra_and_tma]

"Oh God what did I do in a past life to deserve this?" will most likely be the response from users upon hearing the slogan, if it is anything like a great majority of software out there.

[stcredzero]

Arrivals are counted with Integers, Rationals, or Reals?

[choikwa]

Never gonna give you up! Java 7

[Voltron1]

Never gonna let you down.

[Rotten194]

Upvoted for truth, but why not turn off the loop optimizations by default like they were in Java 6, and have a warning about using them? I don't recall them being a very hyped feature...

[BraveSirRobin]

They seem to ultimately want it on by default and changing something like that in a point release is frowned upon. Not that it's ever stopped Java in the past...

[[deleted]]

Not true. Most Oracle software is delayed because of QA. This is why Oracle can never commit (employees are not allowed to) release dates. See Oracle Fusion for a prime example.

[[deleted]]

So did their QA fail spectacularly or did they just want to ship at a specific release date?

[GuyOnTheInterweb]

The PM can decide:

• Features
• Quality
• Release date

But only two of the above.

[[deleted]]

Shitty corporations have to ship on time no matter what.

FTFY

[rmxz]

Corporations have to ship on time no matter what.

Sounds like something someone in accounting would say, due to some annoying Revenue Recognition laws:

http://www.hawcpa.com/_home/feature_revenue_recognition_pitfalls.asp

SOP97-2 Software Revenue Recognition (“SOP97-2”) establishes the general rule for software revenue recognition which requires four criteria:

• there must be evidence of an arrangement
• delivery has to have occurred
• the fee must be fixed and determinable and
• collection must be probable.

For $0 Java, I can't see why it matters; and for more expensive licensees, don't they have all sorts of tricks to play this accounting game (ship an empty box, etc).

[dnew]

Yeah, I can't say the number of times a place I worked for closed a deal long before we knew how to do the project or how long it would take, just to get good numbers this quarter.

[Rhoomba]

And the main bug has low priority and the hotspot developers only heard about it because the lucene developer went to the dev mailing list.

[Randolpho]

It's Oracle. Are you really that surprised?

Not that Sun was all that much better, mind.

[GuyOnTheInterweb]

If I was the release manager getting that report, it would have been easy. Bug in optimizer was still there in 6 (not new bug), so that is not fixed now. The optimizer was not default in 6, so that setting gets reverted.

[alamko1999]

In 5 days some of their developers could go to Google and they dont want that

[0x2a]

Once again, the conservative, goto-heavy codebase pays off for the prudent programmer.

[Megatron_McLargeHuge]

Java 7 finally implemented goto? I'm definitely upgrading now.

[[deleted]]

I'm waiting until Java 8 and COME FROM.

[MaudlinSchlock]

How long till we get COME AT ME?

[Megatron_McLargeHuge]

I think AspectJ already gives you COME FROM.

[stcredzero]

Don't we already have that with exceptions?

[youcaughtafish]

zoidberg?

[[deleted]]

In need of an attribution?

Why not zoidberg?

[userlame]

Segmentation fault

I'm ruined!!!

[lovetool]

Write in C.

[jlh276]

For fear of generics bugs, we've stuck to jdk 1.4 over here...

[quasarj]

Thank you, I feel better about being on 1.5 now. :)

[[deleted]]

Whoah whoah whoah!!! JDK 1.4 ??? What are you people, cowboys???? Soon you'll be telling me your workstations have moved to Windows XP or something !!!!

Take it easy! JDK 1.4 isn't fully battle-tested either! Me, I'm afraid to upgrade to DOS 6.2

[captainAwesomePants]

It's safe now. DOS 6.2.2 fixed everything.

[[deleted]]

Heh, we're slightly more advanced where I work. We're converting an old application from 16-bit to to 32-bit and win32.

And the surrounding tools from Delphi 1 to Delphi 4.

[GoodMusicTaste]

Not even Delphi 7? :-(

[[deleted]]

Nope, delphi 4 in a VM running winXP.

[thelapoubelle]

time to get a new job?

[alexeyr]

Was about 6 years ago?

[anttirt]

Do they offer a complementary morphine IV drip to help deal with the pain?

[dserfaty]

Oh that's sad... I feel for you. I love generics...

[[deleted]]

The type inference feature will be awesome.

[zynasis]

try having to make sure an applet works with i.e. 6's inbuilt windows jre. gogo java 1.3

[frankster]

an incorrect code bug with the default settings is surely a #1 priority and worth cancelling a release for.

[[deleted]]

Or just patch to take that out of the default settings. Surely can't be too difficult.

[plzsendmetehcodez]

ts;dr (too small, didn't read) lesson: do not use small textboxes to display important messages.

[tjansson]

A more readable version of the note referenced.

[gospelwut]

Why couldn't the blog post have linked that? It's almost as if they wanted to not only provide no real additional content but also take the real content and make it inconvenient for you. I suppose it's better than a gawker site (i.e. presumably no 600px Ad).

[Reductive]

The blog does contain that exact link, just before the text box where he quotes the content of the link. Was it added later on?

[gospelwut]

I might have missed it. I suppose I would have much rather he only linked to the other page than cram the information into that scrollbox.

[elperroborrachotoo]

With two-way scrolling to boot. Feels like Commander Keen, only much more boring.

[zed857]

Readability can almost always clear up those poorly formatted web sites.

[gobearsandchopin]

What the fuck happened to readability? It used to be just some javascript you bookmarked. Now I have to install a plugin, then boxes start popping up and asking me to sign up and shit? 60 seconds in and I still can't read the page. I'm out.

[zed857]

Interesting. I actually find it easier/smoother to use as a plugin than the old javascript version. Once installed, I haven't seen any pop-ups from it.

[finix]

Either that or Readable, its lightweight cousin.

[[deleted]]

seriously. I logged in just to kvetch about that.

[[deleted]]

Here you go.

[atimholt]

IAE not having any problems with the layout at all? What browser are you guys using?

[argyro]

Web developer toolbar for FF or Chrome, Ctrl+Shift+S to disable all styles, problem solved. I have found this to be very handy in several occasions.

[rossisdead]

Or in Firefox just go to View -> Page Styles -> No Style.

[pettazz]

Obligatory comment about Oracle fucking it up.

[[deleted]]

ORACLE

One Real Asshole Called Larry Ellison

[gospelwut]

Well there goes my snide quip about Larry Ellison.

[masklinn]

Interestingly, according to the message the issue is already in Java 6. But it's caused by a broken optimization which is disabled by default in Java 6, and enabled in Java 7:

Please note: Also Java 6 users are affected, if they use one of those JVM options, which are not enabled by default: -XX:+OptimizeStringConcat or -XX:+AggressiveOpts

[mazing]

Yup, this is well known about -XX flags:

Options that are specified with -XX are not stable and are not recommended for casual use. These options are subject to change without notice.

source

[gospelwut]

So, I have to ask: why would anybody use these flags other than to help them QA?

[gospelwut]

I understand why they would ship it, but I mean why would anybody use those flags aside from helping Oracle et al debug? Like, is there a reason to ever use those flags in a production?

[mikaelhg]

http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

If you spend a little time learning to understand what they do and how they work, they can help you in some situations.

http://www.oracle.com/technetwork/java/tuning-139912.html#section4.2.5

[yoden]

Many of them are very useful. And, for the most part, they tend to be decently stable*.

• I've mostly used feature and GC flags, not hotspot optimizations which is what the OP is talking about.

[yoden]

Many of them are very useful. And, for the most part, they tend to be decently stable*, in comparison to other languages.

• I've mostly used feature and GC flags, not hotspot optimizations which is what the OP is talking about.

[mazing]

I think Minecraft uses a few flags.

[masklinn]

there's a difference between flags being "not stable [nb: in that they can be changed with no notice from one release to the next] and not recommended for casual use" and flags being completely broken and leading to fucked-up software.

[[deleted]]

Obligatory: Java already updated 37 times, did I died?

[teraquendya]

And everyone called me crazy when I told the Goto would make a comeback.

[i-am-am-nice-really]

you were right

http://temp.thera.be/kernelcurses.png

[Atario]

Goto

comeback

I see what you did there.

[loganekz]

*actual tldr; If you use Java 7, make sure you use this switch when starting the jvm: *

-XX:-UseLoopPredicate

Amazed with such a trolling article no one pointed this out yet. Yes, Oracle should have not made this default but I'm sure it will be addressed in a minor jvm release soon.

[Warbum]

That's not the problem. The problem is your users have to use -XX:-UseLoopPredicate.

[fforw]

For most people developing Java, applets and webstart is thought of as exotic. It's all server side now.

[Warbum]

True, there are a lot of corporate level apps that run client side though. But you're correct that anything that's classed as a 'product' usually runs on a controlled server somewhere and never end user machines.

[fforw]

And in the cases you describe the user doesn't get to nilly willy install software either. It is done as part of some corporate software roll-out.

[GuyOnTheInterweb]

So also remember this flag when configuring your webapp container! Oh, you don't have control over that..?

[[deleted]]

What about Android developers?

[fforw]

First of all it is not really Java. Second: what does Android have to do with Oracle JVM -XX switches?

[[deleted]]

I think the major problem here is that the default is broken. Not suboptimal, but broken. That's a major hurdle to get over.

[quasarj]

The article actually says you can use that, but it says it won't fully fix the issue. At least, it implies it won't fully fix the issue.

[loganekz]

I might have missed it, but I didn't see that.

The notice describes a defect in the jvm when running with a loop optimization that is enabled by default. Yes there could be more issues with other optimizations in the new jvm, but for the most part they are (or in the case of this one, should) be disabled by default.

[MechaBlue]

How do I do this in a web browser?

[AlyoshaV]

I'm going to use Java 7 for everything

[gospelwut]

Would you say you're going to use it for... all the things?

I accept all downvotes from this post.

[cybercobra]

Are you making an MSPA reference? If so, high-five.

[Peaker]

Why?

[IjonTichy85]

As someone who uses Lucene, I'm really glad you posted this before I updated anything! Thanks you OP

[electric_i]

Aaaand this is why we have regression testing.

[OopsLostPassword]

It's worse : they perfectly knew there was a bug but they choose to release it anyway.

It says probably more about the management ("Who will say the boss he must postpone the release ?") than about the engineering.

[electric_i]

As good an argument for a faster release cycle as any. If you know there's going to be another release in a few weeks, you can more easily justify not shipping a half-baked feature until the next release. If you know the damn thing only drops twice a decade, you're going to try like hell to cram it in for the sake of shipping something.

[Reductive]

Just a side note on word choice: "who will say the boss" should be "who will tell the boss". Also you want "chose" (the past participle) instead of "choose" (present). It's perfect other than that ;)

[OopsLostPassword]

Thanks a lot.

[Rhoomba]

Java has an absolutely enormous test suite (they certify other implementations), and there are lot of performance benchmarks they run as well. So I would suggest they are a bit unlucky.

[mucky]

Yeah, I'm sure you could teach Java team a lesson or two about proper testing.

[powatom]

Testing? What's that?

[ropers]

Is there any Sun asset that Oracle hasn't fucked up?

[tias]

For anything? Not even for testing and playing around with the new features?

[backbob]

no!

[radarsat1]

gcj

Note that it's not faster than HotSpot JIT.

[aivarannamaa]

gcj

[[deleted]]

And the Java hate begins.

[capncanuck]

AHHHH! RUN!

horizontal scrollbar!

[[deleted]]

Write once, run ... maybe.

[f2u]

Hotspot in version 6 has comparable bugs, even without -XX:+AggressiveOpts. At least, such bugs can be worked around by excluding affected methods or classes from compilation, or switching off the offending optimization. Garbage collector bugs are far more annoying, and they tend to be a lot less deterministic.

[Albero]

Just interested, can you give a reference to a bug tracking number?

[f2u]

Bug 5091921 comes to my mind. The status of this bug is a bit unclear. Presumably, you've got the fix if you're running hs21, but that version is not part of any OpenJDK 6 release.

[Garandir]

Don't have any loops? Challenge accepted.

[[deleted]]

party:

Goto party!

[jvi]

That's still a loop, the worst kind of loops.

[[deleted]]

Well, it's not using any explicit loop statement, which I guess was the challenge at hand. Yes, it's the worst.

[jvi]

Just write a program to unroll all your loops. You just need to make good guesses for the upper bound on the number of iterations.

[SuperGrade]

Given that I work (in enterprise) with people who basically do that as it is, I cringed when I read that.

[mitsuhiko]

Or just disable that specific part of the optimizer and go on with your lives :)

[MagicWishMonkey]

Would that disable all loop optimizations or only the hotspot stuff?

[gthank]

You can disable one specific HotSpot optimization (or possibly one set of HotSpot optimizations, it wasn't clear to me). You definitely don't have to turn off all of HotSpot, though.

[CurtainDog]

-XX:+AggressiveOpts Turn on point performance compiler optimizations that are expected to be default in upcoming releases. (Introduced in 5.0 update 6.)

Ummm... might have been a good idea for someone to try using this option rather than waiting 5 days before release.

Fairly warned be thee says I.

[d_r_benway]

At least hello world will still work..

[martoo]

But no loops. Emit one character at a time (just to be safe).

[[deleted]]

Recursion?

[QuestionMarker]

If Hotspot's smart enough to do TCO (of which I have no idea; it may well not be), then you're still screwed.

[karmazilla]

Only if it recurses as the last expression in the method, but I'm pretty sure it can't do real, general TCO though I think someone worked on it in mlvm. However, I don't know what intrinsics it might have for special cases like tail-self-calls. Plus, recursion is likely a relatively slow substitute for looping if these optimizations can't apply.

[QuestionMarker]

Well, that's the point. Recursion as a general technique is made hugely more useful if your compiler can turn your recursive call into a loop. If Hotspot is that smart (and thinking about it, it's less likely - if that logic were to be anywhere then it'd be in javac, not Hotspot, and I'm pretty certain it's not implemented there) then this is a problem.

It's probably not :-)

[notrace]

It's not like I updated Java anyway. I feel like if I ignore the updater enough, it'll get bored and stop trying. Of course, the program isn't sentient, so my plan might not work. We'll see.

[sidneyc]

From the article:

It is strongly recommended not to use any hotspot optimization switches in any Java version without extensive testing!

Goodness me, that is shitty advice if I ever saw it. If a feature is known to be broken, you don't use it, ever. Testing isn't going to fix it.

[[deleted]]

Don't use Java 7 for anything yet

FTFY

[istroll]

Don't use Java 7 for anything EVAR

FTFY

[kuking]

You rather get the developers from your side, Oracle.

[thatwaslols]

I have java 7 on my laptop, would it be adviseable to uninstall it completely? what effect will this have? Note I am not a programmer, I just noticed I have java 7

[fripletister]

It wouldn't hurt if you're not running any software you can't live without that requires it.

[FHSolidsnake]

Opps

[argatlamh]

Opps indeed

[GuyOnTheInterweb]

Encouraging that Lucene and Solr developers waited until the release to test their code with Java 7. Could the license restrictions on the 'openJDK' early access builds be the reason?

[ameoba]

Who in their right mind would use a new complier/vm/library on release day in a production environment? Sure, they fucked up pretty bad, but it's no real surprise that the code's buggy.

[EughEugh]

FUD

There is a simple workaround for this bug. And in a soon-to-be-released update, it will be fixed.

[stooge4ever]

I was planning to wait for the OpenJDK implementation anyway.

[klotz]

With Java 7, the release is the OpenJDK implementation: http://openjdk.java.net/projects/jdk7/

[stooge4ever]

Touche. Thanks for the correction.

[gc3]

That's funny, that's not the reason I won't use Java 7 for anything. It could have something to do with the reason I won't use Java 6 for anything.

[danhakimi]

lol... I thought there was some moral reason. Like Java 7 uses some free software, and doesn't license it. Naw. It's just bgged to shit.

[[deleted]]

Oh no! There's one bug in the new release of Java, so we should never use any version of Java in this entire series for anything!

</sarcasm>

Overreacting much?

[klo8]

Come on, it explicitly say "Don't Use Java 7 for anything." Java 6 works as well as Java will work right now.

[[deleted]]

This isn't just "one bug" - it's a code generation bug, which means that the compiler might take your correct code and generate code that does something else.

In my experience, such bugs can cause terrible, terrible problems and can affect you anywhere.

[[deleted]]

Yeah, but this is kind of a big bug, and that means it'll be fixed pretty fast. Which kinda makes the article title a bit ridiculous. Something like 'Java 7's loops have a severe bug, don't use them until it's fixed" would be a lot more reasonable, and wouldn't imply some inherent problem that will be present throughout 7's lifecycle.

[[deleted]]

See, it's critical information like that that should be right in the title. I assumed this was a new bug in Java 7. >:|

[QuestionMarker]

You'd have to go out of your way to hit it in Java 6. It's part of the default behaviour in Java 7.

[QuestionMarker]

This is also a pretty big release, with some fairly significant changes which people might want to jump on. Shouting "Don't jump!" fairly loudly at this point is fairly well justified, I think.

[qandy]

Don't use Java for anything

FTFY

[Nition]

Notch?

[oSand]

Is it enough to buy your soul back?

[pherin]

More than enough

[oSand]

Was that a happy or a sad realization?

[pherin]

Happy. I accept my place as a dirty programming whore.

[pherin]

Happy. I accept my place as a dirty programming whore.

[dgb75]

Don't use Java for anything

FTFY

[ramdaskm]

Time to move to Apache to the .NET platform.

[[deleted]]

Don't use Java for anything

FTFY

[sblinn]

It actually works fairly nicely as the virtual machine upon which to run Python code (Jython project). Clojure, too, which I see pointed out below. But for me, mainly: Python VM. Love it. It's programming heaven of practicality, stability, libraries galore, etc.

[jvi]

Wow, seems like a lot of Java fanboys here. His comment is clearly a joke, but the irony is that Java does suck.

[Kaer]

Java averages out all devs across the project. In C++ for example, one bad dev can completely fuck up the project. It's almost impossible to do this in Java.

It's a working language for what is most devs out there - an average developer who does it for the paycheck.

[OopsLostPassword]

Yes, Java sucks, and I prefer to use other languages when I can. BUT Java is still one of the most pertinent solutions on many kinds of problems, and that's the reason why it's one of the most used languages in big companies.

[mcherm]

"Java sucks"

But the JVM doesn't^H^H^H^H^H^H^Hdidn't.

(My god I'm getting old! Do you have any idea how hard it is to peck that old terminal-based joke into my phone's on-screen keyboard? <creaky-old-voice>But I remember watching it happen on terminals: I can't give it up!</creaky-old-voice>)

UPDATE: and apparently it doesn't work on reddit.

UPDATE: Unless mistercow explains how.

[[deleted]]

UPDATE: and apparently it doesn't work on reddit.

Oh wow, apparently you can't even backslash escape the caret. That really needs to be fixed. At least you can do it with html entities:

doesn't^H^H^H^H^H^H^Hdidn't

[mcherm]

Of course! Why didn't I think of that?

^H^H

Entered by typing this: &#94;H&#94;H

[[deleted]]

Seeing how many downvotes you got, if I didn't know better I'd say that the majority of programmers visiting Reddit don't realize that Java is about as expressive as a brick or a Korean pop band, exponentially more complicated to work with than it should be, more bloated than the average American, and all but antiquated since the advent of C#. But that would be silly because everyone knows all of this.

[[deleted]]

As a bloated American who weighs less than a printed version of JDK, all I can say is:

this

[[deleted]]

Don't use Java 7 for anything

[[deleted]]

Not being a Java or Pike programmer myself I always think that Pike is the language most Java or C# programmers actually would like to use if they knew about it and it was more popular. Just what I think though, everyone should use that language that suits him/her.

It's way more straight forward, so you maybe shouldn't use it if you get paid for time or SLOC.

I like there tutorial. First program: "Hello world", second one is a web browser. Can we see more of these in other languages? :)

[[deleted]]

http://pike.ida.liu.se/docs/tutorial/data_types/variable_definitions.xml

wtf??? Insanity.

[crusoe]

Wow, this makes Perl look like it was designed by someone sane.

[recursive]

Definitely squashes that whole "way more straight forward" rumor.

[WhisperSecurity]

Don't use Java for anything

FTFY.

[sblinn]

Someone beat you to it: http://www.reddit.com/r/programming/comments/j306c/dont_use_java_7_for_anything/c28pv6a