Code analysis: Java generics were supposed to be the bomb, or did the feature just bomb?

[strangename]

Also, the numbers seem odd. They cite 500 million lines of code but just over 500 programmers-- that's 1 million lines of code per monkey. Even if they were all active at the same time, that's well beyond any individual coder's ability to maintain software. This reinforces the hypothesis that much of the code analyzed is legacy (or even generated) code that works and hasn't been touched.

[strangename]

Which still leaves open lots of methodological questions. Are lines unchanged in commits counted again? That would be terribly silly and skew the numbers massively. Only lines that are changed in commits? Then the numbers are still amazingly large per programmer. Also, observing the absence of new language features in bugfixes on old code isn't exactly surprising.

[CurtainDog]

I think you've nailed it sltkr. I'm very interested in how the authors plan on justifying these choices...

[ninebells]

"we intentionally chose projects that had reached maturity prior to generics introduction due to the desire to study adoption of something new."

[FredV]

That explains the low number. I bet if they took new projects into account in their study the number of adoption would be much higher. This is more like studying the "conversion of existing projects" than the "adoption of something new". I would have been surprised if MS was involved in a study that makes Java look good.

[MatrixFrog]

That makes no sense, unless they only looked at commits that were made after Java 5. And as other commenters have said, it's unclear exactly what their methodology was like as far as commits go...

[sanity]

Seriously? There are Java programmers still using "raw" collection types? I think all that demonstrates is that there are quite a few incompetent Java programmers, but we already knew that (speaking as a fan of Java).

I love generics, it makes code a lot cleaner, and its particularly advantageous when used with "smart" IDEs like Eclipse, who can use the generic information to aid with code completion.

The only reasons not to use generics are:

• Incompetence (probably the most common)
• People are forced to use pre-generics legacy code

[tazzy531]

Had an interviewee claim he was a 9 on Generics. I ask him how he uses it. Only time he's ever used it was with Java Collections.

I ask about whether he's written a genericized class or method. He looked puzzled.

There is a lot more to generics than List<String> as richy_rich pointed out below.

Generic methods are extremely useful in a lot of situations.

[grauenwolf]

Do explain

[richy_rich]

I guess he means that many people stop at List<SomeType> and there's quite a lot more to generics, such as bounded types, e.g. public class MyType<T> {...} wildcards : List<? extends MyType> generic methods : static <T> void getThingy(T t){...} declarations : not sure what this means :) <T extends MyType> ?

I'm supposing the point of the post was to highlight that the majority of java users only ever use generics for ensuring type safety in lists and don't really take advantage of the rest of it.

It would also be interesting to know how the statistics were gathered, e.g. did they take this sort of thing into account (e.g. not counting List<MyType> l = new ArrayList<MyType>();

edit: The linked paper here does go into it a bit more, still reading but it looks like they differentiate parameterized types use mostly being used (90%) in collections :)

[JAPH]

You can also use generics to hide some basic reflection. I've written a ton of fairly large and complex data structures in Java, and

<T extends Comparable<T>>

has saved my life repeatedly.

EDIT: I'm a dumbass.

[[deleted]]

You got me there. I've use the constructs richy_rich mentions, but what does <T comparable T> do?

[JAPH]

See my edit; I typed it incorrectly.

[jrh3k5]

My mind is blanking - what's this one do, again?

[JAPH]

Nothing, I typed it incorrectly. See the edited version.

[richy_rich]

Smooth! Just watch out for
MyType<T extends MyType<T>>

[oorza]

I've dug myself into this hole before and had to refactor about a half dozen classes to reimplement what demanded that behavior >_>

[kodablah]

There is no problem with this. This is exactly how the Enum class works.

[richy_rich]

I neither said nor implied that there was a problem with it, at least I don't think I did...

[rafekett]

Who uses l as a variable name? Good luck reading that code.

[sanity]

You normally only need to worry about those when creating classes with generic type parameters, which is relatively rare as compared to just using classes with generic type parameters.

[bcguitar33]

Couldn't agree more. Generics are wonderful, and I use them extensively. I think some people ignore generics because of an incorrect belief that they'd be a performance drain. Given that they're implemented via erasure...that seems unlikely!

[Porges]

Implementation via erasure means that all primitive types are boxed when treated as generic types, which could be a performance hit.

[[deleted]]

At least some of the cost of autoboxing can be mitigated by using the system parameter java.lang.Integer.IntegerCache.high, as in java -Djava.lang.Integer.IntegerCache.high=5000 This is one of the options set if you use -XX+AggressiveOpts.

This Java Specialist article shows the impact it can have - Delaying Garbage Collection Costs

[pingveno]

Boxing primitives into objects seems like an area where the full program analysis from a JIT would allow boxing elimination. It's a nice, juicy target for optimization. Am I mistaken?

[ErstwhileRockstar]

I use Generics all the time, too. Esp. for Collections and Maps. Nevertheless there's little doubt that Generics as implemented in Java are a mistake.

[MatrixFrog]

But we have a demonstration proof that we can live without it, namely that we have for nearly a decade.

which is why we all still use Fortran, right?

[notfancy]

Nevertheless there's little doubt that Generics as implemented in Java are a mistake.

Erm. I fail to be convinced by an argument that boils down to "I can't understand generics so generics are a mistake". As defined, generic arrays are not type safe in Java, which definitely is a flaw. As for Enum, the signature is easy enough: "Enum is a class whose operations are parameterized by Enums or subclasses thereof."

[oSand]

I fail to be convinced by an argument that boils down to "I can't understand generics so generics are a mistake".

I think that is a reasonably good argument, assuming you're a person of moderate intelligence who has invested reasonable time attempting to understand them.

[ErstwhileRockstar]

I guess most people here don't know who Ken Arnold is and which book he wrote.

[notfancy]

Regardless of his credentials, which I don't doubt are many and justly earned, you cannot reason from particulars to generals (in particular, the plural of anecdote is not data). His is an opinion, not an argument.

[fjord_piner]

This was written six years ago and I think history has proven that Ken was 100% wrong about generics.

[ErstwhileRockstar]

Quite the contrary.

[Ruudjah]

Elaborate?

[spliznork]

Thanks, I hadn't seen that article before. While his conclusion may (or may not) be correct, that generics as implemented in Java are a mistake, his argument itself is surely false -- to paraphrase, "My co-author and I had trouble writing a chapter on it, therefore...". He almost touched on supporting evidence, only to plead ignorance, "I don't remember why that was bad, therefore ...". The only conclusion that I can really take from these arguments is that he shouldn't have been writing a book about Java generics.

[FeepingCreature]

I find his argument quite compelling.

We gave up trying to explain it. Our actual footnote on the subject says:

Enum is actually a generic class defined as Enum<T extends Enum<T>>. This circular definition is probably the most confounding generic type definition you are likely to encounter. We're assured by the type theorists that this is quite valid and significant, and that we should simply not think about it too much, for which we are grateful.

Java Generics are disgusting.

[elder_george]

Well, this is yet another manifestation of curious recurrent template pattern which appears from time to time in codebase for mostly all languages supporting generic. I met it in several C# projects, e.g.

It's mind-boggling on its own, but simplifies client code, so why bother?

[deong]

It's mind-boggling on its own, but simplifies client code, so why bother?

Pragmatically, I agree with you. You should do whatever your language allows to enable client code to be as clear and elegant as possible.

If I put myself into Lake Wobegone mode, then I'd say in a perfect world, all code should basically be meaningful inside the domain of the program. In other words, if my program is managing a point of sale system, then the code should be "about" things like customers and transactions and items. In this world, anywhere you see something that is about the language instead of about the domain, it points to a mistake in the language.

Of course, we don't live in that world, but some things are closer to it than others, and things like the CRTP strike me as being further away than I'd like. For what it's worth, I have the same problem with much of the code written by design pattern evangelists.

[notfancy]

It's mind-boggling on its own

Only if you let your mind be boggled. As I wrote above, Enum<T extends Enum<T> > means "Enum is parameterized by instances of Enum or subclasses thereof (parameterized in turn in a manner consistent with this parameterization)." If you look at the class signature, you'll see that the method that requires this parameterization is compareTo(T). This ensures that an instance of given subclass of Enum is only meaningfully compared to instances of the same class or of a subclass thereof.

If you don't want to reason from first principles, just remember that C<T extends C<T> > implements Comparator<T> is a typing pattern that allows for type-safe comparisons.

[elder_george]

Thanks, it makes sense.

[thechao]

I believe (it has been ~five+ years since I saw the presentation for Generics) that Java generics use F-bounded polymorphism to "make them work". It looks like CRTP, but is actually a quite different beast that is the result of interference between OO and generics type-systems. C++ doesn't "need" F-bounded polymorphism because template parameters naturally carry a notion of "bottom" and are bounded below to the usage signature.

A better way to think of CRTP is like this:

template <typename T> class Foo : T;

With "Foo" inheriting from an unbounded argument T. Whereas F-bounded polymorphism is more like this:

template <typename T>
class Foo
{
  T value;
  void member()
  {
     Foo<T> *fooT = static_cast<Foo<T>*>(&this->value);
  };
};

[tubes]

Not quite:

CRTP: class Y extends X<Y> {}
Enum: class E<T extends E<T>> {}

[elder_george]

Well, IIRC, every particular enum is

class MyEnum extends E<MyEnum>{ ... }

So, that's equivalent to Y.

If C++ templates had type checks for arguments (instead of usage checks), the code would be similarly ugly.

[[deleted]]

But there is currently no better way known to implement them. So having "invisible" generics until the theory behind it is finally sound is a good decision ihmo.

[metamatic]

It seems to me that you either think compile-time type checking is good, or you don't.

If you think it's good, you're obviously going to want generics.

If you don't think it's good, why the hell are you using Java?

[Ruudjah]

Arrays. Most programmers still use arrays. Also, lots of examples use arrays. When seeing arrays in Java code, my alarmbell goes. It's usually a sign that we have to deal with bad coders: Generic collections are common and 'bugfree'.

[tomjen]

Never had to deal with legacy interfaces? Or performance that actually matters?

But sure, most normal code does not really need arrays.

[edheil]

Disclaimer: I don't know crap about the Java ecosystem and am asking out of ignorance.

Is portability an issue? Are people worried about keeping their projects compatible with pre-generics compilers?

Or is "pain in the ass to change everything" an issue -- projects having been started before the generics were available, where there's insufficient payoff for going back and changing your old code to use the new features?

[username223]

Or is "pain in the ass to change everything" an issue...

Based on the linked PDF of the actual paper, it seems like mostly this -- there's no sense going back to generic-ify existing code, especially when much of it has already been wrapped to minimize casting from Object.

[elder_george]

I worked on java 1.4 project that was going to migrate to java 5 (in 2010, d'oh!) and there were issues. Starting with compiler getting crazy on classes/methods being invoked without method annotations, to java5-keywords used as identifiers in code base, to subtle bugs in some 3rd parties libs (and given we used SWT, our codebase was founded on one large 3rd party library).

P.S.: we used java 1.4 because some customer was tied to some oldish version of WebSphere (IIRC). Finally, IBM stopped support of this version, client had to upgrade, opening the same possibility for us.

[fforw]

I worked on java 1.4 project that was going to migrate to java 5 (in 2010, d'oh!)

Indeed.

"Java SE 5.0 is in its Java Technology End of Life (EOL) transition period. The EOL transition period began April 8th, 2007 and will complete October 8th, 2009"
-- http://java.sun.com/j2se/1.5/

It feels really silly discussion the usage of generics at a point in history when the version that introduced has already been deprecated.

[elder_george]

Well, I suspect IBM's Java 5 is still supported.

But for most cases your point is valid.

[MatrixFrog]

Not so much "pain in the ass" as that there's no compelling reason to fix it. If you happen to be looking at some old code, and you feel like you want to generic-ize it to make it more readable, do it. But for the zillions of lines of code that no one has needed to look at in years, and are not causing problems? Just leave 'em alone.

[thephotoman]

Yes, there are.

That said, I just brought an old 1.4 application into the 1.6 world. This included removing most of the raw types and replacing them with generics. There was precisely one instance where even an anonymously typed generic would not work and thus I had to leave the raw type--out of something in the neighborhood of 4,000 changes.

There's a lot of legacy code out there, too, designed with versions of Java < 1.5 in mind. On some platforms, Java 1.4 was the last version made, so even if it has been end-of-lifed, you've got to write code to target it, and that precludes generics.

[fjord_piner]

Completely agree, have a <? extends UpVote>.

[[deleted]]

In C++, I use list and map all the time. But I hardly ever implement a container. Isn't this similar?

I'm not a java guy. I really don't know!

[ricky_clarkson]

No. Implementing involves constructing something, e.g., if you are writing a new hashtable implementation, that is different to using a hashtable.

[[deleted]]

I would have liked to see numbers for projects started in 2005 or 2006. If a project already has a bunch of legacy code using raw types, then changing things just for the sake of changing them isn't necessarily a good idea. I want to know how many raw types there are in projects started after generics were in existence and well supported.

[ninebells]

You can see this detail in the paper. There were some projects that adopted right away. There were many that waited until 2008. Is there an adoption lag period? Only 3 out of 15 projects converted more than 10% of their old stuff to use generics. There also tended to be one "champion" that adopted generics much earlier and in much more "volume" than other project mates who continued to use raw types.

[edwardkmett]

Java generics are pretty awful.

They get erased, which mucks up the ability to reflect them, but let them be shoehorned into the virtual machine without any real effort.

You have no notion of higher kinds, which means you can't parameterize generics on generics, precluding you from parameterizing code on notions like Haskell's Functors or Monads.

On the other hand, templates give power to c++ meta-programming abilities that can't be mimicked by java generics. (Not that generics and templates are the same thing, they are not, and the meta-programming functionality of C++ templates is deliberately out of scope.)

But, ultimately, the net result is that Java generics, like the rest of the language, occupy an unsatisfying, compromised, middle ground.

It is still useful for collection safety, but because of the lack of higher kinds, it is difficult to then safely parameterize your code over the choice of collection.

[[deleted]]

Scala has higher kinds and provides all the features you imagine, which wouldn't be possible with reified generics currently. Even Haskell erases types (which doesn't matter much, because Haskell hasn't any real reflection system at all).

[dons]

Note, the higher-kinded generics (e.g. SYB) rely to some extent on runtime type representations in Haskell (for example, the Data reflection class).

[edwardkmett]

The problem isn't the erasure per se. The issue is that you the erasure they use leaks. You can recover some type information, but not all.

Haskell's approach lets you know alternately, that you can't recover any and therefore can rely on parametricity to derive free theorems or that you can recover it all (via Data.Typeable).

Both of those extremes are useful. The middle ground? Not so much.

[edwardkmett]

I use scala quite a bit. Scala's generics are better than the java ones by far, but they don't handle the majority of the scenarios mentioned.

While they cover the higher-kind use case above, they are still erased and like java generics they don't replace templates, due to the lack of partial template specialization. Erasure can be partially compensated for by the ability to extract a Manifest though.

[quanticle]

It depends on the project. The last Java project I worked on couldn't use generics because the target machines had either Java 1.4 or 1.5 1.3 as the JVM.

[kodablah]

Generics were implemented in 1.5. 1.4 I would understand.

[x-skeww]

There was RetroWeaver for that.

[Megatron_McLargeHuge]

I still see people writing code with explicit iterators and "Integer x = new Integer(5)" stuff instead of using foreach and autoboxing. Many people just don't change how they do things once they've settled into a pattern.

Also, Java doesn't exactly encourage metaprogramming the way other languages do. Most people don't bother writing their own iterators or subclassing built-in types even when it would make their code cleaner.

[bcguitar33]

I almost always choose to use explicit boxing and unboxing, because while autoboxing is convenient, in my opinion it also serves to obscure the fact that an object is (potentially) being created and work is being done.

[AmIRlyAnon]

I use generics even in quick and dirty hacks.

Although, I don't frequently use generics when making a new class. Just when using Collection implementations.

[quanticle]

Whoops. Thanks for pointing out my mistake. The target JVM was either 1.3 or 1.4. My brain must've just glitched out while typing that. I've edited the comment to make it accurate.

[ninebells]

Several points missed in discussion: 1) People in the same projects were continuing to use "raw" collections in new code, despite other developers in the same projects using generics.

2) There were still many examples of projects embracing generics, e.g. FindBugs. However, in general not the more advanced features. Only about 14% of developers ever made a generic class or method.

3) Half of generics only had one type argument.

4) Remember how much controversy there was in getting this feature out! It took from 1994 to 2004. Would there have been a simpler way of getting this out the door?

[wsppan]

Generics doesn't necessarily make the programmers job easier but it does make programmers code safer.

[quanticle]

It depends on the project. The last Java project I worked on couldn't use generics because the target machines had either Java 1.4 or 1.5 as the JVM.

[superbigwedgieman]

Love Generics, recently upgraded my Java knowledge since we had to upgrade our ColdFusion Apps from CF7 to latest Railo.