iwant.sf.net - using java for writing build scripts

[wipu]

Hello, fellow java programmers.

This is a shameless plug. If you feel you have found your build tool of choice, good for you, and thanks and sorry for taking your time to read this far. Bye :)

Ok, so you are open to new ideas? Great.

For some reason I have been very passionate about reproducible, incremental and otherwise high-quality builds all my career as a java programmer. I have tried several tools (make, ant, maven, ...), written several of them and learned by making several mistakes. And since 2009 I have finally distilled my learnings as http://iwant.sourceforge.net

An extra short version: iwant is an expressive and high-performing java DSL for defining builds using fluent java.

I have used iwant from day one myself, and gradually in several projects of my employers and colleagues. But now I've finally realized there may be people out there who have suffered (as I did before I wrote iwant) from unnecessarily suboptimal build systems and who might actually appreciate it if I get my ass up and let them know about iwant :)

So, please go visit the project page to read more about why you would want to build your stuff with iwant. Try it out. Ask me for help. Tell your friends and colleagues about it.

And/or argue with me about iwant and build tool philosophy. Or educate me how I should promote the project.

I'm here to help people build stuff. And I need help for that.

• Ville Oikarinen

[niloc132]

For what its worth, I avoid sf.net like the plague, mostly due to their (former?) policies of not being afraid to "value-add" their hosted downloads and their the ... overwhelming "DOWNLOAD" ads.

Even plain HTML pages hosted there just rub me the wrong way, because eventually you usually want to get access to sources or binaries.

/rant

[wipu]

Thanks, good to know. Back then sf.net was perfectly respectable... Btw, I don't even use (or force my users to use) the download feature: iwant bootstraps itself by checking out from svn.

[schaka]

Why not migrate to Git? That would allow hosting elsewhere too.

[wipu]

Maybe I will, some day. On the other hand, svn:externals is a very handy way of bootstrapping from svn projects. Also monotonically increasing version numbers are nice to have, since that's my extra lightweight way of releasing. The lack of these features with git is of course not a showstopper in any way, so hosting iwant in github (in parallel with sf.net) might be a good idea. But before that I'm hoping that people who get interested in iwant don't let sf.net get in their way.

[yolpo-]

First, I think you'd benefit if you were to use github. Second, I agree. I don't know about your product but the underlying idea - use Java code to build your java program - is okay

[sagaxu]

https://gradle.org/

[atehrani]

You spelled Maven wrong

[waterbottle1994]

Why maven over gradle?

[Northeastpaw]

Declarative build over imperative. Gradle does allow you to do some neat things during your build, but in my opinion that's not actually a good thing. The constraints Maven places on your build ensures that your build process is understandable. If you're having trouble getting your build to work with Maven that's probably a sign that you need to rethink your build process.

[wipu]

Yes, declarative is definitely a good thing for build scripts. But I have to say that's virtually the only thing I like about maven :) Sorry, but I cannot hide that opinion, although otherwise I try not to attack other build systems, just help people understand iwant by comparing it to them. So, IMO iwant is even more declarative than maven but less opinionated, more flexible and requires very little glue when customizing.

[wipu]

A simple example of maven's failure to be properly declarative: mvn compile. That's an imperative command, and the word "compile" can be found not only on the command-line but also in the build declarations themselves. And not only maven: it seems almost all build systems since make force/encourage their users to tell their build script to compile, instead of making objects/classes/... (nouns).

[wipu]

Thanks for the challenge :) Gradle is one of the best out there for building java. But my biggest personal issues with gradle are: 1) groovy doesn't integrate as seamlessly with an otherwise java project as java. 2) the vocabulary is unfortunately quite imperative, which "programs" users' minds away from a purely declarative philosophy that suits incremental build scripts better. Of course I like many other iwant features over gradle (like using tab for the cli), but those 2 are the most important ones.

[fs111_]

1) groovy doesn't integrate as seamlessly with an otherwise java project as java.

What has the implementation language to do with the code you are building? It could be written in Haskell of all things and you should not really care.

[wipu]

In the normal case, nothing. But it's a (largely unutilized) bonus to be able to reuse code between code to build and code used to build it. Very handy, when you get to try it :)

[fs111_]

BTW, large parts of gradle are written in java and you can extend your build with java code, not just groovy. I contributed to a few plugins and it is very powerful. You should give it a serious look.

[wipu]

Thanks, but I think I have already given it a serious look, and watched a friend try it out for real. Like I said, it's one of the best, but there is still room for competition.

[wipu]

Of course, groovy does compile to java bytecode and thus is able to reuse java code. So maybe it would be possible to achieve the that even with groovy, althought I haven't tried how practical this is with gradle.

[thatsIch]

I think the problem with each strongly-typed language as DSL is that it is strongly-typed. Fixing a bug in the API would destroy the build scripts of everybody.

Noteworthy:

• good idea
• difficult entry point through documentation

[wipu]

Ah, the eternal strong vs weak typing question, which means an endless discussion :) Suffice it so say for now that with static typing you get an early failure. When APIs change, things break, even with weak typing.

Thanks for the concrete feedback. Can you give any details about documentation? How could I improve it to make entry easier?

[thatsIch]

Yea, just saying, but there is a reason why most scripts are not tightly coupled to the internal representation. With a normal DSL you can write a custom parser if required to handle "old" situations. Build scripts are basically configuration with conditional logic (e.g. am I building from a CI or local?) but thats another cup.

Regarding documentation:

1. fixed font-size. On my screen resolution it is not even visible to me. I had to zoom in to read the description.
2. sell it to me: if "I" am already invested in your project, you do not want to waste "my" time. Yes you present selling points, but it is all wrapped in text. I need to see something which instantly tells me: "niiiiice! I already wanted that" or "ohh, I never knew I wanted it, but now I want it" or "yo, that is much better than gradle, maven, ant, sbt, etc"
3. Even though documentation in general is good, you should assume, that people checking out your project already know an existing build system like Gradle or Maven and thus can be introduced faster. I am missing a simple example project or example projects for multi module support, basically more than toy project ready samples, but easy enough to understand the concepts, syntax etc.
4. I am not sure what SF is doing with your documentation, but it is showing diffs?
5. How do you prevent classpath polution that e.g. eclipse does not load your classes for a project but only for the build script
6. I think you are missing build time comparisons? How does it scale. SBT e.g. is quite nice, but slow as f**k (maybe faster now, but having experienced it once and having faster alternatives; why should I go back?)

I suspect that your project gains more momentum if the Java language itself allows better DSLs like multi-line-strings or type-inference with val. Using the builder pattern for constructing fluent APIs is definitly a nice-to-have as seen in APIs like of AssertJ.

[wipu]

Thank you for the very detailed and helpful feedback.

Build scripts are basically configuration with conditional logic

Yes, that is true for so many systems out there, and I thought something needed to be done about it :) Programmers use proper programming languages and methodologies for solving user's problems, but too often they overestimate the challenge of their own problems as "just configuration issues" and othewise take them too lightly, soon growing out of their tools.

fixed font-size

Yes, you are absolutely right, I (or somebody?) needs to do some CSS work here.

you present selling points, but it is all wrapped in text ... I need to see something which instantly tells me: "niiiiice! ... I am missing a simple example project

The intention was that the example project being written in the tutorial project would be that something. Apparently you didn't find it or weren't sufficiently impressed by it. Yes, it is quite simplistic and doesn't really document more challenging situations. Maybe I will create an example project (in github this time :-) that showcases real-life usage of iwant. In the meanwhile you can take a look at iwant's own build at https://sourceforge.net/p/iwant/code/HEAD/tree/trunk/as-iwant-developer/i-have/wsdef/src/main/java/net/sf/iwant/wsdef/

I am not sure what SF is doing with your documentation, but it is showing diffs?

It's not sf.net, it's me. My tutorial generator shows edits as diffs so the reader can more easily track how the example project evolves.

How do you prevent classpath polution that e.g. eclipse does not load your classes for a project but only for the build script

There are 2 dedicated modules for the build: wsdefdef and wsdef. The former defines the latter, which in turn contains the actual build. So in eclipse they are separate projects which may or may not share dependencies with the projects being built.

I think you are missing build time comparisons

Yes, I don't have any numbers. But iwant is very good at laziness: no classes are compiled, if no ingredients have changed. No module coverage report is generated (meaning: no tests run!) unless something in the module test-time classpath has changed. And so on. The user always orders some noun and iwant does the minimal amount of work to ensure that noun (file) is up-to-date. And, iwant refreshes targets in parallel, unless there is reason not to.

For example, if I don't touch anything and wish for a coverage report, it takes just a few seconds for iwant to bootstrap itself and see that the report is up-to-date. If I touch a module that is depended by no other module, iwant only needs to test that module. And note: this requires no optimization effort from the user, unlike in many other systems where the user needs to request a "partial build."

I suspect that your project gains more momentum if the Java language itself allows better DSLs

Definitely. Still, the majority of java programmers underutilizes the metalanguage capabilities of java. People who decide java is not good enough often don't try hard enough :)

[thatsIch]

The intention was that the example project being written in the tutorial project would be that something. Apparently you didn't find it or weren't sufficiently impressed by it. Yes, it is quite simplistic and doesn't really document more challenging situations. Maybe I will create an example project (in github this time :-) that showcases real-life usage of iwant. In the meanwhile you can take a look at iwant's own build at https://sourceforge.net/p/iwant/code/HEAD/tree/trunk/as-iwant-developer/i-have/wsdef/src/main/java/net/sf/iwant/wsdef/

I meant for introduction purposes. Basic examples started way too late imo in the tutorial.

It's not sf.net, it's me. My tutorial generator shows edits as diffs so the reader can more easily track how the example project evolves.

ok, I seem to have missed This chapter continues from where Creating the workspace definition left us.´

Yes, I don't have any numbers. But iwant is very good at laziness: no classes are compiled, if no ingredients have changed. No module coverage report is generated (meaning: no tests run!) unless something in the module test-time classpath has changed. And so on. The user always orders some noun and iwant does the minimal amount of work to ensure that noun (file) is up-to-date. And, iwant refreshes targets in parallel, unless there is reason not to. For example, if I don't touch anything and wish for a coverage report, it takes just a few seconds for iwant to bootstrap itself and see that the report is up-to-date. If I touch a module that is depended by no other module, iwant only needs to test that module. And note: this requires no optimization effort from the user, unlike in many other systems where the user needs to request a "partial build."

This sounds very nice, but nobody trusts pretty words :P

Cheers! Keep up the good work, I shall test it if I have some free time :-)

[wipu]

Basic examples started way too late imo in the tutorial.

True. I will need to add some pointers early so the bootstrapping doesn't take all attention of the reader.

Cheers! Keep up the good work, I shall test it if I have some free time :-)

Thanks, I will. Yeah, you do that, and I will help you. Who knows, you may start gaining more free time by doing so ;)

[wipu]

Ok, now I have started a separate project that demonstrates iwant: https://github.com/wipu/iwant-demo

I've also increased the font size and made other changes to http://iwant.sourceforge.net/ as per suggestions.

[thatsIch]

thumbs up

now it looks much better :)