Tuesday, July 13, 2010

Can we build it? Yes we can!

Thomas Hallgren, Michal Ruzicka, and Kenn have been toiling away since EclipseCon (Michal fulltime, Thomas and Kenn intermittently) on the committer community's build problems. We haven't been blogging on this topic as often as we should have for Thomas and Michal to get the recognition they deserve. So here's a quick summary of their contributions to the "build commons" over the last few months.

Building Helios

First, we had to get the Helios Builder (the b3 aggregator) working. In theory, it worked perfectly, except that it had the irritating habit of not always doing what David Williams expected it to do. I think David was happy with the result in the end, but we had to spend a lot of time fixing and babysitting it along the way (Filip Hrbek also helped out here).

Building Modeling, RAP, Mylyn, etc.

Second, we Buckminsterized a bunch of projects, most notably within the Modeling domain. The Modeling PMC adopted Buckminster as the standard for project builds a few months back, and we agreed to jumpstart the initiative by Buckminsterizing a number of subprojects. To date, we've finished migrating nearly half of the active projects, including many (most?) of the higher visibility projects such as Xtext, Acceleo, CDO, and EMF Core. Some of these projects have already cut over to the new build; others will be cutting over this month, now that Helios is behind them.

We also found time to build a few projects outside of Modeling. We finished RAP last month and we'll be Buckminsterizing the restructured Mylyn project this month as part of leading the new Mylyn Build subproject.

Up to this point, we've done all the work ourselves. But we've found that most projects can be Buckminsterized in under a day, so we'll generally be limiting our role to supporting the project teams from this point forward.

Building the Platform

Finally, while everyone else was watching the World Cup (Sweden was knocked out early), Thomas Hallgren had plenty of time for a singleminded pursuit of the shiniest trophy in his Eclipse domain - the Eclipse Platform build!

Thomas was able to get it working in just under two weeks. It actually turned out to be not that difficult, since Buckminster is able to reuse most of the work that's gone into the current Platform build. For example, to bring things into the workspace for the build, we simply used a Buckminster RMAP (resource map) provider that appoints the platform's existing releng maps. The SWT parts of the build were a bit tricky, since they have to be repeatedly called with different settings (which hopefully underscores the importance of fixing this bug).

Other than that, the platform build fit very well within the Buckminster defaults (source bundle/feature generation, qualifier replacement, etc.). So one really nice consequence will be that the new build should be minimally disruptive to those using or depending on the current one.

Thomas got the build working *in the lab* the night before leaving for the traditional Swedish extended summer vacation (which basically kills July over there). He'll have a few details to polish when he gets back, but once he has everything all prettied up and packaged, we'll publish the results for anyone who wants to inspect/repeat the results for themselves. So stay tuned for more.

You might wonder why we took the time to do this, given that the current Platform build seems to work just fine. One reason is that it seemed like a good challenge. We often hear that the true test of a build system is whether it can build the Platform cleanly. We always thought Buckminster could do it, but it seemed like an academic exercise, since it wasn't clear anyone was interested in an alternative build.

Recently, however, there's been a lot of discussion about the need for builds that can be run outside the Platform project team, as well as greater openness in the build domain in general. Many organizations want to fix bugs themselves but can't build the result, or worry who will build the older versions of Eclipse they depend on. There even have been rumblings about IBM wanting to share responsibility for the Platform build. So it seemed like a good time to try.

Kenn also caught up with some Sonatype folks at the Toronto DemoCamp the week before last, and learned that they've been trying to build the Platform as well (using Tycho). Alternatives are always a good thing. 

Note: for readers without kids in your environment, the title and picture are references to Bob the Builder.