Can we build it? Yes we can!

Thomas Hallgren, Michal Ruzicka, and Kenn have been toiling away since EclipseCon (Michal fulltime, Thomas and Kenn intermittently) on the committer community's build problems. We haven't been blogging on this topic as often as we should have for Thomas and Michal to get the recognition they deserve. So here's a quick summary of their contributions to the "build commons" over the last few months.



Building Helios

First, we had to get the Helios Builder (the b3 aggregator) working. In theory, it worked perfectly, except that it had the irritating habit of not always doing what David Williams expected it to do. I think David was happy with the result in the end, but we had to spend a lot of time fixing and babysitting it along the way (Filip Hrbek also helped out here).

Building Modeling, RAP, Mylyn, etc.


Second, we Buckminsterized a bunch of projects, most notably within the Modeling domain. The Modeling PMC adopted Buckminster as the standard for project builds a few months back, and we agreed to jumpstart the initiative by Buckminsterizing a number of subprojects. To date, we've finished migrating nearly half of the active projects, including many (most?) of the higher visibility projects such as Xtext, Acceleo, CDO, and EMF Core. Some of these projects have already cut over to the new build; others will be cutting over this month, now that Helios is behind them.



We also found time to build a few projects outside of Modeling. We finished RAP last month and we'll be Buckminsterizing the restructured Mylyn project this month as part of leading the new Mylyn Build subproject.

Up to this point, we've done all the work ourselves. But we've found that most projects can be Buckminsterized in under a day, so we'll generally be limiting our role to supporting the project teams from this point forward.

Building the Platform

Finally, while everyone else was watching the World Cup (Sweden was knocked out early), Thomas Hallgren had plenty of time for a singleminded pursuit of the shiniest trophy in his Eclipse domain - the Eclipse Platform build!

Thomas was able to get it working in just under two weeks. It actually turned out to be not that difficult, since Buckminster is able to reuse most of the work that's gone into the current Platform build. For example, to bring things into the workspace for the build, we simply used a Buckminster RMAP (resource map) provider that appoints the platform's existing releng maps. The SWT parts of the build were a bit tricky, since they have to be repeatedly called with different settings (which hopefully underscores the importance of fixing this bug).

Other than that, the platform build fit very well within the Buckminster defaults (source bundle/feature generation, qualifier replacement, etc.). So one really nice consequence will be that the new build should be minimally disruptive to those using or depending on the current one.

Thomas got the build working *in the lab* the night before leaving for the traditional Swedish extended summer vacation (which basically kills July over there). He'll have a few details to polish when he gets back, but once he has everything all prettied up and packaged, we'll publish the results for anyone who wants to inspect/repeat the results for themselves. So stay tuned for more.

You might wonder why we took the time to do this, given that the current Platform build seems to work just fine. One reason is that it seemed like a good challenge. We often hear that the true test of a build system is whether it can build the Platform cleanly. We always thought Buckminster could do it, but it seemed like an academic exercise, since it wasn't clear anyone was interested in an alternative build.



Recently, however, there's been a lot of discussion about the need for builds that can be run outside the Platform project team, as well as greater openness in the build domain in general. Many organizations want to fix bugs themselves but can't build the result, or worry who will build the older versions of Eclipse they depend on. There even have been rumblings about IBM wanting to share responsibility for the Platform build. So it seemed like a good time to try.

Kenn also caught up with some Sonatype folks at the Toronto DemoCamp the week before last, and learned that they've been trying to build the Platform as well (using Tycho). Alternatives are always a good thing. 

Note: for readers without kids in your environment, the title and picture are references to Bob the Builder.

Do well by doing good at Eclipse

For those who aren't familiar with Eclipse/Tycho, it's a new project from Sonatype intended to make it easy to do Maven builds from within Eclipse.

Here's a quote from the project proposal they submitted last month:

"There are existing solutions at Eclipse in the same space at Tycho. Among these solution [sic] are Buckminster, B3, PDE Build, and Athena. Tycho will collaborate with these projects on P2 interoperability issues, but aside from that Tycho competes with these projects."

For those of us who've been reading Eclipse project proposals for the last six years, the language is somewhat jarring. New projects generally go out of their way to talk about how they intend to be complementary to everything, collaborate with everyone, etc. Of course, this doesn't always happen. But generally that's because of the coordination challenges, rather than because the intention to collaborate wasn't there in the first place.

I've been sitting on this topic for a few weeks. People have been pinging us to say the proposal was surprising and ask why we hadn't responded. We didn't want to get into some "Maven versus everything else" religious battle. It's a turn-off for users. And it's not like the build domain is likely to end up a technical monoculture, despite what anyone wants, so why waste everyone's time getting worked up about it?

Besides, I couldn't put my finger on what, if anything, there was to object to. There's nothing wrong with competition per se, obviously. We're all driven to excel in this community (i.e., inclined to compete). Hopefully that makes us work harder and produce better products. And there's no point acting as though Eclipse projects aren't sometimes in competition. Maybe that competition isn't as widespread or explicit as it is at Apache, but it definitely exists.

Plus, you have to give the Tycho crew an A for honesty. Apparently, we're behind 2 of the 4 projects they think are standing in their path. At least they let everyone know that right up front! Compare this to Kepler, the last big Maven-to-Eclipse project. The EMO told the Kepler team to collaborate with the Buckminster guys (us), so they added us to the proposal as committers. But they never meant it and ended up wasting everyone's time.

But the more I thought about it, the more I decided there is some greater principle at stake. (Plus, I needed a first post for my new blog. :-) ) It comes down to the difference between using Eclipse as a better way to create technology for yourself and others (the interests of the "commons") instead of primarily a way to claim ownership of a "market. While the former has generally paid off (IMHO) for those who've tried it, as well as for the ecosystem as a whole, the latter generally has not. (I can think of a few exceptions, but I'd argue it's generally true.)

Anyone remember ALF? That project came out of the "market for sale" phase of the Eclipse ecosystem (circa 2004/5). All you had to do was start a top-level project in whatever market space you wanted to dominate, and then you'd end up the "Red Hat" of that space. That's just one example; I can come up with many. The fallacy, of course, is that the developer community only cares about how real the technology is, and the user community ends up following the developers. Projects that turn out good technology tend to succeed; others don't.

In fact, when I think of commercial successes in the ecosystem, it seems like commitment to good technology almost always comes first, and commercial opportunities come after that. Consider Mylyn and Tasktop - the technology was great long before it was commercialized. Or itemis and modeling - supporting innovative modeling projects gives them credibility with clients; technical strategy within the projects is independent of the marketing considerations. These are only a few examples. I can come up with many.

So what's the moral? That you're more likely to get a return on your Eclipse commitments if your goal is to get technology that's better than you could develop privately. Conversely, you're likely to be disappointed if you see Eclipse as some giant marketing channel for an essentially proprietary agenda. Expected ROI on pure marketing agenda: negative. Expected ROI on pure technology agenda: positive. And an ancillary benefit is that you get to focus on the interest of the commons, unlike the commercial domain where you can't. (I'm assuming you enjoy contributing to the commons, by the way; not everyone does.)

So how does this relate to the proposal? Because it forecloses the potential for collaboration. If your inclination is to see everyone else as a competitor, rather than a potential collaborator, you'll never collaborate. In some cases collaboration may not be possible. But in this case, it may well have been. We've always seen Maven as an important technology with lots of users that should be treated as first class citizens. Our inclination has always been to ask, what does Maven already do, and what can we do that's different and hopefully complementary?

But no one asked us to collaborate. They just decided that whatever we're doing is in the way and decided we were "competitors." Developer cycles and good code will be wasted on all sides. Users will be confused. And the community will probably end up with something that isn't as good as it could have been.

If that kind of thing bothers you, then it's a bummer.

first post...