Posts Tagged parallel

Stacking GWT 1.6 workers with ant parallel for super-fast builds

Having just built a new machine around a Core i7 950 enthusiastically overclocked to 4.2gz, I couldn’t help but explore the possibilities of a lightning-fast GWT compile. I regularly build a project that involves a core services war that’s pure Java along with a GWT 1.6 war consisting of two modules that requires 3 permutations each, all wrapped into an ear at the end.

Historically this took about 3 minutes running serially on a reasonable Core 2 Duo. It would run the core services war build, then the GWT build one module at a time, each using GWT 1.6′s -localWorkers 2 to at least make use of both cores.

The potential of 8 concurrent threads across the i7′s four cores and the presence of two GWT modules plus another build compelled me to resurect some old-school Ant parallelization to stack as much work as possible during the build. With a bit of tweaking it’s easy to have ant running multiple targets – some GWT and some non-GWT – at the same time, so that entire modules, not just permutations, are built in parallel. For example, something like

<target name="gwt">
	<java classname="com.google.gwt.dev.Compiler" fork="true" failonerror="false">
		<arg value="module1" />
		<arg value="-localWorkers" />
		<arg value="1" />
	</java>
	<java classname="com.google.gwt.dev.Compiler" fork="true" failonerror="false">
		<arg value="module2" />
		<arg value="-localWorkers" />
		<arg value="1" />
	</java>
</target>

becomes

<target name="gwt">
	<parallel threadsperprocessor="1">
		<java classname="com.google.gwt.dev.Compiler" fork="true" failonerror="false">
			<arg value="module1" />
			<arg value="-localWorkers" />
			<arg value="4" />
		</java>
		<java classname="com.google.gwt.dev.Compiler" fork="true" failonerror="false">
			<arg value="module2" />
			<arg value="-localWorkers" />
			<arg value="4" />
		</java>
	</parallel>
</target>

The result was phenomenal. My 3 minute build now takes only 40 seconds! If you have lots of cores available be sure to consider stacking ant parallel with GWT workers for a big improvement. Keep in mind that running many modules in parallel may also require ridiculous amounts of memory (around 600mb per module in my case), but on an i7 system you should have at least 6gb anyway right? :)

, , ,

4 Comments