Convincing darwinports to buid in parallel

Posted by Roy Hooper Sun, 25 Mar 2007 22:57:00 GMT

Sometimes you want to use all of your available CPU power for building. If Darwinports were like FreeBSD ports, it would be really easy to pass -jN to the build process (make -jN install). With a little bit of playing around, I managed to sort this out.

Darwinports isn’t quite like FreeBSD ports, and doesn’t depend on makefiles, so you can’t directly pass -jN to the build process. In order to trick it into passing -jN flags to make at build time, you need to edit /opt/local/etc/ports/ports.conf to add:

extra_env                       MAKEFLAGS

From that point on, you can define flags to be passed into make as a part of the build process:

MAKEFLAGS=-j6 port build <portname>

You’ll run into weirdness if you try to install a port with that MAKEFLAGS set, and some ports won’t build with -jN flag set. For a dual-CPU system, I reccomend -j3. For a quad-CPU system, I reccomend -j5 or -j6.

Here’s a timing comparison:

monopoly:~ rhooper# time sudo port build ImageMagick
--->  Fetching ImageMagick
--->  Verifying checksum(s) for ImageMagick
--->  Extracting ImageMagick
--->  Configuring ImageMagick
--->  Building ImageMagick with target all

real    5m2.928s
user    2m24.148s
sys     2m19.407s

And here it is with -j5:

monopoly:~ rhooper# sudo port clean ImageMagick
--->  Cleaning ImageMagick

monopoly:~ rhooper# MAKEFLAGS=-j5 bash -c "time sudo port build ImageMagick" 
--->  Fetching ImageMagick
--->  Verifying checksum(s) for ImageMagick
--->  Extracting ImageMagick
--->  Configuring ImageMagick
--->  Building ImageMagick with target all

real    2m38.003s
user    2m43.868s
sys     3m22.793s

And just to compare, here it is with -j6

monopoly:~ rhooper# sudo port clean ImageMagick
--->  Cleaning ImageMagick

monopoly:~ rhooper# MAKEFLAGS=-j6 bash -c "time port build ImageMagick" 

--->  Fetching ImageMagick
--->  Verifying checksum(s) for ImageMagick
--->  Extracting ImageMagick
--->  Configuring ImageMagick
--->  Building ImageMagick with target all

real    2m32.642s
user    2m45.351s
sys     3m26.030s

This will be a real timesaver with those bigger ports!