Cluster for Batch Photo Processing

I was considering stealing some cycles from an old P4 Box I have set up as a NAS, my Core 2 Duo Laptop, and my Core Duo Mac Mini. Details on each machine at bottom of post.

Has anyone attempted to use a cluster to resize photos? I have 10's of 1,000s of photos to resize and I was wondering if clustering together made sense in this case?

My questions...

1. Which should be the master (does the master get fully utilized? - i.e. or should it be the slowest machine?)
2. Has anyone tried Imagemagick on a cluster environment?
3. The first machine (NAS below) only has a 10/100 mb/s LAN connection - should this not be included in the cluster b/c my options are to crossover the laptop/mini at a 1000 mb/s connection or run all three on a 100 mb/s router (WRT54GL)
4. Are there any obvious bottlenecks in the machines below?
5. Can a script be created to say send - the first file to node 1, second to node 2, third to node 3 and then start all over?

Here is the detail on each Machine

* NAS Single core P4 (1.8 Ghz) 512 Mb Ram (Unknown Speed) / Ubuntu / Striped RAID Array
** Core 2 Duo Laptop (2.0 Ghz) 800 MHz FSB / 3 GB Ram (PC2-6400) / Vista / 7200 RPM HD
*** Core Duo Mac Mini (1.8 Ghz) 667 MHz FSB / 2 GB Ram (PC2-5300) / Mac OS X 10.4 / 5400 RPM HD

Welcome to the forums!

It sure does make sense to spread big operations out over multiple machines, but the way you approach it you make it very difficult for yourself. You have not only very different machines with different specs (which isn't a big problem), but also they operate on very different platforms.

I'm sure this wont mean it's a 'no go', but it does mean you will probably spend more time tweaking your setup than actually running the show. That's because you will have to modify every system with custom settings. It seems to me you want to use existing (but very different) systems for a particular task, whereas I'd say your life will become much easier if you can dedicate those systems to that particular task.


That said, it seems your Vista box is the best one for the master node. But here's the irony. It's got Vista. So it can only act as slave node. Second in command seems the Mac, but then, it's a Mac. I don't know if the mac is capable of being master?
So if you ask me, the slowest and thus poorest choice is best as master node. As far as my knowledge reaches it's the only one capable of doing what you have in mind. But as I said, it's sub-optimal.

It's your choice. You could just divide your photo's over the three machines manually and script some mogrify loop and let them run all night.
Or you can build your NFS server, add two customized initrd's, a little kernel, passwordless SSH connections to match and PXE netboot the Vista machine and Mac to become nice slave nodes doing all your work for you.

(The initrd && PXE netboot combo will cope with the problem of having three different platforms, thus saving you the trouble of installing Linux over OSX and Vista. But it also adds to the complexity of building the cluster.)


And then of course there's LOTS of testing before the machines respond well to each other and co-operate. But it is not impossible.