I wanted to know what is the best way to cluster computers running linux of the size of about 40 nodes.
I looked up online and found rocks (http://www.rocksclusters.org)
does anyone know about how good/stable this is? how easy it is to install this for a person who has never installed any cluster before? also are there any problems with using this?
or is there any other easy to install clustering software that also supports queuing jobs?
There's Beowulf! It's been around for quite a while and seem to be the way to go. I don't have first hand experience with clusters, but have heard good things about Beowulf clusters.
Beowulf is kind of a generic term for a high performance compute cluster running on commodity hardware and free software. There used to be an actual distribution called Beowulf but Scyld seemingly has the rights to it and sells it now. Rocks is supposedly super easy to use. Warewulf/Perceus looks pretty nice for node provisioning and server side images. Sun Grid Engine looks nice for the computing part (if you're looking towards HPC not HA).
What you use depends on what type of cluster you are talking, high availability/load balancing or high performance computing.
Thanks likwid and bidi for the replies.
I am talking about an HPC cluster.
I read more about Warewulf/Perceus, OSCAR, and Clustermatic and am a bit more confused now.
not sure which is the best way to go for someone who does not have firsthand experience with building a cluster before.
Well I really recommend picking up "Building Clustered Linux Systems" from HP Press. Especially if you are tasked with this professionally. I picked it up out of a personal interest in high performance computing and about 30 sun workstations laying around. The guy who wrote it built clusters for pretty big research labs. He takes it real slow too so if you have no idea where to start, it's good.
great .. will do that.
Thanks for the help!