Find the answer to your Linux question:
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 13
ok, i wish to cluster a few boxes. the boxes are all pretty much the same, clustering them would not really be a problem, except all i really want them ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    May 2006
    Posts
    9

    a simple cluster


    ok, i wish to cluster a few boxes. the boxes are all pretty much the same, clustering them would not really be a problem, except all i really want them to do is file serve. i would just be utilizing the ethernet and the hard disks (all other hardware is completly arbitrary) all of the boxes would be booted from the master node via the network then mount the disks via NFS or whatever.

    now this may seem simple, but i can not find anything on the situation, i have been looking for two weeks. i mean there must be a way. or maybe i am missing something.

    i guess my question is does anyone know a way?

    i just want my files.

  2. #2
    Linux Enthusiast
    Join Date
    Apr 2004
    Location
    UK
    Posts
    682
    I've never done anything with clustering so I might not be able to help out, but I do have a quick question that might help; What kind of cluster are you trying to set up? Do you want a failover setup or do you want to combine all of the availble space into a single logical storage device, or something else I havn't thought of.

    Let us know how you get on,

    Chris...
    To be good, you must first be bad. "Newbie" is a rank, not a slight.

  3. #3
    Just Joined!
    Join Date
    May 2006
    Posts
    9
    maybe cluster is the wrong word. i just want to have the space, but i want to beable to boot the boxes through the lan. i plan to clear the drives of operating systems. i dont want bells and whistles, just the space. no ssh no rsh, nothing like that. it would all be behind the master node so security is not an issue.

    basicly, pxe or etherboot via dhcpd, load up a the kernel, mount the hard drives via nfs to the master node.

    i keep trying to think of very simple ways to do this. but ive got nothing.

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Enthusiast
    Join Date
    Apr 2004
    Location
    UK
    Posts
    682
    You could try out the Linux Terminal Services Project. It comes with everything you need to boot over a network. It may be overkill for what you want, but it ought to cover your bases. After it's working you can tinker with the client settings to allow a minimal setup.

    I'm pretty sure it uses network block devices to allow the clients to use swap space on the server which you could possibly re-jig to make the client's space available on the server.

    Let us know how you get on,

    Chris...
    To be good, you must first be bad. "Newbie" is a rank, not a slight.

  6. #5
    Linux User IsaacKuo's Avatar
    Join Date
    Feb 2005
    Location
    Baton Rouge, LA, USA
    Posts
    406
    Exactly why do you want to do this with netbooting? If you've got the hard drives in each machine anyway, you can avoid single points of failure by installing the OS locally on each machine. This will help performance also.

    Personally, I'd put all of the hard drives in one machine and maybe set up a second machine as a spare full backup. With all drives in one computer, you have a lot of flexibility to set up a robust RAID5 or RAID1 or whatever combination suits your needs.
    Isaac Kuo, ICQ 29055726 or Yahoo mechdan

  7. #6
    Just Joined!
    Join Date
    May 2006
    Posts
    9
    i will give a close look to Linux Terminal Services Project, it seems plausable. last update was from 2004 but i guess i could just read it over, maybe find something.

    it may come down to building it all from scratch, i guess thats fine and all, but where to begain? i will just keep reading utill a few more things click into place. i know what i want, its just making everything work together.

    andy

  8. #7
    Just Joined!
    Join Date
    May 2006
    Posts
    9
    Quote Originally Posted by IsaacKuo
    Exactly why do you want to do this with netbooting? If you've got the hard drives in each machine anyway, you can avoid single points of failure by installing the OS locally on each machine. This will help performance also.

    Personally, I'd put all of the hard drives in one machine and maybe set up a second machine as a spare full backup. With all drives in one computer, you have a lot of flexibility to set up a robust RAID5 or RAID1 or whatever combination suits your needs.
    i have considered alot of options, the main thing is, i would not rather have to maintain the other machines, i mean besides writting files back and forth. i want them to turn on and mount the file systems automaticly to the master node. hence the net booting. if i boot them from the net i can do all of this from one machine, proformance is not an issue, and i would rather conserve the space then add another operating sysem.

    i have considered a live cd for each machine, but then i take a hit on memory,

    i am thinking the avg space each machine has is about 360gigs. they all can boot from the eth cards. i would ad more drives to each one, but they are at their limit now. the machines are all AMD athlon xp between 900mhz and 1.5ghz i have 4 of them(and if i do not utilize them i will go insane) i actually think the motherboards are exactly the same in atleast two of them and all of the motherboards are from the same manufacture. so hardware shouldnt be issue here.

    i could go the way of something like damn small linux, but i would not rather have to configure each machine, and configure each and every time i add a machine to my network, i do not need video, sound, usb, no other hardware then the network cards and the hard drives.

    i mean it just seems silly to waste all of that space on things i just dont need, the kernel could just be loaded at boot time into memory. if the machine acts up, hit reboot. if nothing happens it would be odvious its hardware.

    i feel like there has to be a simple solution besides cluster suits and cluster distros. they are way overkill for what i want. i want to just add it to my already working set up with out to much hassel and adding minmal things to run it.

    i use "elive" installed to the hard drive. i like it because it fits nicely with the testing packs in debian i have been using elive 0.4.2 since it came out and have had no major issues except with my usb sound card. and that isue is it does no use my 5.1 suround sound so i have stereo insted. that and no KDE or GNOME. they are useless when you can $ anyways.

  9. #8
    Linux User IsaacKuo's Avatar
    Join Date
    Feb 2005
    Location
    Baton Rouge, LA, USA
    Posts
    406
    Quote Originally Posted by andykryst
    i have considered alot of options, the main thing is, i would not rather have to maintain the other machines, i mean besides writting files back and forth. i want them to turn on and mount the file systems automaticly to the master node. hence the net booting. if i boot them from the net i can do all of this from one machine, proformance is not an issue, and i would rather conserve the space then add another operating sysem.
    If you install Debian Stable on each machine, you will NEVER have to mess with them after the install and initial configuration of nfs shares. Better yet, just do one Debian Stable install, and simply copy the OS partition for the other systems. Then, the only post-configuration you have to deal with is manually changing the static IP address in /etc/network/interfaces. This assumes you want the same basic partition structure for each machine. If not, then you'll have to manually edit the /etc/fstab and /etc/exports also.

    I really don't see how you can make things easier with net-booting. The biggest problem with net-booting is the headache of figuring out a way to give each machine its own static IP. Also, if there's any sort of configuration which needs to be done differently on any machine, then you've got some more big headaches.

    Net-booting multiple machines really only works well when you don't care which one is which. For example, in a cluster computing solution, the individual nodes are completely interchangeable. For your application, though, the identity of each node does matter because you want to be able to locate your files in the same place every time.

    i am thinking the avg space each machine has is about 360gigs.
    A Debian install will cost you less than half a gig.

    they all can boot from the eth cards. i would ad more drives to each one, but they are at their limit now. the machines are all AMD athlon xp between 900mhz and 1.5ghz i have 4 of them(and if i do not utilize them i will go insane) i actually think the motherboards are exactly the same in atleast two of them and all of the motherboards are from the same manufacture. so hardware shouldnt be issue here.
    Linux can be relatively insensitive to differences in hardware, for some things. Fortunately for you, this includes IDE drives, a basic console interface, and usually the ethernet card--as long as all the necessary modules are loaded. Unfortunately, the size of these modules are such that you have to manually pick and choose exactly which ethernet modules you want to load out if you're net-booting. This isn't a problem if all of the ethernet cards are identical.

    However, I can't see any advantages to net-booting. It's more of a headache to set up and maintain, with only the most puny of disk space savings.

    i mean it just seems silly to waste all of that space on things i just dont need, the kernel could just be loaded at boot time into memory. if the machine acts up, hit reboot.
    With individual installs, when a machine acts up, just hit reboot--and then the machine actually reboots with the same static IP address as before. Thus the same nfs share location. Thus its files end up in the same location as before.

    i feel like there has to be a simple solution besides cluster suits and cluster distros. they are way overkill for what i want. i want to just add it to my already working set up with out to much hassel and adding minmal things to run it.
    If you want minimum hassle, with less than a dozen computers, then avoid net-booting.

    Just install Debian Stable on one machine, and set up nfs as per any number of simple tutorials on nfs with Debian. After you get it working to your satisfaction, simply copy that OS partition to the other three computers, and manually set the static IP address for each.

    This is the ABSOLUTE MINIMUM effort solution you can come up with. Even if you manage to get a netbooting "cluster" solution working, you WILL have to individually configure a unique IP address for each machine (unless you LIKE having your files arbitrarily move from one share to another upon reboots).
    Isaac Kuo, ICQ 29055726 or Yahoo mechdan

  10. #9
    Just Joined!
    Join Date
    May 2006
    Posts
    9
    Quote Originally Posted by IsaacKuo
    If you install Debian Stable on each machine, you will NEVER have to mess with them after the install and initial configuration of nfs shares. Better yet, just do one Debian Stable install, and simply copy the OS partition for the other systems. Then, the only post-configuration you have to deal with is manually changing the static IP address in /etc/network/interfaces. This assumes you want the same basic partition structure for each machine. If not, then you'll have to manually edit the /etc/fstab and /etc/exports also.

    I really don't see how you can make things easier with net-booting. The biggest problem with net-booting is the headache of figuring out a way to give each machine its own static IP. Also, if there's any sort of configuration which needs to be done differently on any machine, then you've got some more big headaches.

    Net-booting multiple machines really only works well when you don't care which one is which. For example, in a cluster computing solution, the individual nodes are completely interchangeable. For your application, though, the identity of each node does matter because you want to be able to locate your files in the same place every time.



    A Debian install will cost you less than half a gig.



    Linux can be relatively insensitive to differences in hardware, for some things. Fortunately for you, this includes IDE drives, a basic console interface, and usually the ethernet card--as long as all the necessary modules are loaded. Unfortunately, the size of these modules are such that you have to manually pick and choose exactly which ethernet modules you want to load out if you're net-booting. This isn't a problem if all of the ethernet cards are identical.

    However, I can't see any advantages to net-booting. It's more of a headache to set up and maintain, with only the most puny of disk space savings.



    With individual installs, when a machine acts up, just hit reboot--and then the machine actually reboots with the same static IP address as before. Thus the same nfs share location. Thus its files end up in the same location as before.



    If you want minimum hassle, with less than a dozen computers, then avoid net-booting.

    Just install Debian Stable on one machine, and set up nfs as per any number of simple tutorials on nfs with Debian. After you get it working to your satisfaction, simply copy that OS partition to the other three computers, and manually set the static IP address for each.

    This is the ABSOLUTE MINIMUM effort solution you can come up with. Even if you manage to get a netbooting "cluster" solution working, you WILL have to individually configure a unique IP address for each machine (unless you LIKE having your files arbitrarily move from one share to another upon reboots).
    yeah i am finding out that yes, netbooting is a nightmare, i was actually just playing with "fai". to many configuration files all over the place. seems kind of silly.

    it seems i will not get away with this with out some configuration, i think this is starting to become a pride thing. i know it seems all so very silly. but hey, i try to make things happen the way i want them to go.

    i think the thing i am not conveying is that, i want to set this up so that you plug a system into the network, boot it up, and have the other drives auto-mounted to the masternode. if the files are here or there is not important. the master-node would autoname the file-nodes as they boot. since the system is so small booting 4 file-nodes one at a time, would really not make things to confusing.

    mostly the files are all media, mostly wav files. i do losts and lots of audio production, its just i would not rather fight with my systems just to get the files over to my work station to mix a single track.

    i have about 12-15 other hard drives not even hooked up, back in waiting, i want to beable to shut down a node, take out all of the hard drives, and load up the ones i want.

    i may just have to read as much as i can about each thing i want it to do, and write it from scratch. im sure it wouldnt take to much effort. it just seems crazy that no one has thought of doing this. or maybe they have and i just havnt read it. i mean knoppix has a thing going on where it boots via network(it works well)but diskless clusters are pointless in this situation.

    but honestly, thanx for the ideas. the more i think about it the more i can see how hard things are going to be. pride is a *****.

  11. #10
    Linux User IsaacKuo's Avatar
    Join Date
    Feb 2005
    Location
    Baton Rouge, LA, USA
    Posts
    406
    Hmm...forgive me if you think this solution is too easy...but have you considered using a set of USB enclosures/adapters? That turns any IDE hard drive into a hot-pluggable removeable drive. It's fast enough to keep up with 100BaseT. The challenge would be in transparently making hot-plugged USB drives shared over the network.

    But I guess you want to do net-boot as a matter of pride. Maybe this tutorial I once found/bookmarked could help you: http://homex.subnet.at/~max/diskless/index.php
    Isaac Kuo, ICQ 29055726 or Yahoo mechdan

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •