Find the answer to your Linux question:
Results 1 to 2 of 2
To save you a long story my college math department has the opportunity to apply to a $100,000 grant I suggested they use it to make a linux cluster computer ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Apr 2012
    Posts
    10

    $100,000 cluster computer


    To save you a long story my college math department has the opportunity to apply to a $100,000 grant I suggested they use it to make a linux cluster computer they really liked that idea and are going to use it... but they know nothing about building computers I really want us to get one and thus I need to flesh out what exactly a $100,000 cluster computer would be. I have experience with building desktop rigs but obviously this is a whole new animal!

    I've talked with the IT department and there is room and power for around 100 1U server chassis, so all the money should be able to go directly into the actual computer.
    From my research it seems that 5 to 10 teraflops is a reasonable ballpark figure for the computer ($10-$20 per gigaflop).

    My main questions to anyone who has experience with cluster computing is:
    - what's a good balance in regard to number of nodes vs quality of each node? Purely by the numbers hooking up 2,000 raspberry-pi's (claimed 24 gigaflops a piece) should yield ~50 teraflops, which is way more than this budget is expected to provide! ... but common sense would suggest that's probably not a good idea.
    - is it practical to try to do GPU accelerated processing at this level? (the cheapest purpose built ones by NVIDIA start at $2000 a piece, 1/50th total budget per card). And if so is a beefy CPU beneficial or does it just act as a controller for the GPU and thus a waste of money?
    - Is gigabit ethernet fast enough to not be a concern or depending on the nodes will it be a bottle neck and therefore fiber could be needed?
    - And probably the most important of all: is it stupid to try to do a DIY build on this scale and really buying a professional prebuilt system is a much smarter move?

    Sorry to ask such basic questions, usually the web has tons to offer and I don't waste time posting things that have been answered somewhere already but I haven't found any relative to a cluster computer of this budget.

    Thanks,
    code_ape

  2. #2
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, or in a galaxy far, far away.
    Posts
    11,175
    Quote Originally Posted by code_ape View Post
    To save you a long story my college math department has the opportunity to apply to a $100,000 grant I suggested they use it to make a linux cluster computer they really liked that idea and are going to use it... but they know nothing about building computers I really want us to get one and thus I need to flesh out what exactly a $100,000 cluster computer would be. I have experience with building desktop rigs but obviously this is a whole new animal!

    I've talked with the IT department and there is room and power for around 100 1U server chassis, so all the money should be able to go directly into the actual computer.
    From my research it seems that 5 to 10 teraflops is a reasonable ballpark figure for the computer ($10-$20 per gigaflop).

    My main questions to anyone who has experience with cluster computing is:
    - what's a good balance in regard to number of nodes vs quality of each node? Purely by the numbers hooking up 2,000 raspberry-pi's (claimed 24 gigaflops a piece) should yield ~50 teraflops, which is way more than this budget is expected to provide! ... but common sense would suggest that's probably not a good idea.
    - is it practical to try to do GPU accelerated processing at this level? (the cheapest purpose built ones by NVIDIA start at $2000 a piece, 1/50th total budget per card). And if so is a beefy CPU beneficial or does it just act as a controller for the GPU and thus a waste of money?
    - Is gigabit ethernet fast enough to not be a concern or depending on the nodes will it be a bottle neck and therefore fiber could be needed?
    - And probably the most important of all: is it stupid to try to do a DIY build on this scale and really buying a professional prebuilt system is a much smarter move?

    Sorry to ask such basic questions, usually the web has tons to offer and I don't waste time posting things that have been answered somewhere already but I haven't found any relative to a cluster computer of this budget.

    Thanks,
    code_ape
    Great project!

    GPUs == awesome floating point processing.
    CPUs/cores == better general purpose processing

    Try to find a balance between FP (GPU) and GP (CPU) processing needs. Where that balance will lay is dependent a lot upon the sort of processing load you are going to run. As for nVidia GPU gear, don't go with the purpose-built ones. Commodity cards have a LOT of FPU capability at a very low price-point ($100 USD for a card with 256+ FPUs and a gig or two of RAM).

    CPU<->Memory<->bus I/O will be your biggest bottleneck. This is an issue you will want to investigate as it is going to be a major gating factor in how well your cluster performs overall.

    Anyway, have fun!
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •