Find the answer to your Linux question:
Results 1 to 4 of 4
Hello, Perhaps an odd question: I would like to rate limit a device and haven't found a neat (or any, to be honest) way of doing it. The background to ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Oct 2011
    Posts
    2

    Rate limiting Linux devices


    Hello,

    Perhaps an odd question: I would like to rate limit a device and haven't found a neat (or any, to be honest) way of doing it.

    The background to this is that I'm currently doing some academic research into grid computing, specifically experimenting with some modifications to the MapReduce algorithm (see the Apache project).

    Ideally, I want to simulate a more pronounced difference between HDD speed of the cluster nodes and their CPU capability. So, if I've got a HDD partition, say, /dev/sdb2, that contains the application data, I'd like to rate limit this in a way that doesn't effect the other partitions (e.g. that store the O/S etc.) and is completely transparent to the code (Java mostly) that reads the data from it.

    The cluster setup is Fedora14 (can be changed, but would rather keep it the same if possible).

    Many thanks in advance!

    Russ.

  2. #2
    Linux Newbie nplusplus's Avatar
    Join Date
    Apr 2010
    Location
    Charlotte, NC, USA
    Posts
    106
    Hi, Russ,

    My only suggestion would be to mount a network storage device as your app partition and rate limit the network communications.

    N

  3. #3
    Linux Newbie reginaldperrin's Avatar
    Join Date
    Oct 2010
    Location
    Christchurch, New Zealand
    Posts
    123
    Unless controlled by software, devices will simply work to the limits that their hardware allows.
    Rate-limiting a device usually happens by some sort of software.
    In the case of a network connection, for example, the software would do something like: count up to 1000 bits passing through, then drop the bits until time >= 1 second, then reset the time and start allowing bits to pass through again, counting up to 1000 bits.

    All of this requires some overhead of a program running at the same time, so if you are looking at benchmarking a system's performance, this will inevitably skew the results.

    You might be better off altogether by using some sort of traffic generator software which will emulate the HDD cluster nodes and send bits through the device at a predetermined rate. The device will be operating to it's best abilities, but the rate will be whatever you want it to be. You might want to investigate the mobo's manufacturers stated performance of the internal bus, the cables which connect the HDD cluster nodes, the read/write speed of the HDDs themselves.
    The lowest value will be the maximum speed at which your system can operate, and you may be able to use this speed in a traffic generator.

    Hope this helps.

  4. #4
    Just Joined!
    Join Date
    Oct 2011
    Posts
    2

    Rate Limiting Devices

    Quote Originally Posted by reginaldperrin View Post
    Unless controlled by software, devices will simply work to the limits that their hardware allows.
    Rate-limiting a device usually happens by some sort of software.
    In the case of a network connection, for example, the software would do something like: count up to 1000 bits passing through, then drop the bits until time >= 1 second, then reset the time and start allowing bits to pass through again, counting up to 1000 bits.

    All of this requires some overhead of a program running at the same time, so if you are looking at benchmarking a system's performance, this will inevitably skew the results.

    You might be better off altogether by using some sort of traffic generator software which will emulate the HDD cluster nodes and send bits through the device at a predetermined rate. The device will be operating to it's best abilities, but the rate will be whatever you want it to be. You might want to investigate the mobo's manufacturers stated performance of the internal bus, the cables which connect the HDD cluster nodes, the read/write speed of the HDDs themselves.
    The lowest value will be the maximum speed at which your system can operate, and you may be able to use this speed in a traffic generator.

    Hope this helps.
    Hi All,

    Thanks a lot for the advice!

    Regards,
    Russ.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •