Software RAID-1 way too slow

[jakobwenzel]

Hi there!

I am currently running Debian Squeeze on a headless system mainly used for backup.
It has a RAID-1 made up of two 1TB SATA-Disks which can transfer about 100 MB/s reading and writing. Yesterday I noticed that one of the disks was missing from the RAID configuration. After re-adding the drive and doing a rebuild (ran with 80-100MB/s) I noticed massive speed issues.
Running "time cat largefile > /dev/null" on a 1GB file takes about one minute with this output:

real	0m56.347s
user	0m0.024s
sys	0m1.896s

So I have a reading speed of 17 MB/s, far below of what it should be.

Running cat /proc/mdstat gives:

Personalities : [raid1] 
md0 : active raid1 sda1[0] sdb1[1]
      976759936 blocks [2/2] [UU]
      
unused devices: <none>

mdadm --detail /dev/md0 gives:

        Version : 0.90
  Creation Time : Tue Jul 28 17:08:01 2009
     Raid Level : raid1
     Array Size : 976759936 (931.51 GiB 1000.20 GB)
  Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Mon Mar 22 16:22:13 2010
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           UUID : 426f1bd3:2d1b0f31:2ce552e4:6d63ea58
         Events : 0.276751

    Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1
       1       8       17        1      active sync   /dev/sdb1

Do You have any idea what is slowing down the system and how I can bring it back up to normal speed?

Thanks in advance,
jakobwenzel

[jakobwenzel]

Yes, i meant rebuild.
I am wondering right now why I didn't write rebuild, changed that up there...

[HROAdmin26]

What do these results show?

hdparm -tT /dev/sda1

hdparm -tT /dev/sdb1

* Don't use any other switches with hdparm - data loss can occur.

[jakobwenzel]

/dev/sda1:
 Timing cached reads:   1190 MB in  2.00 seconds = 595.02 MB/sec
 Timing buffered disk reads:  308 MB in  3.00 seconds = 102.65 MB/sec
/dev/sdb1:
 Timing cached reads:     2 MB in  2.15 seconds = 954.32 kB/sec
 Timing buffered disk reads:  290 MB in  3.01 seconds =  96.45 MB/sec

After I couldn't believe the cached reads for sdb1, i reran the test:

/dev/sdb1:
 Timing cached reads:   1176 MB in  2.00 seconds = 587.49 MB/sec
 Timing buffered disk reads:   86 MB in  3.01 seconds =  28.61 MB/sec

Strange again…

[HROAdmin26]

No, not strange at all. Unless this disk was under some heavy load, there's not much reason for inconsistent numbers like this. You said the disk "disappeared" from the RAID set.

Thought process after finding drive missing from RAID:

Wrong: "Oh wow, drive's missing...whoops. OK, let me just add it back in and assume everything's working great."

Right: "Uh-oh, drives don't disappear from a RAID set unless something bad happened. Maybe I should take a look at the *logs* and assume something malfunctioned."

You need to look at the messages log, smartctl, etc. There is likely a problem with the drive, controller, or cabling (in that order.)

[mande01]

Sorry for waking this old thread up but I have a very similar issue except no hardware issue to note.

Can you provide some guidance as to what i should be looking for in the log files?
smartctl is not reporting anything as for some reason, I cannot access the smart data. Any suggestion on that one.

When I get home this evening i'll report my output as above, and i'll plug the drives into a different machine to try and get the smart data.

My box is a little netgear stora (Media NAS box with linux raid-1)

Thanks,
Derry

[oz]

Hello and welcome!

We close old threads that are brought back to the top, but please feel free to start a fresh thread of your own with the details if you are having any issues with Linux.

Thank you.