Please help w/ mdadm - two disk failures

[jz_]

Hi everyone,

On Saturday I received an alert from our server setup with software RAID1 using mdadm.

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdb1[1] sda1[0]
3905472 blocks [2/2] [UU]

md1 : active raid1 sdb2[1] sda2[2](F)
1461231552 blocks [2/1] [_U]

I removed sda2 and then tried to rebuild it, but received a FailSpare event alert shortly after.

Last night I received another mail alerting of a failure, but this time on md0.

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdb1[1] sda1[2](F)
3905472 blocks [2/1] [_U]

md1 : active raid1 sda2[2](F) sdb2[1]
1461231552 blocks [2/1] [_U]

I removed both failed partitions using mdadm --remove /dev/md1 /dev/sda2 and
mdadm --remove /dev/md0 /dev/sda1. I am now in a degraded array state with cat /proc/mdadm showing

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active raid1 sda2[1]
1461231552 blocks [2/1] [_U]

md0 : active raid1 sda1[1]
3905472 blocks [2/1] [_U]

I am still able to log on to the server, but can anyone please help me rebuild these drives? I appreciate your help. Thanks.

[Rubberman]

Unfortunately, a lot of RAID systems experience multiple drive failures within a very short time frame since their discs are usually purchased/installed together from the same manufacturing batch. When that happens, you are SOL. Just one reason why I generally don't bother with RAID unless I manually install them and make sure that all discs are from different manufacturing batches. I know this doesn't help you a lot, but then there's not much I can do other than suggest you contact the RAID device manufacturer to see what they recommend.

[jz_]

I appreciate the response. Both disks were purchased from the same vendor at the same time and were from a bad batch.

[ptkobe]

1- What you mean 2 disks failures? I only see failures on sda.

2- If you --remove sdaX, why sdbX is not showing on the last /proc/mdstat (not /proc/mdadm, right?), but only the sdaX you did (maybe not) removed? Why is not sdbX showing?

The path is:
say on md0 -> sda2(F)

mdadm /dev/md0 --fail /dev/sda2
mdadm /dev/md0 --remove /dev/sda2

Now the array will be in degraded mode (not a biggy) showing only sdb partitions.

Remove faulty disk, buy a new one, connect it, create a new sda2 partition on that disk, and run
mdadm /dev/md0 --add /dev/sda2

After sync, it will be ok. So please check my first questions, and we may enter on some more detail and verify the statements, but that's the general idea, AFAIK.

Regards
Luis

PS: I would try to use the server as little as possible.
And be hopefull

[ptkobe]

... contact the RAID device manufacturer ...

I don't think there's a RAID device here.

To the OP: Is there? Do you have any RAID card or are using the motherboard RAID?