Find the answer to your Linux question:
Results 1 to 4 of 4
I have a pretty huge (16x1TB disks) RAID6 LVM array on a Red Hat box which has really thrown a spanner in its works and I'm in a right fix: ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Dec 2007
    Location
    Wellington, New Zealand
    Posts
    3

    RAID 6 LVM recovery from interrupted reconstruction


    I have a pretty huge (16x1TB disks) RAID6 LVM array on a Red Hat box which has really thrown a spanner in its works and I'm in a right fix: I was part-way through rebuilding the array with one replacement disk when a second died and a third seemed to have fallen from the array. I was replacing a disk which had been showing signs through SMART of probable failure (uncorrectable errors) and I noticed the second disk was just starting to give similar but slightly less numerous errors.

    Now my logic based on a gradually growing understanding of the workings of the LVM is that I stand a chance of resurrecting things as the data held on the machine has been static for a while (a few days) before my attempted drive replacement and none of the disks have actually FAILED. I have replaced the unit I removed so the physical devices are as they were before the exercise. I clearly need to carefully convince the configuration that it's "how it was" and there are a few meta-data files in the /etc/lvm/archive I believe I can use to re-"set the scene". Presently the machine fails to boot completely and it's patiently waiting in single-user mode for my next visit. Thankfully the whole office is on holiday until 10th January although it will be needed from then.

    I intend investigating as much as possible prior to any action to avoid sudden and catastrophic complete loss of data on the array but would VERY much like to have any guidance.


    Many thanks in advance,
    Rik

  2. #2
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,663
    Unfortunately, a lot of people think that RAIDs don't need to be backed up... In reality, they need to be backed up regularly just like normal discs/arrays (LVM). Their main benefit to the user is higher availability as they will continue to work even if one disc fails. Caveate User! Most arrays have discs from the same manufacturing batch, which means that when one fails, more are likely to follow suit. Result? Unrecoverable systems/data. Sorry, but I don't have a solution for you, other than to try to save your data to an external array/device/system(s). Not so helpfull, I know, but what is, is. So, wishing you a prosperous New Year, I remain unhelpfully yours...
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

  3. #3
    Just Joined!
    Join Date
    Dec 2007
    Location
    Wellington, New Zealand
    Posts
    3
    I appreciate your comments and regrettably acknowledge the vulnerability of even RAID 6 and consequential need for Proper Backups. However, I am trying to delve into the bowels of the existing system and need some guidance as to how best attempt recovery of what I believe is still available, albeit a tricky operation ... Logically I believe I can recover the data without the need to resort to forensics or a specialist recovery service.

    Alas this is clearly a significant Weak Link in the existing configuration and it has snapped in a particularly inconvenient manner but thankfully at a "quiet time"!


    Cheers for now,
    Rik.

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined!
    Join Date
    Dec 2007
    Location
    Wellington, New Zealand
    Posts
    3

    Solving the problem ...

    I managed to piece the array back together and will write up a full account of the steps presently.

    BIG PHEW.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •