help: lvm partition disappeared

[mkaut]

Hello,

how do I re-create an LVM raid 1 partition, without destroying data on the discs?

The full story: I have a 650GB data partition which is a raid 1 array with ext3. Two days ago, the system (Ubuntu 9.04) started to refuse to write to it, claiming no space left on device - even if there is ca. 102GB free left, if the disk is 85% full (according to df). Interestingly, removing a couple of GB did not help, after reboot the disk was again full..
I did the "tune2fs -m 0" trick and then forced file check on next reboot by "sudo touch /forcefsck" .. and the result is that the raid partition is gone I have the two physical drives /dev/sd*, unmounted, but the /dev/md1 is no longer there. What can I do to re-create it, without losing the data?

Btw: I ran tune2fs on the physical partitions /dev/sd* - was I supposed to run it on /dev/md1?

Thanks a lot in advance.

Regards,
Michal

[ptkobe]

Please post
$ sudo fdisk -l

and
$ cat /proc/mdstat

(if necessary, will look into the LVM after)

also, what ubuntu version are you using.

I think the best way to analyze and fix this kind of problems is to use a Live CD (or have an OS on a pen)

If you use the live option from your ubuntu install CD, you may
Boot from the live CD and run:

$ sudo -s
# apt-get install mdadm lvm2
# mdadm --auto-detect
# lvchange -ay /dev/vg... (your logical volumes)

Btw: I ran tune2fs on the physical partitions /dev/sd* - was I supposed to run it on /dev/md1?

In you case, the filesystem resides on your lvm logical device.

So, if tunefs should be used at all (I use xfs, don't ask me why), i'll say it should be on the logical volume (/dev/mapper/vg-something).
Not on the raid, surely not on the sd devices.

Also see man tunefs:
"The tunefs utility cannot be run on an active file system. To change an active file system, it must be downgraded to read-only or unmounted."

Others will help on this better

Regards
Luis

[mkaut]

Hello Luis,

thanks for the reply, I will try this tonight (it is my home machine and I am at work now). The machine runs Ubuntu 9.04 x64.

But I am wondering: do I really need to use a live CD, if the partition contains only data (used to mounted as /data)?

Btw: is it safe to mount one of the physical partitions read only (and make a backup of the most important data), or can this create problems with creation of the raid partition (since the files will have different access-times on the two discs)?

Thanks,
Michal

[RDU]

If it's only a data disk, you can boot your system normaly, no need to use a live cd.
You can safely try to mount with ro option. Access time is not written to READONLY mounted part.
I agree that the fsck has to be done at the logical level (this is where the filesystem is), never on the physical layer under Raid or LVM.

[ptkobe]

Right.

Please post the request files after you got home

you can always run
# mdadm --auto-detect
it detetcs raid at kernel level. see man mdadm

if the necessary raid partition exist, but not the lv, try
# lvchange -ay /dev/vg... (your logical volumes)
to activate the volume

If no luck with the raid, you may try to assemble the array "by hand" with
# mdadm --assemble ...
see man mdadm --assemble --help

It's mostly a question if tunefs did something with the superblocks or not, I think.

Good luck
PS: You may also check your /etc/mdadm/mdadm.conf file

[mkaut]

I just came home, started the computer and the partition was back
Then I did "sudo touch /forcefsck" and restarted .. and it was gone again .. and after next restart (without file check) it was back

So it looks like the file check does something to the LVM/RAID setup, strange..

Anyway, the original problem of missing 100GB is still there, but I put it into a new thread: linuxforums.org/forum/ubuntu-help/169176-missing-15-disk-capacity.html

Thanks for the help.

Michal