Find the answer to your Linux question:
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 15
Hi Linux gurrus hope you all will be doing well. Well I am facing a problem first time in my sys admin experience. I am using RHEL-4, I am unable ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Jan 2008
    Posts
    49

    /boot is 100% full


    Hi Linux gurrus

    hope you all will be doing well.

    Well I am facing a problem first time in my sys admin experience. I am using RHEL-4, I am unable to access /boot partition. when i try to access this partion or want to show its contents, It seems /boot is corrupted. see the following outputs

    #ls /boot/
    9?^?????tdܰ??????4?j?

    #df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/mapper/VolGroup00-LogVol00
    145G 98G 40G 72% /
    /dev/sda1 1.4T 1.4T 0 100% /boot
    none 501M 0 501M 0% /dev/shm

    Note: its showing the wrong entries /dev/sda1 (1.4T) and its 100% full.

    #fdisk -l

    Disk /dev/sda: 80.0 GB, 80000000000 bytes
    255 heads, 63 sectors/track, 9726 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot Start End Blocks Id System
    /dev/sda1 * 1 13 104391 83 Linux
    /dev/sda2 14 9726 78019672+ 8e Linux LVM

    Disk /dev/sdb: 80.0 GB, 80000000000 bytes
    255 heads, 63 sectors/track, 9726 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot Start End Blocks Id System
    /dev/sdb1 * 1 9726 78124063+ 8e Linux LVM

    Can any body help me whats wrong with /boot? Number of applications are being run (24x7) on this server. I cant reboot it and cant interrupt the running applications. Can any body tell me what should i do now?

    Server's info:

    #cat /etc/redhat-release
    Red Hat Enterprise Linux ES release 4 (Nahant Update 5)

    # uname -r
    2.6.9-11.ELsmp

    waiting for an early response.

    Thanks
    Sohail

  2. #2
    Linux Engineer
    Join Date
    Nov 2004
    Location
    home
    Posts
    796
    Have you fscked it? (I'm not trying to make a joke here) Perhaps, if unable to fix the fs, you might have to recreate the partition/fs, install the kernel and boot loader on it manually to fix it. If you don't have another copy of your current kernel or config, you will have to build another one or temporarily use the default until another can be built. Luckily it's only /boot and not something non replacable.

  3. #3
    Just Joined!
    Join Date
    Jan 2008
    Posts
    49
    Thanks for the reply.
    No, I didnt check the fsck. I just wanted to discuss before doing any thing because this is the most important server cant afford to loose the running applications.
    Can you please help me out? do tell me step wise what should i do now? from where to troubleshoot?
    1- fsck /dev/sda1 ??
    2- can i recover /boot folder with all its contents? As i have backup of grub.conf and other imp boot files in the /boot folder no where to go?
    I have other RHEL4 servers but as i joined new here, dont know are they have same boot setup or not?
    thanks for you time and waiting for your help.

    Sam

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Newbie sdimhoff's Avatar
    Join Date
    Jan 2007
    Posts
    191
    1.) I think that looking at what fsck throws back at you is the first step.

    The "good" thing about your problem is that there shouldn't be anything in the boot partition that is irreplaceable or should be necessary for any of the currently running applications (I assume that most of the time it isn't even mounted.). Additional note, if you just want to see what fsck would output you can give it the N flag. fsck -n (unless it is reiserfs or minx)

    2.) If fsck fixes the problem then great, you are done (although you should check files to be sure that fixing the drive errors didn't screw up any data). If not, then I think Valan was on the right track. You may be forced to reformat the /boot partition then reconfigure the bootloader and copy over your kernel.

    If worst comes to worst, the next time the server restarts, it won't boot. In that case you can just use a liveCD to fix it.

  6. #5
    Just Joined!
    Join Date
    Jan 2008
    Posts
    49
    # fsck -n /boot
    fsck 1.35 (28-Feb-2004)
    fsck.ext3: Unable to resolve 'LABEL=/boot'

    Now?

  7. #6
    Just Joined!
    Join Date
    Jan 2008
    Posts
    49
    can any of you tell me step-by-step what can i know? or can recommend any tutorial/pdf for this issue? I know if i rebooted the server it wouldnt boot it up

  8. #7
    Administrator MikeTbob's Avatar
    Join Date
    Apr 2006
    Location
    Texas
    Posts
    7,864
    I think you need to use the device point and not the mount point.
    fsck -n /dev/sda1
    EDIT: The best info on fsck is the man pages, open a terminal window and type this command
    man fsck
    Try Google also.
    fsck.ext3 - Google Search
    Last edited by MikeTbob; 01-22-2008 at 02:22 PM. Reason: More NFO
    I do not respond to private messages asking for Linux help, Please keep it on the forums only.
    All new users please read this.** Forum FAQS. ** Adopt an unanswered post.

    I'd rather be lost at the lake than found at home.

  9. #8
    Just Joined!
    Join Date
    Jan 2008
    Posts
    49
    # fsck -n /dev/sda1
    fsck 1.35 (28-Feb-2004)
    e2fsck 1.35 (28-Feb-2004)
    Warning! /dev/sda1 is mounted.
    Couldn't find ext2 superblock, trying backup blocks...
    Superblock has a bad ext3 journal (inode .
    Clear? no

    fsck.ext2: Illegal inode number while checking ext3 journal for /boot


    Now what next ??

  10. #9
    Administrator MikeTbob's Avatar
    Join Date
    Apr 2006
    Location
    Texas
    Posts
    7,864
    I would try unmounting the drive, then try fsck again, if you get the same errors, then maybe it's time to think that it really is trashed and needs to be rebuilt. I am no expert, so you need to use your best judgement here.
    I do not respond to private messages asking for Linux help, Please keep it on the forums only.
    All new users please read this.** Forum FAQS. ** Adopt an unanswered post.

    I'd rather be lost at the lake than found at home.

  11. #10
    Linux Newbie sdimhoff's Avatar
    Join Date
    Jan 2007
    Posts
    191
    Sorry I wasn't specific on the commands you needed, but it looks like Mike clarified things.

    Additional Advice:

    From what you've said, I think you were right in your estimation that the server will NOT reboot if you tried now. The system will run fine until the next reboot (unless you have a failing hard drive) therefore you should plan out your attack and notify your users BEFORE the downtime will begin.

    It also seems like you don't really know what sort of plan of attack you should do so I've laid out what I would do. I want to be clear that the following is simply what I would do, and that you have to be ready to deal with the consequences if this doesn't work out. (Of course, you can always post new problems here along the way!)

    1.) Do the fsck check but let it fix the errors.
    umount /boot
    fsck /dev/sda1

    2.) IF the fsck continues to fail then you have to be sure that no users are trying to access your files, because this will require a reboot AFTER re-making the boot partition.

    3.) Reformat /dev/sda1 (this will wipe the boot partition, so you should be sure it is what you want to do.)

    4.) Mount the new boot partition and reconfigure the bootloader and copy over your kernel. (I'm sure there is documentation for RHEL server)

    5.) Be sure that your bootloader is pointed to your newly copied over kernel, and only after you are either sure that things are correct, or your users know there will be downtime can you reboot.

    6.) If you have configured your bootloader incorrectly and have rebooted, you can always boot to a liveCD to fix it.

    7.) If problems continue to occur then this may be a symptom of a harddrive failure.

    Good luck and remember that you have plenty of time to fix this (the system is currently running and shouldn't need the boot partition until reboot). It is always difficult to fix problems when a server is needed by many people, so you have to plan and notify your users if there will be downtime.

    __

    Also, if any other people spy an error in what I've written then please feel free to correct me.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •