File-system corruption caused by expanding file-size

[ashish.narmen]

I experienced this problem today while using VMWare Workstation 6.0 on Suse 10.2. The exact details of the software are as follows

VMWare Workstation : 6.0.1 build-55017
Host: Suse Linux: 2.6.18.2-34-default #1 SMP Mon Nov 27 11:46:27 UTC 2006 x86_64 x86_64 x86_64 GNU/Linux

I started off to create a virtual machine for the Solaris Express Developer Edition. The default option for the virtual hard disk was "Split disk into 2 GB files". However, I unchecked that option such the virtual hard disk would dynamically expand.


Guest: Solaris Express Developer Edition


During the course of installation (which failed), the virtual machine rebooted in a semi-installed state and the vm displayed the grub. Also, the entire GNOME desktop was painfully slow and simple actions were taking a lot of time. (The size of the vmdisk had expanded to about 4 GB by this time)


Upon rebooting, I was not able to log in to my user account. After a couple of failed attempts, I had to login using root. However, the default location of the virtual machine /home/username/vmware was accessible using read-only mode and I was unable to delete the Solaris Express virtual machine.


Another attempt to reboot and login in FailSafe mode did not mount that partition and repeated attempts to load the partition using the mount command were unsuccessful. After that I had to initiate a clean up of the file system using fsck which was able to detect and correct errors. Then I was able to mount the partition and delete the error causing file after which I was able to


I concur that the problem was caused by the file size that had grown beyond 4 gigs. But according to what I have read, ext3 supports greater file sizes too. Also, if this was the cause of the issue why did the kernel not shutdown the errant process (vmware) which was trying to write a file greater than the max allowed size. If not, does vmware run under the context of root due to which it was able to continue writing to the vmdk file causing the file-system to crash.


Looking forward to a response which would help me in drilling down to the root cause of this issue.


/etc/fstab listing

/dev/hda1 / ext3 acl,user_xattr 1 1
/dev/hda3 /home ext3 acl,user_xattr 1 2
/dev/hda5 swap swap defaults 0 0
proc /proc proc defaults 0 0
sysfs /sys sysfs noauto 0 0
debugfs /sys/kernel/debug debugfs noauto 0 0
usbfs /proc/bus/usb usbfs noauto 0 0
devpts /dev/pts devpts mode=0620,gid=5 0 0
/dev/fd0 /media/floppy auto noauto,user,sync 0 0
/dev/VolGroup00/LogVol00 /home/ashin/data ext3 defaults 1 2

Log file snippet
Sep 30 12:09:59 ashin-suse kernel: EXT3-fs error (device hda3): ext3_free_blocks: Freeing blocks in system zones - Block = 33992, count = 1
Sep 30 12:09:59 ashin-suse kernel: Aborting journal on device hda3.
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_free_blocks_sb: Journal has aborted
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3): ext3_free_blocks: Freeing blocks in system zones - Block = 1180160, count = 1
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_free_blocks_sb: Journal has aborted
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_free_blocks_sb: Journal has aborted
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3): ext3_free_blocks: Freeing blocks not in datazone - block = 368853200, count = 1
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3): ext3_free_blocks: Freeing blocks not in datazone - block = 4294934784, count = 1
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_free_blocks_sb: Journal has aborted
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_free_blocks_sb: Journal has aborted
Sep 30 12:10:00 ashin-suse kernel: ext3_abort called.
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3): ext3_journal_start_sb: Detected aborted journal
Sep 30 12:10:00 ashin-suse kernel: Remounting filesystem read-only
Sep 30 12:10:00 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_free_blocks_sb: Journal has aborted
.
.
.
Sep 30 12:10:05 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_reserve_inode_write: Journal has aborted
Sep 30 12:10:05 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_truncate: Journal has aborted
Sep 30 12:10:05 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_reserve_inode_write: Journal has aborted
Sep 30 12:10:05 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_orphan_del: Journal has aborted
Sep 30 12:10:05 ashin-suse kernel: EXT3-fs error (device hda3) in ext3_reserve_inode_write: Journal has aborted
Sep 30 12:10:05 ashin-suse kernel: /dev/vmmon5125: host clock rate change request 19 -> 0
Sep 30 12:10:05 ashin-suse kernel: vmmon: Had to deallocate locked 127763 pages from vm driver ffff810007e38000
Sep 30 12:10:05 ashin-suse kernel: vmmon: Had to deallocate AWE 4937 pages from vm driver ffff810007e38000
Sep 30 12:10:05 ashin-suse kernel: __journal_remove_journal_head: freeing b_committed_data
Sep 30 12:10:05 ashin-suse kernel: __journal_remove_journal_head: freeing b_committed_data
....


.....


.........
Sep 30 12:10:10 ashin-suse kernel: vmmon: Had to deallocate AWE 16 pages from vm driver ffff810007e38000
Sep 30 12:11:44 ashin-suse gconfd (ashin-4251): Failed to write "/home/ashin/.gconf/apps/nautilus/preferences/%gconf.xml": Failed to open "/home/ashin/.gconf/apps/nautilus/preferences/%gconf.xml.new": Read-only file system
Sep 30 12:11:44 ashin-suse gconfd (ashin-4251): Failed to sync one or more sources: Failed to write some configuration data to disk
Sep 30 12:12:00 ashin-suse gconfd (ashin-4251): Failed to open saved state file: Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system)
Sep 30 12:12:00 ashin-suse gconfd (ashin-4251): Could not open saved state file '/home/ashin/.gconfd/saved_state.tmp' for writing: Read-only file system
Sep 30 12:12:00 ashin-suse gconfd (ashin-4251): Failed to write "/home/ashin/.gconf/apps/gnome-session/options/%gconf.xml": Failed to open "/home/ashin/.gconf/apps/gnome-session/options/%gconf.xml.new": Read-only file system
Sep 30 12:12:00 ashin-suse gconfd (ashin-4251): Failed to write "/home/ashin/.gconf/apps/nautilus/preferences/%gconf.xml": Failed to open "/home/ashin/.gconf/apps/nautilus/preferences/%gconf.xml.new": Read-only file system
Sep 30 12:12:00 ashin-suse gconfd (ashin-4251): Failed to sync one or more sources: Failed to write some configuration data to disk
Sep 30 12:12:04 ashin-suse gconfd (ashin-4251): Exiting
Sep 30 12:12:16 ashin-suse gdm3924: Couldn't authenticate user
Sep 30 12:12:49 ashin-suse gdm3924: run_session_child: Could not open ~/.xsession-errors
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): starting (version 2.14.0), pid 5569 user 'ashin'
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Resolved address "xml:readonly:/etc/opt/gnome/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Resolved address "xml:readwrite:/home/ashin/.gconf" to a read-only configuration source at position 1
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Resolved address "xml:readonly:/etc/opt/gnome/gconf/gconf.xml.defaults" to a read-only configuration source at position 2
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Resolved address "xml:readonly:/etc/opt/gnome/gconf/gconf.xml.schemas" to a read-only configuration source at position 3
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): None of the resolved addresses are writable; saving configuration settings will not be possible
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): No writable config sources successfully resolved, may not be able to save some configuration changes
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Failed to open saved state file: Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system)
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Failed to open saved state file: Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system)
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Failed to open saved state file: Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system)
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Failed to open saved state file: Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system)
Sep 30 12:13:21 ashin-suse gconfd (ashin-5569): Failed to open saved state file: Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system)
Sep 30 12:13:22 ashin-suse gconfd (ashin-5569): Failed to open saved state file: Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system)
Sep 30 12:13:22 ashin-suse gconfd (ashin-5569): Failed to log addition of listener gnome-session (Failed: Failed to open gconfd logfile; won't be able to restore listeners after gconfd shutdown (Read-only file system));will not be able to restore this listener on gconfd restart, resulting in unreliable notification of configuration changes.

[gogalthorp]

1) Are you running 64bit Linux?
2) If the VM interface crashes or you simply close the window the VM continues to run. You must either suspend or shutdown the VM to stop it.
3) what processes are you running in the VM that might cause expansion?

I never tried dynamic Virtual disks I like to have control.
I also have a Special partition for my VM's. This keeps things separate and neat.

This might be a bug in VMware. Have you check on the VMware boards?

[ashish.narmen]

Hi
1. Yes, I am running 64 bit Linux
2. The VM interface did not crash but the VM rebooted during the installation process. (I am referring to the OS installation on the virtual machine over here).
3. This might be a bug in VMWare too. I have posted this query on the vmware forums but did not get a reply as of now. (VMware Communities: File-system corruption caused by ...)

Initially, I created a virtual machine and unselected the split into 2 gigs options as well as the allocate all disk space now option.
Then I started installing the Solaris Express Operating System on the newly created virtual machine. The file-size kept growing as the virtual machine operating system was getting installed.

I wanted to know the following
1. how to calculate the maximum file size that my file system will accommodate.
2. how to determine the security context in which the vm is running.

Thanks.