mv overwrite doesn't free hard drive space

[ianjmarsh]

I have just overwritten an 11GB text file with a 10MB file using the mv command i.e...

file_a = 11GB
file_b = 10MB

mv file_b file_a

However no extra space has been freed up on the hard drive, guessing I should have used... rm file_a ...first before the move command but what's done is done!

Anyone know how that space can be released or if it happens at a particular time e.g. on restart etc? I haven't tried restarting yet as its on a live server so would prefer not to take down our site unless I know this will definitely work.

Thanks.

[Rubberman]

What makes you think it hasn't been freed up? Question: was the original file_a a real file, a hard link, or a soft link?

[ianjmarsh]

Hi, thanks for the response.

I checked the disk space using df -h before I performed the move and again afterwards and no difference was seen. I was obviously expecting the file system to increase in free space by about 11GB.

Its a log file that holds lines of text characters. As it had grown to such a large size, and I only really need the latest entries in it, I wrote the last 10,000 lines of it into another file and moved this new file to overwrite the old file. (I know there are better options for keeping only the latest information!)

[Cabhan]

Are you certain that the original file_a was actually a file? In Linux, you can have things that look like files that are actually called soft/symbolic links. These provide an entry in the filesystem that looks like a file, but actually simply refers to another file. In which case you haven't actually deleted anything, but simply removed a reference to another file.

[ianjmarsh]

Sorry, I didn't answer that clearly last time.

Unless there's some other method that I'm not aware of to see if a file is a soft or hard link... at the time using ls -l and ls -F showed no symbolic link path, denoted by an @ sign, or any other detail to make me think otherwise, which led me to believe I could replace it how I did.

[Cabhan]

Alrighty then.

Well, this is kind of weird. There is a facility called a hard link where two names in the filesystem are actually are the same file (there's no way of distinguishing the two different names), and the file isn't actually deleted until all hard links are. However, hard links are relatively rare these days.

Because of that, I'm not sure what to tell ya. Maybe someone else will come along who has an idea.

[ianjmarsh]

Does seem odd doesn't it! Luckily I'm not pressed for space on that server but I wouldn't mind my 11GB back if anyone else has any ideas!

Thanks for the interest and responses though.

[Rubberman]

The thing about hard links is that they have to be to a file or other hard link in the same file system. Besides the directory entry, each link will share the inode, and each will increment the inode reference count. Copying to a hard link will keep the link, and all links will contain the same copied data. Moving a file to replace a hard link will unlink the file, and rename the moved file to the old name. This does not affect the data in the other links to the original data. Example:

ln fileA fileB

These two files will share an inode and the inode reference count will be incremented.

cp fileC fileA

This has copied the data from fileC to fileA, and since fileB is hard-linked to fileA, it also contains the same data as fileA which is the data copied from fileC.

mv fileD fileA

This will unlink fileA, decrementing the inode count of hard linked fileB by one, and then rename fileD to fileA but it's inode will be left untouched. If fileD is in another file system, then it will create a new fileA and copy the data to that before unlinking fileD from the other file system. When the reference count of a file's inode reaches 0, only then will the operating system physically remove the file. This is why I think you have moved your data over a hard linked file, and the original data is still on the disc somewhere. You should be able to use the find command to locate files of the appropriate size which can be subsequently examined and removed if it is the correct one.

[HROAdmin26]

Google: linux lsof disk space

[Rubberman]

Google: linux lsof disk space

Good point. Open files also increment the inode reference/link count, keeping them from being physically deleted even if there are no directory entries for them. Sorry, brain fart. Didn't even consider that possibility... doh! Anyway, using the lsof command as shown in the linked document will help determine this possibility. If other tools cannot find where the data is (such as the 'du' command), then one has to consider this scenario.