append files, or low level 'cat'

[Stooofer]

Hi all
Everything is a first once, and here is my first post.

I'm curious about the low level functionality of cat. The background is that I have a script that calibrates datafiles and after calibration appends them total calibrated file.
The problem is that this collected file quickly grows and easily reach >500Mb, when running the append command
> cat COLLECTED_FILE NEW_FILE > NEW_COLLECTED_FILE
I can go and have a cuppa because the computer completely grinds to a halt for a while.

My understanding is that cat reads in the old file, the new file and then append them thus there is a lot of reading and writing when the files grow.
Is there a way to speed up this process, or do I have to do these things at night?

Thanks

[vsemaska]

Just append to the existing file.

# cat NEW_FILE >> COLLECTED_FILE

[i92guboj]

Yep. That way you avoid having to read one of the files.

Also, you can use "nice" to set up a better niceness for that process so it's softer in terms of cpu. If udma is on, your box shouldn't be irresponsive when doing this though. If so, then there's some problem at some level.

Using a sane fs (read, anything but reiserfs) will probably also save you some cpu cycles.

[wje_lf]

Just append to the existing file.

Faster, but not recommended if you're worried about the system or the process dying before the appending is finished. It depends on how bulletproof this situation needs to be.

[Stooofer]

*thump*, the sound of my head against the desk...

I knew there was a better way! Thanks a lot.