Cron jobs and swapping memory

[vanderkerkoff]

Hello everyone

I'm using Ubuntu 8.04 server and I'm running into a problem.

I have a rake task that grabs alot of data out of a csv file and imports it into our mysql database. When I run the rake task from in the console the amount of memory needed to complete the task exceeds that available memory and swapping occurs to complete the task.

When I run this job from within cron, at the moment when swapping needs to occur to complete the task the task fails and the swapping does not occur.

It's as if I need to tell cron to allow memory swapping.

Has anyone seen anything like this before or have any ideas on how I tell cron to allow memory swapping to complete it's tasks?

All help is greatly appreciated.

[Lakshmipathi]

Has anyone seen anything like this before or have any ideas on how I tell cron to allow memory swapping to complete it's tasks?

That's quite unusual for me. I may be wrong,but here is my view:
I think memory activities are obviously handled by kernel.Kernel decides to use swap space,only when i ran out of primary memory (RAM)

In your case , It might be simple case of RAM has enough memory to execute the process ,so swapping is not used.

I would suggest you to check the process timing using
time command

man time

You can redirected the output from crontab to file and check the timing.

note:
Remember Swap and CPU time all depends on other processes too.

If you have 10 process already running and when you execute your program,clearly it will take more time and it may use swap.

If there is very few running process (say 2) then you mayn't need swap,processing time will quick.

[Rubberman]

Swapping is built into the OS. If your process is failing under cron, then I think you likely do not have enough swap space allocated. Have you looked into the cronlog and system (dmesg) log file(s) to see what happened? FWIW, cron has no relationship to swap space. If you ran out of swap, then the process will fail, simply enough.

ROT (Rule Of Thumb): calculate 1.5x the memory required by the OS, normally running processes, plus that required by any regularly scheduled process. If that exceeds the available RAM, then allocate sufficient swap space for up to 1.5x the available RAM or 1.5x the calculated space required - whichever is greater.

[b2bwild]

ROT (Rule Of Thumb): calculate 1.5x the memory required by the OS, normally running processes, plus that required by any regularly scheduled process. If that exceeds the available RAM, then allocate sufficient swap space for up to 1.5x the available RAM or 1.5x the calculated space required - whichever is greater.

Important thing about swap,
For Desktop, or Small server.
Swap should be minimum 32 MB and maximum 2GB.
You swap should be 2.0x of RAM, if your RAM is less than 512.
Your swap should be 1.5x of RAM, if your RAM is less than or equal to 1GB
Your swap should be 1x of RAM, if ram is <= 2GB
If RAM is more than 2GB, Swap file should go decreasing.

Swap will be used when you are out of RAM,
While you have enough space on your RAM, swapping is an idiocy.
You don't need cron to swap memory.
Swapping everything on swap will affect system stability.

Swapping is controlled by swappiness.

cat /proc/sys/vm/swappiness

If you want to swap memory more increase swappiness.

echo 100 > /proc/sys/vm/swappiness

If you don't want memory to be swapped.

echo 0 > /proc/sys/vm/swappiness

Increasing swappiness will put more load on CPU.

[Lakshmipathi]

I tried to re-write it. are these general rules ? Let me know whether is this correct?

You swap should be 2.0x of RAM, if your RAM is less than 512.

if(RAM_SIZE <=512MB)
swap = 2 x RAM_SIZE

Your swap should be 1.5x of RAM, if your RAM is less than or equal to 1GB

if (RAM_SIZE > 512MB && RAM_SIZE <=1GB
swap = 1.5 x RAM_SIZE

Your swap should be 1x of RAM, if ram is <= 2GB

if (RAM_SIZE > 1GB && RAM_SIZE <=2GB )
swap = RAM_SIZE

If RAM is more than 2GB, Swap file should go decreasing.

if (RAM_SIZE > 2GB)
swap = RAM_SIZE - x

[b2bwild]

if (RAM_SIZE > 512MB && RAM_SIZE <=1GB)
swap = 1.5 x RAM_SIZE

Correct algorithm, You only forgot a ) in if condition, X_X lol

These are standard rules defined by Red Hat, applies to most of the Linux Distros.

[Lakshmipathi]

[QUOTE=b2bwild;700159]Correct algorithm, You only forgot a ) in if condition [QUOTE]
yes,its a syntax error