Application monitoring

[bornon2303]

Hi,

The requirement is to monitor an application which should be executing all the time, and if the application is down for any reason, it should be automatically started by the monitoring script.
For this, currently we have written a monitoring script which is added to a crontab and it is scheduled for execution every minute. This(monitoring script) should start the application if it is down. Currently it is not working as expected for some reason. We have a command in the script which uses 'nohup' to bring up the application, this command is working fine when issued thru the command prompt, but not working when it is used in the shell script.
I need help in finding out what the issue could be with the shell script and also if there is an alternate method to achieve the objective stated.

Thanks in advance.

[watael]

hi,

without the scripts, we'll just have to guess.
it may take a very long time, until we hit it right !

as crontab does not have tty, nohup is useless.
check path for every command is in PATH.
if an application can be down, it should be started from inittab.

[bornon2303]

Hi,

Below is the script contents.

#!/bin/bash
# check daemon
ps -ef | grep -v grep | grep comon.App
# if not found - equals to 1, start it
if [ $? -eq 1 ]
then
nohup   /app/Oracle/Middleware/jdk160_21/bin/java -cp /app/abcd/watch/LogS.jar logs.comon.App &
else
echo " already running - do nothing"
fi

If the following code is executed manually the application starts...

nohup   /app/Oracle/Middleware/jdk160_21/bin/java -cp /app/abcd/watch/LogS.jar logs.comon.App &

So not sure what the issue is.

[atreyu]

Hello and welcome to LinuxForums!

I would use screen to to do this, instead of nohup.

It will launch a screen session in the background, with your named process running in the foreground of that screen. Here's an example of how to call it:

screen -dm -S Java_App /app/Oracle/Middleware/jdk160_21/bin/java -cp /app/abcd/watch/LogS.jar logs.comon.App

To list running screens for a given user, do:

screen -ls

to attach to the pid of a screen (returned by the above command):

screen -r <PID>

or in the above case, you can use the screen name like:

screen -r Java_App

if you need to clean screens:

screen -wipe

Also, to trouble-shoot your original issue, i would add logging to the cronjob, e.g.:

10 0 * * * /path/to/java_script.sh > /tmp/script-cronjob.log 2>&1

then check that log file for errors that you might normally not be seeing, when cron runs the job.

[watael]

better is to not use ps, because it shows itself too.
instead use pidof, or pgrep.

[bornon2303]

Can this be implemented in a production environment ?

[atreyu]

Can this be implemented in a production environment ?

I'm not sure why not. It is not much different from your nohup method.

Before pushing anything into production, though, I would be testing it out the wazoo on a system that is hopefully identical in hardware and software under a similar load.