Services Start then Stop (Fedora 10)

[billsinc]

I have several services set to start in init 3 on a cloud provider. The init scripts appear to be in the correct folder but the services are never started after a reboot. I finally watched the console and all the services start correctly then shutdown after a few moments. Does anyone what causes this behavior? Thanks!

[Rubberman]

What are the services, and have you reviewed their log files?

[billsinc]

They are several, but the major ones are httpd and mysqld. Which log files should I look at?

[Rubberman]

The log files for services in Linux are under /var/log, so for httpd you would look at the files in /var/log/httpd.

[billsinc]

I've already looked there and just did again. I was thinking there might be something somewhere else. The mysqld log just shows it starting and stopping normal when it's supposed to. The httpd logs just show the accesses and errors, which really don't help. The boot log is empty.

[Rubberman]

Some services start up and then stop at boot time just to initialize stuff. They are started for real by inetd / xinetd when a request for them comes in from a client somewhere. The ftp, ssh, and similar services are started that way. Look in /etc/xinetd.d for a file for mysql. For example, cvs on my system has the following in the /etc/xinetd.d/cvs file:

service cvspserver
{
        disable                 = yes
        port                    = 2401
        socket_type             = stream
        protocol                = tcp
        wait                    = no
        user                    = root
        passenv                 = PATH
        server                  = /usr/bin/cvs
        env                     = HOME=/var/cvs
        server_args             = -f --allow-root=/var/cvs pserver
#       bind                    = 127.0.0.1
}

If a file for mysql is there, then that will contain a similar block of information that allows the xinetd daemon to start it when a connection request is received by the server system.

[billsinc]

My xinetd.d folder is empty. How would mysql and httpdfunction that way? My problem is that when the server reboots, httpd and mysql are not started automatically (well they are and then they shutdown). So if someone tries to go to our site, they can't. I have to login via ssh and manually start the services.

[Rubberman]

My xinetd.d folder is empty. How would mysql and httpdfunction that way? My problem is that when the server reboots, httpd and mysql are not started automatically (well they are and then they shutdown). So if someone tries to go to our site, they can't. I have to login via ssh and manually start the services.

Well, then you need to find out why they are shutting down. Since they are started on boot, they have to have an services script in /etc/init.d, such as /etc/init.d/mysqld. If you are running RHEL or CentOS, then that will be a link to /etc/rc.d/init.d, and the contents of each /etc/rc.d/rcN.d directory will be links to the actual scripts in /etc/rc.d/init.d, but they have a letter and number prefexed to the file/link name, such as K36mysqld. The letter/number combo is so that each run-level can start each script in the proper order.

What I am trying to say in a long-winded way is that something that your http and mysql servers depend upon may not be starting up first like they need, or have not had time to register with the system so that mysql or apache/tomcat cannot properly start, and detecting that they then shut down.

So, question 1: Is there a mysqld and httpd script in /etc/init.d?
question 2: If so, is there a link to those scripts in /etc/rc.d/rcN.d (N is a number from 0-6)?

[billsinc]

I've looked in here before but didn't know what order they needed to be in. Do I need to change the order? Thanks by the way!

lrwxrwxrwx 1 root root 19 2009-05-08 20:26 K05saslauthd -> ../init.d/saslauthd
lrwxrwxrwx 1 root root 19 2009-05-08 20:26 K10dc_server -> ../init.d/dc_server
lrwxrwxrwx 1 root root 19 2009-05-08 20:26 K12dc_client -> ../init.d/dc_client
lrwxrwxrwx 1 root root 20 2009-05-08 20:26 K50netconsole -> ../init.d/netconsole
lrwxrwxrwx 1 root root 21 2009-05-08 20:26 K87restorecond -> ../init.d/restorecond
lrwxrwxrwx 1 root root 18 2009-05-08 20:26 K89netplugd -> ../init.d/netplugd
lrwxrwxrwx 1 root root 15 2009-05-08 20:26 K89rdisc -> ../init.d/rdisc
lrwxrwxrwx 1 root root 19 2009-05-08 20:26 K92ip6tables -> ../init.d/ip6tables
lrwxrwxrwx 1 root root 18 2009-05-08 20:26 S08iptables -> ../init.d/iptables
lrwxrwxrwx 1 root root 17 2009-05-08 20:26 S10network -> ../init.d/network
lrwxrwxrwx 1 root root 17 2009-05-08 20:26 S12rsyslog -> ../init.d/rsyslog
lrwxrwxrwx 1 root root 19 2009-05-08 20:26 S15mdmonitor -> ../init.d/mdmonitor
lrwxrwxrwx 1 root root 20 2009-05-08 20:26 S22messagebus -> ../init.d/messagebus
lrwxrwxrwx 1 root root 15 2009-05-08 20:26 S25netfs -> ../init.d/netfs
lrwxrwxrwx 1 root root 19 2009-05-08 20:26 S26udev-post -> ../init.d/udev-post
lrwxrwxrwx 1 root root 14 2009-05-08 20:26 S55sshd -> ../init.d/sshd
lrwxrwxrwx 1 root root 16 2009-05-08 20:26 S64mysqld -> ../init.d/mysqld
lrwxrwxrwx 1 root root 18 2009-05-08 20:26 S80sendmail -> ../init.d/sendmail
lrwxrwxrwx 1 root root 19 2009-05-08 20:26 S85denyhosts -> ../init.d/denyhosts
lrwxrwxrwx 1 root root 13 2009-05-08 20:26 S85gpm -> ../init.d/gpm
lrwxrwxrwx 1 root root 15 2009-05-08 20:26 S85httpd -> ../init.d/httpd
lrwxrwxrwx 1 root root 15 2009-05-08 20:26 S90crond -> ../init.d/crond
lrwxrwxrwx 1 root root 15 2009-05-08 20:26 S98monit -> ../init.d/monit
lrwxrwxrwx 1 root root 17 2009-05-08 20:26 S99anacron -> ../init.d/anacron
lrwxrwxrwx 1 root root 24 2009-03-28 00:25 S99killall -> /etc/rc.d/init.d/killall
lrwxrwxrwx 1 root root 11 2009-03-28 00:25 S99local -> ../rc.local
lrwxrwxrwx 1 root root 16 2009-05-08 20:26 S99webmin -> ../init.d/webmin

[Rubberman]

I've looked in here before but didn't know what order they needed to be in. Do I need to change the order? Thanks by the way!

I don't think you need to change the order, but you might want to add a bit of a delay in the script before it really sets up the server. As I recall, you said that if you do a service start mysqld manually then it doesn't shut down. Besides dependencies on other services that might not have fully initialized, your boot-time problem could also be due to something like the fact that the boot-time scripts have no tty attached to stdin, stdout, stderr and I have seen in cases in the past when that would cause a server to quit, because it wasn't coded to accommodate that possibility. That's just a SWAG though, and probably not the real reason why your problem is happening.