Cron Daemon: opt/plesk-billing/task-manager/scripts/run-all.sh problem

Quote:

Originally Posted by atreyu

Which makes sense if you want/need others (i.e., the rest of the world) to access a webserver (or plesk) on your system. Is that the case? If not, then limit port 80 to only your ip address as well, via iptables. Even if you do want to leave port 80 open, maybe block it temporarily and see if the problem goes away.

Also, did you check the log files to look for malicious/suspicious behavior? You might want to consider fail2ban.org, too.

Well yes, I do website hosting. But I think its because I did something in SSH. I ran some commands and after that it suddently happend, I'm 99% sure it was me who caused this.
I ran some commands but I forgot exactly what I did.

Quote:

Originally Posted by Senethic

Well yes, I do website hosting. But I think its because I did something in SSH. I ran some commands and after that it suddently happend, I'm 99% sure it was me who caused this.
I ran some commands but I forgot exactly what I did.

Have you tried looking in your bash history? just run the command:

Code:

history

and it will show you all the commands you've run in a terminal, as that user.

Quote:

Originally Posted by atreyu

Have you tried looking in your bash history? just run the command:

Code:

history

and it will show you all the commands you've run in a terminal, as that user.

Oh wow, how did I not know that command LOL.

I also ran a repair but it didn't log that.
Anyway I filtered the useless stuff such as cd / rm -rf / ls etc.
Also I'm 99% sure that this isn't what caused it but mayby you can figure something out from this.

914 netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
918 service httpd restart
919 cd /usr/local/psa/admin
925 sync && echo 3 > /proc/sys/vm/drop_caches
927 echo 1 > /proc/sys/vm/drop_caches
929 crontab
930 crontab -l
931 less /etc/crontab
932 iftop -F *censored*/24
934 yum install iftop
935 iftop

Also thank you for your help so far !

Quote:

Originally Posted by Senethic

Also I'm 99% sure that this isn't what caused it but mayby you can figure something out from this.

yeah, you're right. nothing there. have you checked your plesk log, web logs, and /var/log/secure to see if any attempts to access your system coincide w/each other? i am totally unfamiliar w/plesk, so I can't offer any experienced help there.

Quote:

Originally Posted by atreyu

yeah, you're right. nothing there. have you checked your plesk log, web logs, and /var/log/secure to see if any attempts to access your system coincide w/each other? i am totally unfamiliar w/plesk, so I can't offer any experienced help there.

I've checked /usr/local/psa/admin/logs and that gave me a huge list.
I'm not sure what this is doing but looking at the time stamps it keeps repeating itself every second.

Code:

::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:26:12 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:26:12 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:30:17 +0100] "POST /check-plesk.php HTTP/1.1" 200 642 "-" "Dr.Web Daemon/Plesk Edition" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:31:13 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:31:13 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:36:14 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:36:14 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:40:32 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 992 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:41:14 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:41:14 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:46:15 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:46:15 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:51:16 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:51:16 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:56:17 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:03:56:17 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:00:33 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 992 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:01:17 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:01:17 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:06:18 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:06:18 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:11:19 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:11:19 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:16:19 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:16:19 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:20:34 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 992 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:21:20 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:21:20 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:26:20 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:26:20 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:31:20 +0100] "POST /check-plesk.php HTTP/1.1" 200 664 "-" "Dr.Web Daemon/Plesk Edition" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:31:20 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:31:20 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:36:21 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:36:21 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:40:35 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 992 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:41:21 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:41:21 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:46:22 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:46:22 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:51:22 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:51:23 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:56:22 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:04:56:23 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:00:36 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 992 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:01:22 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:01:22 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:06:23 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:06:23 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:11:24 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:11:24 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:16:25 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:16:25 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:20:36 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 992 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:21:26 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:21:26 +0100] "POST /plesk-billing/task-manager.php HTTP/1.1" 200 49 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.13.1.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2" ::ffff:127.0.0.1 localhost:8443 - [06/Jan/2013:05:26:26 +0100] "POST /plesk-billing/mbapi/remote.php HTTP/1.1" 200 1001 "-" "-"

Also I've looked up on the cron.d

Code:

[root@*censored* cron.d]# cat billing-task-manager */5 * * * * psaadm /opt/plesk-billing/task-manager/scripts/run-all.sh --config /opt/plesk-billing/task-manager/config/config.ini

I think thats very related to the emails I'm getting.

Emails:

Code:

/opt/plesk-billing/task-manager/scripts/run-all.sh: fork: retry: Resource temporarily unavailable /opt/plesk-billing/task-manager/scripts/run-all.sh: fork: retry: Resource temporarily unavailable /opt/plesk-billing/task-manager/scripts/run-all.sh: fork: retry: Resource temporarily unavailable /opt/plesk-billing/task-manager/scripts/run-all.sh: fork: retry: Resource temporarily unavailable /opt/plesk-billing/task-manager/scripts/run-all.sh: fork: Resource temporarily unavailable

Code:

execl: couldn't exec `/bin/sh' execl: Resource temporarily unavailable

Oh also I've did some more researching.
Just to define the error some more

"The error is generated because of recursive defunct processes going on in the memory and is unable to allocate specific amount of memory for the newly created process. Basically the general structure of process generation is as followed. "

So my best guess would be that its taking cron jobs which can't be run because of lack of memory.
(Thats why swap memory is at 100%)

I would like to thank everybody for the support.
Unfortunately I couldn't fix it so I reinstalled my server.

Oh well.. it was time for a reinstall anyway.

Closed at request of O.P.