uh-uh - broke my server

[Mistoffeles]

I was going through a hardening procedure from the Center for Internet Security:

https://community.cisecurity.org/download/

The one for RHEL5, v.1.1.2 (benchmark version).

It contains this script:

ls -la /etc/pam.d/* > $tmpcis/CIS_8.1.Before.tmp
cd /etc/pam.d
for FILE in `find . -type f -exec grep -l rhosts_auth {} \;`; do
  echo "Removing .rhosts support in ${FILE}."
  grep -v rhosts_auth $FILE > $tmpcis/${FILE}.tmp
  /bin/cp -f $tmpcis/${FILE}.tmp $FILE
  chown root:root $FILE
  chmod 0644 $FILE
done
ls -la /etc/pam.d/* > $tmpcis/CIS_8.1.After.tmp
echo "The following entries changed under '/etc/pam.d'"
echo "diff $tmpcis/CIS_8.1.Before.tmp $tmpcis/CIS_8.1.After.tmp"
diff $tmpcis/CIS_8.1.Before.tmp $tmpcis/CIS_8.1.After.tmp
chmod -R 0400 $tmpcis/*
cd $cishome

which is for removing rhosts support from PAM (*shrug?*), which upon running gave me so many '/proc' filesystem write errors that it scrolled right out of the PuTTY buffer. It also disabled the ability to run *ANY* command from the session and now I can't log into any account on the server.

There isn't even a reference to '/proc' in the script, and none of the links in the directory used in the script point to anything in the '/proc' filesystem.

I don't know if this has just disabled the ability to log in remotely, or totally bolluxed my proc filesystem and made the server completely unusable (i.e.: time for a reformat).

[Mistoffeles]

No suggestions?

[drl]

Hi.

If variable tmpcis was not defined, then the command

chmod -R 0400 $tmpcis/*

would have been effectively:

chmod -R 0400 /*

making all files read-only for the owner.

I have no idea if this would produce the symptoms you see. I usually place

set -o nounset

in my [bash] scripts to prevent problems like this.

Strictly from a cursory look at your script and strictly a guess.

Perhaps booting into single-user might provide more information.

If it's any consolation, I once ran a script on a remote server that disabled networking, so I had to drive 2 cities away to reset the box. I was very careful afterwards ... cheers, drl

[Mistoffeles]

ouch.

Getting to this server is only a half-day trip, including the drive both ways and a couple hours' work as needed. If I push it, and bring a ready replacement, I could technically swap it out in 3 hours including checks to make sure everything is working. (It's not production yet, I have just been building a personal hardening procedure as I go along, trying various things).