vBulletin server goes into read-only mode.

Hi

I've a popular vbulletin forum (10k max in vbulletin stats 'whos online') runnin in two servers like this:

1) Web server running vBulletin.
Dual Intel E5620 at 2.40GHz (shows 16 cpus)
12 GB RAM
2 x 500 GB SATA (one for system/web and other for backups)
CentOS 5.6 64 bit
Apache/2.2.17
PHP 5.2.16

2) MySQL 5 server
Dual Intel E5620 at 2.40GHz (shows 16 cpus)
16 GB RAM
2 x 500 GB SATA (one for system/web and other for backups)
CentOS 5.6 64 bit
MySQL 5.0.77

Web server connects to MySQL using private LAN.
Web-php server works with xcache as recommended by vBulletin.
Image/avatars and other static files (css, js, etc) are on a CDN
MySQL server uses Memcached for caching common queries.

The problem I'm facing, is from time to time (let's say one or two times a week) the Web server goes into read only mode... then the server has to be rebooted in order to get online normally and be able to work. From the logs I've seen this:

Code:
server:~ dmesg | grep ext3 -i

sizeof(ext3inode)=760 bytes
EXT3-fs: mounted filesystem with ordered data mode.
EXT3 FS on sda3, internal journal
EXT3 FS on sdb1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
EXT3 FS on loop0, internal journal
ext3_orphan_cleanup: deleting unreferenced inode 27
ext3_orphan_cleanup: deleting unreferenced inode 26
ext3_orphan_cleanup: deleting unreferenced inode 25
ext3_orphan_cleanup: deleting unreferenced inode 24
ext3_orphan_cleanup: deleting unreferenced inode 23
ext3_orphan_cleanup: deleting unreferenced inode 22
ext3_orphan_cleanup: deleting unreferenced inode 21
ext3_orphan_cleanup: deleting unreferenced inode 20
ext3_orphan_cleanup: deleting unreferenced inode 19
ext3_orphan_cleanup: deleting unreferenced inode 18
ext3_orphan_cleanup: deleting unreferenced inode 17
ext3_orphan_cleanup: deleting unreferenced inode 16
ext3_orphan_cleanup: deleting unreferenced inode 15
ext3_orphan_cleanup: deleting unreferenced inode 14
ext3_orphan_cleanup: deleting unreferenced inode 13
ext3_orphan_cleanup: deleting unreferenced inode 12
EXT3-fs: loop0: 16 orphan inodes deleted
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.

This is a new server, and in the old server the 'read only mode' issue happened very often too... so I guess it's not a HD issue.. it's just the HD have a heavy I/O usage. What do you think?

So, I am thinking about this solutions to improve the webserver performance:

1) 75% of the vbulletin users are anonymous users, so I've been thinking about using Cloudflare as a cache for all that traffic, that would reduce the php-webserver process a lot.

2) Using an external email server is a good option I guess... as right now the webservers also handles the email notifications for all the vbulletin users and it is constantly sending email everywhere. So, email will be stopped to gain server performance.

3) Mounting partitions with 'noatime' at /etc/fstab seems to improve I/O performance according to what I've read and tested a few times but I'm not totally sure. What do you think?

Do you have any other ideas about this?

I appreciate your help.

Thanks!