Results 1 to 10 of 12
I had a question from a friend that runs 2 redhat systems one on west coast and one on east coast. He is looking for some software (pay or free, ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
- 03-25-2010 #1
- Join Date
- Oct 2008
Looking for site replication software
He is looking for some software (pay or free, prefer pay with support options) that he can use to keep these 2 systems (each has local disk no SAN disks) in sync with each other.
One will be primary and one backup. But real time with a minimum of data loss should one of them go down.
Basically he process real time transactions on the primary system and needs to keep the backup system in sync over the network.
They are running mysql for the DB.
Is there any good sw or tools to do this?
If this is to much over the net he may consider getting a 3rd system local to the primary machine but that kinda defeats a site outage issue but still covers a local hdw issue should one arise.
- 03-25-2010 #2
you could load a software like GoodSync or Dropbox but you can just easily write a small script for a WebDAV server/client. The main question is how they are connected - via a private network (VPN) or Internet and if the IP address stays the same all time (which I assume in the described scenario). When you have an Apache web server installed on the one server you can use WebDAV (you may enable it - see Apache documentation). Use Cadaver WebDAV client on the other machine and write a script which is executed every few seconds/minutes (cron) to download the files from the other side. Haven't tried it for a MySQL database yet... Goodsync is easier and can be scheduled as well. Hope this helps.
- 03-25-2010 #3
- Join Date
- Nov 2007
You cannot "sync" a database by only considering the filesystem. Databases have transactions in memory, in logs, and in the DB files themselves. If they are not all in sync, flushed to disk, and then quiesced properly, any backup/copy of the DB files is useless. Of course, if you shut down the DB, and then backup the DB files, it is fine (called a "cold backup.")
You didn't state what "your friend" wants to keep in sync. The database? The filesystem? Both?
Filesystems are easy to sync using rsync. How often it runs is up to you. Database "syncing" is called replication. MySQL (like many) supports its own replication methods internally.
If DB replication is beyond your skillset, another method would be to dump the DB to a file every so often and then rsync the MySQL dump to the backup server. Getting the backup DB online would require "importing" the dump file into the DB. But then this could also be automated/scripted.
* If you prefer a "pay option with support," a MySQL support contract for a replicated database setup would be the most professional.
- 03-26-2010 #4
- Join Date
- Jan 2009
I Completely agree with HROAdmin26. You can set mysql replication which is not so hard or you can write a script to dump yourmysql database and then use rsync to backup the database dump.
- 03-26-2010 #5
HROAdmin26 and upanwar are correct. I ignored that fact and it is essential. As my experience with databases are limited, I backu up a bit.
Once you do a data dump of the database and sync those files, you can use the methods I described, but there are additional steps and it might be difficult to be reliable synced up at the end.
Anybody here with the knowledge on how to sync MySQL databases?
- 03-26-2010 #6
something like High-Availability Storage With GlusterFS On Fedora 12 - Automatic File Replication (Mirror) Across Two Storage Servers | HowtoForge - Linux Howtos and Tutorials
It's a free software.
disclosure: I'm part of this open source file system team
You cannot "sync" a database by only considering the filesystem.
- 03-26-2010 #7
- Join Date
- Jun 2007
what about RSYNC?
- 03-26-2010 #8
i don't believe rsync is intended for real time for failover capabilities
- 03-26-2010 #9
- Join Date
- Nov 2007
I have heard of Gluster but not used it. I have more first hand experience with VCS, VxFS, and VVR replication. VVR and HW-based replication are based on the block level *below* the filesystem.
Searching for any information about replicated MySQL on GlusterFS, I find no information that Gluster has any integration to ensure DB data consistency. I found several mailing list threads that said "*don't* put MySQL on Gluster volumes."
* A free block-level replication tech available for Linux is DRBD. Here is an older example of using MySQL with DRBD. Across large WAN links (West > East US), the network lag will cause large delays in sync'ing. As noted, even with this, a MySQL recovery is likely to be needed on the secondary instance.
Last edited by HROAdmin26; 03-26-2010 at 05:04 PM.
- 03-26-2010 #10
- Join Date
- Aug 2006
- North Dakota, USA
If this is a website along with the database that is doing online transaction processing (as opposed to database-only), there are several things to consider:
First, there's a difference between server name vs. role. "Primary" and "Backup" are each physical servers; "master" identifies the server currently running all the active services, and "slave" is the server that is acting as a backup (i.e. syncing data from master). Get your mind wrapped around this concept, or you'll get very confused. "... no, no, backup is primary, and primary is backup! Primary went down..." Using non-role-based server names (like names from a particular cartoon, demon/angel names, greek mythology, etc) will avoid confusion: "... no, no, Mephisto was the master, Belphegor was the slave. Mephisto went down...". Doesn't seem that big of a deal, but trust me, personal experience has taught me otherwise.
Next, there are several pieces that make up the whole. Each part must have proper consideration or there will be big problems. A backup solution that isn't properly planned will go haywire in bad ways if there isn't enough fore-thought.
* DNS: if one server goes down, users must get redirected to the backup (and vice-versa)
* Re-sync Primary: once the backup server becomes "master", the primary becomes a slave. The primary server must sync any missing transactions from the master before it can again become the master (i.e. if the primary is down for 3 hours, then comes back, you want it to be up-to-date & not 3 hours behind).
* Website changes: the backup server must have code kept in sync with the primary, and considerations need to be made so the backup website doesn't try to contact the primary database (i.e. make sure the hostname is something that will resolve to the local database at each site)
* MISC. FILE CHANGES: if there is any sort of uploads (i.e. users uploading inventory files), make sure those get synced (if they get entered into the database & then file gets deleted, make sure there's logic to avoid partial runs and to handle if an already-processed file gets copied back to the master server).
Experience tells me that sometimes recovering from disaster can lead to another disaster. Imagine the backup plan is perfect, and the slave server (s1) becomes master (previously p1). So s1 is master, p1 is now slave, and p1 finally recovers from its catastrophic failure: it seems like all there's left to do is bring it back online after being down for 3 hours... but suddenly it comes online and, although s1 was storing all the transactional data during p1's downtime, now p1 came online with data from 3 hours ago. If it isn't caught fast enough, it is very likely that the data in question could be automatically erased during a routine backup from p1 to s1 (master to slave). Whoops!