IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 1 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New Database Replication.algorithms?
This is a bit of a stab in the dark, but I was wondering if anyone here has been involved in creating a database replication algorithm. We need to come up with something in the next few months and we'd really like to avoid re-inventing what could be a quite tricky wheel.

To give some background, we need a geographically distributed database behind our application. Under guaranteed connectivity, we can arrange to have all writes to be made to a master database which then hot-copies to a collection of slaves. Unfortunately, the connectivity cannot be guaranteed, but local databases are still required. So some sort of replication will be required. (We already know we will need the application backend to manage it as it will have the knowledge to sort out conflicts that the database won't have.)

I've started Googling, but the IWETHEY Group Mind would be useful to plumb, too.

Wade.

Is it enough to love
Is it enough to breathe
Somebody rip my heart out
And leave me here to bleed
 
Is it enough to die
Somebody save my life
I'd rather be Anything but Ordinary
Please

-- "Anything but Ordinary" by Avril Lavigne.

New What DB you using?
MySQL has replication now, you can select from the local copy, writes are to the master and immediately broadcast. Postgres was supposed to be close to having it when I last used it a year-and-a-half ago. Oracle of course it's standard. So what are *you* using?
===

Implicitly condoning stupidity since 2001.
New MySQL at the moment.
And yes, we've experimented with MySQL's replication. For a high-load situation requiring several servers all next to one another, it would suit perfectly.

Unfortunately, this will not suit one of our newest clients. The local sites have a variety of Internet connectivity - one of the more difficult has a satellite downlink for inbound and a modem for outbound. MySQL replication may be okay with that, but we can't assume that.

Wade.

Is it enough to love
Is it enough to breathe
Somebody rip my heart out
And leave me here to bleed
 
Is it enough to die
Somebody save my life
I'd rather be Anything but Ordinary
Please

-- "Anything but Ordinary" by Avril Lavigne.

New Might work for you
You'd have to write your apps to do all reads locally and all writes to the master server. The big issue you'd have -- and I'm fairly sure you would have it -- is that there is no mechanism for checking if a particular write has been propogated to the slaves.

eg: Our users update the status of an order. The page submits the update query to the master DB server, then it queries the slave to draw the page. If the update hasn't been replicated yet, the page will come up still showing the old status. If another user has just submitted a large insert, like inserting a multi-MB blob, your insert would be queued. The query to the slave would then execute before the replication was done.

The way we worked around it is when submitting an update and then doing a query on the same page, query the master server. This causes higher load on the main server, but perceived responsiveness is the same as if there were a way to wait for the replication before issuing thw query. Depending on how your application is written, this might or might not be possible.
===

Implicitly condoning stupidity since 2001.
New We were going to explore application level replication.
That is, the application layer is aware of a local and hence fast server that it should by default use, and a remote server that access to is not 100% guaranteed. Changes it has made to "the database" it keeps track of and periodically informs an application instance local to the master database of them and vice-versa. Synchronisation of updates would be policed by timestamps and I'm thinking of handling new entries using a partitioned number space to prevent collisions and having them self-update as they fill up.

Wade.

Is it enough to love
Is it enough to breathe
Somebody rip my heart out
And leave me here to bleed
 
Is it enough to die
Somebody save my life
I'd rather be Anything but Ordinary
Please

-- "Anything but Ordinary" by Avril Lavigne.

     Database Replication.algorithms? - (static) - (4)
         What DB you using? - (drewk) - (3)
             MySQL at the moment. - (static) - (2)
                 Might work for you - (drewk) - (1)
                     We were going to explore application level replication. - (static)

Powered (for now) by thermodynamics!
58 ms