IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New First step is to realize this is not a technology project
You have multiple similar representations of the same real-world entities. Those representations are similar, but not identical. Until they are identical, they can not be used to generate unambiguously correct reports. Yes, you can get close, but you can not verify the correctness.

Once the people who matter -- ie: the ones who pay the bills -- are willing to accept that the data must be cleaned up first, then any one of several options for doing the reporting becomes clear. This exact project has been done over and over, and the people who don't have to make it work always think there is some magic wand you can wave over shit data that makes it all fit together. There isn't. You can't.




Now, once you've got the green light to fix the data, you need a way to keep multiple DB platforms in sync. If the only two are Oracle and SQL Server, you can use update triggers on all tables that need to be syncronized. These triggers will have to call routines -- probably written in C/C++ -- to do the synchronization.

I'm not going to suggest picking one authoritative customer table and deleting the other two. That would involve re-writing everything that uses it. That's a whole other project. Unless that is open for discussion.
===

Purveyor of Doc Hope's [link|http://DocHope.com|fresh-baked dog biscuits and pet treats].
[link|http://DocHope.com|http://DocHope.com]
New Double plus good
Having played this game more then once I have to agree with Drew. First, track down the differences in the databases and data and figure out how to reconcile them at a buisness level. You will probably have to send some of those issues back up to your manager to setup cross department meetings to figure out how to reconcile them.

After that the technological problem is actually much simpler. Depending on how much data you need for your reports, how current they need to be, and how often they are generated, you have a bunch of options.

Jay
New Thanks :)
Just wanted to say thank you, for your advices and replies.

I think, I was really hoping, that there would be a magical tool (well not in a naiive way of course), since as drewk mentioned this system was done many times before.

So to sum things up.
1. Make sure all the databases have the same information
2. Use db triggers to synchronize, from that point forward
3. Build new functionality on top of that

Cool, and thanks again :)
New Note well
that making sure the databases have the 'same' information will not be easy. This is a political problem; who gets to decide the info on a customer... finance or sales?

I'd suggest that if you're going to sql server, that you might want to consider changing that a bit:

  • Meet with stakeholders over time to resolve information issues.

  • Create a new schema for a new'n'improved system that includes all data.

  • Harmonise data in the various databases. Create triggers that keep information harmonised following above schema. Run this way for a while to ensure that schema meets organisation's needs. Use this phase to refine schema according to users' needs.

  • Implement new and improved schema on central high powered db. I'd strongly consider postgres (free licensing, support available from developers for money, it runs on Windows too, has ports to many versions of unix so when growth is such that windows keels over you can scale to a better platform with little pain).

  • Move data into central db using above schema for organisation. Migrate clients to software to access this db instead of the group of old databases.

  • Keep both old and new systems synched for a while to ensure that no problems pop up that require reverting to old system

  • Decomission the old system.

Lots of folks here have more experience with this stuff than I do, I'm sure that they'll chime in if I've egregiously misrepresented/fucked up anywhere.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New No... you did very well. One thing to add.
After: Decomission the old system

I'd re-deploy the old systems as report "processing" off-loaders and/or compute nodes.

Obviously they are pretty good machines, currently, seeing they run Oracle and MS-SQL. That way you could get more life out of the machines *AND* make sure the old system(s) go away...

Get rid of the old and augment the new at the same time. That way departments don't feel like they are being robbed, you could even dedicate the old machine to them for specific things, making them feel special.

All part of "Protecting the Company's Investment". Yeah... err... YEAH!
--
[link|mailto:greg@gregfolkert.net|greg],
[link|http://www.iwethey.org/ed_curry|REMEMBER ED CURRY!] @ iwethey
Freedom is not FREE.
Yeah, but 10s of Trillions of US Dollars?
SELECT * FROM scog WHERE ethics > 0;

0 rows returned.
     General DB integration question - (systems) - (6)
         First step is to realize this is not a technology project - (drewk) - (4)
             Double plus good - (JayMehaffey)
             Thanks :) - (systems) - (2)
                 Note well - (jake123) - (1)
                     No... you did very well. One thing to add. - (folkert)
         stovepiping is your friend - (boxley)

Escape from the prison planet!
62 ms