Post #19,360
11/25/01 8:57:28 PM
|
I Dunno where this belongs, so I'm putting it here
I am about to have a project thrust upon me.
Currently 2 years old.
It should have gone into "production" in 6 months.
The current due date is Jan 1. Hah!
It is based on a mixture of Oracle PL/SQL, Perl
(written by a poor novice), and a few system utilities.
Much of the PL/SQL confuses me (I've only done lite PL/SQL
on purpose, I like Perl much better. I will, though, code
some very tight update loops in it to gain bulk fetch
and update speeds).
I avoid multi level selects. This system has hundreds
of them. Sometimes created by dynamically building up
the SQL scripts in PL/SQL, so there is no clear statements
unless the script is actually running.
Doing a simple dir search, there are 209 perl scripts,
2920 sql scripts, and 71 sql loader control files.
There is an huge amount of overlap in these scripts, but
they can be large (hundreds of poorly coded lines), and the
overlap can be difficult to determine.
The currently Oracle database size is about 800GB, and total
system disk size is 2TB. The system is on a Sun 3500 with 4
CPUs. Total hardware cost so far about $300,000, and I assume
personel cost at about $300,000 also.
The current claim is that the base data is loaded, and now
there is a twice a week production that will need to be run.
And that the current process is merely too slow.
Bullshit.
The current person who interviewed the clients, designed the
system, coded most of it, directed others to code the rest, and
is now the day to day overseer of it has way too much need to have
it seem to work.
He may choose not to stick around, which means the whole house of
cards comes tumbling down.
Docs? Yeah, right.
Or he may choose to hide critical pieces of it.
I figure I need to duplicate the entire system on another box to
start off with. Just so happens I have a Sun 450 with 2TB spare at
the moment.
I then need to parallel run anything he does, comparing reports and
outputs. This might take about 2 months to get to the point I
actually believe I understand what its being run. Not that I'll
understand WHY, just that no one ran anything without me being
aware of it (last second hacks, etc).
At the same time, I should find the current production bottlenecks.
If they are for output, I can attempt to optimize in place, but
if they are for input/update, it has to be done against the parallel
system to make sure I don't hurt current data.
I'll have to review all email and previous specs, interview clients
and in-house personnel, and all the stake holders. There might be
many levels of political tension, which I'm famous for "putting out
the fire, with GASOLINE!", so I'm really out of my element there.
If I need to create from scratch, I assume a 6 month projects based
on what I've observed and the data volumes involved. Which is what
it should have been in the 1st place.
How can I make this a win/win?
The president's job is on the line, depending on how he salvages this
project. He let the previous guy run amok for way to long, and the CEO
knows it.
Any suggestions?
|
Post #19,362
11/25/01 9:11:12 PM
|
Book a farewell lunch for the President?
I was in a similar situation once, but mercifully on a much smaller scale. We had an Evil Contractor(tm) who had worked most of his way through a project, but things had turned very nasty and he had to be escorted out of the building by security. So there wasn't exactly a lot of opportunity for handover, you might say.
In this case, it was easier to actually start from scratch, and build the thing in about four weeks (the Evil Contractor had been there about three months I think). Amazingly, the end result actually worked, and the product was sold on to another company, marking one of my proudest working moments :)
Anyway. I don't envy you one bit. Sounds like a horror. I think the first thing I'd do is get the implemtation date changed. Basically, lots of 'expectation management' required. The sad thing I guess is that you'll probably need to spend a fair bit of time covering your own arse to make sure none of these stuff-ups can be attributed to you.
Or shift the onus back on managment - get them to make the 'fix it' or 'rebuild it' call, based on your estimates of how long each would take. But I guess, make it abundantly clear that neither would be possible by Jan 1.
Anyway, guess I've just made it abundantly clear I know nothing about project managment. But the very best of luck all the same! :)
On and on and on and on,
and on and on and on goes John.
|
Post #19,373
11/25/01 10:02:45 PM
|
You need the president on side.
The president's job is on the line, depending on how he salvages this
project. He let the previous guy run amok for way to long, and the CEO
knows it.
Managing (people) is not one of my skills and I've not done any project management either. But I quite agree with Meerkat that getting it all sorted by Jan 1 is going to be Very Difficult. I would recommend you convince him that Starting Over is going to be the best solution. However, it rather depends on what you meant by "win/win" for the people involved other than yourself.
Wade.
"All around me are nothing but fakes
Come with me on the biggest fake of all!"
|
Post #19,377
11/26/01 2:40:09 AM
|
Don't start over...
...as such. I'd be more inclined to take the bits that /do/ work, and prop them up with your own stuff - I guess in this case it'd be the PL/SQL. You're going to be working late and weekends to do this by COP Jan 1.
However, totally starting over is not a runner. The system you're trying to rescue is just too big to reimplement in a little over 5 weeks.
However, due to the timeframe involved, you've basically got to decide /really quickly/ what it is you're going to do, and then run with it. But you knew that already :-)
Best of luck...
Peter
Shill For Hire
[link|http://www.kuro5hin.org|There is no K5 Cabal]
|
Post #19,455
11/26/01 5:58:20 PM
|
That advice is optimistic
Barry has been around the block a few times.
If he thinks the system can't fly by Jan 1, then it would be stupid for him to volunteer for a death march to the inevitable belly flop.
Personally if I was in Barry's position, I would draw up the different options, and I would offer upper and lower limits to how long he guesses that they would take. Give big error bars, and estimates of what point in each option he would be able to give better estimates. If you want to do well by the President, go up to him and tell him in advance, This is basically what I am going to have to say. But if you can figure out how to say it so that I get the facts across but cause a smaller mess, I will say it that way.
Throw that in everyone's lap. A key point would be, I refuse to touch this project unless people are planning on these figures. If I take this project and people attempt to impose anything else on me, I will drop it again like a hot potato. If you think that anyone else can do it more reliably or faster, let them. Let the crap fall where it may.
My goals would be to first protect the principle that other people's misjudgements shouldn't become your nightmares. Second, make sure that people get bad news early, rather than late, so that they have as long to deal with it as possible. Third, protect your reputation as a person who does what you say you can. Fourth, protect your reputation as a person who can be depended on to say things as they are. Other things come lower.
This won't make the President happy by giving him the politically convenient lie. But the truth will out anyways as the project grinds on, and it is only with a realistic view of the situation that the president can figure out what choices to make.
Cheers,
Ben
PS There is a reason I don't talk to clients. :-)
|
Post #19,388
11/26/01 9:24:04 AM
|
An embarassment of riches
I figure I need to duplicate the entire system on another box to start off with. Just so happens I have a Sun 450 with 2TB spare at the moment. Where do you work, man? I mean, the concept of a "spare" 2TB on a server is an oxymoron where I come from! Would indicate you're well funded.... I like your idea about the parallel tracking. I would suggest you take that "spare" box, tho, and begin cleanup there, and start merging cleaned up code into the main (production?) systes as soon as you get it working (they DO have an SCCS-equivalent, don't they? If not, make that the first thing you get!). Keep up informed...and good luck. jb4
(Resistance is not futile...)
|
Post #19,430
11/26/01 3:59:22 PM
|
Maybe candidate for top-down refactoring
Expectation managment has been covered. As for the technical solution ...
Just start off with a main loop that contains a stub for 'doEverything()' that calls the existing mess. Then figure out how the report is run (sounded like you were saying there's one main report this whole mess does?) and write a stub for 'runReport()' that calls that mess.
Every time I've taken over someone else's project, I've re-written to one extent or another. The first step is always just figuring out what the hell it currently does. The top-down refactor is the best way I've found to wrap my head around this. It has the beneficial side-effect of sometimes -- OK, rarely, but still -- being useful as a framework for the eventual new system.
With over 3,000 files in your project, I don't see how you could do anything else without first figuring out what does what. Actually, you might be able to outomate this inspection somewhat. Try creating a Perl script to grep through the existing files, catalogue all the classes and functions, and map out what calls what. Knowing that 'foo()' calls 'bar()' and that 'bar()' is only called by 'foo()' goes a long way to getting things organized.
Me.cents --
Me.cents --
We have to fight the terrorists as if there were no rules and preserve our open society as if there were no terrorists. -- [link|http://www.nytimes.com/2001/04/05/opinion/BIO-FRIEDMAN.html|Thomas Friedman]
|
Post #19,451
11/26/01 5:41:31 PM
|
SQL Subqueries
I'm not quite sure what you mean by "multi-level selects", but I suspect you mean correlated subqueries. Similar to sentence fragments.
Useful device. Will be used. More later.
Like most syntactic outliers, subqueries are useful. Used judiciously. The can simplify some convoluted coding, reduce extra steps, and occasionally improve performance. They can also to the reverse of all in spades.
My usual preference is to keep subqueries restricted to a depth of one, possibly two in exceptional cases. Anything more than this and I start abstracting out the queries, either as views or additional CREATE TABLE queries (use your workspace for these). The set nature of SQL means that, if you can work out the logic and parameters of the queries themselves, you can replace a subquery with the equivalent table or view.
I have to admit complete ignorance of PL/SQL, but I've used SQL extensively in generating reports with SAS, where many intermediate steps can often be saved with some only mildly convoluted query logic.
--
Karsten M. Self [link|mailto:kmself@ix.netcom.com|kmself@ix.netcom.com]
What part of "gestalt" don't you understand?
|
Post #19,502
11/27/01 9:25:50 AM
|
Maybe not (subqueries that is)
I've seen queries similar to
select d.(stuff) from a, b, c, d
where a.(stuff) = b.(stuff) and b.(stuff) = c.(stuff) and c.(stuff) = d.(stuff)
and have seen query "optimizers" make stupid decisions about what to select from where and when. This is particularly evident when you upgrade from one version of a database to another; often times they've "improved" the optimizer enough to shoot the hell out of your carefully crafted queries that worked with the old one.
(Most databases seem to have pathetic query analyzers and diagnostics, but I digress.)
Not that this helps in this particular problem. I ran into a report converting tool ("automatically" converted from one 4GL with one flavor of database selection syntax to another) that did some Gawdawful stuff to the SQL, but because there were so many reports it wasn't possible to go through and optimize each report. I have every reason to think that those reports (in the tortured SQL version) still exist.
"Beware of bugs in the above code; I have only proved it correct, not tried it."
-- Donald Knuth
|
Post #19,458
11/26/01 6:09:34 PM
|
Oracle and PL/SQL tips
I've found that even though you list the tables in a specific order, Oracle may not use that order when filtering the data. This can result in large run times. ie:
select * from item_ven where vendor = 'ACME';
results take 2 seconds
select * from item_file where lead_time >=7 and leadtime < 14;
results take 5 minutes
select i.item,
i.description,
v.vendor
from item_ven v,
item_file i
where v.vendor = 'ACME'
and i.item = v.item
and i.lead_time >= 7
and i.lead_time < 14;
results take 5 minutes
even though you, as the programmer, may know that VENDOR is more restrictive and thus put item_ven first, Oracle may decide to resequence the query and hit item_file first. Putting /*+ ordered */ tells it to hit the tables in the order you listed them - ie:
select /*+ ordered */
i.item,
i.description,
v.vendor
from item_ven v,
item_file i
where v.vendor = 'ACME'
and i.item = v.item
and i.lead_time >= 7
and i.lead_time < 14;
results take 3 seconds
You can also create/delete indexes within PL/SQL. Useful for reporting on large active files that you don't want to bog down with extra index overhead during normal processing hours. The creation of an index is quite fast and can significantly speed up queries. I can send you sample code if you need it. Darrell Spice, Jr.
[link|http://home.houston.rr.com/spiceware/|SpiceWare] - We don't do Windows, it's too much of a chore
|
Post #19,587
11/27/01 11:07:10 PM
|
Bang! Dodge - whish
The sound of me dodging a bullet.
I'm split between 2 "critical" projects right now. The need for me in one of them will be winding down in about 6 weeks, which means that until then, it is my priority. Which means that the success or failure of the other one is not my issue, and I won't be expected to pick ANYTHING up until 2 weeks after it is already in "production". I'll spend the next 6 weeks merely reviewing and minor training, since I can't let the other one slip.
The manager in charge of the current project screamed loud enough to make sure he didn't lose me until then.
I gotta admit, it's an ego boost when the various execs fight over me.
As far at the other comments, thanks. All are appreciated.
Right now I'm doign a Veritas updgrade remotely. Never done it before. As I fiddle with OS disk drivers and reboot the box, ya think it'll actually come back up?
|
Post #19,589
11/27/01 11:39:44 PM
|
Phew. Yay!
On and on and on and on,
and on and on and on goes John.
|
Post #19,655
11/28/01 2:24:52 PM
|
Lucky you...
But I suggest that if the process of going into production drags as you suspect it will, you may wish to provide "advance hints" that when you switch priorities, you shouldn't be expected to do something impossible.
Cheers,
Ben
|
