Not exposing the "structure" of the database

Post #48,033 by gdaustin 8/4/02 11:08:07 AM Reply	Not exposing the "structure" of the database I'm not buying this argument, Todd, because I've programmed in about 4 flavors of SQL ( Sybase, MS, Oracle, DB2 )for about 6-7 years now. Yes, you can objectivize the structure of the database away using object oriented techniques, but this is a very bad thing, performance-wise. When you're writing software to do mass purges (purging a million rows in 2-3 days) out of a 100 million row database, you can't just "let the object-oriented libraries handle it". There are very specific types of SQL statements you have to write to make it function correctly, or it will perform VERY SLOW, or run the system out of log space. I did that in a previous job. When the structure of the database is abstracted away from the developer, then very bad things happen, like queries that result in table scans on 100 million row tables. This same thing happens when people use things like Rational-Rose, and it happened back in my mainframe days, when people used IEF. The "modeling" tools have no idea what indexes the database has, how many rows certain tables have, etc. and then the "modeling" tool ends up writing code that performs like crap. Bottom line, you HAVE to know the structure/behavior of the database (keys, indexes, rowcounts, access hot spots) to be successful. To the extent that modeling tools remove you from the knowledge of that structure, they are evil. Also, the added layer of object orientation between business logic and data is being preached in n-tier mantras all the time, many times when it is not necessary. I argue against it many times for specific reasons. If the project is entirely internal, with a single interface and a single customer, then it is simply faster to do 2 or 3 tier. OO is useful for interfaces where a non-computer literate customer is attempting to program. A good example is our hourly people at work dropping controls on a screen in Delphi to attempt to build a program. n-tier interfaces need to be defined in complex systems where multiple customers process the same data, and multiple sub-systems are used by multiple customers, and the company needs to reduce costs by sharing the functionality of subsystems, rather than duplicating them (and their costs). Software design needs to be exactly like architecting a building. The interface points, entry/exits for people, water, electicity, gas, and air need to be well defined, carefully placed (where water and electricity are kept apart, for example) in order for a building to be successful. Good software has to be built the same way. In small, homogeneous environments, 2 tier is fine. Glen Austin
Post #48,043 by tuberculosis 8/4/02 1:37:55 PM Reply	Re: Not exposing the "structure" of the database Yes, you can objectivize the structure of the database away using object oriented techniques, but this is a very bad thing, performance-wise. You can't make that statement. Its a huge blanket statement and it can't possibly cover every scenario. For the average crud app, query for a list of objects, do some master-detail navigation, update or insert some data and save, OR Mapping with lazy faulting is the only way to fly. When you're writing software to do mass purges (purging a million rows in 2-3 days) out of a 100 million row database, you can't just "let the object-oriented libraries handle it". Uh huh. How many apps that the average developer is going to write are going to do that? Exactly Zero. That's a db maintenance function to be performed by an administrator and yes you write SQL for it. I wouldn't do it from Java either. I'd use sqlplus or whatever. Incidentally, I'm not arguing for 2 tier - rather 3+tier. If you've had trouble with mapping layers, then you're using the wrong software or bad technique. This stuff can and does work when judiciously applied. But you have to know what you're doing when designing the object model. Your statements are somewhat reminiscent of the objects of the anti-garbage collection crowd. Now java has gc and suddenly we're OK with it. Yeah yeah yeah. GC, like OR mapping, can be done well or poorly, but when properly done it outperforms manual memory management - despite the irreational fears of the control freaks that insist you can only get performance with manual memory management. Software design needs to be exactly like architecting a building. The interface points, entry/exits for people, water, electicity, gas, and air need to be well defined, carefully placed (where water and electricity are kept apart, for example) in order for a building to be successful. Good software has to be built the same way. Absolutely agree - but I don't want to have to go to the basement and shovel coal everytime I want the heat adjusted. I am out of the country for the duration of the Bush administration. Please leave a message and I'll get back to you when democracy returns.
Post #48,076 by gdaustin 8/4/02 10:06:51 PM Reply	Average Crud App >For the average crud app, query for a list of objects, do some master-detail >navigation, update or insert some data and save, OR Mapping with lazy faulting >is the only way to fly. But, back in the MS-DOS days there were tools like Clipper which allowed complete idiots to write CRUD apps... My boss at work is intent on proving that hourly people can write CRUD apps in Delphi with minimum help from "real" programmers. Come up with something like "Clipper for Windows" and you can convince management that they don't even need us high cost prima-donnas. Oops, I guess that is MS-Access and VB Office. I think that's the current management phase. Why do we need you $40 an hour (loaded with benefits) prima-donnas when I can get a smart hourly person making $10 - $12 an hour with no benefits use some Clipper or Access tool to build me a database. Seems like I'm wasting a lot of money with you high falutin' "Software Engineers"! My current management's sentiment exactly. I don't want to write CRUD apps for a living. That's why I LIKE writing things like purges on 100 million row databases. Transaction processing/large databases, that's what makes me tick. How do you get 1 million or 10 million transactions a day safely through a set of Unix servers and through a tuned relational database without it dying? >Uh huh. How many apps that the average developer is going to write are going >to do that? Exactly Zero. That's a db maintenance function to be performed by >an administrator and yes you write SQL for it. I wouldn't do it from Java >either. I'd use sqlplus or whatever. Maybe I'm not an average developer, then. I'm probably going exactly where I need to be in working on being a DBA and developing tools for database management and transaction processing. >If you've had trouble with mapping layers, then you're using the wrong >software or bad technique. This stuff can and does work when judiciously >applied. But you have to know what you're doing when designing the object >model. I don't have a problem, because I always make sure I get the database requirements while I'm developing the code. But I've seen a LOT of other people fail, because they assume away WAY too much control to the OO design or structured design tool. >Absolutely agree - but I don't want to have to go to the basement and shovel >coal everytime I want the heat adjusted. If shoveling coal is the fastest way to get the 10 story building from 50 to 75 degrees, then I'm shoveling coal. I agree that the thermostat is nice for regulating heat, but only when it works. Glen Austin
Post #48,094 by tuberculosis 8/5/02 2:43:12 AM Reply	Not like Clipper! Look, I've done clipper programming (a little) and it also exposes and embeds SQL directly in the app which makes the app brittle and sensitive to database changes. It also encourages you to replicate your business logic in the screens. If you read what I wrote on my 2 steps forwards one step back piece I explain this. 2 tier is evil in a multiuser environment. Example - some Sybase consultant thought it would be a good trick to get an organization I was working in hooked on PowerBuilder. So he wrote a little app for service reps to handle some business task. Of course it worked fine for him. Only you get no locking mechanism with these tools. Its just select into a widget, let the user play with the data in the widget, then write code to do updates back to the database based on what changed in the widget. The OR mapping layers do optimistic locking by default. In fact you specify which fields are significant for locking in the model definition. So right there they save you significant work and prevent bugs. As far as getting more done with less - we get a couple of top developers to do the schema and the object model, then get the drones to build UI on top of the business objects. I think you'd prefer to be the developer doing the model (its what I specialize in). Regarding your big operations that have to be done in the database - yes you write stored procs for stuff like that and then expose them as methods in the object model (usually class methods for sweeping stuff like that). Its not a big deal and by putting that api on your database you avoid integrity issues, isolate apps from schema changes, minimize bugs, and still leverage the power of a cast of thousands (of screen painting dolts as management prefers). I am out of the country for the duration of the Bush administration. Please leave a message and I'll get back to you when democracy returns.
Post #48,122 by gdaustin 8/5/02 9:43:52 AM Reply	I'm Amazed that management still thinks there is a "magic bullet". I agree. Doing the schema, the database admin, and the object model is certainly more rewarding than the screens. It's rewarding professionally and it pays better. It just pains me to run into managment (like my current administration), who believes that schemas, administration, and object models aren't important. Then, again, I'm getting paid really well to clean up his prior programmer's Delphi mess. Glen Austin
Post #48,128 by tuberculosis 8/5/02 10:13:18 AM Reply	(Mis)Management Just out of curiosity - how did your management end up management? Whats their background? Career path? Why is that guy in charge and not you? Something about what that guy knows is important to the business owners. Maybe. This is one of the puzzles I'm trying to work out. How come the idiots end up on top? Why are the clueless defining the objectives and the means to reach them in the absence of any real skillset? I wish I knew. I am out of the country for the duration of the Bush administration. Please leave a message and I'll get back to you when democracy returns.
Post #48,139 by gdaustin 8/5/02 10:57:54 AM Reply	Re: (Mis)Management My boss IS the owner of the company. It's a small 50-60 employee company. Glen Austin
Post #48,150 by orion 8/5/02 11:44:01 AM Reply	This is an easy one Either they are a major stock holder, or a friend or relative of one, or the owner of the company. Failing that, they are an employee that worked their way up the ranks by sucking the right dcks, going to happy hour with the other management and getting sht faced, perticipating in spouse-swapping parties, attending those "adult" parties thrown by management (Getting into S&M, hardcore stuff), and bluffing their way into management by pretending that they know what they are doing (The Pointy haired Boss). That is why Idiots end up in management as the majority, they do these things because their job skills suck. They usually have an MBA or something that allows them to get promoted over other coworkers that don't have one. Nobody knows how they were smart enough to get an MBA, it is either fake, or they paid someone to take the classes for them to earn it. Or they were smart at one time, but burned out their brain on drugs after getting the degree? *I am free now, to choose my own destiny.*
Post #48,277 by static 8/5/02 9:45:26 PM Reply	Sales skills. There is a point above most technical positions that require a mix of sales and (people) management skills. Often there is actually a band where the positions also require technical skills. Hence the problems of managers who know enough to be dangerous but not enough to be truly useful. I found the best explanation in, of all places, a Dungeons & Dragons guide about the structure of armies. At the very bottom, you will have Level 0 and Level 1 Fighters. You may have some Level-1 Fighters as Coporals but they'd probably be Level 2 or higher. As you get up towards Generals and Colonels, the levels get higher until you might have a Level 16 Fighter as your army's General. But if they're responsible to a king or prince, you may well have a Level 0 or 1 character as the General or who the General reports to - that's your prince or king. (And their most trusted advisors will be a mix of levels and classes, too.) Quite simply, there is a point where the skillset required changes. In modern large company parlance, when you get to the top of the techs, you have one who is capable of managing other techs. He is not entirely a political animal, but he will play some of it. His boss will be less of a tech, or perhaps a tech with old knowledge, but will be capabe of managing and interacting with other managers who aren't techs. By this level, you find people with sales skills because either the people skills for sales and for management are very similar, or they are similar enough that sales people can learn them better than most. Makes job advancement prospects poor for us techs, though. Wade. "Ah. One of the difficult questions."
Post #48,357 by tuberculosis 8/6/02 10:29:19 AM Reply	Yeah well there's a problem here I got a degree in petroleum engineering. So please bear in mind that I have been trained to bore holes up to 2 miles into the earth, guess how much stuff is down there by sucking carefully on one or two holes in cycles over a couple months, and figure out whether what we've found is gold or dust. One of the more interesting classes was engineering economics. Basically, you're given a set of well test data, and are told to figure out how to maximize the return. You design the production plan, calculate the costs, returns, cash flows, etc. Its all about fiscal responsibility because if your plan is too risky or you don't budget your expenditures you end up sitting on a stack of gold bars too heavy to move. Not very valuable. I think techies - and I know this holds for engineers - have had the business training to manage organizations and large projects effectively. But we're not considered for the roles - even if we lobby for them. Something about a boys club I think. Shared ignorance yields camaraderie. That's what I'm finding. I am out of the country for the duration of the Bush administration. Please leave a message and I'll get back to you when democracy returns.
Post #48,362 by boxley 8/6/02 10:38:34 AM Reply	Fear factor If you have the ability to promote you promote someone you are comfortable with, not nescessarily the one best suited for the job. It is a club in a way so schmoozing is a big part of getting and staying there. thanx, bill ."Once, in the wilds of Afghanistan, I had to subsist on food and water for several weeks." W.C. Fields
Post #48,397 by orion 8/6/02 1:16:50 PM Reply	Yeah sort of like that Norm is a good worker, hard working, honest, and his programs don't see to crash the system. Too bad he doesn't go to Happy Hour with us on Fridays and get sht faced. Now Chris isn't that good of a worker, tends to joke around with the other coworkers a lot, his programs take down servers and are hard to debug. But he gets sht faced with us at Happy Hour. I need to promote one of them to be a Project Leader, but which one? Well I feel more comfortable with Chris than Norman, besides Norman is such a good worker that it is most likely some other firm will hire him away in a year or two anyway. Let's promote Chris instead! *I am free now, to choose my own destiny.*
Post #48,147 by orion 8/5/02 11:36:15 AM Reply	VB can create crud apps too Just ask me about all the VB apps I had to debug for my coworkers that crashed systems. Heck, the ASP apps did it too, only they took down servers. Nothing like spending hours over someone else's code looking for code that causes memory leaks and places they forgot to close off record sets and set objects back into nothing to free up memory. Also managled up HTML parsing in ASP was a loser as well. They kept these people but got rid of me? WTF? *I am free now, to choose my own destiny.*
Post #48,154 by admin 8/5/02 12:01:44 PM Reply	Wrong crud... CRUD == Create Read Update Delete Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."

Welcome to IWETHEY!