IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New My sympathies, Drew.
Unprotected development is like walking on eggs.

Did you leave some documentation around the not quite dead code?

Believe it or not, there is the other extreme. Endless meetings with barely knowledgeable people with veto power on moving code from development to test levels never mind moving code to release levels.
Alex

"No man's life, liberty, or property are safe while the legislature is in session."\t-- Mark Twain
New What we came up with
Keep in mind, this is guided by a boss who is a Visual C++ developer, and thinks in terms of compiled executables.

We will no longer have common or shared files included in anything. If you know anything about PHP you know that you write a function (or class), save it to a file, and everywhere you need it you include the file. All these common files go in a directory that is in the PHP include path. In the new regime, whenever a file needs to include a "common" file, that "common" file will get copied to the local directory.

If anyone needs to make a change to a "common" file they can work on it in isolation, test exactly what will be published, and roll it out as tested. If anyone needs to make a change to a "common" file, that progrmmer will be responsible to track down all other instances of that file and schedule the relevant application for a new round of testing against the changes.

So for instance if the DB abstraction class is included by every single page on the site (which it is) then every single directory will have a local copy of that class. If you need to make changes to that class, it's not going to be sufficient to create test cases and roll out the change to a shared file. You will need to individually re-test every single page on the site to make sure it works against the changes before rolling out the change to that directory.

Yes, we've just re-invented DLL hell.

Then we have to talk about development, testing and/or staging environments; CVS; formal rollout/rollback procedures; etc. Whee.
===
Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link|http://z.iwethey.org/forums/render/content/show?contentid=38978|Andrew Grygus]
New VC++ or not, your boss is a moron. Get the heck outta there!
New Stupid.
But about the best you could manage without version control and a managed build system.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New Oh, and ...
One of the other programmers, the only other one who would talk about CVS in the meeting, said that "every other shop in the world" uses it, and that the way they use it is that when one user checks out a file no one else can touch it. Actually every place I've used it or seen it used, everyone keeps a complete copy of the repository checked out to their local development environment, and does update & commit whenever they want to move something up into testing. But hey, what do I know.
===
Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link|http://z.iwethey.org/forums/render/content/show?contentid=38978|Andrew Grygus]
New Actually that's how we work here.
When you lock a file, no one else can change it until you either commit a change or release the lock.

We do have a complete local copy of the repository, but it's read-only.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New Would probably work here, too
The point was, he couldn't conceive of any other way to do it. There was a lot of "this is how it's done" floating around the room, and precious little, "yeah, I could see that too".
===
Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link|http://z.iwethey.org/forums/render/content/show?contentid=38978|Andrew Grygus]
New Gotcha.
Although requiring locks does completely remove the problem of reconciling concurrent changes.

Of course, we don't actually use CVS. PVCS here.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New Same here
In fact, I've never seen a version control system that doesn't work that way: You check out a file, you work on it, and while you are working on it, nobody else can futz with it. When you check it in, then cahn others make their changes. Different VCSs have different rules as to what version one can check out, but I've never seen one that purposely lests more than one user check out the same version of a file for revision simultaneously. (If it did, it'd be no different than the chaos you currently describe.)
jb4
"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning."
Rich Cook
New the one we used in Siemens did
if you went to the file directly using the OS instead of the program, it wouldnt be locked or changes kept.
thanx,
bill
will work for cash and other incentives [link|http://home.tampabay.rr.com/boxley/resume/Resume.html|skill set]

questions, help? [link|mailto:pappas@catholic.org|email pappas at catholic.org]
New That's the 'C' in CVS
Concurrent Versioning System. How I've used it:

There is a base repository, and multiple users each check out a complete copy of it. A copy is also checked out to the staging server. Each user makes whatever modifications to his local copy and, when ready for testing, commits his changes to the repository. Before committing, the user first does an update to see if anyone else has updated it since he checked out his copy. The update is then pulled down to the testing server. If anything is wrong, it is rolled back. The user who committed the changes can either make the fixes and commit again, or back the changes out of the repository.

Typical workflow for me as a user:

  • Check out a complete copy of the repository. I now have an exact mirror of the production system.

  • Make changes, test my local version until I'm happy with it.

  • Do a 'cvs update' to check the repository. Any changes since the last time I pulled an update are merged into what I've got locally. Conflicts are flagged so I can fix them.

  • Once there are no conflicts (there were none or I've fixed them) I do 'cvs commit' and my changes are merged into the repository.

  • Testers then do 'cvs update' to pull the changes down into their copy to test.

  • Changes are tested and look good. They are copied up to the production server

  • -- or --
  • There is a problem. Testers do a rollback to the last known good state.

  • I make minor corrections and commit again.

  • -- or --
  • I roll back the repository version to before my commit. I still have my version localy and can continue to work on it.
This way multiple users can work on the same stuff at the same time and it's handled (somewhat) sanely. In practice you still generally don't want to have peple actively developing the same parts at the same time. But this makes it possible to do system-wide changes like changing a fieldname in the database. One user can make the chnge system-wide, and every user gets the changes next time they do an update.
===
Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link|http://z.iwethey.org/forums/render/content/show?contentid=38978|Andrew Grygus]
New Impossible in some situations.
Conflict resolution is a pain in the ass. We have 70 developers here, and the system is in a constant state of flux. Using the procedure you just described we'd spend all of our time resolving the conflicts.

workflow here:

1) Create a build directory. This contains a local readonly copy of all the source.
2) Lock a file you want to change.
3) When finished, check it back in. The file is now available for other people to see, but it isn't part of the build yet.
4) When your changes in a particular module are ready to be released to the DEV environment, you release the module. This tags all the current files you have checked out for that module as belonging to a particular release. The build system here will warn you if a) there have been interim releases while you have been doing your work, in which case you need to pull down the the newer releases, then your head revisions of the files you changed, then release, or b) if you release while someone else has a file in that module checked out (this is more or less just a warning to let them know).
5) At night, all modules that have been released that day get promoted to the DEV build and built with everything else. If you release a module and it breaks the build that night, you have to fix it the next day.
6) When something has been building/running in DEV for a bit, it can be promoted to the TST environment. Same deal. Promotion tags the module as part of the current TST build.
7) Repeat 6 for QA and PRD (PRD is more complex, but outside the context of this discussion).

We also have validation environments (DEV7, DEV8) that are shadows of DEV1. Every day the test suites get run on DEV7/8 to make sure everything is kosher. If a test fails, the last person to release that module gets an automated nastygram. ;-)

There is still a little conflict resolution to be done if someone releases a module you are working in, but in practice this happens quite infrequently. When it does, the fix is usually as simple as checking out the head revisions of the new files and re-releasing.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New must be annoying to work for a sane/logical group :-)
will work for cash and other incentives [link|http://home.tampabay.rr.com/boxley/resume/Resume.html|skill set]

questions, help? [link|mailto:pappas@catholic.org|email pappas at catholic.org]
New Hee hee hee.
It's great fun here.

The build system is one of the best I've ever worked with. It's a combination of PVCS (could be any vcs, really) and a number of perl scripts that manage the entire process.

We can also run a single command to build the entire system starting with an empty directory, database and all (creates the tablespaces, everything), generate a zip file, then remotely install that zip file of code on the web and process servers.

Saves a helluva lot of time.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New Is your system available/useful to anyone else?
It sounds like you've already solved much of what we need to do. Is there any chance you could share/release the build system, or is that considered trade secret stuff?
===
Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link|http://z.iwethey.org/forums/render/content/show?contentid=38978|Andrew Grygus]
New Available? I don't know? Useful? Partially.
Some of it is definitely relevant to our systems only. A fair portion of it is geared heavily towards our use of Oracle, as well.

So I guess the answer is, I don't know. I've never asked.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New I just wonder...
How often you do have to wait for the file to be released before you check it out? In practice?
--

We have only 2 things to worry about: That
things will never get back to normal, and that they already have.
New I've been here 1.5 years...
... and I've had to wait for one file that I can remember. I think I've made someone else wait a few times at the most.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New Possible in more situations than most would think
Most people when they see CVS think that something like that can't possibly work for them.

Some are right.

Most aren't.

I am not saying that you are wrong. But in several years of use with an admittedly smaller team I have seen conflicts..once. And if your work is scattered across enough files that you almost never run across concurrent locks (as you describe below), then I would be willing to bet large sums that conflicts would not be an issue in practice.

Integration with your build system is, however, a different issue entirely...

Cheers,
Ben
"good ideas and bad code build communities, the other three combinations do not"
- [link|http://archives.real-time.com/pipermail/cocoon-devel/2000-October/003023.html|Stefano Mazzocchi]
New Here's the issue:
I would be willing to bet large sums that conflicts would not be an issue in practice.
The only reason they aren't here is because the locking system prevents it.

Let me explain: if people were allowed to have copies where they could change anything, we'd end up with a number of people who rarely updated. When someone finally did remember to update, there would be mass chaos. When you don't have to lock something, there is a tendency to just make the change and forget it. Eventually you update a large number of files. The result of this in practice is that your window of conflict goes up for all of the files you have changed, for the entire period between updates.

With a locking system, the tendency is to grab the file, make your change, and check it back in. The window of conflict for that piece is very very small.

In practice, the above has been borne out. During my recent porting stint, I have kept large numbers of files checked out for long periods of time; weeks for many of them. I have gotten a lot more requests that I unlock/finish with a file this past month or so than I ever have; the statistics I gave in the other post were from when I was using the system normally, not from porting. Porting efforts aren't normal as they touch large portions of the system at a time, and in tandem.

To sum up: non-locking systems with large numbers of developers will tend to widen the window of conflicts, as the effective contention period becomes much longer. Locking systems encourage short bursts of contention. I suppose an analogy would be a long running transaction vs. a short one.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New That behaviour issue is addressible
Get people in the habit of proactively updating at least daily. It isn't that hard to get them to do it. And it means that the possibility of conflict is addressed early and often. And if you choose not to do this you mainly create problems for yourself. :-)

For porting, create a branch and then run a command to detect conflicts every day. Similarly if you suspect that developers are not keeping good habits, you can run a cron job to find conflicts so you can come by and give them the speesh.

It can work. Many groups make it work for them. Bad developers can make it really suck, but they can do the same with any other source control system as well...

Cheers,
Ben
"good ideas and bad code build communities, the other three combinations do not"
- [link|http://archives.real-time.com/pipermail/cocoon-devel/2000-October/003023.html|Stefano Mazzocchi]
New That behaviour issue is addressible - in some orgs.
Get people in the habit of proactively updating at least daily.
Same issue. Daily window of multiple files vs. typically an hour window or so for a single file.

And if you choose not to do this you mainly create problems for yourself.
No, you create issues for the entire group as multiple files need to be reconciled before the project can be built.

For porting, create a branch and then run a command to detect conflicts every day.
Not feasible. I would have detected dozens of conflicts each day.

Similarly if you suspect that developers are not keeping good habits, you can run a cron job to find conflicts so you can come by and give them the speesh.
Easier said than done, and not in keeping with the culture we have here.

Many groups make it work for them.
Perhaps. Perhaps many groups make it work for them because they don't know any better. Not many groups have a code base consisting of 2M lines of code and 70 very active developers. Even large open source projects don't have the same level of activity that we do; people are working on this code base all day long for 8-12 hours a day. As a comparison, this is about twice as many loc as the gcc 3.2.1 C++ compiler, and nearly 30 times larger than Apache.
Regards,

-scott anderson

"Welcome to Rivendell, Mr. Anderson..."
New I'm not trying to convince you, honest
Part of what makes CVS work is that it is usually OK if multiple people edit the same file at the same time. Because usually people are editing unrelated parts of the file.

The claim that the person who keeps the file out loses is based on the fact that they can't checkin until they have noted and resolved the conflict. Most conflicts actually turn out to be easy to resolve. I believe that you can also arrange for some key people to be notified in the event of conflict so that you can be sure that people resolve it correctly.

As for the question of who makes it work and why, you have not described any conflict issue yet that makes me believe that CVS is unworkable for you. However I suspect that raw performance issues from some of CVS' algorithmic mistakes are a good reason for you not to use it.

Cheers,
Ben

PS The cron job that I mentioned is trivial to write. Whether your culture is such that people would take kindly to feedback is another story, but a group of developers that cannot give each other feedback is a group that is likely to wind up with climbing tension levels...
"good ideas and bad code build communities, the other three combinations do not"
- [link|http://archives.real-time.com/pipermail/cocoon-devel/2000-October/003023.html|Stefano Mazzocchi]
New That's how we use CVS
(Concurrent VS)

In practice, it's very rare that 2 developers who work on different problems produce conflicts. And those conflicts are easily resolved when developers talk to each other.

A really serious problem arises when you do major refactoring. Well, for those case one may want to lock the tree, after all.
--

We have only 2 things to worry about: That
things will never get back to normal, and that they already have.
New Same now, used to be better
We use Visual SourceSafe, and we disallow simultaneous updates, also.

One person checks it out at a time.

However, when I worked on the mainframe, we wrote our own source control system and here's what IT did.

1. When subsequent persons checked out the program, they would be subscribed to a list of updaters, the program would be checked out, but it would e-mail (publish) to all current "users", that another user had the file also.
2. When any other person checked in, un-did checkout, etc., it would email the list of persons who had checked out the file. It was publish/subscribe, or rather subscribe/publish.
3. Also, remember this was mainframe days, it would notify all subscribers when you changed the version of the program to the primary test system, or to any of the departmental test systems. It tracked program versions on some 120 working test subsystems, called "virtual machines" back then.
4. This was a lifesaver, because you alway knew exactly who else was working on the same code as you, and you "worked it out".
5. Each version of a program module was tracked through the heirarchy of test systems, until it was finally put into the production system, in SABRE.
6. Finally, this was all done in 1986 in VM and Profs for 370 Assembler Language computer systems, over 15 years ago. It's still in use at SABRE/EDS/American Airlines, and I haven't seen a better system, yet.

It was called AIS, written by three programmers at American, and I'm not even sure one of them had a college degree.
Expand Edited by gdaustin Feb. 15, 2003, 01:00:35 AM EST
Expand Edited by gdaustin Feb. 15, 2003, 01:00:43 AM EST
     Saying "I told you so" isn't nearly so satisfying ... - (drewk) - (34)
         My sympathies, Drew. - (a6l6e6x) - (24)
             What we came up with - (drewk) - (23)
                 VC++ or not, your boss is a moron. Get the heck outta there! -NT - (CRConrad)
                 Stupid. - (admin) - (21)
                     Oh, and ... - (drewk) - (20)
                         Actually that's how we work here. - (admin) - (19)
                             Would probably work here, too - (drewk) - (1)
                                 Gotcha. - (admin)
                             Same here - (jb4) - (16)
                                 the one we used in Siemens did - (boxley)
                                 That's the 'C' in CVS - (drewk) - (12)
                                     Impossible in some situations. - (admin) - (11)
                                         must be annoying to work for a sane/logical group :-) -NT - (boxley) - (3)
                                             Hee hee hee. - (admin) - (2)
                                                 Is your system available/useful to anyone else? - (drewk) - (1)
                                                     Available? I don't know? Useful? Partially. - (admin)
                                         I just wonder... - (Arkadiy) - (1)
                                             I've been here 1.5 years... - (admin)
                                         Possible in more situations than most would think - (ben_tilly) - (4)
                                             Here's the issue: - (admin) - (3)
                                                 That behaviour issue is addressible - (ben_tilly) - (2)
                                                     That behaviour issue is addressible - in some orgs. - (admin) - (1)
                                                         I'm not trying to convince you, honest - (ben_tilly)
                                 That's how we use CVS - (Arkadiy)
                                 Same now, used to be better - (gdaustin)
         I work solo on most projects... - (ChrisR) - (4)
             I think that's an underlying issue here - (drewk) - (3)
                 Dead code removal is a vital function. - (admin) - (2)
                     Agreed - (ChrisR) - (1)
                         We have both, would you like some of ours? -NT - (drewk)
         Speaking on a related topic... - (ben_tilly) - (2)
             Yup: We unplug the HP3000s - (drewk) - (1)
                 Better than you thought, huh? - (ben_tilly)
         Prototype folder - (orion)

Sorry. I don't see the benefit for me in that arrangement.
103 ms