Impossible in some situations.

Post #81,648 by admin 2/14/03 10:41:32 AM Reply	Impossible in some situations. Conflict resolution is a pain in the ass. We have 70 developers here, and the system is in a constant state of flux. Using the procedure you just described we'd spend all of our time resolving the conflicts. workflow here: 1) Create a build directory. This contains a local readonly copy of all the source. 2) Lock a file you want to change. 3) When finished, check it back in. The file is now available for other people to see, but it isn't part of the build yet. 4) When your changes in a particular module are ready to be released to the DEV environment, you release the module. This tags all the current files you have checked out for that module as belonging to a particular release. The build system here will warn you if a) there have been interim releases while you have been doing your work, in which case you need to pull down the the newer releases, then your head revisions of the files you changed, then release, or b) if you release while someone else has a file in that module checked out (this is more or less just a warning to let them know). 5) At night, all modules that have been released that day get promoted to the DEV build and built with everything else. If you release a module and it breaks the build that night, you have to fix it the next day. 6) When something has been building/running in DEV for a bit, it can be promoted to the TST environment. Same deal. Promotion tags the module as part of the current TST build. 7) Repeat 6 for QA and PRD (PRD is more complex, but outside the context of this discussion). We also have validation environments (DEV7, DEV8) that are shadows of DEV1. Every day the test suites get run on DEV7/8 to make sure everything is kosher. If a test fails, the last person to release that module gets an automated nastygram. ;-) There is still a little conflict resolution to be done if someone releases a module you are working in, but in practice this happens quite infrequently. When it does, the fix is usually as simple as checking out the head revisions of the new files and re-releasing. Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."
Post #81,651 by boxley 2/14/03 10:44:35 AM Reply	must be annoying to work for a sane/logical group :-) will work for cash and other incentives [link\|http://home.tampabay.rr.com/boxley/resume/Resume.html\|skill set] questions, help? [link\|mailto:pappas@catholic.org\|email pappas at catholic.org]
Post #81,654 by admin 2/14/03 10:50:11 AM Reply	Hee hee hee. It's great fun here. The build system is one of the best I've ever worked with. It's a combination of PVCS (could be any vcs, really) and a number of perl scripts that manage the entire process. We can also run a single command to build the entire system starting with an empty directory, database and all (creates the tablespaces, everything), generate a zip file, then remotely install that zip file of code on the web and process servers. Saves a helluva lot of time. Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."
Post #81,668 by drewk 2/14/03 12:09:12 PM Reply	Is your system available/useful to anyone else? It sounds like you've already solved much of what we need to do. Is there any chance you could share/release the build system, or is that considered trade secret stuff? === Microsoft offers them the one thing most business people will pay any price for - the ability to say "we had no choice - everyone's doing it that way." -- [link\|http://z.iwethey.org/forums/render/content/show?contentid=38978\|Andrew Grygus]
Post #81,677 by admin 2/14/03 12:18:32 PM Reply	Available? I don't know? Useful? Partially. Some of it is definitely relevant to our systems only. A fair portion of it is geared heavily towards our use of Oracle, as well. So I guess the answer is, I don't know. I've never asked. Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."
Post #81,663 by Arkadiy 2/14/03 11:36:49 AM Reply	I just wonder... How often you do have to wait for the file to be released before you check it out? In practice? -- We have only 2 things to worry about: That things will never get back to normal, and that they already have.
Post #81,676 by admin 2/14/03 12:16:51 PM Reply	I've been here 1.5 years... ... and I've had to wait for one file that I can remember. I think I've made someone else wait a few times at the most. Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."
Post #81,738 by ben_tilly 2/14/03 6:15:58 PM Reply	Possible in more situations than most would think Most people when they see CVS think that something like that can't possibly work for them. Some are right. Most aren't. I am not saying that you are wrong. But in several years of use with an admittedly smaller team I have seen conflicts..once. And if your work is scattered across enough files that you almost never run across concurrent locks (as you describe below), then I would be willing to bet large sums that conflicts would not be an issue in practice. Integration with your build system is, however, a different issue entirely... Cheers, Ben "good ideas and bad code build communities, the other three combinations do not" - [link\|http://archives.real-time.com/pipermail/cocoon-devel/2000-October/003023.html\|Stefano Mazzocchi]
Post #81,742 by admin 2/14/03 6:27:11 PM Reply	Here's the issue: I would be willing to bet large sums that conflicts would not be an issue in practice. The only reason they aren't here is because the locking system prevents it. Let me explain: if people were allowed to have copies where they could change anything, we'd end up with a number of people who rarely updated. When someone finally did remember to update, there would be mass chaos. When you don't have to lock something, there is a tendency to just make the change and forget it. Eventually you update a large number of files. The result of this in practice is that your window of conflict goes up for all of the files you have changed, for the entire period between updates. With a locking system, the tendency is to grab the file, make your change, and check it back in. The window of conflict for that piece is very very small. In practice, the above has been borne out. During my recent porting stint, I have kept large numbers of files checked out for long periods of time; weeks for many of them. I have gotten a lot more requests that I unlock/finish with a file this past month or so than I ever have; the statistics I gave in the other post were from when I was using the system normally, not from porting. Porting efforts aren't normal as they touch large portions of the system at a time, and in tandem. To sum up: non-locking systems with large numbers of developers will tend to widen the window of conflicts, as the effective contention period becomes much longer. Locking systems encourage short bursts of contention. I suppose an analogy would be a long running transaction vs. a short one. Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."
Post #81,763 by ben_tilly 2/14/03 7:28:14 PM Reply	That behaviour issue is addressible Get people in the habit of proactively updating at least daily. It isn't that hard to get them to do it. And it means that the possibility of conflict is addressed early and often. And if you choose not to do this you mainly create problems for yourself. :-) For porting, create a branch and then run a command to detect conflicts every day. Similarly if you suspect that developers are not keeping good habits, you can run a cron job to find conflicts so you can come by and give them the speesh. It can work. Many groups make it work for them. Bad developers can make it really suck, but they can do the same with any other source control system as well... Cheers, Ben "good ideas and bad code build communities, the other three combinations do not" - [link\|http://archives.real-time.com/pipermail/cocoon-devel/2000-October/003023.html\|Stefano Mazzocchi]
Post #81,772 by admin 2/14/03 8:04:10 PM Reply	That behaviour issue is addressible - in some orgs. Get people in the habit of proactively updating at least daily. Same issue. Daily window of multiple files vs. typically an hour window or so for a single file. And if you choose not to do this you mainly create problems for yourself. No, you create issues for the entire group as multiple files need to be reconciled before the project can be built. For porting, create a branch and then run a command to detect conflicts every day. Not feasible. I would have detected dozens of conflicts each day. Similarly if you suspect that developers are not keeping good habits, you can run a cron job to find conflicts so you can come by and give them the speesh. Easier said than done, and not in keeping with the culture we have here. Many groups make it work for them. Perhaps. Perhaps many groups make it work for them because they don't know any better. Not many groups have a code base consisting of 2M lines of code and 70 very active developers. Even large open source projects don't have the same level of activity that we do; people are working on this code base all day long for 8-12 hours a day. As a comparison, this is about twice as many loc as the gcc 3.2.1 C++ compiler, and nearly 30 times larger than Apache. Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."
Post #82,022 by ben_tilly 2/16/03 10:02:49 AM Reply	I'm not trying to convince you, honest Part of what makes CVS work is that it is usually OK if multiple people edit the same file at the same time. Because usually people are editing unrelated parts of the file. The claim that the person who keeps the file out loses is based on the fact that they can't checkin until they have noted and resolved the conflict. Most conflicts actually turn out to be easy to resolve. I believe that you can also arrange for some key people to be notified in the event of conflict so that you can be sure that people resolve it correctly. As for the question of who makes it work and why, you have not described any conflict issue yet that makes me believe that CVS is unworkable for you. However I suspect that raw performance issues from some of CVS' algorithmic mistakes are a good reason for you not to use it. Cheers, Ben PS The cron job that I mentioned is trivial to write. Whether your culture is such that people would take kindly to feedback is another story, but a group of developers that cannot give each other feedback is a group that is likely to wind up with climbing tension levels... "good ideas and bad code build communities, the other three combinations do not" - [link\|http://archives.real-time.com/pipermail/cocoon-devel/2000-October/003023.html\|Stefano Mazzocchi]

Welcome to IWETHEY!