Died and gone to heaven, part 2

Post #158,520 by broomberg 6/5/04 10:34:00 AM Reply	Died and gone to heaven, part 2 It is too early to fund a Blade Server project, but we have something coming down the pike that we need hardware for, FAST! This was initially designed as 2 NT file servers, and a bunch of NT compute servers. 1 file server would be accessed by general interactive windows users for the specific project. The other file server would have to hold at least a TB of files, as the compute server read/writes in a lot of bursts, using the results from the 1st server all well. The compute server is running a 3rd party app that costs a LOT. It also runs on EITHER Linux or NT. YES! Someone else did the initial design, having all NT. I pointed out that it was my job to automate the compute app, which would be a lot easier if we used the Linux version. Boss: "It runs under Linux?" Me: "Yup" Boss: "Cool, compute server is now Linux" And then, since the most critical piece was running under Linux, we then reviewed the need for NT servers vs Linux Samba/NFS servers. Suddenly, everything was Linux. I then pointed out that using cheap compute servers made no sense since the cost of the app was per box, which means we should buy the fastest multi-CPU box we could afford for it. Which ends up being an HP Quad Opteron 2.2Ghz. 10GB of RAM. YEAHYEAHYEAHYEAHYEAH!!! The vendor says the app is VERY compute intensive, and we should expect to bottleneck on CPU. We have multiple people submitting these jobs on an ongoing basis. We currently do this processing on our mainframe. I expect a 40-100 times speed increase as compared to our current MF processing. This is because we've tested this app on regular 1.7Ghz PCs and seen it running about 5 times faster than the MF process. But once it is running THAT fast, it means either the jobs will finish so much sooner that the box is not being used 98% of the time, or we wouldn't mind if they ran a LITTLE slower and made use of the CPUs for more general purpose jobs. We really want them to run as fast as possible since this is a critical step on our print production process. The issue is how fast can we run as many of these jobs when they are ready to run. But I can sacrifice 5% of CPU based on the huge difference. And once we consider this box, do we REALLY need separate file servers? The goal here is to maximize the compute performance, which in turn is much happier for local file system than over NFS. And now that we have over double the local performance available than in initial design (4 Opterons VS 2 Xeons, do we really need a ANY separate file servers? HELL NO! We had multiple single points of failure before, now we have one, which we can then mirror to a slower box if we want to. We buy the "6 hour to fix" service, so a worst case scenario should be 7 hours. I like separate software style rsync mirrors, which I'll maintain to some additional storage, just in case. The file server aspect is very small and bursty to I'm confident this will not cause a problem. So we've simplified the design, lost a bunch of software costs, sped of the processing (FC attached dedicated array, faster CPUs), and gained additional application scalability since this will run over twice as much work for the same app license cost, while totally dropping the NT costs. Oh, and the total hardware cost is less than the initial design. The only thing that is new to me is core usage of Samba, for Windows desktop usage, for users across multiple Windows domains. For this I believe we NEED Samba 3. Anyone have experience with it for this type of usage?
Post #158,600 by folkert 6/5/04 10:49:10 PM Reply	\\o/ <- Me Waving I can send you my smb.conf for use on linux and 8 smal scripts I wrote to make the SAMBA PDC seem nearly identiucal to a NT Domain. Means no AD though (whoopty who care) I also have a good reference I am re-writing to be more "distro neutral" I am \|\| this far away from having LDAP in place for golbal authentication... and then I can replicate away on everything. The only things I am not doing(yet) IS ldap and NTLMv2 as I have an OLD win95 box that will NOT use it, even though it can. There are TONS TONS of things you can do. Right now I have cups as my print backend, and I am just starting to load up the drivers for all the printers. I have some basic policies in place (some user and a few group) allowing me to control a few things. Roaming Profiles work great. I can use svrmgr.exe to manage the domain now (cept a few thing) I can add users and they will get UNIX account (if I want, which I do) and they will be the same passwd. I can add groups with specific SID. I have a software depot share, I can add users to groups (and be reflected in unix and samba) and remove them as well. I can establish domain trusts, workstation accounts(to join the domain) reset passwords remotely on workstations using swat, I also could if I wanted to have samba control the dynamicDNS updates. There are more things that I can't even begin to think of samba is now doing. Just make sure you start with 3.0.4 or later. Trust me, I just spent the entire afternoon rebuilding an upgrade gone sour from 3.0.2a to 3.0.4 because of the recent GDMFAHBSSM$ using QUALITY TESTING they use. "Hey! It builds! SHIP IT!" Twas a hung process that I didn't catch soon enough. Corrupting nearly all of the tdbs. The hung process was during a logrotate event. SUXXORZ. But SAMBA did still run, just after about 4-6 hours it would stop allowing NEW domain logins, but would still allow share mapping (which was the workaround) Enough... SAMBA is plenty light weight especially with LDAP and no winbind being needed (winbind does the external AD or domain authentication and automagic homedir creation etc...) Will be putting a neutered version of my smb.conf up for perusal. Sometime soon. -- [link\|mailto:greg@gregfolkert.net\|greg], [link\|http://www.iwethey.org/ed_curry\|REMEMBER ED CURRY!] @ *i_wethey* I've decided to become a perfectionist. That way I'll have more reasons to hate people. Your recycled electrons annoy me. Please use new electrons.
Post #158,652 by broomberg 6/6/04 3:35:20 PM Reply	Sounds good I'm considering moving my consulting to an 80% Linux/Samba server environment, so this might help. With my regular job, though, the key issue is AD integration.
Post #158,683 by folkert 6/6/04 9:07:50 PM Reply	Well, quick question. Why would you need MADS when all of your Servers are SAMBA? You can join a Win 2K3/2K servers to a samba domain, easy peasy. You can manage them with MMC just like you could before. You just can't use MMC with the SAMBA Domain (for that SRVTOOLS are needed) Since you can do ~ 90% of the stuff in AD with a DOMAIN you might wanna consider going without AD. You can get the Kerberos to issue proper tickets and have LDAP do the replication stuff. Amoung other things. -- [link\|mailto:greg@gregfolkert.net\|greg], [link\|http://www.iwethey.org/ed_curry\|REMEMBER ED CURRY!] @ *i_wethey* I've decided to become a perfectionist. That way I'll have more reasons to hate people. Your recycled electrons annoy me. Please use new electrons.
Post #158,709 by broomberg 6/6/04 11:52:50 PM Reply	No, you don't understand My regular job has LOTS of W2K servers. We have an AD FARM!!!!! I mean it. We have 12 servers DEDICATED to AD. We have 3 domains, which were kind of screwed up. And then Exchange started having problems. So the M$ company that was reviewing our setup said we NEEDED 12 boxes to handle our AD traffic in correct isolation. We ended up with 12 dual Xeon boxes JUST for that. People were NOT happy, but they were backed in a corner. I don't mind too much, because I now use it as an example of crappy technology put in by novices, which then burns you when you depend on it and it fails. Very convincing argument to doing things the right way.
Post #158,738 by pwhysall 6/7/04 2:09:48 AM Reply	Twelve? I could understand two per domain, but FOUR? Someone was getting some fine commission on THAT hardware deal. (If they're just DCs, dual processors aren't required, either) Peter [link\|http://www.debian.org\|Shill For Hire] [link\|http://www.kuro5hin.org\|There is no K5 Cabal] [link\|http://guildenstern.dyndns.org\|Blog]
Post #158,761 by broomberg 6/7/04 6:41:56 AM Reply	Yup: Twelve And I believe it was 3 per domain, plus a bit over overhead. And yes, I'm sure that they are sitting 90% idle. But I'm not in the position to look at it too close. It was an emergency install, the whole thing done in about 48 hours, from start to finish, and they guarantee our email will never go down due to AD issues ever again. We will be consolidating domains in about 6 months, at which point most of the hardware will be redeployed. On hardware pricing side, we got a good deal. Same cost from this all-in-one vendor as our lowest discounted CDW price. These are HPs.
Post #158,783 by folkert 6/7/04 10:28:11 AM Reply	OUCH. Sounds like you have your SAMBA servers already. Use Winbind for the intergration... it just works REEL-GUUD. It can be configured to handle the three domains as well. Nifty schpifty. 12 machines... someone got schnookered. -- [link\|mailto:greg@gregfolkert.net\|greg], [link\|http://www.iwethey.org/ed_curry\|REMEMBER ED CURRY!] @ *i_wethey* I've decided to become a perfectionist. That way I'll have more reasons to hate people. Your recycled electrons annoy me. Please use new electrons.
Post #158,800 by drewk 6/7/04 11:09:13 AM Reply	D'oh! Why didn't I ask here sooner (new thread) Created as new thread #158799 titled [link\|/forums/render/content/show?contentid=158799\|D'oh! Why didn't I ask here sooner] === Implicitly condoning stupidity since 2001.
Post #158,919 by broomberg 6/7/04 10:19:11 PM Reply	Ordered The IT dept manager is used to getting HP stuff within a week, I told her she'd be lucky to get it in a month. We split the difference, betting $10. She said it will ship within 2 weeks, I say over. Easy money. These things are on indefinite backorder.
Post #158,921 by drewk 6/7/04 10:27:10 PM Reply	Talk about a no-lose situation === Implicitly condoning stupidity since 2001.
Post #159,383 by broomberg 6/10/04 3:58:19 PM Reply	And I lost!!!! It shipped yesterday. But 2 CPUs are on backorder. So I offered to pay, since I'll have a working box. But the IT manager refused, saying the CPUs will ship before the 18th.
Post #159,002 by tonytib 6/8/04 11:38:18 AM Reply	From what the The Inquirer says, with good reason The Inquirer's had a bunch of articles on how the HP quad Opteron eats Xeon's for lunch. So I wouldn't be surprised if they were back ordered. Tony
Post #159,101 by broomberg 6/8/04 6:58:12 PM Reply	See the 8 way blurb? I WANT!
Post #159,245 by tonytib 6/9/04 3:07:36 PM Reply	Well, for personal use, how about the Dual Opteron SFF? [link\|http://www.iwill.net/zmax/zmaxdp_1.htm\|Dual Opteron Small Form Factor PC from Iwill] Very interesting for a personal workstations. It'll be interesting to see how the 8-way Opteron servers affect the server space (especially if Dell stays Intel-only). I also wonder how much the motherboard affects performance. Tyan has a $1600 quad Opteron motherboard; it'd be interesting to see how it compares to HP's quad Opteron. Tony
Post #159,251 by folkert 6/9/04 3:17:36 PM Reply	I want that one for work... :) -- [link\|mailto:greg@gregfolkert.net\|greg], [link\|http://www.iwethey.org/ed_curry\|REMEMBER ED CURRY!] @ *i_wethey* I've decided to become a perfectionist. That way I'll have more reasons to hate people. Your recycled electrons annoy me. Please use new electrons.

Welcome to IWETHEY!