multi-instance...

Post #123,227 by slugbug 10/29/03 10:34:11 AM Reply	multi-instance... Actually, yes, I have broken the data down into smaller sizes and am running multiple instances now. However, I'd like to better take advantage of available processors (14 of them) and memory (42GB). This is a repeatable process to parse some data prior to updating a data warehouse, fyi...
Post #123,232 by ben_tilly 10/29/03 10:43:47 AM Reply	If this is a Unixy system, definitely go with fork() The bulk of the interpreter will remain shared between children, saving memory. I don't know the threading implementation well enough to know if that holds there. Also the fact that these are separate processes is a big hint to the OS that it doesn't even have to think about concurrency issues. Some versions of *nix actively try to make it more likely that threads will stay on one processor to minimize locking/unlocking costs. Processes are more likely to be migrated freely to balance the load. Plus fork() has been around for longer, and so has better guarantees of stability. Cheers, Ben "good ideas and bad code build communities, the other three combinations do not" - [link\|http://archives.real-time.com/pipermail/cocoon-devel/2000-October/003023.html\|Stefano Mazzocchi]
Post #123,247 by slugbug 10/29/03 11:33:16 AM Reply	Yes... ....in this particular runtime, there is a correlation between threads and processors. Not overly familiar with fork(), but will read...thanks again for the help. -Slugbug If you think you can, or you think you can't, you're right. -Henry Ford
Post #123,238 by deSitter 10/29/03 11:00:52 AM Reply	Re: multi-instance... !!! What are you working on, the Forbin project?? 14 processors! Holy Cran! Must be an E10000 with a lot of cooling fans! -drl
Post #123,240 by admin 10/29/03 11:07:31 AM Reply	Er... We've got a 6000 with 24 processors here, IIRC... :-) Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."
Post #123,242 by deSitter 10/29/03 11:14:20 AM Reply	DAMMIT! I WANT TOYS! -drl
Post #123,243 by slugbug 10/29/03 11:20:40 AM Reply	toys, toys, toys... ...we have 60+ systems and most of these are 10-36 processor range with tons of disk and memory. Fun to partition some of these and run Linux (except for upgrade issues); these boxes can run multiple OSes concurrently and can shift memory, processor, and disk based on scheduled settings. By early next year, these same boxes will be able to shift memory, processor, and disk dynamically based on workload at any given time. Reminds me.....its about time to get some more personal toys for my lab :-) -Slugbug
Post #123,245 by deSitter 10/29/03 11:25:50 AM Reply	Re: toys, toys, toys... Need any help? Have truck and bike will travel :) -drl
Post #123,250 by admin 10/29/03 11:34:11 AM Reply	Re: toys, toys, toys... (new thread) Created as new thread #123249 titled [link\|/forums/render/content/show?contentid=123249\|Re: toys, toys, toys...] Regards, -scott anderson "Welcome to Rivendell, Mr. Anderson..."

Welcome to IWETHEY!