IWETHEY v. 0.3.0 | TODO
1,095 registered users | 0 active users | 0 LpH | Statistics
Login | Create New User
IWETHEY Banner

Welcome to IWETHEY!

New How can an external process tell?
New AFAIK, it can't.
There's a certain amount of space that's set aside for references. If you're running out of that, it could cause the effect you describe. You could try changing the amount of heap available and see if it affects how long the process lasts before it goes tits up. Alternatively, on warp you could use theseus to monitor memory usage while it's running.

I'm not sure that you can find out without sticking some debug type messages in the code where you might think there's a problem.

To set the heap size, use the following options on the java command line:

-Xms<size> set initial Java heap size
-Xmx<size> set maximum Java heap size
-Xss<size> set java thread stack size

The one you're interested in is probably the -Xmx option. Try setting it low and see if the program cacks out early. If that happens, it could mean that references aren't being dereferenced properly; could be that sockets aren't being released after they're done, or it could be some other object not being destroyed after the program's finished with it.

Hard to say without looking at the code. Which I'm sure you don't have.

Does the system otherwise behave itself? Your statement about being able to ssh in seems to indicate that it does... do you have to reboot the machine to get it back to normal operation, or kill and restart the "java someclass" process? If you have to reboot the box, that indicates that it's outside the actual server program; if stopping and restarting the VM fixes it that means it's internal to either the version of the JVM you're using or internal to the actual program itself.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New Erm?
[...] or it could be some other object not being destroyed after the program's finished with it.

I thought that couldn't happen with Java, as its vaunted garbage collection is supposed to collect and destroy all objects after the program is finished with them (and sometimes, before the program's finished with them...).

After all, Java's all about protecting the poor, benighted (l)user from just such an occurrence, innit?


[edit: tyop]
jb4
shrub●bish (Am., from shrub + rubbish, after the derisive name for America's 43 president; 2003) n. 1. a form of nonsensical political doubletalk wherein the speaker attempts to defend the indefensible by lying, obfuscation, or otherwise misstating the facts; GIBBERISH. 2. any of a collection of utterances from America's putative 43rd president. cf. BULLSHIT

Expand Edited by jb4 Dec. 9, 2005, 02:07:09 PM EST
New No, not necessarily.
It destroys things for you automagically if there's no possibility of it being referenced again. However, if say you add some new objects to a persistent collection while someone's connected, and don't remove them after they've disconnected, then those objects will hang out till doomsday as there is the possibility of them being referenced from the collection.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New And not necessarily again
Garbage collection runs partly based on how much memory pressure there is. If you have lots of memory available then it doesn't bother collecting garbage aggressively, which can be a problem when other kinds of scarce resources are being tied up.

This is particularly a problem if there is, say, a tight loop that ties up a handle on every iteration. You can easily tie up a lot of handles between garbage collection runs, and it isn't obvious that you are leaking handles.

Cheers,
Ben
I have come to believe that idealism without discipline is a quick road to disaster, while discipline without idealism is pointless. -- Aaron Ward (my brother)
New A) Thanks. B) Harrumph!
A) Thanks for the info. I had no idea that Java was even more benighted than I had originally thought.

B) Harrumph! In spite of all the raves by Blanchard and others who tend to equate garbage collection and other associated protect-one-from-one's-self flotsam with the (next) coming of Christ, it seems that once again, all the whistles, bells and protecting from oneself any PhD candidate or someone named after a goose can possibly throw into a language processor all gets trumped by simple, good ol-fashioned disciplined programming.
jb4
shrub●bish (Am., from shrub + rubbish, after the derisive name for America's 43 president; 2003) n. 1. a form of nonsensical political doubletalk wherein the speaker attempts to defend the indefensible by lying, obfuscation, or otherwise misstating the facts; GIBBERISH. 2. any of a collection of utterances from America's putative 43rd president. cf. BULLSHIT

New Well, of course you need disciplined programming
Just because you don't have to deal directly with memory management doesn't mean that therefore you can act like it doesn't exist.

A bad algorithm is still a bad algorithm and will take longer and use more resources than a good one. Everybody's got more resources now than they used to, but they're not infinite, and it's entirely possible to use them up.

do i over someArray
do j over someArray
do k over someArray
someArray[k] += 1
end
someArray[j] += 1
end
someArray[i] += 1
end

is going to suck no matter what language you do it in.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New Stupid "never took a CS course" question
So how else do you iterate over every possible combination?
===

Purveyor of Doc Hope's [link|http://DocHope.com|fresh-baked dog biscuits and pet treats].
[link|http://DocHope.com|http://DocHope.com]
New If you have to iterate over every possible combination
you're kinda fucked. The real task is figuring out how NOT to iterate over every possible combination.


For example, the closest pair of points problem. How do you take a collection of points and figure out which pair are the closest together? Well, one way is to compare every point's distance to every other point.

Assume each point is stored in an array foo (this is in object rexx, still my most familiar language;) and you're going to store the distances in array bar

do i = 1 to foo~items\n   do j = 1 to foo~items\n      if i = j then do\n         bar[i][j] = 0\n         iterate\n         end\n      bar[i][j] = foo~distance(i, j) /* assuming a method for finding distance exists for array foo */\n      end\n   end\n\ndo i = 1 to bar~items\n   sort(bar[i])\n   end\n\nsort(bar)


This really sucks. It's an n^2 complexity, and will take a long time for any set of points that's more than trivially large.

However, another way to do it is to break up the problem into manageable subsets. For example, sort the array of points by the x position (ie- each point is delineated by (x,y,z) coordinates).

To compare a set of three points takes three operations, so you use that as your base case; break the array of points sorted by the x coord into sets of three, and find the smallest distance in each. Now, you will have an idea of where the divisions are between them (they will be x1+x2/2 for the adjacent points on each side of the division) and you can then compare the the distance of the closest pair of points in the set of three to their distance to the nearest division. If it's closer than that, you then have to compare the closest point to each division to see if they are closer to each other than to any of the points in their set of three.

The neat thing is, if you use a merge sort approach, you can do this automagically as you break the set up and merge them together again. This will result in a complexity of O(nlogn) instead of O(n^2), which is something that can be done in a reasonable amount of time.

This is from an actual problem I solved for a friend of mine in biochem here at Queen's. He was doing the naive method above... in excel. He let the spreadsheet sit for a week on a set of three thousand or so points before he gave up and killed it. An entire week with his PC running balls to the wall the whole time... and for all I know, it may have hit only a few percent of the space in that time. When you do things like that, the amount of computation involved in getting an answer goes up really fast as the problem grows larger.

Once you get into the O(n^2) or even worse O(2^n) levels of complexity, you either figure out how to get an answer that's good enough if not the best, or you don't bother; as soon as you get into the exponential levels, you can easily be talking about heat death of the universe problems like the travelling salesman problem (which can't be solved for a sufficiently large value of n before the heat death of the universe). For reference, a sufficiently large value of n in the travelling salesman problem is something like seven or eight cities. If you get a manager type asking you to solve that problem, you need to show them why it's not solvable unless you're a genius mathematician who will make their entire career by solvingn it.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New I like your answer! (Esp. the 1st sentence...)
jb4
shrub●bish (Am., from shrub + rubbish, after the derisive name for America's 43 president; 2003) n. 1. a form of nonsensical political doubletalk wherein the speaker attempts to defend the indefensible by lying, obfuscation, or otherwise misstating the facts; GIBBERISH. 2. any of a collection of utterances from America's putative 43rd president. cf. BULLSHIT

New Well, it's not like it's not true...
Of course, as one of the profs here put it, 'being able to know when you have no choice but to do every possible combination is an important career skill in this field'.

His claim was that it could be very important to be able to explain to a TLA type why neither you nor anyone else was going to be able to do what he wanted done.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New Okay, about what I thought
Though I had no idea the travelling salesman got so hard so fast.





Wait, that didn't come out right.
===

Purveyor of Doc Hope's [link|http://DocHope.com|fresh-baked dog biscuits and pet treats].
[link|http://DocHope.com|http://DocHope.com]
Expand Edited by drewk Dec. 14, 2005, 09:45:56 PM EST
New Not only hard
but rilly rilly big too.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New You overstate the case significantly
With 3000 points at 1000 operations/second (and if it is only doing 1000 operations/second, then you have another problem...) it will finish in 3000^2/1000=9000 seconds, or just a few hours. Your friend was doing something else wrong if it hosed his computer for a week.

Also travelling salesman is not as bad as you make it out to be. Travelling salesman takes O(n*n!) with the naive algorithm. (For n locations you can just try each possible order of visiting locations, and calculating each answer takes order n work. If you want to be smarter about it, you can stop looking at a path as soon as it takes longer than the best you have. In the real world that tends to reduce the problem complexity by quite a bit, but it doesn't help in the worst case.) So travelling salesman of 7 takes 35,280 steps. 8 takes 322,560 steps. And so on.

At the measly 1000 operations/second, you can solve it for 10 in a half-day. At a more optimistic 1,000,000 operations per second, you can solve it for 13 in about a day. It is also a highly parallelizable problem, so you can easily split the work across multiple computers working in parallel, so you can realistically do a bit more than that.

Of course if you wanted to work out, say, 20 in a year, it would take about 15,000 computers that were each doing a hundred million calculations per second to do it...

Cheers,
Ben
I have come to believe that idealism without discipline is a quick road to disaster, while discipline without idealism is pointless. -- Aaron Ward (my brother)
New Yeah, that's true
but it can be fun to polemicize on something completely trivial like that.
--\n-------------------------------------------------------------------\n* Jack Troughton                            jake at consultron.ca *\n* [link|http://consultron.ca|http://consultron.ca]                   [link|irc://irc.ecomstation.ca|irc://irc.ecomstation.ca] *\n* Kingston Ontario Canada               [link|news://news.consultron.ca|news://news.consultron.ca] *\n-------------------------------------------------------------------
New Disciplined programming, eh?
HAHAHAHAHAHA


Peter
[link|http://www.no2id.net/|Don't Let The Terrorists Win]
[link|http://www.kuro5hin.org|There is no K5 Cabal]
[link|http://guildenstern.dyndns.org|Home]
Use P2P for legitimate purposes!
New Sounds like you may never have heard of it?
jb4
shrub●bish (Am., from shrub + rubbish, after the derisive name for America's 43 president; 2003) n. 1. a form of nonsensical political doubletalk wherein the speaker attempts to defend the indefensible by lying, obfuscation, or otherwise misstating the facts; GIBBERISH. 2. any of a collection of utterances from America's putative 43rd president. cf. BULLSHIT

New Nah, just that I've rarely *seen* it
===

Purveyor of Doc Hope's [link|http://DocHope.com|fresh-baked dog biscuits and pet treats].
[link|http://DocHope.com|http://DocHope.com]
     Need to QC an active linux box. - (broomberg) - (20)
         My guess would be handles - (jake123) - (18)
             How can an external process tell? -NT - (broomberg) - (17)
                 AFAIK, it can't. - (jake123) - (16)
                     Erm? - (jb4) - (15)
                         No, not necessarily. - (jake123) - (14)
                             And not necessarily again - (ben_tilly) - (13)
                                 A) Thanks. B) Harrumph! - (jb4) - (12)
                                     Well, of course you need disciplined programming - (jake123) - (8)
                                         Stupid "never took a CS course" question - (drewk) - (7)
                                             If you have to iterate over every possible combination - (jake123) - (6)
                                                 I like your answer! (Esp. the 1st sentence...) -NT - (jb4) - (1)
                                                     Well, it's not like it's not true... - (jake123)
                                                 Okay, about what I thought - (drewk) - (1)
                                                     Not only hard - (jake123)
                                                 You overstate the case significantly - (ben_tilly) - (1)
                                                     Yeah, that's true - (jake123)
                                     Disciplined programming, eh? - (pwhysall) - (2)
                                         Sounds like you may never have heard of it? -NT - (jb4) - (1)
                                             Nah, just that I've rarely *seen* it -NT - (drewk)
         do a netstat -a | grep -i waiting - (boxley)

A vampire walrus would look just like a regular walrus.
234 ms