Did a google search and [link|http://folk.uio.no/hakonon/oracle/iouga/doc/html/148.html|this] has promising looking information about what to adjust before recompiling a kernel with more shared memory handles.
have you increased the number in /proc/sys/kernel/shmmni ? It's supposed to control the maximum number of segments allowed. It is shared memory mappings running out, not semaphores or msgs right?