kernel.pid_maxhas been reached. (The default is 32K, at least on my CentOS 5 box.)
With threaded Apache httpd on Linux/Unix, the
pthread_create()failure can be logged as
Cannot allocate memory: apr_thread_create: unable to create worker thread
The most common cause of this failure is still the thread stack size, but if you're creating thousands of threads and shrinking the stack size doesn't help, see if increasing
kernel.pid_maxallows more threads to be created.
Hats off to a colleague at &bigco; for finding this solution.