Hello-
Our portal box crashed with the below error:
[Thr 5659] Mon Aug 20 14:23:09 2012
[Thr 5659] *** WARNING => IcmReadFromConn(id=32/208743): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 1543] Mon Aug 20 14:23:12 2012
[Thr 1543] *** ERROR => HttpJ2EETriggerServer: alloc failed: out of MPI blocks [http_j2ee2_m 1646]
[Thr 3342] Mon Aug 20 14:23:18 2012
[Thr 3342] *** WARNING => IcmReadFromConn(id=44/208745): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 4372] *** WARNING => IcmReadFromConn(id=20/208746): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 5145] Mon Aug 20 14:23:19 2012
[Thr 5145] *** WARNING => IcmReadFromConn(id=11/208747): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 3085] Mon Aug 20 14:23:20 2012
[Thr 3085] *** WARNING => IcmReadFromConn(id=31/208748): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 5402] Mon Aug 20 14:23:22 2012
[Thr 5402] *** WARNING => IcmReadFromConn(id=25/208749): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 1800] Mon Aug 20 14:23:23 2012
[Thr 1800] *** ERROR => HttpJ2EETriggerServer: alloc failed: out of MPI blocks [http_j2ee2_m 1646]
[Thr 2828] Mon Aug 20 14:23:28 2012
[Thr 2828] *** WARNING => IcmReadFromConn(id=26/208751): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 4886] Mon Aug 20 14:23:32 2012
[Thr 4886] *** WARNING => IcmReadFromConn(id=16/208752): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 2571] Mon Aug 20 14:23:34 2012
[Thr 2571] *** ERROR => HttpJ2EETriggerServer: alloc failed: out of MPI blocks [http_j2ee2_m 1646]
[Thr 2828] Mon Aug 20 14:23:35 2012
[Thr 2828] *** WARNING => IcmReadFromConn(id=26/208751): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 3342] *** WARNING => IcmReadFromConn(id=25/208749): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 1286] *** WARNING => IcmReadFromConn(id=31/208748): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 3085] *** WARNING => IcmReadFromConn(id=11/208747): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 4372] *** WARNING => IcmReadFromConn(id=20/208746): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 5145] *** WARNING => IcmReadFromConn(id=44/208745): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658]
[Thr 5402] *** WARNING => IcmReadFromConn(id=32/208743): temporarily out of MPI buffers -> roll out [icxxthrio_mt 2658
I found OSS notes 737625 and 715400 that are relavant , but I know not quite confident as to what all parameters and by how much the size of those parameters should be increased? I understand that the mpi/total_size_MB need to be increased.
Also for AIX sidadm ulimit -n we currently have "no files(descriptors) 2000" , how much should this be changed?
Please suggest all the parameters/sizes that need to be edited. We have around 6000 users that use the system heavily.
Our default values:
rdisp/elem_per_queue = 4000
mpi/total_size_MB = 500
mpi/buffer_size = 65536
mpi/max_pipes = 16000
Thank you!