Replies by Roger Andre for InterSystems Developer Community

Roger Andre · Jul 15, 2022

Any MSM/MUMPS experts?

Roger Andre · Jul 8, 2022

Hi Ron, this MSM/WINDOWS system has how many users working simultaneously?
Are Activate and Workstation services active?
Today we have about 1100 concurrent users and we started to notice drops in Activate/Workstation services when the number of users is at this level, but we couldn't identify the reason for the drops, because activate/workstation does not generate logs for verification.

go to post

Roger Andre · Jul 7, 2022

I need to understand if the configuration of these parameters is reflecting in the constant drops we have in the Activate/Workstation services and also in the freezing of MSM/UNIX.

go to post

Roger Andre · Jun 21, 2022

MSM/MUMPS is running on a Unix IBM/AIX 7.

Following are logs from the console.log from IBM/AIX and logs from the command below

alog -o -t console

errpt -a > errptSISMED.txt

0 Tue Jun 21 03:38:01 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 03:38:01 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:01 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 03:38:01 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:01 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 03:38:01 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:02 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 03:38:02 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:03 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 03:38:03 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:08 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 03:38:08 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:17 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 03:38:17 2022
Use local problem reporting procedures.

qdaemon: errno = 4: A system call received an interrupt.
0 Tue Jun 21 03:38:43 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtf2:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prtf2 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/ttRnaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Feb 11 09:21:58 2022. Current Time: Jun 21 03:38:43 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:43 -03 2022 qdaemon: (WARNING): 0781-088 Queue prthc:@172_16_108_15 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_108_15 -P prthc -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tfv4Maa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 3 07:25:52 2020. Current Time: Jun 21 03:38:43 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:47 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt03:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt03 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/trfAMea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 3 11:06:38 2022. Current Time: Jun 21 03:38:47 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 03:38:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:53 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 03:38:52 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:54 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtn3:@172_16_107_22 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_107_22 -P prtn3 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tLkiqaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 19 05:16:40 2021. Current Time: Jun 21 03:38:54 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:54 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt31:@129_156_10_165 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_10_165 -P prt31 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t5-Waaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 9 08:11:21 2021. Current Time: Jun 21 03:38:54 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:55 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 03:38:55 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:55 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 03:38:55 2022
Use local problem reporting procedures.

0 Tue Jun 21 03:38:57 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 03:38:57 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:02 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 06:30:02 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:45 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt31:@129_156_10_165 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_10_165 -P prt31 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t5-Waaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 9 08:11:21 2021. Current Time: Jun 21 06:30:45 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:47 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 06:30:47 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:48 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtn3:@172_16_107_22 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_107_22 -P prtn3 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tLkiqaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 19 05:16:40 2021. Current Time: Jun 21 06:30:48 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:49 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 06:30:49 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:49 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 06:30:49 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 06:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:30:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 06:30:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtf2:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prtf2 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/ttRnaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Feb 11 09:21:58 2022. Current Time: Jun 21 06:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prthc:@172_16_108_15 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_108_15 -P prthc -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tfv4Maa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 3 07:25:52 2020. Current Time: Jun 21 06:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 06:31:36 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt03:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt03 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/trfAMea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 3 11:06:38 2022. Current Time: Jun 21 06:31:36 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:24:44 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt98:@172_16_136_35 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_35 -P prt98 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tFb4Mea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 21 07:23:14 2022. Current Time: Jun 21 07:24:44 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:04 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 07:30:04 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:48 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 07:30:48 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:48 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt31:@129_156_10_165 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_10_165 -P prt31 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t5-Waaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 9 08:11:21 2021. Current Time: Jun 21 07:30:48 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtn3:@172_16_107_22 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_107_22 -P prtn3 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tLkiqaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 19 05:16:40 2021. Current Time: Jun 21 07:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 07:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 07:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 07:30:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:30:53 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 07:30:53 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtf2:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prtf2 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/ttRnaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Feb 11 09:21:58 2022. Current Time: Jun 21 07:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prthc:@172_16_108_15 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_108_15 -P prthc -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tfv4Maa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 3 07:25:52 2020. Current Time: Jun 21 07:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 07:31:38 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt03:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt03 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/trfAMea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 3 11:06:38 2022. Current Time: Jun 21 07:31:38 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:04 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 08:30:04 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:47 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt31:@129_156_10_165 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_10_165 -P prt31 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t5-Waaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 9 08:11:21 2021. Current Time: Jun 21 08:30:47 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:48 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 08:30:48 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:49 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 08:30:49 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 08:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtn3:@172_16_107_22 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_107_22 -P prtn3 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tLkiqaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 19 05:16:40 2021. Current Time: Jun 21 08:30:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 08:30:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:30:55 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 08:30:55 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtf2:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prtf2 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/ttRnaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Feb 11 09:21:58 2022. Current Time: Jun 21 08:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prthc:@172_16_108_15 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_108_15 -P prthc -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tfv4Maa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 3 07:25:52 2020. Current Time: Jun 21 08:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 08:31:38 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt03:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt03 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/trfAMea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 3 11:06:38 2022. Current Time: Jun 21 08:31:38 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:04 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 09:30:04 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:47 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt31:@129_156_10_165 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_10_165 -P prt31 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t5-Waaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 9 08:11:21 2021. Current Time: Jun 21 09:30:47 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:49 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 09:30:49 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 09:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtn3:@172_16_107_22 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_107_22 -P prtn3 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tLkiqaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 19 05:16:40 2021. Current Time: Jun 21 09:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 09:30:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 09:30:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:30:53 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 09:30:53 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:31:32 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtf2:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prtf2 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/ttRnaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Feb 11 09:21:58 2022. Current Time: Jun 21 09:31:32 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:31:32 -03 2022 qdaemon: (WARNING): 0781-088 Queue prthc:@172_16_108_15 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_108_15 -P prthc -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tfv4Maa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 3 07:25:52 2020. Current Time: Jun 21 09:31:32 2022
Use local problem reporting procedures.

0 Tue Jun 21 09:31:39 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt03:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt03 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/trfAMea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 3 11:06:38 2022. Current Time: Jun 21 09:31:39 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:03 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 10:30:03 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:48 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 10:30:48 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:49 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt31:@129_156_10_165 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_10_165 -P prt31 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t5-Waaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 9 08:11:21 2021. Current Time: Jun 21 10:30:49 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:49 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtn3:@172_16_107_22 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_107_22 -P prtn3 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tLkiqaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 19 05:16:40 2021. Current Time: Jun 21 10:30:49 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 10:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 10:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 10:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:30:52 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 10:30:52 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtf2:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prtf2 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/ttRnaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Feb 11 09:21:58 2022. Current Time: Jun 21 10:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prthc:@172_16_108_15 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_108_15 -P prthc -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tfv4Maa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 3 07:25:52 2020. Current Time: Jun 21 10:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 10:31:35 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt03:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt03 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/trfAMea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 3 11:06:38 2022. Current Time: Jun 21 10:31:35 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:04 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt75:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt75 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/tZouaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Oct 16 10:07:29 2021. Current Time: Jun 21 12:30:04 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:46 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt31:@129_156_10_165 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_10_165 -P prt31 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t5-Waaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 9 08:11:21 2021. Current Time: Jun 21 12:30:46 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:48 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtn3:@172_16_107_22 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_107_22 -P prtn3 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tLkiqaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 19 05:16:40 2021. Current Time: Jun 21 12:30:48 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:49 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtff:@129_156_11_135 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 129_156_11_135 -P prtff -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/t2uXqea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 21 09:47:14 2020. Current Time: Jun 21 12:30:49 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt29:@172_16_127_30 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_127_30 -P prt29 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tWST7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jul 1 07:34:49 2019. Current Time: Jun 21 12:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:50 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt43:@172_16_136_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_136_90 -P prt43 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/teaX7ea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Apr 26 07:18:10 2021. Current Time: Jun 21 12:30:50 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:51 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt64:@172_16_150_90 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_150_90 -P prt64 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tVwg7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jan 18 06:27:49 2022. Current Time: Jun 21 12:30:51 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:30:53 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtg1:@172_20_104_10 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_20_104_10 -P prtg1 -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tBVS7aa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Dec 15 12:55:22 2021. Current Time: Jun 21 12:30:53 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prtf2:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prtf2 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/ttRnaaa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Feb 11 09:21:58 2022. Current Time: Jun 21 12:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:31:31 -03 2022 qdaemon: (WARNING): 0781-088 Queue prthc:@172_16_108_15 went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S 172_16_108_15 -P prthc -N /usr/lib/lpd/aixshort \
/var/spool/qdaemon/tfv4Maa
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Nov 3 07:25:52 2020. Current Time: Jun 21 12:31:31 2022
Use local problem reporting procedures.

0 Tue Jun 21 12:31:35 -03 2022 qdaemon: (WARNING): 0781-088 Queue prt03:@iprint went down, job is still queued:
Backend: /usr/lib/lpd/rembak -S iprint.ghc.com.br -P prt03 -N /usr/lib/lpd/bsdshort \
/var/spool/qdaemon/trfAMea
Backend Exit Value: EXITFATAL (0100)
Job Submit Time: Jun 3 11:06:38 2022. Current Time: Jun 21 12:31:35 2022
Use local problem reporting procedures.

---------------------------------------------------------------------------
LABEL: CL_VAR_FULL
IDENTIFIER: E5899EEB

Date/Time:       Tue Jun 21 10:29:46 -03 2022
Sequence Number: 21318
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   CAA (for RSCT)

Description
/var filesystem is running low on space

Probable Causes
Unknown

Failure Causes
Unknown

   Recommended Actions
   RSCT could malfunction if /var gets full
   Increase the filesystem size or delete unwanted files

Detail Data
Percent full
          75
Percent threshold
          75
---------------------------------------------------------------------------
LABEL:          TTY_TTYHOG
IDENTIFIER:     0873CF9F

Date/Time:       Tue Jun 21 10:29:46 -03 2022
Sequence Number: 21317
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            TEMP
WPAR:            Global
Resource Name:   pts/58

Description
TTYHOG OVER-RUN

Failure Causes
EXCESSIVE LOAD ON PROCESSOR

   Recommended Actions
   REDUCE SYSTEM LOAD.
   REDUCE SERIAL PORT BAUD RATE

Duplicates
Number of duplicates
           1
Time of first duplicate
Tue Jun 21 10:29:36 -03 2022
Time of last duplicate
Tue Jun 21 10:29:46 -03 2022
---------------------------------------------------------------------------
LABEL:          TTY_TTYHOG
IDENTIFIER:     0873CF9F

Date/Time:       Tue Jun 21 10:29:36 -03 2022
Sequence Number: 21316
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            TEMP
WPAR:            Global
Resource Name:   pts/58

Description
TTYHOG OVER-RUN

Failure Causes
EXCESSIVE LOAD ON PROCESSOR

   Recommended Actions
   REDUCE SYSTEM LOAD.
   REDUCE SERIAL PORT BAUD RATE

---------------------------------------------------------------------------
LABEL: CL_VAR_FULL
IDENTIFIER: E5899EEB

Date/Time:       Tue Jun 21 10:15:05 -03 2022
Sequence Number: 21315
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   CAA (for RSCT)

Description
/var filesystem is running low on space

Probable Causes
Unknown

Failure Causes
Unknown

   Recommended Actions
   RSCT could malfunction if /var gets full
   Increase the filesystem size or delete unwanted files

Detail Data
Percent full
          75
Percent threshold
          75
---------------------------------------------------------------------------
LABEL:          CL_VAR_FULL
IDENTIFIER:     E5899EEB

Date/Time:       Tue Jun 21 09:59:46 -03 2022
Sequence Number: 21314
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   CAA (for RSCT)

Description
/var filesystem is running low on space

Probable Causes
Unknown

Failure Causes
Unknown

   Recommended Actions
   RSCT could malfunction if /var gets full
   Increase the filesystem size or delete unwanted files

Detail Data
Percent full
          75
Percent threshold
          75
---------------------------------------------------------------------------
LABEL:          STORAGERM_STARTED_S
IDENTIFIER:     EDFF8E9B

Date/Time:       Mon Jun 20 16:53:21 -03 2022
Sequence Number: 21313
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   StorageRM

Description
IBM.StorageRM daemon has started.

Probable Causes
The RSCT Storage Resource Manager daemon (IBM.StorageRMd) has been started.

User Causes
The RSCT Storage Resource Manager daemon (IBM.StorageRMd) has been started.

Recommended Actions
None

Detail Data
DETECTING MODULE
RSCT,IBM.StorageRMd.C,1.52,326
ERROR ID

REFERENCE CODE

---------------------------------------------------------------------------
LABEL:          CONFIGRM_ONLINE_ST
IDENTIFIER:     3B16518D

Date/Time:       Mon Jun 20 16:53:21 -03 2022
Sequence Number: 21312
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            INFO
WPAR:            Global
Resource Name:   ConfigRM

Description
The node is online in the domain indicated in the detail data.

Probable Causes
A user ran the 'startrpdomain' or 'startrpnode' commands.
The node rebooted while the node was online.
The configuration manager recycled the node through an offline/online
transition to resynchronize the domain configuration or to recover
from some other failure.

User Causes
A user ran the 'startrpdomain' or 'startrpnode' commands.
The node rebooted while the node was online.
The configuration manager recycled the node through an offline/online
transition to resynchronize the domain configuration or to recover
from some other failure.

Recommended Actions
None

Detail Data
DETECTING MODULE
RSCT,PeerDomain.C,1.99.34.6,26599
ERROR ID

REFERENCE CODE

Peer Domain Name
CL_SISMED
---------------------------------------------------------------------------
LABEL:          CONFIGRM_HASQUORUM_
IDENTIFIER:     4BDDFBCC

Date/Time:       Mon Jun 20 16:53:21 -03 2022
Sequence Number: 21311
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            INFO
WPAR:            Global
Resource Name:   ConfigRM

Description
The operational quorum state of the active peer domain has changed to HAS_QUORUM.
In this state, cluster resources may be recovered and controlled as needed by
management applications.

Probable Causes
One or more nodes have come online in the peer domain.

User Causes
One or more nodes have come online in the peer domain.

Recommended Actions
None

Detail Data
DETECTING MODULE
RSCT,PeerDomain.C,1.99.34.6,20532
ERROR ID

REFERENCE CODE

---------------------------------------------------------------------------
LABEL:          RMCD_INFO_3_ST
IDENTIFIER:     C369AE20

Date/Time:       Mon Jun 20 16:53:21 -03 2022
Sequence Number: 21310
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   RMCdaemon

Description
The daemon is started.

Probable Causes
The Resource Monitoring and Control daemon has been started.

User Causes
The startsrc -s ctrmc command has been executed or
the rmcctrl -s command has been executed.

Recommended Actions
Confirm that the daemon should be started.

Detail Data
DETECTING MODULE
RSCT,rmcd_rmcproc.c,1.165.1.3,1018
ERROR ID
6UsOO11l.BgW/9oX/33.2g0...................
REFERENCE CODE

rmcd_flags
C4C9 101C
---------------------------------------------------------------------------
LABEL:          GS_START_ST
IDENTIFIER:     AFA89905

Date/Time:       Mon Jun 20 16:53:14 -03 2022
Sequence Number: 21309
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   cthags

Description
Group Services daemon started

Probable Causes
Daemon started during system startup
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user

User Causes
Daemon started manually by user

Recommended Actions
Check that Group Services daemon is running

Detail Data
DETECTING MODULE
RSCT,pgsd.C,1.62.1.41,750
ERROR ID
63Y7ej0e.BgW/V8n/33.2g0...................
REFERENCE CODE

DIAGNOSTIC EXPLANATION
HAGS daemon started by SRC. Log file is /var/ct/3ZDzY69geHvP12DXtDlzeM/log/cthags/trace.
---------------------------------------------------------------------------
LABEL:          CONFIGRM_STARTED_ST
IDENTIFIER:     DE84C4DB

Date/Time:       Mon Jun 20 16:52:44 -03 2022
Sequence Number: 21308
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   ConfigRM

Description
IBM.ConfigRM daemon has started.

Probable Causes
The RSCT Configuration Manager daemon (IBM.ConfigRMd) has been started.

User Causes
The RSCT Configuration Manager daemon (IBM.ConfigRMd) has been started.

Recommended Actions
None

Detail Data
DETECTING MODULE
RSCT,IBM.ConfigRMd.C,1.76.1.1,422
ERROR ID

REFERENCE CODE

---------------------------------------------------------------------------
LABEL:          RMCD_INFO_0_ST
IDENTIFIER:     A6DF45AA

Date/Time:       Mon Jun 20 16:52:42 -03 2022
Sequence Number: 21307
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   RMCdaemon

Description
The daemon is started.

Probable Causes
The Resource Monitoring and Control daemon has been started.

User Causes
The startsrc -s ctrmc command has been executed or
the rmcctrl -s command has been executed.

Recommended Actions
Confirm that the daemon should be started.

Detail Data
DETECTING MODULE
RSCT,rmcd.c,1.134.2.1,255
ERROR ID
6eKora08.BgW/IA/133.2g0...................
REFERENCE CODE

---------------------------------------------------------------------------
LABEL:          REBOOT_ID
IDENTIFIER:     2BFA76F6

Date/Time:       Mon Jun 20 16:52:07 -03 2022
Sequence Number: 21304
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            TEMP
WPAR:            Global
Resource Name:   SYSPROC

Description
SYSTEM SHUTDOWN BY USER

Probable Causes
SYSTEM SHUTDOWN

Detail Data
USER ID
           0
0=SOFT IPL 1=HALT 2=TIME REBOOT
           0
TIME TO REBOOT (FOR TIMED REBOOT ONLY)
           0
---------------------------------------------------------------------------
LABEL:          ERRLOG_ON
IDENTIFIER:     9DBCFDEE

Date/Time:       Mon Jun 20 16:52:25 -03 2022
Sequence Number: 21303
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   errdemon

Description
ERROR LOGGING TURNED ON

Probable Causes
ERRDEMON STARTED AUTOMATICALLY

User Causes
/USR/LIB/ERRDEMON COMMAND

Recommended Actions
NONE

---------------------------------------------------------------------------
LABEL: ERRLOG_OFF
IDENTIFIER: 192AC071

Date/Time:       Mon Jun 20 16:50:41 -03 2022
Sequence Number: 21302
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   errdemon

Description
ERROR LOGGING TURNED OFF

Probable Causes
ERRSTOP COMMAND

User Causes
ERRSTOP COMMAND

   Recommended Actions
   RUN ERRDEAD COMMAND
   TURN ERROR LOGGING ON

---------------------------------------------------------------------------
LABEL: LDMP_COMPLETE
IDENTIFIER: AEA055D0

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21301
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            INFO
WPAR:            Global
Resource Name:   livedump

Description
Live dump complete

Detail Data
File name and message text
/var/adm/ras/livedump/lvm.medvgbkp.202206201950.00.DZ

---------------------------------------------------------------------------
LABEL: LDMP_COMPLETE
IDENTIFIER: AEA055D0

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21300
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            INFO
WPAR:            Global
Resource Name:   livedump

Description
Live dump complete

Detail Data
File name and message text
/var/adm/ras/livedump/lvm.medvg.202206201950.00.DZ

---------------------------------------------------------------------------
LABEL: LVM_GS_CHILDGONE
IDENTIFIER: 4B219AEA

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21299
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Concurrent LVM daemon forced Volume Group offline

Probable Causes
Unrecoverable event detected by Concurrent LVM daemon

Failure Causes
Lost communication with remote nodes
Lost quorum

   Recommended Actions
   Ensure Cluster daemons are running
   CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
   Attempt to bring the Concurrent Volume Group back online
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
Volume Group ID
00F9 8DEE 0000 4C00 0000 014B 506D D812
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL: LVM_SA_QUORCLOSE
IDENTIFIER: CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21298
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0028 0000 0000
QUORUM COUNT
           2
ACTIVE COUNT
       65535
SENSE DATA
0000 0000 0000 0A9E 00F9 8DEE 0000 4C00 0000 014B 506D D812 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_QUORCLOSE
IDENTIFIER:     CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21297
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21296
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Concurrent LVM daemon forced Volume Group offline

Probable Causes
Unrecoverable event detected by Concurrent LVM daemon

Failure Causes
Lost communication with remote nodes
Lost quorum

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21295
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Unable to start gsclvmd

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Unable to establish communication with Cluster daemons

   Recommended Actions
   Ensure Cluster daemons are running
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL: LVM_SA_QUORCLOSE
IDENTIFIER: CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21294
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21293
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Concurrent LVM daemon forced Volume Group offline

Probable Causes
Unrecoverable event detected by Concurrent LVM daemon

Failure Causes
Lost communication with remote nodes
Lost quorum

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21292
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Unable to start gsclvmd

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Unable to establish communication with Cluster daemons

   Recommended Actions
   Ensure Cluster daemons are running
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21291
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Concurrent LVM daemon forced Volume Group offline

Probable Causes
Unrecoverable event detected by Concurrent LVM daemon

Failure Causes
Lost communication with remote nodes
Lost quorum

Detail Data
Volume Group ID
00F9 8DEE 0000 4C00 0000 014A F455 2593
MAJOR/MINOR DEVICE NUMBER
0025 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL: LVM_SA_QUORCLOSE
IDENTIFIER: CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21290
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0025 0000 0000
QUORUM COUNT
           6
ACTIVE COUNT
       65535
SENSE DATA
0000 0000 0000 0A9E 00F9 8DEE 0000 4C00 0000 014A F455 2593 00FB 42D7 15E1 EEBA
---------------------------------------------------------------------------
LABEL:          LVM_SA_QUORCLOSE
IDENTIFIER:     CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21289
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21288
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Concurrent LVM daemon forced Volume Group offline

Probable Causes
Unrecoverable event detected by Concurrent LVM daemon

Failure Causes
Lost communication with remote nodes
Lost quorum

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21287
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Unable to start gsclvmd

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Unable to establish communication with Cluster daemons

   Recommended Actions
   Ensure Cluster daemons are running
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21286
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21285
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Concurrent LVM daemon forced Volume Group offline

Probable Causes
Unrecoverable event detected by Concurrent LVM daemon

Failure Causes
Lost communication with remote nodes
Lost quorum

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21284
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Unable to start gsclvmd

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Unable to establish communication with Cluster daemons

   Recommended Actions
   Ensure Cluster daemons are running
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21283
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            INFO
WPAR:            Global
Resource Name:   livedump

Description
Live dump complete

Detail Data
File name and message text
/var/adm/ras/livedump/lvm.ghchome.202206201950.00.DZ

---------------------------------------------------------------------------
LABEL: LVM_SA_QUORCLOSE
IDENTIFIER: CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21282
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21281
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21280
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Group Services detected a failure

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Concurrent Volume Group forced offline

   Recommended Actions
   CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21279
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Group Services detected a failure

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Concurrent Volume Group forced offline

   Recommended Actions
   CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21278
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   StorageRM

Description
IBM.StorageRM daemon has been stopped.

Probable Causes
The RSCT Storage Resource Manager daemon(IBM.StorageRMd) has been stopped.

User Causes
The stopsrc -s IBM.StorageRM command has been executed.

Recommended Actions
Confirm that the daemon should be stopped. Normally, this daemon should
not be stopped explicitly by the user.

Detail Data
DETECTING MODULE
RSCT,StorageRMDaemon.C,1.66,333
ERROR ID

REFERENCE CODE

---------------------------------------------------------------------------
LABEL:          LVM_GS_CONNECTIVITY
IDENTIFIER:     DB14100E

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21277
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Group Services detected a failure

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Concurrent Volume Group forced offline

   Recommended Actions
   CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
Volume Group ID
00F9 8DEE 0000 4C00 0000 014A F430 EF07
MAJOR/MINOR DEVICE NUMBER
0024 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL: LVM_SA_QUORCLOSE
IDENTIFIER: CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21276
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0024 0000 0000
QUORUM COUNT
           3
ACTIVE COUNT
       65535
SENSE DATA
0000 0000 0000 0A9E 00F9 8DEE 0000 4C00 0000 014A F430 EF07 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_QUORCLOSE
IDENTIFIER:     CAD234BE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21275
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           H
Type:            UNKN
WPAR:            Global
Resource Name:   LVDD
Resource Class: NONE
Resource Type:   NONE
Location:

Description
QUORUM LOST, VOLUME GROUP CLOSING

Probable Causes
PHYSICAL VOLUME UNAVAILABLE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21274
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class: NONE
Resource Type:   NONE
Location:

Description
Group Services detected a failure

Probable Causes
Unable to establish communication with Cluster daemons

Failure Causes
Concurrent Volume Group forced offline

   Recommended Actions
   CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
   IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Date/Time:       Mon Jun 20 16:50:40 -03 2022
Sequence Number: 21273
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   ConfigRM

Description
IBM.ConfigRM daemon has been stopped.

Probable Causes
The RSCT Configuration Manager daemon(IBM.ConfigRMd) has been stopped.

User Causes
The stopsrc -s IBM.ConfigRM command has been executed.

   Recommended Actions
   Confirm that the daemon should be stopped. Normally, this daemon should
   not be stopped explicitly by the user.

Detail Data
DETECTING MODULE
RSCT,ConfigRMDaemon.C,1.30,221
ERROR ID

REFERENCE CODE

---------------------------------------------------------------------------
LABEL:          OPMSG
IDENTIFIER:     AA8AB241

Date/Time:       Mon Jun 20 16:50:31 -03 2022
Sequence Number: 21272
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   OPERATOR

Description
OPERATOR NOTIFICATION

User Causes
ERRLOGGER COMMAND

Recommended Actions
REVIEW DETAILED DATA

Detail Data
MESSAGE FROM ERRLOGGER COMMAND
clexit.rc : clstrmgrES terminated during AIX shutdown.
---------------------------------------------------------------------------
LABEL: CORE_DUMP
IDENTIFIER: A924A5FC

Date/Time:       Mon Jun 20 16:26:55 -03 2022
Sequence Number: 21271
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSPROC

Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED

Probable Causes
SOFTWARE PROGRAM

User Causes
USER GENERATED SIGNAL

Recommended Actions
CORRECT THEN RETRY

Failure Causes
SOFTWARE PROGRAM

   Recommended Actions
   RERUN THE APPLICATION PROGRAM
   IF PROBLEM PERSISTS THEN DO THE FOLLOWING
   CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
SIGNAL NUMBER
           4
USER'S PROCESS ID:
              64028842
FILE SYSTEM SERIAL NUMBER
          18
INODE NUMBER
                     2
CORE FILE NAME
/msmmed/core
PROGRAM NAME
mumsm
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
??
PROCESSOR ID
hw_fru_id: 0
hw_cpu_id: 1

ADDITIONAL INFORMATION
??
??
Unable to generate symptom string.
---------------------------------------------------------------------------
LABEL: DMPCHK_TOOSMALL
IDENTIFIER: E87EF1BE

Date/Time:       Mon Jun 20 15:00:00 -03 2022
Sequence Number: 21270
Machine Id:      00FB42D74C00
Node Id:         SISMED1
Class:           O
Type:            PEND
WPAR:            Global
Resource Name:   dumpcheck

Description
The largest dump device is too small.

Probable Causes
Neither dump device is large enough to accommodate a system dump at this time.

Recommended Actions
Increase the size of one or both dump devices.

Detail Data
Largest dump device
lg_dumplv
Largest dump device size in kb
     1048576
Current estimated dump size in kb
     1092382

go to post

Roger Andre · Jun 2, 2022

Hello, I'm having problems on my server where it has freezes in the service responsible for the MUMPS server on our IBM/AIX. I have noticed that the MUMPS service freezes when the server load average increases. At certain times of the day the load average increases to values above 30. At the moment it is at 14.81.
SISMED1:/var/tmp> uptime
03:09PM up 34 days, 22:57, 50 users, load average: 14.81, 12.31, 13.20

When it increases a lot, the service responsible for MUMPS/MSM freezes and we have to kill the service to restart the MUMPS system. Is there anything I can check or is there any system tuning to improve processor performance?

Here are some data

SISMED1:/var/tmp> oslevel -r
7100-05
SISMED1:/var/tmp> uname -a
AIX SISMED1 1 7 00FB42D74C00

SISMED1:/var/tmp> prtconf | grep -i "Processor Type"
Processor Type: PowerPC_POWER8
SISMED1:/var/tmp> lsattr -El sys0 | grep -i model
modelname IBM,8286-41A Machine name False
SISMED1:/var/tmp> lsattr -El proc0|grep -E "freq|type"
frequency 3724000000 Processor Speed False
type PowerPC_POWER8 Processor type False

SISMED1:/var/tmp> uptime
03:11PM up 34 days, 23 hrs, 49 users, load average: 21.55, 15.95, 14.47
SISMED1:/var/tmp> ps -ef | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 20
root 2294088 21692518 58 10:04:01 - 86:09 mumsm 1
root 60162070 21692518 4 10:03:59 - 0:09 muserver 132
root 25690180 21692518 3 10:04:01 - 0:23 muserver 1008
root 15925388 21692518 3 10:04:01 - 0:14 muserver 773
root 15007768 21692518 3 10:04:01 - 0:13 muserver 824
root 26345658 21692518 3 10:04:01 - 0:12 muserver 775
root 18678218 21692518 3 10:04:01 - 0:08 muserver 782
root 32178256 21692518 3 10:04:00 - 0:31 muserver 388
root 16908700 21692518 3 10:04:00 - 0:17 muserver 395
root 28049756 21692518 3 10:04:00 - 0:03 muserver 356
root 34013198 21692518 3 10:03:59 - 8:09 muserver 134
root 6160786 21692518 3 10:03:59 - 0:08 muserver 142
root 25886922 21692518 2 10:04:01 - 0:01 muserver 835
root 39649530 21692518 2 10:04:00 - 0:10 muserver 583
root 1245528 21692518 2 10:04:00 - 0:08 muserver 660
root 34078832 21692518 2 10:04:00 - 0:06 muserver 330
root 26608036 21692518 2 10:04:00 - 0:06 muserver 304
root 28901636 21692518 2 10:03:59 - 0:06 muserver 60
root 14876948 21692518 1 10:04:01 - 0:13 muserver 893
root 29950292 21692518 1 10:04:01 - 0:09 muserver 846

SISMED1:/var/tmp> lparstat 5 2

System configuration: type=Shared mode=Uncapped smt=Off lcpu=2 mem=16384MB psize=8 ent=2.00

%user %sys %wait %idle physc %entc lbusy vcsw phint
----- ----- ------ ------ ----- ----- ------ ----- -----
27.8 26.3 0.4 45.4 1.15 57.4 58.5 4881 0
28.0 29.9 0.0 42.1 1.22 61.0 61.7 4620 0
SISMED1:/var/tmp> uptime
03:14PM up 34 days, 23:02, 46 users, load average: 42.96, 23.78, 17.47

SISMED1:/var/tmp> uptime
03:18PM up 34 days, 23:07, 47 users, load average: 85.81, 61.00, 35.38
SISMED1:/var/tmp> ps -ef | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 20
root 2294088 21692518 60 10:04:01 - 88:35 mumsm 1
root 25690180 21692518 1 10:04:01 - 0:25 muserver 1008
root 721204 21692518 1 10:04:01 - 0:18 muserver 800
root 22348084 21692518 1 10:04:01 - 0:16 muserver 874
root 16384190 21692518 1 10:04:01 - 0:09 muserver 977
root 27656196 21692518 1 10:04:01 - 0:09 muserver 887
root 13041828 21692518 1 10:04:01 - 0:07 muserver 732
root 33489084 21692518 1 10:04:01 - 0:07 muserver 700

SISMED1:/var/tmp> uptime
03:24PM up 34 days, 23:12, 50 users, load average: 94.80, 81.23, 51.82
SISMED1:/var/tmp> vmstat -Iwt 2

System configuration: lcpu=2 mem=16384MB ent=2.00

kthr memory page faults cpu time
----------- --------------------- ------------------------------------ ------------------ ----------------------- --------
r b p avm fre fi fo pi po fr sr in sy cs us sy id wa pc ec hr mi se
38 1 0 1265158 6972 240 241 0 0 0 0 9747 393136 43404 24 32 44 0 1.19 59.6 15:24:23
122 0 0 1265758 5982 140 121 0 0 0 0 8057 325536 36254 30 26 43 0 1.20 59.8 15:24:25
55 0 0 1265758 5431 184 209 0 0 0 0 9258 364265 42399 25 29 45 0 1.16 58.0 15:24:27
74 0 0 1265827 5028 105 156 0 0 0 0 9636 382992 43773 25 30 46 0 1.16 58.1 15:24:29
83 0 0 1265862 5647 73 97 0 0 467 476 8847 437987 39165 34 36 30 0 1.44 72.0 15:24:31
80 0 0 1265611 5503 110 103 0 0 0 0 9467 397141 43378 27 33 40 0 1.25 62.7 15:24:33
141 0 0 1265611 5263 65 86 0 0 0 0 8545 347205 38507 28 27 45 0 1.15 57.7 15:24:35
172 0 0 1265612 4966 87 71 0 0 0 0 9080 369104 41146 27 29 44 0 1.19 59.3 15:24:37
41 0 0 1265618 5138 72 50 0 0 0 0 7969 393872 36325 26 28 45 0 1.16 57.8 15:24:39
155 0 0 1265635 7455 114 18 0 0 1293 1362 9790 423290 44117 28 34 38 0 1.30 65.1 15:24:41

SISMED1:/var/tmp> lparstat -i
Node Name : SISMED1
Partition Name : SISMED1
Partition Number : 9
Type : Shared
Mode : Uncapped
Entitled Capacity : 2.00
Partition Group-ID : 32777
Shared Pool ID : 0
Online Virtual CPUs : 2
Maximum Virtual CPUs : 2
Minimum Virtual CPUs : 1
Online Memory : 16384 MB
Maximum Memory : 16384 MB
Minimum Memory : 2048 MB
Variable Capacity Weight : 192
Minimum Capacity : 0.50
Maximum Capacity : 2.00
Capacity Increment : 0.01
Maximum Physical CPUs in system : 8
Active Physical CPUs in system : 8
Active CPUs in Pool : 8
Shared Physical CPUs in system : 8
Maximum Capacity of Pool : 800
Entitled Capacity of Pool : 540
Unallocated Capacity : 0.00
Physical CPU Percentage : 100.00%
Unallocated Weight : 0
Memory Mode : Dedicated
Total I/O Memory Entitlement : -
Variable Memory Capacity Weight : -
Memory Pool ID : -
Physical Memory in the Pool : -
Hypervisor Page Size : -
Unallocated Variable Memory Capacity Weight: -
Unallocated I/O Memory entitlement : -
Memory Group ID of LPAR : -
Desired Virtual CPUs : 2
Desired Memory : 16384 MB
Desired Variable Capacity Weight : 192
Desired Capacity : 2.00
Target Memory Expansion Factor : -
Target Memory Expansion Size : -
Power Saving Mode : Disabled
Sub Processor Mode : -

Configuration information
=========================
System: AIX 7.1 Node: SISMED1 Machine: 00FB42D74C00
Tprof command was:
tprof -skex sleep 10
Trace command was:
/usr/bin/trace -ad -M -L 955614412 -T 500000 -j 00A,001,002,003,38F,005,006,134,210,139,5A2,5A5,465,234,5D8, -o -
Total Samples = 2002
Traced Time = 10.02s (out of a total execution time of 10.02s)
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Process Freq Total Kernel User Shared Other
======= ==== ===== ====== ==== ====== =====
wait 1 45.50 45.50 0.00 0.00 0.00
muserver 213 34.17 29.52 2.15 2.50 0.00
mumsm 14 15.03 6.49 8.14 0.40 0.00
/usr/sbin/ftpd 5 1.55 1.05 0.00 0.50 0.00
gil 4 1.30 1.30 0.00 0.00 0.00
/usr/bin/ps 2 0.75 0.60 0.00 0.15 0.00
/usr/sbin/syncd 1 0.25 0.25 0.00 0.00 0.00
/bin/ksh 4 0.20 0.10 0.05 0.05 0.00
/usr/bin/ftp 4 0.20 0.15 0.00 0.05 0.00
/usr/bin/rm 2 0.10 0.10 0.00 0.00 0.00
/usr/bin/tprof 2 0.10 0.05 0.00 0.05 0.00
/usr/sbin/inetd 2 0.10 0.10 0.00 0.00 0.00
/usr/bin/sh 1 0.05 0.05 0.00 0.00 0.00
/usr/sbin/rsct/bin/hagsd 1 0.05 0.05 0.00 0.00 0.00
/usr/sbin/snmpd 1 0.05 0.05 0.00 0.00 0.00
/usr/sbin/getty 1 0.05 0.05 0.00 0.00 0.00
/usr/sbin/syslogd 1 0.05 0.00 0.00 0.05 0.00
/usr/sbin/zabbix_agentd: 1 0.05 0.05 0.00 0.00 0.00
/usr/sbin/clcomd 1 0.05 0.05 0.00 0.00 0.00
/var/opt/tivoli/ep/_jvm/jre/bin/java 1 0.05 0.05 0.00 0.00 0.00
/usr/bin/awk 1 0.05 0.05 0.00 0.00 0.00
/usr/bin/lp 1 0.05 0.05 0.00 0.00 0.00
lock_rcv 1 0.05 0.05 0.00 0.00 0.00
lrud 1 0.05 0.05 0.00 0.00 0.00
/usr/bin/grep 1 0.05 0.05 0.00 0.00 0.00
/usr/bin/enq 1 0.05 0.05 0.00 0.00 0.00
mutio 1 0.05 0.05 0.00 0.00 0.00
======= ==== ===== ====== ==== ====== =====
Total 269 100.00 85.91 10.34 3.75 0.00

Process PID TID Total Kernel User Shared Other
======= === === ===== ====== ==== ====== =====
wait 131076 131077 45.50 45.50 0.00 0.00 0.00
mumsm 2294088 75431989 13.94 5.44 8.14 0.35 0.00
muserver 27066692 84869175 0.85 0.85 0.00 0.00 0.00
muserver 5374448 37879949 0.70 0.65 0.00 0.05 0.00
muserver 11862232 34537611 0.65 0.60 0.05 0.00 0.00
muserver 49152014 66978043 0.65 0.60 0.05 0.00 0.00
muserver 16580696 48300045 0.65 0.60 0.05 0.00 0.00
muserver 16973934 111149099 0.60 0.60 0.00 0.00 0.00
/usr/sbin/ftpd 19464572 56361167 0.60 0.30 0.00 0.30 0.00
muserver 24248320 33357851 0.60 0.40 0.05 0.15 0.00
muserver 43253952 95486057 0.60 0.55 0.05 0.00 0.00
muserver 45416552 87490577 0.55 0.55 0.00 0.00 0.00
muserver 53084204 64290915 0.55 0.55 0.00 0.00 0.00
muserver 16056398 67960967 0.55 0.40 0.05 0.10 0.00
muserver 721204 69206233 0.50 0.45 0.05 0.00 0.00
muserver 31195350 110559441 0.50 0.40 0.10 0.00 0.00
/usr/sbin/ftpd 30605782 93651129 0.50 0.35 0.00 0.15 0.00
muserver 5570968 83099851 0.50 0.45 0.00 0.05 0.00
muserver 30147068 38928523 0.50 0.40 0.00 0.10 0.00
muserver 47251700 30801975 0.45 0.40 0.05 0.00 0.00
muserver 53936288 64618743 0.45 0.35 0.00 0.10 0.00
gil 1179684 2031679 0.45 0.45 0.00 0.00 0.00
muserver 6291886 56295613 0.40 0.40 0.00 0.00 0.00
muserver 46334152 79757475 0.40 0.35 0.05 0.00 0.00
muserver 18481254 121372847 0.40 0.30 0.10 0.00 0.00
muserver 13566426 96141445 0.40 0.35 0.00 0.05 0.00
/usr/bin/ps 8781864 66584713 0.40 0.35 0.00 0.05 0.00
muserver 35651758 58065115 0.35 0.30 0.05 0.00 0.00
muserver 3604744 46203009 0.35 0.30 0.00 0.05 0.00
gil 1179684 2228293 0.35 0.35 0.00 0.00 0.00
muserver 46792944 79167653 0.35 0.25 0.10 0.00 0.00
muserver 16384404 36307015 0.35 0.30 0.00 0.05 0.00
muserver 23331206 87883877 0.35 0.35 0.00 0.00 0.00
/usr/bin/ps 21627128 75759809 0.35 0.25 0.00 0.10 0.00
muserver 20185222 97517799 0.35 0.25 0.00 0.10 0.00
muserver 48365694 77856947 0.30 0.30 0.00 0.00 0.00
muserver 28180592 35586265 0.30 0.30 0.00 0.00 0.00
muserver 30867942 114688219 0.30 0.30 0.00 0.00 0.00
muserver 64159960 38797389 0.30 0.25 0.00 0.05 0.00
gil 1179684 2097217 0.30 0.30 0.00 0.00 0.00
muserver 63635656 72614107 0.30 0.25 0.00 0.05 0.00
muserver 18546936 53346491 0.30 0.15 0.10 0.05 0.00
muserver 26345658 74907735 0.30 0.25 0.05 0.00 0.00
muserver 52232434 101515471 0.30 0.30 0.00 0.00 0.00
muserver 62652590 70189201 0.30 0.30 0.00 0.00 0.00
muserver 10616916 72024079 0.30 0.20 0.05 0.05 0.00
muserver 8257962 109445239 0.30 0.30 0.00 0.00 0.00
muserver 15401232 52560121 0.25 0.25 0.00 0.00 0.00
muserver 6947142 44040267 0.25 0.25 0.00 0.00 0.00
muserver 15728686 119013629 0.25 0.20 0.05 0.00 0.00
muserver 2621768 40304781 0.25 0.20 0.05 0.00 0.00
muserver 38142142 20906023 0.25 0.25 0.00 0.00 0.00
/usr/sbin/syncd 2228388 5177503 0.25 0.25 0.00 0.00 0.00
muserver 1638756 84803663 0.25 0.20 0.00 0.05 0.00
muserver 25624748 58523669 0.25 0.20 0.00 0.05 0.00
muserver 28639606 90636289 0.25 0.20 0.05 0.00 0.00
muserver 9044138 39518369 0.25 0.25 0.00 0.00 0.00
muserver 44236920 108527727 0.20 0.15 0.05 0.00 0.00
muserver 10944750 107085867 0.20 0.20 0.00 0.00 0.00
muserver 61734940 39911671 0.20 0.15 0.00 0.05 0.00
muserver 40960240 81527021 0.20 0.15 0.05 0.00 0.00
muserver 10027062 103809067 0.20 0.20 0.00 0.00 0.00
muserver 60620828 90439765 0.20 0.20 0.00 0.00 0.00
muserver 40697860 49283091 0.20 0.10 0.05 0.05 0.00
muserver 19398814 78053403 0.20 0.20 0.00 0.00 0.00
muserver 11403676 87752889 0.20 0.20 0.00 0.00 0.00
gil 1179684 2162755 0.20 0.20 0.00 0.00 0.00
mumsm 2294088 119734315 0.20 0.20 0.00 0.00 0.00
muserver 14942606 108134419 0.20 0.10 0.05 0.05 0.00
muserver 63242428 81133791 0.20 0.15 0.05 0.00 0.00
muserver 10682694 65536225 0.20 0.20 0.00 0.00 0.00
muserver 40239112 103153843 0.20 0.10 0.00 0.10 0.00
/usr/sbin/ftpd 8781868 66584717 0.15 0.15 0.00 0.00 0.00
muserver 19333248 35258471 0.15 0.15 0.00 0.00 0.00
muserver 23920896 106168429 0.15 0.05 0.05 0.05 0.00
muserver 34013198 79429743 0.15 0.15 0.00 0.00 0.00
muserver 66322500 113377421 0.15 0.15 0.00 0.00 0.00
muserver 22282458 81199333 0.15 0.10 0.00 0.05 0.00
muserver 44564698 78119137 0.15 0.15 0.00 0.00 0.00
muserver 1900960 57016537 0.15 0.10 0.05 0.00 0.00
muserver 23134420 56950831 0.15 0.05 0.00 0.10 0.00
muserver 22085640 96731181 0.15 0.15 0.00 0.00 0.00
muserver 18678218 114950351 0.15 0.15 0.00 0.00 0.00
muserver 56295424 97452237 0.15 0.15 0.00 0.00 0.00
muserver 2818426 61735107 0.15 0.15 0.00 0.00 0.00
muserver 25428026 97124547 0.15 0.10 0.00 0.05 0.00
muserver 14418366 80675005 0.15 0.10 0.00 0.05 0.00
mumsm 2294088 86179985 0.15 0.15 0.00 0.00 0.00
muserver 23593028 80347291 0.15 0.10 0.05 0.00 0.00
muserver 3932596 79626303 0.15 0.05 0.00 0.10 0.00
muserver 45744302 86376515 0.15 0.15 0.00 0.00 0.00
mumsm 2294088 33816771 0.15 0.15 0.00 0.00 0.00
muserver 61014238 73400491 0.15 0.10 0.05 0.00 0.00
muserver 22348084 78774447 0.15 0.15 0.00 0.00 0.00
muserver 9699654 69664859 0.15 0.15 0.00 0.00 0.00
muserver 10158576 97189983 0.15 0.15 0.00 0.00 0.00
muserver 63766532 65863777 0.15 0.15 0.00 0.00 0.00
/usr/sbin/ftpd 19464574 56361169 0.15 0.15 0.00 0.00 0.00
muserver 49872932 54657207 0.15 0.10 0.00 0.05 0.00
muserver 13959404 34865335 0.15 0.15 0.00 0.00 0.00
/usr/sbin/ftpd 19464570 56361165 0.15 0.10 0.00 0.05 0.00
muserver 27852926 69075099 0.10 0.10 0.00 0.00 0.00
muserver 9830652 84082745 0.10 0.05 0.05 0.00 0.00
muserver 26345944 63373505 0.10 0.10 0.00 0.00 0.00
muserver 15663334 74383395 0.10 0.10 0.00 0.00 0.00
muserver 55574738 114491429 0.10 0.05 0.00 0.05 0.00
muserver 61603860 113180825 0.10 0.10 0.00 0.00 0.00
muserver 31523000 118882323 0.10 0.10 0.00 0.00 0.00
muserver 66519224 43843665 0.10 0.10 0.00 0.00 0.00
muserver 40632382 117768391 0.10 0.05 0.05 0.00 0.00
muserver 2490864 28967115 0.10 0.10 0.00 0.00 0.00
muserver 14156226 36700179 0.10 0.10 0.00 0.00 0.00
mumsm 2294088 92078305 0.10 0.10 0.00 0.00 0.00
muserver 13107444 121897047 0.10 0.10 0.00 0.00 0.00
muserver 26804260 68354137 0.10 0.10 0.00 0.00 0.00
muserver 55836774 45744275 0.10 0.10 0.00 0.00 0.00
muserver 42729546 27066457 0.10 0.10 0.00 0.00 0.00
muserver 27591006 82051161 0.10 0.10 0.00 0.00 0.00
mumsm 2294088 52756621 0.10 0.10 0.00 0.00 0.00
muserver 54460564 107282451 0.10 0.05 0.00 0.05 0.00
muserver 20578576 104399057 0.10 0.05 0.00 0.05 0.00
muserver 27197486 71827665 0.10 0.10 0.00 0.00 0.00
muserver 64356574 105316399 0.10 0.10 0.00 0.00 0.00
muserver 20119806 60555277 0.10 0.10 0.00 0.00 0.00
muserver 16318514 36569231 0.10 0.10 0.00 0.00 0.00
muserver 1049084 34144287 0.10 0.10 0.00 0.00 0.00
muserver 21889432 83755219 0.10 0.10 0.00 0.00 0.00
muserver 65142926 55509123 0.10 0.10 0.00 0.00 0.00
muserver 11469196 28311769 0.10 0.05 0.00 0.05 0.00
muserver 49021144 104923279 0.10 0.10 0.00 0.00 0.00
muserver 15532044 72679557 0.10 0.05 0.05 0.00 0.00
muserver 54067364 59834465 0.10 0.10 0.00 0.00 0.00
muserver 12845250 78381155 0.10 0.10 0.00 0.00 0.00
muserver 11468830 45088869 0.10 0.00 0.00 0.10 0.00
muserver 5636386 68223195 0.10 0.05 0.05 0.00 0.00
muserver 27263212 116719835 0.10 0.10 0.00 0.00 0.00
muserver 43516128 42860645 0.10 0.05 0.05 0.00 0.00
muserver 4259860 112459911 0.10 0.10 0.00 0.00 0.00
muserver 7143732 90112109 0.10 0.10 0.00 0.00 0.00
muserver 54263978 7602401 0.10 0.10 0.00 0.00 0.00
muserver 12058794 96206963 0.10 0.00 0.00 0.10 0.00
muserver 45154356 120389857 0.10 0.10 0.00 0.00 0.00
muserver 45940984 69337315 0.10 0.10 0.00 0.00 0.00
muserver 46530578 81264755 0.10 0.00 0.05 0.05 0.00
muserver 23396586 49414259 0.10 0.10 0.00 0.00 0.00
muserver 29950292 42336489 0.10 0.10 0.00 0.00 0.00
muserver 59572300 40108269 0.10 0.10 0.00 0.00 0.00
muserver 56426554 57344009 0.10 0.10 0.00 0.00 0.00
muserver 4719062 104726647 0.10 0.10 0.00 0.00 0.00
muserver 21299590 45351031 0.10 0.10 0.00 0.00 0.00
muserver 23724038 44433535 0.10 0.10 0.00 0.00 0.00
muserver 31981778 45809915 0.10 0.10 0.00 0.00 0.00
muserver 16449832 116129815 0.05 0.05 0.00 0.00 0.00
muserver 28770496 96272527 0.05 0.05 0.00 0.00 0.00
mumsm 2294088 36241641 0.05 0.05 0.00 0.00 0.00
muserver 53739592 40370263 0.05 0.05 0.00 0.00 0.00
muserver 60883156 20316187 0.05 0.05 0.00 0.00 0.00
mumsm 2294088 55443677 0.05 0.00 0.00 0.05 0.00
muserver 17367052 61800593 0.05 0.05 0.00 0.00 0.00
mumsm 2294088 25624693 0.05 0.05 0.00 0.00 0.00
muserver 64094438 90898521 0.05 0.05 0.00 0.00 0.00
muserver 43909230 102367381 0.05 0.05 0.00 0.00 0.00
mumsm 2294088 92405875 0.05 0.05 0.00 0.00 0.00
muserver 28770730 35848221 0.05 0.00 0.00 0.05 0.00
muserver 3473770 94371929 0.05 0.05 0.00 0.00 0.00
mumsm 2294088 31326303 0.05 0.05 0.00 0.00 0.00
mumsm 2294088 39452765 0.05 0.05 0.00 0.00 0.00
muserver 58720510 70320285 0.05 0.05 0.00 0.00 0.00
muserver 50200586 31653909 0.05 0.05 0.00 0.00 0.00
muserver 22020434 35651743 0.05 0.05 0.00 0.00 0.00
muserver 28705178 54722719 0.05 0.05 0.00 0.00 0.00
muserver 57409562 103284827 0.05 0.05 0.00 0.00 0.00
muserver 17498590 88211619 0.05 0.00 0.05 0.00 0.00
mumsm 2294088 98631737 0.05 0.05 0.00 0.00 0.00
lrud 262152 983071 0.05 0.05 0.00 0.00 0.00
muserver 43122848 103219293 0.05 0.05 0.00 0.00 0.00
muserver 17826284 91619503 0.05 0.05 0.00 0.00 0.00
muserver 13631608 87031901 0.05 0.05 0.00 0.00 0.00
lock_rcv 3145842 3997839 0.05 0.05 0.00 0.00 0.00
muserver 19071246 86835295 0.05 0.05 0.00 0.00 0.00
muserver 13828562 120848479 0.05 0.05 0.00 0.00 0.00
ep/_jvm/jre/bin/java 7995446 25428021 0.05 0.05 0.00 0.00 0.00
muserver 4522274 58916899 0.05 0.05 0.00 0.00 0.00
/sbin/zabbix_agentd: 6488100 11075761 0.05 0.05 0.00 0.00 0.00
/usr/sbin/syslogd 3604652 7078027 0.05 0.00 0.00 0.05 0.00
/usr/sbin/snmpd 5570740 9109527 0.05 0.05 0.00 0.00 0.00
/sbin/rsct/bin/hagsd 8061182 24641541 0.05 0.05 0.00 0.00 0.00
muserver 29360306 57409723 0.05 0.00 0.05 0.00 0.00
/usr/sbin/inetd 8781868 66584717 0.05 0.05 0.00 0.00 0.00
muserver 39190608 86704317 0.05 0.05 0.00 0.00 0.00
/usr/sbin/inetd 19464570 56361165 0.05 0.05 0.00 0.00 0.00
/usr/sbin/getty 4915392 9961623 0.05 0.05 0.00 0.00 0.00
muserver 40828966 78839999 0.05 0.05 0.00 0.00 0.00
muserver 43450464 88539329 0.05 0.05 0.00 0.00 0.00
muserver 53805086 40566977 0.05 0.05 0.00 0.00 0.00
muserver 16187566 75825187 0.05 0.05 0.00 0.00 0.00
muserver 9961480 34799717 0.05 0.05 0.00 0.00 0.00
/usr/sbin/clcomd 6029314 10420333 0.05 0.05 0.00 0.00 0.00
muserver 21561478 44695657 0.05 0.05 0.00 0.00 0.00
/usr/bin/tprof 13697312 100663359 0.05 0.05 0.00 0.00 0.00
/usr/bin/tprof 8323134 79102135 0.05 0.00 0.00 0.05 0.00
muserver 14876948 29884613 0.05 0.00 0.05 0.00 0.00
muserver 29688068 121438221 0.05 0.05 0.00 0.00 0.00
muserver 19726354 85524521 0.05 0.05 0.00 0.00 0.00
/usr/bin/sh 21627130 75759811 0.05 0.05 0.00 0.00 0.00
/usr/bin/rm 21626882 75759819 0.05 0.05 0.00 0.00 0.00
/usr/bin/rm 21626888 75759825 0.05 0.05 0.00 0.00 0.00
muserver 49545242 36962507 0.05 0.05 0.00 0.00 0.00
/usr/bin/lp 21627132 75759813 0.05 0.05 0.00 0.00 0.00
/usr/bin/grep 41025666 69468403 0.05 0.05 0.00 0.00 0.00
muserver 7602560 74121225 0.05 0.05 0.00 0.00 0.00
/usr/bin/ftp 21626896 75759833 0.05 0.05 0.00 0.00 0.00
muserver 7537068 112001141 0.05 0.05 0.00 0.00 0.00
muserver 19857616 97648847 0.05 0.05 0.00 0.00 0.00
/usr/bin/ftp 21627134 75759815 0.05 0.05 0.00 0.00 0.00
/usr/bin/ftp 21626890 75759827 0.05 0.00 0.00 0.05 0.00
/usr/bin/ftp 21626884 75759821 0.05 0.05 0.00 0.00 0.00
muserver 20578546 105775313 0.05 0.05 0.00 0.00 0.00
muserver 43712622 57671727 0.05 0.00 0.00 0.05 0.00
/usr/bin/enq 41025676 69468413 0.05 0.05 0.00 0.00 0.00
muserver 22413456 78971003 0.05 0.05 0.00 0.00 0.00
muserver 51183684 41746555 0.05 0.05 0.00 0.00 0.00
muserver 30540254 92602493 0.05 0.00 0.00 0.05 0.00
muserver 49086484 59768835 0.05 0.05 0.00 0.00 0.00
muserver 55640098 89194621 0.05 0.05 0.00 0.00 0.00
muserver 44695732 74973395 0.05 0.05 0.00 0.00 0.00
muserver 17039628 74317951 0.05 0.05 0.00 0.00 0.00
muserver 51970132 55246897 0.05 0.05 0.00 0.00 0.00
muserver 13631924 56492249 0.05 0.05 0.00 0.00 0.00
/usr/bin/awk 9895992 36765779 0.05 0.05 0.00 0.00 0.00
muserver 24314218 69927041 0.05 0.05 0.00 0.00 0.00
/bin/ksh 21626900 75759837 0.05 0.05 0.00 0.00 0.00
/bin/ksh 8781876 66584725 0.05 0.05 0.00 0.00 0.00
/bin/ksh 9895994 36765781 0.05 0.00 0.05 0.00 0.00
/bin/ksh 8781872 66584721 0.05 0.00 0.00 0.05 0.00
muserver 30474486 92799107 0.05 0.05 0.00 0.00 0.00
muserver 55312582 116654099 0.05 0.05 0.00 0.00 0.00
muserver 42008818 77922353 0.05 0.05 0.00 0.00 0.00
muserver 11600248 106954801 0.05 0.05 0.00 0.00 0.00
muserver 20054410 53739775 0.05 0.05 0.00 0.00 0.00
muserver 22217126 84279359 0.05 0.05 0.00 0.00 0.00
muserver 21758398 121307187 0.05 0.00 0.05 0.00 0.00
muserver 61079744 92733499 0.05 0.05 0.00 0.00 0.00
muserver 50987140 56623239 0.05 0.05 0.00 0.00 0.00
muserver 8126560 58392649 0.05 0.05 0.00 0.00 0.00
muserver 10748258 38731851 0.05 0.00 0.05 0.00 0.00
muserver 13041828 39387365 0.05 0.05 0.00 0.00 0.00
muserver 28639290 65339647 0.05 0.05 0.00 0.00 0.00
muserver 39649530 118947859 0.05 0.05 0.00 0.00 0.00
muserver 30867526 48693299 0.05 0.05 0.00 0.00 0.00
muserver 38731876 115278057 0.05 0.05 0.00 0.00 0.00
muserver 54591634 69009463 0.05 0.05 0.00 0.00 0.00
muserver 37093562 118751285 0.05 0.05 0.00 0.00 0.00
muserver 14549178 88866839 0.05 0.05 0.00 0.00 0.00
muserver 30081178 49479919 0.05 0.00 0.00 0.05 0.00
muserver 59506758 30474481 0.05 0.05 0.00 0.00 0.00
muserver 54132902 48169201 0.05 0.05 0.00 0.00 0.00
muserver 12582944 76546289 0.05 0.05 0.00 0.00 0.00
muserver 42598484 57278707 0.05 0.05 0.00 0.00 0.00
muserver 28246022 73138419 0.05 0.05 0.00 0.00 0.00
muserver 16449666 49676339 0.05 0.05 0.00 0.00 0.00
mumsm 2294088 109903907 0.05 0.05 0.00 0.00 0.00
muserver 19661242 28704783 0.05 0.05 0.00 0.00 0.00
muserver 57999392 96862231 0.05 0.05 0.00 0.00 0.00
muserver 19595674 118620411 0.05 0.05 0.00 0.00 0.00
muserver 29229498 91881495 0.05 0.05 0.00 0.00 0.00
muserver 33685616 61669391 0.05 0.05 0.00 0.00 0.00
muserver 53215370 37421109 0.05 0.00 0.00 0.05 0.00
mutio 21365218 61079677 0.05 0.05 0.00 0.00 0.00
======= === === ===== ====== ==== ====== =====
Total 100.00 85.91 10.34 3.75 0.00

Total % For All Processes (KERNEL) = 82.22

Subroutine % Source
========== ====== ======
h_cede_end_point 36.06 hcalls.s
.unlock_enable_mem 9.04 low.s
.enable 6.84 misc.s
hypervisor_call_asm_end_point 6.29 hcalls.s
sc_msr_2_point 3.75 low.s
svc_rfid_ret32 2.80 low.s
.disable_lock 1.50 low.s
._hkeyset_restore_userkeys 1.20 64/skeys.s
hk_update_ukeys_accr_point 0.85 64/skeys.s
.trchook64 0.80 trchka64.s
.umem_move 0.75 low.s
.mtrchook2 0.60 low.s
.hkeyset_restore 0.45 low.s
ovlya_addr_pcs_glue_xon2 0.45 vmvcs.s
.rsemop 0.40 bos/kernel/ipc/sem.c
.atomic 0.40 bos/kernel/ipc/sem.c
.mtrchook1 0.40 low.s
._check_lock 0.40 low.s
.dac_refmon_ipc 0.35 /refmon/dac_refmon.c
._getf_onethread 0.35 bos/kernel/lfs/fio.c
.waitproc 0.30 rnel/proc/dispatch.c
.cs 0.30 low.s
.simple_unlock_mem 0.30 low.s
.soereceive_base 0.30 kernel/uipc/socket.c
.copyout 0.25 /kernel/vmm/userio.c
.simple_lock 0.25 low.s
.svc_instr 0.25 low.s
.hkey_legacy_gate 0.25 low.s
._erecv 0.20 rnel/uipc/syscalls.c
.copyin 0.20 /kernel/vmm/userio.c
.sc_debug_fixup 0.20 low.s
.thr_crref 0.15 kernel/s/auth/cred.c
.hkeyset_add 0.15 low.s
.sigprocmask 0.15 el/proc/sigsvcs_pn.c
.ipcaccess 0.15 bos/kernel/ipc/ipc.c
.rtsem_tsleep_thread 0.15 bos/kernel/ipc/sem.c
._semconv 0.15 bos/kernel/ipc/sem.c
.getuidx_cr 0.10 /kernel/s/auth/uid.c
.mtrchook4 0.10 low.s
.syncHashList 0.10 kernel/j2/j2_inode.c
.net_malloc_cpu 0.10 el/net/kern_malloc.c
.sounlock 0.10 kernel/uipc/socket.c
.m_copydata 0.10 s/kernel/uipc/mbuf.c
.kernel_add_gate_cstack 0.10 low.s
.drw_lock_read 0.10 low.s
.sbdrop 0.10 ernel/uipc/socket2.c
sc_exit_etrc_point 0.10 low.s
.drw_lock_done 0.10 low.s
.hkeyset_replace 0.10 low.s
.mtrchook5 0.10 low.s
._ptrgl 0.10 low.s
.semsleep 0.10 bos/kernel/ipc/sem.c
set_smt_pri_syscall_ret 0.05 low.s
.convert_tb 0.05 ernel/proc/curtime.c
.vm_handle 0.05 ernel/vmm/vmmisc64.c
.thread_twakeup 0.05 kernel/proc/sleep3.c
.xmfree_frontend 0.05 ernel/alloc/xmfree.c
.e_block_thread 0.05 kernel/proc/sleep2.c
.thread_setmymask_fast 0.05 kernel/proc/thread.c
.uiomove 0.05 bos/kernel/ios/uio.c
.as_getadsp64 0.05 /kernel/vmm/vmadsp.c
.sigaction 0.05 el/proc/sigsvcs_pn.c
.v_psizeok 0.05 kernel/vmm/vmpsize.c
.v_lookup_mpss 0.05 ernel/vmm/v_lookup.c
.v_lookup_swpft 0.05 ernel/vmm/v_lookup.c
.v_mflrulist 0.05 rnel/vmm/v_lrulist.c
.i_eoi 0.05 os/kernel/ios/intr.c
vcs_p4_point 0.05 vmvcs.s
.sig_deliver32 0.05 flih_util.s
.v_delpft 0.05 kernel/vmm/v_lists.c
.v_inspft 0.05 kernel/vmm/v_lists.c
.v_delscb 0.05 rnel/vmm/v_scblist.c
.v_scan_compute_weights 0.05 /kernel/vmm/vmscan.c
.v_pageout 0.05 rnel/vmm/v_putsubs.c
.pile_alloc 0.05 s/kernel/pile/pile.c
._hkeyset_restore 0.05 low.s
.iActivate 0.05 kernel/j2/j2_inode.c
.pagerCopyIn 0.05 kernel/j2/j2_pager.c
.bmRelease 0.05 ernel/j2/j2_bufmgr.c
.v_pdtqiox_common 0.05 nel/vmm/v_disksubs.c
.net_free 0.05 el/net/kern_malloc.c
.waitproc_find_run_queue 0.05 rnel/proc/dispatch.c
.j2_close 0.05 kernel/j2/j2_close.c
.j2_seek 0.05 /kernel/j2/j2_seek.c
.kwpar_get_aacct_ker_data 0.05 /kernel/proc/aacct.c
.v_pfget 0.05 el/vmm/v_getsubs64.c
.rn_genmatch 0.05 s/kernel/net/radix.c
.cpuget_from_list 0.05 low.s
.mclpFreeCnt 0.05 /kernel/uipc/mpool.c
.uiocopyin 0.05 /kernel/vmm/userio.c
.fetch_and_addlp 0.05 low.s
._esend 0.05 rnel/uipc/syscalls.c
.fetch_and_add_h 0.05 low.s
.memcmp 0.05 memcmp.s
.lseek 0.05 os/kernel/lfs/seek.c
.refmon_o_ipc_access@AF49_1 0.05 el/s/refmon/refmon.c
_hkeyset_update_userkeys 0.05 64/skeys.s
.fetch_and_or 0.05 low.s
.hmcr_manage_vipc 0.05 nel/corral/mcr_api.c
.fetch_and_add 0.05 low.s
.getf 0.05 bos/kernel/lfs/fio.c
.fp_getdevno 0.05 ernel/lfs/getdevno.c
.rtipc_tsleep_block 0.05 s/kernel/ipc/rtipc.c
.cpu_speculation_barrier 0.05 low.s
_tls_ 0.05 low.s
.hypervisor_call 0.05 nel/ios/POWER/lpar.c
sc_msr_1_point 0.05 low.s

Millicode Subroutine % Source
========== ====== ======
.memcmp_overlay 0.25 low.s
.strcmp_overlay 0.10 low.s
.memccpy_overlay 0.05 low.s
.strcmp64_overlay 0.05 low.s
.memmove64_overlay 0.05 low.s
.strlen_overlay 0.05 low.s
.strlen64_overlay 0.05 low.s

Total % For All Processes (KEX) = 3.70

Kernel Ext %
========== ======
/usr/lib/drivers/netinet 1.85
UnknownBinary 0.65
/usr/lib/drivers/vioentdd 0.60
/usr/lib/drivers/if_en 0.30
/usr/lib/drivers/eth_demux 0.10
/usr/lib/drivers/vfcdd 0.10
/etc/drivers/scsidiskpin 0.05
/usr/lib/drivers/vdev_busdd 0.05

Profile: /usr/lib/drivers/netinet

Total % For All Processes (/usr/lib/drivers/netinet) = 1.85

Subroutine % Source
========== ====== ======
.tcp_input0 0.40 /lib/drivers/netinet
.tcp_output 0.35 /lib/drivers/netinet
.arpresolve_common 0.15 /lib/drivers/netinet
.tcp_input 0.10 /lib/drivers/netinet
.in_pcblookup2 0.10 /lib/drivers/netinet
.trchook64 0.10 glink64.s
.ipintr_noqueue 0.10 /lib/drivers/netinet
.in_cksum 0.10 /lib/drivers/netinet
.ip_output 0.05 /lib/drivers/netinet
.ip_output_post_fw 0.05 /lib/drivers/netinet
.tcp_usrreq 0.05 /lib/drivers/netinet
.tcp_xmit_timer 0.05 /lib/drivers/netinet
.net_malloc 0.05 glink64.s
.in_pcbhashlookup3 0.05 /lib/drivers/netinet
.ipintr_noqueue_post_fw 0.05 /lib/drivers/netinet
.gids_ok 0.05 /lib/drivers/netinet
.in_flip_and_run 0.05 /lib/drivers/netinet

Profile: UnknownBinary

Total % For All Processes (UnknownBinary) = 0.65

Subroutine % Source
========== ====== ======
<0xF1000000C03D5AE8> 0.05
<0xF1000000C03CF780> 0.05
<0xF1000000C03CF700> 0.05
<0xF1000000C03CECD8> 0.05
<0xF1000000C01B28E8> 0.05
<0xF1000000C03E7094> 0.05
<0xF1000000C01A25F0> 0.05
<0x5B94CAC> 0.05
<0xF1000000C0160380> 0.05
<0xF1000000C015FD20> 0.05
<0xF1000000C01384A4> 0.05
<0x5BA01B8> 0.05
<0x5B95400> 0.05

Profile: /usr/lib/drivers/vioentdd

Total % For All Processes (/usr/lib/drivers/vioentdd) = 0.60

Subroutine % Source
========== ====== ======
.vioent_output 0.30 oent/vioent_output.c
.vioent_receive 0.15 ent/vioent_receive.c
.vioent_slih 0.05 ent/vioent_receive.c
.getcaller 0.05 glink64.s
.h_send_logical_lan 0.05 lib/libsysp/hcalls.c

Profile: /usr/lib/drivers/if_en

Total % For All Processes (/usr/lib/drivers/if_en) = 0.30

Subroutine % Source
========== ====== ======
.en_output 0.25 sr/lib/drivers/if_en
.arpresolve 0.05 sr/lib/drivers/if_en

Profile: /usr/lib/drivers/eth_demux

Total % For All Processes (/usr/lib/drivers/eth_demux) = 0.10

Subroutine % Source
========== ====== ======
.eth_receive 0.10 ib/drivers/eth_demux

Profile: /usr/lib/drivers/vfcdd

Total % For All Processes (/usr/lib/drivers/vfcdd) = 0.10

Subroutine % Source
========== ====== ======
.vfc_output 0.05 sr/lib/drivers/vfcdd
.vfc_process_response 0.05 sr/lib/drivers/vfcdd

Profile: /etc/drivers/scsidiskpin

Total % For All Processes (/etc/drivers/scsidiskpin) = 0.05

Subroutine % Source
========== ====== ======
.scsidisk_start_in_progress 0.05 ext/disk/scsidiskb.c

Profile: /usr/lib/drivers/vdev_busdd

Total % For All Processes (/usr/lib/drivers/vdev_busdd) = 0.05

Subroutine % Source
========== ====== ======
.d_map_page_tce 0.05 ci_busdd/d_map_tce.c

Total % For All Processes (SH-LIBs) = 3.75

Shared Object %
============= ======
/usr/lib/libpthreads.a[shr.o] 2.00
/usr/lib/libc.a[shr.o] 1.35
/usr/lib/libc.a[shr_64.o] 0.15
/usr/lib/libpthreads.a[shr_xpg5.o] 0.15
/usr/lib/libpthreads.a[shr_xpg5_64.o] 0.05
/usr/lib/libtrace.a[shr_64.o] 0.05

Profile: /usr/lib/libpthreads.a[shr.o]

Total % For All Processes (/usr/lib/libpthreads.a[shr.o]) = 2.00

Subroutine % Source
========== ====== ======
._global_lock_common 0.35 reads/pth_spinlock.c
._kernel_unlock_slow 0.30 reads/pth_spinlock.c
._p_sigaction 0.25 threads/pth_signal.c
.cs 0.25 glink.s
._sigaction_common 0.20 threads/pth_signal.c
._kernel_lock_slow 0.20 reads/pth_spinlock.c
._lib_geterrno_addr 0.15 bpthreads/pth_libc.c
.sigprocmask 0.10 glink.s
._lib_mutex_unlock 0.05 bpthreads/pth_libc.c
._sigsetmask 0.05 threads/pth_signal.c
.global_lock_ppc_mp 0.05 pth_locks_ppc_mp.s
.sigaction 0.05 threads/pth_signal.c

Profile: /usr/lib/libc.a[shr.o]

Total % For All Processes (/usr/lib/libc.a[shr.o]) = 1.35

Subroutine % Source
========== ====== ======
.semop 0.35 lib/libc/ipc_stubs.c
.recv 0.20 lib/libc/net_stubs.c
.compare_rec_char 0.15 CAL/common_compare.c
._erecv 0.10 glink.s
.strpbrk 0.10 s/lib/libc/strpbrk.c
.__semop 0.05 glink.s
.__is_wctype_sb 0.05 ibc/__is_wctype_sb.c
.send 0.05 lib/libc/net_stubs.c
._rec_mutex_unlock 0.05 lib/libc/rec_mutex.c
.method_getgrset 0.05 AL/method_getgrset.c
.strtol 0.05 cs/lib/libc/strtol.c
.sigemptyset 0.05 cs/lib/libc/sigops.c
.malloc_y 0.05 /lib/libc/malloc_y.c
.read_colon_rec 0.05 /LOCAL/common_file.c

Profile: /usr/lib/libc.a[shr_64.o]

Total % For All Processes (/usr/lib/libc.a[shr_64.o]) = 0.15

Subroutine % Source
========== ====== ======
.compare_rec_char 0.05 CAL/common_compare.c
.__fd_select 0.05 /ccs/lib/libc/iosl.c
._doprnt 0.05 cs/lib/libc/doprnt.c

Profile: /usr/lib/libpthreads.a[shr_xpg5.o]

Total % For All Processes (/usr/lib/libpthreads.a[shr_xpg5.o]) = 0.15

Subroutine % Source
========== ====== ======
.global_unlock_ppc_mp 0.10 pth_locks_ppc_mp.s
.pthread_cleanup_pop 0.05 hreads/pth_pthread.c

Profile: /usr/lib/libpthreads.a[shr_xpg5_64.o]

Total % For All Processes (/usr/lib/libpthreads.a[shr_xpg5_64.o]) = 0.05

Subroutine % Source
========== ====== ======
.global_unlock_ppc_mp 0.05 pth_locks_ppc_mp.s

Profile: /usr/lib/libtrace.a[shr_64.o]

Total % For All Processes (/usr/lib/libtrace.a[shr_64.o]) = 0.05

Subroutine % Source
========== ====== ======
.ischartype 0.05 b/libtrace/trceval.c

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

go to post

Roger Andre · Jun 1, 2022

Hello, follow msmlog logs

[Jun-01-22 10:03:41] And mumsm ended (proc=#20316374, status=90009)
.................... And ...abnormal termination, signal # = 9
[Jun-01-22 10:03:41] very ended (proc=#14221630, status=0)
[Jun-01-22 10:03:41] very ended (proc=#8257564, status=0)
[Jun-01-22 10:03:41] very ended (proc=#64749588, status=0)
[Jun-01-22 10:03:41] very ended (proc=#44957844, status=0)
[Jun-01-22 10:03:41] very ended (proc=#64684234, status=0)
[Jun-01-22 10:03:41] very ended (proc=#42401856, status=0)
[Jun-01-22 10:03:41] very ended (proc=#65405012, status=0)
[Jun-01-22 10:03:41] very ended (proc=#39125020, status=0)
.................... And ...abnormal termination, signal # = 9
[Jun-01-22 10:03:44] And muserver ended (proc=#61014236, status=90009)
.................... And ...abnormal termination, signal # = 9
[Jun-01-22 10:03:44] And muserver ended (proc=#32571450, status=90009)
.................... And ...abnormal termination, signal # = 9
[Jun-01-22 10:03:44] And muserver ended (proc=#27132320, status=90009)
.................... And ...abnormal termination, signal # = 9
[Jun-01-22 10:03:44] And muserver ended (proc=#33685614, status=90009)
.................... And ...abnormal termination, signal # = 9
[Jun-01-22 10:03:44] And muserver ended (proc=#23789816, status=90009)
.................... And ...abnormal termination, signal # = 9
[Jun-01-22 10:03:44] And muserver ended (proc=#4194584, status=90009)
.................... And ...abnormal termination, signal # = 9

[MGR,SYS]>D license^MSASRV()

[MGR,SYS]>W
%MERR=""
Licinfo("ExpDate")="76658"
Licinfo("MaxBackground")="553"
Licinfo("MaxLAT")="0"
Licinfo("MaxNet")="1104"
Licinfo("MaxUsers")="1104"

go to post

Roger Andre · May 25, 2022

Hi, here are some msmlog logs when the drops occur.

Logs

[May-24-22 16:38:47]   Begin MSM process: ./mutio, id=1124, Proc#=31785338
[May-24-22 16:40:16]   Begin MSM process: ./mutio, id=1170, Proc#=51446006
[May-24-22 16:40:24]   Begin MSM process: ./mutio, id=1111, Proc#=10682396
[May-24-22 16:41:06]   Begin MSM process: ./mutio, id=1164, Proc#=14155800
[May-24-22 16:41:14] E recv peek fail, errno=4
[May-24-22 16:41:22]   Begin MSM process: ./mutio, id=1168, Proc#=59441312
[May-24-22 16:41:32]   mutio ended (proc=#10027276, status=0)
[May-24-22 16:41:36]   Begin MSM process: ./mutio, id=1136, Proc#=11141126
[May-24-22 16:41:38] E recv peek fail, errno=4
[May-24-22 16:43:05]   Begin MSM process: ./mutio, id=1112, Proc#=21168612
[May-24-22 16:43:42] E recv peek fail, errno=4
[May-24-22 16:44:03]   Begin MSM process: ./mutio, id=1125, Proc#=9240856
[May-24-22 16:44:06]   mutio ended (proc=#59441312, status=0)
[May-24-22 16:45:31]   Begin MSM process: ./mutio, id=1168, Proc#=30998900
[May-24-22 16:47:52]   mutio ended (proc=#54853690, status=0)
[May-24-22 16:48:11]   mutio ended (proc=#53543120, status=0)
[May-24-22 16:49:17]   Begin MSM process: ./mutio, id=1153, Proc#=31457702
[May-24-22 16:49:21]   Begin MSM process: ./mutio, id=1152, Proc#=30671178
[May-24-22 16:49:28]   mutio ended (proc=#14155800, status=0)
[May-24-22 16:49:39]   mutio ended (proc=#11141126, status=0)
[May-24-22 16:50:08]   mutio ended (proc=#23200234, status=0)
[May-24-22 16:50:29] E recv peek fail, errno=4
[May-24-22 16:50:32]   mutio ended (proc=#32833614, status=0)
[May-24-22 16:50:51]   Begin MSM process: ./mutio, id=1117, Proc#=5177690
[May-24-22 16:51:24]   Begin MSM process: ./mutio, id=1119, Proc#=23200130
[May-24-22 16:52:21]   mutio ended (proc=#14745610, status=0)
[May-24-22 16:53:14]   mutio ended (proc=#5177690, status=0)
[May-24-22 16:54:00]   Begin MSM process: ./mutio, id=1117, Proc#=35520534
[May-24-22 16:54:23] E recv peek fail, errno=11
[May-24-22 16:54:34] E recv peek fail, errno=4
[May-24-22 16:55:00]   Begin MSM process: ./mutio, id=1120, Proc#=14155776
[May-24-22 16:55:32]   Begin MSM process: ./mutio, id=1136, Proc#=65929218
[May-24-22 16:55:51]   mutio ended (proc=#30998900, status=0)
[May-24-22 16:55:56] E recv peek fail, errno=4
[May-24-22 16:56:20] E recv peek fail, errno=4
[May-24-22 16:56:27]   mutio ended (proc=#14155776, status=0)
[May-24-22 16:57:17]   mutio ended (proc=#62324752, status=0)
[May-24-22 16:57:28]   Begin MSM process: ./mutio, id=1116, Proc#=14155866
[May-24-22 16:57:45]   mutio ended (proc=#10682396, status=0)
[May-24-22 16:57:45] E recv peek fail, errno=4
[May-24-22 16:58:35]   mutio ended (proc=#7537144, status=0)
[May-24-22 16:58:48]   Begin MSM process: ./mutio, id=1148, Proc#=13893868
[May-24-22 16:58:49]   Begin MSM process: ./mutio, id=1111, Proc#=30998818
[May-24-22 16:59:27]   mutio ended (proc=#13893868, status=0)
[May-24-22 17:00:30]   Begin MSM process: ./mutio, id=1148, Proc#=44367872
[May-24-22 17:00:48]   mutio ended (proc=#44367872, status=0)
[May-24-22 17:01:00]   Begin MSM process: ./mutio, id=1148, Proc#=27984208
[May-24-22 17:01:00]   mutio ended (proc=#27984208, status=0)
[May-24-22 17:01:44] E recv peek fail, errno=4
[May-24-22 17:05:03] E recv peek fail, errno=11
[May-24-22 17:05:06] E recv peek fail, errno=11
[May-24-22 17:05:07] E recv peek fail, errno=11
[May-24-22 17:05:07] E recv peek fail, errno=11
[May-24-22 17:05:08] E recv peek fail, errno=11
[May-24-22 17:05:12] E recv peek fail, errno=11
[May-24-22 17:05:13] E recv peek fail, errno=11
[May-24-22 17:05:13] E recv peek fail, errno=11
[May-24-22 17:05:20] E recv peek fail, errno=11
[May-24-22 17:05:20] E recv peek fail, errno=11
[May-24-22 17:05:23] E recv peek fail, errno=11
[May-24-22 17:05:30] E mumsm ended (proc=#66519134, status=40084)
.................... E ...abnormal termination, signal # = 4
[May-24-22 17:05:30]   muserver ended (proc=#8978906, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#18874850, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#25100716, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#19399112, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#30081174, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#4063724, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#11862122, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#8126616, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#5112160, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#21495946, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#6488484, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#524728, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#23003562, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#2490640, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#2556250, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#16515534, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#4325650, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#29688274, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#59506928, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#5898316, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#25362742, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#2949410, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#1704248, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#18416098, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#13763028, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#12058982, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#15925502, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#26935570, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#28770802, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#16908316, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#62521452, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#22741372, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#14942620, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#13500706, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#31719578, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#33161436, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10617336, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#66584768, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#29753550, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#26607926, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#28442782, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#19660898, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#52887672, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#34668612, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#55967948, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#20906300, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#56885364, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#15401038, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#61669382, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#2687468, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#23593436, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#28246120, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#66125980, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#20316306, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#46661682, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#19726506, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#28704830, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#16056560, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#21299346, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#26411490, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#18481602, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#24248428, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#31981654, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#16515178, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#39518238, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#45023448, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#13959218, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10486008, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#12583058, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#356, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#12714158, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#39387234, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#30605654, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#26083548, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#5374320, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#49742022, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#46989370, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#28114954, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#60883162, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#61341866, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#5046592, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#14680250, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10027048, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#17826080, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#28639242, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#64815104, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10551604, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#22675830, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#66060410, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#22479186, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#30802360, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#6422984, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#6684964, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#5243048, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#5832924, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#6357026, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#6553734, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#6881326, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#7667878, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#8192056, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#8519758, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#8585376, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#8781918, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#8912990, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#8978650, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#9109706, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#9240628, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#9371654, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#9502932, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10092630, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10223700, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10354734, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10551420, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10879154, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10944732, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#15466990, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#11731104, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#10420322, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#11403482, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#11534356, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#11600010, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#12124200, status=0)
[May-24-22 17:05:30]   muserver ended (proc=#12386430, status=0)

[May-25-22 11:00:50]   muserver ended (proc=#55115956, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#57212952, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#262442, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#3801412, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#5701916, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#9633818, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#24576016, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#38928454, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#55443470, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#58589214, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#4915498, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#7143716, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#22085986, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#23790038, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#24510720, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#19530218, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#26542428, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#49217764, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#57016444, status=0)
[May-25-22 11:00:50]   mubij ended (proc=#16842906, status=0)
[May-25-22 11:00:50]   mujrnl ended (proc=#62455808, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#65863786, status=0)
[May-25-22 11:00:50]   muserver ended (proc=#58064978, status=0)
[May-25-22 11:00:51]   muserver ended (proc=#65732856, status=0)
[May-25-22 11:00:52] E muserver ended (proc=#10551422, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#10420324, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#11534358, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#12124202, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#12910634, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#13238494, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#15859718, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#17301554, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#17825858, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#18153548, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#18743452, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#18940080, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#19791998, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#20971698, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#21037220, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#23265532, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#24444940, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#33554466, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#9306158, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#31064242, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#12845234, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#9830636, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#12451866, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#11665522, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#9961720, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#32047192, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#7012392, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#10158088, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#24379512, status=90009)
.................... E ...abnormal termination, signal # = 9
[May-25-22 11:00:52] E muserver ended (proc=#14352584, status=90009)
.................... E ...abnormal termination, signal # = 9

Social networks

InterSystems resources