ninipig 发表于 2015-2-2 10:25:19

AIX 6.1 RAC 11.2.0.3.6 节点2的ACFS故障

$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       oradb1                                       
               ONLINE  ONLINE       oradb2                                       
ora.LISTENER.lsnr
               ONLINE  ONLINE       oradb1                                       
               ONLINE  ONLINE       oradb2                                       
ora.asm
               ONLINE  ONLINE       oradb1                   Started            
               ONLINE  ONLINE       oradb2                   Started            
ora.gsd
               OFFLINE OFFLINE      oradb1                                       
               OFFLINE OFFLINE      oradb2                                       
ora.net1.network
               ONLINE  ONLINE       oradb1                                       
               ONLINE  ONLINE       oradb2                                       
ora.ons
               ONLINE  ONLINE       oradb1                                       
               ONLINE  OFFLINE      oradb2                                       
ora.registry.acfs
               ONLINE  ONLINE       oradb1                                       
               ONLINE  UNKNOWN      oradb2                                       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       oradb2                                       
ora.cvu
      1        ONLINE  ONLINE       oradb2                                       
ora.oc4j
      1        ONLINE  ONLINE       oradb1                                       
ora.oradb1.vip
      1        ONLINE  ONLINE       oradb1                                       
ora.oradb2.vip
      1        ONLINE  ONLINE       oradb2                                       
ora.orcl.db
      1        ONLINE  ONLINE       oradb1                   Open               
      2        ONLINE  ONLINE       oradb2                   Open               
ora.scan1.vip
      1        ONLINE  ONLINE       oradb2                                       
$

$ crsctl start resource ora.registry.acfs -n oradb2
CRS-2679: Attempting to clean 'ora.registry.acfs' on 'oradb2'
CRS-5016: Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "clean" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
CRS-2680: Clean of 'ora.registry.acfs' on 'oradb2' failed
CRS-4000: Command Start failed, or completed with errors.

ninipig 发表于 2015-2-2 10:30:25

日志见附件，谢谢

ninipig 发表于 2015-2-2 12:02:38

$ crsctl start resource ora.registry.acfs -n oradb2
CRS-2679: Attempting to clean 'ora.registry.acfs' on 'oradb2'
CRS-5016: Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "clean" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
CRS-2680: Clean of 'ora.registry.acfs' on 'oradb2' failed
CRS-4000: Command Start failed, or completed with errors.

$ crsctl start resource ora.ons -n oradb2
CRS-2672: Attempting to start 'ora.ons' on 'oradb2'
CRS-5016: Process "/u01/grid/11.2.0/grid/opmn/bin/onsctli" spawned by agent "/u01/grid/11.2.0/grid/bin/oraagent.bin" for action "start" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/oraagent_grid/oraagent_grid.log"
CRS-2674: Start of 'ora.ons' on 'oradb2' failed
CRS-4000: Command Start failed, or completed with errors.

ninipig 发表于 2015-2-2 12:02:54

another log

ninipig 发表于 2015-2-2 16:06:56

节点2的CRS日志，一直在报这个错误：

2015-02-02 15:06:02.310
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:06:32.308
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:07:02.325
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:07:32.323
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:08:02.317
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:08:32.439
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:09:02.331
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:09:32.324
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:10:02.322
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:10:32.319
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:11:02.319
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"
2015-02-02 15:11:32.322
CRS-5016:Process "/u01/grid/11.2.0/grid/bin/acfsregistrymount" spawned by agent "/u01/grid/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log"

ninipig 发表于 2015-2-2 16:07:12

$ tail -200 /u01/grid/11.2.0/grid/log/oradb2/agent/crsd/orarootagent_root/orarootagent_root.log
2015-02-02 15:10:02.322: {2:10686:2} execCmd ret = 255
2015-02-02 15:10:02.322: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 31654062 Memory fault(coredump)
2015-02-02 15:10:02.322: {2:10686:2} (:CLSN00010:)
2015-02-02 15:10:02.322: {2:10686:2} RegistryAcfsAgent:: Check type: CLSAGFW_PERIODIC_CHECK
2015-02-02 15:10:02.323: {2:10686:2} RegistryAcfsAgent:: Check Return: 255
2015-02-02 15:10:02.323: [ AGFW] {2:10686:2} Received unknown resource status code: 255
2015-02-02 15:10:19.941: TM is changing desired thread # to 5. Current # is 6
2015-02-02 15:10:19.941: {0:2:24149} Worker thread is exiting in TM to meet the desired count of 5. New count is 5
2015-02-02 15:10:19.942: TM is changing desired thread # to 6. Current # is 5
2015-02-02 15:10:30.960: [ AGFW] {2:10686:2} Agent received the message: AGENT_HB ID 12293:3589927
2015-02-02 15:10:32.193: {2:10686:2} Filling Environment Map with _ORA_CHECK_TIMEOUT = 600
2015-02-02 15:10:32.194: {2:10686:2} Filling Environment Map with _ORA_CHECK_TYPE = CLSAGFW_PERIODIC_CHECK
2015-02-02 15:10:32.194: {2:10686:2} RegistryAcfsAgent:: Calling check
2015-02-02 15:10:32.194: {2:10686:2} Filling Environment Map with ORACLE_HOME = /u01/grid/11.2.0/grid
2015-02-02 15:10:32.194: {2:10686:2} Filling Environment Map with ORACLE_SID = +ASM2
2015-02-02 15:10:32.194: {2:10686:2} Arg Value = check
2015-02-02 15:10:32.195: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:10:32.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:10:32.195: {2:10686:2} Utils::execCmd 1 USR_ORA_ENV: oracleHome:/u01/grid/11.2.0/grid CrsHome:/u01/grid/11.2.0/grid
2015-02-02 15:10:32.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:10:32.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:10:32.195: {2:10686:2} Adding Environment Variables ORACLE_HOME=/u01/grid/11.2.0/grid
2015-02-02 15:10:32.196: {2:10686:2} Adding Environment Variables ORACLE_SID=+ASM2
2015-02-02 15:10:32.196: {2:10686:2} Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2015-02-02 15:10:32.196: {2:10686:2} Adding Environment Variables _ORA_CHECK_TIMEOUT=600
2015-02-02 15:10:32.196: {2:10686:2} Adding Environment Variables _ORA_CHECK_TYPE=CLSAGFW_PERIODIC_CHECK
2015-02-02 15:10:32.196: {2:10686:2} Adding Environment Variables __IS_HASD_AGENT=
2015-02-02 15:10:32.196: {2:10686:2} Utils:execCmd action = 3 flags = 6 ohome = (null) cmdname = acfsregistrymount.
2015-02-02 15:10:32.196: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:10:32.318: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 31654070 Memory fault(coredump)
2015-02-02 15:10:32.318: {2:10686:2} (:CLSN00010:)
2015-02-02 15:10:32.319: {2:10686:2} (:CLSN00010:)Utils:execCmd scls_process_join() uret 255

2015-02-02 15:10:32.319: {2:10686:2} execCmd ret = 255
2015-02-02 15:10:32.319: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 31654070 Memory fault(coredump)
2015-02-02 15:10:32.319: {2:10686:2} (:CLSN00010:)
2015-02-02 15:10:32.319: {2:10686:2} RegistryAcfsAgent:: Check type: CLSAGFW_PERIODIC_CHECK
2015-02-02 15:10:32.319: {2:10686:2} RegistryAcfsAgent:: Check Return: 255
2015-02-02 15:10:32.320: [ AGFW] {2:10686:2} Received unknown resource status code: 255
2015-02-02 15:10:49.942: TM is changing desired thread # to 5. Current # is 6
2015-02-02 15:10:49.942: {0:2:24151} Worker thread is exiting in TM to meet the desired count of 5. New count is 5
2015-02-02 15:10:59.741: TM is changing desired thread # to 6. Current # is 5
2015-02-02 15:11:02.192: {2:10686:2} Filling Environment Map with _ORA_CHECK_TIMEOUT = 600
2015-02-02 15:11:02.193: {2:10686:2} Filling Environment Map with _ORA_CHECK_TYPE = CLSAGFW_PERIODIC_CHECK
2015-02-02 15:11:02.193: {2:10686:2} RegistryAcfsAgent:: Calling check
2015-02-02 15:11:02.193: {2:10686:2} Filling Environment Map with ORACLE_HOME = /u01/grid/11.2.0/grid
2015-02-02 15:11:02.193: {2:10686:2} Filling Environment Map with ORACLE_SID = +ASM2
2015-02-02 15:11:02.193: {2:10686:2} Arg Value = check
2015-02-02 15:11:02.194: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:11:02.194: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:11:02.194: {2:10686:2} Utils::execCmd 1 USR_ORA_ENV: oracleHome:/u01/grid/11.2.0/grid CrsHome:/u01/grid/11.2.0/grid
2015-02-02 15:11:02.194: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:11:02.194: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:11:02.195: {2:10686:2} Adding Environment Variables ORACLE_HOME=/u01/grid/11.2.0/grid
2015-02-02 15:11:02.195: {2:10686:2} Adding Environment Variables ORACLE_SID=+ASM2
2015-02-02 15:11:02.195: {2:10686:2} Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2015-02-02 15:11:02.195: {2:10686:2} Adding Environment Variables _ORA_CHECK_TIMEOUT=600
2015-02-02 15:11:02.195: {2:10686:2} Adding Environment Variables _ORA_CHECK_TYPE=CLSAGFW_PERIODIC_CHECK
2015-02-02 15:11:02.195: {2:10686:2} Adding Environment Variables __IS_HASD_AGENT=
2015-02-02 15:11:02.195: {2:10686:2} Utils:execCmd action = 3 flags = 6 ohome = (null) cmdname = acfsregistrymount.
2015-02-02 15:11:02.196: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:11:02.318: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 31654134 Memory fault(coredump)
2015-02-02 15:11:02.319: {2:10686:2} (:CLSN00010:)
2015-02-02 15:11:02.319: {2:10686:2} (:CLSN00010:)Utils:execCmd scls_process_join() uret 255

2015-02-02 15:11:02.319: {2:10686:2} execCmd ret = 255
2015-02-02 15:11:02.319: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 31654134 Memory fault(coredump)
2015-02-02 15:11:02.319: {2:10686:2} (:CLSN00010:)
2015-02-02 15:11:02.320: {2:10686:2} RegistryAcfsAgent:: Check type: CLSAGFW_PERIODIC_CHECK
2015-02-02 15:11:02.320: {2:10686:2} RegistryAcfsAgent:: Check Return: 255
2015-02-02 15:11:02.320: [ AGFW] {2:10686:2} Received unknown resource status code: 255
2015-02-02 15:11:19.944: TM is changing desired thread # to 5. Current # is 6
2015-02-02 15:11:19.944: {0:2:24153} Worker thread is exiting in TM to meet the desired count of 5. New count is 5
2015-02-02 15:11:23.741: TM is changing desired thread # to 6. Current # is 5
2015-02-02 15:11:30.952: [ AGFW] {2:10686:2} Agent received the message: AGENT_HB ID 12293:3589939
2015-02-02 15:11:32.193: {2:10686:2} Filling Environment Map with _ORA_CHECK_TIMEOUT = 600
2015-02-02 15:11:32.193: {2:10686:2} Filling Environment Map with _ORA_CHECK_TYPE = CLSAGFW_PERIODIC_CHECK
2015-02-02 15:11:32.193: {2:10686:2} RegistryAcfsAgent:: Calling check
2015-02-02 15:11:32.194: {2:10686:2} Filling Environment Map with ORACLE_HOME = /u01/grid/11.2.0/grid
2015-02-02 15:11:32.194: {2:10686:2} Filling Environment Map with ORACLE_SID = +ASM2
2015-02-02 15:11:32.194: {2:10686:2} Arg Value = check

ninipig 发表于 2015-2-2 16:07:40

2015-02-02 15:11:32.194: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:11:32.194: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:11:32.194: {2:10686:2} Utils::execCmd 1 USR_ORA_ENV: oracleHome:/u01/grid/11.2.0/grid CrsHome:/u01/grid/11.2.0/grid
2015-02-02 15:11:32.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:11:32.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:11:32.195: {2:10686:2} Adding Environment Variables ORACLE_HOME=/u01/grid/11.2.0/grid
2015-02-02 15:11:32.195: {2:10686:2} Adding Environment Variables ORACLE_SID=+ASM2
2015-02-02 15:11:32.195: {2:10686:2} Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2015-02-02 15:11:32.195: {2:10686:2} Adding Environment Variables _ORA_CHECK_TIMEOUT=600
2015-02-02 15:11:32.196: {2:10686:2} Adding Environment Variables _ORA_CHECK_TYPE=CLSAGFW_PERIODIC_CHECK
2015-02-02 15:11:32.196: {2:10686:2} Adding Environment Variables __IS_HASD_AGENT=
2015-02-02 15:11:32.196: {2:10686:2} Utils:execCmd action = 3 flags = 6 ohome = (null) cmdname = acfsregistrymount.
2015-02-02 15:11:32.196: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:11:32.321: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 47251702 Memory fault(coredump)
2015-02-02 15:11:32.322: {2:10686:2} (:CLSN00010:)
2015-02-02 15:11:32.322: {2:10686:2} (:CLSN00010:)Utils:execCmd scls_process_join() uret 255

2015-02-02 15:11:32.322: {2:10686:2} execCmd ret = 255
2015-02-02 15:11:32.322: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 47251702 Memory fault(coredump)
2015-02-02 15:11:32.323: {2:10686:2} (:CLSN00010:)
2015-02-02 15:11:32.323: {2:10686:2} RegistryAcfsAgent:: Check type: CLSAGFW_PERIODIC_CHECK
2015-02-02 15:11:32.323: {2:10686:2} RegistryAcfsAgent:: Check Return: 255
2015-02-02 15:11:32.323: [ AGFW] {2:10686:2} Received unknown resource status code: 255
2015-02-02 15:11:49.945: TM is changing desired thread # to 5. Current # is 6
2015-02-02 15:11:49.946: {0:2:24155} Worker thread is exiting in TM to meet the desired count of 5. New count is 5
2015-02-02 15:11:50.751: TM is changing desired thread # to 6. Current # is 5
2015-02-02 15:12:00.956: [ AGFW] {2:10686:2} Agent received the message: AGENT_HB ID 12293:3589947
2015-02-02 15:12:02.193: {2:10686:2} Filling Environment Map with _ORA_CHECK_TIMEOUT = 600
2015-02-02 15:12:02.194: {2:10686:2} Filling Environment Map with _ORA_CHECK_TYPE = CLSAGFW_PERIODIC_CHECK
2015-02-02 15:12:02.194: {2:10686:2} RegistryAcfsAgent:: Calling check
2015-02-02 15:12:02.194: {2:10686:2} Filling Environment Map with ORACLE_HOME = /u01/grid/11.2.0/grid
2015-02-02 15:12:02.194: {2:10686:2} Filling Environment Map with ORACLE_SID = +ASM2
2015-02-02 15:12:02.194: {2:10686:2} Arg Value = check
2015-02-02 15:12:02.194: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:12:02.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:12:02.195: {2:10686:2} Utils::execCmd 1 USR_ORA_ENV: oracleHome:/u01/grid/11.2.0/grid CrsHome:/u01/grid/11.2.0/grid
2015-02-02 15:12:02.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:12:02.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:12:02.195: {2:10686:2} Adding Environment Variables ORACLE_HOME=/u01/grid/11.2.0/grid
2015-02-02 15:12:02.195: {2:10686:2} Adding Environment Variables ORACLE_SID=+ASM2
2015-02-02 15:12:02.196: {2:10686:2} Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2015-02-02 15:12:02.196: {2:10686:2} Adding Environment Variables _ORA_CHECK_TIMEOUT=600
2015-02-02 15:12:02.196: {2:10686:2} Adding Environment Variables _ORA_CHECK_TYPE=CLSAGFW_PERIODIC_CHECK
2015-02-02 15:12:02.196: {2:10686:2} Adding Environment Variables __IS_HASD_AGENT=
2015-02-02 15:12:02.196: {2:10686:2} Utils:execCmd action = 3 flags = 6 ohome = (null) cmdname = acfsregistrymount.
2015-02-02 15:12:02.196: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:12:02.306: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 32964832 Memory fault(coredump)
2015-02-02 15:12:02.307: {2:10686:2} (:CLSN00010:)
2015-02-02 15:12:02.307: {2:10686:2} (:CLSN00010:)Utils:execCmd scls_process_join() uret 255

2015-02-02 15:12:02.307: {2:10686:2} execCmd ret = 255
2015-02-02 15:12:02.308: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 32964832 Memory fault(coredump)
2015-02-02 15:12:02.308: {2:10686:2} (:CLSN00010:)
2015-02-02 15:12:02.308: {2:10686:2} RegistryAcfsAgent:: Check type: CLSAGFW_PERIODIC_CHECK
2015-02-02 15:12:02.308: {2:10686:2} RegistryAcfsAgent:: Check Return: 255
2015-02-02 15:12:02.308: [ AGFW] {2:10686:2} Received unknown resource status code: 255
2015-02-02 15:12:19.959: TM is changing desired thread # to 5. Current # is 6
2015-02-02 15:12:19.960: TM is changing desired thread # to 7. Current # is 6
2015-02-02 15:12:32.193: {2:10686:2} Filling Environment Map with _ORA_CHECK_TIMEOUT = 600
2015-02-02 15:12:32.193: {2:10686:2} Filling Environment Map with _ORA_CHECK_TYPE = CLSAGFW_PERIODIC_CHECK
2015-02-02 15:12:32.194: {2:10686:2} RegistryAcfsAgent:: Calling check
2015-02-02 15:12:32.194: {2:10686:2} Filling Environment Map with ORACLE_HOME = /u01/grid/11.2.0/grid
2015-02-02 15:12:32.194: {2:10686:2} Filling Environment Map with ORACLE_SID = +ASM2
2015-02-02 15:12:32.194: {2:10686:2} Arg Value = check
2015-02-02 15:12:32.194: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:12:32.194: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:12:32.195: {2:10686:2} Utils::execCmd 1 USR_ORA_ENV: oracleHome:/u01/grid/11.2.0/grid CrsHome:/u01/grid/11.2.0/grid
2015-02-02 15:12:32.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:12:32.195: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:12:32.195: {2:10686:2} Adding Environment Variables ORACLE_HOME=/u01/grid/11.2.0/grid
2015-02-02 15:12:32.195: {2:10686:2} Adding Environment Variables ORACLE_SID=+ASM2
2015-02-02 15:12:32.195: {2:10686:2} Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2015-02-02 15:12:32.196: {2:10686:2} Adding Environment Variables _ORA_CHECK_TIMEOUT=600
2015-02-02 15:12:32.196: {2:10686:2} Adding Environment Variables _ORA_CHECK_TYPE=CLSAGFW_PERIODIC_CHECK
2015-02-02 15:12:32.196: {2:10686:2} Adding Environment Variables __IS_HASD_AGENT=
2015-02-02 15:12:32.196: {2:10686:2} Utils:execCmd action = 3 flags = 6 ohome = (null) cmdname = acfsregistrymount.
2015-02-02 15:12:32.196: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:12:32.322: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 31654110 Memory fault(coredump)
2015-02-02 15:12:32.323: {2:10686:2} (:CLSN00010:)
2015-02-02 15:12:32.323: {2:10686:2} (:CLSN00010:)Utils:execCmd scls_process_join() uret 255

ninipig 发表于 2015-2-2 16:08:05

2015-02-02 15:12:32.324: {2:10686:2} execCmd ret = 255
2015-02-02 15:12:32.324: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 31654110 Memory fault(coredump)
2015-02-02 15:12:32.324: {2:10686:2} (:CLSN00010:)
2015-02-02 15:12:32.324: {2:10686:2} RegistryAcfsAgent:: Check type: CLSAGFW_PERIODIC_CHECK
2015-02-02 15:12:32.324: {2:10686:2} RegistryAcfsAgent:: Check Return: 255
2015-02-02 15:12:32.324: [ AGFW] {2:10686:2} Received unknown resource status code: 255
2015-02-02 15:12:50.749: TM is changing desired thread # to 6. Current # is 7
2015-02-02 15:12:50.750: {0:2:24159} Worker thread is exiting in TM to meet the desired count of 6. New count is 6
2015-02-02 15:13:00.959: [ AGFW] {2:10686:2} Agent received the message: AGENT_HB ID 12293:3589957
2015-02-02 15:13:02.194: {2:10686:2} Filling Environment Map with _ORA_CHECK_TIMEOUT = 600
2015-02-02 15:13:02.194: {2:10686:2} Filling Environment Map with _ORA_CHECK_TYPE = CLSAGFW_PERIODIC_CHECK
2015-02-02 15:13:02.194: {2:10686:2} RegistryAcfsAgent:: Calling check
2015-02-02 15:13:02.194: {2:10686:2} Filling Environment Map with ORACLE_HOME = /u01/grid/11.2.0/grid
2015-02-02 15:13:02.195: {2:10686:2} Filling Environment Map with ORACLE_SID = +ASM2
2015-02-02 15:13:02.195: {2:10686:2} Arg Value = check
2015-02-02 15:13:02.196: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:13:02.196: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:13:02.196: {2:10686:2} Utils::execCmd 1 USR_ORA_ENV: oracleHome:/u01/grid/11.2.0/grid CrsHome:/u01/grid/11.2.0/grid
2015-02-02 15:13:02.196: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:13:02.196: {2:10686:2} Utils::getCrsHome crsHome /u01/grid/11.2.0/grid
2015-02-02 15:13:02.197: {2:10686:2} Adding Environment Variables ORACLE_HOME=/u01/grid/11.2.0/grid
2015-02-02 15:13:02.197: {2:10686:2} Adding Environment Variables ORACLE_SID=+ASM2
2015-02-02 15:13:02.197: {2:10686:2} Adding Environment Variables _ORA_AGENT_ACTION=TRUE
2015-02-02 15:13:02.197: {2:10686:2} Adding Environment Variables _ORA_CHECK_TIMEOUT=600
2015-02-02 15:13:02.197: {2:10686:2} Adding Environment Variables _ORA_CHECK_TYPE=CLSAGFW_PERIODIC_CHECK
2015-02-02 15:13:02.197: {2:10686:2} Adding Environment Variables __IS_HASD_AGENT=
2015-02-02 15:13:02.197: {2:10686:2} Utils:execCmd action = 3 flags = 6 ohome = (null) cmdname = acfsregistrymount.
2015-02-02 15:13:02.198: {2:10686:2} getOracleHomeAttrib: oracle_home = /u01/grid/11.2.0/grid
2015-02-02 15:13:02.320: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 32964690 Memory fault(coredump)
2015-02-02 15:13:02.320: {2:10686:2} (:CLSN00010:)
2015-02-02 15:13:02.320: {2:10686:2} (:CLSN00010:)Utils:execCmd scls_process_join() uret 255

2015-02-02 15:13:02.321: {2:10686:2} execCmd ret = 255
2015-02-02 15:13:02.321: {2:10686:2} (:CLSN00010:)/u01/grid/11.2.0/grid/lib/acfstoolsdriver.sh: 32964690 Memory fault(coredump)
2015-02-02 15:13:02.321: {2:10686:2} (:CLSN00010:)
2015-02-02 15:13:02.321: {2:10686:2} RegistryAcfsAgent:: Check type: CLSAGFW_PERIODIC_CHECK
2015-02-02 15:13:02.321: {2:10686:2} RegistryAcfsAgent:: Check Return: 255
2015-02-02 15:13:02.322: [ AGFW] {2:10686:2} Received unknown resource status code: 255
页: [1]
查看完整版本: AIX 6.1 RAC 11.2.0.3.6 节点2的ACFS故障