gzsujj

0 积分	0 好友	3 主题

发消息

[SQL调优] node 2 instance crash

1^#

发表于 2014-8-14 23:27:46 | 查看: 3353| 回复: 5

8.13日下午16:45，node 2 实例crash，awr中有checkpoint not complete等待事件
数据库版本:11.
以下为各数据库日志情况:
oracle alert:
Wed Aug 13 16:00:43 2014
Thread 2 cannot allocate new log, sequence 50796
Checkpoint not complete
  Current log# 14 seq# 50795 mem# 0: +REDODG/racdb/onlinelog/group_14
Thread 2 advanced to log sequence 50796 (LGWR switch)
  Current log# 15 seq# 50796 mem# 0: +REDODG/racdb/onlinelog/group_15
Wed Aug 13 16:01:18 2014
Thread 2 cannot allocate new log, sequence 50797
Checkpoint not complete
  Current log# 15 seq# 50796 mem# 0: +REDODG/racdb/onlinelog/group_15
Thread 2 advanced to log sequence 50797 (LGWR switch)
  Current log# 11 seq# 50797 mem# 0: +REDODG/racdb/onlinelog/group_11
Wed Aug 13 16:01:54 2014
Thread 2 cannot allocate new log, sequence 50798
Checkpoint not complete
  Current log# 11 seq# 50797 mem# 0: +REDODG/racdb/onlinelog/group_11
Thread 2 advanced to log sequence 50798 (LGWR switch)
  Current log# 12 seq# 50798 mem# 0: +REDODG/racdb/onlinelog/group_12
Wed Aug 13 16:02:26 2014
Thread 2 cannot allocate new log, sequence 50799
Checkpoint not complete
  Current log# 12 seq# 50798 mem# 0: +REDODG/racdb/onlinelog/group_12
Thread 2 advanced to log sequence 50799 (LGWR switch)
  Current log# 13 seq# 50799 mem# 0: +REDODG/racdb/onlinelog/group_13

Wed Aug 13 16:34:50 2014
kkjcre1p: unable to spawn jobq slave process
Errors in file /opt/oracle/diag/rdbms/racdb/racdb2/trace/racdb2_cjq0_5508.trc:

Wed Aug 13 16:44:47 2014
ORA-609 : opiodr aborting process unknown ospid (7910_1)

Errors in file /opt/oracle/diag/rdbms/racdb/racdb2/trace/racdb2_smon_3581.trc:
ORA-00482: LMD* process terminated with error

Wed Aug 13 16:44:57 2014
ORA-1092 : opiodr aborting process unknown ospid (17233_1)
Wed Aug 13 16:44:58 2014
ORA-1092 : opitsk aborting process
Wed Aug 13 16:44:58 2014
ORA-1092 : opiodr aborting process unknown ospid (5380_1)
Errors in file /opt/oracle/diag/rdbms/racdb/racdb2/trace/racdb2_ckpt_3579.trc:

Wed Aug 13 16:45:07 2014
License high water mark = 1055
System State dumped to trace file /opt/oracle/diag/rdbms/racdb/racdb2/trace/racdb2_diag_3477.trc
Wed Aug 13 16:45:13 2014
Instance terminated by LMD0, pid = 3499
USER (ospid: 8266): terminating the instance
Instance terminated by USER, pid = 8266

Wed Aug 13 16:45:21 2014
Starting ORACLE instance (normal)

crs alert：
2014-08-13 16:44:44.321
[cssd(2012)]CRS-1612:node db1 (1) at 50% heartbeat fatal, eviction in 14.712 seconds
2014-08-13 16:44:45.321
[cssd(2012)]CRS-1612:node db1 (1) at 50% heartbeat fatal, eviction in 13.712 seconds

crsd.log
2014-08-13 16:44:58.990: [  CRSRES][955883] In stateChanged, ora.racdb.racdb2.inst target is ONLINE
2014-08-13 16:44:58.997: [  CRSRES][955883] ora.racdb.racdb2.inst on db2 went OFFLINE unexpectedly
2014-08-13 16:44:59.018: [  CRSRES][955883] StopResource: setting CLI values
2014-08-13 16:44:59.378: [  CRSRES][955883] Attempting to stop `ora.racdb.racdb2.inst` on member `db2`
2014-08-13 16:45:19.835: [  CRSRES][955883] Stop of `ora.racdb.racdb2.inst` on member `db2` succeeded.
2014-08-13 16:45:19.836: [  CRSRES][955883] ora.racdb.racdb2.inst RESTART_COUNT=0 RESTART_ATTEMPTS=5
2014-08-13 16:45:19.836: [  CRSRES][955883] Restarting ora.racdb.racdb2.inst on db2
2014-08-13 16:45:19.842: [  CRSRES][955883] startRunnable: setting CLI values
2014-08-13 16:45:19.843: [  CRSRES][955883] Attempting to start `ora.racdb.racdb2.inst` on member `db2`
2014-08-13 16:48:44.297: [  CRSRES][955883] Start of `ora.racdb.racdb2.inst` on member `db2` succeeded.
2014-08-13 16:48:44.297: [  CRSRES][955883] Successfully restarted ora.racdb.racdb2.inst on db2, RESTART_COUNT=1
2014-08-13 16:48:44.316: [  CRSRES][955883] ora.racdb.racdb2.inst Updated LAST_RESTART time in ocr

分享0