oracle 服务器掉电 重启后观察日志情况
大家好
今天机房突然断电了,启动之后oralce警告日志有好多包报错
oracle版本是 11.2.3 安装在IBM p720 操作系统是 AIX 6100-06-08-1216
以下是报错信息,请大家帮忙分析下
SQL> select group#,sequence#,status from v$log;
GROUP# SEQUENCE# STATUS
---------- ---------- ----------------
1 2197 CURRENT
2 2194 INACTIVE
3 189 INACTIVE
4 190 INACTIVE
5 2195 INACTIVE
6 2196 INACTIVE
7 191 INACTIVE
8 192 INACTIVE
8 rows selected.
$ cat alert_CWEOL1.log |grep ORA
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE from environment = /oracle/product/ora11gr2
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE from environment = /oracle/product/ora11gr2
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE from environment = /oracle/product/ora11gr2
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00313: open failed for members of log group 5 of thread 1
ORA-00313: open failed for members of log group 5 of thread 1
ORA-00313: open failed for members of log group 5 of thread 1
ORA-00313: open failed for members of log group 6 of thread 1
ORA-00313: open failed for members of log group 6 of thread 1
ORA-00313: open failed for members of log group 6 of thread 1
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE from environment = /oracle/product/ora11gr2
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
ORA-12012: error on auto execute of job "ORACLE_OCM"."MGMT_CONFIG_JOB_2_1"
ORA-29280: invalid directory path
ORA-06512: at "ORACLE_OCM.MGMT_DB_LL_METRICS", line 2436
ORA-06512: at line 1
ORA-12012: error on auto execute of job "ORACLE_OCM"."MGMT_CONFIG_JOB_2_1"
ORA-29280: invalid directory path
ORA-06512: at "ORACLE_OCM.MGMT_DB_LL_METRICS", line 2436
ORA-06512: at line 1
ORA-12012: error on auto execute of job "ORACLE_OCM"."MGMT_CONFIG_JOB_2_1"
ORA-29280: invalid directory path
ORA-06512: at "ORACLE_OCM.MGMT_DB_LL_METRICS", line 2436
ORA-06512: at line 1
ORA-12012: error on auto execute of job "ORACLE_OCM"."MGMT_CONFIG_JOB_2_1"
ORA-29280: invalid directory path
ORA-06512: at "ORACLE_OCM.MGMT_DB_LL_METRICS", line 2436
ORA-06512: at line 1
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
ORA-12012: error on auto execute of job "ORACLE_OCM"."MGMT_CONFIG_JOB_2_1"
ORA-29280: invalid directory path
ORA-06512: at "ORACLE_OCM.MGMT_DB_LL_METRICS", line 2436
ORA-06512: at line 1
ORA-01555 caused by SQL statement below (SQL ID: 7dg78f8rg7xgk, Query Duration=0 sec, SCN: 0x0000.0325d6f7):
WARNING: inbound connection timed out (ORA-3136)
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE from environment = /oracle/product/ora11gr2
ORA-12012: error on auto execute of job "ORACLE_OCM"."MGMT_CONFIG_JOB_2_1"
ORA-29280: invalid directory path
ORA-06512: at "ORACLE_OCM.MGMT_DB_LL_METRICS", line 2436
ORA-06512: at line 1
WARNING: inbound connection timed out (ORA-3136)
ORA-12012: error on auto execute of job "ORACLE_OCM"."MGMT_CONFIG_JOB_2_1"
ORA-29280: invalid directory path
ORA-06512: at "ORACLE_OCM.MGMT_DB_LL_METRICS", line 2436
ORA-06512: at line 1
ORA-27072: File I/O error
ORA-00202: control file: '+DATA/cweol/controlfile/current.260.797570831'
ORA-15081: failed to submit an I/O operation to a disk
ORA-27063: number of bytes read/written is incorrect
ORA-27063: number of bytes read/written is incorrect
ORA-15080: synchronous I/O operation to a disk failed
ORA-15080: synchronous I/O operation to a disk failed
ORA-00345: redo log write error block 93936 count 63
ORA-00312: online log 6 thread 1: '+DATA/cweol/onlinelog/group_6.264.797570853'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-00345: redo log write error block 93936 count 63
ORA-00312: online log 6 thread 1: '+RECO/cweol/onlinelog/group_6.260.797570855'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-00340: IO error processing online log 6 of thread 1
ORA-00345: redo log write error block 93936 count 63
ORA-00312: online log 6 thread 1: '+DATA/cweol/onlinelog/group_6.264.797570853'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-00345: redo log write error block 93936 count 63
ORA-00312: online log 6 thread 1: '+RECO/cweol/onlinelog/group_6.260.797570855'
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-1092 : opitsk aborting process
ORA-1092 : opitsk aborting process
Starting ORACLE instance (normal)
ORACLE_HOME = /oracle/product/ora11gr2/db_1
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
Reusing ORACLE_BASE from an earlier startup = /oracle/product/ora11gr2
$ cat alert_CWEOL1.log |grep fail
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00313: open failed for members of log group 1 of thread 1
ORA-00313: open failed for members of log group 1 of thread 1
Checker run found 1 new persistent data failures
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00313: open failed for members of log group 2 of thread 1
ORA-00313: open failed for members of log group 5 of thread 1
ORA-00313: open failed for members of log group 5 of thread 1
ORA-00313: open failed for members of log group 5 of thread 1
ORA-00313: open failed for members of log group 6 of thread 1
ORA-00313: open failed for members of log group 6 of thread 1
ORA-00313: open failed for members of log group 6 of thread 1
WARNING: failed to read mirror side 1 of virtual extent 4 logical extent 0 of file 260 in group from disk DATA_0002 allocation unit 227 reason error; if possible, will try another mirror side
ORA-15081: failed to submit an I/O operation to a disk
ORA-15080: synchronous I/O operation to a disk failed
WARNING: failed to write mirror side 1 of virtual extent 45 logical extent 0 of file 264 in group 1 on disk 1 allocation unit 358
ORA-15080: synchronous I/O operation to a disk failed
WARNING: failed to write mirror side 1 of virtual extent 45 logical extent 0 of file 260 in group 3 on disk 1 allocation unit 467
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
ORA-15081: failed to submit an I/O operation to a disk
Instance termination failed to kill one or more processes
本帖最后由 xindahai 于 2012-12-7 18:01 编辑
补充 : 这是RAC环境,另外一个节点数据库没有启动。 连接的存储是IBM DS5020.
下面是系统的情况
SQL> select * from v$logfile;
GROUP# STATUS TYPE MEMBER IS_
---------- ------- ------- ------------------------------ ---
6 ONLINE +DATA/cweol/onlinelog/group_6. NO
264.797570853
6 ONLINE +RECO/cweol/onlinelog/group_6. YES
260.797570855
5 ONLINE +DATA/cweol/onlinelog/group_5. NO
263.797570845
5 ONLINE +RECO/cweol/onlinelog/group_5. YES
259.797570849
GROUP# STATUS TYPE MEMBER IS_
---------- ------- ------- ------------------------------ ---
2 ONLINE +DATA/cweol/onlinelog/group_2. NO
262.797570841
2 ONLINE +RECO/cweol/onlinelog/group_2. YES
258.797570843
1 ONLINE +DATA/cweol/onlinelog/group_1. NO
261.797570835
1 ONLINE +RECO/cweol/onlinelog/group_1. YES
GROUP# STATUS TYPE MEMBER IS_
---------- ------- ------- ------------------------------ ---
257.797570837
3 ONLINE +DATA/cweol/onlinelog/group_3. NO
267.797571139
3 ONLINE +RECO/cweol/onlinelog/group_3. YES
261.797571143
4 ONLINE +DATA/cweol/onlinelog/group_4. NO
268.797571147
GROUP# STATUS TYPE MEMBER IS_
---------- ------- ------- ------------------------------ ---
4 ONLINE +RECO/cweol/onlinelog/group_4. YES
262.797571149
7 ONLINE +DATA/cweol/onlinelog/group_7. NO
269.797571151
7 ONLINE +RECO/cweol/onlinelog/group_7. YES
263.797571153
8 ONLINE +DATA/cweol/onlinelog/group_8. NO
270.797571157
GROUP# STATUS TYPE MEMBER IS_
---------- ------- ------- ------------------------------ ---
8 ONLINE +RECO/cweol/onlinelog/group_8. YES
264.797571161
16 rows selected.
# fcstat fcs0
FIBRE CHANNEL STATISTICS REPORT: fcs0
Device Type: 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) (adapter/pciex/df1000f114108a0)
Serial Number: 1B80313ED5
Option ROM Version: 02781135
ZA: U2D1.10X5
World Wide Node Name: 0x20000000C9A25E70
World Wide Port Name: 0x10000000C9A25E70
FC-4 TYPES:
Supported: 0x0000012000000000000000000000000000000000000000000000000000000000
Active: 0x0000010000000000000000000000000000000000000000000000000000000000
Class of Service: 3
Port Speed (supported): 8 GBIT
Port Speed (running): 8 GBIT
Port FC ID: 0x010300
Port Type: Fabric
Seconds Since Last Reset: 31289
Transmit Statistics Receive Statistics
------------------- ------------------
Frames: 13281208 16309158
Words: 4571927296 5622437376
LIP Count: 0
NOS Count: 0
Error Frames: 1
Dumped Frames: 0
Link Failure Count: 0
Loss of Sync Count: 5
Loss of Signal: 0
Primitive Seq Protocol Error Count: 0
Invalid Tx Word Count: 8
Invalid CRC Count: 1
IP over FC Adapter Driver Information
No DMA Resource Count: 0
No Adapter Elements Count: 62
FC SCSI Adapter Driver Information
No DMA Resource Count: 0
No Adapter Elements Count: 62
No Command Resource Count: 0
IP over FC Traffic Statistics
Input Requests: 0
Output Requests: 0
Control Requests: 0
Input Bytes: 0
Output Bytes: 0
FC SCSI Traffic Statistics
Input Requests: 2496484
Output Requests: 1519369
Control Requests: 2353
Input Bytes: 21986818303
Output Bytes: 17840344576
页:
[1]