Oracle数据库数据恢复、性能优化

找回密码
注册
搜索
热搜: 活动 交友 discuz
Oracle数据库数据恢复、性能优化»论坛 › Oracle › Oracle数据库管理 › ORA-00600 错误
发新帖
lesliewang

8

积分

0

好友

0

主题

ORA-00600 错误

1#
发表于 2012-2-27 20:14:53 | 查看: 10793| 回复: 11
ORACLE 10.2.0.4 RAC 出现一个节点频繁重启问题，alert.log报
Mon Feb 27 19:38:55 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_8995.trc:
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:38:56 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_8995.trc:
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:38:56 2012
Trace dumping is performing id=[cdmp_20120227193856]
Mon Feb 27 19:38:56 2012
SUCCESS: diskgroup ORA_ARCHIVELOG was mounted
SUCCESS: diskgroup ORA_ARCHIVELOG was dismounted
SUCCESS: diskgroup ORA_ARCHIVELOG was mounted
Mon Feb 27 19:38:57 2012
Thread 2 advanced to log sequence 171708 (archiver wakeup)
Mon Feb 27 19:38:57 2012
Database Characterset is ZHS16GBK
Mon Feb 27 19:38:57 2012
SUCCESS: diskgroup ORA_ARCHIVELOG was dismounted
Mon Feb 27 19:38:57 2012
Opening with Resource Manager plan: INTERNAL_PLAN
where NUMA PG = 2, CPUs = 8
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=142, OS id=9107
Mon Feb 27 19:38:58 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_8995.trc:
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:38:58 2012
Completed: ALTER DATABASE OPEN
Mon Feb 27 19:38:59 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_8995.trc:
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:39:02 2012
Doing block recovery for file 64 block 925405
Block recovery from logseq 211705, block 333 to scn 71136015173
Mon Feb 27 19:39:02 2012
Recovery of Online Redo Log: Thread 1 Group 10 Seq 211705 Reading mem 0
  Mem# 0: +ORA_DG_1/sctestdb/onlinelog/group_10.273.729222079
Block recovery completed at rba 211705.352.16, scn 16.2416538438
Mon Feb 27 19:39:02 2012
SMON: slave died unexpectedly, downgrading to serial recovery
Mon Feb 27 19:39:02 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_8881.trc:
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Mon Feb 27 19:39:04 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_8881.trc:
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Mon Feb 27 19:39:04 2012
Trace dumping is performing id=[cdmp_20120227193904]
Mon Feb 27 19:39:05 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_8881.trc:
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00607: Message 607 not found; No message file for product=RDBMS, facility=ORA


手动重启CRS及服务器后，数据库两个节点都无法启动，日志：
Starting up ORACLE RDBMS Version: 10.2.0.4.0.
System parameters with non-default values:
  processes                = 2000
  sga_max_size             = 10737418240
   

SUCCESS: diskgroup ORA_DATA_01 was mounted
ARC1: Archival started
LGWR: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=102, OS id=11509
Mon Feb 27 19:42:26 2012
Thread 1 advanced to log sequence 211706 (thread open)
Thread 1 opened at log sequence 211706
  Current log# 2 seq# 211706 mem# 0: +ORA_DG_1/sctestdb/onlinelog/group_2.267.729214243
Successful open of redo thread 1
Mon Feb 27 19:42:26 2012
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
Mon Feb 27 19:42:26 2012
ARC0: Becoming the 'no FAL' ARCH
ARC0: Becoming the 'no SRL' ARCH
Mon Feb 27 19:42:26 2012
SMON: enabling cache recovery
Mon Feb 27 19:42:26 2012
ARC1: Becoming the heartbeat ARCH
Mon Feb 27 19:42:26 2012
Successfully onlined Undo Tablespace 1.
Mon Feb 27 19:42:26 2012
SMON: enabling tx recovery
Mon Feb 27 19:42:27 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_11472.trc:
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:42:29 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_11472.trc:
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:42:29 2012
Trace dumping is performing id=[cdmp_20120227194229]
Mon Feb 27 19:42:29 2012
SUCCESS: diskgroup ORA_ARCHIVELOG was mounted
SUCCESS: diskgroup ORA_ARCHIVELOG was dismounted
SUCCESS: diskgroup ORA_ARCHIVELOG was mounted
Mon Feb 27 19:42:29 2012
Database Characterset is ZHS16GBK
Mon Feb 27 19:42:29 2012
SUCCESS: diskgroup ORA_ARCHIVELOG was dismounted
Mon Feb 27 19:42:29 2012
Opening with Resource Manager plan: INTERNAL_PLAN
where NUMA PG = 2, CPUs = 8
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=144, OS id=11620
Mon Feb 27 19:42:30 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_11472.trc:
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:42:30 2012
Completed: alter database open
Mon Feb 27 19:42:31 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_p002_11472.trc:
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00607: Internal error occurred while making a change to a data block
ORA-00602: internal programming exception
ORA-07445: exception encountered: core dump [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8] [] []
ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
Mon Feb 27 19:42:34 2012
Doing block recovery for file 64 block 925405
Block recovery from logseq 211706, block 332 to scn 71136035741
Mon Feb 27 19:42:34 2012
Recovery of Online Redo Log: Thread 1 Group 2 Seq 211706 Reading mem 0
  Mem# 0: +ORA_DG_1/sctestdb/onlinelog/group_2.267.729214243
Block recovery completed at rba 211706.351.16, scn 16.2416559006
Mon Feb 27 19:42:34 2012
SMON: slave died unexpectedly, downgrading to serial recovery
Mon Feb 27 19:42:34 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_9989.trc:
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Mon Feb 27 19:42:35 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_9989.trc:
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Mon Feb 27 19:42:35 2012
Trace dumping is performing id=[cdmp_20120227194235]
Mon Feb 27 19:42:37 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_9989.trc:
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
2#
发表于 2012-2-27 20:29:16
以下mos中的

ORA-600[kghstack_free2] Raised by Corrupt Datafile [ID 748310.1]
     修改时间 11-DEC-2008     类型 PROBLEM     状态 MODERATED     
In this Document
  Symptoms
  Cause
  Solution
  References

This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process, and therefore has not been subject to an independent technical review.
Applies to:

Oracle Server - Enterprise Edition - Version: 10.2.0.3
This problem can occur on any platform.
Symptoms

The following errors are seen in the alert log:
     ORA-00600: internal error code, arguments: [kghstack_free2], [], [], [], [], [], [], []
     ORA-07445: exception encountered: core dump [] [] [] [] [] []

The call stack generated by the ora-600[kghstack_free2] can look like this:
     kghstack_free ksmfrs kdxcom kdxlcnu kco_issue_callback ...

This error may result in instance termination.


Cause

This issue can be caused by a corrupt datafile(s) which may be confirmed by running DBVerify.  
See Note 35512.1 for assistance with DBVerify.

Solution

Take corrective actions against the corrupt datafile(s) (e.g. recovery).



References

NOTE:35512.1 - DBVERIFY - Database file Verification Utility (7.3.2 - 10.2)

[ 本帖最后由 popowin 于 2012-2-27 20:30 编辑 ]

回复 只看该作者 道具 举报

3#
发表于 2012-2-27 20:33:25
请你  压缩后 上传 /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_9989.trc 这个 trace文件

回复 只看该作者 道具 举报

4#
发表于 2012-2-27 20:38:40

sctestdb1_smon_9989.trc

/oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_9989.trc  文件

sctestdb1_smon_9989.rar

318.47 KB, 下载次数: 1048

回复 只看该作者 道具 举报

5#
发表于 2012-2-27 21:32:51
ODM Finding:


Version :10.2.0.4           platform : HP-UX
  1. Parallel Transaction recovery caught exception 12805
  2. *** 2012-02-27 19:42:34.357
  3. SMON: slave died unexpectedly, downgrading to serial recovery
  4. Parallel Transaction recovery caught error 30311
  5. Corrupted stack chunk header at location 9ffffffffd0434f0


  8. 9FFFFFFFFD0434F0 B3C9B6BC 2C004A02 C107FF02 C132FFFF  [....,.J......2..]

  10. stack call

  12. kgesin()+112         call     kgesinv()            6000000000031340 ?
  13.                                                    9FFFFFFFFD1925C0 ?
  14.                                                    9FFFFFFFFD1925D0 ?
  15.                                                    6000000000032770 ?
  16.                                                    9FFFFFFFFFFF5F80 ?
  17. $cold_kghstack_free  call     kgesin()             6000000000031340 ?
  18. ()+416                                             9FFFFFFFFD1925C0 ?
  19.                                                    4000000001589180 ?
  20.                                                    000000000 ?
  21.                                                    C000000000000B1D ?
  22.                                                    4000000003540E70 ?
  23.                                                    6000000000031340 ?
  24. ksmfrs()+48          call     $cold_kghstack_free  6000000000031340 ?
  25.                               ()                   000001000 ?
  26. kdbcpss()+2768       call     ksmfrs()             9FFFFFFFFD043510 ?
  27.                                                    C000000000001938 ?
  28.                                                    4000000002B7EAD0 ?
  29.                                                    60000000000B5D60 ?
  30.                                                    C000000000000491 ?
  31.                                                    4000000002FDBBC0 ?
  32. kdourp2()+4416       call     kdbcpss()            C0000003FDF42014 ?
  33.                                                    000000000 ? 000000011 ?
  34.                                                    000001FE8 ?
  35.                                                    60000000000B5D60 ?
  36.                                                    C000000000009C40 ?
  37.                                                    4000000003087F00 ?
  38.                                                    00001011D ?
  39. kdourp()+208         call     kdourp2()            9FFFFFFFFD045638 ?
  40.                                                    9FFFFFFFFD04768A ?
  41.                                                    000000005 ?
  42.                                                    C0000003FDF42014 ?
  43.                                                    000000003 ?
  44.                                                    9FFFFFFFFFFF7E70 ?
  45. kco_issue_callback(  call     kdourp()             9FFFFFFFFD045638 ?
  46. )+368                                              9FFFFFFFFD04768A ?
  47.                                                    000000005 ?
复制代码

回复 只看该作者 道具 举报

6#
发表于 2012-2-27 21:37:09
尝试找出 坏块，使用rman 执行以下脚本，并贴出输出：
  1. 先 startup mount ;



  5. rman target /

  7. run {
  8. allocate channel d1 type disk;
  9. allocate channel d2 type disk;
  10. allocate channel d3 type disk;
  11. allocate channel d4 type disk;
  12. backup validate check logical database;
  13. }
复制代码

回复 只看该作者 道具 举报

7#
发表于 2012-2-27 22:09:34
===============alter log============
Mon Feb 27 21:51:15 2012
SUCCESS: diskgroup ORA_DG_11X was dismounted
Mon Feb 27 21:51:26 2012
Error backing up file 12, block 442563: logical corruption
Mon Feb 27 21:54:26 2012
SUCCESS: diskgroup ORA_DG_25 was mounted
Mon Feb 27 21:54:31 2012
SUCCESS: diskgroup ORA_DG_29 was dismounted

Mon Feb 27 21:54:57 2012
SUCCESS: diskgroup ORA_DG_20 was dismounted
Mon Feb 27 21:56:46 2012
Error backing up file 64, block 925405: logical corruption

Mon Feb 27 21:58:00 2012

回复 只看该作者 道具 举报

8#
发表于 2012-2-27 22:13:40
SQL> select * from v$database_block_corruption;

查一下这个视图

回复 只看该作者 道具 举报

9#
发表于 2012-2-27 22:17:42
SQL> desc v$database_block_corruption;
  Name                                     Null?    Type
----------------------------------------- -------- ----------------------------
FILE#                                              NUMBER
BLOCK#                                             NUMBER
BLOCKS                                             NUMBER
CORRUPTION_CHANGE#                                 NUMBER
CORRUPTION_TYPE                                    VARCHAR2(9)
SQL>
SQL> select * from v$database_block_corruption;
     FILE#     BLOCK#     BLOCKS CORRUPTION_CHANGE# CORRUPTIO
---------- ---------- ---------- ------------------ ---------
        12     442563          1         7.1136E+10 LOGICAL
        64     925405          1         7.1136E+10 LOGICAL
SQL>

回复 只看该作者 道具 举报

10#
发表于 2012-2-27 22:22:55
FILE 12  BLOCK 442563         
FILE 64 BLOCK  925405  存在 logical corruption 逻辑讹误，


1. 尝试用物理备份恢复这2个数据块
2. 如果没有备份 或者是由bug 引起的logical corruption的话，可以考虑禁用smon transaction recovery 来绕过 该问题：

设置 10531 event:
  1. shutdown immediate;

  3. startup nomount;

  5. alter system set fast_start_parallel_rollback=false;
  6. alter system set events '10513 trace name context forever, level 2';

  8. alter database mount;
  9. alter database open;
复制代码

回复 只看该作者 道具 举报

11#
发表于 2012-2-27 22:31:24
SUCCESS: diskgroup ORA_ARCHIVELOG was mounted
SUCCESS: diskgroup ORA_ARCHIVELOG was dismounted
SUCCESS: diskgroup ORA_ARCHIVELOG was mounted
SUCCESS: diskgroup ORA_ARCHIVELOG was dismounted
Mon Feb 27 22:27:56 2012
Successfully onlined Undo Tablespace 1.
Mon Feb 27 22:27:56 2012
SMON: enabling tx recovery
Mon Feb 27 22:27:56 2012
Database Characterset is ZHS16GBK
Mon Feb 27 22:27:56 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_35967.trc:
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Opening with Resource Manager plan: INTERNAL_PLAN
where NUMA PG = 2, CPUs = 8
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
QMNC started with pid=108, OS id=36259
Mon Feb 27 22:27:58 2012
Trace dumping is performing id=[cdmp_20120227222758]
Mon Feb 27 22:27:58 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_35967.trc:
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Mon Feb 27 22:28:00 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_35967.trc:
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00607: Message 607 not found; No message file for product=RDBMS, facility=ORA
ORA-00602: Message 602 not found; No message file for product=RDBMS, facility=ORA
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Mon Feb 27 22:28:01 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_smon_35967.trc:
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00607: Message 607 not found; No message file for product=RDBMS, facility=ORA
ORA-00602: Message 602 not found; No message file for product=RDBMS, facility=ORA
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00607: Message 607 not found; No message file for product=RDBMS, facility=ORA
ORA-00602: Message 602 not found; No message file for product=RDBMS, facility=ORA
ORA-07445: Message 7445 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_err()+288] [SIGSEGV] [Address not mapped to object] [0xFF08C9CCD2B5D6D8]
ORA-00600: Message 600 not found; No message file for product=RDBMS, facility=ORA; arguments: [kghstack_free2]
Mon Feb 27 22:28:01 2012
Completed: alter database open
Mon Feb 27 22:28:04 2012
Errors in file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_pmon_35869.trc:
ORA-00474: Message 474 not found; No message file for product=RDBMS, facility=ORA
Mon Feb 27 22:28:04 2012
PMON: terminating instance due to error 474
Mon Feb 27 22:28:05 2012
System state dump is made for local instance
System State dumped to trace file /oracle/app/oracle/admin/sctestdb/bdump/sctestdb1_diag_35916.trc
Mon Feb 27 22:28:05 2012
Trace dumping is performing id=[cdmp_20120227222805]
Mon Feb 27 22:28:08 2012
Instance terminated by PMON, pid = 35869
Mon Feb 27 22:29:04 2012
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0

回复 只看该作者 道具 举报

12#
发表于 2012-2-27 22:52:41
正在恢复数据文件，有结果后再回复，谢谢谢谢！！！

回复 只看该作者 道具 举报

返回列表
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 注册

QQ|手机版|Archiver|Oracle数据库数据恢复、性能优化

GMT+8, 2024-12-23 23:59 , Processed in 0.057039 second(s), 25 queries .

Powered by Discuz! X2.5

© 2001-2012 Comsenz Inc.

回顶部
TEL/電話+86 13764045638
Email service@parnassusdata.com
QQ 47079569