clot09 发表于 2013-12-27 12:56:29

ORA-07445 core dump [0000F4F4] [SIGSEGV]错误,导致ogg连节oracle中断

hi 各位,
今天早上ogg报连不上oracle,然后中断了;
数据库报了个ORA-07445: exception encountered: core dump [] [];
基本信息如下:$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on Fri Dec 27 12:41:18 2013

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi
PL/SQL Release 10.2.0.4.0 - Production
CORE    10.2.0.4.0      Production
TNS for IBM/AIX RISC System/6000: Version 10.2.0.4.0 - Productio
NLSRTL Version 10.2.0.4.0 - Production



$ ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.1.1.0.28 Build 001
AIX 5L, ppc, 64bit (optimized), Oracle 10.2 on Mar 12 2011 19:21:59

Copyright (C) 1995, 2010, Oracle and/or its affiliates. All rights reserved.
ogg日志报错如下:2013-12-27 09:00:12  WARNING OGG-00869  Oracle GoldenGate Delivery for Oracle, reperp.prm:  OCI Error ORA-03113: end-of-file on comm
unication channel (status = 3113), SQL <UPDATE /*+ RESTRICT_ALL_REF_CONS */ "SAPR3"."ZERMS_1" SET "JSON" = :a5,"METADATA" = :a6,"XML
STR" = :a7,"ZDATE" = :a8,"EFILE_TITLE" = :a9,"EFILE_TIME" = :a10,"DOC_OPERATION" = :a11,"DOC_CREATOR" = :a12>.
2013-12-27 09:00:12  WARNING OGG-01004  Oracle GoldenGate Delivery for Oracle, reperp.prm:  Aborted grouped transaction on 'SAPR3.ZE
RMS_1', Database error 3114 (ORA-03114: not connected to ORACLE).alert日志如下:Thread 1 advanced to log sequence 183196 (LGWR switch)
  Current log# 4 seq# 183196 mem# 0: /oracle/PRD/origlogB/log_g14m1.dbf
  Current log# 4 seq# 183196 mem# 1: /oracle/PRD/mirrlogB/log_g14m2.dbf
Fri Dec 27 08:54:32 2013
Completed checkpoint up to RBA , SCN: 1669885094
Fri Dec 27 08:55:20 2013
Completed checkpoint up to RBA , SCN: 1669888343
Fri Dec 27 08:58:05 2013
Errors in file /oracle/PRD/saptrace/usertrace/prd_ora_360586.trc:
ORA-07445: exception encountered: core dump [] []
Fri Dec 27 08:58:27 2013
Completed checkpoint up to RBA , SCN: 1669891863
Fri Dec 27 09:17:18 2013
Beginning log switch checkpoint up to RBA , SCN: 1669896877
Fri Dec 27 09:17:18 2013
Thread 1 advanced to log sequence 183197 (LGWR switch)
  Current log# 1 seq# 183197 mem# 0: /oracle/PRD/origlogA/log_g11m1.dbf
  Current log# 1 seq# 183197 mem# 1: /oracle/PRD/mirrlogA/log_g11m2.dbf附件中为日志和trc文件,请各位帮忙,谢谢!

kevinlin.ora 发表于 2013-12-27 14:49:04

按call stack搜索,比如:
000044F0             ?        00000000            
upduaw+00a4          bl       updgrh               FFFFFFFFFFEA680 ?
                                                   700000226EEA0C8 ?
kdusru+09f8          bl       01F93874            
kauupd+0230          bl       kdusru               700000210B6FA08 ?
                                                   100400000000 ? 000000000 ?
                                                   000000000 ?
updrow+10fc          bl       kauupd               11050F940 ? 000000000 ?
                                                   110511FA8 ?
                                                   4004828010000FD8 ?
                                                   700000217F99588 ?
                                                   1A000000000000 ?
                                                   48523FFFF6870 ?
                                                   700000226B555E0 ?
qerupRowProcedure+0  bl       updrow               000000000 ? 7FFF00000000 ?
050                                                
qerupFetch+053c      bl       03F24EB4            
updaul+0e1c          bl       01F93874            
updThreePhaseExe+0f  bl       updaul               700000217F9D5E8 ?
10                                                 FFFFFFFFFFF7FA8 ? 000000000 ?
updexe+02f8          bl       updThreePhaseExe     07FFFFFF8 ? 104A27300 ?
                                                   104A272E4 ? 000000000 ?
opiexe+27d8          bl       updexe               110D72750 ? 3FFFF86B8 ?
kpoal8+0edc          bl       opiexe               1100DD628 ? 000000018 ?

clot09 发表于 2013-12-27 14:59:07

kevinlin.ora 发表于 2013-12-27 14:49 static/image/common/back.gif
按call stack搜索,比如:
000044F0             ?        00000000            
upduaw+00a4          bl ...

额,水平比较菜,没理解,麻烦说的详细点吧,谢谢了~

kevinlin.ora 发表于 2013-12-27 15:03:59

MOS里的7445文档工具里,左边是按版本及错误的argument查找,右边有按stack搜索的。

clot09 发表于 2013-12-27 17:01:31

kevinlin.ora 发表于 2013-12-27 15:03 static/image/common/back.gif
MOS里的7445文档工具里,左边是按版本及错误的argument查找,右边有按stack搜索的。 ...

根据你提供的方法,试了下,似乎只找到了
Bug 4904890 - Dump (upduaw) from UPDATE (文档 ID 4904890.8)
thanks!

Liu Maclean(刘相兵 发表于 2013-12-27 17:35:56

Exception signal: 11 (SIGSEGV), code: 51 (Invalid permissions for mapped object), addr: 0x11028effc, PC:
Registers:

UPDATE /*+ RESTRICT_ALL_REF_CONS */ "SAPR3"."ZERMS_1" SET "JSON" = :a5,"METADATA" = :a6,"XMLSTR" = :a7,"ZDATE" = :a8,"EFILE_TITLE" = :a9,"EFILE_TIME" = :a10,"DOC_OPERATION" = :a11,"DOC_CREATOR" = :a12,"TS_STATUS" = :a13,"TS_TIME" = :a14,"ERMS_EFILE_ID" = :a15,"RET_RESULT_ID" = :a16,"RET_MESSAGE" = :a17,"CXTS_NUM" = :a18,"REMARK" = :a19,"RESERVE1" = :a20,"RESERVE2" = :a21,"RESERVE3" = :a22,"RESERVE4" = :a23,"RESERVE5" = :a24 WHERE "MANDT" = :b0 AND "UNIT_ID" = :b1 AND "SYS_NAME" = :b2 AND "TP_ID" = :b3 AND "EFILE_ID" = :b4


qerupFetch qerupRowProcedure updrow kauupd kdusru upduaw ssexhd

Liu Maclean(刘相兵 发表于 2013-12-27 17:39:42

ODM FINDING:

Symptoms

You could be getting either of the following errors in the alert.log or all of them while running an update statement:


ORA-07445: exception encountered: core dump [] []

ORA-07445: exception encountered: core dump [] []

ORA-07445: exception encountered: core dump []

ORA-7445: exception encountered: core dump
[] []

ORA-07445: exception encountered: core dump [] []


The call stack extracted from the trace file for the above error(s) will look similar to the one below:


updgrh upduaw kdusru kauupd updrow qerupRowProcedure qerupFetch updaul updThreePhaseExe updexe opiexe kpoal8 opiodr ttcpip opitsk opiino opiodr opidrv sou2o opimai_real main main_opd_entry


If you enable the 1551 event tracing, a trace file is generated at the time of the internal error reporting the following:

Errors in file /u10/app/oracle/admin/sid/udump/sid_ora_x.trc:
ORA-01551: extended rollback segment, pinned blocks released
Changes

Upgraded to 9.2.0.8 or 10.2.0.4
Cause

This problem is due to unpublished BUG 5868257 : Dump / memory corruption from UPDATE DML

Details:
There is a problem with the fix from Bug 4549673 which is included in 9.2.0.8 and 10.2.0.4. With that fix applied it is possible to get a dump under updgrh and/or private memory corruption. This issue can also cause malformed RCI information in the redo stream causing problems for log miner and Streams.
eg:
Corruption / dumps can occur when updating a migrated row where one of the columns is to be updated to NULL and an internal 1551 error occurs during the DML.(1551 errors are not visible to customers and are caught and handled internally)

This problem is raised in the following bugs which are closed as Duplicate of

Unpublished BUG 5868257.

BUG 7138239 - ORA-07445 FOR UPDATE
BUG 7118779 - VARIOUS ORA-600 / ORA-7445 ERRORS FROM REMOTE APPLY SIDE
BUG 7136414 - ORA-07445 ERROR RUNNING UPDATE
BUG 7423332 - MEMORY CORRUPTIONS WITH VARIOUS ORA-600 AND ORA-7445'S UNTIL PMON FAILURE
Bug 7390551 ORA-7445 EXCEPTION ENCOUNTERED: CORE DUMP

Solution

1. Unpublished Bug 5868257 is fixed in patchsets 10.2.0.5 and 11.1.0.6 and higher. Therefore upgrade to one of the Oracle Versions where the bug has been fixed.

OR

2. Apply the one-off Patch 5868257 if available on My Oracle Support for your Oracle Version and platform.

If your database version is 10.2.0.4 on HP-UX Itanium, then make sure you apply the patch which is dated 23-OCT-2008 (or later) because previous versions of this patch on this platform & DB version showed some problems as seen in Bug 7423332.


OR

3. As a workaround, you can help to avoid getting migrated rows by ensuring that PCTFREE is set sufficiently high such that subsequent UPDATE of rows in a data block has space to fit. It seems likely that you could minimize the risk of this issue by rebuilding the main affected table with a sufficient PCTFREE to avoid row migration.
References

Liu Maclean(刘相兵 发表于 2013-12-27 17:39:53

. Apply the one-off Patch 5868257 if available on My Oracle Support for your Oracle Version and platform.

clot09 发表于 2013-12-27 17:40:19

Liu Maclean(刘相兵 发表于 2013-12-27 17:35 static/image/common/back.gif
Exception signal: 11 (SIGSEGV), code: 51 (Invalid permissions for mapped object), addr: 0x11028effc, ...

刘大,请问这是使用到的函数吧?那根据Bug 4904890 - Dump (upduaw) from UPDATE (文档 ID 4904890.8)。能推断这是bug引起的吧?

clot09 发表于 2013-12-27 17:40:55

Liu Maclean(刘相兵 发表于 2013-12-27 17:39 static/image/common/back.gif
. Apply the one-off Patch 5868257 if available on My Oracle Support for your Oracle Version and plat ...

好的,谢谢刘大
页: [1]
查看完整版本: ORA-07445 core dump [0000F4F4] [SIGSEGV]错误,导致ogg连节oracle中断