9204升级到10204之后出现的7445错误

风三

同事遇到的，升级之后数据库几乎每天down一次，错误信息如下：
ORA-07445: exception encountered: core dump [ksuxdp()+431] [SIGSEGV] [Address not mapped to object] [0x000000008] [] []
Wed May 30 23:01:06 2012
ARC1: terminating instance due to error 472
Instance terminated by ARC1, pid = 19326

alertandtrace.rar (373.2 KB, 下载次数: 795)

2012-5-31 10:10 上传

点击文件名下载附件

这是alert和trace文件
metalink上没搜到类似的信息，请各位帮忙给看下，谢了。

Maclean Liu(刘相兵

7445 和 性能调优有什么关系吗？ 为什么要放到这个版来？！

所有的 7445 600 故障诊断都在管理版，  所有的性能问题 sql slow都在调优版！

Maclean Liu(刘相兵

ODM DATA:


10.2.0.4.0 + Linux  x86_64


Errors in file /oracle/admin/mon/bdump/mon_pmon_19241.trc:
ORA-07445: exception encountered: core dump [ksuxdp()+431] [SIGSEGV] [Address not mapped to object] [0x000000008] [] []
Wed May 30 23:01:06 2012
ARC1: terminating instance due to error 472
Instance terminated by ARC1, pid = 19326

pmon stack call:
opirip => ksbrdp => ksucln =>  ksuxdpg => ksuxdp=>__funlockfile => 报错


PROCESS STATE
-------------
Process global information:
     process: 0x2d8228bc8, call: 0x2d8362520, xact: (nil), curses: 0x2d932b308, usrses: 0x2d932b308
  ----------------------------------------
  SO: 0x2d8228bc8, type: 2, owner: (nil), flag: INIT/-/-/0x00
  (process) Oracle pid=2, calls cur/top: 0x2d8362520/0x2d8362520, flag: (e) SYSTEM
            int error: 0, call error: 0, sess error: 0, txn error 0
  (post info) last post received: 0 0 33
              last post received-location: ksrpublish
              last process to post me: 2d822b350 7 0
              last post sent: 0 0 34
              last post sent-location: ksrmdone
              last process posted by me: 2d822b350 7 0
    (latch info) wait_event=0 bits=1
      holding    (efd=4) 2df289b08 Child session switching level=0 child#=1
        Location from where latch is held: ksuxdp:
        Context saved from call: 0
        state=busy, wlstate=free
          waiters [orapid (seconds since: put on list, posted, alive check)]:
           19 (0, 1338390056, 0)
           waiter count=1
    Process Group: DEFAULT, pseudo proc: 0x2da2878b0
    O/S info: user: oracle, term: UNKNOWN, ospid: 19241
    OSD pid info: Unix process pid: 19241, image: oracle@monstd (PMON)



pmon hold  Child session switching latch

    SO: 0x2d8362520, type: 3, owner: 0x2d8228bc8, flag: INIT/-/-/0x00
    (call) sess: cur 2d932b308, rec 0, usr 2d932b308; depth: 0
    ----------------------------------------
    SO: 0x2da3e94b0, type: 16, owner: 0x2d8228bc8, flag: INIT/-/-/0x00
    (osp req holder)



pmon 持有  Child session switching latch   并block pid=19的 服务进程

PID =19 之前出现过 virtual circuit status ， 可能是  shared server process ，其最后在等 latch free

AA9297BB:10F07631    19     0 10005   3 KSL POST RCVD poster=16 loc='kmcmbf: not KMCVCFTOS' id1=0 id2=0 name=   type=0 fac#=5 facpost=1
AA9298F8:10F07638    19     0 10005   4 KSL POST SENT postee=16 loc='kmcpdp' id1=0 id2=0 name=   type=0
AA929907:10F07639    19     0 10005   1 KSL WAIT BEG [virtual circuit status] -1/0xffffffffffffffff 2/0x2 0/0x0
AA929C88:10F07640    19     0 10005   2 KSL WAIT END [virtual circuit status] -1/0xffffffffffffffff 2/0x2 0/0x0 time=897
AA929C89:10F07641    19     0 10005   3 KSL POST RCVD poster=15 loc='kmcmbf: not KMCVCFTOS' id1=0 id2=0 name=   type=0 fac#=5 facpost=1
AA929CD1:10F07642    19   490 10021  11 0x00000000A97E2E92 0x00000000A97E2E91
AA929CD4:10F07643    19   490 10005   4 KSL POST SENT postee=7 loc='ksasnd' id1=0 id2=0 name=   type=0
AA929CF3:10F07647    19   490 10021  12 0x00000000A97E2E92 0x00000000A97E2E93
AA929CF6:10F07648    19   490 10005   1 KSL WAIT BEG [log file sync] 12317/0x301d 0/0x0 0/0x0
AA929E34:10F0764E    19   490 10005   2 KSL WAIT END [log file sync] 12317/0x301d 0/0x0 0/0x0 time=319
AA929E36:10F0764F    19   490 10005   3 KSL POST RCVD poster=7 loc='kslpsr' id1=109 id2=0 name=EV type=0 fac#=0 facpost=1
AA929E51:10F07650    19     0 10005   4 KSL POST SENT postee=15 loc='kmcpdp' id1=0 id2=0 name=   type=0
AA929E5E:10F07651    19     0 10005   1 KSL WAIT BEG [virtual circuit status] -1/0xffffffffffffffff 2/0x2 0/0x0
AA929EC6:10F07658    19     0 10005   2 KSL WAIT END [virtual circuit status] -1/0xffffffffffffffff 2/0x2 0/0x0 time=103
AA929EC6:10F07659    19     0 10005   3 KSL POST RCVD poster=16 loc='kmcmbf: not KMCVCFTOS' id1=0 id2=0 name=   type=0 fac#=5 facpost=1
AA929F0B:10F0765A    19   374 10021  11 0x00000000A97E2E94 0x00000000A97E2E93
AA929F0D:10F0765B    19   374 10005   4 KSL POST SENT postee=7 loc='ksasnd' id1=0 id2=0 name=   type=0
AA929F23:10F0765F    19   374 10021  12 0x00000000A97E2E94 0x00000000A97E2E95
AA929F26:10F07660    19   374 10005   1 KSL WAIT BEG [log file sync] 12319/0x301f 0/0x0 0/0x0
AA92A040:10F07666    19   374 10005   2 KSL WAIT END [log file sync] 12319/0x301f 0/0x0 0/0x0 time=283
AA92A042:10F07667    19   374 10005   3 KSL POST RCVD poster=7 loc='kslpsr' id1=109 id2=0 name=EV type=0 fac#=0 facpost=1
AA92A074:10F07668    19   374 10005   1 KSL WAIT BEG [latch free] 12333914888/0x2df289b08 5/0x5 0/0x0

Maclean Liu(刘相兵

就TRACE信息来看 与5642296 9.2.0.7 RDBMS 9.2.0.7 VOS PRODID-5 PORTID-59 ORA-7445 十分相似

Hdr: 5642296 9.2.0.7 RDBMS 9.2.0.7 VOS PRODID-5 PORTID-59 ORA-7445
Abstract: DATABASE CRASHED AFTER  ORA-7445 KSUXDP


PROBLEM:
--------
Database crashed with the following errors:

  ORA-7445: exception encountered: core dump [ksuxdp()+1052] [SIGSEGV]
[unknown    code] [0x20000000A] [] []
  ORA-472: PMON  process terminated with error


and the pmon trace file shows:

   Dump file /u01/app/oracle/admin/PBMLOL/bdump/pbmlol_pmon_2282.trc
    Oracle9i Enterprise Edition Release 9.2.0.7.0 - 64bit Production
    With the Partitioning option
    JServer Release 9.2.0.7.0 - Production
   
    *** 2006-11-01 11:12:31.225
    *** ID 1.1) 2006-11-01 11:12:31.161
    Pseudo-process owns s.o. c00000044f0376e8 which is not a session:
    Exception signal: 11 (SIGSEGV), code: 0 (unknown code), addr:
0x000000020000000a, PC:
    [0x4000000001922e74, ksuxdp()+1052]
   
    *** 2006-11-01 11:12:31.551
    ksedmp: internal or fatal error
    ORA-7445: exception encountered: core dump [ksuxdp()+1052] [SIGSEGV]
[unknown code] [0x20000000A]
    [] []
    ----- Call Stack Trace -----
    ksedmp ssexhd _sigreturn ksuxdp ksuxdpg ksucln ksbrdp opirip opidrv sou2o
main
   
   
   
        (session) trans: 0000000000000000, creator: c0000004136ad9d0, flag:
(51) USR/- BSY/-/-/-/-/-
                  DID: 0001-0002-00000002, short-term DID: 0000-0000-00000000
                  txn branch: 0000000000000000
                  oct: 0, prv: 0, sql: c000000462fa53a0, psql:
c000000462fa53a0, user: 0/SYS
        last wait for 'pmon timer' blocking sess=0x0 seq=28654
wait_time=2927371
                    duration=12c, =0, =0
        temporary object counter: 0



The error has raised during delete sessions hanging off pseudo that has gone.
   
     << ksuxdp ksuxdpg ksucln >>

Error not reproducible. Situation has raised one time

WORKAROUND:
-----------
none


ksuxdp ksuxdpg ksucln ksbrdp   stack call几乎一致

这可能是 10.2.0.4 一个隐藏的比较深的bug ， 建议你确认 该系统是否有XA事务， 另外参数显示：

  session_max_open_files   = 20
  cursor_sharing           = SIMILAR
  shared_servers           = 5
  max_shared_servers       = 10
  max_dispatchers          = 10
sort_area_size           = 524288
这些是否是 9i 历史留下的参数？  该系统是否真的是用 shared server 模式， 是否可以考虑 禁用 shared server 。


建议1：
禁用shared server ， 删除legacy 参数 譬如 sort_area_size cursor_sharing  = SIMILAR

建议2：

升级到 10.2.0.5.7

风三

不好意思，好久没来，生疏了。抱歉抱歉。

风三

先把shared_server关了，貌似几天没出现问题了，再观察一段时间看看。多谢了。