Oracle数据库数据恢复、性能优化»论坛 › Oracle › Oracle数据库管理 › 11g R2从11.2.0.3.3升级到11.2.0.3.6，执行最后一步时CR ...

0 积分	2 好友	7 主题

发消息

[Exadata常规管理] 11g R2从11.2.0.3.3升级到11.2.0.3.6，执行最后一步时CRS起不来

1^#

发表于 2013-8-26 13:06:14 | 查看: 5271| 回复: 12

本帖最后由 renjie_41 于 2013-8-26 13:11 编辑

操作系统版本：
6100-08
数据库版本：打补丁前，11.2.0.3.3；
         打补丁后，11.2.0.3.6
打补丁过程中间无报错，manual local方式打的补丁
[grid@xjcrmdb1:/u01/app/11.2.0/grid/OPatch]$ opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.4
Copyright (c) 2012, Oracle Corporation.  All rights reserved.

Oracle Home    : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
from          : /u01/app/11.2.0/grid/oraInst.loc
OPatch version : 11.2.0.3.4
OUI version    : 11.2.0.3.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-08-26_12-47-01PM_1.log

Lsinventory Output file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2013-08-26_12-47-01PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure                                        11.2.0.3.0
There are 1 products installed in this Oracle Home.

Interim patches (2) :

Patch  16056266    : applied on Thu Aug 22 12:00:14 GMT+08:00 2013
Unique Patch ID:  15978825
Patch description:  "Database Patch Set Update : 11.2.0.3.6 (16056266)"
Created on 13 Mar 2013, 05:49:56 hrs PST8PDT
Sub-patch  14727310; "Database Patch Set Update : 11.2.0.3.5 (14727310)"
Sub-patch  14275605; "Database Patch Set Update : 11.2.0.3.4 (14275605)"
Sub-patch  13923374; "Database Patch Set Update : 11.2.0.3.3 (13923374)"
Sub-patch  13696216; "Database Patch Set Update : 11.2.0.3.2 (13696216)"
Sub-patch  13343438; "Database Patch Set Update : 11.2.0.3.1 (13343438)"
但执行最后一步报错，CRS资源拉不起来
xjcrmdb1:root:/># /u01/app/11.2.0/grid/crs/install/rootcrs.pl -patch
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
CRS-4123: Oracle High Availability Services has been started.
Failed to write the checkpoint:'' with status:FAIL.Error code is 256
Oracle Grid Infrastructure stack start initiated but failed to complete at /u01/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 11627.

rootcrs_xjcrmdb1.txt.txt (207.13 KB, 下载次数: 886)

alert_ ASM.txt.txt

222.54 KB, 下载次数: 911

alertxjcrmdb1.txt.txt

122.78 KB, 下载次数: 899

分享0

收藏0 回复只看该作者道具举报

Maclean Liu(刘相兵

2^#

发表于 2013-8-26 19:49:10

2013-08-24 15:32:05: Executing cmd: /u01/app/11.2.0/grid/bin/crsctl check has
2013-08-24 15:32:06: Command output:
> CRS-4639: Could not contact Oracle High Availability Services
>End Command output

需要OHASD和OCSSD的日志

以及

df -h的信息

回复只看该作者道具举报

renjie_41

3^#

发表于 2013-8-27 10:13:25

xjcrmdb1:root:/># df -g
Filesystem GB blocks    Free %Used Iused %Iused Mounted on
/dev/hd4          4.00    3.27 19% 12249    2% /
/dev/hd2          4.00    0.96 77% 59077 21% /usr
/dev/hd9var       1.00    0.55 46%    8714    7% /var
/dev/hd3          6.00    5.42 10%    321    1% /tmp
/dev/hd1          1.00    0.99 1%    94    1% /home
/dev/hd11admin    1.00    1.00 1%       5    1% /admin
/proc                -       - -       -    -  /proc
/dev/hd10opt    1.00    0.39 62% 12009 12% /opt
/dev/livedump    1.00    1.00 1%       4    1% /var/adm/ras/livedump
/dev/lv_u01    80.00    22.94 72% 115117    3% /u01
/dev/lvarch1    300.00 276.90 8%    84    1% /archcrmdb1
/dev/lvbackup 150.00    45.36 70% 14498    1% /backup

回复只看该作者道具举报

renjie_41

4^#

发表于 2013-8-27 10:14:25

xjcrmdb1:root:/># /u01/app/11.2.0/grid/bin/crsctl check has
CRS-4638: Oracle High Availability Services is online

回复只看该作者道具举报

renjie_41

5^#

发表于 2013-8-27 10:25:19

谢谢刘老师了

3.82 MB, 下载次数: 736

1.1 MB, 下载次数: 830

6^#

发表于 2013-8-27 11:02:42

急求刘老师及各位朋友的帮助

回复只看该作者道具举报

renjie_41

7^#

发表于 2013-8-27 11:13:48

[grid@xjcrmdb1:/u01/app/11.2.0/grid]$ crsctl stat res -t -init
--------------------------------------------------------------------------------
NAME          TARGET  STATE       SERVER                STATE_DETAILS
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
   1       ONLINE  ONLINE    xjcrmdb1                Started
ora.cluster_interconnect.haip
   1       ONLINE  ONLINE    xjcrmdb1
ora.crf
   1       ONLINE  ONLINE    xjcrmdb1
ora.crsd
   1       ONLINE  INTERMEDIATE xjcrmdb1
ora.cssd
   1       ONLINE  ONLINE    xjcrmdb1
ora.cssdmonitor
   1       ONLINE  ONLINE    xjcrmdb1
ora.ctssd
   1       ONLINE  ONLINE    xjcrmdb1                OBSERVER
ora.diskmon
   1       OFFLINE OFFLINE
ora.drivers.acfs
   1       ONLINE  OFFLINE
ora.evmd
   1       ONLINE  INTERMEDIATE xjcrmdb1
ora.gipcd
   1       ONLINE  ONLINE    xjcrmdb1
ora.gpnpd
   1       ONLINE  ONLINE    xjcrmdb1
ora.mdnsd
   1       ONLINE  ONLINE    xjcrmdb1
[grid@xjcrmdb1:/u01/app/11.2.0/grid]$
[grid@xjcrmdb1:/u01/app/11.2.0/grid]$
[grid@xjcrmdb1:/u01/app/11.2.0/grid]$
[grid@xjcrmdb1:/u01/app/11.2.0/grid]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager

回复只看该作者道具举报

Maclean Liu(刘相兵

8^#

发表于 2013-8-27 19:59:39

Oracle Database 11g Clusterware Release 11.2.0.3.0 - Production Copyright 1996, 2011 Oracle. All rights reserved.
2013-08-27 10:12:51.916: [GIPCXCPT][5408]gipchaLowerRecv: partial header received, discarding len 52 < hdr 80, from 'mcast://224.0.0.251:0/0.0.0.80', ret gipcretFail (1)
2013-08-27 10:12:51.916: [GIPCHALO][5408]gipchaLowerCallback: EXCEPTION[ ret gipcretFail (1) ]  error while processing req 114bde508 { type gipcreqtypeRecv, endp 0000000000000617, ret gipcretSuccess, endpA 0000000000000000, peer 'mcast://224.0.0.251:0/0.0.0.80' buf 1154469d8, len 10240, olen 52 }, hctx 1113921b0 [0000000000000010] { gipchaContext : host 'xjcrmdb1', name 'CLSFRAME_xjcrmdb', luid '5292db51-00000000', numNode 0, numInf 1, usrFlags 0x0, flags 0x67 }
2013-08-27 10:12:51.916: [GIPCXCPT][5408]gipchaLowerRecv: partial header received, discarding len 52 < hdr 80, from 'mcast://230.0.1.0:0/0.0.0.80', ret gipcretFail (1)
2013-08-27 10:12:51.916: [GIPCHALO][5408]gipchaLowerCallback: EXCEPTION[ ret gipcretFail (1) ]  error while processing req 114bde5d0 { type gipcreqtypeRecv, endp 0000000000000620, ret gipcretSuccess, endpA 0000000000000000, peer 'mcast://230.0.1.0:0/0.0.0.80' buf 114e5f898, len 10240, olen 52 }, hctx 1113921b0 [0000000000000010] { gipchaContext : host 'xjcrmdb1', name 'CLSFRAME_xjcrmdb', luid '5292db51-00000000', numNode 0, numInf 1, usrFlags 0x0, flags 0x67 }
2013-08-27 10:12:51.916: [GIPCXCPT][5408]gipchaLowerRecv: partial header received, discarding len 52 < hdr 80, from 'mcast://224.0.0.251:0/0.0.0.80', ret gipcretFail (1)
2013-08-27 10:12:51.916: [GIPCHALO][5408]gipchaLowerCallback: EXCEPTION[ ret gipcretFail (1) ]  error while processing req 114bde698 { type gipcreqtypeRecv, endp 0000000000000617, ret gipcretSuccess, endpA 0000000000000000, peer 'mcast://224.0.0.251:0/0.0.0.80' buf 114a4c718, len 10240, olen 52 }, hctx 1113921b0 [0000000000000010] { gipchaContext : host 'xjcrmdb1', name 'CLSFRAME_xjcrmdb', luid '5292db51-00000000', numNode 0, numInf 1, usrFlags 0x0, flags 0x67 }
2013-08-27 10:12:51.916: [GIPCXCPT][5408]gipchaLowerRecv: partial header received, discarding len 52 < hdr 80, from 'mcast://230.0.1.0:0/0.0.0.80', ret gipcretFail (1)
2013-08-27 10:12:51.916: [GIPCHALO][5408]gipchaLowerCallback: EXCEPTION[ ret gipcretFail (1) ]  error while processing req 114bde760 { type gipcreqtypeRecv, endp 0000000000000620, ret gipcretSuccess, endpA 0000000000000000, peer 'mcast://230.0.1.0:0/0.0.0.80' buf 114e9f0f8, len 10240, olen 52 }, hctx 1113921b0 [0000000000000010] { gipchaContext : host 'xjcrmdb1', name 'CLSFRAME_xjcrmdb', luid '5292db51-00000000', numNode 0, numInf 1, usrFlags 0x0, flags 0x67 }
2013-08-27 10:12:51.916: [GIPCXCPT][5408]gipchaLowerRecv: partial header received, discarding len 52 < hdr 80, from 'mcast://224.0.0.251:0/0.0.0.80', ret gipcretFail (1)

The issue has been investigated in BUG 16277504, BUG 16285675 and BUG 16264027. With collaboration by IBM the root cause has been identified to be an issue on both AIX 6.1 TL08 SP01 and AIX 7.1 TL02 SP01 with truncation of multicast packets.

The solution is to install the following AIX HIPER efix:

AIX 6.1 TL08 SP01

APAR IV35888: UDP MULTICAST: SHORT PACKET FOR SOME LISTENERS. 13/02/08 PTF PECHANGE
http://www-01.ibm.com/support/do ... omponent&mync=E

AIX 7.1 TL02 SP01

APAR IV35893: UDP MULTICAST: SHORT PACKET FOR SOME LISTENERS. 13/02/08 PTF PECHANGE
http://www-01.ibm.com/support/do ... omponent&mync=E

HISTORY

13-Feb-2013 - original version

REFERENCES

BUG:16277504 - GI STACK NOT STARTING POST APPLY OF 11.2.0.3.5 PSU + AIX 6.1 TL8

回复只看该作者道具举报

Maclean Liu(刘相兵

9^#

发表于 2013-8-27 20:05:37

这是IBM AIX的BUG 需要打补丁 AIX 6.1 TL08 SP01

AIX 6.1 TL08 SP01
APAR IV35888: UDP MULTICAST: SHORT PACKET FOR SOME LISTENERS. 13/02/08 PTF
PECHANGE
http://www-01.ibm.com/support/do ... omponent&mync=E
.
and then update the bug about whether the issue persists after applying APAR.

The issue also documented in note 1528452.1

we applied the AIX fix and now the 11.2.0.3.5 is working properly.

回复只看该作者道具举报