RAC监听都停止，vip跑到对方主机，请教

136156188

rac出现异常，登录后，查看crs状态：

发现vip都跑到对方主机上了，监听也停止了。


SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi
PL/SQL Release 10.2.0.4.0 - Production
CORE    10.2.0.4.0      Production
TNS for Linux: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production
SQL>

[oracle@hgrac1 ~]$ crsctl query crs activeversion
CRS active version on the cluster is [10.2.0.4.0]
[oracle@hgrac1 ~]$


cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1       localhost.localdomain   localhost
192.100.180.1   hgrac1
192.100.180.2   hgrac2
192.100.180.3   hgrac1-vip
192.100.180.4   hgrac2-vip
172.250.250.1   hgrac1-priv
172.250.250.2   hgrac2-priv

大约在16:25:16我重启了crs_start -all
然后rac正常了
附上各种日志，斑斑帮忙分析一下，问题出在哪里？
新文件夹.rar (558.16 KB, 下载次数: 902)

2012-3-11 18:29 上传

点击文件名下载附件
日志

[ 本帖最后由 136156188 于 2012-3-11 18:29 编辑 ]

Maclean Liu(刘相兵

请把2个节点上 $ORA_CRS_HOME/log/$hostname/racg 目录下的日志上传

136156188

谢谢，racg日志附上。

Maclean Liu(刘相兵

ODM Finding:

2012-03-11 01:04:05.063: [    RACG][1952268992] [19814][1952268992][ora.hgrac1.vip]: bounce eth0 (host=hgrac2)

2012-03-11 03:19:42.861: [    RACG][1935295168] [31329][1935295168][ora.hgrac1.vip]: IP:192.100.180.3 is already up in the network (host=hgrac2)
IP:192.100.180.3 is already up in the network (host=hgrac2)
ping to 192.100.180.254 via eth0 failed, rc = 1 (host=hgrac2)
ping to 192.100.180.254 via eth0 failed, rc = 1 (host=hgrac2)

2012-03-11 03:19:42.861: [    RACG][1935295168] [31329][1935295168][ora.hgrac1.vip]: get_lock: Failed to get lock for eth0_1 (host=hgrac2)
get_lock: Failed to get lock for eth0_2 (host=hgrac2)
get_lock: Failed to get lock for eth0_3 (host=hgrac2)

2012-03-11 03:41:32.706: [    RACG][2836095680] [10167][2836095680][ora.hgrac1.vip]: ping to 192.100.180.254 via eth0 failed, rc = 1 (host=hgrac2)
ping to 192.100.180.254 via eth0 failed, rc = 1 (host=hgrac2)
Interface eth0 checked failed (host=hgrac2)
Invalid parameters, or failed to bring up VIP (host=hgrac2)

2012-03-11 03:41:32.706: [    RACG][2836095680] [10167][2836095680][ora.hgrac1.vip]: clsrcexecut: env ORACLE_CONFIG_HOME=/opt/ora10g/product/10.2.0/crs_1

2012-03-11 03:41:32.706: [    RACG][2836095680] [10167][2836095680][ora.hgrac1.vip]: clsrcexecut: cmd = /opt/ora10g/product/10.2.0/crs_1/bin/racgeut -e _USR_ORA_DEBUG=0 54 /opt/ora10g/product/10.2.0/crs_1/bin/racgvip check hgrac1

2012-03-11 03:41:32.706: [    RACG][2836095680] [10167][2836095680][ora.hgrac1.vip]: clsrcexecut: rc = 1, time = 6.270s

2012-03-11 03:41:32.706: [    RACG][2836095680] [10167][2836095680][ora.hgrac1.vip]: end for resource = ora.hgrac1.vip, action = check, status = 1, time = 6.310s

2012-03-11 03:55:22.020: [    RACG][1522299584] [14766][1522299584][ora.hgrac1.vip]: ping to 192.100.180.254 via eth0 failed, rc = 1 (host=hgrac2)
ping to 192.100.180.254 via eth0 failed, rc = 1 (host=hgrac2)



bounce eth0 => eth0 的network interface 出现过问题

ping to 192.100.180.254 via eth0 广播失败

即ora.hgrac1.vip 在 hgrac1 上 发现  public network interface eth0 发生失败 ， 漂移(drift)到  hgrac2上
而ora.hgrac2.vip 在 hgrac2 上也发现 public network interface eth0 发生失败，漂移(drift)到hgrac1

从10.2.0.4 开始 vip因为public network fail 而发生 漂移， 即便public network 恢复 正常， vip也不会 漂移会原节点。(VIP does not relocate back to the original node starting from 10.2.0.4 and 11.1 even after the public network problem is resolved. [ID 805969.1])

建议你检查 eth0 这个interface 192.100.180.0 网段是否正常， 然后重启 vip资源。