RAC 11gR2 VIP的Mac不能及时更新问题

limouse

环境：
两台主机+SAN存储+Cisco 7609+oracle11.2.0.2

故障现象：
一台主机的Public接口线路中断后，VIP飘移到另一台主机，数据库连接正常。
当把网线插回原处public接口重新通讯后，VIP飘回原位，但Ping 该VIP不通，数据库连接中断
经查7609上VIP的mac地址没有更新，clear 该接口的arp信息后，该VIP的MAC更新，一切正常。

请问老大这是什么原因是配置问题还是oracle的一个BUG

Maclean Liu(刘相兵

是什么操作系统？　Windows?

$CRS_HOME/log/<nodename>/crsd/*.log
$CRS_HOME/log/<nodename>/racg/*.log

把这2个目录下的日志上传

limouse

操作系统都是Redhat 5.4，日志附上，13.24分左右作了一次测试

Maclean Liu(刘相兵

ODM Data:

1. Linux: ARP cache issues with Red Hat "balance-alb (mode 6)" bonding driver [ID 756259.1]
   
2. 
   
3. Applies to:
   
4. Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 11.1.0.7 - Release: 10.1 to 11.1
   
5. Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 11.1.0.7   [Release: 10.1 to 11.1]
   
6. Linux x86
   
7. Linux x86-64
   
8. Oracle Server Enterprise Edition - Version: 10.1.0.2.0 to 11.1.0.7.0
   
9. Oracle Clusterware when using bonding on Public network
   
10. Oracle Clusterware when using bonding on Private network
    
11. Symptoms
    
12. When using "balance-alb (mode 6)" bonding driver in an Oracle Clusterware setup, if the VIP fails over to another node, the ping requests for the relocated VIP fail unless the ARP cache is manually cleared.
    
13. 
    
14. This causes sessions to hang as the session retries to connect to the VIP but it is unsuccessful due to the stale ARP cache entries.
    
15. Cause
    
16. 
    
17. This hang is due to the failover mode configured for NIC bonding. Specifically  "balance-alb (mode 6)" bonding driver requires ARP to be cleaned manually. This is not an Oracle Clusterware issue. It's recommended to use other mode if there's difficulty to fix the OS issue.
    
18. Additionally the on failover "balance-alb" driver issues arp requests every two seconds. This is the way "balance-alb" bonding driver was designed to work in failover situations however this may cause sessions to hang therefore rendering the Oracle Clusterware VIP failover useless.
    
19. Solution
    
20. This is not an Oracle Clusterware issue. Different NIC bonding modes may be incompatible with certain switches, interface cards and drivers, Customers should test VIP failover in their environment with different modes and choose the one that fits their requirements.
    
21. For more information refer on bonding modes refer to bonding.txt which is supplied by the linux vendor.

复制代码

确认你的环境中是否使用了"balance-alb (mode 6)" bonding driver。




另一个类似的case

1. Continuous Ping Fails For a Short Time Following SCAN Relocation [ID 1061722.1]
   
2. 
   
3. Applies to:
   
4. Oracle Server - Enterprise Edition - Version: 11.2.0.1.0 to 11.2.0.1.0 - Release: 11.2 to 11.2
   
5. Information in this document applies to any platform.
   
6. Symptoms
   
7. After SCAN VIP relocation form one node to another by any means (manual relocation or CRS or node shutdown), the relocated SCAN VIP is not pingable for a small amount of time.
   
8. 
   
9. By running any packet analyzer utility on the failed ping packets sent to the relocated SCAN VIP, it should show something like the following:
   
10. 
    
11. Frame 760 (60 bytes on wire, 60 bytes captured)
    
12. Ethernet II, Src: 00:19:bb:cf:7b:dc (00:19:bb:cf:7b:dc), Dst:
    
13. ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
    
14. Address Resolution Protocol (reply/gratuitous ARP)
    
15. Hardware type: Ethernet (0x0001)
    
16. Protocol type: IP (0x0800)
    
17. Hardware size: 6
    
18. Protocol size: 4
    
19. Opcode: reply (0x0002)
    
20. Sender MAC address: 00:19:bb:cf:7b:dc (00:19:bb:cf:7b:dc)
    
21. Sender IP address: 10.3.2.74 (10.3.2.74)
    
22. Target MAC address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
    
23. Target IP address: 10.3.2.74 (10.3.2.74)
    
24. 
    
25. Target MAC address filed is ff:ff:ff:ff:ff:ff and not 00:19:bb:cf:7b:dc as it should be.
    
26. Changes
    
27. 11.2.0.1 Grid Infrastructure (a.k.a CRS) version, with the public interface connected to layer 3 switches or routers that are RFC compliant, specifically that are unable to violate the RFC2002.
    
28. Cause
    
29. After the start of VIP on a node, the vipagent sends a gratuitous ARP request, there was a problem in the packet sent as gratuitous ARP.
    
30. 
    
31. RFC2002:
    
32. 
    
33. A Gratuitous ARP is an ARP packet sent by a node in order to
    
34. spontaneously cause other nodes to update an entry in their ARP
    
35. cache. A gratuitous ARP MAY use either an ARP Request or an ARP
    
36. Reply packet. In either case, the ARP Sender Protocol Address
    
37. and ARP Target Protocol Address are both set to the IP address
    
38. of the cache entry to be updated, and the ARP Sender Hardware
    
39. Address is set to the link-layer address to which this cache
    
40. entry should be updated. When using an ARP Reply packet, the
    
41. Target Hardware Address is also set to the link-layer address to
    
42. which this cache entry should be updated (this field is not used
    
43. in an ARP Request packet)
    
44. 
    
45. 
    
46. Solution
    
47. The fix for this code defect Bug:9109880 has been included in the GRID Infrastructure PSU2 patch of 11.2.0.1 version and in version 11.2.0.2.

复制代码

但是oracle 宣称在 11.2.0.2 中已经修复以上1061722.1问题

limouse

关于案例一：
我这边public网卡没有做Bond，所以也不存在这个问题。

案例二：我这边是11.2.0.2不存在这个问题。

郁闷中

Maclean Liu(刘相兵

建议你检查 ora.vip的 相关日志 ， 没有上传过。

此外 实际在10.2.0.3以后和11gR1中 public network挂掉， 默认vip不会自动failover了。

Starting from 10.2.0.4 and 11.1, VIP does not fail-over back to the original node even after the public network problem is resolved.  This behavior is the default behavior in 10.2.0.4 and 11.1 and is different from that of 10.2.0.3

http://www.oracledatabase12g.com ... original%20node.htm