ORA-12805: parallel query server died unexpectedly

wanglixue2012

DB：11.2.0.3.6
OS:linux 6.3 64BIT


创建索引时报的错误日志：(应用上的报错)


Script Execution is unsuccessful For create_missing_indexes_au_tables_ps3.source

ORA-20001: ERROR ENCOUNTERED WHILE CREATING INDEX ORA-20001: INDEX CREATION

FAILED FOR I_AU_PURCHASED_DISCOUNT__IDORA-12805: parallel query server died

unexpectedly

ORA-12805: parallel query server died unexpectedly

ORA-06512: at line 159

ORA-20001: INDEX CREATION FAILED FOR I_AU_PURCHASED_DISCOUNT__IDORA-12805:

parallel query server died unexpectedly

ORA-12805: parallel query server died unexpectedly

Maclean Liu(刘相兵

你没说时间点 只能猜一个时间点：

Thu May 30 15:16:00 2013
Dumping diagnostic data in directory=[cdmp_20130530151354], requested by (instance=3, osid=38364 (J000)), summary=[incident=64001].
Thu May 30 15:18:48 2013
IPC Send timeout detected. Sender: ospid 44155 [oracle@aaadbrac10 (P076)]
Receiver: inst 3 binc 27 ospid 36148
Thu May 30 15:26:57 2013
Dumping diagnostic data in directory=[cdmp_20130530152647], requested by (instance=3, osid=47143 (W000)), summary=[incident=64003].
Thu May 30 15:29:13 2013
Dumping diagnostic data in directory=[cdmp_20130530152913], requested by (instance=2, osid=21289 (CKPT)), summary=[incident=112259].
Thu May 30 15:33:38 2013
Dumping diagnostic data in directory=[cdmp_20130530153110], requested by (instance=3, osid=2429 (J000)), summary=[incident=64004].
Thu May 30 16:11:42 2013
IPC Send timeout detected. Sender: ospid 30632 [oracle@aaadbrac10 (P019)]
Receiver: inst 3 binc 27 ospid 28243
Errors in file /u01/app/oracle/diag/rdbms/aaadbrac/aaadbrac1/trace/aaadbrac1_p019_30632.trc  (incident=112611):
ORA-00600: internal error code, arguments: [15733], [5], [0xCE1098F88], [70], [3], [], [], [], [], [], [], []
ORA-10388: parallel query server interrupt (failure)


需要/u01/app/oracle/diag/rdbms/aaadbrac/aaadbrac1/trace/aaadbrac1_p019_30632.trc

wanglixue2012

哈哈，不好意思。问题大概是这样的。
7个节点不约而同的报了和inst3的 IPC问题。
IPC Send timeout detected. Sender: ospid 23542 [oracle@aaadbrac16 (P019)]
Receiver: inst 3 binc 27 ospid 4317


第三个节点：

skgxpvfymtu: mtu_type: 61 process 48797 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 3)
Errors in file /u01/app/oracle/diag/rdbms/aaadbrac/aaadbrac3/trace/aaadbrac3_q003_48797.trc  (incident=64115):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: sskgxpsnd2
opidrv aborting process Q003 ospid (48797) as a result of ORA-603

trace里有一段SKGXPVFYMTU: mtu_type: 61 Could not create context which satisfies the minimum MTU requirement of 32768 bytes

大致意思就是内存不能创建连续的32K的空间满足MTU的，导致节点间通信不正常，所以其他节点探测到节点3有问题


后来发现根本原因是因为，操作系统的报错：
May 30 15:12:59 aaadbrac12 kernel: nf_conntrack: table full, dropping packet.

网上有很多解决方案。但是终极原因是iptables 在LINUX 6.3 诡异的开启着，chkconfig 配置过iptables 全部为OFF了，但是系统启动后还是把iptables 开启了。

所以我只能在系统启动时 service iptabels stop

kevinyang

遇到过系统上开启了keepalived就会自动打开iptables的问题，不知道楼主有没有开keepalived