oracle rac 11.2.0.3 是否有内存泄露

hpf99

安装了oracle rac 11g 数据库版本是11.2.0.3 ,操作系统是redhat 5.4 ，通过top 命令发现orarootagent.bi 进程占用虚拟内存大，网上说   11.2.0.1 有本进程内存泄露，建议升级到  11.2.0.3，解决内存泄露，而实际上并没有解决。不知各位遇是否到此种情况                                                                       
                                                                                                                                                                                 
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                            
3836 root      15   0 1645m  17m  11m S  0.0  0.8   3:31.69 orarootagent.bi                                                                                                     
5153 oracle    18   0 1254m 277m  20m S  0.3 12.8   0:34.96 java            

Maclean Liu(刘相兵

pmap -d 3836  看一下

hpf99

Maclean Liu(刘相兵 发表于 2013-4-3 10:17
pmap -d 3836  看一下

[root@rac1 ~]# pmap -d 3836
3836:   /u01/app/11.2.0/grid/bin/orarootagent.bin
Address           Kbytes Mode  Offset           Device    Mapping
0000000000400000    5516 r-x-- 0000000000000000 008:00003 orarootagent.bin
0000000000a62000      96 rw--- 0000000000562000 008:00003 orarootagent.bin
0000000000a7a000      80 rw--- 0000000000a7a000 000:00000   [ anon ]
000000001de82000    5732 rw--- 000000001de82000 000:00000   [ anon ]
00000000418b6000       4 ----- 00000000418b6000 000:00000   [ anon ]
00000000418b7000   32768 rwx-- 00000000418b7000 000:00000   [ anon ]
00000000438b7000       4 ----- 00000000438b7000 000:00000   [ anon ]
00000000438b8000   32768 rwx-- 00000000438b8000 000:00000   [ anon ]
00000000458b8000       4 ----- 00000000458b8000 000:00000   [ anon ]
00000000458b9000   32768 rwx-- 00000000458b9000 000:00000   [ anon ]
00000000478b9000       4 ----- 00000000478b9000 000:00000   [ anon ]
00000000478ba000   32768 rwx-- 00000000478ba000 000:00000   [ anon ]
00000000498ba000       4 ----- 00000000498ba000 000:00000   [ anon ]
00000000498bb000   32768 rwx-- 00000000498bb000 000:00000   [ anon ]
000000004b8bb000       4 ----- 000000004b8bb000 000:00000   [ anon ]
000000004b8bc000   32768 rwx-- 000000004b8bc000 000:00000   [ anon ]
000000004d8bc000       4 ----- 000000004d8bc000 000:00000   [ anon ]
000000004d8bd000   32768 rwx-- 000000004d8bd000 000:00000   [ anon ]
000000004f8bd000       4 ----- 000000004f8bd000 000:00000   [ anon ]
000000004f8be000   32768 rwx-- 000000004f8be000 000:00000   [ anon ]
00000000518be000       4 ----- 00000000518be000 000:00000   [ anon ]
00000000518bf000   32768 rwx-- 00000000518bf000 000:00000   [ anon ]
00000000538bf000       4 ----- 00000000538bf000 000:00000   [ anon ]
00000000538c0000   32768 rwx-- 00000000538c0000 000:00000   [ anon ]
00000000558c0000       4 ----- 00000000558c0000 000:00000   [ anon ]
00000000558c1000   32768 rwx-- 00000000558c1000 000:00000   [ anon ]
00000000598c2000       4 ----- 00000000598c2000 000:00000   [ anon ]
00000000598c3000   32768 rwx-- 00000000598c3000 000:00000   [ anon ]
000000005b8c3000       4 ----- 000000005b8c3000 000:00000   [ anon ]
000000005b8c4000   32768 rwx-- 000000005b8c4000 000:00000   [ anon ]
000000005d8c4000       4 ----- 000000005d8c4000 000:00000   [ anon ]
000000005d8c5000   32768 rwx-- 000000005d8c5000 000:00000   [ anon ]
000000005f8c5000       4 ----- 000000005f8c5000 000:00000   [ anon ]
000000005f8c6000   32768 rwx-- 000000005f8c6000 000:00000   [ anon ]
00000000618c6000       4 ----- 00000000618c6000 000:00000   [ anon ]
00000000618c7000   32768 rwx-- 00000000618c7000 000:00000   [ anon ]
00000000638c7000       4 ----- 00000000638c7000 000:00000   [ anon ]
00000000638c8000   32768 rwx-- 00000000638c8000 000:00000   [ anon ]
00000000658c8000       4 ----- 00000000658c8000 000:00000   [ anon ]
00000000658c9000   32768 rwx-- 00000000658c9000 000:00000   [ anon ]
00000000678c9000       4 ----- 00000000678c9000 000:00000   [ anon ]
00000000678ca000   32768 rwx-- 00000000678ca000 000:00000   [ anon ]
00000000698ca000       4 ----- 00000000698ca000 000:00000   [ anon ]
00000000698cb000   32768 rwx-- 00000000698cb000 000:00000   [ anon ]
000000006b8cb000       4 ----- 000000006b8cb000 000:00000   [ anon ]
000000006b8cc000   32768 rwx-- 000000006b8cc000 000:00000   [ anon ]
000000006d8cc000       4 ----- 000000006d8cc000 000:00000   [ anon ]
000000006d8cd000   32768 rwx-- 000000006d8cd000 000:00000   [ anon ]
000000006f8cd000       4 ----- 000000006f8cd000 000:00000   [ anon ]
000000006f8ce000   32768 rwx-- 000000006f8ce000 000:00000   [ anon ]
00000000718ce000       4 ----- 00000000718ce000 000:00000   [ anon ]
00000000718cf000   32768 rwx-- 00000000718cf000 000:00000   [ anon ]
00000000738cf000       4 ----- 00000000738cf000 000:00000   [ anon ]
00000000738d0000   32768 rwx-- 00000000738d0000 000:00000   [ anon ]
00000000758d0000       4 ----- 00000000758d0000 000:00000   [ anon ]
00000000758d1000   32768 rwx-- 00000000758d1000 000:00000   [ anon ]
00000000778d1000       4 ----- 00000000778d1000 000:00000   [ anon ]
00000000778d2000   32768 rwx-- 00000000778d2000 000:00000   [ anon ]
00000000798d2000       4 ----- 00000000798d2000 000:00000   [ anon ]
00000000798d3000   32768 rwx-- 00000000798d3000 000:00000   [ anon ]
000000007b8d3000       4 ----- 000000007b8d3000 000:00000   [ anon ]
000000007b8d4000   32768 rwx-- 000000007b8d4000 000:00000   [ anon ]
000000007d8d4000       4 ----- 000000007d8d4000 000:00000   [ anon ]
000000007d8d5000   32768 rwx-- 000000007d8d5000 000:00000   [ anon ]
000000300f000000     112 r-x-- 0000000000000000 008:00003 ld-2.5.so
000000300f21b000       4 r---- 000000000001b000 008:00003 ld-2.5.so
000000300f21c000       4 rw--- 000000000001c000 008:00003 ld-2.5.so
000000300f400000    1336 r-x-- 0000000000000000 008:00003 libc-2.5.so
000000300f54e000    2044 ----- 000000000014e000 008:00003 libc-2.5.so
000000300f74d000      16 r---- 000000000014d000 008:00003 libc-2.5.so
000000300f751000       4 rw--- 0000000000151000 008:00003 libc-2.5.so
000000300f752000      20 rw--- 000000300f752000 000:00000   [ anon ]
000000300f800000     520 r-x-- 0000000000000000 008:00003 libm-2.5.so
000000300f882000    2044 ----- 0000000000082000 008:00003 libm-2.5.so
000000300fa81000       4 r---- 0000000000081000 008:00003 libm-2.5.so
000000300fa82000       4 rw--- 0000000000082000 008:00003 libm-2.5.so
000000300fc00000       8 r-x-- 0000000000000000 008:00003 libdl-2.5.so
000000300fc02000    2048 ----- 0000000000002000 008:00003 libdl-2.5.so
000000300fe02000       4 r---- 0000000000002000 008:00003 libdl-2.5.so
000000300fe03000       4 rw--- 0000000000003000 008:00003 libdl-2.5.so
0000003010000000      88 r-x-- 0000000000000000 008:00003 libpthread-2.5.so
0000003010016000    2044 ----- 0000000000016000 008:00003 libpthread-2.5.so
0000003010215000       4 r---- 0000000000015000 008:00003 libpthread-2.5.so
0000003010216000       4 rw--- 0000000000016000 008:00003 libpthread-2.5.so
0000003010217000      16 rw--- 0000003010217000 000:00000   [ anon ]
0000003013000000      84 r-x-- 0000000000000000 008:00003 libnsl-2.5.so
0000003013015000    2044 ----- 0000000000015000 008:00003 libnsl-2.5.so
0000003013214000       4 r---- 0000000000014000 008:00003 libnsl-2.5.so
0000003013215000       4 rw--- 0000000000015000 008:00003 libnsl-2.5.so
0000003013216000       8 rw--- 0000003013216000 000:00000   [ anon ]
0000003013400000      12 r-x-- 0000000000000000 008:00003 libcap.so.1.10
0000003013403000    2048 ----- 0000000000003000 008:00003 libcap.so.1.10
0000003013603000       4 rw--- 0000000000003000 008:00003 libcap.so.1.10
000000301fa00000      52 r-x-- 0000000000000000 008:00003 libgcc_s-4.1.2-20080825.so.1
000000301fa0d000    2048 ----- 000000000000d000 008:00003 libgcc_s-4.1.2-20080825.so.1
000000301fc0d000       4 rw--- 000000000000d000 008:00003 libgcc_s-4.1.2-20080825.so.1
0000003023200000     920 r-x-- 0000000000000000 008:00003 libstdc++.so.6.0.8
00000030232e6000    2044 ----- 00000000000e6000 008:00003 libstdc++.so.6.0.8
00000030234e5000      24 r---- 00000000000e5000 008:00003 libstdc++.so.6.0.8
00000030234eb000      12 rw--- 00000000000eb000 008:00003 libstdc++.so.6.0.8
00000030234ee000      72 rw--- 00000030234ee000 000:00000   [ anon ]
00002aaaaaaab000      40 r-x-- 0000000000000000 008:00003 libnque11.so
00002aaaaaab5000    1020 ----- 000000000000a000 008:00003 libnque11.so
00002aaaaabb4000       4 rw--- 0000000000009000 008:00003 libnque11.so
00002aaaaabb5000     520 rw--- 00002aaaaabb5000 000:00000   [ anon ]
00002aaaac000000     724 rw--- 00002aaaac000000 000:00000   [ anon ]
00002aaaac0b5000   64812 ----- 00002aaaac0b5000 000:00000   [ anon ]
00002aaab0000000     388 rw--- 00002aaab0000000 000:00000   [ anon ]
00002aaab0061000   65148 ----- 00002aaab0061000 000:00000   [ anon ]
00002aaab4000000       4 ----- 00002aaab4000000 000:00000   [ anon ]
00002aaab4001000   32768 rwx-- 00002aaab4001000 000:00000   [ anon ]
00002aaab6001000       4 ----- 00002aaab6001000 000:00000   [ anon ]
00002aaab6002000   32768 rwx-- 00002aaab6002000 000:00000   [ anon ]
00002aaab8002000       4 ----- 00002aaab8002000 000:00000   [ anon ]
00002aaab8003000   32768 rwx-- 00002aaab8003000 000:00000   [ anon ]
00002aaaba003000       4 ----- 00002aaaba003000 000:00000   [ anon ]
00002aaaba004000   32768 rwx-- 00002aaaba004000 000:00000   [ anon ]

hpf99

hpf99 发表于 2013-4-3 16:55
[root@rac1 ~]# pmap -d 3836
3836:   /u01/app/11.2.0/grid/bin/orarootagent.bin
Address           Kb ...

00002aaabc004000       4 ----- 00002aaabc004000 000:00000   [ anon ]
00002aaabc005000   32768 rwx-- 00002aaabc005000 000:00000   [ anon ]
00002aaabe005000       4 ----- 00002aaabe005000 000:00000   [ anon ]
00002aaabe006000   32768 rwx-- 00002aaabe006000 000:00000   [ anon ]
00002aaac0006000       4 ----- 00002aaac0006000 000:00000   [ anon ]
00002aaac0007000   32768 rwx-- 00002aaac0007000 000:00000   [ anon ]
00002aaac2007000       4 ----- 00002aaac2007000 000:00000   [ anon ]
00002aaac2008000   32768 rwx-- 00002aaac2008000 000:00000   [ anon ]
00002aaac4008000       4 ----- 00002aaac4008000 000:00000   [ anon ]
00002aaac4009000   32768 rwx-- 00002aaac4009000 000:00000   [ anon ]
00002aaac6009000       4 ----- 00002aaac6009000 000:00000   [ anon ]
00002aaac600a000   32768 rwx-- 00002aaac600a000 000:00000   [ anon ]
00002aaac800a000       4 ----- 00002aaac800a000 000:00000   [ anon ]
00002aaac800b000   32768 rwx-- 00002aaac800b000 000:00000   [ anon ]
00002aaaca00b000       4 ----- 00002aaaca00b000 000:00000   [ anon ]
00002aaaca00c000   32768 rwx-- 00002aaaca00c000 000:00000   [ anon ]
00002aaacc00c000       4 ----- 00002aaacc00c000 000:00000   [ anon ]
00002aaacc00d000   32768 rwx-- 00002aaacc00d000 000:00000   [ anon ]
00002aaad0000000     244 rw--- 00002aaad0000000 000:00000   [ anon ]
00002aaad003d000   65292 ----- 00002aaad003d000 000:00000   [ anon ]
00002b3036a4e000       8 rw--- 00002b3036a4e000 000:00000   [ anon ]
00002b3036a50000     668 r-x-- 0000000000000000 008:00003 libocr11.so
00002b3036af7000    1024 ----- 00000000000a7000 008:00003 libocr11.so
00002b3036bf7000      12 rw--- 00000000000a7000 008:00003 libocr11.so
00002b3036bfa000     652 r-x-- 0000000000000000 008:00003 libocrb11.so
00002b3036c9d000    1020 ----- 00000000000a3000 008:00003 libocrb11.so
00002b3036d9c000       8 rw--- 00000000000a2000 008:00003 libocrb11.so
00002b3036d9e000      44 r-x-- 0000000000000000 008:00003 libocrutl11.so
00002b3036da9000    1024 ----- 000000000000b000 008:00003 libocrutl11.so
00002b3036ea9000       4 rw--- 000000000000b000 008:00003 libocrutl11.so
00002b3036eaa000       4 rw--- 00002b3036eaa000 000:00000   [ anon ]
00002b3036eab000    6952 r-x-- 0000000000000000 008:00003 libhasgen11.so
00002b3037575000    1020 ----- 00000000006ca000 008:00003 libhasgen11.so
00002b3037674000     136 rw--- 00000000006c9000 008:00003 libhasgen11.so
00002b3037696000      24 rw--- 00002b3037696000 000:00000   [ anon ]
00002b303769c000   38728 r-x-- 0000000000000000 008:00003 libclntsh.so.11.1
00002b3039c6e000    2044 ----- 00000000025d2000 008:00003 libclntsh.so.11.1
00002b3039e6d000    1624 rw--- 00000000025d1000 008:00003 libclntsh.so.11.1
00002b303a003000     148 rw--- 00002b303a003000 000:00000   [ anon ]
00002b303a028000       8 r-x-- 0000000000000000 008:00003 libskgxn2.so
00002b303a02a000    1020 ----- 0000000000002000 008:00003 libskgxn2.so
00002b303a129000       4 rw--- 0000000000001000 008:00003 libskgxn2.so
00002b303a143000       8 rw--- 00002b303a143000 000:00000   [ anon ]
00002b303a145000     652 r-x-- 0000000000000000 008:00003 libasmclntsh11.so
00002b303a1e8000    1020 ----- 00000000000a3000 008:00003 libasmclntsh11.so
00002b303a2e7000      84 rw--- 00000000000a2000 008:00003 libasmclntsh11.so
00002b303a2fc000     392 r-x-- 0000000000000000 008:00003 libcell11.so
00002b303a35e000    1024 ----- 0000000000062000 008:00003 libcell11.so
00002b303a45e000      40 rw--- 0000000000062000 008:00003 libcell11.so
00002b303a468000       8 rw--- 00002b303a468000 000:00000   [ anon ]
00002b303a46a000     864 r-x-- 0000000000000000 008:00003 libskgxp11.so
00002b303a542000    1024 ----- 00000000000d8000 008:00003 libskgxp11.so
00002b303a642000       8 rw--- 00000000000d8000 008:00003 libskgxp11.so
00002b303a644000    2596 r-x-- 0000000000000000 008:00003 libnnz11.so
00002b303a8cd000    1020 ----- 0000000000289000 008:00003 libnnz11.so
00002b303a9cc000     264 rw--- 0000000000288000 008:00003 libnnz11.so
00002b303aa0e000       8 rw--- 00002b303aa0e000 000:00000   [ anon ]
00002b303aa10000    1284 r-x-- 0000000000000000 008:00003 libgns11.so
00002b303ab51000    1020 ----- 0000000000141000 008:00003 libgns11.so
00002b303ac50000      20 rw--- 0000000000140000 008:00003 libgns11.so
00002b303ac55000       4 rw--- 00002b303ac55000 000:00000   [ anon ]
00002b303ac56000     136 r-x-- 0000000000000000 008:00003 libeons.so
00002b303ac78000    1020 ----- 0000000000022000 008:00003 libeons.so
00002b303ad77000       8 rw--- 0000000000021000 008:00003 libeons.so
00002b303ad79000     104 r-x-- 0000000000000000 008:00003 libonsx.so
00002b303ad93000    1020 ----- 000000000001a000 008:00003 libonsx.so
00002b303ae92000       8 rw--- 0000000000019000 008:00003 libonsx.so
00002b303ae94000       4 rw--- 00002b303ae94000 000:00000   [ anon ]
00002b303ae95000      72 r-x-- 0000000000000000 008:00003 libeonsserver.so
00002b303aea7000    1020 ----- 0000000000012000 008:00003 libeonsserver.so
00002b303afa6000       8 rw--- 0000000000011000 008:00003 libeonsserver.so
00002b303afa8000       4 rw--- 00002b303afa8000 000:00000   [ anon ]
00002b303afa9000       4 r-x-- 0000000000000000 008:00003 libaio.so.1.0.1
00002b303afaa000    2044 ----- 0000000000001000 008:00003 libaio.so.1.0.1
00002b303b1a9000       4 rw--- 0000000000000000 008:00003 libaio.so.1.0.1
00002b303b1aa000     152 rw--- 00002b303b1aa000 000:00000   [ anon ]
00002b303b1e9000      40 r-x-- 0000000000000000 008:00003 libnss_files-2.5.so
00002b303b1f3000    2044 ----- 000000000000a000 008:00003 libnss_files-2.5.so
00002b303b3f2000       4 r---- 0000000000009000 008:00003 libnss_files-2.5.so
00002b303b3f3000       4 rw--- 000000000000a000 008:00003 libnss_files-2.5.so
00002b303b3f4000    4100 rw--- 00002b303b3f4000 000:00000   [ anon ]
00007fff1600c000     140 rwx-- 00007ffffffdc000 000:00000   [ stack ]
ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
mapped: 1726176K    writeable/private: 1423820K    shared: 0K
[root@rac1 ~]#

hpf99

hpf99 发表于 2013-4-3 16:56
00002aaabc004000       4 ----- 00002aaabc004000 000:00000   [ anon ]
00002aaabc005000   32768 rwx- ...

上边两个加在一起

Maclean Liu(刘相兵

writeable/private: 1423820K  ==> 1gb

orarootagent.bin 用来监控 一些network 、等资源的， 用这么多内存必然也有原因




可以通过kill orarootagent.bin进程来释放内存，orarootagent.bin进程被KILL之后会被重新respawned启动。

更多信息可以直接参考 ：

1. How many orarootagent.bin processes should be running in GI environment?

By default there should be two orarootagent.bin.

One is spawned by ohasd and the other by crsd.

root:/> ps -ef |grep oraroot

root 26083426 1 0 Apr 22 - 68:11 /opt/rtl/11.2.0/grid/bin/orarootagent.bin
root 35979384 1 2 Apr 30 - 63:55 /opt/rtl/11.2.0/grid/bin/orarootagent.bin


2. Is it normal to have zombie/defunct processes associated with orarootagent.bin process?

Yes ,there are usually some zombie processes associated with the orarootagent.bin process.
They are spawned when orarootagent.bin process tries to restart HAIP resource.



3. How to check for Zombie processes?

- ps -ef |grep defunct
root 8781978 42795142 0 09:56:29 pts/0 0:00 grep defunct
root 17825906 26083426 2 0:00 <defunct>
root 41222186 26083426 1 0:00 <defunct>
root 41222186 26083426 1 0:00 <defunct>

All of them have the same parent, process id 26083426.

- “ps aux” and look for a Z in the STAT column.
- ps aux | awk '{ print $8 " " $2 }' | grep -w Z
- Also in top output


4. What is a Zombie process?

A zombie is a dead process, i.e, a child process that has completed execution but has not yet been reaped by the parent process.
The reason for zombies is so that the zombie's parent (process) can retrieve the zombie's exit status and resource usage statistics.
The parent process then signals the operating system that it no longer needs the zombie by using one of the wait() system calls.

A Zombie process has no resources (memory , cpu) allocated to it, but it still has an entry in the process tree.


5. Should Zombie processes be removed?

Usually no action is necessary as Zombies don't influence GI or RAC and should clear out eventually.
However, it is possible that under heavy load, many zombies can cause the system to run out of process ID numbers.
In such cases, Zombies should be removed.


6. How to remove Zombie processes ?

Since they are already dead processes, Zombies cannot be killed using "kill -9".

They can be removed by sending a SIGCHLD signal to the parent:
kill -s SIGCHLD <parent_pid>
Ideally this will cause parents to reap their zombie children.

If the parent process still refuses to reap the zombie, the next step is to remove the parent process.
kill -9 <parent_pid>
Killing the parent process causes all of the parent’s children to be adopted by the init process (pid 1), which periodically runs wait() to reap any zombie children.


Please note that killing orarootagent.bin process will not cause an issue , since it is automatically respawned.