asm diskgroup新增disk后 down

sunstar3000

redhat5.4_64  11.2.0.3 单机asm + udev


SQL> select group_number,disk_number,name,mount_status,header_status,state,total_mb,free_mb,path from v$asm_disk;

GROUP_NUMBER DISK_NUMBER NAME                           MOUNT_S HEADER_STATU STATE      TOTAL_MB    FREE_MB PATH
------------ ----------- ------------------------------ ------- ------------ -------- ---------- ---------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
           0           0                                CLOSED  CANDIDATE    NORMAL            0          0 /dev/raw/raw2
           1           0 DDD_0000                       CACHED  MEMBER       NORMAL       583435     136459 /dev/raw/raw1

SQL> alter diskgroup DDD add disk '/dev/raw/raw2';

Diskgroup altered.

SQL> select group_number,disk_number,name,mount_status,header_status,state,total_mb,free_mb,path from v$asm_disk;

GROUP_NUMBER DISK_NUMBER NAME                           MOUNT_S HEADER_STATU STATE      TOTAL_MB    FREE_MB PATH
------------ ----------- ------------------------------ ------- ------------ -------- ---------- ---------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
           1           0 DDD_0000                       CACHED  MEMBER       NORMAL       583435     137235 /dev/raw/raw1
           1           1 DDD_0001                       CACHED  MEMBER       NORMAL       154783     154004 /dev/raw/raw2

SQL> select group_number gno,name,block_size,state,type,total_mb,free_mb from v$asm_diskgroup;

       GNO NAME                           BLOCK_SIZE STATE       TYPE     TOTAL_MB    FREE_MB
---------- ------------------------------ ---------- ----------- ------ ---------- ----------
         1 DDD                                  4096 MOUNTED     EXTERN     738218     291239


SQL> select group_number,operation,state,est_work,sofar,est_rate,est_minutes from v$asm_operation;

GROUP_NUMBER OPERA STAT   EST_WORK      SOFAR   EST_RATE EST_MINUTES
------------ ----- ---- ---------- ---------- ---------- -----------
           1 REBAL RUN       77482       7320       2909          24


SQL> select group_number,operation,state,est_work,sofar,est_rate,est_minutes from v$asm_operation;

no rows selected


数据库此时down



SQL> select group_number,disk_number,name,mount_status,header_status,state,total_mb,free_mb,path from v$asm_disk;

GROUP_NUMBER DISK_NUMBER NAME                           MOUNT_S HEADER_STATU STATE      TOTAL_MB    FREE_MB PATH
------------ ----------- ------------------------------ ------- ------------ -------- ---------- ---------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
           0           1                                CLOSED  MEMBER       NORMAL            0          0 /dev/raw/raw1
           0           0                                CLOSED  MEMBER       NORMAL            0          0 /dev/raw/raw2



SQL> alter diskgroup ddd mount;

Diskgroup altered.

SQL> select group_number,disk_number,name,mount_status,header_status,state,total_mb,free_mb,path from v$asm_disk;

GROUP_NUMBER DISK_NUMBER NAME                           MOUNT_S HEADER_STATU STATE      TOTAL_MB    FREE_MB PATH
------------ ----------- ------------------------------ ------- ------------ -------- ---------- ---------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
           1           1 DDD_0001                       CACHED  MEMBER       NORMAL       154783      77307 /dev/raw/raw2
           1           0 DDD_0000                       CACHED  MEMBER       NORMAL       583435     213932 /dev/raw/raw1



[oracle@asm ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Tue Aug 21 01:16:49 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DDD/jzhj/spfilejzhj.ora'
ORA-17503: ksfdopn:2 Failed to open file +DDD/jzhj/spfilejzhj.ora
ORA-15056: additional error message
ORA-17503: ksfdopn:2 Failed to open file +DDD/jzhj/spfilejzhj.ora
ORA-15001: diskgroup "DDD" does not exist or is not mounted
ORA-06512: at line 4



SQL> select group_number gno,name,block_size,state,type,total_mb,free_mb from v$asm_diskgroup;

       GNO NAME                           BLOCK_SIZE STATE       TYPE     TOTAL_MB    FREE_MB
---------- ------------------------------ ---------- ----------- ------ ---------- ----------
         0 DDD                                  4096 DISMOUNTED                  0          0

[ 本帖最后由 sunstar3000 于 2012-8-20 18:10 编辑 ]

Maclean Liu(刘相兵

ODM DATA:

11.2.0.3.0 + ASM

NOTE: starting rebalance of group 1/0x3858d29f (DDD) at power 1
SUCCESS: alter diskgroup DDD add disk '/dev/raw/raw2'
Starting background process ARB0
Tue Aug 21 00:42:10 2012
ARB0 started with pid=21, OS id=22276
NOTE: assigning ARB0 to group 1/0x3858d29f (DDD) with 1 parallel I/O
NOTE: Attempting voting file refresh on diskgroup DDD
Tue Aug 21 01:09:20 2012
Errors in file /home/u01/app/grid/diag/asm/+asm/+ASM/trace/+ASM_arb0_22276.trc:
ORA-27072: File I/O error
Additional information: 4
Additional information: 1398276112
WARNING: Read Failed. group:1 disk:0 AU:682752 offset:8192 size:4096
WARNING: cache failed reading from group=1(DDD) dsk=0 blk=1538 count=1 from disk= 0(DDD_0000) kfkist=0x20 status=0x02 file=kfc.c line=11555
ERROR: cache failed to read group=1(DDD) dsk=0 blk=1538 from disk(s): 0(DDD_0000)
ORA-15080: synchronous I/O operation to a disk failed
NOTE: cache initiating offline of disk 0 group DDD
NOTE: process _arb0_+asm (22276) initiating offline of disk 0.3915915888 (DDD_0000) with mask 0x7e in group 1
WARNING: Disk 0 (DDD_0000) in group 1 in mode 0x7f is now being taken offline on ASM inst 1
NOTE: initiating PST update: grp = 1, dsk = 0/0xe9682270, mask = 0x6a, op = clear
Tue Aug 21 01:09:20 2012
GMON updating disk modes for group 1 at 8 for pid 21, osid 22276
ERROR: Disk 0 cannot be offlined, since diskgroup has external redundancy.
ERROR: too many offline disks in PST (grp 1)
Tue Aug 21 01:09:21 2012
NOTE: cache dismounting (not clean) group 1/0x3858D29F (DDD)





*** 2012-08-21 01:09:20.907
ORA-27072: File I/O error
Additional information: 4
Additional information: 1398276112
WARNING: Read Failed. group:1 disk:0 AU:682752 offset:8192 size:4096
path:/dev/raw/raw1
         incarnation:0xe9682270 synchronous result:'I/O error'
         subsys:System iop:0x2b3998b5d000 bufp:0x635dc000 osderr:0x0 osderr1:0x0
WARNING: cache failed reading from group=1(DDD) dsk=0 blk=1538 count=1 from disk= 0(DDD_0000) kfkist=0x20 status=0x02 file=kfc.c line=11555
ERROR: cache failed to read group=1(DDD) dsk=0 blk=1538 from disk(s): 0(DDD_0000)
CE: (0x0x637ce178) group=1 (DDD) dsk=0 blk=1538
    hashFlags=0x0000 lid=0x0002 lruFlags=0x0000 bastCount=1
    mirror=0
    flags_kfcpba=0x49 copies=1 blockIndex=2 AUindex=0 AUcount=1 loctr fcn=0.0
    copy #0:  disk=0  au=682752 flags=01
BH: (0x0x6379df60) bnum=4073 type=reading state=reading chgSt=not modifying pageIn=current
    flags=0x00000000 pinmode=excl lockmode=excl bf=0x635dc000
    kfbh_kfcbh.fcn_kfbh = 0.0 lowAba=0.0 highAba=0.0
    last kfcbInitSlot return code=null chgCount=0 cpkt lnk is null ralFlags=0x00000000
    PINS:
    (kfcbps) pin=29520 get by kfd.c line 25320 mode=excl
             dsk=0 blk=1538 status=pinned
             flags=0x40000000 flags2=0x00000000
             class=1400 type=ALLOCTBL stateWanted=current
             bastCount=1 waitStatus=0x00000000 relocCount=0
             scanBastCount=0 scanBxid=0 scanSkipCode=0
             last released by kfd.c 25313
ORA-15080: synchronous I/O operation to a disk failed
----- Abridged Call Stack Trace -----
ksedsts()+461<-kfcReadBlk()+1934<-kfcReadBuffer()+2329<-kfcGet0()+8441<-kfcGet1Priv()+1532<-kfcGetPriv()+900<-kfdAtGetPriv()+397<-kfdRebalZones()+4016<-kfdExecute()+1294<-kfgbRebalExecute()+874<-kfgbDriver()+1899<-ksbabs()+771<-kfgbRun()+838<-ksbrdp()+971<-opirip()+623
<-opidrv()+603<-sou2o()+103<-opimai_real()+266<-ssthrdmain()+252<-main()+201<-__libc_start_main()+244<-_start()+36
----- End of Abridged Call Stack Trace -----



/dev/raw/raw1 IO 失败造成DDD DISKGROUP 过多ASM DISK OFFLINE，造成DDD DISKGROUP 强制DISMOUNT


该问题应当与你的LUN的可用性有问题

sunstar3000

感谢版主解答，LUN的可用性有问题 可以理解为块有损坏吗?这个问题有解决的可能性吗。

Read Failed. group:1 disk:0 AU:682752,raw1最大为583435，AU为682752这个是什么原因

xifenfei

WARNING: Read Failed. group:1 disk:0 AU:682752 offset:8192 size:4096

表示读到682752 这个au的时候，出问题，不是说你的au总数

oracn

最近在搞asm目前正在测试