Oracle数据库数据恢复、性能优化»论坛 › Oracle › Oracle数据库管理 › Exadata flashcache

ligle

22 积分	0 好友	0 主题

发消息

Exadata flashcache

1^#

发表于 2012-1-11 18:57:41 | 查看: 8395| 回复: 17

请教maclean关于exadata的一个小疑问：flashcard、flashcache在Exadata的存储节点上发挥的什么作用呢？望帮忙解惑下...

分享0

收藏0 回复只看该作者道具举报

Maclean Liu(刘相兵

2^#

发表于 2012-1-11 21:20:36

一些资料:

DatabaseSmart Flash Cache
For Linux and Solaris Servers

2012-1-11 21:21 上传
下载附件 (46.46 KB)

•Uses flash disks or cards in database host

•Transparently extends buffer cache

•Eliminates most I/Os and speeds query performance

•Lower power required with high ROI

Exadata Smart FlashCache
Extreme Performance for OLTP

2012-1-11 21:21 上传
下载附件 (103.56 KB)

•Full rack has 5 TB of Smart Flash Cache

•Can process over 1 million IOs per second

•50 GB/sec query throughput on uncompressed data

•5x more I/Os than 1000 Disk Enterprise Storage Array

回复只看该作者道具举报

ligle

3^#

发表于 2012-1-11 21:29:51

5x more I/Os than 1000 Disk Enterprise Storage Array 中 5x 是个什么概念？

可以把这份资料share下吗？谢谢！

回复只看该作者道具举报

ligle

4^#

发表于 2012-1-11 21:35:02

另外,现在有个疑惑是:
这有3个存储cell节点，现在只有存储节点cell3安装了flash cache，而其他两个存储cell没有安装flashcache，这是正常的吗？

回复只看该作者道具举报

Maclean Liu(刘相兵

5^#

发表于 2012-1-11 21:49:34

repeat this action plan on cells:

action plan:

cellcli

list flashcache
list flashcache detail
LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
list physicaldisk where status = "not present" detail
list alerthistory

回复只看该作者道具举报

ligle

6^#

发表于 2012-1-11 21:55:02

cell1:
CellCLI> list flashcache

CellCLI> list flashcache detail

CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
      [1:0:0:0]    5080020000f1552FMOD0 normal
      [1:0:1:0]    5080020000f1552FMOD1 normal
      [1:0:2:0]    5080020000f1552FMOD2 normal
      [1:0:3:0]    5080020000f1552FMOD3 normal
      [2:0:0:0]    5080020000efa0aFMOD0 normal
      [2:0:1:0]    5080020000efa0aFMOD1 normal
      [2:0:2:0]    5080020000efa0aFMOD2 normal
      [2:0:3:0]    5080020000efa0aFMOD3 normal
      [3:0:0:0]    5080020000d830aFMOD0 normal
      [3:0:1:0]    5080020000d830aFMOD1 normal
      [3:0:2:0]    5080020000d830aFMOD2 normal
      [3:0:3:0]    5080020000d830aFMOD3 normal
      [4:0:0:0]    5080020000d889eFMOD0 normal
      [4:0:1:0]    5080020000d889eFMOD1 normal
      [4:0:2:0]    5080020000d889eFMOD2 normal
      [4:0:3:0]    5080020000d889eFMOD3 normal

CellCLI> list physicaldisk where status = "not present" detail

CellCLI> list alerthistory
      9_1    2011-07-09T20:04:30+08:00    warning       "A generic sensor has detected an error.  Component Name    : /SYS/ACPI  Trap Additional Info : S0/G0: working"

cell2:
CellCLI> list flashcache

CellCLI> list flashcache detail

CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
      [1:0:0:0]    5080020000d8604FMOD0 normal
      [1:0:1:0]    5080020000d8604FMOD1 normal
      [1:0:2:0]    5080020000d8604FMOD2 normal
      [1:0:3:0]    5080020000d8604FMOD3 normal
      [2:0:0:0]    5080020000f1486FMOD0 normal
      [2:0:1:0]    5080020000f1486FMOD1 normal
      [2:0:2:0]    5080020000f1486FMOD2 normal
      [2:0:3:0]    5080020000f1486FMOD3 normal
      [3:0:0:0]    5080020000d8318FMOD0 normal
      [3:0:1:0]    5080020000d8318FMOD1 normal
      [3:0:2:0]    5080020000d8318FMOD2 normal
      [3:0:3:0]    5080020000d8318FMOD3 normal
      [4:0:0:0]    5080020000efac2FMOD0 normal
      [4:0:1:0]    5080020000efac2FMOD1 normal
      [4:0:2:0]    5080020000efac2FMOD2 normal
      [4:0:3:0]    5080020000efac2FMOD3 normal

CellCLI> list physicaldisk where status = "not present" detail

CellCLI> list alerthistory
      8_1    2011-07-09T12:36:26+08:00    warning       "A generic sensor has detected an error.  Component Name    : /SYS/ACPI  Trap Additional Info : S0/G0: working"
      17_1 2011-12-01T01:57:22+08:00    critical       "File system "/" is 80% full, which is above the 80% threshold. Accelerated space reclamation has started.  This alert will be cleared when file system "/" becomes less than 75% full."

回复只看该作者道具举报

ligle

7^#

发表于 2012-1-11 21:55:21

cell03:
CellCLI> list flashcache
      dm01cel03_FLASHCACHE critical

CellCLI> list flashcache detail
      name:                dm01cel03_FLASHCACHE
      cellDisk:
      creationTime:          2011-02-16T20:38:16+08:00
      degradedCelldisks:    FD_13_dm01cel03,FD_04_dm01cel03,FD_05_dm01cel03,FD_09_dm01cel03,FD_07_dm01cel03,FD_01_dm01cel03,FD_08_dm01cel03,FD_12_dm01cel03,FD_15_dm01cel03,FD_11_dm01cel03,FD_00_dm01cel03,FD_06_dm01cel03,FD_14_dm01cel03,FD_03_dm01cel03,FD_10_dm01cel03,FD_02_dm01cel03
      effectiveCacheSize:    0
      id:                   143c2a31-3769-4a1e-a7a4-7becc1d4e786
      size:                365.25G
      status:                critical

CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
      [3:0:0:0]    5080020000ef72eFMOD0 not present
      [3:0:1:0]    5080020000ef72eFMOD1 not present
      [3:0:2:0]    5080020000ef72eFMOD2 not present
      [3:0:3:0]    5080020000ef72eFMOD3 not present
      [4:0:0:0]    5080020000ef748FMOD0 not present
      [4:0:1:0]    5080020000ef748FMOD1 not present
      [4:0:2:0]    5080020000ef748FMOD2 not present
      [4:0:3:0]    5080020000ef748FMOD3 not present

CellCLI> list physicaldisk where status = "not present" detail
      name:                [3:0:0:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef72eFMOD0
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 5; FDOM: 0"
      status:                not present

      name:                [3:0:1:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef72eFMOD1
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 5; FDOM: 1"
      status:                not present

      name:                [3:0:2:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef72eFMOD2
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 5; FDOM: 2"
      status:                not present

      name:                [3:0:3:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef72eFMOD3
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 5; FDOM: 3"
      status:                not present

      name:                [4:0:0:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef748FMOD0
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 2; FDOM: 0"
      status:                not present

      name:                [4:0:1:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef748FMOD1
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 2; FDOM: 1"
      status:                not present

      name:                [4:0:2:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef748FMOD2
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 2; FDOM: 2"
      status:                not present

      name:                [4:0:3:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef748FMOD3
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 2; FDOM: 3"
      status:                not present

回复只看该作者道具举报

ligle

8^#

发表于 2012-1-11 21:57:45

CellCLI> list alerthistory
      11_1 2011-04-25T07:13:26+08:00    warning       "A generic sensor has detected an error.  Component Name    : /SYS/ACPI  Trap Additional Info : S0/G0: working"
      12_1 2011-05-11T03:51:24+08:00    critical       "Flash disk status changed to poor performance.  Status                   : POOR PERFORMANCE  Manufacturer             : MARVELL  Model Number             : SD88SA02  Size                      : 23G  Serial Number             : 5080020000ef772FMOD1  Firmware                   : D20Y  Slot Number                : PCI Slot: 1; FDOM: 1  Cell Disk                : FD_01_dm01cel03  Grid Disk                : No grid disks exist  Flash Cache                : Present  Reason for poor performance : no IOs completed in TIMEOUT seconds, even though there were outstanding IOs"
      12_2 2011-05-11T03:54:40+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef772FMOD1  Firmware    : D20Y  Slot Number : PCI Slot: 1; FDOM: 1  Cell Disk    : FD_01_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      14_1 2011-05-11T03:54:38+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef772FMOD0  Firmware    : D20Y  Slot Number : PCI Slot: 1; FDOM: 0  Cell Disk    : FD_00_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      15_1 2011-05-11T03:54:42+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef772FMOD2  Firmware    : D20Y  Slot Number : PCI Slot: 1; FDOM: 2  Cell Disk    : FD_02_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      16_1 2011-05-11T03:54:44+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef772FMOD3  Firmware    : D20Y  Slot Number : PCI Slot: 1; FDOM: 3  Cell Disk    : FD_03_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      17_1 2011-05-11T03:54:45+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef6feFMOD0  Firmware    : D20Y  Slot Number : PCI Slot: 4; FDOM: 0  Cell Disk    : FD_08_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      18_1 2011-05-11T03:54:46+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef6feFMOD1  Firmware    : D20Y  Slot Number : PCI Slot: 4; FDOM: 1  Cell Disk    : FD_09_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      19_1 2011-05-11T03:54:48+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef6feFMOD2  Firmware    : D20Y  Slot Number : PCI Slot: 4; FDOM: 2  Cell Disk    : FD_10_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      20_1 2011-05-11T03:54:49+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef6feFMOD3  Firmware    : D20Y  Slot Number : PCI Slot: 4; FDOM: 3  Cell Disk    : FD_11_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      28    2012-01-11T17:41:33+08:00    info          "BBU on disk contoller at adapter 0 is going into a learn cycle. All Logical Volumes on harddisks will go into WriteThrough caching mode. Write Throughput will be lower."
      29    2012-01-11T17:42:03+08:00    critical       "Could not sync the following change in disk state with Cellsrv.  Lost luns :  2_0 2_1 2_2 2_3 5_0 5_1 5_2 5_3"
      30_1 2012-01-11T17:43:14+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef748FMOD0  Firmware    : D20Y  Slot Number : PCI Slot: 2; FDOM: 0  Cell Disk    : FD_04_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      31_1 2012-01-11T17:43:15+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef748FMOD1  Firmware    : D20Y  Slot Number : PCI Slot: 2; FDOM: 1  Cell Disk    : FD_05_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      32_1 2012-01-11T17:43:16+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef748FMOD2  Firmware    : D20Y  Slot Number : PCI Slot: 2; FDOM: 2  Cell Disk    : FD_06_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      33_1 2012-01-11T17:43:17+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef748FMOD3  Firmware    : D20Y  Slot Number : PCI Slot: 2; FDOM: 3  Cell Disk    : FD_07_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      34_1 2012-01-11T17:43:17+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef72eFMOD0  Firmware    : D20Y  Slot Number : PCI Slot: 5; FDOM: 0  Cell Disk    : FD_12_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      35_1 2012-01-11T17:43:18+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef72eFMOD1  Firmware    : D20Y  Slot Number : PCI Slot: 5; FDOM: 1  Cell Disk    : FD_13_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      36_1 2012-01-11T17:43:18+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef72eFMOD2  Firmware    : D20Y  Slot Number : PCI Slot: 5; FDOM: 2  Cell Disk    : FD_14_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"
      37_1 2012-01-11T17:43:21+08:00    critical       "Flash disk removed.  Status       : NOT PRESENT  Manufacturer  : MARVELL  Model Number  : SD88SA02  Size       : 23G  Serial Number : 5080020000ef72eFMOD3  Firmware    : D20Y  Slot Number : PCI Slot: 5; FDOM: 3  Cell Disk    : FD_15_dm01cel03  Grid Disk    : No grid disks exist  Flash Cache : Present"

回复只看该作者道具举报

Maclean Liu(刘相兵

9^#

发表于 2012-1-11 22:00:35

CellCLI> LIST PHYSICALDISK WHERE DISKTYPE=flashdisk
      [3:0:0:0]    5080020000ef72eFMOD0 not present
      [3:0:1:0]    5080020000ef72eFMOD1 not present
      [3:0:2:0]    5080020000ef72eFMOD2 not present
      [3:0:3:0]    5080020000ef72eFMOD3 not present
      [4:0:0:0]    5080020000ef748FMOD0 not present
      [4:0:1:0]    5080020000ef748FMOD1 not present
      [4:0:2:0]    5080020000ef748FMOD2 not present
      [4:0:3:0]    5080020000ef748FMOD3 not present

CellCLI> list physicaldisk where status = "not present" detail
      name:                [3:0:0:0]
      diskType:             FlashDisk
      luns:
      makeModel:             "MARVELL SD88SA02"
      notPresentSince:       2012-01-11T17:41:33+08:00
      physicalFirmware:    D20Y
      physicalInsertTime:    2010-12-24T06:46:13+08:00
      physicalInterface:    sas
      physicalSerial:       5080020000ef72eFMOD0
      physicalSize:          22.8880615234375G
      slotNumber:          "PCI Slot: 5; FDOM: 0"
      status:                not present

cell3 上的flashdisk 显示状态为"not present" ,

notPresentSince:       2012-01-11T17:41:33+08:00

2012-01-11 17:41:33+08:00 该时间点是否有过什么操作？

建议你将该cell03的信息提交SR ，可能存在bad flash disk

回复只看该作者道具举报

ligle

10^#

发表于 2012-1-11 22:01:16

需要说明一点：
现在Exadata storage cell3节点的flashcard missing了，今天晚上要准备打补丁解决该问题。所以就不用太关心cell3上的报错了，因为今天晚上就要解决这个问题了。

另外，主要就是想明白，为什么cell1 和cell2上没有falshcache呢？这正常吗？

回复只看该作者道具举报

ligle

11^#

发表于 2012-1-11 22:06:33

回复 9# 的帖子

今天没做任何的操作，我是今天下午18点左右发现的，早在一个月前，有两块flashcard missing了，今天我发现剩余的两块flashcards 也missing了，我也很奇怪。这个现在已诊断出问题了，今天晚上要通过Oracle提供的fix包解决该问题。

回复只看该作者道具举报

vicky72

12^#

发表于 2012-1-11 22:24:28

能贴一下这个的输出结果吗？在cell1，cell2上
list celldisk
如果FD 开头的盘是normal状态就ok
就是没有执行建立flashcache 的命令

create flashcache all
看是否报错。因为看你前面的输出，flash model的状态应该正常

[ 本帖最后由 vicky72 于 2012-1-11 22:25 编辑 ]

回复只看该作者道具举报

Maclean Liu(刘相兵

13^#

发表于 2012-1-11 22:28:40

回复 3# 的帖子

你要的文档已经上传：

http://t.askmaclean.com/thread-232-1-1.html

回复只看该作者道具举报

ligle

14^#

发表于 2012-1-11 22:28:53

在cell1和cell2上执行list celldisk 显示的结果中FD开头的磁盘状态都是normal。

这是生产库，也不敢轻易执行create flashcache all命令。
如果可以create flashcache的话，前提应该是相应的storage cell节点安装了flash cards吧？如果是的话，那如果确实这个两个storage cell是否安装了flashcards呢？

回复只看该作者道具举报

vicky72

15^#

发表于 2012-1-11 22:33:48

你FD的状态都normal了，那说明flash card状态正常，就是安装时没有建立flashcache，这个我们安装的时候都会建立的，为什么没建立，或者被删除了？

回复只看该作者道具举报

ligle

16^#

发表于 2012-1-11 22:33:55

回复 13# 的帖子

thank you!

回复只看该作者道具举报

ligle

17^#

发表于 2012-1-11 22:39:59

回复 15# 的帖子

这个只能问当初实施的人了，汗~，找时间问下当初实施的人，究竟是遗漏了还是别的原因？
非常感谢！

回复只看该作者道具举报

syhnd

18^#

发表于 2012-3-20 21:59:28

当你搞清楚了flash cache的作用，你就明白了并不是所有的cell层都非要创建flash cache.

flash cache对于OLTP系统而言就非常重要；而对于DW系统而言，都是些非常大的表，则它的功效就没有smart scan重要了，所以如果你的系统是纯DW系统，则没有flash cache就很正常。

回复只看该作者道具举报

返回列表

		自动登录	找回密码
密码			注册