Exadata Flash Based Grid Disks

• Table: SYSTEM.MYOBJ
• Size (Gb): 14.8
• Blocks: 1,940,352
• Tablespace Name: USERS
• ASM Disk Group: DATA_CM01
• ASM AU Size: 4MB 

To ASM, Flash-Based Grid Disks are just like normal Grid Disks – they’re chunks of usable storage upon which to create ASM disk groups. In order to configure these, we need to first “DROP FLASHCACHE”, then re-create it (which will allocate some space for Smart Flash Cache), and then create Flash-Based Grid Disks on the storage cell flash cards. See below, and I’ve even provided some syntax errors to show that this is real testing =):

Now let’s create Flash-Based Grid Disks:

Next, we’ll create an ASM disk group on this:

Now we’ll create a tablespace on this ASM disk group:

Then we’ll create MYOBJ_FLASH and store it in FLASH_CM01:

Now we can do our test:

As we can see, the elapsed time was 2.57 seconds, an improvement from the 4.30-second average time with non-Flash Grid Disk storage against the MYOBJ table. The time waited (119,320) was about (.1/.25) =~ 40% that of non-Flash Grid disks, which is a pretty decent improvement. We can summarize this testing to say that Flash-Based Grid Disks will provide improvement, but in our test case we used almost 800GB of storage for our ASM disk group, and while this helped our test, it certainly will have an impact on a largely loaded system with high degrees of concurrency, since we’ve removed this 800GB from Smart Flash Cache.

Summary

IO performance for segments residing in tablespaces part of Grid Disks residing on Flash-Based Grid disks will be better than those residing on normal drives, but be wary of using Flash for Grid Disks. Logically, “best practices” would say to use your flash cards entirely for Smart Flash Cache and allow Oracle to cache as it sees appropriate.