Resizing Exadata Grid Disks and ASM Disk Groups

In this post, we’ll show you how to resize your Exadata storage cell Grid Disks and the ASM disk groups that reside on them. We’ll assume we’ve a storage environment comprised of different ASM disk groups for different application and database purposes, and further, different Grid Disks to support these ASM disk group requirements. Let’s imagine we want to re-baseline and resize everything.
First, we’ll “clean up” storage created from our various test case data, drop tablespaces, drop ASM disk groups, and resize the DATA_CM01 ASM disk group to add the vacated capacity.
We’ll first start with removing the FLASH_CM01 ASM disk group and the Flash-Based Grid Disks, and after re-create FLASHCACHE:

 

 

 

 

Now we’ll drop Grid Disks, but first let’s list them:

 

 

From this, we’ll create a dcli script to drop them – some of the output is below:

 

 

Next we’ll re-create Flash Cache:

 

 

At this point, or cell flash cards are back to the “best practices” configuration and are wholly purposed for Smart Flash Cache. Let’s try to start our database:

 

 

Uh-oh, when we dropped the ASM disk group FLASH_CM01, it didn’t remove this as a required resource for ora.dbm.db. Let’s check dependencies:

 

 

We can see in the above that the ASM dependencies include FLASH_CM01.dg ASM disk group, as well as DATA_CM01, RECO_CM01, NOSSDATA_CM01, AU1MDATA_CM01, and AU8MDATA_CM01.

We can fix this with a series of srvctl commands:

 

 

OK, we’re still having problems, and it’s because we forgot to drop the FLASH_CM01 tablespace. What’s amazing is that with all my focus on doing “Exadata testing” I’ve forgotten some pretty basic Oracle DBA stuff.

Let’s fix this as below:

 

Let’s now move on to the Grid Disks that were created for the various bits of testing above. Below, we’ll drop tables, drop tablespaces (this time let’s remember it), drop ASM disk groups, and drop Grid Disks:

 

 

Then we’ll move on to dropping the ASM disk groups, followed by removing the resource dependencies. I threw in some security issues (with resolution) to keep it real:

 

 

 

 

The failures on the “srvctl disable diskgroup” commands were expected since we dropped the ASM disk groups, this was just a sanity check. Now let’s start the database back up:

 

 

Our DBM database and its instances are up. We’ve got large chunks of space on the cell servers reserved for the testing-related Grid Disks, so let’s get rid of them:

 

 

 

Now we’re going to create “DATA” Grid Disks on the remaining space. We’ll do this using the CREATE GRIDDISK statement in cellcli with a “DATA” prefix.

 

As we can see, it’s complaining on the “create griddisk all prefix=DATA” statement because no available space exists on the Exadata flash cards – recall, we did a “create flashcache all” previously. Once we create Flash Grid Disks, we’ll need to worry about stuff like this.

So let’s try this:

 

 

As we can see, it was unable to create additional Grid Disks and the reason is because we specified the same Grid Disk prefix.

So let’s try with a different prefix:

 

 

We have success. So we need to keep in mind that when we do a “create griddisk all”, we need to make sure to use a different prefix. Of course, we probably should have resized the existing “DATA_%” Grid Disks. So let’s drop the DATA2″ Grid Disks again (output withheld, but it’s like some of the previous examples) and build a script to resize the existing Grid Disks:

 

 

We’ll copy this into a shell script and run it to resize the Grid Disks to 425GB in each case:

 

 

Let’s now check the size of a few of these from cellcli:

 

 

If you recall from previous sections of this document, the size of our DATA_CM01 ASM disk group was 7,372,800 MB. Running this query again, we see the same:

 

 

 

Let’s now resize the ASM disks and rebalance the ASM disk group:

 

 

 

After the rebalance is complete, let’s check space:

 

 

Summary

The steps in this post should help an Exadata Oracle DBA understand the sequence and dependencies for resizing Grid Disks and ASM disk groups.