4 Administering Oracle ASM Disk Groups

Using the CREATE DISKGROUP SQL Statement

The CREATE DISKGROUP SQL statement is used to create disk groups. When creating a disk group, you:

• Assign a unique name to the disk group.

  The specified disk group name is not case sensitive and is always converted to uppercase when stored internally.

  Note:

  Oracle does not recommend using identifiers for database object names that must be quoted. While these quoted identifiers may be valid as names in the SQL CREATE statement, such as CREATE DISKGROUP "1DATA", the names may not be valid when using other tools that manage the database object.

• Specify the redundancy level of the disk group.

  For Oracle ASM to mirror files, specify the redundancy level as NORMAL REDUNDANCY (2-way mirroring by default for most file types) or HIGH REDUNDANCY (3-way mirroring for all files). Specify EXTERNAL REDUNDANCY if you do not want mirroring by Oracle ASM. For example, you might choose EXTERNAL REDUNDANCY to use storage array protection features.

  After a disk group is created, you cannot alter the redundancy level of the disk group. To change the redundancy level, you must create another disk group with the appropriate redundancy and then move the files to the new disk group.

  Oracle recommends that you create failure groups of equal size to maintain space balance and even distribution of mirror data.

  For more information about redundancy levels, refer to "Mirroring, Redundancy, and Failure Group Options".

• Specify the disks that are to be formatted as Oracle ASM disks belonging to the disk group.

  The disks can be specified using operating system dependent wildcard characters in search strings that Oracle ASM then uses to find the disks. You can specify names for the disks with the NAME clause or use the system-generated names.

• Optionally specify the disks as belonging to specific failure groups.

  For information about failure groups, refer to "Understanding Oracle ASM Concepts" and "Mirroring, Redundancy, and Failure Group Options".

• Optionally specify the type of failure group.

  For information about QUORUM and REGULAR failure groups, refer to "Oracle Cluster Registry and Voting Files in Oracle ASM Disk Groups".

• Optionally specify disk group attributes, such as software compatibility or allocation unit size.

Oracle ASM programmatically determines the size of each disk. If for some reason this is not possible, or to restrict the amount of space used on a disk, you can specify a SIZE clause for each disk. Oracle ASM creates operating system–independent names for the disks in a disk group that you can use to reference the disks in other SQL statements. Optionally, you can provide your own name for a disk using the NAME clause. Disk names are available in the V$ASM_DISK view.

The Oracle ASM instance ensures that any disk in a newly created disk group is addressable and is not currently a member of another disk group. You must use FORCE only when adding a disk that was dropped with FORCE. If a disk is dropped with NOFORCE, then you can add it with NOFORCE. For example, a disk might have failed and was dropped from its disk group. After the disk is repaired, it is no longer part of any disk group, but Oracle ASM still recognizes that the disk had been a member of a disk group. You must use the FORCE flag to include the disk in a new disk group. In addition, the disk must be addressable, and the original disk group must not be mounted. Otherwise, the operation fails.

The CREATE DISKGROUP statement mounts the disk group for the first time, and adds the disk group name to the ASM_DISKGROUPS initialization parameter if a server parameter file is being used. If a text initialization parameter file is being used and you want the disk group to be automatically mounted at instance startup, then you must remember to add the disk group name to the ASM_DISKGROUPS initialization parameter before you shut down and restart the Oracle ASM instance. You can also create disk groups with Oracle Enterprise Manager. Refer to "Creating Disk Groups".

Example: Creating a Disk Group

The following examples assume that the ASM_DISKSTRING initialization parameter is set to the '/devices/*' string. Oracle ASM disk discovery identifies disks in the /devices directory, including the following disks:

Controller 1:

The SQL statement in Example 4-1 creates a disk group named data with normal redundancy consisting of two failure groups controller1 or controller2 with four disks in each failure group. The data disk group is typically used to store database data files.

CREATE DISKGROUP data NORMAL REDUNDANCY
  FAILGROUP controller1 DISK
    '/devices/diska1' NAME diska1,
    '/devices/diska2' NAME diska2,
    '/devices/diska3' NAME diska3,
    '/devices/diska4' NAME diska4
  FAILGROUP controller2 DISK
    '/devices/diskb1' NAME diskb1,
    '/devices/diskb2' NAME diskb2,
    '/devices/diskb3' NAME diskb3,
    '/devices/diskb4' NAME diskb4
  ATTRIBUTE 'au_size'='4M',
    'compatible.asm' = '11.2', 
    'compatible.rdbms' = '11.2',
    'compatible.advm' = '11.2';

In Example 4-1, the NAME clauses enable you to explicitly assign names to the disks rather than the default system-generated names. The system-generated names are in the form diskgroup_nnnn, where nnnn is the disk number for the disk in the disk group. For ASMLib disks, the disk name defaults to the ASMLib name that is the user label of the disk; for example, mydisk is the default Oracle ASM disk name for ORCL:mydisk.

When creating the disk group in Example 4-1, the values of following disk group attributes were explicitly set:

• AU_SIZE

  Specifies the size of the allocation unit for the disk group. For information about allocation unit size and extents, see "Extents".

  You can view the value of the AU_SIZE disk group attribute in the ALLOCATION_UNIT_SIZE column of the V$ASM_DISKGROUP view.

• COMPATIBLE.ASM

  Determines the minimum software version for any Oracle ASM instance that uses a disk group. For information about the COMPATIBLE.ASM attribute, see "COMPATIBLE.ASM".

• COMPATIBLE.RDBMS

  Determines the minimum software version for any database instance that uses a disk group. For information about the COMPATIBLE.RDBMS attribute, see "COMPATIBLE.RDBMS".

• COMPATIBLE.ADVM

  Determines whether the disk group can contain Oracle ASM volumes. For information about the COMPATIBLE.ADVM attribute, see "COMPATIBLE.ADVM".

In Example 4-2, the fra disk group (typically created for the fast recovery area) is created with the default disk group attribute values. Names are not specified for the Oracle ASM disks and failure groups are not explicitly specified. This example assumes that diskc1 through diskc9 are present in the /devices directory.

CREATE DISKGROUP fra NORMAL REDUNDANCY
  DISK '/devices/diskc*';

Creating Disk Groups for a New Oracle Installation

This section describes the basic procedure to follow when creating disk groups during a new installation of Oracle Restart and Oracle Database. This information also applies to an Oracle Grid Infrastructure installation.

The procedure assumes that the data disk group is used for the storage of the database data files and the fra disk group is used for storage of the fast recovery area files. Detailed information about installation with Oracle Universal Installer (OUI) and database creation with Database Configuration Assistant (DBCA) is available in the installation guides for your specific operating system.

1. Install Oracle Restart with OUI, following the screen prompts.

   During the installation, create the data disk group for storing database files such as the data and control files.

   This OUI process is similar to creating a disk group with Oracle ASM Configuration Assistant (ASMCA). For information about using ASMCA to create disk groups, see "Managing Disk Groups with Oracle ASM Configuration Assistant".

   Note that the data disk group is the disk group used for storing Oracle Cluster Registry (OCR) and voting files in an Oracle Grid Infrastructure installation. See "Oracle Cluster Registry and Voting Files in Oracle ASM Disk Groups"

2. After Oracle Restart is installed, use ASMCA to create the fra disk group for storing the fast recovery area files.

   Create the fra disk group to hold the fast recovery area files.

   At this time, you can also update the data disk group if necessary. For information about using ASMCA to create or alter disk groups, see "Managing Disk Groups with Oracle ASM Configuration Assistant".

   You can also create the fra disk group with SQL*Plus or ASMCMD commands run from the Oracle Restart home. For information, see "Using the CREATE DISKGROUP SQL Statement" and "mkdg".

   See Also:

   • Oracle Database Backup and Recovery User's Guide for information about configuring the fast recovery area

   • Oracle Database Administrator's Guide for information about specifying a fast recovery area

3. Install the Oracle Database software with OUI, following the screen prompts.

4. After the database software has been installed, run DBCA to create a database, following the screen prompts.

   During the creation of the database, make the appropriate selections to use Oracle ASM for storage of data files and fast recovery area files. When prompted:

   • Store database data files in the data disk group

   • Store fast recovery area files in the fra disk group

Specifying the Allocation Unit Size

Oracle recommends that the allocation unit (AU) size for a disk group be set to 4 megabytes (MB). In addition to this AU size recommendation, the operating system (OS) I/O size should be set to the largest possible size.

Some benefits of a 4 MB allocation unit are:

• Increased I/O through the I/O subsystem if the I/O size is increased to the AU size.

• Reduced SGA size to manage the extent maps in the database instance.

• Faster datafile initialization if the I/O size is increased to the AU size.

• Increased file size limits.

• Reduced database open time.

The allocation unit size is specified with the disk group attribute AU_SIZE. The AU size cannot be altered after a disk group is created. Example 4-1 shows how the AU_SIZE is specified with the CREATE DISKGROUP SQL statement.

Specifying the Sector Size for Drives

You can use the optional SECTOR_SIZE disk group attribute with the CREATE DISKGROUP SQL statement to specify disks with the sector size set to the value of SECTOR_SIZE for the disk group. Oracle ASM provides support for 4 KB sector disk drives without negatively affecting performance. The SECTOR_SIZE disk group attribute can be set only during disk group creation.

The values for SECTOR_SIZE can be set to 512, 4096, or 4K if the disks support those values. The default value is platform dependent. The COMPATIBLE.ASM and COMPATIBLE.RDBMS disk group attributes must be set to 11.2 or higher to set the sector size to a value other than the default value.

The following validations apply to the sector size disk group attribute.

• Oracle ASM prevents disks of different sector sizes from being added to the same disk group. This validation occurs during CREATE DISKGROUP, ALTER DISKGROUP ADD DISK, and ALTER DISKGROUP MOUNT operations.

• If the SECTOR_SIZE attribute is explicitly specified when creating a disk group, then Oracle ASM attempts to verify that all disks discovered through disk search strings have a sector size equal to the specified value. If one or more disks were found to have a sector size different from the specified value, or if Oracle ASM was not able to verify a disk sector size, then the create operation fails.

  Oracle ASM also attempts to verify disk sector size during the mount operation and the operation fails if one or more disks have a sector size different than the value of the SECTOR_SIZE attribute.

• If the SECTOR_SIZE attribute is not specified when creating a disk group and Oracle ASM can verify that all discovered disks have the same sector value, then that value is assumed for the disk group sector size that is created. If the disks have different sector sizes, the create operation fails.

• When new disks are added to an existing disk group using the ALTER DISKGROUP .. ADD DISK SQL statement, you must ensure that the new disks to be added have the same value as the SECTOR_SIZE disk group attribute. If the new disks have different sector sizes, the alter operation fails.

• You can determine the sector size value that has either been assumed or explicitly set for a successful disk group creation by querying the V$ASM_ATTRIBUTE view or run the ASMCMD lsattr command. You can also query the SECTOR_SIZE column in the V$ASM_DISKGROUP view.

  SQL> SELECT name, value FROM V$ASM_ATTRIBUTE 
       WHERE name = 'sector_size' AND group_number = 1;
  NAME                        VALUE
  --------------------------- -----------------------
  sector_size                 512
  
  SQL> SELECT group_number, sector_size FROM V$ASM_DISKGROUP 
       WHERE group_number = 1;
  
  GROUP_NUMBER SECTOR_SIZE
  ------------ -----------
             1         512
  

• Not all disks support all of the possible SECTOR_SIZE values. The sector size attribute setting must be compatible with the physical hardware.

As shown in Example 4-3, you can use the SECTOR_SIZE attribute with the CREATE DISKGROUP SQL statement to specify the sector size of the disk drive on which the Oracle ASM disk group is located.

CREATE DISKGROUP data NORMAL REDUNDANCY
FAILGROUP controller1 DISK
'/devices/diska1',
'/devices/diska2',
'/devices/diska3',
'/devices/diska4'
FAILGROUP controller2 DISK
'/devices/diskb1',
'/devices/diskb2',
'/devices/diskb3',
'/devices/diskb4'
ATTRIBUTE 'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2',
          'sector_size'='4096';

Oracle Cluster Registry and Voting Files in Oracle ASM Disk Groups

You can store Oracle Cluster Registry (OCR) and voting files in Oracle ASM disk groups. The voting files and OCR are two important components of Oracle Clusterware.

Voting files manage information about node membership. OCR is a system that manages cluster and Oracle Real Application Clusters (Oracle RAC) database configuration information. A quorum failure group is a special type of failure group and disks in these failure groups do not contain user data. A quorum failure group is not considered when determining redundancy requirements in respect to storing user data. However, a quorum failure group counts when mounting a disk group. For information about failure groups, see "Oracle ASM Failure Groups".

You can manage and monitor OCR and voting files with the following:

• The CRSCTL and ocrconfig command-line tools

  The CRSCTL and ocrconfig commands enable the placement of OCR storage and Cluster Synchronization Services (CSS) voting files inside the disk groups managed by Oracle ASM.

• CREATE/ALTER DISKGROUP SQL Statements

  The SQL keywords QUORUM and REGULAR enable the specification of disk and failure groups when creating or altering disk groups.

  See Example 4-4.

• V$ASM views

  The FAILGROUP_TYPE column in both the V$ASM_DISK and V$ASM_DISK_STAT views specifies failure group type. The value for this column is REGULAR for regular failure groups and QUORUM for quorum failure groups.

  The VOTING_FILE column in both the V$ASM_DISK and V$ASM_DISK_STAT views specifies whether a disk contains a voting file. The value for this column is either Y if the disk contains a voting file or N if not.

  Note that the value of USABLE_FILE_MB in V$ASM_DISKGROUP and V$ASM_DISKGROUP_STAT does not consider any free space that is present in QUORUM disks because that space is not available for client data files.

  See "Views Containing Oracle ASM Disk Group Information".

The QUORUM and REGULAR keywords provide an additional qualifier for failure group or disk specifications when creating or altering a disk group. QUORUM disks (or disks in QUORUM failure groups) cannot have client data files, whereas REGULAR disks (or disks in non-quorum failure groups) have no such restriction.

You can use these keywords before the keyword FAILGROUP if a failure group is being explicitly specified. If the failure group is implicitly implied, you can use these keywords (QUORUM/REGULAR) before the keyword DISK. When failure groups are explicitly specified, it is an error to specify these keywords (QUORUM/REGULAR) immediately before the keyword DISK. REGULAR is the default failure group type.

When performing operations on existing disks or failure groups, the qualifier specified in the SQL must match the qualifier that was specified when the disks or failure groups were added to the disk group.

Example 4-4 shows the creation of a disk group with a QUORUM failure group. For Oracle Clusterware files a minimum of three disk devices or three failure groups is required with a normal redundancy disk group. A QUORUM failure group is not considered when determining redundancy requirements in respect to storing user data.

The COMPATIBLE.ASM disk group compatibility attribute must be set to 11.2 or greater to store OCR or voting files in a disk group.

CREATE DISKGROUP ocr_data NORMAL REDUNDANCY
   FAILGROUP fg1 DISK '/devices/diskg1'
   FAILGROUP fg2 DISK '/devices/diskg2'
   QUORUM FAILGROUP fg3 DISK '/devices/diskg3'
   ATTRIBUTE 'compatible.asm' = '11.2.0.0.0';

Managing Volumes in a Disk Group

You can create an Oracle ASM Dynamic Volume Manager (Oracle ADVM) volume in a disk group. The volume device associated with the dynamic volume can then be used to host an Oracle ACFS file system.

The compatibility parameters COMPATIBLE.ASM and COMPATIBLE.ADVM must be set to 11.2 or higher for the disk group. See "Disk Group Compatibility Attributes".

The ALTER DISKGROUP VOLUME SQL statements enable you to manage Oracle ADVM volumes, including the functionality to add, modify, resize, disable, enable, and drop volumes. The following are examples of the ALTER DISKGROUP VOLUME statement.

SQL> ALTER DISKGROUP data ADD VOLUME volume1 SIZE 10G;
Diskgroup altered.

SQL> ALTER DISKGROUP data RESIZE VOLUME volume1 SIZE 15G;
Diskgroup altered.

SQL> ALTER DISKGROUP data DISABLE VOLUME volume1;
Diskgroup altered.

SQL> ALTER DISKGROUP data ENABLE VOLUME volume1;
Diskgroup altered.

SQL> ALTER DISKGROUP ALL DISABLE VOLUME ALL;
Diskgroup altered.

SQL> ALTER DISKGROUP data DROP VOLUME volume1;
Diskgroup altered.

If the volume is hosting an Oracle ACFS file system, then you cannot resize that volume with the SQL ALTER DISKGROUP statement. Instead you must use the acfsutil size command. For information, see "acfsutil size".

For information about Oracle ADVM, see "Overview of Oracle ASM Dynamic Volume Manager". For information about managing Oracle ADVM volumes with ASMCMD, see "ASMCMD Volume Management Commands". For information about managing Oracle ADVM volumes with ASMCA, see "Managing Oracle ADVM Volumes with Oracle ASM Configuration Assistant". For information about managing Oracle ADVM volumes with Oracle Enterprise Manager, see "Managing Oracle ACFS with Oracle Enterprise Manager".

Adding Disks to a Disk Group

You can use the ADD clause of the ALTER DISKGROUP statement to add a disk or a failure group to a disk group. You can use the same syntax with the ALTER DISKGROUP statement that you use to add a disk or failure group with the CREATE DISKGROUP statement. For an example of the CREATE DISKGROUP SQL statement, refer to Example 4-1. After you add new disks, the new disks gradually begin to accommodate their share of the workload as rebalancing progresses.

Oracle ASM behavior when adding disks to a disk group is best illustrated through"Adding Disks to a Disk Group with SQL Statements". You can also add disks to a disk group with Oracle Enterprise Manager, described in "Adding Disks to Disk Groups".

ALTER DISKGROUP data1 ADD DISK
     '/devices/diska*';

ALTER DISKGROUP data1 ADD DISK
     '/devices/diska5' NAME diska5,
     '/devices/diska6' NAME diska6,
     '/devices/diska7' NAME diska7,
     '/devices/diska8' NAME diska8;

ALTER DISKGROUP data1 ADD DISK
     '/devices/disk*4';

ALTER DISKGROUP data1 ADD DISK
      '/devices/diskd*'
       REBALANCE POWER 5 WAIT;

ALTER DISKGROUP data2 ADD DISK
     '/devices/diskc3' FORCE;

Dropping Disks from Disk Groups

To drop disks from a disk group, use the DROP DISK clause of the ALTER DISKGROUP statement. You can also drop all of the disks in specified failure groups using the DROP DISKS IN FAILGROUP clause.

When a disk is dropped, the disk group is rebalanced by moving all of the file extents from the dropped disk to other disks in the disk group. A drop disk operation might fail if not enough space is available on the other disks. The best approach is to perform both the add and drop operation with the same ALTER DISKGROUP statement. This has the benefit of rebalancing data extents only one time and ensuring that there is enough space for the rebalance operation to succeed.

If you specify the FORCE clause for the drop operation, the disk is dropped even if Oracle ASM cannot read or write to the disk. You cannot use the FORCE flag when dropping a disk from an external redundancy disk group.

You can also drop disks from a disk group with Oracle Enterprise Manager. See "Dropping Disks from Disk Groups".

The statements in Example 4-6 demonstrate how to drop disks from the disk group data1 described in "Adding Disks to a Disk Group with SQL Statements". The first example drops diska5 from disk group data1. The second example drops diska5 from disk group data1, and also illustrates how multiple actions are possible with one ALTER DISKGROUP statement.

ALTER DISKGROUP data1 DROP DISK diska5;

ALTER DISKGROUP data1 DROP DISK diska5
     ADD FAILGROUP failgrp1 DISK '/devices/diska9' NAME diska9;

Intelligent Data Placement

Intelligent Data Placement enables you to specify disk regions on Oracle ASM disks for best performance. Using the disk region settings, you can ensure that frequently accessed data is placed on the outermost (hot) tracks which have greater speed and higher bandwidth. In addition, files with similar access patterns are located physically close, reducing latency. Intelligent Data Placement also enables the placement of primary and mirror extents into different hot or cold regions.

Intelligent Data Placement settings can be specified for a file or in disk group templates. The disk region settings can be modified after the disk group has been created. The disk region setting can improve I/O performance by placing more frequently accessed data in regions furthest from the spindle, while reducing your cost by increasing the usable space on a disk.

Intelligent Data Placement works best for the following:

• Databases with data files that are accessed at different rates. A database that accesses all data files in the same way is unlikely to benefit from Intelligent Data Placement.

• Disk groups that are more than 25% full. If the disk group is only 25% full, the management overhead is unlikely to be worth any benefit.

• Disks that have better performance at the beginning of the media relative to the end. Because Intelligent Data Placement leverages the geometry of the disk, it is well suited to JBOD (just a bunch of disks). In contrast, a storage array with LUNs composed of concatenated volumes masks the geometry from Oracle ASM.

The COMPATIBLE.ASM and COMPATIBLE.RDBMS disk group attributes must be set to 11.2 or higher to use Intelligent Data Placement.

Intelligent Data Placement can be managed with the ALTER DISKGROUP ADD or MODIFY TEMPLATE SQL statements and the ALTER DISKGROUP MODIFY FILE SQL statement.

• The ALTER DISKGROUP TEMPLATE SQL statement includes a disk region clause for setting hot/mirrorhot or cold/mirrorcold regions in a template:

  ALTER DISKGROUP data ADD TEMPLATE datafile_hot
    ATTRIBUTE ( 
      HOT
      MIRRORHOT);
  

• The ALTER DISKGROUP ... MODIFY FILE SQL statement that sets disk region attributes for hot/mirrorhot or cold/mirrorcold regions:

  ALTER DISKGROUP data MODIFY FILE '+data/orcl/datafile/users.259.679156903'
    ATTRIBUTE ( 
      HOT
      MIRRORHOT);
  

  When you modify the disk region settings for a file, this action applies to new extensions of the file, but existing file contents are not affected until a rebalance operation. To apply the new Intelligent Data Placement policy for existing file contents, you can manually initiate a rebalance. A rebalance operation uses the last specified policy for the file extents. For information on the rebalance operation, see "Manually Rebalancing Disk Groups".

Oracle ASM Configuration Assistant (ASMCA) supports Intelligent Data Placement with template creation during disk group alterations. See "Managing Disk Groups with Oracle ASM Configuration Assistant".

Oracle Enterprise Manager supports Intelligent Data Placement from the Templates page launched from the disk group page. See "Managing Disk Group Templates with Oracle Enterprise Manager".

To display information about Intelligent Data Placement regions, see "Viewing Disk Region Information" and "iostat".

Resizing Disks in Disk Groups

The RESIZE clause of ALTER DISKGROUP enables you to perform the following operations:

• Resize all disks in the disk group

• Resize specific disks

• Resize all of the disks in a specified failure group

If you do not specify a new size in the SIZE clause, then Oracle ASM uses the size of the disk as returned by the operating system. The new size is written to the Oracle ASM disk header and if the size of the disk is increasing, then the new space is immediately available for allocation. If the size is decreasing, rebalancing must relocate file extents beyond the new size limit to available space below the limit. If the rebalance operation can successfully relocate all extents, then the new size is made permanent, otherwise the rebalance fails.

The following example resizes all of the disks in failure group failgrp1 of disk group data1. If the new size is greater than disk capacity, the statement fails.

ALTER DISKGROUP data1 
     RESIZE DISKS IN FAILGROUP failgrp1 SIZE 100G;

Undropping Disks in Disk Groups

The UNDROP DISKS clause of the ALTER DISKGROUP statement enables you to cancel all pending drops of disks within disk groups. If a drop disk operation has completed, then this statement cannot be used to restore it. This statement cannot be used to restore disks that are being dropped as the result of a DROP DISKGROUP statement, or for disks that are being dropped using the FORCE clause.

The following example cancels the dropping of disks from disk group data1:

ALTER DISKGROUP data1 UNDROP DISKS;

Manually Rebalancing Disk Groups

You can manually rebalance the files in a disk group using the REBALANCE clause of the ALTER DISKGROUP statement. This would normally not be required, because Oracle ASM automatically rebalances disk groups when their configuration changes. You might want to do a manual rebalance operation to control the speed of what would otherwise be an automatic rebalance operation.

The POWER clause of the ALTER DISKGROUP ... REBALANCE statement specifies the degree of parallelism, and thus the speed of the rebalance operation. It can be set to a minimum value of 0 which halts a rebalancing operation until the statement is either implicitly or explicitly re-run. A higher value increases the speed of the rebalance operation.

The default rebalance power is set by the ASM_POWER_LIMIT initialization parameter. The range of values for the POWER clause is the same for the ASM_POWER_LIMIT initialization parameter. For information about the ASM_POWER_LIMIT initialization parameter, refer to "ASM_POWER_LIMIT".

The power level of an ongoing rebalance operation can be changed by entering the rebalance statement with a new level.

The ALTER DISKGROUP...REBALANCE command by default returns immediately so that you can run other commands while the rebalance operation takes place asynchronously in the background. You can query the V$ASM_OPERATION view for the status of the rebalance operation.

To cause the ALTER DISKGROUP...REBALANCE command to wait until the rebalance operation is complete before returning, add the WAIT keyword to the REBALANCE clause. The wait functionality is especially useful in scripts. The command also accepts a NOWAIT keyword, which invokes the default behavior of conducting the rebalance operation asynchronously. You can interrupt a rebalance running in wait mode by typing CTRL-C on most platforms. This causes the command to return immediately with the message ORA-01013: user requested cancel of current operation, and then continue the rebalance operation asynchronously.

Additional rules for the rebalance operation include the following:

• An ongoing rebalance command is restarted if the storage configuration changes either when you alter the configuration, or if the configuration changes due to a failure or an outage. Furthermore, if the new rebalance fails because of a user error, then a manual rebalance may be required.

• The ALTER DISKGROUP...REBALANCE statement runs on a single node even if you are using Oracle Real Application Clusters (Oracle RAC).

• Oracle ASM can perform one disk group rebalance at a time on a given instance. If you have initiated multiple rebalances on different disk groups on a single node, then Oracle processes these operations in parallel on additional nodes if available; otherwise the rebalances are performed serially on the single node. You can explicitly initiate rebalances on different disk groups on different nodes in parallel.

• Rebalancing continues across a failure of the Oracle ASM instance performing the rebalance.

• The REBALANCE clause (with its associated POWER and WAIT/NOWAIT keywords) can also be used in ALTER DISKGROUP commands that add, drop, or resize disks.

  Note:

  Oracle restarts the processing of an ongoing rebalance operation if the storage configuration changes. If the next rebalance operation fails because of a user error, then a manual rebalance may be required.

The following example manually rebalances the disk group data2. The command does not return until the rebalance operation is complete.

ALTER DISKGROUP data2 REBALANCE POWER 5 WAIT;

For more information about rebalancing operations, refer to "Tuning Rebalance Operations".

Tuning Rebalance Operations

If the POWER clause is not specified in an ALTER DISKGROUP statement, or when rebalance is implicitly run by adding or dropping a disk, then the rebalance power defaults to the value of the ASM_POWER_LIMIT initialization parameter. You can adjust the value of this parameter dynamically.

The higher the power limit, the more quickly a rebalance operation can complete. Rebalancing takes longer with lower power values, but consumes fewer processing and I/O resources which are shared by other applications, such as the database.

Oracle ASM tries to keep a rebalance I/O for each unit of power. Each I/O requires PGA memory for the extent involved in the relocation.

The default value of 1 minimizes disruption to other applications. The appropriate value is dependent on your hardware configuration, performance requirements, and availability requirements.

If a rebalance is in progress because a disk is manually or automatically dropped, then increasing the power of the rebalance shortens the time frame during which redundant copies of that data on the dropped disk are reconstructed on other disks.

The V$ASM_OPERATION view provides information for adjusting ASM_POWER_LIMIT and the resulting power of rebalance operations. The V$ASM_OPERATION view also gives an estimate in the EST_MINUTES column of the amount of time remaining for the rebalance operation to complete. You can see the effect of changing the rebalance power by observing the change in the time estimate.

The range of values for the POWER clause is the same for the ASM_POWER_LIMIT initialization parameter. For information about the ASM_POWER_LIMIT initialization parameter, refer to "ASM_POWER_LIMIT".

For more information about rebalancing disk groups, refer to "Manually Rebalancing Disk Groups".

How A Disk is Discovered

When an Oracle ASM instance is initialized, Oracle ASM discovers and examines the contents of all of the disks that are in the paths that you designated with values in the ASM_DISKSTRING initialization parameter.

Disk discovery also occurs when you:

• Run the following SQL statements

  • Mount a disk group with ALTER DISKGROUP ... MOUNT

  • Online a disk with ALTER DISKGROUP ... ONLINE DISK

  • Add a disk to a disk group with CREATE or ALTER DISKGROUP...ADD DISK

  • Resize a disk in a disk group with ALTER DISKGROUP...RESIZE DISK

  • Query with SELECT ... FROM V$ASM_DISKGROUP or V$ASM_DISK views

• Run Oracle Enterprise Manager or Oracle ASM Configuration Assistant (ASMCA) operations that invoke the SQL statements previously listed

• Run ASMCMD commands that perform the same operations as the SQL statements previously listed

After Oracle ASM successfully discovers a disk, the disk appears in the V$ASM_DISK view. Disks that belong to a disk group, that is, disks that have a disk group name in the disk header, show a header status of MEMBER. Disks that were discovered, but that have not yet been assigned to a disk group, have a status of either CANDIDATE or PROVISIONED. Disks that previously belonged to a disk group and were dropped cleanly from the disk group have a status of FORMER.

The PROVISIONED status implies that an additional platform-specific action has been taken by an administrator to make the disk available for Oracle ASM. For example, on Windows computers, the administrator might have used asmtool or asmtoolg to stamp the disk with a header. On Linux computers, the administrator might have used ASMLib to prepare the disk for Oracle ASM.

Example 4-10 shows a SQL query on V$ASM_DISK that displays the header status of a group of disks.

SQL> SELECT name, header_status, path FROM V$ASM_DISK 
     WHERE path LIKE '/devices/disk0%'

NAME      HEADER_STATUS PATH
--------- ------------- ---------------------
          FORMER        /devices/disk02
          FORMER        /devices/disk01
          CANDIDATE     /devices/disk07
DISK06    MEMBER        /devices/disk06
DISK05    MEMBER        /devices/disk05
DISK04    MEMBER        /devices/disk04
DISK03    MEMBER        /devices/disk03
7 rows selected.

Disk Discovery Rules

The rules for discovering Oracle ASM disks are as follows:

• Oracle ASM can discover up to 10,000 disks. That is, if more than 10,000 disks match the ASM_DISKSTRING initialization parameter, then Oracle ASM discovers only the first 10,000.

• Oracle ASM only discovers disk partitions. Oracle ASM does not discover partitions that include the partition table.

• From the perspective of the installation, candidate disks are those that have the CANDIDATE, PROVISIONED, or FORMER header status. These disks with a CANDIDATE, PROVISIONED, or FORMER status can be added to Oracle ASM disk groups without using the FORCE flag.

• When adding a disk, the FORCE option must be used if Oracle ASM recognizes that the disk was managed by Oracle. Such a disk appears in the V$ASM_DISK view with a status of FOREIGN. In this case, you can only add the disk to a disk group by using the FORCE keyword.

• MEMBER disks can usually be added to a disk group by specifying the FORCE flag, if the disks are not part of a currently mounted disk group.

In addition, Oracle ASM identifies the following configuration errors during discovery:

• Multiple paths to the same disk

  In this case, if the disk is part of a disk group, then disk group mount fails. If the disk is being added to a disk group with the ADD DISK or CREATE DISKGROUP command, then the command fails. To correct the error, adjust the ASM_DISKSTRING value so that Oracle ASM does not discover multiple paths to the same disk. Or if you are using multipathing software, then ensure that you include only the pseudo-device name in the ASM_DISKSTRING value. See "Oracle ASM and Multipathing".

• Multiple Oracle ASM disks with the same disk header

  This can be caused by having copied one disk onto another. In this case, the disk group mount operation fails.

Improving Disk Discovery Time

The value for the ASM_DISKSTRING initialization parameter is an operating system–dependent value that Oracle ASM uses to limit the set of paths that the discovery process uses to search for disks. When a new disk is added to a disk group, each Oracle ASM instance that has the disk group mounted must be able to discover the new disk using its ASM_DISKSTRING.

In many cases, the default value (NULL) is sufficient. Using a more restrictive value might reduce the time required for Oracle ASM to perform discovery, and thus improve disk group mount time or the time for adding a disk to a disk group. Oracle may dynamically change the ASM_DISKSTRING before adding a disk so that the new disk is discovered through this parameter.

The default value of ASM_DISKSTRING might not find all disks in all situations. If your site is using a third-party vendor ASMLib, then the vendor might have discovery string conventions that you must use for ASM_DISKSTRING. In addition, if your installation uses multipathing software, then the software might place pseudo-devices in a path that is different from the operating system default. See "Oracle ASM and Multipathing" and consult the multipathing vendor documentation for details.

Negative Values of USABLE_FILE_MB

Due to the relationship between FREE_MB, REQUIRED_MIRROR_FREE_MB, and USABLE_FILE_MB, USABLE_FILE_MB can become negative. Although this is not necessarily a critical situation, it does mean that:

• Depending on the value of FREE_MB, you may not be able to create new files.

• The next failure might result in files with reduced redundancy.

If USABLE_FILE_MB becomes negative, it is strongly recommended that you add more space to the disk group as soon as possible.

Mirroring, Redundancy, and Failure Group Options

If you specify mirroring for a file, then Oracle ASM automatically stores redundant copies of the file extents in separate failure groups. Failure groups apply only to normal and high redundancy disk groups. You can define the failure groups for each disk group when you create or alter the disk group.

There are three types of disk groups based on the Oracle ASM redundancy level. Table 4-1 lists the types with their supported and default mirroring levels. The default mirroring levels indicate the mirroring level with which each file is created unless a different mirroring level is designated.

The redundancy level controls how many disk failures are tolerated without dismounting the disk group or losing data. Each file is allocated based on its own redundancy, but the default comes from the disk group.

The redundancy levels are:

• External redundancy

  Oracle ASM does not provide mirroring redundancy and relies on the storage system to provide RAID functionality. Any write error causes a forced dismount of the disk group. All disks must be located to successfully mount the disk group.

• Normal redundancy

  Oracle ASM provides two-way mirroring by default, which means that all files are mirrored so that there are two copies of every extent. A loss of one Oracle ASM disk is tolerated. You can optionally choose three-way or unprotected mirroring.

  A file specified with HIGH redundancy (three-way mirroring) in a NORMAL redundancy disk group provides additional protection from a bad disk sector, not protection from a disk failure.

• High redundancy

  Oracle ASM provides triple mirroring by default. A loss of two Oracle ASM disks in different failure groups is tolerated.

If there are not enough online failure groups to satisfy the file mirroring (redundancy attribute value) specified in the disk group file type template, Oracle ASM allocates as many mirrors copies as possible and subsequently allocates the remaining mirrors when sufficient online failure groups are available. For information about specifying Oracle ASM disk group templates, see "Managing Disk Group Templates".

Failure groups enable the mirroring of metadata and user data. System reliability can diminish if your environment has an insufficient number of failure groups.

This section contains these topics:

• Oracle ASM Failure Groups

• How Oracle ASM Manages Disk Failures

• Guidelines for Using Failure Groups

• Failure Group Frequently Asked Questions

Oracle ASM Recovery from Read and Write I/O Errors

Read errors can be the result of a loss of access to the entire disk or media corruptions on an otherwise a healthy disk. Oracle ASM tries to recover from read errors on corrupted sectors on a disk. When a read error by the database or Oracle ASM triggers the Oracle ASM instance to attempt bad block remapping, Oracle ASM reads a good copy of the extent and copies it to the disk that had the read error.

• If the write to the same location succeeds, then the underlying allocation unit (sector) is deemed healthy. This might be because the underlying disk did its own bad block reallocation.

• If the write fails, Oracle ASM attempts to write the extent to a new allocation unit on the same disk. If this write succeeds, the original allocation unit is marked as unusable. If the write fails, the disk is taken offline.

One unique benefit on Oracle ASM based mirroring is that the database instance is aware of the mirroring. For many types of logical corruptions such as a bad checksum or incorrect System Change Number (SCN), the database instance proceeds through the mirror side looking for valid content and proceeds without errors. If the process in the database that encountered the read can obtain the appropriate locks to ensure data consistency, it writes the correct data to all mirror sides.

When encountering a write error, a database instance sends the Oracle ASM instance a disk offline message.

• If database can successfully complete a write to at least one extent copy and receive acknowledgment of the offline disk from Oracle ASM, the write is considered successful.

• If the write to all mirror side fails, database takes the appropriate actions in response to a write error such as taking the tablespace offline.

When the Oracle ASM instance receives a write error message from a database instance or when an Oracle ASM instance encounters a write error itself, the Oracle ASM instance attempts to take the disk offline. Oracle ASM consults the Partner Status Table (PST) to see whether any of the disk's partners are offline. If too many partners are offline, Oracle ASM forces the dismounting of the disk group. Otherwise, Oracle ASM takes the disk offline.

The ASMCMD remap command was introduced to address situations where a range of bad sectors exists on a disk and must be corrected before Oracle ASM or database I/O. For information about the remap command, see "remap".

Oracle ASM Fast Mirror Resync

Restoring the redundancy of an Oracle ASM disk group after a transient disk path failure can be time consuming. This is especially true if the recovery process requires rebuilding an entire Oracle ASM failure group. Oracle ASM fast mirror resync significantly reduces the time to resynchronize a failed disk in such situations. When you replace the failed disk, Oracle ASM can quickly resynchronize the Oracle ASM disk extents.

Any problems that make a failure group temporarily unavailable are considered transient failures that can be recovered by the Oracle ASM fast mirror resync feature. For example, transient failures can be caused by disk path malfunctions, such as cable failures, host bus adapter failures, controller failures, or disk power supply interruptions.

Oracle ASM fast resync keeps track of pending changes to extents on an OFFLINE disk during an outage. The extents are resynced when the disk is brought back online.

By default, Oracle ASM drops a disk in 3.6 hours after it is taken offline. You can set the DISK_REPAIR_TIME disk group attribute to delay the drop operation by specifying a time interval to repair the disk and bring it back online. The time can be specified in units of minutes (m or M) or hours (h or H). If you omit the unit, then the default unit is hours. The DISK_REPAIR_TIME disk group attribute can only be set with the ALTER DISKGROUP SQL statement.

If the attribute is not set explicitly, then the default value (3.6h) applies to disks that have been set to OFFLINE mode without an explicit DROP AFTER clause. Disks taken offline due to I/O errors do not have a DROP AFTER clause.

The default DISK_REPAIR_TIME attribute value is an estimate that should be adequate for most environments. However, ensure that the attribute value is set to the amount of time that you think is necessary in your environment to fix any transient disk error, and that you are willing to tolerate reduced data redundancy.

The elapsed time (since the disk was set to OFFLINE mode) is incremented only when the disk group containing the offline disks is mounted. The REPAIR_TIMER column of V$ASM_DISK shows the amount of time left (in seconds) before an offline disk is dropped. After the specified time has elapsed, Oracle ASM drops the disk. You can override this attribute with the ALTER DISKGROUP OFFLINE DISK statement and the DROP AFTER clause.

If an offline disk is taken offline for a second time, then the elapsed time is reset and restarted. If another time is specified with the DROP AFTER clause for this disk, the first value is overridden and the new value applies. A disk that is in OFFLINE mode cannot be dropped with an ALTER DISKGROUP DROP DISK statement; an error is returned if attempted. If for some reason the disk must be dropped (such as the disk cannot be repaired) before the repair time has expired, a disk can be dropped immediately by issuing a second OFFLINE statement with a DROP AFTER clause specifying 0h or 0m.

You can use ALTER DISKGROUP to set the DISK_REPAIR_TIME attribute to a specified hour or minute value, such as 4.5 hours or 270 minutes. For example:

ALTER DISKGROUP data SET ATTRIBUTE 'disk_repair_time' = '4.5h'
ALTER DISKGROUP data SET ATTRIBUTE 'disk_repair_time' = '270m'

After you repair the disk, run the SQL statement ALTER DISKGROUP ONLINE DISK. This statement brings a repaired disk group back online to enable writes so that no new writes are missed. This statement also starts a procedure to copy of all of the extents that are marked as stale on their redundant copies.

If a disk goes offline when the Oracle ASM instance is in rolling upgrade mode, the disk remains offline until the rolling upgrade has ended and the timer for dropping the disk is stopped until the Oracle ASM cluster is out of rolling upgrade mode. See "Using Oracle ASM Rolling Upgrade". Examples of taking disks offline and bringing them online follow.

The following example takes disk DATA_001 offline and drops it after five minutes.

ALTER DISKGROUP data OFFLINE DISK DATA_001 DROP AFTER 5m;

The next example takes the disk DATA_001 offline and drops it after the time period designated by DISK_REPAIR_TIME elapses:

ALTER DISKGROUP data OFFLINE DISK DATA_001;

This example takes all of the disk in failure group FG2 offline and drops them after the time period designated by DISK_REPAIR_TIME elapses. IF you used a DROP AFTER clause, then the disks would be dropped after the specified time:

ALTER DISKGROUP data OFFLINE DISK IN FAILGROUP FG2;

The next example brings all of the disks in failure group FG2 online:

ALTER DISKGROUP data ONLINE DISK IN FAILGROUP FG2;

This example brings only disk DATA_001 online:

ALTER DISKGROUP data ONLINE DISK DATA_001;

This example brings all of the disks in disk group DATA online:

ALTER DISKGROUP data ONLINE ALL;

Querying the V$ASM_OPERATION view while you run any of these types of ALTER DISKGROUP ... ONLINE statements displays the name and state of the current operation that you are performing. For example, the query:

SELECT GROUP_NUMBER, OPERATION, STATE FROM V$ASM_OPERATION;

Displays output similar to the following:

GROUP_NUMBER OPERA STAT 
------------ ----- ---- 
           1 ONLIN RUN 

An OFFLINE operation is not displayed in a V$ASM_OPERATION view query.

Preferred Read Failure Groups

When you configure Oracle ASM failure groups, it might be more efficient for a node to read from an extent that is closest to the node, even if that extent is a secondary extent. In other words, you can configure Oracle ASM to read from a secondary extent if that extent is closer to the node instead of Oracle ASM reading from the primary copy which might be farther from the node. Using the preferred read failure groups feature is most useful in extended clusters.

To use this feature, Oracle recommends that you configure at least one mirrored extent copy from a disk that is local to a node in an extended cluster. However, a failure group that is preferred for one instance might be remote to another instance in the same Oracle RAC database. The parameter setting for preferred read failure groups is instance specific.

Both the Oracle ASM clients and Oracle ASM require Oracle Database 11g release 1 (11.1) or higher to use preferred read failure groups.

Determining the Number of Disk Groups

Use the following criteria to determine the number of disk groups to create:

• Disks in a given disk group should have similar size and performance characteristics. If you have several different types of disks in terms of size and performance, then create several disk groups that contain similar characteristics.

• Create separate disk groups for your database files and fast recovery area for backup files. This configuration allows fast recovery should a disk group failure occur.

For information about creating disk groups for a new installation, refer to "Creating Disk Groups for a New Oracle Installation".

Performance Characteristics When Grouping Disks

Oracle ASM load balances the file activity by uniformly distributing file extents across all of the disks in a disk group. For this technique to be effective it is important that disks in a disk group be of similar performance characteristics. For example, the newest and fastest disks might reside in a disk group reserved for the database work area, and slower drives could reside in a disk group reserved for the fast recovery area.

There might be situations where it is acceptable to temporarily have disks of different sizes and performance characteristics coexist in a disk group. This would be the case when migrating from an old set of disks to a new set of disks. The new disks would be added and the old disks dropped. As the old disks are dropped, their storage is migrated to the new disks while the disk group is online.

Oracle ASM Storage Limits

Oracle ASM provides near unlimited capacity for future growth, but does have some storage limits. For example, Oracle ASM has the following limits on the number of disk groups, disks, and files:

• 63 disk groups in a storage system

• 10,000 Oracle ASM disks in a storage system

• 1 million files for each disk group

Without any Oracle Exadata Storage, Oracle ASM has these storage limits:

• 2 terabytes (TB) maximum storage for each Oracle ASM disk

• 20 petabytes (PB) maximum for the storage system

With all Oracle Exadata Storage, Oracle ASM has these storage limits:

• 4 PB maximum storage for each Oracle ASM disk

• 40 exabytes (EB) maximum for the storage system

The maximum size limit of a disk group equals the maximum disk size multiplied by the maximum number of disks in a disk group (10,000).

The maximum number of disks across all disk groups is 10,000. The 10,000 disks can be in one disk group or distributed across a maximum of 63 disk groups. This is a limitation on the number of Oracle ASM disks, not necessarily the number of spindles. A storage array could group multiple spindles into a LUN that is used as a single Oracle ASM disk. However Oracle ASM is currently limited to 2 TB in a single disk unless using Oracle Exadata storage.

File size limits are dependent on the value of the disk group compatibility attributes. Oracle ASM supports file sizes greater than 128 TB in any redundancy mode when the COMPATIBLE.RDBMS disk group attribute is set greater than10.1.

If COMPATIBLE.RDBMS is set to 10.1, the file size limits are less. For example, with COMPATIBLE.RDBMS equal to 10.1 and the AU size equal to 1 MB, Oracle ASM file size limits are:

• External redundancy: 16 TB

• Normal redundancy: 5.8 TB

• High redundancy: 3.9 TB

For information about Oracle ASM disk group compatibility attributes, see "Disk Group Compatibility". For information about Oracle ASM file size limits, see Table 4-4.

Overview of Disk Group Compatibility

Advancing the disk group compatibility settings enables you to use the new Oracle ASM features that are available in a later release. For example, a disk group with the disk group compatibility attributes set to 11.2 can take advantage of new Oracle 11g release 2 (11.2) features, such as Oracle ASM volumes in disk groups and Oracle ASM File Access Control. See Table 4-3 for the features enabled for combinations of compatibility attribute settings.

The disk group compatibility feature also enables environments to interoperate when they use disk groups from both Oracle Database 10g and Oracle Database 11g. For example, disk group compatibility settings that are set to Oracle Database 10g enable an Oracle 10g client to access a disk group created with Oracle ASM 11g.

The disk group attributes that determine compatibility are COMPATIBLE.ASM, COMPATIBLE.RDBMS. and COMPATIBLE.ADVM. The COMPATIBLE.ASM and COMPATIBLE.RDBMS attribute settings determine the minimum Oracle Database software version numbers that a system can use for Oracle ASM and the database instance types respectively. For example, if the Oracle ASM compatibility setting is 11.2, and RDBMS compatibility is set to 11.1, then the Oracle ASM software version must be at least 11.2, and the Oracle Database client software version must be at least 11.1. The COMPATIBLE.ADVM attribute determines whether the Oracle ASM Dynamic Volume Manager feature can create an volume in a disk group.

When you create a disk group, you can specify the disk group compatibility attribute settings in the CREATE DISKGROUP SQL statement. The ALTER DISKGROUP SQL statement can update the compatible attribute settings for existing disk groups. If not specified when using the CREATE DISKGROUP SQL statement, 10.1 is the default setting for both the COMPATIBLE.ASM and COMPATIBLE.RDBMS attributes for Oracle ASM in Oracle Database 11g. The COMPATIBLE.ADVM attribute is empty if it is not set. See Table 4-2 for examples of valid combinations of compatible attribute settings.

You can set and update disk group attributes with the ASMCMD setattr command. For information about the ASMCMD setattr command, see "setattr".

Disk Group Compatibility Attributes

The disk group compatibility attributes specify the disk group compatibility settings for Oracle ASM and database instances. These attributes are described under the following topics:

• COMPATIBLE.ASM

• COMPATIBLE.RDBMS

• COMPATIBLE.ADVM

Setting Disk Group Compatibility Attributes

This section discusses the settings of the disk group compatibility attributes and how to set the attribute values with the CREATE DISKGROUP or ALTER DISKGROUP SQL statement.

This section contains these topics:

• Valid Combinations of Compatibility Attribute Settings

• Using CREATE DISKGROUP with Compatibility Attributes

• Using ALTER DISKGROUP with Compatibility Attributes

• Viewing Compatibility Attribute Settings

• Features Enabled By Disk Group Compatibility Attribute Settings

• Reverting Disk Group Compatibility

You can also set the disk group compatibility settings with Oracle Enterprise Manager, Oracle ASM command-line utility (ASMCMD), and Oracle ASM Configuration Assistant (ASMCA). See Chapter 9, "Administering Oracle ASM with Oracle Enterprise Manager", Chapter 11, "Oracle ASM Configuration Assistant", and Chapter 12, "Oracle ASM Command-Line Utility".

CREATE DISKGROUP data1 DISK '/dev/sd*' 
       ATTRIBUTE 'compatible.asm' = '11.2';

CREATE DISKGROUP data2 DISK '/dev/sd*' 
       ATTRIBUTE 'compatible.asm' = '11.2', 'compatible.rdbms' = '11.2',
                 'compatible.advm' = '11.2';

ALTER DISKGROUP data3 SET ATTRIBUTE 'compatible.asm' = '11.2';

ALTER DISKGROUP data3 SET ATTRIBUTE 'compatible.rdbms' = '11.2',
ALTER DISKGROUP data3 SET ATTRIBUTE 'compatible.advm' = '11.2';

Considerations When Setting Disk Group Compatibility in Replicated Environments

If you advance disk group compatibility, then you could enable the creation of files that are too large to be managed by a previous Oracle database release. You must be aware of the file size limits because replicated sites cannot continue using the software from a previous release to manage these large files. The disk group compatibility settings should be the same for all replicated environments.

Table 4-4 shows the maximum Oracle ASM file sizes supported for COMPATIBLE.RDBMS settings when the AU_SIZE attribute is set to one megabyte for a disk group.

Table 4-4 shows that Oracle Database 10g can only support a file size of up to 16 TB for external redundancy. If you advance the COMPATIBILE.RDBMS attribute to 11.1 or greater, then a file can grow beyond 16 TB. However, the larger size causes the file to be unusable in a replicated and disaster recovery site if the disaster recovery site has a disk group COMPATIBLE.RDBMS setting that is incompatible with the larger size.

For information about Oracle ASM storage sizes, see "Oracle ASM Storage Limits".

About Oracle ASM File Access Control

Oracle ASM File Access Control restricts the access of files to specific Oracle ASM clients that connect as SYSDBA. An Oracle ASM client is typically a database, which is identified as the user that owns the database instance home. Oracle ASM File Access Control uses this user name to identify a database. Oracle ASM File Access Control restricts access based on the operating system effective user identification number of a database owner. For example, in Table 3-2, "Separated operating system groups and privileges for Oracle ASM users" the databases are identified as oracle1 and oracle2.

Oracle ASM uses file access control to determine the additional privileges that are given to a database that has been authenticated AS SYSDBA on the Oracle ASM instance. These additional privileges include the ability to modify and delete certain files, aliases, and user groups.

You can set up user groups to specify the list of databases that share the same access permissions to Oracle ASM files. User groups are lists of databases and any database that authenticates AS SYSDBA can create a user group. However, only the creator of a group can delete it or modify its membership list.

Each Oracle ASM file has three categories of privileges: owner, group, and other. Each category can have no permission, read-only permission, or read-write permission.

The file owner is usually the creator of the file and can assign permissions for the file in any of the owner, group, or other categories. The owner can also change the group associated with the file.

When administering Oracle ASM File Access Control, Oracle recommends that you connect as SYSDBA to the database instance that is the owner, or planned owner, of the files in the disk group.

To set up Oracle ASM File Access Control for files in a disk group, perform the following steps:

1. Alter a new or existing disk group to set the Oracle ASM File Access Control disk group attributes.

   For a newly-created disk group, you should set the disk group attributes before creating any files in the disk group.

   See "Setting Disk Group Attributes for Oracle ASM File Access Control".

2. For files that exist in a disk group before setting the Oracle ASM File Access Control disk group attributes, you must explicitly set the permissions and ownership on those existing files.

   Ensure that the user exists before setting ownership or permissions on a file. The file must be closed before setting the ownership or permissions.

   See ALTER DISKGROUP SET PERMISSION and ALTER DISKGROUP SET OWNERSHIP in "Using SQL Statements to Manage Oracle ASM File Access Control".

3. Optionally, you can create user groups that are groups of database users that share the same access permissions to Oracle ASM files.

   See ALTER DISKGROUP ADD USERGROUP in "Using SQL Statements to Manage Oracle ASM File Access Control".

Setting Disk Group Attributes for Oracle ASM File Access Control

To manage Oracle ASM File Access Control for a disk group, you must set the ACCESS_CONTROL.ENABLED and ACCESS_CONTROL.UMASK disk group attributes. You can set the attributes by altering the disk group with the ALTER DISKGROUP SQL statement or you can set the attributes with the ASMCMD setattr command. For information about the ASMCMD setattr command, see "setattr".

When you set up file access control on an existing disk group, the files previously created remain accessible by everyone, unless you run the ALTER DISKGROUP SET PERMISSION SQL statement or the ASMCMD chmod command to restrict the permissions. For information about the ASMCMD chmod command, see "chmod".

The COMPATIBLE.ASM and COMPATIBLE.RDBMS disk group attributes must be set to 11.2 or higher to enable Oracle ASM File Access Control. For information about disk group compatibility attributes, see "Disk Group Compatibility".

The disk group attributes that control Oracle ASM File Access Control are the following:

• ACCESS_CONTROL.ENABLED

  This attribute determines whether Oracle ASM File Access Control is enabled for a disk group.

  The value can be true or false. The default is false.

  If the attribute is set to true, accessing Oracle ASM files is subject to access control. If false, any user can access every file in the disk group. All other operations behave independently of this attribute.

• ACCESS_CONTROL.UMASK

  This attribute determines which permissions are masked out on the creation of an Oracle ASM file for the user that owns the file, users in the same user group, and others not in the user group. This attribute applies to all files on a disk group.

  The values can be combinations of three digits {0|2|6} {0|2|6} {0|2|6}. The default is 066.

  Setting to 0 masks out nothing. Setting to 2 masks out write permission. Setting to 6 masks out both read and write permissions.

  Before setting the ACCESS_CONTROL.UMASK disk group attribute, you must set the ACCESS_CONTROL.ENABLED attribute to true to enable Oracle ASM File Access Control.

Example 4-11 shows how to enable Oracle ASM File Access Control for a disk group using SQL*Plus. In this example, the permissions setting is 026 which enables read-write access for the owner, read access for users in the group, and no access to others not in the group

ALTER DISKGROUP data1 SET ATTRIBUTE 'access_control.enabled' = 'true';
ALTER DISKGROUP data1 SET ATTRIBUTE 'access_control.umask' = '026';

Using SQL Statements to Manage Oracle ASM File Access Control

You can use the ALTER DISKGROUP SQL statement to manage file access control for Oracle ASM disk groups. These SQL statements are available for both database and Oracle ASM instances.

The SQL statements that support disk group access control are:

• ALTER DISKGROUP ADD USERGROUP ... WITH MEMBER

  Adds an Oracle ASM user group to a disk group. The user group name is limited to a maximum of 30 characters. The databases identified in the MEMBER clause must be in the disk group, as shown by V$ASM_USER, or the command returns an error. Any users authenticated as SYSASM or SYSDBA can create new user groups. For example:

  SQL> SELECT group_number, os_name FROM V$ASM_USER;
  
  GROUP_NUMBER OS_NAME
  ------------ ----------------------------------------------------------------
             1 oracle1
             1 oracle2
  ...
  
  SQL> ALTER DISKGROUP data ADD USERGROUP 'test_grp1' 
       WITH MEMBER 'oracle1','oracle2';
  

• ALTER DISKGROUP DROP USERGROUP

  Drops an Oracle ASM user group from a disk group. Dropping a group might leave some files without a valid group. For those files to have a valid group, you must manually update the group associated with those files to a valid group.

  SQL> ALTER DISKGROUP data DROP USERGROUP 'test_grp1';
  

• ALTER DISKGROUP MODIFY USERGROUP ADD MEMBER

  Adds users to the specified user group. The users must be in the disk group, as shown by V$ASM_USER, or the command returns an error. Only the creator of the group or the Oracle ASM administrator can modify group membership.

  SQL> ALTER DISKGROUP data MODIFY USERGROUP 'test_grp2' ADD MEMBER 'oracle2';
  

• ALTER DISKGROUP MODIFY USERGROUP DROP MEMBER

  Removes users from the specified user group. If a member is not in the user group, then an error is returned. Only the creator of the group or the Oracle ASM administrator can modify group membership.

  SQL> ALTER DISKGROUP data MODIFY USERGROUP 'test_grp2' DROP MEMBER 'oracle2';
  

• ALTER DISKGROUP ADD USER

  Adds operating system (OS) users to an Oracle ASM disk group, so that these users can have access privileges on the disk group. The users must be existing operating system users, and their user names must have a corresponding operating system user ID or system ID. If a user exists in the disk group, as shown by V$ASM_USER, then the command records an error and continues to add other users, if any.

  The operating system user of a running database instance is automatically added to a disk group when the database instance accesses that disk group and creates files. However, for a database instance to read files in a disk group without creating any files, then you must use the ADD USER clause to add that database user to the disk group. Also, you can use this clause to add a database user to an existing disk group immediately after setting the Oracle ASM File Access Control disk group attributes and before creating new files.

  SQL>  ALTER DISKGROUP DATA ADD USER 'oracle1';
  

• ALTER DISKGROUP DROP USER

  Drops operating system users from an Oracle ASM disk group. If a user is not in the disk group, then this command records an error and continues to drop other users, if any.

  If the user owns any files on the same Oracle ASM disk group, then this command fails with an error, unless the CASCADE keyword is specified. If the latter case, then the user is deleted, along with all the files that the user owns.

  If any files owned by the user are currently open, then the DROP USER command fails, and no files are deleted.

  SQL>  ALTER DISKGROUP DATA DROP USER 'oracle1';
  

• ALTER DISKGROUP SET PERMISSION

  Modifies permissions of an Oracle ASM file. Setting read only permission to a file that has read write permission revokes the write permission. Only the file owner or the Oracle ASM administrator can change the permissions of a file. You cannot change the permissions on an open file.

  SQL> ALTER DISKGROUP data SET PERMISSION OWNER=read write, GROUP=read only,
       OTHER=none FOR FILE '+data/controlfile.f';
  

• ALTER DISKGROUP SET OWNERSHIP

  Changes the owner or group of a file to the specified user or user group name, respectively. If the specified user or user group name does not exist, this command fails with an error. Only the owner of the file or the Oracle ASM administrator can run this command, and only the Oracle ASM administrator can change the owner. Also, the user group name must exist, and the owner of the file must be a member of that group. You cannot change the ownership of an open file.

  SQL> ALTER DISKGROUP data SET OWNERSHIP OWNER='oracle1', GROUP='test_grp1'
       FOR FILE '+data/controlfile.f';
  

Mounting Disk Groups Using the FORCE Option

Oracle ASM provides a MOUNT FORCE option with ALTER DISKGROUP to enable Oracle ASM disk groups to be mounted in normal or high redundancy modes even though some Oracle ASM disks may be unavailable to the disk group at mount time. The default behavior without the FORCE option is to fail to mount a disk group that has damaged or missing disks.

The MOUNT FORCE option is useful in situations where a disk is temporarily unavailable and you want to mount the disk group with reduced redundancy while you correct the situation that caused the outage.

To successfully mount with the MOUNT FORCE option, Oracle ASM must be able to find at least one copy of the extents for all of the files in the disk group. In this case, Oracle ASM can successfully mount the disk group, but with potentially reduced redundancy.

The disks that Oracle ASM cannot access are placed in an offline mode. Oracle ASM then begins timing the period that these disks are in an offline mode. If the disk offline time period exceeds the timer threshold set by DISK_REPAIR_TIME disk group attribute, then those disks are permanently dropped from the disk group. You can change the offline timer after a disk is put in an offline state by using the ALTER DISKGROUP OFFLINE statement. For more information about setting the DISK_REPAIR_TIME disk group attribute, see "Oracle ASM Fast Mirror Resync".

In clustered Oracle ASM environments, if an Oracle ASM instance is not the first instance to mount the disk group, then using the MOUNT FORCE statement fails. This is because the disks have been accessed by another instance and the disks are not locally accessible.

If all disks are available, then using the FORCE option causes the MOUNT command to fail. This discourages unnecessary and improper use of the feature.

The following example shows how to use the FORCE option to force the mount of the data1 disk group:

ALTER DISKGROUP data1 MOUNT FORCE