Web-based Oracle Enterprise Manager Database Control and Grid Control interfaces let you manage Oracle Real Application Clusters (Oracle RAC) databases. The Enterprise Manager console is a central point of control for the Oracle environment. Use the Database Control console to initiate cluster database management tasks. Use the Grid Control console to administer multiple Oracle RAC databases and cluster nodes.
This chapter describes how to administer your Oracle RAC environment. It explains the startup and shutdown tasks for database components and how to administer parameters and parameter files in Oracle RAC. This chapter includes the following sections:
Starting and Stopping Oracle RAC Databases and Database Instances
About Oracle Real Application Clusters Initialization Parameters
Oracle Real Application Clusters (Oracle RAC) provides technology that links two or more individual computers so that they function as one system. Oracle RAC enables each computer that is a member of a cluster, or node, to share access to the Oracle database. If one cluster node fails or is taken offline, then the other cluster nodes continue operating and the entire Oracle RAC database remains available. Two or more inexpensive computers can appear to applications as if they were a single, much more powerful and more expensive, computer.
To increase the performance of an Oracle RAC database, you can add cluster nodes. Each additional node can help speed up application processing, support more users or processes, or both. In addition, you can also add cluster nodes to increase the availability and reliability of a two-node Oracle RAC database. The more nodes that your Oracle RAC environment has, the less the impact of the loss of any individual node on the database.
Note:
If you are using Oracle Database Standard Edition, then your cluster must adhere to the license restrictions. See Oracle Database Licensing Information for specific details on these restrictions.An Oracle RAC database requires three components: cluster nodes, shared storage, and Oracle Clusterware. Although you can choose how many nodes your cluster should have and what type of shared storage to use, this guide describes one specific configuration for a two-node cluster. This two-node configuration uses Oracle Automatic Storage Management (Oracle ASM) for storage management and Recovery Manager (RMAN) for the backup and recovery strategy.
Most administration tasks are the same for Oracle single-instance and Oracle RAC databases. This guide provides additional instructions for database administration tasks specific to Oracle RAC, and recommendations for managing Oracle RAC databases.
Caution:
By default, any named user may create a server pool. To restrict the operating system users that have this privilege, Oracle strongly recommends that you add specific users to the CRS Administrators list. See Oracle Clusterware Administration and Deployment Guide for more information about adding users to the CRS Administrators list.Oracle RAC One Node databases are administered slightly differently from Oracle RAC or single-instance databases. For administrator-managed Oracle RAC One Node databases, you must monitor the candidate node list and make sure a server is always available for failover if possible. Candidate servers reside in the Generic server pool and the database and its services will fail over to one of those servers. For policy-managed Oracle RAC One Node databases, you must ensure that the server pools are configured such that a server will be available for the database to fail over to in case its current node becomes unavailable. Also, for policy-managed Oracle RAC One Node databases, the destination node for online database relocation must be located in the database's server pool.
Oracle Real Application Clusters One Node (Oracle RAC One Node) is a single instance of an Oracle Real Application Clusters (Oracle RAC) database that runs on one node in a cluster. Instead of stopping and starting instances, you use Oracle RAC One Node online database relocation to relocate the Oracle RAC One Node instance to another server.
See Also:
Oracle Real Application Clusters Administration and Deployment Guide for more information about managing Oracle RAC One Node databasesThe Web-based Oracle Enterprise Manager Database Control console and the Oracle Enterprise Manager Grid Control console let you manage Oracle RAC and Oracle RAC One Node databases. Enterprise Manager is a central point of control for the Oracle environment that you access by way of a graphical user interface (GUI). You can use Enterprise Manager to create and modify services, and to start and stop the cluster database instances and the cluster database. Use Enterprise Manager Database Control for cluster database management tasks. Use Enterprise Manager Grid Control to administer your entire Oracle RAC environment, not just the Oracle RAC database.
When you log in to Enterprise Manager using a client browser, the Cluster Database Home page appears. The Cluster Database Home page is similar to a single-instance Database Home page. However, on the Cluster Database Home page, Enterprise Manager displays the system state and availability of the entire Oracle RAC environment. This includes a summary about alert messages and job activity, and the status of and links to all the database and Oracle Automatic Storage Management (Oracle ASM) instances. By clicking the Cluster tab on this page you can view the Cluster Home page, to view the status of and alerts for the underlying cluster.
See Also:
Oracle Real Application Clusters Administration and Deployment Guide for more information about monitoring Oracle RAC performance
Typically, you start and shut down the cluster database from the Enterprise Manager Cluster Database Home page. By using this page for cluster database startup and shutdown operations, you ensure that all the instances that belong to the Oracle RAC database are in a consistent state. This enables you to more easily manage an Oracle RAC database.
You can start and stop individual instances in an Oracle RAC database. Starting and stopping one instance in an Oracle RAC database does not stop or start the other database instances. To completely stop an Oracle RAC database, you must shut down all of its instances.
To start and stop an entire Oracle RAC database, assuming you are using a server parameter file (SPFILE):
Go to the following URL and log in to Enterprise Manager:
http://hostname:portnumber/em
For example, http://racnode1.example.com:1158/em
.
On the Cluster Database Home page, in the General section, click Startup if the database is down, or Shutdown if the database is started.
The Startup/Shutdown: Specify Credentials page appears.
Enter the host credentials for the cluster nodes. The host credentials are the user name and password for a user who is a member of the OSDBA
or OSOPER
operating system group.
The Startup/Shutdown: Select Operation page appears.
Click Select All to select all the instances, or then click Shutdown to stop all the database instances or Startup to start all the database instances.
The Startup/Shutdown: Confirmation page appears.
Click Yes.
To start and stop individual instances, go to the Startup/Shutdown: Select Operation page and select the database instances, then click Startup or Shutdown to perform the desired operation on the selected database instances. You can also start and shut down instances using SQL*Plus or Server Control (SRVCTL).
Note:
You can start and shut down individual instances from each instance's home page. However, it is easier to perform instance startup and shutdown operations directly from the Startup/Shutdown: Select Operation page.See Also:
Oracle Real Application Clusters Administration and Deployment Guide for more information about using command-line interfaces to start and stop Oracle RAC database instances
Managing initialization parameters for an Oracle RAC database is essentially the same as managing them for a single-instance Oracle database. Note the following differences for parameters in Oracle RAC databases:
Parameters that are cluster-specific have the value Cluster Database in the Category column.
Parameters that are the same on each instance in the Oracle RAC database are identified in the Instance column with an asterisk (*).
Parameters that are set to different values on each instance of an Oracle RAC database are listed by instance name.
The administration of initialization parameters in an Oracle RAC environment is slightly different from single-instance database parameter administration. For example, if you change a parameter setting that is marked by an asterisk, which indicates that the parameter is a clusterwide database initialization parameter, then you change that parameter's setting for all the instances in your Oracle RAC database. If you change an initialization parameter prefixed with an instance name, or an instance-specific initialization parameter, then the change affects only that instance; the change does not affect the parameter's settings on other database instances.
This section contains the following topics:
About Configuring Initialization Parameters for an Oracle RAC Database
Editing Initialization Parameter Settings for an Oracle RAC Database
About the Server Parameter File for Oracle Real Application Clusters
See Also:
Oracle Real Application Clusters Administration and Deployment Guide for more information about initialization parameters in an Oracle RAC environment
A server parameter file (SPFILE) is a type of repository for initialization parameters that is maintained on the server where the Oracle database server runs, or on shared storage for an Oracle RAC database. Initialization parameters stored in a server parameter file are persistent, in that any changes made to the parameters while an instance is running can persist across instance shutdown and startup.
An initialization parameter file is a text file that contains initialization parameter settings. In contrast to the SPFILE, this parameter file is not binary and does not need to be located on the database server. The text-based initialization parameter file can be read by the database, but it is not written to by the database.
By default, Oracle Database sets most parameters to a default value and this value is the same across all instances. However, many initialization parameters can also have different values on different instances as described in Oracle Database Reference. Other parameters must be either unique or identical across instances, as described in the following sections:
Parameters that Must Have Identical Settings on All Instances
Parameters that Should Have Identical Settings on All Instances
Certain initialization parameters that are critical at database creation or that affect certain database operations must have the same value for every instance in an Oracle RAC database. Specify these parameter values in the SPFILE, or within the individual PFILEs for each instance. The following list contains the parameters that must be identical on every instance:
For the following parameters, the parameter must be set on all instances to a value that is unique to that instance:
ROLLBACK_SEGMENTS
(if the UNDO_MANAGEMENT
initialization parameter is not set to AUTO
)
UNDO_TABLESPACE
(if UNDO_MANAGEMENT
is set to AUTO
)
Oracle Database uses the INSTANCE_NUMBER
parameter to distinguish among instances at startup. Oracle Database uses the INSTANCE_NAME
parameter to assign redo log groups to specific instances. To simplify administration, use the same number for both the INSTANCE_NAME
and INSTANCE_NUMBER
parameters. With Oracle Database 11.2 using Grid Plug and Play, the instance name parameter is no longer required. If you do not specify the instance name, then instance name defaults to db_unique_name
_number
.
Oracle recommends that you set the values for the following parameters to the same value on all instances. Although you can have different settings for these parameters on different instances, setting each parameter to the same value on all instances simplifies administration:
The SERVICE_NAMES
initialization parameter specifies one or more names by which clients can connect to the instance. The instance registers its service names with the listener. When a client requests a service, the listener determines which instances offer the requested service and routes the client to the appropriate instance.
In an Oracle RAC database, you should not modify this parameter directly. Instead, define services for your database and database instances using the Clustered Managed Database Services page in Oracle Enterprise Manager. If you must change a service, then you can use either Enterprise Manager or SRVCTL. When you use either Oracle Enterprise Manager or SRVCTL to create and start the service, the SERVICE_NAMES
parameter is updated automatically once the service is active.
See Also:
When you create the database, Oracle creates an SPFILE in the file location that you specify. This location can be an Oracle ASM disk group or a file on a cluster file system. In the environment described by this guide, the SPFILE is created on an Oracle ASM disk group.
All instances in the cluster database use the same SPFILE at startup. Oracle RAC uses a traditional parameter file only if an SPFILE does not exist or if you specify PFILE
in your STARTUP
command. Oracle recommends that you use an SPFILE to simplify administration, maintain parameter setting consistency, and to guarantee parameter setting persistence across database shutdown and startup events. In addition, you can configure RMAN to back up your SPFILE.
See Also:
Oracle Real Application Clusters Administration and Deployment Guide for more information about using a server parameter file in an Oracle Real Application Clusters environment
Oracle Database Backup and Recovery User's Guide for information about using RMAN to backup the SPFILE
You can use Enterprise Manager to view and edit the initialization parameter settings for your Oracle RAC database.
To view or modify the initialization parameters using Enterprise Manager:
On the Cluster Database Home page, while logged in as a SYSDBA user, click Server.
The Server page appears.
Under the heading Database Configuration, select Initialization Parameters.
The Initialization Parameters page appears.
Select either the Current or SPFile subpage to modify the parameter settings.
See either:
The Current subpage of the Initialization Parameters page contains a list of configuration parameters for that instance and database. You can set these parameters to particular values to initialize many of the memory and process settings of an Oracle instance. When you modify initialization parameters using the Current tab, the changes are applied only to the running instances, not the SPFILE, unless the "Apply changes in current running instance(s) mode to SPFile" option is selected.
The Instance column shows the instances for which the parameter has the value listed in the table. An asterisk (*) indicates that the parameter has the same value for all remaining instances of the cluster database. For example, if open_cursors = 250
for node1
and open_cursors = 300
for node2
, then the Instance column for open_cursors = 250
displays an asterisk, but the Instance column for open_cursors = 300
contains "node2". This shorthand notation saves space when the cluster databases has many instances.
You can filter the Initialization Parameters page to show only those parameters that meet the criteria of the filter you enter in the Filter by name field. Optionally, you can select Show All to display on one page all parameters currently used by the running instance(s).
To modify parameter values using the Current tab:
Select a parameter from the Select column and do one of the following:
Click Add to add the selected parameter to a different instance.
When the new row appears in the table, choose the instance name to which the new setting applies, and enter the new value for the parameter in the Value field. Optionally, you can put text in the Comment field to indicate why the instance has a different value from the other instances.
Click Reset to reset the value of the selected parameter. Note that using this option you can reset only those parameters that do not have an asterisk in the Instance column.
When you click Reset, the value of the selected parameter is reset to the value of the remaining instances (that is, the row with the asterisk in the Instance column) and the row that has the local instance name in the Instance field is removed from the table.
For example, select the parameter OPEN_CURSORS
, then click Add. In the new entry for OPEN_CURSORS
, select an Instance (for example sales1
or RACDB2
), change the Value field to 300.
After you make changes to one or more of the parameters, click Apply to accept and implement the changes.
You can also add or reset parameters using the SPFile tab. When you modify initialization parameters using the SPFile tab, the changes are applied only to the SPFILE, not the currently running instances, unless the "Apply changes in SPFile mode to the current running instance(s)" option is selected.
Resetting parameters using the SPFile tab is different than resetting the same parameters using the Current tab. You can either reset the parameter value for an instance back to the default value for all instances, or you can delete the default parameter setting (unset the parameter) for all instances.
If you reset a parameter with an asterisk in the Instance column, then the entry is deleted from both the SPFILE and the table in Database Control. Only parameters without an asterisk (or instance-specific parameters) remain.
If you reset the only entry for a nonasterisk parameter, then it is deleted from both the SPFILE and the table in Database Control, but the parameter is replaced by a dummy parameter with an empty value field and an asterisk in the Instance column. This enables you to specify a new value for the parameter, add new instance-specific entries for the parameter, and so on.
Resetting a parameter that is set for only one instance results in the parameter being unset for all instances.
To modify parameter values using the SPFile tab:
Choose a parameter using the Select column.
For more information about each parameter, click the information icon in the Help column next to the parameter.
Click Add, then edit the entry to change the target instance or parameter value. You can optionally add a comment describing the reason for the change.
Click Apply to apply the changes to the SPFILE, or to the SPFile and the currently running instance if you selected the "Apply changes in SPFile mode to the current running instance(s)" option.
On the SPFile subpage of the Initialization Parameters page, select a parameter from the Select column and do one of the following:
Click Add to add the selected parameter to a different instance.
When the new row appears in the table, choose the instance name to which the new setting applies, and then enter the new value for the parameter in the Value field. Optionally, you can put text in the Comment field describing the reason for the change.
Click Reset to reset the value of the selected parameter.
When you click Reset, one of the following actions is performed:
If the entry you selected was for a specific instance, then the value of the selected parameter for that instance is reset to the value of the remaining instances (indicated by an asterisk in the Instance column). The entry that has the local instance name in the Instance field is deleted.
If the entry you selected to reset was the default values for all instances (indicated by an asterisk in the Instance column), then the value of the selected parameter is unset for all instances, but any instance-specific parameter entries for the same parameter are not changed.
If you reset the only entry for a parameter, regardless of whether the entry applies to all instances or a single instance, then the parameter is unset for all instances in the cluster database.
After you make changes to one or more of the parameters, click Apply to accept and implement the changes.
Suppose that the open_cursors
parameter has two entries in the SPFILE:
*.open_cursors = 200 RACDB2.open_cursors = 250
Using the Initialization Parameters page with the SPFile tab selected, if you click Reset for *.open_cursors
, then Enterprise Manager deletes that entry from both the SPFILE and the displayed list of parameters, leaving only RACDB2.open_cursors = 250
displayed.
If you click Reset for RACDB2.open_cursors
, then Enterprise Manager also deletes this parameter entry from both the SPFILE and the displayed list of parameters, but then a new entry, *.open_cursors = <NULL>
is added to the displayed list of parameters for the reset parameter.
See Also:
Oracle Real Application Clusters Administration and Deployment Guide for more information about using a server parameter file in an Oracle Real Application Clusters environment
Most administration tasks for managing storage are the same for Oracle single-instance and Oracle RAC databases. This section provides additional information for using Enterprise Manager to manage some storage structures of an Oracle RAC database.
This section describes the following topics:
Oracle RAC automatically manages undo segments within a specific undo tablespace that is assigned to an instance. Only the instance assigned to the undo tablespace can modify the contents of that tablespace. However, each instance can read the undo data blocks created by any instance. Also, when performing transaction recovery, any instance can update any undo tablespace, if that undo tablespace is not currently being used by another instance for undo generation or transaction recovery. You assign undo tablespaces in your Oracle RAC database by specifying a different value for the UNDO_TABLESPACE
parameter for each instance in your SPFILE or individual PFILEs. You cannot simultaneously use automatic undo management and manual undo management in an Oracle RAC database. In other words, all instances of an Oracle RAC database must operate in the same undo mode.
See Also:
Oracle Database 2 Day DBA for more information about managing the undo data for your database
Oracle Automatic Storage Management (Oracle ASM) automatically optimizes storage to maximize performance by managing the storage configuration across the disks that Oracle ASM manages. Oracle ASM does this by evenly distributing the storage load across all the available storage within your cluster database environment. Oracle ASM partitions your total disk space requirements into uniformly sized units across all the disks in a disk group. Oracle ASM can also automatically mirror data to prevent data loss. Because of these features, Oracle ASM also significantly reduces your administrative overhead.
As in single-instance Oracle databases, using Oracle ASM in Oracle RAC does not require I/O tuning. The following topics describe Oracle ASM and Oracle ASM administration:
When you configure Oracle ASM after installing Oracle Grid Infrastructure for a cluster, one Oracle ASM instance is created on each node in your Oracle RAC environment if one does not exist. Each Oracle ASM instance has either an SPFILE or PFILE type parameter file. For the environment described in this guide, the Oracle ASM instances use SPFILEs.
When you create a disk group for a cluster, or add new disks to an existing clustered disk group, you must prepare only the underlying physical storage on shared disks. The shared disk requirement is the only substantial difference between using Oracle ASM in an Oracle RAC database compared to using it in a single-instance Oracle database. Oracle ASM automatically rebalances the storage load after you add or delete a disk or disk group.
In a cluster, each Oracle ASM instance manages the metadata updates to the disk groups for the node on which it is running. In addition, each Oracle ASM instance coordinates disk group metadata with other nodes in the cluster. See Figure 1-2, "Oracle Instances and Oracle ASM Storage" for an illustration of the relationship between Oracle RAC instances, Oracle ASM instances, and the shared storage. As in single-instance Oracle databases, you can use Enterprise Manager, Oracle ASM configuration assistant (ASMCA), Oracle ASM command-line utility (ASMCMD), SQL*Plus, and SRVCTL to administer disk groups for Oracle ASM in an Oracle RAC environment.
See Also:
Oracle Automatic Storage Management Administrator's Guide for information about how to use SQL*Plus to administer Oracle ASM instances
You can configure Oracle ASM during installation of Oracle Grid Infrastructure for a cluster. If you choose not to configure Oracle ASM during installation, then you must manually configure Oracle ASM, or perform standalone Oracle ASM disk group management.
To manually configure Oracle ASM, you must first run ASMCA to create the Oracle ASM instances and create at least one diskgroup. Once Oracle ASM is running, then you can use DBCA to create an Oracle RAC database that uses Oracle ASM for storage.
Note:
If you installed the Oracle Cluster Registry (OCR) and Voting Disks on Oracle ASM as part of your Oracle Grid Infrastructure for a cluster install, then the Oracle ASM instances are created by OUI and you do not have to run ASMCA. You must use ASMCA only if you did not specify Oracle ASM storage for the OCR and Voting disks during installation.You do not have to create a database to modify Oracle ASM storage properties.
See Also:
Oracle Automatic Storage Management Administrator's Guide for information about how to use the Oracle Automatic Storage Management command-line utility
You can perform administrative operations on Oracle ASM disk groups using Enterprise Manager such as adding and deleting them. You can also monitor Oracle ASM disk group performance and control disk group availability at the instance level. For example, Enterprise Manager provides some Oracle RAC-specific features for Oracle ASM, such as the following:
When you add a disk group, the disk group definition includes a check box to indicate whether the disk group is automatically mounted to all the cluster database instances.
The default Disk Group Performance page displays instance-level performance details when you click a performance characteristic such as Write Response Time or I/O Throughput.
When you mount and dismount Oracle ASM disk groups, you can use a check box to indicate which instances should mount or dismount a particular Oracle ASM Disk Group.
Managing redo log files in Oracle RAC environments is similar to managing redo log files in single-instance Oracle Database environments. This section provides an overview of some of the additional concepts and procedures for configuring redo log files in Oracle RAC environments.
Redo logs contain a record of changes that have been made to data files. In a single-instance Oracle database, redo logs are stored in two or more redo log file groups. Each of these groups contains a redo log file and possibly one or more mirrored copies of that file. In an Oracle RAC database, each instance requires its own set of redo log groups, which is known as a redo thread. Mirrored copies of the redo log files provide your system with extra protection against data loss that is due to hardware failures or data corruption. If a redo log file is unreadable, then the Oracle Database attempts to access its mirrored copy. The redo log file mirrors should be located on different disk devices from the primary redo log files.
Figure 4-1 Illustration of Redo Threads for an Oracle RAC Database
Each instance's redo thread must contain at least two redo log groups. Each redo log group should contain at least two members: a redo log and its mirrored copy. If you create your Oracle RAC database using DBCA, then your Oracle RAC database automatically implements a configuration that meets the Oracle recommendations.
You should create redo log groups only if you are using administrator-managed databases. For policy-managed databases, if an instance starts due to a change in server pool cardinality, then Oracle Database automatically creates redo log files, enables a redo thread for the instance if there is not a redo thread allocated to that instance, and creates the undo tablespace if there is not an undo tablespace allocated to that instance. The database must be using Oracle Managed Files and Oracle ASM in this situation. See Oracle Real Application Clusters Administration and Deployment Guide for more information.
In an Oracle RAC database, all the redo log files reside on shared storage. In addition, each instance must have access to the redo log files of all the other instances in the cluster. If your Oracle RAC database uses Oracle ASM, then Oracle ASM manages the shared storage for the redo log files and the access to those files.
Note:
DBCA creates redo threads and undo tablespaces up to the maximum instance parameter at the time of creation for better performance.In an Oracle RAC database, each instance writes and archives the redo log groups in its redo thread in the same manner that single-instance Oracle databases do. However, in recovery mode, the instance performing the recovery can read and process all the redo threads for the database, regardless of which instance generated the redo thread. Being able to read all the redo threads enables a running instance to recover the work completed by one or more failed instances.
In case of instance failure, a surviving instance can read the redo logs of the failed instance. Users can continue to access and update the database without waiting for the failed instance to be restarted. For example, assume that you have an Oracle RAC database with two instances, instance A and instance B. If instance A is down, then instance B can read the redo log files for both instance A and B to ensure a successful recovery.
See Also:
"Administering Redo Logs in Oracle RAC"On the Redo Log Groups page, you can create additional redo log groups and add members to the redo log group. The Thread column identifies the instance, or redo thread, to which a redo log file belongs.
To access the redo log file groups using Enterprise Manager:
On the Cluster Database Home Page, select Server.
The Server page appears.
In the Storage section, select Redo Log Groups.
The Redo Log Groups page appears.
On this page you can perform the following tasks:
View the status, number of members, thread, file size, status, and archive statue of each redo log group
Create or delete log groups
Edit a redo log group to add or remove members
Perform other redo log group management tasks, including clearing log files, duplicating a log group, generating sizing advice, and forcing a log switch
See Also:
Oracle Real Application Clusters Administration and Deployment Guide for additional information about redo threads in an Oracle RAC environment
Oracle Database 2 Day DBA for more information about creating online redo log files