4 Adding and Deleting Cluster Nodes

Adding a Cluster Node on Linux and UNIX Systems

This procedure describes how to add a node to your cluster. This procedure assumes that:

• There is an existing cluster with two nodes named node1 and node2

• You are adding a node named node3 using a virtual node name, node3-vip, that resolves to an IP address, if you are not using Grid Naming Service (GNS)

• You have successfully installed Oracle Clusterware on node1 and node2 in a local (non-shared) home, where Grid_home represents the successfully installed home

To add a node:

1. Ensure that you have successfully installed Oracle Clusterware on at least one node in your cluster environment. To perform the following procedure, Grid_home must identify your successfully installed Oracle Clusterware home.

   See Also:

   Oracle Grid Infrastructure Installation Guide for Oracle Clusterware installation instructions

2. Verify the integrity of the cluster and node3:

   $ cluvfy stage -pre nodeadd -n node3 [-fixup [-fixupdir fixup_dir]] [-verbose]
   

   You can specify the -fixup option and a directory into which CVU prints instructions to fix the cluster or node if the verification fails.

3. To extend the Grid Infrastructure home to the node3, navigate to the Grid_home/oui/bin directory on node1 and run the addNode.sh script as the user that installed Oracle Clusterware using the following syntax, where node3 is the name of the node that you are adding and node3-vip is the VIP name for the node:

   If you are using Grid Naming Service (GNS), run the following command:

   $ ./addNode.sh "CLUSTER_NEW_NODES={node3}"
   

   If you are not using GNS, run the following command:

   $ ./addNode.sh "CLUSTER_NEW_NODES={node3}" "CLUSTER_NEW_VIRTUAL_
   HOSTNAMES={node3-vip}"
   

   Note:

   You can specify multiple nodes for the CLUSTER_NEW_NODES and the CLUSTER_NEW_VIRTUAL_HOSTNAMES parameters by entering a comma-separated list of nodes between the braces. For example:

   "CLUSTER_NEW_NODES={node3,node4,node5}"
   "CLUSTER_NEW_VIRTUAL_HOSTNAMES={node3-vip,node4-vip,node5-vip}"
   

   Alternatively, you can specify the entries shown in Example 4-1 in a response file, where file_name is the name of the file, and run the addNode.sh script, as follows:

   $ ./addNode.sh -responseFile file_name
   

   Example 4-1 Response File Entries for Adding Oracle Clusterware Home

   RESPONSEFILE_VERSION=2.2.1.0.0
   
   CLUSTER_NEW_NODES={node3}
   CLUSTER_NEW_VIRTUAL_HOSTNAMES={node3-vip}
   

   See Also:

   Oracle Universal Installer and OPatch User's Guide for details about how to configure command-line response files

   Notes:

   • If you are not using Oracle Grid Naming Service (GNS), then you must add the name and IP address of node3 to DNS.

   • Command-line values always override response file values.

4. If you have an Oracle RAC or Oracle RAC One Node database configured on the cluster and you have a local Oracle home, then do the following to extend the Oracle database home to node3:

   1. Navigate to the Oracle_home/oui/bin directory on node1 and run the addNode.sh script as the user that installed Oracle RAC using the following syntax:

      $ ./addNode.sh "CLUSTER_NEW_NODES={node3}"
      

   2. Run the Oracle_home/root.sh script on node3 as root, where Oracle_home is the Oracle RAC home.

   If you have a shared Oracle home that is shared using Oracle Automatic Storage Management Cluster File System (Oracle ACFS), then do the following as the user that installed Oracle RAC to extend the Oracle database home to node3:

   1. Run the Grid_home/root.sh script on node3 as root, where Grid_home is the Oracle Grid Infrastructure home.

   2. Start the Oracle ACFS resource on the new node by running the following command as root from the Grid_home/bin directory:

      # srvctl start filesystem -d volume_device_name [-n node_name]
      

      Note:

      .Make sure the Oracle ACFS resources, including Oracle ACFS registry resource and Oracle ACFS file system resource where the ORACLE_HOME is located, are online on the newly added node.

   3. Navigate to the Oracle_home/oui/bin directory on node1 and run the addNode.sh script as the user that installed Oracle RAC using the following syntax:

      $ ./addNode.sh -noCopy "CLUSTER_NEW_NODES={node3}"
      

      Note:

      Use the -noCopy option because the Oracle home on the destination node is already fully populated with software.

   If you have a shared Oracle home on a shared file system that is not Oracle ACFS, then you must first create a mount point for the Oracle RAC database home on the target node, mount and attach the Oracle RAC database home, and update the Oracle Inventory, as follows:

   1. Run the srvctl config db -d db_name command on an existing node in the cluster to obtain the mount point information.

   2. Run the following command as root on node3 to create the mount point:

      # mkdir -p mount_point_path
      

   3. Mount the file system that hosts the Oracle RAC database home.

   4. Run the following command as the user that installed Oracle RAC from the Oracle_home/oui/bin directory on the node you are adding to add the Oracle RAC database home:

      $ ./runInstaller -attachHome ORACLE_HOME="ORACLE_HOME" "CLUSTER
      _NODES={node_list}" LOCAL_NODE="node_name"
      

   5. Update the Oracle Inventory as the user that installed Oracle RAC, as follows:

      $ ./runInstaller -updateNodeList ORACLE_HOME=mount_point_path "CLUSTER
      _NODES={node_list}"
      

      In the preceding command, node_list refers to a list of all nodes where the Oracle RAC database home is installed, including the node you are adding.

5. Run the Grid_home/root.sh script on the node3 as root and run the subsequent script, as instructed.

6. Run the following CVU command as the user that installed Oracle Clusterware to check cluster integrity. This command verifies that any number of specified nodes has been successfully added to the cluster at the network, shared storage, and clusterware levels:

   $ cluvfy stage -post nodeadd -n node3 [-verbose]
   

   See Also:

   "cluvfy stage [-pre | -post] nodeadd" for more information about this CVU command

Check whether either a policy-managed or administrator-managed Oracle RAC database is configured to run on node3 (the newly added node). If you configured an administrator-managed Oracle RAC database, you may need to use DBCA to add an instance to the database to run on this newly added node.

Deleting a Cluster Node on Linux and UNIX Systems

This section describes the procedure for deleting a node from a cluster.

To delete a node from a cluster:

1. Ensure that Grid_home correctly specifies the full directory path for the Oracle Clusterware home on each node, where Grid_home is the location of the installed Oracle Clusterware software.

2. Run the following command as either root or the user that installed Oracle Clusterware to determine whether the node you want to delete is active and whether it is pinned:

   $ olsnodes -s -t
   

   If the node is pinned, then run the crsctl unpin css command. Otherwise, proceed to the next step.

3. Disable the Oracle Clusterware applications and daemons running on the node. Run the rootcrs.pl script as root from the Grid_home/crs/install directory on the node to be deleted, as follows:

   Note:

   Before you run this command, you must stop the EMAGENT, as follows:

   $ emctl stop dbconsole
   

   # ./rootcrs.pl -deconfig -force
   

   If you are deleting multiple nodes, then run the rootcrs.pl script on each node that you are deleting.

   If you are deleting all nodes from a cluster, then append the -lastnode option to the preceding command to clear OCR and the voting disks, as follows:

   # ./rootcrs.pl -deconfig -force -lastnode
   

   Caution:

   Only use the -lastnode option if you are deleting all cluster nodes because that option causes the rootcrs.pl script to clear OCR and the voting disks of data.

   Note:

   If you do not use the -force option in the preceding command or the node you are deleting is not accessible for you to execute the preceding command, then the VIP resource remains running on the node. You must manually stop and remove the VIP resource using the following commands as root from any node that you are not deleting:

   # srvctl stop vip -i vip_name -f
   # srvctl remove vip -i vip_name -f
   

   Where vip_name is the VIP for the node to be deleted. If you specify multiple VIP names, then separate the names with commas and surround the list in double quotation marks ("").

4. From any node that you are not deleting, run the following command from the Grid_home/bin directory as root to delete the node from the cluster:

   # crsctl delete node -n node_to_be_deleted
   

5. On the node you want to delete, run the following command as the user that installed Oracle Clusterware from the Grid_home/oui/bin directory where node_to_be_deleted is the name of the node that you are deleting:

   $ ./runInstaller -updateNodeList ORACLE_HOME=Grid_home "CLUSTER_NODES=
   {node_to_be_deleted}" CRS=TRUE -silent -local
   

   Caution:

   Before you proceed to the next step, check the inventory.xml to ensure that it has not been updated. If you perform this step and mistakenly update the inventory.xml file, or you fail to complete this step and move on to the next step, then the next step will deinstall the entire cluster.

6. Depending on whether you have a shared or local Oracle home, complete one of the following procedures as the user that installed Oracle Clusterware:

   • If you have a shared home, then run the following commands in the following order on the node you want to delete.

     Run the following command to deconfigure Oracle Clusterware:

     $ Grid_home/perl/bin/perl Grid_home/crs/install/rootcrs.pl -deconfig
     

     Run the following command from the Grid_home/oui/bin directory to detach the Grid home:

     $ ./runInstaller -detachHome ORACLE_HOME=Grid_home -silent -local
     

     Manually delete the following files:

     /etc/oraInst.loc
     /etc/oratab
     /etc/oracle/
     /opt/ORCLfmap/
     $OraInventory/
     

   • For a local home, deinstall the Oracle Clusterware home from the node that you want to delete, as follows, by running the following command, where Grid_home is the path defined for the Oracle Clusterware home:

     $ Grid_home/deinstall/deinstall –local
     

     Caution:

     If you do not specify the -local flag, then the command removes the Grid Infrastructure home from every node in the cluster.

7. On any node other than the node you are deleting, run the following command from the Grid_home/oui/bin directory where remaining_nodes_list is a comma-delimited list of the nodes that are going to remain part of your cluster:

   $ ./runInstaller -updateNodeList ORACLE_HOME=Grid_home "CLUSTER_NODES=
   {remaining_nodes_list}" CRS=TRUE -silent
   

   Notes:

   • You must run this command a second time where ORACLE_HOME=ORACLE_HOME, and CRS=TRUE -silent is omitted from the syntax, as follows:

     $ ./runInstaller -updateNodeList ORACLE_HOME=ORACLE_HOME
      "CLUSTER_NODES={remaining_nodes_list}"
     

   • If you have a shared Oracle Grid Infrastructure home, then append the -cfs option to the command example in this step and provide a complete path location for the cluster file system.

8. Run the following CVU command to verify that the specified nodes have been successfully deleted from the cluster:

   $ cluvfy stage -post nodedel -n node_list [-verbose]
   

   See Also:

   "cluvfy stage -post nodedel" for more information about this CVU command

Adding a Node to a Cluster on Windows Systems

Ensure that you complete the prerequisites listed in "Prerequisite Steps for Adding Cluster Nodes" before adding nodes.

This procedure describes how to add a node to your cluster. This procedure assumes that:

• There is an existing cluster with two nodes named node1 and node2

• You are adding a node named node3

• You have successfully installed Oracle Clusterware on node1 and node2 in a local home, where Grid_home represents the successfully installed home

To add a node:

1. Verify the integrity of the cluster and node3:

   C:\>cluvfy stage -pre nodeadd -n node3 [-fixup [-fixupdir fixup_dir]] [-verbose]
   

   You can specify the -fixup option and a directory into which CVU prints instructions to fix the cluster or node if the verification fails.

2. On node1, go to the Grid_home\oui\bin directory and run the addNode.bat script, as follows:

   C:\>addNode.bat "CLUSTER_NEW_NODES={node3}"
   "CLUSTER_NEW_VIRTUAL_HOSTNAMES={node3-vip}"
   

   You can alternatively specify the entries shown in Example 4-2 in a response file and run addNode.bat as follows:

   C:\>addNode.bat -responseFile filename
   

   Example 4-2 Response File Entries for Adding a Node

   CLUSTER_NEW_NODES={"node3"}
   CLUSTER_NEW_VIRTUAL_HOSTNAMES={"node3-vip"}
   

   Command-line values always override response file values.

   See Also:

   Oracle Universal Installer and OPatch User's Guide for details about how to configure command-line response files

3. Run the following command on the new node:

   C:\>Grid_home\crs\config\gridconfig.bat
   

4. Run the following command to verify the integrity of the Oracle Clusterware components on all of the configured nodes, both the preexisting nodes and the nodes that you have added:

   C:\>cluvfy stage -post crsinst -n all [-verbose]
   

After you complete the procedure in this section for adding nodes, you can optionally extend Oracle Database with Oracle Real Application Clusters (Oracle RAC) components to the new nodes, making them members of an existing Oracle RAC database.

Deleting a Cluster Node on Windows Systems

This section describes how to delete a cluster node on Windows systems. This procedure assumes that Oracle Clusterware is installed on node1, node2, and node3, and that you are deleting node3 from the cluster.

To delete a cluster node on Windows systems:

1. Before you delete a node, you must disable the Oracle Clusterware applications and services running on the node.

   On the node you want to delete, run the rootcrs.pl -deconfig script as a member of the Administrators groups, as follows:

   C:\>Grid_home\perl\bin\perl -I$Grid_home\perl\lib -I$Grid_home\crs\install
   Grid_home\crs\install\rootcrs.pl -deconfig -force
   

   If you are deleting multiple nodes, then run this script on each node that you are deleting.

2. On a node that you are not deleting, run the following command:

   C:\>Grid_home\bin\crsctl delete node -n node_to_be_deleted
   

3. Only if you have a local home, on the node you want to delete, run the following command with -local option to update the node list:

   C:\>Grid_home\oui\bin\setup.exe -updateNodeList ORACLE_HOME=Grid_home "CLUSTER_NODES=
   {node_to_be_deleted}" CRS=TRUE -local
   

4. Run the deinstall tool located in the Grid_home\deinstall directory to deinstall and deconfigure the Oracle Clusterware home, as follows:

   C:\>deinstall.bat
   

5. On any node that you are not deleting, run the following command from the Grid_home\oui\bin directory where remaining_nodes_list is a comma-delimited list of the nodes that are going to remain part of your cluster:

   C:\>setup.exe –updateNodeList ORACLE_HOME=Grid_home
   "CLUSTER_NODES={remaining_nodes_list}" CRS=TRUE -silent
   

   Notes:

   • You must run this command a second time where ORACLE_HOME=ORACLE_HOME, and CRS=TRUE -silent is omitted from the syntax, as follows:

     C:\>setup.exe -updateNodeList ORACLE_HOME=ORACLE_HOME
      "CLUSTER_NODES={remaining_nodes_list}"
     

   • If you have a shared Oracle Grid Infrastructure home, then append the -cfs option to the command example in this step and provide a complete path location for the cluster file system.

6. Run the following CVU command to verify that the specified nodes have been successfully deleted from the cluster:

   C:\>cluvfy stage -post nodedel -n node_list [-verbose]
   

   See Also:

   "cluvfy stage -post nodedel"