2 Advanced Installation Oracle Grid Infrastructure for a Cluster Preinstallation Tasks

2.1.1 Completing a Minimal Linux Installation

To complete a minimal Linux installation, select one of the minimal installation options (either a custom installation where you select the Minimal option from Package Group Selection, or where you deselect all packages except for the Base pack). This installation lacks many RPMs required for database installation, so you must use an RPM package for your Oracle Linux release to install the required packages. The package you use depends on your Linux release, and your support status with Unbreakable Linux Network (ULN).

Refer to the following URL for documentation regarding installation of a reduced set of packages:

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=728346.1

SSH is required for Oracle Grid Infrastructure installation. OpenSSH should be included in the Linux distribution minimal installation. To confirm that SSH packages are installed, enter the following command:

# rpm -qa |grep ssh

If you do not see a list of SSH packages, then install those packages for your Linux distribution.

2.1.2 Completing a Default Linux Installation

If you do not install the Oracle Preinstallation RPM, then Oracle recommends that you install your Linux operating system with the default software packages (RPMs). This installation includes most of the required packages and helps you limit manual verification of package dependencies. Oracle recommends that you do not customize the RPMs during installation.

For information about a default installation, log on to My Oracle Support:

https://support.oracle.com

Search for "default rpms linux installation," and look for your Linux distribution. For example:

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=ID 401167.1

After installation, review system requirements for your distribution to ensure that you have all required kernel packages installed, and complete all other configuration tasks required for your distribution and system configuration.

2.1.3 About Oracle Linux and the Unbreakable Enterprise Kernel

Oracle's Unbreakable Enterprise Kernel delivers the latest innovations from upstream development to customers who run Red Hat Enterprise Linux (RHEL) 5 or Oracle Linux 5 in the data center. The Unbreakable Enterprise Kernel is included and enabled by default starting with Oracle Linux 5 Update 6.

The Unbreakable Enterprise Kernel is based on a recent stable mainline development Linux kernel, and also includes optimizations developed in collaboration with Oracle Database, Oracle middleware, and Oracle hardware engineering teams to ensure stability and optimal performance for the most demanding enterprise workloads.

Oracle highly recommends deploying the Oracle Unbreakable Enterprise Kernel in your Linux environment, especially if you run enterprise applications. However, using Oracle Unbreakable Enterprise Kernel is optional. If you require strict RHEL kernel compatibility, then Oracle Linux also includes a kernel compatible with the RHEL Linux kernel, compiled directly from the RHEL source code.

You can obtain more information about the Oracle Unbreakable Enterprise Kernel for Linux at the following URL:

http://www.oracle.com/us/technologies/linux

The Oracle Unbreakable Enterprise Kernel for Linux is the standard kernel used with Oracle products. The build and QA systems for Oracle Database and other Oracle products use the Oracle Unbreakable Enterprise Kernel for Linux exclusively. The Oracle Unbreakable Enterprise Kernel for Linux is also the kernel used in Oracle Exadata and Oracle Exalogic systems. Oracle Unbreakable Enterprise Kernel for Linux is used in all benchmark tests on Linux in which Oracle participates, as well as in the Oracle Preinstallation RPM program for x86-64.

Ksplice, which is part of Oracle Linux, updates the Linux operating system (OS) kernel, while it is running, without requiring restarts or any interruption. Ksplice is available only with Oracle Linux.

2.1.4 About the Oracle Preinstallation and Oracle Validated RPMs

If your Linux distribution is Oracle Linux and you are an Oracle Linux customer, then you can complete most preinstallation configuration tasks by using the Oracle Preinstallation RPM or Oracle Validated RPM for your release, which is available from the Oracle Linux Network, or available on the Oracle Linux DVDs. Using the Oracle Preinstallation RPM is not required, but Oracle recommends you use it to save time in setting up your cluster servers.

When it is installed, the Oracle Preinstallation RPM or Oracle Validated RPM does the following:

• Automatically downloads and installs any additional RPM packages needed for installing Oracle Grid Infrastructure and Oracle Database, and resolves any dependencies

• Creates an oracle user, and creates the oraInventory (oinstall) and OSDBA (dba) groups for that user

• As needed, sets sysctl.conf settings, system startup parameters, and driver parameters to values based on recommendations from the Oracle Preinstallation RPM program

• Sets hard and soft resource limits

• Sets other recommended parameters, depending on your kernel version

To become an Oracle Linux Network customer, contact your sales representative, or purchase a license from the Oracle Linux store:

https://shop.oracle.com/product/oraclelinux

To register your server on the Unbreakable Linux Network, or to find out more information, see the following URL:

https://linux.oracle.com

If you are using Oracle Linux 5.2 and higher, then the Oracle Preinstallation RPM is included on the install media.

You can obtain Linux kernels from the Oracle Linux public Yum repository. The public Yum repository for Oracle Linux 4, Oracle Linux 5 and Oracle Linux 6 is available at the following URL:

http://public-yum.oracle.com/

To become an Unbreakable Linux Network customer, contact your sales representative, or purchase a license from the Unbreakable Linux store:

https://shop.oracle.com/product/oraclelinux

To register your server on the Unbreakable Linux Network, or to find out more information, refer to the following URL:

https://linux.oracle.com

If you are using Oracle Linux 4.7 and higher, or Oracle Linux 5.2 and higher, then the Oracle Preinstallation or Oracle Validated RPM is included on the install media.

2.1.5 Installing the Oracle Preinstallation or Oracle Validated RPM with ULN Support

Use the following procedure to subscribe to Unbreakable Linux Network (ULN) Oracle Linux channels, and to add the Oracle Linux channel that distributes the Oracle Preinstallation RPM or Oracle Validated RPM for your release:

1. Register your server with Unbreakable Linux Network (ULN). By default, you are registered for the Oracle Linux Latest channel for your operating system and hardware.

2. Log in to Unbreakable Linux Network:

   https://linux.oracle.com

3. Click the Systems tab, and in the System Profiles list, select a registered server. The System Details window opens and displays the subscriptions for the server.

4. Click Manage Subscriptions. The System Summary window opens.

5. From the Available Channels list, select the Linux installation media copy and update patch channels corresponding to your Oracle Linux distribution. For example, if your distribution is Oracle Linux 5 Update 6 for x86_64, then select the following:

   • Oracle Linux 5 Update 6 installation media copy (x86_64)

   • Oracle Linux 5 Update 6 Patch (x86_64)

6. Click Subscribe.

7. Start a terminal session and enter the following command as root, depending on your platform.

   Oracle Linux 6:

   # yum install oracle-rdbms-server-11gR2-preinstall
   

   Oracle Linux 5 or Oracle Linux 4:

   # yum install oracle-validated
   

   You should see output indicating that you have subscribed to the Oracle Linux channel, and that packages are being installed. For example:

   el5_u6_i386_base
   el5_u6_x86_64_patch
   

   Oracle Linux automatically creates a standard (not role-allocated) Oracle installation owner and groups, and sets up other kernel configuration settings as required for Oracle installations.

8. Repeat steps 1 through 8 on all other servers in your cluster.

   Note:

   Check the RPM log file to review the system configuration changes. For example:

   Oracle Linux 6:

   /var/log/oracle-rdbms-server-11gR2-preinstall/results/orakernel.log
   

   Oracle Linux 5 and Oracle Linux 4:

   /var/log/oracle-validated/results/orakernel.log
   

2.1.6 Installing the Oracle Preinstallation RPM From DVDs or Images

Use the following procedure to install the Oracle Preinstallation RPM from the Oracle Linux distribution:

1. Obtain Oracle Linux disks either by ordering the Oracle Linux media pack from Oracle Store, or by downloading disk images from the Oracle Software Delivery Cloud website for Oracle Linux and Oracle VM.

   Oracle Store:

   https://shop.oracle.com
   

   Oracle Software Delivery Cloud website:

   http://edelivery.oracle.com/linux
   

2. Start the Oracle Linux installation.

3. Review the first software selection screen, which lists task-specific software options. At the bottom of the screen, there is an option to customize now or customize later. Select Customize now, and click Next.

4. Select Base System on the Customization page, in the list on the left side of the screen, and then select System Tools on the right side of the screen. Then click Optional Packages.

   The Packages in System Tools window opens.

5. Select the Oracle Preinstallation RPM package box from the package list, and click Next.

6. Complete the other screens to finish installing Oracle Linux.

   Oracle Linux automatically creates a standard (not role-allocated) Oracle installation owner and groups, and sets up other kernel configuration settings as required for Oracle installations.

7. Repeat steps 2 through 6 on all other cluster member nodes.

2.1.7 Using Ksplice to Perform a Zero Downtime Update

Ksplice Uptrack updates provide Linux security and bug fix updates, repackaged in a form that allows these updates to be applied without restarting the kernel.

To use Ksplice Uptrack:

1. Obtain or verify your Oracle Linux premium support subscription from Unbreakable Linux Network:

   https://linux.oracle.com

2. Log in as root.

3. Ensure that you have access to the Internet on the server where you want to use Ksplice. For example, if you are using a proxy server, then set the proxy server and port values in the shell with commands similar to the following:

   # export http_proxy=http://proxy.example.com:port
   # export https_proxy=http://proxy.example.com:port
   

4. Download the Ksplice Uptrack repository RPM package:

   https://www.ksplice.com/yum/uptrack/ol/ksplice-uptrack-release.noarch.rpm

5. Run the following commands:

   rpm -i ksplice-uptrack-release.noarch.rpm
   yum -y install uptrack
   

6. Open /etc/uptrack/uptrack.conf with a text editor, enter your premium support access key, and save the file. You must use the same access key for all of your systems.

7. Run the following command to carry out a zero downtime update of your kernel:

   uptrack-upgrade -y
   

   See Also:

   • The Oracle Ksplice Uptrack website for more information:

     http://www.ksplice.com

   • Oracle Ksplice for Oracle Linux:

     http://oss.oracle.com/ksplice/docs/ksplice-quickstart.pdf

2.5.1 Determining If the Oracle Inventory and Oracle Inventory Group Exists

When you install Oracle software on the system for the first time, OUI creates the oraInst.loc file. This file identifies the name of the Oracle Inventory group (by default, oinstall), and the path of the Oracle Central Inventory directory. An oraInst.loc file has contents similar to the following:

inventory_loc=central_inventory_location
inst_group=group

In the preceding example, central_inventory_location is the location of the Oracle central inventory, and group is the name of the group that has permissions to write to the central inventory (the OINSTALL group privilege).

If you have an existing Oracle central inventory, then ensure that you use the same Oracle Inventory for all Oracle software installations, and ensure that all Oracle software users you intend to use for installation have permissions to write to this directory.

To determine if you have an Oracle central inventory directory (oraInventory) on your system:

Enter the following command:

# more /etc/oraInst.loc

If the oraInst.loc file exists, then the output from this command is similar to the following:

inventory_loc=/u01/app/oracle/oraInventory
inst_group=oinstall

In the previous output example:

• The inventory_loc group shows the location of the Oracle Inventory.

• The inst_group parameter shows the name of the Oracle Inventory group (in this example, oinstall).

Use the command grep groupname /etc/group to confirm that the group specified as the Oracle Inventory group still exists on the system. For example:

$ grep oinstall /etc/group
oinstall:x:1000:grid,oracle

2.5.2 Creating the Oracle Inventory Group If an Oracle Inventory Does Not Exist

If the oraInst.loc file does not exist, then create the Oracle Inventory group by entering a command similar to the following:

# /usr/sbin/groupadd -g 1000 oinstall

The preceding command creates the oraInventory group oinstall, with the group ID number 1000. Members of the oraInventory group are granted privileges to write to the Oracle central inventory (oraInventory).

By default, if an oraInventory group does not exist, then the installer lists the primary group of the installation owner for the Oracle Grid Infrastructure for a Cluster software as the oraInventory group. Ensure that this group is available as a primary group for all planned Oracle software installation owners.

2.5.3 Creating the Oracle Grid Infrastructure User

You must create a software owner for Oracle Grid Infrastructure in the following circumstances:

• If an Oracle software owner user does not exist; for example, if this is the first installation of Oracle software on the system

• If an Oracle software owner user exists, but you want to use a different operating system user, with different group membership, to separate Oracle Grid Infrastructure administrative privileges from Oracle Database administrative privileges.

  In Oracle documentation, a user created to own only Oracle Grid Infrastructure software installations is called the grid user. A user created to own either all Oracle installations, or only Oracle database installations, is called the oracle user.

# id oracle

uid=501(oracle) gid=501(oinstall) groups=502(dba),503(oper)

2.5.4 Creating the Oracle Base Directory Path

The Oracle base directory for the grid installation owner is the location where diagnostic and administrative logs, and other logs associated with Oracle ASM and Oracle Clusterware are stored.

If you have created a path for the Oracle Clusterware home that is compliant with Oracle Optimal Flexible Architecture (OFA) guidelines for Oracle software paths then you do not need to create an Oracle base directory. When OUI finds an OFA-compliant path, it creates the Oracle base directory in that path.

For OUI to recognize the path as an Oracle software path, it must be in the form u[00-99]/app, and it must be writable by any member of the oraInventory (oinstall) group. The OFA path for the Oracle base is /u01/app/user, where user is the name of the software installation owner.

Oracle recommends that you create an Oracle Grid Infrastructure Grid home and Oracle base homes manually, particularly if you have separate Oracle Grid Infrastructure for a cluster and Oracle Database software owners, so that you can separate log files.

For example:

# mkdir -p  /u01/app/11.2.0/grid
# mkdir -p /u01/app/grid
# mkdir -p /u01/app/oracle
# chown grid:oinstall /u01/app/11.2.0/grid
# chown grid:oinstall /u01/app/grid
# chown oracle:oinstall /u01/app/oracle
# chmod -R 775 /u01/
# chown -R grid:oinstall /u01

2.5.5 Creating Job Role Separation Operating System Privileges Groups and Users

A job role separation privileges configuration of Oracle ASM is a configuration with groups and users that divide administrative access privileges to the Oracle ASM installation from other administrative privileges users and groups associated with other Oracle installations. Administrative privileges access is granted by membership in separate operating system groups, and installation privileges are granted by using different installation owners for each Oracle installation.

If you prefer, you can allocate operating system user privileges so that you can use one administrative user and one group for operating system authentication for all system privileges on the storage and database tiers.

For example, you can designate the oracle user to be the installation owner for all Oracle software, and designate oinstall to be the group whose members are granted all system privileges for Oracle Clusterware, Oracle ASM, and all Oracle Databases on the servers, and all privileges as installation owners. This group must also be the Oracle Inventory group.

Oracle recommends that you use at least two groups: A system privileges group whose members are granted administrative system privileges, and an installation owner group (the oraInventory group) to provide separate installation privileges (the OINSTALL privilege). To simplify using the defaults for Oracle tools such as Cluster Verification Utility, if you do choose to use a single operating system group to grant all system privileges and the right to write to the oraInventory, then that group name should be oinstall.

• Overview of Creating Job Role Separation Groups and Users

• Creating Database Groups and Users with Job Role Separation

# /usr/sbin/groupadd -g 1031 dba

# /usr/sbin/groupadd -g 1032 oper1

# /usr/sbin/groupadd -g 1020 asmadmin

# /usr/sbin/groupadd -g 1022 asmoper

# /usr/sbin/groupadd -g 1021 asmdba

# id oracle

uid=501(oracle) gid=501(oinstall) groups=502(dba),503(oper)

# /usr/sbin/usermod -g oinstall -G dba,asmdba oracle

2.5.6 Example of Creating Standard Groups, Users, and Paths

The following is an example of how to create the Oracle Inventory group (oinstall), and a single group (dba) as the OSDBA, OSASM and OSDBA for Oracle ASM groups. In addition, it shows how to create the Oracle Grid Infrastructure software owner (grid), and one Oracle Database owner (oracle) with correct group memberships. This example also shows how to configure an Oracle base path compliant with OFA structure with correct permissions:

# groupadd -g 1000 oinstall
# groupadd -g 1031 dba
# useradd -u 1100 -g oinstall -G dba grid
# useradd -u 1101 -g oinstall -G dba oracle
# mkdir -p  /u01/app/11.2.0/grid
# mkdir -p /u01/app/grid
# chown -R grid:oinstall /u01
# mkdir /u01/app/oracle
# chown oracle:oinstall /u01/app/oracle
# chmod -R 775 /u01/

After running these commands, you have the following groups and users:

• An Oracle central inventory group, or oraInventory group (oinstall). Members who have the central inventory group as their primary group, are granted the OINSTALL permission to write to the oraInventory directory.

• A single system privileges group that is used as the OSASM, OSDBA, OSDBA for ASM, and OSOPER for ASM group (dba), whose members are granted the SYSASM and SYSDBA privilege to administer Oracle Clusterware, Oracle ASM, and Oracle Database, and are granted SYSASM and OSOPER for ASM access to the Oracle ASM storage.

• An Oracle grid installation for a cluster owner (grid), with the oraInventory group as its primary group, and with the OSASM group as the secondary group, with its Oracle base directory /u01/app/grid.

• An Oracle Database owner (oracle) with the oraInventory group as its primary group, and the OSDBA group as its secondary group, with its Oracle base directory /u01/app/oracle.

• /u01/app owned by grid:oinstall with 775 permissions before installation, and by root after the root.sh script is run during installation. This ownership and permissions enables OUI to create the Oracle Inventory directory, in the path /u01/app/oraInventory.

• /u01 owned by grid:oinstall before installation, and by root after the root.sh script is run during installation.

• /u01/app/11.2.0/grid owned by grid:oinstall with 775 permissions. These permissions are required for installation, and are changed during the installation process.

• /u01/app/grid owned by grid:oinstall with 775 permissions before installation, and 755 permissions after installation.

• /u01/app/oracle owned by oracle:oinstall with 775 permissions.

2.5.7 Example of Creating Role-allocated Groups, Users, and Paths

The following is an example of how to create role-allocated groups and users that is compliant with an Optimal Flexible Architecture (OFA) deployment:

# groupadd -g 1000 oinstall
# groupadd -g 1020 asmadmin
# groupadd -g 1021 asmdba
# groupadd -g 1031 dba1
# groupadd -g 1041 dba2
# groupadd -g 1022 asmoper
# useradd -u 1100 -g oinstall -G asmadmin,asmdba grid
# useradd -u 1101 -g oinstall -G dba1,asmdba oracle1
# useradd -u 1102 -g oinstall -G dba2,asmdba oracle2
# mkdir -p  /u01/app/11.2.0/grid
# mkdir -p /u01/app/grid
# chown -R grid:oinstall /u01
# mkdir -p /u01/app/oracle1
# chown oracle1:oinstall /u01/app/oracle1
# mkdir -p /u01/app/oracle2
# chown oracle2:oinstall /u01/app/oracle2
# chmod -R 775 /u01

After running these commands, you have the following groups and users:

• An Oracle central inventory group, or oraInventory group (oinstall), whose members that have this group as their primary group are granted permissions to write to the oraInventory directory.

• A separate OSASM group (asmadmin), whose members are granted the SYSASM privilege to administer Oracle Clusterware and Oracle ASM.

• A separate OSDBA for ASM group (asmdba), whose members include grid, oracle1 and oracle2, and who are granted access to Oracle ASM.

• A separate OSOPER for ASM group (asmoper), whose members are granted limited Oracle ASM administrator privileges, including the permissions to start and stop the Oracle ASM instance.

• An Oracle grid installation for a cluster owner (grid), with the oraInventory group as its primary group, and with the OSASM (asmadmin), OSDBA for ASM (asmdba) group as a secondary group.

• Two separate OSDBA groups for two different databases (dba1 and dba2) to establish separate SYSDBA privileges for each database.

• Two Oracle Database software owners (oracle1 and oracle2), to divide ownership of the Oracle database binaries, with the OraInventory group as their primary group, and the OSDBA group for their database (dba1 or dba2) and the OSDBA for ASM group (asmdba) as their secondary groups.

• An OFA-compliant mount point /u01 owned by grid:oinstall before installation.

• An Oracle base for the grid installation owner /u01/app/grid owned by grid:oinstall with 775 permissions, and changed during the installation process to 755 permissions.

• An Oracle base /u01/app/oracle1 owned by oracle1:oinstall with 775 permissions.

• An Oracle base /u01/app/oracle 2 owned by oracle2:oinstall with 775 permissions.

• A Grid home /u01/app/11.2.0/grid owned by grid:oinstall with 775 (drwxdrwxr-x) permissions. These permissions are required for installation, and are changed during the installation process to root:oinstall with 755 permissions (drwxr-xr-x).

• /u01/app/oraInventory. This path remains owned by grid:oinstall, to enable other Oracle software owners to write to the central inventory.

2.6.1 General Server Requirements

• Ensure that the server is started with run level 3 or run level 5.

• Ensure servers run the same operating system binary. Oracle Grid Infrastructure installations and Oracle Real Application Clusters (Oracle RAC) support servers with different hardware in the same cluster.

2.6.2 Server Hardware Requirements

Each system must meet the following minimum hardware requirements:

• At least 1024 x 768 display resolution, so that OUI displays correctly.

• 1 GB of space in the /tmp directory.

• 6.6 GB of space for the Oracle Grid Infrastructure for a Cluster home (Grid home) This includes Oracle Clusterware and Oracle ASM files and log files, ACFS log files, and includes the Cluster Health Monitor repository, if available for your platform.

• Upto 10 GB of additional space in the Oracle base directory of the Grid Infrastructure owner for diagnostic collections generated by Trace File Analyser and Collector.

• On Linux x86 platforms, if you intend to install Oracle Database, then allocate 4 GB of space for the Oracle base.

• For Linux x86_64 platforms, if you intend to install Oracle Database, then allocate 4.6 GB of disk space for the Oracle base.

If you are installing Oracle Database, then you require additional space, either on a file system or in an Oracle Automatic Storage Management disk group, for the Fast Recovery Area if you choose to configure automated database backups.

2.6.3 Server Memory Requirements

Ensure that your system meets the following requirements, depending on your system architecture

• 64-bit System and Linux on System z Memory Requirements

• Memory Requirements

2.6.4 Checking Server Hardware and Memory Configuration

To ensure that each system meets these requirements, follow these steps:

1. To determine the physical RAM size, enter the following command:

   # grep MemTotal /proc/meminfo
   

   If the size of the physical RAM installed in the system is less than the required size, then you must install more memory before continuing.

2. To determine the size of the configured swap space, enter the following command:

   # grep SwapTotal /proc/meminfo
   

   If necessary, refer to your operating system documentation for information about how to configure additional swap space.

3. To determine the amount of space available in the /tmp directory, enter the following command:

   # df -k /tmp
   

   This command displays disk space in 1 kilobyte blocks. On most systems, you can use the df command with the -h flag (df -h) to display output in "human-readable" format, such as "24G" and "10M." If there is less than 1 GB of disk space available in the /tmp directory (less than 1048576 1-k blocks), then complete one of the following steps:

   • Delete unnecessary files from the /tmp directory to make available the space required.

   • Extend the file system that contains the /tmp directory. If necessary, contact your system administrator for information about extending file systems.

4. To determine the amount of free RAM and disk swap space on the system, enter the following command:

   # free
   

   Note:

   • Oracle recommends that you take multiple values for the available RAM and swap space before finalizing a value. This is because the available RAM and swap space keep changing depending on the user interactions with the computer.

   • Contact your operating system vendor for swap space allocation guidance for your server. The vendor guidelines supersede the swap space requirements listed in this guide.

5. To determine if the system architecture can run the software, enter the following command:

   # uname -m
   

   Note:

   This command displays the processor type. If you intend to install on a 64-bit architecture, then the output should be x86_64.

2.7.1 Network Hardware Requirements

The following is a list of requirements for network configuration:

• Each node must have at least two network adapters or network interface cards (NICs): one for the public network interface, and one for the private network interface (the interconnect).

  With Redundant Interconnect Usage, you should identify multiple interfaces to use for the cluster private network, without the need of using bonding or other technologies. This functionality is available starting with Oracle Database 11g Release 2 (11.2.0.2).

  When you define multiple interfaces, Oracle Clusterware creates from one to four highly available IP (HAIP) addresses. Oracle RAC and Oracle ASM instances use these interface addresses to ensure highly available, load-balanced interface communication between nodes. The installer enables Redundant Interconnect Usage to provide a high availability private network.

  By default, Oracle Grid Infrastructure software uses all of the HAIP addresses for private network communication, providing load-balancing across the set of interfaces you identify for the private network. If a private interconnect interface fails or become non-communicative, then Oracle Clusterware transparently moves the corresponding HAIP address to one of the remaining functional interfaces.

  Note:

  If you define more than four interfaces as private network interfaces, be aware that Oracle Clusterware activates only four of the interfaces at a time. However, if one of the four active interfaces fails, then Oracle Clusterware transitions the HAIP addresses configured to the failed interface to one of the reserve interfaces in the defined set of private interfaces.

  When you upgrade a node to Oracle Grid Infrastructure 11g release 2 (11.2.0.2) and later, the upgraded system uses your existing network classifications.

  To configure multiple public interfaces, use a third-party technology for your platform to aggregate the multiple public interfaces before you start installation, and then select the single interface name for the combined interfaces as the public interface. Oracle recommends that you do not identify multiple public interface names during Oracle Grid Infrastructure installation. Note that if you configure two network interfaces as public network interfaces in the cluster without using an aggregation technology, the failure of one public interface on a node does not result in automatic VIP failover to the other public interface.

  Oracle recommends that you use the Redundant Interconnect Usage feature to make use of multiple interfaces for the private network. However, you can also use third-party technologies to provide redundancy for the private network.

• If you install Oracle Clusterware using OUI, then the public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes. This restriction does not apply if you use cloning, either to create a new cluster, or to add nodes to an existing cluster.

  For example: With a two-node cluster, you cannot configure network adapters on node1 with eth0 as the public interface, but on node2 have eth1 as the public interface. Public interface names must be the same, so you must configure eth0 as public on both nodes. You should configure the private interfaces on the same network adapters as well. If eth1 is the private interface for node1, then eth1 should be the private interface for node2.

  See Also:

  Oracle Clusterware Administration and Deployment Guide for information about how to add nodes using cloning

• For the public network, each network adapter must support TCP/IP.

• For the private network, the interface must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP (minimum requirement 1 Gigabit Ethernet).

  Note:

  UDP is the default interface protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware. You must use a switch for the interconnect. Oracle recommends that you use a dedicated switch.

  Oracle does not support token-rings or crossover cables for the interconnect.

• Each node's private interface for interconnects must be on the same subnet, and those subnets must connect to every node of the cluster. For example, if the private interfaces have a subnet mask of 255.255.255.0, then your private network is in the range 192.168.0.0--192.168.0.255, and your private addresses must be in the range of 192.168.0.[0-255]. If the private interfaces have a subnet mask of 255.255.0.0, then your private addresses can be in the range of 192.168.[0-255].[0-255].

  For clusters using Redundant Interconnect Usage, each private interface should be on a different subnet. However, each cluster member node must have an interface on each private interconnect subnet, and these subnets must connect to every node of the cluster. For example, you can have private networks on subnets 192.168.0 and 10.0.0, but each cluster member node must have an interface connected to the 192.168.0 and 10.0.0 subnets.

• For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. There should be no node that is not connected to every private network interface. You can test if an interconnect interface is reachable using ping.

2.7.2 IP Address Requirements

Before starting the installation, you must have at least two interfaces configured on each node: One for the private IP address and one for the public IP address.

You can configure IP addresses with one of the following options:

• Dynamic IP address assignment using Oracle Grid Naming Service (GNS). If you select this option, then network administrators assign static IP address for the physical host name and dynamically allocated IPs for the Oracle Clusterware managed VIP addresses. In this case, IP addresses for the VIPs are assigned by a DHCP and resolved using a multicast domain name server configured as part of Oracle Clusterware within the cluster. If you plan to use GNS, then you must have the following:

  • A DHCP service running on the public network for the cluster

  • Enough addresses on the DHCP to provide 1 IP address for each node's virtual IP, and 3 IP addresses for the cluster used by the Single Client Access Name (SCAN) for the cluster

• Static IP address assignment. If you select this option, then network administrators assign a fixed IP address for each physical host name in the cluster and for IPs for the Oracle Clusterware managed VIPs. In addition, domain name server (DNS) based static name resolution is used for each node. Selecting this option requires that you request network administration updates when you modify the cluster.

root@node1]$ nslookup mycluster-scan
Server:         dns.example.com
Address:        192.0.2.001
 
Name:   mycluster-scan.example.com
Address: 192.0.2.201
Name:   mycluster-scan.example.com
Address: 192.0.2.202
Name:   mycluster-scan.example.com
Address: 192.0.2.203

2.7.3 Broadcast Requirements for Networks Used by Oracle Grid Infrastructure

Broadcast communications (ARP and UDP) must work properly across all the public and private interfaces configured for use by Oracle Grid Infrastructure release 2 patchset 1 (11.2.0.2) and later releases.

The broadcast must work across any configured VLANs as used by the public or private interfaces.

2.7.4 Multicast Requirements for Networks Used by Oracle Grid Infrastructure

With Oracle Grid Infrastructure release 2 (11.2), on each cluster member node, the Oracle mDNS daemon uses multicasting on all interfaces to communicate with other nodes in the cluster.

With Oracle Grid Infrastructure release 2 patchset 1 (11.2.0.2) and later releases, multicasting is required on the private interconnect. For this reason, at a minimum, you must enable multicasting for the cluster:

• Across the broadcast domain as defined for the private interconnect

• On the IP address subnet ranges 224.0.0.0/24 and 230.0.1.0/24

You do not need to enable multicast communications across routers.

2.7.5 DNS Configuration for Domain Delegation to Grid Naming Service

If you plan to use GNS, then before Oracle Grid Infrastructure installation, you must configure your domain name server (DNS) to send to GNS name resolution requests for the subdomain GNS serves, which are the cluster member nodes. The following is an overview of what needs to be done for domain delegation. Your actual procedure may be different from this example.

Configure the DNS to send GNS name resolution requests using delegation:

1. In the DNS, create an entry for the GNS virtual IP address, where the address uses the form gns-server.CLUSTERNAME.DOMAINNAME. For example, where the cluster name is mycluster, and the domain name is example.com, and the IP address is 192.0.2.1, create an entry similar to the following:

   mycluster-gns.example.com  A  192.0.2.1
   

   The address you provide must be routable.

2. Set up forwarding of the GNS subdomain to the GNS virtual IP address, so that GNS resolves addresses to the GNS subdomain. To do this, create a BIND configuration entry similar to the following for the delegated domain, where cluster01.example.com is the subdomain you want to delegate:

    cluster01.example.com  NS  mycluster-gns.example.com
   

3. When using GNS, you must configure resolve.conf on the nodes in the cluster (or the file on your system that provides resolution information) to contain name server entries that are resolvable to corporate DNS servers. The total timeout period configured—a combination of options attempts (retries) and options timeout (exponential backoff)—should be less than 30 seconds. For example, where xxx.xxx.xxx.42 and xxx.xxx.xxx.15 are valid name server addresses in your network, provide an entry similar to the following in /etc/resolv.conf:

   options attempts: 2
   options timeout: 1
   
   search cluster01.example.com example.com
   nameserver xxx.xxx.xxx.42
   nameserver xxx.xxx.xxx.15
   

   /etc/nsswitch.conf controls name service lookup order. In some system configurations, the Network Information System (NIS) can cause problems with Oracle SCAN address resolution. Oracle recommends that you place the nis entry at the end of the search list. For example:

   /etc/nsswitch.conf
        hosts:    files   dns   nis
   

   Note:

   Be aware that use of NIS is a frequent source of problems when doing cable pull tests, as host name and username resolution can fail.

2.7.6 Grid Naming Service Configuration Example

If you use GNS, then you need to specify a static IP address for the GNS VIP address, and delegate a subdomain to be delegated to that static GNS IP address.

As nodes are added to the cluster, your organization's DHCP server can provide addresses for these nodes dynamically. These addresses are then registered automatically in GNS, and GNS provides resolution within the subdomain to cluster node addresses registered with GNS.

Because allocation and configuration of addresses is performed automatically with GNS, no further configuration is required. Oracle Clusterware provides dynamic network configuration as nodes are added to or removed from the cluster. The following example is provided only for information.

With a two node cluster where you have defined the GNS VIP, after installation you might have a configuration similar to the following for a two-node cluster, where the cluster name is mycluster, the GNS parent domain is example.com, the subdomain is grid.example.com, 192.0.2 in the IP addresses represent the cluster public IP address network, and 192.168.0 represents the private IP address subnet:

^Footnote 1 Node host names may resolve to multiple addresses, including VIP addresses currently running on that host.

2.7.7 Manual IP Address Configuration Example

If you choose not to use GNS, then before installation you must configure public, virtual, and private IP addresses. Also, check that the default gateway can be accessed by a ping command. To find the default gateway, use the route command, as described in your operating system's help utility.

For example, with a two node cluster where each node has one public and one private interface, and you have defined a SCAN domain address to resolve on your DNS to one of three IP addresses, you might have the configuration shown in the following table for your network interfaces:

^Footnote 1 Node host names may resolve to multiple addresses.

You do not need to provide a private name for the interconnect. If you want name resolution for the interconnect, then you can configure private IP names in the hosts file or the DNS. However, Oracle Clusterware assigns interconnect addresses on the interface defined during installation as the private interface (eth1, for example), and to the subnet used for the private subnet.

The addresses to which the SCAN resolves are assigned by Oracle Clusterware, so they are not fixed to a particular node. To enable VIP failover, the configuration shown in the preceding table defines the SCAN addresses and the public and VIP addresses of both nodes on the same subnet, 192.0.2.

2.7.8 Network Interface Configuration Options

The precise configuration you choose for your network depends on the size and use of the cluster you want to configure, and the level of availability you require.

If certified Network-attached Storage (NAS) is used for Oracle RAC and this storage is connected through Ethernet-based networks, then you must have a third network interface for NAS I/O. Failing to provide three separate interfaces in this case can cause performance and stability problems under load.

2.7.9 Multiple Private Interconnects and Enterprise Linux

In kernel 2.6.31 and above, which also includes Oracle Unbreakable Enterprise Kernel 2.6.32, a bug has been fixed in the Reverse Path Filtering. As a consequence of this correction, Oracle RAC systems that use multiple NICs for the private interconnect now require specific settings for the rp_filter parameter. This requirement also applies to all Exadata systems that are running Linux kernel 2.6.32 and above. Without these rp_filter parameter settings systems, interconnect packets can be blocked or discarded.

The rp_filter values set the Reverse Path filter to no filtering (0), to strict filtering (1), or to loose filtering (2). Set the rp_filter value for the private interconnects to either 0 or 2. Setting the private interconnect NIC to 1 can cause connection issues on the private interconnect. It is not considered unsafe to disable or relax this filtering, because the private interconnect should be on a private and isolated network.

For example, where eth1 and eth2 are the private interconnect NICs, and eth0 is the public network NIC, set the rp_filter of the private address to 2 (loose filtering), the public address to 1 (strict filtering), using the following entries in /etc/sysctl.conf:

net.ipv4.conf.eth2.rp_filter = 2
net.ipv4.conf.eth1.rp_filter = 2
net.ipv4.conf.eth0.rp_filter = 1

Enterprise Linux 5.6 (Enterprise Linux 5 Update 6) includes a fix using initscripts-8.45.33-1.0.4.el5.i386.rpm, which sets the kernel parameter net.ipv4.conf.default.rp_filter to 2 (relaxed mode). For that reason, after you apply Unbreakable Linux Kernel on top of Enterprise Linux 5.6, you may not need to make manual changes, because the rp_filter value of all NICs is set to 2. If you require more strict reverse path filtering on the public network, then set the public NIC rp_filter to 1.

https://support.oracle.com

2.7.10 Enabling the Name Service Cache Daemon

To allow Oracle Clusterware to better tolerate network failures with NAS devices or NFS mounts, enable the Name Service Cache Daemon (nscd).

To check to see if nscd is set to load when the system is restarted, enter the command chkconfig --list nscd. For example:

# chkconfig --list nscd
nscd                      0:off  1:off  2:off  3:on   4:off  5:off   6:off 

In the preceding example, nscd is turned on for run level 3, and turned off for run level 5. The nscd should be turned on for both run level 3 and run level 5.

To change the configuration to ensure that nscd is on for both run level 3 and run level 5, enter one of the following command as root:

On Red Hat and Oracle Linux 4 and 5:

# chkconfig --level 35 nscd on

On SUSE SLES10 and11:

# chkconfig --level 35 nscd on

To start up nscd in the current session, enter the command as root:

# service nscd start

To restart nscd with the new setting, enter the following command as root:

# service nscd restart

2.8.1 Software Requirements List for x86 Linux Platforms

For installations of Oracle Grid Infrastructure, ensure that you have the kernel versions and packages listed in Table 2-5 and Table 2-6.

Ensure that OpenSSH is installed on your servers. OpenSSH is the required SSH software.

If you intend to install Oracle Database or Oracle RAC in addition to Oracle Grid Infrastructure, then check Table 2-7 to determine if you must install additional packages for the features you plan to use.

binutils-2.15.92.0.2
compat-libstdc++-33-3.2.3
elfutils-libelf-0.97
elfutils-libelf-devel-0.97
gcc-3.4.6
gcc-c++-3.4.6
glibc-2.3.4-2.41
glibc-common-2.3.4
glibc-devel-2.3.4
glibc-headers-2.3.4
libaio-devel-0.3.105
libaio-0.3.105
libgcc-3.4.6
libstdc++-3.4.6
libstdc++-devel-3.4.6
make-3.80
numactl-0.6.4.i386
pdksh-5.2.14
sysstat-5.0.5
unixODBC-2.2.11
unixODBC-devel-2.2.11

binutils-2.17.50.0.6
compat-libstdc++-33-3.2.3
elfutils-libelf-0.125
elfutils-libelf-devel-0.125
elfutils-libelf-devel-static-0.125
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.5-24
glibc-common-2.5
glibc-devel-2.5
glibc-headers-2.5
kernel-headers-2.6.18
ksh-20060214
libaio-0.3.106
libaio-devel-0.3.106 
libgcc-4.1.2
libgomp-4.1.2
libstdc++-4.1.2 
libstdc++-devel-4.1.2
make-3.81
sysstat-7.0.2
unixODBC-2.2.11
unixODBC-devel-2.2.11

binutils-2.20.51.0.2-5.11.el6.i686
compat-libcap1-1.10-1.i686
compat-libstdc++-33-3.2.3-69.el6.i686
gcc-4.4.4-13.el6.i686
gcc-c++-4.4.4-13.el6.i686
glibc-2.12-1.7.el6.i686
glibc-devel-2.12-1.7.el6.i686
ksh
libgcc-4.4.4-13.el6.i686
libstdc++-4.4.4-13.el6.i686
libstdc++-devel-4.4.4-13.el6.i686
libaio-0.3.107-10.el6.i686
libaio-devel-0.3.107-10.el6.i686
make-3.81-19.el6.i686
sysstat-9.0.4-11.el6.i686

binutils-2.16.91.0.5
compat-libstdc++-5.0.7
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.4-31.63
glibc-devel-2.4.31.63
ksh-93r-12.9
libaio-0.3.104
libaio-devel-0.3.104
libelf-0.8.5
libgcc-4.1.2
libstdc++-4.1.2
libstdc++-devel-4.1.2
make-3.80
sysstat-8.0.4

binutils-2.19
gcc-4.3
gcc-c++-4.3
glibc-2.9
glibc-devel-2.9
ksh-93t
libcap1-1.10-6.10
libstdc++33-3.3.3
libstdc++43-4.3.3_20081022
libstdc++43-devel-4.3.3_20081022
libaio-0.3.104
libaio-devel-0.3.104
libgcc43-4.3.3_20081022
libstdc++-devel-4.3
make-3.81
sysstat-8.1.5

http://www.unixodbc.org

unixODBC-2.2.11 (32 bit) or later
unixODBC-devel-2.2.11 (32 bit) or later

unixODBC-2.2.11 (32 bit) or laterunixODBC-devel-2.2.11 (32 bit) or later

unixODBC-2.2.14-11.el6.i686 or laterunixODBC-devel-2.2.14-11.el6.i686 or later

unixODBC-32bit-2.2.11 (32 bit) or later
unixODBC-devel-32bit-2.2.11 (32 bit) or later

unixODBC-32bit-2.2.12 (32-bit) or later
unixODBC-devel-32bit-2.2.12 (32 bit) or later

modinfo ocfs2

rpm -qa| grep ocfs2

http://oss.oracle.com/projects/ocfs2/

2.8.2 Software Requirements List for x86-64 Linux Platforms

For installations only of Oracle Grid Infrastructure, ensure that you have the kernel versions and packages listed in Table 2-8 and Table 2-9.

Ensure that OpenSSH is installed on your servers. OpenSSH is the required SSH software.

If you intend to install Oracle Database or Oracle RAC in addition to Oracle Grid Infrastructure, then check Table 2-10 to determine if you must install additional packages for the features you plan to use.

binutils-2.23.52.0.1-12.el7.x86_64 
compat-libcap1-1.10-3.el7.x86_64 
gcc-4.8.2-3.el7.x86_64 
gcc-c++-4.8.2-3.el7.x86_64 
glibc-2.17-36.el7.i686 
glibc-2.17-36.el7.x86_64 
glibc-devel-2.17-36.el7.i686 
glibc-devel-2.17-36.el7.x86_64 
ksh
libaio-0.3.109-9.el7.i686 
libaio-0.3.109-9.el7.x86_64 
libaio-devel-0.3.109-9.el7.i686 
libaio-devel-0.3.109-9.el7.x86_64 
libgcc-4.8.2-3.el7.i686 
libgcc-4.8.2-3.el7.x86_64 
libstdc++-4.8.2-3.el7.i686 
libstdc++-4.8.2-3.el7.x86_64 
libstdc++-devel-4.8.2-3.el7.i686 
libstdc++-devel-4.8.2-3.el7.x86_64 
libXi-1.7.2-1.el7.i686 
libXi-1.7.2-1.el7.x86_64 
libXtst-1.2.2-1.el7.i686 
libXtst-1.2.2-1.el7.x86_64 
make-3.82-19.el7.x86_64 
sysstat-10.1.5-1.el7.x86_64

binutils-2.20.51.0.2-5.11.el6 (x86_64)
compat-libcap1-1.10-1 (x86_64)
compat-libstdc++-33-3.2.3-69.el6 (x86_64)
compat-libstdc++-33-3.2.3-69.el6.i686
gcc-4.4.4-13.el6 (x86_64)
gcc-c++-4.4.4-13.el6 (x86_64)
glibc-2.12-1.7.el6 (i686)
glibc-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6.i686
ksh
libgcc-4.4.4-13.el6 (i686)
libgcc-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6.i686
libstdc++-devel-4.4.4-13.el6 (x86_64)
libstdc++-devel-4.4.4-13.el6.i686
libaio-0.3.107-10.el6 (x86_64)
libaio-0.3.107-10.el6.i686
libaio-devel-0.3.107-10.el6 (x86_64)
libaio-devel-0.3.107-10.el6.i686
make-3.81-19.el6
sysstat-9.0.4-11.el6 (x86_64)

binutils-2.17.50.0.6
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.125
elfutils-libelf-devel-0.125
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.5-24
glibc-2.5-24 (32 bit)
glibc-common-2.5
glibc-devel-2.5
glibc-devel-2.5 (32 bit)
glibc-headers-2.5
ksh-20060214
libaio-0.3.106
libaio-0.3.106 (32 bit)
libaio-devel-0.3.106
libaio-devel-0.3.106 (32 bit)
libgcc-4.1.2
libgcc-4.1.2 (32 bit)
libstdc++-4.1.2
libstdc++-4.1.2 (32 bit)
libstdc++-devel-4.1.2
make-3.81
sysstat-7.0.2
unixODBC-2.2.11
unixODBC-2.2.11 (32 bit)
unixODBC-devel-2.2.11
unixODBC-devel-2.2.11 (32 bit)
coreutils-5.97-23.el5_4.1 or later

binutils-2.15.92.0.2
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.97
elfutils-libelf-devel-0.97
expat-1.95.7
gcc-3.4.6
gcc-c++-3.4.6
glibc-2.3.4-2.41
glibc-2.3.4-2.41 (32 bit)
glibc-common-2.3.4
glibc-devel-2.3.4
glibc-headers-2.3.4
libaio-0.3.105
libaio-0.3.105 (32 bit)
libaio-devel-0.3.105
libaio-devel-0.3.105 (32 bit)
libgcc-3.4.6
libgcc-3.4.6 (32-bit)
libstdc++-3.4.6
libstdc++-3.4.6 (32 bit)
libstdc++-devel 3.4.6
make-3.80
numactl-0.6.4.x86_64
pdksh-5.2.14
sysstat-5.0.5
unixODBC-2.2.11
unixODBC-2.2.11 (32 bit)
unixODBC-devel-2.2.11
unixODBC-devel-2.2.11 (32 bit)

binutils-2.25.0-13.1
gcc-4.8-6.189
gcc48-4.8.5-24.1
glibc-2.19-31.9
glibc-32bit-2.19-31.9
glibc-devel-2.19-31.9.x86_64
glibc-devel-32bit-2.19-31.9.x86_64
libaio1-0.3.109-17.15
libaio-devel-0.3.109-17.15
libcap1-1.10-59.61
libstdc++48-devel-4.8.5-24.1.x86_64
libstdc++48-devel-32bit-4.8.5-24.1.x86_64
libstdc++6-5.2.1+r226025-4.1.x86_64
libstdc++6-32bit-5.2.1+r226025-4.1.x86_64
libstdc++-devel-4.8-6.189.x86_64
libstdc++-devel-32bit-4.8-6.189.x86_64
libgcc_s1-5.2.1+r226025-4.1.x86_64
libgcc_s1-32bit-5.2.1+r226025-4.1.x86_64
mksh-50-2.13
make-4.0-4.1.x86_64
sysstat-10.2.1-3.1.x86_64
xorg-x11-driver-video-7.6_1-14.30.x86_64
xorg-x11-server-7.6_1.15.2-36.21.x86_64
xorg-x11-essentials-7.6_1-14.17.noarch
xorg-x11-Xvnc-1.4.3-7.2.x86_64
xorg-x11-fonts-core-7.6-29.45.noarch
xorg-x11-7.6_1-14.17.noarch
xorg-x11-server-extra-7.6_1.15.2-36.21.x86_64
xorg-x11-libs-7.6-45.14.noarch
xorg-x11-fonts-7.6-29.45.noarch

binutils-2.19
gcc-4.3
gcc-32bit-4.3
gcc-c++-4.3
glibc-2.9
glibc-32bit-2.9
glibc-devel-2.9
glibc-devel-32bit-2.9
ksh-93t
libaio-0.3.104
libaio-32bit-0.3.104
libaio-devel-0.3.104
libaio-devel-32bit-0.3.104
libstdc++33-3.3.3
libstdc++33-32bit-3.3.3
libstdc++43-4.3.3_20081022
libstdc++43-32bit-4.3.3_20081022
libstdc++43-devel-4.3.3_20081022
libstdc++43-devel-32bit-4.3.3_20081022
libgcc43-4.3.3_20081022
libstdc++-devel-4.3
make-3.81
sysstat-8.1.5

binutils-2.16.91.0.5
compat-libstdc++-5.0.7 (not needed with IBM:Linux on System z)
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.4-31.63
glibc-devel-2.4
glibc-devel-32bit-2.4
ksh-93r-12.9
libaio-0.3.104
libaio-32bit-0.3.104
libaio-devel-0.3.104
libaio-devel-32bit-0.3.104
libelf-0.8.5
libgcc-4.1.2
libstdc++-4.1.2
make-3.80
numactl-0.9.6.x86_64
sysstat-8.0.4

binutils-2.23.52.0.1-55.el7.x86_64
compat-libcap1-1.10-7.el7.x86_64
gcc-4.8.5-4.el7.ns7.01.x86_64
gcc-c++-4.8.5-4.el7.ns7.01.x86_64
glibc-2.17-105.el7.ns7.01.i686
glibc-2.17-105.el7.ns7.01.x86_64
glibc-devel-2.17-105.el7.ns7.01.i686
glibc-devel-2.17-105.el7.ns7.01.x86_64
ksh-20120801-22.el7_1.2.x86_64
libaio-0.3.109-13.el7.i686
libaio-0.3.109-13.el7.x86_64
libaio-devel-0.3.109-13.el7.i686
libaio-devel-0.3.109-13.el7.x86_64
libgcc-4.8.5-4.el7.ns7.01.i686
libgcc-4.8.5-4.el7.ns7.01.x86_64
libstdc++-4.8.5-4.el7.ns7.01.i686
libstdc++-4.8.5-4.el7.ns7.01.x86_64
libstdc++-devel-4.8.5-4.el7.ns7.01.i686
libstdc++-devel-4.8.5-4.el7.ns7.01.x86_64
libXi-1.7.4-2.el7.i686
libXi-1.7.4-2.el7.x86_64
libXtst-1.2.2-2.1.el7.i686
libXtst-1.2.2-2.1.el7.x86_64
make-3.82-21.el7.x86_64
sysstat-10.1.5-7.el7.x86_64

binutils-2.20.51.0.2-5.36.el6 (x86_64) 
compat-libcap1-1.10-1 (x86_64) 
compat-libstdc++-33-3.2.3-69.el6 (x86_64) 
compat-libstdc++-33-3.2.3-69.el6 (i686) 
gcc-4.4.7-4.el6 (x86_64) 
gcc-c++-4.4.7-4.el6 (x86_64) 
glibc-2.12-1.132.el6 (i686) 
glibc-2.12-1.132.el6 (x86_64) 
glibc-devel-2.12-1.132.el6 (x86_64) 
glibc-devel-2.12-1.132.el6 (i686) 
ksh 
libgcc-4.4.7-4.el6 (i686) 
libgcc-4.4.7-4.el6 (x86_64) 
libstdc++-4.4.7-4.el6 (x86_64) 
libstdc++-4.4.7-4.el6 (i686) 
libstdc++-devel-4.4.7-4.el6 (x86_64) 
libstdc++-devel-4.4.7-4.el6 (i686) 
libaio-0.3.107-10.el6 (x86_64) 
libaio-0.3.107-10.el6 (i686) 
libaio-devel-0.3.107-10.el6 (x86_64) 
libaio-devel-0.3.107-10.el6 (i686) 
make-3.81-20.el6
sysstat-9.0.4-22.el6 (x86_64)

http://www.unixodbc.org

unixODBC-2.2.11 (32 bit) or later
unixODBC-devel-2.2.11 (64 bit) or later
unixODBC-2.2.11 (64 bit ) or later

unixODBC-devel-2.2.11 (32 bit) or later
unixODBC-devel-2.2.11 (64 bit) or later
unixODBC-2.2.11 (64 bit) or later

unixODBC-2.2.14-11.el6 (x86_64) or later
unixODBC-2.2.14-11.el6.i686 or later
unixODBC-devel-2.2.14-11.el6 (x86_64) or later
unixODBC-devel-2.2.14-11.el6.i686 or later

unixODBC-2.3.1-6.el7.x86_64 or later
unixODBC-2.3.1-6.el7.i686 or later
unixODBC-devel-2.3.1-6.el7.x86_64 or later
unixODBC-devel-2.3.1-6.el7.i686 or later

unixODBC-32bit-2.2.11 (32 bit) or later
unixODBC-2.2.11 (64 bit ) or later
unixODBC-devel-2.2.11 (64 bit) or later

unixODBC-2.2.12 or later
unixODBC-devel-2.2.12 or later
unixODBC-32bit-2.2.12 (32 bit) or later

unixODBC-2.3.1-11.el7.i686 or later
unixODBC-2.3.1-11.el7.x86_64 or later
unixODBC-devel-2.3.1-11.el7.i686 or later
unixODBC-devel-2.3.1-11.el7.x86_64 or later

unixODBC-2.2.14-12.el6_3.i686 or later
unixODBC-2.2.14-12.el6_3.x86_64 or later
unixODBC-devel-2.2.14-12.el6_3.x86_64 or later
unixODBC-devel-2.2.14-12.el6_3.i686 or later

modinfo ocfs2

rpm -qa| grep ocfs2

http://oss.oracle.com/projects/ocfs2/

http://oss.oracle.com/projects/ocfs2/

2.8.3 Software Requirements list for IBM: Linux on System z

Ensure that OpenSSH is installed on your servers. OpenSSH is the required SSH software.

binutils-2.23.52.0.1-30.el7.s390x
compat-libcap1-1.10-7.el7.s390x
cpp-4.8.2-16.el7.s390x
gcc-4.8.3-9.el7.s390x
gcc-c++-4.8.3-9.el7.s390x
glibc-2.17-78.el7.s390
glibc-2.17-78.el7.s390x
glibc-devel-2.17-78.el7.s390
glibc-devel-2.17-78.el7.s390x
glibc-headers-2.17-55.el7.s390x
ksh-20120801-22.el7.s390x
libaio-0.3.109-12.el7.s390
libaio-0.3.109-12.el7.s390x
libaio-devel-0.3.109-12.el7.s390
libaio-devel-0.3.109-12.el7.s390x
libgcc-4.8.3-9.el7.s390
libgcc-4.8.3-9.el7.s390x
libstdc++-4.8.3-9.el7.s390
libstdc++-4.8.3-9.el7.s390x
libstdc++-devel-4.8.3-9.el7.s390
libstdc++-devel-4.8.3-9.el7.s390x
libXi-1.2.2-2.1.el7.s390
libXi-1.2.2-2.1.el7.s390x
libXtst-1.2.2-2.1.el7.s390
libXtst-1.2.2-2.1.el7.s390x
make-3.82-21.el7.s390x
mpfr-3.1.1-4.el7.s390x
sysstat-10.1.5-7.el7.s390x

binutils-2.20.51.0.2-5.28 (s390x)
compat-libstdc++-33-3.2.3-69.el6 (s390)
gcc-4.4.6-3.el6 (s390x) 
gcc-c++-4.4.6-3.el6 (s390x)
glibc-2.12-1.47.el6 (s390)
glibc-2.12-1.47.el6 (s390x)
glibc-devel-2.12-1.47.el6 (s390)
glibc-devel-2.12-1.47.el6 (s390x)
ksh
libaio-0.3.107-10.el6 (s390)
libaio-0.3.107-10.el6 (s390x)
libaio-devel-0.3.107-10.el6 (s390x)
libgcc-4.4.6-3.el6 (s390)
libgcc-4.4.6-3.el6 (s390x)
libstdc++-4.4.6-3.el6 (s390x)
make-3.81-19.el6 (s390x)
sysstat-9.0.4-18.el6 (s390x)
unixODBC-2.2.14-11.el6 (s390x) (Only needed if using ODBC)
unixODBC-devel-2.2.14-11.el6 (s390x) (Only needed if using ODBC)

binutils-2.17.50.0.6-12.el5 (s390x)
compat-libstdc++-33-3.2.3-61 (s390)
compat-libstdc++-33-3.2.3-61 (s390x)
gcc-4.1.2-46.el5 (s390x)
gcc-c++-4.1.2-46.el5 (s390x)
glibc-2.5-42(s390)
glibc-2.5-42 (s390x)
glibc-devel-2.5-42 (s390)
glibc-devel-2.5-42 (s390x)
ksh
libaio-0.3.106-3.2 (s390)
libaio-0.3.106-3.2 (s390x)
libaio-devel-0.3.106-3.2 (s390)
libaio-devel-0.3.106-3.2 (s390x)
libgcc-4.1.2-46.el5 (s390)
libgcc-4.1.2-46.el5 (s390x)
libstdc++-4.1.2-46.el5 (s390)
libstdc++-4.1.2-46.el5 (s390x)
libstdc++-devel-4.1.2-46.el5 (s390x)
make-3.81
sysstat-7.0.2-3.el5 (s390x)
unixODBC-2.2.11 (Only needed if using ODBC)
unixODBC-devel-2.2.11 (Only needed if using ODBC)

binutils-2.15.92.0.2-25 (s390x)
compat-libstdc++-33-3.2.3-47.3 (s390)
compat-libstdc++-33-3.2.3-47.3 (s390x)
gcc-3.4.6-11 (s390x)
gcc-c++-3.4.6-11 (s390x)
glibc-2.3.4-2.43 (s390)
glibc-2.3.4-2.43 (s390x)
glibc-devel-2.3.4-2.43 (s390)
glibc-devel-2.3.4-2.43 (s390x)
libaio-0.3.105-2 (s390)
libaio-0.3.105-2 (s390x)
libaio-devel-0.3.105-2 (s390)
libaio-devel-0.3.105-2 (s390x)
libgcc-3.4.6-11 (s390)
libgcc-3.4.6-11 (s390x)
libstdc++-3.4.6-10.0.1
libstdc++-3.4.6-10.0.1 (32-bit)
libstdc++-devel-3.4.6-10.0.1
libstdc++-devel-3.4.6-10.0.1 (x86_64)
make-3.80
pdksh
sysstat-5.0.5-25.el4 (s390x)
unixODBC-2.2.11 (Only needed if using ODBC)
unixODBC-devel-2.2.11 (Only needed if using ODBC)

binutils-2.20.0-0.7.9 (s390x)
gcc-4.3-62.198 (s390x)
gcc-c++-4.3-62.198 (s390x)
glibc-2.11.1-0.17.4 (s390x)
glibc-32bit-2.11.1-0.17.4 (s390x)
glibc-devel-2.11.1-0.17.4 (s390x)
glibc-devel-32bit-2.11.1-0.17.4 (s390x)
ksh
libaio-0.3.109-0.1.46 (s390x)
libaio-32bit-0.3.109-0.1.46 (s390x)
libaio-devel-0.3.109-0.1.46 (s390x)
libaio-devel-32bit-0.3.109-0.1.46 (s390x)
libstdc++43-4.3.4_20091019-0.7.35 (s390x)
libstdc++43-32bit-4.3.4_20091019-0.7.35 (s390x)
libstdc++43-devel-4.3.4_20091019-0.7.35 (s390x)
libstdc++43-devel-32bit-4.3.4_20091019-0.7.35 (s390x)
libgcc43-4.3.4_20091019-0.7.35
make-3.81
sysstat-8.1.5-7.9.56 (s390x)
unixODBC-2.2.11 (Only needed if using ODBC)
unixODBC-devel-2.2.11 (Only needed if using ODBC)

binutils-32bit-2.16.91.0.5-23.34.33 (s390x)
gcc-4.1.2_20070115-0.29.6 (s390x)
gcc-c++-4.1.2_20070115-0.29.6 (s390x)
glibc-2.4-31.74.1 (s390x)
glibc-32bit-2.4-31.74.1 (s390x)
glibc-devel-2.4-31.74.1 (s390x)
glibc-devel-32bit-2.4-31.74.1 (s390x)
ksh
libaio-0.3.104-14.2 (s390x)
libaio-32bit-0.3.104-14.2 (s390x)
libaio-devel-0.3.104-14.2 (s390x)
libaio-devel-32bit-0.3.104-14.2 (s390x)
libgcc-4.1.2_20070115-0.29.6 (s390x)
libstdc++-4.1.2_20070115-0.29.6 (s390x)
libstdc++-devel-4.1.2_20070115-0.29.6 (s390x)
make-3.80-202.2 (s390x)
sysstat-8.0.4-1.7.27 (s390x)
unixODBC-2.2.11 (Only needed if using ODBC)
unixODBC-devel-2.2.11 (Only needed if using ODBC)

http://www.unixodbc.org

unixODBC-2.2.11 (32 bit) or later
unixODBC-devel-2.2.11 (64 bit) or later
unixODBC-2.2.11 (64 bit ) or later

unixODBC-2.2.11 (32 bit) or later
unixODBC-devel-2.2.11 (64 bit) or later
unixODBC-2.2.11 (64 bit) or later

unixODBC-2.2.14-11 (32 bit) or later
unixODBC-devel-2.2.14-11(64 bit) or later
unixODBC-2.2.14-11 (64 bit) or later

unixODBC-2.3.1-11 (32 bit) or later
unixODBC-2.3.1-11 (64 bit) or later
unixODBC-devel-2.3.1-11 (32 bit) or later
unixODBC-devel-2.3.1-11 (64 bit) or later

unixODBC-32bit-2.2.11 (32 bit) or later
unixODBC-2.2.11 (64 bit ) or later
unixODBC-devel-2.2.11 (64 bit) or later

unixODBC-2.2.12 or later
unixODBC-devel-2.2.12 or later
unixODBC-32bit-2.2.12 (32 bit) or later

2.13.1 Requirements for Enabling IPMI

You must have the following hardware and software configured to enable cluster nodes to be managed with IPMI:

• Each cluster member node requires a Baseboard Management Controller (BMC) running firmware compatible with IPMI version 1.5 or greater, which supports IPMI over LANs, and configured for remote control using LAN.

• Each cluster member node requires an IPMI driver installed on each node.

• The cluster requires a management network for IPMI. This can be a shared network, but Oracle recommends that you configure a dedicated network.

• Each cluster member node's Ethernet port used by BMC must be connected to the IPMI management network.

• Each cluster member must be connected to the management network.

• Some server platforms put their network interfaces into a power saving mode when they are powered off. In this case, they may operate only at a lower link speed (for example, 100 MB, instead of 1 GB). For these platforms, the network switch port to which the BMC is connected must be able to auto-negotiate down to the lower speed, or IPMI will not function properly.

2.13.2 Configuring the IPMI Management Network

You can configure the BMC for DHCP, or for static IP addresses. Oracle recommends that you configure the BMC for dynamic IP address assignment using DHCP. To use this option, you must have a DHCP server configured to assign the BMC IP addresses.

2.13.3 Configuring the IPMI Driver

For Oracle Clusterware to communicate with the BMC, the IPMI driver must be installed permanently on each node, so that it is available on system restarts. The IPMI driver is available on the Asianux Linux, Oracle Linux, Red Hat Enterprise Linux, and SUSE Enterprise Linux distributions supported with this release.

2.15.1 Environment Requirements for Oracle Grid Infrastructure Software Owner

You must make the following changes to configure the Oracle Grid Infrastructure software owner environment:

• Set the installation software owner user (grid, oracle) default file mode creation mask (umask) to 022 in the shell startup file. Setting the mask to 022 ensures that the user performing the software installation creates files with 644 permissions.

• Set ulimit settings for file descriptors and processes for the installation software owner (grid, oracle)

• Set the software owner's environment variable DISPLAY environment variables in preparation for the Oracle Grid Infrastructure installation

2.15.2 Procedure for Configuring Oracle Software Owner Environments

To set the Oracle software owners' environments, follow these steps, for each software owner (grid, oracle):

1. Start a new terminal session; for example, start an X terminal (xterm).

2. Enter the following command to ensure that X Window applications can display on this system:

   $ xhost + hostname
   

   The hostname is the name of the local host.

3. If you are not already logged in to the system where you want to install the software, then log in to that system as the software owner user.

4. If you are not logged in as the user, then switch to the software owner user you are configuring. For example, with the grid user:

   $ su - grid
   

5. To determine the default shell for the user, enter the following command:

   $ echo $SHELL
   

6. Open the user's shell startup file in any text editor:

   • Bash shell (bash):

     $ vi .bash_profile
     

   • Bourne shell (sh) or Korn shell (ksh):

     $ vi .profile
     

   • C shell (csh or tcsh):

     % vi .login
     

7. Enter or edit the following line, specifying a value of 022 for the default file mode creation mask:

   umask 022
   

8. If the ORACLE_SID, ORACLE_HOME, or ORACLE_BASE environment variables are set in the file, then remove these lines from the file.

9. Save the file, and exit from the text editor.

10. To run the shell startup script, enter one of the following commands:

    • Bash shell:

      $ . ./.bash_profile
      

    • Bourne, Bash, or Korn shell:

      $ . ./.profile
      

    • C shell:

      % source ./.login
      

11. If you are not installing the software on the local system, then enter a command similar to the following to direct X applications to display on the local system:

    • Bourne, Bash, or Korn shell:

      $ DISPLAY=local_host:0.0 ; export DISPLAY
      

    • C shell:

      % setenv DISPLAY local_host:0.0
      

    In this example, local_host is the host name or IP address of the system (your workstation, or another client) on which you want to display the installer.

12. If you determined that the /tmp directory has less than 1 GB of free space, then identify a file system with at least 1 GB of free space and set the TEMP and TMPDIR environment variables to specify a temporary directory on this file system:

    Note:

    You cannot use a shared file system as the location of the temporary file directory (typically /tmp) for Oracle RAC installation. If you place /tmp on a shared file system, then the installation fails.

    1. Use the df -h command to identify a suitable file system with sufficient free space.

    2. If necessary, enter commands similar to the following to create a temporary directory on the file system that you identified, and set the appropriate permissions on the directory:

       $ su - root
       # mkdir /mount_point/tmp
       # chmod 775 /mount_point/tmp
       # exit
       

    3. Enter commands similar to the following to set the TEMP and TMPDIR environment variables:

       • Bourne, Bash, or Korn shell:

         $ TEMP=/mount_point/tmp
         $ TMPDIR=/mount_point/tmp
         $ export TEMP TMPDIR
         

       • C shell:

         % setenv TEMP /mount_point/tmp
         % setenv TMPDIR /mount_point/tmp
         

13. To verify that the environment has been set correctly, enter the following commands:

    $ umask
    $ env | more
    

    Verify that the umask command displays a value of 22, 022, or 0022 and that the environment variables you set in this section have the correct values.

2.15.3 Check Resource Limits for the Oracle Software Installation Users

For each installation software owner, check the resource limits for installation, using the following recommended ranges:

To check resource limits:

1. Log in as an installation owner.

2. Check the soft and hard limits for the file descriptor setting. Ensure that the result is in the recommended range. For example:

   $ ulimit -Sn
   1024
   $ ulimit -Hn
   65536
   

3. Check the soft and hard limits for the number of processes available to a user. Ensure that the result is in the recommended range. For example:

   $ ulimit -Su
   2047
   $ ulimit -Hu
   16384
   
   

4. Check the soft limit for the stack setting. Ensure that the result is in the recommended range. For example:

   $ ulimit -Ss
   10240
   $ ulimit -Hs
   32768
   

5. Repeat this procedure for each Oracle software installation owner.

2.15.4 Setting Display and X11 Forwarding Configuration

If you are on a remote terminal, and the local node has only one visual (which is typical), then use the following syntax to set the DISPLAY environment variable:

Bourne, Korn, and Bash shells

$ export DISPLAY=hostname:0

C shell:

$ setenv DISPLAY hostname:0

For example, if you are using the Bash shell, and if your host name is node1, then enter the following command:

$ export DISPLAY=node1:0

To ensure that X11 forwarding will not cause the installation to fail, create a user-level SSH client configuration file for the Oracle software owner user, as follows:

1. Using any text editor, edit or create the software installation owner's ~/.ssh/config file.

2. Make sure that the ForwardX11 attribute is set to no. For example:

   Host *
         ForwardX11 no
   

2.15.5 Preventing Installation Errors Caused by Terminal Output Commands

During an Oracle Grid Infrastructure installation, OUI uses SSH to run commands and copy files to the other nodes. During the installation, hidden files on the system (for example, .bashrc or .cshrc) will cause makefile and other installation errors if they contain terminal output commands.

To avoid this problem, you must modify these files in each Oracle installation owner user home directory to suppress all output on STDOUT or STDERR (for example, stty, xtitle, and other such commands) as in the following examples:

• Bourne, Bash, or Korn shell:

  if [ -t 0 ]; then
     stty intr ^C
  fi
  

• C shell:

  test -t 0
  if ($status == 0) then
     stty intr ^C
  endif
  

  Note:

  When SSH is not available, the Installer uses the rsh and rcp commands instead of ssh and scp.

  If there are hidden files that contain stty commands that are loaded by the remote shell, then OUI indicates an error and stops the installation.