D How to Complete Installation Prerequisite Tasks Manually

This appendix provides instructions for how to complete configuration tasks manually that Cluster Verification Utility (CVU) and the installer (OUI) normally complete during installation. Use this appendix as a guide if you cannot use the fixup script.

This appendix contains the following information:

D.1 Configuring SSH Manually on All Cluster Nodes

Passwordless SSH configuration is a mandatory installation requirement. SSH is used during installation to configure cluster member nodes, and SSH is used after installation by configuration assistants, Oracle Enterprise Manager, Opatch, and other features.

Automatic Passwordless SSH configuration using OUI creates RSA encryption keys on all nodes of the cluster. If you have system restrictions that require you to set up SSH manually, such as using DSA keys, then use this procedure as a guide to set up passwordless SSH.

In the examples that follow, the Oracle software owner listed is the grid user.

If SSH is not available, then OUI attempts to use rsh and rcp instead. However, these services often are disabled for security.

This section contains the following:

D.1.1 Checking Existing SSH Configuration on the System

To determine if SSH is running, enter the following command:

$ pgrep sshd

If SSH is running, then the response to this command is one or more process ID numbers. In the home directory of the installation software owner (grid, oracle), use the command ls -al to ensure that the .ssh directory is owned and writable only by the user.

You need either an RSA or a DSA key for the SSH protocol. RSA is used with the SSH 1.5 protocol, while DSA is the default for the SSH 2.0 protocol. With OpenSSH, you can use either RSA or DSA. The instructions that follow are for SSH1. If you have an SSH2 installation, and you cannot use SSH1, then refer to your SSH distribution documentation to configure SSH1 compatibility or to configure SSH2 with DSA.

D.1.2 Configuring SSH on Cluster Nodes

To configure SSH, you must first create RSA or DSA keys on each cluster node, and then copy all the keys generated on all cluster node members into an authorized keys file that is identical on each node. Note that the SSH files must be readable only by root and by the software installation user (oracle, grid), as SSH ignores a private key file if it is accessible by others. In the examples that follow, the DSA key is used.

You must configure SSH separately for each Oracle software installation owner that you intend to use for installation.

To configure SSH, complete the following:

D.1.2.1 Create SSH Directory, and Create SSH Keys On Each Node

Complete the following steps on each node:

  1. Log in as the software owner (in this example, the grid user).

  2. To ensure that you are logged in as grid, and to verify that the user ID matches the expected user ID you have assigned to the grid user, enter the commands id and id grid. Ensure that Oracle user group and user and the user terminal window process you are using have group and user IDs are identical. For example:

    $ id 
    uid=502(grid) gid=501(oinstall) groups=501(oinstall),502(grid,asmadmin,asmdba)
    $ id grid
    uid=502(grid) gid=501(oinstall) groups=501(oinstall),502(grid,asmadmin,asmdba)
    
  3. If necessary, create the .ssh directory in the grid user's home directory, and set permissions on it to ensure that only the oracle user has read and write permissions:

    $ mkdir ~/.ssh
    $ chmod 700 ~/.ssh
    

    Note:

    SSH configuration will fail if the permissions are not set to 700.
  4. Enter the following command:

    $ /usr/bin/ssh-keygen -t dsa
    

    At the prompts, accept the default location for the key file (press Enter).

    Note:

    SSH with passphrase is not supported for Oracle Clusterware 11g release 2 and later releases.

    This command writes the DSA public key to the ~/.ssh/id_dsa.pub file and the private key to the ~/.ssh/id_dsa file.

    Never distribute the private key to anyone not authorized to perform Oracle software installations.

  5. Repeat steps 1 through 4 on each node that you intend to make a member of the cluster, using the DSA key.

D.1.2.2 Add All Keys to a Common authorized_keys File

Complete the following steps:

  1. On the local node, change directories to the .ssh directory in the Oracle Grid Infrastructure owner's home directory (typically, either grid or oracle).

    Then, add the DSA key to the authorized_keys file using the following commands:

    $ cat id_dsa.pub >> authorized_keys
    $ ls
    

    In the .ssh directory, you should see the id_dsa.pub keys that you have created, and the file authorized_keys.

  2. On the local node, use SCP (Secure Copy) or SFTP (Secure FTP) to copy the authorized_keys file to the oracle user .ssh directory on a remote node. The following example is with SCP, on a node called node2, with the Oracle Grid Infrastructure owner grid, where the grid user path is /home/grid:

    [grid@node1 .ssh]$ scp authorized_keys node2:/home/grid/.ssh/
    

    You are prompted to accept a DSA key. Enter Yes, and you see that the node you are copying to is added to the known_hosts file.

    When prompted, provide the password for the grid user, which should be the same on all nodes in the cluster. The authorized_keys file is copied to the remote node.

    Your output should be similar to the following, where xxx represents parts of a valid IP address:

    [grid@node1 .ssh]$ scp authorized_keys node2:/home/grid/.ssh/
    The authenticity of host 'node2 (xxx.xxx.173.152) can't be established.
    DSA key fingerprint is 7e:60:60:ae:40:40:d1:a6:f7:4e:zz:me:a7:48:ae:f6:7e.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'node1,xxx.xxx.173.152' (dsa) to the list
    of known hosts
    grid@node2's password:
    authorized_keys       100%             828             7.5MB/s      00:00
    
  3. Using SSH, log in to the node where you copied the authorized_keys file. Then change to the .ssh directory, and using the cat command, add the DSA keys for the second node to the authorized_keys file, clicking Enter when you are prompted for a password, so that passwordless SSH is set up:

    [grid@node1 .ssh]$ ssh node2
    [grid@node2 grid]$ cd .ssh
    [grid@node2 ssh]$ cat id_dsa.pub  >> authorized_keys
    

    Repeat steps 2 and 3 from each node to each other member node in the cluster.

    When you have added keys from each cluster node member to the authorized_keys file on the last node you want to have as a cluster node member, then use scp to copy the authorized_keys file with the keys from all nodes back to each cluster node member, overwriting the existing version on the other nodes.

    To confirm that you have all nodes in the authorized_keys file, enter the command more authorized_keys, and determine if there is a DSA key for each member node. The file lists the type of key (ssh-dsa), followed by the key, and then followed by the user and server. For example:

    ssh-dsa AAAABBBB . . . = grid@node1
    

    Note:

    The grid user's /.ssh/authorized_keys file on every node must contain the contents from all of the /.ssh/id_dsa.pub files that you generated on all cluster nodes.

D.1.3 Enabling SSH User Equivalency on Cluster Nodes

After you have copied the authorized_keys file that contains all keys to each node in the cluster, complete the following procedure, in the order listed. In this example, the Oracle Grid Infrastructure software owner is named grid:

  1. On the system where you want to run OUI, log in as the grid user.

  2. Use the following command syntax, where hostname1, hostname2, and so on, are the public host names (alias and fully qualified domain name) of nodes in the cluster to run SSH from the local node to each node, including from the local node to itself, and from each node to each other node:

    [grid@nodename]$ ssh hostname1 date
    [grid@nodename]$ ssh hostname2 date
        .
        .
        .
    

    For example:

    [grid@node1 grid]$ ssh node1 date
    The authenticity of host 'node1 (xxx.xxx.100.101)' can't be established.
    DSA key fingerprint is 7z:60:60:zz:48:48:z1:a0:f7:4e.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'node1,xxx.xxx.100.101' (DSA) to the list of
    known hosts.
    Mon Dec 4 11:08:13 PST 2006
    [grid@node1 grid]$ ssh node1.example.com date
    The authenticity of host 'node1.example.com (xxx.xxx.100.101)' can't be
    established.
    DSA key fingerprint is 7z:60:60:zz:48:48:z1:a0:f7:4e.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'node1.example.com,xxx.xxx.100.101' (DSA) to the
    list of known hosts.
    Mon Dec 4 11:08:13 PST 2006
    [grid@node1 grid]$ ssh node2 date
    Mon Dec 4 11:08:35 PST 2006
    .
    .
    .
    

    At the end of this process, the public host name for each member node should be registered in the known_hosts file for all other cluster nodes.

    If you are using a remote client to connect to the local node, and you see a message similar to "Warning: No xauth data; using fake authentication data for X11 forwarding," then this means that your authorized keys file is configured correctly, but your SSH configuration has X11 forwarding enabled. To correct this issue, proceed to Section 2.12.4, "Setting Display and X11 Forwarding Configuration."

  3. Repeat step 2 on each cluster node member.

If you have configured SSH correctly, then you can now use the ssh or scp commands without being prompted for a password. For example:

[grid@node1 ~]$ ssh node2 date
Mon Feb 26 23:34:42 UTC 2009
[grid@node1 ~]$ ssh node1 date
Mon Feb 26 23:34:48 UTC 2009

If any node prompts for a password, then verify that the ~/.ssh/authorized_keys file on that node contains the correct public keys, and that you have created an Oracle software owner with identical group membership and IDs.

D.2 Configuring Kernel Parameters Manually

Note:

The kernel parameter values shown in this section are recommended values only. For production database systems, Oracle recommends that you tune these values to optimize the performance of the system. See your operating system documentation for more information about tuning kernel parameters.

On HP-UX 11.31 (version 3), the following parameters are not valid:

  • msgmap

  • msgseg

  • tcp_smallest_anon_port

  • udp_smallest_anon_port

D.2.1 Minimal Kernel Parameter Values

On all cluster nodes, verify that the kernel parameters shown in Table D-1 are set either to the formula shown, or to values greater than or equal to the recommended value shown. The procedure following the table describes how to verify and set the values.

Table D-1 Minimal HP-UX Kernel Parameter Values

Parameter Minimum Required Value

ksi_alloc_max

32768

executable_stack

0

max_thread_proc

1024

maxdsiz

1073741824 (1 GB)

maxdsiz_64bit

2147483648 (2 GB)

maxfiles

1024

maxfiles_lim

32767

maxssiz

134217728 (128 MB)

maxssiz_64bit

1073741824 (1 GB)

maxuprc

3686

msgmni

4096

msgtql

4096

ncsize

35840

nflocks

4096

ninode

34816

nkthread

7184

nproc

4096

semmni

4096

semmns

8192

semmnu

4096

semvmx

32767

shmmax

1073741824

shmmni

4096

shmseg

512

tcp_largest_anon_port

65500

udp_largest_anon_port

65500


Ensure that you set the TCP and UDP kernel parameters by following the procedure described in the section Setting UDP and TCP Kernel Parameters Manually.

Note:

If the current value for any parameter is higher than the value listed in this table, then do not change the value of that parameter.

D.2.2 Checking Kernel Parameters Manually

To view the current value or formula specified for these kernel parameters, and to change them if necessary, use kctune, HP System Management Homepage (HP SMH), or HP-UX Kernel Configuration (kcweb).

D.2.2.1 Modifying Kernel Settings Using KCTUNE

Use /usr/sbin/kctune to view or change kernel parameters. For example, to view kernel parameters, enter kctune with the -d flag. For example:

# /usr/sbin/kctune -d

To modify a kernel parameter, enter kctune variable=value. For example:

# /usr/sbin/kctune aio_listio_max=512

Complete this procedure on all other cluster nodes.

D.2.2.2 Modifying Kernel Settings Using SMH

The following procedure describes how to modify kernel settings using HP System Management Homepage ( SMH):

  1. Optionally, set the DISPLAY environment variable to specify the display of the local system:

    • Bourne, Bash, or Korn shell:

      # DISPLAY=local_host:0.0 ; export DISPLAY
      
    • C shell:

      # setenv DISPLAY local_host:0.0
      
  2. Start SMH:

    # /usr/sbin/smh
    
  3. Choose the Kernel Configuration area, then choose the Configurable Parameters area.

  4. Check the value or formula specified for each of these parameters and, if necessary, modify that value or formula.

    If necessary, refer to the SMH online Help for more information about completing this step.

    Note:

    If you modify the value of a parameter that is not dynamic, then you must restart the system.
  5. If necessary, when the system restarts, log in and switch user to root.

  6. Complete this procedure on all other cluster nodes.

D.2.2.3 Modifying Kernel Settings Using KCWEB

The following procedure shows how to

  1. Enter the following command to start the kcweb application:

    # /usr/sbin/kcweb -F
    
  2. Check the value or formula specified for each of these parameters and, if necessary, modify that value or formula.

    If necessary, refer to the kcweb online Help for more information about completing this step.

    Note:

    If you modify the value of a parameter that is not dynamic, then you must restart the system.
  3. If necessary, when the system restarts, log in and switch user to root.

  4. Complete this procedure on all other cluster nodes.

D.3 Setting UDP and TCP Kernel Parameters Manually

Use NDD to ensure that the HP-UX kernel TCP/IP ephemeral port range is broad enough to provide enough ephemeral ports for the anticipated server workload. Set the port range high enough to avoid reserved ports for any applications you may intend to use.

For example:

# /usr/bin/ndd /dev/tcp tcp_largest_anon_port
65535

In the preceding example, tcp_largest_anon_port is set to the default value.

If necessary, edit the file /etc/rc.config.d/nddconf and add entries to update the UDP and TCP ephemeral port values. For example:

TRANSPORT_NAME[0]=tcp
NDD_NAME[0]=tcp_largest_anon_port
NDD_VALUE[0]=65500
 
TRANSPORT_NAME[1]=udp
NDD_NAME[1]=udp_largest_anon_port
NDD_VALUE[1]=65500
 

Ensure that the entries are numbered in proper order. For example, if there are two entries present for the TCP and UDP ports in nddconf, then ensure that they are numbered 0 and 1: TRANSPORT_NAME[0]=tcp and TRANSPORT_NAME[1]=udp.

Note:

The tcp_smallest_anon_port and udp_smallest_anon_port parameters are obsolete and you do not need to set them.