![]() ![]() ![]() |
Chapter 2: Planning and Installing HACMP for Linux
This chapter describes how to plan and install HACMP for Linux. It contains the following sections:
Cluster Hardware
This section lists examples of IBM hardware that you can use for cluster nodes, cluster networks and cluster storage disks. For complete information, see the IBM Portal on Linux website:
Hardware for Cluster Nodes
HACMP for Linux lets you configure up to eight HACMP nodes. You can use:
Selected models of IBM System p™ servers For more information, see: http://www.ibm.com/systems/p/linux/
Also, for descriptions of IBM hardware that you can use as HACMP cluster nodes in AIX 5L, see the HACMP for AIX 5L Planning Guide.
Hardware for Cluster Networks
HACMP for Linux supports the following interconnection networks for clusters:
Selected modes of 10/100 Mbps Ethernet Selected models of Gigabit Ethernet Token Ring. An Ethernet or a Token Ring network can be used as an HACMP cluster IP-based network.
Hardware for Cluster Storage
HACMP for Linux does not provide high availability for storage resources in your cluster configuration. However you can use NFS or IBM TotalStorage disk subsystems as the storage options in your cluster.
No Automatic NFS and Volume Management
Although you can have disks and filesystems configured in the same environment in which your HACMP for Linux cluster is configured, HACMP for Linux does not support NFS filesystems. You cannot include filesystems associated with the application into the resource groups.
This means that the filesystems are not kept highly available by HACMP for Linux. In particular, during fallovers, when applications are moved to other nodes, HACMP for Linux does not automatically unmount the associated filesystems on one node and mount them on the takeover node. Similarly, HACMP for Linux does not automatically perform any volume management or volume group operations for volume groups that a particular application needs to access.
However, if you want to manage storage in the cluster, you can still use NFS or GPFS to control it. To ensure that your NFS filesystems work within the cluster, you must manage NFS manually, that is, completely outside of your HACMP for Linux cluster.
For example, for a two-node cluster, you can have an NFS server configured somewhere at your site, and make it to export the filesystem to your cluster nodes. Both nodes will need to mount the filesystem at boot time. The filesystem will be also mounted on another cluster node, the one to which the resource group may potentially fall over in cases of failures. Your application and service IP label will be running on one node. On fallover, the application and service IP label will move to the takeover node where the NFS filesystem has also been mounted since boot time. This way, your application has access to the filesystem regardless of which node is currently hosting the application. However, the NFS filesystems service provided to your application is not kept highly available by HACMP.
As an alternative, here is a cluster configuration that lets you have high availability of your NFS filesystem in the HACMP for Linux cluster. You can configure an NFS server on a separate two-node cluster, with both nodes running HACMP for AIX 5L, specifically, the nodes should run HACMP’s NFS component (it is part of HACMP for AIX 5L). You can then export the filesystem from this highly available NFS server to the nodes of your separate HACMP for Linux cluster.
Cluster Software
The HACMP for Linux cluster software can be described in these two categories:
Software that you need sot that you can install and run the cluster. In particular, HACMP for Linux requires RSCT (IBM Reliable Scalable Cluster Technology) subsystem to be installed on the nodes. For complete information on what software you need to install, see the installation section. The application that you plan to make highly available with the use of HACMP. It can be a database or another service. The following link provides an overview of software available for the Linux platform: Planning the HACMP Configuration
Plan to have the following components in an HACMP cluster:
An application Up to eight nodes Resource groups Networks. Planning Applications
Once you put an application under HACMP’s control, HACMP starts it on the node(s) and periodically polls the application’s status, if you define application monitors. In cases of component failures, HACMP moves the application to other nodes while the process is invisible to application’s end users.
Plan to have the following for your application:
Customized application start and stop scripts and their locations. The scripts should contain all pre- and post-processing you want HACMP to do so that it starts and stops the applications on the nodes cleanly and according to your requirements. You define these scripts as the application server in WebSMIT. Customized scripts you may want to use in HACMP for monitoring the application’s successful startup, and for periodically checking the application’s running process. You define these scripts to HACMP as application monitors in WebSMIT. If you have a complex production environment with tiered applications that require dependencies between their startup, or a staged production environment where some applications should start only if their “supporting” applications are already running, HACMP supports these configurations by letting you configure multiple types of dependencies between resource groups in WebSMIT. To configure a working cluster that will support such dependent applications, first plan the dependencies for all the services that you want to make highly available. For examples of such planning, see the HACMP for AIX 5L Planning Guide and Administration Guide (sections on multi-tiered applications and resource group dependencies).
In HACMP 5.4, you can use WebSMIT to take an application out of HACMP’s control temporarily without disrupting it, and then restart HACMP on the nodes that currently run the application.
Planning HACMP Nodes
HACMP for Linux lets you configure up to eight HACMP nodes.
For each critical application, be mindful of the resources required by the application, including its processing and data storage requirements. For example, when you plan the size of your cluster, include enough nodes to handle the processing requirements of your application after a node fails.
Keep in mind the following considerations when determining the number of cluster nodes and planning the nodes:
An HACMP cluster can be made up of any combination of supported workstations, LPARs, and other machines. See Hardware for Cluster Nodes. Ensure that all cluster nodes do not share components that could be a single point of failure (for example, a power supply). Similarly, do not place nodes on a single rack. Create small clusters that consist of nodes that perform similar functions or share resources. Smaller, simple clusters are easier to design, implement, and maintain. For performance reasons, it may be desirable to use multiple nodes to support the same application. To provide mutual takeover services, the application must be designed in a manner that allows multiple instances of the application to run on the same node. For example, if an application requires that the dynamic data reside in a directory called /data, chances are that the application cannot support multiple instances on the same processor. For such an application (running in a non-concurrent environment), try to partition the data so that multiple instances of the application can run—each accessing a unique database.
Furthermore, if the application supports configuration files that enable the administrator to specify that the dynamic data for instance1 of the application reside in the data1 directory, instance2 resides in the data2 directory, and so on, then multiple instances of the application are probably supported.
In certain configurations, including additional nodes in the cluster design can increase the level of availability provided by the cluster; it also gives you more flexibility in planning node fallover and reintegration. Choose cluster nodes that have enough I/O slots to support redundant network interface cards and disk adapters. Ensure you have enough cluster nodes in your cluster. Although this adds to the cost of the cluster, we highly recommend to support redundant hardware, (such as enough I/O slots for network interface cards and disk adapters). This will increase the availability of your application.
Use nodes with similar processing speed. Use nodes with the sufficient CPU cycles and I/O bandwidth to allow the production application to run at peak load. Remember, nodes should have enough capacity to allow HACMP to operate. To plan for this, benchmark or model your production application, and list the parameters of the heaviest expected loads. Then choose nodes for an HACMP cluster that will not exceed 85% busy, when running your production application.
Planning for Resource Groups in an HACMP Cluster
To make your applications highly available in an HACMP cluster, plan and configure resource groups. Resource groups must include resources related to the application, such as its start and stop script (application server) and the service IP label for the application.
Plan the following for resource groups in HACMP for Linux:
The nodelist for the resource groups must contain all or some nodes from the cluster. These are the nodes on which you “allow” HACMP to host your application. The first node in the nodelist is the default node, or the home node for the resource group that contains the application. You define the nodelist in WebSMIT. You can use any set of resource group policies for a resource group startup, fallover and fallback. In WebSMIT, HACMP lets you combine only valid sets of these policies and prevents you from configuring non-working scenarios. HACMP for Linux supports only non-concurrent resource groups. HACMP for Linux does not support the fallover policy Fallover using Dynamic Node Priority policy. HACMP for Linux does not support cluster sites. If your applications are dependent on other applications, you may need to plan for dependencies between resource groups. HACMP lets you have node-collocated resource groups, resource groups that always must reside on different nodes, and also child resource groups that do not start before their parent resource groups are active (parent/child dependencies). Make a diagram of your dependent applications to better plan dependencies that you want to configure for resource groups, and then define them in WebSMIT. HACMP processes the resource groups in parallel by default. HACMP for Linux does not allow dynamic changes to the cluster resources or resource groups (also known as dynamic reconfiguration or DARE). This means that you must stop the cluster services, before changing the resource groups or their resources. For complete planning information, see the guidelines in Chapter 6: Planning Resource Groups in the HACMP Cluster in the HACMP for AIX 5L Planning Guide.
Resource Group Policies: Overview
HACMP allows you to configure only valid combinations of startup, fallover, and fallback behaviors for resource groups. The following table summarizes the basic startup, fallover, and fallback behaviors you can configure for resource groups in HACMP for Linux v. 5.4:
Startup Behavior Fallover Behavior Fallback Behavior Online only on home node (first node in the nodelist) Online on first available node Any of these:
Planning IP Networks and Network Interfaces
Plan to configure the following networks and IP interfaces:
A heartbeating IP-based network. An HACMP cluster requires at least one network that will be used for the cluster heartbeating traffic. A heartbeating serial network, such as RS232. An IP-based network that lets you connect from the application’s client machine to the nodes. The nodes serve as the application’s servers and run HACMP. To configure this network, plan to configure a client machine with a network adapter and a NIC compatible with at least one of the networks configured on the cluster nodes. Two HACMP cluster networks. These are TCP/IP-based networks used by HACMP for inter-node communication. HACMP utilities use them to synchronize information between the nodes and propagate cluster changes across the cluster nodes. For each HACMP cluster network, on each cluster node plan to configure two IP labels that will be available at boot time, will be configured on different subnets, and will be used for IPAT via IP aliasing. See Planning IP Labels for IPAT via IP Aliasing.
On the cluster node that will serve as a Web server, set up a network connection to access WebSMIT. Typically, you set up WebSMIT to be accessible from the cluster’s internal network that is not reachable from the Internet. To securely run WebSMIT on a node, you must ensure HTTP(S)/SSL connectivity to that node; it is not handled automatically by WebSMIT or HACMP. See Security Considerations. Planning IP Labels for IPAT via IP Aliasing
IP address takeover via IP aliasing is the default method of taking over the IP address and is supported in HACMP for Linux. IPAT via IP aliasing allows one node to acquire the IP label and the IP address of another node in the cluster, using IP aliases.
To enable that IP Address Takeover via IP aliases can be used in the HACMP for Linux networks configuration, configure NICs for the two HACMP cluster networks that meet the following requirements:
Plan to configure more than one boot-time IP label on the service network interface card on each cluster node. Subnet requirements: Multiple boot-time addresses configured on a node should be defined on different subnets. Service IP addresses must be configured on a different subnet from all non-service addresses (such as boot) defined for that network on the cluster node. Multiple service labels can coexist as aliases on a given interface. The netmask for all IP labels in an HACMP network must be the same. Manually add the IP labels described in this section into the /etc/hosts file on each node. This must be done before you proceed to configure an HACMP cluster in WebSMIT. HACMP non-service labels are defined on the nodes as the boot-time addresses, assigned by the operating system after a system boot and before the HACMP software is started. When you start the HACMP software on a node, the node’s service IP label is added as an alias onto one of the NICs that has a non-service label.
When using IPAT via IP Aliases, the node’s NIC must meet the following conditions:
The NIC has both the boot-time and service IP addresses configured, where the service IP label is an alias placed on the interface. The boot-time address is never removed from a NIC, simply an alias is added on the NIC in addition to the boot-time address. If the node fails, a takeover node acquires the failed node’s service address as an alias on one of its non-service interfaces on the same HACMP network. During a node fallover event, the service IP label that is moved is placed as an alias on the target node’s NIC in addition to any other service labels that may already be configured on that NIC. When using IPAT via IP Aliases, service IP labels are acquired using all available non-service interfaces. If there are multiple interfaces available to host the service IP label, the interface is chosen according to the number of IP labels currently on that interface. If multiple service IP labels are acquired and there are multiple interfaces available, the service IP labels are distributed across all the available interfaces.
Installing HACMP for Linux
HACMP for Linux uses the RPM tool for the installation. For information on RPM, see:
http://www.rpm.org/max-rpm/ch-rpm-commands.html
Once you install HACMP for Linux, proceed to configure WebSMIT for access to the cluster configuration user interface.
Software Prerequisites for Installation
When you install HACMP for Linux, make sure that the following software is installed on the cluster nodes:
Red Hat™ Enterprise Linux (RHEL) 4 or SUSE™ LINUX Enterprise Server (SLES) 9 (both with latest updates). Read the readme file for WebSMIT/usr/es/sbin/cluster/wsm/README for information on specific Apache V1 and V2 requirements, and for information on specific issues related to RHEL or SUSE Linux distribution.
RSCT 2.4.5.2. For the latest information about RSCT levels and the latest available APARs for RSCT, check the HACMP for Linux v. 5.4 Release Notes. Apache WebServer V1 and V2 (provided with the Linux distribution). ksh93. A compliant version of ksh. Ensure that the ksh version you have installed is ksh93 compliant. The ksh93 environment is a prerequisite for the RHEL distribution, and HACMP for Linux checks for it prior to the installation. You can download ksh93 from the Web. The fileset name is similar to the following: ksh-20050202-1.ppc.rpm.
Contents of the Installation Media
The HACMP for Linux installation media provides the following .rpm files:
hacmp.license
hacmp.license-5.4.0.0.ppc.rpm HACMP for Linux License Package. hacmp.license provides the software License Agreement functions for the HACMP for Linux software.
hacmp.doc
hacmp.doc.html-5.4.0.0.ppc.rpm HACMP for Linux HTML documentation—U.S. English hacmp.doc.pdf-5.4.0.0.ppc.rpm HACMP for Linux PDF documentation—U.S. English
Installation Process Overview
Install the HACMP for Linux software on each cluster node (server). Perform the installation process as the root user.
Installing HACMP for Linux RPMs
Before you install, ensure that you have installed all the prerequisites for the installation. See Software Prerequisites for Installation.
To install HACMP for Linux:
1. Insert the HACMP for Linux CD ROM and install the hacmp.license.rpm RPM:
This RPM provides a utility that lets you accept the License Agreement for HACMP for Linux v.5.4, and complete the installation.
Note: You can install the HACMP for Linux documentation without accepting the License Agreement.
2. Run the HACMP installation script /usr/es/sbin/cluster/install_hacmp.
This script has two options:
The /usr/es/sbin/cluster/install_hacmp installation script launches the License Agreement Program (LAP) and the License Agreement acceptance dialog appears.
3. Read and accept the License Agreement. The software places a key on your system to identify that you accepted the license agreement.
You can also accept the license without installing the rest of the filesets by running the /usr/es/sbin/cluster/hacmp_license script. You can then use the RPM tool to install the remaining RPMs.
The usr/es/sbin/cluster/install_hacmp installation script checks for the following prerequisites for the HACMP for Linux software:
rsct.basic-x.x.x.x rsct.core-x.x.x.x. rsct.core.utils-x.x.x.x An appropriate version of ksh93 (such as ksh-20050202-1.ppc.rpm) Perl 5 (an RSCT prerequisite. perl-5.8.3 is installed with RHEL) src-1.3.0.1 (an RSCT prerequisite) 4. Check the required RSCT levels in the HACMP for Linux v.5.4 Release Notes, or in the section Software Prerequisites for Installation.
You can install HACMP for Linux when prerequisites are already installed, or together with the prerequisites.
The usr/es/sbin/cluster/install_hacmp installation script runs the rpm command to install HACMP for Linux RPMs:
rpm -ivh hacmp.*5. Verify the installed cluster software. Verify that the RPMs have correct version numbers and other specific information. The RPMs cannot be installed when prerequisites are not installed.
6. Configure WebSMIT. See /usr/es/sbin/cluster/wsm/README for information, as well as the section Installing and Configuring WebSMIT in this chapter.
7. Read the HACMP for Linux v. 5.4 Release Notes /usr/es/sbin/cluster/release_notes.linux, for information that does not appear in the product documentation.
Installing and Configuring WebSMIT
WebSMIT is a Web-based user interface that provides consolidated access to all functions of HACMP configuration and management, interactive cluster status, and the HACMP documentation.
WebSMIT is:
Supported on Mozilla-based browsers (Mozilla 1.7.3 for AIX and FireFox 1.5.0.2), Supported on Internet Explorer, versions 5.0, 5.5 and 6.0. Requires that JavaScript is enabled in your browser. Requires network access between the browser and the cluster node that serves as a Web server. To run WebSMIT on a node, you must ensure HTTP(S)/SSL connectivity to that node; it is not handled automatically by WebSMIT or HACMP. To launch the WebSMIT interface:
1. Configure and run a Web server process, such as Apache server, on the cluster node(s) to be administered.
2. See the /usr/es/sbin/cluster/wsm/README file for information on basic Web server configuration, the default security mechanisms in place when installing HACMP, and the configuration files available for customization.
You can run WebSMIT on a single node. Note that WebSMIT will be unavailable if a node failure occurs. To provide better availability, you can setup WebSMIT to run on multiple nodes. Since WebSMIT is retrieving and updating information from the HACMP cluster, that information should be available from all nodes in the cluster.
Typically, you set up WebSMIT to be accessible from the cluster’s internal network that is not reachable from the Internet.
Since the WebSMIT interface runs in a Web browser, you can access it from any platform. For information on WebSMIT security, see Security Considerations.
For more information about installing WebSMIT, see the section Installing and Configuring WebSMIT in the HACMP for AIX 5L Installation Guide.
Integration of WebSMIT with the Apache Server on Different Linux Distributions
The WebSMIT readme file /usr/es/sbin/cluster/wsm/README contains different template files and instructions to enable you to handle variations in packaging, when integrating WebSMIT with the Apache server on different Linux distributions.
Verifying the Installed Cluster Software
After the HACMP for Linux software is installed on all nodes, verify the configuration. Use the verification functions of the RPM utility: your goal is to ensure that the cluster software is the same on all nodes.
Verify that the information returned by the rpm command is accurate:
rpm -qi hacmp.server rpm -qi hacmp.client rpm -qi hacmp.license rpm -qi hacmp.doc.html rpm -qi hacmp.doc.pdfEach command should return information about each RPM. In particular, the Name, Version, Vendor, Summary and Description fields should contain appropriate information about each package.
HACMP modifies different system files during the installation process (such as /etc/inittab, /etc/services, and others). To view the details of the installation process, see the log file file/var/hacmp/log/ hacmp.install.log.<date>.
Example of the Installation Using RPM
Here is an example of the installation using rpm:
# rpm -iv hacmp* Preparing packages for installation... Cluster services are not active on this node. hacmp.client-5.4.0.0-06128 Cluster services are not active on this node. hacmp.server-5.4.0.0-06128 May 8 2006 22:26:18 Starting execution of /usr/es/sbin/cluster/etc/rc.init with parameters: May 8 2006 22:26:18 Completed execution of /usr/es/sbin/cluster/etc/rc.init with parameters: . Exit status = 0 Installation of HACMP for Linux is complete.After installation, use the rpm command to view the information about the installed product:
ppstest3:~ # rpm -qa | grep hacmp hacmp.server-5.4.0.0-06128 hacmp.client-5.4.0.0-06128 ppstest3:~ # rpm -qi hacmp.server-5.4.0.0-06128 Name : hacmp.server Relocations: (not relocatable) Version : 5.4.0.0 Vendor: IBM Corp. Release : 06128 Build Date: Mon May 8 21:21:09 2006 Install date: Tue May 9 13:03:20 2006 Build Host: bldlnx18.ppd.pok.ibm.com Group : System Environment/Base Source RPM: hacmp.server-5.4.0.0-06128.nosrc.rpm Size : 48627953 License: IBM Corp. Signature : (none) Packager : IBM Corp. <hafeedbk@us.ibm.com> URL : http://www.ibm.com/systems/p/ha/ Summary : High Availability Cluster Multi-Processing - server part Description : hacmp.server provides the server side functions for HACMP. Service information for this package can be found at http://techsupport.services.ibm.com/server/cluster Product ID 5765-G71 Distribution: (none)Entries Added to System Directories after Installation
After you install HACMP for Linux, the installation process adds the following lines to the /usr/es/sbin/cluster/etc/inittab file:
SRC definitions are added (run lssrc -s <subsystem>):
Addressing Problems during Installation
If you experience problems during the installation, refer to the RPM documentation for information on a cleanup process after an unsuccessful installation and other issues.
To view the details of the installation process, see the following log file:
/var/hacmp/log/ hacmp.install.log.<date>.
Security Considerations
HACMP for Linux uses the Cluster Communications utility, clcomd to ensure secure communication between the cluster nodes. For complete information on the clcomd utility, see the HACMP on AIX 5L Administration Guide. clcomd in HACMP for Linux does not provide the options for message encryption supplied by the CtSec software package from IBM.
Note: WebSMIT relies on its own security utility for all commands that are run with the root privilege. In HACMP for AIX 5L, the WebSMIT security utility uses a set of security APIs specific to AIX 5L, such as updates for the count of failed logins, checks for the expiration of user passwords and checks for who is allowed to login. These commands are not available on the HACMP for Linux platform. Use the Pluggable Authentication Module package (PAM) on Linux to provide similar functions.
WebSMIT Security
The network access is required between the browser and the cluster node that serves as a Web server: To run WebSMIT on a node, you must ensure HTTP(S)/SSL connectivity to that node; it is not handled automatically by WebSMIT or HACMP.
We recommend using the Internet Protocol SECurity (IPSEC) technology that provides a secure tunnel into a network, allowing you to connect separate networks through an untrusted connection. Note that IPSEC is not included in many Linux distributions.
You may consider using an open source VPN project that will either be used to connect to an existing network that uses the IPSEC standard, or provide a VPN doorway into your network. Check with your Linux distribution provider for availability.
Where You Go from Here
After you install the software on each cluster node, you are ready to configure HACMP for Linux:
Definitions of cluster terminology and main configuration components are listed in Chapter 1: HACMP for Linux Cluster Overview Overview of basic configuration tasks is included in Chapter 3: Common Task Summary Additional tasks for configuring a cluster in WebSMIT are listed in Chapter 4: Other Cluster Configuration Tasks.
![]() ![]() ![]() |