![]() ![]() ![]() |
Chapter 1: Administering an HACMP Cluster
This chapter provides a list of the tasks you perform to configure, maintain, monitor, and troubleshoot an HACMP system, related administrative tasks, and a list of AIX 5L files modified by HACMP.
The main sections in this chapter include:
Options for Configuring an HACMP Cluster
In HACMP, you can configure a cluster using one of the following HACMP tools:
HACMP SMIT user interface. WebSMIT utility. For information on using this utility, see Chapter 2: Administering a Cluster Using WebSMIT. Online Planning Worksheets (OLPW). This tool provides a convenient method for documenting your cluster configuration: You can use the tool to configure a new cluster or to document an existing cluster. For instructions, see the chapter on Using Online Planning Worksheets in the Planning Guide. Two-Node Cluster Configuration Assistant. Use this tool to configure a basic two-node HACMP cluster. You supply the minimum information required to define a cluster, and HACMP discovers the remainder of the information for you. See the section on Using the Two-Node Cluster Configuration Assistant in the chapter on Creating a Basic HACMP Cluster in the Installation Guide. General Configuration Smart Assist. Start with your installed application and configure a basic cluster (any number of nodes). If you are configuring a WebSphere, DB2 UDB or Oracle application, see the corresponding HACMP Smart Assist guide. See Chapter 3: Configuring an HACMP Cluster (Standard). Cluster Snapshot Utility. If you have a snapshot of the HACMP cluster configuration taken before an upgrade to HACMP 5.4, you can use the Cluster Snapshot utility to perform the initial configuration. For more information, see Chapter 18: Saving and Restoring Cluster Configurations. Configuration Tasks
The HACMP configuration tasks are described in detail in subsequent chapters. You can choose to use either the standard or the extended path for the initial configuration, although the standard configuration path is recommended. The major steps in the process are:
First, configure the cluster topology, and then HACMP resources and resource groups using the standard configuration path First, configure the cluster topology, and then HACMP resources and resource groups using the extended configuration path.
(Optional) Configure pre- and post-events, remote notification, HACMP File Collections, cluster verification with automatic corrective action, and other optional settings. Verify and synchronize the HACMP configuration. Test the cluster. Configuring HACMP Using the Standard Configuration Path
Using the options under the Initialization and Standard Configuration SMIT menu, you can add the basic components of the HACMP cluster to the HACMP Configuration Database (ODM) in a few steps. This configuration path significantly automates the discovery and selection of configuration information and chooses default behaviors.
The prerequisites and default settings of this path are:
Connectivity for communication must already be established between all cluster nodes. Automatic discovery of cluster information runs by default. That is, once you have configured communication interfaces/devices and established communication paths to other nodes, HACMP automatically collects HACMP-related information and automatically configures the cluster nodes and networks based on physical connectivity. All discovered networks are added to the cluster configuration. This helps you in the configuration process. To understand how HACMP maintains the security of incoming connections, see the section Managing Cluster Security and Inter-Node Communications in this chapter.
IP aliasing is used as the default mechanism for binding service IP labels/addresses to network interfaces. For more information, see the chapter on Planning Cluster Network Connectivity in the Planning Guide. You can configure the most common types of resources. Customization of resource group fallover/fallback behavior supports the most common scenarios. Chapter 3: Configuring an HACMP Cluster (Standard) takes you through the configuration process if you plan to use the Initialization and Standard Configuration path in SMIT. Once you have configured the basic components, you can use the Extended Configuration path to customize your configuration.
Configuring HACMP Using the Extended Configuration Path
In order to configure the less common HACMP elements, or if connectivity to each of the cluster nodes is unavailable, you can manually enter the information. When using the menu panels under the Extended Configuration SMIT path, if any components are on remote nodes, you must manually initiate the discovery of cluster information. That is, the discovery process used by HACMP is optional when using this path (rather than automatic, as it is when using the Initialization and Standard Configuration SMIT path).
Using the options under the Extended Configuration SMIT menu, you can add basic components to the HACMP Configuration Database (ODM), as well as additional types of resources. Use the Extended Configuration path to customize the cluster for all the components, policies, and options that are not included in the standard configuration menus.
Configuring Topology and Resources
Chapter 3: Configuring an HACMP Cluster (Standard), describes all the SMIT menus and options available for configuring cluster topology and all the various types of resources supported by the software.
There is an option to configure a distribution preference for the aliases of the service IP labels that are placed under HACMP control. A distribution preference for service IP label aliases is a network-wide attribute used to control the placement of the service IP label aliases on the physical network interface cards on the cluster nodes.
For more information, see the section Distribution Preference for Service IP Label Aliases: Overview in Chapter 4: Configuring HACMP Cluster Topology and Resources (Extended).
Configuring Resource Groups and Assigning Resources
Chapter 5: Configuring HACMP Resource Groups (Extended) describes how to configure different types of resource groups. The Extended Configuration menus include options for configuring various runtime policies for resource groups as well as for customizing fallover, fallback and startup behavior. It also includes the procedure for adding resources to a resource group.
Configuring Dynamic LPAR and Capacity Upgrade on Demand Resources
Appendix E: Using DLPAR and CUoD in an HACMP Cluster describes how to plan, integrate, configure, and troubleshoot application provisioning for HACMP through the use of dynamic LPAR (DLPAR) and Capacity Upgrade on Demand (CUoD) functions available on some pSeries servers. It also includes examples and recommendations about customizing your existing pre- and post-event scripts.
Configuring Cluster Events
The HACMP system is event-driven. An event is a change of status within a cluster. When the Cluster Manager detects a change in cluster status, it executes the designated script to handle the event and initiates any user-defined customized processing.
To configure customized cluster events, you indicate the script that handles the event and any additional processing that should accompany an event. Chapter 6: Configuring Cluster Events describes the procedures for customization of event handling in HACMP.
Configuring Remote Notification for Cluster Events
The remote notification function allows you to direct SMS text-message notifications to any address including your cell phone.
With previous versions of HACMP, you could alter event scripts to send email when connected to the Internet. Alternately, the remote notification subsystem could send numeric or alphanumeric pages through a dialer modem, which uses the standard Telocator Alphanumeric Protocol (TAP) protocol.
For more information, see the section Defining a New Remote Notification Method in Chapter 6: Configuring Cluster Events.
Verifying and Synchronizing the Configuration
Verifying the cluster configuration assures you that all resources used by HACMP are validly configured, and that ownership and takeover of those resources are defined and are in agreement across all nodes. By default, if the verification is successful, the configuration is automatically synchronized. You should verify the configuration after making changes to a cluster or node. Chapter 7: Verifying and Synchronizing an HACMP Cluster, describes the SMIT menus for verification, explains the contents and uses of the clverify.log file, and describes how to verify your cluster.
Chapter 7: Verifying and Synchronizing an HACMP Cluster also explains how to create and maintain HACMP File Collections. Using the HACMP File Collections utility, you can request that a list of files is automatically kept synchronized across the cluster. You no longer have to manually copy an updated file to every cluster node, verify that the file is properly copied, and confirm that each node has the same version of it. If you use the HACMP File Collections utility, HACMP can detect and warn you if one or more files in a collection is deleted or has a zero value on one or more cluster nodes during cluster verifications.
Testing the Cluster
HACMP includes the Cluster Test Tool to help you test the recovery procedures for a new cluster before the cluster becomes part of your production environment. You can also use the tool to test configuration changes in an existing cluster, when the cluster services are not running. Chapter 8: Testing an HACMP Cluster explains how to use the Cluster Test Tool.
Maintaining an HACMP Cluster
The following maintenance tasks for an HACMP system are described in detail in subsequent chapters:
Starting and Stopping Cluster Services
Various methods for starting and stopping cluster services are available. Chapter 9: Starting and Stopping Cluster Services describes how to start and stop HACMP on server and client nodes.
Maintaining Shared Logical Volume Manager Components
Any changes to logical volume components must be synchronized across all nodes in the cluster. Chapter 11: Managing Shared LVM Components, and Chapter 12: Managing Shared LVM Components in a Concurrent Access Environment describe how to maintain cluster LVM components. Using C-SPOC (the Cluster Single Point of Control) to configure the cluster components on one node and then synchronize the cluster saves you time and effort.
Managing the Cluster Topology
Any changes to cluster topology require updating the cluster across all nodes. Chapter 13: Managing the Cluster Topology describes how to modify cluster topology after the initial configuration. You can make most changes on one node and then synchronize the cluster.
This chapter also includes information about the HACMP Communication Interface Management SMIT menu that lets you configure communication interfaces/devices to AIX 5L without leaving HACMP SMIT.
Managing Cluster Resources
Any changes to cluster resources require updating the cluster across all nodes. You can make most changes on one node and then synchronize the cluster. Chapter 14: Managing the Cluster Resources describes how to modify cluster resources after the initial configuration.
Managing Cluster Resource Groups
Chapter 15: Managing Resource Groups in a Cluster describes how to modify cluster resource groups after the initial configuration. You can add or delete resources and change the runtime policies of resource groups.
You can dynamically migrate resource groups to other nodes and take them online or offline, using the Resource Group Management utility (clRGmove) from the command line or through SMIT.
Managing Users and Groups in a Cluster
HACMP lets you manage user accounts for a cluster from a Single Point of Control (C-SPOC). Use C-SPOC to create, change, or remove users and groups from all cluster nodes by executing a C-SPOC command on any single cluster node.
For information, see Chapter 16: Managing User and Groups.
Managing Cluster Security and Inter-Node Communications
You can protect access to your HACMP cluster by setting up security for cluster communications between nodes. HACMP provides security for connections between nodes, with higher levels of security for inter-node communications provided through Kerberos (on SP nodes only) or through virtual private networks (VPN). In addition, you can configure authentication and encryption of the messages sent between nodes.
For information, see Chapter 17: Managing Cluster Security.
Understanding the /usr/es/sbin/cluster/etc/rhosts File
This section explains how and when HACMP uses the /usr/es/sbin/cluster/etc/rhosts file, which HACMP uses for inter-node communications. It also describes how this file relates to the ~/.rhosts file.
The /usr/es/sbin/cluster/etc/rhosts file
A Cluster Communications daemon (clcomd) runs on each HACMP node to transparently manage inter-node communications for HACMP. In other words, HACMP manages connections for you automatically:
If the /usr/es/sbin/cluster/etc/rhosts file is empty (this is the initial state of this file, upon installation), then clcomd accepts the first connection from another node and adds entries to the /etc/rhosts file. Since this file is empty upon installation, the first connection from another node adds IP addresses to this file. The first connection usually is performed for verification and synchronization purposes, and this way, for all subsequent connections, HACMP already has entries for node connection addresses in its Configuration Database.
clcomd validates the addresses of the incoming connections to ensure that they are received from a node in the cluster. The rules for validation are based on the presence and contents of the /usr/es/sbin/cluster/etc/rhosts file.
In addition, HACMP includes in the /usr/es/sbin/cluster/etc/rhosts file the addresses for all network interface cards from the communicating nodes. If the /usr/es/sbin/cluster/etc/rhosts file is not empty, then clcomd compares the incoming address with the addresses/labels found in the HACMP Configuration Database (ODM) and then in the /usr/es/sbin/cluster/etc/rhosts file and allows only listed connections. In other words, after installation, HACMP accepts connections from another HACMP node and adds the incoming address(es) to the local file, thus allowing you to configure the cluster without ever editing the file directly. If the /usr/es/sbin/cluster/etc/rhosts file is not present, clcomd rejects all connections Typically, you do not manually add entries to the /usr/es/sbin/cluster/etc/rhosts file unless you have specific security needs or concerns.
If you are especially concerned about network security (for instance, you are configuring a cluster on an unsecured network), then prior to configuring the cluster, you may wish to manually add all the IP addresses/labels for the nodes to the empty /usr/es/sbin/cluster/etc/rhosts file. For information on how to do it, see Manually Configuring /usr/es/sbin/cluster/etc/rhosts file on Individual Nodes in Chapter 17: Managing Cluster Security.
After you synchronize the cluster, you can empty the /usr/es/sbin/cluster/etc/rhosts file (but not remove it), because the information present in the HACMP Configuration Database would be sufficient for all future connections.
If the configuration for AIX 5L adapters was changed after the cluster has been synchronized, HACMP may issue an error. See the section Troubleshooting the Cluster Communications Daemon, or the Troubleshooting Guide for information on refreshing the clcomd utility and updating /usr/es/sbin/cluster/etc/rhosts.
The ~/.rhosts File
~/.rhosts is only needed during the migration from pre-5.1 versions of HACMP. Once migration is completed, we recommend removing ~/.rhosts, if no other applications need rsh for inter-node communication.
Saving and Restoring HACMP Cluster Configurations
After you configure the topology and resources of a cluster, you can save the cluster configuration by taking the cluster snapshot. This saved configuration can later be used to restore the configuration if this is needed by applying the cluster snapshot. A cluster snapshot can also be applied to an active cluster to dynamically reconfigure the cluster. Chapter 18: Saving and Restoring Cluster Configurations describes how to use the Cluster Snapshot utility.
Additional HACMP Maintenance Tasks
Additional tasks that you can perform to maintain an HACMP system include changing the log file attributes for a node and performance tuning. For information on these tasks, see the section on Troubleshooting an HACMP Cluster in this chapter.
Monitoring the Cluster
By design, failures of components in the cluster are handled automatically, but you need to be aware of all such events. Chapter 10: Monitoring an HACMP Cluster describes various tools you can use to check the status of an HACMP cluster, the nodes, networks, and resource groups within that cluster, and the daemons that run on the nodes.
The HACMP software includes the Cluster Information Program (Clinfo), based on SNMP. The HACMP for AIX software provides the HACMP for AIX 5L MIB, associated with and maintained by HACMP. Clinfo retrieves this information from the HACMP for AIX Management Information Base (MIB).
The Cluster Manager gathers information relative to cluster state changes of nodes and interfaces. The Cluster Information Program (Clinfo) gets this information from the Cluster Manager and allows clients communicating with Clinfo to be aware of a cluster’s state changes. This cluster state information is stored in the HACMP MIB.
Clinfo runs on cluster server nodes and on HACMP client machines. It makes information about the state of an HACMP cluster and its components available to clients and applications via an application programming interface (API). Clinfo and its associated APIs enable you to write applications that recognize and respond to changes within a cluster.
The Clinfo program, the HACMP MIB, and the APIs are described in the Programming Client Applications Guide.
Although the combination of HACMP and the high availability features built into the AIX 5L system keeps single points of failure to a minimum, there are still failures that, although detected, can cause other problems.
For suggestions on customizing error notification for various problems not handled by the HACMP events, the Planning Guide.
Troubleshooting an HACMP Cluster
It is useful to follow guidelines for troubleshooting. You should be aware of all the diagnostic tools available from HACMP and AIX 5L. See Chapter 1: Troubleshooting HACMP Clusters in the Troubleshooting Guide for suggested troubleshooting guidelines, as well as for information on tuning the cluster for best performance.
When you become aware of a problem, the first place to look for helpful diagnostic information is the log files. Chapter 2: Using Cluster Log Files in the Troubleshooting Guide describes how to use the various log files. This chapter also contains information on viewing and maintaining log file parameters and instructions for redirecting log files.
If log files do not help you resolve the issue, you may need to check cluster components. See Chapter 3: Investigating System Components and Solving Common Problems in the Troubleshooting Guide for suggested strategies as well as for a list of solutions to common problems that may occur in an HACMP environment.
For information specific to Reliable Scalable Cluster Technology (RSCT) subsystems, see the following IBM publications:
IBM Reliable Scalable Cluster Technology for AIX 5L and Linux: Group Services Programming Guide and Reference, SA22-7888 IBM Reliable Scalable Cluster Technology for AIX 5L and Linux: Administration Guide, SA22-7889 IBM Reliable Scalable Cluster Technology for AIX 5L: Technical Reference, SA22-7890 IBM Reliable Scalable Cluster Technology for AIX 5L: Messages, GA22-7891 Related Administrative Tasks
The tasks below, while not specifically discussed in this book, are essential for effective system administration.
Backing Up Your System
The practice of allocating multiple copies of a logical volume can enhance high availability in a cluster environment, but it should not be considered a replacement for regular system backups. Although HACMP is designed to survive failures within the cluster, it cannot survive a catastrophic failure where multiple points of failure leave data on disks unavailable. Therefore, to ensure data reliability and to protect against catastrophic physical volume failure, you must have a backup procedure in place and perform backups of your system on a regular basis.
To maintain your HACMP environment, you must back up the root volume group (which contains the HACMP software) and the shared volume groups (which contain the data for highly available applications) regularly. HACMP is like other AIX 5L environments from this perspective. Back up all nodes.
Documenting Your System
As your HACMP system grows and changes, it differs from its initial cluster configuration. It is your responsibility as system administrator to document all aspects of the HACMP system unique to your environment. This responsibility includes documenting procedures concerning the highly available applications, recording changes that you make to the configuration scripts distributed with HACMP, documenting any custom scripts you write, recording the status of backups, maintaining a log of user problems, and maintaining records of all hardware. This documentation, along with the output of various display commands and cluster snapshots, will be useful for you, as well as for IBM support, to help resolve problems.
Starting with HACMP 5.2, you can use the report supplied with the Online Planning Worksheet program to generate a of a cluster configuration, then save and print the report to document the system.
Maintaining Highly Available Applications
As system administrator, you should understand the relationship between your applications and HACMP. To keep the applications highly available, HACMP starts and stops the applications that are placed under HACMP control in response to cluster events. Understanding when, how, and why this happens is critical to keeping the applications highly available, as problems can occur that require corrective actions.
For a discussion of strategies for making your applications highly available, see the planning chapters and Appendix B on Applications and HACMP in the Planning Guide.
Helping Users
As the resident HACMP expert, you can expect to receive many questions from end users at your site about HACMP. The more you know about HACMP, the better you are able to answer these questions. If you cannot answer questions about your HACMP cluster environment, contact your IBM support representative.
AIX 5L Files Modified by HACMP
The following AIX 5L files are modified to support HACMP. They are not distributed with HACMP.
/etc/hosts
The cluster event scripts use the /etc/hosts file for name resolution. All cluster node IP interfaces must be added to this file on each node.
HACMP may modify this file to ensure that all nodes have the necessary information in their /etc/hosts file, for proper HACMP operations.
If you delete service IP labels from the cluster configuration using SMIT, we recommend that you also remove them from /etc/hosts. This reduces the possibility of having conflicting entries if the labels are reused with different addresses in a future configuration.
Note that DNS and NIS are disabled during HACMP-related name resolution. This is why HACMP IP addresses must be maintained locally.
/etc/inittab
The /etc/inittab file is modified in each of the following cases:
HACMP is configured for IP address takeover The Start at System Restart option is chosen on the SMIT System Management (C-SPOC) > Manage HACMP Services > Start Cluster Services panel Concurrent Logical Volume Manager (CLVM) is installed with HACMP Starting with HACMP 5.3, the /etc/inittab file has the following entry in the /user/es/sbin/cluster/etc/rc.init: Modifications to the /etc/inittab File due to IP Address Takeover
The following entry is added to the /etc/inittab file for HACMP network startup with IP address takeover:
When IP address takeover is enabled, the system edits /etc/inittab to change the rc.tcpip and inet-dependent entries from run level “2” (the default multi-user level) to run level “a”. Entries that have run level “a” are processed only when the telinit command is executed specifying that specific run level.
Modifications to the /etc/inittab File due to System Boot
The /etc/inittab file is used by the init process to control the startup of processes at boot time.
When the system boots, the /etc/inittab file calls the /usr/es/sbin/cluster/etc/rc.cluster script to start HACMP. The entry is added to the /etc/inittab file if the Start at system restart option is chosen on the SMIT System Management (C-SPOC) > Manage HACMP Services > Start Cluster Services panel or when the system boots:
hacmp:2:once:/usr/es/sbin/cluster/etc/rc.init
This starts the HACMP Communications Daemon, clcomd, and the clstrmgr subsystem.
Because the inet daemons must not be started until after HACMP-controlled interfaces have swapped to their service IP address, HACMP also adds the following entry to the end of the /etc/inittab file to indicate that /etc/inittab processing has completed:
clinit:a:wait:/bin/touch /usr/es/sbin/cluster/.telinit #HACMP for AIX These must be the last entry in run level “a” in inittab! pst_clinit:a:wait:/bin/echo Created /usr/es/sbin/cluster/ .telinit > /dev/console #HACMP for AIX These must be the last entry in run level “a” in inittab!See Chapter 9: Starting and Stopping Cluster Services, for more information about the files involved in starting and stopping HACMP.
/etc/rc.net
The /etc/rc.net file is called by cfgmgr, (cfgmgr is the AIX 5L utility that configures devices and optionally installs device software into the system), to configure and start TCP/IP during the boot process. It sets hostname, default gateway, and static routes. The following entry is added at the beginning of the file for a node on which IP address takeover is enabled:
# HACMP for AIX \ # HACMP for AIX These lines added by HACMP for AIX software [ "$1" = "-boot" ] && shift || { # HACMP for AIX ifconfig lo0 127.0.0.1 up; # HACMP for AIX /bin/uname -S`hostname|sed 's/\..*$//'`; # HACMP for AIX exit 0; # HACMP for AIX } # HACMP for AIX #The HACMP entry prevents cfgmgr from reconfiguring boot and service IP addresses while HACMP is running.
/etc/services
The /etc/services file defines the sockets and protocols used for network services on a system. The ports and protocols used by the HACMP components are defined here.
#clinfo_deadman 6176/tcp #clsmuxpd 6270/tcp #clm_lkm 6150/tcp #clm_smux 6175/tcp #godm 6177/tcp #topsvcs 6178/udp #grpsvcs 6179/udp #emsvcs 6180/udp #clver 6190/tcp #clcomd 6191/tcpNote: If, in addition to HACMP, you install HACMP/XD for GLVM, the following entry for the port number and connection protocol is automatically added to the /etc/services file on each node on the local and remote sites on which you installed the software: rpv 6192/tcp. This default value enables the RPV server and RPV client to start immediately after they are configured, that is, to be in the available state. For more information, see HACMP/XD for GLVM Planning and Administration Guide.
/etc/snmpd.conf
The SNMP daemon reads the /etc/snmpd.conf configuration file when it starts up and when a refresh or kill -1 signal is issued. This file specifies the community names and associated access privileges and views, hosts for trap notification, logging attributes, snmpd-specific parameter configurations, and SMUX configurations for the snmpd. The HACMP installation process adds a clsmuxpd password to this file. The following entry is added to the end of the file, to include the HACMP MIB supervised by the Cluster Manager:
HACMP supports SNMP Community Names other than “public.” That is, HACMP will function correctly if the default SNMP Community Name has been changed in /etc/snmpd.conf to be anything other than “public” (the default). The SNMP Community Name used by HACMP is the first name found that is not “private” or “system” using the lssrc -ls snmpd command.
The Clinfo service also gets the SNMP Community Name in the same manner. The Clinfo service supports the -c option for specifying SNMP Community Name but its use is not required. The use of the -c option is considered a security risk because doing a ps command could find the SNMP Community Name. If it is important to keep the SNMP Community Name protected, change permissions on /tmp/hacmp.out, /etc/snmpd.conf, /smit.log and /usr/tmp/snmpd.log to not be world readable.
Note: See the AIX documentation for full information on the /etc/snmpd.conf file. Version 3 (default for AIX 5.2 and up) has some differences from Version 1.
/etc/snmpd.peers
The /etc/snmpd.peers file configures snmpd SMUX peers. During installation, HACMP adds the following entry to include the clsmuxpd password to this file:
/etc/syslog.conf
The /etc/syslog.conf configuration file is used to control output of the syslogd daemon, which logs system messages. During the install process HACMP adds entries to this file that direct the output from HACMP-related problems to certain files.
# example: # "mail messages, at debug or higher, go to Log file. File must exist." # "all facilities, at debug and higher, go to console" # "all facilities, at crit or higher, go to all users" # mail.debug /usr/spool/mqueue/syslog # *.debug /dev/console # *.crit * # HACMP Critical Messages from HACMP local0.crit /dev/console # HACMP Informational Messages from HACMP local0.info /usr/es/adm/cluster.log # HACMP Messages from Cluster Scripts user.notice /usr/es/adm/cluster.log # HACMP/ES for AIX Messages from Cluster Daemons daemon.notice /usr/es/adm/cluster.logThe /etc/syslog.conf file should be identical on all cluster nodes.
/etc/trcfmt
The /etc/trcfmt file is the template file for the system trace logging and report utility, trcrpt. The installation process adds HACMP tracing to the trace format file. HACMP tracing is performed for the clstrmgr and clinfo daemons.
Note: HACMP 5.3 and up no longer uses the clsmuxpd daemon; the SNMP server functions are included in the Cluster Manager—the clstrmgr daemon.
/var/spool/cron/crontab/root
The /var/spool/cron/crontab/root file contains commands needed for basic system control. The installation process adds HACMP logfile rotation to the file.
HACMP Scripts
The HACMP software contains the following scripts.
Startup and Shutdown Scripts
The HACMP software uses each of the following scripts during starting and stopping the cluster services:
/usr/es/sbin/cluster/utilities/clstart
The /usr/es/sbin/cluster/utilities/clstart script, which is called by the /usr/es/sbin/cluster/etc/rc.cluster script, invokes the AIX 5L System Resource Controller (SRC) facility to start the cluster daemons. The clstart script starts HACMP with the options currently specified on the System Management (C-SPOC) > Manage HACMP Services > Start Cluster Services SMIT panel.
There is a corresponding C-SPOC version of this script that starts cluster services on each cluster node. The /usr/es/sbin/cluster/sbin/cl_clstart script calls the HACMP clstart script.
At cluster startup, clstart looks for the file /etc/rc.shutdown. The system file /etc/rc.shutdown can be configured to run user-specified commands during processing of the AIX 5L /usr/sbin/shutdown command.
Newer versions of the AIX 5L /usr/sbin/shutdown command automatically call HACMP's /usr/es/sbin/cluster/etc/rc.shutdown, and subsequently call the existing /etc/rc.shutdown (if it exists).
Older versions of the AIX 5L /usr/sbin/shutdown command do not have this capability. In this case, HACMP manipulates the /etc/rc.shutdown script, so that both /usr/es/sbin/cluster/etc/rc.shutdown and the existing /etc/rc.shutdown (if it exists) are run. Since HACMP needs to stop cluster services before the shutdown command is run, on cluster startup, rc.cluster replaces any user supplied /etc/rc.shutdown file with the HACMP version. The user version is saved and is called by the HACMP version prior to its own processing. When cluster services are stopped, the clstop command restores the user's version of rc.shutdown.
/usr/es/sbin/cluster/utilities/clstop
The /usr/es/sbin/cluster/utilities/clstop script, which is called from the SMIT Stop Cluster Services panel, invokes the SRC facility to stop the cluster daemons with the options specified on the Stop Cluster Services panel.
There is a corresponding C-SPOC version of this script that stops cluster services on each cluster node. The /usr/es/sbin/cluster/sbin/cl_clstop script calls the HACMP clstop script.
Also see the notes on /etc/rc.shutdown in the section on clstart above for more information.
/usr/es/sbin/cluster/utilities/clexit.rc
If the SRC detects that the clstrmgr daemon has exited abnormally, it executes the /usr/es/sbin/cluster/utilities/clexit.rc script to halt the system. If the SRC detects that any other HACMP daemon has exited abnormally, it executes the clexit.rc script to stop these processes, but does not halt the node.
You can change the default behavior of the clexit.rc script by configuring the /usr/es/sbin/cluster/etc/hacmp.term file to be called when the HACMP cluster services terminate abnormally. You can customize the hacmp.term file so that HACMP will take actions specific to your installation. See the hacmp.term file for full information.
/usr/es/sbin/cluster/etc/rc.cluster
If the Start at system restart option is chosen on the System Management (C-SPOC) > Manage HACMP Services > Start Cluster Services SMIT panel, the /usr/es/sbin/cluster/etc/rc.cluster script is called by the /etc/inittab file to start HACMP. The /usr/es/sbin/cluster/etc/rc.cluster script does some necessary initialization and then calls the usr/es/sbin/cluster/utilities/clstart script to start HACMP.
The /usr/es/sbin/cluster/etc/rc.cluster script is also used to start the clinfo daemon on a client.
A corresponding C-SPOC version of this script starts cluster services on each cluster node. The /usr/es/sbin/cluster/sbin/cl_rc.cluster script calls the HACMP rc.cluster script.
See the man page for rc.cluster for more information.
/etc/rc.net
The /etc/rc.net script is called by the /usr/es/sbin/cluster/etc/rc.cluster script to configure and start the TCP/IP interfaces and to set the required network options. The /etc/rc.net script is used in the boot process to retrieve interface information from the ODM and to configure all defined interfaces. If IP address takeover is configured, the /etc/rc.net script is called from the /usr/es/sbin/cluster/etc/rc.cluster script at cluster startup instead of during the boot process.
Event Scripts
The node, network, resource group, server, site, and other event scripts are called by the cluster daemons to respond to cluster events. The event scripts are found in the /usr/es/sbin/cluster/events directory.
For more information about these scripts, see the chapter on planning cluster events in the Planning Guide, and Chapter 6: Configuring Cluster Events in this Guide.
/usr/es/sbin/cluster/etc/clinfo.rc Script
The /usr/es/sbin/cluster/etc/clinfo.rc script, which is invoked by the clinfo daemon whenever a network or node event occurs, updates the system’s ARP cache. You can customize this script for additional processing. There must be a copy of the /usr/es/sbin/cluster/etc/clinfo.rc script on each node in the cluster. See the clinfo.rc man page for additional information.
![]() ![]() ![]() |