Index
-
A -
B -
C -
D -
E -
F -
G -
H -
I -
J -
L -
M -
N -
O -
P -
Q -
R -
S -
T -
U -
V -
W
/etc/hosts file
check before starting cluster
listing IP labels
loopback and localhosts as aliases
missing entries for netmon.cf
/etc/locks file
/etc/netsvc.conf file
editing for nameserving
/etc/rc.net script
checking the status of
/etc/syslogd file
redirecting output
/tmp/cspoc.log file
recommended use
/tmp/emuhacmp.out file
message format
understanding messages
viewing its contents
/tmp/hacmp.out file
correcting sparse content
displaying event summaries
first node up gives network error message
message formats
recommended use
selecting verbose script output
troubleshooting TCP/IP
understanding messages
/usr
becomes too full
/usr/es/sbin/cluster/cl_event_summary.txt
/usr/es/sbin/cluster/clinfo daemon
Clinfo
/usr/es/sbin/cluster/clstrmgrES daemon
Cluster Manager
/usr/es/sbin/cluster/cspoc/clencodearg utility
/usr/es/sbin/cluster/etc/clhosts file
invalid hostnames/addresses
on client
updating IP labels and addresses
/usr/es/sbin/cluster/etc/rhosts
troubleshooting
/usr/es/sbin/cluster/events/utils directory
/usr/es/sbin/cluster/history/cluster.mmddyyyy file
/usr/es/sbin/cluster/server.status
/usr/es/sbin/cluster/snapshots/clsnapshot.log
/usr/es/sbin/cluster/utilities/celpp utility
/usr/es/sbin/cluster/utilities/clRGinfo command
example output
/usr/es/sbin/cluster/utilities/clRGmove command
/usr/es/sbin/cluster/utilities/clruncmd command
/usr/es/sbin/cluster/utilities/clsnapshot utility
clsnapshot
/usr/es/sbin/cluster/utilities/cltopinfo command
/usr/es/sbin/rsct/bin/hatsdmsinfo command
/usr/sbin/snap command
snap-e collects HACMP information
/var/ha/log/grpglsm log file
/var/ha/log/grpsvcs
recommended use
/var/ha/log/topsvcs
/var/hacmp/clverify/clverify.log file 1 2
/var/hacmp/log/clutils.log 1 2 3
/var/hacmp/utilities/cl_configassist.log
/var/spool/lpd/qdir
/var/spool/qdaemon
A
AIX 5L network interface
disconnected from HACMP IP label
recovery
application monitoring
troubleshooting
applications
fail on takeover node
identifying problems
inaccessible to clients
troubleshooting
ARP cache
flushing
arp command 1 2
checking IP address conflicts
assigning
persistent IP labels
ATM
arp command
LAN emulation
troubleshooting
troubleshooting
automatic cluster configuration monitoring
configuring
automatic error notification
failure
B
baud rate
for tty 1 2
C
C-SPOC
checking shared filesystems
checking shared logical volumes
checking shared volume groups
C-SPOC commands
creating
encoding/decoding arguments
C-SPOC scripts
using CEL
CDE
hangs after IPAT on HACMP startup
CEL
Command Execution Language
CEL guide
CEL constructs
writing execution plans
CEL plan
example
celpp
converting execution plan to ksh script
cfgmgr command
unwanted behavior in cluster
changing
network modules
checking
cluster services and Processes
clcheck_server
cluster snapshot file
HACMP cluster
logical volume definitions
shared filesystem definitions
shared volume group definitions
volume group definitions
checking cluster configuration with Online Planning Worksheets
cl_activate_fs utility
cl_activate_nfs utility
cl_activate_vgs utility
cl_convert
not run due to failed installation
cl_convert utility
cl_deactivate_fs utility
cl_deactivate_nfs utility
cl_deactivate_vgs utility
cl_disk_available utility
cl_echo utility
cl_export_fs utility
cl_fs2disk utility
cl_get_disk_vg_fs_pvids utility
cl_init.cel file
cl_is_array utility
cl_is_scsidisk utility
cl_log utility
cl_lsfs command
checking shared filesystem definitions
cl_lslv command
checking logical volume definitions
cl_lsvg command
checking shared volume group definitions
cl_nfskill command
unmounting NFS filesystems
cl_path .cel file
cl_raid_vg utility
cl_rsh remote shell command
cl_scsidiskreset command
fails and writes errors to /tmp/hacmp.out file
cl_scsidiskreset utility
cl_scsidiskrsrv utility
cl_swap_HPS_IP_address utility
cl_swap_HW_address utility
cl_swap_IP_address utility
cl_sync_vgs utility
cl_unswap_HW_address
clcheck_server
checking
cluster services and processes
clcmod.log
clcmoddiag.log
clcomd
logging
refreshing
tracing
troubleshooting
clcomd.log file
clcomddiag.log
clcomdES and clstrmgrES fail to start
newly installed AIX 5L nodes
clearing
SSA disk fence registers
clhosts file
editing on client nodes
clients
cannot access applications
connectivity problems
not able to find clusters
Clinfo
checking the status of
exits after starting
not reporting that a node is down
not running
restarting to receive traps
trace ID
tracing 1 2
clRGinfo command
reference page
clRGmove utility
syntax
clsetenvgrp script
used for serial processing
clsnapshot utility 1 2
clsnapshot.log file
clstat utility
finding clusters
clstrmgrES and clinfoES daemons
user-level applications controlled by SRC
cluster
checking configuration with cluster snapshot utility
troubleshooting configuration
tuning performance parameters
cluster configuration
automatic cluster verification
cluster events
emulating
resetting customizations
cluster history log file
message format and content
Cluster Information Program
Clinfo 1 2
cluster log files
redirecting
Cluster Manager
cannot process CPU cycles
checking the status of
hangs during reconfiguration
trace ID
tracing
troubleshooting common problems 1 2
will not start
cluster security
troubleshooting configuration 1 2
cluster services
not starting
starting on a node after a DARE
Cluster SMUX Peer
checking the status of
failure
trace ID
tracing 1 2
cluster snapshot
checking during troubleshooting
creating prior to resetting tunable values
files
information saved
ODM data file
using to check configuration
cluster verification utility
automatic cluster configuration monitoring
checking a cluster configuration
tasks performed
troubleshooting a cluster configuration
cluster.log file
message format
recommended use
viewing its contents
cluster.mmddyyyy file
cluster history log
recommended use
clverify log file
logging to console
clverify.log file
collecting
data from HACMP clusters
Command Execution Language
CEL guide
command-line arguments
encoding/decoding C-SPOC commands
commands
arp 1 2 3
cl_convert
cl_nfskill
cl_rsh
cl_scsidiskreset
clRGinfo
clRGmove
clruncmd 1 2
cltopinfo
configchk
df
dhb_read
diag 1 2 3
errpt
fsck
ifconfig 1 2
lsattr
lsdev
lsfs
lslv
lspv 1 2
lssrc
lsvg
mount
netstat
ping 1 2
snap -e
varyonvg
communications daemon
tracing
config_too_long message
configchk command
returns an Unknown Host message
configuration files
merging during installation 1 2
configuring
automatic cluster configuration monitoring
checking with snapshot utility
log file parameters
network modules
runtime parameters
configuring cluster
restoring saved configurations
conversion
failed installation
cron jobs
making highly available
cspoc.log file
message format
viewing its contents
custom scripts
print queues
samples
customizing
cluster log files
D
daemon.notice output
redirecting to /usr/tmp/snmpd.log
daemons
clinfo
exits after starting
monitoring
trace IDs
tracing
deadman switch
avoiding
cluster performance tuning
definition
fails due to TCP traffic
releasing TCP traffic
time to trigger
tuning virtual memory management
debug levels
setting on a node
dependent resource groups
processing
df command
checking filesystem space
dhb_read
testing a disk heartbeating network
dhb_read command
diag command
checking disks and adapters
testing the system unit
diagnosing problems
recommended procedures
discovery
disk adapters
troubleshooting
disk enclosure failure detection
disk heartbeating
disk fencing
job type
disk heartbeating
disk enclosure failure detection
failure detection
troubleshooting
disk heartbeating network
testing
diskhb networks
failure detection
troubleshooting
disks
troubleshooting
Distributed SMIT (DSMIT)
unpredictable results
domain merge
error message displayed
handling node isolation
dynamic reconfiguration
emulating
lock
E
emulating
cluster events
enabling
I/O pacing
Enterprise Storage System (ESS)
automatic error notification
error messages
console display of
errors
mail notification of
errpt command 1 2
event duration time
customizing
event emulator
log file
event preamble
example
event summaries
cl_event_summary.txt file too large
examples with job types
reflecting resource groups parallel processing
reflecting resource groups serial processing
resource group information does not display
sample hacmp.out contents
viewing
event_error event 1 2
events
changing custom events processing
displaying event summaries
emulating dynamic reconfiguration
emulating events
event_error 1 2
processing replicated resource groups
unusual events
execution plan
converting to ksh script
exporting
volume group information
F
F1 help fails to display
failure detection
disk heartbeating
failure detection rate
changing
changing to avoid DMS timeout
failures
non-ip network, network adapter or node failures
fallback timer
example in hacmp.out event summary
Fast Failover Detection
decreasing fallover time using
Fast Failure Detection
using disk heartbeating to reduce node failure detection time.
filesystems
change not recognized by lazy update
failure to unmount
mount failures
troubleshooting
flushing
ARP cache
forced varyon
failure
fsck command
Device open failed message
fuser command
using in scripts
G
generating
trace report
Geo_Primary
problem after uninstall
Geo_Secondary
problem after uninstall
gigabit Ethernet adapters
fail with HWAT
H
HACMP
troubleshooting components
HACMP Configuration Database
security changes for HACMP
HACMP for AIX 5L
commands
syntax conventions
hacmp.out event summary
example for settling time
fall back timer example
hacmp.out log file
event summaries
setting output format
HAGEO
network issues after uninstallation
hardware address swapping
message appears after node_up_local fails
heartbeating over IP Aliases
checking
HWAT
gigabit Ethernet adapters fail
I
I/O pacing
enabling
tuning 1 2
ifconfig command 1 2
initiating a trace session
installation issues
cannot find filesystem at boot-time
unmerged configuration files
IP address
listing in arp cache
IP address takeover
applications fail on takeover node
J
job types
examples in the event summaries
fencing
parallel processing of resource groups
L
LANG variable
lazy update
filesystem changes not recognized
lock
set by dynamic reconfiguration
log
WebSmit
log files
/tmp/emuhacmp.out 1 2
/tmp/hacmp.out file 1 2
/usr/es/sbin/cluster/snapshots/clsnapshot.log
/var/hacmp/log/clutils.log 1 2 3
changing default pathnames
changing parameters on a node
clcomd
cluster.log file 1 2
cluster.mmddyyyy 1 2
collecting for problem reporting
recommended use
redirecting
system error log 1 2
types of
with cluster messages
logical volume manager (LVM)
troubleshooting
logical volumes
troubleshooting
lsattr command
lsdev command
for SCSI disk IDs
lsfs command 1 2
lslv command
for logical volume definitions
lspv command
checking physical volumes
for logical volume names
lssrc command
checking the inetd daemon status
checking the portmapper daemon status
lsvg command
checking volume group definitions
LVM
troubleshooting
M
mail
used for event notification
maxfree
migration
ODM security changes
PSSP File Collections issue
minfree
monitoring applications
troubleshooting
monitoring cluster configuration
cluster verification automatic monitor
mount command
listing filesystems
N
netmon.cf
netstat command
network interface and node status
NetView
deleted or extraneous objects in map
network
troubleshooting
network failure after MAU reconnect
will not reintegrate when reconnecting bus
network error message
network modules
changing or showing parameters
configuring
failure detection parameters
networks
diskhb
Ethernet
modify Geo networks definition
reintegration problem 1 2
single adapter configuration
Token-Ring 1 2
troubleshooting
cannot communicate on ATM Classic IP
cannot communicate on ATM LANE
Token-Ring thrashes
unusual events when simple switch not supported
NIC failure
switched networks
NIM process of RSCT being blocked
nodes
troubleshooting
cannot communicate with other nodes
configuration problems
dynamic node removal affects rejoining
non-IP networks
failure detection
O
Object Data Manager (ODM)
updating
ODM see Object Data Manager
Online Planning Worksheets
P
parallel processing
tracking resource group activity
partitioned cluster
avoiding
PCI network card
recovering from failure 1 2
permissions
on HACMP ODM files
persistent node IP label
physical volumes
troubleshooting
ping command
checking node connectivity
flushing the ARP cache
print queues
custom script
making highly available
problem reporting
process_resources event script
PSSP File Collections
migration issue
Q
quorum loss of
does not trigger selective fallover
R
rebooting
failover attempt fails
redirection
of cluster log files
refreshing the cluster communications daemon
replicated resources
event processing
resetting HACMP tunable values
Resource Group Migration
utilities
resource group recovery on node_up
troubleshooting
resource groups
monitoring status and location
clRMupdate and clRGinfo commands
processed serially unexpectedly
processing messages in hacmp.out
tracking parallel and serial processing in hacmp.out
rg_move event
multiple resource groups
rhosts
troubleshooting
RS/6000 SP system
See SP
RSCT command
dhb_read
S
scripts
activating verbose mode
making print queues highly available
recovering from failures
sample custom scripts
setting debug levels
tape_resource_start_example
SCSI devices
troubleshooting
scsidiskutil utility
security
ODM changes that may affect upgrading HACMP
selective fallover
not triggered by loss of quorum
serial processing
resource groups
serial processing of resource groups
tracking in event summaries
server.status file (see /usr/sbin/cluster/server.status)
service IP labels
listed in /etc/hosts file
setting
I/O Pacing
syncd frequency rate
settling time
example in hacmp.out event summary
single-adapter networks and netmon configuration file
sites
processing resource groups
SMIT
help fails to display with F1
open on remote node
snap command
snapshot
checking cluster snapshot file
definition
SP Switch
SP Utilities
SSA disk fence registers
clearing
ssa_clear utility
ssa_clear_all utility
ssa_configure utility
ssa_fence utility
stabilizing a node
starting
cluster services on a node after a DARE
status icons
displaying
stopping HACMP
with unmanaged resource groups
switched networks
NIC failure
syncd
setting frequency rate
syntax conventions
HACMP for AIX 5L commands
system components
checking
investigating
system error log file
message formats
recommended use
understanding its contents
system panic
invoked by deadman switch
T
tape_resource_start_example script
tape_resource_stop_example script
target mode SCSI
failure to reintegrate
TCP traffic
releasing
deadman switch
TCP/IP services
troubleshooting
Token-Ring
network thrashes
node failure detection takes too long
topsvcs daemon
messages on interface states
tracing HACMP for AIX 5L daemons
enabling tracing using SMIT
tracing HACMP for AIX 5L daemons
disabling using SMIT
generating a trace report using SMIT
overview
sample trace report
specifying a trace report format
specifying a trace report output file
specifying content of trace report
starting a trace session using SMIT
stopping a trace session using SMIT
trace IDs
using SMIT
tracing HACMP for AIX daemons
initiating a trace session
triggering
deadman switch
troubleshooting
AIX 5L operating system
applications
clcomd errors
cluster communications
cluster configuration
cluster security configuration
disk heartbeating
Ethernet networks
filesystems
guidelines for
HACMP components
heartbeating over IP Aliases
investigating system components
LVM entities
networks
recommended procedures
resource group processing
rhosts
SCSI disks and adapters
snap -e command
solving common problems
system hardware
TCP/IP subsystem
Token-Ring networks
volume groups
VPN
tty baud rate
TTY baud rate
changing
tuning
I/O pacing
virtual memory management
tuning parameters
cluster performance
resetting
U
unmerged configuration files
installing
unmounting
filesystems
upgrading
pre- and post-event scripts
utilities
C-SPOC (see also C-SPOC utility)
checking shared filesystems
checking shared logical volumes
checking shared vgs
cl_activate_fs
cl_activate_nfs
cl_activate_vgs
cl_deactivate_fs
cl_deactivate_nfs
cl_deactivate_vgs
cl_disk_available
cl_echo
cl_export_fs
cl_fs2disk
cl_getdisk_vg_fs_pvids
cl_is_array
cl_is_scsidisk
cl_log
cl_nfskill
cl_raid_vg
cl_scdiskreset
cl_scsidiskrsrv
cl_swap_HPS_IP_address
cl_swap_HW_address
cl_swap_IP_address
cl_sync_vgs
cl_unswap_HW_address
clsnapshot 1 2
clstat
cluster verification
event emulation
scripts
scsidiskutil
ssa_clear
ssa_clear_all
ssa_configure
ssa_fence
stop_clmarketdemo
V
varyon
checking passive or active mode
varyonvg command
fails during takeover
fails if volume group varied on
troubleshooting
verbose script output
activating
viewing
cluster.log file
cspoc.log file
emuhacmp.out log file
event summaries
virtual memory management
tuning deadman switch
VLAN
troubleshooting
vmstat command
vmtune command
volume groups
checking varyon state
disabling autovaryon at boot
troubleshooting
VPN
troubleshooting
W
WebSMIT
problems viewing the cluster
WebSmit log