PreviousNextIndex

Appendix B: Command Execution Language Guide


This appendix describes how you can use the HACMP for AIX 5L Command Execution Language (CEL) to create additional Cluster Single Point of Control (C-SPOC) commands for your cluster. It also explains CEL constructs and how to convert a command’s “execution plan” written in CEL into a Korn shell (ksh) C-SPOC script.

You should be familiar with Korn shell programming (and with programming concepts in general) before attempting to create C-SPOC commands.

Overview

CEL is a programming language that lets you integrate the dsh command’s distributed functionality into each C-SPOC script the CEL preprocessor (celpp) generates. When you invoke a C-SPOC script from a single cluster node to perform an administrative task, the script is automatically executed on all nodes in the cluster. Without C-SPOC’s distributed functionality, you must execute each administrative task separately on each cluster node, which can lead to inconsistent node states within the cluster.

Appendix C: HACMP for AIX 5L Commands in the Administration Guide provides a list of all C-SPOC commands provided with the HACMP for AIX 5L software.

Creating C-SPOC Commands

C-SPOC commands are written as execution plans in CEL. Each plan contains constructs to handle one or more underlying AIX 5L tasks (a command, executable, or script) with a minimum of user input. An execution plan becomes a C-SPOC command when the /usr/es/sbin/cluster/utilities/celpp utility converts it into a cluster aware ksh script, meaning the script uses the C-SPOC distributed mechanism—the C-SPOC Execution Engine—to execute the underlying AIX 5L commands on cluster nodes to complete the defined tasks.

C-SPOC commands provide a means for controlling the execution of specific tasks on cluster nodes, collecting status and log information in the /tmp/cspoc.log file, and responding to errors generated by its script. Each command corresponds to an existing, underlying AIX 5L system administration task that lets you maintain user accounts, maintain shared Logical Volume Manager (LVM) components, or control HACMP cluster services on a cluster-wide basis.

To create a C-SPOC command:

    1. Write an execution plan for the command.
    2. Run celpp to convert the plan (.cel file) into a ksh script and make the script executable.
    3. Store the script in a user-defined directory.

The following sections describe each step.

Writing an Execution Plan

In a C-SPOC cluster environment, various commands let you manage user and group accounts, logical volumes, and cluster services. You can expand this command set by creating new commands for specific tasks that meet your administrative needs.

To create a new command, you first write an execution plan for the command. The execution plan contains ksh and CEL constructs (statements and clauses) that together describe the tasks the script will perform. CEL constructs are shown in bold in the following pseudo execution plan example. Execution plans you create should be similar in format and content to this example.

For an explanation of each construct and to see an actual execution plan for the cl_chuser command see the CEL Constructs section.

################################################################### 
#Other CEL scripts may be included. This one defines routines used 
#by all C-SPOC scripts (e.g. initialization, verification, logging, 
#etc.) 
%include cl_path.cel 
%include cl_init.cel 
#The following variables must be defined when including cl_init.cel in 
an execution plan: 
_CSPOC_OPT_STR=""       # Define valid C-SPOC option flags 
_OPT_STR=""             # Define valid option flags for the generated 
script 
_USAGE=""               # Define the usage for the generated script 
%define ERROR_CODE 2 
#An error that is not handled within a try statement using %except 
#can be handled using a global %others statement. 
%others 
  print "ERROR: Command failed!" 
%end 
#Each plan must contain a try statement to execute commands across 
#nodes in the cluster. A %try_parallel executes commands on all 
#nodes simultaneously. Note that only a single line ksh statement 
#is allowed after a %try_parallel or %try_serial. 
%try_parallel 
  ls -l $* 
  %except ERROR_CODE 
    print "ERROR: Unable to open: $*" 
  %end 
%end 
################################################################### 

For a description of option string variables used in the preceding example, refer to the cl_init.cel file in the /usr/es/sbin/cluster/samples/cspoc directory. The cl_init.cel file provides examples of functionality required in any execution plan you create; it should be included in each .cel file.

The initialization and verification routines in the cl_init.cel file provide the following functionality:

  • Get a list of nodes in the cluster.
  • Get a list of target nodes for command execution.
  • Determine nodes associated with any resource groups specified.
  • Process and implement the standard C-SPOC flags (-f, -n, and -g).
  • Validate option strings. Requires several environment variables to be set within the plan (_OPT_STR, CSPOC_OPT_STR, and _USAGE).
  • Save log file entries upon command termination.
  • Perform C-SPOC verification as follows:
  • Ensure dsh is available in $PATH
  • Check the HACMP version on all nodes.
  • The cl_path .cel file sets the PATH variable so that the C-SPOC and HACMP functions can be found at runtime. This is essential for execution plans that make use of any HACMP command, or the C-SPOC try_serial or try_parallel operations.

    Encoding and Decoding Command-Line Arguments

    If the command execution plans you create require that you pass additional arguments (or perform additional processing to the existing command-line arguments), you must use the C-SPOC clencodearg and cldecodearg programs located in /usr/es/sbin/cluster/cspoc directory to code and decode a list of arguments. This ensures that the CEL execution engine handles the embedded spacing and quotes within arguments across multiple invocations of the shell.

    For example, in the following command, the argument “John Smith” contains an embedded space:

    chuser gecos="John Smith" jsmith 
    

    Thus, you should encode all command-line arguments to C-SPOC commands unless they begin with a dash. Arguments that begin with a dash are generally command-line flags and do not contain spaces or quotes. If a string that begins with a dash is passed to the clencodearg or cldecodearg program, the string is passed through without being encoded or decoded.

    The following script fragments shows how to encode and decode a list of arguments.

    To encode a list of args:

    ENCODED_ARGS="" 
    for ARG in "$@" 
    do 
      ENCODED_ARGS="$ENCODED_ARGS $(print ARG | clencodearg)" 
    done 
    To decode a list of args: 
    UNENCODED_ARGS="" 
    for ARG in $ENCODED_ARGS 
    do 
      UNENCODED_ARGS="$UNENCODED_ARGS $(print $ARG | cldecodearg)" 
    done 
    
    Warning: You must use the clencodearg or cldecodearg program to process command-line arguments that are to be passed to any commands contained within a %try_serial or %try_parallel statement because the C-SPOC Execution Engine (cdsh) tries to decode all command-line arguments before executing the command. See CEL Constructs for more information about %try_serial and %try_parallel statements.
    Warning: Any arguments obtained from the environment variables set by routines within cl_init.cel, such as _getopts(), will have been encoded. If a command contained within a %try_serial or %try_parallel statement includes arguments generated within the execution plan, then they must first be encoded. For example, most execution plans pass command-line arguments as follows:
    %try_parallel
    chuser $_CMD_ARGS
    %end

    The following CEL plan uses the clencodearg and cldecodearg programs.

    Example CEL Plan

    #Initialize C-SPOC variables required by cl_init.cel 
      _CMD_NAME=$(basename $0) # Specify the name of this script. 
      _CSPOC_OPT_STR="d:f"     # Specify valid C-SPOC option flags. 
      _OPT_STR="+2"            # Specify valid AIX command option flags. 
    #Specify a Usage statement for this script. 
      _USAGE="USAGE: cl_chuser [-cspoc -f] Attr=Value...Username" 
    #Initialize variables local to this script. The default return code 
    #is '0' for success. 
      RETCODE=0 
    #Include the required C-SPOC Initialization and Verification 
    #Routines 
    
    %include cl_path.cel
    %include cl_init.cel
    #This define makes the following exception clause more readable. 
      %define USER_NOT_FOUND 2 
    #Get the username, which is the last command line arg and perform 
    #some crude checks for validity. Note: This is an example of 
    #decoding args within a script. 
      user=${_CMD_ARGS##*[     ]} # The []'s contain a tab & a space! 
      case $user in 
      -*|"") 
            print "$_USAGE" 
            exit 2 
            ;; 
      esac 
    #Since cl_init.cel has encoded all the args we must decode it to 
    # use it. 
        Duser=$(print $user | cldecodearg) 
    #Construct a Gecos field based on the 411 entry for the username. 
    #Note: 411 is a local script that prints the following 
    # tab-separated fields: username   Firstname Lastname  Work_phone 
    # Home_phone 
    #This plan cuts the "Firstname Lastname" to put into the Gecos 
    #field of /etc/passwd 
      GECOS=$(/usr/local/bin/411 $Duser | cut -d'     ' -f2) 
    #Construct a new set of command line args. Note the following: 
    # 1) We put the 'gecos=' arg just before the username so that it 
    #    will supercede any that may have been specified on the 
    #    command line. 
    # 2) We must encode the 'gecos=' arg explicitly. 
    # 3) This is an example of encoding args specified inside a script. 
     NEW_CMD_ARGS=${_CMD_ARGS%[      ]*} # []'s contain a tab & a space 
     NEW_CMD_ARGS="$NEW_CMD_ARGS $(print gecos="$GECOS" | HR>    
    clencodearg) $user" 
    #Perform a check that the username exists on all nodes. The check 
    #is not performed when the C-SPOC force flag is specified. 
      if [[ -z "${_SPOC_FORCE}" ]] 
      then 
    #Check if user exists across all cluster nodes 
      %try_parallel _NODE _TARGET_NODES silent_err silent_out 
        lsuser $user 
        %except USER_NOT_FOUND 
          print "${_CMD_NAME}: User ${Duser} does " 
          print "not exist on node ${_NODE}" >&2" 
          RETCODE=1 
        %end 
      %end 
      fi 
    #If the username does not exist on a node then exit immediately 
      if [[ ${RETCODE} -ne 0 ]] 
      then 
            exit ${RETCODE} 
      fi 
    #Execute the chuser command in parallel on all nodes in the #cluster. 
      %try_parallel _NODE _TARGET_NODES 
            chuser $NEW_CMD_ARGS 
            %others 
              # If chuser returned an error on any node 
              # set the return code to '1' to indicate  
              # that one or more nodes failed. 
                 RETCODE=1 
            %end 
      %end 
    #Exit with the appropriate value. 
      exit ${RETCODE} 
    

    CEL Constructs

    You can use the following CEL constructs (statements or clauses) in a command’s execution plan. All C-SPOC commands must contain the %include statement, a %try_parallel or %try_serial statement, and an %end statement.

    The %include statement is used to access ksh libraries within the cl_init.cel file. These libraries make C-SPOC commands “cluster aware.” The %except and %others clauses are used typically for error handling in response to a command’s execution on one or more cluster nodes.

    %define statement:

    %define key value 
    

    For improved readability, the %define statement (keyword) is used to provide descriptive names for error_id values in %except clauses. The error_id given in the define statement is inserted in place of the given ID in any subsequent %except clauses.

    %include statement:

    %include filename 
    

    The %include statement allows a copy of a file to be included in an execution plan, which means that common code can be shared among execution plans. The %include statement also allows C-SPOC commands with similar algorithms to share the same plans. Using %include, CEL statements can be used within library routines included in any execution plan.

    %try_parallel statement:

    %try_parallel node nodelist [silent_err] [silent_out] 
    	ksh statement 
    	[ %except clause ... ] 
    [%end | %end_p ] 
    

    The %try_parallel statement executes the enclosed ksh statement simultaneously across all nodes in the nodelist. It waits until a command completes its execution on all cluster nodes before checking for errors.

    The %try_parallel statement is equivalent to the following pseudo-code:

            for each node in nodelist 
                execute ksh statement in the background on each node 
            end 
            wait for all background execution to complete on all nodes 
            for each node in nodelist 
                check status for each node and execute %except 
    	    clause(s) 
            end 
    

    %try_serial statement:

    %try_serial node nodelist [silent_err] [silent_out] 
    	ksh statement 
    	[ %except clause ... ] 
    	[%others clause] 
    [%end | %end_s ] 
    

    The %try_serial statement executes the enclosed ksh statement consecutively on each node specified in the nodelist. The command must complete before %try_serial continues its execution on the next node. All %try_serial statements sequentially check for errors after a command completes on each node in the list.

    The %try_serial statement is equivalent to the following pseudo-code:

    for each node in nodelist

    execute ksh statement in the on each node

    end

    for each node in nodelist

    check status for each node and execute %except

    clause(s)

    end

    Note: Any non-flag arguments (those not preceded by a dash) used inside a try statement must be encoded using the /usr/es/sbin/cluster/cspoc/clencodearg utility; otherwise, arguments will be decoded incorrectly.

    %except clause:

    %except error_id 
    	ksh code 
    [ %end | %end_e ] 
    

    The %except clause is used for error checking on a per-node basis, and for the execution of the ksh statement contained within a %try_parallel or %try_serial statement. If the ksh statement fails or times out, the %except clauses define actions to take for each return code value.

    If an error_id is defined using the %define statement, the return code of the ksh statement (the status code set by a command’s execution on a particular node) is compared to the error_id value. If a match is found, the ksh code within the %except statement is executed.

    If the %others clause is used within the %except clause, the ksh code within the %except clause is executed only if the return code value did not match an error_id defined in previous %except clauses.

    An %except clause can be defined within or outside a try statement. If it is defined within a try statement, the clause’s scope is local to the %try_parallel or %try_serial statement.

    If the %except clause is defined outside a try statement, it is used in any subsequent %try_serial or %try_parallel statements that do not already provide one for the specified error_id or %others clause. In this case, the %except clause’s scope is global until an %unexcept statement is found. Global %except clauses are used to simplify and standardize repetitive types of error handling.

    %stop_try clause:

    %except error_id 
    	ksh code 
    	[ %stop_try clause ] 
    	ksh code 
    [ %end | %end_e ] 
    

    The %stop_try clause can be used within an %except clause; it causes an exit from the enclosing try statement. This statement has the effect of a break statement in other languages.

    The %stop_try clause has a different effect that depends on whether it is defined with a %try_parallel versus a %try_serial statement.

    In a %try_serial statement, defining a %stop_try clause prevents further execution of the ksh statement on all cluster nodes; it also prevents further error checking defined by the %except clause.

    In a %try_parallel statement, defining a %stop_try clause prevents error checking on all cluster nodes, since execution of the try statement happens simultaneously on all nodes. The commands within the ksh statement will have completed on other nodes by the time the %except clauses are evaluated.

    %others clause:

    %others error_id 
    	ksh code 
    	[ %stop_try clause ] 
    	ksh code 
    [ %end | %end_o ] 
    

    The %others clause is the default error action performed if a command’s return code does not match the return code of a specific %except clause. The %others clause can be used in a try statement, or globally within an execution plan.

    %end statement:

    %end 
    

    The %end statement is used with all CEL constructs. Ensure that your .cel file includes an %end statement with each statement or clause. A construct like %try_parallel, for example, can be ended with %end or with %end_p, where p represents parallel.

    Actual Execution Plan

    The following example shows an actual execution plan, cl_chuser.cel, written in CEL that uses the required constructs to create the cl_chuser command. The CEL constructs are shown in bold. Other examples of execution plans are in the /usr/es/sbin/cluster/samples/cspoc directory.

    ################################################################### 
    # Name: 
    #       cl_chuser.cel 
    # 
    # Description: 
    #   The cl_chuser command changes user attributes for a user on all  # nodes in an 	 
    HACMP cluster. 
    # 
    #   Usage: cl_chuser [-cspoc "[-f]"] Attribute=Value ... Name 
    # 
    # Arguments: 
    #   The cl_chuser command arguments include all options and 
    #   arguments that are valid for the chuser command as well as 
    #   C-SPOC specific arguments. The C-SPOC specific arguments are as #follows: 
    #      	-f			C-SPOC force flag 
    # 
    # Return Values: 
    #       0       success 
    #       1       failure 
    # 
    ################################################################### 
    # Initialize variables 
    _CMD_NAME=`basename $0` 
    _CSPOC_OPT_STR="d:f" 
    _OPT_STR="" 
    _USAGE="Usage: cl_chuser [-cspoc #7f>[-f]] Attribute=Value ... Name" 
    _MSET=12 
    _RETCODE=0 
    # Default exception handling for a COULD_NOT_CONNECT error 
    %except 1000 
            nls_msg -l $cspoc_tmp_log ${_MSET} CDSH_UNABLE_TO_CONNECT "${_CMD_NAME}: 
    Unable to 
     connect to node ${_NODE}" >& 2 
            if [[ -z "${_SPOC_FORCE}" ]] 
            then 
                    exit 1 
            fi 
    %end 
    # C-SPOC Initialization and Verification 
    %include cl_path.cel 
    %include cl_init.cel 
    %define USER_NOT_FOUND 2 
    user=${_CMD_ARGS##* } 
    if [[ -z "${_SPOC_FORCE}" ]] 
    then 
    	# 
    	# Check if user exists across all cluster nodes 
    	# 
    	%try_parallel _NODE _TARGET_NODES silent_err silent_out 
    		lsuser $user 
    		%except USER_NOT_FOUND 
    	        	nls_msg -l $cspoc_tmp_log ${_MSET} 1 "${_CMD_NAME}: User ${user} does not 
    exist on node ${_NODE}" ${_CMD_NAME} ${user} ${_NODE} >& 2 
    			_RETCODE=1 
            	%end 
    	%end 
    fi 
    # If user does not exist on a node, exit 1 
    if [[ ${_RETCODE} -ne 0 ]]  
    then 
    	exit ${_RETCODE} 
    fi 
    # Run chuser across the cluster nodes 
    %try_parallel _NODE _TARGET_NODES 
    	chuser $_CMD_ARGS 
    	%others 
    		# If chuser returned an error on any node, exit 1 
    		_RETCODE=1 
    	%end 
    %end 
    exit ${_RETCODE} 
    

    Converting Execution Plans to ksh Scripts

    After writing an execution plan for a command, use the CEL preprocessor (celpp) to convert it into a ksh script; then execute the chmod command on the script to make it executable. An executable ksh script includes all code required to implement the command across cluster nodes. The preprocessor generates much of the code you would otherwise have to write.

    To convert an execution plan into a ksh script:

      1. Change directories to the one containing the execution plan (.cel file) for the command you want to convert.
      2. Enter the following command to run celpp, specifying the .cel file as the input file and the command to be generated as the output file:
    celpp [-i inputfile] [-o outputfile] [-I IncludeDirectory]
    where the elements of this command line are:

    • -i inputfile
    Uses inputfile for input, or stdin by default.
    • -o outputfile
    Uses outputfile for output, or stdout by default.
    • -I IncludeDirectory
    Uses the specified directory name to locate .cel files (execution plans). You can specify multiple -I options. The cl_init.cel and cl_path.cel files are installed in /usr/es/sbin/cluster/samples/cspoc; The IncludeDirectory should normally be specified.


    For example, enter:

    celpp -i cl_chuser.cel -o cl_chuser -I /usr/es/sbin/cluster/samples/cspoc]
    This command converts the cl_chuser.cel file into a ksh script.
      3. Enter the following command to make the generated ksh script executable:
    chmod +x inputfile
    For example, enter:
    chmod +x cl_chuser

    You can now invoke the C-SPOC command. If you change a command’s execution plan after converting it, repeat the preceding steps to generate a new ksh script. Then make the script executable. Note that the preceding steps can be included in a Makefile.

    Storing C-SPOC Commands

    The HACMP for AIX 5L software must be installed in your cluster to use C-SPOC commands. You can store C-SPOC commands you create in any directory.

    Handling Command Output

    When you are ready to use a C-SPOC command, be aware that all error output (messages) resulting from the command’s execution across the cluster is returned to stdout and stderr. By default, stdout and stderr remain directed to your display. You can override this action by using the silent_err or silent_out arguments with the try statements in a command’s execution plan. If the silent_err flag is defined, stderr from the enclosed try ksh statement will appear only in the $try_err file. If the silent_out flag is defined, stdout from the enclosed try ksh statement will appear only in the $try_out file. Refer to Writing an Execution Plan for an example of how you use these arguments.

    Whatever way you direct error output, all error messages are prefixed with the name of the node on which the command was executed. When the command completes on each node, error messages are logged in the cspoc.log file—the log file for all C-SPOC commands. for more information about C-SPOC-related log entries, see Chapter 2: Using Cluster Log Files.


    PreviousNextIndex