PreviousNextIndex

Chapter 7: Defining Shared LVM Components


This chapter describes how to define the LVM components shared by the nodes in an HACMP cluster. This chapter contains the following sections:

  • Overview
  • Defining Shared LVM Components for Non-Concurrent Access
  • Defining LVM Components for Concurrent Access.
  • Overview

    Creating the volume groups, logical volumes, and filesystems shared by multiple nodes in an HACMP cluster requires that you perform steps on all those nodes. In general, you define the components on one node (referred to in the text as the source node) and then import the volume group onto each of the other nodes that will be part of the resource group (referred to as destination nodes). This ensures that the definitions in the HACMP configuration database for the shared components are the same on all the nodes that may access them.

    Using SMIT, you can collect information about all volume groups available either on a local node, or on all nodes in the cluster defined to the resource group. Later, you can automatically import these volume groups onto all the destination nodes, if needed. As you configure shared resources in SMIT, when you add a volume group to a resource group, you can select the option to automatically import the volume group onto all the destination nodes in the participating resource group.

    HACMP non-concurrent access environments typically use journaled filesystems to manage data, while concurrent access environments use raw logical volumes. This chapter provides instructions for both types of environments.

    While configuring filesystems for a resource group, you can select one or more individual filesystems to be mounted in the particular resource group. In addition, you can specify that all filesystems in the specified shared volume groups to be mounted at runtime. Using this option allows you to add or delete filesystems in the particular shared volume group without resynchronizing the cluster resources after each update.

    Defining Shared LVM Components for Non-Concurrent Access

    HACMP non-concurrent access environments typically use journaled filesystems to manage data, although some database applications may bypass the journaled filesystem and access the logical volume directly.

    The key consideration is whether the environment uses mirrors. Shared logical volumes residing on SCSI devices, and IBM 7133 SSA devices should be mirrored in AIX 5L to eliminate the disk as a single point of failure. Shared volume groups residing on an IBM 2105 E10 and E20 Enterprise Storage Server RAID devices and IBM TotalStorage DS6000 and DS8000 should not be AIX 5L mirrored; these systems provide their own data redundancy.

    The following figures list the tasks you complete to define the shared LVM components with and without mirrors. Each task is described throughout the pages following the figures. Refer to your completed copies of the shared volume group worksheets as you define the shared LVM components.

    Defining Shared LVM Components

    You can define shared LVM components with or without mirroring.

    To define shared LVM components on the source node:

      1. Create the volume group.
      2. Create the journaled filesystem.
      3. Rename the jfslog and the logical volume, so that they are unique across the cluster.
      4. Mirror the jfslog and logical volume (only with AIX 5L mirroring on one node in the cluster).
      5. Vary off the volume group.

    To define shared LVM components (with or without AIX 5L mirroring) on the other nodes that will access this volume group:

      1. Import the volume group.
      2. Change the volume group options so that it does not vary on at system startup.
      3. Vary off the volume group.

    Creating a Shared Volume Group on Source Node

    To create a shared volume group on a source node:

      1. Enter smit mkvg
      2. Use the default field values unless your configuration has other requirements, or unless you are specifically instructed otherwise here. (Only fields with specific HACMP instructions are shown.)
    Note: If you are creating a mirrored volume group, select two or three times as many disks for the volume group as you will need for your data, depending on the number of mirrors you select.
    VOLUME GROUP name
    The name of the shared volume group must be unique within the cluster and distinct from the service IP label/address and resource group names; it should relate to the application it serves, as well as to any corresponding device, such as websphere_service_address.
    Activate volume group AUTOMATICALLY at system restart?
    Set to no so that the volume group can be activated as appropriate by the cluster event scripts.
    Volume Group MAJOR NUMBER
    If you will be using NFS, make sure to use the same major number on all nodes. Use the lvlstmajor command on each node to determine a free major number common to all nodes.


    Creating a Shared Filesystem on Source Node

    To create a shared filesystem on a source node:

      1. Enter smit crjfs
    When you create a journaled filesystem, AIX 5L creates the corresponding logical volume. Therefore, you do not need to define a logical volume.
      2. Rename both the logical volume and the log logical volume for the filesystem and volume group.
      3. Review the settings for the following fields:
    Mount AUTOMATICALLY at
    system restart?
    Make sure this field is set to no.
    Start Disk Accounting
    Set this field to no unless you want disk accounting.

    Renaming jfslogs and Logical Volumes on Source Node

    AIX 5L assigns a logical volume name to each logical volume it creates. Examples of logical volume names are /dev/lv00 and /dev/lv01. Within an HACMP cluster, the name of any shared logical volume must be unique. Also, the journaled filesystem log (jfslog) is a logical volume that requires a unique name in the cluster.

    To make sure that logical volumes have unique names, rename the logical volume associated with the filesystem and the corresponding jfslog logical volume. Use a naming scheme that indicates the logical volume is associated with a certain filesystem. For example, lvsharefs could name a logical volume for the /sharefs filesystem

    To rename jfslogs and logical volumes:

      1. Use the lsvg -l volume_group_name command to determine the name of the logical volume and the log logical volume (jfslog) associated with the shared volume groups. In the command output:
  • Look for the logical volume name that has type jfs. This is the logical volume.
  • Then look for the logical volume name that has type jfslog. This is the log logical volume.
    1. 2. Use the smit chlv fastpath to rename the logical volume and the log logical volume.
      3. After renaming the jfslog or a logical volume:
  • Check the /etc/filesystems file to make sure the dev and log attributes reflect the change.
  • Check the log attribute for each filesystem in the volume group and make sure that it has the new jfslog name.
  • Check the dev attribute for the logical volume you renamed and make sure that it has the new logical volume name.
  • Adding Copies to Logical Volume on Source Node

    Note: This step is not needed for disk subsystems that provide their own mirroring, such as the IBM 2105 Enterprise Storage Server and IBM Total Storage DS6000 and DS8000.

    To add logical volume copies on a source node:

      1. Use the smit mklvcopy fastpath to add copies to a logical volume. Add copies to both the jfslog log logical volume and the logical volumes in the shared filesystems. To avoid space problems, first mirror the jfslog log logical volume and then the shared logical volumes.
    The copies should reside on separate disks that are controlled by different disk adapters and are located in separate drawers or units, if possible.
      2. Review the number of logical volume copies. Enter:
    lsvg -l volume_group_name

    In the command output, locate the line for the logical volume for which you just added copies. Notice that the number in the physical partitions column is x times the number in the logical partitions column, where x is the number of copies.

      3. To review the placement of logical volume copies, enter:
    lspv -l hdiskx

    where hdiskx is the name of each disk to which you assigned copies. That is, you enter this command for each disk. In the command output, locate the line for the logical volume for which you just added copies. For copies placed on separate disks, the numbers in the logical partitions column and the physical partitions column should be equal. Otherwise, the copies were placed on the same disk and the mirrored copies will not protect against disk failure.

    Testing a Filesystem

    To run a consistency check on each filesystem’s information:

      1. Enter:
    fsck /filesystem_name
      2. Ensure that you can mount the filesystem by entering:
    mount /filesystem_name
      3. Ensure that you can unmount the filesystem by entering:
    umount /filesystem_name

    Varying Off a Volume Group on the Source Node

    After completing the previous tasks, use the varyoffvg command to deactivate the shared volume group. You vary off the volume group so that it can be properly imported onto the destination node and activated as appropriate by the HACMP event scripts. Enter the following command:

    varyoffvg volume_group_name 
    

    Collecting Information on Current Volume Group Configuration

    HACMP can collect information about all volume groups available across all nodes in the cluster as part of the discovery process. HACMP collects information about all shared volume groups that are currently available on the nodes in the cluster. It then determines which volume groups can be imported to the other nodes in the resource group. HACMP filters out those volume groups that are already included in any other resource groups. You can use this information later to import discovered volume groups onto other nodes in the resource group that do not have these volume groups.

    To collect the information about volume group configuration:

      1. Enter smit hacmp
      2. In SMIT, select Extended Configuration > Discover HACMP-related Information from Configured Nodes and press Enter. The information on current volume group configuration is collected, as part of the general configuration information.

    Importing a Volume Group onto Destination Nodes

    Importing the volume group onto the destination nodes synchronizes the definition in the HACMP configuration database for the volume group on each node on which it is imported.

    When adding a volume group to the resource group, you can manually import a volume group onto the destination nodes or automatically import it onto all the destination nodes in the resource group.

    Importing a Volume Group Automatically

    You can set up automatic import of a volume group.

    To automatically import a volume group:

      1. Enter smit hacmp
      2. In SMIT, select Extended Configuration > Extended Resource Configuration > HACMP Extended Resource Group Configuration > Change/Show all Resources for a Resource Group and press enter.

    This option enables HACMP to automatically import shareable volume groups onto all the destination nodes in the resource group. Automatic import allows you to create a volume group and then add it to the resource group immediately, without manually importing it onto each of the destination nodes in the resource group.

    Before automatically importing volume groups, make sure that you have collected the information on appropriate volume groups, using the Discover HACMP-related Information from Configured Nodes option.

    Note: Each volume group is assigned a major number when it is created. When HACMP automatically imports a volume group, the major number already assigned to the volume group will be used if it is available on all the destination nodes. Otherwise, any free major number will be used. NFS fallover requires that the major number be the same on all nodes. For more information, see Using NFS with HACMP in the Planning Guide.

    Prerequisites and Notes

    For HACMP to import available volume groups, ensure that the following conditions are met:

  • Volume group names must be the same across cluster nodes, and unique to the cluster.
  • Logical volumes and filesystems must have unique names.
  • All physical disks must be known to AIX 5L and have PVIDs assigned.
  • A volume group is available for auto import onto other nodes only if the physical disks on which it resides are available to all of the nodes in the resource group.
  • Importing a Volume Group Automatically

    To add volume groups to a resource group via automatic import:

      1. Enter smit hacmp
      2. In SMIT, select Extended Configuration > Extended Resource Configuration > HACMP Extended Resource Group Configuration > Change/Show all Resources for a Resource Group and press enter.
      3. Select the resource group for which you want to define a volume group and press Enter. A panel appears with fields for all types of resources.
      4. The Automatically Import Volume Groups flag is set to False by default. Set this flag to True.
      5. In the Volume Groups field, select the volume groups from the picklist or enter volume groups names.

    If you ran the discovery process before starting this procedure, review the list of all existing volume groups cluster-wide (excluding rootvg). The list includes all shared volume groups in the resource group and the volume groups that are currently available for import onto the resource group nodes. HACMP filters out from the cluster-wide list those volume groups that are already included in any of the resource groups.

      6. Press Enter. HACMP determines whether the volume groups that you entered or selected in the Volume Groups field need to be imported to any of the nodes that you defined for the resource group, and proceeds to import them, as needed.

    Final State of Automatically Imported Volume Groups

    When HACMP automatically imports volume groups, the final state (varied on or varied off) depends on the initial state of the volume group and whether the resource group is online or offline when the import occurs.

    In all cases, the volume group will end up varied on after the resource group is started or after the cluster resources are synchronized, even if it is varied off at some point during the import process.

    The following table shows the initial condition of the volume group after its creation, the state of the resource group when the import occurs, and the resulting state of the imported volume group:

    Initial Volume Group State
    Resource Group State
    Auto Imported Volume Group State
    Varied ON
    OFFLINE
    Remains varied ON
    Varied ON
    ONLINE
    Varied ON
    Varied OFF
    OFFLINE
    Varied OFF until the resource group is started
    Varied OFF
    ONLINE
    Varied ON

    Importing a Volume Group Manually

    If you do not want HACMP to import your volume group automatically upon adding it to the resource group, in SMIT make sure that the HACMP Automatically Import Volume Groups flag is set to False (this is the default).

    To set values for the volume group:

      1. Enter smit importvg
      2. Enter field values as follows:
    VOLUME GROUP name
    Enter the name of the volume group that you are importing. Make sure the volume group name is the same name that you used on the source node.
    PHYSICAL VOLUME name
    Enter the name of a physical volume that resides in the volume group. Note that a disk may have a different physical name on different nodes. Make sure that you use the disk name as it is defined on the destination node.
    Volume Group MAJOR NUMBER
    If you are using NFS, use the same major number on all nodes. Use the lvlstmajor command on each node to determine a free major number common to all nodes.

    Changing a Volume Group’s Startup Status

    By default, a volume group that has just been imported is configured to automatically become active at system restart. In an HACMP environment, a volume group should be varied on as appropriate by the cluster event scripts. Therefore, after importing a volume group, use the Change a Volume Group panel to reconfigure the volume group so that it is not activated automatically at system restart.

    To change the volume group’s startup status:

      1. Enter smit chvg
      2. Select the volume group from the list.
      3. Enter these field values as follows:
    Activate volume group Automatically at system restart?
    Set this field to no.
    A QUORUM of disks required to keep the volume group on-line?
    This field is site-dependent. See the chapter on Planning Shared Disk and Tape Devices in the Planning Guide for a discussion of quorum.

    Varying Off Volume Group on Destination Nodes

    Use the varyoffvg command to deactivate the shared volume group so that it can be imported onto another destination node or activated as appropriate by the cluster event scripts. Enter:

    varyoffvg volume_group_name 
    

    Defining LVM Components for Concurrent Access

    Using concurrent access with HACMP requires installing an additional fileset. For more information, see the section Prerequisites in Chapter 4: Installing HACMP on Server Nodes.

    Concurrent access does not support filesystems. Instead, use logical volumes.

    This section describes the procedure for defining shared LVM components for a concurrent access environment. Concurrent access is supported on all devices supported for HACMP.

    Refer to your completed copies of the concurrent volume group worksheets as you define the concurrent LVM components.

    With HACMP 5.1 and up, you can create enhanced concurrent volume groups. Enhanced concurrent volume groups can be used for both concurrent and non-concurrent access.

    Creating a Concurrent Access Volume Group on a Source Node

    Take the following steps on the source and destination nodes in an HACMP cluster to create a volume group that HACMP can vary on in concurrent access mode.

    Step 1. Complete Prerequisite Tasks

    The physical volumes (hdisks) should be installed, configured, and available. You can review the disk status using the lsdev -Cc disk command.

    Step 2. Create a Concurrent Access Volume Group on Source Node

    AIX 5L v.5.2 and up supports enhanced concurrent mode and creates concurrent volume groups in enhanced concurrent mode by default.

    Creating a Concurrent Access Volume Group

    To use a concurrent access volume group, create it as a concurrent capable volume group. A concurrent capable volume group can be activated (varied on) in either non-concurrent mode or concurrent access mode. To define logical volumes on a concurrent capable volume group, it must be varied on in non-concurrent mode.

    To create a concurrent capable volume group from the command line, use the mkvg command. The following example creates an enhanced concurrent volume group on hdisk1 and hdisk2:

    mkvg -n -s 4 -C -y myvg hdisk1 hdisk2  
    

    The mkvg flags do the following:

    -n
    Ensures that the volume group does not vary on at boot time.
    -s
    Specifies the partition size.
    -C
    Creates an enhanced concurrent volume group.
    -y
    Specifies the volume group name.

    You can also use SMIT to run the mkvg command.

    To create a concurrent access volume group:

      1. Enter smit cl_convg
      2. Select Create a Concurrent Volume Group.
    SMIT provides picklists for the node name and physical volume ids (PVIDs) of the disks to be used.
    The Add a Volume Group panel displays. Some fields will already be filled and cannot be changed.
      3. Enter field values as follows:
    VOLUME GROUP name
    Specify name of volume group.
    Physical partition SIZE in megabytes
    Accept the default.
    Volume Group Major Number
    Accept the default.
      4. Press Enter.

    SMIT prompts you to confirm your selections.

      5. Press Enter.

    Step 3. Import Concurrent Capable Volume Group Information on Destination Nodes

    Importing the volume group into the destination nodes synchronizes the definition in the HACMP configuration database for the volume group on each node on which it is imported.

    When you add a volume group to the resource group, you may choose to import a volume group manually onto the destination nodes or you may automatically import it onto all the destination nodes in the resource group.

    Run the discovery process to make picklists available. See the section Collecting Information on Current Volume Group Configuration for information.

    The list of volume groups will include only the concurrent capable volume groups. This list will not include rootvg, and any volume groups already defined to another resource group.

    Final State of Automatically Imported Volume Groups

    When HACMP automatically imports volume groups, the final state (varied on or varied off) depends on the initial state of the volume group and whether the resource group is online or offline when the import occurs.

    In all cases, the volume group will end up varied ON after the resource group is started or after the cluster resources are synchronized, even if it is varied off at some point during the import process.

    This table shows the initial condition of the volume group after its creation, the state of the resource group when the import occurs, and the resulting state of the imported volume group:

    Initial Volume Group State
    Resource Group State
    Auto Imported Volume Group State
    Varied ON
    OFFLINE
    Remains varied ON
    Varied ON
    ONLINE
    Varied ON
    Varied OFF
    OFFLINE
    Varied OFF until the resource group is started
    Varied OFF
    ONLINE
    Varied ON

    Importing a Volume Group Manually

    In SMIT, if you do not want HACMP to import your volume group automatically upon adding it to the resource group, make sure that the Automatically Import Volume Groups flag is set to False (this is the default), and use the importvg fastpath.

    On each destination node, manually import the volume group, using the importvg command, as in the following example:

    importvg -C -y vg_name physical_volume_name 
    

    Specify the name of any disk in the volume group as an argument to the importvg command. By default, AIX 5L automatically varies on non-concurrent capable volume groups when they are imported. AIX 5L does not automatically vary on concurrent capable volume groups when they are imported.

    You can also build the importvg command through SMIT using the procedure that follows.

    To import a concurrent capable volume group using SMIT:

      1. Enter smit importvg
    The Import a Volume Group SMIT panel appears.
      2. Enter field values as follows (for other fields, use the defaults or the appropriate entries for your operation):
    VOLUME GROUP name
    Enter the name of the volume group that you are importing. Make sure the volume group name is the same name that you used on the source node.
    PHYSICAL VOLUME name
    Enter the name of one of the physical volumes that resides in the volume group. Note that a disk may have a different hdisk number on different nodes. Make sure that you use the disk name as it is defined on the destination node. Use the lspv command to map the hdisk number to the PVID. The PVID uniquely identifies a disk.
    ACTIVATE volume group after it is imported?
    Set the field to no.
    Volume Group MAJOR NUMBER
    Accept the default.
    Make this VG concurrent capable
    Accept the default.
    Make default varyon of VG concurrent
    Accept the default.
      3. Press Enter to commit the information.

    If your cluster uses SCSI external disks (including RAID devices) and the import of the volume group fails, check that no reserve exists on any disk in the volume group by executing the following command, only after installing the HACMP software as described in Chapter 4: Installing HACMP on Server Nodes:

    /usr/es/sbin/cluster/events/utils/cl_scdiskreset /dev/hdiskn ... 
    

    For example, if the volume group consists of hdisk1 and hdisk2, enter:

    /usr/es/sbin/cluster/events/utils/cl_scdiskreset /dev/hdisk1 /dev/hdisk2  
    

    Step 4. Vary on the Concurrent Capable Volume Group in Non-Concurrent Mode

    Use the varyonvg command to activate a volume group in non-concurrent mode. To create logical volumes, the volume group must be varied on in non-concurrent access mode. For example, to vary on the concurrent capable volume group myvg in non-concurrent access mode, enter the following command:

    varyonvg myvg  
    

    You can also use SMIT to run the varyonvg command by using the following procedure.

      1. To vary on a concurrent capable volume group in non-concurrent mode, enter smit varyonvg
    The Activate a Volume Group SMIT panel appears.
      2. Enter field values as follows:
    VOLUME GROUP name
    Specify the name of volume group.
    RESYNCHRONIZE stale physical partitions?
    Set this field to no.
    Activate volume group in SYSTEM MANAGEMENT mode?
    Accept the default.
    FORCE activation of the volume group?
    Accept the default.
    Varyon volume group in concurrent mode
    Accept the default. To create logical volumes on the volume group, it must be varied on in non-concurrent mode. (AIX 5L v.5.2 field shown here; default is no.)
      3. Press Enter.

    SMIT prompts you to confirm your selections.

      4. Press Enter.

    Step 5. Create Logical Volumes on Concurrent Capable Volume Group on Source Node

    Create logical volumes on the volume group, specifying logical volume mirrors to provide data redundancy. If the volume group is varied on in concurrent access mode, you will not be able to create logical volumes. A concurrent-capable volume group must be varied on in non-concurrent access mode to create logical volumes on it.

    For more information about creating logical volumes, see the AIX 5L System Management Guide: Operating System and Devices.

    To create logical volumes on concurrent-capable volume group on source node:

      1. Enter smit cl_conlv
      2. Specify the size of the logical volume as the number of logical partitions. Accept default values for all other fields except the following:
    Logical volume name
    Specify name of logical volume. The name must be the same on all cluster nodes.
    PHYSICAL VOLUME names?
    Specify the physical volumes you want the logical volume to include.
    Number of COPIES of each
    logical partition
    Specify 1, 2, or 3 mirror copies. Specifying 1 means no mirroring.
    Mirror Write Consistency
    Set this value to no.
    ENABLE BAD BLOCK relocation
    Set this field to no to disable bad block relocation for concurrent environments (applies to RAID devices).

    Step 6. Vary Off Volume Group on Source Node

    After creating the logical volume, vary off the volume group using the varyoffvg command so that it can be varied on by the HACMP scripts. Enter:

    varyoffvg volume_group_name 
    

    Step 7. Change Volume Group to Remain Dormant at Startup on Destination Nodes

    By default, AIX 5L configures an imported volume group to become active automatically at system restart. In the HACMP system, a volume group should be varied on as appropriate by the HACMP scripts. Therefore, if you did not use the smit cl_ fastpaths to do your configuration, you must reconfigure the volume group so that it remains dormant at startup.

    To change the startup state of a volume group from the command line, enter:

    chvg -a n volume_group_name 
    

    To use SMIT to change the startup state of a volume group:

      1. Enter smit chvg
      2. Set the Activate volume group Automatically at system restart? field to no. For other fields use the defaults or the appropriate entries for your operation.
      3. Press Enter to commit this change.
      4. Exit SMIT.

    Step 8. Complete Follow-up Tasks

    Complete any follow-up tasks as needed. For example, ensure that the HACMP scripts can activate the concurrent capable volume group as a concurrent cluster resource.


    PreviousNextIndex