You can set up your devices so that the server automatically moves data from one device to another, or one media type to another, based on characteristics such as file size or storage capacity. To do this, you set up different primary storage pools to form a storage pool hierarchy. A typical implementation may have disk storage pointing to tape storage. When a client backs up a file, the server may initially store the file on disk according to the policy for that file. Later, the server may move the file to tape when the disk becomes full. This action by the server is called migration. You can also place a size limit on files that are stored on disk, so that large files are stored initially on tape instead of on disk.
For example, your fastest devices are disks, but you do not have enough space on these devices to store all data that needs to be backed up over the long term. You have tape drives, which are slower to access, but have much greater capacity. You can define a hierarchy so that files are initially stored on the fast disk volumes in one storage pool. This provides clients with quick response to backup requests and some recall requests. As the disk storage pool becomes full, the server migrates, or moves, data to volumes in the tape storage pool.
Migration of files from disk to sequential storage pool volumes is particularly useful because the server migrates all the files for a single node together. This gives you partial collocation for clients. Migration of files is especially helpful if you decide not to enable collocation for sequential storage pools. See Keeping a Client's Files Together: Collocation for details.
You can set up a storage pool hierarchy when you configure devices by using the Device Configuration utility. You can also go back to this utility to change the storage hierarchy.
You establish a hierarchy by identifying the next storage pool, sometimes called the subordinate storage pool. The server migrates data to the next storage pool if the original storage pool is full or unavailable. See Migration of Files in a Storage Pool Hierarchy for detailed information on how migration between storage pools works.
Restrictions:
When you are adding clients, you can set up a storage pool hierarchy. You can also use the Storage Pool Hierarchy wizard in the Server Utilities.
You can also rearrange the hierarchy by using commands. The following shows one example.
For this example, suppose that you have determined that an engineering department requires a separate storage hierarchy. You set up policy so that the server initially stores backed up files for this department to a disk storage pool. When that pool fills, you want the server to migrate files to a tape storage pool. You want the pools to have the following characteristics:
You can define the storage pools in a storage pool hierarchy from the top down or from the bottom up. Defining the hierarchy from the bottom up requires fewer steps. To define the hierarchy from the bottom up, perform the following steps:
define stgpool backtape tape description='tape storage pool for engineering backups' maxsize=nolimit collocate=yes maxscratch=100
define stgpool engback1 disk description='disk storage pool for engineering backups' maxsize=5M nextstgpool=backtape highmig=85 lowmig=40
If you have already defined the storage pool at the top of the hierarchy, you can update the storage hierarchy to include a new storage pool. You can update the storage pool by using the UPDATE STGPOOL command or by using the Server Utilities, which includes a wizard. The wizard allows you to change your storage pool hierarchy by using a drag and drop interface.
To use the Storage Pool Hierarchy wizard in the Utilities, do the following:
You can also rearrange the hierarchy by using commands. The following shows one example.
For example, suppose that you had already defined the ENGBACK1 disk storage pool. Now you have decided to set up a tape storage pool to which files from ENGBACK1 can migrate. Perform the following steps to define the new tape storage pool and update the hierarchy:
define stgpool backtape tape description='tape storage pool for engineering backups' maxsize=nolimit collocate=yes maxscratch=100
update stgpool engback1 nextstgpool=backtape
When a user backs up or archives files from a client node, the server may group multiple client files into an aggregate (a single physical file). The size of the aggregate depends on the sizes of the client files being stored, and the number of bytes and files allowed for a single transaction. Two options, one in the server options file and one in the client options file, affect the number of bytes and files allowed for a single transaction:
This option sets a target size for the aggregate file. An aggregate file will usually be smaller than the value specified by the TXNBYTELIMIT option. A logical file (a single user's file) that is larger than the value specified by TXNBYTELIMIT option will not become part of an aggregate, but will be stored as a single physical file.
The recommended value is 25600.
Together these options allow you to control the size of aggregate files stored by the server. For more information on using options to tune performance, look for the performance tuning guide on the product Web site ( http://www.tivoli.com/support/storage_mgr/tivolimain.html ).
When a Tivoli Space Manager client (HSM client) migrates files to the server, the files are not grouped into an aggregate.
When a user backs up, archives, or migrates a file from a client node, the server looks at the management class that is bound to the file. The management class specifies the destination, the storage pool in which to store the file. The server then checks that storage pool to determine the following:
Using these factors, the server determines if the file can be written to that storage pool or the next storage pool in the hierarchy.
ADSM Version 2 Clients: When an ADSM Version 2 client backs up or archives files, the server must estimate the size of the aggregate file that the client will send. The server bases the estimate on earlier transactions with the client. The server uses the estimated size to check whether the storage pool has enough space to store the file. Because the server uses the estimated size rather than the actual size for ADSM Version 2 clients, the server may not always store files in the storage pool that you expect.
As an example of how the server stores files in a storage hierarchy, assume a company has a storage pool hierarchy as shown in Figure 14.
Figure 14. Storage Hierarchy Example
The storage pool hierarchy consists of two storage pools:
Assume a user wants to archive a 5MB file that is named FileX. FileX is bound to a management class that contains an archive copy group whose storage destination is DISKPOOL, see Figure 14.
When the user archives the file, the server determines where to store the file based on the following process:
The maximum file size applies to the physical file being stored, which may be a single client file or an aggregate file. The maximum file size allowed in DISKPOOL is 3MB. FileX is a 5MB file and therefore cannot be stored in DISKPOOL.
If the DISKPOOL storage pool has no maximum file size specified, the server checks for enough space in the pool to store the physical file. If there is not enough space for the physical file, the server uses the next storage pool in the storage hierarchy to store the file.
Copy storage pools enable you to back up your primary storage pools for an additional level of data protection for clients. See Backing Up Storage Pools for details. Copy storage pools are not part of a storage hierarchy.
For efficiency, it is strongly recommended that you use one copy storage pool to back up all primary storage pools that are linked to form a storage hierarchy. By backing up all primary storage pools to one copy storage pool, you do not need to recopy a file when the file migrates from its original primary storage pool to another primary storage pool in the storage hierarchy.
In most cases, a single copy storage pool can be used for backup of all primary storage pools. The number of copy storage pools you need depends on whether you have more than one primary storage pool hierarchy and on what type of disaster recovery protection you want to implement.
Multiple copy storage pools may be needed to handle particular situations, including:
A common way to use the storage hierarchy is to initially store client data on disk, then let the server migrate the data to tape. Typically you would need to ensure that you have enough disk storage to handle one night's worth of the clients' incremental backups. While not always possible, this guideline proves to be valuable when considering storage pool backups.
For example, if you have enough disk space for nightly incremental backups for clients and have tape devices, you can set up the following pools:
You can then schedule the following steps every night:
Backing up disk storage pools before migration processing allows you to copy as many files as possible while they are still on disk. This saves mount requests while performing your storage pool backups.
When this migration completes, raise the high migration threshold back to 100%.
The tape primary storage pool must still be backed up to catch any files that might have been missed in the backup of the disk storage pools (for example, large files that went directly to sequential media).
See Estimating Space Needs for Storage Pools for more information about storage pool space.