Banner Home Previous Next Index Help



Using Intelligent Calculation to Optimize Calculation


This chapter provides information on how to use Intelligent Calculation to optimize the performance of Hyperion Essbase calculations. For additional information on optimizing overall database calculations, see the following:

This chapter includes the following sections:


Go to top Introducing Intelligent Calculation

When you do a full calculation of a database, Hyperion Essbase tracks which data blocks it has calculated. If you then load a small subset of data, on subsequent calculations, you can choose to calculate only those data blocks that Hyperion Essbase has not calculated or that require recalculating. In Hyperion Essbase, this process is called Intelligent Calculation.


Go to top Benefits of Intelligent Calculation

Intelligent Calculation is designed to provide significant calculation performance benefits for a full calculation of a database (CALC ALL), or when you use a calc script that calculates all members in one CALC DIM command. For more information, see Intelligent Calculation and Data Block Status. If you cannot use Intelligent Calculation for all of a database calculation, you may be able to use it for part of the calculation.

For example, consider a case in which you calculate a database by doing a default consolidation and then an allocation of data. To significantly improve your calculation performance in this case, turn on Intelligent Calculation for the default consolidation and then turn off Intelligent Calculation for the allocation.

Assuming that Intelligent Calculation is turned on (the default), use a calc script to accomplish the following:

  1. Do a CALC ALL of a database with Intelligent Calculation turned on.
  2. Use the SET UPDATECALC command to turn off Intelligent Calculation for the duration of the calc script.
  3. Allocate Headquarters Costs across all products.

By default, Intelligent Calculation is turned on. You can change this default setting in the ESSBASE.CFG file. You can also turn Intelligent Calculation on or off in a calc script. For more information, see Turning Intelligent Calculation On and Off. For more information on the ESSBASE.CFG file, see the online Technical Reference in the DOCS directory.


Go to top Intelligent Calculation and Data Block Status

To provide Intelligent Calculation, Hyperion Essbase uses the status of the data blocks in a database. Data blocks have a calculation status of either clean or dirty. Hyperion Essbase marks a data block as clean after certain calculations.

When Intelligent Calculation is turned on, Hyperion Essbase calculates only dirty blocks and their dependent parents. Turning off Intelligent Calculation means that Hyperion Essbase calculates all data blocks, regardless of whether they are marked as clean or dirty.

Hyperion Essbase marks data blocks as clean in the following calculations:

When a calc script causes Hyperion Essbase to do two or more calculation passes through a database, you need to consider carefully the effects of Intelligent Calculation. For more information, see Using the SET CLEARUPDATESTATUS Command. For information on calculation passes, see Defining the Calculation Order.

Hyperion Essbase does not mark calculated data blocks as clean in any calculations other than the situations described above, unless you use the SET CLEARUPDATESTATUS command in a calc script. For more information, see Using the SET CLEARUPDATESTATUS Command.

Hyperion Essbase marks a data block as dirty in the following situations:

Intelligent Calculation works on a data block level and not on a cell level. For example, if you load a data value into one cell in a data block, the whole data block is marked as dirty.

Maintaining Clean and Dirty Status

If you want to use Intelligent Calculation when calculating a subset of a database or when performing multiple calculation passes through a database, consider carefully the implications of how Hyperion Essbase marks data blocks as clean. When using Intelligent Calculation, carefully maintain the clean and dirty status of the data blocks to ensure that Hyperion Essbase recalculates the database as efficiently as possible.

For example, when you calculate a subset of a database, the newly calculated data blocks are not marked as clean by default. You can ensure that the newly calculated blocks are marked as clean by using the SET CLEARUPDATESTATUS AFTER command in a calc script. To ensure accurate calculation results, you need to consider the information in Using the SET CLEARUPDATESTATUS Command and the online Technical Reference in the DOCS directory.


Go to top Limitations of Intelligent Calculation

Consider the following limitations when using Intelligent Calculation:


Go to top Using Intelligent Calculation

This section provides information on turning Intelligent Calculation on and off and on using Intelligent Calculation with different types of calculations.


Go to top Turning Intelligent Calculation On and Off

By default, Intelligent Calculation is turned on. You can change the default by using the UPDATECALC setting in the ESSBASE.CFG file.

You can turn Intelligent Calculation on and off for the duration of a calc script by using the SET UPDATECALC command in a calc script. Turning on Intelligent Calculation means that Hyperion Essbase calculates only dirty blocks and their dependent parents. Turning off Intelligent Calculation means that Hyperion Essbase calculates all data blocks, regardless of whether they are marked as clean or dirty.

For more information on these commands and on ESSBASE.CFG, see the online Technical Reference in the DOCS directory.


Go to top Using Intelligent Calculation for a Default, Full Calculation

Intelligent Calculation provides significant performance benefits when you do a full calculation (CALC ALL) of a database. If you do a full calculation of a database, leave Intelligent Calculation turned on (the default) to take advantage of the performance benefits it provides.

Unless you have changed the default, a full calculation (CALC ALL) is the default calculation for a database. You can check the default calculation setting by selecting Database > Set Default Calc in Hyperion Essbase Application Manager.

CAUTION: When using Intelligent Calculation, note the information in Limitations of Intelligent Calculation.

Calculating for the First Time

When you do a full calculation of a database for the first time, Hyperion Essbase calculates every existing block. The performance is the same whether you have Intelligent Calculation turned on or off.

Recalculating

When you do a full recalculation of a database with Intelligent Calculation turned on, Hyperion Essbase checks each block to see if it is marked as clean or dirty. Checking the data blocks has a 5% to 10% performance overhead. For more information on clean and dirty, see Intelligent Calculation and Data Block Status.

During most recalculations, this small performance overhead is insignificant when compared to the performance gained by turning on Intelligent Calculation.

However, if you recalculate a database in which more than approximately 80% of the values have changed, the overhead of Intelligent Calculation may outweigh the benefits. In this case, you can turn off Intelligent Calculation.


Go to top Using Intelligent Calculation for a Partial, Calc Script Calculation

Hyperion Essbase marks a data block as clean when it calculates the data block on a full calculation (CALC ALL) or when it calculates all dimensions in one CALC DIM command. For more information, see Intelligent Calculation and Data Block Status.

In any other calculations, Hyperion Essbase does not mark calculated data blocks as clean, unless you use the SET CLEARUPDATESTATUS command in a calc script. For example, if you calculate a subset of a database or calculate a database in two calculation passes, Hyperion Essbase does not mark the calculated blocks as clean, unless you use the SET CLEARUPDATESTATUS command.

The following calc scripts do not cause Hyperion Essbase to mark the calculated data blocks as clean:

FIX("New York")
CALC DIM(Product, Measures);
ENDFIX
CALC DIM(Measures, Product);
CALC DIM(Market, Year, Scenario);

Go to top Using the SET CLEARUPDATESTATUS Command

In some cases, Hyperion Essbase does not mark calculated blocks as clean; for example, if you calculate a subset of a database or calculate a database in two calculation passes. To manually mark data blocks as clean for purposes of Intelligent Calculation, use the SET CLEARUPDATESTATUS command in a calc script. For more information, see Intelligent Calculation and Data Block Status.

The SET CLEARUPDATESTATUS command has three parameters: AFTER > ONLY > OFF.


Go to top Considerations for Using SET CLEARUPDATESTATUS

When you use the SET CLEARUPDATESTATUS command to mark calculated data blocks as clean, it is imperative that you consider carefully the following questions:

Which data blocks are calculated?

Only calculated data blocks are marked as clean. For more information, see Calculating Data Blocks.

Do concurrent calculations affect the same data blocks?

Do not use the SET CLEARUPDATESTATUS AFTER command with concurrent calculations unless you are certain that the concurrent calculations do not need to calculate the same data block or blocks. If concurrent calculations attempt to calculate the same data blocks, with Intelligent Calculation turned on, Hyperion Essbase may not recalculate the data blocks because the blocks are already marked as clean. For more information, see Handling Concurrent Calculations.

Does Hyperion Essbase recalculate the same data blocks on a second calculation pass through a database?

When Hyperion Essbase calculates data blocks on a first calculation pass through a database, it marks the data blocks as clean. If you then attempt to calculate the same data blocks on a subsequent pass with Intelligent Calculation turned on, Hyperion Essbase does not recalculate the data blocks because they are already marked as clean.


Go to top Examples Using SET CLEARUPDATESTATUS

The following examples are based on the Sample Basic database. Assume the following:

Example 1

SET CLEARUPDATESTATUS AFTER;
FIX("New York")
CALC DIM(Product);
ENDFIX

In this example, Hyperion Essbase searches for dirty parent data blocks for New York (for example New York->Colas, in which Colas is a parent member). It calculates these dirty blocks and marks them as clean. (The calculation is based on the Product dimension.) Hyperion Essbase does not mark the level 0 data blocks as clean because they are not calculated. For information on level 0 blocks, see Defining the Calculation Order.

Example 2

SET CLEARUPDATESTATUS ONLY;
FIX("New York")
CALC DIM(Product);
ENDFIX

This example differs from the first example in that the SET CLEARUPDATESTATUS ONLY command is used instead of SET CLEARUPDATESTATUS AFTER. Hyperion Essbase searches for dirty parent data blocks for New York (for example New York->Colas, in which Colas is a parent member on the Product dimension). It marks them as clean. It does not calculate the data blocks. It does not mark the level 0 data blocks as clean because they are not calculated. For example, if New York->100-10 is dirty, it remains dirty.

Example 3

SET CLEARUPDATESTATUS OFF;
CALC ALL;
CALC TWOPASS;
SET CLEARUPDATESTATUS ONLY;
CALC ALL;

In this example, Hyperion Essbase first calculates all the dirty data blocks in the database. The calculated data blocks remain dirty. Hyperion Essbase does not mark them as clean. Hyperion Essbase then calculates those members tagged as Two-Pass on the dimension tagged as accounts. Because the data blocks are still marked as dirty, Hyperion Essbase recalculates them. Again, it does not mark the calculated data blocks as clean. Hyperion Essbase then searches for all the dirty blocks in the database and marks them as clean. It does not calculate the blocks, even though a CALC ALL command is used.


Go to top Calculating Data Blocks

Hyperion Essbase creates a data block for each unique combination of sparse dimension members, provided that at least one data value exists for the combination. Each data block represents all the dense dimension member values for that unique combination of sparse dimension members. For example, in the Sample Basic database, the Market and Product dimensions are sparse. The data block New York->Colas represents all the member values on the Year, Measures, and Scenario dimensions for the sparse combination New York->Colas.

The following information assumes that you are familiar with the concepts of upper level, level 0, and input data blocks. For more information on how Hyperion Essbase creates data blocks, see Defining the Calculation Order.


Go to top Calculating a Dense Dimension

When you calculate a dense dimension and do not use a FIX command, Hyperion Essbase calculates at least some of the data values in every data block in a database. For example, the following calc script is based on the Sample Basic database:

SET CLEARUPDATESTATUS AFTER;
CALC DIM(Year);

This script calculates the Year dimension, which is a dense dimension. Because Year is dense, every data block in the database includes members of the Year dimension. Therefore, Hyperion Essbase calculates data values in every data block. Because the script uses the SET CLEARUPDATESTATUS AFTER command, Hyperion Essbase marks all the data blocks as clean.


Go to top Calculating a Sparse Dimension

When you calculate a sparse dimension, Hyperion Essbase may not need to calculate every data block in a database. For example, the following calc script is based on the Sample Basic database:

SET CLEARUPDATESTATUS AFTER;
CALC DIM(Product);

This script calculates the Product dimension, which is a sparse dimension. Because Product is sparse, separate data blocks exist for each member on the Product dimension. For example, one data block exists for New York->Colas and another for New York->100-10. The data block New York->100-10 is a level 0 block, which means that it does not represent a parent member on either sparse dimension. The data values for New York->100-10 are input values; they are loaded into the database. Therefore, Hyperion Essbase does not need to calculate this data block. It does not mark it as clean, even though the script uses the SET CLEARUPDATESTATUS AFTER command.

The upper level data block New York->Colas represents Colas, which is a parent level member on the Product dimension. Hyperion Essbase needs to calculate values for Colas, so Hyperion Essbase calculates this data block. Because the script uses the SET CLEARUPDATESTATUS AFTER command, Hyperion Essbase marks this data block as clean.

When Hyperion Essbase calculates a sparse dimension, it does not calculate the level 0 data blocks (unless a formula is applied to one of the sparse level 0 members). Because it does not calculate these level 0 data blocks, they are not marked as clean, even when you use the SET CLEARUPDATESTATUS AFTER command.

When Hyperion Essbase calculates a sparse dimension, it recalculates an upper level data block if the block is dependent on one or more child blocks that are dirty.

If you load data into a database, the level 0 data blocks into which you load data are marked as dirty. If you subsequently calculate only a sparse dimension or dimensions, these level 0 blocks remain dirty, because Hyperion Essbase does not calculate them. Therefore, when you recalculate only a sparse dimension or dimensions, Hyperion Essbase recalculates all the upper level data blocks because the upper level blocks are marked as dirty if their child blocks are dirty, even though the upper level blocks were originally clean.

You can avoid unnecessary calculation by ensuring that you calculate at least one dense dimension. When you calculate a dense dimension and do not use the FIX command, data values are calculated in every data block, including the level 0 blocks. So the level 0 blocks are marked as clean.


Go to top Handling Concurrent Calculations

If concurrent calculations attempt to calculate the same data blocks and Intelligent Calculation is turned on, Hyperion Essbase may not recalculate the data blocks because they are already marked as clean.

Do not use the SET CLEARUPDATESTATUS AFTER command with concurrent calculations unless you are certain that the concurrent calculations do not calculate the same data block or blocks.

Consider the following example, which is based on the Sample Basic database. Actual and Budget are members of the dense Scenario dimension. Because Scenario is dense, each data block in the database contains both Actual and Budget values.

SET CLEARUPDATESTATUS AFTER;
FIX("New York", Actual)
CALC DIM(Product, Year);
ENDFIX

If User One runs the above calc script, Hyperion Essbase calculates the Actual values for all data blocks that represent New York. Hyperion Essbase marks the calculated data blocks as clean, even though not all the data values in each calculated block have been calculated. For example, the Budget values have not yet been calculated.

SET CLEARUPDATESTATUS AFTER;
FIX("New York", Budget)
CALC DIM(Product, Year);
ENDFIX

If User Two runs the previous calc script to calculate the Budget values for New York, Hyperion Essbase does not recalculate the specified data blocks, because they are already marked as clean. The calculation results are not correct.

One way to solve this problem is to make the Scenario dimension sparse; then the Actual and Budget values are in different data blocks, for example, New York->Colas->Actual and New York->Colas->Budget. In this case, the second calc script correctly calculates the data blocks.


Go to top Handling Multiple-Pass Calculations

Whenever possible, Hyperion Essbase calculates a database in one calculation pass through the database. For information on calculation passes, see Defining the Calculation Order.

When you use a calc script to calculate a database, the number of calculation passes Hyperion Essbase performs depends upon the calc script. For more information, see Intelligent Calculation and Data Block Status, and for more information on grouping formulas and calculations, see Developing Calc Scripts.

Consider a situation in which you calculate data blocks on a first calculation pass through a database and Hyperion Essbase marks them as clean. If you then attempt to calculate the same data blocks on a subsequent pass with Intelligent Calculation turned on, Hyperion Essbase does not recalculate the data blocks because they are already marked as clean.

The following examples describe situations in which you obtain incorrect calculation results. The examples provide solutions for these situations. The examples are based on the Sample Basic database and assume that Intelligent Calculation is turned on.

Example 1

Consider the following calc script:

CALC ALL;
CALC TWOPASS;

Hyperion Essbase calculates the dirty data blocks in the database and marks all the data blocks as clean. Hyperion Essbase then needs to recalculate those members tagged as Two-Pass in the dimension tagged as accounts. However, Hyperion Essbase does not recalculate the specified data blocks because they are already marked as clean. The calculation results are not correct.

You can calculate the correct results by turning off Intelligent Calculation for the Two-Pass calculation.

Example 2

This calc script calculates data values for New York. The calculation is based on the Product dimension:

SET CLEARUPDATESTATUS AFTER;
FIX("New York")
CALC DIM(Product);
ENDFIX
CALC TWOPASS;

Hyperion Essbase performs the following processes:

  1. Because the SET CLEARUPDATESTATUS AFTER command is used, Hyperion Essbase marks the calculated blocks as clean, even though only part of the database is being calculated.
  2. Hyperion Essbase cycles through the database calculating the dirty data blocks that represent New York. The calculation is based on the Product dimension. Thus, Hyperion Essbase calculates only those blocks that represent a parent member on the Product dimension (for example, New York->Colas, New York->Root Beer, and New York->Fruit Soda). Hyperion Essbase calculates only the aggregations and formulas for the Product dimension. Hyperion Essbase marks the calculated data blocks as clean, even though all the data values in each calculated block have not been calculated.
  3. Hyperion Essbase needs to recalculate those members tagged as Two-Pass in the dimension tagged as accounts. However, some of these data blocks are already marked as clean from the calculation in step 2. Hyperion Essbase does not recalculate the data blocks that are already marked as clean. The calculation results are not correct.

You can calculate the correct results by turning off Intelligent Calculation for the Two-Pass calculation. For detailed information on using Two-Pass calculations, see Defining the Calculation Order.

Example 3

This calc script bases the database calculation on the Product and Year dimensions. Because two CALC DIM commands are used, Hyperion Essbase does two calculation passes through the database:

SET CLEARUPDATESTATUS AFTER;
CALC DIM(Product);
CALC DIM(Year);

Hyperion Essbase performs the following processes:

  1. Because the SET CLEARUPDATESTATUS AFTER command is used, Hyperion Essbase marks the calculated blocks as clean, even though only part of the database is being calculated.
  2. Hyperion Essbase cycles through the database calculating the dirty data blocks. The calculation is based on the Product dimension, as in Example 2. Hyperion Essbase marks the calculated data blocks as clean, even though all the data values in each calculated block have not been calculated.
  3. Hyperion Essbase needs to recalculate the data blocks. The recalculation is based on the Year dimension. However, as a result of the calculation in step 2, some of the data blocks are already marked as clean. Hyperion Essbase does not recalculate the data blocks that are already marked as clean. The calculation results are not correct.

You can calculate the correct results by using one CALC DIM command to calculate both the Product and Year dimensions. Hyperion Essbase then calculates both dimensions in one calculation pass through the database. The following calc script calculates the correct results:

SET CLEARUPDATESTATUS AFTER;
CALC DIM(Product, Year);
Note:   When you calculate several dimensions in one CALC DIM command, Hyperion Essbase calculates the dimensions in the default calculation order and not in the order in which you list them in the command. For more information, see Defining the Calculation Order.

Example 4

Consider another example, which calculates data values for New York but calculates based on two different dimensions. The first calc script calculates the Product dimension:

SET CLEARUPDATESTATUS AFTER;
FIX("New York")
CALC DIM(Product);
ENDFIX

Hyperion Essbase calculates the data blocks that include New York. The calculation is based on the Product dimension. Thus, Hyperion Essbase calculates only the dirty blocks that include a parent member on the Product dimension (for example, New York->Colas, New York->Root Beer, and New York->Fruit Soda). It calculates only the aggregations and formulas for the Product dimension. Hyperion Essbase marks the calculated data blocks as clean, even though all the data values in each calculated block have not been calculated.

The second calc script calculates the Year dimension:

SET CLEARUPDATESTATUS AFTER;
FIX("New York") CALC DIM(Year); ENDFIX

Hyperion Essbase calculates the data blocks that represent New York. The calculation is based on the Year dimension. Year is a dense dimension, which means that Hyperion Essbase needs to calculate all data blocks that include New York. Hyperion Essbase calculates only the aggregations and formulas for the Year dimension.

However, as a result of the previous calculation, some of the data blocks for New York are already marked as clean. Hyperion Essbase does not recalculate these data blocks because they are already marked as clean. The calculation results are not correct.

You can calculate the correct results by telling Hyperion Essbase not to mark the calculated data blocks as clean. The following calc script calculates the correct results:

SET CLEARUPDATESTATUS OFF;
FIX("New York")
CALC DIM(Product); ENDFIX SET CLEARUPDATESTATUS AFTER; FIX("New York") CALC DIM(Year); ENDFIX

The SET CLEARUPDATESTATUS OFF command tells Hyperion Essbase to calculate the dirty data blocks, but not to mark them as clean. The SET CLEARUPDATESTATUS AFTER command tells Hyperion Essbase to mark the data blocks as clean.

This solution assumes that the data blocks are not already marked as clean from a previous partial calculation of the database.

You can ensure that all data blocks are calculated, irrespective of their clean or dirty status, by turning off Intelligent Calculation. The following calc script calculates all the specified data blocks, irrespective of their clean or dirty status:

SET UPDATECALC OFF;
FIX("New York")
CALC DIM(Year, Product);
ENDFIX

Because you have not used the SET CLEARUPDATESTATUS AFTER command, Hyperion Essbase does not mark the calculated data blocks as clean.


Go to top Considering the Effects of Intelligent Calculation

Using Intelligent Calculation may have implications for the way you administer a database. This section discusses the implications of each of the following:


Go to top Changing a Formula or Accounts Property

Changing a formula in the database outline, or changing an accounts property in the database outline, does not cause Hyperion Essbase to restructure the database. Thus, the data blocks affected by the change are not marked as dirty, and when you subsequently run a default calculation with Intelligent Calculation turned on, your changes are not calculated. For example, if you change a time balance tag in the dimension tagged as accounts, Hyperion Essbase does not restructure the database and does not mark the affected blocks as dirty.

You must recalculate the appropriate data blocks. You can accomplish this by using a calc script to do any of the following:

For more detailed information, see Developing Calc Scripts.


Go to top Using Relationship and Financial Functions

If you use relationship functions (for example, @PRIOR or @NEXT) or financial functions (for example, @NPV or @INTEREST) in a formula on a sparse dimension, Hyperion Essbase always recalculates the data block that contains the formula, even if the data block is marked as clean.

For more information on relationship functions and financial functions, see the online Technical Reference in the DOCS directory.


Go to top Restructuring a Database

When you restructure a database (for example, by adding a member to a dense dimension), all data blocks potentially need recalculating. Therefore, Hyperion Essbase marks all data blocks as dirty. When you calculate the restructured database, all blocks are calculated.

Note:   Changing a formula in the database outline or changing an accounts property in the database outline does not cause Hyperion Essbase to restructure the database. You must recalculate the appropriate data blocks. For more information, see Changing a Formula or Accounts Property.

Go to top Copying and Clearing Data

When you copy values to a data block by using the DATACOPY command, the resulting data block is marked as dirty. Hyperion Essbase calculates the block when you recalculate a database.

When you clear data values by using the CLEARDATA and CLEARBLOCK commands, Hyperion Essbase clears all the blocks regardless of whether they are marked as clean or dirty.

For more information on these commands, see the online Technical Reference in the DOCS directory.


Go to top Converting Currencies

When you convert currencies using the CCONV command, the resulting data blocks are marked as dirty. Hyperion Essbase calculates all the converted blocks when you recalculate a database.

For more information on this command, see the online Technical Reference in the DOCS directory.


Home Previous Next Index Help Banner


Copyright © 1991-2000 Hyperion Solutions Corporation. All rights reserved.