Analyzing execution results

When you execute a test, the execution details are saved in an Execution Result. You can view the results immediately, or you can later access the results through a test execution record's history.

Overview

The top section of the Execution Result displays the details of the test run:

  • Actual Result
  • Host Name
  • Owner
  • Test Milestone
  • Test Case
  • Test Script
  • Test Data
  • Weight
Some of these fields are editable, depending on the type of script that is run. For manual and automated tests, you can only edit the Actual Result field in this section. If you execute a test without a script, all fields are editable until you save the result. After you save the result, only the Actual Result field is editable. If you execute a test without a script, you must first save the result in order to open the associated test case.
Note: You must have permission to edit fields in the Execution Result. For more information on permissions, see Permissions and roles

Rational Quality Manager uses a default set of test execution states. The Actual Result field displays one of the following execution states:

  • Passed (Pass)
  • Incomplete (Incomplete)
  • Inconclusive (Inconclusive)
  • Partially Blocked (Partially blocked)
  • Failed (Fail)
  • Error (Error)
  • Blocked (Blocked)
  • PermFailed (PermFailed)
  • Deferred (Deferred)
To see standard definitions of test execution states, go to Test execution states.
Tip: Execution states can be removed or renamed by the administrator in System Properties > Execution States.

Weight Distribution

Weight is a measure of execution effort and can be based on tester hours, units of work, or priority. For example, a test case that requires twice as much time to execute could have double the weight as another test case. When running reports to measure progress, weight value is taken into account when determining status.

You can use the sliders in this section to manually edit weight distributions for execution outcomes. For instance, even if a test fails, you might want to track that 80% of the test actually passed. If the test case has a weight of 100 points, you record that 80 points passed, while 20 points failed.

The following formulas are used to calculate the combined weight distributions for tests:

  • Pass + Fail + Inconclusive <= Total Weight
  • Pass + Blocked <= Total Weight
  • Attempted >= Pass + Fail + Inconclusive

Test Environment

This section lists the software and hardware platforms specified in the test execution record. You can link to the test environment directly from the result.

Defects

This section displays the defects that were opened during execution. You can also create new defects (Add new defect), add existing defects (Add existing defect), or change the blocking status of defects (Change blocking status). The defects associated with the Execution Result are listed in the Related Defects panel on the right side of the page.

Results Details

For a manual test, this section outlines the steps executed in a test run, along with its result. You can use this information to see at what step a test failed or a defect was reported. You can open the defect by clicking the Defect (Defect) icon.

For other script types, you can click the link in this section to view the report in HTML format.

If you run a test without a script, this section will display execution details in a rich-text editor. You can edit these detail if you have the appropriate permissions.


Feedback