Rational TestManager
Release Notes
Release 2002.05.20
Copyright © 2002 Rational Software and its subsidiaries. All rights reserved.
June 2002About These Release Notes
- Trademarks and Copyrights
- Crystal Reports Information
Getting Started
- Installation Information and System Requirements
- Installation Issues
- Installation Directories
- Message About Required Internet Explorer (IE) Upgrade
- Evaluation Artifacts
- Installation of Rational Testing Products on Non-English Versions of Operating Systems
- Upgrading from Earlier Rational Testing Products Releases
- Uninstalling the Nutcracker Operating Environment Before Upgrade
- Upgrading Test Datastores
- Upgrading Reports
- Launching Agents on Windows XP or Windows NT with Auto-Login
- Netscape Support
- Creating the Rational Test Datastore
New and Changed Features
- General Enhancements
- New Testing Tool: RobotJ
- Test Impact Analysis
- Enhanced Filtering of Test Input Sources
- New Built-in Test Input Type: Microsoft Excel
- Defining Test Script Options
- New Test Scripts View
- Defining Test Tool Options
- Support for Red Hat Linux Version 7.1 Agents
- New Actions in the Test Script Console and Test Input Adapters
- Test Asset Management and Reporting Enhancements
- Importing and Exporting Test Assets
- Changes to the Test Case Reports
- Performance Testing Enhancements
- New Reports
- Improved License Allocation of Virtual Testers
Late Changes to Documentation
- Description of GetPath Function
- Saving and Copying Test Case Reports
Guidelines and Restrictions
- Using Excel Test Inputs
- Submitting Defects in Test Log
- Updating Suspicion from TestManager
- Embedded Quotation Marks in Datapool Columns
- Test Script Playback
- General
- VU
- Command Line
- Manual
- Visual Basic
- Generated Visual Basic and Java Test Scripts
- Supported Agents
- Solaris Test Agent Installation
- F5 (Refresh All) Does Not Work in Test Asset Workspace
- Resource Monitoring
- Report Generation
- SAP Limitations
- SQL Anywhere Server
- Rational Administrator
- Unified Change Management (UCM)
- UCM on Windows NT with ClearCase 5.0
- Associating a Rose Test Input Source with a Project
- Editing Reports with Long Path Names
- Performance Testing
- Response vs. Time Reports
- Suspended Users in Transactors
- ODBC Limitations
- Testing CORBA Applications
- Rational TestManager Help
Contacting Rational Technical Support
About These Release Notes
This release incorporates several enhancements to the Rational TestManager product. These Release Notes provide information that is not available in the printed documentation or the Help for Rational TestManager.
Trademarks and Copyrights
A complete online trademark and copyright information page for all Rational products resides in the /doc subfolder of your installation folder. The file is called copyright.html, and its default location is C:/Program Files/Rational/doc.
Crystal Reports Information
As of May 1, 2002, licenses for Rational Products no longer include a copy and/or license of Crystal Reports. Licenses of Rational Products acquired prior to May 1, 2002 that included a copy and/or license of Crystal Reports are not affected.
Any reference to Crystal Reports in the Rational Product documentation applies only to copies and/or licenses of Crystal Reports you have rightfully licensed.
Please contact Crystal Decisions, Inc. for information concerning purchasing Crystal Reports Professional Version 8.0 for use with Rational Products. You can purchase Crystal Reports Professional Version 8.0 from the Crystal Decisions, Inc. web site at www.crystaldecisions.com or by contacting them directly:
Crystal Decisions, Inc.
Vancouver, BC
Canada
www.crystaldecisions.comGetting Started
Installation Information and System Requirements
You will find complete installation instructions and system requirements in Rational Testing Products Installation Guide, which is available as a printed book and electronically as a PDF file on the Rational Solutions for Windows Online Documentation CD, which also includes a version of the Acrobat Reader.
Installation Issues
Installation Directories
Do not install Rational testing products to the root directory of a drive.
Do not install to a directory path that contains non-ASCII or double-byte characters.
Message About Required Internet Explorer (IE) Upgrade
You may see a message that indicates that you must upgrade to IE 5.0 Service Pack 2 before you can install the Rational testing products. This message is incorrect. The testing products support IE versions 5.0 and 4.0. Click on Ignore, and you will not be prevented from installing.
Evaluation Artifacts
Evaluation artifacts are not included as a part of this installation procedure. As a result, when you select the Evaluator option on the Software Use page of the Rational Software Setup program, sample data will not be installed.
To obtain the sample data, download the Rational Suite Tutorial from http://www.rational.com/documentation.
Installation of Rational Testing Products on Non-English Versions of Operating Systems
When installing Rational testing products on non-English versions of operating systems, do not install to a directory path that contains non-ASCII characters. The directory path should contain ASCII characters only.
Upgrading from Earlier Rational Testing Products Releases
Uninstalling the Nutcracker Operating Environment Before Upgrade
If you are upgrading to this release (2002.05) from a previous release of any Rational testing product, you must remove the Nutcracker operating environment and restart your computer before installing the current version.
To remove the Nutcracker executable:
Note: If you are uninstalling a version of Nutcracker installed with a version of Rational Testing products prior to release 2002.05, select Nutcracker, and click Remove.
- Click Start > Settings > Control Panel.
- Click Add/Remove Programs.
- Select MKS Platform Components 7.x, and click Remove.
- Click OK.
- Restart your computer before installing the current version of the Rational testing product.
Upgrading Test Datastores
Datastores created in Rational Test Releases 2001A.04 or 2001.03 are not compatible with the latest version, 2002.05. Before TestManager or Robot Versions 2002.5 can connect to and operate against the test assets contained in the datastores, you must first use Rational Administrator to upgrade them as follows:
In the Administrator, select File > Upgrade Project Assets.
Select the .RSP file to upgrade the datastores associated with the project.
The program displays a progress dialog. When the process is complete, you can view the log generated during the upgrade.
After the datastore has been upgraded, it cannot be used with earlier versions of TestManager or Robot.
Refer to the Rational Suite Administrator's Guide for more information on creating and upgrading datastores.
Refer to the Rational TestManager User's Guide for more information on new features associated with the upgrade.
Upgrading Reports
If you upgrade from a former release to release 2002.05.20, reports that you created in TestManager in the former release appear in release 2002.05.20. However, when you run the old reports, you may encounter problems completing the run.
Rational provides these reports after an upgrade mostly for use as reference in editing. Use the report editor to copy and paste elements between reports.
Launching Agents on Windows XP or Windows NT with Auto-Login
When auto-logging on to a Windows XP or Windows NT system, you may see a Nutcraker service error message. If this occurs, do not use auto-login when launching the system, and try waiting at the login screen for several extra seconds, thus allowing the Nutcraker service to finish starting.
Netscape Support
When you execute a manual test script using a Netscape Navigator browser, the latest versions of Netscape available from Netscape.com for Windows and Sun Microsystems for UNIX are not supported. However, Netscape version 4.7 is supported.
Creating the Rational Test Datastore
When creating the Rational test datastore from Rational Administrator, do not specify a path containing non-ASCII characters. Also, do not use non-ASCII characters when naming any projects or log folders. If your test datastore path contains non-ASCII characters, you will experience problems of varying severity, including abnormal test script termination, datapool access anomalies, and problems with verification points.
New and Changed Features
For details on new features, click Help > Contents and Index > What's New in Rational TestManager 2002.05.00.
General Enhancements
New Testing Tool: RobotJ
Rational RobotJ is an object-oriented automated testing tool that enables you to test Java and HTML applications running on the Windows NT, Windows 2000, Windows XP, and Linux platforms. Use RobotJ to:
- Perform full functional testing. Record and play back scripts that navigate through your application and test the state of objects through verification points.
- Create and edit simple and easy to read object-oriented test scripts. In addition to automatically recording test scripts, RobotJ contains wizards for generating code; for example, for automatically creating a verification point. RobotJ's test scripts are implemented as Java programs.
- Perform cross-browser record and play back of Web applications.
Note: Information about RobotJ is in a separate release notes file. The RobotJ release notes contain important information for use of this new product, including information about the installation, the RobotJ documentation, and known technical issues.
- Uses ScriptAssure object recognition technology to provide robust and flexible scripts.
- Creates an object map that lets you update object properties once and propagate the updates to all scripts that use that map.
- Includes integration with Rational ClearCase.
To access the RobotJ Release Notes:
RobotJ and TestManager
When you associate a RobotJ datastore with a Rational project, you can:
- Create and play back scripts, including RobotJ scripts, from Rational TestManager. Use the TestManager Agent to play back scripts on a remote computer.
- View scripts from the Test Scripts window of Rational TestManager.
You can use the Test Log window of TestManager to:
- Open a test log to view a result.
- Filter the data of a test log to view only the information you need.
- Open the test script that generates a log event in the appropriate test script development tool. For example, if you create a RobotJ script and run it from TestManager, when you click on a log event, Rational RobotJ opens and displays the script.
- Analyze the results of a test log using the Verification Point Comparator. If you use Rational RobotJ to record scripts, you can analyze the results in the Verification Point Comparator to determine why a verification point may have failed.
- Submit a defect for a failed test case or for a failed log event. The test log automatically fills in build, configuration, and script information in the Rational ClearQuest defect form.
- Preview or print data displayed in the active test log in the Test Log window.
Test Impact Analysis
If a test input changes, TestManager now informs you of the impact of that change on your test plan. For example, if a requirement changes in some meaningful way, you need to know which test cases in your test plans are affected so that you can update them to reflect requirement changes. TestManager automatically identifies and marks the affected test cases as "suspect" for ReqPro and Excel Test Input sources.
Enhanced Filtering of Test Input Sources
The Test Input Adapter Extensibility feature of TestManager now supports the ability to save test input source filters and Test Case Coverage report filters. Test input sources and Test Case Coverage reports are unfiltered by default. When you set a filter for a test input source, the last filter that you set persists in the Test Inputs window and Test Input Select dialog box. When you set a filter for a Test Input source, TestManager saves the filter with the test input source. When you set a filter for a Test Case Coverage report, TestManager saves the filter with the report.
New Built-in Test Input Type: Microsoft Excel
You can now use one or more Excel spreadsheets as test input sources. This test input adapter can be useful in many ways. Many teams use Excel to manage information in their projects. For example, some teams may document requirements or capture project planning information in an Excel spreadsheet. In either case, the new Excel test input adapter allows teams to track the testing effort with respect to their information in an Excel spreadsheet. Specifically, they could associate test cases with tasks defined in rows of a spreadsheet so that a coverage report could be generated to show progress or completion of those tasks.
Defining Test Script Options
You can now define additional test script options controlling how test script execution adapters execute test scripts.
New Test Scripts View
You can now view all your test scripts sorted by type from the Test Scripts View. From this window you can perform test management operations on test scripts. You can also perform basic version control operations (check-in, checkout, etc.) on registered test scripts that support these operations.
Defining Test Tool Options
You can now set test tool options to control how a particular test tool plays back a test script. You can define a test tool option in the test script execution adapter that you have written for a particular test tool.
Support for Red Hat Linux Version 7.1 Agents
You can now play back VU, Java, Command Line, and custom test scripts on Red Hat Linux version 7.1 agents, as well as on version 6.x agents supported in earlier versions of TestManager.
New Actions in the Test Script Console and Test Input Adapters
Test script console adapter and test input adapter developers can now add their own custom actions, or commands. You can run these custom actions or commands in TestManager.
Test Asset Management and Reporting Enhancements
Importing and Exporting Test Assets
You can move or copy existing test assets between and within Rational projects, use third party tools to create importable versions of test assets, or use third party tools to analyze exported test assets. Importing and exporting Test Assets are subject to the following restrictions:
- Logs, log details, and test case results are not supported by TestManager's import functionality.
- TestManager does not support importing and exporting queries.
- Test Suite implementations are not imported or exported. Test Suite reference names can be imported or exported provided the reference name exists in the import destination Test Datastore.
Changes to the Test Case Reports
You can now run test case reports that display test case suspicion status as presented by the test impact analysis capability. Using the test case suspicion feature you can track meaningful changes to test inputs so that you can update the associated test cases or their implementations.
Also, TestManager now has several new pre-defined test case coverage reports.
Performance Testing Enhancements
New Reports
There are two new performance testing reports in TestManager: Command Data and Command Trace. Command Data reports let you identify the differences between expected and actual responses in a test run. They show the commands sent and the response data that is written to the test log. The Command Trace report lists the activity of a suite run and enables you to examine in detail unusual or unexpected events. This report formats raw data from the suite run without performing statistical analysis.
Improved License Allocation of Virtual Testers
You can now combine multiple non-pooled virtual tester packs. Thus, if you had a 100 virtual tester pack and a 500 virtual tester pack, you could execute a single run of as many as 600 virtual testers.
Late Changes to Documentation
Description of GetPath Function
The documentation for the GetPath function incorrectly implies that it returns the actual storage location where executing test scripts and attached log files are stored. In fact, the service returns the root path of the source. The exact storage location is relative to the root path.
Descriptions for this service are in several manuals:
The manuals will be corrected in the next release.
Saving and Copying Test Case Reports
The Rational TestManager User's Guide states that, "After you run a test case report, you can print, save or copy it to the Clipboard." This is not true for Tree Style test case reports.
Tree Style test case reports can be printed only; they cannot be saved or copied. The statement remains true for all other test case reports
Guidelines and Restrictions
Using Excel Test Inputs
In TestManager you can view test inputs that have been defined in Microsoft Excel spreadsheets. If, while viewing those test inputs, you or another user creates a new test input or modifies an existing test input in Excel, you must refresh the TestManager test input view to see those changes in TestManager.
If you are changing the "Date Modified" column in an Excel spreadsheet and the TestManager datastore and spreadsheet are in different time zones, the "Update Suspicion" feature does not work properly.
Submitting Defects in Test Log
If after submitting a defect in the Test Log Window, you want to log into a different datastore or project, or log in as a different user, TestManager may not display the login dialog after restart. In this case you must either log off the operating system and log back in (if you are running on Windows NT or later), or bring up Task Manager, and terminate the CQINTSRV11.EXE process. You can then log in to another datastore or project, or log in as a different user.
Updating Suspicion from TestManager
Requisite Pro projects that use a Microsoft Access database may experience a delay between the entry of the data and the display of the modified data. This delay is caused by a defect in the Microsoft MDAC component. A similar delay is apparent from TestManager when Requisite Pro test inputs are involved. The most noticeable case is when suspicion is updated and cleared frequently in the Test Input View.
Embedded Quotation Marks in Datapool Columns
The TSSDatapool.DatapoolValue (Visual Basic) and TSSDatapool.datapoolValue() (Java) methods take a datapool column argument. If the value of the column specified by this argument contains double quotes (for example, "Name"), you must manually edit the datapool column and add a second pair of double quotes around the value (""Name"").
Test Script Playback
General
- Messages logged with the VU log_msg() function, or with an analogous TSS call, are truncated at 30 kb.
VU
- Playback of VU test scripts is not supported for Windows Me or Windows 98, whether the system is a Local computer or an Agent computer.
- To play back Oracle 7 with Oracle 7 libraries on HP-UX, you must set the following environment variable before invoking TestManager:
RT_MASTER_USE_STATE_FOR_SQLCommand Line
- When running command line test scripts or custom test script types that use the command line adapter on Windows NT systems as Agents, TestManager may report test script failure even though the test scripts executed successfully. To avoid this, install Service Pack 6a for Windows NT 4 on the Agent.
- If you write a test script on a Windows computer in an interpreted scripting language such as Perl or Python and execute that test script on a UNIX agent, the test script might fail with an error message such as sh: filename: not found even though the file does exist and has the correct permissions. This problem is caused by the carriage return - line feed sequence that DOS text files use to mark line endings. The Notepad editor creates files of this type. To avoid this problem, use an editor that will save text files in the UNIX text file format that marks line endings with a line feed character.
Manual
When playing back manual test scripts from TestManager, be sure to exit Rational ManualTest to terminate the script. The alternative, terminating the user from the TestManager console, is not necessary and may lead to unexpected results.
Visual Basic
Test Script Services (TSS) support test script playback from a proxy TSS server. However, TestManager does not provide Visual Basic bindings to those proxy TSS servers.
Generated Visual Basic and Java Test Scripts
Visual Basic and Java test scripts that are automatically generated (by the Robot Session Recorder or by QualityArchitect) do not record script line numbers. As a result, the TestManager Script view feature (the Script check box under Tools > Options) does not work with these test scripts. If you manually add method calls to your Visual Basic and Java test scripts that pass test script line numbers to TestManager, the Script view will work. The relevant methods are TSSMonitor.PositionSet (Visual Basic) and TSSMonitor.positionSet (Java).
Supported Agents
The following table lists supported agent operating systems and indicates which built-in test script types can be played back on each.
Important note: Effective with this release, Windows 95 is no longer a supported platform for the Rational Test Agent.
The Dynix operating system version of the Rational Test Agent is no longer delivered or supported as of this release. Users of this test agent should not upgrade to this release.
Solaris Test Agent Installation
Before running RTsagt to start the Solaris agent, check the content of the LD_LIBRARY_PATH variable (echo $LD_LIBRARY_PATH) and make sure that /usr/openwin/lib is included. If it is not, then add it using the following sequence:
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/openwin/lib export LD_LIBRARY_PATHF5 (Refresh All) Does Not Work in Test Asset Workspace
You can press the F5 key to refresh all in the Test Plan window and in the Test Input window. This key does not work in the Test Asset Workspace. You must right-click a folder in each Test Asset Workspace tab or click View > Refresh Workspace from the main menu to refresh all the Test Asset Workspace.
Resource Monitoring
- Monitoring system resources during test runs is not supported for Windows Me or Windows 98, Local or Agent.
- To monitor resources on an Agent computer that is running AIX, you must install the bos.acct package on that Agent if it is not already installed.
Report Generation
TestManager provides a "previous" button allowing you to change the test log selection after you have selected the virtual tester, script, and commandID filters to be used in generating a report. This causes a problem because the previous test log is not closed and conflicts with the new test log. Workaround: if you need to change the test log selection after you have selected filters, start over rather than use the "previous" button.
SAP Limitations
The following list gives SAP testing limitations in TestManager:
- UNIX Agents are not supported for the SAP protocol.
- The personalizing option (the multicolored circle in the upper right of SAPGUI) is not supported.
- Multiple SAP sessions with a single client are not supported.
- Only the active screen is rendered, while SAPGUI sometimes renders inactive screens.
- The control box in table controls is not supported.
- Transaction IQ03 is not supported.
- Automatic logon load balancing, as performed by SAPLOGON.EXE and the SAP Session Manager, is not supported.
- The Manager's Desktop functionality under the "Human Resources" menu is not supported.
SQL Anywhere Server
During the installation of a SQL Anywhere Server, select Replace these files with English equivalents independent of the language environment in which you are working. Microsoft recommends this because selected files must be replaced with their English equivalents.
Rational Administrator
Unified Change Management (UCM)
If you use Unified Change Management (UCM) with Rational Suite under these conditions:
- You use Rational ClearQuest to manage UCM activities, and
- You place a RequisitePro project or a Rational Test datastore under configuration control
Make sure that you do not set the UCM policy, "Check Assignment Before Work On." (The default is for the policy not to be set.)
If you set the policy, you cannot work on an activity unless you are the owner of that activity. However, in the current implementation of UCM for RequisitePro and Rational Test, an entire team of workers share the same activity. Therefore, if the policy is set, only one person, not the entire team, can work in a RequisitePro project or in a Rational Test datastore.
If the policy is mistakenly set, the error message "Source control could not set activity." appears. To work around this error:
- Start the Rational Administrator and connect to a Rational project.
- Right-click the UCM project and click Properties.
- On the ClearQuest tab, unset the policy.
For further details, see the Rational Suite Administrator's Guide.
UCM on Windows NT with ClearCase 5.0
A possible problem has been identified on Windows NT 4.0 when ClearCase 5.0 is installed. After performing a UCM datastore operation from the Rational Administrator (such as create a datastore, check-in all, add to or remove from source control), we suggest you shut down and restart Rational Administrator before attempting another such operation.
Associating a Rose Test Input Source with a Project
The Rational Administrator includes a Configure Project dialog box. When associating a Rational Rose model with a Rational project, it is not automatically defined as a test input source for TestManager.
To register a Rational Rose model as a test input, use the Manage Test Input Types dialog box from TestManager. To reach this dialog, click Tools > Manage > Test Input Types.
Editing Reports with Long Path Names
Crystal Reports is used to generate or edit Listing reports. You can run a report regardless of the length of its pathname. However, if you attempt to edit a report whose pathname is longer than 80 characters, the message "Dictionary not found" appears. This is a Crystal Reports restriction. If you create a new or customize an existing design layout, keep the root path to the test datastore as short as possible (less than 80 characters).
Performance Testing
The issues described in this section are specific to performance testing.
Response vs. Time Reports
The Show Resource dialog box for data points in the Response vs. Time report for a Performance Testing Report, refers to the MASTER computer. This should read Local computer.
Suspended Users in Transactors
When you monitor a TestManager suite and suspend a virtual tester in an independent transactor, upon restarting, the tester will try to catch up on the missed transactions. If you suspend a virtual tester in a coordinated transactor, the remaining testers will compensate by attempting to maintain the desired transactor rate. If the rate cannot be maintained, as testers are restarted, they will try to catch up on the missed transactions.
ODBC Limitations
When playing back a SQL statement that generates multiple result sets, TestManager logs only the data for the first fetch. The other fetches show the number of rows returned, but do not log the data actually returned.
If a sqlfetch_cursor is run multiple times in succession with ALL_ROWS and EXPECT_ROWS 0, an error for the second sqlfetch_cursor is generated stating:
ERROR -1: {error} STATE=S1010, CODE=0, MSG=[Microsoft][ODBC Driver Manager] Function sequence errorWhen logging data returned from sqlfetch_cursor, sometimes the data is truncated and the next SQL command appears on the same line as a row of data. This is a logging problem only; the test script is executing correctly.
Testing CORBA Applications
To test CORBA applications with TestManager, you must have the following software installed:
- Windows 2000 or Windows NT Version 4.0 with at least Service Pack 4
- VisiBroker (C++ or Java) Version 3.3.
CORBA Playback
Before playback, you must import into the project the IDL for each interface referenced in a test script.
TestManager provides special IIOP monitoring states in the state histogram monitor window.
TestManager does not support playback of ASCII hex (opaque) data between computers of differing byte ordering. Byte ordering specification is marked in the IIOP protocol. It is possible that data may be transferred with big-endian byte order, even on Windows NT systems. This case is more likely to occur if the application is Java. The result is that data specified opaquely in these cases may not play back correctly, even though the recording computer and the playback computer have the same byte order.
Supported and Unsupported CORBA Data Types for VU Test Scripts
TestManager supports playing back the following CORBA data types:
- char
- octet
- short
- unsigned short
- long
- unsigned long
- float
- double
- boolean
- enum
- struct
- union
- array
- sequence
The following data types are not supported, and will result in methods being scripted opaquely if they occur as a parameter data type in a method:
UNIX Agents
Only Solaris is supported for UNIX Agent playback of VU language CORBA/IIOP scripts recorded by Rational Robot. The VisiBroker bind modus is not supported for playback. This limitation means that "VisiBroker" cannot be used to locate object references at play back. Therefore, another method, such as reading IORs (Inter-operable Object Reference) from a file, or specifying them explicitly in VU test scripts, must be used to locate objects when playing back test scripts from a Solaris Agent.
Rational TestManager Help
A paragraph in the Help topic, What's New in Rational TestManager 2002.05.00, contains a typographical error. A corrected version of the paragraph follows.
Enhanced filtering - The Test Input Adapter Extensibility feature of TestManager now supports the ability to save test input source filters and Test Case Coverage report filters. When you set a filter for a test input source, the last filter that you set affects test inputs that appear in the Test Inputs window and Test Input Select dialog box until you change the filter. When you set a filter for a Test Input source, TestManager saves the filter with the test input source. When you set a filter for a Test Case Coverage report, TestManager saves the filter with the report.
Contacting Rational Technical Support
If you have questions about installing, using, or maintaining this product, contact Rational Technical Support as follows:
Note: When you contact Rational Technical Support, please be prepared to supply the following information:
|
Rational Software Corporation http://www.rational.com sales@rational.com info@rational.com |