Copyright © 2000 Rational Software and its subsidiaries. All rights reserved.
November 2000
These Release Notes provide information that is not available in the printed documentation or the Help for Rational TestManager
This release incorporates major changes and additions to the Rational TestManager product.
Rational TestManager has been expanded. It now contains the Test Log window and incorporates the functions that were previously found in Rational LoadTest.
TestManager is now the one place in the Suites to manage all testing activities-planning, design, development, execution, and analysis. TestManager ties together testing with the rest of the development effort, joining your testing assets and tools to provide a single point from which to understand the exact state of your project. TestManager can be purchased as part of any Suite, part of Robot or TeamTest, or as a stand-alone product.
You will find complete installation instructions and system requirements in the Installing Rational Testing Products manual, which is available as a printed book and electronically as a PDF file on the Rational Solutions for Windows Online Documentation CD, which also includes a version of the Acrobat Reader.
If you install Rational Suite Enterprise on German Windows 98, you may see the following error message prior to rebooting:
A system error occurred while copying <CD-ROM Drive>:\coverage98\win98\testfactory.mkr to <Destination Drive>:\coverage98\testfactory.mkr: Zugriff wurde verweigert.
You can ignore this message since the file in question, testfactory.mkr, is not needed to run Rational applications and does not corrupt the install.
The Minimal Install functionality of Rational products is not supported in this release. You must choose the Typical or Custom/Full options to install Rational TestManager. For additional information about installing Rational products, see your product's Installation manual.
Note that the installation manuals may include information on minimal installations. That information is no longer valid.
Evaluation artifacts are not included as a part of this installation procedure. As a result, when you select the Evaluator option on the Software Use page of the Rational Software Setup program, sample data will not be installed.
To obtain the sample data, download the Rational Suite Tutorial from http://www.rational.com/documentation.
Before you install the current Rational testing product(s), you must uninstall any earlier versions of the following products:
Note: Rational LoadTest functionality is now included within Rational TestManager. Previous versions of Rational LoadTest will not work in conjunction with any testing products from the current release and must be uninstalled before installing this release.
A Rational project is an entity that manages the association of Rational Test datastores, RequisitePro projects, ClearQuest databases, and Rose models. Repositories are no longer created by the Administrator or used by any Rational Suite tools.
You can migrate repositories from Rational Suite version 1.0, 1.1, 1.5, 2000, or from any test products marked as 7.x, such as Rational Robot 7.x, to projects. In addition, a special Rational v. 2000 CD is available that converts your 6.x repositories to v. 2000. This converted repository can then be migrated to v. 2001.
The Rational Administrator provides access to many of the features of Rational Unified Change Management (UCM). UCM is the Rational Software approach to managing change to requirements, design models, documentation, components, test cases, source code, and so on.
UCM is unique because it links the activities used to plan and track project progress to the artifacts undergoing change. UCM is based on two key concepts: activity-based Configuration Management (CM) and parallel development.
You can place the information in a Rational project under configuration management using Rational Administrator by associating a Rational project with a UCM project. Use Rational ClearCase to create a UCM project first, and then create a Rational project and UCM-enable it. A version of ClearCase ships with the Rational Suites.
When you create a project, you must also configure it. Configuring a project associates Rational Test datastores, RequisitePro projects, ClearQuest databases, and Rose models with the project and optionally brings them under configuration management.
A Rational Test datastore contains all test assets (test scripts, suites, datapools, logs, reports, test plans, build information, and more) except users and groups, which are themselves contained in a separate datastore.
As part of the UCM support in Rational Suite, both the Rational Test datastore and the RequisitePro project may be UCM-enabled. Once UCM-enabled, they can be integrated with ClearCase so that baselines of the test assets and requirements may be created and then associated with a new Rational Administrator project.
Rational TestManager has been redesigned with a much broader scope. TestManager now contains the Test Log window and incorporates the functions that were previously in Rational LoadTest. For example, from TestManager you create, run, and monitor suites (formerly called schedules).
The major changes in terminology include the following:
The expansion of TestManager more fully integrates the planning process into performance testing. Although you can still design and run a suite (formerly called schedule) directly, you can now also plan your performance tests through TestManager. The procedure for planning and running a test in TestManager is as follows:
The names of the performance testing reports have changed as follows:
The name of the Performance report has not changed.
In Rational Suite PerformanceStudio and Rational LoadTest, workloads were represented as "schedules". With this release of Rational Suite TestManager, "suites" perform the same functions as schedules. In addition, you can:
You can run command line test scripts (including batch files, perl scripts, or executable files created with any programming language) in single or multi-user configurations as part of your tests.
Playback on Netscape Navigator has been enhanced. In order to work with controls that do not present an adequate object model, this new feature uses a form of pattern recognition that retrieves data from rendered pixels.
TestManager supports the DCOM protocol. When you monitor a suite (formerly called a schedule) that includes Visual Basic test scripts, you can view a histogram specifically designed for DCOM.
Note: When you record a DCOM session in Rational Robot, Robot generates Visual Basic test scripts from the session files.
You can now play back test scripts that access SQL Server 2000 and Oracle 8.1. For playing back test scripts against the SQL Server, the DB-LIB and ODBC APIs are now supported.
TestManager now supports test scripts written in Java and Visual Basic. This means that VB and Java test scripts can be played back right from TestManager. Information on writing test scripts for Java and VB is described in the manuals Rational Test Script Services for Java and Rational Test Script Services for Visual Basic. These manuals are available as printed books and electronically as PDF files on the Documentation CD.
In addition, you can run the online documentation through the Windows Start menu - for example:
Start > Programs > Rational ProductName > Rational Test > API > DocumentName
In the example, ProductName is the name of your Rational software product, and DocumentName is either TSS for Java or TSS for Visual Basic.
The folder Rational/Rational Test/rtsdk/Java/Samples contains some Java sample test scripts.
The folder Rational/Rational Test/rtsdk/VB/Samples contains some Visual Basic sample test scripts. However, the sample test scripts for Visual Basic are currently empty.
At this time, TestManager supports the following JDKs for Java script execution:
You can play back (but not record) Java language test scripts against Enterprise Java Beans.
TestManager now includes APIs that allow you to add support for a variety of additional test input sources and for test scripts written in new languages. This flexibility allows you to create and execute tests most appropriate for your application, environment, and resources. Methods for making use of this extensibility are discussed in the manual Rational TestManager Extensibility Reference. This manual is available as a printed book and electronically as a PDF file on the Documentation CD.
In addition, you can run the online documentation through the Windows Start menu - for example:
Start > Programs > Rational ProductName > Rational Test > API > TestManager Extensibility
In the example, ProductName is the name of your Rational software product.
Rational LoadTest functionality is now part of Rational TestManager. TestManager also contains the Test Log window (formerly called LogViewer) and other new testing features. For information on performance testing changes, see the section Rational TestManager on page 6.
In the Test Script Services for Visual Basic documentation, the section "Results of a Compile" in chapter 2 state that when a VB test script is compiled to a DLL, the name of the computer where the test script was recorded is prepended to the DLL name, in the format mmm_sss (machineName_scriptName). In this release, the word MASTER is being prepended to the test script name-the computer name is not being used.
When you run a Java test script from TestManager, or run a suite containing a Java test script, TestManager calls whatever Java compiler you are using to compile the test script.Information on how to set Java compiler options is in the online help but not in the printed documentation.
You can specify whatever compiler options your Java compiler supports. For these, see the documentation for the Java compiler you are using.You specify Java compiler options from the Test Script Type Properties dialog. Navigate to this dialog from the TestManager Tools menu as follows:
Note: If you are using a non-standard Java package (third-party or your own), set up the CLASSPATH environment variable to accommodate the package. This should be done on all Local and Agent computers that run the Java package.
Page 6 of Rational Test Script Services for Java explains how to register test script source folders. Step 4 should read as follows:
This name will be added to the list that is displayed by TestManager when you select File > Run Test Script, File > New Test Script, or File > Open Test Script.
Chapter 1 of the TestManager Extensibility Reference explains how to add a new test script type that uses the Command Line adapter. The following clarifications should be noted in step 3 of the procedure:
When you save a new test script use Save As rather than Save, and Browse to the location you will specify in step 5 of this procedure.
At release time, the revised documents Try It! for Performance Testing of SAP Applications and Try It! for GUI Testing of SAP Applications are unavailable. The documents will be available for download from ftp://ftp.rational.com/exchange/outgoing/SAP_DOCS_v2001/ on or before February 1, 2001. To access this ftp site, type anonymous as the user name, and your email address as the password.
Use of the Rational Administrator under Windows NT and Windows 2000 systems requires system Administrator privileges. If you attempt to connect to a project without write access privileges to the project, Rational Administrator may crash.
In ClearQuest, there is a problem with the Script Properties button on the Test Data tab. As a workaround, you can view script properties in Robot.
The TSSServerStart method starts a Test Script Services proxy server. However, proxy servers are not supported for Visual Basic and Java test script types.
Playback of Visual Basic test scripts is not supported for Windows Me, Windows 98, or Windows 95.
Playback of Java test scripts is not supported for Windows Me, Windows 98, or Windows 95.
After adding new test script types, you may need to exit and restart TestManager to be able to select the new test script types from File > Open Test Script and File > New Test Script menu items.
The user may only rename a Test Plan from the Planning Tab of the Test Asset Workspace. The rename will not be completed if the Test Plan is open.
When working with suites in TestManager, be careful to save a suite before checking it. Accordingly, the "Check suite when saving" option on the Create Suite tab of the Options dialog box should be deselected.
The user can press the F5 key to refresh all in the Test Planning View and in the Test Input View. This key will not work in the Test Asset Workspace. The user will need to right click on a folder in each of the Test Asset Workspace tabs or go to View > Refresh from the main menu to refresh all the Test Asset Workspace.
The Test Script Properties dialog does not reference the Robot session from which the VU test script was generated. To view this information, go to the Session Properties dialog in Rational Robot to look for the test script as a contained script.
Occasionally Monitor menu items may be unavailable. To access monitoring options in this case, use the Monitor toolbar.
Monitoring system resources during test runs is not supported for Windows Me, Windows 98, or Windows 95, Local or Agent.
This means that when running configured test cases in parallel on multiple computers, depending on test case order, all configured test cases man not be run even if computers matching defined configurations are available. To work around this, limit running configured test cases to one computer at a time, and repeat over the list of desired computers in sequential order.
The Show Resource dialog box for data points in the Response vs. Time report refers to the MASTER computer. This should read Local computer.
The following list gives SAP testing limitations in TestManager:
TestManager does not support BLOB (Binary Large OBject) transfer. If you are recording a Sybase or SQL Server session that transfers BLOB data, the connection is ignored and the generated test script does not refer to BLOB transfer activity.
When you monitor a TestManager suite (formerly called a LoadTest schedule), and suspend a virtual tester in an independent transactor, upon restarting the tester will try to catch up on the missed transactions. If you suspend a virtual tester in a coordinated transactor, the remaining testers will pick up the slack and try to maintain the desired transactor rate. If the rate cannot be maintained, then as testers are restarted, they will try to catch up on the missed transactions.
The following list gives Oracle testing limitations in TestManager:
Set the variable in the Environment tab of the System Properties Control Panel.
When playing back a SQL statement that generates multiple result sets, LoadTest logs only the data for the first fetch. The other fetches show the number of rows returned, but will not log the data actually returned.
If a sqlfetch_cursor is run multiple times in a row with ALL_ROWS, and expect rows 0, an error for the second sqlfetch_cursor is generated stating:
ERROR -1: {error} STATE=S1010, CODE=0, MSG=[Microsoft][ODBC Driver
Manager] Function sequence error
When logging data returned from sqlfetch_cursor, sometimes the data is truncated and the next SQL command appears on the same line as a row of data. This is a logging problem only; the test script is executing correctly.
Named pipes are not supported for network recording.
Generated Visual Basic test scripts do not pass script line numbers to TestManager. As a result, the TestManager Script view feature (the Script check box under Tools > Options) does not work with generated Visual Basic test scripts. If you manually add method calls to your Visual Basic and Java test scripts that pass test script line numbers to TestManager, the Script view will work. The relevant methods are TSSMonitor.PositionSet (Visual Basic) and TSSMonitor.positionSet (Java).
For Windows NT only, if you are using a Madge token ring card for network recording, make sure that the Traffic Statistics Gathering option is set to enabled. To enable this option, click Start > Settings > Control Panel, and click the Network icon. Then click the Adapter tab, click the Properties button, and enable the Traffic Statistics Gathering option.
When TestManager creates test scripts that contain PeopleSoft traffic (Tuxedo or Jolt) it automatically correlates any PeopleSoft certificate information required for proper client authentication during playback. The correlations take the form of string variables, declared with the prefix psft_cert, and various operations on them. The following code shows a typical example of certificate correlation in a PeopleSoft/Jolt test script:
1 sock_send
2 jolt_request(0, jolt_sessionid_1, jolt_handlerid_1, 3,
3 jolt_dataxfer(jolt_sessionid_1, JOLT_CALL_RQST,
4 jolt_setatt_name("GetCertificate") +
5 jolt_setatt_version(1) +
6 jolt_setatt_flags(4) +
7 jolt_setatt_data(
8 jolt_setpar_carray(1,
9 "`000000480102030400000000000002bcffffffff0000001f0004`SCTX"
10 "`0a`JavaClient`02`PS`02`PS`00000000000000150008`SCertReq`0"
11 "2`PS`02`PS`00`"))));
12
13 sock_nrecv ["suzy027h"] jolt_response_header();
14 sock_nrecv ["suzy027b"] jolt_response_body();
15 { string psft_cert_1; }
16 psft_cert_1 = "`" + substr(mixed2hexstring(jolt_getpar_carray(1)),
72, 32) +
17 "`";
18 set Think_avg = 230;
19
20 sock_send
21 jolt_request(0, jolt_sessionid_1, jolt_handlerid_1, 4,
22 jolt_dataxfer(jolt_sessionid_1, JOLT_GETDEF_RQST,
22 jolt_setatt_repname("SVC`2f`StmGetTimeOut")));
23
24 sock_nrecv ["suzy028h"] jolt_response_header();
25 sock_nrecv ["suzy028b"] jolt_response_body();
26 set Think_avg = 240;
27
28 sock_send
29 jolt_request(0, jolt_sessionid_1, jolt_handlerid_1, 5,
30 jolt_dataxfer(jolt_sessionid_1, JOLT_CALL_RQST,
31 jolt_setatt_name("StmGetTimeOut") +
32 jolt_setatt_version(1) +
33 jolt_setatt_flags(4) +
34 jolt_setatt_data(
35 jolt_setpar_carray(1,
36 "`0000005a0102030400000000000002bcffffffff0000002f0004`SCTX"
37 "`0a`JavaClient`02`PS`02`PS`10`"+ psft_cert_1 +
38 "`00000000000017000e`SGetTimeOutReq`02`PS"))));\
In lines 15 and 16 a certificate correlation variable, psft_cert_1, is declared and assigned a value based on the contents of a previous Jolt response message. The variable is then properly applied to the contents of a subsequent Jolt request message in line 37.
Because this feature is necessary for proper playback of PeopleSoft test scripts, do not change the contents of the certification correlation variables or change the variable's position within the context of the surrounding message data. Doing so improperly may cause corrupt messages to be sent to the application-under-test, leading to unpredictable results. Should correlation variable manipulation be necessary, however, all automatically scripted variables represent 16 bytes of certificate data, in hexstring form, delimited by single backquotes ("`"), for a total of 34 text characters (16 * 2 + 2). For example, "`664d689999999999c99a9ac9c9c9c9c9`".
To test CORBA applications with TestManager, you must have the following software installed:
IDL (Interface Definition Language) defines an application's interfaces, including method names, parameters, data types, and exceptions. IDL can be used to map to many languages for implementation. For Robot to produce and TestManager to play back test scripts with methods, parameter names, and data described in a meaningful way, you must import the application's IDL into TestManager before test script generation.
To import IDL into TestManager:
The information imported is stored in the repository for use by TestManager at test script generation and play back. You will not be able to play back test scripts until the IDL has been imported into TestManager.
IDL information is stored within the repository based upon the interface name.
If you do not load all the IDL prior to test script generation, some methods may script opaquely. That is, method parameter names will not be listed. Instead, only a single parameter with the name "*" is listed, and all data is asciified hexadecimal.
Sometimes, IDL files will include other IDL files. If include path information is not relative, you may have to specify an IDL include path. You may specify such a path using the IDLINCLUDE environment variable. For example, IDLINCLUDE=d:\myidl;d:\otheridl.
Recording of CORBA applications is similar to recording of other environments. Both Network and API recording methods are supported.
Before recording, you must select IIOP in the Generator Filtering tab of the Virtual User Record Options dialog box in Rational Robot. IIOP traffic between client and server is used to produce VU test scripts.
After recording, a VU test script is produced that describes the interactions between client and server. There are two common IIOP VU emulation commands:
You can limit the scripting of client traffic to a particular computer using the LT7_IIOPCLIENTIP environment variable. For example, LT7_IIOPCLIENTIP=137.162.195.135.
By default, IDL sequences are scripted with a limit of ten members. You can increase this number using the LT7_DEFAULT_SEQUENCE_SIZE environment variable. For example, LT7_DEFAULT_SEQUENCE_SIZE=20.
In each case, the variable is set in the Environment tab of the System Properties dialog box under the Control Panel.
Before playback, all interfaces referenced in a test script must have their IDL imported into TestManager.
Special IIOP monitoring states are provided for the state histogram monitor window.
Playback of ASCII hex (opaque) data between computers of differing byte ordering is not supported. Byte ordering specification is marked in the IIOP protocol. It is possible that data may be transferred with big-endian byte order, even on Windows NT systems. This case is more likely to occur if the application is Java. The result is that data specified opaquely in these cases may not play back correctly, even though the recording computer and the playback computer have the same byte order.
TestManager supports record and playback of the following data types:
The following data types are not supported, and will result in methods being scripted opaquely if they occur as a parameter data type in a method:
Only Solaris is supported for UNIX Agent playback. The Visibroker bind modus is not supported for playback.This limitation means that "Visibroker" cannot be used to locate object references at play back. Therefore, another method, such as reading IORs (Inter-operable Object Reference) from a file, or specifying them explicitly in VU test scripts, must be used to locate objects when playing back test scripts from a Solaris Agent.
When a Java test script named as part of a package is played back on an Agent, a datapool referenced in the DatapoolConfig inner class is not transferred with the script. Playback fails with the message "[22.20.8.2]: TSS datapool initialization failure." To avoid this, add another Java test script at the script source root that references the same datapool in its DatapoolConfig inner class.
In TestManager analysis reports, if you select Test Plans in the Distributed Over list box, and click Values, the icons that are used to represent the test plan root nodes do not appear. In addition, if you generate a test input report, icons are not displayed in the report.
Customized test case properties do not function as intended when used as reporting criteria.
Test cases have three customizable properties, accessed from the Tools menu (Tools > Customize > Test Case). The default names for these properties are Custom1, Custom2, and Custom3.
The user is able to modify the names and values of these properties. The new names and any available values will then appear in the Custom tab of the Test Case Properties dialog. The user is also able to select these properties as Distribute Over or Stack variables when generating reports.
However, the reporting functionality in TestManager does not recognize the new names for the custom properties for sorting test case data. The result is that reports using the renamed custom test case properties as reporting criteria will display no results.
TestManager Logviewer Edition is a special version of TestManager that is distributed with Rational QualityArchitect.
TestManager Logviewer Edition cannot open suite logs generated from TestManager. Attempting to view a TestManager suite log with TestManager Logviewer Edition will cause a program crash. Note that suite logs can be viewed with the standard TestManager product.
Occasionally, the Test Log window has trouble retrieving properties of an event that includes associated data. The General tab of the Event Properties windows displays an "Application Warning" event with an error message as the property. This will be corrected in a forthcoming product update.
TestManager may crash when you attempt to run a suite that has been converted from a LoadTest schedule, when that schedule also contains scenarios. For the time being, Rational recommends that you use the following workaround:
For any schedule that contained scenarios, before ever saving or running the converted Suite, that is, the first time you open it for editing, do the following for each scenario:
If you have questions about installing, using, or maintaining this product, contact Rational Technical Support as follows:
Note: When you contact Rational Technical Support, please be prepared to supply the following information: