Tutorial: Creating a custom test execution environment


Objectives:

To create a new Test Execution type using the Eclipse Test and Performance Tools Platform (TPTP).

Time required

1 hour

Before you begin

Before you start this tutorial, you need to

  1. Install Eclipse and the Eclipse Test and Performance Tools Platform (TPTP).
  2. Have a basic understanding of Eclipse Plug-in Development
  3. Configure and run the Agent Controller which corresponds to your TPTP version. For more information, see Getting started with the Agent Controller.
  4. Review the Test Execution Environment documentation.


Description

In this tutorial, you create a new test execution type building on top of the TPTP Test framework.

This tutorial guides you through the following procedures:

  1. Create the Plug-in Projects
  2. Define the Test Type
  3. Define the New Test Execution Launch Type
  4. Define the New Test Execution Adapter and Runner
  5. Implement the New Test Execution Adapter and Runner
  6. Export the New Test Type and update the Agent Controller classpath

Create the Plug-in Projects


In this section, you will create the framework for the new Test Execution plug-ins.
  1. Create a new Plug-in Project for the core plug-in containing two separate source folders, one for code run in the workbench and the other for code deployed to the runner agent of the Agent Controller.
    • From the File menu choose File > New > Project.... The New Project dialog appears.
    • In the Wizards list, select Plug-in Project and click Next. The Create a New Plug-in project wizard appears.
    • Type org.eclipse.tptp.test.execution.example.core for the Project name and click Next.
    • Make sure the option This plug-in will make contributions to the UI is not checked.
    • Click Finish. The org.eclipse.tptp.test.execution.example.core plug-in project appears in the Package Explorer view.
    • Open the Dependencies tab of the Manifest file in the Plug-in Manifest Editor. Add the following Required Plug-ins and save the changes:
      • org.eclipse.hyades.execution
      • org.eclipse.hyades.test.core
      • org.eclipse.hyades.test.tools.core
      • org.eclipse.tptp.platform.models
    • Select the plug-in in the Package Explorer view. From the File menu choose File > New > Source Folder. Enter runner for the Folder name and click Finish.
  2. Create a new Plug-in Project for the UI plug-in and a separate folder for the icons.
    • From the File menu choose File > New > Project.... The New Project dialog appears.
    • In the Wizards list, select Plug-in Project and click Next. The Create a New Plug-in project wizard appears.
    • Type org.eclipse.tptp.test.execution.example.ui for the Project name and click Next.
    • Make sure the option This plug-in will make contributions to the UI is checked.
    • Click Finish. The org.eclipse.tptp.test.execution.example.ui plug-in project appears in the Package Explorer view.
    • Open the Dependencies tab of the Manifest file in the Plug-in Manifest Editor. Add the following Required Plug-ins and save the changes:
      • org.eclipse.hyades.ui
      • org.eclipse.hyades.test.ui
      • org.eclipse.hyades.test.core
    • Select the plug-in in the Package Explorer view. From the File menu choose File > New > Folder. Enter icons for the Folder name and click Finish.
    • Open the Build tab of the Manifest file in the Plug-in Manifest Editor. Add icons to the Binary Build by selecting the entry in the list.

Define the Test Type


In this section, you will add support for the new test type by extending the org.eclipse.hyades.ui.typeDescriptions extension point. After adding support for the new test type, wizards, editors, launch configurations and executions can be defined.
  1. Add the extension point for the test suite in the UI.
    • From the org.eclipse.tptp.test.execution.example.ui plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Click Add, to add a new extension. In the New Extension dialog, enter org.eclipse.hyades.ui.typeDescriptions in the filter, select the extension point and click Finish.
    • Select the typeDescription node of the org.eclipse.hyades.ui.typeDescriptions extension and update the following values:
      • type: org.eclipse.tptp.test.execution.example.testSuite
      • extension: testSuite
      • name: TST_SUITE_EXECUTION_EXAMPLE_NAME
      • icon: icons/tsuite.gif
      • description: TST_SUITE_EXECUTION_EXAMPLE_DESC
    • Save the Manifest file.
  2. Add the extension point for the new test suite wizard.
    • From the org.eclipse.tptp.test.execution.example.ui plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Click Add, to add a new extension. In the New Extension dialog, enter org.eclipse.ui.newWizards in the filter, select the extension point and click Finish.
    • Right click the org.eclipse.ui.newWizards extension and select New > wizard. Select the resulting name (wizard) node of the extension and update the following values:
      • id: ExecutionExampleTestSuiteNewWizard
      • name: TPTP Execution Example Test Suite
      • class: org.eclipse.tptp.test.execution.example.ui.wizard.ExecutionExampleTestSuiteNewWizard
      • icon: icons/new_executionExamplesuite.gif
      • category: org.eclipse.tptp.test.ui.wizards.new/testSuite
      Right click the TPTP Execution Example Test Suite wizard node and select New > description. Select the resulting description node and add Create a new TPTP Execution Example Test Suite as the body text.
    • Save the Manifest file.
  3. Define the new test suite wizard by implementing the wizard class defined in the extension and extending TestSuiteNewWizard in the package org.eclipse.tptp.test.execution.example.ui.wizard
    • From the org.eclipse.tptp.test.execution.example.ui plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Select the TPTP Execution Example Test Suite (wizard) extension entry. Click on the class link to open the New Java Class wizard.
    • In the Java Class wizard, browse to the superclass org.eclipse.hyades.test.ui.wizard.TestSuiteNewWizard2, remove any interface entries, and deselect adding constructors from the superclass. Click Finish on the wizard.
    • Update the overridden getType() method so it looks like the following:
      @Override
      protected String getType() {
      	return "testExecutionExampleType";
      }
      		
    • From the Run menu, choose Run > Run Configurations.... Select Eclipse Application and create a new launch configuration. Under the Plug-ins tab, ensure the workspace plug-ins are included in the launch. Click Run.
    • In the launched workbench, from the File menu choose File > New > Other. Expand Other and select TPTP Execution Example Test Suite. Follow the prompts to complete creating the new test suite. When you are done, close the Eclipse Application launch configuration.

Define the New Test Execution Launch Type


This section will add a new launch configuration type. The org.eclipse.hyades.test.core.launchconfigLaunchableType can be used to create new launch configurations, adding new tabs to the launch configuration and any additional behaviors prior to the invocation of the test execution harness.
  1. From the org.eclipse.tptp.test.execution.example.ui plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Click Add, to add a new extension. In the New Extension dialog, enter org.eclipse.hyades.test.core.launchconfigLaunchableType in the filter, select the extension point and click Finish.
  2. Select the resulting launchableType node of the extension and update the following values:
    • type: testExecutionExampleType
    • kind: test
  3. Expand the launchableType node, select the supportedMode child node and update the following values:
    • mode: run
    • shortcutAvailable: true
  4. Optionally, create a launch type for debug mode.
    Right click the launchableType node and select New > supportedMode. Select the newly created node and update the following values:
    • mode: debug
    • shortcutAvailable: true
  5. Save the Manifest file.
  6. Re-launch the Eclipse Application launch configuration. Select the test suite created from the test suite definition section, from the Run menu, choose Run > Run Configurations.... Select Test and create a new launch configuration. In the Test tab, ensure that your test suite appears. Close the dialog and Eclipse Application launch configuration.

Define the New Test Execution Adapter and Runner


In this section the execution component and adapter are registered for the new test type by re-using the TPTP execution components to create an adapter and runner for the new test type. See the Execution Environment documentation for more details.
  1. Extend org.eclipse.hyades.test.core.RegisteredExecutionComponentImpl in org.eclipse.tptp.test.execution.example.core and register the execution components for re-use.
    1. From the org.eclipse.tptp.test.execution.example.core plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Click Add, to add a new extension. In the New Extension dialog, enter org.eclipse.hyades.test.core.RegisteredExecutionComponentImpl in the filter, select the extension point and click Finish.
    2. Select the resulting org.eclipse.hyades.test.core.RegisteredExecutionComponentImpl node of the extension and update the following values:
      • ID: RegisteredExecutionComponentImpl.JAVA
      • Name: JAVA
    3. Select the child node, RegisteredExecutionComponentImpl of the extension and update the following values:
      • id: ExecutionEnvironmentImpl
      • type: ENVIRONMENT
      • implClass: org.eclipse.hyades.execution.core.impl.ExecutionEnvironmentImpl
      • stubClass: org.eclipse.hyades.execution.local.ExecutionEnvironmentStub
      • skeletonClass: org.eclipse.hyades.execution.remote.ExecutionEnvironmentSkeleton
      • name: JAVA_ENVIRONMENT
      Save the Manifest file. Right click the JAVA_ENVIRONMENT(RegisteredExecutionComponentImpl) node and select New > Supported Test Type. Select the resulting SupportedTestType node and update the name value with testExecutionExampleType.
    4. Select the org.eclipse.hyades.test.core.RegisteredExecutionComponentImpl node of the extension, right click and choose New > RegisteredExecutionComponentImpl. Update the resulting child node with the following values:
      • id: ProcessExecutorImpl
      • type: EXECUTOR
      • implClass: org.eclipse.hyades.execution.core.impl.ProcessExecutorImpl
      • stubClass: org.eclipse.hyades.execution.harness.TestExecutionHarnessExecutorStub
      • skeletonClass: org.eclipse.hyades.execution.remote.JavaProcessExecutorSkeleton
      • name: JAVA_EXECUTOR
      Save the Manifest file. Right click the JAVA_EXECUTOR(RegisteredExecutionComponentImpl) node and select New > Supported Test Type. Select the resulting SupportedTestType node and update the name value with testExecutionExampleType.
    5. Select the org.eclipse.hyades.test.core.RegisteredExecutionComponentImpl node of the extension, right click and choose New > RegisteredExecutionComponentImpl. Update the resulting child node with the following values:
      • id: JavaTaskRemoteHyadesComponentImpl
      • type: AGENT
      • implClass: org.eclipse.hyades.execution.core.impl.JavaTaskRemoteHyadesComponentImpl
      • stubClass: org.eclipse.hyades.execution.local.JavaTaskRemoteHyadesComponentStub
      • skeletonClass: org.eclipse.hyades.execution.remote.JavaTaskRemoteHyadesComponentSkeleton
      • name: JAVA_AGENT
      Save the Manifest file. Right click the JAVA_AGENT(RegisteredExecutionComponentImpl) node and select New > Supported Test Type. Select the resulting SupportedTestType node and update the name value with testExecutionExampleType.
    6. Select the org.eclipse.hyades.test.core.RegisteredExecutionComponentImpl node of the extension, right click and choose New > RegisteredExecutionComponentImpl. Update the resulting child node with the following values:
      • id: JavaProcessExecutableObjectImpl
      • type: EXECUTABLEOBJECT
      • implClass: org.eclipse.hyades.execution.core.impl.JavaProcessExecutableObjectImpl
      • stubClass: org.eclipse.hyades.execution.local.JavaProcessExecutableObjectStub
      • skeletonClass: org.eclipse.hyades.execution.remote.JavaProcessExecutableObjectSkeleton
      • name: JAVA_EXECUTABLEOBJECT
      Save the Manifest file. Right click the JAVA_AGENT(RegisteredExecutionComponentImpl) node and select New > Supported Test Type. Select the resulting SupportedTestType node and update the name value with testExecutionExampleType.
    7. Save the Manifest file.
  2. Register the adapters to implement for the new Test type.
    • From the org.eclipse.tptp.test.execution.example.core plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Click Add, to add a new extension. In the New Extension dialog, enter org.eclipse.hyades.test.core.ExecutableObjectAdapter in the filter, select the extension point and click Finish.
    • Select the resulting org.eclipse.hyades.test.core.ExecutableObjectAdapter node of the extension and update the following values:
      • ID: RegisteredExecutionComponentImpl.ExecExample
      • Name: Test Execution Example
    • Select the child node, ExecutableObjectAdapter of the extension and update the following values:
      • id: TestExecExampleExecutableObjectAdapter
      • name: TEST_EXEC_EXAMPLE_EXECUTABLE_OBJECT_ADAPTER
      • class: org.eclipse.tptp.test.exec.example.execution.execExampleExecutableObjectAdapter
      Save the Manifest file. Right click the TEST_EXEC_EXAMPLE_EXECUTABLE_OBJECT_ADAPTER(ExecutableObjectAdapter) node and select New > Supported Test Type. Select the resulting SupportedTestType node and update the name value with testExecutionExampleType.
    • From the org.eclipse.tptp.test.execution.example.core plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Click Add, to add a new extension. In the New Extension dialog, enter org.eclipse.hyades.test.core.ExecutionEnvironmentAdapter in the filter, select the extension point and click Finish.
    • Select the resulting org.eclipse.hyades.test.core.ExecutionEnvironmentAdapter node of the extension and update the following values:
      • ID: ExecutionEnvironmentAdapter.ExecExample
      • Name: Test Execution Example
    • Select the child node, ExecutionEnvironmentAdapter of the extension and update the following values:
      • id: TestExecExampleExecutionEnvironmentAdapter
      • name: EXEC_EXAMPLE_EXECUTION_ENVIRONMENT_ADAPTER
      • class: org.eclipse.tptp.test.exec.example.execution.TestExecExampleExecutionEnvironmentAdapter
      Save the Manifest file. Right click the EXEC_EXAMPLE_EXECUTION_ENVIRONMENT_ADAPTER(ExecutionEnvironmentAdapter) node and select New > Supported Test Type. Select the resulting SupportedTestType node and update the name value with testExecutionExampleType.
    • From the org.eclipse.tptp.test.execution.example.core plug-in, open the Extensions tab of the Manifest file in the Plug-in Manifest Editor. Click Add, to add a new extension. In the New Extension dialog, enter org.eclipse.hyades.test.core.ExecutionDeploymentAdapter in the filter, select the extension point and click Finish. This will define the deployment behavior of the test type.
    • Select the resulting org.eclipse.hyades.test.core.ExecutionDeploymentAdapter node of the extension and update the following values:
      • ID: ExecutionDeploymentAdapter.ExecExample
      • Name: Test Execution Example
    • Select the child node, ExecutionEnvironmentAdapter of the extension and update the following values:
      • id: TestExecExampleExecutionDeploymentAdapter
      • name: EXEC_EXAMPLE_EXECUTION_DEPLOYMENT_ADAPTER
      • class: org.eclipse.tptp.test.exec.example.execution.TestExecExampleExecutionDeploymentAdapter
      Select the SupportedTestType child node of EXEC_EXAMPLE_EXECUTION_ENVIRONMENT_ADAPTER(ExecutionDeploymentAdapter) and update the name value with testExecutionExampleType.
    • Save the Manifest file.

Implement the New Test Execution Adapter and Runner


In this section the adapters are implemented to create the extensions to setup the classpath for the Execution Environment. This example will not make any test specific modifications to the classpath other than adding the runner to the Agent Controller classpath. This is done by creating the Execution Environment Adapter class but not modifying the Execution Environment.
  1. Implement the Execution Deployment Adapter class, TestExecExampleExecutionDeploymentAdapter, which is responsible for deploying the files required to run the test. In this example, there are no files to deploy to during the test so the inherited methods will remain unimplemented.
    • Select the org.eclipse.tptp.test.execution.example.core plug-in project in the Package Explorer. From the File menu select New > Class and enter the following values:
      • Package: org.eclipse.tptp.test.exec.example.execution
      • Name: TestExecExampleExecutionDeploymentAdapter
      • Interfaces: org.eclipse.hyades.execution.harness.IExecutionDeploymentAdapterExtended
  2. Implement the Execution Environment Adapter class, TestExecExampleExecutionEnvironmentAdapter, which is responsible for adding additional elements to the classpath for the test. In this example, there are no entries to add to the classpath so the inherited methods will remain unimplemented.
    • Select the org.eclipse.tptp.test.execution.example.core plug-in project in the Package Explorer. From the File menu select New > Class and enter the following values:
      • Package: org.eclipse.tptp.test.exec.example.execution
      • Name: TestExecExampleExecutionEnvironmentAdapter
      • Interfaces: org.eclipse.hyades.execution.harness.IExecutionEnvironmentAdapter
  3. Implement the Executable Object Adapter to pass the JVM arguments, main class and arguments to launch the new runner.
    • Select the org.eclipse.tptp.test.execution.example.core plug-in project in the Package Explorer. From the File menu select New > Class and enter the following values:
      • Package: org.eclipse.tptp.test.exec.example.execution
      • Name: ExecExampleExecutableObjectAdapter
      • Interfaces: IExecutableObjectAdapter
    • Update the ExecExampleExecutableObjectAdapter class to contain the following:
      public void setupExecutableObject(IExecutableObject execObj, CFGClass rootResource, IImplementor theImplementor,
      	TPFDeployment deployment) throws ClassCastException
      {
      	JavaProcessExecutableObjectStub objectStub = (JavaProcessExecutableObjectStub)execObj;
      	objectStub.setArgs("org.eclipse.tptp.test.execution.example.runner.ExecExampleRunner" + theImplementor.getResource());
      }
      		
  4. Implement the runner org.eclipse.tptp.test.execution.example.runner.ExecExampleRunner and save the class in the src-runner source folder. The runner is called by the executor using the passed in classpath and command line arguments. This class should extend the org.eclipse.hyades.test.common.runner.HyadesRunner to implement behaviors expected by the TPTP execution component. This should include:
    • Creating an instance of itself
    • Launching the test
    • Returning execution history events to the model (using writeExecEvents)
    Execution history events are stored as XML fragments when working with the workbench. The utility classes in the org.eclipse.hyades.test.common.event package of the org.eclipse.hyades.test.common plug-in can be used to create the XML fragments. They can also be created using custom fragments adhering to the Test Execution Event Schema.
    The runner should be exported to a runner.jar to be used from the Agent Controller plug-ins directory.

Export the New Test Type and update the Agent Controller classpath


  1. Export the runner plug-in and add the runner jar to the Agent Controller classpath.
  2. Add it to the classpath, by placing the core plug-in's runner jar in a directory in the Agent Controller lib directory.
  3. Create a config/pluginconfig.xml within the core plug-in's directory containing the runner jar and update the path so it appends the runner jar to the classpath similar to the following fragment.
    <Variable name="CLASSPATH" position="append" value="%PLUGINS_HOME%\execExample\runner.jar"/>
    		
    Alternatively, the pluginconfig.xml can be generated by creating a class called SetConfig in a src-config source folder which implements org.eclipse.tptp.platform.agentcontroller.config.SetConfigSkeleton.

  4. A sample structure for the Agent Controller plug-ins folder would be: