Troubleshooting Guide

Table of contents

Security troubleshooting checklist

  1. TLS or SSL troubleshooting steps
  2. SSH troubleshooting steps
  3. Do you receive COMM662, COMM663, or COMM666 error messages?
  4. Are you editing the registry if drivers load when starting IKEYMAN
  5. Are you using two smartcard readers in IKEYMAN
  6. Review adding a client certificate
  7. Review creating self-signed certificates on Smartcards
  8. Is the certificate exported using z/OS utility gskkyman appears to be invalid or corrupted
  9. On the Windows operating system, after Host On-Demand is upgraded to a new level, the Certificate Wizard window does not appear when the Certificate Wizard is started
  10. Review UTF-8 support for Host On-Demand Version 7 server and Host On-Demand Versions 5 and 6 cached clients
  11. Does Keytool.exe produce errors for the Czech Republic language?
  12. Review Certificate management utility limitation on AIX
  13. Review using Host On-Demand with other security products


  1. TLS or SSL security troubleshooting steps

    If you are not able to establish a TLS or SSL connection to the server, check the following:

    1. What kind of certificate do you use?
      • Self-signed
      • CA
    2. Where is the certificate stored (for example, is it in the server's keyring)?
    3. Have you added the certificate to the Host On-Demand server's keyring database?   Issue the keyrng command from a command prompt. The syntax is:

      keyrng x connect server_name:port_number ftp

      where:

      • x is a generic class name.
      • server_name is the name of the Host On-Demand server.
      • port_number is the port on which the server is listening. For non-FTP connections, the default is 443. For FTP connections, the default is 990.
      • ftp indicates that a connection is being made to an FTP server

      Press enter at the password prompt. A list of all the certificates in the server's keyring database appears.

    4. If you added a certificate, check the validity dates on the certificate to determine if the certificate has expired.
    5. If the certificate is still valid, add it to the Host On-Demand server keyring. Stop and restart the Service Manager.
    6. Add the certificate to the client keyring to make it available to the clients.
    7. Use the keyrng utility to verify the correct certificate and validity dates. For example:
      keyrng CustomizedCAs verify
    8. Use the keyrng command to connect to the server on the 12173 SSL port. For example:
      keyrng x connect servername:12173 
    9. Configure a TLS or SSL session on the client to connect to a server on an SSL port (such as 12173).
    10. Try removing the Host On-Demand cached client, deleting temporary internet files, and try again. Deleting the temporary internet files removes the current CustomizedCAs.class.
    11. If possible, verify connectivity by trying to connect. (See Client does not connect for troubleshooting hints.)
    12. On the Server, verify that the HODServerKeyDb.* files are present in the \hostondemand\HOD directory. This is the key database file that is used by the Host On-Demand Redirector, and it contains certificates from well known CAs. It also contains the private key and any personal certificates for the server.
    13. If you use a self-signed certificate, verify that the CustomizedCAs.p12 file (if it exists) and CustomizedCAs.class are present in the \hostondemand\HOD directory. If you use a self-signed certificate or a certificate issued by an unknown CA, extract the certificate and include it in the CustomizedCAs.p12 (if it exists) and CustomizedCAs.class files. This file resides in the Host On-Demand publish directory so that the clients can get to it. It is downloaded every time the Host On-Demand client is downloaded.
    14. If you experience problems with SSL on the Redirector, verify that the Host On-Demand server key database and the CustomizedCAs.p12 file (if it exists) and CustomizedCAs.class have been created and are present on the computer running the Host On-Demand Redirector. See Redirector Troubleshooting Checklist for details.
  2. SSH troubleshooting steps

    If you are not able to establish an SSH connection to the server, check the following:

    1. Make sure the address and port number of the server are correct. The standard port number of SSH server is 22, but it can be different.
    2. Check the JVM level on the client. Host On-Demand SSH function requires a Java 2 JVM with Java Cryptography Extension (JCE), which is a part of JCE since JDK 1.4 on client side. If you use an older JVM, for example JDK 1.1 or JDK 1.3 level of JVM without JCE, upgrade your JVM to JDK 1.4 level.
    3. Make sure SSH Version 2 protocol is enabled on the server. Older SSH servers support only SSH Version 1.x protocol, which Host On-Demand does not understand.
    4. If you have enabled public-key authentication, disable it and try to use password authentication only because the configuration for public-key authentication is more complicated and error-prone.
    5. If password authentication works, double-check the configuration for public-key authentication. In many cases, the configuration on the server is not correct, for example file permissions. The procedure is different depending on your SSH server. For more information, refer to your SSH server documentation.
    6. The private/public key-pair used for public key authentication is saved on the file system of each client. If you need to use public key authentication from more than two clients, you need to copy KeyStore file (usually it is .keystore file in user's home directory) to other machines.
    7. Many SSH Servers have their own debug modes. If you have difficulties in problem identification, you might want to use them. For more information, refer to your SSH server documentation.
  3. Do you receive COMM662, COMM663, or COMM666 error messages?

    Refer to the following COMM error messages for more information:

    1. COMM662: The signer certificate of the host's site certificate has not been added to the CustomizedCAs.p12 file or CustomizedCAs.class on the Host On-Demand server. This error can occur when using a self-signed certificate if you specified a password other then hod when creating the customizedCAs.p12 file. This error can also occur if the permission bits for the CustomizedCAs.p12 file are not set to 755.
    2. COMM663: The certificate on the server might use a name that does not match its Internet name.
    3. COMM666: The certificate might have expired.

  4. Editing the registry if drivers load when starting IKEYMAN

    When starting IKEYMAN on a Host On-Demand Server on Windows 2000, an error message occurs when loading slbck.dll during startup. A Schlumberger smart card reader must first be installed and then uninstalled. Some Schlumberger entries might remain in the registry. To get rid of this message, a user must clear all Schlumber entries out of the registry, or they must edit a file in Host On-Demand.

    Host On-Demand Certificate Management uses the PKCS11 interface to access smartcard functions. This interface is used mostly for creating self-signed certificates in smartcards, or downloading a certificate in a .pfx or .p12 file to a smartcard.

    Before the smartcard can be accessed, additional configuration might be required. When Host On-Demand is installed, it determines if any smartcards are present in the system. Currently, Host On-Demand recognizes the IBM Security Card and the Schlumberger Reflex readers installed with the Cryptoflex Security Kit V3.0c.

    IBM Certificate Management reads all its parameters from an initialization file, ikminit_hod.properties, stored in hostondemand\bin. If Host On-Demand recognizes the IBM Security Card, the following line appears in the properties file:

    DEFAULT_CRYPTOGRAPHIC_MODULE=w32pk2ig.dll

    This tells IBM Certificate Manager to load this dll when smartcard functions are needed.

    If Host On-Demand recognizes a Schlumberger card, a line similar to the following appears in the properties file:

    DEFAULT_CRYPTOGRAPHIC_MODULE=C:\\Program
    Files\\Schlumberger\\Smart Cards and Terminals\\Common Files\\slbck.dll

    These are the only security devices that have been tested with IBM Certificate Management. If another security device implements the PKCS11 interface through a dll, you can test it by changing the name and location of the dll in the ikminit_hod.properties file.

    If the security device is removed from the system, IBM Certificate Management reports the following error at startup:

    Cryptographic token initialization failed.

    To prevent this error, remove the DEFAULT_CRYPTOGRAPHIC_MODULE statement from the ikminit_hod.properties file.

  5. Are you using two smartcard readers in IKEYMAN?

    Installing more than one smartcard on the same computer might cause Host On-Demand smartcard support to function incorrectly.

    For example, if the Host On-Demand Certificate Manager cannot open the IBM Security Card and a Schlumberger smartcard was previously installed on your computer, there might be values left in your registry causing the IBM Security Card drivers to function incorrectly.

    To remedy this problem, make a backup of your registry and carefully delete any of the following keys that remain after you uninstall the Schlumberger card:

  6. Review adding a client certificate

    When the Host On-Demand client contacts an SSL server that requests a client certificate, such as Communications Server for Windows NT, Communications Server for AIX, or Communications Server for OS/390 in client authentication mode, the Host On-Demand client might invoke the MSCAPI interface to request all available client certificates. MSCAPI returns all registered certificates, whether they are stored completely in the MSCAPI database, or are associated through MSCAPI with some security device, such as a smartcard or thumbprint reader. The list of certificates that are currently registered in a MSCAPI database can be displayed in the following way:

    1. Start the Internet Explorer 5.x browser.
    2. From the menu bar, choose Tools then Internet Options.
    3. Across the top of the Internet Options panel, choose the Content tab.
    4. On the Content panel, click the Certificates button.
    5. Across the top of the Certificates panel, choose the Personal tab; if it is not already chosen. These are the certificates that will appear in the drop down list on the Host On-Demand session configuration panel and the Server Requesting Certificate panel. If the certificate is not in this list, it cannot be used by Host On-Demand for client authentication.

    Any smartcard or security device that is recognized by MSIE can be used by Host On-Demand for client authentication. Certificates are usually obtained by visiting a Web page with the MSIE browser, filling out a form on the Web page, and then storing the new certificate in either the browser's database or a security device.

    For example, load http://freecerts.entrust.com/webcerts/ag_browser_req.htm into the MSIE browser. Fill out the information requested, press Proceed to Step 2 and then Proceed to Step 3. At the bottom of this page is a drop down list that lets you specify where to put the certificate.

    Choosing Microsoft Base Cryptographic Provider 1.0 puts the certificate into the MSCAPI database. No extra hardware is needed to access it.

    Choosing Schlumberger Cryptographic Service Provider or Gemplus GemSAFE Card CSP v1.0 puts the certificate into a smartcard. If you choose this destination, the name of the certificate appears in the MSIE Certificates window; just like a certificate that has been put into the MSCAPI database. However, the certificate will only be accessible if you have plugged in the smartcard by which the certificate was downloaded to.

    You should use the certificate obtained from freecerts.entrust.com for testing purposes only. After downloading the certificate, go to the the MSIE Certificates window and click the Trusted Root Certification Authorities tab. Scroll down the list until you find a certificate issued to Entrust PKI Demonstration Certificates. Highlight this certificate and export it to a file. Then add the exported file to the trusted list of your client authenticating SSL server. With this configuration, the SSL server should trust the Entrust certificate if it is returned by the Host On-Demand client. You should only use this exercise for testing purposes, and you should remove the Entrust PKI Demonstration Certificate from any production server.

  7. Review creating self-signed certificates on Smartcards

    Host On-Demand Certificate Management uses the PKCS11 interface to access smartcard functions. This interface is used mostly for creating self-signed certificates in smartcards, or downloading a certificate in a .pfx or .p12 file to a smartcard. (Note: The IBM Security Card supports the creation of a self-signed certificate, but not the downloading of an existing certificate in a .pfx or .p12 file.)

    Before the smartcard can be accessed, additional configuration may need to be done. When Host On-Demand is installed, it tries to determine if any smartcards are present in the system. Currently the only smartcards that are recognized are the IBM Security Card and the Schlumberger Reflex readers installed with the Cryptoflex Security Kit V3.0c.

    IBM Certificate Management reads all its parameters from an initialization file named ikminit_hod.properties that is stored in the hostondemand\bin directory. If the IBM Security Card was recognized, the following line will appear in the properties file:

        DEFAULT_CRYPTOGRAPHIC_MODULE=w32pk2ig.dll

    This tells IBM Certificate Manager to load this dll when smartcard functions are needed.

    If no IBM Security Card was detected, but a Schlumberger card was, the line will be similar to

        DEFAULT_CRYPTOGRAPHIC_MODULE=C:\\Program Files\\Schlumberger\\Smart Cards and Terminals\\Common Files\\slbck.dll

    These are the only security devices that have been tested with IBM Certificate Management. If you have another security device that implements the PKCS11 interface through a dll, it can be tested by changing the name and location of the dll in the ikminit_hod.properties file. If the smartcards are ever removed from the system, these lines should be removed from ikminit_hod.properties.

    With this configuration, a self-signed certificate can be created in the smartcard with the following steps:

    1. Start IBM Certificate Management
    2. On the menu bar, select Cryptographic Token
    3. On the Cryptographic Token panel, type in the PIN number of the smartcard, and clear the external database and press OK
    4. The Cryptographic Token (smartcard) is now open.

    Both the IBM Security Card and Schlumberger cards can create self-signed certificates. The Schlumberger card can also have a certificate in a .p12 or .pfx file imported to the card.

    If self-signed certificates are created, then the public portion of the certificates must be extracted (not exported) and added to the trusted list of the SSL server that will request the certificate.

    If a self-signed certificate is created in the IBM Security Card, it must be registered with MSCAPI. To do this, start the GemSAFE Card Details Tool. It will check the card, see that the certificate in the card has not been registered with MSCAPI, and ask if you want to register it.

    In our testing, not all readers supported all operations on all platforms. Here is a table of what readers were tested on which platforms.

      Entrust Self-signed Add .p12 Windows 98/NT operating sytem Windows 2000 operating system
    IBM Security Card PCMCIA Reader X X   X  
    IBM Security Card Serial Reader X     X  
    Schlumberger Reflex 20 Reader X X X X X
    Schlumberger Reflex 72 Reader X X X X  
    Schlumberger Reflex Lite X X X   X

  8. Is the certificate exported using z/OS utility gskkyman appears to be invalid or corrupted

    Host On-Demand and its utilities will not read PKCS12 files exported using the z/OS utility gskkyman. The problem is that gskkyman uses PFX v1 format for PKCS12 files, whereas Host On-Demand and its utilities use PFX v3 format for PKCS12 files.

    Here is an example of a failing scenario:

    1. On a z/OS server, the system administrator:
      1. Uses gskkyman to create a self-signed certificate to serve as a personal certificate for a Host On-Demand client.
      2. Uses gskkyman to export the key and the certificate to a PKCS12 file. (The function on the gskkyman Export Key Menu is 'Export keys to a PKCS12 file'.
      3. Transmits the PKCS12 file to the user.

    2. On the client, the user:
      1. Starts a 3270 Display session that requires that the client send a certificate to the host, and that specifies that the certificate source be a PKCS12 or PFX file.
      2. Provides, when prompted by the 3270 Display session startup program, the path of the PKCS12 file that was received from the system administrator.

    3. On the client, the 3270 Display session startup program displays the following error message:
      Certificate password was incorrect or certificate found at <path of PKCS12 file> was corrupted. (ECL0034)

    Another failing scenario may be that the certificate cannot be read by any of the Host On-Demand certificate utilities.

  9. The fix is to convert the PKCS12 file to PFX v3 format before sending the PKCS12 file to a user and before using the PKCS12 file with any Host On-Demand utility or session. To convert the format, take the following steps:

    1. Download the certificate to a workstation that has a Netscape 4.x or Netscape 6.x browser installed.
    2. Use Netscape to import the certificate from the PKCS12 file. Netscape can read the PFX v1 format. To import the file using Netscape 4.x:
      1. Click Communicator > Tools > Security Info.
      2. Under Certificates, click Yours.
      3. Click Import a Certificate... and follow the instructions to import the certificate from a PKCS12 file.
    3. Use Netscape to export the certificate to a PKCS12 file. Netscape will export the certificate in PFX v3 format. To export the file using Netscape 4.x:
      1. Click Communicator > Tools > Security Info.
      2. Under Certificates, click Yours.
      3. Click the imported certificate.
      4. Click Export a Certificate... and follow the instructions to export the certificate to a PKCS12 file.

  10. On the Windows operating systems, after Host On-Demand is upgraded to a new level, the Certificate Wizard window does not appear when the Certificate Wizard is started

    On the Windows operating systems, after Host On-Demand is upgraded to a new level, the Certificate Wizard panel may not appear when the Certificate Wizard is started.

    This problem is caused by the fact that program processes associated with the Certificate Wizard may have been left running during the upgrade. On Windows 2000 operating systems these program processes are:

    As a temporary workaround, you can use the Certificate Manager.

    To actually fix this problem using the Windows 2000 operating system, follow these steps:

    1. Stop the processes if they are still running.
      1. Type Ctrl-Alt-Delete.
      2. Click Task Manager.
      3. On the Task Manager panel, click the Applications tab.
      4. On the Task Manager panel, click Host On-Demand Certificate Wizard, then click End Task. The entry for Host On-Demand Certificate Wizard should disappear.
      5. On the Task Manager panel, click File > Exit Task Manager.
    2. Uninstall Host On-Demand using the Windows 2000 operating system's Add/Remove Programs wizard.
    3. Reinstall Host On-Demand.

  11. Review UTF-8 support for Host On-Demand Version 7 server and Host On-Demand Versions 5 and 6 cached clients

    When using Host On-Demand Versions 5 and 6 cached clients, in order to support UTF-8 encoding with the self-signed certificates created with the Host On-Demand Version 7 server, download either the 6.0.4a or 5.0.8a Interim Service Build (ISB) from the IBM Software Internet Service Delivery site at http://www6.software.ibm.com/aim/home.html. You will need to register on this site if you have not already done so.

  12. Does Keytool.exe produce errors for the Czech Republic language?
  13. Keytool.exe is a binary executable for Windows included with the JRE installed with Host On-Demand. When running keytool.exe, there are translation errors for the Czech Republic language.

    To resolve this problem, upgrade to the latest IBM JRE on the Host On-Demand Service Key Web site.

  14. Review Certificate management utility limitation on AIX

    The certificate management utility on AIX requires JRE 1.1.8. If you are running JVM 1.3, you will receive the following error message:
    Exception in thread "main" java.lang.VerifyError

    To use the certificate management utility on AIX with JRE 1.1.8, set the JAVA_HOME environment variable to point to the Java 1.1.8 installation before running the "CertificateManagement" script.

  15. Review using Host On-Demand with other security products

    When using other vendors' security products that lock or overwrite files, such as Netscape's Mission Control, be aware that the edited client configuration files may cause problems when upgrading to a newer version of Host On-Demand.

    For example, if the signed.db file is locked or overwritten, the prior version of Host On-Demand's signed certificate is presented. Consequently, because the incorrect version of the certificate continues to be presented, users are prompted to grant or deny access to the newer version's Host On-Demand applets each time they try to log in. Selecting the "Remember this decision" checkbox has no effect. Other symptoms include blank lines or undefined hexadecimal certificate information in Netscape's Java/Javascript Certificate list.

    To resolve this, follow the security program's instructions on how to recapture the configuration to use the newer version of Host On-Demand's signed certificate before distributing to users.

    Top of page Table of contents