cqtsdbcrawler

The cqtsdbcrawler command runs the full-text search record-extractor.

Synopsis

cqperl cqtsdbcrawler.pl { -batch | -update | -optimize | -repair repair_file } CQ-db_name-dbset_name.xml [ -site site_name ]
cqperl cqtsdbcrawler.pl -encrypt -s dbset -d dbname -u user -p password -kf output_key_file_name -pf output_encrypted_password_file_name -jvmpar "jvm_options"

Description

Although the index is updated continuously after full-text search has been configured, there are circumstances in which you need to use the cqtsdbcrawler.pl command:
  • To index records for the first time.
  • To update the index explicitly.
  • To re-index records that have not been processed correctly.
  • To generate an encrypted password and key file.
  • To optimize the full-text search index file to improve performance.

Records that have not been processed correctly are logged to a repair file that is named record_type_time_stamp.xml and written to the same directory to which the ClearQuest search properties file, CQ-dbname-dbset-name.xml, is written. You use the file as input to cqtsdbcrawler.pl –repair.

When run in batch or updates modes, cqtsdbcrawler.pl records the last record that it successfully processed in the files, .batch_recovery.properties and .update_recovery.properties, respectively. These files provide starting points for processing when the command execution is stopped and restarted. They are written to the current user's home directory.

cqtsdbcrawler.pl is installed in the default product installation directory.

Options and Arguments

–batch
Runs the extractor in batch mode. This option read all records that have been configured for full-text search and passes them to Solr for indexing by Lucene. Processing is CPU- and IO-bound; to avoid degrading performance, therefore, you may want to run batch mode when system use is minimal.
–update
Runs the extractor in update mode. This option monitors the database for the addition, deletion, or modification of records. Any such changes to the database are read and passed to Solr for indexing. (This option is the default run mode for the WebSphere Application Server Express instance of the extractor; it is provided to enable the same functionality from the command line.)
–optimize
Optimizes the search engine index to improve performance.
–repair repair_file
Runs the extractor in repair mode. Records listed in repair_file are those that were not correctly processed in batch or update modes.
cq_search_properties_file
The ClearQuest search properties file, which is generated by the command, cqtssetup.pl ... –o CQ.
–site site
The default site is <local>, which means that the database is not replicated. This option is required if the database is replicated: specify the (MultiSite) site from which to gather the configuration parameters for record extraction.
–s dbset
Name of the database set.
–d dbname
Name of the database.
-u user
The user name.
-p password
The password.
-encrypt password
Encrypts the password.
-kf key_file
The file to which the password encryption key is to be written.
-pf password_file
The file to which the encrypted password is to be written.
-jvmpar "jvm_options"
Use to set the Java Virtual Machine (JVM) options.

Examples

Run cqtsdbcrawler.pl in batch mode:
cqperl cqtsdbcrawler.pl -batch CQ_SAMPLE-TextSearch.xml -site "<local>"
Run cqtsdbcrawler.pl in repair mode:
cqperl cqtsdbcrawler.pl -repair ~bert/Defect207294053937.xml ~bert/CQ_SAMPLE-TextSearch.xml
Create the encrypted password and key file:
cqperl cqtsdbcrawler.pl -s TextSearch -d SAMPL -u admin -p admin_password -kf key.txt -pf password.txt
Optimize the full-text search index file:
cqperl cqtsdbcrawler.pl -optimize CQ-SAMPL-TextSearch.xml
Run cqtsdbcrawler.pl in batch mode with the JVM option:
cqperl cqtsdbcrawler.pl -jvmpar "-Xms512m -Xmx1024m" -batch CQ_SAMPLE-TextSearch.xml -site "<local>"

See also

cqtssetup.pl, setmasterpropertyfromfile


Feedback