|
Enterprise Information Portal APIs |
||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--com.ibm.gcs.db.component.UrlCrawlTableDef
This class provides the constants for the column names of the table containing the urls and crawl info, URLCRAWLTABLE It also provides a method to construct the sql string to create the URLCRAWLTABLE as follows:
create table URLCRAWLTABLE ( URL varchar(250) primary key not null, DEPTH int not null default 0, CRAWL_PATTERN_ID varchar(100), STATE varchar(40) default 'VIRGIN', TIME timestamp not null default current timestamp, TIME_CRAWLED timestamp, EXCEPTION varchar(1000), HIDE smallint not null default 0, CRAWL_FREQ date, STATE_ID smallint not null default 0, PRIORITY smallint not null default 0 )
Field Summary | |
static java.lang.String |
CRAWL_PATTERN_ID
The key for the CrawlPattern: CRAWL_PATTERN_ID . |
static java.lang.String |
DEPTH
The recursion depth of this url in the crawl: DEPTH . |
static java.lang.String |
EXCEPTION
Exception message for failed states: EXCEPTION . |
static java.lang.String |
FREQ
How often should this url be crawled? : CRAWL_FREQ . |
static java.lang.String |
HIDE
Flag to be used to specify whether the url is visible in the collection: HIDE . |
static java.lang.String |
LAST_MODIFIED
The time the source was last modified, specifically as returned by the http response: LAST_MODIFIED . |
static java.lang.String |
PRIORITY
The crawl priority.: PRIORITY . |
static java.lang.String |
STATE
The current state: STATE . |
static java.lang.String |
STATE_ID
What state is this url in?: STATE_ID . |
static java.lang.String |
TABLE
The name of the table: URLCRAWLTABLE . |
static java.lang.String |
TIME
The time of the current state: TIME . |
static java.lang.String |
URL_KEY
The name of the url: URL . |
static java.lang.String |
VISIT_TIME
The time crawled: VISIT_TIME . |
Constructor Summary | |
UrlCrawlTableDef()
|
Method Summary | |
static void |
createTable(Transaction t)
Given a Transaction object, executes the create statements (which create the table and associated indexes on the table). |
static void |
dropTable(Transaction t)
Given a Transaction object, executes the drop statement. |
static java.util.Enumeration |
getCreateIndexSQL()
Returns the SQL CREATE statements to create the necessary indexes on the URLCRAWLTABLE. |
static java.lang.String |
getCreateSQL()
Returns the SQL CREATE statement to create the URLCRAWLTABLE. |
Methods inherited from class java.lang.Object |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
public static final java.lang.String TABLE
URLCRAWLTABLE
.public static final java.lang.String URL_KEY
URL
.
VARCHAR(250) NOT NULL PRIMARY KEY
public static final java.lang.String DEPTH
DEPTH
.
INT NOT NULL DEFAULT 0
public static final java.lang.String CRAWL_PATTERN_ID
CRAWL_PATTERN_ID
.
VARCHAR(40)
public static final java.lang.String TIME
TIME
.
TIMESTAMP NOT NULL DEFAULT CURRENT TIMESTAMP
public static final java.lang.String STATE
STATE
.
VARCHAR(16) NOT NULL
public static final java.lang.String VISIT_TIME
VISIT_TIME
.
DATE
public static final java.lang.String LAST_MODIFIED
LAST_MODIFIED
.
DATE
public static final java.lang.String EXCEPTION
EXCEPTION
.
VARCHAR(4000)
public static final java.lang.String HIDE
HIDE
.
SMALL INT NOT NULL DEFAULT 0
public static final java.lang.String FREQ
CRAWL_FREQ
.
DATE
public static final java.lang.String STATE_ID
STATE_ID
.
SMALLINT NOT NULL DEFAULT 0
for conversions.
public static final java.lang.String PRIORITY
PRIORITY
.
SMALLINT NOT NULL DEFAULT 0
Constructor Detail |
public UrlCrawlTableDef()
Method Detail |
public static java.lang.String getCreateSQL()
create table URLCRAWLTABLE( URL varchar(250) primary key not null, DEPTH int not null default 0, CRAWL_PATTERN_ID varchar(100), STATE varchar(40) default 'VIRGIN', TIME timestamp not null default current timestamp, VISIT_TIME timestamp, LAST_MODIFIED timestamp, EXCEPTION varchar(1000), HIDE smallint not null default 0, CRAWL_FREQ date, STATE_ID smallint not null default 0, PRIORITY smallint not null default 0 )Note: As a result of column dependencies, the treetable must be created before the urlpoolstable.
public static java.util.Enumeration getCreateIndexSQL()
CREATE INDEX FIFO ON URLCRAWLTABLE
(TIME, STATE_ID, HIDE)
CREATE INDEX URL_STAR ON URLCRAWLTABLE
(URL_KEY, DEPTH, CRAWL_PATTERN_ID, STATE, ..)
public static void createTable(Transaction t) throws TransactionException
t
- The transaction object through which to execute
the create statements.TransactionException
- on failed SQL update or if
ConfigTable does not already exist.public static void dropTable(Transaction t) throws TransactionException
t
- The transaction object through which to execute
the drop statements.TransactionException
- on failed SQL update.
|
EIP Web Crawler APIs | ||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |