Package translate :: Package search :: Package indexing :: Module XapianIndexer :: Class XapianDatabase
[hide private]
[frames] | no frames]

Class XapianDatabase

source code


interface to the xapian (http://xapian.org) indexer

Nested Classes [hide private]
  QUERY_TYPE
override this with the query class of the implementation
Instance Methods [hide private]
 
__init__(self, basedir, analyzer=None, create_allowed=True)
initialize or open a xapian database
source code
 
flush(self, optimize=False)
force to write the current changes to disk immediately
source code
xapian.Query
_create_query_for_query(self, query)
generate a query based on an existing query object
source code
xapian.Query
_create_query_for_string(self, text, require_all=True, analyzer=None)
generate a query for a plain term of a string query
source code
xapian.Query
_create_query_for_field(self, field, value, analyzer=None)
generate a field query
source code
xapian.Query
_create_query_combined(self, queries, require_all=True)
generate a combined query
source code
xapian.Document
_create_empty_document(self)
create an empty document to be filled and added to the index later
source code
 
_add_plain_term(self, document, term, tokenize=True)
add a term to a document
source code
 
_add_field_term(self, document, field, term, tokenize=True)
add a field term to a document
source code
 
_add_document_to_index(self, document)
add a prepared document to the index database
source code
 
begin_transaction(self)
begin a transaction
source code
 
cancel_transaction(self)
cancel an ongoing transaction
source code
 
commit_transaction(self)
submit the changes of an ongoing transaction
source code
XapianIndexer.CommonEnquire
get_query_result(self, query)
return an object containing the results of a query
source code
 
delete_document_by_id(self, docid)
delete a specified document
source code
list of dicts
search(self, query, fieldnames)
return a list of the contents of specified fields for all matches of a query
source code
 
_prepare_database(self, writable=False)
reopen the database as read-only or as writable if necessary
source code

Inherited from CommonIndexer.CommonDatabase: delete_doc, get_field_analyzers, index_document, make_query, set_field_analyzers

Inherited from object: __delattr__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __str__

Class Variables [hide private]
  INDEX_DIRECTORY_NAME = 'xapian'
override this with a string to be used as the name of the indexing directory/file in the filesystem

Inherited from CommonIndexer.CommonDatabase: ANALYZER_DEFAULT, ANALYZER_EXACT, ANALYZER_PARTIAL, ANALYZER_TOKENIZE, field_analyzers

Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self, basedir, analyzer=None, create_allowed=True)
(Constructor)

source code 

initialize or open a xapian database

Parameters:
  • basedir (str) - the parent directory of the database
  • analyzer (int) - bitwise combination of possible analyzer flags to be used as the default analyzer for this database. Leave it empty to use the system default analyzer (self.ANALYZER_DEFAULT). see self.ANALYZER_TOKENIZE, self.ANALYZER_PARTIAL, ...
  • create_allowed (bool) - create the database, if necessary; default: True
Raises:
  • ValueError - the given location exists, but the database type is incompatible (e.g. created by a different indexing engine)
  • OSError - the database failed to initialize
Overrides: object.__init__

flush(self, optimize=False)

source code 

force to write the current changes to disk immediately

Parameters:
  • optimize (bool) - ignored for xapian
Overrides: CommonIndexer.CommonDatabase.flush

_create_query_for_query(self, query)

source code 

generate a query based on an existing query object

basically this function should just create a copy of the original

Parameters:
  • query (xapian.Query) - the original query object
Returns: xapian.Query
the resulting query object
Overrides: CommonIndexer.CommonDatabase._create_query_for_query

_create_query_for_string(self, text, require_all=True, analyzer=None)

source code 

generate a query for a plain term of a string query

basically this function parses the string and returns the resulting query

Parameters:
  • text (str) - the query string
  • require_all (bool) - boolean operator (True -> AND (default) / False -> OR)
  • analyzer (int) - Define query options (partial matching, exact matching, tokenizing, ...) as bitwise combinations of CommonIndexer.ANALYZER_???. This can override previously defined field analyzer settings. If analyzer is None (default), then the configured analyzer for the field is used.
Returns: xapian.Query
resulting query object
Overrides: CommonIndexer.CommonDatabase._create_query_for_string

_create_query_for_field(self, field, value, analyzer=None)

source code 

generate a field query

this functions creates a field->value query

Parameters:
  • field (str) - the fieldname to be used
  • value (str) - the wanted value of the field
  • analyzer (int) - Define query options (partial matching, exact matching, tokenizing, ...) as bitwise combinations of CommonIndexer.ANALYZER_???. This can override previously defined field analyzer settings. If analyzer is None (default), then the configured analyzer for the field is used.
Returns: xapian.Query
the resulting query object
Overrides: CommonIndexer.CommonDatabase._create_query_for_field

_create_query_combined(self, queries, require_all=True)

source code 

generate a combined query

Parameters:
  • queries (list of xapian.Query) - list of the original queries
  • require_all (bool) - boolean operator (True -> AND (default) / False -> OR)
Returns: xapian.Query
the resulting combined query object
Overrides: CommonIndexer.CommonDatabase._create_query_combined

_create_empty_document(self)

source code 

create an empty document to be filled and added to the index later

Returns: xapian.Document
the new document object
Overrides: CommonIndexer.CommonDatabase._create_empty_document

_add_plain_term(self, document, term, tokenize=True)

source code 

add a term to a document

Parameters:
  • document (xapian.Document) - the document to be changed
  • term (str) - a single term to be added
  • tokenize (bool) - should the term be tokenized automatically
Overrides: CommonIndexer.CommonDatabase._add_plain_term

_add_field_term(self, document, field, term, tokenize=True)

source code 

add a field term to a document

Parameters:
  • document (xapian.Document) - the document to be changed
  • field (str) - name of the field
  • term (str) - term to be associated to the field
  • tokenize (bool) - should the term be tokenized automatically
Overrides: CommonIndexer.CommonDatabase._add_field_term

_add_document_to_index(self, document)

source code 

add a prepared document to the index database

Parameters:
  • document (xapian.Document) - the document to be added
Overrides: CommonIndexer.CommonDatabase._add_document_to_index

begin_transaction(self)

source code 

begin a transaction

Xapian supports transactions to group multiple database modifications. This avoids intermediate flushing and therefore increases performance.

Overrides: CommonIndexer.CommonDatabase.begin_transaction

cancel_transaction(self)

source code 

cancel an ongoing transaction

no changes since the last execution of 'begin_transcation' are written

Overrides: CommonIndexer.CommonDatabase.cancel_transaction

commit_transaction(self)

source code 

submit the changes of an ongoing transaction

all changes since the last execution of 'begin_transaction' are written

Overrides: CommonIndexer.CommonDatabase.commit_transaction

get_query_result(self, query)

source code 

return an object containing the results of a query

Parameters:
  • query (xapian.Query) - a pre-compiled xapian query
Returns: XapianIndexer.CommonEnquire
an object that allows access to the results
Overrides: CommonIndexer.CommonDatabase.get_query_result

delete_document_by_id(self, docid)

source code 

delete a specified document

Parameters:
  • docid (int) - the document ID to be deleted
Overrides: CommonIndexer.CommonDatabase.delete_document_by_id

search(self, query, fieldnames)

source code 

return a list of the contents of specified fields for all matches of a query

Parameters:
  • query (xapian.Query) - the query to be issued
  • fieldnames (string | list of strings) - the name(s) of a field of the document content
Returns: list of dicts
a list of dicts containing the specified field(s)
Overrides: CommonIndexer.CommonDatabase.search

_prepare_database(self, writable=False)

source code 

reopen the database as read-only or as writable if necessary

this fixes a xapian specific issue regarding open locks for writable databases

Parameters:
  • writable (bool) - True for opening a writable database