class Ai4r::Experiment::ClassifierEvaluator

The ClassifierEvaluator is useful to compare different classifiers algorithms. The evaluator builds the Classifiers using the same data examples, and provides methods to evalute their performance in parallel. It is a nice tool to compare and evaluate the performance of different algorithms, the same algorithm with different parameters, or your own new algorithm against the classic classifiers.

Attributes

build_times[R]
classifiers[R]
eval_times[R]

Public Class Methods

new() click to toggle source
# File lib/ai4r/experiment/classifier_evaluator.rb, line 19
def initialize
  @classifiers = []
end

Public Instance Methods

<<(classifier)
Alias for: add_classifier
add_classifier(classifier) click to toggle source

Add a classifier instance to the test batch

# File lib/ai4r/experiment/classifier_evaluator.rb, line 24
def add_classifier(classifier)
  @classifiers << classifier
  return self
end
Also aliased as: <<
build(data_set) click to toggle source

Build all classifiers, using data examples found in data_set. The last attribute of each item is considered as the item class. Building times are measured by separate, and can be accessed through #build_times attribute reader.

# File lib/ai4r/experiment/classifier_evaluator.rb, line 36
def build(data_set)
  @build_times = []
  @classifiers.each do |classifier|
    @build_times << Benchmark.measure { classifier.build data_set }
  end
  return self
end
eval(data) click to toggle source

You can evaluate new data, predicting its class. e.g.

classifier.eval(['New York',  '<30', 'F'])  
=> ['Y', 'Y', 'Y', 'N', 'Y', 'Y', 'N']

Evaluation times are measured by separate, and can be accessed through #eval_times attribute reader.

# File lib/ai4r/experiment/classifier_evaluator.rb, line 50
def eval(data)
  @eval_times = []
  results = []
  @classifiers.each do |classifier|
    @eval_times << Benchmark.measure { results << classifier.eval(data) }
  end
  return results
end
test(data_set) click to toggle source

Test classifiers using a data set. The last attribute of each item is considered as the expected class. Data items are evaluated using all classifiers: evalution times, sucess rate, and quantity of classification errors are returned in a data set. The return data set has a row for every classifier tested, and the following attributes:

["Classifier", "Testing Time", "Errors", "Success rate"]
# File lib/ai4r/experiment/classifier_evaluator.rb, line 66
def test(data_set)
  result_data_items = []
  @classifiers.each do |classifier|
    result_data_items << test_classifier(classifier, data_set)
  end
  return Ai4r::Data::DataSet.new(:data_items => result_data_items, 
    :data_labels => ["Classifier","Testing Time","Errors","Success rate"])
end

Private Instance Methods

test_classifier(classifier, data_set) click to toggle source
# File lib/ai4r/experiment/classifier_evaluator.rb, line 76
def test_classifier(classifier, data_set)
  data_set_size = data_set.data_items.length
  errors = 0
  testing_times = Benchmark.measure do 
    data_set.data_items.each do |data_item|
      data = data_item[0...-1]
      expected_result = data_item.last
      result = classifier.eval data
      errors += 1 if result != expected_result
    end
  end
  return [classifier, testing_times.real, errors,
    ((data_set_size-errors*1.0)/data_set_size)]
end