class Glove::Workers::TrainingWorker

Performs the traing process on the word vector matrix, as well as word biases

Attributes

indices[RW]

@!attribute [r] indices

@return [Glove::Corpus] Shuffled co-occurrence matrix slots

@!attribute [r] word_vec

@return [GSL::Matrix] Clone of @caller.word_vec

@!attribute [r] word_biases

@return [GSL::Vector] Clone of @caller.word_biases
word_biases[RW]

@!attribute [r] indices

@return [Glove::Corpus] Shuffled co-occurrence matrix slots

@!attribute [r] word_vec

@return [GSL::Matrix] Clone of @caller.word_vec

@!attribute [r] word_biases

@return [GSL::Vector] Clone of @caller.word_biases
word_vec[RW]

@!attribute [r] indices

@return [Glove::Corpus] Shuffled co-occurrence matrix slots

@!attribute [r] word_vec

@return [GSL::Matrix] Clone of @caller.word_vec

@!attribute [r] word_biases

@return [GSL::Vector] Clone of @caller.word_biases

Public Class Methods

new(caller, indices) click to toggle source

Create a {Glove::Workers::TrainingWorker} instance @param [Glove::Model] caller Caller class @param [Array<(Integer, Integer)>] indices Shuffled indices of non-zero

elements in the model's co-occurence matrix
# File lib/glove/workers/training_worker.rb, line 21
def initialize(caller, indices)
  @caller, @indices = caller, indices
  @word_vec     = @caller.word_vec.dup
  @word_biases  = @caller.word_biases.dup
end

Public Instance Methods

apply_weights(w1, w2, loss, word_a_norm, word_b_norm) click to toggle source

Applies calculated weights to @word_vec and @word_biases. MUST be called in a Mutex#synchronize block

@param [Integer] w1 Row index @param [Integer] w2 Column index @param [Float] loss Loss value @param [Float] word_a_norm Norm of word at row w1 @param [Float] word_b_norm Norm of word at col w2

# File lib/glove/workers/training_worker.rb, line 93
def apply_weights(w1, w2, loss, word_a_norm, word_b_norm)
  word_vec.each_col do |col|
    col[w1] = (col[w1] - learning_rate * loss * col[w2]) / word_a_norm
    col[w2] = (col[w2] - learning_rate * loss * col[w2]) / word_b_norm
  end

  word_biases[w1] -= learning_rate * loss
  word_biases[w2] -= learning_rate * loss
end
calc_weights(w1, w2, prediction=0.0, word_a_norm=0.0, word_b_norm = 0.0) click to toggle source

Calculates loss, and norms for word1 (row) and word2 (column) by given indices

@param [Integer] w1 Row index @param [Integer] w2 Column index @param [Float] prediction (0.0) Initial predication value @param [Float] word_a_norm (0.0) Initial norm of word at row w1 @param [Float] word_b_norm (0.0) Initial norm of word at col w2 @return [Float, Float, Float] Array of loss, word_a_norm, word_b_norm

# File lib/glove/workers/training_worker.rb, line 64
def calc_weights(w1, w2, prediction=0.0, word_a_norm=0.0, word_b_norm = 0.0)
  count = cooc_matrix[w1, w2]

  word_vec.each_col do |col|
    w1_context = col[w1]
    w2_context = col[w2]

    prediction = prediction + w1_context + w2_context
    word_a_norm += w1_context * w1_context
    word_b_norm += w2_context * w2_context
  end

  prediction = prediction + word_biases[w1] + word_biases[w2]
  word_a_norm = Math.sqrt(word_a_norm)
  word_b_norm = Math.sqrt(word_b_norm)
  entry_weight = [1.0, (count/max_count)].min ** alpha
  loss = entry_weight * (prediction - Math.log(count))

  [loss, word_a_norm, word_b_norm]
end
run() click to toggle source

Runs the calculations @return [GSL::Matrix, GSL::Vector] Weighted word vectors and word biases

# File lib/glove/workers/training_worker.rb, line 29
def run
  mutex = Mutex.new
  slice_size = indices.size / threads

  workers = indices.each_slice(slice_size).map do |slice|
    Thread.new{ work(slice, mutex) }
  end
  workers.each(&:join)

  [word_vec, word_biases]
end
work(slice, mutex) click to toggle source

Perform a full train iteration on the word vectors and word biases @param [Array] slice Shuffled co-occurrence matrix slots @param [Mutex] mutex Thread-safe lock on apply_weights

# File lib/glove/workers/training_worker.rb, line 44
def work(slice, mutex)
  slice.each do |slot|
    w1, w2 = slot
    loss, word_a_norm, word_b_norm = calc_weights(w1, w2)

    mutex.synchronize do
      apply_weights(w1, w2, loss, word_a_norm, word_b_norm)
    end
  end
end