module MapReduce::Reduceable
The MapReduce::Reduceable
mixin allows to reduce an arbitrary chunk using the specified map-reduce implementation.
Private Instance Methods
reduce_chunk(chunk, implementation) { |prev_item| ... }
click to toggle source
Reduces the specified chunk, e.g. some enumerable, using the specified map-reduce implementation using a lookahead of one to detect key changes. The reduce implementation is called up until a key change is detected, because the key change signals that the reduce operation is finished for the particular key, such that it will then be yielded.
@param chunk The chunk to be reduced. Can e.g. be some enumerable. @param implementation The map-reduce implementation.
# File lib/map_reduce/reduceable.rb, line 17 def reduce_chunk(chunk, implementation) return enum_for(:reduce_chunk, chunk, implementation) unless block_given? last_item = chunk.inject do |prev_item, cur_item| prev_key = prev_item[0] # Here we can compare without serializing the keys to json first, # because we reduce a chunk which includes a deserialization step. if prev_key == cur_item[0] [prev_key, implementation.reduce(prev_key, prev_item[1], cur_item[1])] else yield(prev_item) cur_item end end yield(last_item) if last_item end