class LogStash::Filters::Memoize

This filter provides en.wikipedia.org/wiki/Memoization[memoization] to wrapped filter. Internally, It based on en.wikipedia.org/wiki/Cache_replacement_policies#LRU[LRU] cache algorithm.

See below an example of how this filter might be used.

source,ruby

filter {

memoize {
  key => "%{host}" <1>
  fields => ["host_owner", "host_location"] <2>
  filter_name => "elasticsearch" <3>
  filter_options => { <4>
    query => "host:%{host}"
    index => "known_host"
    fields => {
      "host_owner" => "host_owner"
      "host_location" => "host_location"
    }
  }
}

}


  • When an event with a new <1> key comes in, execute wrapped <3> <4> filter and caches the <2> fields value.

  • When an event with a same <1> key comes in, sets cached value to target <2> fields without wrapped <3> <4> filter execution.

Public Instance Methods

filter(event) click to toggle source
# File lib/logstash/filters/memoize.rb, line 62
def filter(event)
  formattedKey = event.sprintf(@key);
  result = @cache[formattedKey]

  if !result.nil?
    @logger.debug("Cached value found.", :key => formattedKey, :value => result) if @logger.debug?
    @fields.each { |field| event.set(field, result[field]) }
  else
    @logger.debug("Cached value not found. Do filter.", :key => formattedKey, :filter => @filter) if @logger.debug?
    @filter.filter(event)
    @fields.each { |field| (result ||= {})[field] = event.get(field) }
    @cache[formattedKey] = result
  end

  # filter_matched should go in the last line of our successful code
  filter_matched(event)
end
register() click to toggle source
# File lib/logstash/filters/memoize.rb, line 55
def register
  @filter = LogStash::Plugin.lookup("filter", @filter_name).new(@filter_options)
  @filter.register
  @cache = ::LruRedux::TTL::ThreadSafeCache.new(@cache_size, @ttl)
end