class CartonDb::ListMapDb

A map with strings as keys and lists of strings as contents.

This is suitable for storing a total number of elements as large as the low millions, with each entry containing a number of elements in the hundreds or low thousands.

Constants

FILE_ENCODING_OPTS

Attributes

name[RW]

Public Class Methods

new(name) click to toggle source

Initializes an instance that interacts with the database identified by the given name, which is the full path to a directory in the filesystem.

The directory for the database will be created if it does not already exist.

The parent directory is assumed to already exist, and an exception will be raised if it does not.

Other instance methods assume that the directory exists but make no other assumptions about the state of the persisted data, and an empty directory is a valid representation of an empty database.

This is a very fast operation.

@param name [String] The full path of the directory in the

filesystem in which the data is stored or will be stored.
# File lib/carton_db/list_map_db.rb, line 43
def initialize(name)
  self.name = name
  FileUtils.mkdir name unless File.directory?(name)
end

Public Instance Methods

[](key) click to toggle source

Returns the content of the entry identified by the given key or nil if no such entry exists.

This operation is fast, but may be slower for a larger database.

@param key [String] The key identifying the entry. @return [Array<String>] if a matching entry exists. @return [nil] if no matching entry exists.

# File lib/carton_db/list_map_db.rb, line 78
def [](key)
  key_d = CartonDb::Datum.for_plain(key)
  segment = segment_containing(key_d)
  segment.collect_content(key_d, Array)
end
[]=(key, content) click to toggle source

Creates a new entry or replaces the contents of the existing entry identified by the given key.

The is a fairly fast operation, but can be somewhat slower in a large database. Note that appending and concatenating may be faster than assignment.

@param key [String] The key identifying the entry. @param content [Array<String>] An array or other

enumerable collection of 0 or more list element string
values to be stored.
# File lib/carton_db/list_map_db.rb, line 59
def []=(key, content)
  key_d = CartonDb::Datum.for_plain(key)
  segment = segment_containing(key_d)
  if segment.empty?
    concat_elements key_d.plain, content
  else
    replace_entry_in_file segment, key_d, content
  end
end
append_element(key, element) click to toggle source

Appends an element string to the content of an entry. If the entry does not already exist, then one is created with a list containing the given element as its content.

Since this will only append text to a file within the database, it is a very fast operation.

@param key [String] The key identifying the entry. @param element [String] The element to be appended to the

content of the entry.
# File lib/carton_db/list_map_db.rb, line 235
def append_element(key, element)
  key_d = CartonDb::Datum.for_plain(key)
  element_d = CartonDb::Datum.for_plain(element)
  segment = segment_containing(key_d)
  segment.write_key_element_d key_d, element_d
end
clear() click to toggle source

Removes all entries from the database, leaving it empty.

This operation can be somewhat slow for a large database.

# File lib/carton_db/list_map_db.rb, line 167
def clear
  ListMapDb::Segment.clear_all_in_db name
end
concat_any_elements(key, elements) click to toggle source

Appends any number of element strings to the content of an entry. If the given elements collection is empty or nil, then the database is unchanged, and a new entry is not created if one did not exist previously.

This is a fast operation since it only appends text to an existing file.

@param key [String] The key identifying the entry. @param elements [Array<String>] An array or other

enumerable collection of elements to append.
# File lib/carton_db/list_map_db.rb, line 274
def concat_any_elements(key, elements)
  key_d = CartonDb::Datum.for_plain(key)
  segment = segment_containing(key_d)
  segment.write_key_d_elements key_d, elements
end
concat_elements(key, elements) click to toggle source

Appends any number of element strings to the content of an entry. If the entry does not already exist, then one is created with the given list as its content.

Appending an empty or nil collection is equivalent to invoking `db.touch key, optimization: :small`.

When appending a non-empty collection, this is a fast operation since it only appends text to an existing file.

@param key [String] The key identifying the entry. @param elements [Array<String>] An array or other

enumerable collection of elements to append.
# File lib/carton_db/list_map_db.rb, line 255
def concat_elements(key, elements)
  if empty_collection?(elements)
    touch key, optimization: :small
  else
    concat_any_elements key, elements
  end
end
count() click to toggle source

Returns the number of entries in the map.

This operation scans the entire database to count the keys, so it can be be a slow operation if the database is large.

@return [Fixnum]

# File lib/carton_db/list_map_db.rb, line 130
def count
  key_count = 0
  ListMapDb::Segment.each_in_db name do |segment|
    key_count += segment.key_count
  end
  key_count
end
delete(key) click to toggle source

Removes an entry from the database. Has no effect if the entry already does not exist.

This operation is fast, but may be slower for a larger database.

@param key [String] The key identifying the entry to be

deleted.
# File lib/carton_db/list_map_db.rb, line 214
def delete(key)
  key_d = CartonDb::Datum.for_plain(key)
  segment = segment_containing(key_d)
  return if segment.empty?

  segment.replace do |repl_io|
    segment.copy_entries_except key_d, repl_io
  end
end
each() { |key, content| ... } click to toggle source

Yields each entry in the database as a key/array pair.

This operation can take a lot of total time for a large database, but yields entries pretty rapidly regardless of database size.

@yieldparam key [String] The key of the entry. @yeildparam array [Array<String>] The elements of the list

entry's content.
# File lib/carton_db/list_map_db.rb, line 180
def each
  key_arrays_slice = {}
  ListMapDb::Segment.each_in_db name do |segment|
    segment.each_entry do |key, content|
      yield key, content
    end
  end
end
each_first_element() { |key, element| ... } click to toggle source

For each entry in the database, yields the first element in the entry's content or nil if the content is an empty list.

Performance is pretty much identical to that of each.

@yieldparam key [String] The key of the entry. @yeildparam array [String, nil] The first element of the

entry's content.
# File lib/carton_db/list_map_db.rb, line 198
def each_first_element
  ListMapDb::Segment.each_in_db name do |segment|
    segment.each_first_element do |key, element|
      yield key, element
    end
  end
end
element?(key, element) click to toggle source

Returns trus if an entry with the given key exists and its content includes at least one element with the given element value.

Performance is similar to key?

@param key [String] The key identifying the entry. @param element [String] The element value to match.

# File lib/carton_db/list_map_db.rb, line 105
def element?(key, element)
  key_d = CartonDb::Datum.for_plain(key)
  element_d = CartonDb::Datum.for_plain(element)
  segment = segment_containing(key_d)
  segment.element_d?(key_d, element_d)
end
empty?() click to toggle source

Returns true if the map has no entries.

This is a fairly fast operation.

@return [Boolean]

# File lib/carton_db/list_map_db.rb, line 117
def empty?
  ListMapDb::Segment.each_in_db name do |segment|
    return false unless segment.empty?
  end
  true
end
key?(key) click to toggle source

Returns true if an entry with the given key exists.

Performance is similar to [] but may be somewhat faster when a key is found since it doesn't need to ensure that it has read all of the elements for an entry.

@param key [String] The key identifying the entry.

# File lib/carton_db/list_map_db.rb, line 91
def key?(key)
  key_d = CartonDb::Datum.for_plain(key)
  segment = segment_containing(key_d)
  segment.key_d?(key_d)
end
merge_elements(key, elements) click to toggle source

Performs a bag-wise merge of the given elements with the content of an entry. Appends whatever elements are necessary so the content has an element corresponding to each of the given elements.

Performance is similar to [] when no new elements need to be added and similar to [] followed by concat_elements when one or more new elements needs to be added.

@param key [String] The key identifying the entry. @param elements [Array<String>] An array or other

enumerable collection of elements to be appended as
applicable.
# File lib/carton_db/list_map_db.rb, line 306
def merge_elements(key, elements)
  key_d = CartonDb::Datum.for_plain(key)
  element_ds = elements.map { |el|
    CartonDb::Datum.for_plain(el)
  }
  segment = segment_containing(key_d)
  segment.each_element_for_d key_d do |ed|
    eds_idx = element_ds.index(ed)
    element_ds.delete_at(eds_idx) if eds_idx
  end

  concat_elements key_d, element_ds
end
touch(key, optimization: :small) click to toggle source

Creates an entry with an empty list as its content if no entry exists for the given key. Has no effect on the content of the entry if it already exists.

Using :small optimization (the default), the operation may be slow for a large database since it checks for the existence of the key before marking its existence.

Using :fast optimization, the operation will always be fast, but it will add a mark for the existence of the key even if that is redundant, thus adding to the size of the stored data.

@param key [String] The key identifying the entry. @param optimization[:small, :fast] The optimization mode.

# File lib/carton_db/list_map_db.rb, line 153
def touch(key, optimization: :small)
  if optimization != :small && optimization != :fast
    raise ArgumentError, "Invalid optimization value. Must be :small or :fast"
  end

  key_d = CartonDb::Datum.for_plain(key)
  segment = segment_containing(key_d)
  segment.touch_d key_d, optimization
end
touch_element(key, element) click to toggle source

Appends an element to the content of an entry if no element with the same value already exists in the content.

@param key [String] The key identifying the entry. @param element [String] The element to be appended to the

content of the entry if applicable.
# File lib/carton_db/list_map_db.rb, line 286
def touch_element(key, element)
  key_d = CartonDb::Datum.for_plain(key)
  element_d = CartonDb::Datum.for_plain(element)
  return if element?(key_d, element_d)
  append_element key_d, element_d
end

Private Instance Methods

empty_collection?(collection) click to toggle source
# File lib/carton_db/list_map_db.rb, line 339
def empty_collection?(collection)
  return true if collection.nil?
  occupied = collection.any? { true }
  ! occupied
end
replace_entry_in_file(segment, key_d, content) click to toggle source
# File lib/carton_db/list_map_db.rb, line 324
def replace_entry_in_file(segment, key_d, content)
  segment.replace do |repl_io|
    segment.copy_entries_except key_d, repl_io
    element_count = 0
    count = write_key_elements(key_d, content, repl_io)
    repl_io << "#{key_d.escaped}\n" if count.zero?
  end
end
segment_containing(key) click to toggle source
# File lib/carton_db/list_map_db.rb, line 333
def segment_containing(key)
  ListMapDb::Segment.in_db_for_hashcode(
    name, key.storage_hashcode
  )
end
write_key_elements(key_d, elements, to_io) click to toggle source
# File lib/carton_db/list_map_db.rb, line 345
def write_key_elements(key_d, elements, to_io)
  count = 0
  elements.each do |element|
    element_d = CartonDb::Datum.for_plain(element)
    count += 1
    to_io << "#{key_d.escaped}\t#{element_d.escaped}\n"
  end
  count
end