class Innodb::DataDictionary

Constants

COLUMN_MTYPE

A hash of InnoDB’s internal type system to the values stored for each type.

COLUMN_MTYPE_BY_VALUE

A hash of COLUMN_MTYPE keys by value.

COLUMN_PRTYPE_FLAG

A hash of InnoDB ‘precise type’ bitwise flags.

COLUMN_PRTYPE_FLAG_BY_VALUE

A hash of COLUMN_PRTYPE keys by value.

COLUMN_PRTYPE_MYSQL_TYPE_MASK

The bitmask to extract the MySQL internal type from the InnoDB ‘precise type’.

DATA_DICTIONARY_RECORD_DESCRIBERS

A hash of hashes of table name and index name to describer class.

INDEX_TYPE_FLAG

A hash of InnoDB’s index type flags.

INDEX_TYPE_FLAG_BY_VALUE

A hash of INDEX_TYPE_FLAG keys by value.

MYSQL_TYPE

A hash of MySQL’s internal type system to the stored values for those types, and the ‘external’ SQL type. rubocop:disable Layout/HashAlignment rubocop:disable Layout/CommentIndentation

MYSQL_TYPE_BY_VALUE

A hash of MYSQL_TYPE keys by value :value key.

MysqlType

Attributes

system_space[R]

Public Class Methods

mtype_prtype_to_data_type(mtype, prtype, len, prec) click to toggle source

Return a full data type given an mtype and prtype, such as [‘VARCHAR(10)’, :NOT_NULL] or [:INT, :UNSIGNED].

# File lib/innodb/data_dictionary.rb, line 214
def self.mtype_prtype_to_data_type(mtype, prtype, len, prec)
  type = mtype_prtype_to_type_string(mtype, prtype, len, prec)
  raise "Unsupported type (mtype #{mtype}, prtype #{prtype})" unless type

  data_type = [type]
  data_type << :NOT_NULL if prtype & COLUMN_PRTYPE_FLAG[:NOT_NULL] != 0
  data_type << :UNSIGNED if prtype & COLUMN_PRTYPE_FLAG[:UNSIGNED] != 0

  data_type
end
mtype_prtype_to_type_string(mtype, prtype, len, prec) click to toggle source

Return the ‘external’ SQL type string (such as ‘VARCHAR’ or ‘INT’) given the stored mtype and prtype from the InnoDB data dictionary. Note that not all types are extractable into fully defined SQL types due to the lossy nature of the MySQL-to-InnoDB interface regarding types.

# File lib/innodb/data_dictionary.rb, line 174
def self.mtype_prtype_to_type_string(mtype, prtype, len, prec)
  mysql_type = prtype & COLUMN_PRTYPE_MYSQL_TYPE_MASK
  internal_type = MYSQL_TYPE_BY_VALUE[mysql_type]
  external_type = MYSQL_TYPE[internal_type].type

  case external_type
  when :VARCHAR
    # One-argument: length.
    "%s(%i)" % [external_type, len]
  when :FLOAT, :DOUBLE
    # Two-argument: length and precision.
    "%s(%i,%i)" % [external_type, len, prec]
  when :CHAR
    if COLUMN_MTYPE_BY_VALUE[mtype] == :MYSQL
      # When the mtype is :MYSQL, the column is actually
      # stored as VARCHAR despite being a CHAR. This is
      # done for CHAR columns having multi-byte character
      # sets in order to limit size. Note that such data
      # are still space-padded to at least len.
      "VARCHAR(%i)" % [len]
    else
      "CHAR(%i)" % [len]
    end
  when :DECIMAL
    # The DECIMAL type is designated as DECIMAL(M,D)
    # however the M and D definitions are not stored
    # in the InnoDB data dictionary. We need to define
    # the column as something which will extract the
    # raw bytes in order to read the column, but we
    # can't figure out the right decimal type. The
    # len stored here is actually the on-disk storage
    # size.
    "CHAR(%i)" % [len]
  else
    external_type
  end
end
new(system_space) click to toggle source
# File lib/innodb/data_dictionary.rb, line 227
def initialize(system_space)
  @system_space = system_space
end

Public Instance Methods

_make_column_description(type, record) click to toggle source

Produce a Innodb::RecordDescriber-compatible column description given a type (:key, :row) and data dictionary SYS_COLUMNS record.

# File lib/innodb/data_dictionary.rb, line 548
def _make_column_description(type, record)
  {
    type: type,
    name: record["NAME"],
    description: self.class.mtype_prtype_to_data_type(
      record["MTYPE"],
      record["PRTYPE"],
      record["LEN"],
      record["PREC"]
    ),
  }
end
clustered_index_name_by_table_name(table_name) click to toggle source

Return the name of the clustered index (usually ‘PRIMARY’, but not always) for a given table name.

# File lib/innodb/data_dictionary.rb, line 538
def clustered_index_name_by_table_name(table_name)
  table_record = table_by_name(table_name)
  raise "Table #{table_name} not found" unless table_record

  index_record = object_by_two_fields(:each_index, "TABLE_ID", table_record["ID"], "TYPE", 3)
  index_record["NAME"] if index_record
end
column_by_name(table_name, column_name) click to toggle source

Lookup a column by table name and column name.

# File lib/innodb/data_dictionary.rb, line 410
def column_by_name(table_name, column_name)
  table = table_by_name(table_name)
  return unless table

  object_by_two_fields(:each_column, "TABLE_ID", table["ID"], "NAME", column_name)
end
data_dictionary_index(table_name, index_name) click to toggle source

Return an Innodb::Index object initialized to the internal data dictionary index with an appropriate record describer so that records can be recursed.

# File lib/innodb/data_dictionary.rb, line 284
def data_dictionary_index(table_name, index_name)
  raise "Data Dictionary not found; is the MySQL version supported?" unless found?

  table_entry = data_dictionary_indexes[table_name]
  raise "Unknown data dictionary table #{table_name}" unless table_entry

  index_root_page = table_entry[index_name]
  raise "Unknown data dictionary index #{table_name}.#{index_name}" unless index_root_page

  # If we have a record describer for this index, load it.
  record_describer = data_dictionary_index_describer(table_name, index_name)

  system_space.index(index_root_page, record_describer)
end
data_dictionary_index?(table_name, index_name) click to toggle source
# File lib/innodb/data_dictionary.rb, line 269
def data_dictionary_index?(table_name, index_name)
  return false unless data_dictionary_table?(table_name)

  DATA_DICTIONARY_RECORD_DESCRIBERS[table_name.to_sym].include?(index_name.to_sym)
end
data_dictionary_index_describer(table_name, index_name) click to toggle source
# File lib/innodb/data_dictionary.rb, line 275
def data_dictionary_index_describer(table_name, index_name)
  return unless data_dictionary_index?(table_name, index_name)

  DATA_DICTIONARY_RECORD_DESCRIBERS[table_name.to_sym][index_name.to_sym].new
end
data_dictionary_index_ids() click to toggle source
# File lib/innodb/data_dictionary.rb, line 243
def data_dictionary_index_ids
  raise "Data Dictionary not found; is the MySQL version supported?" unless found?

  return @data_dictionary_index_ids if @data_dictionary_index_ids

  # TODO: This could probably be done a lot more Ruby-like.
  @data_dictionary_index_ids = {}
  data_dictionary_indexes.each do |table, indexes|
    indexes.each do |index, root_page_number|
      root_page = system_space.page(root_page_number)
      next unless root_page

      @data_dictionary_index_ids[root_page.index_id] = {
        table: table,
        index: index,
      }
    end
  end

  @data_dictionary_index_ids
end
data_dictionary_indexes() click to toggle source

A helper method to reach inside the system space and retrieve the data dictionary index locations from the data dictionary header.

# File lib/innodb/data_dictionary.rb, line 234
def data_dictionary_indexes
  system_space.data_dictionary_page.data_dictionary_header[:indexes]
end
data_dictionary_table?(table_name) click to toggle source
# File lib/innodb/data_dictionary.rb, line 265
def data_dictionary_table?(table_name)
  DATA_DICTIONARY_RECORD_DESCRIBERS.include?(table_name.to_sym)
end
each_column() { |fields| ... } click to toggle source

Iterate through the records in the SYS_COLUMNS data dictionary table.

# File lib/innodb/data_dictionary.rb, line 350
def each_column
  return enum_for(:each_column) unless block_given?

  each_record_from_data_dictionary_index(:SYS_COLUMNS, :PRIMARY) do |record|
    yield record.fields
  end

  nil
end
each_column_by_table_id(table_id) { |record| ... } click to toggle source

Iterate through all columns in a table by table ID.

# File lib/innodb/data_dictionary.rb, line 488
def each_column_by_table_id(table_id)
  return enum_for(:each_column_by_table_id, table_id) unless block_given?

  each_column do |record|
    yield record if record["TABLE_ID"] == table_id
  end

  nil
end
each_column_by_table_name(table_name, &block) click to toggle source

Iterate through all columns in a table by table name.

# File lib/innodb/data_dictionary.rb, line 499
def each_column_by_table_name(table_name, &block)
  return enum_for(:each_column_by_table_name, table_name) unless block_given?
  raise "Table #{table_name} not found" unless (table = table_by_name(table_name))

  each_column_by_table_id(table["ID"], &block)

  nil
end
each_column_description_by_index_name(table_name, index_name) { |_make_column_description(:key, record)| ... } click to toggle source

Iterate through Innodb::RecordDescriber-compatible column descriptions for a given index by table name and index name.

# File lib/innodb/data_dictionary.rb, line 563
def each_column_description_by_index_name(table_name, index_name)
  return enum_for(:each_column_description_by_index_name, table_name, index_name) unless block_given?

  unless (index = index_by_name(table_name, index_name))
    raise "Index #{index_name} for table #{table_name} not found"
  end

  columns_in_index = {}
  each_column_in_index_by_name(table_name, index_name) do |record|
    columns_in_index[record["NAME"]] = 1
    yield _make_column_description(:key, record)
  end

  if (index["TYPE"] & INDEX_TYPE_FLAG[:CLUSTERED]).zero?
    clustered_index_name = clustered_index_name_by_table_name(table_name)

    each_column_in_index_by_name(table_name, clustered_index_name) do |record|
      yield _make_column_description(:row, record)
    end
  else
    each_column_by_table_name(table_name) do |record|
      yield _make_column_description(:row, record) unless columns_in_index.include?(record["NAME"])
    end
  end

  nil
end
each_column_in_index_by_name(table_name, index_name) { |column_by_name(table_name, record)| ... } click to toggle source

Iterate through all columns in an index by table name and index name.

# File lib/innodb/data_dictionary.rb, line 509
def each_column_in_index_by_name(table_name, index_name)
  return enum_for(:each_column_in_index_by_name, table_name, index_name) unless block_given?

  each_field_by_index_name(table_name, index_name) do |record|
    yield column_by_name(table_name, record["COL_NAME"])
  end

  nil
end
each_column_not_in_index_by_name(table_name, index_name) { |record| ... } click to toggle source

Iterate through all columns not in an index by table name and index name. This is useful when building index descriptions for secondary indexes.

# File lib/innodb/data_dictionary.rb, line 521
def each_column_not_in_index_by_name(table_name, index_name)
  return enum_for(:each_column_not_in_index_by_name, table_name, index_name) unless block_given?

  columns_in_index = {}
  each_column_in_index_by_name(table_name, index_name) do |record|
    columns_in_index[record["NAME"]] = 1
  end

  each_column_by_table_name(table_name) do |record|
    yield record unless columns_in_index.include?(record["NAME"])
  end

  nil
end
each_data_dictionary_index() { |table_name, index_name, data_dictionary_index(table_name, index_name)| ... } click to toggle source

Iterate through all data dictionary indexes, yielding the table name, index name, and the index itself as an Innodb::Index.

# File lib/innodb/data_dictionary.rb, line 315
def each_data_dictionary_index
  return enum_for(:each_data_dictionary_index) unless block_given?

  data_dictionary_indexes.each do |table_name, indexes|
    indexes.each_key do |index_name|
      yield table_name, index_name,
        data_dictionary_index(table_name, index_name)
    end
  end

  nil
end
each_data_dictionary_index_root_page_number() { |table_name, index_name, root_page_number| ... } click to toggle source

Iterate through all data dictionary indexes, yielding the table name, index name, and root page number.

# File lib/innodb/data_dictionary.rb, line 301
def each_data_dictionary_index_root_page_number
  return enum_for(:each_data_dictionary_index_root_page_number) unless block_given?

  data_dictionary_indexes.each do |table_name, indexes|
    indexes.each do |index_name, root_page_number|
      yield table_name, index_name, root_page_number
    end
  end

  nil
end
each_field() { |fields| ... } click to toggle source

Iterate through the records in the SYS_FIELDS data dictionary table.

# File lib/innodb/data_dictionary.rb, line 372
def each_field
  return enum_for(:each_field) unless block_given?

  each_record_from_data_dictionary_index(:SYS_FIELDS, :PRIMARY) do |record|
    yield record.fields
  end

  nil
end
each_field_by_index_id(index_id) { |record| ... } click to toggle source

Iterate through all fields in an index by index ID.

# File lib/innodb/data_dictionary.rb, line 465
def each_field_by_index_id(index_id)
  return enum_for(:each_field_by_index_id, index_id) unless block_given?

  each_field do |record|
    yield record if record["INDEX_ID"] == index_id
  end

  nil
end
each_field_by_index_name(table_name, index_name, &block) click to toggle source

Iterate through all fields in an index by index name.

# File lib/innodb/data_dictionary.rb, line 476
def each_field_by_index_name(table_name, index_name, &block)
  return enum_for(:each_field_by_index_name, table_name, index_name) unless block_given?

  index = index_by_name(table_name, index_name)
  raise "Index #{index_name} for table #{table_name} not found" unless index

  each_field_by_index_id(index["ID"], &block)

  nil
end
each_index() { |fields| ... } click to toggle source

Iterate through the records in the SYS_INDEXES dictionary table.

# File lib/innodb/data_dictionary.rb, line 361
def each_index
  return enum_for(:each_index) unless block_given?

  each_record_from_data_dictionary_index(:SYS_INDEXES, :PRIMARY) do |record|
    yield record.fields
  end

  nil
end
each_index_by_space_id(space_id) { |record| ... } click to toggle source

Iterate through indexes by space ID.

# File lib/innodb/data_dictionary.rb, line 431
def each_index_by_space_id(space_id)
  return enum_for(:each_index_by_space_id, space_id) unless block_given?

  each_index do |record|
    yield record if record["SPACE"] == space_id
  end

  nil
end
each_index_by_table_id(table_id) { |record| ... } click to toggle source

Iterate through all indexes in a table by table ID.

# File lib/innodb/data_dictionary.rb, line 442
def each_index_by_table_id(table_id)
  return enum_for(:each_index_by_table_id, table_id) unless block_given?

  each_index do |record|
    yield record if record["TABLE_ID"] == table_id
  end

  nil
end
each_index_by_table_name(table_name, &block) click to toggle source

Iterate through all indexes in a table by table name.

# File lib/innodb/data_dictionary.rb, line 453
def each_index_by_table_name(table_name, &block)
  return enum_for(:each_index_by_table_name, table_name) unless block_given?

  table = table_by_name(table_name)
  raise "Table #{table_name} not found" unless table

  each_index_by_table_id(table["ID"], &block)

  nil
end
each_record_from_data_dictionary_index(table, index, &block) click to toggle source

Iterate through records from a data dictionary index yielding each record as a Innodb::Record object.

# File lib/innodb/data_dictionary.rb, line 330
def each_record_from_data_dictionary_index(table, index, &block)
  return enum_for(:each_record_from_data_dictionary_index, table, index) unless block_given?

  data_dictionary_index(table, index).each_record(&block)

  nil
end
each_table() { |fields| ... } click to toggle source

Iterate through the records in the SYS_TABLES data dictionary table.

# File lib/innodb/data_dictionary.rb, line 339
def each_table
  return enum_for(:each_table) unless block_given?

  each_record_from_data_dictionary_index(:SYS_TABLES, :PRIMARY) do |record|
    yield record.fields
  end

  nil
end
found?() click to toggle source

Check if the data dictionary indexes are all available.

# File lib/innodb/data_dictionary.rb, line 239
def found?
  data_dictionary_indexes.values.map(&:values).flatten.none?(&:zero?)
end
index_by_id(index_id) click to toggle source

Lookup an index by index ID.

# File lib/innodb/data_dictionary.rb, line 418
def index_by_id(index_id)
  object_by_field(:each_index, "ID", index_id)
end
index_by_name(table_name, index_name) click to toggle source

Lookup an index by table name and index name.

# File lib/innodb/data_dictionary.rb, line 423
def index_by_name(table_name, index_name)
  table = table_by_name(table_name)
  return unless table

  object_by_two_fields(:each_index, "TABLE_ID", table["ID"], "NAME", index_name)
end
object_by_field(method, field, value) click to toggle source

A helper to iterate the method provided and return the first record where the record’s field matches the provided value.

# File lib/innodb/data_dictionary.rb, line 384
def object_by_field(method, field, value)
  send(method).select { |o| o[field] == value }.first
end
object_by_two_fields(method, field1, value1, field2, value2) click to toggle source

A helper to iterate the method provided and return the first record where the record’s fields f1 and f2 match the provided values v1 and v2.

# File lib/innodb/data_dictionary.rb, line 390
def object_by_two_fields(method, field1, value1, field2, value2)
  send(method).select { |o| o[field1] == value1 && o[field2] == value2 }.first
end
record_describer_by_index_id(index_id) click to toggle source

Return an Innodb::RecordDescriber object describing the records in a given index by index ID.

# File lib/innodb/data_dictionary.rb, line 622
def record_describer_by_index_id(index_id)
  if (dd_index = data_dictionary_index_ids[index_id])
    return data_dictionary_index_describer(dd_index[:table], dd_index[:index])
  end

  unless (index = index_by_id(index_id))
    raise "Index #{index_id} not found"
  end

  unless (table = table_by_id(index["TABLE_ID"]))
    raise "Table #{INDEX['TABLE_ID']} not found"
  end

  record_describer_by_index_name(table["NAME"], index["NAME"])
end
record_describer_by_index_name(table_name, index_name) click to toggle source

Return an Innodb::RecordDescriber object describing records for a given index by table name and index name.

# File lib/innodb/data_dictionary.rb, line 593
def record_describer_by_index_name(table_name, index_name)
  return data_dictionary_index_describer(table_name, index_name) if data_dictionary_index?(table_name, index_name)

  unless (index = index_by_name(table_name, index_name))
    raise "Index #{index_name} for table #{table_name} not found"
  end

  describer = Innodb::RecordDescriber.new

  if (index["TYPE"] & INDEX_TYPE_FLAG[:CLUSTERED]).zero?
    describer.type :secondary
  else
    describer.type :clustered
  end

  each_column_description_by_index_name(table_name, index_name) do |column|
    case column[:type]
    when :key
      describer.key column[:name], *column[:description]
    when :row
      describer.row column[:name], *column[:description]
    end
  end

  describer
end
table_by_id(table_id) click to toggle source

Lookup a table by table ID.

# File lib/innodb/data_dictionary.rb, line 395
def table_by_id(table_id)
  object_by_field(:each_table, "ID", table_id)
end
table_by_name(table_name) click to toggle source

Lookup a table by table name.

# File lib/innodb/data_dictionary.rb, line 400
def table_by_name(table_name)
  object_by_field(:each_table, "NAME", table_name)
end
table_by_space_id(space_id) click to toggle source

Lookup a table by space ID.

# File lib/innodb/data_dictionary.rb, line 405
def table_by_space_id(space_id)
  object_by_field(:each_table, "SPACE", space_id)
end