class GitLab::Exporter::Database::RowCountCollector

A helper class that executes the query its given and returns an int of the row count This class works under the assumption you do COUNT(*) queries, define queries in the QUERIES constant. If in doubt how these work, read construct_query rubocop:disable Metrics/ClassLength

Constants

MIRROR_QUERY
QUERIES
WHERE_MIRROR_ENABLED

We ignore mirrors with a next_execution_timestamp before 2020-03-28 because this is when we stopped processing mirrors for private projects on the free plan. Skipping those can significantly improve query performance: gitlab.com/gitlab-org/gitlab/-/issues/216252#note_334514544

Public Class Methods

new(selected_queries: nil, **args) click to toggle source
Calls superclass method GitLab::Exporter::Database::Base::new
# File lib/gitlab_exporter/database/row_count.rb, line 137
def initialize(selected_queries: nil, **args)
  super(**args)

  @selected_queries = Set.new(selected_queries.map(&:to_sym)) unless selected_queries.nil?
end

Public Instance Methods

run() click to toggle source
# File lib/gitlab_exporter/database/row_count.rb, line 143
def run
  results = Hash.new(0)

  QUERIES.each do |key, query_hash|
    next if query_hash[:check] && !successful_check?(query_hash[:check])
    next if !@selected_queries.nil? && !@selected_queries.include?(key)

    results[key] = count_from_query_hash(query_hash)
  end

  results
end

Private Instance Methods

construct_query(query) click to toggle source

Not private so I can test it without meta programming tricks

# File lib/gitlab_exporter/database/row_count.rb, line 185
def construct_query(query)
  query_string = "SELECT COUNT(*)"
  (query[:fields] || []).each do |key, value|
    query_string << ", "
    query_string << "(#{value[:definition]}) AS " if value[:definition]
    query_string << key.to_s
  end
  query_string << " FROM #{query[:select]}"
  query_string << " #{query[:joins]}"       if query[:joins]
  query_string << " WHERE #{query[:where]}" if query[:where]
  query_string << " GROUP BY " + query[:fields].keys.join(", ") if query[:fields]
  query_string << ";"
end
count_from_query_hash(query_hash) click to toggle source
# File lib/gitlab_exporter/database/row_count.rb, line 158
def count_from_query_hash(query_hash)
  result = execute(construct_query(query_hash))
  return [{ "count": 0, "labels": {} }] unless result

  result.map do |row|
    labels = {}
    (query_hash[:fields] || []).each do |key, _| labels[key] = row[key.to_s] end
    { "count": row["count"], "labels": labels }
  end
end
execute(query) click to toggle source
# File lib/gitlab_exporter/database/row_count.rb, line 176
def execute(query)
  with_connection_pool do |conn|
    conn.exec(query)
  end
rescue PG::UndefinedTable, PG::UndefinedColumn
  nil
end
successful_check?(query) click to toggle source
# File lib/gitlab_exporter/database/row_count.rb, line 169
def successful_check?(query)
  result = execute("SELECT EXISTS (#{query})")
  return unless result

  result[0]["exists"]
end