class DbBlaster::Configuration

Configuration class for providing credentials, topics, and customizations. Either the `sns_topic` or `s3_bucket' must be set

Constants

ATTRIBUTE_S3_META_FORMAT
DEFAULT_BATCH_SIZE
DEFAULT_DATETIME_FORMAT
DEFAULT_MAX_MESSAGE_SIZE_IN_KILOBYTES
DEFAULT_S3_KEY
EITHER_OR_FIELDS

exactly one of these fields needs to be set

INLINE_S3_META_FORMAT
REQUIRED_FIELDS

The required configuration fields

Attributes

aws_access_key[RW]
aws_access_secret[RW]
aws_region[RW]
batch_size[RW]

Optional db_blaster will select and then publish `batch_size` rows at a time Default value is 100

extra_sns_message_attributes[RW]

Optional Applicable only when `sns_topic` is set Extra [SNS message_attributes](docs.aws.amazon.com/sns/latest/dg/sns-message-attributes.html) Attributes set here will be included in every published message example: config.extra_sns_message_attributes = {'infra_id' => {data_type: 'String', value: '061'}}

ignore_source_tables[RW]

Optional If set, ignore source tables specified. example: config.ignore_source_tables = ['active_storage_blobs']

ignored_column_names[RW]

Global list of column names not to include in published SNS messages example: config.ignored_column_names = ['email', 'phone_number']

max_message_size_in_kilobytes[RW]

Optional DbBlaster will publish no messages larger than this value Default value is 256

only_source_tables[RW]

Optional If set, only publish tables specified. example: config.only_source_tables = ['posts', 'tags', 'comments']

s3_bucket[RW]

The s3 bucket name

s3_key[RW]

Optional Applicable only when `s3_bucket` is set The S3 key. The following values will get substituted: <batch_date_time> - date time when batch started <batch_date> - date when batch started <batch_time - time when batch started <date_time> - the datetime just before pushing to S3 <table_name> - the name of the table associated with the S3 body <uuid> - a universal identifier '<batch_timestamp>/kcp-api/001/<table_name>/<uuid>.json'

s3_meta[RW]

Optional Applicable only when `s3_bucket' is set The value set here will be included in every payload pushed to S3 example: config.s3_meta = {'infra_id' => '061', 'source_app' => 'kcp-api'}}

s3_meta_format[RW]

Optional Options: ['attribute', 'inline'] Defaults to 'attribute' 'attribute' payload: { meta: `s3_meta`, records: [source_table_records] } 'inline' payload: records.collect{|record| record.merge(meta) }

s3_tags[RW]

Optional Applicable only when `s3_bucket` is set S3 Tags example: config.s3_tags = { infra_id: '001', source_app: 'kcp-api', source_table: 'meetings' }

sns_topic[RW]

The topic to which messages will be published

source_table_options[RW]

Optional Customize batch_size and/or ignored_columns example: config.source_table_options = [{ source_table_name: 'posts', batch_size: 100, ignored_column_names: ['email'] },

{ source_table_name: 'comments', ignored_column_names: ['tags'] }]

Public Instance Methods

verify!() click to toggle source

Raises error if a required field is not set

# File lib/db_blaster/configuration.rb, line 96
def verify!
  verify_required
  verify_either_or
end
verify_either_or() click to toggle source
# File lib/db_blaster/configuration.rb, line 101
def verify_either_or
  either_or = EITHER_OR_FIELDS.select do |attribute|
    send(attribute).nil? || send(attribute).strip.empty?
  end
  raise "only one of [#{either_or.join(', ')}] should be set" unless either_or.length == 1
end
verify_required() click to toggle source
# File lib/db_blaster/configuration.rb, line 108
def verify_required
  no_values = REQUIRED_FIELDS.select do |attribute|
    send(attribute).nil? || send(attribute).strip.empty?
  end
  raise "missing configuration values for [#{no_values.join(', ')}]" unless no_values.empty?
end