class DbBlaster::Configuration
Configuration
class for providing credentials, topics, and customizations. Either the `sns_topic` or `s3_bucket' must be set
Constants
- ATTRIBUTE_S3_META_FORMAT
- DEFAULT_BATCH_SIZE
- DEFAULT_DATETIME_FORMAT
- DEFAULT_MAX_MESSAGE_SIZE_IN_KILOBYTES
- DEFAULT_S3_KEY
- EITHER_OR_FIELDS
exactly one of these fields needs to be set
- INLINE_S3_META_FORMAT
- REQUIRED_FIELDS
The required configuration fields
Attributes
Optional db_blaster will select and then publish `batch_size` rows at a time Default value is 100
Optional Applicable only when `sns_topic` is set Extra [SNS message_attributes](docs.aws.amazon.com/sns/latest/dg/sns-message-attributes.html) Attributes set here will be included in every published message example: config.extra_sns_message_attributes = {'infra_id' => {data_type: 'String', value: '061'}}
Optional If set, ignore source tables specified. example: config.ignore_source_tables = ['active_storage_blobs']
Global list of column names not to include in published SNS messages example: config.ignored_column_names = ['email', 'phone_number']
Optional DbBlaster
will publish no messages larger than this value Default value is 256
Optional If set, only publish tables specified. example: config.only_source_tables = ['posts', 'tags', 'comments']
The s3 bucket name
Optional Applicable only when `s3_bucket` is set The S3 key. The following values will get substituted: <batch_date_time> - date time when batch started <batch_date> - date when batch started <batch_time - time when batch started <date_time> - the datetime just before pushing to S3 <table_name> - the name of the table associated with the S3 body <uuid> - a universal identifier '<batch_timestamp>/kcp-api/001/<table_name>/<uuid>.json'
Optional Applicable only when `s3_bucket' is set The value set here will be included in every payload pushed to S3 example: config.s3_meta = {'infra_id' => '061', 'source_app' => 'kcp-api'}}
Optional Options: ['attribute', 'inline'] Defaults to 'attribute' 'attribute' payload: { meta: `s3_meta`, records: [source_table_records] } 'inline' payload: records.collect{|record| record.merge(meta) }
The topic to which messages will be published
Optional Customize batch_size
and/or ignored_columns example: config.source_table_options = [{ source_table_name: 'posts', batch_size
: 100, ignored_column_names
: ['email'] },
{ source_table_name: 'comments', ignored_column_names: ['tags'] }]
Public Instance Methods
Raises error if a required field is not set
# File lib/db_blaster/configuration.rb, line 96 def verify! verify_required verify_either_or end
# File lib/db_blaster/configuration.rb, line 101 def verify_either_or either_or = EITHER_OR_FIELDS.select do |attribute| send(attribute).nil? || send(attribute).strip.empty? end raise "only one of [#{either_or.join(', ')}] should be set" unless either_or.length == 1 end
# File lib/db_blaster/configuration.rb, line 108 def verify_required no_values = REQUIRED_FIELDS.select do |attribute| send(attribute).nil? || send(attribute).strip.empty? end raise "missing configuration values for [#{no_values.join(', ')}]" unless no_values.empty? end