class FasterS3Url::Builder

Signing algorithm based on Amazon docs at docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html , as well as some interactive code reading of Aws::Sigv4::Signer github.com/aws/aws-sdk-ruby/blob/6114bc9692039ac75c8292c66472dacd14fa6f9a/gems/aws-sigv4/lib/aws-sigv4/signer.rb as used by Aws::S3::Presigner github.com/aws/aws-sdk-ruby/blob/6114bc9692039ac75c8292c66472dacd14fa6f9a/gems/aws-sdk-s3/lib/aws-sdk-s3/presigner.rb

Constants

ALGORITHM
DEFAULT_EXPIRES_IN
FIFTEEN_MINUTES
MAX_CACHED_SIGNING_KEYS
METHOD
ONE_WEEK
SERVICE
SIGNED_HEADERS

Attributes

access_key_id[R]
bucket_name[R]
host[R]
region[R]

Public Class Methods

new(bucket_name:, region:, access_key_id:, secret_access_key:, host:nil, default_public: true, cache_signing_keys: false) click to toggle source

@option params [String] :bucket_name required

@option params [String] :region eg “us-east-1”, required

@option params :host optional, host to use in generated URLs. If empty, will construct default AWS S3 host for bucket name and region.

@option params [String] :access_key_id required at present, change to allow look up from environment using standard aws sdk routines?

@option params [String] :secret_access_key required at present, change to allow look up from environment using standard aws sdk routines?

@option params [boolean] :default_public (true) default value of `public` when instance method url is called.

@option params [boolean] :cache_signing_keys (false). If set to true, up to five signing keys used for presigned URLs will

be cached and re-used, improving performance when generating mulitple presigned urls with a single Builder by around 50%.
NOTE WELL: This will make the Builder no longer technically concurrency-safe for sharing between multiple threads, is one
reason it is not on by default.
# File lib/faster_s3_url/builder.rb, line 41
def initialize(bucket_name:, region:, access_key_id:, secret_access_key:, host:nil, default_public: true, cache_signing_keys: false)
  @bucket_name = bucket_name
  @region = region
  @host = host || default_host(bucket_name)
  @default_public = default_public
  @access_key_id = access_key_id
  @secret_access_key = secret_access_key
  @cache_signing_keys = cache_signing_keys
  if @cache_signing_keys
    @signing_key_cache = {}
  end


  @canonical_headers = "host:#{@host}\n"
end

Public Instance Methods

presigned_url(key, time: nil, expires_in: DEFAULT_EXPIRES_IN, response_cache_control: nil, response_content_disposition: nil, response_content_encoding: nil, response_content_language: nil, response_content_type: nil, response_expires: nil, version_id: nil) click to toggle source

Generates a presigned GET URL for a specified S3 object key.

@param [String] key The S3 key to create a URL pointing to.

@option params [Time] :time (Time.now) The starting time for when the

presigned url becomes active.

@option params [String] :response_cache_control

Adds a `response-cache-control` query param to set the `Cache-Control` header of the subsequent response from S3.

@option params [String] :response_content_disposition

Adds a `response-content-disposition` query param to set the `Content-Disposition` header of the subsequent response from S3

@option params [String] :response_content_encoding

Adds a `response-content-encoding` query param to set `Content-Encoding` header of the subsequent response from S3

@option params [String] :response_content_language

Adds a `response-content-language` query param to sets the `Content-Language` header of the subsequent response from S3

@option params [String] :response_content_type

Adds a `response-content-type` query param to sets the `Content-Type` header of the subsequent response from S3

@option params [String] :response_expires

Adds a `response-expires` query param to sets the `Expires` header of of the subsequent response from S3

@option params [String] :version_id

Adds a `versionId` query param to reference a specific version of the object from S3.
# File lib/faster_s3_url/builder.rb, line 88
def presigned_url(key, time: nil, expires_in: DEFAULT_EXPIRES_IN,
                    response_cache_control: nil,
                    response_content_disposition: nil,
                    response_content_encoding: nil,
                    response_content_language: nil,
                    response_content_type: nil,
                    response_expires: nil,
                    version_id: nil)
  validate_expires_in(expires_in)

  canonical_uri = "/" + uri_escape_key(key)

  now = time ? time.dup.utc : Time.now.utc # Uh Time#utc is mutating, not nice to do to an argument!
  amz_date  = now.strftime("%Y%m%dT%H%M%SZ")
  datestamp = now.strftime("%Y%m%d")

  credential_scope = datestamp + '/' + region + '/' + SERVICE + '/' + 'aws4_request'

  canonical_query_string_parts = [
      "X-Amz-Algorithm=#{ALGORITHM}",
      "X-Amz-Credential=" + uri_escape(@access_key_id + "/" + credential_scope),
      "X-Amz-Date=" + amz_date,
      "X-Amz-Expires=" + expires_in.to_s,
      "X-Amz-SignedHeaders=" + SIGNED_HEADERS,
    ]

  extra_params = {
    :"response-cache-control" => response_cache_control,
    :"response-content-disposition" => response_content_disposition,
    :"response-content-encoding" => response_content_encoding,
    :"response-content-language" => response_content_language,
    :"response-content-type" => response_content_type,
    :"response-expires" => convert_for_timestamp_shape(response_expires),
    :"versionId" => version_id
  }.compact


  if extra_params.size > 0
    # These have to be sorted, but sort is case-sensitive, and we have a fixed
    # list of headers we know might be here... turns out they are already sorted?
    extra_param_parts = extra_params.collect {|k, v| "#{k}=#{uri_escape v}" }.join("&")
    canonical_query_string_parts << extra_param_parts
  end

  canonical_query_string = canonical_query_string_parts.join("&")



  canonical_request = ["GET",
    canonical_uri,
    canonical_query_string,
    @canonical_headers,
    SIGNED_HEADERS,
    'UNSIGNED-PAYLOAD'
  ].join("\n")

  string_to_sign = [
    ALGORITHM,
    amz_date,
    credential_scope,
    Digest::SHA256.hexdigest(canonical_request)
  ].join("\n")

  signing_key = retrieve_signing_key(datestamp)
  signature = OpenSSL::HMAC.hexdigest("SHA256", signing_key, string_to_sign)

  return "https://" + self.host + canonical_uri + "?" + canonical_query_string + "&X-Amz-Signature=" + signature
end
public_url(key) click to toggle source
# File lib/faster_s3_url/builder.rb, line 57
def public_url(key)
  "https://#{self.host}/#{uri_escape_key(key)}"
end
url(key, public: @default_public, **options) click to toggle source

just a convenience method that can call public_url or presigned_url based on flag

  signer.url(object_key, public: true)
    #=> forwards to signer.public_url(object_key)

  signer.url(object_key, public: false, response_content_type: "image/jpeg")
     #=> forwards to signer.presigned_url(object_key, response_content_type: "image/jpeg")

Options (sucn as response_content_type) that are not applicable to #public_url
are ignored in public mode.

The default value of `public` can be set by initializer arg `default_public`, which
is itself default true.

    builder = FasterS3Url::Builder.new(..., default_public: false)
    builder.url(object_key) # will call #presigned_url
# File lib/faster_s3_url/builder.rb, line 173
def url(key, public: @default_public, **options)
  if public
    public_url(key)
  else
    presigned_url(key, **options)
  end
end

Private Instance Methods

aws_get_signature_key(key, date_stamp, region_name, service_name) click to toggle source

`def get_signature_key` `from python example at docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html

# File lib/faster_s3_url/builder.rb, line 245
def aws_get_signature_key(key, date_stamp, region_name, service_name)
  k_date = aws_sign("AWS4" + key, date_stamp)
  k_region = aws_sign(k_date, region_name)
  k_service = aws_sign(k_region, service_name)
  aws_sign(k_service, "aws4_request")
end
aws_sign(key, data) click to toggle source

`def sign` from python example at docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html

# File lib/faster_s3_url/builder.rb, line 253
def aws_sign(key, data)
  OpenSSL::HMAC.digest("SHA256", key, data)
end
convert_for_timestamp_shape(arg) click to toggle source

Crazy kind of reverse engineered from aws-sdk-ruby, for compatible handling of Expires header.

This honestly seems to violate the HTTP spec, the result will be that for an `response-expires` param, subsequent S3 response will include an Expires header in ISO8601 instead of HTTP-date format.… but for now we'll make our tests pass by behaving equivalently to aws-sdk-s3 anyway? filed with aws-sdk-s3: github.com/aws/aws-sdk-ruby/issues/2415

Switch last line from `.utc.iso8601` to `.httpdate` if you want to be more correct than aws-sdk-s3?

# File lib/faster_s3_url/builder.rb, line 276
def convert_for_timestamp_shape(arg)
  return nil if arg.nil?

  time_value = case arg
    when Time
      arg
    when Date, DateTime
      arg.to_time
    when Integer, Float
      Time.at(arg)
    else
      Time.parse(arg.to_s)
  end
  time_value.utc.iso8601
end
default_host(bucket_name) click to toggle source
# File lib/faster_s3_url/builder.rb, line 235
def default_host(bucket_name)
  if region == "us-east-1"
    # use legacy one without region, as S3 seems to
    "#{bucket_name}.s3.amazonaws.com".freeze
  else
    "#{bucket_name}.s3.#{region}.amazonaws.com".freeze
  end
end
make_signing_key(datestamp) click to toggle source
# File lib/faster_s3_url/builder.rb, line 184
def make_signing_key(datestamp)
  aws_get_signature_key(@secret_access_key, datestamp, @region, SERVICE)
end
retrieve_signing_key(datestamp) click to toggle source

If caching of signing keys is turned on, use and cache signing key, while making sure not to cache more than MAX_CACHED_SIGNING_KEYS

Otherwise if caching of signing keys is not turned on, just generate and return a signing key.

# File lib/faster_s3_url/builder.rb, line 193
def retrieve_signing_key(datestamp)
  if @cache_signing_keys
    if value = @signing_key_cache[datestamp]
      value
    else
      value = @signing_key_cache[datestamp] =  make_signing_key(datestamp)
      while @signing_key_cache.size > MAX_CACHED_SIGNING_KEYS
        @signing_key_cache.delete(@signing_key_cache.keys.first)
      end
      value
    end
  else
    make_signing_key(datestamp)
  end
end
uri_escape(string) click to toggle source

Becaues CGI.escape in MRI is written in C, this really does seem to be the fastest way to get the semantics we want, starting with CGI.escape and doing extra gsubs. Alternative would be using something else in pure C that has the semantics we want, but does not seem available.

# File lib/faster_s3_url/builder.rb, line 214
def uri_escape(string)
  if string.nil?
    nil
  else
    CGI.escape(string.encode('UTF-8')).gsub('+', '%20').gsub('%7E', '~')
  end
end
uri_escape_key(string) click to toggle source

like uri_escape but does NOT escape `/`, leaves it alone. The appropriate escaping algorithm for an S3 key turning into a URL.

Faster to un-DRY the code with uri_escape. Yes, faster to actually just gsub %2F back to /

# File lib/faster_s3_url/builder.rb, line 227
def uri_escape_key(string)
  if string.nil?
    nil
  else
    CGI.escape(string.encode('UTF-8')).gsub('+', '%20').gsub('%7E', '~').gsub("%2F", "/")
  end
end
validate_expires_in(expires_in) click to toggle source
# File lib/faster_s3_url/builder.rb, line 257
def validate_expires_in(expires_in)
  if expires_in > ONE_WEEK
    raise ArgumentError.new("expires_in value of #{expires_in} exceeds one-week maximum.")
  elsif expires_in <= 0
    raise ArgumentError.new("expires_in value of #{expires_in} cannot be 0 or less.")
  end
end