Amazon S3 Ruby library that leverages EventMachine and em-http-request.
By using EventMachine Happening
does not block on S3 downloads/uploads thus allowing for a higher concurrency.
Happening
was developed by Peritor for usage inside Nanite/EventMachine. Alternatives like RightAws block during the HTTP calls thus blocking the Nanite-Agent.
For now it only supports GET, PUT and DELETE operations on S3 items. The PUT operations support S3 ACLs/permissions. Happening
will handle redirects and retries on errors by default.
Installation¶ ↑
gem install happening
Usage¶ ↑
require 'happening' EM.run do item = Happening::S3::Item.new('bucket', 'item_id') item.get # non-authenticated download, works only for public-read content item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret') item.get # authenticated download item.put("The new content") item.delete end
The above examples are a bit useless, as you never get any content back. You need to specify a callback that interacts with the http response:
EM.run do item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret') item.get do |response| puts "the response content is: #{response.response}" EM.stop end end
This will enqueue your download and run it in the EventMachine event loop.
You can also react to errors:
EM.run do on_error = Proc.new {|response| puts "An error occured: #{response.response_header.status}"; EM.stop } item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret') item.get(:on_error => on_error) do |response| puts "the response content is: #{response.response}" EM.stop end end
If you don’t supply an error handler yourself, Happening
will be default raise an Exception.
Downloading many files could look like this:
EM.run do count = 100 on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop if count <= 0} on_success = Proc.new {|http| puts "the response is: #{http.response}"; EM.stop if count <= 0} count.times do |i| item = Happening::S3::Item.new('bucket', "item_#{i}", :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret') item.get(:on_success => on_success, :on_error => on_error) end end
Upload¶ ↑
Happening
supports the simple S3 PUT upload:
EM.run do on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop } item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret', :on_success => on_success, :on_error => on_error) item.put( File.read('/etc/passwd'), :on_error => on_error ) do |response| puts "Upload finished!"; EM.stop end end
Setting permissions looks like this:
EM.run do on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop } on_success = Proc.new {|http| puts "the response is: #{http.response}"; EM.stop } item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret', :permissions => 'public-write') item.get(:on_success => on_success, :on_error => on_error) end
Custom headers:
EM.run do on_error = Proc.new {|http| puts "An error occured: #{http.response_header.status}"; EM.stop } on_success = Proc.new {|http| puts "the response is: #{http.response}"; EM.stop } item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret', :permissions => 'public-write') item.put(:on_success => on_success, :on_error => on_error, :headers => { 'Cache-Control' => "max-age=252460800", 'Content-Type' => 'text/html', 'Expires' => 'Fri, 16 Nov 2018 22:09:29 GMT', 'x-amz-meta-abc' => 'ABC' }) end
Deleting¶ ↑
Happening
support the simple S3 PUT upload:
EM.run do on_error = Proc.new {|response| puts "An error occured: #{response.response_header.status}"; EM.stop } item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret') item.delete(:on_error => on_error) do |response| puts "Deleted!" EM.stop end end
Amazon returns no content on delete, so having a success handler is usually not needed for delete operations.
Head¶ ↑
You can also just load the headers of an S3 item:
EM.run do on_error = Proc.new {|response| puts "An error occured: #{response.response_header.status}"; EM.stop } item = Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret') item.head(:on_error => on_error) do |response| puts "Headers: #{response.inspect}" EM.stop end end
Streaming¶ ↑
The response data can also be streamed:
EM.run do item = Happening::S3::Item.new( bucket... item.get(:on_error => on_error, :on_success => on_success ).stream do |chunk| # .. handle the individual chunk end end
List Objects in Bucket¶ ↑
EM.run do bucket = Happening::S3::Bucket.new('bucket', :aws_access_key_id => 'Your-ID', :aws_secret_access_key => 'secret') bucket.get do |response| puts "Response content: #{response.response}" end end
SSL Support¶ ↑
Happening
will use SSL/HTTPS by default. What it cannot do by default is verify the SSL certificate. This means that traffic is encrypted but nobody can say if the SSL-endpoint is the one you except. In order to verify the SSL certificate you need to provide Happening
with the path to a certificate CA collection in PEM format:
Happening::S3.ssl_options[:cert_chain_file] = '/etc/ca-bundle.crt'
You can also set this option on each item:
Happening::S3::Item.new('bucket', 'item_id', :aws_access_key_id => 'A', :aws_secret_access_key => 'B', :ssl => { :cert_chain_file => '/etc/ca-bundle.crt' }
Or even on the request:
item.get(:ssl => {:cert_chain_file => '/etc/ca-bundle.crt'})
The SSL options are directly passed to EventMachine, see the EventMachine documentation for more information on the SSL support.
Credits¶ ↑
The AWS signing and canonical request description is based on RightAws.
License¶ ↑
Happening
is licensed under the Apache 2.0 license. See LICENSE.txt
About¶ ↑
Happening
was written by Jonathan Weiss for Peritor.