module S3MediaServerApi::Uploader
Module with file uplod finctionality implementation
Public Class Methods
upload(file_path)
click to toggle source
uploads file to amazon s3 and create AwsFile
object parameter : filepath - file path in file system returns: [AwsFile object]
Example
file = S3MediaServerApi::Uploader.upload(/home/vasya/my_awesome_file.awesome)
# File lib/s3_media_server_api/uploader.rb, line 32 def upload(file_path) parts = [] file = AwsFile.create_from_path(file_path) default_part_size = file.default_part_size aws_file_uuid = file.uuid uploads_count = file.uploads_count parts = compute_parts(file_path, default_part_size) Parallel.each(parts, in_threads: S3MediaServerApi::Config.upload_thread_count) do |part| signed_upload_url = AwsFile.get_signed_upload_url(aws_file_uuid, part[:part_number]) raise PartUploadError.new("Part #{part[:part_number]} wasn't uploaded") unless upload_part(signed_upload_url, part[:body].read) end AwsFile.complete_upload(aws_file_uuid) ensure close_all_parts(parts) end
upload_from_url(url)
click to toggle source
# File lib/s3_media_server_api/uploader.rb, line 17 def upload_from_url(url) file = S3MediaServerApi::Downloader.download_by_url(url) upload(file) ensure S3MediaServerApi::Downloader.remove_if_exists(file) end
Private Class Methods
close_all_parts(parts)
click to toggle source
closes all parts of file that were used in multipart upload to prevent memory leak
# File lib/s3_media_server_api/uploader.rb, line 54 def close_all_parts(parts) parts.each do |part| part[:body].close end end
compute_parts(source, default_part_size)
click to toggle source
divides file into parts for multipart upload parameter:
source - path of source file default_part_size - wanted part size (in bytes)
returns: [ array of parts ]
# File lib/s3_media_server_api/uploader.rb, line 66 def compute_parts(source, default_part_size) size = File.size(source) offset, part_number, parts = 0, 1, [] while offset < size parts << { part_number: part_number, body: FilePart.new(source: source, offset: offset, size: part_size(size, default_part_size, offset)) } part_number += 1 offset += default_part_size end parts end
part_size(total_size, part_size, offset)
click to toggle source
calculates size of one part - last part often has smaller size than other parts parameters: total_size - totol size of file (in bytes)
part_size - default size of one part (in bytes) offset - offset from beginning of file (in bytes)
returns: part size
# File lib/s3_media_server_api/uploader.rb, line 87 def part_size(total_size, part_size, offset) if offset + part_size > total_size total_size - offset else part_size end end
upload_part(url, data)
click to toggle source
uploads data to specified url parameters: url - upload url
data - data to upload
returns: true if upload was successful
false otherwise
# File lib/s3_media_server_api/uploader.rb, line 101 def upload_part(url, data) conn = Faraday.new(url: url) do |faraday| faraday.adapter :net_http end resp = conn.put do |req| req.body = data # to prevent Faraday from adding garbage header req.headers['Content-Type'] = '' end resp.success? end