class FReCon::Scraper

The default scraper scrapes other FReCon instances. To scrape a different source, a descendant scraper should be used.

Public Class Methods

new(base_uri) click to toggle source
# File lib/frecon/scraper.rb, line 17
def initialize(base_uri)
        @base_uri = base_uri
end

Public Instance Methods

get(model = nil, query = {}) click to toggle source

If no arguments are passed, will import the whole other database. If only one argument is passed, will import all of that model. If two arguments are passed, will import the models that match the query params.

# File lib/frecon/scraper.rb, line 59
def get(model = nil, query = {})
        # Turns something like 'team' into Team.
        model = ('FReCon::' + model.capitalize).constantize if model.is_a?(String)

        # The route name for the model branch.
        route_name = model.name.gsub(/FReCon::/, '').downcase.pluralize if model

        if !model && query.empty?
                type = :dump
                data = HTTP.get("http://#{@base_uri}/dump")
        elsif model && query.empty?
                type = :index
                data = HTTP.get("http://#{@base_uri}/#{route_name}")
        else
                type = :single
                data = HTTP.get("http://#{@base_uri}/#{route_name}", { query: query })
        end

        read data.body, model: model, type: type
end
read(data, context = {}) click to toggle source

Reads and imports a data string. Determines what to do with information in the `context` hash.

# File lib/frecon/scraper.rb, line 23
def read(data, context = {})
        # `data` will be a string, so we need to convert it from JSON.
        data = JSON.parse(data)

        if context[:type] == :single && data.empty?
                return [404, 'Could not find a model with that query.']
        elsif
                puts 'Just a heads up: you are importing an empty array of data.'
        end

        # Here we want `context` to tell us what model we are making.
        if context[:model]
                result = context[:model].controller.create(nil, nil, data)
                result.first == 201 ? result.first : JSON.parse(result.last)
        else
                # Therefore, we must be dealing with a dump.
                statuses = data.map do |key, value|
                        begin
                                unless value.empty?
                                        model = ('FReCon::' + key.singularize.capitalize).constantize
                                        result = model.controller.create(nil, nil, value)
                                        result.first == 201 ? result.first : JSON.parse(result.last)
                                end
                        rescue Moped::Errors::OperationFailure => e
                                RequestError.new(422, "A model already exists in your database with the key you are trying to import.\nPerhaps you should clear your database?").return_value
                        end
                end
                statuses.delete(nil)
                statuses
        end
end