class PassiveTotal::API
The API
class wraps the PassiveTotal.org web API
for all the verbs that it supports See api.passivetotal.org/api/docs/ for the API
documentation.
Constants
- TLDS
The
TLDS
array helps the interface detect valid domains. This list was generated by parsing the NS records from a zone transfer of the root The same list could have been downloaded from data.iana.org/TLD/tlds-alpha-by-domain.txt
Public Class Methods
initialize a new PassiveTotal::API
object username: the email address associated with your PassiveTotal
API
key. apikey: is 64-hexcharacter string endpoint: base URL for the web service, defaults to api.passivetotal.org/v2/
# File lib/passivetotal/api.rb, line 31 def initialize(username, apikey, endpoint = 'https://api.passivetotal.org/v2/') unless apikey =~ /^[a-fA-F0-9]{64}$/ raise ArgumentError.new("apikey must be a 64 character hex string") end @username = username @apikey = apikey @endpoint = endpoint end
Public Instance Methods
Account : Get account details your account.
# File lib/passivetotal/api.rb, line 41 def account get('account') end
Account History : Get history associated with your account.
# File lib/passivetotal/api.rb, line 46 def account_history get('account/history') end
Account organization : Get details about the organization your account is associated with.
# File lib/passivetotal/api.rb, line 54 def account_organization get('account/organization') end
Account organization teamstream : Get the teamstream for the organization your account is associated with.
# File lib/passivetotal/api.rb, line 62 def account_organization_teamstream get('account/organization/teamstream') end
Account sources : Get source details for a specific source.
# File lib/passivetotal/api.rb, line 70 def account_sources(source) get('account/sources', {'source' => source}) end
Add a user-tag to an IP or domain query: A domain or IP address to tag tag: Value used to tag query value. Should only consist of alphanumeric, underscores and hyphen values
# File lib/passivetotal/api.rb, line 192 def add_tag(query, tag) is_valid_with_error(__method__, [:ipv4, :domain], query) is_valid_with_error(__method__, [:tag], tag) post('actions/tags', { 'query' => query, 'tags' => [tag] }) end
Get the classification for a query in bulk query: An array of domains or IP address to query
# File lib/passivetotal/api.rb, line 225 def bulk_classification(query) if query.class != Array query = [query] end query.map do |q| is_valid_with_error(__method__, [:ipv4, :domain], q) if domain?(q) q = normalize_domain(q) end q end get_with_data('actions/bulk/classification', { 'query' => query }) end
Enrichment bulk : Enrich each of the given queries with metadata query: An array of domains or IP addresses to query
# File lib/passivetotal/api.rb, line 116 def bulk_enrichment(query) if query.class != Array query = [query] end query.map do |q| is_valid_with_error(__method__, [:ipv4, :domain], q) if domain?(q) q = normalize_domain(q) end q end get_with_data('enrichment/bulk', { 'query' => query }) end
malware bulk: get sample information based from domains query: An array of domains or IP addresses to query
# File lib/passivetotal/api.rb, line 396 def bulk_malware(query) if query.class != Array query = [query] end query.map do |q| is_valid_with_error(__method__, [:ipv4, :domain], q) if domain?(q) q = normalize_domain(q) end q end get_with_data('enrichment/bulk/malware', { 'query' => query }) end
osint bulk : Enrich each of the given queries with metadata query: An array of domains or IP addresses to query
# File lib/passivetotal/api.rb, line 142 def bulk_osint(query) if query.class != Array query = [query] end query.map do |q| is_valid_with_error(__method__, [:ipv4, :domain], q) if domain?(q) q = normalize_domain(q) end q end get_with_data('enrichment/bulk/osint', { 'query' => query }) end
PassiveTotal
uses the notion of classifications to highlight table rows a certain color based on how they have been rated. PassiveTotal::API#classification()
queries if only one argument is given, and sets if both are given query: A domain or IP address to query
# File lib/passivetotal/api.rb, line 210 def classification(query, set=nil) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end if set.nil? get('actions/classification', {'query' => query}) else is_valid_with_error(__method__.to_s, [:classification], set) post('actions/classification', { 'query' => query, 'classification' => set }) end end
PassiveTotal
tracks some interesting metadata about a host query: a hostname or ip address
# File lib/passivetotal/api.rb, line 359 def components(query) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('host-attributes/components', {'query' => query}) end
PassiveTotal
allows users to notate if a domain is associated with a dynamic DNS provider. PassiveTotal::API#dynamic()
queries if only one argument is given, and sets if both are given query: A domain to query set: a boolean flag
# File lib/passivetotal/api.rb, line 262 def dynamic(query, set=nil) is_valid_with_error(__method__, [:domain], query) query = normalize_domain(query) if set.nil? get('actions/dynamic-dns', {'query' => query}) else is_valid_with_error(__method__, [:bool], set) post('actions/dynamic-dns', { 'query' => query, 'status' => set }) end end
Enrichment : Enrich the given query with metadata query: A domain or IP address to query
# File lib/passivetotal/api.rb, line 103 def enrichment(query) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('enrichment', {'query' => query}) end
PassiveTotal
allows users to notate if a domain or IP address have ever been compromised. These values aid in letting users know that a site may be benign, but it was used in an attack at some point in time. PassiveTotal::API#ever_compromised()
queries if only one argument is given, and sets if both are given query: A domain or IP address to query set: a boolean flag
# File lib/passivetotal/api.rb, line 243 def ever_compromised(query, set=nil) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end if set.nil? get('actions/ever-compromised', {'query' => query}) else is_valid_with_error(__method__, [:bool], set) post('actions/ever-compromised', { 'query' => query, 'status' => set }) end end
malware: get sample information based from domain query: ip or domain
# File lib/passivetotal/api.rb, line 386 def malware(query) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('enrichment/malware', {'query' => query}) end
PassiveTotal
allows users to notate if an ip or domain is “monitored”. PassiveTotal::API#monitor()
queries if only one argument is given, and sets if both are given query: A domain to query set: a boolean flag
# File lib/passivetotal/api.rb, line 277 def monitor(query, set=nil) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end if set.nil? get('actions/monitor', {'query' => query}) else is_valid_with_error(__method__, [:bool], set) post('actions/monitor', { 'query' => query, 'status' => set }) end end
osint: Get opensource intelligence data query: A domain or IP address to query
# File lib/passivetotal/api.rb, line 132 def osint(query) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('enrichment/osint', {'query' => query}) end
Passive provides a complete passive DNS picture for a domain or IP address including first/last seen values, deconflicted values, sources used, unique counts and enrichment for all values. query: A domain or IP address to query
# File lib/passivetotal/api.rb, line 80 def passive(query) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('dns/passive', {'query' => query}) end
Passive provides a complete passive DNS picture for a domain or IP address including first/last seen values, deconflicted values, sources used, unique counts and enrichment for all values. query: A domain or IP address to query
# File lib/passivetotal/api.rb, line 90 def passive_unique(query) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('dns/passive/unique', {'query' => query}) end
Remove a user-tag to an IP or domain query: A domain or IP address to remove a tag from tag: Value used to tag query value. Should only consist of alphanumeric, underscores and hyphen values
# File lib/passivetotal/api.rb, line 201 def remove_tag(query, tag) is_valid_with_error(__method__, [:ipv4, :domain], query) is_valid_with_error(__method__, [:tag], tag) delete('actions/tags', { 'query' => query, 'tags' => [tag] }) end
whois: Get reputation data for a domain or IP address query: A domain or IP address to query
# File lib/passivetotal/api.rb, line 181 def reputation(query) is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('reputation', {'query' => query}) end
PassiveTotal
allows users to notate if an IP address is a known sinkhole. These values are shared globally with everyone in the platform. PassiveTotal::API#sinkhole()
queries if only one argument is given, and sets if both are given query: An IP address to set as a sinkhole or not set: a boolean flag
# File lib/passivetotal/api.rb, line 298 def sinkhole(query, set=nil) is_valid_with_error(__method__, [:ipv4], query) if set.nil? get('actions/sinkhole', {'query' => query}) else is_valid_with_error(__method__, [:bool], set) post('actions/sinkhole', { 'query' => query, 'status' => set }) end end
ssl_certificate
: returns details about SSL certificates query: SHA-1 has to query, or, if field is set, a valid value for that field field: the certificate field to query upon
certificate fields: issuer_surname, subject_organizationName, issuer_country, issuer_organizationUnitName, fingerprint, subject_organizationUnitName, serialNumber, subject_emailAddress, subject_country, issuer_givenName, subject_commonName, issuer_commonName, issuer_stateOrProvinceName, issuer_province, subject_stateOrProvinceName, sha1, sslVersion, subject_streetAddress, subject_serialNumber, issuer_organizationName, subject_surname, subject_localityName, issuer_streetAddress, issuer_localityName, subject_givenName, subject_province, issuer_serialNumber, issuer_emailAddress
# File lib/passivetotal/api.rb, line 347 def ssl_certificate(query, field=nil) if field.nil? is_valid_with_error(__method__, [:hash], query) get('ssl-certificate', {'query' => query}) else is_valid_with_error(__method__, [:ssl_field], field) get_params('ssl-certificate/search', { 'query' => query, 'field' => field }) end end
PassiveTotal
collects and provides SSL certificates as an enrichment point when possible. Beyond the certificate data itself, PassiveTotal
keeps a record of the IP address of where the certificate was found and the time in which it was collected. query: A SHA-1 hash to query
# File lib/passivetotal/api.rb, line 338 def ssl_certificate_history(query) is_valid_with_error(__method__, [:ipv4, :hash], query) get('ssl-certificate/history', {'query' => query}) end
subdomains: Get subdomains using a wildcard query query: A domain with wildcard, e.g., *.passivetotal.org
# File lib/passivetotal/api.rb, line 158 def subdomains(query) get('enrichment/subdomains', {'query' => query}) end
teamstream is an alias for account_organization_teamstream
trackers: Get all tracking codes for a domain or IP address. query: ip or domain, or, if type is supplied, a valid tracker ID type: A valid tracker type to search:
tracker types: YandexMetricaCounterId, ClickyId, GoogleAnalyticsAccountNumber, NewRelicId, MixpanelId, GoogleAnalyticsTrackingId
# File lib/passivetotal/api.rb, line 371 def trackers(query, type=nil) if type.nil? is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('host-attributes/trackers', {'query' => query}) else is_valid_with_error(__method__, [:tracker_type], type) get('trackers/search', {'query' => query, 'type' => type}) end end
whois: Get WHOIS data for a domain or IP address query: ipv4, domain, or, if you specify a field, any value for that field field: field name to query if not the default ip/domain field
field names: domain, email, name, organization, address, phone, nameserver
# File lib/passivetotal/api.rb, line 166 def whois(query, field=nil) if field is_valid_with_error(__method__, [:whois_field], field) get('whois/search', {'field' => field, 'query' => query}) else is_valid_with_error(__method__, [:ipv4, :domain], query) if domain?(query) query = normalize_domain(query) end get('whois', {'query' => query, 'compact_record' => 'false'}) end end
Private Instance Methods
returns true is the given object matches true or false
# File lib/passivetotal/api.rb, line 443 def bool?(b) not ['true', 'false'].index(b.to_s).nil? end
returns true if the given string matches a valid classification
# File lib/passivetotal/api.rb, line 438 def classification?(c) not ["malicious", "non-malicious", "suspicious", "unknown"].index(c).nil? end
helper function to perform an HTTP DELETE against the web API
# File lib/passivetotal/api.rb, line 501 def delete(api, params) url2json(:DELETE, "#{@endpoint}#{api}", params) end
returns true if the given string looks like a domain and ends with a known top-level domain (TLD)
# File lib/passivetotal/api.rb, line 422 def domain?(domain) return false if domain.nil? domain = normalize_domain(domain) domain =~ /^[a-zA-Z0-9\-\.]{3,255}$/ and TLDS.index(domain.split(/\./).last) end
helper function to perform an HTTP GET against the web API
# File lib/passivetotal/api.rb, line 482 def get(api, params={}) url2json(:GET, "#{@endpoint}#{api}", params) end
helper function to perform an HTTP GET against the web API
# File lib/passivetotal/api.rb, line 487 def get_params(api, params) url2json(:GET, "#{@endpoint}#{api}", params) end
# File lib/passivetotal/api.rb, line 491 def get_with_data(api, params={}) url2json(:GET_DATA, "#{@endpoint}#{api}", params) end
returns true if the given string looks like a SHA-1 hash, i.e., 40 character hex string
# File lib/passivetotal/api.rb, line 429 def hash?(hash) return false if hash.nil? if hash =~ /^[a-fA-F0-9]{40}$/ return true end false end
returns true if the given string is a dotted quad IPv4 address
# File lib/passivetotal/api.rb, line 414 def ipv4?(ip) if ip =~ /^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$/ return true end false end
tests an item to see if it matches a valid type
# File lib/passivetotal/api.rb, line 569 def is_valid?(types, item) types.each do |type| if type == :ipv4 return true if ipv4?(item) elsif type == :domain return true if domain?(item) elsif type == :hash return true if hash?(item) elsif type == :classification return true if classification?(item) elsif type == :tag return true if tag?(item) elsif type == :bool return true if bool?(item) elsif type == :ssl_field return true if ssl_field?(item) elsif type == :whois_field return true if whois_field?(item) elsif type == :tracker_type return true if tracker_type?(item) end end return false end
tests an item to see if it matches a valid type and raises an ArgumentError if invalid
# File lib/passivetotal/api.rb, line 595 def is_valid_with_error(methname, types, item) valid = is_valid?(types, item) unless valid raise ArgumentError.new("#{methname} requires arguments of type: #{types.join(",")}") end valid end
lowercases and removes a trailing period (if one exists) from a domain name
# File lib/passivetotal/api.rb, line 477 def normalize_domain(domain) return domain.downcase.gsub(/\.$/,'') end
helper function to perform an HTTP POST against the web API
# File lib/passivetotal/api.rb, line 496 def post(api, params) url2json(:POST, "#{@endpoint}#{api}", params) end
# File lib/passivetotal/api.rb, line 456 def ssl_field?(f) return false if f.nil? not ["issuerSurname", "subjectOrganizationName", "issuerCountry", "issuerOrganizationUnitName", "fingerprint", "subjectOrganizationUnitName", "serialNumber", "subjectEmailAddress", "subjectCountry", "issuerGivenName", "subjectCommonName", "issuerCommonName", "issuerStateOrProvinceName", "issuerProvince", "subjectStateOrProvinceName", "sha1", "sslVersion", "subjectStreetAddress", "subjectSerialNumber", "issuerOrganizationName", "subjectSurname", "subjectLocalityName", "issuerStreetAddress", "issuerLocalityName", "subjectGivenName", "subjectProvince", "issuerSerialNumber", "issuerEmailAddress"].index(f).nil? end
returns true if the given string looks like a valid tag
# File lib/passivetotal/api.rb, line 448 def tag?(t) return false if t.nil? if t =~ /^[a-zA-Z][\w\_\-]+[a-zA-Z]$/ return true end false end
# File lib/passivetotal/api.rb, line 471 def tracker_type?(t) return false if t.nil? not ["YandexMetricaCounterId", "ClickyId", "GoogleAnalyticsAccountNumber", "NewRelicId", "MixpanelId", "GoogleAnalyticsTrackingId"].index(t).nil? end
main helper function to perform HTTP interactions with the web API
.
# File lib/passivetotal/api.rb, line 506 def url2json(method, url, params) if method == :GET url << "?" + params.map{|k,v| "#{k}=#{v}"}.join("&") end url = URI.parse url http = Net::HTTP.new(url.host, url.port) http.use_ssl = (url.scheme == 'https') http.verify_mode = OpenSSL::SSL::VERIFY_NONE http.verify_depth = 5 request = nil if method == :GET request = Net::HTTP::Get.new(url.request_uri) elsif method == :GET_DATA request = Net::HTTP::Get.new(url.request_uri) form_data = params.to_json request.content_type = 'application/json' request.body = form_data elsif method == :POST request = Net::HTTP::Post.new(url.request_uri) form_data = params.to_json request.content_type = 'application/json' request.body = form_data elsif method == :DELETE request = Net::HTTP::Delete.new(url.request_uri) form_data = params.to_json request.content_type = 'application/json' request.body = form_data elsif method == :HEAD request = Net::HTTP::Head.new(url.request_uri) request.set_form_data(params) elsif method == :PUT request = Net::HTTP::Put.new(url.request_uri) request.set_form_data(params) end request.basic_auth(@username, @apikey) request.add_field("User-Agent", "Ruby/#{RUBY_VERSION} passivetotal rubygem v#{PassiveTotal::VERSION}") t1 = Time.now response = http.request(request) delta = (Time.now - t1).to_f data = JSON.parse(response.body) obj = Transaction.new( Query.new(method, params['query'], params[method] || params['tag'], url, params), Response.new(response.body, response.code == '200', data), delta ) if data['error'] message = data['error']['message'] case message when "API key provided does not match any user." raise InvalidAPIKeyError.new(obj) when "Quota has been exceeded!" raise ExceededQuotaError.new(obj) else raise APIUsageError.new(obj) end end return obj end
# File lib/passivetotal/api.rb, line 466 def whois_field?(f) return false if f.nil? not ["domain", "email", "name", "organization", "address", "phone", "nameserver"].index(f).nil? end