class Telly::TestRailTeller

Constants

BLOCKED
CREDENTIALS_FILE
FAILED
PASSED

Testrail Status IDs

TESTCASE_ID_REGEX

Used for extracted the test case ID from beaker scripts

TESTRAIL_URL

Public Instance Methods

beaker_test_path(junit_file_path, junit_result) click to toggle source

Calculates the path to a beaker test file by combining the junit file path with the test name from the junit results. Makes the assumption that junit folder that beaker creates will always be 2 directories up from the beaker script base directory. TODO somewhat hacky, maybe a config/command line option

@param [String] junit_file_path Path to a junit xml file @param [String] junit_result Path to a junit xml file

@return [String] The path to the beaker script from the junit test result

@example load_junit_results(“~/junit/latest/beaker_junit.xml”)

# File lib/telly/test_rail_teller.rb, line 296
def beaker_test_path(junit_file_path, junit_result)
  beaker_folder_path = junit_result[:classname]
  test_filename = junit_result[:name]

  File.join(File.dirname(junit_file_path), "../../../", beaker_folder_path, test_filename)
end
do_stub_test(credentials) click to toggle source
# File lib/telly/test_rail_teller.rb, line 42
def do_stub_test(credentials)
  api = get_testrail_api(credentials)

  api.send_post
end
get_testrail_api(credentials) click to toggle source

Returns a testrail API object that talks to testrail

@param [Hash] credentials A hash containing at least two keys, testrail_username and testrail_password

@return [TestRail::APIClient] The API object for talking to TestRail

@example api = get_testrail_api(load_credentials)

# File lib/telly/test_rail_teller.rb, line 110
def get_testrail_api(credentials)
  client = TestRail::APIClient.new(TESTRAIL_URL)
  client.user = credentials["testrail_username"]
  client.password = credentials["testrail_password"]

  return client
end
load_credentials(credentials_file) click to toggle source

Load testrail credentials from file

@return [Hash] Contains testrail_username and testrail_password

@example password = load_credentials()

# File lib/telly/test_rail_teller.rb, line 89
def load_credentials(credentials_file)
  begin
    YAML.load_file(File.expand_path(credentials_file))
  rescue
    puts "Error: Could not find #{credentials_file}"
    puts "Create #{credentials_file} with the following:"
    puts "testrail_username: your.username\ntestrail_password: yourpassword"

    exit

  end
end
load_junit_results(junit_file) click to toggle source

Loads the results of a beaker run. Returns hash of failures, passes, and skips that each hold a list of junit xml objects

@param [String] junit_file Path to a junit xml file

@return [Hash] A hash containing xml objects for the failures, skips, and passes

@example load_junit_results(“~/junit/latest/beaker_junit.xml”)

# File lib/telly/test_rail_teller.rb, line 256
def load_junit_results(junit_file)
  junit_doc = Nokogiri::XML(File.read(junit_file))

  failures = junit_doc.xpath('//testcase[failure]')
  skips = junit_doc.xpath('//testcase[skip]')
  passes = junit_doc.xpath('//testcase[not(failure) and not(skip)]')

  return {failures: failures, skips: skips, passes: passes}
end
make_testrail_time(seconds_string) click to toggle source

Returns a string that testrail accepts as an elapsed time Input from beaker is a float in seconds, so it rounds it to the nearest second, and adds an 's' at the end

Testrail throws an exception if it gets “0s”, so it returns a minimum of “1s”

@param [String] seconds_string A string that contains only a number, the elapsed time of a test

@return [String] The elapsed time of the test run, rounded and with an 's' appended

@example puts make_testrail_time(“2.34”) # “2s”

# File lib/telly/test_rail_teller.rb, line 233
def make_testrail_time(seconds_string)
  # If time is 0, make it 1
  rounded_time = [seconds_string.to_f.round, 1].max
  # Test duration
  time_elapsed = "#{rounded_time}s"

  return time_elapsed
end
send_to_testrail() click to toggle source

Run the importer

@return [Void]

@example password = Telly::main(parse_opts)

# File lib/telly/test_rail_teller.rb, line 57
def send_to_testrail()
  options = ArgParser.parse_opts
  # Get pass/fail/skips from junit file
  results = load_junit_results(options[:junit_file])

  puts "Run results:"
  puts "#{results[:passes].length} Passing"
  puts "#{results[:failures].length} Failing or Erroring"
  puts "#{results[:skips].length} Skipped"

  # Set results in testrail
  bad_results = set_testrail_results(results, options[:junit_file], options[:testrun_id], options[:dry_run])

  # Print error messages
  if not bad_results.empty?
    puts "Error: There were problems processing these test scripts:"
    bad_results.each do |test_script, error|
      puts "#{test_script}:\n\t#{error}"
    end
  end
end
set_testrail_results(results, junit_file, testrun_id, dry_run = false) click to toggle source

Sets the results in testrail. Tests that have testrail API exceptions are kept track of in bad_results

@param [Hash] results A hash of lists of xml objects from the junit output file. @param [String] junit_file The path to the junit xml file

Needed for determining the path of the test file in add_failure, etc

@param [String] testrun_id The TestRail test run ID @param [Boolean] dry_run Do not create api connection when dry_run = true

@return [Void]

# File lib/telly/test_rail_teller.rb, line 129
def set_testrail_results(results, junit_file, testrun_id, dry_run = false)
  if not dry_run
    credentials = load_credentials(CREDENTIALS_FILE)
    api = get_testrail_api(credentials)
  else
    api = get_testrail_api( { "testrail_username" => 'name', "testrail_password" => 'pass' } )
  end


  # Results that couldn't be set in testrail for some reason
  bad_results = {}

  # passes
  results[:passes].each do |junit_result|
    begin
      submit_result(api, PASSED, junit_result, junit_file, testrun_id, dry_run)
    rescue MissingTestRailId, TestRail::APIError => e
      bad_results[junit_result[:name]] = e.message
    end
  end

  # Failures
  results[:failures].each do |junit_result|
    begin
      submit_result(api, FAILED, junit_result, junit_file, testrun_id, dry_run)
    rescue MissingTestRailId, TestRail::APIError => e
      bad_results[junit_result[:name]] = e.message
    end
  end

  # Skips
  results[:skips].each do |junit_result|
    begin
      submit_result(api, BLOCKED, junit_result, junit_file, testrun_id, dry_run)
    rescue MissingTestRailId, TestRail::APIError => e
      bad_results[junit_result[:name]] = e.message
    end
  end

  return bad_results
end
submit_result(api, status, junit_result, junit_file, testrun_id, dry_run = false) click to toggle source

Submits a test result to TestRail

@param [TestRail::APIClient] api TestRail API object @param [int] status The testrail status to set @param [Nokogiri::XML::Element] junit_result The nokogiri node that holds the junit result @param [String] junit_file Path to the junit file the test result originated from @param [String] testrun_id The testrun ID @param [Boolean] dry_run When true do not send results to TestRail

@return [Void]

@raise [TestRail::APIError] When there is a problem with the API request, testrail raises

this exception. Should be caught for error reporting

@example submit_result(api, BLOCKED, junit_result, junit_file, testrun_id)

# File lib/telly/test_rail_teller.rb, line 186
def submit_result(api, status, junit_result, junit_file, testrun_id, dry_run = false)
  test_file_path = beaker_test_path(junit_file, junit_result)

  puts junit_result.class
  testcase_id = testcase_id_from_beaker_script(test_file_path)

  time_elapsed = make_testrail_time(junit_result[:time])

  # Make appropriate comment for testrail
  case status
  when FAILED
    error_message = junit_result.xpath('./failure').first[:message]
    testrail_comment = "Failed with message:\n#{error_message}"
  when BLOCKED
    skip_message = junit_result.xpath('system-out').first.text
    testrail_comment = "Skipped with message:\n#{skip_message}"
  else
    testrail_comment = "Passed"
  end

  puts "\nSetting result for test case: #{testcase_id}"
  puts "Adding comment:\n#{testrail_comment}"

  if not dry_run
    api.send_post("add_result_for_case/#{testrun_id}/#{testcase_id}",
      {
        status_id: status,
        comment: testrail_comment,
        elapsed: time_elapsed,
      }
    )
  end
end
testcase_id_from_beaker_script(beaker_file) click to toggle source

Extracts the test case id from the test script

@param [String] beaker_file Path to a beaker test script

@return [String] The test case ID

@example testcase_id_from_beaker_script(“~/tests/test_the_things.rb”) # 1234

# File lib/telly/test_rail_teller.rb, line 274
def testcase_id_from_beaker_script(beaker_file)
  # Find first matching line
  match = File.readlines(beaker_file).map { |line| line.match(TESTCASE_ID_REGEX) }.compact.first

  raise MissingTestRailId, 'Testcase ID could not be found in file' if match.nil?

  match[:testrun_id]
end