File fetchers

File fetchers are used to fetch the relevant dependency files for a project (e.g., the Gemfile and Gemfile.lock). They are also responsible for checking whether a repo has an admissible set of requirement files.

There is a Dependabot::FileFetchers class for each language Dependabot supports.

Public API

Each Dependabot::FileFetchers class implements the following methods:

Method Description
`.required_files_in?` Checks an array of filenames (string) and returns a boolean describing whether the language-specific dependency files required for an update run are present.
`.required_files_message` Returns a static error message which can be displayed to a user if `required_files_in?` returns false.
`#files` Fetches the language-specific dependency files for the repo this instance was created with. Returns an array of `Dependabot::DependencyFile` instances.
`#commit` Returns the commit SHA-1 hash at the time the dependency files were fetched. If called before `files`, the returned value will be used in subsequent calls to `files`.

An integration might look as follows:

require 'octokit'
require 'dependabot/file_fetchers'
require 'dependabot/source'

target_repo_name = 'dependabot/dependabot-core'
source = Dependabot::Source.new(provider: 'github', repo: target_repo_name)

client = Octokit::Client.new
fetcher_class = Dependabot::FileFetchers::Ruby::Bundler
filenames = client.contents(target_repo_name).map(&:name)

unless fetcher_class.required_files_in?(filenames)
  raise fetcher_class.required_files_message
end

fetcher = fetcher_class.new(source: source, credentials: [])

puts "Fetched #{fetcher.files.map(&:name)}, at commit SHA-1 '#{fetcher.commit}'"

Writing a file fetcher for a new language

All new file fetchers should inherit from Dependabot::FileFetchers::Base and implement the following methods:

Method Description
`.required_files_in?` See Public API section.
`.required_files_message` See Public API section.
`#fetch_files` Private method to fetch the required files from GitHub. For each required file, you can use the `fetch_file_from_host(filename)` method from `Dependabot::FileFetchers::Base` to do the fetching.

To ensure the above are implemented, you should include it_behaves_like "a dependency file fetcher" in your specs for the new file fetcher.

File fetchers tend to get complicated when the file requirements for an update to run are non-trivial - for example, for Ruby we could accept [Gemfile, Gemfile.lock] or [Gemfile, example.gemspec], but not just [Gemfile.lock]. When adding a new language, it's normally easiest to pick a single case and implement it for all the update steps (parsing, update checking, etc.). You can then return and add other cases later.