class Chef::RunContext::CookbookCompiler

Implements the compile phase of the chef run by loading/eval-ing files from cookbooks in the correct order and in the correct context.

Attributes

events[R]
logger[R]
run_list_expansion[R]

Public Class Methods

new(run_context, run_list_expansion, events) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 36
def initialize(run_context, run_list_expansion, events)
  @run_context = run_context
  @events = events
  @run_list_expansion = run_list_expansion
  @cookbook_order = nil
  @logger = run_context.logger.with_child(subsystem: "cookbook_compiler")
end

Public Instance Methods

compile() click to toggle source

Run the compile phase of the chef run. Loads files in the following order:

  • Libraries

  • Ohai

  • Attributes

  • LWRPs

  • Resource Definitions

  • Recipes

Recipes are loaded in precisely the order specified by the expanded run_list.

Other files are loaded in an order derived from the expanded run_list and the dependencies declared by cookbooks' metadata. See cookbook_order for more information.

# File lib/chef/run_context/cookbook_compiler.rb, line 73
def compile
  compile_libraries
  compile_ohai_plugins
  compile_attributes
  compile_lwrps
  compile_resource_definitions
  compile_recipes
end
compile_attributes() click to toggle source

Loads attributes files from cookbooks. Attributes files are loaded according to cookbook_order; within a cookbook, default.rb is loaded first, then the remaining attributes files in lexical sort order.

# File lib/chef/run_context/cookbook_compiler.rb, line 132
def compile_attributes
  @events.attribute_load_start(count_files_by_segment(:attributes, "attributes.rb"))
  cookbook_order.each do |cookbook|
    load_attributes_from_cookbook(cookbook)
  end
  @events.attribute_load_complete
end
compile_libraries() click to toggle source

Loads library files from cookbooks according to cookbook_order.

# File lib/chef/run_context/cookbook_compiler.rb, line 100
def compile_libraries
  @events.library_load_start(count_files_by_segment(:libraries))
  cookbook_order.each do |cookbook|
    load_libraries_from_cookbook(cookbook)
  end
  @events.library_load_complete
end
compile_lwrps() click to toggle source

Loads LWRPs according to cookbook_order. Providers are loaded before resources on a cookbook-wise basis.

# File lib/chef/run_context/cookbook_compiler.rb, line 142
def compile_lwrps
  lwrp_file_count = count_files_by_segment(:providers) + count_files_by_segment(:resources)
  @events.lwrp_load_start(lwrp_file_count)
  cookbook_order.each do |cookbook|
    load_lwrps_from_cookbook(cookbook)
  end
  @events.lwrp_load_complete
end
compile_ohai_plugins() click to toggle source

Loads Ohai Plugins from cookbooks, and ensure any old ones are properly cleaned out

# File lib/chef/run_context/cookbook_compiler.rb, line 110
def compile_ohai_plugins
  ohai_plugin_count = count_files_by_segment(:ohai)
  @events.ohai_plugin_load_start(ohai_plugin_count)
  FileUtils.rm_rf(Chef::Config[:ohai_segment_plugin_path])

  cookbook_order.each do |cookbook|
    load_ohai_plugins_from_cookbook(cookbook)
  end

  # Doing a full ohai system check is costly, so only do so if we've loaded additional plugins
  if ohai_plugin_count > 0
    # FIXME(log): figure out what the ohai logger looks like here
    ohai = Ohai::System.new.run_additional_plugins(Chef::Config[:ohai_segment_plugin_path])
    node.consume_ohai_data(ohai)
  end

  @events.ohai_plugin_load_complete
end
compile_recipes() click to toggle source

Iterates over the expanded run_list, loading each recipe in turn.

# File lib/chef/run_context/cookbook_compiler.rb, line 161
def compile_recipes
  @events.recipe_load_start(run_list_expansion.recipes.size)
  run_list_expansion.recipes.each do |recipe|
    begin
      path = resolve_recipe(recipe)
      @run_context.load_recipe(recipe)
      @events.recipe_file_loaded(path, recipe)
    rescue Chef::Exceptions::RecipeNotFound => e
      @events.recipe_not_found(e)
      raise
    rescue Exception => e
      @events.recipe_file_load_failed(path, e, recipe)
      raise
    end
  end
  @events.recipe_load_complete
end
compile_resource_definitions() click to toggle source

Loads resource definitions according to cookbook_order

# File lib/chef/run_context/cookbook_compiler.rb, line 152
def compile_resource_definitions
  @events.definition_load_start(count_files_by_segment(:definitions))
  cookbook_order.each do |cookbook|
    load_resource_definitions_from_cookbook(cookbook)
  end
  @events.definition_load_complete
end
cookbook_collection() click to toggle source

Chef::CookbookCollection object for the current run

# File lib/chef/run_context/cookbook_compiler.rb, line 50
def cookbook_collection
  @run_context.cookbook_collection
end
cookbook_order() click to toggle source

Extracts the cookbook names from the expanded run list, then iterates over the list, recursing through dependencies to give a run_list ordered array of cookbook names with no duplicates. Dependencies appear before the cookbook(s) that depend on them.

# File lib/chef/run_context/cookbook_compiler.rb, line 86
def cookbook_order
  @cookbook_order ||= begin
    ordered_cookbooks = []
    seen_cookbooks = {}
    run_list_expansion.recipes.each do |recipe|
      cookbook = Chef::Recipe.parse_recipe_name(recipe).first
      add_cookbook_with_deps(ordered_cookbooks, seen_cookbooks, cookbook)
    end
    logger.debug("Cookbooks to compile: #{ordered_cookbooks.inspect}")
    ordered_cookbooks
  end
end
definitions() click to toggle source

Resource Definitions from the compiled cookbooks. This is populated by calling compile_resource_definitions (which is called by compile)

# File lib/chef/run_context/cookbook_compiler.rb, line 56
def definitions
  @run_context.definitions
end
node() click to toggle source

Chef::Node object for the current run.

# File lib/chef/run_context/cookbook_compiler.rb, line 45
def node
  @run_context.node
end
reachable_cookbooks() click to toggle source

All cookbooks in the dependency graph, returned as a Set.

# File lib/chef/run_context/cookbook_compiler.rb, line 186
def reachable_cookbooks
  @reachable_cookbooks ||= Set.new(cookbook_order)
end
unreachable_cookbook?(cookbook_name) click to toggle source

Whether or not a cookbook is reachable from the set of cookbook given by the run_list plus those cookbooks' dependencies.

# File lib/chef/run_context/cookbook_compiler.rb, line 181
def unreachable_cookbook?(cookbook_name)
  !reachable_cookbooks.include?(cookbook_name)
end

Private Instance Methods

add_cookbook_with_deps(ordered_cookbooks, seen_cookbooks, cookbook) click to toggle source

Builds up the list of ordered_cookbooks by first recursing through the dependencies of cookbook, and then adding cookbook to the list of ordered_cookbooks. A cookbook is skipped if it appears in seen_cookbooks, otherwise it is added to the set of seen_cookbooks before its dependencies are processed.

# File lib/chef/run_context/cookbook_compiler.rb, line 304
def add_cookbook_with_deps(ordered_cookbooks, seen_cookbooks, cookbook)
  return false if seen_cookbooks.key?(cookbook)

  seen_cookbooks[cookbook] = true
  each_cookbook_dep(cookbook) do |dependency|
    add_cookbook_with_deps(ordered_cookbooks, seen_cookbooks, dependency)
  end
  ordered_cookbooks << cookbook
end
count_files_by_segment(segment, root_alias = nil) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 314
def count_files_by_segment(segment, root_alias = nil)
  cookbook_collection.inject(0) do |count, cookbook_by_name|
    count + cookbook_by_name[1].segment_filenames(segment).size + (root_alias ? cookbook_by_name[1].files_for(:root_files).select { |record| record[:name] == root_alias }.size : 0)
  end
end
each_cookbook_dep(cookbook_name, &block) click to toggle source

Yields the name, as a symbol, of each cookbook depended on by cookbook_name in lexical sort order.

# File lib/chef/run_context/cookbook_compiler.rb, line 328
def each_cookbook_dep(cookbook_name, &block)
  cookbook = cookbook_collection[cookbook_name]
  cookbook.metadata.dependencies.keys.sort.map { |x| x.to_sym }.each(&block)
end
files_in_cookbook_by_segment(cookbook, segment) click to toggle source

Lists the local paths to files in cookbook of type segment (attribute, recipe, etc.), sorted lexically.

# File lib/chef/run_context/cookbook_compiler.rb, line 322
def files_in_cookbook_by_segment(cookbook, segment)
  cookbook_collection[cookbook].files_for(segment).map { |record| record[:full_path] }.sort
end
load_attribute_file(cookbook_name, filename) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 214
def load_attribute_file(cookbook_name, filename)
  logger.trace("Node #{node.name} loading cookbook #{cookbook_name}'s attribute file #{filename}")
  attr_file_basename = ::File.basename(filename, ".rb")
  node.include_attribute("#{cookbook_name}::#{attr_file_basename}")
rescue Exception => e
  @events.attribute_file_load_failed(filename, e)
  raise
end
load_attributes_from_cookbook(cookbook_name) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 192
def load_attributes_from_cookbook(cookbook_name)
  list_of_attr_files = files_in_cookbook_by_segment(cookbook_name, :attributes).dup
  root_alias = cookbook_collection[cookbook_name].files_for(:root_files).find { |record| record[:name] == "root_files/attributes.rb" }
  default_file = list_of_attr_files.find { |path| File.basename(path) == "default.rb" }
  if root_alias
    if default_file
      logger.error("Cookbook #{cookbook_name} contains both attributes.rb and and attributes/default.rb, ignoring attributes/default.rb")
      list_of_attr_files.delete(default_file)
    end
    # The actual root_alias path decoding is handled in CookbookVersion#attribute_filenames_by_short_filename
    load_attribute_file(cookbook_name.to_s, "default")
  elsif default_file
    list_of_attr_files.delete(default_file)
    load_attribute_file(cookbook_name.to_s, default_file)
  end

  list_of_attr_files.each do |filename|
    next unless File.extname(filename) == ".rb"
    load_attribute_file(cookbook_name.to_s, filename)
  end
end
load_libraries_from_cookbook(cookbook_name) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 223
def load_libraries_from_cookbook(cookbook_name)
  files_in_cookbook_by_segment(cookbook_name, :libraries).each do |filename|
    next unless File.extname(filename) == ".rb"
    begin
      logger.trace("Loading cookbook #{cookbook_name}'s library file: #{filename}")
      Kernel.require(filename)
      @events.library_file_loaded(filename)
    rescue Exception => e
      @events.library_file_load_failed(filename, e)
      raise
    end
  end
end
load_lwrp_provider(cookbook_name, filename) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 248
def load_lwrp_provider(cookbook_name, filename)
  logger.trace("Loading cookbook #{cookbook_name}'s providers from #{filename}")
  Chef::Provider::LWRPBase.build_from_file(cookbook_name, filename, self)
  @events.lwrp_file_loaded(filename)
rescue Exception => e
  @events.lwrp_file_load_failed(filename, e)
  raise
end
load_lwrp_resource(cookbook_name, filename) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 257
def load_lwrp_resource(cookbook_name, filename)
  logger.trace("Loading cookbook #{cookbook_name}'s resources from #{filename}")
  Chef::Resource::LWRPBase.build_from_file(cookbook_name, filename, self)
  @events.lwrp_file_loaded(filename)
rescue Exception => e
  @events.lwrp_file_load_failed(filename, e)
  raise
end
load_lwrps_from_cookbook(cookbook_name) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 237
def load_lwrps_from_cookbook(cookbook_name)
  files_in_cookbook_by_segment(cookbook_name, :providers).each do |filename|
    next unless File.extname(filename) == ".rb"
    load_lwrp_provider(cookbook_name, filename)
  end
  files_in_cookbook_by_segment(cookbook_name, :resources).each do |filename|
    next unless File.extname(filename) == ".rb"
    load_lwrp_resource(cookbook_name, filename)
  end
end
load_ohai_plugins_from_cookbook(cookbook_name) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 266
def load_ohai_plugins_from_cookbook(cookbook_name)
  target = Chef::Config[:ohai_segment_plugin_path]
  files_in_cookbook_by_segment(cookbook_name, :ohai).each do |filename|
    next unless File.extname(filename) == ".rb"

    logger.trace "Loading Ohai plugin: #{filename} from #{cookbook_name}"
    target_name = File.join(target, cookbook_name.to_s, File.basename(filename))

    FileUtils.mkdir_p(File.dirname(target_name))
    FileUtils.cp(filename, target_name)
  end
end
load_resource_definitions_from_cookbook(cookbook_name) click to toggle source
# File lib/chef/run_context/cookbook_compiler.rb, line 279
def load_resource_definitions_from_cookbook(cookbook_name)
  files_in_cookbook_by_segment(cookbook_name, :definitions).each do |filename|
    next unless File.extname(filename) == ".rb"

    begin
      logger.trace("Loading cookbook #{cookbook_name}'s definitions from #{filename}")
      resourcelist = Chef::ResourceDefinitionList.new
      resourcelist.from_file(filename)
      definitions.merge!(resourcelist.defines) do |key, oldval, newval|
        logger.info("Overriding duplicate definition #{key}, new definition found in #{filename}")
        newval
      end
      @events.definition_file_loaded(filename)
    rescue Exception => e
      @events.definition_file_load_failed(filename, e)
      raise
    end
  end
end
resolve_recipe(recipe_name) click to toggle source

Given a recipe_name, finds the file associated with the recipe.

# File lib/chef/run_context/cookbook_compiler.rb, line 334
def resolve_recipe(recipe_name)
  cookbook_name, recipe_short_name = Chef::Recipe.parse_recipe_name(recipe_name)
  cookbook = cookbook_collection[cookbook_name]
  cookbook.recipe_filenames_by_name[recipe_short_name]
end