module CircleCI::Parallel
Provides simple APIs for syncing CircleCI
parallel nodes and transferring files between the nodes.
@example
merged_data = {} CircleCI::Parallel.configure do |config| config.on_every_node.before_sync do data = do_something json = JSON.generate(data) File.write('data.json', json) end config.on_master_node.after_download do Dir.glob('*/data.json') do |path| json = File.read(path) data = JSON.parse(json) node_name = File.dirname(path) merged_data[node_name] = data end end end CircleCI::Parallel.sync p merged_data
Constants
- BASE_DATA_DIR
@api private
- DOWNLOAD_MARKER_FILE
@api private
- SYNC_MARKER_FILE
@api private
- WORK_DIR
@api private
Public Class Methods
Returns the download data directory where all node data will be downloaded. Note that only master node downloads data from other slave node. When the downloads are complete, the directory structure on the master node will be the following:
. ├── node0 │ └── node_specific_data_you_saved_on_node0.txt ├── node1 │ └── node_specific_data_you_saved_on_node1.txt └── node2 └── node_specific_data_you_saved_on_node2.txt
@return [String] the download data directory
@example
Dir.chdir(CircleCI::Parallel.download_data_dir) do merged_data = Dir['*/data.json'].each_with_object({}) do |path, merged_data| data = JSON.parse(File.read(path)) node_name = File.dirname(path) merged_data[node_name] = data end end
@see CircleCI::Parallel::MasterNodeConfiguration#before_download @see CircleCI::Parallel::MasterNodeConfiguration#after_download
# File lib/circleci/parallel.rb, line 165 def download_data_dir BASE_DATA_DIR.tap do |path| FileUtils.makedirs(path) unless Dir.exist?(path) end end
@deprecated Use `.sync` instead.
# File lib/circleci/parallel.rb, line 178 def join sync end
Returns the local data directory where node specific data should be saved in.
@return [String] the local data directory
@example
path = File.join(CircleCI::Parallel.local_data_dir, 'data.json') File.write(path, JSON.generate(some_data))
@see CircleCI::Parallel::MasterNodeConfiguration#before_sync @see CircleCI::Parallel::MasterNodeConfiguration#after_sync @see CircleCI::Parallel::SlaveNodeConfiguration#before_sync @see CircleCI::Parallel::SlaveNodeConfiguration#after_sync
# File lib/circleci/parallel.rb, line 133 def local_data_dir current_node.data_dir.tap do |path| FileUtils.makedirs(path) unless Dir.exist?(path) end end
@api private
# File lib/circleci/parallel.rb, line 172 def reset! environment.clean @environment = nil end
Private Class Methods
# File lib/circleci/parallel.rb, line 184 def environment @environment ||= Environment.new end