MaestroPipeline {maestro} | R Documentation |
Class for an individual maestro pipeline A pipeline is defined as a single R script with a schedule or input
Class for an individual maestro pipeline A pipeline is defined as a single R script with a schedule or input
new()
Create a new Pipeline object
MaestroPipeline$new( script_path, pipe_name, frequency = NA_character_, start_time = lubridate::NA_POSIXct_, tz = NA_character_, hours = NULL, days = NULL, months = NULL, skip = FALSE, log_level = "INFO", inputs = NULL )
script_path
path to the script
pipe_name
name of the pipeline
frequency
frequency of the pipeline (e.g., 1 day)
start_time
start time of the pipeline
tz
time zone of the pipeline
hours
specific hours of the day
days
specific days of week or month
months
specific months of year
skip
whether to skip the pipeline regardless of scheduling
log_level
log level of the pipeline
inputs
unused
MaestroPipeline object
print()
Prints the pipeline
MaestroPipeline$print()
run()
Runs the pipeline
MaestroPipeline$run( resources = list(), log_file = tempfile(), quiet = FALSE, log_file_max_bytes = 1e+06, ... )
resources
named list of arguments and values to pass to the pipeline
log_file
path to the log file for logging
quiet
whether to silence console output
log_file_max_bytes
maximum bytes of the log file before trimming
...
additional arguments (unused)
invisible
get_pipe_name()
Get the pipeline name
MaestroPipeline$get_pipe_name()
pipeline_name
get_schedule()
Get the schedule as a data.frame
MaestroPipeline$get_schedule()
data.frame
check_timeliness()
Check whether a pipeline is scheduled to run based on orchestrator frequency and current time
MaestroPipeline$check_timeliness( orch_unit, orch_n, check_datetime = lubridate::now(), ... )
orch_unit
unit of the orchestrator (e.g., day)
orch_n
number of units of the frequency
check_datetime
datetime against which to check the timeliness of the pipeline (should almost always be now)
...
unused
MaestroPipeline
get_status()
Get status of the pipeline as a data.frame
MaestroPipeline$get_status()
data.frame
get_artifacts()
Get artifacts (return values) from the pipeline
MaestroPipeline$get_artifacts()
list
get_errors()
Get list of errors from the pipeline
MaestroPipeline$get_errors()
list
get_warnings()
Get list of warnings from the pipeline
MaestroPipeline$get_warnings()
list
get_messages()
Get list of messages from the pipeline
MaestroPipeline$get_messages()
list
clone()
The objects of this class are cloneable with this method.
MaestroPipeline$clone(deep = FALSE)
deep
Whether to make a deep clone.