MaestroPipeline {maestro}R Documentation

Class for an individual maestro pipeline A pipeline is defined as a single R script with a schedule or input

Description

Class for an individual maestro pipeline A pipeline is defined as a single R script with a schedule or input

Class for an individual maestro pipeline A pipeline is defined as a single R script with a schedule or input

Methods

Public methods


Method new()

Create a new Pipeline object

Usage
MaestroPipeline$new(
  script_path,
  pipe_name,
  frequency = NA_character_,
  start_time = lubridate::NA_POSIXct_,
  tz = NA_character_,
  hours = NULL,
  days = NULL,
  months = NULL,
  skip = FALSE,
  log_level = "INFO",
  inputs = NULL
)
Arguments
script_path

path to the script

pipe_name

name of the pipeline

frequency

frequency of the pipeline (e.g., 1 day)

start_time

start time of the pipeline

tz

time zone of the pipeline

hours

specific hours of the day

days

specific days of week or month

months

specific months of year

skip

whether to skip the pipeline regardless of scheduling

log_level

log level of the pipeline

inputs

unused

Returns

MaestroPipeline object


Method print()

Prints the pipeline

Usage
MaestroPipeline$print()
Returns

print


Method run()

Runs the pipeline

Usage
MaestroPipeline$run(
  resources = list(),
  log_file = tempfile(),
  quiet = FALSE,
  log_file_max_bytes = 1e+06,
  ...
)
Arguments
resources

named list of arguments and values to pass to the pipeline

log_file

path to the log file for logging

quiet

whether to silence console output

log_file_max_bytes

maximum bytes of the log file before trimming

...

additional arguments (unused)

Returns

invisible


Method get_pipe_name()

Get the pipeline name

Usage
MaestroPipeline$get_pipe_name()
Returns

pipeline_name


Method get_schedule()

Get the schedule as a data.frame

Usage
MaestroPipeline$get_schedule()
Returns

data.frame


Method check_timeliness()

Check whether a pipeline is scheduled to run based on orchestrator frequency and current time

Usage
MaestroPipeline$check_timeliness(
  orch_unit,
  orch_n,
  check_datetime = lubridate::now(),
  ...
)
Arguments
orch_unit

unit of the orchestrator (e.g., day)

orch_n

number of units of the frequency

check_datetime

datetime against which to check the timeliness of the pipeline (should almost always be now)

...

unused

Returns

MaestroPipeline


Method get_status()

Get status of the pipeline as a data.frame

Usage
MaestroPipeline$get_status()
Returns

data.frame


Method get_artifacts()

Get artifacts (return values) from the pipeline

Usage
MaestroPipeline$get_artifacts()
Returns

list


Method get_errors()

Get list of errors from the pipeline

Usage
MaestroPipeline$get_errors()
Returns

list


Method get_warnings()

Get list of warnings from the pipeline

Usage
MaestroPipeline$get_warnings()
Returns

list


Method get_messages()

Get list of messages from the pipeline

Usage
MaestroPipeline$get_messages()
Returns

list


Method clone()

The objects of this class are cloneable with this method.

Usage
MaestroPipeline$clone(deep = FALSE)
Arguments
deep

Whether to make a deep clone.


[Package maestro version 0.3.0 Index]