mcProgressBar {mcprogress} | R Documentation |
Uses echo
to safely output a progress bar to Rstudio or Linux console
during parallel processing.
mcProgressBar(
val,
len = 1L,
cores = 1L,
subval = NULL,
title = "",
spinner = FALSE,
eta = TRUE,
start = NULL,
sensitivity = 0.01
)
closeProgress(start = NULL, title = "", eta = TRUE)
val |
Integer measuring progress |
len |
Total number of processes to be executed overall. |
cores |
Number of cores used for parallel processing. |
subval |
Optional subvalue ranging from 0 to 1 to enable granularity
during long processes. Especially useful if |
title |
Optional title for the progress bar. |
spinner |
Logical whether to show a spinner which moves when each core
completes a process. More useful for relatively long processes where the
length of time for each process to complete is variable. Not shown if
|
eta |
Logical whether to show estimated time to completion. |
start |
Used to pass the system time from the start of the call to show a total time elapsed. See the example below. |
sensitivity |
Determines maximum sensitivity with which to report
progress for situations where |
This package provides 2 main methods to show progress during parallelised
code using mclapply()
. If X
(the list object looped over in a call to
mclapply()
) has many elements compared to the number of cores, then it is
easiest to use pmclapply()
. However, in some use cases the length of X
is
comparable to the number of cores and each process may take a long time. For
example, machine learning applied to each of 8 folds on an 8-core machine
will open 8 processes from the outset. Each process will often complete at
roughly the same time. In this case pmclapply()
is much less informative as
it only shows completion at the end of 1 round of processes so it will go
from 0% to 100%. In this example, if each process code is long and
subprogress can be reported along the way, for example during nested loops,
then mcProgressBar()
provides a way to show the subprogress during the
inner loop. The example below shows how to write code involving an outer call
to mclapply()
and an inner loop whose subprogress is tracked via calls to
mcProgressBar()
.
Technically only 1 process can be tracked. If cores
is set to 4 and
subval
is invoked, then the 1st, 5th, 9th, 13th etc process is tracked.
Subprogress of this process is computed as part of the number of blocks of
processes required. ETA is approximate. As part of minimising overhead, it is
only updated with each change in progress (i.e. each time a block of
processes completes) or when subprogress changes. It is not updated by
interrupt.
No return value. Prints a progress bar to the console if called within an Rstudio or Linux environment.
Myles Lewis
if (Sys.info()["sysname"] != "Windows") {
## Example function with mclapply wrapped around another nested function
library(parallel)
my_fun <- function(x, cores) {
start <- Sys.time()
mcProgressBar(0, title = "my_fun") # initialise progress bar
res <- mclapply(seq_along(x), function(i) {
# inner loop of calculation
y <- 1:4
inner <- lapply(seq_along(y), function(j) {
Sys.sleep(0.2 + runif(1) * 0.1)
mcProgressBar(val = i, len = length(x), cores, subval = j / length(y),
title = "my_fun")
rnorm(4)
})
inner
}, mc.cores = cores)
closeProgress(start, title = "my_fun") # finalise the progress bar
res
}
res <- my_fun(letters[1:4], cores = 2)
## Example of long function
longfun <- function(x, cores) {
start <- Sys.time()
mcProgressBar(0, title = "longfun") # initialise progress bar
res <- mclapply(seq_along(x), function(i) {
# long sequential calculation in parallel with 3 major steps
Sys.sleep(0.2)
mcProgressBar(val = i, len = length(x), cores, subval = 0.33,
title = "longfun") # 33% complete
Sys.sleep(0.2)
mcProgressBar(val = i, len = length(x), cores, subval = 0.66,
title = "longfun") # 66% complete
Sys.sleep(0.2)
mcProgressBar(val = i, len = length(x), cores, subval = 1,
title = "longfun") # 100% complete
return(rnorm(4))
}, mc.cores = cores)
closeProgress(start, title = "longfun") # finalise the progress bar
res
}
res <- longfun(letters[1:2], cores = 2)
}