A Basic Usage of ‘LazyArray’

Zhengjia Wang

library(lazyarray)

Initialize

Create a blank array and assign R object

# Sample data (~24 MB)
x <- rnorm(3e6); dim(x) <- c(10, 100, 100, 30)

# Save array to a path
path <- tempfile()
arr <- create_lazyarray(path, 'double', dim(x), multipart = TRUE)
arr[] <- x

Load existing array

# Load existing array
arr <- load_lazyarray(path)

The loaded array is read-only by default. However, they can be set writable.

# Make loaded array writable
arr$make_writable()
arr$can_write
#> [1] TRUE

S3 methods

  1. Set dimension names
arr$make_writable()
dimnames(arr) <- list(
  A = 1:10,
  B = 1:100,
  C = 1:100,
  D = 1:30
)
  1. Subset and subset assign
# Subset/read array
y1 <- arr[]              
y2 <- arr[,,,3]          

# Write to slice of data, writing to slices along the 
# last dimension is optimized
arr[,,,1] <- seq_len(1e5)
  1. Subset by formula
sub <- subset(arr, A ~ A <= 2, B ~ B == 10)
dim(sub)
#> [1]   2   1 100  30

Remove Arrays

Data created via lazyarray does not remove automatically. You need to finalize array by yourself. This is because multiple lazy array instances might point to a same dataset. If one of the object is garbage collected, you might not want to remove the data on hard drive as this will invalidate the other instances. To manually remove data, use

arr$remove_data()
#> [1] TRUE