to_sdf {sparklyr.flint} | R Documentation |
Construct a Spark dataframe containing time series data from a TimeSeriesRDD
to_sdf(ts_rdd)
toSDF(ts_rdd)
ts_rdd |
A TimeSeriesRDD object |
A Spark dataframe containing time series data exported from 'ts_rdd'
Other Spark dataframe utility functions:
collect.ts_rdd()
,
from_rdd()
,
from_sdf()
,
spark_connection.ts_rdd()
,
spark_dataframe.ts_rdd()
,
spark_jobj.ts_rdd()
,
ts_rdd_builder()
Other Spark dataframe utility functions:
collect.ts_rdd()
,
from_rdd()
,
from_sdf()
,
spark_connection.ts_rdd()
,
spark_dataframe.ts_rdd()
,
spark_jobj.ts_rdd()
,
ts_rdd_builder()
library(sparklyr)
library(sparklyr.flint)
sc <- try_spark_connect(master = "local")
if (!is.null(sc)) {
sdf <- copy_to(sc, tibble::tibble(t = seq(10), v = seq(10)))
ts <- from_sdf(sdf, is_sorted = TRUE, time_unit = "SECONDS", time_column = "t")
ts_avg <- summarize_avg(ts, column = "v", window = in_past("3s"))
# now export the average values from `ts_avg` back to a Spark dataframe
# named `sdf_avg`
sdf_avg <- ts_avg %>% to_sdf()
} else {
message("Unable to establish a Spark connection!")
}