svem_significance_test_parallel {SVEMnet} | R Documentation |
SVEM Significance Test (Parallel Version)
Description
Performs a whole-model significance test using the SVEM framework, handling both continuous and categorical predictors, with parallel computation.
Usage
svem_significance_test_parallel(
formula,
data,
nPoint = 2000,
nSVEM = 7,
nPerm = 200,
percent = 85,
nBoot = 200,
glmnet_alpha = c(1),
weight_scheme = c("SVEM"),
objective = c("wAIC", "wSSE"),
verbose = TRUE,
nCore = parallel::detectCores(),
seed = NULL,
...
)
Arguments
formula |
A formula specifying the model to be tested. |
data |
A data frame containing the variables in the model. |
nPoint |
The number of random points to generate in the factor space (default: 2000). |
nSVEM |
The number of SVEM models to fit to the original data (default: 7). |
nPerm |
The number of SVEM models to fit to permuted data for reference distribution (default: 200). |
percent |
The percentage of variance to capture in the SVD (default: 85). |
nBoot |
The number of bootstrap iterations within SVEM (default: 200). |
glmnet_alpha |
The alpha parameter(s) for glmnet (default: |
weight_scheme |
The weight scheme to use in SVEM (default: "SVEM"). |
objective |
Character; the objective function to use in |
verbose |
Logical; if |
nCore |
The number of CPU cores to use for parallel processing (default: all available cores). |
seed |
An integer seed for random number generation (default: NULL). |
... |
Additional arguments passed to the underlying |
Details
The 'svem_significance_test_parallel' function implements a whole-model test designed to gauge the significance of a fitted SVEM model compared to the null hypothesis of a constant response surface, with parallel computation. This method helps identify responses that have relatively stronger or weaker relationships with study factors.
The test constructs standardized predictions by centering the SVEM predictions (obtained from SVEMnet()
) by the response mean and scaling by the ensemble standard deviation. A reference distribution is created by fitting the SVEM model to multiple randomized permutations of the response vector. The Mahalanobis distances of the original and permuted models are calculated using a reduced-rank singular value decomposition.
The R code to perform this test (using matrices of nSVEM and nPerm predictions) is taken from the supplementary material of Karl (2024).
The SVEMnet
parameter debias
is hard coded to FALSE
for this test. Unpublished simulation work suggests that setting debias=TRUE
reduces the power of the test (without affecting the Type I error rate).
Value
A list containing the test results.
Acknowledgments
Development of this package was assisted by GPT o1-preview, which helped in constructing the structure of some of the code and the roxygen documentation. The code for the significance test is taken from the supplementary material of Karl (2024) (it was handwritten by that author).
References
Karl, A. T. (2024). A randomized permutation whole-model test heuristic for Self-Validated Ensemble Models (SVEM). Chemometrics and Intelligent Laboratory Systems, 249, 105122. doi:10.1016/j.chemolab.2024.105122
Examples
# Simulate data
set.seed(1)
n <- 30
X1 <- runif(n)
X2 <- runif(n)
X3 <- runif(n)
y <- 1 + X1 + X2 + X1 * X2 + X1^2 + rnorm(n)
data <- data.frame(y, X1, X2, X3)
#CRAN requires a max of nCore=2 for example. Recommend using default nCore to use entire CPU.
# Perform the SVEM significance test
test_result <- svem_significance_test_parallel(
y ~ (X1 + X2 + X3)^2 + I(X1^2) + I(X2^2) + I(X3^2),
data = data,
nPoint = 2000,
nSVEM = 9,
nPerm = 250,
nBoot = 200,
nCore = 2
)
# View the p-value
print(test_result)
test_result2 <- svem_significance_test_parallel(
y ~ (X1 + X2 )^2 + I(X1^2) + I(X2^2),
data = data,
nPoint = 2000,
nSVEM = 9,
nPerm = 250,
nBoot = 200,
nCore = 2
)
# View the p-value
print(test_result2)
# Plot the Mahalanobis distances
plot(test_result,test_result2)