svem_significance_test {SVEMnet} | R Documentation |
SVEM Significance Test
Description
Performs a whole-model significance test using the SVEM framework, handling both continuous and categorical predictors.
Usage
svem_significance_test(
formula,
data,
nPoint = 2000,
nSVEM = 5,
nPerm = 125,
percent = 90,
nBoot = 200,
glmnet_alpha = c(1),
weight_scheme = c("SVEM", "FWR"),
debias = FALSE,
objective = c("wSSE", "wAIC"),
verbose = FALSE,
...
)
Arguments
formula |
A formula specifying the model to be tested. |
data |
A data frame containing the variables in the model. |
nPoint |
The number of random points to generate in the factor space (default: 2000). |
nSVEM |
The number of SVEM models to fit to the original data (default: 5). |
nPerm |
The number of SVEM models to fit to permuted data for reference distribution (default: 125). |
percent |
The percentage of variance to capture in the SVD (default: 90). |
nBoot |
The number of bootstrap iterations within SVEM (default: 200). |
glmnet_alpha |
The alpha parameter(s) for glmnet (default: |
weight_scheme |
The weight scheme to use in SVEM (default: "SVEM"). Valid options are "SVEM" and "FWR". |
debias |
Logical; debiasing option passed to |
objective |
Character; the objective function to use in |
verbose |
Logical; if |
... |
Additional arguments passed to the underlying |
Details
The 'svem_significance_test' function implements a whole-model test designed to gauge the significance of a fitted SVEM model compared to the null hypothesis of a constant response surface. This method helps identify responses that have relatively stronger or weaker relationships with study factors.
The test constructs standardized predictions by centering the SVEM predictions by the response mean and scaling by the ensemble standard deviation. A reference distribution is created by fitting the SVEM model to multiple randomized permutations of the response vector. The Mahalanobis distances of the original and permuted models are calculated using a reduced-rank singular value decomposition.
The R code to perform this test (using matrices of nSVEM and nPerm predictions) is taken from the supplementary material of Karl (2024).
Value
A list containing the test results.
Acknowledgments
Development of this package was assisted by GPT o1-preview, which helped in constructing the structure of some of the code and the roxygen documentation. The code for the significance test is taken from the supplementary material of Karl (2024) (it was handwritten by that author).
References
Karl, A. T. (2024). A randomized permutation whole-model test heuristic for Self-Validated Ensemble Models (SVEM). Chemometrics and Intelligent Laboratory Systems, 249, 105122. doi:10.1016/j.chemolab.2024.105122
Examples
# Simulate data
set.seed(0)
n <- 21
X1 <- runif(n)
X2 <- runif(n)
X3 <- runif(n)
y <- 1 + X1 + X2 + X1 * X2 + X1^2 + rnorm(n)
data <- data.frame(y, X1, X2, X3)
# Perform the SVEM significance test
test_result <- svem_significance_test(
y ~ (X1 + X2 + X3)^2 + I(X1^2) + I(X2^2) + I(X3^2),
data = data,
nPoint = 2000,
nSVEM = 5,
nPerm = 125,
nBoot = 200
)
# View the p-value
test_result$p_value
test_result2 <- svem_significance_test(
y ~ (X1 + X2 )^2 + I(X1^2) + I(X2^2),
data = data,
nPoint = 2000,
nSVEM = 5,
nPerm = 125,
nBoot = 200
)
# View the p-value
test_result2$p_value
# Plot the Mahalanobis distances
plot(test_result,test_result2)