svem_significance_test {SVEMnet}R Documentation

SVEM Significance Test

Description

Performs a whole-model significance test using the SVEM framework, handling both continuous and categorical predictors.

Usage

svem_significance_test(
  formula,
  data,
  nPoint = 2000,
  nSVEM = 5,
  nPerm = 125,
  percent = 85,
  nBoot = 200,
  glmnet_alpha = c(1),
  weight_scheme = c("SVEM"),
  objective = c("wAIC", "wSSE"),
  verbose = TRUE,
  ...
)

Arguments

formula

A formula specifying the model to be tested.

data

A data frame containing the variables in the model.

nPoint

The number of random points to generate in the factor space (default: 2000).

nSVEM

The number of SVEM models to fit to the original data (default: 5).

nPerm

The number of SVEM models to fit to permuted data for reference distribution (default: 125).

percent

The percentage of variance to capture in the SVD (default: 85).

nBoot

The number of bootstrap iterations within SVEM (default: 200).

glmnet_alpha

The alpha parameter(s) for glmnet (default: c(1)).

weight_scheme

The weight scheme to use in SVEM (default: "SVEM").

objective

Character; the objective function to use in SVEMnet. Options are "wAIC" or "wSSE" (default: "wAIC").

verbose

Logical; if TRUE, displays progress messages (default: TRUE).

...

Additional arguments passed to the underlying SVEMnet() and then glmnet() functions.

Details

The 'svem_significance_test' function implements a whole-model test designed to gauge the significance of a fitted SVEM model compared to the null hypothesis of a constant response surface. This method helps identify responses that have relatively stronger or weaker relationships with study factors.

The test constructs standardized predictions by centering the SVEM predictions (obtained from SVEMnet()) by the response mean and scaling by the ensemble standard deviation. A reference distribution is created by fitting the SVEM model to multiple randomized permutations of the response vector. The Mahalanobis distances of the original and permuted models are calculated using a reduced-rank singular value decomposition.

The R code to perform this test (using matrices of nSVEM and nPerm predictions) is taken from the supplementary material of Karl (2024).

This function assumes that there are no restrictions among the factors (e.g. mixture factors). The method will work with restrictions, but the code would need to be changed to ensure the nPoint points respect the factor restriction(s). For example, rdirichlet() could be used for the mixture factors.

The SVEMnet parameter debias is hard coded to FALSE for this test. Unpublished simulation work suggests that setting debias=TRUE reduces the power of the test (without affecting the Type I error rate).

Value

A list containing the test results.

Acknowledgments

Development of this package was assisted by GPT o1-preview, which helped in constructing the structure of some of the code and the roxygen documentation. The code for the significance test is taken from the supplementary material of Karl (2024) (it was handwritten by that author).

References

Karl, A. T. (2024). A randomized permutation whole-model test heuristic for Self-Validated Ensemble Models (SVEM). Chemometrics and Intelligent Laboratory Systems, 249, 105122. doi:10.1016/j.chemolab.2024.105122

Examples


# Simulate data
set.seed(1)
n <- 30
X1 <- runif(n)
X2 <- runif(n)
X3 <- runif(n)
y <- 1 + X1 +  X2 + X1 * X2 + X1^2 + rnorm(n)
data <- data.frame(y, X1, X2, X3)

# Perform the SVEM significance test
test_result <- svem_significance_test(
  y ~ (X1 + X2 + X3)^2 + I(X1^2) + I(X2^2) + I(X3^2),
  data = data,
  nPoint = 2000,
  nSVEM = 7,
  nPerm = 150,
  nBoot = 200

)

# View the p-value
print(test_result)


test_result2 <- svem_significance_test(
  y ~ (X1 + X2 )^2 + I(X1^2) + I(X2^2),
  data = data,
  nPoint = 2000,
  nSVEM = 7,
  nPerm = 150,
  nBoot = 200
)

# View the p-value
print(test_result2)


# Plot the Mahalanobis distances
plot(test_result,test_result2)



[Package SVEMnet version 1.3.0 Index]