Working with Tuning Parameters

Numeric Parameters

An example of a numeric tuning parameter is the cost-complexity parameter of CART trees, otherwise known as \(C_p\). A parameter object for \(C_p\) can be created in dials using:

library(dials)
cost_complexity()
#> Cost-Complexity Parameter (quantitative)
#> Transformer: log-10 [1e-100, Inf]
#> Range (transformed scale): [-10, -1]

Note that this parameter is handled in log units and the default range of values is between 10^-10 and 0.1. The range of possible values can be returned and changed based on some utility functions. We’ll use the pipe operator here:

library(dplyr)
cost_complexity() %>% range_get()
#> $lower
#> [1] 1e-10
#> 
#> $upper
#> [1] 0.1
cost_complexity() %>% range_set(c(-5, 1))
#> Cost-Complexity Parameter (quantitative)
#> Transformer: log-10 [1e-100, Inf]
#> Range (transformed scale): [-5, 1]

# Or using the `range` argument
# during creation
cost_complexity(range = c(-5, 1))
#> Cost-Complexity Parameter (quantitative)
#> Transformer: log-10 [1e-100, Inf]
#> Range (transformed scale): [-5, 1]

Values for this parameter can be obtained in a few different ways. To get a sequence of values that span the range:

# Natural units:
cost_complexity() %>% value_seq(n = 4)
#> [1] 1e-10 1e-07 1e-04 1e-01

# Stay in the transformed space:
cost_complexity() %>% value_seq(n = 4, original = FALSE)
#> [1] -10  -7  -4  -1

Random values can be sampled too. A random uniform distribution is used (between the range values). Since this parameter has a transformation associated with it, the values are simulated in the transformed scale and then returned in the natural units (although the original argument can be used here):

set.seed(5473)
cost_complexity() %>% value_sample(n = 4)
#> [1] 6.91e-09 8.46e-04 3.45e-06 5.90e-10

For CART trees, there is a discrete set of values that exist for a given data set. It may be a good idea to assign these possible values to the object. We can get them by fitting an initial rpart model and then adding the values to the object. For mtcars, there are only three values:

library(rpart)
cart_mod <- rpart(mpg ~ ., data = mtcars, control = rpart.control(cp = 0.000001))
cart_mod$cptable
#>         CP nsplit rel error xerror  xstd
#> 1 0.643125      0     1.000  1.064 0.258
#> 2 0.097484      1     0.357  0.687 0.180
#> 3 0.000001      2     0.259  0.576 0.126
cp_vals <- cart_mod$cptable[, "CP"]

# We should only keep values associated with at least one split:
cp_vals <- cp_vals[ cart_mod$cptable[, "nsplit"] > 0 ]

# Here the specific Cp values, on their natural scale, are added:
mtcars_cp <- cost_complexity() %>% value_set(cp_vals)
#> Error in `value_set()`:
#> ! Some values are not valid: 0.0974840733898344 and 1e-06.

The error occurs because the values are not in the transformed scale:

mtcars_cp <- cost_complexity() %>% value_set(log10(cp_vals))
mtcars_cp
#> Cost-Complexity Parameter (quantitative)
#> Transformer: log-10 [1e-100, Inf]
#> Range (transformed scale): [-10, -1]
#> Values: 2

Now, if a sequence or random sample is requested, it uses the set values:

mtcars_cp %>% value_seq(2)
#> [1] 0.097484 0.000001
# Sampling specific values is done with replacement
mtcars_cp %>% 
  value_sample(20) %>% 
  table()
#> .
#>              1e-06 0.0974840733898344 
#>                  9                 11

Any transformations from the scales package can be used with the numeric parameters, or a custom transformation generated with scales::trans_new().

trans_raise <- scales::trans_new(
  "raise", 
  transform = function(x) 2^x , 
  inverse = function(x) -log2(x)
)
custom_cost <- cost(range = c(1, 10), trans = trans_raise)
custom_cost
#> Cost (quantitative)
#> Transformer: raise [-Inf, Inf]
#> Range (transformed scale): [1, 10]

Note that if a transformation is used, the range argument specifies the parameter range on the transformed scale. For this version of cost(), parameter values are sampled between 1 and 10 and then transformed back to the original scale by the inverse -log2(). So on the original scale, the sampled values are between -log2(10) and -log2(1).

-log2(c(10, 1))
#> [1] -3.32  0.00
value_sample(custom_cost, 100) %>% range()
#> [1] -3.3172 -0.0314

Discrete Parameters

In the discrete case there is no notion of a range. The parameter objects are defined by their discrete values. For example, consider a parameter for the types of kernel functions that is used with distance functions:

weight_func()
#> Distance Weighting Function  (qualitative)
#> 'rectangular', 'triangular', 'epanechnikov', 'biweight', 'triweight', 'cos', ...

The helper functions are analogues to the quantitative parameters:

# redefine values
weight_func() %>% value_set(c("rectangular", "triangular"))
#> Distance Weighting Function  (qualitative)
#> 'rectangular' and 'triangular'
weight_func() %>% value_sample(3)
#> [1] "triangular" "inv"        "triweight"

# the sequence is returned in the order of the levels
weight_func() %>% value_seq(3)
#> [1] "rectangular"  "triangular"   "epanechnikov"