sklift.metrics.average_squared_deviation

sklift.metrics.metrics.average_squared_deviation(y_true_train, uplift_train, treatment_train, y_true_val, uplift_val, treatment_val, strategy='overall', bins=10)[source]

Compute the average squared deviation.

The average squared deviation (ASD) is a model stability metric that shows how much the model overfits the training data. Larger values of ASD mean greater overfit.

Parameters:
  • y_true_train (1d array-like) – Correct (true) target values for training set.

  • uplift_train (1d array-like) – Predicted uplift for training set, as returned by a model.

  • treatment_train (1d array-like) – Treatment labels for training set.

  • y_true_val (1d array-like) – Correct (true) target values for validation set.

  • uplift_val (1d array-like) – Predicted uplift for validation set, as returned by a model.

  • treatment_val (1d array-like) – Treatment labels for validation set.

  • strategy (string, ['overall', 'by_group']) –

    Determines the calculating strategy. Default is ‘overall’.

    • 'overall':

      The first step is taking the first k observations of all test data ordered by uplift prediction (overall both groups - control and treatment) and conversions in treatment and control groups calculated only on them. Then the difference between these conversions is calculated.

    • 'by_group':

      Separately calculates conversions in top k observations in each group (control and treatment) sorted by uplift predictions. Then the difference between these conversions is calculated

  • bins (int) – Determines the number of bins (and the relative percentile) in the data. Default is 10.

Returns:

average squared deviation

Return type:

float

References

René Michel, Igor Schnakenburg, Tobias von Martens. Targeting Uplift. An Introduction to Net Scores.