· Aug 4, 2021

Cross Validation in IntegratedML


I'd like to know if the VALIDATE MODEL statement in IntegratedML performs a cross validation (CV) procedure internally. If it doesn't, is there a ready to use method for CV?


Discussion (2)1
Log in or sign up to continue

No, the VALIDATE MODEL statement does not do cross validation. It calculates validation metrics for the given trained model and dataset. As described in the "Model Selection Process" section of the documentation, however, the TRAIN MODEL statement does this to some extent for classification models when using the AutoML provider:

These scoring metrics are then computed for each model using Monte Carlo cross validation, with three training/testing splits of 70%/30%, to determine the best model.

I also believe that the DataRobot provider incorporates cross validation into its training. I'm not sure about H2O.