Question 21
Domain 3You created an ML pipeline with multiple input parameters. You want to investigate the tradeoffs between different parameter combinations. The parameter options are Input dataset Max tree depth of the boosted tree regressor Optimizer learning rate You need to compare the pipeline performance of the different parameter combinations measured in F1 score, time to train, and model complexity. You want your approach to be reproducible, and track all pipeline runs on the same platform. What should you do?
Correct answer: D
Explanation
Vertex AI Experiments is designed to “track all pipeline runs on the same platform” and compare metrics across runs. By creating a pipeline with the investigated parameters and submitting multiple runs to the same experiment, you can reproducibly evaluate tradeoffs in “F1 score, time to train, and model complexity” for different parameter combinations.
Why each option is right or wrong
A. 1. Use BigQueryML to create a boosted tree regressor, and use the hyperparameter tuning capability. 2. Configure the hyperparameter syntax to select different input datasets: max tree depths, and optimizer learning rates. Choose the grid search option.
B. 1. Create a Vertex AI pipeline with a custom model training job as part of the pipeline. Configure the pipeline’s parameters to include those you are investigating. 2. In the custom training step, use the Bayesian optimization method with F1 score as the target to maximize.
C. 1. Create a Vertex AI Workbench notebook for each of the different input datasets. 2. In each notebook, run different local training jobs with different combinations of the max tree depth and optimizer learning rate parameters. 3. After each notebook finishes, append the results to a BigQuery table.
D. 1. Create an experiment in Vertex AI Experiments. 2. Create a Vertex AI pipeline with a custom model training job as part of the pipeline. Configure the pipeline’s parameters to include those you are investigating. 3. Submit multiple runs to the same experiment, using different values for the parameters.
Vertex AI Experiments is the service intended to log and compare repeated training or pipeline executions under a single experiment context, so creating one experiment first gives a common place to store the run metadata and metrics. In Vertex AI Pipelines, pipeline parameters are explicitly supported as runtime inputs, and submitting multiple executions with different parameter values lets you compare the resulting F1 score, training time, and model complexity across runs in a reproducible way on the same platform.