Question 13
Domain 4: Evaluation, Experimentation, and Data AnalysisIn A/B testing for LLM deployment, what is the recommended traffic distribution approach?
Correct answer: B
Explanation
A/B testing compares two deployed variants under the same conditions, so both models should be live and receive comparable traffic. Serving traffic equally avoids bias from uneven exposure and lets performance differences reflect the models rather than the routing split.
Why each option is right or wrong
A. Send all traffic to the new model immediately
B. Deploy both models and serve traffic equally
A/B testing requires both variants to be live at the same time so their outputs are measured under the same operating conditions; in deployment practice, the standard split is 50/50 traffic unless a different experimental design is explicitly specified. An unequal routing split would confound the comparison by giving one model more exposure and more stable estimates, so the correct setup is to deploy both models and allocate traffic evenly.
C. Use only internal testing
D. Gradually increase traffic over months