Question 28
Domain 2: Core Machine Learning, AI, and Transformer FoundationsIn the context of training recurrent neural networks (RNNs) for sequence modeling, what is 'backpropagation through time' (BPTT)?
Correct answer: B
Explanation
Backpropagation through time is the method used to train RNNs by treating the network as a sequence of repeated layers across time. It "unfolds the RNN across time steps" and then applies standard backpropagation to compute gradients for the parameters "shared across all timesteps."
Why each option is right or wrong
A. A technique that reverses the processing order of input sequences during the forward pass to capture backward temporal dependencies, enabling bidirectional context modeling within a single recurrent pass
B. An algorithm that unfolds the RNN across time steps and applies standard backpropagation to compute gradients with respect to parameters shared across all timesteps
BPTT is the standard training procedure for an RNN: the recurrent cell is unrolled across the sequence length, creating a chain of time-indexed copies of the same computation graph, and ordinary backpropagation is then run through that unrolled graph to accumulate gradients. The key point is that the weights are shared at every timestep, so the gradient contributions from each step are summed for the same parameter set rather than learned separately for each position in the sequence.
C. A weight initialization strategy specifically designed for recurrent architectures that sets initial hidden states and weight matrices to prevent gradient vanishing during early training iterations
D. A cyclic learning rate scheduling method that systematically decreases the step size according to a predetermined decay schedule across training epochs to improve recurrent network convergence