AWS Certified Machine Learning Engineer Associate · Free Practice Question Medium
Question 21
Which of the following best describes how Amazon SageMaker handles data parallelism during distributed training?
-
A
Splits the dataset and model across multiple machines, ensuring both are distributed equally
-
B
Splits the dataset across multiple machines, each training a different model
-
C
Splits the model across multiple machines and synchronizes the model parameters
-
D
Splits the dataset across multiple machines, each training the same model on a portion of the data
Reveal correct answer
Correct answer: D
A.
Hybrid parallelism refers to splitting both the dataset and model, but in data parallelism, only the dataset is divided, not the model.
B.
Each machine trains the same model on a portion of the data, not a different model.
C.
This describes model parallelism, not data parallelism. In model parallelism, the model is split across multiple machines.
D.
In data parallelism, the dataset is divided into chunks, and each chunk is assigned to a different machine (or worker). Each worker trains the same model on its portion of the data, and then SageMaker synchronizes the gradients across all workers to update the model.
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
