AWS Certified Machine Learning Specialty · Free Practice Question Medium
Question 15
An analytics company wants to create a text summarization model based on the SageMaker seq2seq algorithm. However the training data is in the form of 1TB of flat files whereas seq2seq only expects RecordIO-Protobuf format.
As an ML Specialist, which of the following solutions would you recommend?
-
A
Use Kinesis Data Firehose to transform the data into RecordIO-Protobuf format
-
B
Use AWS Glue to create an ETL job to write the data in RecordIO-Protobuf format on S3. This data can be used by seq2seq based model for training
-
C
Use AWS Step function with Lambda to write the data in RecordIO-Protobuf format on S3. This data can be used by seq2seq based model for training
-
D
Spin-up an Apache Spark EMR cluster to transform the data from flat files into RecordIO-Protobuf format and save it on S3. This data can be used by seq2seq based model for training
Reveal correct answer
Correct answer: D
Explanation
Correct option:
Spin-up an Apache Spark EMR cluster to transform the data from flat files into RecordIO-Protobuf format and save it on S3. This data can be used by seq2seq based model for training
Apache Spark is a distributed processing framework and programming model that helps you do machine learning, stream processing, or graph analytics using Amazon EMR clusters. Amazon EMR is a web service that makes it easy for you to process and analyze vast amounts of data using applications in the Hadoop ecosystem, including Hive, Pig, HBase, Presto, Impala, and others.
Apache Spark (running on the EMR cluster in this use-case) can write the output in RecorIO-Protobuf format.
Incorrect options:
Use AWS Glue to create an ETL job to write the data in RecordIO-Protobuf format on S3. This data can be used by seq2seq based model for training - AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. AWS Glue provides both visual and code-based interfaces to make data integration easier. Users can easily find and access data using the AWS Glue Data Catalog. Data engineers and ETL (extract, transform, and load) developers can visually create, run, and monitor ETL workflows with a few clicks in AWS Glue Studio. Glue cannot write the output in RecordIO-Protobuf format.
Use AWS Step function with Lambda to write the data in RecordIO-Protobuf format on S3. This data can be used by seq2seq based model for training - AWS Step Functions is a low-code visual workflow service used to orchestrate AWS services, automate business processes, and build serverless applications. Lambda is not suited for long-running processes such as the task of transforming 1TB data into RecordIO-Protobuf format.
Use Kinesis Data Firehose to transform the data into RecordIO-Protobuf format - Kinesis Data Firehose can capture, transform, and load streaming data into Amazon S3, Amazon Redshift, Amazon Elasticsearch Service, and Splunk, enabling near real-time analytics. It is not meant to be used for batch processing use cases and it cannot write data in RecorIO-Protobuf format.
Reference:
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
