AWS Certified Data Engineer Associate · Free Practice Question Medium
Question 40
A company's real-time streaming application is running on AWS. As the data is ingested, a job runs on the data and takes 30 minutes to complete. The workload frequently experiences high latency due to the large volume of incoming data. A data engineer needs to design a scalable and serverless solution to enhance performance.
Which combination of steps do you recommend? (Select two)
-
A
Set up Amazon Kinesis Data Streams to ingest the data
-
B
Set up AWS Database Migration Service (AWS DMS) to ingest the data
-
C
Set up AWS Lambda with AWS Step Functions to process the data
-
D
Set up AWS Fargate with Amazon ECS to process the data
-
E
Provision Amazon EC2 instances in an Auto Scaling group to process the data
Reveal correct answers
Correct answers: A, D
Explanation
Correct options:
Set up Amazon Kinesis Data Streams to ingest the data
Set up AWS Fargate with Amazon ECS to process the data
Amazon Kinesis Data Streams (KDS) is a massively scalable and durable real-time data streaming service. KDS can continuously capture gigabytes of data per second from hundreds of thousands of sources such as website clickstreams, database event streams, financial transactions, social media feeds, IT logs, and location-tracking events. The data collected is available in milliseconds to enable real-time analytics use cases such as real-time dashboards, real-time anomaly detection, dynamic pricing, and more.
AWS Fargate is a serverless compute engine for containers that works with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS). Fargate makes it easy for you to focus on building your applications. Fargate removes the need to provision and manage servers, lets you specify and pay for resources per application, and improves security through application isolation by design.
For the given use case, we can use Kinesis Data Streams as the ingestion layer and the containerized ECS application on AWS Fargate as the processing layer. Both these components are serverless and can scale to offer the desired performance.
Incorrect options:
Set up AWS Database Migration Service (AWS DMS) to ingest the data - AWS Database Migration Service helps you migrate databases to AWS quickly and securely. DMS cannot be used for real-time data ingestion. Hence, this option is incorrect.
Set up AWS Lambda with AWS Step Functions to process the data - The maximum timeout value for any AWS Lambda function is 15 minutes. When the specified timeout is reached, AWS Lambda terminates the execution of your Lambda function. Since the use case talks about a job that runs for 30 minutes, AWS Lambda is not the right fit.
Provision Amazon EC2 instances in an Auto Scaling group to process the data - The given requirement is for a serverless solution to process the data. Hence, provisioning an Amazon EC2 instance is clearly not the right solution.
Reference:
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
