AWS Certified Data Engineer Associate · Free Practice Question Medium
Question 8
A company wants to ensure data accuracy and trustworthiness in its ETL pipeline. Which AWS service offers features to establish data lineage and track data movement across multiple AWS services?
-
A
Amazon S3
-
B
Amazon Kinesis
- C AWS Glue
-
D
AWS DataSync
Reveal correct answer
Correct answer: C
Explanation
AWS Glue, is the best choice for ensuring data accuracy and trustworthiness through data lineage and tracking capabilities because:
Data Catalog: AWS Glue provides a managed data catalog service that serves as a central metadata repository. This catalog stores metadata about data sources, transformations, and targets, which aids in understanding and tracking how data moves and transforms across the entire ETL pipeline.
Data Lineage: AWS Glue supports data lineage tracking, enabling visibility into the lifecycle and transformation of data. This feature helps in understanding data dependencies, troubleshooting data processing jobs, and ensuring data governance and compliance, which are crucial for maintaining data accuracy and trustworthiness.
Integration and Automation: Glue integrates with various AWS services, making it ideal for managing and orchestrating ETL jobs. It automates much of the effort in connecting, cleansing, and optimizing data movement, which enhances the reliability and consistency of data operations.
The other options, while useful in their respective functions, do not provide comprehensive data lineage and governance capabilities:
(AWS DataSync): DataSync is used for moving large volumes of data across AWS storage services or from on-premises storage to AWS efficiently. However, it does not provide features for tracking data lineage or managing data governance.
(Amazon S3): Amazon S3 is a scalable storage service and is often used as a data lake or storage endpoint in ETL pipelines but lacks built-in capabilities for data lineage and governance without integration with other services like AWS Glue.
(Amazon Kinesis): Kinesis is great for real-time data streaming and analytics but does not inherently offer data lineage features. It focuses more on data ingestion and processing rather than governance and tracking.
A. Amazon S3 is a scalable object storage service that allows users to store and retrieve data. While it is commonly used in ETL pipelines for storing data, it does not offer specific features for establishing data lineage and tracking data movement across multiple AWS services.
B. Amazon Kinesis is a platform for streaming data on AWS, allowing users to ingest, process, and analyze real-time data. While it is useful for real-time data processing, it does not provide features for establishing data lineage and tracking data movement in ETL pipelines.
C. AWS Glue is a fully managed extract, transform, and load (ETL) service that provides features to establish data lineage and track data movement across multiple AWS services. It can catalog data, generate metadata, and create data lineage to ensure data accuracy and trustworthiness in the ETL pipeline.
D. AWS DataSync is a service used for transferring data between on-premises storage and Amazon S3, Amazon EFS, or Amazon FSx. While it facilitates data transfer, it does not offer features specifically designed for establishing data lineage and tracking data movement across multiple AWS services.
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
