Databricks Certified Data Engineer Professional · Free Practice Question Medium
Question 6
The data engineering team has a pipeline that ingest Kafka source data into a Multiplex bronze table. This Delta table is partitioned based on the topic and month columns.
A new data engineer notices that the ‘user_activity’ topic contains Personal Identifiable Information (PII) that needs to to be deleted every two months based on the company’s Service-Level Agreement (SLA).
Which statement describes how table partitioning can help to meet this requirement ?
-
A
None of the above statements is correct. Table partitioning can not help to meet the specified requirement.
-
B
Table partitioning allows immediate files deletion without running VACUUM command
-
C
Table partitioning allows delete queries to leverage partition boundaries.
-
D
Table partitioning reduces query latency when deleting large data files
-
E
Table partitioning does not allow to time travel the PII data after deletion
Reveal correct answer
Correct answer: C
Explanation
Partitioning on datetime columns can be leveraged when removing data older than a certain age from the table. For example, you can decide to delete previous months data. In this case, file deletion will be cleanly along partition boundaries.
Similarly, data could be archived and backed up at partition boundaries to a cheaper storage tier. This drives a huge savings on cloud storage.
Reference: https://delta.io/blog/2023-01-18-add-remove-partition-delta-lake/
Study materials from our exam preparation course on Udemy:
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
