Databricks Certified Data Engineer Professional · Free Practice Question Hard
Question 7
A data engineer is using the following spark configurations in a pipeline to enable Optimized Writes and Auto Compaction:
- spark.conf.set("spark.databricks.delta.optimizeWrite.enabled", True)
- spark.conf.set("spark.databricks.delta.autoCompact.enabled", True)
They also want to enable Z-order indexing with Auto Compaction to leverage data skipping on all the pipeline’s tables.
Which of the following solutions allows the data engineer to complete this task ?
-
A
Use spark.conf.set("spark.databricks.delta.autoZorder.enabled", True)
-
B
Use spark.conf.set("spark.databricks.delta.autoCompact.zorder.enabled", True)
-
C
Z-order indexing with Auto Compaction can only be enabled on each table separately using:
ALTER TABLE table_name
SET TBLPROPERTIES (delta.autoOptimize.zorder.enabled = true)
-
D
There is no need for extra configurations. Z-Ordering is enabled by default with Auto Compaction
-
E
There is no way to enable Z-order indexing with Auto Compaction since it does not support Z-Ordering
Reveal correct answer
Correct answer: E
Explanation
Auto Compaction does not support Z-Ordering as Z-Ordering is significantly more expensive than just compaction.
Reference:
https://docs.databricks.com/delta/tune-file-size.html#when-to-run-optimize
Study materials from our exam preparation course on Udemy:
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
