Databricks Certified Data Engineer Associate · Free Practice Question Easy
Question 24
You are asked to setup an AUTO LOADER to process the incoming data, this data arrives in JSON format and get dropped into cloud object storage and you are required to process the data as soon as it arrives in cloud storage, which of the following statements is correct
-
A
AUTO LOADER is native to DELTA lake it cannot support external cloud object storage
-
B
AUTO LOADER has to be triggered from an external process when the file arrives in the cloud storage
-
C
AUTO LOADER needs to be converted to a Structured stream process
-
D
AUTO LOADER can only process continuous data when stored in DELTA lake
-
E
AUTO LOADER can support file notification method so it can process data as it arrives
Reveal correct answer
Correct answer: E
Explanation
Auto Loader supports two modes when ingesting new files from cloud object storage
Directory listing: Auto Loader identifies new files by listing the input directory, and uses a directory polling approach.
File notification: Auto Loader can automatically set up a notification service and queue service that subscribe to file events from the input directory.
File notification is more efficient and can be used to process the data in real-time as data arrives in cloud object storage.
Choosing between file notification and directory listing modes | Databricks on AWS
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
