Associate Data Practitioner · Free Practice Question Medium
Question 2
When migrating terabytes of historical data from a relational database to BigQuery, which loading approach would be most efficient?
-
A
Using the bq command-line tool with streaming inserts
-
B
Exporting to CSV and using BigQuery batch loading
-
C
Using Dataflow with JDBC source and BigQuery sink
-
D
Setting up Datastream for continuous replication
Reveal correct answer
Correct answer: B
Explanation
Option B is CORRECT. For large historical data migrations, exporting to CSV files and using BigQuery's batch loading functionality provides the most efficient approach. Batch loading is optimized for high-throughput ingestion of large datasets, allows for parallel loading of multiple files, and is significantly more cost-effective than streaming inserts for large volumes of historical data. This approach is simple and leverages BigQuery's native batch loading capabilities.
Option A is INCORRECT. Streaming inserts through the bq command-line tool would be inefficient and expensive for terabytes of historical data. Streaming inserts are designed for real-time data, have higher per-record costs, and would take substantially longer than batch loading for large datasets.
Option C is INCORRECT. While Dataflow with JDBC could work, it introduces additional complexity and potential bottlenecks compared to direct batch loading. For a one-time migration of historical data (as opposed to ongoing synchronization), the added flexibility of Dataflow isn't necessary and could slow down the migration.
Option D is INCORRECT. Datastream is designed for ongoing replication of changes rather than one-time migration of historical data. Setting up Datastream for a historical data migration would be overkill and less efficient than using BigQuery's batch loading capabilities directly.
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
