dbt
Mid-levelWhy does my first dbt or Spark query of the day take forever?
Auto-suspend is a real cost control, not a bug - but it means the first query after idle time pays to wake something back up, and how much depends entirely on what actually went to sleep.
"Slow first query" means something completely different depending on whether it's a suspended Snowflake warehouse waking up or a terminated EMR cluster provisioning new nodes from scratch.
What auto-suspend is actually trading off
Snowflake/BigQuery warehouses, and EMR/Databricks clusters configured to terminate on idle, suspend specifically to stop billing for compute nobody's querying - that's a deliberate, real cost control, not an oversight. The trade is that the next query after idle time pays to bring that compute back, instead of running against something already warm.
How big the cold start actually is
Resuming a suspended Snowflake or BigQuery warehouse is usually single-digit seconds. A terminated EMR or Databricks cluster is a different order of magnitude entirely - minutes to provision new nodes from scratch. "First query of the day is slow" means something very different depending on which layer actually went cold, and the fix for one does nothing for the other.
The two real fixes, and their actual cost
A scheduled keep-warm query - a trivial SELECT 1 on a cron just before the first real workload - trades a small, predictable compute cost for eliminating the cold start; it only makes sense if that cost is genuinely smaller than what auto-suspend was saving. The alternative, lengthening the auto-suspend timeout, pays a bigger idle-time cost across the board instead of a small targeted one.
See what this looks like on your own pipeline.
Upload a real Spark event log, dbt run_results.json, or Flink metrics export and get
concrete, approve-before-apply recommendations back - not another rule of thumb.