Why isn't Spark's adaptive query execution actually helping my job?

Two settings decide whether AQE does anything at all, and both can be set to defeat it without an error, a warning, or any sign in the Spark UI that it happened.

What AQE actually changes mid-query

After each shuffle stage completes, AQE has real partition sizes to work with - not the planner's upfront estimate. It uses that to coalesce many small shuffle partitions into fewer larger ones, switch a sort-merge join to a broadcast join if a side turns out small enough, and split a partition that's disproportionately large (skew) into several smaller tasks.

Why it's silently a no-op

The two usual culprits: spark.sql.adaptive.enabled explicitly set to false somewhere in a shared config, or spark.sql.shuffle.partitions hardcoded alongside coalescePartitions.enabled=false - which disables the one mechanism AQE applies automatically on almost every job. Neither failure shows up as an error; the job just runs exactly as it would have on Spark 2.x.

Where AQE can't save you

AQE only adapts after a shuffle boundary - a single-stage job with no shuffle at all gets nothing from it, and a job whose initial partitioning is bad well before any shuffle still pays that cost up front. AQE fixes what happens after a shuffle runs, not whether the read itself was partitioned sensibly.