Why isn't Spark pruning partitions even though I filtered on the partition column?

A query that filters on the exact partition column can still scan every partition in the table - and nothing about the error (there isn't one) tells you why.

What partition pruning requires to trigger

The filter has to be a direct, literal comparison on the partition column itself, in a type that matches how the partition is actually stored. The planner needs to be able to resolve which partition directories can be skipped before it reads a single file - anything that requires reading data first to evaluate isn't pruning, it's filtering after the fact.

The type-mismatch gotcha

A partition column stored as a string ("2026-11-02") compared against a date-typed literal - or vice versa - forces an implicit cast. Once the filter expression is wrapped in that cast, the planner generally can't push it down to partition pruning anymore; it reads every partition's files and filters afterward, silently, with no warning that pruning stopped applying.

Derived filters don't prune either

Filtering on a column derived from the partition column - date_trunc('month', event_date) on a table partitioned by event_date - isn't recognized as the same predicate by the planner. Only a direct comparison to the raw partition column, in its native type, actually prunes.