Why I stopped guessing at Spark and dbt config values
I've spent more than a decade building data pipelines, and the part nobody warns you about isn't the pipeline logic. It's the tuning. Executor memory, shuffle partitions, cluster size, thread counts.
Aug 8, 20264 min read17
