Which statement about broadcast joins in PySpark is true?

Which statement about broadcast joins in PySpark is true?

• A. They require the smaller DataFrame to be cached on each executor.
• B. They are always the fastest option regardless of data sizes.
• C. They avoid any data movement by using local joins only.
• D. They broadcast the smaller DataFrame to all workers to avoid shuffling.

Answer: (D)

Broadcast joins in PySpark work by sending the smaller DataFrame to every executor, so each partition can perform the join locally and you avoid shuffling the large side. This is typically done with a broadcast() helper or via automatic broadcast when the smaller side is under a configured size threshold. It’s advantageous when one side is small enough to fit in memory on each worker, reducing network I/O and the cost of a full shuffle, which can significantly speed up the join. The other statements aren’t right because: you don’t have to cache the smaller DataFrame on each executor as part of a broadcast; broadcasting uses a broadcast variable, not mandatory caching. It isn’t guaranteed to be the fastest option in every scenario—if the small DataFrame is still large or memory is constrained, broadcasting can cause memory pressure. And it isn’t true that no data movement occurs—the small DataFrame is replicated to all workers, which is data movement, even though it avoids shuffling the large DataFrame.

Broadcast joins in PySpark work by sending the smaller DataFrame to every executor, so each partition can perform the join locally and you avoid shuffling the large side. This is typically done with a broadcast() helper or via automatic broadcast when the smaller side is under a configured size threshold. It’s advantageous when one side is small enough to fit in memory on each worker, reducing network I/O and the cost of a full shuffle, which can significantly speed up the join.

The other statements aren’t right because: you don’t have to cache the smaller DataFrame on each executor as part of a broadcast; broadcasting uses a broadcast variable, not mandatory caching. It isn’t guaranteed to be the fastest option in every scenario—if the small DataFrame is still large or memory is constrained, broadcasting can cause memory pressure. And it isn’t true that no data movement occurs—the small DataFrame is replicated to all workers, which is data movement, even though it avoids shuffling the large DataFrame.