Which description best matches a broadcast join in Spark?

Which description best matches a broadcast join in Spark?

• A. It sorts both DataFrames by key before joining.
• B. It performs a Cartesian product to complete the join.
• C. It sends the smaller DataFrame to each executor to perform local joins.
• D. It duplicates the large DataFrame on every partition.

Answer: (C)

Broadcast join focuses on using a small dataset to drive the join by distributing it to every worker. Spark ships the smaller DataFrame to all executors so each executor can join its local partition of the larger DataFrame with that in-memory copy, avoiding shuffling the big dataset across the cluster. This approach reduces network I/O and speeds up the join when one side is small enough to fit in each executor’s memory. It’s most effective when the smaller DataFrame truly is compact; Spark can auto-broadcast or you can hint the system to do so. That explains why the description matches: the smaller DataFrame is sent to each executor to perform local joins. The other options describe other join behaviors—sorting both sides before joining (a sort-merge style), performing a Cartesian product (which is not a typical or efficient join), or duplicating the large DataFrame on every partition (which would be inefficient and not how broadcast joins operate).

Broadcast join focuses on using a small dataset to drive the join by distributing it to every worker. Spark ships the smaller DataFrame to all executors so each executor can join its local partition of the larger DataFrame with that in-memory copy, avoiding shuffling the big dataset across the cluster. This approach reduces network I/O and speeds up the join when one side is small enough to fit in each executor’s memory. It’s most effective when the smaller DataFrame truly is compact; Spark can auto-broadcast or you can hint the system to do so.

That explains why the description matches: the smaller DataFrame is sent to each executor to perform local joins. The other options describe other join behaviors—sorting both sides before joining (a sort-merge style), performing a Cartesian product (which is not a typical or efficient join), or duplicating the large DataFrame on every partition (which would be inefficient and not how broadcast joins operate).