What does OPTIMIZE do in Lakehouse Delta lake table maintenance?

What does OPTIMIZE do in Lakehouse Delta lake table maintenance?

• A. Consolidates multiple Parquet files into a large file
• B. Removes old files no longer referenced by a Delta table log
• C. Applies optimized sorting, encoding, and compression to Delta parquet files to enable fast read operations
• D. Allows users to reference data without copying it

Answer: (A)

OPTIMIZE is used to compact small Parquet files into larger ones to improve query performance. When data is written incrementally, many tiny files can accumulate, which increases the overhead of reading data—more file open/close actions and more metadata to scan. By rewriting these fragments into fewer, larger Parquet files, Delta Lake reduces the number of files a query must touch, speeding up scans. You can also pair OPTIMIZE with ZORDER to physically cluster related data, further boosting performance for range queries. The other options describe different maintenance tasks—removing older files, changing sorting/encoding/compression in ways not specific to this operation, or referencing data without copying—that aren’t what OPTIMIZE primarily does.

OPTIMIZE is used to compact small Parquet files into larger ones to improve query performance. When data is written incrementally, many tiny files can accumulate, which increases the overhead of reading data—more file open/close actions and more metadata to scan. By rewriting these fragments into fewer, larger Parquet files, Delta Lake reduces the number of files a query must touch, speeding up scans. You can also pair OPTIMIZE with ZORDER to physically cluster related data, further boosting performance for range queries. The other options describe different maintenance tasks—removing older files, changing sorting/encoding/compression in ways not specific to this operation, or referencing data without copying—that aren’t what OPTIMIZE primarily does.