spark.sql("OPTIMIZE .delta 'path'") dt = DeltaTable.forPath(spark, 'path')
Then get the history and operations from
A quicker way would be to pull this data immediately after the query:
data = Spark.sql("OPTIMIZE .delta 'path'")
And grab metrics from data, but there are no metrics for p25FileSize, p50FileSize, p75FileSize. Is there a way to get these metrics without calling the .history() command?