[iceberg] `Comet execution only takes Arrow Arrays, but got class org.apache.iceberg.spark.data.vectorized.ColumnVectorWithFilter`

[hsiang-c]

Describe the bug

• Take from GitHub CI of fix: [iceberg] Switch to OSS Spark and run Iceberg Spark tests in parallel #1987

TestMergeOnReadUpdate > testUpdateRefreshesRelationCache() > catalogName = testhadoop, implementation = org.apache.iceberg.spark.SparkCatalog, config = {type=hadoop}, format = PARQUET, vectorized = true, distributionMode = hash, fanout = true, branch = null, planningMode = LOCAL, formatVersion = 2 FAILED
    org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 632.0 failed 1 times, most recent failure: Lost task 0.0 in stage 632.0 (TID 732) (localhost executor driver): org.apache.spark.SparkException: Comet execution only takes Arrow Arrays, but got class org.apache.iceberg.spark.data.vectorized.ColumnVectorWithFilter

Steps to reproduce

SparkSession configs used:

            .config("spark.plugins", "org.apache.spark.CometPlugin")
            .config("spark.shuffle.manager", "org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager")
            .config("spark.comet.explainFallback.enabled", "true")
            .config("spark.sql.iceberg.parquet.reader-type", "COMET")
            .config("spark.memory.offHeap.enabled", "true")
            .config("spark.memory.offHeap.size", "10g")
            .config("spark.comet.use.lazyMaterialization", "false")
            .config("spark.comet.schemaEvolution.enabled", "true")

Expected behavior

No response

Additional context

No response

[hsiang-c]

take

[hsiang-c]

This is related to delete filters in Iceberg.

[hsiang-c]

This issue should be fixed by #2346

[andygrove]

@parthchandra @hsiang-c I think that this can be closed now. Please reopen if I am mistaken.