Delta Does not Build Under Java > 8

[anoopj]

Bug

Describe the problem

Steps to reproduce

1. git clone https://github.com/delta-io/delta.git
2. build/sbt compile (with a newer JVM: I can repro this with Java 11 and Java 17, both on OSX and Ubuntu

Observed results

Compilation errors

[error] /home/anoop/delta/iceberg/src/main/scala/org/apache/spark/sql/delta/icebergShaded/IcebergConversionTransaction.scala:35:8: not found: object shadedForDelta
[error] import shadedForDelta.org.apache.iceberg.{AppendFiles, DeleteFiles, OverwriteFiles, PendingUpdate, RewriteFiles, Transaction => IcebergTransaction}

This happened because the build script used in Uniform (delta/icebergShaded/generate_iceberg_jars.py) fails because of a Gradle error.

With JDK8, the build succeeds cleanly.

Expected results

Users should be able to build/test Delta with newer JDK versions.

[anoopj]

I think this happened because delta/icebergShaded/generate_iceberg_jars.py checks out an older version of Iceberg with a specific commit hash. In that version, Iceberg Gradle build requires JDK 8 or 11 to be used. Newer versions of Iceberg support JDK 11, 17 and 21.

So this is pretty much an Iceberg upgrade problem.