[AMORO-3943] Shade all third party classes for spark runtime

[turboFei]

Why are the changes needed?

Close #3943.

all not shaded class

jar -tf amoro-format-mixed/amoro-mixed-spark/v3.5/amoro-mixed-spark-runtime-3.5/target/amoro-format-mixed-spark-runtime-3.5_2.12-0.9-SNAPSHOT.jar |grep '.class$' |grep -v amoro|grep -v shade|grep -v 'org/apache/spark'|wc -l                      
    1077

org.apache.thrift

jar -tf amoro-format-mixed/amoro-mixed-spark/v3.5/amoro-mixed-spark-runtime-3.5/target/amoro-format-mixed-spark-runtime-3.5_2.12-0.9-SNAPSHOT.jar |grep '.class$' |grep -v amoro|grep -v shade|grep -v 'org/apache/spark'|grep  'org/apache/thrift'|wc -l
     179

org.apache.commons.io

jar -tf amoro-format-mixed/amoro-mixed-spark/v3.5/amoro-mixed-spark-runtime-3.5/target/amoro-format-mixed-spark-runtime-3.5_2.12-0.9-SNAPSHOT.jar |grep '.class$' |grep -v amoro|grep -v shade|grep -v 'org/apache/spark'|grep  'org/apache/commons/io' | wc -l
     201

com.github.benmanes

jar -tf amoro-format-mixed/amoro-mixed-spark/v3.5/amoro-mixed-spark-runtime-3.5/target/amoro-format-mixed-spark-runtime-3.5_2.12-0.9-SNAPSHOT.jar |grep '.class$' |grep -v amoro|grep -v shade|grep -v 'org/apache/spark'|grep  'com/github/benmanes' | wc -l
     692

org.threeten from orc-core https://github.com/apache/orc/tree/main/java/core/src/java/org/threeten

jar -tf amoro-format-mixed/amoro-mixed-spark/v3.5/amoro-mixed-spark-runtime-3.5/target/amoro-format-mixed-spark-runtime-3.5_2.12-0.9-SNAPSHOT.jar |grep '.class$' |grep -v amoro|grep -v shade|grep -v 'org/apache/spark'|grep  'org/threeten' | wc -l      
       4

others:

jar -tf amoro-format-mixed/amoro-mixed-spark/v3.5/amoro-mixed-spark-runtime-3.5/target/amoro-format-mixed-spark-runtime-3.5_2.12-0.9-SNAPSHOT.jar |grep '.class$' |grep -v amoro|grep -v shade|grep -v 'org/apache/spark'|grep -v 'org/apache/thrift' |grep -v 'org/apache/commons/io' |grep -v 'com/github/benmanes' |grep -v 'org/threeten'

META-INF/versions/9/module-info.class

Brief change log

• Shade all third party classes for spark runtime

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

./mvnw clean package -Pspark-3.5 -Pscala-2.12 -pl amoro-format-mixed/amoro-mixed-spark/v3.5/amoro-mixed-spark-runtime-3.5 -am -DskipTests     

all classes shaded

./mvnw clean package -Pspark-3.3 -Pscala-2.12 -pl amoro-format-mixed/amoro-mixed-spark/v3.3/amoro-mixed-spark-runtime-3.3 -am -DskipTests     

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[turboFei]

cc @zhoujinsong @xxubai

[turboFei]

cc @zhoujinsong

[zhoujinsong]

I left a comment, PTAL.

[zhoujinsong]

Should we manege the filter in the plugin manager like :https://github.com/apache/amoro/blob/master/pom.xml#L1560

[turboFei]

It would not work once we override the configuration in sub module