[#1591] feat(spark): Support Spark 3.5.1 (#1592)
### What changes were proposed in this pull request?
Support Spark 3.5.1
The abstract class `TaskContext` introduced a new method `isFailed()` after Spark version 3.5.1. Due to this addition, it is not possible to compile the `MockTaskContext` class using Spark 3.5.1 without modifying the code. Consequently, we also added the `isFailed()` method to `MockTaskContext`. However, to maintain compatibility with versions of Spark prior to 3.5.1, we did not annotate this method with `@Override`.
Compiling old code with Spark 3.5.1 results in the following error:
```error: MockTaskContext is not abstract and does not override abstract method isFailed() in TaskContext.```
Additionally, adding the `@Override` annotation leads to another error:
```error: method does not override or implement a method from a supertype.```
### Why are the changes needed?
Fix: #1591
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests.
diff --git a/integration-test/spark3/src/test/java/org/apache/uniffle/test/GetReaderTest.java b/integration-test/spark3/src/test/java/org/apache/uniffle/test/GetReaderTest.java
index 32deea3..e1b5dc0 100644
--- a/integration-test/spark3/src/test/java/org/apache/uniffle/test/GetReaderTest.java
+++ b/integration-test/spark3/src/test/java/org/apache/uniffle/test/GetReaderTest.java
@@ -297,6 +297,12 @@
return false;
}
+ // The following method is only available after Spark 3.5.1, and in order to be compatible
+ // with the version before Spark 3.5.1, the annotation @Override is not added.
+ public boolean isFailed() {
+ return false;
+ }
+
@Override
public Properties getLocalProperties() {
return null;
diff --git a/pom.xml b/pom.xml
index 2491416..69dc4d8 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1761,7 +1761,7 @@
<id>spark3.5</id>
<properties>
<scala.binary.version>2.12</scala.binary.version>
- <spark.version>3.5.0</spark.version>
+ <spark.version>3.5.1</spark.version>
<client.type>3</client.type>
<jackson.version>2.15.2</jackson.version>
<log4j.version>2.20.0</log4j.version>
