[ZEPPELIN-6701] Support Spark 4.1, 4.2 and 4.3

### What is this PR for?

Adds official Spark 4.1, 4.2 and 4.3 support.

1. Adds `spark-4.1`, `spark-4.2` and `spark-4.3` build profiles, using Spark `4.1.3`, `4.2.0` and `4.3.0`.
2. Extends the supported Spark version range to `3.3`-`4.3`; `4.4` is the unsupported-future boundary.
3. Restructures the `spark-test` job as a matrix-driven job covering Spark, Scala, Java, and Python combinations.
4. Runs Spark 4.1, 4.2 and 4.3 tests with JDK 17 and Python 3.14.
5. Adds a Python 3.14 conda lock resolved from conda-forge, and pins the interpreter in the test environment YAMLs.
6. Updates the Spark build documentation.

### What type of PR is it?

Feature

### How should this be tested?

CI validates all supported combinations through the matrix-driven `spark-test` job. The Python 3.14 runs install `testing/locks/python-3.14-linux-64.conda.lock`, which carries the Spark-derived dependency constraints, including `pandas>=2.2,<3.0`, `numpy>=1.21`, `pyarrow>=18`, and `grpcio>=1.76,<2`.

Locally verified:

```bash
./mvnw package -pl spark/interpreter -am -DskipTests -Pspark-4.1 -Pspark-scala-2.13
./mvnw package -pl spark/interpreter -am -DskipTests -Pspark-4.2 -Pspark-scala-2.13
./mvnw test -pl spark/interpreter -Pspark-4.3 -Pspark-scala-2.13 -Dtest=SparkVersionTest -DfailIfNoTests=false
```

Also verified the Python 3.14 lock solve and installation.

### Screenshots

N/A

### How this resolves?

Provides the Maven profiles, runtime version boundary, CI coverage, and documentation needed to test and build against Spark 4.1, 4.2 and 4.3.

### Questions

1. Does the licenses file need to update? No
2. Is there breaking changes for older versions? No
3. Does this need documentation? Yes


Closes #5465 from pan3793/spark-41-42.

Signed-off-by: Jongyoul Lee <jongyoul@gmail.com>