1.8.0 release of datafu-spark
diff --git a/RELEASE.md b/RELEASE.md
index 1f81ae2..0e97898 100644
--- a/RELEASE.md
+++ b/RELEASE.md
@@ -133,7 +133,7 @@
./gradlew uploadArchives -PnexusUsername=yourNexusUsername -PnexusPassword=yourNexusPassword
-The above command assumes you have configured `$HOME/.gradle/gradle.properties` with your GPG key information.
+The above command assumes you have configured `$HOME/.gradle/gradle.properties` with your GPG key information. If this fails with _Cannot invoke method readPassword() on null object_ or _Cannot invoke method readLine() on null object_, this is because the Gradle daemon is running. Disable it with _--no-daemon_ and try again. Please note that sometimes your JVM settings will prevent Gradle from launching without forking a daemon. Removing them from the gradlew script may help.
If you now visit the [Apache Nexus Repository](https://repository.apache.org) and click on Staging Repositories, you should see a repository named orgapachedatafu-xxxx, where xxxx is some number. Select the repository and browse the content to make sure the set of files looks right. If it looks correct then Close the repository. The repository is now ready for testing. If you look at the summary there is a URL for the repository that may be used to fetch the archives.
diff --git a/datafu-spark/README.md b/datafu-spark/README.md
index 0de223f..40b6ece 100644
--- a/datafu-spark/README.md
+++ b/datafu-spark/README.md
@@ -12,7 +12,7 @@
* [Count distinct up to](https://github.com/apache/datafu/blob/spark-tmp/datafu-spark/src/main/scala/datafu/spark/SparkUDAFs.scala#L224) - an efficient implementation when you just want to verify that a certain minimum of distinct rows appear in a table
-It has been tested on Spark releases from 2.2.0 to 2.4.3, using Scala 2.11 and 2.12. You can check if your Spark/Scala version combination has been tested by looking [here.](https://github.com/apache/datafu/blob/master/datafu-spark/build_and_test_spark.sh#L20)
+It has been tested on Spark releases from 2.2.2 to 2.4.5, using Scala 2.11 and 2.12. You can check if your Spark/Scala version combination has been tested by looking [here.](https://github.com/apache/datafu/blob/master/datafu-spark/build_and_test_spark.sh#L20)
-----------
@@ -21,9 +21,9 @@
First, call pyspark with the following parameters
```bash
-export PYTHONPATH=datafu-spark_2.11_2.3.0-1.7.0.jar
+export PYTHONPATH=datafu-spark_2.11-1.8.0.jar
-pyspark --jars datafu-spark_2.11_2.3.0-1.7.0.jar --conf spark.executorEnv.PYTHONPATH=datafu-spark_2.11_2.3.0-1.7.0.jar
+pyspark --jars datafu-spark_2.11-1.8.0.jar --conf spark.executorEnv.PYTHONPATH=datafu-spark_2.11-1.8.0.jar
```
The following is an example of calling the Spark version of the datafu _dedup_ method
diff --git a/datafu-spark/build.gradle b/datafu-spark/build.gradle
index d07e2c2..878fc10 100644
--- a/datafu-spark/build.gradle
+++ b/datafu-spark/build.gradle
@@ -155,6 +155,7 @@
systemProperty 'datafu.spark.version', sparkVersion
+// removed so the daemon is cancelled
maxHeapSize = "2G"
}
diff --git a/gradle.properties b/gradle.properties
index f6814f8..9455899 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -16,7 +16,7 @@
# under the License.
group=org.apache.datafu
-version=1.8.0-SNAPSHOT
+version=1.8.0
gradleVersion=5.6.4
sparkCompatVersion=2.4
sparkVersion=2.4.4
diff --git a/site/source/blog/2023-07-08-datafu-1-8-0-released.markdown b/site/source/blog/2023-07-08-datafu-1-8-0-released.markdown
new file mode 100644
index 0000000..f7b597d
--- /dev/null
+++ b/site/source/blog/2023-07-08-datafu-1-8-0-released.markdown
@@ -0,0 +1,46 @@
+---
+title: Apache DataFu-Spark 1.8.0 Released
+author: Eyal Allweil
+license: >
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+I'd like to announce the release of Apache DataFu-Spark 1.8.0.
+
+Many thanks to Arpit Bhardwaj and Shaked Aharon, who worked on this version.
+
+<br>
+
+**Improvements**
+
+* dedupWithCombiner method now supports a list of columns in the order / group by params (DATAFU-171)
+* Scala Python bridge now uses secure gateway (DATAFU-167)
+
+**Breaking changes**
+
+* Spark 2.2.0, 2.2.1, and 2.3.0. no longer supported
+
+<br>
+
+The source release can be obtained from:
+
+http://www.apache.org/dyn/closer.cgi/datafu/apache-datafu-1.8.0/apache-datafu-sources-1.8.0.tgz
+
+Artifacts for DataFu are published in Apache's Maven Repository:
+
+https://repository.apache.org/content/groups/public/org/apache/datafu/
+
+Please visit the [Download](/docs/download.html) page for instructions on building from source or retrieving the artifacts in your build system.
diff --git a/site/source/docs/download.html.markdown.erb b/site/source/docs/download.html.markdown.erb
index 7252ddc..188395b 100644
--- a/site/source/docs/download.html.markdown.erb
+++ b/site/source/docs/download.html.markdown.erb
@@ -1,7 +1,7 @@
---
title: Download - Apache DataFu
section_name: Getting Started
-version: 1.7.0
+version: 1.8.0
license: >
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
diff --git a/site/source/docs/spark/getting-started.html.markdown.erb b/site/source/docs/spark/getting-started.html.markdown.erb
index 47f70af..58f28bd 100644
--- a/site/source/docs/spark/getting-started.html.markdown.erb
+++ b/site/source/docs/spark/getting-started.html.markdown.erb
@@ -1,6 +1,6 @@
---
title: Apache DataFu Spark - Getting Started
-version: 1.7.0
+version: 1.8.0
section_name: Getting Started
license: >
Licensed to the Apache Software Foundation (ASF) under one or more
diff --git a/site/source/docs/spark/guide.html.markdown.erb b/site/source/docs/spark/guide.html.markdown.erb
index 8cc593c..e94047a 100644
--- a/site/source/docs/spark/guide.html.markdown.erb
+++ b/site/source/docs/spark/guide.html.markdown.erb
@@ -1,6 +1,6 @@
---
title: Guide - Apache DataFu Spark
-version: 1.7.0
+version: 1.8.0
section_name: Apache DataFu Spark
license: >
Licensed to the Apache Software Foundation (ASF) under one or more
