added support for MCF Docker Repo, added debug port 5005, support for Maven deps publicly available
diff --git a/README.md b/README.md
index 01faccd..e3c2f3e 100644
--- a/README.md
+++ b/README.md
@@ -1,19 +1,35 @@
-# Apache ManifoldCF SDK 0.0.4
+# Apache ManifoldCF SDK 0.0.5
This is the SDK project of Apache ManifoldCF dedicated to developers that need to extend the platform with new components, connectors or integrations using Maven and Docker. On the other hand this SDK can be extremely useful also for all the developers who want to contribute to the ManifoldCF project.
This project has started with an initial contribution by @OpenPj and @binduwavell.
The SDK will take care of the following tasks:
+* Creating custom containers using the official ManifoldCF Docker Images
* Creating the Docker Volumes for the MCF Maven Repo and the MCF installation
* Preparing the Docker Image for executing the build installing Ant and Maven
* Running the container for executing the building process with Ant and then with Maven
* Copying the entire Maven Repo in the Maven target folder in the host machine (locally)
* Generating the MCF official site
+ManifoldCF SDK is using the ManifoldCF Docker repository:
+[https://hub.docker.com/r/apache/manifoldcf](https://hub.docker.com/r/apache/manifoldcf)
+
+What's new in v0.0.5
+* The Maven profile `docker-hub` now is enabled by default
+* MCF SDK now by default is building the image from the official Docker images
+* Added a debug port (5005) for accelerate connectors development with remote debugging
+* Default Maven dependencies version now is 2.27-SNAPSHOT (publicly available)
+
+If you need to build your custom Docker image using dependencies built from the MCF source code and stored in the target folder (using the `init` script mentioned below), please use the `local` Maven profile.
+
Examples of commands using the main bash script for a typical usage:
* `./run.sh init 2.26 ga` -> start the init process for ManifoldCF 2.26 GA
-* `./run.sh build_start` -> build extensions and run everything with Docker locally
-* `./run.sh start` -> start all the containers (MCF and PostgreSQL)
-* `./run.sh stop` -> stop all the containers (MCF and PostgreSQL)
+* `./run.sh init custom-version local /<USER_HOME>/Documents/workspaces/manifoldcf/custom-project` -> start the init process for a custom ManifoldCF project
+* `./run.sh build_start` -> build extensions and run everything with Docker locally using an official ManifoldCF distribution
+* `./run.sh build_start_local` -> build extensions and run everything with Docker locally using a custom local ManifoldCF source code repository
+* `./run.sh start` -> start all the containers (MCF dist and PostgreSQL)
+* `./run.sh stop` -> stop all the containers (MCF dist and PostgreSQL)
+* `./run.sh start_local` -> start all the containers (custom MCF project and PostgreSQL)
+* `./run.sh stop_local` -> stop all the containers (custom MCF project and PostgreSQL)
Examples of commands for developing and testing MCF itself:
* `./run.sh init 2.22.1 rc` -> start the init process for ManifoldCF 2.22.1 RC
@@ -28,7 +44,7 @@
* `./run-site.sh clean` -> Start the mcf-sdk-site in order to clean the project
* `./run-site.sh stop` -> Stop the mcf-sdk-site
-# Initializing the local Maven repository using Docker
+# Initializing the local Maven repository using Docker (optional)
In order to compile extensions with Maven, the SDK provides an initialization script that will download the source code of ManifoldCF and then build it in the container. At the end of the build process the Maven repository included in the container will be copied in the local folder `target/mcf-maven-repo`. The local Maven repo will be used as the main reference for compiling the extension code.
The initialization command consists of the following arguments:
diff --git a/mcf-docker-overlay/pom.xml b/mcf-docker-overlay/pom.xml
index 6e98871..b544486 100644
--- a/mcf-docker-overlay/pom.xml
+++ b/mcf-docker-overlay/pom.xml
@@ -9,13 +9,16 @@
<parent>
<groupId>org.apache.manifoldcf</groupId>
<artifactId>mcf-sdk</artifactId>
- <version>0.0.4</version>
+ <version>0.0.5</version>
</parent>
<properties>
<docker.tomcat.port.external>8345</docker.tomcat.port.external>
<docker.tomcat.port.internal>8345</docker.tomcat.port.internal>
+
+ <docker.debug.port.external>5005</docker.debug.port.external>
+ <docker.debug.port.internal>5005</docker.debug.port.internal>
<docker.mcf.database.port.internal>5432</docker.mcf.database.port.internal>
<docker.mcf.database.port.external>5432</docker.mcf.database.port.external>
@@ -144,6 +147,7 @@
</network>
<ports>
<port>${docker.tomcat.port.external}:${docker.tomcat.port.internal}</port>
+ <port>${docker.debug.port.external}:${docker.debug.port.internal}</port>
</ports>
<wait>
<http>
@@ -232,7 +236,120 @@
</plugins>
</pluginManagement>
</build>
-
+ </profile>
+ <profile>
+ <id>local</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ </activation>
+ <build>
+ <plugins>
+ <plugin>
+ <artifactId>maven-resources-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>copy-and-filter-docker-resources</id>
+ <phase>package</phase>
+ <goals>
+ <goal>copy-resources</goal>
+ </goals>
+ <configuration>
+ <outputDirectory>${project.parent.basedir}/target</outputDirectory>
+ <resources>
+ <resource>
+ <directory>src/main/docker/local</directory>
+ <filtering>true</filtering>
+ <excludes>
+ <exclude>**/*.jar</exclude>
+ <exclude>**/*.so</exclude>
+ <exclude>**/*.gz</exclude>
+ </excludes>
+ </resource>
+ </resources>
+ </configuration>
+ </execution>
+ <execution>
+ <id>copy-and-filter-docker-resources-non-filtered</id>
+ <phase>package</phase>
+ <goals>
+ <goal>copy-resources</goal>
+ </goals>
+ <configuration>
+ <outputDirectory>${project.parent.basedir}/target</outputDirectory>
+ <resources>
+ <resource>
+ <directory>src/main/docker/local</directory>
+ <filtering>false</filtering>
+ <includes>
+ <include>**/*.jar</include>
+ <include>**/*.so</include>
+ <include>**/*.gz</include>
+ </includes>
+ </resource>
+ </resources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ <profile>
+ <id>docker-hub</id>
+ <activation>
+ <activeByDefault>true</activeByDefault>
+ </activation>
+ <build>
+ <plugins>
+ <plugin>
+ <artifactId>maven-resources-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>copy-and-filter-docker-resources</id>
+ <phase>package</phase>
+ <goals>
+ <goal>copy-resources</goal>
+ </goals>
+ <configuration>
+ <outputDirectory>${project.parent.basedir}/target</outputDirectory>
+ <resources>
+ <resource>
+ <directory>src/main/docker/hub</directory>
+ <filtering>true</filtering>
+ <excludes>
+ <exclude>**/*.jar</exclude>
+ <exclude>**/*.so</exclude>
+ <exclude>**/*.gz</exclude>
+ </excludes>
+ </resource>
+ </resources>
+ </configuration>
+ </execution>
+ <execution>
+ <id>copy-and-filter-docker-resources-non-filtered</id>
+ <phase>package</phase>
+ <goals>
+ <goal>copy-resources</goal>
+ </goals>
+ <configuration>
+ <outputDirectory>${project.parent.basedir}/target</outputDirectory>
+ <resources>
+ <resource>
+ <directory>src/main/docker/hub</directory>
+ <filtering>false</filtering>
+ <includes>
+ <include>**/*.jar</include>
+ <include>**/*.so</include>
+ <include>**/*.gz</include>
+ </includes>
+ </resource>
+ </resources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
</profile>
</profiles>
diff --git a/mcf-docker-overlay/src/main/docker/hub/Dockerfile b/mcf-docker-overlay/src/main/docker/hub/Dockerfile
new file mode 100644
index 0000000..75e1d82
--- /dev/null
+++ b/mcf-docker-overlay/src/main/docker/hub/Dockerfile
@@ -0,0 +1,16 @@
+FROM apache/manifoldcf:2.27-RC1
+LABEL maintainer="The Apache ManifoldCF Project"
+
+ARG MCF_VERSION=2.27-RC1
+ARG MCF_USER=manifoldcf
+
+ARG MCF_GROUP=manifoldcf
+
+ARG APP_DIR=/usr/share/manifoldcf
+
+COPY connectors.xml ${APP_DIR}
+COPY properties.xml ${APP_DIR}/example
+COPY start-options.env.unix ${APP_DIR}/example
+COPY extensions/*.jar ${APP_DIR}/connector-lib/
+
+USER ${MCF_USER}
\ No newline at end of file
diff --git a/mcf-docker-overlay/src/main/docker/connectors.xml b/mcf-docker-overlay/src/main/docker/hub/connectors.xml
similarity index 100%
rename from mcf-docker-overlay/src/main/docker/connectors.xml
rename to mcf-docker-overlay/src/main/docker/hub/connectors.xml
diff --git a/mcf-docker-overlay/src/main/docker/properties.xml b/mcf-docker-overlay/src/main/docker/hub/properties.xml
similarity index 100%
rename from mcf-docker-overlay/src/main/docker/properties.xml
rename to mcf-docker-overlay/src/main/docker/hub/properties.xml
diff --git a/mcf-docker-overlay/src/main/docker/hub/start-options.env.unix b/mcf-docker-overlay/src/main/docker/hub/start-options.env.unix
new file mode 100644
index 0000000..5fde963
--- /dev/null
+++ b/mcf-docker-overlay/src/main/docker/hub/start-options.env.unix
@@ -0,0 +1,8 @@
+-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=0.0.0.0:5005
+-Xms512m
+-Xmx512m
+-Dorg.apache.manifoldcf.configfile=./properties.xml
+-Dorg.apache.manifoldcf.jettyshutdowntoken=secret_token
+-Djava.security.auth.login.config=
+-cp
+.:../lib/mcf-core.jar:../lib/mcf-agents.jar:../lib/mcf-pull-agent.jar:../lib/mcf-ui-core.jar:../lib/mcf-jetty-runner.jar:../lib/http2-client-9.4.48.v20220622.jar:../lib/http2-common-9.4.48.v20220622.jar:../lib/http2-hpack-9.4.48.v20220622.jar:../lib/http2-http-client-transport-9.4.48.v20220622.jar:../lib/jetty-alpn-client-9.4.48.v20220622.jar:../lib/jetty-alpn-java-client-9.4.48.v20220622.jar:../lib/jetty-client-9.4.48.v20220622.jar:../lib/jetty-continuation-9.4.48.v20220622.jar:../lib/jetty-http-9.4.48.v20220622.jar:../lib/jetty-io-9.4.48.v20220622.jar:../lib/jetty-jndi-9.4.48.v20220622.jar:../lib/jetty-jsp-9.2.30.v20200428.jar:../lib/jetty-jsp-jdt-2.3.3.jar:../lib/jetty-plus-9.4.48.v20220622.jar:../lib/jetty-schemas-3.1.M0.jar:../lib/jetty-security-9.4.48.v20220622.jar:../lib/jetty-server-9.4.48.v20220622.jar:../lib/jetty-servlet-9.4.48.v20220622.jar:../lib/jetty-util-9.4.48.v20220622.jar:../lib/jetty-webapp-9.4.48.v20220622.jar:../lib/jetty-xml-9.4.48.v20220622.jar:../lib/hsqldb-2.3.2.jar:../lib/postgresql-42.1.3.jar:../lib/commons-codec-1.10.jar:../lib/commons-collections-3.2.2.jar:../lib/commons-collections4-4.4.jar:../lib/commons-discovery-0.5.jar:../lib/commons-el-1.0.jar:../lib/commons-exec-1.3.jar:../lib/commons-fileupload-1.3.3.jar:../lib/commons-io-2.11.0.jar:../lib/commons-lang-2.6.jar:../lib/commons-lang3-3.12.0.jar:../lib/commons-logging-1.2.jar:../lib/ecj-4.3.1.jar:../lib/gson-2.8.0.jar:../lib/guava-25.1-jre.jar:../lib/httpclient-4.5.13.jar:../lib/httpcore-4.4.15.jar:../lib/jasper-6.0.35.jar:../lib/jasper-el-6.0.35.jar:../lib/javax.mail-1.6.2.jar:../lib/javax.servlet-api-3.1.0.jar:../lib/jna-5.10.0.jar:../lib/jna-platform-5.10.0.jar:../lib/json-simple-1.1.1.jar:../lib/jsp-api-2.1-glassfish-2.1.v20091210.jar:../lib/juli-6.0.35.jar:../lib/log4j-1.2-api-2.17.2.jar:../lib/log4j-api-2.17.2.jar:../lib/log4j-core-2.17.2.jar:../lib/netty-handler-4.1.100.Final.jar:../lib/serializer-2.7.1.jar:../lib/slf4j-api-1.7.36.jar:../lib/slf4j-simple-1.7.36.jar:../lib/velocity-engine-core-2.3.jar:../lib/xalan-2.7.1.jar:../lib/xercesImpl-2.12.2.jar:../lib/xml-apis-1.4.01.jar:../lib/zookeeper-3.9.1.jar:../lib/zookeeper-jute-3.9.1.jar:../lib/javax.activation-1.2.0.jar:../lib/javax.activation-api-1.2.0.jar:
diff --git a/mcf-docker-overlay/src/main/docker/Dockerfile b/mcf-docker-overlay/src/main/docker/local/Dockerfile
similarity index 100%
rename from mcf-docker-overlay/src/main/docker/Dockerfile
rename to mcf-docker-overlay/src/main/docker/local/Dockerfile
diff --git a/mcf-docker-overlay/src/main/docker/connectors.xml b/mcf-docker-overlay/src/main/docker/local/connectors.xml
similarity index 100%
copy from mcf-docker-overlay/src/main/docker/connectors.xml
copy to mcf-docker-overlay/src/main/docker/local/connectors.xml
diff --git a/mcf-docker-overlay/src/main/docker/properties.xml b/mcf-docker-overlay/src/main/docker/local/properties.xml
similarity index 100%
copy from mcf-docker-overlay/src/main/docker/properties.xml
copy to mcf-docker-overlay/src/main/docker/local/properties.xml
diff --git a/mcf-extensions-jar/pom.xml b/mcf-extensions-jar/pom.xml
index f1d3d94..4ff9fb8 100644
--- a/mcf-extensions-jar/pom.xml
+++ b/mcf-extensions-jar/pom.xml
@@ -7,7 +7,7 @@
<parent>
<groupId>org.apache.manifoldcf</groupId>
<artifactId>mcf-sdk</artifactId>
- <version>0.0.4</version>
+ <version>0.0.5</version>
</parent>
<dependencyManagement>
diff --git a/pom.xml b/pom.xml
index b2c62a5..3dad005 100644
--- a/pom.xml
+++ b/pom.xml
@@ -3,23 +3,24 @@
<groupId>org.apache.manifoldcf</groupId>
<artifactId>mcf-sdk</artifactId>
<name>Apache ManifoldCF SDK</name>
- <version>0.0.4</version>
+ <version>0.0.5</version>
<packaging>pom</packaging>
<properties>
<webVersion>4.0</webVersion>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
- <manifoldcf.version>2.26</manifoldcf.version>
- <maven.enforcer.plugin.version>3.4.1</maven.enforcer.plugin.version>
- <docker-maven-plugin.version>0.43.4</docker-maven-plugin.version>
- <maven.compiler.plugin.version>3.11.0</maven.compiler.plugin.version>
+ <manifoldcf.version>2.27-SNAPSHOT</manifoldcf.version>
+ <maven.enforcer.plugin.version>3.5.0</maven.enforcer.plugin.version>
+ <docker-maven-plugin.version>0.44.0</docker-maven-plugin.version>
+ <maven.compiler.plugin.version>3.13.0</maven.compiler.plugin.version>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<maven.resources.plugin.version>3.3.1</maven.resources.plugin.version>
- <maven.dependency.plugin.version>3.6.1</maven.dependency.plugin.version>
- <maven.clean.plugin.version>3.3.2</maven.clean.plugin.version>
+ <maven.dependency.plugin.version>3.7.1</maven.dependency.plugin.version>
+ <maven.clean.plugin.version>3.4.0</maven.clean.plugin.version>
<maven.lifecycle.mapping.plugin.version>1.0.0</maven.lifecycle.mapping.plugin.version>
+ <maven.versions.plugin.version>2.16.2</maven.versions.plugin.version>
<postgresql.version>13.1</postgresql.version>
</properties>
@@ -56,6 +57,11 @@
<artifactId>docker-maven-plugin</artifactId>
<version>${docker-maven-plugin.version}</version>
</plugin>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>versions-maven-plugin</artifactId>
+ <version>${maven.versions.plugin.version}</version>
+ </plugin>
</plugins>
</pluginManagement>
<plugins>
@@ -101,11 +107,17 @@
</includes>
<excludes>
<exclude>mcf-maven-repo/**</exclude>
+ <exclude>mcf-local/**</exclude>
+ <exclude>mcf-dist/**</exclude>
</excludes>
</fileset>
</filesets>
</configuration>
</plugin>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>versions-maven-plugin</artifactId>
+ </plugin>
</plugins>
</build>
@@ -114,14 +126,38 @@
<module>mcf-docker-overlay</module>
</modules>
- <repositories>
- <repository>
- <id>mcf-maven-repo</id>
- <name>ManifoldCF Local Repo</name>
- <layout>default</layout>
- <url>file:target/mcf-maven-repo/repository</url>
- </repository>
- </repositories>
-
+ <profiles>
+ <!-- For using the Maven artifacts installed locally by the Maven build triggered from build container -->
+ <profile>
+ <id>local</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ </activation>
+ <repositories>
+ <repository>
+ <id>mcf-maven-repo</id>
+ <name>ManifoldCF Local Repo</name>
+ <layout>default</layout>
+ <url>file:target/mcf-maven-repo/repository</url>
+ </repository>
+ </repositories>
+ </profile>
+
+ <!-- For using with the official ManifoldCF Docker Images provided in the ASF Docker repository -->
+ <profile>
+ <id>docker-hub</id>
+ <activation>
+ <activeByDefault>true</activeByDefault>
+ </activation>
+ <repositories>
+ <repository>
+ <id>asf-maven-repo</id>
+ <name>ASF Maven Repo</name>
+ <layout>default</layout>
+ <url>https://repository.apache.org/content/groups/snapshots</url>
+ </repository>
+ </repositories>
+ </profile>
+ </profiles>
</project>
\ No newline at end of file
diff --git a/run.sh b/run.sh
index 5d20b97..96bca82 100755
--- a/run.sh
+++ b/run.sh
@@ -1,14 +1,86 @@
#!/bin/sh
mcfversion=$2
mcfdist=$3
+mcfdisturl=$4
-init() {
- echo "Starting Apache ManifoldCF SDK initialization for version $mcfversion"
+initLocal() {
+ echo "Starting Apache ManifoldCF SDK initialization for version $mcfversion - Init Local"
mcfversionInFuction=$1
- echo "Init - 1 = $1"
+ echo "Init - 1 - MCF version = $1"
+
mcfdistInFunction=$2
- echo "Init - 2 = $2"
+ echo "Init - 2 - MCF DIST URL = $2"
+
+ mcfdistTypeInFunction=$3
+ echo "Init - 3 - MCF DIST TYPE = $3"
+
+ echo "Updating the ManifoldCF version inside the pom.xml"
+ mvn versions:set-property -Dproperty=manifoldcf.version -DnewVersion=$mcfversionInFuction
+
+ echo "Creating Docker Volume for the ManifoldCF Maven Repository"
+ docker volume create --name mcf-maven-repo
+
+ echo "Creating Docker Volume for the ManifoldCF Installation..."
+ docker volume create --name mcf-app-volume
+
+ echo "Creating simlink for the local MCF source code repository: $mcfdistInFunction"
+ mkdir target
+ cd target
+ ln -s $mcfdistInFunction mcf-local
+ cd ..
+
+ echo "Building the Docker Container for executing Ant and Maven build..."
+ docker build src/main/docker-local -t mcf-sdk-init
+
+ echo "Copying local source code into the container..."
+ docker run -itd --rm --name mcf-sdk-init-container -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init bash
+ docker cp target/mcf-local/. mcf-sdk-init-container:/usr/src/manifoldcf
+ docker stop mcf-sdk-init-container
+
+ echo "Starting the mcf-sdk-build-container for executing the Maven and Ant building process..."
+ docker run -it --rm --name mcf-sdk-init-container -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init /bin/bash -c "ant make-core-deps make-deps image; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-api-service -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-api-service.war; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-authority-service -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-authority-service.war; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-combined-service -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-combined-service.war; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-crawler-ui -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-crawler-ui.war
+ mvn -B -e -T 1C clean install -DskipTests -DskipITs"
+
+ echo "Starting the mcf-sdk-init-container to export the Maven repository in target/mcf-maven-repo..."
+ docker run --rm --name mcf-sdk-init-container -itd -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init bash
+
+ echo "Copying the ManifoldCF Maven Repo in target/mcf-maven-repo..."
+ mkdir target
+ docker cp mcf-sdk-init-container:/root/.m2/repository target/mcf-maven-repo
+
+ mkdir target/mcf-dist
+ echo "Copying the mcf-dist in target/mcf-dist"
+ docker cp mcf-sdk-init-container:/usr/src/manifoldcf/dist/apache-manifoldcf-$mcfdistInFunction-dev-bin.tar.gz target/mcf-dist
+
+ tar -xvz target/mcf-dist/apache-manifoldcf-$mcfdistInFunction-dev-bin.tar.gz -C target/mcf-dist
+ rm apache-manifoldcf-$mcfdistInFunction-dev-bin.tar.gz
+
+ echo "Stopping and removing the existent building container..."
+ docker stop mcf-sdk-init-container
+
+ #echo "Merging the ManifoldCF Maven Repository with the local Maven Repo"
+ #cp -n target/mcf-maven-repo $HOME/.m2
+}
+
+init226() {
+ echo "Starting Apache ManifoldCF SDK initialization for version $mcfversion - Init 2.26"
+
+ mcfversionInFuction=$1
+ echo "Init - 1 - MCF version = $1"
+
+ mcfdistInFunction=$2
+ echo "Init - 2 - MCF DIST URL = $2"
+
+ mcfdistTypeInFunction=$3
+ echo "Init - 3 - MCF DIST TYPE = $3"
+
+ echo "Updating the ManifoldCF version inside the pom.xml"
+ mvn versions:set-property -Dproperty=manifoldcf.version -DnewVersion=$mcfversionInFuction
echo "Creating Docker Volume for the ManifoldCF Maven Repository"
docker volume create --name mcf-maven-repo
@@ -18,7 +90,8 @@
echo "Building the Docker Container for executing Ant and Maven build..."
docker build --build-arg MCF_VERSION=$mcfversionInFuction --build-arg MCF_DIST_URL=$mcfdistInFunction src/main/docker -t mcf-sdk-init
-
+
+
echo "Starting the mcf-sdk-build-container for executing the Maven and Ant building process..."
docker run -it --rm --name mcf-sdk-init-container -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init /bin/bash -c "sed -i -e 's/<fileset dir=\"test-materials\" excludes/<fileset dir=\"test-materials\" erroronmissingdir=\"false\" excludes/g' connectors/solr/build.xml; \
sed -i -e 's/<?xml version=\"1\.0\" encoding=\"UTF-8\"?>/ /g' connectors/csv/pom.xml; \
@@ -31,7 +104,7 @@
mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-combined-service -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-combined-service.war; \
mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-crawler-ui -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-crawler-ui.war
mvn -B -e -T 1C clean install -DskipTests -DskipITs"
-
+
echo "Starting the mcf-sdk-init-container to export the Maven repository in target/mcf-maven-repo..."
docker run --rm --name mcf-sdk-init-container -itd -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init bash
@@ -46,21 +119,87 @@
#cp -n target/mcf-maven-repo $HOME/.m2
}
+init() {
+ echo "Starting Apache ManifoldCF SDK initialization for version $mcfversion - Init"
+
+ mcfversionInFuction=$1
+ echo "Init - 1 - MCF version = $1"
+
+ mcfdistInFunction=$2
+ echo "Init - 2 - MCF DIST URL = $2"
+
+ mcfdistTypeInFunction=$3
+ echo "Init - 3 - MCF DIST TYPE = $3"
+
+ echo "Updating the ManifoldCF version inside the pom.xml"
+ mvn versions:set-property -Dproperty=manifoldcf.version -DnewVersion=$mcfversionInFuction
+
+ echo "Creating Docker Volume for the ManifoldCF Maven Repository"
+ docker volume create --name mcf-maven-repo
+
+ echo "Creating Docker Volume for the ManifoldCF Installation..."
+ docker volume create --name mcf-app-volume
+
+ echo "Building the Docker Container for executing Ant and Maven build..."
+ docker build --build-arg MCF_VERSION=$mcfversionInFuction --build-arg MCF_DIST_URL=$mcfdistInFunction src/main/docker -t mcf-sdk-init
+
+ echo "Starting the mcf-sdk-build-container for executing the Maven and Ant building process..."
+ docker run -it --rm --name mcf-sdk-init-container -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init /bin/bash -c "ant make-core-deps make-deps build; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-api-service -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-api-service.war; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-authority-service -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-authority-service.war; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-combined-service -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-combined-service.war; \
+ mvn install:install-file -DgroupId=org.apache.manifoldcf -DartifactId=mcf-crawler-ui -Dversion=$mcfversion -Dpackaging=war -Dfile=/usr/src/manifoldcf/dist/web/war/mcf-crawler-ui.war
+ mvn -B -e -T 1C clean install -DskipTests -DskipITs"
+
+
+ echo "Starting the mcf-sdk-init-container to export the Maven repository in target/mcf-maven-repo..."
+ docker run --rm --name mcf-sdk-init-container -itd -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init bash
+
+ echo "Copying the ManifoldCF Maven Repo in target/mcf-maven-repo..."
+ mkdir target
+ docker cp mcf-sdk-init-container:/root/.m2/repository target/mcf-maven-repo
+
+ echo "Stopping and removing the existent building container..."
+ docker stop mcf-sdk-init-container
+
+ #echo "Merging the ManifoldCF Maven Repository with the local Maven Repo"
+ #cp -n target/mcf-maven-repo $HOME/.m2
+}
+
+
+
build_start() {
echo "Building with Maven skipping tests and run Docker containers"
mvn clean install docker:build docker:start -DskipTests -DskipITs
}
+build_start_local() {
+ echo "Building with Maven skipping tests and run Docker containers"
+ mkdir target/mcf-dist
+ cp -R target/mcf-local/dist/. target/mcf-dist
+ mvn clean install docker:build docker:start -DskipTests -DskipITs -Plocal
+}
+
build() {
echo "Building with Maven skipping tests"
mvn clean install -DskipTests -DskipITs
}
+build_local() {
+ echo "Building with Maven skipping tests"
+ mvn clean install -DskipTests -DskipITs -Plocal
+}
+
build_test() {
echo "Building with Maven executing tests"
mvn clean install
}
+build_test_local() {
+ echo "Building with Maven executing tests"
+ mvn clean install -Plocal
+}
+
unittests() {
echo "Executing unit tests"
mvn clean test
@@ -71,16 +210,31 @@
mvn clean integration-test
}
+integrationtests_local() {
+ echo "Executing integration tests"
+ mvn clean integration-test -Plocal
+}
+
start() {
echo "Run ManifoldCF Docker containers"
mvn docker:start
}
+start_local() {
+ echo "Run ManifoldCF Docker containers"
+ mvn docker:start -Plocal
+}
+
stop() {
echo "Stop ManifoldCF Docker containers"
mvn docker:stop
}
+stop_local() {
+ echo "Stop ManifoldCF Docker containers"
+ mvn docker:stop -Plocal
+}
+
test() {
echo "Starting the mcf-sdk-test-container to execute unit and integration tests"
docker run -it --rm -v mcf-maven-repo:/root/.m2 -v mcf-app-volume:/usr/src/manifoldcf -t mcf-sdk-init ant test
@@ -100,30 +254,46 @@
case "$1" in
init)
- if [ ! -n "$mcfversion" ]
+ if [ ! -n "$mcfversion" ] && [ "$mcfdist" != "local" ];
then
- echo "$0 - Error ManifoldCF version is not set. Usage for using ManifoldCF 2.26: $0 init 2.26 ga"
+ echo "$0 - Error ManifoldCF version is not set. Usage for using ManifoldCF 2.26: $0 init 2.26 ga or $0 init 2.27 rc"
exit 1
else
echo "ManifoldCF version is correctly set"
fi
- if [ ! -n "$mcfdist" ]
+ if [ ! -n "$mcfdist" ];
then
- echo "$0 - Error ManifoldCF release type is not set, it should be ga or rc. Usage for using ManifoldCF 2.21: $0 init 2.21 ga or $0 init 2.21 rc"
+ echo "$0 - Error ManifoldCF release type is not set, it should be ga or rc. Usage for using ManifoldCF 2.21: $0 init 2.21 ga or $0 init 2.21 rc or $0 init local YOUR_LOCAL_PATH"
exit 1
else
echo "ManifoldCF release type is correctly set and now starting Apache ManifoldCF SDK initialization script..."
fi
- if [ "$mcfdist"=="ga" ];
- mcfdisturl=""
- then
+
+ if [ "$mcfdist" -eq "ga" ];
+ then
mcfdisturl="https://dlcdn.apache.org/manifoldcf"
echo "Building a GA release with artifact base URL: $mcfdisturl"
- else
+ elif [ "$mcfdist" -eq "rc" ];
+ then
mcfdisturl="https://dist.apache.org/repos/dist/dev/manifoldcf"
echo "Building a RC release with artifact base URL: $mcfdisturl"
+ elif [ "$mcfdist" -eq "local" ];
+ then
+ echo "Building a local build: $mcfdisturl"
fi
- init $mcfversion $mcfdisturl
+
+ if [ "$mcfdist"=="ga" ] && [ "$mcfversion" -eq "2.26" ];
+ then
+ echo "mcfversion=$mcfversion | mcfdisturl: $mcfdisturl | mcfdist: $mcfdist"
+ init226 $mcfversion $mcfdisturl $mcfdist
+ elif [ "$mcfdist"=="local" ];
+ then
+ echo "mcfversion=$mcfversion | mcfdisturl: $mcfdisturl | mcfdist: $mcfdist"
+ initLocal $mcfversion $mcfdisturl $mcfdist
+ else
+ echo "mcfversion=$mcfversion | mcfdisturl: $mcfdisturl | mcfdist: $mcfdist"
+ init $mcfversion $mcfdisturl $mcfdist
+ fi
;;
clean)
clean
@@ -131,18 +301,30 @@
build_start)
build_start
;;
+ build_start_local)
+ build_start_local
+ ;;
start)
start
;;
+ start_local)
+ start_local
+ ;;
stop)
stop
;;
+ stop_local)
+ stop_local
+ ;;
unittests)
unittests
;;
integrationtests)
integrationtests
;;
+ integrationtests_local)
+ integrationtests_local
+ ;;
test)
test
;;
@@ -150,5 +332,5 @@
purge
;;
*)
- echo "Usage: $0 {init|clean|build_start|start|stop|unittests|integrationtests|test|purge}"
+ echo "Usage: $0 {init|clean|build_start|build_start_local|start|start_local|stop|stop_local|unittests|integrationtests|integrationtests_local|test|purge}"
esac
\ No newline at end of file
diff --git a/src/main/docker-local/Dockerfile b/src/main/docker-local/Dockerfile
new file mode 100644
index 0000000..ee6ba91
--- /dev/null
+++ b/src/main/docker-local/Dockerfile
@@ -0,0 +1,20 @@
+FROM maven:3.9.6-eclipse-temurin-11
+
+ARG MCF_SOURCE_DIR=/usr/src/manifoldcf
+
+ARG ANT_VERSION=1.10.12
+ARG ANT_HOME=/opt/ant
+
+RUN apt-get update && apt-get install -y --no-install-recommends wget openjfx
+
+RUN wget --no-cookies http://archive.apache.org/dist/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz \
+ && wget --no-check-certificate --no-cookies http://archive.apache.org/dist/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz.sha512 \
+ && echo "$(cat apache-ant-${ANT_VERSION}-bin.tar.gz.sha512) apache-ant-${ANT_VERSION}-bin.tar.gz" | sha512sum -c \
+ && tar -zvxf apache-ant-${ANT_VERSION}-bin.tar.gz -C /opt/ \
+ && ln -s /opt/apache-ant-${ANT_VERSION} /opt/ant \
+ && rm -f apache-ant-${ANT_VERSION}-bin.tar.gz \
+ && rm -f apache-ant-${ANT_VERSION}-bin.tar.gz.sha512 \
+ && update-alternatives --install "/usr/bin/ant" "ant" "/opt/ant/bin/ant" 1 \
+ && update-alternatives --set "ant" "/opt/ant/bin/ant"
+
+WORKDIR ${MCF_SOURCE_DIR}
\ No newline at end of file
diff --git a/src/main/docker/Dockerfile b/src/main/docker/Dockerfile
index ee23298..3f5803d 100644
--- a/src/main/docker/Dockerfile
+++ b/src/main/docker/Dockerfile
@@ -1,4 +1,4 @@
-FROM maven:3.9.5-eclipse-temurin-11
+FROM maven:3.9.6-eclipse-temurin-11
ARG MCF_VERSION=2.26
ARG MCF_SOURCE_DIR=/usr/src/manifoldcf
