A docker image for the standalone openwhisk (#4782)
* use system properties to set standalone host ip and name
* show values
* plaground support and dockerfile
* fixes to the standalone docker image
* making scancode happy
making scalafmt happy
* using stop, disabling launcher, publishing image, updated docs
* licenses
* adding cors support
* fixed start.sh and docs
* scancode
* removing extra push
* support for CONTAINER_EXTRA_ENV and JVM_EXTRA_ARGS
* update readme
* fixes - wait for the server ready
* forgot file
* rebuild
* Update core/standalone/README.md
Co-Authored-By: rodric rabbah <rodric@gmail.com>
* Update core/standalone/README.md
Co-Authored-By: rodric rabbah <rodric@gmail.com>
* Update core/standalone/README.md
Co-Authored-By: rodric rabbah <rodric@gmail.com>
Co-authored-by: rodric rabbah <rodric@gmail.com>
diff --git a/core/standalone/Dockerfile b/core/standalone/Dockerfile
new file mode 100644
index 0000000..6b01383
--- /dev/null
+++ b/core/standalone/Dockerfile
@@ -0,0 +1,33 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+FROM scala
+ARG OPENWHISK_JAR
+ENV DOCKER_VERSION=18.06.3-ce
+ENV WSK_VERSION=1.0.0
+ADD init /
+ADD stop waitready /bin/
+RUN chmod +x /bin/stop /bin/waitready ;\
+ curl -sL \
+ https://download.docker.com/linux/static/stable/x86_64/docker-${DOCKER_VERSION}.tgz \
+ | tar xzvf - -C /usr/bin --strip 1 docker/docker ;\
+ curl -sL \
+ https://github.com/apache/openwhisk-cli/releases/download/${WSK_VERSION}/OpenWhisk_CLI-${WSK_VERSION}-linux-amd64.tgz \
+ | tar xzvf - -C /usr/bin wsk
+ADD ${OPENWHISK_JAR} /openwhisk-standalone.jar
+WORKDIR /
+EXPOSE 8080
+ENTRYPOINT ["/init"]
diff --git a/core/standalone/README.md b/core/standalone/README.md
index 4149508..ff19351 100644
--- a/core/standalone/README.md
+++ b/core/standalone/README.md
@@ -65,6 +65,12 @@
$ ./gradlew :core:standalone:bootRun --args='-m runtimes.json'
```
+You can also build a standalone docker image with:
+
+```bash
+$ ./gradlew :core:standalone:distDocker
+```
+
### Usage
OpenWhisk standalone server support various launch options
@@ -380,6 +386,35 @@
}
```
+## Launching OpenWhisk standalone with Docker
+
+If you have docker and bash installed, you can launch the standalone openwhisk from the docker image with just:
+
+`bash <(curl -sL https://s.apache.org/openwhisk.sh)`
+
+If you do not want to execute arbitrary code straight from the net, you can look at [this script](start.sh), check it and run it when you feel safe.
+
+The script will start the standalone controller with Docker, and will also try to open the playground. It was tested on Linux, OSX and Windows with Git Bash. If a browser does not open with playground, access it at `http://localhost:3232`.
+
+You can then install the [wsk CLI](https://github.com/apache/openwhisk-cli/releases) and retrieve the command line to configure `wsk` with:
+
+`docker logs openwhisk | grep 'wsk property'`
+
+To properly shut down OpenWhisk and containers it creates, use [this script](stop.sh) or run the command:
+
+`docker exec openwhisk stop`
+
+### Extra Args for the Standalone OpenWhisk Docker Image
+
+When running OpenWhisk Standalone using the docker image, you can set environment variables to pass extra args with the `-e` flag.
+
+Extra args are useful to configure the JVM running OpenWhisk and to propagate additional environment variables to containers running images. This feature is useful for example to enable debugging for actions.
+
+You can pass additional parameters (for example set system properties) to the JVM running OpenWhisk setting the environment variable `JVM_EXTRA_ARGS`. For example `-e JVM_EXTRA_ARGS=-Dconfig.loads` allows to enable tracing of configuration. You can set any OpenWhisk parameter with feature.
+
+You can also set additional environment variables for each container running actions invoked by OpenWhisk by setting `CONTAINER_EXTRA_ENV`. For example, setting `-e CONTAINER_EXTRA_ENV=__OW_DEBUG_PORT=8081` enables debugging for those images supporting starting the action under a debugger, like the typescript runtime.
+
+
[1]: https://github.com/apache/incubator-openwhisk/blob/master/docs/cli.md
[2]: https://github.com/apache/incubator-openwhisk/blob/master/docs/samples.md
[3]: https://github.com/apache/incubator-openwhisk-apigateway
diff --git a/core/standalone/build.gradle b/core/standalone/build.gradle
index 0027785..6e97466 100644
--- a/core/standalone/build.gradle
+++ b/core/standalone/build.gradle
@@ -25,6 +25,11 @@
id 'com.gorylenko.gradle-git-properties' version '2.0.0'
}
+ext.dockerImageName = 'standalone'
+ext.dockerBuildArgs = [ "OPENWHISK_JAR=build/libs/openwhisk-standalone-${version}.jar"]
+apply from: '../../gradle/docker.gradle'
+distDocker.dependsOn 'bootJar'
+
project.archivesBaseName = "openwhisk-standalone"
task copySwagger(type: Copy) {
@@ -153,6 +158,7 @@
compile "io.github.embeddedkafka:embedded-kafka_${gradle.scala.depVersion}:2.4.0"
compile "org.scala-lang:scala-reflect:${gradle.scala.version}"
+ compile "ch.megard:akka-http-cors_${gradle.scala.depVersion}:0.4.2"
testCompile "junit:junit:4.11"
testCompile "org.scalatest:scalatest_${gradle.scala.depVersion}:3.0.8"
diff --git a/core/standalone/init b/core/standalone/init
new file mode 100755
index 0000000..76f0ab1
--- /dev/null
+++ b/core/standalone/init
@@ -0,0 +1,27 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+if ! test -e /var/run/docker.sock
+then echo "Please launch this image with the option -v /var/run/docker.sock:/var/run/docker.sock"
+ exit 1
+fi
+JVM_ARGS="$JVM_EXTRA_ARGS\
+ -Dwhisk.standalone.host.name=$(hostname)\
+ -Dwhisk.standalone.host.internal=$(hostname -i)\
+ -Dwhisk.standalone.host.external=localhost"
+set -x
+java $JVM_ARGS -jar openwhisk-standalone.jar --no-browser "$@"
diff --git a/core/standalone/src/main/resources/standalone.conf b/core/standalone/src/main/resources/standalone.conf
index dd2d203..5c8d7cf 100644
--- a/core/standalone/src/main/resources/standalone.conf
+++ b/core/standalone/src/main/resources/standalone.conf
@@ -124,5 +124,21 @@
apache-client {
retry-no-http-response-exception = true
}
+ container-factory {
+ container-args {
+ extra-args {
+ env += ${?CONTAINER_EXTRA_ENV}
+ }
+ }
+ }
+}
+akka-http-cors {
+ allow-generic-http-requests = yes
+ allow-credentials = yes
+ allowed-origins = "*"
+ allowed-headers = "*"
+ allowed-methods = ["GET", "POST", "HEAD", "OPTIONS"]
+ exposed-headers = []
+ max-age = 1800 seconds
}
diff --git a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PlaygroundLauncher.scala b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PlaygroundLauncher.scala
index 146ccc7..bac8cc9 100644
--- a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PlaygroundLauncher.scala
+++ b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PlaygroundLauncher.scala
@@ -20,11 +20,12 @@
import java.nio.charset.StandardCharsets.UTF_8
import akka.actor.ActorSystem
-import akka.http.scaladsl.model.{ContentType, HttpCharsets, HttpEntity, MediaTypes, StatusCodes}
+import akka.http.scaladsl.model._
import akka.http.scaladsl.server.Route
import akka.http.scaladsl.server.directives.FileAndResourceDirectives.ResourceFile
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.{Sink, Source}
+import ch.megard.akka.http.cors.scaladsl.CorsDirectives._
import org.apache.commons.io.IOUtils
import org.apache.commons.lang3.SystemUtils
import org.apache.openwhisk.common.{Logging, TransactionId}
@@ -34,15 +35,20 @@
import scala.concurrent.duration._
import scala.concurrent.{Await, ExecutionContext}
-import scala.util.{Failure, Success, Try}
import scala.sys.process._
+import scala.util.{Failure, Success, Try}
-class PlaygroundLauncher(host: String, controllerPort: Int, pgPort: Int, authKey: String, devMode: Boolean)(
- implicit logging: Logging,
- ec: ExecutionContext,
- actorSystem: ActorSystem,
- materializer: ActorMaterializer,
- tid: TransactionId) {
+class PlaygroundLauncher(host: String,
+ extHost: String,
+ controllerPort: Int,
+ pgPort: Int,
+ authKey: String,
+ devMode: Boolean,
+ noBrowser: Boolean)(implicit logging: Logging,
+ ec: ExecutionContext,
+ actorSystem: ActorSystem,
+ materializer: ActorMaterializer,
+ tid: TransactionId) {
private val interface = loadConfigOrThrow[String]("whisk.controller.interface")
private val jsFileName = "playgroundFunctions.js"
private val jsContentType = ContentType(MediaTypes.`application/javascript`, HttpCharsets.`UTF-8`)
@@ -58,7 +64,8 @@
private val jsFileContent = {
val js = resourceToString(jsFileName, "ui")
- val content = js.replace("window.APIHOST='http://localhost:3233'", s"window.APIHOST='http://$host:$controllerPort'")
+ val content =
+ js.replace("window.APIHOST='http://localhost:3233'", s"window.APIHOST='http://$extHost:$controllerPort'")
content.getBytes(UTF_8)
}
@@ -88,8 +95,10 @@
if (!devMode) {
prePullDefaultImages()
}
- launchBrowser(pgUrl)
- logging.info(this, s"Launched browser $pgUrl")
+ if (!noBrowser) {
+ launchBrowser(pgUrl)
+ logging.info(this, s"Launched browser $pgUrl")
+ }
}.failed.foreach(t => logging.warn(this, "Failed to launch browser " + t))
}
@@ -112,12 +121,14 @@
object PlaygroundService extends BasicHttpService {
override def routes(implicit transid: TransactionId): Route =
path(PathEnd | Slash | pg) { redirect(s"/$pg/ui/index.html", StatusCodes.Found) } ~
- pathPrefix(pg / "ui" / Segment) { fileName =>
- get {
- if (fileName == jsFileName) {
- complete(HttpEntity(jsContentType, jsFileContent))
- } else {
- getFromResource(s"$uiPath/$fileName")
+ cors() {
+ pathPrefix(pg / "ui" / Segment) { fileName =>
+ get {
+ if (fileName == jsFileName) {
+ complete(HttpEntity(jsContentType, jsFileContent))
+ } else {
+ getFromResource(s"$uiPath/$fileName")
+ }
}
}
}
diff --git a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PreFlightChecks.scala b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PreFlightChecks.scala
index 3d430c5..454d04c 100644
--- a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PreFlightChecks.scala
+++ b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/PreFlightChecks.scala
@@ -45,6 +45,9 @@
println(separator)
println("Running pre flight checks ...")
println()
+ println(s"Local Host Name: ${StandaloneDockerSupport.getLocalHostName()}")
+ println(s"Local Internal Name: ${StandaloneDockerSupport.getLocalHostInternalName()}")
+ println()
checkForDocker()
checkForWsk()
checkForPorts()
@@ -113,6 +116,7 @@
val apihost = "wsk property get --apihost".!!.trim
val requiredHostValue = s"http://${StandaloneDockerSupport.getLocalHostName()}:${conf.port()}"
+ val externalHostValue = s"http://${StandaloneDockerSupport.getExternalHostName()}:${conf.port()}"
//We can use -o option to get raw value. However as its a recent addition
//using a lazy approach where we check if output ends with one of the configured auth keys or
@@ -130,7 +134,7 @@
case Some((ns, guestAuth)) =>
println(s"$warn Configure wsk via below command to connect to this server as [$ns]")
println()
- println(clr(s"wsk property set --apihost '$requiredHostValue' --auth '$guestAuth'", MAGENTA, clrEnabled))
+ println(clr(s"wsk property set --apihost '$externalHostValue' --auth '$guestAuth'", MAGENTA, clrEnabled))
case None =>
}
}
diff --git a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneDockerSupport.scala b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneDockerSupport.scala
index 1662b6a..8b6eb5b 100644
--- a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneDockerSupport.scala
+++ b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneDockerSupport.scala
@@ -106,28 +106,45 @@
}
/**
+ * Returns the hostname to access the playground.
+ * It defaults to localhost but it can be overriden
+ * and it is useful when the standalone is run in a container.
+ */
+ def getExternalHostName(): String = {
+ sys.props.get("whisk.standalone.host.external").getOrElse(getLocalHostName())
+ }
+
+ /**
* Returns the address to be used by code running outside of container to connect to
* server. On non linux setups its 'localhost'. However for Linux setups its the ip used
* by docker for docker0 network to refer to host system
*/
def getLocalHostName(): String = {
- if (SystemUtils.IS_OS_LINUX) hostIpLinux
- else "localhost"
+ sys.props
+ .get("whisk.standalone.host.name")
+ .getOrElse(if (SystemUtils.IS_OS_LINUX) hostIpLinux
+ else "localhost")
}
def getLocalHostIp(): String = {
- if (SystemUtils.IS_OS_MAC || SystemUtils.IS_OS_WINDOWS)
- hostIpNonLinux
- else hostIpLinux
+ sys.props
+ .get("whisk.standalone.host.ip")
+ .getOrElse(
+ if (SystemUtils.IS_OS_MAC || SystemUtils.IS_OS_WINDOWS)
+ hostIpNonLinux
+ else hostIpLinux)
}
/**
* Determines the name/ip which code running within container can use to connect back to Controller
*/
def getLocalHostInternalName(): String = {
- if (SystemUtils.IS_OS_MAC || SystemUtils.IS_OS_WINDOWS)
- "host.docker.internal"
- else hostIpLinux
+ sys.props
+ .get("whisk.standalone.host.internal")
+ .getOrElse(
+ if (SystemUtils.IS_OS_MAC || SystemUtils.IS_OS_WINDOWS)
+ "host.docker.internal"
+ else hostIpLinux)
}
def prePullImage(imageName: String)(implicit logging: Logging): Unit = {
diff --git a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneOpenWhisk.scala b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneOpenWhisk.scala
index b77d759..3fb2abb 100644
--- a/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneOpenWhisk.scala
+++ b/core/standalone/src/main/scala/org/apache/openwhisk/standalone/StandaloneOpenWhisk.scala
@@ -104,11 +104,14 @@
val devKcf = opt[Boolean](descr = "Enables KubernetesContainerFactory for local development")
val noUi = opt[Boolean](descr = "Disable Playground UI", noshort = true)
+
val uiPort = opt[Int](
descr = s"Playground UI server port. If not specified then $preferredPgPort or some random free port " +
s"(if $StandaloneOpenWhisk is busy) would be used",
noshort = true)
+ val noBrowser = opt[Boolean](descr = "Disable Launching Browser", noshort = true)
+
val devUserEventsPort = opt[Int](
descr = "Specify the port for the user-event service. This mode can be used for local " +
"development of user-event service by configuring Prometheus to connect to existing running service instance")
@@ -564,7 +567,14 @@
conf: Conf)(implicit logging: Logging, as: ActorSystem, ec: ExecutionContext, materializer: ActorMaterializer) = {
implicit val tid: TransactionId = TransactionId(systemPrefix + "playground")
val pgPort = getPort(conf.uiPort.toOption, preferredPgPort)
- new PlaygroundLauncher(StandaloneDockerSupport.getLocalHostName(), owPort, pgPort, systemAuthKey, conf.devMode())
+ new PlaygroundLauncher(
+ StandaloneDockerSupport.getLocalHostName(),
+ StandaloneDockerSupport.getExternalHostName(),
+ owPort,
+ pgPort,
+ systemAuthKey,
+ conf.devMode(),
+ conf.noBrowser())
}
private def systemAuthKey: String = {
diff --git a/core/standalone/start.sh b/core/standalone/start.sh
new file mode 100755
index 0000000..c81ec76
--- /dev/null
+++ b/core/standalone/start.sh
@@ -0,0 +1,31 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+USER="${1:-openwhisk}"
+shift
+docker run --rm -d \
+ -h openwhisk --name openwhisk \
+ -p 3233:3233 -p 3232:3232 \
+ -v //var/run/docker.sock:/var/run/docker.sock \
+ $USER/standalone "$@"
+docker exec openwhisk waitready
+case "$(uname)" in
+ (Linux) xdg-open http://localhost:3232 ;;
+ (Darwin) open http://localhost:3232 ;;
+ (MINGW*) start http://localhost:3232 ;;
+ (*) echo Please use http://localhost:3232 for playground ;;
+esac
diff --git a/core/standalone/stop b/core/standalone/stop
new file mode 100755
index 0000000..99edfcf
--- /dev/null
+++ b/core/standalone/stop
@@ -0,0 +1,18 @@
+#!/bin/sh
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+ps -o comm,pid ax | awk '/^java/ { print $2 }' | xargs kill
diff --git a/core/standalone/stop.sh b/core/standalone/stop.sh
new file mode 100755
index 0000000..2aa7e2a
--- /dev/null
+++ b/core/standalone/stop.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+docker exec openwhisk stop
diff --git a/core/standalone/waitready b/core/standalone/waitready
new file mode 100755
index 0000000..78c876f
--- /dev/null
+++ b/core/standalone/waitready
@@ -0,0 +1,22 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+AUTH="23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP"
+wsk property set --apihost "http://$(hostname):3233" --auth "$AUTH"
+until wsk action list 2>/dev/null >/dev/null
+do sleep 1 ; echo server still not ready - retrying
+done
