changed signature, implemented basic tests, refactored compiler
diff --git a/README.md b/README.md
index 303ee5d..74c44d9 100644
--- a/README.md
+++ b/README.md
@@ -59,10 +59,10 @@
This will build the images:
-* `actionloop-golang-v1.10`: an image supporting Go sources
-* `actionloop`: the base image, supporting generic executables
+* `actionloop-golang-v1.11`: an image supporting Go sources
+* `actionloop`: the base image, supporting generic executables ans shell script
-The `actionloop` image is used as a basis also for the `actionloop-swift` image. It can be used for supporting other compiled programming languages as long as they implement a `compile` script and the *action loop* protocol described below.
+The `actionloop` image can be used for supporting other compiled programming languages as long as they implement a `compile` script and the *action loop* protocol described below.
To run tests
```
@@ -75,7 +75,7 @@
If you want to develop the proxy and run tests natively, you can on Linux or OSX.
Tested on Ubuntu Linux (14.04) and OSX 10.13. Probably other distributions work, maybe even Windows with WSL, but since it is not tested YMMMV.
-You need of course [go 1.10.2](https://golang.org/doc/install)
+You need of course [go 1.11.0](https://golang.org/doc/install)
Then you need a set of utilities used in tests:
@@ -93,39 +93,36 @@
# Using it with Go Sources
-The image can execute, compiling them on the fly, Golang OpenWhisk actions in source format. An action must be a Go source file, placed in the `action` package, implementing the `Main` function (or the function specified as `main`).
+The image can execute, compiling them on the fly, Golang OpenWhisk actions in source format.
-The expected signature is:
+An action must be a Go source file, placed in the `main` package and your action.
-`func Main(event json.RawMessage) (json.RawMessage, error)`
+Since `main.main` is reserved in Golang for the entry point of your program, and the entry point is used by support code, your action must be named `Main` (with capital `M`) even if your specify `main` as the name of the action (or you do not specify it, defaulting to `main`). If you specify a function name different than `main` the name of your functino does not need to be capitalized.
-Note the name of the function must be capitalised, because it needs to be exported from the `action` package. You can say the name of the function also in lower case, it will be capitalised anyway.
+The expected signature for a `main` function is:
+
+`func Main(event map[string]interface{}) map[string]interface{}`
For example:
```go
-package action
+package main
-import (
- "encoding/json"
- "log"
-)
+import "log"
// Main is the function implementing the action
-func Main(event json.RawMessage) (json.RawMessage, error) {
- // decode the json
- var obj map[string]interface{}
- json.Unmarshal(event, &obj)
+func Main(obj map[string]interface{}) map[string]interface{} {
// do your work
name, ok := obj["name"].(string)
if !ok {
name = "Stranger"
}
- msg := map[string]string{"message": ("Hello, " + name + "!")}
+ msg := make(map[string]interface{})
+ msg["message"] = "Hello, " + name + "!"
// log in stdout or in stderr
log.Printf("name=%s\n", name)
// encode the result back in json
- return json.Marshal(msg)
+ return msg
}
```
@@ -137,16 +134,23 @@
Compiling sources on the image can take some time when the images is initialised. You can speed up precompiling the sources using the image as an offline compiler. You need `docker` for doing that.
-The images accepts a `compile` command expecting sources in `/src`. It will then compile them and place the result in `/out`.
+The images accepts a `-compile <main>` flag, and expects you provide sources in standard input. It will then compile them, emit the binary in standard output and errors in stderr.
If you have docker, you can do it this way:
-- place your sources under `src` folder in current directory
-- create an `out` folder to receive the binary
-- run: `docker run -v $PWD/src:/src -v $PWD/out openwhisk/actionloop-golang-v1.10 compile`
-- you can now use `wsk` to publish the `out/main` executable
+If you have a single source maybe in file `main.go`, with a function named `Main` just do this:
-If you have a function named in a different way, for example `Hello`, specify `compile hello`. It will produce a binary named `out/hello`
+`cat main.go | docker run openwhisk/actionloop-golang-v1.11 -compile main >main`
+
+If you have multiple sources in current directory, even with a subfolder with sources, you can compile it all with:
+
+`zip -r - * | docker run openwhisk/actionloop-golang-v1.11 -compile main >main`
+
+The generated executable is suitable to be deployed in openwhisk:
+
+`wsk action create my/action main -docker openwhisk/actionloop-golang-v1.11`
+
+Note that the output is always in Linux AMS64 format so the executable can be run only inside a Docker Linux container.
<a name="generic"/>
@@ -158,8 +162,28 @@
Repeat forever:
- read one line from standard input (file descriptor 0)
-- parse the line as a json object
-- execute the action, logging in standard output and in standard error (file descriptor 1 and 2)
+- parse the line as a json object, that will be in format:
+
+```{
+ "value": JSON,
+ "namespace": String,
+ "action_name": String,
+ "api_host": String,
+ "api_key": String,
+ "activation_id": String,
+ "deadline": Number
+}```
+
+Note that if you use libraries, those will expect the values in enviroment variables:
+
+- `__OW_NAMESPACE`
+- `__OW_ACTION_NAME`
+- `__OW_API_HOST`
+- `__OW_API_KEYS`
+- `__OW_ACTIVATION_ID`
+- `__OW_DEADLINE`
+
+- execute the action, using the `value` that contains the payload provided by the used and logging in standard output and in standard error (file descriptor 1 and 2)
- write an answer in json format as a single line (without embedding newlines - newlines in strings must be quoted)
The `actionloop` image works actually with executable in unix sense, so also scripts are acceptable. In the actionloop image there is `bash` and the `jq` command, so you can for example implement the actionloop with a shell script:
@@ -170,7 +194,7 @@
while read line
do
# parse the in input with `jq`
- name="$(echo $line | jq -r .name)"
+ name="$(echo $line | jq -r .name.value)"
# log in stdout
echo msg="hello $name"
# produce the result - note the fd3
@@ -178,5 +202,5 @@
done
```
-Note the `actionloop` image will accept any source and will try to run it (if it is possible), while the `actionloop-golang` and `actionloop-swift` images will try to compile the sources instead.
+Note the `actionloop` image will accept any source and will try to run it (if it is possible), while the `actionloop-golang` images will try to compile the sources instead.
diff --git a/actionProxyLoop/Dockerfile b/actionProxyLoop/Dockerfile
index d81db8a..1560470 100644
--- a/actionProxyLoop/Dockerfile
+++ b/actionProxyLoop/Dockerfile
@@ -1,9 +1,11 @@
-FROM golang:1.10.2
+FROM ubuntu:xenial
RUN apt-get update && apt-get install -y \
curl \
jq \
+ zsh \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir /action
WORKDIR /action
-ADD proxy /proxy
-CMD /proxy
+ADD proxy /bin/proxy
+ENTRYPOINT [ "/bin/proxy" ]
+
diff --git a/common/gobuild.py b/common/gobuild.py
new file mode 100755
index 0000000..fe616e6
--- /dev/null
+++ b/common/gobuild.py
@@ -0,0 +1,101 @@
+#!/usr/bin/python
+"""Golang Action Compiler
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""
+
+import os
+import re
+import sys
+import codecs
+import subprocess
+
+def sources(launcher, source_dir, main):
+
+ # if you choose main, then it must be upper case
+ func = "Main" if main == "main" else main
+
+ # copy the uploaded main code, if it exists
+ src = "%s/%s" % (source_dir, main)
+ has_main = None
+ if os.path.isfile(src):
+ dst = "%s/func_%s_.go" % (source_dir, func)
+ with codecs.open(dst, 'w', 'utf-8') as d:
+ with codecs.open(src, 'r', 'utf-8') as s:
+ body = s.read()
+ has_main = re.match(r".*package\s+main\W.*func\s+main\s*\(\s*\)", body, flags=re.DOTALL)
+ d.write(body)
+
+
+ # copy the launcher fixing the main
+ if not has_main:
+ dst = "%s/launch_%s_.go" % (source_dir, func)
+ with codecs.open(dst, 'w', 'utf-8') as d:
+ with codecs.open(launcher, 'r', 'utf-8') as e:
+ code = e.read()
+ code = code.replace("Main", func)
+ d.write(code)
+
+ return os.path.abspath(dst)
+
+def build(parent, source_dir, target):
+ # compile...
+ p = subprocess.Popen(
+ ["go", "build", "-i", "-o", target],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ cwd=source_dir,
+ env={ "GOPATH": os.path.abspath(parent), "PATH": os.environ["PATH"]})
+ (o, e) = p.communicate()
+
+ # stdout/stderr may be either text or bytes, depending on Python
+ # version, so if bytes, decode to text. Note that in Python 2
+ # a string will match both types; so also skip decoding in that case
+ if isinstance(o, bytes) and not isinstance(o, str):
+ o = o.decode('utf-8')
+ if isinstance(e, bytes) and not isinstance(e, str):
+ e = e.decode('utf-8')
+
+ # remove the comments mentioning the folder in order to normalize output
+ o = re.sub(r"# .*\n", "", o, flags=re.MULTILINE)
+ e = re.sub(r"# .*\n", "", e, flags=re.MULTILINE)
+
+ if o:
+ sys.stdout.write(o)
+ sys.stdout.flush()
+
+ if e:
+ sys.stderr.write(e)
+ sys.stderr.flush()
+
+def main(argv):
+ if len(argv) < 4:
+ print("usage: <main-file> <source-dir> <target-dir>")
+ sys.exit(1)
+
+ main = argv[1]
+ source_dir = argv[2]
+ target_dir = argv[3]
+
+ parent = os.path.dirname(source_dir)
+ source = sources(argv[0]+".launcher.go", source_dir, main)
+ target = os.path.abspath("%s/%s" % (target_dir, main))
+
+ build(parent, source_dir, target)
+
+if __name__ == '__main__':
+ main(sys.argv)
diff --git a/common/gobuild.py.launcher.go b/common/gobuild.py.launcher.go
new file mode 100644
index 0000000..382b052
--- /dev/null
+++ b/common/gobuild.py.launcher.go
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package main
+
+import (
+ "bufio"
+ "bytes"
+ "encoding/json"
+ "fmt"
+ "io"
+ "log"
+ "os"
+ "strings"
+)
+
+func main() {
+ // debugging
+ var debug = os.Getenv("OW_DEBUG") != ""
+
+ if debug {
+ filename := os.Getenv("OW_DEBUG")
+ f, err := os.OpenFile(filename, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
+ if err == nil {
+ log.SetOutput(f)
+ defer f.Close()
+ }
+ log.Printf("ACTION ENV: %v", os.Environ())
+ }
+
+ // assign the main function
+ type Action func(event map[string]interface{}) map[string]interface{}
+ var action Action
+ action = Main
+
+ // input
+ out := os.NewFile(3, "pipe")
+ defer out.Close()
+ reader := bufio.NewReader(os.Stdin)
+
+ // read-eval-print loop
+ if debug {
+ log.Println("started")
+ }
+ for {
+ // read one line
+ inbuf, err := reader.ReadBytes('\n')
+ if err != nil {
+ if err != io.EOF {
+ log.Println(err)
+ }
+ break
+ }
+ if debug {
+ log.Printf(">>>'%s'>>>", inbuf)
+ }
+ // parse one line
+ var input map[string]interface{}
+ err = json.Unmarshal(inbuf, &input)
+ if err != nil {
+ log.Println(err.Error())
+ fmt.Fprintf(out, "{ error: %q}\n", err.Error())
+ continue
+ }
+ if debug {
+ log.Printf("%v\n", input)
+ }
+ // set environment variables
+ err = json.Unmarshal(inbuf, &input)
+ for k, v := range input {
+ if k == "value" {
+ continue
+ }
+ if s, ok := v.(string); ok {
+ os.Setenv("__OW_"+strings.ToUpper(k), s)
+ }
+ }
+ // get payload if not empty
+ var payload map[string]interface{}
+ if value, ok := input["value"].(map[string]interface{}); ok {
+ payload = value
+ }
+ // process the request
+ result := action(payload)
+ // encode the answer
+ output, err := json.Marshal(&result)
+ if err != nil {
+ log.Println(err.Error())
+ fmt.Fprintf(out, "{ error: %q}\n", err.Error())
+ continue
+ }
+ output = bytes.Replace(output, []byte("\n"), []byte(""), -1)
+ if debug {
+ log.Printf("'<<<%s'<<<", output)
+ }
+ fmt.Fprintf(out, "%s\n", output)
+ }
+}
diff --git a/common/gobuild.sh b/common/gobuild.sh
deleted file mode 100755
index ebb5758..0000000
--- a/common/gobuild.sh
+++ /dev/null
@@ -1,64 +0,0 @@
-#!/bin/bash
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# executable, defaults to main
-exec="${1:-main}"
-# absolute path of target dir or file
-source="${2:-/src}"
-source="$(realpath $source)"
-dest="${3:-/out}"
-dest="$(realpath $dest)"
-# prepare a compilation dir
-compiledir="$(mktemp -d)"
-compilefile="$(mktemp)"
-mkdir -p "$compiledir/src/action" "$compiledir/src/main"
-# capitalized main function name
-main="$(tr '[:lower:]' '[:upper:]' <<< ${exec:0:1})${exec:1}"
-# preparing for compilation
-if test -d "$source"
-# copy all the files unzipped
-then cp -rf "$source"/* "$compiledir/src/"
- mkdir "$compiledir/src/action" 2>/dev/null
- cp "$source"/* "$compiledir/src/action/"
-# if we have a single file action, copy it
-else cp "$source" "$compiledir/src/action/action.go"
-fi
-# prepare the main
-cat <<EOF >$compiledir/src/main/main.go
-package main
-
-import (
- "os"
- "action"
-
- "github.com/apache/incubator-openwhisk-client-go/whisk"
-)
-
-func main() {
- whisk.StartWithArgs(action.$main, os.Args[1:])
-}
-EOF
-# build it
-cd "$compiledir"
-GOPATH="$GOPATH:$compiledir" go build -i action
-GOPATH="$GOPATH:$compiledir" go build -o "$compilefile" main
-# if output is a directory use executable name
-if test -d "$dest"
-then dest="$dest/$exec"
-fi
-cp "$compilefile" "$dest"
-chmod +x "$dest"
diff --git a/golang1.10/Dockerfile b/golang1.10/Dockerfile
deleted file mode 100644
index f3a0877..0000000
--- a/golang1.10/Dockerfile
+++ /dev/null
@@ -1,14 +0,0 @@
-FROM golang:1.10.2
-RUN apt-get update && apt-get install -y \
- curl \
- jq \
- git \
- realpath \
- && rm -rf /var/lib/apt/lists/*
-RUN mkdir /action /home/go
-WORKDIR /action
-ADD proxy /proxy
-ADD gobuild.sh /bin/compile
-ENV COMPILER /bin/compile
-RUN go get github.com/apache/incubator-openwhisk-client-go/whisk
-CMD /proxy
diff --git a/golang1.11/Dockerfile b/golang1.11/Dockerfile
new file mode 100644
index 0000000..b5c7732
--- /dev/null
+++ b/golang1.11/Dockerfile
@@ -0,0 +1,14 @@
+FROM golang:1.11.0
+RUN apt-get update && apt-get install -y \
+ curl \
+ jq \
+ git \
+ realpath \
+ && rm -rf /var/lib/apt/lists/*
+RUN mkdir /action
+WORKDIR /action
+ADD proxy /bin/proxy
+ADD gobuild.py /bin/compile
+ADD gobuild.py.launcher.go /bin/compile.launcher.go
+ENV OW_COMPILER=/bin/compile
+ENTRYPOINT [ "/bin/proxy" ]
diff --git a/golang1.10/build.gradle b/golang1.11/build.gradle
similarity index 80%
rename from golang1.10/build.gradle
rename to golang1.11/build.gradle
index 9db4e57..ea0731c 100644
--- a/golang1.10/build.gradle
+++ b/golang1.11/build.gradle
@@ -15,11 +15,12 @@
* limitations under the License.
*/
-ext.dockerImageName = 'actionloop-golang-v1.10'
+ext.dockerImageName = 'actionloop-golang-v1.11'
apply from: '../gradle/docker.gradle'
distDocker.dependsOn 'copyProxy'
distDocker.dependsOn 'copyCompiler'
+distDocker.dependsOn 'copyEpilogue'
distDocker.finalizedBy('cleanup')
@@ -29,11 +30,17 @@
}
task copyCompiler(type: Copy) {
- from '../common/gobuild.sh'
+ from '../common/gobuild.py'
+ into '.'
+}
+
+task copyEpilogue(type: Copy) {
+ from '../common/gobuild.py.launcher.go'
into '.'
}
task cleanup(type: Delete) {
delete 'proxy'
- delete 'gobuild.sh'
+ delete 'gobuild.py'
+ delete 'gobuild.py.launcher.go'
}
diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties
index abd704b..d7cc33f 100644
--- a/gradle/wrapper/gradle-wrapper.properties
+++ b/gradle/wrapper/gradle-wrapper.properties
@@ -5,4 +5,4 @@
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-4.1-all.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-4.1-bin.zip
diff --git a/main/proxy.go b/main/proxy.go
index 65465c9..0a64e2e 100644
--- a/main/proxy.go
+++ b/main/proxy.go
@@ -25,26 +25,58 @@
"github.com/apache/incubator-openwhisk-runtime-go/openwhisk"
)
-// flag to enable tracing
-var trace = flag.Bool("trace", false, "enable detailed tracing output")
-
// flag to enable debug
var debug = flag.Bool("debug", false, "enable debug output")
-// flag for the compiler
-var compiler = flag.String("compiler", os.Getenv("COMPILER"), "define the compiler on the command line")
+// flag to require on-the-fly compilation
+var compile = flag.String("compile", "", "compile, reading in standard input the specified function, and producing the result in stdout")
+
+// use the runtime as a compiler "on-the-fly"
+func extractAndCompile(ap *openwhisk.ActionProxy) {
+ // read the std in
+ in, err := ioutil.ReadAll(os.Stdin)
+ if err != nil {
+ log.Printf(err.Error())
+ return
+ }
+
+ // extract and compile it
+ file, err := ap.ExtractAndCompile(&in, *compile)
+ if err != nil {
+ log.Printf(err.Error())
+ return
+ }
+
+ // read the file and write it to stdout
+ out, err := ioutil.ReadFile(file)
+ _, err = os.Stdout.Write(out)
+ if err != nil {
+ log.Printf(err.Error())
+ return
+ }
+}
func main() {
flag.Parse()
- if !(*debug || *trace) {
- // hide log unless you are debugging
- log.SetOutput(ioutil.Discard)
+ // debugging
+ if *debug {
+ // set debugging flag, propagated to the actions
+ openwhisk.Debugging = true
+ os.Setenv("OW_DEBUG", "1")
+ }
+
+ // create the action proxy
+ ap := openwhisk.NewActionProxy("./action", os.Getenv("OW_COMPILER"), os.Stdout, os.Stderr)
+
+ // compile on the fly upon request
+ if *compile != "" {
+ extractAndCompile(ap)
+ return
}
// start the balls rolling
- ap := openwhisk.NewActionProxy("./action", *compiler, os.Stdout)
- ap.Debug = *debug || *trace
- ap.Trace = *trace
+ openwhisk.Debug("OpenWhisk Go Proxy: starting")
ap.Start(8080)
+
}
diff --git a/openwhisk/_test/build.sh b/openwhisk/_test/build.sh
index 27a3a7d..db374d8 100755
--- a/openwhisk/_test/build.sh
+++ b/openwhisk/_test/build.sh
@@ -23,6 +23,16 @@
then return
fi
cp $1.src $1.go
+ GOPATH=$PWD go build -a -o $1 $1.go
+ rm $1.go
+}
+
+function build_main {
+ if test -e $1
+ then return
+ fi
+ cp ../../common/gobuild.py.launcher.go $1.go
+ cat $1.src >>$1.go
go build -a -o $1 $1.go
rm $1.go
}
@@ -37,8 +47,6 @@
rm -rf $$
}
-go get github.com/apache/incubator-openwhisk-runtime-go/openwhisk
-
build exec
rm exec.zip
zip -q -r exec.zip exec etc dir
@@ -46,11 +54,15 @@
build hi
zipit hi.zip hi main
-build hello_message
+build_main hello_message
zipit hello_message.zip hello_message main
zipit hello_message1.zip hello_message message
-build hello_greeting
+build_main hello_greeting
zipit hello_greeting.zip hello_greeting main
zipit hello_greeting1.zip hello_greeting greeting
+rm hello.zip
+cd src
+zip -q -r ../hello.zip main hello
+cd ..
diff --git a/openwhisk/_test/hello.sh b/openwhisk/_test/hello.sh
index 214ce63..ff21212 100755
--- a/openwhisk/_test/hello.sh
+++ b/openwhisk/_test/hello.sh
@@ -17,7 +17,7 @@
#
while read line
do
- name="$(echo $line | jq -r .name)"
+ name="$(echo $line | jq -r .value.name)"
if [ "$name" == "*" ]
then echo "Goodbye!" >&2
exit 0
@@ -25,4 +25,3 @@
echo msg="hello $name"
echo '{"hello": "'$name'"}' >&3
done
-
diff --git a/openwhisk/_test/hello.src b/openwhisk/_test/hello.src
index 2bfd8c1..8c8a5e3 100644
--- a/openwhisk/_test/hello.src
+++ b/openwhisk/_test/hello.src
@@ -15,21 +15,17 @@
* limitations under the License.
*/
-package action
+package main
-import (
- "encoding/json"
- "fmt"
-)
+import "fmt"
-func Main(event json.RawMessage) (json.RawMessage, error) {
- var obj map[string]interface{}
- json.Unmarshal(event, &obj)
+func Main(obj map[string]interface{}) map[string]interface{} {
name, ok := obj["name"].(string)
if !ok {
name = "Stranger"
}
fmt.Printf("name=%s\n", name)
- msg := map[string]string{"message": ("Hello, " + name + "!")}
- return json.Marshal(msg)
+ msg := make(map[string]interface{})
+ msg["message"] = "Hello, " + name + "!"
+ return msg
}
diff --git a/golang1.10/build.gradle b/openwhisk/_test/hello/hello.go
similarity index 64%
copy from golang1.10/build.gradle
copy to openwhisk/_test/hello/hello.go
index 9db4e57..1b25198 100644
--- a/golang1.10/build.gradle
+++ b/openwhisk/_test/hello/hello.go
@@ -15,25 +15,24 @@
* limitations under the License.
*/
-ext.dockerImageName = 'actionloop-golang-v1.10'
-apply from: '../gradle/docker.gradle'
+package hello
-distDocker.dependsOn 'copyProxy'
-distDocker.dependsOn 'copyCompiler'
-distDocker.finalizedBy('cleanup')
+import (
+ "fmt"
+)
-
-task copyProxy(type: Copy) {
- from '../actionProxyLoop/proxy'
- into '.'
-}
-
-task copyCompiler(type: Copy) {
- from '../common/gobuild.sh'
- into '.'
-}
-
-task cleanup(type: Delete) {
- delete 'proxy'
- delete 'gobuild.sh'
+// Hello receive an event in format
+// { "name": "Mike"}
+// and returns a greeting in format
+// { "greetings": "Hello, Mike"}
+func Hello(args map[string]interface{}) map[string]interface{} {
+ res := make(map[string]interface{})
+ greetings := "Stranger"
+ name, ok := args["name"].(string)
+ if ok {
+ greetings = name
+ }
+ res["greetings"] = "Hello, " + greetings
+ fmt.Printf("Hello, %s\n", greetings)
+ return res
}
diff --git a/openwhisk/_test/hello1.src b/openwhisk/_test/hello1.src
index 95e1586..e7d0993 100644
--- a/openwhisk/_test/hello1.src
+++ b/openwhisk/_test/hello1.src
@@ -15,26 +15,19 @@
* limitations under the License.
*/
-package action
+package main
import (
- "encoding/json"
"fmt"
)
-func Hello(event json.RawMessage) (json.RawMessage, error) {
- var obj struct {
- Name string `json:",omitempty"`
- }
- err := json.Unmarshal(event, &obj)
- if err != nil {
- return nil, err
- }
- name := obj.Name
- if name == "" {
+func hello(obj map[string]interface{}) map[string]interface{} {
+ name, ok := obj["name"].(string)
+ if !ok {
name = "Stranger"
}
fmt.Printf("name=%s\n", name)
- msg := map[string]string{"hello": ("Hello, " + name + "!")}
- return json.Marshal(msg)
+ msg := make(map[string]interface{})
+ msg["hello"] = "Hello, " + name + "!"
+ return msg
}
diff --git a/golang1.10/build.gradle b/openwhisk/_test/hello2.src
similarity index 64%
copy from golang1.10/build.gradle
copy to openwhisk/_test/hello2.src
index 9db4e57..26dc66f 100644
--- a/golang1.10/build.gradle
+++ b/openwhisk/_test/hello2.src
@@ -15,25 +15,15 @@
* limitations under the License.
*/
-ext.dockerImageName = 'actionloop-golang-v1.10'
-apply from: '../gradle/docker.gradle'
+package main
-distDocker.dependsOn 'copyProxy'
-distDocker.dependsOn 'copyCompiler'
-distDocker.finalizedBy('cleanup')
+import (
+ "fmt"
+ "hello"
+)
-
-task copyProxy(type: Copy) {
- from '../actionProxyLoop/proxy'
- into '.'
-}
-
-task copyCompiler(type: Copy) {
- from '../common/gobuild.sh'
- into '.'
-}
-
-task cleanup(type: Delete) {
- delete 'proxy'
- delete 'gobuild.sh'
+// Main forwading to Hello
+func Main(args map[string]interface{}) map[string]interface{} {
+ fmt.Println("Main")
+ return hello.Hello(args)
}
diff --git a/openwhisk/_test/hello_greeting.src b/openwhisk/_test/hello_greeting.src
index 4163075..d4b490b 100644
--- a/openwhisk/_test/hello_greeting.src
+++ b/openwhisk/_test/hello_greeting.src
@@ -14,17 +14,23 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package main
-import (
- //"log"
- "os"
+func Main(obj map[string]interface{}) map[string]interface{} {
+ return Hello(obj)
+}
- "github.com/apache/incubator-openwhisk-client-go/whisk"
- "github.com/apache/incubator-openwhisk-runtime-go/openwhisk/_test/action"
-)
-
-func main() {
- //log.SetPrefix("hello_greeting: ")
- whisk.StartWithArgs(action.Hello, os.Args[1:])
+// Hello receive an event in format
+// { "name": "Mike"}
+// and returns a greeting in format
+// { "greetings": "Hello, Mike"}
+func Hello(args map[string]interface{}) map[string]interface{} {
+ res := make(map[string]interface{})
+ greetings := "Stranger"
+ name, ok := args["name"].(string)
+ if ok {
+ greetings = name
+ }
+ res["greetings"] = "Hello, " + greetings
+ fmt.Printf("Hello, %s\n", greetings)
+ return res
}
diff --git a/openwhisk/_test/hello_message.src b/openwhisk/_test/hello_message.src
index 3c89484..bce622f 100644
--- a/openwhisk/_test/hello_message.src
+++ b/openwhisk/_test/hello_message.src
@@ -14,29 +14,15 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package main
-import (
- "encoding/json"
- "fmt"
- "os"
-
- "github.com/apache/incubator-openwhisk-client-go/whisk"
-)
-
-func sayHello(event json.RawMessage) (json.RawMessage, error) {
- var obj map[string]interface{}
- json.Unmarshal(event, &obj)
+func Main(obj map[string]interface{}) map[string]interface{} {
name, ok := obj["name"].(string)
if !ok {
name = "Stranger"
}
fmt.Printf("name=%s\n", name)
- msg := map[string]string{"message": ("Hello, " + name + "!")}
- return json.Marshal(msg)
+ msg := make(map[string]interface{})
+ msg["message"] = "Hello, " + name + "!"
+ return msg
}
-func main() {
- //log.SetPrefix("hello_message: ")
- whisk.StartWithArgs(sayHello, os.Args[1:])
-}
diff --git a/golang1.10/build.gradle b/openwhisk/_test/hi1.src
similarity index 64%
copy from golang1.10/build.gradle
copy to openwhisk/_test/hi1.src
index 9db4e57..edb45f0 100644
--- a/golang1.10/build.gradle
+++ b/openwhisk/_test/hi1.src
@@ -15,25 +15,11 @@
* limitations under the License.
*/
-ext.dockerImageName = 'actionloop-golang-v1.10'
-apply from: '../gradle/docker.gradle'
-
-distDocker.dependsOn 'copyProxy'
-distDocker.dependsOn 'copyCompiler'
-distDocker.finalizedBy('cleanup')
+package main
-task copyProxy(type: Copy) {
- from '../actionProxyLoop/proxy'
- into '.'
-}
-
-task copyCompiler(type: Copy) {
- from '../common/gobuild.sh'
- into '.'
-}
-
-task cleanup(type: Delete) {
- delete 'proxy'
- delete 'gobuild.sh'
+func main() {
+ f := bufio.NewWriter(os.Stdout)
+ defer f.Flush()
+ f.Write([]byte("hi\n"))
}
diff --git a/openwhisk/_test/postcompile.sh b/openwhisk/_test/postcompile.sh
index 85be6c7..c25620f 100755
--- a/openwhisk/_test/postcompile.sh
+++ b/openwhisk/_test/postcompile.sh
@@ -17,6 +17,6 @@
#
FILE=${1:?compiled file}
file --mime-type "$FILE" | sed -e 's/x-mach-binary/x-executable/'
-echo '{"name":"Mike"}' | $FILE 3>/tmp/$$
+echo '{"value":{"name":"Mike"}}' | $FILE 3>/tmp/$$
cat /tmp/$$
rm /tmp/$$
diff --git a/openwhisk/_test/precompile.sh b/openwhisk/_test/precompile.sh
index d2b5f01..9f6697f 100755
--- a/openwhisk/_test/precompile.sh
+++ b/openwhisk/_test/precompile.sh
@@ -16,15 +16,14 @@
# limitations under the License.
#
cd "$(dirname $0)"
-SRC=${1:?source}
-ID=${2:?numbe}
-go get github.com/apache/incubator-openwhisk-client-go/whisk
+SRC=${1:?source file or dir}
+ID=${2:?target dir}
+TGT=${3:-main}
+ETC=${4:-}
rm -Rvf compile/$ID >/dev/null
-rm -Rvf output/$ID >/dev/null
-mkdir -p compile/$ID output/$ID
-if test -d "$SRC"
-then cp -r "$SRC" compile/$ID
-else cp $SRC compile/$ID/exec
+mkdir -p compile/$ID/src compile/$ID/bin
+cp $SRC compile/$ID/src/$TGT
+if test -d "$ETC"
+then cp -r "$ETC" compile/$ID/src
fi
-
diff --git a/golang1.10/build.gradle b/openwhisk/_test/src/hello/hello.go
similarity index 63%
copy from golang1.10/build.gradle
copy to openwhisk/_test/src/hello/hello.go
index 9db4e57..1b25198 100644
--- a/golang1.10/build.gradle
+++ b/openwhisk/_test/src/hello/hello.go
@@ -15,25 +15,24 @@
* limitations under the License.
*/
-ext.dockerImageName = 'actionloop-golang-v1.10'
-apply from: '../gradle/docker.gradle'
+package hello
-distDocker.dependsOn 'copyProxy'
-distDocker.dependsOn 'copyCompiler'
-distDocker.finalizedBy('cleanup')
+import (
+ "fmt"
+)
-
-task copyProxy(type: Copy) {
- from '../actionProxyLoop/proxy'
- into '.'
-}
-
-task copyCompiler(type: Copy) {
- from '../common/gobuild.sh'
- into '.'
-}
-
-task cleanup(type: Delete) {
- delete 'proxy'
- delete 'gobuild.sh'
+// Hello receive an event in format
+// { "name": "Mike"}
+// and returns a greeting in format
+// { "greetings": "Hello, Mike"}
+func Hello(args map[string]interface{}) map[string]interface{} {
+ res := make(map[string]interface{})
+ greetings := "Stranger"
+ name, ok := args["name"].(string)
+ if ok {
+ greetings = name
+ }
+ res["greetings"] = "Hello, " + greetings
+ fmt.Printf("Hello, %s\n", greetings)
+ return res
}
diff --git a/golang1.10/build.gradle b/openwhisk/_test/src/main
similarity index 64%
copy from golang1.10/build.gradle
copy to openwhisk/_test/src/main
index 9db4e57..26dc66f 100644
--- a/golang1.10/build.gradle
+++ b/openwhisk/_test/src/main
@@ -15,25 +15,15 @@
* limitations under the License.
*/
-ext.dockerImageName = 'actionloop-golang-v1.10'
-apply from: '../gradle/docker.gradle'
+package main
-distDocker.dependsOn 'copyProxy'
-distDocker.dependsOn 'copyCompiler'
-distDocker.finalizedBy('cleanup')
+import (
+ "fmt"
+ "hello"
+)
-
-task copyProxy(type: Copy) {
- from '../actionProxyLoop/proxy'
- into '.'
-}
-
-task copyCompiler(type: Copy) {
- from '../common/gobuild.sh'
- into '.'
-}
-
-task cleanup(type: Delete) {
- delete 'proxy'
- delete 'gobuild.sh'
+// Main forwading to Hello
+func Main(args map[string]interface{}) map[string]interface{} {
+ fmt.Println("Main")
+ return hello.Hello(args)
}
diff --git a/openwhisk/actionProxy.go b/openwhisk/actionProxy.go
index 50299e9..738e233 100644
--- a/openwhisk/actionProxy.go
+++ b/openwhisk/actionProxy.go
@@ -24,8 +24,15 @@
"os"
)
+// OutputGuard constant string
+const OutputGuard = "XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX"
+
// ActionProxy is the container of the data specific to a server
type ActionProxy struct {
+
+ // is it initialized?
+ initialized bool
+
// current directory
baseDir string
@@ -38,25 +45,22 @@
// theChannel is the channel communicating with the action
theExecutor *Executor
- // log file
- logFile *os.File
-
- // debug
- Debug bool
- Trace bool
+ // out and err files
+ outFile *os.File
+ errFile *os.File
}
// NewActionProxy creates a new action proxy that can handle http requests
-func NewActionProxy(baseDir string, compiler string, logFile *os.File) *ActionProxy {
+func NewActionProxy(baseDir string, compiler string, outFile *os.File, errFile *os.File) *ActionProxy {
os.Mkdir(baseDir, 0755)
return &ActionProxy{
+ false,
baseDir,
compiler,
highestDir(baseDir),
nil,
- logFile,
- false,
- false,
+ outFile,
+ errFile,
}
}
@@ -69,7 +73,7 @@
// find the action if any
highestDir := highestDir(ap.baseDir)
if highestDir == 0 {
- log.Println("no action found")
+ Debug("no action found")
ap.theExecutor = nil
return fmt.Errorf("no valid actions available")
}
@@ -78,15 +82,15 @@
curExecutor := ap.theExecutor
// try to launch the action
- executable := fmt.Sprintf("%s/%d/%s", ap.baseDir, highestDir, main)
+ executable := fmt.Sprintf("%s/%d/bin/%s", ap.baseDir, highestDir, main)
os.Chmod(executable, 0755)
- newExecutor := NewExecutor(ap.logFile, executable)
- log.Printf("starting %s", executable)
+ newExecutor := NewExecutor(ap.outFile, ap.errFile, executable)
+ Debug("starting %s", executable)
err := newExecutor.Start()
if err == nil {
ap.theExecutor = newExecutor
if curExecutor != nil {
- log.Println("stopping old executor")
+ Debug("stopping old executor")
curExecutor.Stop()
}
return nil
@@ -94,9 +98,9 @@
// cannot start, removing the action
// and leaving the current executor running
- if !ap.Debug {
+ if !Debugging {
exeDir := fmt.Sprintf("./action/%d/", highestDir)
- log.Printf("removing the failed action in %s", exeDir)
+ Debug("removing the failed action in %s", exeDir)
os.RemoveAll(exeDir)
}
diff --git a/openwhisk/actionProxy_test.go b/openwhisk/actionProxy_test.go
index 23c18e3..5f909ec 100644
--- a/openwhisk/actionProxy_test.go
+++ b/openwhisk/actionProxy_test.go
@@ -38,44 +38,47 @@
fmt.Print(res)
stopTestServer(ts, cur, log)
// Output:
- // {"ok":true}
+ // {"error":"Missing main/no code to execute."}
// {"error":"Error unmarshaling request: invalid character 'X' looking for beginning of value"}
// {"error":"no action defined yet"}
- // {"error":"Error unmarshaling request: invalid character 'X' looking for beginning of value"}
+ // {"error":"no action defined yet"}
}
-func TestStartLatestAction(t *testing.T) {
- // cleanup
- os.RemoveAll("./action")
+func TestStartLatestAction_emit1(t *testing.T) {
+ os.RemoveAll("./action/t2")
logf, _ := ioutil.TempFile("/tmp", "log")
- ap := NewActionProxy("./action", "", logf)
+ ap := NewActionProxy("./action/t2", "", logf, logf)
+ // start the action that emits 1
+ buf := []byte("#!/bin/sh\nwhile read a; do echo 1 >&3 ; done\n")
+ ap.ExtractAction(&buf, "main", "bin")
+ ap.StartLatestAction("main")
+ ap.theExecutor.io <- []byte("x")
+ res := <-ap.theExecutor.io
+ assert.Equal(t, res, []byte("1\n"))
+ ap.theExecutor.Stop()
+}
- // start an action that terminate immediately
+func TestStartLatestAction_terminate(t *testing.T) {
+ os.RemoveAll("./action/t3")
+ logf, _ := ioutil.TempFile("/tmp", "log")
+ ap := NewActionProxy("./action/t3", "", logf, logf)
+ // now start an action that terminate immediately
buf := []byte("#!/bin/sh\ntrue\n")
- ap.ExtractAction(&buf, "main")
+ ap.ExtractAction(&buf, "main", "bin")
ap.StartLatestAction("main")
assert.Nil(t, ap.theExecutor)
+}
- // start the action that emits 1
- buf = []byte("#!/bin/sh\nwhile read a; do echo 1 >&3 ; done\n")
- ap.ExtractAction(&buf, "main")
- ap.StartLatestAction("main")
- ap.theExecutor.io <- "x"
- assert.Equal(t, <-ap.theExecutor.io, "1")
-
- // now start an action that terminate immediately
- buf = []byte("#!/bin/sh\ntrue\n")
- ap.ExtractAction(&buf, "main")
- ap.StartLatestAction("main")
- ap.theExecutor.io <- "y"
- assert.Equal(t, <-ap.theExecutor.io, "1")
-
+func TestStartLatestAction_emit2(t *testing.T) {
+ os.RemoveAll("./action/t4")
+ logf, _ := ioutil.TempFile("/tmp", "log")
+ ap := NewActionProxy("./action/t4", "", logf, logf)
// start the action that emits 2
- buf = []byte("#!/bin/sh\nwhile read a; do echo 2 >&3 ; done\n")
- ap.ExtractAction(&buf, "main")
+ buf := []byte("#!/bin/sh\nwhile read a; do echo 2 >&3 ; done\n")
+ ap.ExtractAction(&buf, "main", "bin")
ap.StartLatestAction("main")
- ap.theExecutor.io <- "z"
- assert.Equal(t, <-ap.theExecutor.io, "2")
+ ap.theExecutor.io <- []byte("z")
+ assert.Equal(t, <-ap.theExecutor.io, []byte("2\n"))
/**/
ap.theExecutor.Stop()
}
diff --git a/openwhisk/compiler.go b/openwhisk/compiler.go
index bd3e879..bd6c0d9 100644
--- a/openwhisk/compiler.go
+++ b/openwhisk/compiler.go
@@ -52,18 +52,18 @@
buf, err := ioutil.ReadFile(file)
if err != nil {
- log.Println(err)
+ Debug(err.Error())
return false
}
- // if a mac add a matched for mac
+ // if this is mac add a matcher for mac
if runtime.GOOS == "darwin" {
filetype.AddMatcher(mach64Type, mach64Matcher)
}
kind, err := filetype.Match(buf)
- log.Printf("isCompiled: %s kind=%s", file, kind)
+ Debug("isCompiled: %s kind=%s", file, kind)
if err != nil {
- log.Println(err)
+ Debug(err.Error())
return false
}
if kind.Extension == "elf" {
@@ -76,18 +76,25 @@
}
// CompileAction will compile an anction in source format invoking a compiler
-func (ap *ActionProxy) CompileAction(main string, src string, out string) error {
+func (ap *ActionProxy) CompileAction(main string, src_dir string, bin_dir string) error {
if ap.compiler == "" {
return fmt.Errorf("No compiler defined")
}
- log.Printf("compiling: compiler=%s main=%s src=%s out=%s", ap.compiler, main, src, out)
+
+ Debug("compiling: %s %s %s %s", ap.compiler, main, src_dir, bin_dir)
+
var cmd *exec.Cmd
- if out == "" {
- cmd = exec.Command(ap.compiler, main, src, src)
- } else {
- cmd = exec.Command(ap.compiler, main, src, out)
+ cmd = exec.Command(ap.compiler, main, src_dir, bin_dir)
+ cmd.Env = []string{"PATH=" + os.Getenv("PATH")}
+
+ // gather stdout and stderr
+ out, err := cmd.CombinedOutput()
+ Debug("compiler out: %s, %v", out, err)
+ if err != nil {
+ return err
}
- res, err := cmd.CombinedOutput()
- log.Print(string(res))
- return err
+ if len(out) > 0 {
+ return fmt.Errorf("%s", out)
+ }
+ return nil
}
diff --git a/openwhisk/compiler_test.go b/openwhisk/compiler_test.go
index 72ed95e..3fa609e 100644
--- a/openwhisk/compiler_test.go
+++ b/openwhisk/compiler_test.go
@@ -19,168 +19,107 @@
import (
"fmt"
- "io/ioutil"
+ "os"
)
-/* this test confuses gogradle
-func Example_compileAction_wrong() {
- sys("_test/precompile.sh", "hello.sh", "0")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/0", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("_test/compile/0/exec", "exec", ""))
- // Output:
- // exit status 1
-}*/
-
/**
-
-Note to understand tests:
-- tests are run from the openwhisk as the current directory
-- compiler (../../common/gobuild.sh) takes 3 arguments:
- <main> <source-dir-or-file> <target-dir>
-
- You create a proxy (NewActionProxy) with the target dir
- then invoke compilation with the source dir (implicit is 'main' as the function)
-
- You can test if compilation works with
-
- cd openwhisk
- ../common/gobuild.sh main _test/compile/c/exe ./action/c
-
- _test/precompile.sh simply copies files to test folder
- _test/postcompile.sh simply checks if the file is compiled
-
-
+Notes to understand tests:
+- tests are run from the "openwhisk" folder, as the current directory
+- precompile.sh prepare a compilation enviroment:
+ _test/precompile.sh hello.src aaa main
+ produces
+ - _test/compile/src/aaa/src/main
+ - _test/compile/src/aaa/bin/
+ ready for the compiler
+- compiler (../../common/gobuild.py) takes 3 arguments:
+ <main> <source-dir> <target-dir>
+ - it will look for a <source-dir>/<main> file
+ - will generate some files in <source-dir>
+ - compiler output is in <target-dir>/<main>
+ - postcompile.sh will
+ - execute the binary with 3>&1
+ - feed it with the json '{"name":"Mike"}
+ - will print the type of the executable and its output and log
*/
-func Example_isCompiled() {
- sys("_test/precompile.sh", "hello.src", "c")
- file := abs("./_test/compile/c/exec")
- dir := abs("./_test/compile/c")
- fmt.Println(isCompiled(file, "main"))
- fmt.Println(isCompiled(dir, "exec"))
+const (
+ PREP = "_test/precompile.sh"
+ CHECK = "_test/postcompile.sh"
+ TMP = "_test/compile/"
+ COMP = "../common/gobuild.py"
+)
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/c", "../common/gobuild.sh", log)
- ap.CompileAction("main", abs("_test/compile/c/exec"), "")
-
- fmt.Println(isCompiled(file, "main"))
- fmt.Println(isCompiled(dir, "exec"))
+// compile a main
+func Example() {
+ sys(PREP, "hello.src", "0", "main")
+ ap := NewActionProxy(TMP, COMP, os.Stdout, os.Stderr)
+ fmt.Println(isCompiled(TMP+"0/src", "main"))
+ fmt.Println(isCompiled(TMP+"0/src/main", "main"))
+ ap.CompileAction("main", TMP+"0/src", TMP+"0/bin")
+ sys(CHECK, TMP+"0/bin/main")
+ fmt.Println(isCompiled(TMP+"0/bin", "main"))
+ fmt.Println(isCompiled(TMP+"0/bin/main", "main"))
// errors
- fmt.Println(isCompiled(dir, "main"))
- fmt.Println(isCompiled(file+"1", "main"))
-
+ fmt.Println(isCompiled(TMP+"0/bin1/main", "main"))
+ fmt.Println(isCompiled(TMP+"0/bin/main1", "main"))
// Output:
// false
// false
+ // _test/compile/0/bin/main: application/x-executable
+ // name=Mike
+ // {"message":"Hello, Mike!"}
// true
// true
// false
// false
}
-func Example_compileAction_singlefile_main() {
- sys("_test/precompile.sh", "hello.src", "1")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/1", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("main", abs("_test/compile/1/exec"), ""))
- sys("_test/postcompile.sh", "_test/compile/1/exec")
+// compile a not-main (hello) function
+func Example_hello() {
+ N := "1"
+ sys(PREP, "hello1.src", N, "hello")
+ ap := NewActionProxy(TMP, COMP, os.Stdout, os.Stderr)
+ ap.CompileAction("hello", TMP+N+"/src", TMP+N+"/bin")
+ sys(CHECK, TMP+N+"/bin/hello")
// Output:
- // <nil>
- // _test/compile/1/exec: application/x-executable
- // name=Mike
- // {"message":"Hello, Mike!"}
-}
-
-func Example_compileAction_singlefile_main_out() {
- sys("_test/precompile.sh", "hello.src", "1a")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/1a", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("main", abs("_test/compile/1a/exec"), abs("_test/output/1a")))
- sys("_test/postcompile.sh", "_test/output/1a/main")
- // Output:
- // <nil>
- // _test/output/1a/main: application/x-executable
- // name=Mike
- // {"message":"Hello, Mike!"}
-}
-
-func Example_compileAction_singlefile_hello() {
- sys("_test/precompile.sh", "hello1.src", "2")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/2", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("hello", "_test/compile/2/exec", ""))
- sys("_test/postcompile.sh", "_test/compile/2/exec")
- // Output:
- // <nil>
- // _test/compile/2/exec: application/x-executable
+ // _test/compile/1/bin/hello: application/x-executable
// name=Mike
// {"hello":"Hello, Mike!"}
}
-func Example_compileAction_singlefile_hello_out() {
- sys("_test/precompile.sh", "hello1.src", "2a")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/2a", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("hello", "_test/compile/2a/exec", abs("_test/output/2a")))
- sys("_test/postcompile.sh", "_test/output/2a/hello")
+// compile a function including a package
+func Example_package() {
+ N := "2"
+ sys(PREP, "hello2.src", N, "main", "hello")
+ ap := NewActionProxy(TMP, COMP, os.Stdout, os.Stderr)
+ ap.CompileAction("main", TMP+N+"/src", TMP+N+"/bin")
+ sys(CHECK, TMP+N+"/bin/main")
// Output:
- // <nil>
- // _test/output/2a/hello: application/x-executable
- // name=Mike
- // {"hello":"Hello, Mike!"}
-}
-
-func Example_compileAction_multifile_main() {
- sys("_test/precompile.sh", "action", "3")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/3", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("main", "_test/compile/3/", ""))
- sys("_test/postcompile.sh", "_test/compile/3/main")
- // Output:
- // <nil>
- // _test/compile/3/main: application/x-executable
- // Main:
+ // _test/compile/2/bin/main: application/x-executable
+ // Main
// Hello, Mike
// {"greetings":"Hello, Mike"}
}
-func Example_compileAction_multifile_main_out() {
- sys("_test/precompile.sh", "action", "3a")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/3a", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("main", "_test/compile/3a/", abs("_test/output/3a")))
- sys("_test/postcompile.sh", "_test/output/3a/main")
- // Output:
- // <nil>
- // _test/output/3a/main: application/x-executable
- // Main:
- // Hello, Mike
- // {"greetings":"Hello, Mike"}
+func Example_compileError() {
+ N := "6"
+ sys(PREP, "hi1.src", N)
+ ap := NewActionProxy(TMP, COMP, os.Stdout, os.Stderr)
+ err := ap.CompileAction("main", TMP+N+"/src", TMP+N+"/bin")
+ fmt.Printf("%v", removeLineNr(err.Error()))
+ // Unordered output:
+ // ./func_Main_.go::: undefined: bufio
+ // ./func_Main_.go::: undefined: os
}
-func Example_compileAction_multifile_hello() {
- sys("_test/precompile.sh", "action", "4")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/4", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("hello", "_test/compile/4/", ""))
- sys("_test/postcompile.sh", "_test/compile/4/hello")
+func Example_withMain() {
+ N := "7"
+ sys(PREP, "hi.src", N)
+ ap := NewActionProxy(TMP, COMP, os.Stdout, os.Stderr)
+ err := ap.CompileAction("main", TMP+N+"/src", TMP+N+"/bin")
+ fmt.Println(err)
+ sys(TMP + N + "/bin/main")
// Output:
// <nil>
- // _test/compile/4/hello: application/x-executable
- // Hello, Mike
- // {"greetings":"Hello, Mike"}
-}
-
-func Example_compileAction_multifile_hello_out() {
- sys("_test/precompile.sh", "action", "4a")
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/4a", "../common/gobuild.sh", log)
- fmt.Println(ap.CompileAction("hello", "_test/compile/4/", abs("_test/output/4a")))
- sys("_test/postcompile.sh", "_test/output/4a/hello")
- // Output:
- // <nil>
- // _test/output/4a/hello: application/x-executable
- // Hello, Mike
- // {"greetings":"Hello, Mike"}
+ // hi
}
diff --git a/golang1.10/build.gradle b/openwhisk/debug.go
similarity index 65%
copy from golang1.10/build.gradle
copy to openwhisk/debug.go
index 9db4e57..766aabf 100644
--- a/golang1.10/build.gradle
+++ b/openwhisk/debug.go
@@ -14,26 +14,25 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+package openwhisk
-ext.dockerImageName = 'actionloop-golang-v1.10'
-apply from: '../gradle/docker.gradle'
+import "log"
-distDocker.dependsOn 'copyProxy'
-distDocker.dependsOn 'copyCompiler'
-distDocker.finalizedBy('cleanup')
+// Debugging flag
+var Debugging = false
-
-task copyProxy(type: Copy) {
- from '../actionProxyLoop/proxy'
- into '.'
+// Debug emits a debug message
+func Debug(format string, args ...interface{}) {
+ if Debugging {
+ log.Printf(format, args...)
+ }
}
-task copyCompiler(type: Copy) {
- from '../common/gobuild.sh'
- into '.'
-}
-
-task cleanup(type: Delete) {
- delete 'proxy'
- delete 'gobuild.sh'
+// DebugLimit emits a debug message with a limit in lenght
+func DebugLimit(msg string, in []byte, limit int) {
+ if len(in) < limit {
+ Debug("%s:%s", msg, in)
+ } else {
+ Debug("%s:%s...", msg, in[0:limit])
+ }
}
diff --git a/openwhisk/executor.go b/openwhisk/executor.go
index 3f29eee..ad1b549 100644
--- a/openwhisk/executor.go
+++ b/openwhisk/executor.go
@@ -20,7 +20,7 @@
import (
"bufio"
"fmt"
- "log"
+ "io"
"os"
"os/exec"
"runtime"
@@ -36,22 +36,30 @@
// Executor is the container and the guardian of a child process
// It starts a command, feeds input and output, read logs and control its termination
type Executor struct {
- io chan string
+ io chan []byte
log chan bool
exit chan error
_cmd *exec.Cmd
- _input *bufio.Writer
- _output *bufio.Scanner
- _logout *bufio.Scanner
- _logerr *bufio.Scanner
- _logbuf *os.File
+ _input io.WriteCloser
+ _output *bufio.Reader
+ _logout *bufio.Reader
+ _logerr *bufio.Reader
+ _outbuf *os.File
+ _errbuf *os.File
}
// NewExecutor creates a child subprocess using the provided command line,
// writing the logs in the given file.
// You can then start it getting a communication channel
-func NewExecutor(logbuf *os.File, command string, args ...string) (proc *Executor) {
+func NewExecutor(outbuf *os.File, errbuf *os.File, command string, args ...string) (proc *Executor) {
cmd := exec.Command(command, args...)
+ cmd.Env = []string{
+ "__OW_API_HOST=" + os.Getenv("__OW_API_HOST"),
+ }
+ if Debugging {
+ cmd.Env = append(cmd.Env, "OW_DEBUG=/tmp/action.log")
+ }
+ Debug("env: %v", cmd.Env)
stdin, err := cmd.StdinPipe()
if err != nil {
@@ -74,60 +82,70 @@
}
cmd.ExtraFiles = []*os.File{pipeIn}
+ pout := bufio.NewReader(pipeOut)
+ sout := bufio.NewReader(stdout)
+ serr := bufio.NewReader(stderr)
+
return &Executor{
- make(chan string),
+ make(chan []byte),
make(chan bool),
make(chan error),
cmd,
- bufio.NewWriter(stdin),
- bufio.NewScanner(pipeOut),
- bufio.NewScanner(stdout),
- bufio.NewScanner(stderr),
- logbuf,
+ stdin,
+ pout,
+ sout,
+ serr,
+ outbuf,
+ errbuf,
}
}
// collect log from a stream
-func _collect(ch chan string, scan *bufio.Scanner) {
- for scan.Scan() {
- ch <- scan.Text()
+func _collect(ch chan string, reader *bufio.Reader) {
+ for {
+ buf, err := reader.ReadBytes('\n')
+ if err != nil {
+ break
+ }
+ ch <- string(buf)
}
}
// loop over the command executing
// returning when the command exits
func (proc *Executor) run() {
- log.Println("run: start")
+ Debug("run: start")
err := proc._cmd.Start()
if err != nil {
proc.exit <- err
- log.Println("run: early exit")
+ Debug("run: early exit")
proc._cmd = nil // do not kill
return
}
+ Debug("pid: %d", proc._cmd.Process.Pid)
// wait for the exit
proc.exit <- proc._cmd.Wait()
proc._cmd = nil // do not kill
- log.Println("run: end")
+ Debug("run: end")
}
-func (proc *Executor) drain(ch chan string) {
+func drain(ch chan string, out *os.File) {
for loop := true; loop; {
runtime.Gosched()
select {
case buf := <-ch:
- fmt.Fprintln(proc._logbuf, buf)
+ fmt.Fprint(out, buf)
case <-time.After(DefaultTimeoutDrain):
loop = false
}
}
- fmt.Fprintln(proc._logbuf, "XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX")
+ fmt.Fprintln(out, OutputGuard)
}
// manage copying stdout and stder in output
// with log guards
func (proc *Executor) logger() {
- log.Println("logger: start")
+ Debug("logger: start")
// poll stdout and stderr
chOut := make(chan string)
go _collect(chOut, proc._logout)
@@ -137,12 +155,13 @@
// wait for the signal
for <-proc.log {
// flush stdout
- proc.drain(chOut)
+ drain(chOut, proc._outbuf)
// flush stderr
- proc.drain(chErr)
+ drain(chErr, proc._errbuf)
}
- proc._logbuf.Sync()
- log.Printf("logger: end")
+ proc._outbuf.Sync()
+ proc._errbuf.Sync()
+ Debug("logger: end")
}
// main service function
@@ -150,27 +169,35 @@
// and reading in output
// using the provide channels
func (proc *Executor) service() {
- log.Println("service: start")
+ Debug("service: start")
for {
in := <-proc.io
- if in == "" {
- log.Println("terminated upon request")
+ if len(in) == 0 {
+ Debug("terminated upon request")
break
}
- // input/output with the process
- log.Printf(">>>%s\n", in)
- proc._input.WriteString(in + "\n")
- proc._input.Flush()
- if proc._output.Scan() {
- out := proc._output.Text()
- log.Printf("<<<%s\n", out)
- proc.io <- out
- if out == "" {
- break
- }
+ // input to the subprocess
+ DebugLimit(">>>", in, 120)
+ proc._input.Write(in)
+ proc._input.Write([]byte("\n"))
+ Debug("done")
+
+ // ok now give a chance to run to goroutines
+ runtime.Gosched()
+
+ // input to the subprocess
+ out, err := proc._output.ReadBytes('\n')
+ if err != nil {
+ break
+ }
+ DebugLimit("<<<", out, 120)
+ proc.io <- out
+ if len(out) == 0 {
+ Debug("empty input - exiting")
+ break
}
}
- log.Printf("service: end")
+ Debug("service: end")
}
// Start execution of the command
@@ -194,10 +221,10 @@
// Stop will kill the process
// and close the channels
func (proc *Executor) Stop() {
- log.Println("stopping")
+ Debug("stopping")
if proc._cmd != nil {
proc.log <- false
- proc.io <- ""
+ proc.io <- []byte("")
proc._cmd.Process.Kill()
<-proc.exit
proc._cmd = nil
diff --git a/openwhisk/executor_test.go b/openwhisk/executor_test.go
index 5c78bea..6a649ed 100644
--- a/openwhisk/executor_test.go
+++ b/openwhisk/executor_test.go
@@ -23,19 +23,19 @@
func ExampleNewExecutor_failed() {
log, _ := ioutil.TempFile("", "log")
- proc := NewExecutor(log, "true")
+ proc := NewExecutor(log, log, "true")
err := proc.Start()
fmt.Println(err)
proc.Stop()
- proc = NewExecutor(log, "/bin/pwd")
+ proc = NewExecutor(log, log, "/bin/pwd")
err = proc.Start()
fmt.Println(err)
proc.Stop()
- proc = NewExecutor(log, "donotexist")
+ proc = NewExecutor(log, log, "donotexist")
err = proc.Start()
fmt.Println(err)
proc.Stop()
- proc = NewExecutor(log, "/etc/passwd")
+ proc = NewExecutor(log, log, "/etc/passwd")
err = proc.Start()
fmt.Println(err)
proc.Stop()
@@ -48,18 +48,18 @@
func ExampleNewExecutor_bc() {
log, _ := ioutil.TempFile("", "log")
- proc := NewExecutor(log, "_test/bc.sh")
+ proc := NewExecutor(log, log, "_test/bc.sh")
err := proc.Start()
fmt.Println(err)
//proc.log <- true
- proc.io <- "2+2"
- fmt.Println(<-proc.io)
+ proc.io <- []byte("2+2")
+ fmt.Printf("%s", <-proc.io)
// and now, exit detection
- proc.io <- "quit"
+ proc.io <- []byte("quit")
proc.log <- true
select {
case in := <-proc.io:
- fmt.Println(in)
+ fmt.Printf("%s", in)
case <-proc.exit:
fmt.Println("exit")
}
@@ -76,11 +76,11 @@
func ExampleNewExecutor_hello() {
log, _ := ioutil.TempFile("", "log")
- proc := NewExecutor(log, "_test/hello.sh")
+ proc := NewExecutor(log, log, "_test/hello.sh")
err := proc.Start()
fmt.Println(err)
- proc.io <- `{"name":"Mike"}`
- fmt.Println(<-proc.io)
+ proc.io <- []byte(`{"value":{"name":"Mike"}}`)
+ fmt.Printf("%s", <-proc.io)
proc.log <- true
waitabit()
proc.Stop()
@@ -99,10 +99,10 @@
func ExampleNewExecutor_term() {
log, _ := ioutil.TempFile("", "log")
- proc := NewExecutor(log, "_test/hello.sh")
+ proc := NewExecutor(log, log, "_test/hello.sh")
err := proc.Start()
fmt.Println(err)
- proc.io <- `{"name":"*"}`
+ proc.io <- []byte(`{"value":{"name":"*"}}`)
var exited bool
select {
case <-proc.io:
diff --git a/openwhisk/extractor.go b/openwhisk/extractor.go
index e496b07..36a6499 100644
--- a/openwhisk/extractor.go
+++ b/openwhisk/extractor.go
@@ -23,7 +23,6 @@
"fmt"
"io"
"io/ioutil"
- "log"
"os"
"path/filepath"
"strconv"
@@ -102,23 +101,23 @@
// ExtractAction accept a byte array and write it to a file
// it handles zip files extracting the content
-// it stores in a new directory under ./action/XXX whee x is incremented every time
+// it stores in a new directory under ./action/XXX/suffix where x is incremented every time
// it returns the file if a file or the directory if it was a zip file
-func (ap *ActionProxy) ExtractAction(buf *[]byte, main string) (string, error) {
+func (ap *ActionProxy) ExtractAction(buf *[]byte, main string, suffix string) (string, error) {
if buf == nil || len(*buf) == 0 {
return "", fmt.Errorf("no file")
}
ap.currentDir++
- newDir := fmt.Sprintf("%s/%d", ap.baseDir, ap.currentDir)
+ newDir := fmt.Sprintf("%s/%d/%s", ap.baseDir, ap.currentDir, suffix)
os.MkdirAll(newDir, 0755)
kind, err := filetype.Match(*buf)
if err != nil {
return "", err
}
- if kind.Extension == "zip" {
- log.Println("Extract Action, assuming a zip")
- return newDir, unzip(*buf, newDir)
- }
file := newDir + "/" + main
+ if kind.Extension == "zip" {
+ Debug("Extract Action, assuming a zip")
+ return file, unzip(*buf, newDir)
+ }
return file, ioutil.WriteFile(file, *buf, 0755)
}
diff --git a/openwhisk/extractor_test.go b/openwhisk/extractor_test.go
index 9fc3476..6c0b9a1 100644
--- a/openwhisk/extractor_test.go
+++ b/openwhisk/extractor_test.go
@@ -26,44 +26,42 @@
)
func TestExtractActionTest_exec(t *testing.T) {
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/x1", "", log)
+ ap := NewActionProxy("./action/x1", "", os.Stdout, os.Stderr)
// cleanup
assert.Nil(t, os.RemoveAll("./action/x1"))
file, _ := ioutil.ReadFile("_test/exec")
- ap.ExtractAction(&file, "exec")
- assert.Nil(t, exists("./action/x1", "exec"))
+ ap.ExtractAction(&file, "exec", "bin")
+ assert.Nil(t, exists("./action/x1", "bin/exec"))
}
func TestExtractActionTest_exe(t *testing.T) {
- log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/x2", "", log)
+ ap := NewActionProxy("./action/x2", "", os.Stdout, os.Stderr)
// cleanup
assert.Nil(t, os.RemoveAll("./action/x2"))
// match exe
file, _ := ioutil.ReadFile("_test/exec")
- ap.ExtractAction(&file, "exec")
- assert.Equal(t, detect("./action/x2", "exec"), "elf")
+ ap.ExtractAction(&file, "exec", "bin")
+ assert.Equal(t, detect("./action/x2", "bin/exec"), "elf")
}
func TestExtractActionTest_zip(t *testing.T) {
log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/x3", "", log)
+ ap := NewActionProxy("./action/x3", "", log, log)
// cleanup
assert.Nil(t, os.RemoveAll("./action/x3"))
// match exe
file, _ := ioutil.ReadFile("_test/exec.zip")
- ap.ExtractAction(&file, "exec")
- assert.Equal(t, detect("./action/x3", "exec"), "elf")
- assert.Nil(t, exists("./action/x3", "etc"))
- assert.Nil(t, exists("./action/x3", "dir/etc"))
+ ap.ExtractAction(&file, "exec", "bin")
+ assert.Equal(t, detect("./action/x3", "bin/exec"), "elf")
+ assert.Nil(t, exists("./action/x3", "bin/etc"))
+ assert.Nil(t, exists("./action/x3", "bin/dir/etc"))
}
func TestExtractAction_script(t *testing.T) {
log, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy("./action/x4", "", log)
+ ap := NewActionProxy("./action/x4", "", log, log)
buf := []byte("#!/bin/sh\necho ok")
- _, err := ap.ExtractAction(&buf, "exec")
+ _, err := ap.ExtractAction(&buf, "exec", "bin")
//fmt.Print(err)
assert.Nil(t, err)
}
diff --git a/openwhisk/initHandler.go b/openwhisk/initHandler.go
index 41dec5e..9d40f53 100644
--- a/openwhisk/initHandler.go
+++ b/openwhisk/initHandler.go
@@ -24,6 +24,8 @@
"io/ioutil"
"log"
"net/http"
+ "os"
+ "path/filepath"
)
type initBodyRequest struct {
@@ -31,6 +33,7 @@
Binary bool `json:"binary,omitempty"`
Main string `json:"main,omitempty"`
}
+
type initRequest struct {
Value initBodyRequest `json:"value,omitempty"`
}
@@ -48,21 +51,30 @@
func (ap *ActionProxy) initHandler(w http.ResponseWriter, r *http.Request) {
+ if ap.initialized {
+ msg := "Cannot initialize the action more than once."
+ sendError(w, http.StatusForbidden, msg)
+ log.Println(msg)
+ return
+ }
+
// read body of the request
if ap.compiler != "" {
- log.Println("compiler: " + ap.compiler)
+ Debug("compiler: " + ap.compiler)
}
body, err := ioutil.ReadAll(r.Body)
defer r.Body.Close()
if err != nil {
sendError(w, http.StatusBadRequest, fmt.Sprintf("%v", err))
+ return
}
// decode request parameters
- if ap.Trace {
- log.Printf("init: decoding %s\n", string(body))
+ if len(body) < 1000 {
+ Debug("init: decoding %s\n", string(body))
}
+
var request initRequest
err = json.Unmarshal(body, &request)
@@ -72,16 +84,8 @@
}
// request with empty code - stop any executor but return ok
- if ap.Trace {
- log.Printf("request: %v\n", request)
- }
if request.Value.Code == "" {
- if ap.theExecutor != nil {
- log.Printf("stop running action")
- ap.theExecutor.Stop()
- ap.theExecutor = nil
- }
- sendOK(w)
+ sendError(w, http.StatusForbidden, "Missing main/no code to execute.")
return
}
@@ -93,39 +97,77 @@
// extract code eventually decoding it
var buf []byte
if request.Value.Binary {
- log.Printf("binary")
+ Debug("it is binary code")
buf, err = base64.StdEncoding.DecodeString(request.Value.Code)
if err != nil {
sendError(w, http.StatusBadRequest, "cannot decode the request: "+err.Error())
return
}
} else {
- log.Printf("plain text")
+ Debug("it is source code")
buf = []byte(request.Value.Code)
}
- // extract the action,
- file, err := ap.ExtractAction(&buf, main)
- if err != nil || file == "" {
- sendError(w, http.StatusBadRequest, "invalid action: "+err.Error())
+ // if a compiler is defined try to compile
+ _, err = ap.ExtractAndCompile(&buf, main)
+ if err != nil {
+ sendError(w, http.StatusBadGateway, err.Error())
return
}
- // compile it if a compiler is available
- if ap.compiler != "" && !isCompiled(file, main) {
- log.Printf("compiling: %s main: %s", file, main)
- err = ap.CompileAction(main, file, file)
- if err != nil {
- sendError(w, http.StatusBadRequest, "cannot compile action: "+err.Error())
- return
- }
- }
-
- // stop and start
+ // start an action
err = ap.StartLatestAction(main)
if err != nil {
sendError(w, http.StatusBadRequest, "cannot start action: "+err.Error())
return
}
+ ap.initialized = true
sendOK(w)
}
+
+// ExtractAndCompile decode the buffer and if a compiler is defined, compile it also
+func (ap *ActionProxy) ExtractAndCompile(buf *[]byte, main string) (string, error) {
+ // extract in "bin" or in "src" if the runtime can compile
+ suffix := "bin"
+ if ap.compiler != "" {
+ suffix = "src"
+ }
+
+ // extract action
+ file, err := ap.ExtractAction(buf, main, suffix)
+ if err != nil {
+ return "", err
+ }
+ if file == "" {
+ return "", fmt.Errorf("empty filename")
+ }
+ // no compiler, we are done
+ if ap.compiler == "" {
+ return file, nil
+ }
+
+ // some path surgery
+ dir := filepath.Dir(file)
+ parent := filepath.Dir(dir)
+ srcDir := filepath.Join(parent, "src")
+ binDir := filepath.Join(parent, "bin")
+ binFile := filepath.Join(binDir, main)
+ os.Mkdir(binDir, 0755)
+
+ // if the file is already compiled just move it from src to bin
+ if isCompiled(file, main) {
+ os.Rename(file, binFile)
+ return binFile, nil
+ }
+
+ // ok let's try to compile
+ Debug("compiling: %s main: %s", file, main)
+ err = ap.CompileAction(main, srcDir, binDir)
+ if err != nil {
+ return "", err
+ }
+ if !isCompiled(binFile, main) {
+ return "", fmt.Errorf("cannot compile")
+ }
+ return binFile, nil
+}
diff --git a/openwhisk/initHandler_test.go b/openwhisk/initHandler_test.go
index cdb2add..be31b1f 100644
--- a/openwhisk/initHandler_test.go
+++ b/openwhisk/initHandler_test.go
@@ -46,18 +46,20 @@
doRun(ts, "")
doInit(ts, initBinary("_test/hello_message", ""))
doRun(ts, "")
+ stopTestServer(ts, cur, log)
+ ts, cur, log = startTestServer("")
doInit(ts, initBinary("_test/hello_greeting", ""))
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// 200 {"ok":true}
// 200 {"message":"Hello, Mike!"}
- // 200 {"ok":true}
- // 200 {"greetings":"Hello, Mike"}
// name=Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
+ // 200 {"ok":true}
+ // 200 {"greetings":"Hello, Mike"}
// Hello, Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
@@ -68,18 +70,20 @@
doRun(ts, "")
doInit(ts, initBinary("_test/hello_greeting.zip", ""))
doRun(ts, "")
+ stopTestServer(ts, cur, log)
+ ts, cur, log = startTestServer("")
doInit(ts, initBinary("_test/hello_message.zip", ""))
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// 200 {"ok":true}
// 200 {"greetings":"Hello, Mike"}
- // 200 {"ok":true}
- // 200 {"message":"Hello, Mike!"}
// Hello, Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
+ // 200 {"ok":true}
+ // 200 {"message":"Hello, Mike!"}
// name=Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
@@ -95,34 +99,36 @@
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// 200 {"ok":true}
// 200 {"hello": "Mike"}
// 400 {"error":"command exited"}
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// msg=hello Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// Goodbye!
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
-} /**/
+}
func Example_main_nocompiler() {
ts, cur, log := startTestServer("")
doRun(ts, "")
doInit(ts, initBinary("_test/hello_message", "message"))
doRun(ts, "")
+ stopTestServer(ts, cur, log)
+ ts, cur, log = startTestServer("")
doInit(ts, initBinary("_test/hello_greeting", "greeting"))
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// 200 {"ok":true}
// 200 {"message":"Hello, Mike!"}
- // 200 {"ok":true}
- // 200 {"greetings":"Hello, Mike"}
// name=Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
+ // 200 {"ok":true}
+ // 200 {"greetings":"Hello, Mike"}
// Hello, Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
@@ -134,35 +140,38 @@
doInit(ts, initBinary("_test/hello_greeting.zip", "greeting"))
doInit(ts, initBinary("_test/hello_greeting1.zip", "greeting"))
doRun(ts, "")
+ stopTestServer(ts, cur, log)
+
+ ts, cur, log = startTestServer("")
doInit(ts, initBinary("_test/hello_message.zip", "message"))
doInit(ts, initBinary("_test/hello_message1.zip", "message"))
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// 400 {"error":"cannot start action: command exited"}
// 200 {"ok":true}
// 200 {"greetings":"Hello, Mike"}
- // 400 {"error":"cannot start action: command exited"}
- // 200 {"ok":true}
- // 200 {"message":"Hello, Mike!"}
// Hello, Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
+ // 400 {"error":"cannot start action: command exited"}
+ // 200 {"ok":true}
+ // 200 {"message":"Hello, Mike!"}
// name=Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
}
func Example_compile_simple() {
- comp, _ := filepath.Abs("../common/gobuild.sh")
+ comp, _ := filepath.Abs("../common/gobuild.py")
ts, cur, log := startTestServer(comp)
doRun(ts, "")
doInit(ts, initCode("_test/hello.src", ""))
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// 200 {"ok":true}
// 200 {"message":"Hello, Mike!"}
// name=Mike
@@ -171,17 +180,14 @@
}
func Example_compile_withMain() {
- comp, _ := filepath.Abs("../common/gobuild.sh")
+ comp, _ := filepath.Abs("../common/gobuild.py")
ts, cur, log := startTestServer(comp)
- sys("_test/build.sh")
doRun(ts, "")
- doInit(ts, initCode("_test/hello1.src", ""))
doInit(ts, initCode("_test/hello1.src", "hello"))
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
- // 400 {"error":"cannot start action: command exited"}
+ // 500 {"error":"no action defined yet"}
// 200 {"ok":true}
// 200 {"hello":"Hello, Mike!"}
// name=Mike
@@ -190,48 +196,22 @@
}
func Example_compile_withZipSrc() {
- comp, _ := filepath.Abs("../common/gobuild.sh")
+ comp, _ := filepath.Abs("../common/gobuild.py")
ts, cur, log := startTestServer(comp)
doRun(ts, "")
- doInit(ts, initBinary("_test/action.zip", ""))
- doRun(ts, "")
- doInit(ts, initBinary("_test/action.zip", "hello"))
+ doInit(ts, initBinary("_test/hello.zip", ""))
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
// 200 {"ok":true}
// 200 {"greetings":"Hello, Mike"}
- // 200 {"ok":true}
- // 200 {"greetings":"Hello, Mike"}
- // Main:
- // Hello, Mike
- // XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
- // XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
+ // Main
// Hello, Mike
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
// XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
}
-/*
-func Example_compile_withZipSrcDefault() {
- sys("_test/zips.sh")
- comp, _ := filepath.Abs("../common/gobuild.sh")
- ts, cur := startTestServer(comp)
- doRun(ts, "")
- doInit(ts, initBinary("_test/action.zip", ""))
- doRun(ts, "")
- stopTestServer(ts, cur)
- // Output:
- // 400 {"error":"no action defined yet"}
- // 200 {"ok":true}
- // name=Mike
- // 200 {"hello":"Hello, Mike!"}
- // XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
- // XXX_THE_END_OF_A_WHISK_ACTIVATION_XXX
-}
-/**/
-
func Example_badinit_nocompiler() {
ts, cur, log := startTestServer("")
doRun(ts, "")
@@ -243,10 +223,10 @@
doRun(ts, "")
stopTestServer(ts, cur, log)
// Output:
- // 400 {"error":"no action defined yet"}
- // 200 {"ok":true}
+ // 500 {"error":"no action defined yet"}
+ // 403 {"error":"Missing main/no code to execute."}
// 400 {"error":"cannot start action: command exited"}
// 400 {"error":"cannot start action: command exited"}
// 400 {"error":"cannot start action: command exited"}
- // 400 {"error":"no action defined yet"}
+ // 500 {"error":"no action defined yet"}
}
diff --git a/openwhisk/runHandler.go b/openwhisk/runHandler.go
index 14cf614..1495189 100644
--- a/openwhisk/runHandler.go
+++ b/openwhisk/runHandler.go
@@ -18,19 +18,13 @@
package openwhisk
import (
+ "bytes"
"encoding/json"
"fmt"
"io/ioutil"
- "log"
"net/http"
- "strings"
)
-// Params are the parameteres sent to the action
-type Params struct {
- Value json.RawMessage `json:"value"`
-}
-
// ErrResponse is the response when there are errors
type ErrResponse struct {
Error string `json:"error"`
@@ -52,31 +46,27 @@
func (ap *ActionProxy) runHandler(w http.ResponseWriter, r *http.Request) {
// parse the request
- params := Params{}
body, err := ioutil.ReadAll(r.Body)
defer r.Body.Close()
if err != nil {
sendError(w, http.StatusBadRequest, fmt.Sprintf("Error reading request body: %v", err))
return
}
-
- // decode request parameters
- err = json.Unmarshal(body, ¶ms)
- if err != nil {
- sendError(w, http.StatusBadRequest, fmt.Sprintf("Error unmarshaling request: %v", err))
- return
- }
+ Debug("done reading %d bytes", len(body))
// check if you have an action
if ap.theExecutor == nil {
- sendError(w, http.StatusBadRequest, fmt.Sprintf("no action defined yet"))
+ sendError(w, http.StatusInternalServerError, fmt.Sprintf("no action defined yet"))
return
}
+ // remove newlines
+ body = bytes.Replace(body, []byte("\n"), []byte(""), -1)
+
// execute the action
// and check for early termination
- ap.theExecutor.io <- string(params.Value)
- var response string
+ ap.theExecutor.io <- body
+ var response []byte
var exited bool
select {
case response = <-ap.theExecutor.io:
@@ -87,28 +77,27 @@
// check for early termination
if exited {
+ Debug("WARNING! Command exited")
ap.theExecutor = nil
sendError(w, http.StatusBadRequest, fmt.Sprintf("command exited"))
return
}
+ DebugLimit("received:", response, 120)
// flush the logs sending the activation message at the end
ap.theExecutor.log <- true
- // check response
- if response == "" {
- sendError(w, http.StatusBadRequest, fmt.Sprintf("%v", err))
+ // check if the answer is an object map
+ var objmap map[string]*json.RawMessage
+ err = json.Unmarshal(response, &objmap)
+ if err != nil {
+ sendError(w, http.StatusBadGateway, "The action did not return a dictionary.")
return
}
- // return the response
- if !strings.HasSuffix(response, "\n") {
- response = response + "\n"
- }
- log.Print(response)
w.Header().Set("Content-Type", "application/json")
w.Header().Set("Content-Length", fmt.Sprintf("%d", len(response)))
- numBytesWritten, err := w.Write([]byte(response))
+ numBytesWritten, err := w.Write(response)
// flush output
if f, ok := w.(http.Flusher); ok {
diff --git a/openwhisk/util_test.go b/openwhisk/util_test.go
index 6aeeeec..9ebcbc0 100644
--- a/openwhisk/util_test.go
+++ b/openwhisk/util_test.go
@@ -29,7 +29,9 @@
"os"
"os/exec"
"path/filepath"
+ "regexp"
"runtime"
+ "strings"
"testing"
"time"
@@ -46,7 +48,7 @@
log.Printf(dir)
// setup the server
buf, _ := ioutil.TempFile("", "log")
- ap := NewActionProxy(dir, compiler, buf)
+ ap := NewActionProxy(dir, compiler, buf, buf)
ts := httptest.NewServer(ap)
log.Printf(ts.URL)
doPost(ts.URL+"/init", `{value: {code: ""}}`)
@@ -86,6 +88,9 @@
} else {
fmt.Printf("%d %s", status, resp)
}
+ if !strings.HasSuffix(resp, "\n") {
+ fmt.Println()
+ }
}
func doInit(ts *httptest.Server, message string) {
@@ -164,16 +169,28 @@
}
func waitabit() {
- time.Sleep(1000 * time.Millisecond)
+ time.Sleep(2000 * time.Millisecond)
}
+func removeLineNr(out string) string {
+ var re = regexp.MustCompile(`:\d+:\d+`)
+ return re.ReplaceAllString(out, "::")
+}
func TestMain(m *testing.M) {
- sys("_test/build.sh")
- sys("_test/zips.sh")
+ //Debugging = true // enable debug
+ // silence those annoying logs
+ if !Debugging {
+ log.SetOutput(ioutil.Discard)
+ }
+
// increase timeouts for init
DefaultTimeoutInit = 1000 * time.Millisecond
// timeout for drain - shoud less (or you can get stuck on stdout without getting the stderr)
DefaultTimeoutDrain = 100 * time.Millisecond
+ // build some test stuff
+ sys("_test/build.sh")
+ sys("_test/zips.sh")
+ // go ahead
code := m.Run()
os.Exit(code)
}
diff --git a/settings.gradle b/settings.gradle
index 1892607..bdd15bb 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -18,7 +18,7 @@
include 'tests'
include 'actionProxyLoop'
-include 'golang1.10'
+include 'golang1.11'
rootProject.name = 'runtime-golang'
diff --git a/tests/src/test/scala/runtime/actionContainers/ActionLoopBasicGoTests.scala b/tests/src/test/scala/runtime/actionContainers/ActionLoopBasicGoTests.scala
new file mode 100644
index 0000000..5534474
--- /dev/null
+++ b/tests/src/test/scala/runtime/actionContainers/ActionLoopBasicGoTests.scala
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package runtime.actionContainers
+
+import actionContainers.ActionContainer.withContainer
+import actionContainers.{ActionContainer, BasicActionRunnerTests}
+import common.WskActorSystem
+import org.junit.runner.RunWith
+import org.scalatest.junit.JUnitRunner
+
+@RunWith(classOf[JUnitRunner])
+class ActionLoopBasicGoTests extends BasicActionRunnerTests with WskActorSystem {
+
+ val goCompiler = "actionloop-golang-v1.11"
+ val image = goCompiler
+
+ override def withActionContainer(env: Map[String, String] = Map.empty)(code: ActionContainer => Unit) = {
+ withContainer(image, env)(code)
+ }
+
+ def withActionLoopContainer(code: ActionContainer => Unit) = withContainer(image)(code)
+
+ behavior of image
+
+ override val testNoSourceOrExec = TestConfig("")
+
+ override val testNotReturningJson = TestConfig(
+ """
+ |package main
+ |import (
+ | "bufio"
+ | "fmt"
+ | "os"
+ |)
+ |func main() {
+ | reader := bufio.NewReader(os.Stdin)
+ | out := os.NewFile(3, "pipe")
+ | defer out.Close()
+ | reader.ReadBytes('\n')
+ | fmt.Fprintln(out, "\"a string but not a map\"")
+ | reader.ReadBytes('\n')
+ |}
+ """.stripMargin)
+
+ override val testEcho = TestConfig(
+ """|package main
+ |import "fmt"
+ |import "log"
+ |func Main(args map[string]interface{}) map[string]interface{} {
+ | fmt.Println("hello stdout")
+ | log.Println("hello stderr")
+ | return args
+ |}
+ """.stripMargin)
+
+ override val testUnicode = TestConfig(
+ """|package main
+ |import "fmt"
+ |func Main(args map[string]interface{}) map[string]interface{} {
+ | delimiter := args["delimiter"].(string)
+ | str := delimiter + " ☃ " + delimiter
+ | fmt.Println(str)
+ | res := make(map[string]interface{})
+ | res["winter"] = str
+ | return res
+ |}
+ """.stripMargin)
+
+
+ override val testEnv = TestConfig(
+ """
+ |package main
+ |import "os"
+ |func Main(args map[string]interface{}) map[string]interface{} {
+ | res := make(map[string]interface{})
+ | res["api_host"] = os.Getenv("__OW_API_HOST")
+ | res["api_key"] = os.Getenv("__OW_API_KEY")
+ | res["namespace"] = os.Getenv("__OW_NAMESPACE")
+ | res["action_name"] = os.Getenv("__OW_ACTION_NAME")
+ | res["activation_id"] = os.Getenv("__OW_ACTIVATION_ID")
+ | res["deadline"] = os.Getenv("__OW_DEADLINE")
+ | return res
+ |}
+ """.stripMargin)
+
+ override val testInitCannotBeCalledMoreThanOnce = TestConfig(
+ """|package main
+ |func Main(args map[string]interface{}) map[string]interface{} {
+ | return args
+ |}
+ """.stripMargin)
+
+ override val testEntryPointOtherThanMain = TestConfig(
+ """|package main
+ |func niam(args map[string]interface{}) map[string]interface{} {
+ | return args
+ |}
+ """.stripMargin, main="niam")
+
+ override val testLargeInput = TestConfig(
+ """|package main
+ |func Main(args map[string]interface{}) map[string]interface{} {
+ | return args
+ |}
+ """.stripMargin)
+}
diff --git a/tests/src/test/scala/runtime/actionContainers/ActionLoopBasicTests.scala b/tests/src/test/scala/runtime/actionContainers/ActionLoopBasicTests.scala
new file mode 100644
index 0000000..32d43c3
--- /dev/null
+++ b/tests/src/test/scala/runtime/actionContainers/ActionLoopBasicTests.scala
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package runtime.actionContainers
+
+import actionContainers.ActionContainer.withContainer
+import actionContainers.{ActionContainer, BasicActionRunnerTests}
+import common.WskActorSystem
+import org.junit.runner.RunWith
+import org.scalatest.junit.JUnitRunner
+
+@RunWith(classOf[JUnitRunner])
+class ActionLoopBasicTests extends BasicActionRunnerTests with WskActorSystem {
+
+ val image = "actionloop"
+
+ override def withActionContainer(env: Map[String, String] = Map.empty)(code: ActionContainer => Unit) = {
+ withContainer(image, env)(code)
+ }
+
+ def withActionLoopContainer(code: ActionContainer => Unit) = withContainer(image)(code)
+
+ behavior of image
+
+ override val testNoSourceOrExec = TestConfig("")
+
+ override val testNotReturningJson = TestConfig(
+ """#!/bin/bash
+ |read line
+ |echo '"not json"' >&3
+ |read line
+ |""".stripMargin)
+
+ override val testEcho = TestConfig(
+ """|#!/bin/bash
+ |while read line
+ |do
+ | echo "hello stdout"
+ | echo "hello stderr" >&2
+ | echo "$line" | jq -c .value >&3
+ |done
+ """.stripMargin)
+
+ override val testUnicode = TestConfig(
+ """|#!/bin/bash
+ |while read line
+ |do
+ | delimiter="$(echo "$line" | jq -r ".value.delimiter")"
+ | msg="$delimiter ☃ $delimiter"
+ | echo "$msg"
+ | echo "{\"winter\": \"$msg\"}" >&3
+ |done
+ """.stripMargin)
+
+ override val testEnv = TestConfig(
+ """#!/bin/bash
+ |while read line
+ |do
+ | __OW_API_HOST="$(echo "$line" | jq -r .api_host)"
+ | __OW_API_KEY="$(echo "$line" | jq -r .api_key)"
+ | __OW_NAMESPACE="$(echo "$line" | jq -r .namespace)"
+ | __OW_ACTIVATION_ID="$(echo "$line" | jq -r .activation_id)"
+ | __OW_ACTION_NAME="$(echo "$line" | jq -r .action_name)"
+ | __OW_DEADLINE="$(echo "$line" | jq -r .deadline)"
+ | echo >&3 "{ \
+ | \"api_host\": \"$__OW_API_HOST\", \
+ | \"api_key\": \"$__OW_API_KEY\", \
+ | \"namespace\": \"$__OW_NAMESPACE\", \
+ | \"activation_id\": \"$__OW_ACTIVATION_ID\", \
+ | \"action_name\": \"$__OW_ACTION_NAME\", \
+ | \"deadline\": \"$__OW_DEADLINE\" }"
+ | done
+ """.stripMargin)
+
+ val echoSh =
+ """|#!/bin/bash
+ |while read line
+ |do echo "$line" | jq -c .value >&3
+ |done
+ """.stripMargin
+
+ override val testInitCannotBeCalledMoreThanOnce = TestConfig(echoSh)
+
+ override val testEntryPointOtherThanMain = TestConfig(echoSh, main = "niam")
+
+ override val testLargeInput = TestConfig(echoSh)
+}
diff --git a/tests/src/test/scala/runtime/actionContainers/ActionLoopContainerTests.scala b/tests/src/test/scala/runtime/actionContainers/ActionLoopContainerTests.scala
index ef11078..9c2ebfb 100644
--- a/tests/src/test/scala/runtime/actionContainers/ActionLoopContainerTests.scala
+++ b/tests/src/test/scala/runtime/actionContainers/ActionLoopContainerTests.scala
@@ -17,23 +17,21 @@
package runtime.actionContainers
-//import java.util.concurrent.TimeoutException
import actionContainers.{ActionContainer, ActionProxyContainerTestUtils}
import actionContainers.ActionContainer.withContainer
import common.WskActorSystem
import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner
-//import spray.json.JsNumber
import spray.json.{JsObject, JsString}
-//import spray.json.JsBoolean
@RunWith(classOf[JUnitRunner])
class ActionLoopContainerTests extends ActionProxyContainerTestUtils with WskActorSystem {
import GoResourceHelpers._
- // "example" is the image build by /sdk/docker
+ val image = "actionloop"
+
def withActionLoopContainer(code: ActionContainer => Unit) = withContainer("actionloop")(code)
behavior of "actionloop"
@@ -43,7 +41,7 @@
s"""#!/bin/bash
|while read line
|do
- | name="$$(echo $$line | jq -r .name)"
+ | name="$$(echo $$line | jq -r .value.name)"
| if test "$$name" == ""
| then exit
| fi
@@ -63,8 +61,8 @@
it should "run sample with init that does nothing" in {
val (out, err) = withActionLoopContainer { c =>
- c.init(JsObject())._1 should be(200)
- c.run(JsObject())._1 should be(400)
+ c.init(JsObject())._1 should be(403)
+ c.run(JsObject())._1 should be(500)
}
}
diff --git a/tests/src/test/scala/runtime/actionContainers/ActionLoopGoContainerTests.scala b/tests/src/test/scala/runtime/actionContainers/ActionLoopGoContainerTests.scala
index e4c7320..b59e306 100644
--- a/tests/src/test/scala/runtime/actionContainers/ActionLoopGoContainerTests.scala
+++ b/tests/src/test/scala/runtime/actionContainers/ActionLoopGoContainerTests.scala
@@ -33,43 +33,34 @@
import GoResourceHelpers._
- val goCompiler = "actionloop-golang-v1.10"
+ val goCompiler = "actionloop-golang-v1.11"
val image = goCompiler
def withActionLoopContainer(code: ActionContainer => Unit) = withContainer(image)(code)
behavior of image
-
- private def checkresponse(res: Option[JsObject], args: JsObject = JsObject()) = {
- res shouldBe defined
- res.get.fields("error") shouldBe JsString("no action defined yet")
- //res.get.fields("args") shouldBe args
+ def helloGo(main: String, pkg: String = "main") = {
+ val func = if (main == "main") "Main" else main
+ s"""|package ${pkg}
+ |
+ |import "fmt"
+ |
+ |func ${func}(obj map[string]interface{}) map[string]interface{} {
+ | name, ok := obj["name"].(string)
+ | if !ok {
+ | name = "Stranger"
+ | }
+ | fmt.Printf("name=%s\\n", name)
+ | msg := make(map[string]interface{})
+ | msg["${pkg}-${main}"] = "Hello, " + name + "!"
+ | return msg
+ |}
+ |""".stripMargin
}
- private def goCodeHello(file: String, main: String) = Seq(
- Seq(s"${file}.go") ->
- s"""
- |package action
- |
- |import (
- | "encoding/json"
- | "fmt"
- |)
- |
- |func ${main}(event json.RawMessage) (json.RawMessage, error) {
- | var obj map[string]interface{}
- | json.Unmarshal(event, &obj)
- | name, ok := obj["name"].(string)
- | if !ok {
- | name = "Stranger"
- | }
- | fmt.Printf("name=%s\\n", name)
- | msg := map[string]string{"${file}-${main}": ("Hello, " + name + "!")}
- | return json.Marshal(msg)
- |}
- |
- """.stripMargin
+ private def helloSrc(main: String) = Seq(
+ Seq(s"${main}") -> helloGo(main)
)
private def helloMsg(name: String = "Demo") =
@@ -81,168 +72,109 @@
it should "run sample with init that does nothing" in {
val (out, err) = withActionLoopContainer { c =>
- c.init(JsObject())._1 should be(200)
- c.run(JsObject())._1 should be(400)
+ c.init(JsObject())._1 should be(403)
+ c.run(JsObject())._1 should be(500)
}
}
it should "accept a binary main" in {
val exe = ExeBuilder.mkBase64Exe(
- goCompiler, goCodeHello("main", "Main"), "main")
+ goCompiler, helloSrc("main"), "main")
withActionLoopContainer {
c =>
c.init(initPayload(exe))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("main-Main", "Hello, Demo!"))
+ c.run(helloMsg()) should be(okMsg("main-main", "Hello, Demo!"))
}
}
+
//def pr(x: Any) = { println(x) ; x}
- it should "build and run a go main zipped exe" in {
+ it should "accept a zipped main binary" in {
val zip = ExeBuilder.mkBase64Zip(
- goCompiler, goCodeHello("main", "Main"), "main")
+ goCompiler, helloSrc("main"), "main")
withActionLoopContainer {
c =>
c.init(initPayload(zip))._1 should be(200)
- c.run(helloMsg()) should be(okMsg("main-Main", "Hello, Demo!"))
+ c.run(helloMsg()) should be(okMsg("main-main", "Hello, Demo!"))
}
}
- it should "buid and run a go hello exe " in {
+ it should "accept a binary not-main" in {
val exe = ExeBuilder.mkBase64Exe(
- goCompiler, goCodeHello("hello", "Hello"), "hello")
+ goCompiler, helloSrc("hello"), "hello")
withActionLoopContainer {
c =>
c.init(initPayload(exe, "hello"))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("hello-Hello", "Hello, Demo!"))
+ c.run(helloMsg()) should be(okMsg("main-hello", "Hello, Demo!"))
}
}
- it should "build and run a go hello zipped exe" in {
+ it should "accept a zipped binary not-main" in {
val zip = ExeBuilder.mkBase64Zip(
- goCompiler, goCodeHello("hello", "Hello"), "hello")
+ goCompiler, helloSrc("hello"), "hello")
withActionLoopContainer {
c =>
c.init(initPayload(zip, "hello"))._1 shouldBe (200)
+ c.run(helloMsg()) should be(okMsg("main-hello", "Hello, Demo!"))
+ }
+ }
+
+ it should "accept a src main action " in {
+ var src = ExeBuilder.mkBase64Src(helloSrc("main"), "main")
+ withActionLoopContainer {
+ c =>
+ c.init(initPayload(src))._1 shouldBe (200)
+ c.run(helloMsg()) should be(okMsg("main-main", "Hello, Demo!"))
+ }
+ }
+
+ it should "accept a src not-main action " in {
+ var src = ExeBuilder.mkBase64Src(helloSrc("hello"), "hello")
+ withActionLoopContainer {
+ c =>
+ c.init(initPayload(src, "hello"))._1 shouldBe (200)
+ c.run(helloMsg()) should be(okMsg("main-hello", "Hello, Demo!"))
+ }
+ }
+
+
+ it should "accept a zipped src main action" in {
+ var src = ExeBuilder.mkBase64SrcZip(helloSrc("main"), "main")
+ withActionLoopContainer {
+ c =>
+ c.init(initPayload(src))._1 shouldBe (200)
+ c.run(helloMsg()) should be(okMsg("main-main", "Hello, Demo!"))
+ }
+ }
+
+ it should "accept a zipped src not-main action" in {
+ var src = ExeBuilder.mkBase64SrcZip(helloSrc("hello"), "hello")
+ withActionLoopContainer {
+ c =>
+ c.init(initPayload(src, "hello"))._1 shouldBe (200)
+ c.run(helloMsg()) should be(okMsg("main-hello", "Hello, Demo!"))
+ }
+ }
+
+ it should "deploy a zip main src with subdir" in {
+ var src = ExeBuilder.mkBase64SrcZip(Seq(
+ Seq("hello", "hello.go") -> helloGo("Hello", "hello"),
+ Seq("main") ->
+ """
+ |package main
+ |import "hello"
+ |func Main(args map[string]interface{})map[string]interface{} {
+ | return hello.Hello(args)
+ |}
+ """.stripMargin
+ ), "main")
+ withActionLoopContainer {
+ c =>
+ c.init(initPayload(src))._1 shouldBe (200)
c.run(helloMsg()) should be(okMsg("hello-Hello", "Hello, Demo!"))
}
}
-
- val helloSrc =
- """
- |package action
- |
- |import (
- | "encoding/json"
- | "fmt"
- |)
- |
- |func Hello(event json.RawMessage) (json.RawMessage, error) {
- | var obj struct {
- | Name string `json:",omitempty"`
- | }
- | err := json.Unmarshal(event, &obj)
- | if err != nil {
- | return nil, err
- | }
- | name := obj.Name
- | if name == "" {
- | name = "Stranger"
- | }
- | fmt.Printf("name=%s\n", name)
- | msg := map[string]string{"Hello": ("Hello, " + name + "!")}
- | return json.Marshal(msg)
- |}
- """.stripMargin
-
- val mainSrc =
- """
- |package action
- |
- |import (
- | "encoding/json"
- | "fmt"
- |)
- |
- |func Main(event json.RawMessage) (json.RawMessage, error) {
- | var obj map[string]interface{}
- | json.Unmarshal(event, &obj)
- | name, ok := obj["name"].(string)
- | if !ok {
- | name = "Stranger"
- | }
- | fmt.Printf("name=%s\n", name)
- | msg := map[string]string{"Main": ("Hello, " + name + "!")}
- | return json.Marshal(msg)
- |}
- """.stripMargin
-
- it should "deploy a src main action " in {
- var src = ExeBuilder.mkBase64Src(Seq(
- Seq("main") -> mainSrc
- ), "main")
- withActionLoopContainer {
- c =>
- c.init(initPayload(src))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("Main", "Hello, Demo!"))
- }
- }
-
- it should "deploy a src hello action " in {
- var src = ExeBuilder.mkBase64Src(Seq(
- Seq("hello") -> helloSrc
- ), "hello")
- withActionLoopContainer {
- c =>
- c.init(initPayload(src, "hello"))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("Hello", "Hello, Demo!"))
- }
- }
-
- it should "deploy a zip main src action" in {
- var src = ExeBuilder.mkBase64SrcZip(Seq(
- Seq("main.go") -> mainSrc
- ), "main")
- withActionLoopContainer {
- c =>
- c.init(initPayload(src))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("Main", "Hello, Demo!"))
- }
- }
-
- it should "deploy a zip main src subdir action" in {
- var src = ExeBuilder.mkBase64SrcZip(Seq(
- Seq("action", "main.go") -> mainSrc
- ), "main")
- withActionLoopContainer {
- c =>
- c.init(initPayload(src))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("Main", "Hello, Demo!"))
- }
- }
-
- it should "deploy a zip src hello action " in {
- var src = ExeBuilder.mkBase64SrcZip(Seq(
- Seq("hello.go") -> helloSrc
- ), "hello")
- withActionLoopContainer {
- c =>
- c.init(initPayload(src, "hello"))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("Hello", "Hello, Demo!"))
- }
- }
-
-
- it should "deploy a zip src hello action in subdir" in {
- var src = ExeBuilder.mkBase64SrcZip(Seq(
- Seq("action", "hello.go") -> helloSrc
- ), "hello")
- withActionLoopContainer {
- c =>
- c.init(initPayload(src, "hello"))._1 shouldBe (200)
- c.run(helloMsg()) should be(okMsg("Hello", "Hello, Demo!"))
- }
- }
}
diff --git a/tests/src/test/scala/runtime/actionContainers/GoResourceHelpers.scala b/tests/src/test/scala/runtime/actionContainers/GoResourceHelpers.scala
index c001f22..1370fe5 100644
--- a/tests/src/test/scala/runtime/actionContainers/GoResourceHelpers.scala
+++ b/tests/src/test/scala/runtime/actionContainers/GoResourceHelpers.scala
@@ -26,10 +26,11 @@
import collection.JavaConverters._
import actionContainers.ResourceHelpers
+import scala.util.Random
+
object GoResourceHelpers {
-
-
/** Create a temporary directory in your /tmp directory.
+ * /tmp/openwhisk/random-lowercase-string/prefix+suffix
*
* This is needed to use docker volume mounts.
* On mac I need to use the /tmp directory,
@@ -38,7 +39,9 @@
*
*/
def tmpDirectoryFile(prefix: String, suffix: String = "") =
- new File(new File("/tmp", "openwhisk"), prefix+System.currentTimeMillis().toString+suffix)
+ new File(new File(new File("/tmp", "openwhisk"),
+ Random.alphanumeric.take(10).toArray.mkString.toLowerCase /*random filename alphanumeric and lower case*/)
+ , prefix++suffix)
def createTmpDirectory(prefix: String, suffix: String = "") = {
val tmpDir = tmpDirectoryFile(prefix,suffix)
@@ -147,31 +150,29 @@
// The absolute paths of the source file
val (srcDir, srcAbsPaths) = writeSourcesToHomeTmpDirectory(sources)
- val src = srcDir.toFile.getAbsolutePath
+ val src = srcAbsPaths.head.toFile
// A temporary directory for the destination files.
// DO NOT CREATE IT IN ADVANCE or you will get a permission denied
- val outDir = tmpDirectoryFile("out")
- val out = outDir.toPath.toAbsolutePath
+ val binDir = tmpDirectoryFile("bin")
+ binDir.mkdirs()
+ val bin = new File(binDir, main)
// command to compile
- val exe = new File(outDir, main)
-
- val cmd = s"${dockerBin} run " +
- s"-v ${src}:/src " +
- s"-v ${out}:/out:rw " +
- s"${image} compile ${main}"
+ val cmd = s"${dockerBin} run -i ${image} -compile ${main}"
// compiling
+ //println(s"${cmd}\n<${src}\n>${bin}")
+
import sys.process._
- cmd.!
+ (src #> cmd #> bin).!
// result
- exe -> outDir.toPath
+ bin -> binDir.toPath
}
- def mkBase64Exe(image: String, sources: Seq[(Seq[String], String)], main: String) = {
+ def mkBase64Exe(image: String, sources: Seq[(Seq[String] /*lines*/, String /*name*/)], main: String) = {
val (exe, dir) = compile(image, sources, main)
//println(s"exe=${exe.getAbsolutePath}")
ResourceHelpers.readAsBase64(exe.toPath)
@@ -193,8 +194,8 @@
def mkBase64SrcZip(sources: Seq[(Seq[String], String)], main: String) = {
val (srcDir, srcAbsPaths) = writeSourcesToHomeTmpDirectory(sources)
- //println(srcDir)
val archive = makeZipFromDir(srcDir)
+ //println(s"zip=${archive.toFile.getAbsolutePath}")
ResourceHelpers.readAsBase64(archive)
}
}
diff --git a/tools/travis/setup.sh b/tools/travis/setup.sh
index fa7ffeb..7a17bff 100755
--- a/tools/travis/setup.sh
+++ b/tools/travis/setup.sh
@@ -19,8 +19,10 @@
set -e
# add realpath
-sudo apt-get -y update
-sudo apt-get -y install realpath
+if ! which realpath
+then sudo apt-get -y update
+ sudo apt-get -y install realpath
+fi
# Build script for Travis-CI.
