add tutorial, website instructions, use latest instead of absolute versions
diff --git a/src/site/markdown/tutorial/build.md b/src/site/markdown/tutorial/build.md
index 1608b28..a51f2c2 100644
--- a/src/site/markdown/tutorial/build.md
+++ b/src/site/markdown/tutorial/build.md
@@ -56,24 +56,24 @@
Run from your command line:
- `git clone https://github.com/apache/incubator-streams`
- `git clone https://github.com/apache/incubator-streams-examples`
+> git clone https://github.com/apache/incubator-streams
+> git clone https://github.com/apache/incubator-streams-examples
### Build Projects
Run from your command line:
- `export MAVEN_OPTS="-Xmx2G"`
- `cd incubator-streams`
- `mvn clean install -Dmaven.test.skip.exec=true`
+> export MAVEN_OPTS="-Xmx2G"
+> cd incubator-streams
+> mvn clean install -Dmaven.test.skip.exec=true
| Possible result | Explanation |
|-----------------|-------------|
| BUILD SUCCESSFUL | You are all good |
| BUILD FAILED | Check yourself |
- `cd ../incubator-streams-examples`
- `mvn clean package`
+> cd ../incubator-streams-examples
+> mvn clean package
| Possible result | Explanation |
|-----------------|-------------|
diff --git a/src/site/markdown/tutorial/database.md b/src/site/markdown/tutorial/database.md
index b546826..5564ee8 100644
--- a/src/site/markdown/tutorial/database.md
+++ b/src/site/markdown/tutorial/database.md
@@ -8,7 +8,7 @@
Run from your command line:
- `docker ps`
+> $ docker ps
If you see a (possibly empty) list of running containers, you are good.
@@ -16,12 +16,12 @@
Elasticsearch is a great database for storing content from your streams.
- `docker run -d --name elasticsearch elasticsearch`
+> $ docker run -d --name elasticsearch elasticsearch
### Add Elasticsearch container details to your configuration
- `cd $STREAMS`
- `export DOCKERHOST=$(docker-machine ip)`
+> cd $STREAMS
+> export DOCKERHOST=$(docker-machine ip)
Put the following into elasticsearch.conf
diff --git a/src/site/markdown/tutorial/docker.md b/src/site/markdown/tutorial/docker.md
index fa5ffec..2f71e4d 100644
--- a/src/site/markdown/tutorial/docker.md
+++ b/src/site/markdown/tutorial/docker.md
@@ -12,7 +12,7 @@
Run from your command line:
- `docker ps`
+> $ docker ps
If you see a (possibly empty) list of running containers, you are good.
diff --git a/src/site/markdown/tutorial/environment.md b/src/site/markdown/tutorial/environment.md
index b86dd7e..64bb050 100644
--- a/src/site/markdown/tutorial/environment.md
+++ b/src/site/markdown/tutorial/environment.md
@@ -4,13 +4,13 @@
Create a new empty folder there and export the path to a variables.
- `mkdir ~/streams`
- `export STREAMS=$(cd ~streams; pwd)`
+> $ mkdir ~/streams
+> $ export STREAMS=$(cd ~streams; pwd)
### Create a configuration file to hold secrets
- `cd $STREAMS`
- `touch reference.conf`
+> $ cd $STREAMS
+> $ touch reference.conf
To get started, put the following into reference.conf
diff --git a/src/site/markdown/website.md b/src/site/markdown/website.md
new file mode 100644
index 0000000..2e2c8f2
--- /dev/null
+++ b/src/site/markdown/website.md
@@ -0,0 +1,159 @@
+### Website Information
+
+http://streams.incubator.apache.org is a set of interconnected websites generated
+by the maven site plugin. The front page, this page, and most pages linked in the
+top nav of the site are part of incubator-streams-master.
+
+#### Website Content
+
+Pages, diagrams, and other hard-linkable resources are stored throughout the source tree.
+
+#### Markdown
+
+* src/site/markdown
+
+Most HTML pages are generated from markdown. The maven-site-plugin does this during the site build process.
+
+#### Schemas
+
+* src/main/jsonschema
+* src/main/xmlschema
+
+JSON and XML Schemas through-out the project are copied to the web page of their respective modules so they can be
+linked to in other projects.
+
+This allows users to extend the canonical streams schemas incrementally and/or re-use specific fields as they see fit.
+
+#### Resources
+
+* src/main/resources
+* src/test/resources
+
+Other resources including .conf and .properties files are copied to the web page of their respective modules so
+they can be linked to across projects and in external projects.
+
+This allows users to import HOCON from modules outside their sphere of control and adapt to changes upstream.
+
+### Website Changes
+
+The project website(s) are hosted by the Apache foundation and updated via SVN.
+
+Currently pushing website changes is a manual process performed by whomever is making the change.
+
+This typically requires checking out the current website from SVN.
+
+ `svn co https://svn.apache.org/repos/infra/websites/production/streams/content`
+ `cd content`
+
+#### Preparing to publishing a new website version
+
+The instructions below presume:
+
+* you have a shell open in the SVN content directory
+* you know the artifactId and version of the repository you want to publish.
+
+If this is a brand new snapshot or release version, you first need to create a directory corresponding to the new version.
+
+> content$ mkdir site/${project.version}
+> content$ svn add site/${project.version}
+> content$ svn commit -m "svn add site/${project.version}"
+
+The first time a specific site is being published for this version, you must create the directory where it will be published.
+
+> content$ mkdir site/${project.version}/${project.artifactId}
+> content$ svn add site/${project.version}/${project.artifactId}
+> content$ svn commit -m "svn add site/${project.version}/${project.artifactId}"
+
+The folder must exist and be empty for the publish steps to succeed.
+
+Repositories should always be built and published in the following order:
+
+* streams-master
+* streams-project
+* streams-examples
+
+#### Generating and publishing a new website version
+
+The instructions below presume:
+
+* you have a shell open in the root of a project repository
+* you know the artifactId and version of the repository you want to publish.
+
+First, ensure that you have local credentials capable of publishing the site.
+
+ <server>
+ <id>site.streams.{master|project|examples}</id>
+ <username>{your apache ID}</username>
+ <privateKey>{absolute path to your private key</privateKey>
+ <passphrase>{your private key passphrase}</passphrase>
+ <filePermissions>664</filePermissions>
+ <directoryPermissions>775</directoryPermissions>
+ <configuration></configuration>
+ </server>
+
+Next, generate SVG resources for all DOT diagrams in the source tree
+
+> $ for dot in $(find . -name *.dot); do dot -Tsvg $dot -o $dot.svg; done
+
+Then, generate the site that will be published
+
+> $ mvn clean site:site site:stage
+
+At this point you can open target/staging/index.html and do a sanity check on the site you intend to publish.
+
+Finally, publish the site.
+
+> $ mvn scm-publish:publish-scm -Dscmpublish.pubScmUrl=scm:svn:https://svn.apache.org/repos/infra/websites/production/streams/content/site/${project.version}/${project.artifactId}
+
+Note the revision number checked in at the bottom of the maven logs.
+
+You should now be able to access the published site(s) via an absolute URL.
+
+ http://streams.incubator.apache.org/site/${project.version}/${project.artifactId}
+
+For example, website documentation from a recent release:
+
+* http://streams.incubator.apache.org/site/0.2-incubating/streams-project/index.html
+
+Some recent snapshots:
+
+* http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-master/index.html
+* http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/index.html
+* http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/index.html
+
+#### Promoting a new website version
+
+New release or snapshots are immediately published, but visitors to the website won't arrive there from standard links and navigation
+until it has been fully promoted.
+
+The instructions below presume:
+
+* you have a shell open in the SVN content directory
+* you know the artifactId and version of the repository you want to publish.
+
+The convention in place exposes the latest specific site version(s) using redirects maintained in the .htaccess file of project website SVN.
+
+This file can be edited from https://cms.apache.org/streams/
+
+First, click 'Get streams Working Copy'
+
+Next, open .htaccess
+
+If you are promoting sites from all streams repositories simultaneously, the file should end with:
+
+ Redirect /site/latest/ /site/${project.version}
+
+If you want to publish sites at different versions across streams repositories, configure as follows:
+
+ Redirect /site/latest/streams-master /site/0.3-incubating-SNAPSHOT/streams-master
+ Redirect /site/latest/streams-project /site/0.2-incubating/streams-project
+ Redirect /site/latest/streams-examples /site/0.3-incubating-SNAPSHOT/streams-examples
+
+Commit your changes.
+
+Wait a few seconds and click Follow Staging Build.
+
+You should see a new build with a 'Build Successful' message.
+
+Open a new tab and visit http://streams.staging.apache.org for one last check before go-live.
+
diff --git a/src/site/site.xml b/src/site/site.xml
index 4087ae7..fbe2ff1 100644
--- a/src/site/site.xml
+++ b/src/site/site.xml
@@ -71,23 +71,23 @@
</menu>
<menu name="Projects">
<item name="streams-master" />
- <item name="streams-project" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project" />
- <item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/" />
+ <item name="streams-project" href="http://streams.incubator.apache.org/site/latest/streams-project" />
+ <item name="streams-examples" href="http://streams.incubator.apache.org/site/latest/streams-examples/" />
</menu>
<menu name="Getting Started">
- <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-pojo/index.html" />
- <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/streams-provider-twitter/index.html" />
- <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
- <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-components/index.html" />
- <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
- <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
+ <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-pojo/index.html" />
+ <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/streams-provider-twitter/index.html" />
+ <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/index.html" />
+ <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-components/index.html" />
+ <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/index.html" />
+ <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/index.html" />
</menu>
<menu name="Tutorial">
<item name="Build streams and streams-examples from source" href="tutorial/build.html" />
<item name="Create a local folder to configure streams" href="tutorial/environment.html"/>
<item name="Set up a local docker environment" href="tutorial/docker.html" />
<item name="Run a local database to store activity documents" href="tutorial/database.html" />
- <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
+ <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/latest/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
<item name="Browse and search the twitter data you collected" href="tutorial/browse.html" />
</menu>
<menu name="Other Resources">
diff --git a/src/site/site_en.xml b/src/site/site_en.xml
index effd07e..b844c2d 100644
--- a/src/site/site_en.xml
+++ b/src/site/site_en.xml
@@ -71,23 +71,23 @@
</menu>
<menu name="Projects">
<item name="streams-master" />
- <item name="streams-project" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project" />
- <item name="streams-examples" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-examples/" />
+ <item name="streams-project" href="http://streams.incubator.apache.org/site/latest/streams-project" />
+ <item name="streams-examples" href="http://streams.incubator.apache.org/site/latest/streams-examples/" />
</menu>
<menu name="Getting Started">
- <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-pojo/index.html" />
- <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/streams-provider-twitter/index.html" />
- <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
- <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-components/index.html" />
- <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
- <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/0.3-incubating-SNAPSHOT/streams-project/streams-contrib/index.html" />
+ <item name="View the official Apache Streams jsonschema files" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-pojo/index.html" />
+ <item name="View the schemas that describe twitter data" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/streams-provider-twitter/index.html" />
+ <item name="Read about twitter / streams conversion" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/index.html" />
+ <item name="Learn about utility streams components" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-components/index.html" />
+ <item name="Learn about streams interoperability modules" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/index.html" />
+ <item name="Browse streams providers, processors, and persisters" href="http://streams.incubator.apache.org/site/latest/streams-project/streams-contrib/index.html" />
</menu>
<menu name="Tutorial">
<item name="Build streams and streams-examples from source" href="tutorial/build.html" />
<item name="Create a local folder to configure streams" href="tutorial/environment.html"/>
<item name="Set up a local docker environment" href="tutorial/docker.html" />
<item name="Run a local database to store activity documents" href="tutorial/database.html" />
- <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/0.2-incubating-SNAPSHOT/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
+ <item name="Run twitter-history-elasticsearch" href="http://streams.incubator.apache.org/site/latest/streams-examples/streams-examples-local/twitter-history-elasticsearch/index.html" />
<item name="Browse and search the twitter data you collected" href="tutorial/browse.html" />
</menu>
<menu name="Other Resources">
