docs/src/upgrade-release-3.1.x-incubating.asciidoc

////
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements.  See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
////

TinkerPop 3.1.0
===============

image::https://raw.githubusercontent.com/apache/incubator-tinkerpop/master/docs/static/images/gremlin-gangster.png[width=225]

*A 187 On The Undercover Gremlinz*

TinkerPop 3.1.0
---------------

*Release Date: November 16, 2015*

Please see the link:https://github.com/apache/incubator-tinkerpop/blob/3.1.0-incubating/CHANGELOG.asciidoc#tinkerpop-310-release-date-november-16-2015[changelog] for a complete list of all the modifications that are part of this release.

Additional upgrade information can be found here:

* <<_tinkerpop_3_0_2,TinkerPop 3.0.2>>
* <<_tinkerpop_3_0_1,TinkerPop 3.0.1>>

Upgrading for Users
~~~~~~~~~~~~~~~~~~~

Shading Jackson
^^^^^^^^^^^^^^^

The Jackson library is now shaded to `gremlin-shaded`, which will allow Jackson to version independently without
breaking compatibility with dependent libraries or with those who depend on TinkerPop.  The downside is that if a
library depends on TinkerPop and uses the Jackson classes, those classes will no longer exist with the standard
Jackson package naming.  They will have to shifted as follows:

* `org.objenesis` becomes `org.apache.tinkerpop.shaded.objenesis`
* `com.esotericsoftware.minlog` becomes `org.apache.tinkerpop.shaded.minlog`
* `com.fasterxml.jackson` becomes `org.apache.tinkerpop.shaded.jackson`

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-835[TINKERPOP3-835]

PartitionStrategy and VertexProperty
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

`PartitionStrategy` now supports partitioning within `VertexProperty`.  The `Graph` needs to be able to support
meta-properties for this feature to work.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-333[TINKERPOP3-333]

Gremlin Server and Epoll
^^^^^^^^^^^^^^^^^^^^^^^^

Gremlin Server provides a configuration option to turn on support for Netty
link:http://netty.io/wiki/native-transports.html[native transport] on Linux, which has been shown to help improve
performance.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-901[TINKERPOP3-901]

Rebindings Deprecated
^^^^^^^^^^^^^^^^^^^^^

The notion of "rebindings" has been deprecated in favor of the term "aliases".  Alias is a better and more intuitive
term than rebindings which should make it easier for newcomers to understand what they are for.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-913[TINKERPOP3-913],
link:http://tinkerpop.incubator.apache.org/docs/3.1.0-incubating/#_aliases[Reference Documentation - Aliases]

Configurable Driver Channelizer
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The Gremlin Driver now allows the `Channerlizer` to be supplied as a configuration, which means that custom
implementations may be supplied.

See: https://issues.apache.org/jira/browse/TINKERPOP3-680[TINKERPOP3-680]

GraphSON and Strict Option
^^^^^^^^^^^^^^^^^^^^^^^^^^

The `GraphMLReader` now has a `strict` option on the `Builder` so that if a data type for a value is invalid in some
way, GraphMLReader will simply skip that problem value. In that way, it is a bit more forgiving than before especially
with empty data.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-756[TINKERPOP3-756]

Transaction.close() Default Behavior
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The default behavior of `Transaction.close()` is to rollback the transaction.  This is in contrast to previous versions
where the default behavior was commit.  Using rollback as the default should be thought of as a like a safer approach
to closing where a user must now explicitly call `commit()` to persist their mutations.

See link:https://issues.apache.org/jira/browse/TINKERPOP3-805[TINKERPOP3-805] for more information.

ThreadLocal Transaction Settings
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The `Transaction.onReadWrite()` and `Transaction.onClose()` settings now need to be set for each thread (if another
behavior than the default is desired). For gremlin-server users that may be changing these settings via scripts.
If the settings are changed for a sessionless request they will now only apply to that one request. If the settings are
changed for an in-session request they will now only apply to all future requests made in the scope of that session.

See link:https://issues.apache.org/jira/browse/TINKERPOP3-885[TINKERPOP3-885]

Hadoop-Gremlin
^^^^^^^^^^^^^^

* Hadoop1 is no longer supported. Hadoop2 is now the only supported Hadoop version in TinkerPop.
* Spark and Giraph have been split out of Hadoop-Gremlin into their own respective packages (Spark-Gremlin and Giraph-Gremlin).
* The directory where application jars are stored in HDFS is now `hadoop-gremlin-x.y.z-libs`.
** This versioning is important so that cross-version TinkerPop use does not cause jar conflicts.

See link:https://issues.apache.org/jira/browse/TINKERPOP3-616

Spark-Gremlin
^^^^^^^^^^^^^

* Providers that wish to reuse a graphRDD can leverage the new `PersistedInputRDD` and `PersistedOutputRDD`.
** This allows the graphRDD to avoid serialization into HDFS for reuse. Be sure to enabled persisted `SparkContext` (see documentation).

See link:https://issues.apache.org/jira/browse/TINKERPOP3-868,
link:https://issues.apache.org/jira/browse/TINKERPOP3-925

TinkerGraph Serialization
^^^^^^^^^^^^^^^^^^^^^^^^^

TinkerGraph is serializable over Gryo, which means that it can shipped over the wire from Gremlin Server.  This
feature can be useful when working with remote subgraphs.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-728[TINKERPOP3-728]

Deprecation in TinkerGraph
^^^^^^^^^^^^^^^^^^^^^^^^^^

The `public static String` configurations have been renamed. The old `public static` variables have been deprecated.
If the deprecated variables were being used, then convert to the replacements as soon as possible.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-926[TINKERPOP3-926]

Deprecation in Gremlin-Groovy
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The closure wrappers classes `GFunction`, `GSupplier`, `GConsumer` have been deprecated. In Groovy, a closure can be
specified using `as Function` and thus, these wrappers are not needed. Also, the `GremlinExecutor.promoteBindings()`
method which was previously deprecated has been removed.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-879[TINKERPOP3-879],
link:https://issues.apache.org/jira/browse/TINKERPOP3-897[TINKERPOP3-897]

Gephi Traversal Visualization
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The process for visualizing a traversal has been simplified.  There is no longer a need to "name" steps that will
represent visualization points for Gephi.  It is possible to just "configure" a `visualTraversal` in the console:

[source,text]
gremlin> :remote config visualTraversal graph vg

which creates a special `TraversalSource` from `graph` called `vg`.  The traversals created from `vg` can be used
to `:submit` to Gephi.

See: link:http://tinkerpop.incubator.apache.org/docs/3.1.0-SNAPSHOT/#gephi-plugin[Reference Documentation - Gephi]

Alterations to GraphTraversal
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There were a number of changes to `GraphTraversal`.  Many of the changes came by way of deprecation, but some semantics
have changed as well:

* `ConjunctionStrategy` has been renamed to `ConnectiveStrategy` (no other behaviors changed).
* `ConjunctionP` has been renamed to `ConnectiveP` (no other behaviors changed).
* `DedupBijectionStrategy` has been renamed (and made more effective) as `FilterRankingStrategy`.
* The `GraphTraversal` mutation API has change significantly with all previous methods being supported but deprecated.
** The general pattern used now is `addE('knows').from(select('a')).to(select('b')).property('weight',1.0)`.
* The `GraphTraversal` sack API has changed with all previous methods being supported but deprecated.
** The old `sack(mult,'weight')` is now `sack(mult).by('weight')`.
* `GroupStep` has been redesigned such that there is now only a key- and value-traversal. No more reduce-traversal.
** The previous `group()`-methods have been renamed to `groupV3d0()`. To immediately upgrade, rename all your `group()`-calls to `groupV3d0()`.
** To migrate to the new `group()`-methods, what was `group().by('age').by(outE()).by(sum(local))` is now `group().by('age').by(outE().sum())`.
* There was a bug in `fold()`, where if a bulked traverser was provided, the traverser was only represented once.
** This bug fix might cause a breaking change to a user query if the non-bulk behavior was being counted on. If so, used `dedup()` prior to `fold()`.
* Both `GraphTraversal().mapKeys()` and `GraphTraversal.mapValues()` has been deprecated.
** Use `select(keys)` and `select(columns)`. However, note that `select()` will not unroll the keys/values. Thus, `mapKeys()` => `select(keys).unfold()`.
* The data type of `Operator` enums will now always be the highest common data type of the two given numbers, rather than the data type of the first number, as it's been before.

Aliasing Remotes in the Console
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The `:remote` command in Gremlin Console has a new `alias` configuration option.  This `alias` option allows
specification of a set of key/value alias/binding pairs to apply to the remote.  In this way, it becomes possible
to refer to a variable on the server as something other than what it is referred to for purpose of the submitted
script.  For example once a `:remote` is created, this command:

[source,text]
:remote alias x g

would allow "g" on the server to be referred to as "x".

[source,text]
:> x.E().label().groupCount()

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-914[TINKERPOP3-914]

Upgrading for Providers
~~~~~~~~~~~~~~~~~~~~~~~

IMPORTANT: It is recommended that providers also review all the upgrade instructions specified for users. Many of the
changes there may prove important for the provider's implementation.

All providers should be aware that Jackson is now shaded to `gremlin-shaded` and could represent breaking change if
there was usage of the dependency by way of TinkerPop, a direct dependency to Jackson may be required on the
provider's side.

Graph System Providers
^^^^^^^^^^^^^^^^^^^^^^

GraphStep Alterations
+++++++++++++++++++++

* `GraphStep` is no longer in `sideEffect`-package, but now in `map`-package as traversals support mid-traversal `V()`.
* Traversals now support mid-traversal `V()`-steps. Graph system providers should ensure that a mid-traversal `V()` can leverage any suitable index.

See link:https://issues.apache.org/jira/browse/TINKERPOP3-762

Decomposition of AbstractTransaction
++++++++++++++++++++++++++++++++++++

The `AbstractTransaction` class has been abstracted into two different classes supporting two different modes of
operation: `AbstractThreadLocalTransaction` and `AbstractThreadedTransaction`, where the former should be used when
supporting `ThreadLocal` transactions and the latter for threaded transactions.  Of course, providers may still
choose to build their own implementation on `AbstractTransaction` itself or simply implement the `Transaction`
interface.

The `AbstractTransaction` gains the following methods to potentially implement (though default implementations
are supplied in `AbstractThreadLocalTransaction` and `AbstractThreadedTransaction`):

* `doReadWrite` that should execute the read-write consumer.
* `doClose` that should execute the close consumer.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-765[TINKERPOP3-765],
link:https://issues.apache.org/jira/browse/TINKERPOP3-885[TINKERPOP3-885]

Transaction.close() Default Behavior
++++++++++++++++++++++++++++++++++++

The default behavior for `Transaction.close()` is to rollback the transaction and is enforced by tests, which
previously asserted the opposite (i.e. commit on close).  These tests have been renamed to suite the new semantics:

* `shouldCommitOnCloseByDefault` became `shouldCommitOnCloseWhenConfigured`
* `shouldRollbackOnCloseWhenConfigured` became `shouldRollbackOnCloseByDefault`

If these tests were referenced in an `OptOut`, then their names should be updated.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-805[TINKERPOP3-805]

Graph Traversal Updates
+++++++++++++++++++++++

There were numerous changes to the `GraphTraversal` API. Nearly all changes are backwards compatible with respective
"deprecated" annotations. Please review the respective updates specified in the "Graph System Users" section.

* `GraphStep` is no longer in `sideEffect` package. Now in `map` package.
* Make sure mid-traversal `GraphStep` calls are folding `HasContainers` in for index-lookups.
* Think about copying `TinkerGraphStepStrategyTest` for your implementation so you know folding is happening correctly.

Element Removal
+++++++++++++++

`Element.Exceptions.elementAlreadyRemoved` has been deprecated and test enforcement for consistency have been removed.
 Providers are free to deal with deleted elements as they see fit.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-297[TINKERPOP3-297]

VendorOptimizationStrategy Rename
+++++++++++++++++++++++++++++++++

The `VendorOptimizationStrategy` has been renamed to `ProviderOptimizationStrategy`.  This renaming is consistent
with revised terminology for what were formerly referred to as "vendors".

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-876[TINKERPOP3-876]

GraphComputer Updates
+++++++++++++++++++++

`GraphComputer.configure(String key, Object value)` is now a method (with default implementation).
This allows the user to specify engine-specific parameters to the underlying OLAP system. These parameters are not intended
to be cross engine supported. Moreover, if there are not parameters that can be altered (beyond the standard `GraphComputer`
methods), then the provider's `GraphComputer` implementation should simply return and do nothing.

Driver Providers
^^^^^^^^^^^^^^^^

Aliases Parameter
+++++++++++++++++

The "rebindings" argument to the "standard" `OpProcessor` has been renamed to "aliases". While "rebindings" is still
supported it is recommended that the upgrade to "aliases" be made as soon as possible as support will be removed in
the future.  Gremlin Server will not accept both parameters at the same time - a request must contain either one
parameter or the other if either is supplied.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-913[TINKERPOP3-913]

ThreadLocal Transaction Settings
++++++++++++++++++++++++++++++++

If a driver configures the `Transaction.onReadWrite()` or `Transaction.onClose()` settings, note that these settings no
longer apply to all future requests. If the settings are changed for a sessionless request they will only apply to
that one request. If the settings are changed from an in-session request they will only apply to all future requests
made in the scope of that session.

See: link:https://issues.apache.org/jira/browse/TINKERPOP3-885[TINKERPOP3-885]