| //// |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| //// |
| |
| = TinkerPop 3.5.0 |
| |
| image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/images/gremlin-sleeping-beauty.png[width=225] |
| |
| *The Sleeping Gremlin: No. 18 Entr'acte symphonique* |
| |
| == TinkerPop 3.5.0 |
| |
| *Release Date: NOT OFFICIALLY RELEASED YET* |
| |
| Please see the link:https://github.com/apache/tinkerpop/blob/3.5.0/CHANGELOG.asciidoc#release-3-5-0[changelog] for a complete list of all the modifications that are part of this release. |
| |
| === Upgrading for Users |
| |
| ==== Java 11 |
| |
| TinkerPop now builds and is compatible with Java 11. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2076[TINKERPOP-2076] |
| |
| ==== Anonymous Child Traversals |
| |
| TinkerPop conventions for child traversals is to spawn them anonymously from `__`, therefore: |
| |
| [source,groovy] |
| g.addV('person').addE('self').to(__.V(1)) |
| |
| or more succinctly via static import as: |
| |
| [source,groovy] |
| g.addV('person').addE('self').to(V(1)) |
| |
| Some users have chosen to instead write the above as: |
| |
| [source,groovy] |
| g.addV('person').addE('self').to(g.V(1)) |
| |
| which spawns a child traversal from a `GraphTraversalSource`. When spawned this way, a traversal is bound to a "source" |
| and therefore is not anonymous. While the above code worked, it is important that there be less ways to do things |
| with Gremlin so as to avoid confusion in examples, documentations and mailing list answers. |
| |
| As of 3.5.0, attempting to use a traversal spawned from a "source" will result in an exception. Users will need to |
| modify their code if they use the unconventional syntax. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2361[TINKERPOP-2361] |
| |
| ==== GraphSON and JsonBuilder |
| |
| GraphSON serialization support for Groovy's `JsonBuilder` has been present since the first version of GraphSON. That |
| approach to returning results has never materialized as a standardized way to use Gremlin as originally envisioned. |
| While support for this serialization form is still present, the dependency on Groovy in `gremlin-driver` has been |
| changed to `provided` scope, which means that users who wish to continue to return `JsonBuilder` results for some |
| reason must explicitly include `groovy` and `groovy-json` dependencies in their applications. For Maven this would |
| mean adding the following depenencies: |
| |
| [source,xml] |
| ---- |
| <dependency> |
| <groupId>org.codehaus.groovy</groupId> |
| <artifactId>groovy</artifactId> |
| <version>${groovy.version}</version> |
| <classifier>indy</classifier> |
| </dependency> |
| <dependency> |
| <groupId>org.codehaus.groovy</groupId> |
| <artifactId>groovy-json</artifactId> |
| <version>${groovy.version}</version> |
| <classifier>indy</classifier> |
| <exclusions> |
| <!-- exclude non-indy type --> |
| <exclusion> |
| <groupId>org.codehaus.groovy</groupId> |
| <artifactId>groovy</artifactId> |
| </exclusion> |
| </exclusions> |
| </dependency> |
| ---- |
| |
| The `${groovy.version}` should match the version specified in TinkerPop's root |
| link:https://github.com/apache/tinkerpop/blob/3.4.8/pom.xml[pom.xml]. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2460[TINKERPOP-2460] |
| |
| ==== JavascriptTranslator |
| |
| Introduced a `JavascriptTranslator` for Java, which is in `gremlin-core` with the other language translators. It |
| generates a Javascript representation of Gremlin from bytecode. |
| |
| [source,java] |
| ---- |
| // gremlin-core module |
| import org.apache.tinkerpop.gremlin.process.traversal.translator.JavascriptTranslator; |
| |
| GraphTraversalSource g = ...; |
| Traversal<Vertex,Integer> t = g.V().has("person","name","marko"). |
| where(in("knows")). |
| values("age"). |
| map(Lambda.function("it.get() + 1")); |
| |
| Translator.ScriptTranslator javascriptTranslator = JavascriptTranslator.of("g"); |
| System.out.println(javascriptTranslator.translate(t).getScript()); |
| // OUTPUT: g.V().has("person","name","marko").where(__.in_("knows")).values("age").map(() => "it.get() + 1") |
| ---- |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2451[TINKERPOP-2451] |
| |
| ==== DotNetTranslator |
| |
| Introduced a `DotNetTranslator` for Java, which is in `gremlin-core` with the other language translators. It |
| generates a C# representation of Gremlin from bytecode. |
| |
| [source,java] |
| ---- |
| // gremlin-core module |
| import org.apache.tinkerpop.gremlin.process.traversal.translator.DotNetTranslator; |
| |
| GraphTraversalSource g = ...; |
| Traversal<Vertex,Integer> t = g.V().has("person","name","marko"). |
| where(in("knows")). |
| values("age"). |
| map(Lambda.function("it.get() + 1")); |
| |
| Translator.ScriptTranslator dotnetTranslator = DotNetTranslator.of("g"); |
| System.out.println(dotnetTranslator.translate(t).getScript()); |
| // OUTPUT: g.V().Has("person","name","marko").Where(__.In("knows")).Values<object>("age").Map<object>(Lambda.Groovy("it.get() + 1")) |
| ---- |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2452[TINKERPOP-2452] |
| |
| ==== Gryo Usage |
| |
| Since the first release of TinkerPop 3.x, Gryo has been the default serialization format for Gremlin Server and |
| Java Driver. It was also used as the default serialization format for Gremlin Console remote connectivity to Gremlin |
| Server. As of this release, Gryo has been replaced as the default by GraphBinary. All packaged configuration files |
| and programmatic defaults have been modified as such. |
| |
| It is still possible to utilize Gryo as a message serialization format by modifying Gremlin Server configuration files |
| to include the appropriate Gryo configurations. If using Gryo, do not user earlier versions of the driver and server |
| with 3.5.0. Use a 3.5.0 client to connect to a 3.5.0 server. Generally speaking, mixed version combinations will |
| appear to work properly, but problems will likely occur during general course of usage and it is therefore not |
| advisable to take this approach. |
| |
| For best compatibility between 3.4.x and 3.5.x, please use GraphBinary. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2259[TINKERPOP-2259] |
| |
| ==== Configuration Upgrade |
| |
| There is a major breaking change in the use of `Configuration` objects. Prior to 3.5.0, `Configuration` objects were |
| from the Apache Commons `commons-configuration` library, but in this version, they are of `commons-configuration2`. |
| While this is a breaking change, the fix for most implementations will be quite simple, Simply, change the import |
| statements from: |
| |
| [source,text] |
| ---- |
| org.apache.commons.configuration.* |
| ---- |
| |
| to |
| |
| [source,text] |
| ---- |
| org.apache.commons.configuration2.* |
| ---- |
| |
| It is also worth noting that default list handling in configurations is treated differently. TinkerPop largely |
| disabled the default list handling approach in `Configuration` 1.x, but if that functionality is still needed, it can |
| be reclaimed by setting the `LegacyListDelimiterHandler` - details for doing taking this step and other relevant |
| upgrade information can be found in the link:https://commons.apache.org/proper/commons-configuration/userguide/upgradeto2_0.html[2.x Upgrade Documentation]. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2185[TINKERPOP-2185] |
| |
| ==== Use of null |
| |
| Gremlin has traditionally disallowed `null` as a value in traversals and not always in consistent ways: |
| |
| [source,text] |
| ---- |
| gremlin> g.inject(1, null, null, 2, null) |
| java.lang.NullPointerException |
| Type ':help' or ':h' for help. |
| Display stack trace? [yN]n |
| gremlin> g.V().has('person','name','marko').property('age', null) |
| The AddPropertyStep does not have a provided value: AddPropertyStep({key=[age]}) |
| Type ':help' or ':h' for help. |
| Display stack trace? [yN] |
| gremlin> g.addV("person").property("name", 'stephen').property("age", null) |
| ==>v[13] |
| gremlin> g.V().has('person','name','stephen').elementMap() |
| ==>[id:13,label:person,name:stephen] |
| gremlin> g.V().constant(null) |
| gremlin> |
| ---- |
| |
| Note how `null` can produce exception behavior or act as a filter. For 3.5.0, TinkerPop has not only made `null` usage |
| consistent, but has also made it an allowable value within a `Traversal`: |
| |
| [source,text] |
| ---- |
| gremlin> g.inject(1, null, null, 2, null) |
| ==>1 |
| ==>null |
| ==>null |
| ==>null |
| ==>2 |
| gremlin> g.V().constant(null) |
| ==>null |
| ==>null |
| ==>null |
| ==>null |
| ==>null |
| ==>null |
| ---- |
| |
| TinkerGraph can be configured to support `null` as a property value and all graphs may not support this feature (for |
| example, Neo4j does not). Please be sure to check the new `supportsNullPropertyValues()` feature (or the documentation |
| of the graph provider) to determine if the `Graph` implementation allows `null` as a property value. |
| |
| With respect to `null` in relation to properties, there was a bit of inconsistency in the handling of `null` in calls |
| to `property()` depending on the type of mutation being executed demonstrated as follows in earlier versions: |
| |
| [source,text] |
| ---- |
| gremlin> g.V(1).property("x", 1).property("y", null).property("z", 2) |
| The AddPropertyStep does not have a provided value: AddPropertyStep({key=[y]}) |
| Type ':help' or ':h' for help. |
| Display stack trace? [yN]N |
| gremlin> g.addV("test").property("x", 1).property("y", null).property("z", 2) |
| ==>v[13] |
| gremlin> g.V(13).properties() |
| ==>vp[x->1] |
| ==>vp[z->2] |
| ---- |
| |
| This behavior has been altered to become consistent. First, assuming `null` is not supported as a property value, the |
| setting of a property to `null` should have the behavior of removing the property in the same way in which you might |
| do `g.V().properties().drop()`: |
| |
| [source,text] |
| ---- |
| gremlin> g.V(1).property("x", 1).property("y", null).property("z", 2) |
| ==>v[1] |
| gremlin> g.V(1).elementMap() |
| ==>[id:1,label:person,name:marko,x:1,z:2,age:29] |
| gremlin> g.V().hasLabel('person').property('age',null).iterate() |
| gremlin> g.V().hasLabel('person').elementMap() |
| ==>[id:1,label:person,name:marko] |
| ==>[id:2,label:person,name:vadas] |
| ==>[id:4,label:person,name:josh] |
| ==>[id:6,label:person,name:peter] |
| ---- |
| |
| Then, assuming `null` is supported as a property value: |
| |
| [source,text] |
| ---- |
| gremlin> g.addV("person").property("name", 'stephen').property("age", null) |
| ==>v[13] |
| gremlin> g.V().has('person','name','stephen').elementMap() |
| ==>[id:13,label:person,name:stephen,age:null] |
| gremlin> g.V().has('person','age',null) |
| ==>v[13] |
| ---- |
| |
| The above described changes also has an effect on steps like `group()` and `groupCount()` which formerly produced |
| exceptions when keys could not be found: |
| |
| [source,text] |
| ---- |
| gremlin> g.V().group().by('age') |
| The property does not exist as the key has no associated value for the provided element: v[3]:age |
| Type ':help' or ':h' for help. |
| Display stack trace? [yN]n |
| ---- |
| |
| The solution was to filter away vertices that did not have the available key so that such steps would work properly |
| or to write a more complex `by()` modulator to better handle the possibility of a missing key. With the latest changes |
| however none of that is necessary unless desired: |
| |
| [source,text] |
| ---- |
| gremlin> g.V().groupCount().by('age') |
| ==>[null:2,32:1,35:1,27:1,29:1] |
| ---- |
| |
| In conclusion, this change in greater support of `null` may affect the behavior of existing traversals written in past |
| versions of TinkerPop as it is no longer possible to rely on `null` to expect a filtering action for traversers. |
| Please review existing Gremlin carefully to ensure that there are no unintended consequences of this change and that |
| there are no opportunities to improve existing logic to take greater advantage of this expansion of `null` semantics. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2235[TINKERPOP-2235], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2099[TINKERPOP-2099] |
| |
| ==== Remote SideEffects |
| |
| Remote traversals no longer support the retrieval of remote side-effects. Users must therefore directly return |
| side-effects as part of their query if they need that data. Note that server settings for `TraversalOpProcessor`, which |
| formerly held the cache for these side-effects, no longer have any effect and can be removed. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2269[TINKERPOP-2269] |
| |
| ==== ByModulatorOptimizationStrategy |
| |
| The new `ByModulatorOptimizationStrategy` attempts to re-write `by()` modulator traversals to use their more optimized |
| forms which can provide a major performance improvement. As a simple an example, a traversal like `by(id())` would |
| be replaced by `by(id)`, thus replacing a step-based traversal with a token-based traversal. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-1682[TINKERPOP-1682] |
| |
| ==== SeedStrategy |
| |
| The new `SeedStrategy` allows the user to set a seed value for steps that make use of `Random` so that the traversal |
| has the ability to return deterministic results. While this feature is useful for testing and debugging purposes, |
| there are also some practical applications as well. |
| |
| [source,text] |
| ---- |
| gremlin> g.V().values('name').fold().order(local).by(shuffle) |
| ==>[josh,marko,vadas,peter,ripple,lop] |
| gremlin> g.V().values('name').fold().order(local).by(shuffle) |
| ==>[vadas,lop,marko,peter,josh,ripple] |
| gremlin> g.V().values('name').fold().order(local).by(shuffle) |
| ==>[peter,ripple,josh,lop,marko,vadas] |
| gremlin> g.withStrategies(new SeedStrategy(22323)).V().values('name').fold().order(local).by(shuffle) |
| ==>[lop,peter,josh,marko,vadas,ripple] |
| gremlin> g.withStrategies(new SeedStrategy(22323)).V().values('name').fold().order(local).by(shuffle) |
| ==>[lop,peter,josh,marko,vadas,ripple] |
| gremlin> g.withStrategies(new SeedStrategy(22323)).V().values('name').fold().order(local).by(shuffle) |
| ==>[lop,peter,josh,marko,vadas,ripple] |
| ---- |
| |
| ==== by(T) for Property |
| |
| The `Property` interface is not included in the hierarchy of `Element`. This means that an edge property or a |
| meta-property are not considered elements the way that a `VertexProperty` is. As a result, some usages of `T` in |
| relation to properties do not work consistently. One such example is `by(T)`, a token-based traversal, where the |
| following works for a `VertexProperty` but will not for edge properties or meta-properties: |
| |
| [source,text] |
| ---- |
| gremlin> g.V(1).properties().as('a').select('a').by(key) |
| ==>name |
| ==>age |
| ---- |
| |
| For a `Property` you would need to use `key()`-step: |
| |
| [source,text] |
| ---- |
| gremlin> g.E(11).properties().as('a').select(last,'a').by(key()) |
| ==>weight |
| ---- |
| |
| Aside from the inconsistency, this issue also presents a situation where performance is impacted as token-based |
| traversals are inherently faster than step-based ones. In 3.5.0, this issue has been resolved in conjunction with the |
| introduction of `ByModulatorOptimizationStrategy` which will optimize `by(key())` and `by(value())` to their |
| appropriate token versions automatically. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-1682[TINKERPOP-1682] |
| |
| ==== Complex dict Deserialization |
| |
| In Gremlin it is common to return a `dict` or `list` as a key value in another `dict`. The problem for Python is that |
| these values are not hashable and will result in an error. By introducing a `HashableDict` and `Tuple` for those keys |
| (respectively), it is now possible to return these types of results and not have to work around them: |
| |
| [source,text] |
| ---- |
| >>> g.V().has('person', 'name', 'marko').elementMap("name").groupCount().next() |
| {{<T.id: 1>: 1, <T.label: 4>: 'person', 'name': 'marko'}: 1} |
| ---- |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2395[TINKERPOP-2395], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2407[TINKERPOP-2407] |
| |
| ==== Gremlin Server Audit Logging |
| |
| The `authentication.enableAuditlog` configuration property is deprecated, but replaced by the `enableAuditLog` property |
| to also make it available to `Authorizer` implementations. With the new setting enabled, there are slight changes in the |
| formatting of audit log messages. In particular, the name of the authenticated user is included in every message. |
| |
| ==== Gremlin Server Authorization |
| |
| While Gremlin Server has long had authentication options to determine if a user can connect to the server, it now also |
| contains the ability to apply a level of authorization to better control what a particular authenticated user will |
| have access to. Authorization is controlled by the new `Authorizer` interface, which can be implemented by users and |
| graph providers to provide this custom functionality. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2389[TINKERPOP-2389], |
| link:https://tinkerpop.apache.org/docs/3.5.0/reference/#authorization[Reference Documentation] |
| |
| ==== Python 2.x Support |
| |
| The gremlinpython module no longer supports Python 2.x. Users must use Python 3 going forward. For the most part, from |
| a user's perspective, there are no specific API changes to consider as a result of this change. It is also worth |
| noting that Jython support has been removed and that `gremlin-python` no longer produces a JVM-based artifact. This |
| change means that the `GremlinJythonScriptEngine` no longer exists and there is no way to write native Python lambdas. |
| All lambdas should be written using `gremlin-groovy` if they are needed. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2317[TINKERPOP-2317] |
| |
| ==== Python Kerberos Support |
| |
| The Python Driver now supports Kerberos based authentication: |
| |
| [source,python] |
| ---- |
| g = traversal().withRemote(DriverRemoteConnection( |
| 'ws://localhost:8182/gremlin', 'g', kerberized_service='gremlin@hostname.your.org')) |
| ---- |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-1641[TINKERPOP-1641], |
| link:https://tinkerpop.apache.org/docs/current/reference/#gremlin-python-connecting[Reference Documentation] |
| |
| ==== .NET Standard 2.0 Only |
| |
| Gremlin.NET no longer targets .NET Standard 1.3, but only .NET Standard 2.0. Since .NET Core 2.0 and .NET Framework |
| 4.6.1 already support this .NET Standard version, most users should not be impacted by this. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2335[TINKERPOP-2335] |
| |
| ==== Gremlin.NET: New JSON Library |
| |
| Gremlin.NET now uses `System.Text.Json` instead of Newtonsoft.Json as `System.Text.Json` is already included in .NET |
| Core 3.0 and higher which means that we have one dependency less on this platform and because it offers an increased |
| performance. |
| Most users should not notice this change. But users who have implemented their own GraphSON serializers or |
| deserializers probably have to change them accordingly. The same applies to users that let Gremlin.NET return data |
| without deserializing it first as the returned data types will change in this case, for example from Newtonsoft.Json's |
| `JObject` or `JToken` to `JsonElement` with `System.Text.Json`. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2349[TINKERPOP-2349], |
| link:http://tinkerpop.apache.org/docs/3.5.0/dev/provider/#_supporting_gremlin_net_io[Documentation for custom JSON serialization with Gremlin.NET] |
| |
| ==== Neo4j Changes |
| |
| There were two key changes to the neo4j-gremlin module: |
| |
| * The underlying Neo4j version moved from the 3.2.x line to 3.4.x line. Please see the |
| link:https://neo4j.com/guides/upgrade-archive/[Neo4j Upgrade FAQ] for more information as features and |
| configuration options may have changed. |
| * Experimental support for multi/meta-properties in Neo4j which were previously deprecated have now been permanently |
| removed. |
| |
| ==== match() Consistency |
| |
| The `match()` step behavior might have seemed inconsistent those first using it. While there are a number of examples |
| that might demonstrate this issue, the easiest one to consume would be: |
| |
| [source,text] |
| ---- |
| gremlin> g.V().match(__.as("a").out("knows").as("b")) |
| ==>[a:v[1],b:v[2]] |
| ==>[a:v[1],b:v[4]] |
| gremlin> g.V().match(__.as("a").out("knows").as("b")).unfold() |
| gremlin> g.V().match(__.as("a").out("knows").as("b")).identity() |
| ==>[] |
| ==>[] |
| ---- |
| |
| The output is unexpected if there isn't awareness of some underlying optimizations at play, where `match()` as the |
| final step in the traversal implies that the user wants all of the labels as part of the output. With the addition |
| of the extra steps, `unfold()` and `identity()` in the above case, the implication is that the traversal must be |
| explicit in the labels to preserve from match, thus: |
| |
| [source,text] |
| ---- |
| gremlin> g.V().match(__.as("a").out("knows").as("b")).select('a','b').unfold() |
| ==>a=v[1] |
| ==>b=v[2] |
| ==>a=v[1] |
| ==>b=v[4] |
| gremlin> g.V().match(__.as("a").out("knows").as("b")).select('a','b').identity() |
| ==>[a:v[1],b:v[2]] |
| ==>[a:v[1],b:v[4]] |
| ---- |
| |
| Being explicit, as is the preference in writing Gremlin to good form, helps restrict the path history required to |
| execute the traversal and therefore preserves memory. Of course, making `match()` a special form of end step is a |
| confusing approach as the behavior of the step changes simply because another step is in play. Furthermore, correct |
| execution of the traversal, relies on the execution of traversal strategies when the same traversal should produce |
| the same results irrespective of the strategies applied to it. |
| |
| In 3.5.0, we look to better adhere to that guiding design principle and ensure a more consistent output for these types |
| of traversals. While the preferred method is to specify the labels to preserve from `match()` with a following |
| `select()` step as shown above, `match()` will now consistently return all labels when they are not specified |
| explicitly. |
| |
| [source,text] |
| ---- |
| gremlin> g.V().match(__.as("a").out("knows").as("b")) |
| ==>[a:v[1],b:v[2]] |
| ==>[a:v[1],b:v[4]] |
| gremlin> g.V().match(__.as("a").out("knows").as("b")).identity() |
| ==>[a:v[1],b:v[2]] |
| ==>[a:v[1],b:v[4]] |
| gremlin> g.V().match(__.as("a").out("knows").as("b")).unfold() |
| ==>a=v[1] |
| ==>b=v[2] |
| ==>a=v[1] |
| ==>b=v[4] |
| ---- |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2481[TINKERPOP-2481], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2499[TINKERPOP-2499] |
| |
| ==== Deprecation Removal |
| |
| The following deprecated classes, methods or fields have been removed in this version: |
| |
| * `gremlin-core` |
| ** `org.apache.tinkerpop.gremlin.process.computer.bulkdumping.BulkDumperVertexProgram` |
| ** `org.apache.tinkerpop.gremlin.process.computer.bulkloading.BulkLoader` |
| ** `org.apache.tinkerpop.gremlin.process.computer.bulkloading.BulkLoaderVertexProgram` |
| ** `org.apache.tinkerpop.gremlin.process.computer.bulkloading.IncrementalBulkLoader` |
| ** `org.apache.tinkerpop.gremlin.process.computer.bulkloading.OneTimeBulkLoader` |
| ** `org.apache.tinkerpop.gremlin.process.computer.clustering.peerpressure.PeerPressureVertexProgram.Builder#traversal(*)` |
| ** `org.apache.tinkerpop.gremlin.process.computer.ranking.pagerank.PageRankVertexProgram.Builder#traversal(*)` |
| ** `org.apache.tinkerpop.gremlin.process.computer.ranking.pagerank.PageRankVertexProgram.Builder#vertexCount()` |
| ** `org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PageRankVertexProgramStep.modulateBy(*)` |
| ** `org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PageRankVertexProgramStep.modulateTimes()` |
| ** `org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PeerPressureVertexProgramStep.modulateBy(*)` |
| ** `org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.PeerPressureVertexProgramStep.modulateTimes()` |
| ** `org.apache.tinkerpop.gremlin.process.remote.traversal.AbstractRemoteTraversalSideEffects` |
| ** `org.apache.tinkerpop.gremlin.process.remote.traversal.EmbeddedRemoteTraversalSideEffects` |
| ** `org.apache.tinkerpop.gremlin.process.remote.traversal.RemoteTraversalSideEffects` |
| ** `org.apache.tinkerpop.gremlin.process.remote.traversal.RemoteTraversal#getSideEffects()` |
| ** `org.apache.tinkerpop.gremlin.process.traversal.Order.decr` |
| ** `org.apache.tinkerpop.gremlin.process.traversal.Order.incr` |
| ** `org.apache.tinkerpop.gremlin.process.traversal.TraversalSource#withRemote(*)` |
| ** `org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource#withRemote(*)` |
| ** `org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertyMapStep(Traversal.Admin, boolean, PropertyType, String...)` |
| ** `org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertyMapStep#isIncludeTokens()` |
| ** `org.apache.tinkerpop.gremlin.process.traversal.util.BytecodeUtil` |
| ** `org.apache.tinkerpop.gremlin.structure.util.star.StarGraph#builder()` |
| ** `org.apache.tinkerpop.gremlin.structure.util.star.StarGraph.Builder#create()` |
| * `gremlin-driver` |
| ** `org.apache.tinkerpop.gremlin.driver.Tokens#ARGS_SCRIPT_EVAL_TIMEOUT` |
| ** `org.apache.tinkerpop.gremlin.driver.Channelizer#createKeepAliveMessage()` |
| ** `org.apache.tinkerpop.gremlin.driver.Channelizer#supportsKeepAlive()` |
| ** `org.apache.tinkerpop.gremlin.driver.Cluster.Builder#keyCertChainFile(String)` |
| ** `org.apache.tinkerpop.gremlin.driver.Cluster.Builder#keyFile(String)` |
| ** `org.apache.tinkerpop.gremlin.driver.Cluster.Builder#keyPassword(String)` |
| ** `org.apache.tinkerpop.gremlin.driver.Cluster.Builder#maxWaitForSessionClose(Integer)` |
| ** `org.apache.tinkerpop.gremlin.driver.Cluster.Builder#trustCertificateChainFile(String)` |
| ** `org.apache.tinkerpop.gremlin.driver.handler.NioGremlinRequestEncoder` |
| ** `org.apache.tinkerpop.gremlin.driver.handler.NioGremlinResponseDecoder` |
| ** `org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversalSideEffects` |
| ** `org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteTraversal#getSideEffects()` |
| ** `org.apache.tinkerpop.gremlin.driver.simple.NioClient` |
| * `gremlin-python` |
| ** `org.apache.tinkerpop.gremlin.python.jsr223.*` |
| * `gremlin-server` |
| ** `org.apache.tinkerpop.gremlin.server.Settings.scriptEvaluationTimeout` |
| ** `org.apache.tinkerpop.gremlin.server.Settings.SslSettings.keyCertChainFile` |
| ** `org.apache.tinkerpop.gremlin.server.Settings.SslSettings.keyFile` |
| ** `org.apache.tinkerpop.gremlin.server.Settings.SslSettings.keyPassword` |
| ** `org.apache.tinkerpop.gremlin.server.Settings.SslSettings.trustCertificateChainFile` |
| ** `org.apache.tinkerpop.gremlin.server.ResponseHandlerContext` |
| ** `org.apache.tinkerpop.gremlin.server.channel.NioChannelizer` |
| ** `org.apache.tinkerpop.gremlin.server.handler.NioGremlinBinaryRequestDecoder` |
| ** `org.apache.tinkerpop.gremlin.server.handler.NioGremlinResponseFrameEncoder` |
| ** `org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.evalOpInternal(ResponseHandlerContext, Supplier, BindingSupplier)` |
| ** `org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.generateMetaData(ChannelHandlerContext, RequestMessage, ResponseStatusCode, Iterator)` |
| ** `org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(ResponseHandlerContext, Iterator)` |
| ** `org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.makeFrame(ChannelHandlerContext, RequestMessage, MessageSerializer, boolean, List, ResponseStatusCode, Map)` |
| ** `org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.makeFrame(Context, RequestMessage, MessageSerializer, boolean, List, ResponseStatusCode, Map)` |
| ** `org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.makeFrame(ResponseHandlerContext, RequestMessage, MessageSerializer, boolean, List, ResponseStatusCode, Map)` |
| ** `org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.makeFrame(ResponseHandlerContext, RequestMessage, MessageSerializer, boolean, List, ResponseStatusCode, Map, Map)` |
| ** `org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.onSideEffectSuccess(Graph, Context)` |
| ** `org.apache.tinkerpop.gremlin.server.util.SideEffectIterator` |
| * `neo4j-gremlin` |
| ** `org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph#getTrait()` |
| ** `org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph#CONFIG_META_PROPERTIES` |
| ** `org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph#CONFIG_MULTI_PROPERTIES` |
| ** `org.apache.tinkerpop.gremlin.neo4j.structure.trait.MultiMetaNeo4jTrait` |
| ** `org.apache.tinkerpop.gremlin.neo4j.structure.trait.NoMultiNoMetaNeo4jTrait` |
| ** `org.apache.tinkerpop.gremlin.neo4j.structure.trait.Neo4jTrait` |
| |
| Certain elements of the API were not or could not be deprecated in prior versions and were simply renamed for this |
| release: |
| |
| * `org.apache.tinkerpop.gremlin.driver.message.ResponseStatusCode#SERVER_ERROR_SCRIPT_EVALUATION` became `SERVER_ERROR_EVALUATION` |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2080[TINKERPOP-2080], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2231[TINKERPOP-2231], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2233[TINKERPOP-2233], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2239[TINKERPOP-2239], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2269[TINKERPOP-2269], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2273[TINKERPOP-2273], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2455[TINKERPOP-2455], |
| link:https://tinkerpop.apache.org/docs/3.5.0/upgrade/#_ssl_security[3.2.10 Upgrade Documentation for SSL] |
| |
| === Upgrading for Provider |
| |
| ==== Graph System Providers |
| |
| ===== Server Authorization |
| |
| Gremlin Server now supports an extension model that enables authorization. Graph providers are not required to |
| implement this functionality in any way, but it can be helpful for those graphs that wish to provide this functionality |
| through Gremlin Server. Graphs Systems may still choose to rely on their own native authorization functionality if |
| they so choose. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2389[TINKERPOP-2389], |
| link:https://tinkerpop.apache.org/docs/3.5.0/reference/#authorization[Reference Documentation], |
| link:https://tinkerpop.apache.org/docs/3.5.0/dev/provider/#_authentication_and_authorization[Provider Documentation] |
| |
| ===== ScalarMapStep |
| |
| `MapStep` had a single abstract method that needed to be implemented: |
| |
| [source,java] |
| ---- |
| protected abstract E map(final Traverser.Admin<S> traverser); |
| ---- |
| |
| This method made it easy to implement new implementations because it hid certain processing logic and made it so that |
| the implementer only had to reason about how to take the current object from the `Traverser` and transform it to a |
| new value. As 3.5.0 changed semantics around how `null` is processed, this method became a bit of a hindrance to the |
| more complex logic which those semantics entailed. Specifically, this method could not easily communicate to underlying |
| processing what a `null` might mean - is the `null` the end of the traversal stream or should the `null` be promoted |
| down the stream as a value to be processed. |
| |
| Interestingly, the method that enabled the handling of this more complex decision making already existed in |
| `AbstractStep`: |
| |
| [source,java] |
| ---- |
| protected Traverser.Admin<E> processNextStart() |
| ---- |
| |
| It returns a whole `Traverser` object and forces manual retrieval of the "next" `Traverser`. At this level it becomes |
| possible to make choices on `null` and return it if it should be propagated or dismiss it and return an |
| `EmptyTraverser`. To better accommodate the `MapStep` which provides the nice helper `map(Traverser)` method as well |
| as the more flexible version that doesn't need that infrastructure, `ScalarMapStep` was added to extend `MapStep`. The |
| `map(Traverser)` was then moved to `ScalarMapStep` and those steps that could rely on that helper method now extend |
| from it. All other steps of this sort still extend `MapStep` and directly implement `processNextStart()`. |
| |
| Providers will get compile errors if they extended `MapStep`. The easy solution will be to simply modify that code so |
| that their step instead extends `ScalarMapStep`. As a secondary task, providers should then examine their step |
| implementation to ensure that `null` semantics as presented in 3.5.0 apply properly. If they do not, then it is likely |
| that the step should simply implement `MapStep` directly and former `map(Traverser)` logic should be migrated to |
| `processNextStart()`. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2235[TINKERPOP-2235], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2099[TINKERPOP-2099] |
| |
| ===== TraversalStrategy Application |
| |
| The methodology for strategy application has been altered and the change is most easily described by example. Given a |
| traversal with the structure: |
| |
| [source,text] |
| ---- |
| a(b(),c(d())) |
| ---- |
| |
| Strategies were formerly applied in the following order: |
| |
| [source,text] |
| ---- |
| StrategyA on a |
| StrategyB on a |
| StrategyA on b |
| StrategyB on b |
| StrategyA on c |
| StrategyB on c |
| StrategyA on d |
| StrategyB on d |
| ---- |
| |
| This approach has always prevented strategies from performing global operations across the traversal and all decedents |
| effectively as children will not have been processed by preceding strategies yet. As of this release, the approach |
| has been altered to apply strategies as follows: |
| |
| [source,text] |
| ---- |
| StrategyA on a |
| StrategyA on b |
| StrategyA on c |
| StrategyA on d |
| StrategyB on a |
| StrategyB on b |
| StrategyB on c |
| StrategyB on d |
| ---- |
| |
| In this way, strategy B can check if it is being applied to the root traversal and if it is it knows that A has been |
| applied globally. |
| |
| This revised methodology could represent a breaking change for `TraversalStrategy` implementations if they somehow |
| relied on the old ordering of application. It may also present an opportunity to revise how a `TraversalStrategy` is |
| written to gain some processing benefit to the new order. Please be sure to review any custom strategies carefully |
| when upgrading to this version. |
| |
| As part of this change, there have been some adjustments to the `Traversal` and `Traversal.Admin` interfaces which have |
| helped to clarify coding intent. There is now an `isRoot()` method which determines whether or not the traversal has a |
| parent or not. Under revised semantics for 3.5.0, a traversal's parent must be an `EmptyStep` instance and should not |
| be `null`. With this change, provider `TraversalStrategy` implementations should be reviewed to evaluate if `isRoot()` |
| semantics cause any breaks in logic to existing code. |
| |
| In addition, `TraversalStrategies` now implements `Iterable` and exposes an `iterator()` method which may be preferred |
| over the old `toList()` style construction for getting the list of configured strategies. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-1568[TINKERPOP-1568], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2310[TINKERPOP-2310], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2311[TINKERPOP-2311] |
| |
| ===== Null Semantics |
| |
| Graph providers should take note of the changes to `null` semantics described in the "users" section of these upgrade |
| notes. As `null` is now acceptable as a `Traverser` object, this change may affect custom steps. Further note that |
| `null` now works more consistently with mutation steps and graph providers may need to include additional logic to |
| deal with those possible conditions. Please see the console sessions below which uses TinkerGraph to demonstrate the |
| current behavioral expectations. |
| |
| [source,text] |
| ---- |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>false |
| gremlin> g.addV(null).property(id, null).property('name',null) |
| ==>v[0] |
| gremlin> g.V().elementMap() |
| ==>[id:0,label:vertex] |
| ... |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>true |
| gremlin> g.addV(null).property(id, null).property('name',null) |
| ==>v[0] |
| gremlin> g.V().elementMap() |
| ==>[id:0,label:vertex,name:null] |
| ---- |
| |
| In the above example, `addV()` defaults to `Vertex.DEFAULT_LABEL`, the `id` is generated and setting the "name" |
| property to `null` results in the value not being set. If the property value is set to an actual value and then set |
| to `null` TinkerGraph will remove the property key all together: |
| |
| [source,text] |
| ---- |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>false |
| gremlin> g.addV().property('name','stephen') |
| ==>v[0] |
| gremlin> g.V().elementMap() |
| ==>[id:0,label:vertex,name:stephen] |
| gremlin> g.V().has('vertex','name','stephen').property('name',null) |
| ==>v[0] |
| gremlin> g.V().elementMap() |
| ==>[id:0,label:vertex] |
| ... |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>true |
| gremlin> g.addV().property('name','stephen') |
| ==>v[2] |
| gremlin> g.V().has('vertex','name','stephen').property('name',null) |
| ==>v[2] |
| gremlin> g.V().elementMap() |
| ==>[id:2,label:vertex,name:null] |
| ---- |
| |
| The above examples point out the default operations of TinkerGraph, but it can be configured to actually accept the |
| `null` as a property value and it is up to graph providers to decided how they wish to treat a `null` property value. |
| Providers should use the new `supportsNullPropertyValues()` feature to indicate to users how `null` is handled. |
| |
| For edges, the `label` still cannot be defaulted and must be specified, therefore: |
| |
| [source,text] |
| ---- |
| gremlin> g.V(0L).as('a').addE(null).to('a') |
| Label can not be null |
| Type ':help' or ':h' for help. |
| Display stack trace? [yN]n |
| gremlin> g.V(0L).as('a').addE(constant(null)).to('a') |
| Label can not be null |
| Type ':help' or ':h' for help. |
| Display stack trace? [yN] |
| ---- |
| |
| Also, edges have similar behavior to vertices when it comes to setting properties (again, the default configuration for |
| TinkerGraph is being used here): |
| |
| [source,text] |
| ---- |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>false |
| gremlin> g.addV().property('name','stephen') |
| ==>v[0] |
| gremlin> g.V().has('vertex','name','stephen').as('a').addE('knows').to('a').property(id,null).property('weight',null) |
| ==>e[2][0-knows->0] |
| gremlin> g.E().elementMap() |
| ==>[id:2,label:knows,IN:[id:0,label:vertex],OUT:[id:0,label:vertex]] |
| gremlin> g.E().property('weight',0.5) |
| ==>e[2][0-knows->0] |
| gremlin> g.E().elementMap() |
| ==>[id:2,label:knows,IN:[id:0,label:vertex],OUT:[id:0,label:vertex],weight:0.5] |
| gremlin> g.E().property('weight',null) |
| ==>e[2][0-knows->0] |
| gremlin> g.E().elementMap() |
| ==>[id:2,label:knows,IN:[id:0,label:vertex],OUT:[id:0,label:vertex]] |
| ... |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>true |
| gremlin> g.addV().property('name','stephen') |
| ==>v[8] |
| gremlin> g.V().has('vertex','name','stephen').as('a').addE('knows').to('a').property(id,null).property('weight',null) |
| ==>e[10][8-knows->8] |
| gremlin> g.E().elementMap() |
| ==>[id:10,label:knows,IN:[id:8,label:vertex],OUT:[id:8,label:vertex],weight:null] |
| gremlin> g.E().property('weight',0.5) |
| ==>e[10][8-knows->8] |
| gremlin> g.E().elementMap() |
| ==>[id:10,label:knows,IN:[id:8,label:vertex],OUT:[id:8,label:vertex],weight:0.5] |
| gremlin> g.E().property('weight',null) |
| ==>e[10][8-knows->8] |
| gremlin> g.E().elementMap() |
| ==>[id:10,label:knows,IN:[id:8,label:vertex],OUT:[id:8,label:vertex],weight:null] |
| ---- |
| |
| Graphs that support multi/meta-properties have some issues to consider as well as demonstrated with TinkerGraph: |
| |
| [source,text] |
| ---- |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>false |
| gremlin> g.addV().property(list,'foo',"x").property(list,"foo", null).property(list,'foo','bar') |
| ==>v[0] |
| gremlin> g.V().elementMap() |
| ==>[id:0,label:vertex,foo:bar] |
| gremlin> g.V().valueMap() |
| ==>[foo:[x,bar]] |
| gremlin> g.V().property('foo',null) |
| ==>v[0] |
| gremlin> g.V().valueMap(true) |
| ==>[id:0,label:vertex] |
| ... |
| gremlin> g.addV().property(list,'foo','bar','x',1,'y',null) |
| ==>v[0] |
| gremlin> g.V().properties('foo').valueMap(true) |
| ==>[id:1,key:foo,value:bar,x:1] |
| gremlin> g.V().properties('foo').property('x',null) |
| ==>vp[foo->bar] |
| gremlin> g.V().properties('foo').valueMap(true) |
| ==>[id:1,key:foo,value:bar] |
| ... |
| gremlin> g.getGraph().features().vertex().supportsNullPropertyValues() |
| ==>false |
| gremlin> g.addV().property(list,'foo',"x").property(list,"foo", null).property(list,'foo','bar') |
| ==>v[11] |
| gremlin> g.V().elementMap() |
| ==>[id:11,label:vertex,foo:bar] |
| gremlin> g.V().valueMap() |
| ==>[foo:[x,null,bar]] |
| ... |
| gremlin> g.addV().property(list,'foo','bar','x',1,'y',null) |
| ==>v[0] |
| gremlin> g.V().properties('foo').valueMap(true) |
| ==>[id:1,key:foo,value:bar,x:1,y:null] |
| gremlin> g.V().properties('foo').property('x',null) |
| ==>vp[foo->bar] |
| gremlin> g.V().properties('foo').valueMap(true) |
| ==>[id:1,key:foo,value:bar,x:null,y:null] |
| ---- |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2235[TINKERPOP-2235], |
| link:https://issues.apache.org/jira/browse/TINKERPOP-2099[TINKERPOP-2099] |
| |
| ===== AbstractOpProcessor API Change |
| |
| The `generateMetaData()` method was removed as it was deprecated in a previous version. There already was a preferred |
| method called `generateResultMetaData()` that took an extra `Settings` parameter. To fix compilation issues simply |
| replace implementations of the `generateMetaData()` method with `generateResultMetaData()`. Gremlin Server has |
| only been calling `generateResultMetaData()` since the deprecation, so this correction should be straightforward. |
| |
| ===== StoreStep and AggregateStep |
| |
| Note that `StoreStep` has been renamed to `AggregateLocalStep` and `AggregateStep` has been renamed to |
| `AggregateGlobalStep`. The renaming is important to consider if any custom `TraversalStrategies` have been written |
| that rely on the old step names. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2254[TINKERPOP-2254] |
| |
| ===== Session Close |
| |
| TinkerPop drivers no longer send the session "close" message to kill a session. The close of the connection itself |
| should be responsible for the close of the session. It is also expected that a session is bound to the client that |
| created it. Closing the session explicitly by closing the connection will act as a force close where transaction are |
| not explicitly rolled-back by Gremlin Server. Such transactions would be handled by the underlying graph system in the |
| manner that they provide. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2336[TINKERPOP-2336] |
| |
| ==== Graph Driver Providers |
| |
| ===== TraversalOpProcessor Side-effects |
| |
| `TraversalOpProcessor` no longer holds a cache of side-effects and more generally the entire side-effect protocol has |
| been removed and is no longer supported in the server or drivers. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2269[TINKERPOP-2269] |
| |
| ===== Close Message |
| |
| The functionality of the "close" message is no longer in place in Gremlin Server. Sending the message (from older |
| drivers for example) will simply result in a no-op on the server and the expected return of the `NO_CONTENT` message. |
| From 3.5.0 forward, drivers need not send this message to close the session and simply rely on the close of the |
| connection to kill the session. |
| |
| See: link:https://issues.apache.org/jira/browse/TINKERPOP-2336[TINKERPOP-2336] |