Add streaming HTTP response support to gremlin-driver (#3419)

Adds streaming GraphBinary response deserialization to the Java driver.
Instead of buffering the entire HTTP response body before processing,
results are now delivered to the ResultSet as they arrive from the server.
This reduces time-to-first-result for large result sets without requiring
application code changes.

Design
------

The streaming pipeline replaces HttpObjectAggregator + HttpGremlinResponseDecoder
with a single HttpStreamingResponseHandler that feeds HTTP content chunks to a
reader thread via a BlockingQueue<ByteBuf>. The reader deserializes GraphBinary
items one at a time using an InputStream-backed Buffer adapter, allowing existing
TypeSerializer implementations to work unchanged.

New classes:
- HttpStreamingResponseHandler: Netty handler orchestrating the streaming lifecycle
- ByteBufQueueInputStream: bridges event loop to reader thread via blocking queue
- InputStreamBuffer: read-only Buffer over InputStream for TypeSerializers
- GraphBinaryStreamResponseReader: pull-based deserializer on a dedicated thread

Non-GraphBinary serializers automatically fall back to the buffered pipeline.

Connection lifecycle
--------------------

Connection pool return is driven by wire-level completion (LAST_CONTENT_READ_RESPONSE),
not application-level completion (ResultSet.markComplete). This ensures all HTTP bytes
are consumed before the connection is reused, preventing framing corruption.
An AtomicBoolean guard makes returnToPool() idempotent across concurrent error paths.

Error handling races between the event loop and reader thread are resolved by marking
errors on the ResultSet before signaling end-of-stream — CompletableFuture's
single-completion semantics guarantee the correct error surfaces to the caller.

Server-side fixes
-----------------

Prevents connection corruption from double-LastHttpContent when writeError() is
called after response streaming has already terminated. Guards writeError() against
writes in FINISHED/ERROR state and defers state transitions until after serialization.
18 files changed
tree: 4673b0deb64a94468b0d3486250e7ee8c2636b4d
  1. .github/
  2. .skills/
  3. bin/
  4. data/
  5. docker/
  6. docs/
  7. gremlin-annotations/
  8. gremlin-console/
  9. gremlin-core/
  10. gremlin-dotnet/
  11. gremlin-driver/
  12. gremlin-examples/
  13. gremlin-go/
  14. gremlin-groovy/
  15. gremlin-js/
  16. gremlin-language/
  17. gremlin-python/
  18. gremlin-server/
  19. gremlin-shaded/
  20. gremlin-test/
  21. gremlin-tools/
  22. gremlin-util/
  23. hadoop-gremlin/
  24. licenses/
  25. spark-gremlin/
  26. tinkergraph-gremlin/
  27. .asf.yaml
  28. .dockerignore
  29. .gitignore
  30. .mailmap
  31. .travis.install-maven.sh
  32. AGENTS.md
  33. CHANGELOG.asciidoc
  34. CONTRIBUTING.md
  35. LICENSE
  36. NOTICE
  37. pom.xml
  38. README.md
  39. source-release.xml
README.md

Apache TinkerPop

Maven Central NuGet PyPI npm Go.Dev

Codecov

TinkerPop3

Apache TinkerPop™ is a graph computing framework for both graph databases (OLTP) and graph analytic systems (OLAP). It provides the Gremlin graph traversal language, drivers, and tools for working with property graphs across a wide variety of underlying data systems.

Project overview

TinkerPop defines a common interface and language (Gremlin) so that applications can work against many different graph systems without being locked into a single vendor. It includes a reference in‑memory graph database (TinkerGraph), Gremlin Server, language variants, and a rich collection of recipes and documentation.

Key resources:

Building and Testing

TinkerPop uses Maven and requires Java 11/17 for proper building and proper operations. To build, execute unit tests and package Gremlin Console/Server run:

mvn clean install

Please see the Building on Windows section for Windows-specific build instructions.

The zip distributions can be found in the following directories:

  1. gremlin-server/target
  2. gremlin-console/target

Please see the CONTRIBUTING.md file for more detailed information and options for building, test running and developing TinkerPop.

Get Started

Download Gremlin Console (compatible with Java 11/17) and unzip to a directory, then:

$ bin/gremlin.sh

         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----
plugin activated: tinkerpop.server
plugin activated: tinkerpop.utilities
plugin activated: tinkerpop.tinkergraph
gremlin> Gremlin.version()
==>3.8.0
gremlin> graph = TinkerFactory.createModern()
==>tinkergraph[vertices:6 edges:6]
gremlin> g = traversal().with(graph)
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> g.V().has('name','vadas').valueMap()
==>[name:[vadas], age:[27]]

From the Gremlin Console, you can connect to a TinkerGraph instance and run your first traversals. Refer to the Getting Started for detailed walkthroughs and examples.

Using TinkerPop

Common ways to use TinkerPop include:

  • Embedding TinkerGraph in your application for development, testing, or lightweight graph workloads.
  • Connecting to a Gremlin‑enabled graph database via drivers or Gremlin Server.
  • Running Gremlin traversals from the JVM, or via Gremlin Language Variants (Python, .NET, JavaScript, Go, etc.).
  • Using the Gremlin Console for interactive exploration, debugging, and learning.

See the Reference Documentation for supported features, configuration options, and other details.

Documentation

The full TinkerPop documentation is published on the project website and is also maintained in this repository under docs/src/ as AsciiDoc “books.”

When changing or adding documentation, follow the existing AsciiDoc structure in docs/src/** and update the relevant index.asciidoc files so new content is included in the build.

Contributing

Contributions to Apache TinkerPop are welcome. The Developer Documentation and contributing guide describe how to set up a development environment, run tests, and submit changes.

Before opening a pull request, please:

  • Discuss larger changes on the appropriate Apache mailing list.
  • Ensure tests pass locally and, where appropriate, add new tests and documentation.
  • Update CHANGELOG.asciidoc and upgrade docs when behavior or public APIs change.

Using AI and IDE assistants

If you use AI coding agents or IDE assistants when working on TinkerPop, please consult AGENTS.md. That file summarizes:

  • Recommended build and test commands.
  • Code style and testing conventions.
  • “Do and don’t” guidance specific to automated tools.

AGENTS.md is a concise guide for tools and tool‑using contributors, while CONTRIBUTING.md and the Developer Documentation remain the canonical sources for project policies and processes.

License

Apache TinkerPop is an open source project of The Apache Software Foundation and is licensed under the Apache License, Version 2.0. See the LICENSE file in this repository for details.