Fix stale TID errors in VLE path materialization (#2551)
Four problems with the TID-based VLE cache and its version counters, all from
798917c2 ("VLE cache + performance improvements").
1. A path bound by MATCH could not be projected once the same statement deleted
its endpoints (#2549):
MATCH p = (n0)<-[:R*..2]-(n1) DETACH DELETE n0, n1 RETURN p
ERROR: get_vertex_entry_properties: stale TID - ...
That commit replaced the properties Datum in vertex_entry and edge_entry
with a TID fetched lazily at projection. cypher_delete() advances
es_snapshot->curcid past every delete, so a path's own endpoints fail the
visibility test by the time they are read; before, properties were captured
at cache build and a later delete could not affect them.
Such a tuple is still physically present, the deleting transaction having
not committed, so it is read anyway and the path reports the properties it
was matched with. The relaxation is narrow: only a tuple deleted by our own
transaction qualifies, and only while the row still carries the cached
entity, so a recycled line pointer cannot be substituted. Any other
unreachable TID still raises the error, keeping a real invalidation bug
visible. Properties are detoasted under the buffer pin, and the buffer is
now released on the failing path too, since heap_fetch is called with
keep_buf, which leaves it pinned when only visibility fails.
2. VACUUM FULL and CLUSTER rewrite the heap, moving every cached TID, and
announce themselves through no trigger and no version counter. A cached
context then resolved stale TIDs against the new file, giving the stale TID
error or "could not read block", which never reaches AGE's guard.
Both are now intercepted in ag_ProcessUtility_hook as TRUNCATE already was,
and the database-wide forms invalidate every tracked graph. Plain VACUUM and
ANALYZE do not move tuples and are ignored.
3. NULL properties were reported as a stale TID. Label tables are created with
properties NOT NULL, so a NULL means the table was altered out from under
AGE; that is now said plainly rather than blamed on the cache.
4. Version counter slots were never released, making the table a tally of every
graph ever mutated rather than of those that exist. A server cycling graphs
filled it, then warned on every mutation and fell back to snapshot
invalidation. drop_graph() now releases its slot and a freed slot is reused;
a new occupant seeds its version above every value the table has issued, so
a context cached for the previous occupant cannot compare equal. The cap
moves 128 -> 256, about 4 KB of shared memory at 16 bytes per entry; lookups
scan only the entries in use, so unused slots cost nothing.
Both accessors now share one helper, and hardcoded column numbers give way to
the Anum_ag_label_* constants.
cypher_vle gains 14 cases: the reported query, a fan-out that fails
if the result depends on which row is projected first, partial and edge-only
deletes, the edge-list projection, an edge property constraint reaching the
accessor during traversal, zero-length bounds, self-loops, labelled vertices, a
multi-hop chain, a delete from an earlier statement that must not be
resurrected, savepoint and transaction rollback, and out-of-line TOAST asserted
set-identical to a live read.
age_global_graph covers CLUSTER and all three VACUUM FULL spellings (named,
parenthesised, database-wide), plus plain VACUUM, ANALYZE and FULL false which
must not invalidate; NULL properties on a vertex and an edge; 260
create/drop cycles that must stay silent; a rolled-back drop; and a graph
recreated under a dropped name.
Verified on PostgreSQL 18.4 and 18.6: clean build, no warnings; installcheck
43/43 on 18.4 before and after, run twice; and 43/43 on 18.6 with
--enable-cassert, reporting no assertion failure or resource leak.
Fixes #2549
modified: regress/expected/age_global_graph.out
modified: regress/expected/cypher_vle.out
modified: regress/sql/age_global_graph.sql
modified: regress/sql/cypher_vle.sql
modified: src/backend/catalog/ag_catalog.c
modified: src/backend/commands/graph_commands.c
modified: src/backend/utils/adt/age_global_graph.c
modified: src/include/utils/age_global_graph.h
Co-authored-by: GitHub Copilot (Claude Opus 5) <noreply@github.com>Apache AGE is an extension for PostgreSQL that enables users to leverage a graph database on top of the existing relational databases. AGE is an acronym for A Graph Extension and is inspired by Bitnine's AgensGraph, a multi-model database fork of PostgreSQL. The basic principle of the project is to create a single storage that handles both the relational and graph data model so that the users can use the standard ANSI SQL along with openCypher, one of the most popular graph query languages today. There is a strong need for cohesive, easy-to-implement multi-model databases. As an extension of PostgreSQL, AGE supports all the functionalities and features of PostgreSQL while also offering a graph model to boot.
Apache AGE is :
Refer to our latest Apache AGE documentation to learn about installation, features, built-in functions, and Cypher queries.
Install the following essential libraries according to each OS. Building AGE from the source depends on the following Linux libraries:
yum install gcc glibc glib-common readline readline-devel zlib zlib-devel flex bison
dnf install gcc glibc bison flex readline readline-devel zlib zlib-devel
sudo apt-get install build-essential libreadline-dev zlib1g-dev flex bison
Apache AGE is intended to be simple to install and run. It can be installed with Docker and other traditional ways.
You will need to install an AGE compatible version of Postgres, for now AGE supports Postgres 11, 12, 13, 14, 15, 16, 17 & 18. Supporting the latest versions is on AGE roadmap.
You can use a package management that your OS provides to download PostgreSQL.
sudo apt install postgresql
You can download the Postgres source code and install your own instance of Postgres. You can read instructions on how to install from source code for different versions on the official Postgres Website.
Clone the github repository or download the download an official release. Run the pg_config utility and check the version of PostgreSQL. Currently, only PostgreSQL versions 11, 12, 13, 14, 15, 16, 17 & 18 are supported. If you have any other version of Postgres, you will need to install PostgreSQL version 11, 12, 13, 14, 15, 16, 17 & 18.
pg_config
Run the following command in the source code directory of Apache AGE to build and install the extension.
make install
If the path to your Postgres installation is not in the PATH variable, add the path in the arguments:
make PG_CONFIG=/path/to/postgres/bin/pg_config install
docker pull apache/age
docker run \ --name age \ -p 5455:5432 \ -e POSTGRES_USER=postgresUser \ -e POSTGRES_PASSWORD=postgresPW \ -e POSTGRES_DB=postgresDB \ -d \ apache/age
docker exec -it age psql -d postgresDB -U postgresUser
For every connection of AGE you start, you will need to load the AGE extension.
CREATE EXTENSION age;
LOAD 'age';
SET search_path = ag_catalog, "$user", public;
ag_catalog ownershipAGE installs all of its objects into the ag_catalog schema. Install AGE (CREATE EXTENSION age) before granting the CREATE privilege on the database to other roles. A role that can create schemas could otherwise pre-create ag_catalog and own it; CREATE EXTENSION age therefore refuses to install when ag_catalog already exists and is owned by a different role. If you hit that error, drop the stray schema (DROP SCHEMA ag_catalog CASCADE) or transfer its ownership to the installing role, then retry.
If you are using AGE from a database client that does not default to autocommit — most commonly psycopg v3 or JDBC — you must understand how PostgreSQL‘s transaction semantics apply to AGE’s setup and DDL-like functions. Otherwise, you may see graphs or labels that appear to be created successfully, but are not visible from new connections.
This is not a bug in AGE — it is standard PostgreSQL behavior. AGE's DDL-like functions write to the catalog, and catalog writes only become visible to other sessions after the enclosing transaction is committed.
| Statement | Scope | Needs commit to be visible elsewhere? |
|---|---|---|
LOAD 'age' | Session-local (loads the .so into the current backend) | No |
SET search_path = ag_catalog, "$user", public | Session-local | No |
SELECT create_graph('g') | Writes to ag_graph and creates a schema | Yes |
SELECT create_vlabel('g', 'L') / create_elabel(...) | Writes to ag_label and creates a table | Yes |
SELECT drop_graph('g', true) / drop_label(...) | Writes to catalog | Yes |
SELECT load_labels_from_file(...) / load_edges_from_file(...) | Writes to catalog + data | Yes |
cypher('g', $$ CREATE (:L {...}) $$) | Writes data | Yes |
In a client that defaults to autocommit (e.g. psql), every statement commits automatically, so this is never noticed. In a non-autocommit client, the first statement you run implicitly opens a transaction that stays open until you call commit(), rollback(), or close the connection.
The common pitfall is that with connection.transaction(): in psycopg does not start a new top-level transaction when one is already open — it creates a savepoint inside the existing outer transaction. Releasing a savepoint is not a commit, so your create_graph write stays invisible to other sessions until the outer transaction is explicitly committed.
import psycopg params = {"host": "localhost", "port": 5432, "user": "postgres", "password": "pw", "dbname": "mydb"} # --- First connection --- conn = psycopg.connect(**params) conn.execute("LOAD 'age'") # implicitly opens a txn conn.execute("SET search_path = ag_catalog, '$user', public") with conn.transaction(), conn.cursor() as cur: # <-- SAVEPOINT, not a real txn cur.execute("SELECT * FROM create_graph('my_graph')") # outer transaction is STILL OPEN here conn.close() # outer transaction is rolled back on close → my_graph is gone # --- New connection --- conn = psycopg.connect(**params) conn.execute("LOAD 'age'") conn.execute("SET search_path = ag_catalog, '$user', public") with conn.cursor() as cur: cur.execute("SELECT name FROM ag_graph;") # 'my_graph' is NOT in the results
commit() after setupconn = psycopg.connect(**params) conn.execute("LOAD 'age'") conn.execute("SET search_path = ag_catalog, '$user', public") conn.commit() # <-- closes the implicit outer txn with conn.transaction(), conn.cursor() as cur: cur.execute("SELECT * FROM create_graph('my_graph')") # this transaction block is now top-level and commits on exit conn.close()
conn = psycopg.connect(**params, autocommit=True) conn.execute("LOAD 'age'") conn.execute("SET search_path = ag_catalog, '$user', public") conn.execute("SELECT * FROM create_graph('my_graph')") # commits immediately conn.close()
You can also toggle autocommit at runtime with conn.set_autocommit(True).
JDBC connections also default to autocommit true per the JDBC spec, but many frameworks (Spring, etc.) flip it off. If you are running AGE DDL-like calls from JDBC, either:
connection.setAutoCommit(true); // ... LOAD 'age'; SET search_path ...; SELECT create_graph(...);
or keep autocommit off and explicitly commit after DDL-like calls:
stmt.execute("LOAD 'age'"); stmt.execute("SET search_path = ag_catalog, \"$user\", public;"); stmt.execute("SELECT create_graph('my_graph');"); connection.commit(); // make the graph visible to other sessions
If an AGE call creates, drops, or modifies a graph, label, vertex, edge, or property, it is a transactional write. In a non-autocommit client, it will not be visible to other sessions until you explicitly
commit().
To create a graph, use the create_graph function located in the ag_catalog namespace.
SELECT create_graph('graph_name');
To create a single vertex with label and properties, use the CREATE clause.
SELECT * FROM cypher('graph_name', $$ CREATE (:label {property:"Node A"}) $$) as (v agtype);
SELECT * FROM cypher('graph_name', $$ CREATE (:label {property:"Node B"}) $$) as (v agtype);
To create an edge between two nodes and set its properties:
SELECT * FROM cypher('graph_name', $$ MATCH (a:label), (b:label) WHERE a.property = 'Node A' AND b.property = 'Node B' CREATE (a)-[e:RELTYPE {property:a.property + '<->' + b.property}]->(b) RETURN e $$) as (e agtype);
And to query the connected nodes:
SELECT * from cypher('graph_name', $$
MATCH (V)-[R]-(V2)
RETURN V,R,V2
$$) as (V agtype, R agtype, V2 agtype);
Starting with Apache AGE is very simple. You can easily select your platform and incorporate the relevant SDK into your code.
Apache AGE Viewer is a user interface for Apache AGE that provides visualization and exploration of data. This web visualization tool allows users to enter complex graph queries and explore the results in graph and table forms. Apache AGE Viewer is enhanced to proceed with extensive graph data and discover insights through various graph algorithms. Apache AGE Viewer will become a graph data administration and development platform for Apache AGE to support multiple relational databases: https://github.com/apache/age-viewer.
This is a visualization tool. After installing AGE Extension, you may use this tool to get access to the visualization features.
You can also get help from these videos.
You can improve ongoing efforts or initiate new ones by sending pull requests to this repository. Also, you can learn from the code review process, how to merge pull requests, and from code style compliance to documentation by visiting the Apache AGE official site - Developer Guidelines. Send all your comments and inquiries to the user mailing list, users@age.apache.org.