blob: 98cba2ad9e72c0437f542747ab529729b6d08f0b [file] [view]
---
layout: global
title: ASOF JOIN
displayTitle: ASOF JOIN
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---
### Description
`ASOF JOIN` combines each row from the left relation with at most one row from the
right relation. The match is the closest row on the right that satisfies a required
`MATCH_CONDITION` comparison and an optional `ON` or `USING` filter.
`ASOF JOIN` is asymmetric: the left relation drives per-row lookups into the right,
and reversing the operands changes the result.
`ASOF JOIN` is disabled by default. Set `spark.sql.join.asofJoin.enabled` to `true`
to enable the syntax. When disabled, `ASOF JOIN` fails at parse time with
`UNSUPPORTED_FEATURE.ASOF_JOIN`.
Only `INNER` (the default) and `LEFT OUTER` join types are supported.
### Syntax
```sql
[ asof_join_type ] ASOF JOIN right_table_reference join_criteria
asof_join_type
{ [ INNER ] |
LEFT [ OUTER ] }
join_criteria
MATCH_CONDITION ( expr1 comparison_operator expr2 )
[ ON boolean_expression | USING ( column_name [, ...] ) ]
comparison_operator
{ >= | > | <= | < }
```
### Parameters
* **right_table_reference**
The table reference on the right of the join; searched once per left row.
* **asof_join_type**
The outer form of the `ASOF JOIN`.
* `INNER` (default): drops left rows with no qualifying match.
* `LEFT [ OUTER ]`: retains such rows with `NULL` right-side columns.
* **join_criteria**
A required `MATCH_CONDITION` clause and an optional `ON` or `USING` clause that
filters the join input before the closest-match selection. When both `ON` and
`USING` are omitted, every left row is matched against the entire right table on
the `MATCH_CONDITION` alone.
* **MATCH_CONDITION ( expr1 comparison_operator expr2 )**
Specifies the ordered comparison that determines the closest match. One of
`expr1` and `expr2` must reference only the left table; the other must
reference only the right table.
* **expr1**, **expr2**
Scalar expressions of an orderable, mutually comparable type. Each must be
deterministic; subqueries, aggregate functions, window functions, and
generator functions are not permitted. `STRUCT` operands compare
lexicographically field-by-field, enabling multi-column match conditions
(see Examples).
* **comparison_operator**
One of `>=`, `>`, `<=`, `<`. Equality operators (`=` and `<>`) are not
permitted.
* **ON boolean_expression**
An expression with a return type of `BOOLEAN` that specifies how rows from the
two relations are matched before the closest-match selection. If the result is
`true`, the rows are considered a match. See [JOIN](sql-ref-syntax-qry-select-join.html).
* **USING ( column_name [, ...] )**
Matches rows by comparing equality for the list of columns, which must exist in
both relations. See [JOIN](sql-ref-syntax-qry-select-join.html).
### Notes
* **Direction of match.** Let *L* be the operand of `MATCH_CONDITION` that references
the left table and *R* the operand that references the right table. The operator
determines which row on the right is closest:
* *L* `>=` *R* (equivalently *R* `<=` *L*): the right row with the largest *R* not
greater than *L* (last-preceding).
* *L* `>` *R*: the largest *R* strictly less than *L* (last strictly-preceding).
* *L* `<=` *R*: the smallest *R* not less than *L* (first-following).
* *L* `<` *R*: the smallest *R* strictly greater than *L* (first strictly-following).
* **NULL.** A `NULL` in either operand never satisfies the `MATCH_CONDITION`
comparison. Left rows whose operand is `NULL` are dropped under `INNER ASOF` and
retained with `NULL` right-side columns under `LEFT ASOF`.
* **Ties.** If multiple right rows share the same *R* value that is closest to *L*
after applying `ON`/`USING`, the choice among tied rows is not deterministic. Add a
tie-breaker either as an equi-key in `ON`/`USING` or as a trailing field of a
`STRUCT`-valued `MATCH_CONDITION` (see Examples).
### Examples
```sql
SET spark.sql.join.asofJoin.enabled=true;
CREATE OR REPLACE TEMP VIEW trades(trade_time, symbol, quantity) AS
VALUES (TIMESTAMP '2026-06-29 10:00:05', 'AAPL', 100),
(TIMESTAMP '2026-06-29 10:00:11', 'AAPL', 200),
(TIMESTAMP '2026-06-29 10:00:12', 'MSFT', 50),
(TIMESTAMP '2026-06-29 09:59:59', 'GOOG', 30);
CREATE OR REPLACE TEMP VIEW quotes(quote_time, symbol, bid_price) AS
VALUES (TIMESTAMP '2026-06-29 10:00:00', 'AAPL', 180.10),
(TIMESTAMP '2026-06-29 10:00:07', 'AAPL', 180.15),
(TIMESTAMP '2026-06-29 10:00:10', 'AAPL', 180.20),
(TIMESTAMP '2026-06-29 10:00:08', 'MSFT', 420.50);
-- Attach the most recent (last-preceding) quote to each trade.
-- The GOOG trade has no matching quote and is dropped by INNER ASOF JOIN.
SELECT t.trade_time, t.symbol, t.quantity, q.bid_price
FROM trades t
ASOF JOIN quotes q
MATCH_CONDITION (t.trade_time >= q.quote_time)
ON t.symbol = q.symbol;
+-------------------+------+--------+---------+
| trade_time|symbol|quantity|bid_price|
+-------------------+------+--------+---------+
|2026-06-29 10:00:05| AAPL| 100| 180.10|
|2026-06-29 10:00:11| AAPL| 200| 180.20|
|2026-06-29 10:00:12| MSFT| 50| 420.50|
+-------------------+------+--------+---------+
-- Preserve unmatched left rows with LEFT ASOF JOIN.
SELECT t.trade_time, t.symbol, q.bid_price
FROM trades t
LEFT ASOF JOIN quotes q
MATCH_CONDITION (t.trade_time >= q.quote_time)
ON t.symbol = q.symbol;
+-------------------+------+---------+
| trade_time|symbol|bid_price|
+-------------------+------+---------+
|2026-06-29 09:59:59| GOOG| NULL|
|2026-06-29 10:00:05| AAPL| 180.10|
|2026-06-29 10:00:11| AAPL| 180.20|
|2026-06-29 10:00:12| MSFT| 420.50|
+-------------------+------+---------+
-- USING is equivalent to ON symbol equality.
SELECT trade_time, symbol, bid_price
FROM trades
ASOF JOIN quotes
MATCH_CONDITION (trades.trade_time >= quotes.quote_time)
USING (symbol);
-- Find the next scheduled maintenance window for each alert.
-- A <= operator selects the first-following right row.
CREATE OR REPLACE TEMP VIEW alerts(alert_time, host) AS
VALUES (TIMESTAMP '2026-06-29 10:00:00', 'db-01');
CREATE OR REPLACE TEMP VIEW maintenance(window_start, host) AS
VALUES (TIMESTAMP '2026-06-29 08:00:00', 'db-01'),
(TIMESTAMP '2026-06-29 12:00:00', 'db-01');
SELECT a.alert_time, a.host, m.window_start
FROM alerts a
ASOF JOIN maintenance m
MATCH_CONDITION (a.alert_time <= m.window_start)
ON a.host = m.host;
+-------------------+-----+-------------------+
| alert_time| host| window_start|
+-------------------+-----+-------------------+
|2026-06-29 10:00:00|db-01|2026-06-29 12:00:00|
+-------------------+-----+-------------------+
-- Multi-column MATCH_CONDITION using tuple comparison. Both sides carry a
-- (time, seq) pair; ties on time are broken lexicographically by seq.
CREATE OR REPLACE TEMP VIEW deploys(deploy_ts, seq, service, version) AS
VALUES (TIMESTAMP '2026-06-29 10:00:00', 1, 'api', 'v1.0'),
(TIMESTAMP '2026-06-29 10:00:00', 2, 'api', 'v1.1'),
(TIMESTAMP '2026-06-29 10:05:00', 1, 'api', 'v1.2');
CREATE OR REPLACE TEMP VIEW requests(req_ts, seq, service) AS
VALUES (TIMESTAMP '2026-06-29 10:00:00', 5, 'api'),
(TIMESTAMP '2026-06-29 10:03:00', 1, 'api');
SELECT r.req_ts, r.seq, d.version
FROM requests r
ASOF JOIN deploys d
MATCH_CONDITION ((r.req_ts, r.seq) >= (d.deploy_ts, d.seq))
ON r.service = d.service;
+-------------------+---+-------+
| req_ts|seq|version|
+-------------------+---+-------+
|2026-06-29 10:00:00| 5| v1.1|
|2026-06-29 10:03:00| 1| v1.1|
+-------------------+---+-------+
```
### Related Statements
* [SELECT](sql-ref-syntax-qry-select.html)
* [JOIN](sql-ref-syntax-qry-select-join.html)