)]}'
{
  "commit": "0d92be00dd1ee012ea481aaddf81f7225fd971e5",
  "tree": "b141d56a907e00e4802de4adbfc5c2bdf2b3371b",
  "parents": [
    "809692d05306e6da52e91cc75f57788e952ee065"
  ],
  "author": {
    "name": "Calvin Kirs",
    "email": "guoqiang@selectdb.com",
    "time": "Sat Jun 27 10:15:28 2026 +0800"
  },
  "committer": {
    "name": "guoqiang",
    "email": "guoqiang@selectdb.com",
    "time": "Wed Jul 01 18:20:04 2026 +0800"
  },
  "message": "[fix](nereids) partition topn opt requires chosen window func partition key to be a subset of co-located ones (#64764)\n\nWhen a single `LogicalWindow` holds multiple window functions, a filter\nlike `rn \u003c\u003d k` may be converted into a `partitionTopN` and pushed\n**below the whole window node**. The generated `partitionTopN` keeps the\nper-partition top-k of the *chosen* window function, so it prunes the\ninput rows that are **shared by all co-located window functions**.\n\nThis is only correct when the chosen window function\u0027s partition key is\na **subset** of every other co-located window function\u0027s partition key\n(i.e. the chosen one is *coarser*). Otherwise the pruning drops rows the\nother windows still need and produces a wrong result, e.g.\n\n```sql\n-- independent partitions: chosen rn1(g1) prunes rows rn2(g2) needs\nrow_number() over (partition by g1 order by ord_key) as rn1,  -- chosen\nrow_number() over (partition by g2 order by ord_key) as rn2\n\n-- chosen is finer than a co-located window: chosen rn(c1,c2) prunes rows rk(c1) needs\nrow_number() over (partition by c1, c2 order by c3) as rn,    -- chosen (finer)\nrank()       over (partition by c1 order by c3) as rk         -- coarser\n```\n\n`LogicalWindow.getPushDownWindowFuncAndLimit()` already required the\n**order keys** of all co-located window functions to be compatible\n(#56622), but it never checked the **partition keys**. This PR\nadditionally requires\n`windowFunc.getPartitionKeys().containsAll(chosenWindowFunc.getPartitionKeys())`\nfor every co-located window function; otherwise the partition-topn\noptimization is disabled.\n\nThe pruning keeps the per-`P0` (chosen partition) top-k by the order\nkey. For another window function `W` partitioned by `P1`:\n\n- **`P0 ⊆ P1` (chosen coarser) → safe.** Any row that could change `W`\u0027s\nvalue for a surviving row `r` is in the same `P1` partition with a\nsmaller order value; being in the same `P1` it is also in the same `P0`\npartition with order `\u003c\u003d r`, so its `P0`-rank is within top-k and it is\n**kept**. Nothing `W` needs is pruned.\n- **`P0 ⊋ P1` or independent → unsafe.** A finer/independent `P0` can\ndrop a row that ranks early in `P1`, corrupting `W`.\n\nSo equality is unnecessarily strict; the precise safe condition is the\n**subset** relation, expressed with `containsAll` (which also makes the\ncheck order-insensitive in the partition-key list, matching the set\nsemantics of `PARTITION BY`).\n\nSingle window, multiple windows with the **same** partition key, and the\n**subset** case above all keep firing. Example where the chosen\n`rank(partition by g1)` is coarser than `row_number(partition by g1,\ng2)`, so a `VPartitionTopN(partition by g1)` is still generated:\n\n```sql\nselect id, g1, g2, ord_key, rk, rn\nfrom (\n  select id, g1, g2, ord_key,\n    rank()       over (partition by g1 order by ord_key) as rk,   -- chosen (coarser)\n    row_number() over (partition by g1, g2 order by ord_key) as rn\n  from multi_window_cases\n) q\nwhere rk \u003c\u003d 2;\n```\n\n```\n  6:VANALYTIC          partition by: g1, g2   order by: ord_key   \u003c- computes rn\n  4:VANALYTIC          partition by: g1       order by: ord_key   \u003c- computes rk\n  1:VPartitionTopN     partition by: g1       order by: ord_key   partition limit: 2   \u003c- safe (g1 ⊆ {g1,g2})\n  0:VOlapScanNode\n```\n\n```sql\nCREATE TABLE multi_window_cases (\n  id      INT,\n  g1      VARCHAR(8),\n  g2      VARCHAR(8),\n  ord_key INT,\n  amt     INT\n)\nDUPLICATE KEY(id)\nDISTRIBUTED BY HASH(id) BUCKETS 1\nPROPERTIES (\"replication_num\" \u003d \"1\");\n\nINSERT INTO multi_window_cases VALUES\n(1,\u0027A\u0027,\u0027X\u0027,1,10),(2,\u0027A\u0027,\u0027X\u0027,2,20),(3,\u0027A\u0027,\u0027Y\u0027,3,30),(4,\u0027B\u0027,\u0027X\u0027,4,40),\n(5,\u0027B\u0027,\u0027Y\u0027,5,50),(6,\u0027B\u0027,\u0027Y\u0027,6,60),(7,\u0027C\u0027,\u0027X\u0027,7,70),(8,\u0027C\u0027,\u0027Z\u0027,8,80);\n\nSELECT id, g1, g2, ord_key, rn1, rn2\nFROM (\n  SELECT id, g1, g2, ord_key,\n    row_number() OVER (PARTITION BY g1 ORDER BY ord_key) AS rn1,\n    row_number() OVER (PARTITION BY g2 ORDER BY ord_key) AS rn2\n  FROM multi_window_cases\n) q\nWHERE rn1 \u003c\u003d 1\nORDER BY id;\n```\n\nWrong result (before), `rn2` should be `1,3,4`:\n\n```\n+----+----+----+---------+-----+-----+\n| id | g1 | g2 | ord_key | rn1 | rn2 |\n+----+----+----+---------+-----+-----+\n|  1 | A  | X  |       1 |   1 |   1 |\n|  4 | B  | X  |       4 |   1 |   2 |   \u003c- wrong (should be 3)\n|  7 | C  | X  |       7 |   1 |   3 |   \u003c- wrong (should be 4)\n+----+----+----+---------+-----+-----+\n```\n\nCorrect result (after, matches MySQL 8.4):\n\n```\n+------+------+------+---------+-----+-----+\n| id   | g1   | g2   | ord_key | rn1 | rn2 |\n+------+------+------+---------+-----+-----+\n|    1 | A    | X    |       1 |   1 |   1 |\n|    4 | B    | X    |       4 |   1 |   3 |\n|    7 | C    | X    |       7 |   1 |   4 |\n+------+------+------+---------+-----+-----+\n```\n\nA `VPartitionTopN(partition by g1)` is inserted **below both analytic\nnodes**, so it prunes rows before `rn2` is computed:\n\n```\n  8:VSORT              order by: id\n  7:VANALYTIC          partition by: g2,  order by: ord_key   \u003c- computes rn2\n  |  predicates: (rn1 \u003c\u003d 1)\n  6:VSORT              order by: g2, ord_key\n  4:VANALYTIC          partition by: g1,  order by: ord_key   \u003c- computes rn1\n  3:VSORT              order by: g1, ord_key\n  1:VPartitionTopN     partition by: g1,  order by: ord_key   \u003c- prunes input (WRONG)\n  0:VOlapScanNode\n```\n\nNo `VPartitionTopN`; both window functions are computed over the full\ninput and `rn1 \u003c\u003d 1` stays as an ordinary predicate above them:\n\n```\n  7:VSORT              order by: id\n  6:VANALYTIC          partition by: g2,  order by: ord_key   \u003c- computes rn2 (full input)\n  |  predicates: (rn1 \u003c\u003d 1)\n  5:VSORT              order by: g2, ord_key\n  3:VANALYTIC          partition by: g1,  order by: ord_key   \u003c- computes rn1\n  2:VSORT              order by: g1, ord_key\n  0:VOlapScanNode\n```\n\nFix wrong result of multiple window functions\n(`row_number`/`rank`/`dense_rank`) with incompatible partition keys when\na top-n filter (e.g. `rn \u003c\u003d k`) is applied; the partition-topn pushdown\nis now restricted to the cases where it is provably safe (the chosen\nfunction\u0027s partition key is a subset of the co-located ones).\n\n- [x] Test\n- [x] Regression test\n(`regression-test/suites/query_p0/partition_topn/check_partitionkey.groovy`,\n`.../push_down_filter_through_window/push_down_multi_filter_through_window.groovy`)\n- [x] Unit test (`GeneratePartitionTopnFromWindowTest`:\n`testMultipleWindowsWithDifferentPartitions`,\n`testMultipleWindowsSubsetPartitionGeneratesTopn`)\n\n- [x] Behavior changed:\n- [x] Function behavior changed (returns correct results for the cases\nabove)\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "72067d73ef8611e4e0c11be28d43770ee784d681",
      "old_mode": 33188,
      "old_path": "fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalWindow.java",
      "new_id": "fc0217a1b3e78c23e1efa92eefd6107e7903d77c",
      "new_mode": 33188,
      "new_path": "fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalWindow.java"
    },
    {
      "type": "modify",
      "old_id": "00d47bb8b78839f57f72c247f9b27a7c3239da99",
      "old_mode": 33188,
      "old_path": "fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/GeneratePartitionTopnFromWindowTest.java",
      "new_id": "9c3993d4d2fae8bca53a300ffe7d0187f4d60099",
      "new_mode": 33188,
      "new_path": "fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/GeneratePartitionTopnFromWindowTest.java"
    },
    {
      "type": "add",
      "old_id": "0000000000000000000000000000000000000000",
      "old_mode": 0,
      "old_path": "/dev/null",
      "new_id": "c33a0912cc4fdc3482b745e5009b541107c6d29d",
      "new_mode": 33188,
      "new_path": "regression-test/data/nereids_p0/partition_topn/check_partitionkey.out"
    },
    {
      "type": "add",
      "old_id": "0000000000000000000000000000000000000000",
      "old_mode": 0,
      "old_path": "/dev/null",
      "new_id": "14457badd2c128e9eea10d78d3174d8e48196028",
      "new_mode": 33188,
      "new_path": "regression-test/suites/nereids_p0/partition_topn/check_partitionkey.groovy"
    },
    {
      "type": "modify",
      "old_id": "015a6e7fae7d4784d290f45fab2be10bbdbb7e63",
      "old_mode": 33188,
      "old_path": "regression-test/suites/nereids_rules_p0/push_down_filter_through_window/push_down_multi_filter_through_window.groovy",
      "new_id": "607391bab0b86210f4c3e2feb3b46546c1acd0dc",
      "new_mode": 33188,
      "new_path": "regression-test/suites/nereids_rules_p0/push_down_filter_through_window/push_down_multi_filter_through_window.groovy"
    }
  ]
}