commit 01ca8c3f2bb38bc8b738d86d3d88aa8d1ffaad87
author Alexey Serbin <alexey@apache.org> Wed Mar 09 14:57:45 2022 -0800
committer Alexey Serbin <alexey@apache.org> Mon Apr 04 18:22:03 2022 +0000
tree d7e69ca1c0bb75fc03db23e87b43665d83f78215
parent fb6dc60621aa49f214b06d70527860c6a26c0e15

[tools] range rebalancing for 'kudu cluster rebalance'

This patch adds range rebalancing functionality into the
'kudu cluster rebalance' CLI tool.  The implementation is rather an
MVP: the range rebalancing can now be performed only for a single
table per run.  As far as I can see, there is a room for improvement
since it's possible to perform range-aware replica movements even
during standard whole cluster rebalancing.

Below is two snapshots of distribution of the range-specific tablet
replicas in a cluster.  Those are produced by running the tool with
extra --report_only --output_replica_distribution_details flags
before and after range rebalancing for a single table:

  kudu cluster rebalance \
     --enable_range_rebalancing \
     --tables=default.loadgen_auto_6800f4ec4e164b2b8e42db7b5044df09 \
     127.0.0.1:8765

before:
========================================================================

Table: abb2bbf8b4ff4bc0989bc82c78d4ae2b

Number of tablet replicas at servers for each range
 Max Skew | Total Count |   Range Start Key
----------+-------------+----------------------
 8        | 8           |
 8        | 8           | ff80000000000001fff4
 8        | 8           | ff80000000000003ffe8
 8        | 8           | ff80000000000005ffdc

Range start key: ''
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 8
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 0
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 0
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 0

Range start key: 'ff80000000000001fff4'
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 0
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 8
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 0
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 0

Range start key: 'ff80000000000003ffe8'
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 0
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 0
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 8
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 0

Range start key: 'ff80000000000005ffdc'
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 0
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 0
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 0
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 8

after:
========================================================================

Table: abb2bbf8b4ff4bc0989bc82c78d4ae2b

Number of tablet replicas at servers for each range
 Max Skew | Total Count |   Range Start Key
----------+-------------+----------------------
 0        | 8           |
 0        | 8           | ff80000000000001fff4
 0        | 8           | ff80000000000003ffe8
 0        | 8           | ff80000000000005ffdc

Range start key: ''
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2

Range start key: 'ff80000000000001fff4'
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2

Range start key: 'ff80000000000003ffe8'
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2

Range start key: 'ff80000000000005ffdc'
               UUID               | Server address | Replica Count
----------------------------------+----------------+---------------
 15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
 3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
 324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
 a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2

Change-Id: I7d2e19266e993f5e2ae13ba18d323c83db30eac1
Reviewed-on: http://gerrit.cloudera.org:8080/18294
Tested-by: Alexey Serbin <alexey@apache.org>
Reviewed-by: Andrew Wong <awong@cloudera.com>