tree 66638986f6d341b6cb5e3f3ae7a65eb06ef57082
parent ee9823d0dd96fe3e4705a9c1499834a748d91cf9
author Jiawei Wang <jiawei.wang@cloudera.com> 1575497782 -0600
committer Impala Public Jenkins <impala-public-jenkins@cloudera.com> 1576386271 +0000

IMPALA-7550: Add documentation to profile counters

This change changes the way developers define profile counters by
generating counters from a counter registry. All the profile counters
will be register there first and then used in the same way as before.
By doing so, we will be able to manage profile counters in a way that
we can define profile counters documentation. For example:

Declaration:
PROFILE_DEFINE_COUNTER(NumBackends, STABLE_HIGH, TUnit::UNIT,
    "Number of backends running this query.");

Initialization:
COUNTER_SET(PROFILE_NumBackends.Instantiate(query_profile_), num_backends);

This shall be how we define a NumBackends counter. It follows with
its significance, type, description in the declaration part.

Users now will be able to view profile counters documentation under
query_profile page, there is a Profile Documentation button which
leads to /profile_docs.

More details:
This commit did the following refactors on profile counters.
1. Add a singleton registry for runtime profile counters prototypes,
similiar to what Kudu does for metrics. This allows us to generate
profile documentation for all counters from the code. We add
/profile_docs and a correspoding UI for the documentation of profile
counters.

2. Profile counters are also annotated with their significance to users.
* STABLE_HIGH - High level and stable counters, always useful for measuring
query performance and status. Counters that everyone is interested. should
rarely change and if it does we will make some effort to notify users.

* STABLE_LOW - Low level and stable counters. Interesting counters to monitor
 and analyze by machine. It will probably be interesting under some
 circumstances for users.

* Unstable - Unstable but useful. Useful to understand query performance,
but subject to change, particularly if the implementation changes.
E.g. MaterializeTupleTimer

* Debug -  Debugging counters. Generally not useful to users of Impala,
 the main use case is low-level debugging. Can be hidden to reduce noise
 for most consumers of profiles.

3. We have around 250 counters. This commit did the replacement in
scan-node and hdfs-scan-node-base and coordinator.

Concers:
The downside is that we will have duplicate comments of query profiles
both in the header file and the .cc file.
Additionally a (arguably good) limitation is that profile counter names
need to be unique.

Change-Id: Idc03faddb27754001290bb6d899840e2cbe7ccb7
Reviewed-on: http://gerrit.cloudera.org:8080/14776
Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>