Support MySQL monitoring. (#9432)
diff --git a/.github/workflows/skywalking.yaml b/.github/workflows/skywalking.yaml
index 989404e..2561387 100644
--- a/.github/workflows/skywalking.yaml
+++ b/.github/workflows/skywalking.yaml
@@ -509,6 +509,8 @@
config: test/e2e-v2/cases/vm/prometheus-node-exporter/e2e.yaml
- name: So11y
config: test/e2e-v2/cases/so11y/e2e.yaml
+ - name: MySQL Prometheus
+ config: test/e2e-v2/cases/mysql/prometheus-mysql-exporter/e2e.yaml
- name: Rover with Istio Process 1.13.1
config: test/e2e-v2/cases/rover/process/istio/e2e.yaml
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index bb5f385..9fba81c 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -47,6 +47,7 @@
* Fix Zipkin trace query the max size of spans.
* Add `tls` and `https` component IDs for Network Profiling.
* Support Elasticsearch column alias for the compatibility between storage logicSharding model and no-logicSharding model.
+* Support MySQL monitoring.
#### UI
diff --git a/docs/en/setup/backend/backend-mysql-monitoring.md b/docs/en/setup/backend/backend-mysql-monitoring.md
new file mode 100644
index 0000000..d4e5fb9
--- /dev/null
+++ b/docs/en/setup/backend/backend-mysql-monitoring.md
@@ -0,0 +1,39 @@
+# MySQL monitoring
+SkyWalking leverages prometheus/mysqld_exporter for collecting metrics data from MySQL. It leverages OpenTelemetry Collector to transfer the metrics to
+[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
+
+## Data flow
+1. mysqld_exporter collect metrics data from MySQL.
+2. OpenTelemetry Collector fetches metrics from mysqld_exporter via Prometheus Receiver and pushes metrics to SkyWalking OAP Server via the OpenCensus GRPC Exporter.
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+
+## Set up
+1. Set up [mysqld_exporter](https://github.com/prometheus/mysqld_exporter#using-docker).
+2. Set up [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/getting-started/#docker). For details on Prometheus Receiver in OpenTelemetry Collector, refer to [here](../../../../test/e2e-v2/cases/mysql/prometheus-mysql-exporter/otel-collector-config.yaml).
+3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
+
+## MySQL Monitoring
+MySQL monitoring provides monitoring of the status and resources of the MySQL server. MySQL server as a `Service` in OAP, and land on the `Layer: MYSQL`.
+
+### MySQL Supported Metrics
+| Monitoring Panel | Unit | Metric Name | Description | Data Source |
+|-----|------|-----|-----|-----|
+| MySQL Uptime | day | meter_mysql_uptime | The MySQL startup time | mysqld_exporter|
+| Max Connections | | meter_mysql_max_connections | The max number of connections. | mysqld_exporter|
+| Innodb Buffer Pool Size | MB | meter_mysql_innodb_buffer_pool_size | The buffer pool size in Innodb engine | mysqld_exporter|
+| Thread Cache Size | | meter_mysql_thread_cache_size | The size of thread cache | mysqld_exporter|
+| Current QPS| | meter_mysql_qps | Queries Per Second | mysqld_exporter|
+| Current TPS | | meter_mysql_tps | Transactions Per Second | mysqld_exporter|
+| Commands Rate | | meter_mysql_commands_insert_rate <br/>meter_mysql_commands_select_rate<br />meter_mysql_commands_delete_rate<br />meter_mysql_commands_update_rate | The rate of total number of insert/select/delete/update executed by the current server | mysqld_exporter|
+| Threads | | meter_mysql_threads_connected<br />meter_mysql_threads_created<br />meter_mysql_threads_cached<br />meter_mysql_threads_running | The number of currently open connections(threads_connected) <br/> The number of threads created(threads_created) <br/> The number of threads in the thread cache(threads_cached) <br/> The number of threads that are not sleeping(threads_running) | mysqld_exporter|
+| Connects | | meter_mysql_connects_available<br />meter_mysql_connects_aborted | The number of available connections(connects_available)<br/>The number of MySQL instance connection rejections(connects_aborted)| mysqld_exporter|
+| Connection Errors | | meter_mysql_connection_errors_internal </br> meter_mysql_connection_errors_max_connections | Errors due to exceeding the max_connections(connection_errors_max_connections) </br>Error caused by internal system(connection_errors_internal) | mysqld_exporter|
+| Slow Queries Rate | | meter_mysql_slow_queries_rate | The rate of slow queries | mysqld_exporter|
+
+
+## Customizations
+You can customize your own metrics/expression/dashboard panel.
+The metrics definition and expression rules are found in `/config/otel-oc-rules/mysql.yaml`.
+The MySQL dashboard panel configurations are found in `/config/ui-initialized-templates/mysql`.
+
+
diff --git a/docs/en/setup/backend/opentelemetry-receiver.md b/docs/en/setup/backend/opentelemetry-receiver.md
index b6b3c58..b92c772 100644
--- a/docs/en/setup/backend/opentelemetry-receiver.md
+++ b/docs/en/setup/backend/opentelemetry-receiver.md
@@ -35,5 +35,6 @@
|k8s-cluster| Metrics of K8s cluster | otel-oc-rules/k8s-cluster.yaml | K8s kube-state-metrics -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
|k8s-node| Metrics of K8s cluster | otel-oc-rules/k8s-node.yaml | cAdvisor & K8s kube-state-metrics -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
|k8s-service| Metrics of K8s cluster | otel-oc-rules/k8s-service.yaml | cAdvisor & K8s kube-state-metrics -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
+|mysql| Metrics of MYSQL| otel-oc-rules/mysql.yaml | prometheus/mysqld_exporter -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
**Note**: You can also use OpenTelemetry exporter to transport the metrics to SkyWalking OAP directly. See [OpenTelemetry Exporter](./backend-meter.md#opentelemetry-exporter).
diff --git a/docs/menu.yml b/docs/menu.yml
index 33fc872..95119e7 100644
--- a/docs/menu.yml
+++ b/docs/menu.yml
@@ -153,6 +153,8 @@
path: "/en/setup/service-agent/server-agents"
- name: "Compatibility"
path: "/en/setup/service-agent/agent-compatibility"
+ - name: "Virtual Database"
+ path: "/en/setup/service-agent/virtual-database"
- name: "Service Mesh"
catalog:
- name: "Observe Service Mesh"
@@ -171,8 +173,8 @@
path: "/en/setup/service-agent/browser-agent"
- name: "Database Monitoring"
catalog:
- - name: "Virtual Database"
- path: "/en/setup/service-agent/virtual-database"
+ - name: "MySQL Monitoring"
+ path: "/en/setup/backend/backend-mysql-monitoring"
- name: "Self Observability"
catalog:
- name: "OAP Telemetry"
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java
index c0c7f55..624e184 100644
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java
@@ -48,6 +48,7 @@
Layer.OS_LINUX.name(),
Layer.MESH_CP.name(),
Layer.MESH_DP.name(),
+ Layer.MYSQL.name(),
Layer.K8S.name(),
Layer.BROWSER.name(),
Layer.SO11Y_OAP.name(),
diff --git a/oap-server/server-starter/src/main/resources/application.yml b/oap-server/server-starter/src/main/resources/application.yml
index 771df8a..52b34f7 100755
--- a/oap-server/server-starter/src/main/resources/application.yml
+++ b/oap-server/server-starter/src/main/resources/application.yml
@@ -339,7 +339,7 @@
selector: ${SW_OTEL_RECEIVER:default}
default:
enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"oc"}
- enabledOcRules: ${SW_OTEL_RECEIVER_ENABLED_OC_RULES:"istio-controlplane,k8s-node,oap,vm"}
+ enabledOcRules: ${SW_OTEL_RECEIVER_ENABLED_OC_RULES:"istio-controlplane,k8s-node,oap,vm,mysql"}
receiver-zipkin:
selector: ${SW_RECEIVER_ZIPKIN:-}
diff --git a/oap-server/server-starter/src/main/resources/otel-oc-rules/mysql.yaml b/oap-server/server-starter/src/main/resources/otel-oc-rules/mysql.yaml
new file mode 100644
index 0000000..b41aad7
--- /dev/null
+++ b/oap-server/server-starter/src/main/resources/otel-oc-rules/mysql.yaml
@@ -0,0 +1,82 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This will parse a textual representation of a duration. The formats
+# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
+# with days considered to be exactly 24 hours.
+# <p>
+# Examples:
+# <pre>
+# "PT20.345S" -- parses as "20.345 seconds"
+# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
+# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
+# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
+# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
+# "P-6H3M" -- parses as "-6 hours and +3 minutes"
+# "-P6H3M" -- parses as "-6 hours and -3 minutes"
+# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
+# </pre>
+filter: "{ tags -> tags.job_name == 'mysql-monitoring' }" # The OpenTelemetry job name
+expSuffix: tag({tags -> tags.host_name = 'mysql::' + tags.host_name}).service(['host_name'] , Layer.MYSQL)
+metricPrefix: meter_mysql
+metricsRules:
+ # mysql configurations
+ - name: uptime
+ exp: mysql_global_status_uptime
+ - name: innodb_buffer_pool_size
+ exp: mysql_global_variables_innodb_buffer_pool_size
+ - name: max_connections
+ exp: mysql_global_variables_max_connections
+ - name: thread_cache_size
+ exp: mysql_global_variables_thread_cache_size
+
+ # database throughput
+ - name: commands_insert_rate
+ exp: mysql_global_status_commands_total.tagEqual('command','insert').rate('PT1M')
+ - name: commands_select_rate
+ exp: mysql_global_status_commands_total.tagEqual('command','select').rate('PT1M')
+ - name: commands_delete_rate
+ exp: mysql_global_status_commands_total.tagEqual('command','delete').rate('PT1M')
+ - name: commands_update_rate
+ exp: mysql_global_status_commands_total.tagEqual('command','update').rate('PT1M')
+ - name: qps
+ exp: mysql_global_status_queries.rate('PT1M')
+ - name: tps
+ exp: mysql_global_status_commands_total.tagMatch('command','rollback|commit').rate('PT1M')
+
+ # connections
+ ## threads
+ - name: threads_connected
+ exp: mysql_global_status_threads_connected
+ - name: threads_created
+ exp: mysql_global_status_threads_created
+ - name: threads_running
+ exp: mysql_global_status_threads_running
+ - name: threads_cached
+ exp: mysql_global_status_threads_cached
+ ## connect
+ - name: connects_aborted
+ exp: mysql_global_status_aborted_connects
+ - name: connects_available
+ exp: mysql_global_variables_max_connections.sum(['host_name']) - mysql_global_status_threads_connected.sum(['host_name'])
+ - name: connection_errors_max_connections
+ exp: mysql_global_status_connection_errors_total.tagEqual('error','max_connection')
+ - name: connection_errors_internal
+ exp: mysql_global_status_connection_errors_total.tagEqual('error','internal')
+
+ # slow queries
+ - name: slow_queries_rate
+ exp: mysql_global_status_slow_queries.rate('PT1M')
+
diff --git a/oap-server/server-starter/src/main/resources/ui-initialized-templates/mysql/mysql-root.json b/oap-server/server-starter/src/main/resources/ui-initialized-templates/mysql/mysql-root.json
new file mode 100644
index 0000000..b28ae7f
--- /dev/null
+++ b/oap-server/server-starter/src/main/resources/ui-initialized-templates/mysql/mysql-root.json
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+[
+ {
+ "id":"b6e3763c-882d-417f-8556-3cf0187cb8db",
+ "configuration":{
+ "children":[
+ {
+ "x":0,
+ "y":3,
+ "w":24,
+ "h":43,
+ "i":"0",
+ "type":"Widget",
+ "graph":{
+ "type":"ServiceList",
+ "dashboardName":"MySQL-Service",
+ "fontSize":12,
+ "showXAxis":false,
+ "showYAxis":false,
+ "showGroup":true
+ }
+ },
+ {
+ "x":0,
+ "y":0,
+ "w":24,
+ "h":3,
+ "i":"1",
+ "type":"Text",
+ "graph":{
+ "fontColor":"blue",
+ "backgroundColor":"white",
+ "content":"Provide MySQL Server monitoring through OpenTelemetry's Prometheus Receiver",
+ "fontSize":14,
+ "textAlign":"left",
+ "url":"https://skywalking.apache.org/docs/main/latest/en/setup/backend/backend-mysql-monitoring/"
+ }
+ }
+ ],
+ "id":"b6e3763c-882d-417f-8556-3cf0187cb8db",
+ "layer":"MYSQL",
+ "entity":"All",
+ "name":"MySQL-Root",
+ "isRoot":true
+ }
+ }
+]
\ No newline at end of file
diff --git a/oap-server/server-starter/src/main/resources/ui-initialized-templates/mysql/mysql-service.json b/oap-server/server-starter/src/main/resources/ui-initialized-templates/mysql/mysql-service.json
new file mode 100644
index 0000000..7ad4b2a
--- /dev/null
+++ b/oap-server/server-starter/src/main/resources/ui-initialized-templates/mysql/mysql-service.json
@@ -0,0 +1,333 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+[
+ {
+ "id":"2981ef0e-1552-4cf0-a68c-da023b4a0fd8",
+ "configuration":{
+ "children":[
+ {
+ "x":0,
+ "y":0,
+ "w":6,
+ "h":11,
+ "i":"0",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValue"
+ ],
+ "metrics":[
+ "meter_mysql_uptime"
+ ],
+ "graph":{
+ "type":"Card",
+ "fontSize":40,
+ "textAlign":"center",
+ "showUnit":true
+ },
+ "metricConfig":[
+ {
+ "calculation":"secondToDay"
+ }
+ ],
+ "widget":{
+ "title":"MySQL Uptime (day)"
+ }
+ },
+ {
+ "x":0,
+ "y":11,
+ "w":6,
+ "h":14,
+ "i":"1",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValues"
+ ],
+ "metrics":[
+ "meter_mysql_qps"
+ ],
+ "graph":{
+ "type":"Line",
+ "step":false,
+ "smooth":false,
+ "showSymbol":false,
+ "showXAxis":true,
+ "showYAxis":true
+ },
+ "widget":{
+ "title":"Current QPS"
+ },
+ "value":"1",
+ "label":"1"
+ },
+ {
+ "x":12,
+ "y":0,
+ "w":6,
+ "h":11,
+ "i":"2",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValue"
+ ],
+ "metrics":[
+ "meter_mysql_innodb_buffer_pool_size"
+ ],
+ "graph":{
+ "type":"Card",
+ "fontSize":40,
+ "textAlign":"center",
+ "showUnit":true
+ },
+ "metricConfig":[
+ {
+ "calculation":"byteToMB"
+ }
+ ],
+ "widget":{
+ "title":"Innodb Buffer Pool Size (MB)"
+ }
+ },
+ {
+ "x":6,
+ "y":0,
+ "w":6,
+ "h":11,
+ "i":"3",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValue"
+ ],
+ "metrics":[
+ "meter_mysql_max_connections"
+ ],
+ "graph":{
+ "type":"Card",
+ "fontSize":40,
+ "textAlign":"center",
+ "showUnit":true
+ },
+ "widget":{
+ "title":"Max Connections"
+ }
+ },
+ {
+ "x":18,
+ "y":0,
+ "w":6,
+ "h":11,
+ "i":"4",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValue"
+ ],
+ "metrics":[
+ "meter_mysql_thread_cache_size"
+ ],
+ "graph":{
+ "type":"Card",
+ "fontSize":40,
+ "textAlign":"center",
+ "showUnit":true
+ },
+ "widget":{
+ "title":"Thread Cache Size"
+ }
+ },
+ {
+ "x":6,
+ "y":11,
+ "w":6,
+ "h":14,
+ "i":"5",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValues"
+ ],
+ "metrics":[
+ "meter_mysql_tps"
+ ],
+ "graph":{
+ "type":"Line",
+ "step":false,
+ "smooth":false,
+ "showSymbol":false,
+ "showXAxis":true,
+ "showYAxis":true
+ },
+ "widget":{
+ "title":"Current TPS"
+ },
+ "value":"5",
+ "label":"5"
+ },
+ {
+ "x":12,
+ "y":11,
+ "w":6,
+ "h":14,
+ "i":"6",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValues",
+ "readMetricsValues",
+ "readMetricsValues",
+ "readMetricsValues"
+ ],
+ "metrics":[
+ "meter_mysql_commands_insert_rate",
+ "meter_mysql_commands_select_rate",
+ "meter_mysql_commands_delete_rate",
+ "meter_mysql_commands_update_rate"
+ ],
+ "graph":{
+ "type":"Area",
+ "opacity":0.4,
+ "showXAxis":true,
+ "showYAxis":true
+ },
+ "widget":{
+ "title":"Commands Trend"
+ },
+ "value":"6",
+ "label":"6"
+ },
+ {
+ "x":6,
+ "y":25,
+ "w":6,
+ "h":13,
+ "i":"7",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValues",
+ "readMetricsValues",
+ "readMetricsValues",
+ "readMetricsValues"
+ ],
+ "metrics":[
+ "meter_mysql_threads_cached",
+ "meter_mysql_threads_running",
+ "meter_mysql_threads_created",
+ "meter_mysql_threads_connected"
+ ],
+ "graph":{
+ "type":"Area",
+ "opacity":0.4,
+ "showXAxis":true,
+ "showYAxis":true
+ },
+ "widget":{
+ "title":"Threads"
+ },
+ "value":"7",
+ "label":"7"
+ },
+ {
+ "x":0,
+ "y":25,
+ "w":6,
+ "h":13,
+ "i":"8",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValues",
+ "readMetricsValues"
+ ],
+ "metrics":[
+ "meter_mysql_connects_available",
+ "meter_mysql_connects_aborted"
+ ],
+ "graph":{
+ "type":"Line",
+ "step":false,
+ "smooth":false,
+ "showSymbol":false,
+ "showXAxis":true,
+ "showYAxis":true
+ },
+ "widget":{
+ "title":"Connects"
+ },
+ "value":"8",
+ "label":"8"
+ },
+ {
+ "x":18,
+ "y":11,
+ "w":6,
+ "h":14,
+ "i":"10",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValues"
+ ],
+ "metrics":[
+ "meter_mysql_slow_queries_rate"
+ ],
+ "graph":{
+ "type":"Line",
+ "step":false,
+ "smooth":false,
+ "showSymbol":false,
+ "showXAxis":true,
+ "showYAxis":true
+ },
+ "widget":{
+ "title":"Slow Quries Trend"
+ },
+ "value":"10",
+ "label":"10"
+ },
+ {
+ "x":12,
+ "y":25,
+ "w":6,
+ "h":13,
+ "i":"11",
+ "type":"Widget",
+ "metricTypes":[
+ "readMetricsValues",
+ "readMetricsValues"
+ ],
+ "metrics":[
+ "meter_mysql_connection_errors_internal",
+ "meter_mysql_connection_errors_max_connections"
+ ],
+ "graph":{
+ "type":"Line",
+ "step":false,
+ "smooth":false,
+ "showSymbol":false,
+ "showXAxis":true,
+ "showYAxis":true
+ },
+ "widget":{
+ "title":"Connection Errors"
+ },
+ "value":"11",
+ "label":"11"
+ }
+ ],
+ "layer":"MYSQL",
+ "entity":"Service",
+ "name":"MySQL-Service",
+ "id":"2981ef0e-1552-4cf0-a68c-da023b4a0fd8"
+ }
+ }
+]
diff --git a/test/e2e-v2/cases/mysql/expected/metrics-has-value.yml b/test/e2e-v2/cases/mysql/expected/metrics-has-value.yml
new file mode 100644
index 0000000..d9c4985
--- /dev/null
+++ b/test/e2e-v2/cases/mysql/expected/metrics-has-value.yml
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{- contains . }}
+- key: {{ notEmpty .key }}
+ value: {{ ge .value 1 }}
+{{- end }}
\ No newline at end of file
diff --git a/test/e2e-v2/cases/mysql/expected/service.yml b/test/e2e-v2/cases/mysql/expected/service.yml
new file mode 100644
index 0000000..f499c9d
--- /dev/null
+++ b/test/e2e-v2/cases/mysql/expected/service.yml
@@ -0,0 +1,24 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{- contains . }}
+- id: {{ b64enc "mysql::showcase" }}.1
+ name: mysql::showcase
+ shortname: showcase
+ group: mysql
+ normal: true
+ layers:
+ - MYSQL
+{{- end }}
\ No newline at end of file
diff --git a/test/e2e-v2/cases/mysql/mysql-cases.yaml b/test/e2e-v2/cases/mysql/mysql-cases.yaml
new file mode 100644
index 0000000..7f4a79d
--- /dev/null
+++ b/test/e2e-v2/cases/mysql/mysql-cases.yaml
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is used to show how to write configuration files and can be used to test.
+
+ cases:
+ # service list
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service ls
+ expected: expected/service.yml
+ # metrics
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_uptime --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_innodb_buffer_pool_size --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_max_connections --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_thread_cache_size --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_commands_select_rate --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_commands_insert_rate --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_commands_update_rate --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_commands_delete_rate --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_qps --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_tps --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_threads_connected --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_threads_created --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_threads_running --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_connects_aborted --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=meter_mysql_connects_available --service-name=mysql::showcase |yq e 'to_entries' -
+ expected: expected/metrics-has-value.yml
\ No newline at end of file
diff --git a/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/docker-compose.yml b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/docker-compose.yml
new file mode 100644
index 0000000..1d60477
--- /dev/null
+++ b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/docker-compose.yml
@@ -0,0 +1,77 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: '2.1'
+
+services:
+ oap:
+ extends:
+ file: ../../../script/docker-compose/base-compose.yml
+ service: oap
+ environment:
+ SW_OTEL_RECEIVER: default
+ SW_OTEL_RECEIVER_ENABLED_OC_RULES: mysql
+ ports:
+ - 12800
+ mysql:
+ image: mysql:8.0.30
+ networks:
+ - e2e
+ ports:
+ - 3306
+ environment:
+ - MYSQL_ROOT_PASSWORD=password
+ - MYSQL_DATABASE=test2
+
+ mysql-load:
+ image: mysql:8.0.30
+ depends_on:
+ - otel-collector
+ networks:
+ - e2e
+ entrypoint: bash
+ volumes:
+ - ../prometheus-mysql-exporter:/docker
+ command:
+ - -c
+ - "mysql -h mysql -uroot -ppassword < /docker/mock.sql"
+
+ mysql-service:
+ image: prom/mysqld-exporter:v0.14.0
+ ports:
+ - 9104
+ environment:
+ - DATA_SOURCE_NAME=root:password@(mysql:3306)/
+ networks:
+ - e2e
+ depends_on:
+ - mysql
+
+
+ otel-collector:
+ image: otel/opentelemetry-collector:0.50.0
+ networks:
+ - e2e
+ command: [ "--config=/etc/otel-collector-config.yaml" ]
+ volumes:
+ - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
+ expose:
+ - 55678
+ depends_on:
+ oap:
+ condition: service_healthy
+
+networks:
+ e2e:
\ No newline at end of file
diff --git a/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/e2e.yaml b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/e2e.yaml
new file mode 100644
index 0000000..87b91a1
--- /dev/null
+++ b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/e2e.yaml
@@ -0,0 +1,38 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is used to show how to write configuration files and can be used to test.
+
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 20m
+ init-system-environment: ../../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+
+verify:
+ retry:
+ count: 20
+ interval: 3s
+ cases:
+ - includes:
+ - ../mysql-cases.yaml
+
\ No newline at end of file
diff --git a/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/mock.sql b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/mock.sql
new file mode 100644
index 0000000..a3d6070
--- /dev/null
+++ b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/mock.sql
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+CREATE DATABASE IF NOT EXISTS test;
+USE test;
+CREATE TABLE IF NOT EXISTS `t1`(
+ `te1` VARCHAR(100) NOT NULL
+)ENGINE=InnoDB DEFAULT CHARSET=utf8;
+SET GLOBAL event_scheduler = 1;
+
+CREATE EVENT `event_1`
+ON SCHEDULE EVERY 1 SECOND
+DO INSERT INTO t1 values('test');
+
+CREATE EVENT `event_2`
+ON SCHEDULE EVERY 1 SECOND
+DO UPDATE t1 SET `te1` = 1;
+
+CREATE EVENT `event_3`
+ON SCHEDULE EVERY 1 SECOND
+DO DELETE FROM t1;
+
+CREATE EVENT `event_4`
+ON SCHEDULE EVERY 1 SECOND
+DO COMMIT;
diff --git a/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/otel-collector-config.yaml b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/otel-collector-config.yaml
new file mode 100644
index 0000000..3ac29a8
--- /dev/null
+++ b/test/e2e-v2/cases/mysql/prometheus-mysql-exporter/otel-collector-config.yaml
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+receivers:
+ prometheus:
+ config:
+ scrape_configs:
+ - job_name: 'mysql-monitoring'
+ scrape_interval: 5s
+ static_configs:
+ - targets: ['mysql-service:9104']
+ labels:
+ host_name: showcase
+processors:
+ batch:
+
+exporters:
+ opencensus:
+ endpoint: oap:11800
+ tls:
+ insecure: true
+service:
+ pipelines:
+ metrics:
+ receivers:
+ - prometheus
+ processors:
+ - batch
+ exporters:
+ - opencensus
\ No newline at end of file
