2. Monitoring Kyuubi - Server Metrics

Kyuubi has a configurable metrics system based on the Dropwizard Metrics Library. This allows users to report Kyuubi metrics to a variety of kyuubi.metrics.reporters. The metrics provide instrumentation for specific activities and Kyuubi server.

2.1. Configurations

The metrics system is configured via $KYUUBI_HOME/conf/kyuubi-defaults.conf.

Key Default Meaning Type Since
kyuubi.metrics
.enabled
true
Set to true to enable kyuubi metrics system
boolean
1.2.0
kyuubi.metrics
.reporters
JSON
A comma separated list for all metrics reporters
  • CONSOLE - ConsoleReporter which outputs measurements to CONSOLE periodically.
  • JMX - JmxReporter which listens for new metrics and exposes them as MBeans.
  • JSON - JsonReporter which outputs measurements to json file periodically.
  • PROMETHEUS - PrometheusReporter which exposes metrics in prometheus format.
  • SLF4J - Slf4jReporter which outputs measurements to system log periodically.
seq
1.2.0
kyuubi.metrics
.console.interval
PT5S
How often should report metrics to console
duration
1.2.0
kyuubi.metrics.json
.interval
PT5S
How often should report metrics to json file
duration
1.2.0
kyuubi.metrics.json
.location
metrics
Where the json metrics file located
string
1.2.0
kyuubi.metrics
.prometheus.path
/metrics
URI context path of prometheus metrics HTTP server
string
1.2.0
kyuubi.metrics
.prometheus.port
10019
Prometheus metrics HTTP server port
int
1.2.0
kyuubi.metrics.slf4j
.interval
PT5S
How often should report metrics to SLF4J logger
duration
1.2.0

2.2. Metrics

These metrics include:

Metrics Prefix Metrics Suffix Type Since Description
kyuubi
.exec.pool
.threads.alive
gauge 1.2.0
threads keepAlive in the backend executive thread pool
kyuubi
.exec.pool
.threads.active
gauge 1.2.0
threads active in the backend executive thread pool
kyuubi
.connection.total
counter 1.2.0
cumulative connection count
kyuubi
.connection.opened
gauge 1.2.0
current active connection count
kyuubi
.connection.opened
${user} counter 1.2.0
current active connections count requested by a ${user}
kyuubi
.connection.failed
counter 1.2.0
cumulative failed connection count
kyuubi
.connection.failed
${user} counter 1.2.0
cumulative failed connections for a ${user}
kyuubi
.operation.total
counter 1.5.0
cumulative opened operation count
kyuubi
.operation.total
${operationType} counter 1.5.0
cumulative opened count for the operation ${operationType}
kyuubi
.operation.opened
gauge 1.5.0
current opened operation count
kyuubi
.operation.opened
${operationType} counter 1.5.0
current opened count for the operation ${operationType}
kyuubi
.operation.failed
${operationType}
.${errorType}
counter 1.5.0
cumulative failed count for the operation ${operationType} with a particular ${errorType}, e.g. execute_statement.AnalysisException
kyuubi
.engine.total
counter 1.2.0
cumulative created engines
kyuubi
.engine.timeout
counter 1.2.0
cumulative timeout engines
kyuubi
.engine.failed
${user} counter 1.2.0
cumulative explicitly failed engine count for a ${user}
kyuubi
.engine.failed
${errorType} counter 1.2.0
cumulative explicitly failed engine count for a particular ${errorType}, e.g. ClassNotFoundException
kyuubi
.backend_service
.open_session_ms
histogram 1.5.0
kyuubi backend service openSession method time cost histogram statistics
kyuubi
.backend_service
.close_session_ms
histogram 1.5.0
kyuubi backend service closeSession method time cost histogram statistics
kyuubi
.backend_service
.get_info_ms
histogram 1.5.0
kyuubi backend service getInfo method time cost histogram statistics
kyuubi
.backend_service
.execute_statement_ms
histogram 1.5.0
kyuubi backend service executeStatement method time cost histogram statistics
kyuubi
.backend_service
.get_type_info_ms
histogram 1.5.0
kyuubi backend service getTypeInfo method time cost histogram statistics
kyuubi
.backend_service
.get_catalogs_ms
histogram 1.5.0
kyuubi backend service getCatalogs method time cost histogram statistics
kyuubi
.backend_service
.get_schemas_ms
histogram 1.5.0
kyuubi backend service getSchemas method time cost histogram statistics
kyuubi
.backend_service
.get_tables_ms
histogram 1.5.0
kyuubi backend service getTables method time cost histogram statistics
kyuubi
.backend_service
.get_table_types_ms
histogram 1.5.0
kyuubi backend service getTableTypes method time cost histogram statistics
kyuubi
.backend_service
.get_columns_ms
histogram 1.5.0
kyuubi backend service getColumns method time cost histogram statistics
kyuubi
.backend_service
.get_functions_ms
histogram 1.5.0
kyuubi backend service getFunctions method time cost histogram statistics
kyuubi
.backend_service
.get_operation_status_ms
histogram 1.5.0
kyuubi backend service getOperationStatus method time cost histogram statistics
kyuubi
.backend_service
.cancel_operation_ms
histogram 1.5.0
kyuubi backend service cancelOperation method time cost histogram statistics
kyuubi
.backend_service
.close_operation_ms
histogram 1.5.0
kyuubi backend service closeOperation method time cost histogram statistics
kyuubi
.backend_service
.get_result_set_metadata_ms
histogram 1.5.0
kyuubi backend service getResultSetMetadata method time cost histogram statistics
kyuubi
.backend_service
.fetch_results_ms
histogram 1.5.0
kyuubi backend service fetchResults method time cost histogram statistics

Before v1.5.0, if you use these metrics:

  • kyuubi.statement.total

  • kyuubi.statement.opened

  • kyuubi.statement.failed.${errorType}

Since v1.5.0, you can use the following metrics to replace:

  • kyuubi.operation.total.ExecuteStatement

  • kyuubi.operation.opened.ExecuteStatement

  • kyuubi.operation.failed.ExecuteStatement.${errorType}