COUNTER-type metric names do not have the _total suffix #1154
Description
Rules configuration:
rules:
- pattern: kafka.network<type=RequestMetrics, name=(.+), (.+)=(.+), (.+)=(.+)><>Count
name: kafka_network_RequestMetrics_$1
type: COUNTER
labels:
"$2": "$3"
"$4": "$5"
- pattern: kafka.network<type=RequestMetrics, name=(.+), (.+)=(.+), (.+)=(.+)><>(.+)Rate
name: kafka_network_RequestMetrics_$1
type: GAUGE
labels:
"$2": "$3"
"$4": "$5"
rate: "$6"Using the above configuration, metrics with the same name but different types are collected, as shown below:
MatchedRule{name='kafka_network_RequestMetrics_RequestsPerSec', matchName='kafka.network<type=RequestMetrics, name=RequestsPerSec, request=OffsetCommit, version=7><>Count: 13589', type='COUNTER' ... }
MatchedRule{name='kafka_network_RequestMetrics_RequestsPerSec', matchName='kafka.network<type=RequestMetrics, name=RequestsPerSec, request=OffsetCommit, version=7><>OneMinuteRate: 0.1664303423094177', type='GAUGE' ... }
Ideally, for metrics with a type of COUNTER, the metric name should include the _total suffix, resulting in kafka_network_RequestMetrics_RequestsPerSec_total. However, in practice, the suffix is missing.
After debugging the code, I identified the issue in the following section:
// io.prometheus.jmx.MatchedRuleToMetricSnapshotsConverter
public static MetricSnapshots convert(List<MatchedRule> matchedRules) {
Map<String, List<MatchedRule>> rulesByPrometheusMetricName = new HashMap<>();
for (MatchedRule matchedRule : matchedRules) {
List<MatchedRule> matchedRulesWithSameName =
rulesByPrometheusMetricName.computeIfAbsent(
matchedRule.name, name -> new ArrayList<>());
matchedRulesWithSameName.add(matchedRule);
}
/* Code omitted */
MetricSnapshots.Builder result = MetricSnapshots.builder();
for (List<MatchedRule> rulesWithSameName : rulesByPrometheusMetricName.values()) {
result.metricSnapshot(convertRulesWithSameName(rulesWithSameName));
}
return result.build();
}
private static MetricSnapshot convertRulesWithSameName(List<MatchedRule> rulesWithSameName) {
boolean labelsUnique = isLabelsUnique(rulesWithSameName);
switch (getType(rulesWithSameName)) {
case "COUNTER":
/* Code omitted */
case "GAUGE":
/* Code omitted */
default:
/* Code omitted */
}
}
/** If all rules have the same type, that type is returned. Otherwise, "UNKNOWN" is returned. */
private static String getType(List<MatchedRule> rulesWithSameName) {
if (rulesWithSameName.stream().map(rule -> rule.type).distinct().count() == 1) {
return rulesWithSameName.get(0).type;
}
return "UNKNOWN";
}Here, metrics with the same name are grouped together, and when the getType method is called, it returns "UNKNOWN". This causes COUNTER type metrics to be incorrectly processed and results in the _total suffix not being added.