Reverse DNS lookups

[josipanas]

Hi,

We are using jmx_prometheus_httpserver version 0.20.0 in our product with TLS enabled.
Recently, we noticed PTR (reverse DNS lookups) requests occurring during JMX Exporter liveness checks:
Liveness: http-get http://:15020/app-health/locator-jmx-exporter/livez delay=90s timeout=5s period=15s #success=1 #failure=5
Reverse DNS lookups:
~> sudo tcpdump -ni cali5205 udp port 53 | grep -i ptr
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on cali5205, link-type EN10MB (Ethernet), snapshot length 262144 bytes
08:54:57.765646 IP 192.168.128.134.49110 > 10.96.0.10.54: 18906+ PTR? 6.0.0.127.in-addr.arpa. (40)
08:55:06.249839 IP 192.168.128.134.57328 > 10.96.0.10.54: 28672+ PTR? 181.223.168.192.in-addr.arpa. (46)
08:55:08.384820 IP 192.168.128.134.33574 > 10.96.0.10.54: 42376+ PTR? 181.223.168.192.in-addr.arpa. (46)
08:55:12.765568 IP 192.168.128.134.42941 > 10.96.0.10.54: 59040+ PTR? 6.0.0.127.in-addr.arpa. (40)
08:55:27.765632 IP 192.168.128.134.37767 > 10.96.0.10.54: 29585+ PTR? 6.0.0.127.in-addr.arpa.

One of our customers has a DNS server configured in a way that is returning "ServFail" response for these PTR requests:
~> sudo tcpdump -ni cali46f udp port 53
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on cali46f , link-type EN10MB (Ethernet), snapshot length 262144 bytes
09:56:39.154100 IP 192.168.168.76.54527 > 169.254.20.10.54: 3225+ PTR? 6.0.0.127.in-addr.arpa. (40)
09:56:41.155101 IP 169.254.20.10.54 > 192.168.168.76.54527: 3225 ServFail- 0/0/0 (40)
09:56:41.155453 IP 192.168.168.76.35659 > 169.254.20.10.54: 3225+ PTR? 6.0.0.127.in-addr.arpa. (40)
09:56:43.156629 IP 169.254.20.10.54 > 192.168.168.76.35659: 3225 ServFail- 0/0/0 (40)

Is there any way to disable these reverse DNS lookups for the JMX Exporter?

Thank you in advance!

[dhoard]

@josipanas Can you share your exporter YAML configuration?

[josipanas]

Hi @dhoard,

YAML configuration:
hostPort: localhost:1099
lowercaseOutputName: true
ssl: false

JVM arguments:
args:
- /stdout-redirect -container locator-jmx-exporter -logfile
/logs/locator-jmx-exporter.log -redirect all -size 5 -rotate 2 -- /usr/src/app/run_jmx_exporter.py
--web io.prometheus.jmx.WebServer --port 8081 --config /etc/config/dynamic/jmxExporter.yaml
--creds /etc/internal-credentials/internalCredentials.json --java-args
" -classpath /usr/src/app/:/usr/src/app/dependency/ -Xms128m -Xmx128m
-Djava.util.logging.config.file=/etc/jmxExporterLogging.properties
-Dlog4j.configurationFile=/etc/config/dynamic/log4j2-jmx-exporter.yaml -Djava.net.preferIPv6Addresses=true"

If needed I can send you run_jmx_exporter.py script that we use to start the JMX Exporter process.
Thanks!

BR,
Josipa

[josipanas]

Hi @dhoard,

Did you get a chance to look at the provided configuration?
Thank you in advance!

BR,
Josipa

[dhoard]

@josipanas I have looked at the exporter code and don't see anything off hand that should cause a reverse DNS lookup. The exporter uses com.sun.net.httpserver.HttpServer which internally could be causing the call.

Does this happen with the latest version of the exporter?

[josipanas]

@dhoard thank you for your response!

I tested it using the following versions:

        <dependency>
            <groupId>io.prometheus.jmx</groupId>
            <artifactId>collector</artifactId>
            <version>1.4.0</version>
        </dependency>
        <dependency>
            <groupId>io.prometheus.jmx</groupId>
            <artifactId>jmx_prometheus_httpserver</artifactId>
            <version>1.0.1</version>
        </dependency>
    </dependencies>

Reverse DNS lookups still persist.

As you may notice, we've upgraded the collector to version 1.4.0, but retained jmx_prometheus_httpserver at version 1.0.1 - which, to my understanding, is the latest version published to Maven.
Our configuration for fetching JMX Exporter is Maven-based, and I'm unsure how we could upgrade jmx_prometheus_httpserver (now renamed to jmx_prometheus_standalone) to version 1.4.0 without completely changing our current setup.

If you have any suggestions, we'd greatly appreciate them.

Thanks again!

BR,
Josipa

[dhoard]

@josipanas It's not clear/I don't understand what you're trying to do/your usage.

You should either ...

1. Use the Java agent exporter
2. Use the Standalone exporter
3. Use the collector module and write your own supporting code.

The collector module is really intended for internal use.

http://prometheus.github.io/jmx_exporter/1.4.0/collector/

---

When using the collector module to remotely monitor a Java process via JMX, RMI is used, which may be initiating the reverse DNS lookups.

reference: https://docs.oracle.com/javase/8/docs/technotes/guides/rmi/faq.html

[josipanas]

@dhoard, thank you for the clarification.
We started using JMX Exporter a long time ago (back in 2019) and have had the collector dependency ever since.
I removed it, and metrics are still visible on our PM server, so it seems we weren’t actually using it. We'll go ahead and remove it from our pom.xml configuration.
Unfortunately, even without the collector module, reverse DNS lookups are still occurring.