[BUG] - influxdb_database.present failed on first run

[LHozzan]

Description
When InfluxDB is fresh installed / configured, this state failed.

          ID: collectd DB in influxdb exist
    Function: influxdb_database.present
        Name: collectd
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
                  (self._dns_host, self.port), self.timeout, **extra_kw)
                File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection
                  raise err
                File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection
                  sock.connect(sa)
              ConnectionRefusedError: [Errno 111] Connection refused
              
              During handling of the above exception, another exception occurred:
              
              Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
                  chunked=chunked)
                File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
                  conn.request(method, url, **httplib_request_kw)
                File "/usr/lib/python3.7/http/client.py", line 1244, in request
                  self._send_request(method, url, body, headers, encode_chunked)
                File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
                  self.endheaders(body, encode_chunked=encode_chunked)
                File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
                  self._send_output(message_body, encode_chunked=encode_chunked)
                File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
                  self.send(msg)
                File "/usr/lib/python3.7/http/client.py", line 966, in send
                  self.connect()
                File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect
                  conn = self._new_conn()
                File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
                  self, "Failed to establish a new connection: %s" % e)
              urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff495069b00>: Failed to establish a new connection: [Errno 111] Connection refused
              
              During handling of the above exception, another exception occurred:
              
              Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send
                  timeout=timeout
                File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
                  _stacktrace=sys.exc_info()[2])
                File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
                  raise MaxRetryError(_pool, url, error or ResponseError(cause))
              urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8086): Max retries exceeded with url: /query?q=SHOW+DATABASES (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff495069b00>: Failed to establish a new connection: [Errno 111] Connection refused'))
              
              During handling of the above exception, another exception occurred:
              
              Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/salt/state.py", line 1981, in call
                  **cdata['kwargs'])
                File "/usr/lib/python3/dist-packages/salt/loader.py", line 1977, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/states/influxdb_database.py", line 34, in present
                  if not __salt__['influxdb.db_exists'](name, **client_args):
                File "/usr/lib/python3/dist-packages/salt/modules/influxdbmod.py", line 108, in db_exists
                  if name in [db['name'] for db in list_dbs(**client_args)]:
                File "/usr/lib/python3/dist-packages/salt/modules/influxdbmod.py", line 92, in list_dbs
                  return client.get_list_database()
                File "/usr/lib/python3/dist-packages/influxdb/client.py", line 570, in get_list_database
                  return list(self.query("SHOW DATABASES").get_points())
                File "/usr/lib/python3/dist-packages/influxdb/client.py", line 416, in query
                  expected_response_code=expected_response_code
                File "/usr/lib/python3/dist-packages/influxdb/client.py", line 267, in request
                  timeout=self._timeout
                File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
                  resp = self.send(prep, **send_kwargs)
                File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
                  r = adapter.send(request, **kwargs)
                File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
                  raise ConnectionError(e, request=request)
              requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8086): Max retries exceeded with url: /query?q=SHOW+DATABASES (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff495069b00>: Failed to establish a new connection: [Errno 111] Connection refused'))
     Started: 15:34:57.164555
    Duration: 6.737 ms
     Changes:   

Next runs working correctly:

----------
          ID: collectd DB in influxdb exist
    Function: influxdb_database.present
        Name: collectd
      Result: True
     Comment: Database collectd has been created
     Started: 15:58:15.983941
    Duration: 5.666 ms
     Changes:   
              ----------
              collectd:
                  Present

Setup

• Vagrant 2.2.9 (box 'generic/debian10' (v3.0.2))
• SaltStack 3000.3
• States:

influxdb repository exist:
  pkgrepo.managed:
    - name: deb https://repos.influxdata.com/debian buster stable
    - key_url: https://repos.influxdata.com/influxdb.key

influxdb installed:
  pkg.installed:
    - names:
      - influxdb
      - python3-influxdb
    - require:
      - pkgrepo: influxdb repository exist

{% set influxdir = salt['pillar.get']('influxdb:dir','/var/lib/influxdb') %}

influxdb config:
  file.managed:
    - name: /etc/influxdb/influxdb.conf
    - source: salt://influxdb/conf/influxdb.conf
    - template: jinja
    - context:
      dir: {{ influxdir }}
    - require:
      - pkg: influxdb installed

influxdb service:
  service.running:
    - name: influxdb
    - enable: true
    - watch:
      - file: influxdb config
    - require:
      - pkg: influxdb installed
      - file: influxdb config
      - pkg: collectd installed

collectd DB in influxdb exist:
  influxdb_database.present:
    - name: collectd
    - require:
      - service: influxdb service

Steps to Reproduce the behavior

• when first run on fresh box install, still failed
• next runs working correctly

Expected behavior
It seems, that InfluxDB is inicializing in time, when report, that is running. State influxdb_database.present is running in time, when InfluxDB isnt ready accepting connections. Slow can occured on slow mechanical harddrive. Consider add some test, that InfluxDB is accepting new connections.

Versions Report

salt --versions-report

``` Salt Version: Salt: 3000.3

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 2.10
libgit2: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.7.3 (default, Dec 20 2019, 18:57:59)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 17.1.2
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.1

System Versions:
dist: debian 10.4
locale: UTF-8
machine: x86_64
release: 4.19.0-9-amd64
system: Linux
version: debian 10.4

</details>

**Additional context**
Add any other context about the problem here.

[garethgreenaway]

@LHozzan Thanks for the report. When the information from the call to service.running returns it is indicating that the underlying service system. eg. systemd has started the service in question. It is not an indication that the service is available. What would help in this situation is a function that reports if InfluxDB is in a ready state. I'm changing this from a bug to a feature request. Additionally, one thing to look at would be the orchestration state, that will allow you to run states once other states have finished, eg. creating the database once the service is in a ready state.

[LHozzan]

@garethgreenaway Thank you for information.
So, you mean add some state like cmd.wait and test in loop, that InfluxDB is ready to accept coonections? Good idea, I'll consider it.