Skip to content

Vault/Consul unstable with stemcell 621.23 #30

@hfrenzel

Description

@hfrenzel

After stemcell update to use 621.23, the safe deployment (v0.2.1) with vault/consul became very unstable as of following errors:

==> /var/vcap/sys/log/vault/consul.log <==
    2019/11/19 18:43:06 [WARN] raft: Election timeout reached, restarting election
    2019/11/19 18:43:06 [INFO] raft: Node at 10.X.X.X:8300 [Candidate] entering Candidate state in term 11581
    2019/11/19 18:43:07 [ERR] http: Request GET /v1/kv/vault/core/cluster/local/info, error: No cluster leader from=127.0.0.1:52548
    2019/11/19 18:43:10 [ERR] agent: coordinate update error: No cluster leader
    2019/11/19 18:43:10 [ERR] http: Request PUT /v1/session/renew/4469ca4a-7b14-adbf-29e7-b95b06c0af20, error: No cluster leader from=127.0.0.1:52578
    2019/11/19 18:43:11 [ERR] http: Request GET /v1/kv/vault/core/lock, error: No cluster leader from=127.0.0.1:52592
    2019/11/19 18:43:13 [WARN] raft: Election timeout reached, restarting election
    2019/11/19 18:43:13 [INFO] raft: Node at 10.X.X.X:8300 [Candidate] entering Candidate state in term 11582
    2019/11/19 18:43:16 [ERR] agent: failed to sync remote state: No cluster leader
    2019/11/19 18:43:16 [ERR] http: Request GET /v1/kv/vault/core/poison-pill, error: No cluster leader from=127.0.0.1:52594
    2019/11/19 18:43:18 [ERR] http: Request GET /v1/kv/vault/core/lock, error: No cluster leader from=127.0.0.1:52654
    2019/11/19 18:43:18 [ERR] http: Request GET /v1/kv/vault/core/?keys=&separator=%2F, error: No cluster leader from=127.0.0.1:52658

==> /var/vcap/sys/log/vault/vault.log <==
2019-11-19T18:43:18.626Z [ERROR] core: barrier init check failed: error="failed to check for initialization: Unexpected response code: 500"
2019-11-19T18:43:18.626Z [ERROR] core: error checking health: error="failed to check for initialization: Unexpected response code: 500"

After rolling back to the last used stemcell (621.12), the error 500 seems to be gone.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions