Skip to content

tasks: fail to submit workflow start operation #197

@diegodelemos

Description

@diegodelemos

Currently, if there are dangling images in the Kubernetes node(s), the check_predefined_conditions call will fail when we call list_nodes because of kubernetes-client/python#895. The exact reason of this is described in kubernetes-client/python#895 (comment), there has been a fix attempt but it hasn't been tackled yet.

The traceback when this happens:

$ kubectl logs reana-server-xxxxx-yyy scheduler --previous
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/models/v1_container_image.py", line 75, in names  
    raise ValueError("Invalid value for `names`, must not be `None`")  # noqa: E501           
ValueError: Invalid value for `names`, must not be `None` 

Note the usage of --previous, because the process exits you won't see any logs if you do kubectl logs reana-server-xxxxx-yyy as you will see the logs of the new pod.

Since we cannot control this, we should be more resilient and avoid blocking the whole system for this reason.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions