Implement support for GSK Vertical Scaling

[twiebe]

Earlier this year we have implemented vertical scaling for GSK clusters. Vertical scaling allows users to change certain parameters for existing clusters, which were read-only until then. These parameters are:

• k8s_worker_node_cores
• k8s_worker_node_ram
• k8s_worker_node_storage
• k8s_worker_node_storage_type

During the resource change nodes will be replaced one at a time. This will cause your pods to shift between nodes and cluster capacity to be degraded by one node temporarily. If you're running near capacity limits, please consider adding worker nodes to avoid impact on your workloads.

This feature is available with all generally available GSK releases, expected to work with future GSK releases and all existing clusters have been upgraded to releases that support this. Technically, whether it is supported or not can be taken from the immutable-flag for each of the four parameters in the parameters_schema:

'k8s_worker_node_storage_type': {
    'type': 'string',
    'allowed': ['storage', 'storage_high', 'storage_insane'],
    'default': 'storage_insane',
    'required': True,
    'immutable': True,
    'description': 'Storage type'
},

If immutable is False, the parameter can be changed and thus vertical scaling is available.