[Logstash 7.9.1] Error: UPGRADE FAILED: cannot patch "logstash" with kind StatefulSet

[ppan-suchada]

Chart version:
7.9.1

Kubernetes version:
v1.17.9

Kubernetes provider:
Amazon Elastic Kubernetes Service (Amazon EKS)

Helm Version:
v3.0.2

Describe the bug:
I was trying to upgrade Logstash from 7.8.0 to 7.9.1 due to the newer version have fixed the bug related to .Values.extraPorts #765

Steps to reproduce:

1. Run helm upgrade with dry-run and debug mode

helm upgrade --install logstash-centralized-pipeline-logging elastic/logstash --namespace ns --version 7.9.1 --values helm/values-logstash.yaml --dry-run --debug

Here is the output when I run helm upgrade with dry-run mode and it seems should be able to upgrade

history.go:52: [debug] getting history for release logstash-centralized-pipeline-logging
upgrade.go:79: [debug] preparing upgrade for logstash-centralized-pipeline-logging
upgrade.go:87: [debug] performing update for logstash-centralized-pipeline-logging
upgrade.go:220: [debug] dry run for logstash-centralized-pipeline-logging
Release "logstash-centralized-pipeline-logging" has been upgraded. Happy Helming!

2. Run helm upgrade

helm upgrade --install logstash-centralized-pipeline-logging elastic/logstash --namespace ns --version 7.9.1 --values helm/values-logstash.yaml

Expected behavior:
Be able to upgrade Logstash to v.7.9.1

Provide logs and/or server output (if relevant):

Error: UPGRADE FAILED: cannot patch "logstash-centralized-pipeline-logging-logstash" with kind StatefulSet: StatefulSet.apps "logstash" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

Any additional context:

[jmlrt]

Hi @ppan-suchada, this is a known problem which is documented here

You should use helm upgrade --force to do the upgrade.

[ppan-suchada]

Thanks @jmlrt, I've tried using helm upgrade --force already.

However, I received the error as below.
Upgrade "logstash-centralized-pipeline-logging" failed: failed to replace object: Service "logstash-centralized-pipeline-logging-logstash" is invalid: spec.clusterIP: Invalid value: "": field is immutable && failed to replace object: StatefulSet.apps "logstash-centralized-pipeline-logging-logstash" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

Here is the configurations that I'm using currently.

service: 
  annotations: {}
  type: ClusterIP
  ports:
  - name: beats
    port: 5044
    protocol: TCP
    targetPort: 5044
  - name: http
    port: 9600
    protocol: TCP
    targetPort: 9600

[jmlrt]

This has been fixed in #776. I suggest you to upgrade to 7.9.1 instead.

[ppan-suchada]

Yess, that's the version I'm trying to update to, but not sure that I miss something out or not because I've tried with helm upgrade --force and it seems fix the first error. However, I still cannot solve the another error #817 (comment)

[jmlrt]

Hi @ppan-suchada,

I could reproduce your issue with Helm 3, while helm upgrade --force worked successfully with Helm 2.

Looking at helm/helm#7082 (comment), it seems that helm upgrade --force behavior changed with Helm 3.

While helm upgrade --force in Helm 2 was deleting and recreating resources (like an helm uninstall && helm install), in Helm 3 it's working differently and won't try to delete and recreate immutable resources.

It seems the only solution is to do an helm uninstall followed by helm install to recreate the resources, while waiting for a fix coming in Helm 3.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue has been automatically closed because it has not had recent activity since being marked as stale.