[stable/redis-ha] Implement stable sentinel IDs by pregenerating them

[towolf]

What this PR does / why we need it:

In the course of reworking the failover mechanisms in PR #10032, the proliferation of stale sentinel IDs was prevented by recovering the old ID from the old config file (from persistent storage) on startup.

However, the redis-ha chart can be used with volatile data and an emptyDir. The sentinel ID will be lost on termination and again randomly generated on startup.

This PR makes sentinel IDs stable by simple pre-generating them and storing the as ENV vars on the init container.

In this approach I take the unique label combination + StatefulSet pod index and take the SHA1 of that:

    Environment:              
      SENTINEL_ID_0:  e2f7f63e2872ee7884a3887a043457bc6bff1fb4
      SENTINEL_ID_1:  10257ae861ee58ad3a0c32cda22ba7f62887e5c2
      SENTINEL_ID_2:  ba97a7362cc6e46973f0fe45c895cd5b57a60000

However, I could image that this would work just as well, not sure:

    Environment:                                              
      SENTINEL_ID_0:  1111111111111111111111111111111111111111
      SENTINEL_ID_1:  2222222222222222222222222222222222222222
      SENTINEL_ID_2:  3333333333333333333333333333333333333333

Special notes for your reviewer:

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

• DCO signed
• Chart Version bumped
• Variables are documented in the README.md

[towolf]

/assign @ssalaues

We talked about this here before:
#10635 (comment)

[k8s-ci-robot]

@towolf: GitHub didn't allow me to assign the following users: ssalaues.

Note that only helm members and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @ssalaues

We talked about this here before:
#10635 (comment)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@ssalaues: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ssalaues]

@towolf Thanks for the PR! I tested it and works well!

The only caveat is the "ghosts" that happen on upgrade from the previous version to this one. However the "ghost" sentinels don't seem to grow beyond this initial upgrade (with or without persistence) and are still able to reach majority. Also just an FYI for any future on lookers of this PR, a consecutive rolling upgrade seemed to clear out the "ghosts" from the configs.

[ssalaues]

@towolf I'm not sure why the approval didn't go through. I just was able to approve this PR #11085 . But since that one is going through, can you rebase and bump your version again?

I'll delete my test and approve comments and try again when you push.

[towolf]

@ssalaues done.

[ssalaues]

/test

[ssalaues]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ssalaues, towolf

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• stable/redis-ha/OWNERS [ssalaues]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment