operator; enable sequential conduit updates

[zolug]

Description

Users can group Conduits into update groups by annotating Conduit Custom Resources with the update-sync-group key. Within an update group, Conduit updates are executed sequentially rather than in parallel.

The default annotation key can be changed via the operator environment variable CONDUIT_UPDATE_SYNC_GROUP_KEY when deploying the operator.

Locking is achieved using in-memory locks, which are fast and provide a sufficiently good solution. However, this approach is not effective in case of operator crashes during ongoing updates. Such crashes would be considered a bug to be fixed and should not occur in general.

The feature has support for changing the annotation value of a Conduit Custom Resource on the fly. (Including removing it completely.)
However, no immediate control can be expected when changing the update group of a Conduit whose Proxy's Status subresource still indicates underlying changes (e.g. in POD readiness) while the DaemonSet otherwise reflects the desired state. This is because the logic relies on the Status subresource to accurately track the progress of updates and ensure consistency.

Also, existing Conduit Custom Resources can be annotated before upgrading to a Meridio version that supports serialized Conduit updates. In this case, the new operator will respect the update groups of the "old" Conduits from the start.

Note: There is no benefit to having a single Conduit in an update group.

Issue link

#555

Checklist

• Purpose
  • Bug fix
  • New functionality
  • Documentation
  • Refactoring
  • CI
• Test
  • Unit test
  • E2E Test
  • Tested manually
• Introduce a breaking change
  • Yes (description required)
  • No

[zolug]

Example where two conduits belong to the same update group team-a:

cat <<EOF | kubectl apply -f -
apiVersion: meridio.nordix.org/v1
kind: Conduit
metadata:
  name: load-balancer-b1
  namespace: default
  annotations:
    update-sync-group: "team-a"
    resource-template: "small"
  labels:
    trench: trench-a
spec:
  type: stateless-lb
---
apiVersion: meridio.nordix.org/v1
kind: Conduit
metadata:
  name: load-balancer-a1
  namespace: default
  annotations:
    update-sync-group: "team-a"
    resource-template: "medium"
  labels:
    trench: trench-a
spec:
  type: stateless-lb
EOF