Auto-scaling services according to etcd key

[yaronr]

Hi

Yes, it's another one of my crazy ideas, but. :)
Wouldn't it be great if there was a way to run a service in fleet, leaving the specification of 'how many' to run - to something external, like an etcd key.

For example:
fleetctl start kuku@{scale/kukus}.service

Then, you could control how many kukus are running, by setting the etcd key 'scale/kukus'. Setting either manually, or by some sort of process. Like - for example - a process that monitors the load on kuku, or latency.

This could be very very useful for scaling up or down services, and - in turn - clusters.

[bcwaldon]

While this is a fine suggestion as an implementation of a feature, we should start first by designing the feature that is being implemented. What exactly do you need this for?

[yaronr]

Hi @bcwaldon, thanks for responding.

Auto-scaling.
From my POV, auto-scaling has 3 phases, I will call them 'sense-think-act' (borrowed from Robotics).
Sense translates to monitoring, think translates to decision making, act translates to taking action based on decisions.

Taking this to the realm of CoreOS:
Assuming I have the technology to do the first two: monitor the hosts and the containers, and make decisions (for now - a simple decision: how many instances of service X are needed), and put the output of the 'think' phase in an accessible place (etcd), then the 'act' phase needs to be implemented: fleet needs to be able to read this etcd (or something else) key, and create / remove instances of 'service x' accordingly.

[bcwaldon]

@yaronr I believe fleet already gives you all the tools you need to do this today. You already have to teach your scheduler (which fulfills step 2) what to do once it decides it needs to cause a change in fleet's units. Rather than setting a key in etcd, the scheduler can just do the work of creating/destroying the relevant units.

[bcwaldon]

Given the direction of #1055, I believe it is safe to say that fleet itself will not support this.