README claims you can run a "hot spare" synapse, which is incorrect

[djschilling]

In the Readme i read the following sentence:

allowing basic active/backup high-availability with a "hot spare" synapse pointing at the same DB master, as well as enabling DB replication in synapse itself.

But i cannot find any documentation how this ha setup should work.
I also asked in the Matrix Channel Synapse Admins but no one could give me help there.

The only issue i found related is this one: #2123

So the questions that i would like to see answered in the docs are:

• What type of postres replication should i use?
  • I like the setup described here, but i have no experience with synapse und some guidance of typical setups would be nice: https://medium.com/searce/design-a-highly-available-postgresql-cluster-with-patroni-in-gcp-part-1-dfb6ed277806
• How can i achieve a hot spare setup as described in the documentation?
• When possible how could i achieve a "real" ha setup, using two or more matrix servers behind a loadbalancer?

[richvdh]

allowing basic active/backup high-availability with a "hot spare" synapse pointing at the same DB master, as well as enabling DB replication in synapse itself.

I have basically no idea what this is referring to. AFAIK nobody has ever tried setting up a "hot spare" synapse, and I can't see how that would work, due to the lack of cache coherence between the two instances. In short, I think the README is misleading.

The paragraph in question seems to originate from cc5d68f; @ara4n any insights as to what this was meant to be referring to?

[djschilling]

@richvdh

Okay but what is the alternative, if that is not an option?
How is for example the setup for matrix.org?
Is it just down if the server crashes?

[richvdh]

yes

[djschilling]

Wow that is unexpected.

So the cache is the main problem?

But if it is just a cache, then it should be possible to just have a second synapse server running and taking over if the first one crashes, right?

My setup then would be:

• Database -> Postgres Cluster
• Files -> shared file system
• Primary Synapse Server
• Standby Synapse Server

[richvdh]

in theory, maybe. In practice, you'd need a way to reliably make sure that the standby didn't do anything (including populate caches from the database) until the primary was dead.

We're looking at multi-master setups in #6677; now isn't a great time to open up a new battle-front in terms of another axis of redundancy. For now, suffice it to say that the README is inaccurate.

[djschilling]

Okay, i understand.

So what is the current best way to deploy a synapse server if high availability is my focus?

[benlangfeld]

Okay, i understand.

So what is the current best way to deploy a synapse server if high availability is my focus?

@djschilling Did you ever get anywhere with this line of enquiry?

We're considering replacing a proprietary chat system which has no single point of failure with something Matrix-compatible and this is something of a hang-up for us.

[djschilling]

@benlangfeld no i never found a solution to this problem. There still seems to be no way to run synapse in an ha setup

[m-amir-ir]

Hello everyone, have you found a way to solve this problem? Even if it is not exactly high availability, an active/passive solution may be suitable for this purpose