Allow to have a master/replica DB configuration

[pvalsecc]

Basically, this guy switches the DB connection to a replica (readonly) DB if the HTTP method is GET or OPTIONS.

[tsauerwein]

The session is bound to a request, so binding the connection would work (like you say, assuming that no writes are done in GET requests). You are proposing to use a DNS to support multiple read-only instances. We could also allow to specify multiple read-only connections and then use a simple random load-balancing strategy to avoid the name-resolution step.

Maybe you want to add some documentation on how to configure the actual replication in Postgres?

And on a side node: It would be great if we could avoid a historically racially charged term and maybe use "primary - replica/read-only" instead (for example like Drupal or CouchDB).

[pvalsecc]

@tsauerwein, slavery is not necessarily linked to races. Romans were enslaving their own, sometimes. But OK, I'm removing references to our dark past (and present) and I'm using Postgres' other terms: master/replica. And added a link to their doc for how to set it up.

Regarding the DNS or multiple addresses, after discussion with our IT experts, it's not enough. If you want failover, you need to setup two HAProxy:

• one for the master with the functionality to elect a new master and point to it in case of failure
• one for the replicas

I've added that to the doc, but without much details. It's out of my league/responsibilities ;-)

[pvalsecc]

@fredj, I've heard you've worked on that problem in the past. Any opinion?

[tsauerwein]

@tsauerwein, slavery is not necessarily linked to races. Romans were enslaving their own, sometimes. But OK, I'm removing references to our dark past (and present) and I'm using Postgres' other terms: master/replica. And added a link to their doc for how to set it up.

Thanks! 👍

[fredj]

We created 2 distinct engines (one for reads and one for writes) and we used one or the other depending on the operation.
It was not automatic but the principle is the same

[sbrunner]

I don't found the therm replica relay useful, every body know whats a master-slave architecture is and we don't do philosophy...

[sbrunner]

https://www.google.ch/trends/explore#q=master%20replica%2C%20master%20slave&cmpt=q&tz=Etc%2FGMT-2

[tsauerwein]

...like you say, assuming that no writes are done in GET requests.

Here is an example of a GET request that also writes to the database:

c2cgeoportal/c2cgeoportal/views/shortener.py

Lines 55 to 64 in 9425952

	@view_config(route_name="shortener_get")
	def get(self):
	ref = self.request.matchdict["ref"]
	short_urls = DBSession.query(Shorturl).filter(Shorturl.ref == ref).all()
	if len(short_urls) != 1:
	raise HTTPNotFound("Ref '%s' not found" % ref)
	short_urls[0].nb_hits += 1
	short_urls[0].last_hit = datetime.now()

[pvalsecc]

@tsauerwein, you should look at the last version of my patch. I've added the possibility to force the usage of the master or the replica for some URLs. And the shortener is part of it.

[tsauerwein]

I've added the possibility to force the usage of the master or the replica for some URLs. And the shortener is part of it.

Perfect, I missed that.

[pvalsecc]

So... what now? Merge or not?
To: @tsauerwein @sbrunner

[tsauerwein]

LGTM