Lock access to facade_factory in data_watchdog to avoid accessing destructed object

[danpat]

Issue

We have a race condition in the data_watchdog where a request may still be accessing a facade_factory that has been destructed by the watchdog thread due to new traffic data arriving (loaded by osrm-datastore).

Performance before the lock:

$ wrk -c 8 -t 8 -d 20 "http://localhost:5000/route/v1/car/1,1;2,2?exclude=toll"
Running 20s test @ http://localhost:5000/route/v1/car/1,1;2,2?exclude=toll
  8 threads and 8 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   706.66us  224.89us  10.92ms   94.13%
    Req/Sec     1.42k   193.42     1.68k    90.55%
  227791 requests in 20.10s, 207.90MB read
  Socket errors: connect 0, read 439, write 0, timeout 0
Requests/sec:  11332.57
Transfer/sec:     10.34MB

Performance after the lock:

$ wrk -c 8 -t 8 -d 20 "http://localhost:5000/route/v1/car/1,1;2,2?exclude=toll"
Running 20s test @ http://localhost:5000/route/v1/car/1,1;2,2?exclude=toll
  8 threads and 8 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   761.02us    1.49ms  51.52ms   99.72%
    Req/Sec     1.43k   173.74     1.72k    86.07%
  229274 requests in 20.10s, 209.25MB read
  Socket errors: connect 0, read 443, write 0, timeout 0
Requests/sec:  11405.21
Transfer/sec:     10.41MB

My read on this is that the new shared lock has negligible impact on overall performance.

With the shared lock in place, I'm no longer able to reproduce the crash described in

#5841

Tasklist

• CHANGELOG.md entry (How to write a changelog entry)
• update relevant Wiki pages
• add tests (see testing documentation
• review
• adjust for comments
• cherry pick to release branch

Requirements / Relations

Link any requirements here. Other pull requests this PR is based on?