Add support for disabling feature datasets

[mjjbell]

Issue

This change adds support for disabling datasets, such that specific files are not loaded into memory when running OSRM. This enables users to not pay the memory cost for features they do not intend to use.

Initially, there are two options:

• ROUTE_GEOMETRY, for disabling overview, steps, annotations and waypoints.
• ROUTE_STEPS, for disabling steps only.

Attempts to query a feature for which the dataset is disabled will lead to a DisabledDatasetException being returned.

Full details are on a new wiki page: https://github.com/Project-OSRM/osrm-backend/wiki/Disabled-Datasets

Design Choices

The implementation separates the "feature dataset" concept from the actual files loaded into memory. This gives some flexibility in how the data is stored, allowing for changes in the file structure without breaking the user facing API.

The feature datasets to disable are passed as an array argument to osrm-routed, osrm-datastore and the NodeJS engine configuration. Currently, you would only select at most one option (ROUTE_GEOMETRY is a superset of ROUTE_STEPS), but the array interface will allow us to add more options in the future whilst maintaining backwards compatibility.

On every data facade call to optional datasets, we check a boolean to see if its loaded. I attempted other solutions that don't perform a check on every call, such as virtual functions and lambdas. These didn't improve performance and were significantly more complicated, so I stuck with the simplest implementation.

Performance

For a contraction hierarchy dataset, --disable-feature-dataset ROUTE_GEOMETRY reduces RAM usage by ~15%.

There appears to be no impact on query performance. This is likely due to the number of data facade calls affected being very small. Will add full query performance results in a subsequent post.

Future Direction

Some optional datasets are stored in files with required datasets. One such example is OSM node IDs being stored with coordinates. We could reduce memory usage further when disabling ROUTE_GEOMETRY by splitting such files, or refactoring how files are loaded into memory.

Tasklist

• CHANGELOG.md entry (How to write a changelog entry)
• update relevant Wiki pages
• add tests (see testing documentation)
• review
• adjust for comments
• cherry pick to release branch

Requirements / Relations

#5218
#5838
#6045

[SiarheiFedartsou]

Nit.

Suggested change

	#include "set"
	#include <set>

[SiarheiFedartsou]

Nit.

Suggested change

	std::string Dataset() const { return dataset; }
	const std::string& Dataset() const { return dataset; }

[mjjbell]

For completeness, I ran performance tests using the same setup as here with the first three coordinate sets.

All requests had the parameters overview=simplified&annotations=true&steps=true&skip_waypoints=false, to check if there is any performance penalty for checking if a dataset is loaded (optional) in comparison to it always being loaded (master).

The results show no change in performance with this PR. Tests with request parameters that disabled geometries and steps had the same outcome.