- What's a Geocoder
- Getting Started
- Development
- Processes
- Who Uses It ?
- Alternatives
- Resources
- License
- Acknowledgments
Mimirsbrunn (also called Mimir) is an independent geocoding and reverse-geocoding system written in Rust, and built upon Elasticsearch. It can handle addresses, streets, points-of-interest (POI), administrative regions and public transport stops.
Usually geocoding refers to "the process of transforming a physical address description to a location on the Earth's surface". However Mimir is more a geoparser than a geocoder since it can resolve any ambiguous toponym to its correct location.
In other words, a geocoder reads a description (possibly incomplete) of a location, and returns a list of candidate locations (latitude / longitude) matching the input.
Geocoding is traditionally used for autocompleting search fields used in geographic applications.
For example, here is a screenshot of Qwant Maps, where the user enters a search string 20 rue hec mal, and mimir returns possible candidates in a dropdown box.
For an introduction to the project, you can have a look at the Introduction to Mimirsbrunn.
These instructions will give you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on deploying the project on a live system.
Mimirsbrunn is a rust project, so you need to have a rust development environment. Instructions on how to do that are found on the rust website.
Additionally, mimirsbrunn relies on cosmogony to create so called cosmogony files which are used for indexing administrative regions.
For running end to end or unit tests, you need the docker engine available.
You can install Mimirsbrunn from debian packages available as build artifacts on the repository homepage. [FIXME Where are .deb ?]
You can also fetch official images from DockerHub here:
If you want to get the commit ref used to build the latest image you can run the following command:
docker inspect --format='{{index .Config.Labels "org.label-schema.vcs-ref"}}' navitia/mimirsbrunn:latestYou can build Mimirsbrunn manually, as the following instructions explain.
You need to retrieve the project and build it using the rust compiler:
git clone https://github.com/hove-io/mimirsbrunn.git
cd mimirsbrunn
cargo build --release
Having built the project, you can now perform a sanity check. This will serve as a test that the program works, and as a basic example of the two main functionalities: indexing and querying.
So we'll index administrative regions into Elasticsearch, and then search for some of them. We'll focus on one country, lets say... Denmark!
- Download OSM file.
First we download the OSM file for Denmark, from the geofabrik server, and store the file locally.
- Generate cosmogony file.
If you haven't installed cosmogony yet, you need to do so now, by following the instructions here. You can then transform the original OSM PBF file for Denmark (
The following command must be typed in the directory of the cosmogony project)
cargo run --release -- generate -i /path/to/denmark-latest.osm.pbf -o denmark.jsonl.gz
- Start an Elasticsearch docker container.
We'll start an Elasticsearch version 7.13, available on ports 9200 and 9300.
docker run --name mimirsbrunn-test \
-p 9200:9200 -p 9300:9300 \
-e "discovery.type=single-node" \
-d docker.elastic.co/elasticsearch/elasticsearch:7.13.0
You can check that Elasticsearch has correctly started (maybe wait for about 10/20s for Elasticsearch to be available):
curl 'http://localhost:9200'
{
"name": "...",
"cluster_name": "docker-cluster",
...
k "tagline": "You Know, for Search"
}
- Prepare Elasticsearch
We use templates, such that if you create an index that starts with a certain prefix, if a templates is configured for that prefix, it will create the index with the values found in that template. So, before creating any index, you need to import all the templates in Elasticsearch.
cd mimirsbrunn
./target/release/ctlmimir \
-s "elasticsearch.url='http://localhost:9200'" \
-c ./config \
-m testing \
run
You should see all the index templates by querying your Elasticsearch. For example, the template for administrative regions:
curl 'http://localhost:9200/_index_template/mimir-admin*'
{
"index_templates": [
{
"name": "mimir-admin",
"index_template": {
"index_patterns": [
"munin_admin*"
],
...
"composed_of": [
"mimir-base",
"mimir-dynamic-mappings"
],
"priority": 10,
"version": 3
}
}
]
}
- Index cosmogony into Elasticsearch
The result of building the mimirsbrunn project includes several binaries located in
/target/releases, one of which is used to index cosmogony files:
cosmogony2mimir uses several configuration files found in the source code, and they work fine by default. In the following command, we use a setting to make sure that cosmogony2mimir will target the Elasticsearch container we just started: (See here for more details about using cosmogony2mimir)
cd mimirsbrunn
./target/release/cosmogony2mimir \
-c ./config \
-m testing \
-s "elasticsearch.url='http://localhost:9200'" \
-s langs=['en', 'da'] \
-i <path/to/denmark.jsonl.gz> \
run
You can follow in the mimirsbrunn/logs directory.
- Check Elasticsearch
The previous step created an index for administrative regions, and so you should be able to query your Elasticsearch like so.
curl 'http://localhost:9200/_cat/indices'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open munin_admin_fr_20211104_152535_346903898 FrWbs7PiRi26w-cbsIXjbg 1 4 1841 0 16.3mb 16.3mb
-
Start Bragi
-
Query Bragi
Since this is a rust project, we are well instrumented to run all sorts of tests:
- style
- lint
- unit tests
- end to end / integration.
- benchmark
You can run them all at once, and this in the way it is carried out in the CI pipeline, with
make check
See this page for a more in depth introduction to testing this project.
Maybe you find that some the results you get are not ranked correctly, and want to adjust the way Elasticsearch is configured. So you may want to learn how Elasticsearch templates are configured and how to get there.
You want to know more about indexing data into Elasticsearch.
You want to know more about bragi and how to query Elasticsearch via mimir, start here.
You can find more developer oriented documentation here
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
We use Semantic Versioning for versioning. For the versions available, see the tags on this repository.
Mimirsbrunn is a project initially started by Guillaume Pinot and Antoine Desbordes for Navitia.
See also the list of contributors who participated in this project.
If you use it too, feel free to open a pull request, we'll be happy to add your project here!
TODO: add a bit more detail on all the projects
All those projects use quite the same APIs, and you can compare their results using geocoder-tester.
For a more visual comparison, you can also use a comparator.
This project is licensed under the AGPLv3 GNU Affero General Public License - see the LICENSE.md file for details
- Billie Thompson - Provided README Template - PurpleBooth