Documentation: hotosm.github.io/raw-data-api
Source Code: github.com/hotosm/raw-data-api
API Documentation : https://api-prod.raw-data.hotosm.org/v1/redoc
Raw Data API
is a set of high-performant APIs for transforming and exporting OpenStreetMap (OSM) data in different GIS file formats.
-
Fast: Built on top of FastAPI
-
OAUTH 2.0 Authentication with OpenStreetMap(OSM)
-
Multiple GIS formats support via GDAL's ogr2ogr - see table below for currently supported formats. Out of which , GeoJSON Follows Own Raw Data API conversion script
Formats Status Esri Shapefile ✔️ KML ✔️ Mbtiles ✔️ FlatGeoBuf ✔️ CSV ✔️ GeoPackage ✔️ PGDUMP ✔️ GeoJSON ✔️ Pmtiles ✔️ Geoparquet ✔️
Raw Data API consists of two elements:
- A backend database, tools, and scripts: used to import OSM data into a specific database structure and keep it updated.
- An API that is used to serve data from the backend database.
To setup the backend see Backend Installation
Raw Data API can be installed through docker
or locally on your computer.
- To install with docker see docker installation.
- To install locally, see local installation.
Setup the necessary configurations for Raw Data API from configurations.
Setup config.txt in project root or include your env variables in .env
-
Confirm that Authetication works
- Hit the
/auth/login/
endpoint - Hit the
url
returned on the response - You will get an
access_token
- You can use the
access_token
in all endpoints that requires authentication. - To check token pass token in /auth/me/. It should return your OpenStreetMap (OSM) profile
- Hit the
-
Try extracting some data:
You can use the
/raw-data/current-snapshot/
endpoint with the following input to check both authentication, database connection and download the export:
curl -d '{"geometry":{"type":"Polygon","coordinates":[[[83.96919250488281,28.194446860487773],[83.99751663208006,28.194446860487773], [83.99751663208006,28.214869548073377],[83.96919250488281,28.214869548073377],[83.96919250488281,28.194446860487773]]]}}' -H 'Content-Type: application/json' http://127.0.0.1:8000/v2/raw-data/current-snapshot/
We use sozip optimization while we zip the files , SOZip makes it possible to access large compressed files directly from a .zip file without prior decompression. It is not a new file format, but a profile of the existing ZIP format, done in a fully backward compatible way. ZIP readers that are non-SOZip aware can read a SOZip-enabled file normally and ignore the extended features that support efficient seek capability. Learn more about it in attached link
- Raw Data API uses pytest for tests, navigate to the root directory and install package in editable mode:
pip install -e .
- At this point you should have PostgreSQL + PostGIS extension enabled on your computer, now run Pytest:
py.test -v -s
- Running individual tests
py.test -k test function name
Learn about current priorities and work going through Roadmap & see here CONTRIBUTING
see Roadmap
see LICENSE
For version control, We use commitizen, follow docs