Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ocean and land area is missing #720

Closed
BinoyKadakkal opened this issue May 13, 2024 · 13 comments
Closed

Ocean and land area is missing #720

BinoyKadakkal opened this issue May 13, 2024 · 13 comments

Comments

@BinoyKadakkal
Copy link

BinoyKadakkal commented May 13, 2024

Ocean and land area is missing while converting .osm.pbf file to .mbtiles. and shows the error

### Unable to open coastline/water_polygons.shp or coastline/water_polygons.SHP.
Reading shapefile urban_areas
Unable to open landcover/ne_10m_urban_areas/ne_10m_urban_areas.shp or landcover/ne_10m_urban_areas/ne_10m_urban_areas.SHP.
Reading shapefile ice_shelf
Unable to open landcover/ne_10m_antarctic_ice_shelves_polys/ne_10m_antarctic_ice_shelves_polys.shp or landcover/ne_10m_antarctic_ice_shelves_polys/ne_10m_antarctic_ice_shelves_polys.SHP.
Reading shapefile glacier
Unable to open landcover/ne_10m_glaciated_areas/ne_10m_glaciated_areas.shp or landcover/ne_10m_glaciated_areas/ne_10m_glaciated_areas.SHP.

Screenshot from 2024-05-13 16-12-04

Tilemaker is running in docker

@systemed
Copy link
Owner

From the readme:

To include sea tiles, create a directory called coastline in the same place you're running tilemaker from, and then save the files from https://osmdata.openstreetmap.de/download/water-polygons-split-4326.zip in it, such that tilemaker can find a file at coastline/water_polygons.shp.

The error you are getting shows that you don't have a file at the location where tilemaker is expecting to find it. Make sure that you don't have any extra levels of directory in there.

(Please use Github discussions, not issues, for help questions. Thank you :) )

@BinoyKadakkal
Copy link
Author

i saw that and i had created the folder as shown in the image and downloaded the data from https://osmdata.openstreetmap.de.
and done several trials by changing the location, rebuilding the docker image and extracting the file. It still shows the same error.

Screenshot from 2024-05-13 16-25-06

@systemed
Copy link
Owner

Ah, you're using Docker. Sorry, I don't know how the filesystem works there, hopefully someone else will.

@baurzhan
Copy link

baurzhan commented May 13, 2024

Run container without --rm option.
docker run -it --name=tilemaker -v $(pwd):/data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf /data/monaco-latest.mbtiles
Wait for the container to stop. Copy folders into container and restart the container:
docker cp landcover tilemaker:/usr/src/app/
docker cp coastline tilemaker:/usr/src/app/
docker start tilemaker

@BinoyKadakkal
Copy link
Author

BinoyKadakkal commented May 15, 2024

it still shows the same problem. after following the instructions i tried to run
docker run -v $(pwd):/data tilemaker /data/southern-zone-latest.osm.pbf /data/india.mbtiles

It also doesn't work

@systemed
Copy link
Owner

@daniel-j-h Could I trouble you for any thoughts on this?

@daniel-j-h
Copy link
Contributor

daniel-j-h commented May 15, 2024

I believe this has to do with absolute vs relative paths; in your config.json can you check if you have relative paths such as

  • coastline/water_polygons.shp and
  • landcover/ne_10m_urban_areas/ne_10m_urban_areas.shp

and turn them into absolute paths such as

  • /data/coastline/water_polygons.shp and
  • /data/landcover/ne_10m_urban_areas/ne_10m_urban_areas.shp

Then run it like

docker run -it --rm --pull always -v $(pwd):/data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf --output /data/monaco-latest.pmtiles --config /data/config-coastline.json --process /data/process-coastline.lua

I'm not sure where exactly tilemaker looks for those files otherwise - is it relative to the tilemaker binary by chance @systemed?


Two learnings here

  1. Should we have a script users can run to download this data, unzip it, put it in the right place; I have something like that already for my own experimentation and can upstream it if I have a bit more time throughout the week, and
  2. Should tilemaker get flags for coastline and landcover directories; I believe it's okay to have it in the config like this but maybe we need to put it into the documentation

@systemed
Copy link
Owner

I'm not sure where exactly tilemaker looks for those files otherwise - is it relative to the tilemaker binary by chance @systemed?

Everything is relative to the current working directory.

As far as tilemaker is concerned, coastline and landcover are just another datasource - there's no special handling. The config for a particular set of vector tiles might have both, or none, or lots more! So I don't think we'd want to add special handling for these paths.

Would something like a --base-dir option, which specifies the root path for the source data, be helpful? That way you could invoke it with something like

tilemaker input.osm.pbf output.mbtiles --base-dir /data

and it would then look for /data/input.osm.pbf, /data/coastline/water_polygons.shp, etc. etc.

@daniel-j-h
Copy link
Contributor

Not sure we need an extra option for tilemaker when you can change the paths in the config file and use both absolute and relative paths there. In addition there is a -w option (docs) for docker to change the current workdir, so we can just pass -w /data and make sure tilemaker sees the mounted data dir as the current working dir.

With relative paths in the configs and changing the workdir the command looks as follows

docker run -it --rm --pull always -v $(pwd):/data -w /data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf --output /data/monaco-latest.pmtiles --config /data/config-coastline.json --process /data/process-coastline.lua

@baurzhan
Copy link

baurzhan commented May 16, 2024

it still shows the same problem. after following the instructions i tried to run docker run -v $(pwd):/data tilemaker /data/southern-zone-latest.osm.pbf /data/india.mbtiles

It also doesn't work

"docker run" creates new container. In my instructions you shouldn't create new container. Just restart existing container. After docker restart tilemaker run docker logs -f tilemaker to see output

@baurzhan
Copy link

Updated instructions.

  1. Create container:
    docker create -i -t --name=tilemaker -v $(pwd):/data ghcr.io/systemed/tilemaker:master /data/monaco-latest.osm.pbf /data/monaco-latest.mbtiles
  2. Copy folders into container:
    docker cp landcover tilemaker:/usr/src/app/
    docker cp coastline tilemaker:/usr/src/app/
  3. Start container
    docker start tilemaker && docker logs -f tilemaker

@daniel-j-h
Copy link
Contributor

@BinoyKadakkal
Copy link
Author

It works, Thank you for the support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants