Remove the stack?

[jaakla]

The tagline says "... without the stack". It seems to require quite a stack through, any plans to remove some or all these:

• Google Protocol Buffers
• Boost 1.56 or later (for boost::geometry, boost::program_options, boost::filesystem, boost::variant)
• Lua 5.1
• sqlite3
• shapelib

[systemed]

I wouldn't say they form a stack - they're dependencies. Once you've built tilemaker, you have no need to maintain them.

In the majority of vector tile toolchains (as documented by Paul Norman), you need to separately maintain a PostGIS database and continually run a tile-server: that's the stack. With tilemaker, there is no need for a database and no server component (so, for example, you can ship your tiles as part of a mobile app without needing to run a server).

That said, removing dependencies is a noble aim. I would generally say that an acceptable dependency is one that can be installed through apt for a recent Ubuntu LTS release, or through homebrew for macOS. Now that we've removed Luabind, we've reached that happy state and installation should be trivial for Ubuntu and macOS users. If you choose to use another distribution we can hopefully assume you know what you're doing. ;)

Of the dependencies you list, Lua is of course crucial to tilemaker because it provides a programmatic way to create layers; and sqlite3 is required because it enables us to create .mbtiles outputs.

Removing Boost is not feasible as we'd have to write our own geometry library, and that way lies madness. ::geometry, ::program_options and ::variant are header-only, so could in theory be bundled with tilemaker, but ::filesystem is binary and would need to be replaced (which is doable).

Potentially shapelib could be made optional for those who don't want to use shapefiles; and protobuf could be replaced by protozero, which is header-only and therefore could be bundled with tilemaker.

I don't personally have any great ambition to do any of these three, but if someone were so motivated I'd be plenty happy to receive a pull request.

[jaakla]

Thanks for detailed explanation. My context is native (meaning C++) development for mobile OSs (Android, iOS etc), and over there the "real" stack in your terms is totally impossible. Also all the the additional libraries need to be carefully evaluated, and generally avoided, as there are a lot of additional limitations in storage/memory footprint size, possible version conflicts, licenses etc. Our current used library list is already quite long :)

[systemed]

Interesting. The idea of running tilemaker itself on a mobile device had never occurred to me, not least because it's very RAM-intensive. (The programmatic approach taken to generating vector tiles would probably forbid its use on iOS anyway.)

So if you want to use the tiles on mobile (which was partly my reasoning for developing tilemaker in the first place), I'd suggest that you run tilemaker in a server environment and that the mobile app/library then fetches the resulting .mbtiles, rather than the app fetching the .osm.pbf and converting it to vector tiles on-device.

[pnorman]

::geometry, ::program_options and ::variant are header-only, so could in theory be bundled with tilemaker

Please, no. Checking in a third-party library should be avoided.

I'd also agree that right now, tilemaker is stack-free.

[jaakla]

It loads all data to RAM before processing? Right, this could be a show-stopper for mobile case, especially for Android which is in many low-RAM devices. For server side we (CARTO) have already a solution - you can upload your data to CARTO service account and get back vector tiles via its Maps API. Certainly this has own limitations, it covers quite different usecases than tilemaker.

My point was really that there is blurry line between what you call stack and other "small" dependencies, depending on your context, they both can be very useful and/or big troubles.

[systemed]

Yep - it essentially loads the full .pbf into memory, so you need enough RAM to store all the nodes and ways and that's not really going to be likely on a mobile device.

Closing as not actionable but feel free to ask/comment more!

[pnorman]

tilemaker is also designed to work from OSM PBF formatted data. It can consume shapefile data, but that's not its primary goal.