Script for generating cell tower databases for NetworkLocation from the μg Project
The μg Project at https://github.com/microg is building a number of open source components to replace Google Mobile Services on Android phones. One of the older components is called NetworkLocation and it uses an on-phone SQLite database to resolve the locations of cell towers seen by the phone.
The Local-GSM-Backend is a replacement for microg's GSM-based location backend that allows importing from a CSV file. This script generates these CSV files on a per-country basis, the aim is that they can be downloaded easily from its settings screen.
- There is a faster script for generating the database located at https://github.com/sobrus/FastLacellsGenerator
- Recent versions of the Local GSM location backend now save their database within the application data area to reduce the privileges needed. This script still pushes to the old nogapps directory and will need to be modified to put the database where it will actually be used.
- bash - The scripts are written in bash
- wget - Used to pull CSV files from OpenCellId and Mozilla Location Services
- OpenCellID API Key - Needed to pull CSV files from OpenCellID. Get one for free
- SQLite3 - Used to generate the actual database for the phone
- adb - If you want to push the database file to a phone.
The main program is lacells-creator. To get a fresh file on your mobile phone, the program needs to download source files, process them, and upload to the phone. This can all be done by lacells-creator as follows:
API_KEY=xxx ./lacells-creator -d -i -c648 -m -p
The API_KEY is your OpenCellID API key, and 648 is the mobile country code (MCC). The full database is several gigabytes in size, so it's usually a good idea to select only the countries you're interested in. Multiple countries can be separated by comma's. You can lookup country codes in misc/mccs.csv or on Wikipedia.
The output would be something like the following:
Downloading cell tower data from OpenCellID
(...)
Importing mozilla cell data
(...)
Importing opencellid cell data
Filtering rows
Merging cell towers
Ready to push the database to your device? (y/n)
When you press y, the file will be uploaded to your phone.
Pushing database to phone
2500 KB/s (50000000 bytes in 20.000s)
Besides this, it is possible to export CSVs. Please see misc/usage.txt for details.
Please note that you can only download the OpenCellID database once per day.
To keep you from having to type your API_KEY or MCCs each time, this can be specified in a configuration file. It might contain
API_KEY=xxxx
MCC=404,405
Store this as creator.cfg in the same directory as the script, or in ~/.config/lacells/creator.cfg. The script will then pick up these defaults automatically (but you can still override the MCCs on the command-line).
If you'd like to use multiple configurations, please look at the command-line option -C or --config.
To automate it even further, you could create a simple script in the same directory that you can run instead. Open a text file, and put in the full command preceded by a shebang line:
#!/bin/sh
API_KEY=xxx ./lacells-creator -d -i -c404,405 -m -p
Make this file executable (chmod +x <filename>), and then you can run that command to do the full cycle of downloading, processing, uploading.
The total database size generated by this script is significantly larger than the one found through the links at http://forum.xda-developers.com/showthread.php?t=1715375
There are several reasons for that:
- That database was created nearly two years ago and the database at OpenCellID has grown since then.
- This script add towers from Mozilla Location Services.
- I have added a couple of fields.
- I have created indices for faster run time access.
To the first point, the 95,171,584 byte cells-world.db contains 2,486,708 cell towers. As of 20Dec2014, a database from OpenCellId for the whole world will contain 6,891,208 towers (177% increase in tower count).
To the second point, adding Mozilla data increases the non-duplicate tower count to 8,456,327 a total 240% greater than the original database.
To the last point, adding indices for faster run time access increase the non-duplicate combined database from 423,677,952 to 729,165,824 bytes (72% increase).
19Dec2014: If you need a lacells.db file for use by my gsm-location backend located at https://github.com/n76/Local-GSM-Backend then you can use the settings menu for that backend to generate the database on the phone with no external script.
It is significantly faster to generate the database file on a laptop or desktop computer but more convenient to generate it on the phone so I am leaving this script available.