Squashed 'third_party/libosmium/' changes from 80df1d6..40c4a48

40c4a48 Release v2.10.0
6addb2e Search for protozero also in the place where libosmium was found.
9c0d4bb Consistently use lhs and rhs as parameter names in operators etc.
6179759 Update change log.
894eb7d Explicitly use size_t to get no narrowing conversions.
e549e73 Fix CMake config for index_lookup.
ccf0bc2 Fix bug where some compilers deduce the wrong type.
fc3507d Cleaned and documented index example and renamed to osmium_index_lookup.
d3c3036 Rename serdump example to dump_internal and document it.
0e9822e Also forward set_uid_from_signed() function.
29ef95c Rename build_taglist function because it was to similar to build_tag_list.
c088dd0 Move static constant out of class so clang will compile it.
606cdc4 Fix CMake build script: Path for finding libosmium.
65f91fe Removed unused typedef.
07174f5 Use condition_variable to tell producer when thread queue is not full.
b35e957 Some code cleanup in XML parser.
c703dff Fix problem with MemberMeta.
1ccbbef Refactoring CompressionFactory.
6561bd6 Use our own exception type for map factory errors.
56e31fa Throw not_found error directly instead of using helper function.
28230c3 Test empty value for node location store, reorganize tests.
2ba316c Reorganize and fix projection tests.
f949485 Don't run quite as many tests with projections because they are slow.
1bad16d Add information about build environment to benchmark output.
99617bb Change proj_geojson benchmark into mercator benchmark.
553b946 Allow optional parameters on Reader in any order.
dcc3d8f Factor out some common code.
004d8cd Fix forwarding constructor.
9702978 Fix metadata check.
6cfb6c4 Faster implementation of decode_dense_nodes without reading metdata.
4ba4638 Add additional read_metadata parameter to other Reader constructors.
d005937 Optionally ignore metadata when reading file.
63961da Mark all CRC32 update functions as noexcept.
cc4ca75 Refactor set_user() function to speed it up.
38d19dc Update comments with file sizes. OSM has grown...
f7d0824 Add new benchmark that shows performance when main thread is busy.
25070dc Use const ref params.
88e8d96 Mark add_user() as deprecated.
f58d9db Refactor some low-level collection code to clean up code.
4680def Add example showing how to create OSM data out of thin air.
d42fd50 Add an example showing how tags in OSM files can be changed.
49bf5bc Add additional constructors to Builders taking a reference to parent.
7b91d63 Change Builder::add_item() to take a reference instead of pointer.
2957e48 Some cleanup of examples.
d0b458d Calculate size of object at compile time.
3fbb6e7 Use explicit cast.
1851f3d Remove a test that depends on math details.
f6a0802 Various cleanups of example programs.
ba4921f Rename add_user() to set_user().
d7637c9 Various cleanups related to builders.
07827bc Fix add_user().
9a5b395 Also refactor OSMObjectBuilder like ChangesetBuilder before.
b1f423c Use call chaining on the builder.
e49473d Get rid of ObjectBuilder class.
67d70b9 Refactor ChangesetBuilder::add_user().
8199c33 Make ChangesetBuilder derive directly from Builder.
61d1b67 Simplify some code.
d38467a Change derivation hierarchy of some builder classes.
b52f8af Refactor Builder code.
d012bfa Refactored some code setting attributes on objects using builder instead.
6a05f60 Also forward set_removed function from builders.
8d63b7d Return *this from Builder setter functions and test it.
72a1266 Update catch.hpp to newest version.
3424a74 Check GEOS version is <= 3.5.
aee9f9d Cleanup test code.
aef198c Improved asserts in Buffer.
a98b9bf Code cleanup in tests.
a150466 Use GDAL/OGR instead of GEOS to test our WKB implementation.
b04a525 Refactor test.
39aa932 Refactored test_factory_with_projection so it works without GEOS.
648f43a Remove unused dependency on geos from tile test.
f1748ae Add setters to Builder classes forwarding to underlying objects.
8166879 In debug mode check that Builders are correctly called and destructed.
1c4257e Call commit() on buffer only after all builders are destructed.
2618636 Add functions to check availability of relation members.
b45a4d9 Mark RelationMember::set_ref() as deprecated.
7886771 Move "basic" and "buffer" tests to "osm" and "memory", respectively.
b664685 Use functions instead of macros in location test for faster compile.
b4929ac Add more tests for number parser.
02662a7 Merge pull request #171 from lonvia/fix-long-exponentials
5344a6c fix parsing of numbers in e-notiation with many post-comma digits
3aeaff3 Add some typedefs to NodeRefList and memory::Collection.
e750665 Add iterators to IdSetSmall and add docs and tests to IdSet classes.
50ecb2a Add more features to IdSetDense, including unset and iterator.
e3dec78 Make IdSet virtual base class with two implementations.
8ea0153 Use C array instead of std::array in IdSet and clear explicitly.
3ba9905 Deprecate osmium::index::BoolVector in favour of new IdSet.
453d1ca Add osmium::index::IdSet.
c78254e Add function to (temporarily) disable the progress bar.
4d88a9f Better document osmium::io::Header class.
320e3af Look for protozero includes in CMake config.
838d25e Allow optional checking for libosmium version number in CMake config.
6ce60c1 Fix entity_bits static_assert() tests.
f054731 Update change log.
77ac4c2 Make sleep duration for full queues much smaller.
7e39c01 Make some entity_bits functions constexpr.
69ea72f Fix ~ operator on entity_bits and more tests for entity bits.
dafe4cf Update embedded Catch unit test header to version 1.5.7.
a41c832 Fixed parsing of location coordinates starting with decimal dot.
6523bae README cosmetics.
229acac Add tests for some examples.
f1e753d Merge pull request #163 from sebastic/executable-not-elf-or-script
ccea2d5 Remove executable bit from .cpp files.
af77fb4 Changelog formatting fixes.

git-subtree-dir: third_party/libosmium
git-subtree-split: 40c4a48f88d25edace6f0b9e079c306308c7760b

CHANGELOG.md

@@ -13,6 +13,53 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 ### Fixed
 
 
+## [2.10.0] - 2016-11-11
+
+### Added
+
+- The `Reader` can take an additional optional `read_meta` flag. If this is
+  set to false the PBF input will ignore metadata on OSM objects (like version,
+  timestamp, uid, ...) which speeds up file reading by 10 to 20%.
+- New `IdSet` virtual class with two implementations: `IdSetDense` and
+  `IdSetSmall`. Used to efficiently store a set of Ids. This is often needed
+  to track, for instance, which nodes are needed for ways, etc.
+- Added more examples and better documented existing examples.
+- Add a benchmark "mercator" converting all node locations in a file to
+  WebMercator and creating geometries in WKB format.
+
+### Changed
+
+- Better queue handling makes I/O faster in some circumstances.
+- The `FindOsmium.cmake` CMake script can now check a current enough libosmium
+  version is found.
+- Builders can now be constructed with a reference to parent builder.
+- Made builders more robust by adding asserts that will catch common usage
+  problems.
+- Calling `OSMObjectBuilder::add_user()` is now optional, and the method was
+  renamed to `set_user()`. (`add_user()` is marked as deprecated.)
+- Benchmarks now show compiler and compiler options used.
+- `Builder::add_item()` now takes a reference instead of pointer (old version
+  of the function marked as deprecated).
+- GEOS support is deprecated. It does not work any more for GEOS 3.6 or newer.
+  Reason is the changed interface in GEOS 3.6. If there is interest for the
+  GEOS support, we can add support back in later (but probably using the
+  GEOS C API which is more stable than the C++ API). Some tests using GEOS
+  were rewritten to work without it.
+- The `BoolVector` has been deprecated in favour of the new `IdSet` classes.
+- Lots of code cleanups and improved API documentation in many places.
+- The relations collector can now tell you whether a relation member was in
+  the input data. See the new `is_available()` and
+  `get_availability_and_offset()` methods.
+- Updated embedded Catch unit test header to version 1.5.8.
+
+### Fixed
+
+- Parsing of coordinates starting with decimal dot and coordinates in
+  scientific notation.
+- `~` operator for `entity_bits` doesn't set unused bits any more.
+- Progress bar can now be (temporarily) removed, to allow other output.
+
+
 ## [2.9.0] - 2016-09-15
 
 ### Added
@@ -110,7 +157,7 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 - New functions for iterating over specific item types in buffers
   (`osmium::memory::Buffer::select()`), over specific subitems
   (`osmium::OSMObject::subitems()`), and for iterating over all rings of
-  an area (`osmium::Areas::outer_rings(`), `inner_rings()`).
+  an area (`osmium::Areas::outer_rings()`, `inner_rings()`).
 - Debug output optionally prints CRC32 when `add_crc32` file option is set.
 
 ### Changed
@@ -267,9 +314,9 @@ This project adheres to [Semantic Versioning](http://semver.org/).
   one in Writer. Calling flush() on the OutputIterator isn't needed any
   more.
 - Reader now throws when trying to read after eof or an error.
-- I/O functions that used to throw std::runtime_error now throw
-  osmium::io_error or derived.
-- Optional parameters on osmium::io::Writer now work in any order.
+- I/O functions that used to throw `std::runtime_error` now throw
+  `osmium::io_error` or derived.
+- Optional parameters on `osmium::io::Writer` now work in any order.
 
 ### Fixed
 
@@ -410,7 +457,8 @@ This project adheres to [Semantic Versioning](http://semver.org/).
   Doxygen (up to version 1.8.8). This version contains a workaround to fix
   this.
 
-[unreleased]: https://github.com/osmcode/libosmium/compare/v2.9.0...HEAD
+[unreleased]: https://github.com/osmcode/libosmium/compare/v2.10.0...HEAD
+[2.10.0]: https://github.com/osmcode/libosmium/compare/v2.9.0...v2.10.0
 [2.9.0]: https://github.com/osmcode/libosmium/compare/v2.8.0...v2.9.0
 [2.8.0]: https://github.com/osmcode/libosmium/compare/v2.7.2...v2.8.0
 [2.7.2]: https://github.com/osmcode/libosmium/compare/v2.7.1...v2.7.2

CMakeLists.txt

@@ -24,7 +24,7 @@ set(CMAKE_CONFIGURATION_TYPES "Debug;Release;RelWithDebInfo;MinSizeRel;Dev;Cover
 project(libosmium)
 
 set(LIBOSMIUM_VERSION_MAJOR 2)
-set(LIBOSMIUM_VERSION_MINOR 9)
+set(LIBOSMIUM_VERSION_MINOR 10)
 set(LIBOSMIUM_VERSION_PATCH 0)
 
 set(LIBOSMIUM_VERSION
@@ -285,6 +285,10 @@ if(BUILD_DATA_TESTS)
     add_subdirectory(test/data-tests)
 endif()
 
+if(BUILD_EXAMPLES)
+    add_subdirectory(test/examples)
+endif()
+
 
 #-----------------------------------------------------------------------------
 #

README.md

@@ -4,8 +4,8 @@ http://osmcode.org/libosmium
 
 A fast and flexible C++ library for working with OpenStreetMap data.
 
-[![Build Status](https://secure.travis-ci.org/osmcode/libosmium.png)](https://travis-ci.org/osmcode/libosmium)
-[![Build status](https://ci.appveyor.com/api/projects/status/mkbg6e6stdgq7c1b?svg=true)](https://ci.appveyor.com/project/Mapbox/libosmium)
+[![Build Status](https://secure.travis-ci.org/osmcode/libosmium.svg)](https://travis-ci.org/osmcode/libosmium)
+[![Build status](https://ci.appveyor.com/api/projects/status/github/osmcode/libosmium?svg=true)](https://ci.appveyor.com/project/Mapbox/libosmium)
 
 Libosmium is developed on Linux, but also works on OSX and Windows (with some
 limitations).

benchmarks/CMakeLists.txt

@@ -12,6 +12,7 @@ set(BENCHMARKS
     count
     count_tag
     index_map
+    mercator
     static_vs_dynamic_index
     write_pbf
     CACHE STRING "Benchmark programs"
@@ -37,6 +38,8 @@ foreach(benchmark ${BENCHMARKS})
                    @ONLY)
 endforeach()
 
+string(TOUPPER "${CMAKE_BUILD_TYPE}" _cmake_build_type)
+set(_cxx_flags "${CMAKE_CXX_FLAGS_${_cmake_build_type}}")
 foreach(file setup run_benchmarks)
     configure_file(${file}.sh ${CMAKE_CURRENT_BINARY_DIR}/${file}.sh @ONLY)
 endforeach()

benchmarks/download_data.sh

@@ -4,9 +4,9 @@
 #
 
 cd $DATA_DIR
-curl --location --output 1_liechtenstein.osm.pbf http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf   # about   1 MB
-curl --location --output 2_bremen.osm.pbf        http://download.geofabrik.de/europe/germany/bremen-latest.osm.pbf  # about  13 MB
-curl --location --output 3_sachsen.osm.pbf       http://download.geofabrik.de/europe/germany/sachsen-latest.osm.pbf # about 120 MB
-curl --location --output 4_germany.osm.pbf       http://download.geofabrik.de/europe/germany-latest.osm.pbf         # about   2 GB
-curl --location --output 5_planet.osm.pbf        http://planet.osm.org/pbf/planet-latest.osm.pbf                    # about  26 GB
+curl --location --output 1_liechtenstein.osm.pbf http://download.geofabrik.de/europe/liechtenstein-latest.osm.pbf   # about   2 MB
+curl --location --output 2_bremen.osm.pbf        http://download.geofabrik.de/europe/germany/bremen-latest.osm.pbf  # about  16 MB
+curl --location --output 3_sachsen.osm.pbf       http://download.geofabrik.de/europe/germany/sachsen-latest.osm.pbf # about 160 MB
+curl --location --output 4_germany.osm.pbf       http://download.geofabrik.de/europe/germany-latest.osm.pbf         # about   3 GB
+curl --location --output 5_planet.osm.pbf        http://planet.osm.org/pbf/planet-latest.osm.pbf                    # about  35 GB
 

benchmarks/osmium_benchmark_count.cpp

@@ -19,15 +19,15 @@ struct CountHandler : public osmium::handler::Handler {
     uint64_t ways = 0;
     uint64_t relations = 0;
 
-    void node(osmium::Node&) {
+    void node(const osmium::Node&) {
         ++nodes;
     }
 
-    void way(osmium::Way&) {
+    void way(const osmium::Way&) {
         ++ways;
     }
 
-    void relation(osmium::Relation&) {
+    void relation(const osmium::Relation&) {
         ++relations;
     }
 

benchmarks/osmium_benchmark_count_tag.cpp

@@ -18,19 +18,19 @@ struct CountHandler : public osmium::handler::Handler {
     uint64_t counter = 0;
     uint64_t all = 0;
 
-    void node(osmium::Node& node) {
+    void node(const osmium::Node& node) {
         ++all;
         const char* amenity = node.tags().get_value_by_key("amenity");
         if (amenity && !strcmp(amenity, "post_box")) {
             ++counter;
         }
     }
 
-    void way(osmium::Way&) {
+    void way(const osmium::Way&) {
         ++all;
     }
 
-    void relation(osmium::Relation&) {
+    void relation(const osmium::Relation&) {
         ++all;
     }
 

benchmarks/osmium_benchmark_mercator.cpp

@@ -0,0 +1,43 @@
+/*
+
+  The code in this file is released into the Public Domain.
+
+*/
+
+#include <cstdint>
+#include <cstdlib>
+#include <iostream>
+#include <string>
+
+#include <osmium/io/any_input.hpp>
+#include <osmium/handler.hpp>
+#include <osmium/visitor.hpp>
+#include <osmium/geom/wkb.hpp>
+#include <osmium/geom/mercator_projection.hpp>
+
+struct GeomHandler : public osmium::handler::Handler {
+
+    osmium::geom::WKBFactory<osmium::geom::MercatorProjection> factory;
+
+    void node(const osmium::Node& node) {
+        const std::string geom = factory.create_point(node);
+    }
+
+};
+
+
+int main(int argc, char* argv[]) {
+    if (argc != 2) {
+        std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
+        std::exit(1);
+    }
+
+    const std::string input_filename{argv[1]};
+
+    osmium::io::Reader reader{input_filename};
+
+    GeomHandler handler;
+    osmium::apply(reader, handler);
+    reader.close();
+}
+

benchmarks/run_benchmark_mercator.sh

@@ -0,0 +1,22 @@
+#!/bin/sh
+#
+#  run_benchmark_mercator.sh
+#
+
+set -e
+
+BENCHMARK_NAME=mercator
+
+. @CMAKE_BINARY_DIR@/benchmarks/setup.sh
+
+CMD=$OB_DIR/osmium_benchmark_$BENCHMARK_NAME
+
+echo "# file size num mem time cpu_kernel cpu_user cpu_percent cmd options"
+for data in $OB_DATA_FILES; do
+    filename=`basename $data`
+    filesize=`stat --format="%s" --dereference $data`
+    for n in $OB_SEQ; do
+        $OB_TIME_CMD -f "$filename $filesize $n $OB_TIME_FORMAT" $CMD $data 2>&1 >/dev/null | sed -e "s%$DATA_DIR/%%" | sed -e "s%$OB_DIR/%%"
+    done
+done
+

benchmarks/setup.sh

@@ -9,6 +9,10 @@ if [ -z $DATA_DIR ]; then
 fi
 
 OB_DIR=@CMAKE_BINARY_DIR@/benchmarks
+OB_BUILD_TYPE=@CMAKE_BUILD_TYPE@
+OB_COMPILER=@CMAKE_CXX_COMPILER@
+OB_COMPILER_VERSION=`$OB_COMPILER --version | head -1`
+OB_CXXFLAGS="@_cxx_flags@"
 
 OB_RUNS=3
 OB_SEQ=`seq -s' ' 1 $OB_RUNS`
@@ -20,11 +24,17 @@ OB_DATA_FILES=`find -L $DATA_DIR -mindepth 1 -maxdepth 1 -type f | sort`
 
 echo "BENCHMARK: $BENCHMARK_NAME"
 echo "---------------------"
+echo "BUILD:"
+echo "build type\t: $OB_BUILD_TYPE"
+echo "compiler\t: $OB_COMPILER"
+echo "CXX version\t: $OB_COMPILER_VERSION"
+echo "CXX flags\t: $OB_CXXFLAGS"
+echo "---------------------"
 echo "CPU:"
 grep '^model name' /proc/cpuinfo | tail -1
-grep '^cpu MHz' /proc/cpuinfo | tail -1
-grep '^cpu cores' /proc/cpuinfo | tail -1
-grep '^siblings' /proc/cpuinfo | tail -1
+grep '^cpu MHz'    /proc/cpuinfo | tail -1
+grep '^cpu cores'  /proc/cpuinfo | tail -1
+grep '^siblings'   /proc/cpuinfo | tail -1
 
 echo "---------------------"
 echo "MEMORY:"