many duplicate lines from relations

[gruzk]

It seems some relations are processed into duplicate lines. On a full planet import I have around 237 thousand duplicates (all with osm_id negative). My style file is unmodified. I am using osm2pgsql release version 0.92.1 with the typical command line as follows...

osm2pgsql -d <db> -C 34000 --cache-strategy optimized --number-processes 6 --multi-geometry --hstore -U <user> -c --flat-nodes nodes.tmp --slim --unlogged -S openstreetmap-carto.style -E 3395 -H xxx -W planet-160912.osm.pbf

Duplicates are identified by counting the number of features with the same osm_id and way, and for which the count is greater than one. Polygons and points do not contain any duplicates, only line and roads.

The worst example is a 700+km line "Asian Highway AH12" with osm_id -229600. It has 132 duplicates on one particular section. This relation in reality consists of only 79 unique parts but in the database has been created with 1043 parts (over 960 duplicates).

Currently I am removing the duplicates in postgresql with some SQL scripts after import, but it would be preferable not to have these duplicates created in the first place. Is there a fix for osm2pgsql?

[pnorman]

What query did you use to test this?

[gruzk]

Something like this would do it:

select osm_id, way, count(*) from planet_osm_line group by osm_id, way having count(*) > 1;

If it returns any rows then there are duplicates.

[pnorman]

Because AH12 is long enough to trigger long way splitting, I wonder if there's an interaction there.

[lonvia]

osm2pgsql doesn't do multilines for route relations. If the route does not result in a linear geometry, it is split.

Duplicate of #677.

[gruzk]

This is not about split lines, this is many identical lines. It is not the same issue as #677.

[pnorman]

osm2pgsql -d osm2pgsql_test r229600.osm reprodcues what is being described. The problem is a data error, not one with osm2pgsql. Some ways (e.g. 300785287) are a member of the relation many times.