Isochrone Plugin v2 #3652

danpat · 2017-02-04T08:10:41Z

This PR adds an isochrone plugin, which can retrieve travel-time isochrones from the OSRM routing graph.

This is an alternative implementation to #2477 that does not need to load the node-based-graph into memory - instead, it derives a temporary edge-based-graph from the CH. It's not super fast, but it works :-)

It functions as follows:

Calculates a maximum bounding box for the isochrone using the range x MAX_SPEED_METERS_PER_SECOND (200km/h for this implementation)
Fetches everything in that box from the RTree
Identifies all the distinct edge-based-nodes returned from the RTree
Uses the start/end node-based-node IDs to identify potentially directly connected edge-based-nodes
Builds a temporary edge-based-graph by testing the potentially directly connected edge-based-nodes in the CH
Performs a Dijkstra search on the temporary edge-based-graph, terminating when all possible paths up to range have been explored
Returns the result

Usage is:

/isochrone/v1/driving/{lon},{lat}?range={seconds}

A 300 second isochrone on the car profile takes about 350ms to return.

Currently, the plugin returns a GeoJSON FeatureCollection for every edge-based-node geometry within the range.

TODO:

add a type parameter, and support:
- exact - return a custom JSON structure for the full isochrone tree
- points - return a big GeoJSON FeatureCollection of Point features with distance/duration for each point, for use with third-party topography tools, like gdal_contour
- polygon - return the boundary polygon for the isochrone (TODO: what type? convex? concave? other params?)
add flag to limit isochrone max range (big isochrones get really expensive)
review
adjust for comments

This reverts commit 859403c.

… This commit is the first step, the second step is a reverse search, still to come.

…to speed things up.

… step is to calculate a convex hull.

… todo: unpack geometry and backtrack, and calculate concave hull.

Include example viewer for isochrones (just point clouds so far).

…edges in the search radius, not sure why yet.

TheMarex · 2017-02-26T01:36:16Z

src/engine/plugins/isochrone.cpp

+    TIMER_START(TABLE_TIMER);
+    std::sort(phantoms.begin(), phantoms.end(), [](const auto &a, const auto &b) {
+        const auto &a_id =
+            a.forward_segment_id.enabled ? a.forward_segment_id.id : a.reverse_segment_id.id;


Sadly the node ID is not a proxy for contraction level in our CH implementation: We keep the node ID constant starting in the EdgeBasedGraphFactory, because we need to save the ID in the RTree which happens before contraction.

The way you could fix this would be to load the .osrm.level file which contains exactly that (we use it for speeding up the contraction on successive contractions).

Welp, that probably explains the missing edges in the test isochrones I've been making.

I'll see if I can get correct results using the .level file, then we can discuss how we might optionally load it.

Optional loading is also desirable for #2764 (comment), so I'm feeling like there's a pattern here.

Interestingly, the PHAST paper here: https://www.microsoft.com/en-us/research/wp-content/uploads/2010/09/phastTR.pdf

states:

The second subphase scans all vertices in G↓ in descending rank order.*

For correctness, any reverse topological order will do.

I think I see a way to do that kind of sorting, I'll give it a go.

@danpat I'm curious about what you have in mind for the sorting, would you care to share some details ?

Dirk23 · 2017-06-01T09:14:56Z

is it posible to set a max speed like this isochrone server is able to?

Update to align with changes in various interfaces: BasePlugin, DataFacade, PhantomNode, many-to-many search algorithm and others used by the Isochrone plug-in (submitted in Project-OSRM#3652) Note, max_weight extra argument added to many-to-many algorithm routines in the original implementation has not been ported as it appears to be unused.

mloskot · 2017-08-08T15:31:51Z

Here in ml/5.10-ch_isochrone_poc branch is my first attempted to rebase-merge this PR on top of 5.10 branch (current release): https://github.com/mloskot/osrm-backend/tree/ml/5.10-ch_isochrone_poc
which contains single commit resolving all the conflicts (mloskot/osrm-backend@f3d2b19).

This allows to build Isochrone plugin with GCC 7 and VS2017.
I haven't managed to try it out which success though, due to some run-time issues with osrm::util::QueryHeap in debug builds with VS2017, which I'm investigating.

TODO: I'd appreciate if someone reviewed the merge and checked if it didn't loose any crucial parts.

adomas-mazeikis · 2017-08-09T13:58:56Z

@mloskot Have you had any success fixing the issue with QueryHeap?

mloskot · 2017-08-09T16:31:40Z

@adomas-mazeikis Not yet, but the issue seems MSVC-specific annoyance and iterator debug levels interfering with testing execution of the plugin. I have asked for advice on boost-users: [heap] Singular handle issue with MSVC iterator debug facilities and copied to https://stackoverflow.com/q/45608426/151641

~~Another quick weird observations I'm experiencing trying out the plugin is around the "Phase 2" step: accessing these a_id and b_id references in a_id < b_id give me read access violation~~

    std::sort(phantoms.begin(), phantoms.end(), [](const auto &a, const auto &b) {
        const auto &a_id =
            a.forward_segment_id.enabled ? a.forward_segment_id.id : a.reverse_segment_id.id;
        const auto &b_id =
            b.forward_segment_id.enabled ? b.forward_segment_id.id : b.reverse_segment_id.id;

        return a_id < b_id;
    });

SOLVED: Initialisation of reference from bit fields. C++ allows const-reference (via temporary trick). Apparently, MSVC does not allow it (see restrictions on bit fields).

If I run the plugin, I get "type":"FeatureCollection" result with collection of Point geometries and Weight properties. However, I haven't checked correctness of the result.

(Soon, I'll be gone offline for 1-3 weeks, I will continue as soon as I'm back.)

Update to align with changes in various interfaces: BasePlugin, DataFacade, PhantomNode, many-to-many search algorithm and others used by the Isochrone plug-in (submitted in Project-OSRM#3652) Note, max_weight extra argument added to many-to-many algorithm routines in the original implementation has not been ported as it appears to be unused.

mloskot · 2017-08-10T20:54:21Z

src/engine/plugins/isochrone.cpp

+{
+
+    // const auto range = parameters.range;
+    const auto range = 300;


Is range intentionally signed int?

Compare with unsigned IsochroneParameters::range declaration.

The type difference actually does affect the isochrones result (see my next #3652 (comment)).

mloskot · 2017-08-10T21:07:22Z

src/engine/plugins/isochrone.cpp

+                const EdgeWeight to_weight = weight + edge_weight;
+
+                // No need to search further than our maximum radius
+                if (weight > range * 10)


TL;TR: Is it intentional here to mismatch signed/unsigned types and rely on signed int to unsigned int implicit conversion (eg. to filter nodes with negative weights)?

In my previous #3652 (comment) above I asked about signed/unsigned mismatch of range.
This weight > range * 10 test is where the difference matters:

weight is signed int

if range is unsigned int (eg. pulled as const auto range = parameters.range; earlier)
then result of comparison test is different than, I assume, expected.

For example, if weight value is negative, it is implicitly converted to unsigned int, according to the usual arithmetic conversions pattern (C++/5/9), and overflow flips the result loosing some nodes.

Never ascribe to cleverness that which can be explained by oversight.

I probably just didn't notice :-)

Note that this plugin has a serious problem with the reverse graph exploration currently - see the comments by @TheMarex above.

@danpat for a moment, I assumed the obscure aims for actual utilisation of the C++ dark fu to benefit :-) One of those "failed by auto" trips. I will correct in my 5.10/master-based branches of the plugin.

I'm exploring the plugin and trying to get over some basic bumps first.

mloskot · 2017-08-10T22:32:37Z

src/engine/plugins/isochrone.cpp

+        const auto &a_id =
+            a.forward_segment_id.enabled ? a.forward_segment_id.id : a.reverse_segment_id.id;
+        const auto &b_id =
+            b.forward_segment_id.enabled ? b.forward_segment_id.id : b.reverse_segment_id.id;


This initialisation of (const) reference from bit field is not allowed by MSVC (including a program flipping over with access violation at run-time).

It's a grey area. While C++ std does not allow to initialise pointer and non-const reference from a bit field, it does allow it for const references, but via a trick: binding a new copy-constructed temporary. Technically, there is no gain in defining auto& vs plain auto value here.

BTW, I have greped through the whole codebase looking for similar references assigned with SegmentID members and I didn't found any apart from these in isochrone.cpp. Looks good.

mloskot · 2017-08-10T23:27:12Z

@adomas-mazeikis QueryHeap w/ MSVC issue solved in #4396

Update to align with changes in various interfaces: BasePlugin, DataFacade, PhantomNode, many-to-many search algorithm and others used by the Isochrone plug-in (submitted in Project-OSRM#3652) Note, max_weight extra argument added to many-to-many algorithm routines in the original implementation has not been ported as it appears to be unused.

CraigglesO · 2018-01-25T21:03:31Z

Are there any updates on this project? I played with it a little. Some weird issues if anyone else uses this PR:

Setup the map inside the osrm-backend folder, otherwise you get weird errors when using osrm-routed.
for some reason on linux, this PR cannot find stxxl, just direct install it sudo apt-get install libstxxl-dev

Thanks for your work though, it's very interesting.

mloskot · 2018-01-30T20:01:17Z

@ CraigglesO

Are there any updates on this project?

I'm not aware of any. I stopped evaluating this PR in Aug 2017

This PR is far behind the current master (which, n.b., moves foward with light speed) and it would be nice to rebase it, so it's easier to try it out again. I may give it a go some time during next weeks.

Setup the map inside the osrm-backend folder

What do you mean exactly? What map? Setup how?

for some reason on linux, this PR cannot find stxxl, just direct install it sudo apt-get install libstxxl-dev

Current master no longer requires STXXL, it's an optional dependency now. Again, rebase would be nice.

geography · 2018-02-02T15:17:50Z

Hi all, any news about this plugin?

mloskot · 2018-02-02T17:02:25Z

On 2 February 2018 at 16:56, Emanuele Notarnicola ***@***.***> wrote: Hi all, any news about this plugin?

How about reading the very recent comments from 2-3 days ago?

CraigglesO · 2018-02-07T18:00:43Z

LMAO. hey man. Thanks for the response. Sorry for not replying sooner. Got sidetracked and github is pretty gnarly about not showing old notifications.

for setup inside osrm-backend I mean that if you run osrm-extract with a profile setup outside of the osrm-backend folder (right after git clone and cmake....) then you run into issues. However, if you use osrm-extract on a pbf file INSIDE the osrm-backend folder no errors come up. It's just a weird bug or I am just doing something wrong.

Good to hear about stxxl though.

Thanks for your work its inspiring. I wrote an implementation that uses a 1080ti GTX and it takes about 1s for 15min car, 7s for 30min car and about 22-30s for 1hr car. However, if your not in a city, 1hr car takes less then a second BAHAHAA. It's kind of funny. I was having a lot of trouble wrapping my head around all of OSRM so I wrote a basic r-tree for all the nodes and edges and keep all of them on RAM over storage for now. LOTS of improvement to go.

ghost · 2018-08-14T16:50:05Z

I would really, really like to see this functionality implemented. It seems multiple individuals have tried to create this plugin, and it was never accepted. Right now, I'm generating isochrones using a grid and durations table. This is not ideal, either in terms of efficiency or accuracy. With the grid, there is always a trade-off between accuracy and calculation time. Taking 8 minutes to generate isochrones for a 1000x1000 grid is just too long, when using this R-Tree method would seem to be much more efficient and effective. Yet, where is it after all this time? I've searched and searched this repository and documentation. Is this being implemented or planned for this project? How can the efforts of those who have spent a lot of time on this already be rewarded?

mloskot · 2018-08-14T17:07:26Z

@tgrushka Try Valhalla, https://github.com/valhalla/valhalla-docs/blob/master/isochrone/api-reference.md

danpat · 2018-08-14T18:03:15Z

Sadly, this was a side project for me, and I have not had time to revisit it.

If you really need isochrones, I second @mloskot suggestion of using Valhalla.

Another option, which uses the same approach as Valhalla, is https://github.com/mapbox/mapbox-isochrone by @peterqliu - it uses the table/matrix plugin to sample, and calculates a concave polygon around the points (Valhalla does this internally too). This script could be modified to hit an OSRM instance if you like, instead of the Mapbox Matrix API.

This PR was about generating a precise isochrone. Maybe I'll get back to it one day....

DennisOSRM · 2024-05-10T17:15:04Z

closing stale PR. Reopen if still relevant.

danpat added the Work In Progress label Feb 4, 2017

danpat self-assigned this Feb 4, 2017

danpat mentioned this pull request Feb 4, 2017

Isochrone Plugin #2189 #2477

Closed

danpat added 6 commits February 16, 2017 13:58

Proof-of-concept isochrone generation from the CH.

c790c91

Start of CH/PHAST

a22a38e

Catchup with changes on master.

253a8ef

Make API match other services.

5e06735

Revert "Start of CH/PHAST"

c206557

This reverts commit 859403c.

Set default range to 5 minutes.

5018ef2

danpat force-pushed the ch_isochrone_poc branch from 022c931 to 5018ef2 Compare February 16, 2017 21:58

danpat added 7 commits February 16, 2017 21:23

Don't build custom graph - we can do what we need directly on the CH.…

caa12ed

… This commit is the first step, the second step is a reverse search, still to come.

Reverse search from all nodes in the search box. Still todo: memoize …

ebe5e90

…to speed things up.

Do a one-to-many table search instead, and return a point cloud. Next…

3240e67

… step is to calculate a convex hull.

Only search each edge-based-node once - not every road segment. Still…

1359505

… todo: unpack geometry and backtrack, and calculate concave hull.

Search each road in both directions as separate targets

40a119d

Terminate many-to-many search early properly in the backwards phase.

4c43131

Include example viewer for isochrones (just point clouds so far).

PHAST - first cut, seems to roughly work, but is missing a few nodes/…

bcf961c

…edges in the search radius, not sure why yet.

TheMarex reviewed Feb 26, 2017

View reviewed changes

daniel-j-h mentioned this pull request Apr 20, 2017

Capping Distance/Time or just returing K Nearest Points in Table Service #3963

Closed

Fkawala mentioned this pull request May 23, 2017

Detour computation - implementation guidance #4015

Closed

mloskot reviewed Aug 10, 2017

View reviewed changes

wangyoucao577 mentioned this pull request Oct 20, 2020

Isochrone service Telenav/osrm-backend#376

Open

mjjbell mentioned this pull request Dec 13, 2020

Isochrone Support #5910

Open

tomalrussell mentioned this pull request Aug 12, 2021

Planned improvement : batch routing UDST/pandana#120

Open

DennisOSRM closed this May 10, 2024

DennisOSRM deleted the ch_isochrone_poc branch May 10, 2024 17:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Isochrone Plugin v2 #3652

Isochrone Plugin v2 #3652

danpat commented Feb 4, 2017 •

edited

Loading

TheMarex Feb 26, 2017

danpat Feb 26, 2017 •

edited

Loading

danpat Feb 26, 2017 •

edited

Loading

Fkawala Jun 14, 2017

Dirk23 commented Jun 1, 2017

mloskot commented Aug 8, 2017 •

edited

Loading

adomas-mazeikis commented Aug 9, 2017

mloskot commented Aug 9, 2017 •

edited

Loading

mloskot Aug 10, 2017 •

edited

Loading

mloskot Aug 10, 2017 •

edited

Loading

danpat Aug 10, 2017

mloskot Aug 10, 2017 •

edited

Loading

mloskot Aug 10, 2017 •

edited

Loading

mloskot commented Aug 10, 2017

CraigglesO commented Jan 25, 2018

mloskot commented Jan 30, 2018

geography commented Feb 2, 2018

mloskot commented Feb 2, 2018 via email

CraigglesO commented Feb 7, 2018

ghost commented Aug 14, 2018

mloskot commented Aug 14, 2018

danpat commented Aug 14, 2018 •

edited

Loading

DennisOSRM commented May 10, 2024

Isochrone Plugin v2 #3652

Isochrone Plugin v2 #3652

Conversation

danpat commented Feb 4, 2017 • edited Loading

TODO:

TheMarex Feb 26, 2017

Choose a reason for hiding this comment

danpat Feb 26, 2017 • edited Loading

Choose a reason for hiding this comment

danpat Feb 26, 2017 • edited Loading

Choose a reason for hiding this comment

Fkawala Jun 14, 2017

Choose a reason for hiding this comment

Dirk23 commented Jun 1, 2017

mloskot commented Aug 8, 2017 • edited Loading

adomas-mazeikis commented Aug 9, 2017

mloskot commented Aug 9, 2017 • edited Loading

mloskot Aug 10, 2017 • edited Loading

Choose a reason for hiding this comment

mloskot Aug 10, 2017 • edited Loading

Choose a reason for hiding this comment

danpat Aug 10, 2017

Choose a reason for hiding this comment

mloskot Aug 10, 2017 • edited Loading

Choose a reason for hiding this comment

mloskot Aug 10, 2017 • edited Loading

Choose a reason for hiding this comment

mloskot commented Aug 10, 2017

CraigglesO commented Jan 25, 2018

mloskot commented Jan 30, 2018

geography commented Feb 2, 2018

mloskot commented Feb 2, 2018 via email

CraigglesO commented Feb 7, 2018

ghost commented Aug 14, 2018

mloskot commented Aug 14, 2018

danpat commented Aug 14, 2018 • edited Loading

DennisOSRM commented May 10, 2024

danpat commented Feb 4, 2017 •

edited

Loading

danpat Feb 26, 2017 •

edited

Loading

danpat Feb 26, 2017 •

edited

Loading

mloskot commented Aug 8, 2017 •

edited

Loading

mloskot commented Aug 9, 2017 •

edited

Loading

mloskot Aug 10, 2017 •

edited

Loading

mloskot Aug 10, 2017 •

edited

Loading

mloskot Aug 10, 2017 •

edited

Loading

mloskot Aug 10, 2017 •

edited

Loading

danpat commented Aug 14, 2018 •

edited

Loading