You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This builds on #388. The idea here is that Mosaic sends queries to a backend for execution but can only run so many queries simultaneously. With fine grained dependencies, we could increase parallelism and potentially eliminate superfluous queries.
We have to be careful about dependencies so that visualizations don't show stale data. As an example, imagine a dashboard with three charts A, B, and C. The user brushes in A which updates B and C. The brush changes create three queries b_1, b_2, b_3 to update B and c_1, c_2, c_3 to update C. Imagine these come in as b_1, c_1, b_2, c_2, b_3, c_3 and the backend can only run two queries at a time. Right now, Mosaic only resolves queries in the same order they were queued even though b_1 and c_1 could resolve in any order without causing issues. b_2 and c_2 do not really need to be sent to the backend if we already know their results will be superseded by b_3 and c_3 so if we can keep track of that as well, we can basically drop the queries from the queue. There are many more dependencies such as queries that create new tables (such as indexes) which need to run in the correct order. For now (even with #475) exec queries never run in parallel since we don't know whether there is a dependency between them.
This could be related to #461 since we can queue up and drop superseded queries when the chart comes into view again.
The text was updated successfully, but these errors were encountered:
This builds on #388. The idea here is that Mosaic sends queries to a backend for execution but can only run so many queries simultaneously. With fine grained dependencies, we could increase parallelism and potentially eliminate superfluous queries.
We have to be careful about dependencies so that visualizations don't show stale data. As an example, imagine a dashboard with three charts A, B, and C. The user brushes in A which updates B and C. The brush changes create three queries b_1, b_2, b_3 to update B and c_1, c_2, c_3 to update C. Imagine these come in as b_1, c_1, b_2, c_2, b_3, c_3 and the backend can only run two queries at a time. Right now, Mosaic only resolves queries in the same order they were queued even though b_1 and c_1 could resolve in any order without causing issues. b_2 and c_2 do not really need to be sent to the backend if we already know their results will be superseded by b_3 and c_3 so if we can keep track of that as well, we can basically drop the queries from the queue. There are many more dependencies such as queries that create new tables (such as indexes) which need to run in the correct order. For now (even with #475) exec queries never run in parallel since we don't know whether there is a dependency between them.
This could be related to #461 since we can queue up and drop superseded queries when the chart comes into view again.
The text was updated successfully, but these errors were encountered: