Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Project Data Quality view #6109

Closed
emi420 opened this issue Oct 26, 2023 · 9 comments
Closed

Add Project Data Quality view #6109

emi420 opened this issue Oct 26, 2023 · 9 comments
Assignees
Labels
data-quality Issues related to data quality and validation workflow status: in progress type: feature request Adding functionality that currently does not exist

Comments

@emi420
Copy link
Collaborator

emi420 commented Oct 26, 2023

Is your feature request related to a problem? Please describe.
There's no way to way to monitor mapping in near real-time, highlighting quality issues at the same time.

Describe the solution you'd like
Integrate Underpass Data Quality Tools and using Underpass UI components.

Describe alternatives you've considered
Another alternative could be to develop this feature for Tasking Manager only, but there's no way currently of getting data and analyze it every minute without using Underpass. Also this could be done without using Underpass DB/REST API and/or Underpass UI, and develop new endpoints or a new UI, but this components already exists because they are intended to be used on other projects.

Additional context

There were several discussions for the design of this idea:

Finally, we arrived to the conclusion that creating a live monitoring tool, while displaying automatically detected data quality issues, have higher value than other options.

This is the draft design for the view:

image

It could be modified when doing the final implementation.

Technical notes

  • It will be available for 4 countries (1 per Hub) in the first version, in order to have time for testing and improvement
  • Should be located in: components/projectDataQuality.js
  • It will be implemented using Underpass React UI components, including the connection with an Underpass API for getting data
  • It will need need access to some data for the TM project: id, areaOfInterest and mappingTypes for filtering results.
@emi420
Copy link
Collaborator Author

emi420 commented Dec 6, 2023

@emi420 emi420 moved this to In Progress in Underpass 2023-2024 Dec 6, 2023
@emi420 emi420 added the data-quality Issues related to data quality and validation workflow label Dec 7, 2023
@emi420 emi420 moved this from In Progress to Done in Underpass 2023-2024 Dec 7, 2023
@varun2948
Copy link
Contributor

Hi @emi420 are we fetching data in certain intervals? Cause we don't have web socket integration right now to get the realtime data right ?

@emi420
Copy link
Collaborator Author

emi420 commented Dec 9, 2023

Hey @varun2948 , yes, when the "live" function is checked. I'll add websockets soon to the API and the UI (https://github.com/hotosm/underpass-ui)

@varun2948
Copy link
Contributor

Sounds great with the websocket implementation and realtime data.

@emi420
Copy link
Collaborator Author

emi420 commented Dec 12, 2023

We're building an approach to use gRPC instead of websockets. https://hotosm.github.io/tm-admin/communication/

@ramyaragupathy
Copy link
Member

From discussion today, we will include this feature only in expert mode. Once a new PR is up from @emi420, @royallsilwallz will implement the changes to support the view for expert mode features.

#6151 will be closed soon due to unmanageable conflicts.

cc @dakotabenjamin @manjitapandey

@ramyaragupathy
Copy link
Member

Related PR: #6299 - @royallsilwallz you're good to pick on the next steps for expert mode integration here.

@ramyaragupathy
Copy link
Member

Related ticket on expert mode integration to launch this feature in experimental mode #6303

@ramyaragupathy
Copy link
Member

part of v4.7.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data-quality Issues related to data quality and validation workflow status: in progress type: feature request Adding functionality that currently does not exist
Projects
No open projects
Status: Done
Development

No branches or pull requests

3 participants