Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Separate measurement and interpretation #13

Closed
bassosimone opened this issue Jun 5, 2019 · 4 comments
Closed

Separate measurement and interpretation #13

bassosimone opened this issue Jun 5, 2019 · 4 comments
Assignees
@bassosimone
Labels
enhancement New feature or request

Comments

@bassosimone
Copy link
Contributor

From measurement-kit/measurement-kit#1712:

See measurement-kit/measurement-kit#1704 (comment)

See also this message of mine on Slack, which elicited lots of positive feedback:

  1. we can refrain from changing the code that decides whether something is blocked for now, but we should change that based on mining data in the future. I do not like that our data format does not clearly distinguish between (a) this is what I did measure and (b) this is what I believe what I measured means. This is unfortunate. Our data format should clearly distinguish between facts, i.e. what was measured, and inferences based on such facts.
@bassosimone bassosimone added the enhancement New feature or request label Jun 5, 2019
@bassosimone
Copy link
Contributor Author

This seems to me deeply related to #12

@bassosimone
Copy link
Contributor Author

Also, from measurement-kit/measurement-kit#1161 (comment):

  1. it is possible to argue that repeating a measurement is the right thing to do when, say, the network is bad and that repeating a failed measurement could have provided the user with the right result. I disagree with that, and I believe that the right result does not matter much, if the methodology is wrong. What we should actually do is (a) measure and (b) reason on the measurement. In my view, these two pieces of codes should probably be quite decoupled. Repeating a measurement is not bad, provided that we not discard the information that the measurement failed N times, because that is also valuable data.

Therefore, one should be able to see from the results that we did retry the measurement.

@bassosimone bassosimone mentioned this issue Aug 28, 2019
Closed
@bassosimone bassosimone changed the title story: separate measurement and interpretation Separate measurement and interpretation Aug 28, 2019
@bassosimone bassosimone added the P2 label Sep 10, 2019
bassosimone added a commit that referenced this issue Oct 8, 2019
@bassosimone
dash: refactor for testability
abebd02 
1. separate measurement and interpretation (see #13)

2. this will allow us to write separate unit tests for each
phase of the measurement, so it will be easier

3. sprinkle some extra documentation
@bassosimone
Copy link
Contributor Author

We have explained how this could be done in #92. We still need to work onto re-running measurements, tho, as this has not been addressed yet.

@bassosimone bassosimone self-assigned this Oct 8, 2019
bassosimone added a commit that referenced this issue Oct 10, 2019
@bassosimone
dash: refactor for testability
2f150d0 
1. separate measurement and interpretation (see #13)

2. this will allow us to write separate unit tests for each
phase of the measurement, so it will be easier

3. sprinkle some extra documentation
bassosimone added a commit that referenced this issue Oct 10, 2019
@bassosimone
dash: refactor for testability
f446e75 
1. separate measurement and interpretation (see #13)

2. this will allow us to write separate unit tests for each
phase of the measurement, so it will be easier

3. sprinkle some extra documentation
bassosimone added a commit that referenced this issue Oct 10, 2019
@bassosimone
dash: refactor for testability
67c8e6f 
1. separate measurement and interpretation (see #13)

2. this will allow us to write separate unit tests for each
phase of the measurement, so it will be easier

3. sprinkle some extra documentation
@bassosimone
Copy link
Contributor Author

This can also be considered one of the guiding principles of #87. To improve on device data analysis we needed to separate measurements (github.com/ooni/netx) and interpretation (github.com/ooni/probe-engine). While it is true that we will keep a backwards compatible data format, we've made enough progress towards this goal that we can close this rather abstract issue with confidence that the separation is as robust as two different repositories.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bassosimone bassosimone

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bassosimone @hellais