Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement DAP-13 #3436

Open
19 of 20 tasks
branlwyd opened this issue Oct 11, 2024 · 5 comments
Open
19 of 20 tasks

Implement DAP-13 #3436

branlwyd opened this issue Oct 11, 2024 · 5 comments
Assignees

Comments

@branlwyd
Copy link
Contributor

branlwyd commented Oct 11, 2024

DAP-13: https://www.ietf.org/archive/id/draft-ietf-ppm-dap-13.html

From DAP-09, this will require (roughly):

  • Poll collection jobs with GET instead of POST. (DAP-10, PR)
  • Upload reports with POST rather than PUT. (DAP-10, PR)
  • Remove support for multi-collection of batches, removing max_batch_query_count & the leader-selected by_batch_id query. (DAP-11, PR, [1])
  • Remove support for Poplar1. (without multi-collection of batches, Poplar1 is not so useful; DAP-13: drop support for Poplar1 #3480)
  • Remove per-task HPKE configurations. (DAP-12, PR)
  • Rename "query type" to "batch mode", "fixed-size" to "leader-selected". (DAP-12, PR)
  • Remove max_batch_size. (DAP-12, PR)
  • Asynchronous aggregation. (DAP-12, PR, Issue)
  • Update collection-related messages to match aggregation messages. (DAP-12, PR)
  • Replay protection changes. (DAP-12, PR, [2])
  • Re-number PrepareError values. (DAP-12, PR)
  • Update to VDAF-13. (DAP-13, PR, [3])
  • Update to draft-ietf-ppm-dap-taskprov-01. (Spec)
  • Rename "prepare error" to "report error". (DAP-13, PR)
  • Rename "upload extension" to "report extension". (DAP-13, PR)
  • Implement task start time, switch from task end time to task duration. (DAP-13, PR)
  • Verify that we always reject reports outside of the task's validity window. (DAP-13, PR)
  • Update embedded messages in Query, BatchSelector, PartialBatchSelector to include length prefix. (DAP-13, PR)
  • Implement public report extensions. (DAP-13, PR)
  • Bump version tag. (DAP-13, PR)

[1] Do not remove the part_batch_selector field, as it is restored in a later change.
[2] This may not require changes, but we should validate that Janus implements the specified behavior.
[3] This depends on a release of libprio-rs implementing VDAF-13 being available.

@branlwyd
Copy link
Contributor Author

Replay protection changes

For this portion of the work, I think no changes are needed:

  • Janus implements leader-side replay checking by only ever including a given report into a single aggregation job. This is stricter than required by DAP, which requires removing/not including reports already marked when generating aggregation jobs, but does not require the report to be marked until aggregation is complete.
  • Janus implements helper-side replay checking by checking/marking replay for a report at time of aggregate initialization (by way of writing a scrubbed client report). This is stricter than required by DAP, which only requires that the check must occur before completing the aggregation job.

N.B. there are a few places where we MUST check replay of a report, where Janus does not do so because we implement the check at an earlier point in the protocol. I don't think we should make functional changes to Janus; we might consider an editorial change to DAP to say something like "Report replay MUST be resolved if it has not already been resolved."

@branlwyd
Copy link
Contributor Author

Verify that we always reject reports outside of the task's validity window

I verified that we will always reject reports that are outside of the task's valid time window (i.e. task_start to task_end, inclusive).

@branlwyd
Copy link
Contributor Author

I removed the (optional) indication of unknown extension types by the Leader, filing #3505 instead.

@branlwyd
Copy link
Contributor Author

I filed #3510 to add support for Prio3MultihotCountVec; it's not required for a compliant DAP-13 implementation, but there is little reason (other than prioritization) not to support it.

@branlwyd
Copy link
Contributor Author

branlwyd commented Dec 9, 2024

Once #3564 lands, Janus should be able to interop with any DAP-13 aggregator, whether it is using synchronous or asynchronous aggregation.

We should still implement Helper async aggregation (which is my next task), but we should now be DAP-13 complete. (I say "should be" rather than "are" because there is no other DAP-13 aggregator to test against just yet, so we cannot fully validate our implementation.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant