WIP: Split Inactive Ballots by Exhausted Choices bucket into Inactive Fully Ranked Ballots by Exhausted Choices and Inactive Partially Ranked Ballots by Exhausted Choices

[yezr]

Currently we have a row for Inactive Ballots By Exhausted Choices in the summary.csv. We'd like to split this into two buckets

• first bucket, Inactive Fully Ranked Ballots by Exhausted Choices is for exhausted ballots that used ALL of their rankings
• second bucket, Inactive Partially Ranked Ballots by Exhausted Choices is for exhausted ballots that used less than the number of rankings that they were allowed

We will use the number configured in Maximum Number of Candidates That Can Be Ranked to differentiate. This will be required! We don't want to assume the number of maximum rankings. It must be explicitly configured.

Open Questions

• How to handle multi-vendor CVRs that might have different maximum rankings?
  • Feedback is that when we do have multi-vendor races that probably they will all be configured to have the same number of rankings. Can't imagine a jurisdiction allowing a different number of rankings on different hardware for the same election. Could be equal protection issues etc.
• When Maximum Number of Candidates That Can Be Ranked is not configured, do we default back to a single Inactive Ballots By Exhausted Choices bucket in the CVR?
• Does the summary.json show all three buckets (exhausted, exhausted_fully_ranked and exhausted_partially_ranked) and fill in whatever it can? Or does the schema differ depending on whether or not max ranks is configured explicitly? How do we flag that for consumers?
• What if instead of requiring the "Maximum Number of Rankings" we add to the summary_extended.csv new rows for "Inactive Ballots by Exhausted Choices" that group those exhausted ballots by the number of rankings those ballots used. This could provide even more precise information and doesn't require any more explicit configuration before tabulation. In the example we only include rows for a specific number of rankings if at least one ballot with that number of ranks went exhausted. So in the example below there were no ballots that ranked three candidates and had their ballot exhaust.
  

[artoonie]

What if the config sets the max votes to "MAX", but one of the CVRs has a more strict limit? This can happen with Dominion CVRs. It feels like it may be more legible both in code and in the output files to simply exclude any "inactive by __" if no ballots are inactive in that way?

[yezr]

That's right with multi-vendor CVRs the max ranking could be different for each one...hmm. Made this a WIP and added all the open questions to the original description.

[yezr]

We need some more time to process the open questions. Removing from 2.0

[yezr]

Would it be possible as a stopgap until this is implemented, to put into the audit.log how many total ranks a ballot had when we log that it went exhausted?

[artoonie]

As I was reverting the code to defer this, I found a solution I'm somewhat happy with that I'd like to pitch.

The idea is that we always include as much information as we have, and only exclude the line item in the CSV when two conditions are met:

1. The config.maxRanking is set to Max, AND
2. There are either no Dominion files, OR the Dominion-specific Max Ranking is never hit

Concretely, the following table indicates when Inactive Ballots by Exhausted Choices (Fully Ranked) will included in the CSV:

CVR Types	config.maxRanking = Max	config.maxRanking = a number
Dominion only	Include if any ballots match	Include
Dominion and others	Include if any ballots match*	Include
No Dominion	Exclude	Include

The asterisk denotes the potentially strange case: a multi-vendor tabulation where only Dominion ballots can end up in the fully ranked count. I think this is okay, because it seems reasonable to encourage, but not require, the operator to set the config max ranking to a value other than Max.

I know this discussion could go on for much longer, but wanted to get a gut check on whether this interim solution is acceptable.

[artoonie]

Okay, here's a funny thing that will need to be understood if we do any sort of splitting of fully ranked vs partially ranked:

If a CVR ranks 100% undeclared candidates, that should probably be marked as fully ranked -- but that means you can have inactive CVRs on the second round, which looks wrong but isn't.

[yezr]

Added Armin's suggestion for extra rows in summary_extended.csv for any exhausted ballots grouped by the number of rankings they used to the open questions section of the original description.