Fetch results of evening voting sessions

[tillprochaska]

While on most plenary days, there is only one voting session around midday, on some days there is another sesssion in the evening, usually around 17:00. The vote results of the evening sessions are appended to the same source document that also contains the results of the midday votes.

Currently, we only run the RCVListPipeline between 12:00 and 15:00 until we’ve been able to fetch vote results successfully. Once we’ve fetched the results, we do not attempt to fetch them again. That means that so far we did not (automatically) fetch results of the evening voting session.

The easy approach to this problem would be to simply keep running the RCVListPipeline until the evening, even after it has successfully completed for the first time. However, this approach has a few disadvantages:

• We store timestamps of when we accessed the data sources related for a specific vote. Following the easy approach means we’d store the timestamp when the data was last accessed. For diagnostics and analysis, it’s nice to keep the timestamp of the first access.
• Every pipeline triggers multiple requests for each vote. As it’s not uncommon to see hundreds of votes per day, this would be quite wasteful and not appropriate for a well-behaved scraper.

Instead, we do the following:

• Between 12:00 and 15:00, we periodically check if the vote results are available until we’ve been able to fetch the vote results successfully (just as we did before).
• Between 17:00 and 20:00, we periodically check if an updated RCV list has been published. When an updated list has been published, we rerun the entire pipeline and stop the periodic checks afterwards. When the list remains unchanged, we don’t rerun subsequent steps of the pipeline.

This is implemented as follows:

• When a pipeline has run successfully, it can return a checksum. The checksum is stored and is provided to subsequent runs of the pipeline.
• The checksum can be any value, but in case of the RCVListPipeline, it is a simple hash of the contents of the RCV list. (In theory, we could make use of HTTP caching mechanisms and try to e.g. store the response ETag. In practice, many of the official sources do not include an ETag in responses, and this would add unnecessary complexity for irrelevant savings.)
• If during a subsequent run, the hash of RCV list contents hasn’t changed, the pipeline exists early to skip unnecessary requests. Once the pipeline has been run successfully for a second time, we also stop periodically checking if the RCV list contents have changed.

Fixes #917

[tillprochaska]

Note to self: These aren’t actually errors, they are exceptions used for flow control. I think I’ve blindly followed the linter recommendation, but this naming doesn’t really make sense for this case.

According to PEP 8, exception classes should have an Error suffix, but only if they actually are errors, otherwise they should have no special suffix, so in this case DataUnchanged and DataUnavailable would be more appropriate.

https://peps.python.org/pep-0008/#exception-names

It probably makes sense to fix this separately though, there might be other similar cases elsewhere.