Research Request - review RT segments speeds preprocessing stages

[edasmalchi]

Research Question

Single sentence description:
As we work towards an open data release, review the first several steps of the segment speeds pipeline to make sure we're keeping as much usable data as feasible.

How will this research be used?

Will be used to adjust scripts as necessary.

Stakeholders & End-Users

All open data stakeholders

Metrics

tbd

Data sources

existing per #592

Deliverables:

Notebook?

[edasmalchi]

read through much of the pipeline code and looked at several operators plus trip stats in rt_segment_speeds/05_threshold_v2_exploratory.ipynb

reccomending 10 minutes / 70% of segments for that threshold

[edasmalchi]

looks like 5 minutes is currently set: https://github.com/cal-itp/data-analyses/blob/main/rt_segment_speeds/scripts/config.yml

(that works for me too)

[tiffanychu90]

• Implemented 10 min cutoff in Tweak speeds pipeline #795 for May and June dates.
• Feb-Apr dates use 5 min cutoffs...can go back and rerun at later date.
• Jan shouldn't be used because this was pre-timezone/dealing with UTCs/Pacific times, so RT and schedule data won't line up well because only 1 date for RT is downloaded. Now, we download 2 dates and concat to get 1 service date for schedule data