Add variable fares by time or day

[isabelle-dr]

Fares can vary based on the time of day (described using timeframe_id) and day of week/year (described using service_id).
This pull request covers fares that vary by time and day, which is a section of the entire GTFS-Fares v2 proposal.
The changes in this pull request are:

• Add new file: timeframes.txt, to define moments in time where the fare applies.
• Extend fare_leg_rules.txt with from_timeframe_id, and to_timeframe_id to specify that a fare leg rule applies only if the beginning or end of the leg is in a specified timeframe.

Here's a quick example

• A regular fare is required to travel.
• The fare is discounted if riders travel between 6:00 AM and 7:00 AM.
• On New Year's, transit is free (no fare).

Define service days and New Year's using calendar.txt. and calendar_dates.txt

service_id	monday	tuesday	wednesday	thursday	friday	saturday	sunday	start_date	end_date
regular_service	1	1	1	1	1	1	1	20230101	20231231

service_id	date	exception_type
regular_service	20230101	2
new_year_service	20230101	1

Define the discounted timeframe using timeframes.txt

timeframe_id	start_time	end_time	service_id
morning	06:00:00	07:00:00	regular_service
regular	00:00:00	06:00:00	regular_service
regular	07:00:00	24:00:00	regular_service
all_day	00:00:00	24:00:00	new_year_service

Define the time and date-based fares using fare_leg_rules.txt

leg_group_id	from_area_id	to_area_id	network_id	from_timeframe_id	fare_product
leg1	zoneA	zoneB	bus	regular	regular_fare
leg1	zoneA	zoneB	bus	morning	rdiscounted_fare
leg1	zoneA	zoneB	bus	all_day	free_fare

[Original message posted by @isabelle-dr on November 2, 2022. OUTDATED]

Hi everyone!

I am opening this Pull Request as part of the second implementation of Fares v2. I am opening it in the exact same state PR #343 was when closed, and I will update the proposal based on feedback already received so we can resume where we left off. For more information on the plan, please refer to #341 (comment).
This pull request covers fares that vary by time and day, which is a section of the entire GTFS-Fares v2 proposal.

Fares can vary based on the time of day (described using timeframe_id) and day of week/year (described using service_id).

The changes in this pull request are:

• Add new file; timeframes.txt, to define timeframes.
• Extend fare_leg_rules.txt with from_timeframe_id, to_timeframe_id, and service_id to describe time-dependant fares.

The time-dependant variables (timeframe and day information) are modelled into fare legs (and not fare products) since in GTFS-Fares v2, fare_leg_rules.txt models the location and time variables (from/to area, from/to timeframe, and service ID). All other factors are modelled in fare products.

Here's a quick example

• A regular fare is required to travel.
• The fare is discounted if riders travel between 6:00 AM and 7:00 AM.
• On New Year's, transit is free (no fare).

Define service days and New Year's using calendar.txt. and calendar_dates.txt

service_id	monday	tuesday	wednesday	thursday	friday	saturday	sunday	start_date	end_date
regular_service	1	1	1	1	1	1	1	20230101	20231231

service_id	date	exception_type
regular_service	20230101	2
new_year_service	20230101	1

Define the discounted timeframe using timeframes.txt

timeframe_id	start_time	end_time
morning	06:00:00	07:00:00
regular	00:00:00	05:59:59
regular	00:07:01	23:59:59
all_day	00:00:00	23:59:59

Define the time and date-based fares using fare_leg_rules.txt

leg_group_id	from_area_id	to_area_id	network_id	from_timeframe_id	to_timeframe_id	service_id	fare_product
leg1	zoneA	zoneB	bus	regular	all_day	regular_service	regular_fare
leg1	zoneA	zoneB	bus	morning	all_day	regular_service	discounted_fare
leg1	zoneA	zoneB	bus	regular	all_day	new_year_service	free_fare
leg1	zoneA	zoneB	bus	morning	all_day	new_year_service	free_fare

Data consumer: Apple
Data producer:

 ITO World

Please go through the changes and share your thoughts here!
Looking forward to feedback and contribution on this proposal.

For other questions/concerns, don’t hesitate to reach out to [email protected].

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[isabelle-dr]

Hello everyone!

Here is an update on this work.

What was done based on the feedback received on #343

• Updated the timeframe.txt file description to remove the 24h period mention, so that it stays in line with the rest of GTFS (24+h format). Result of @skinkie's feedback.
• Mentioned that start and end times are included in the interval, as a result of @flocsy's feedback.
• Made the end_time from Required to Optional, and added that if left empty, it is considered that the fare only depends on start time. This is because most agencies do not use the trip end times to process time-based fares, and it avoids the hassle of creating a separate entry in timeframes.txt to describe "All day". This is a result of the August 24 roundtable discussion. Edit: I reverted this change, the intention was to have fare_leg_rules.to_timeframe_id default to "All day" when left blank. Since this changes the "empty" behavior used everywhere else, I am waiting for more community feedback before making a change.

Outstanding items

• Timeframe type: the need for a file or field to describe the timeframe type was highlighted since transit services have different ways of defining when a fare is activated (at station entrance, upon boarding, upon departure, etc.).
  One option is to extend fare_leg_rules.txt with timeframe_type with enumerations, and @flocsy asked why this couldn't be added in timeframes.txt. This is a result of the August 24 roundtable discussion.
• Timeframe rules: There is a need to better define rules that apply in timeframe design, mainly:
  • If one timeframe is defined for an interval on a given day, we might need to explicitly mention that the whole day must be covered with timeframes
  • We might add a mention that we must not have overlapping timeframes with the same timeframe_id
• We might want to have trip-level attributes to the fare calculation, for the case when the determination of the peak-ness of a leg is determined by the time that vehicle leaves its origin (or reaches its destination). This is a result of @davidlewis-ito's feedback.

I believe these were all the items discussed, please let me know if you think otherwise.

[flocsy]

There's a typo:
end_time | Time | Optional | Start time for the interval.
it should be "End"

[flocsy]

"Mentioned that start and end times are included in the interval"
I'm not sure if this is what you meant (not what I proposed) or you meant what I proposed but the wording is incorrect...
I meant that when we talk, we always talk about intervals like: peak: from 08:00 to 10:00 ; off-peak: 10:00-, so maybe it's better to modify the description to make this the official format. However according to the current wording we need to write: peak: 08:00 - 09:59, off-peak: 10:00 -

[isabelle-dr]

@flocsy

end_time | Time | Optional | Start time for the interval.
it should be "End"

Thanks, fixed. ✅

I'm not sure if this is what you meant (not what I proposed) or you meant what I proposed but the wording is incorrect...
I meant that when we talk, we always talk about intervals like: peak: from 08:00 to 10:00 ; off-peak: 10:00-, so maybe it's better to modify the description to make this the official format. However according to the current wording we need to write: peak: 08:00 - 09:59, off-peak: 10:00 -

Understood. The start and end times are modeled the same way start_date and end_date are in calendar.txt, or start_time and end_time in frequencies.txt: all field values are included in the intervals. We believe that changing this design for timeframes would lead to additional complexity in the spec. I have specified explicitly that start_time and end_time are included so that people design intervals as 08:00 - 09:59, off-peak: 10:00 - 16:0015:59, even though they talk about peaks as _08:00 to 10:00 ; off-peak: 10:00 - 16:00. I'm happy to add an example below the table.

[flocsy]

LOL, you just proved my point. You wrote:

"so that people design intervals as 08:00 - 09:59, off-peak: 10:00 - 16:00, even though they talk about peaks as 08:00 to 10:00 ; off-peak: 10:00 - 16:00. "

You should've written: 10:00 - 15:59

But I get it. So the important thing is to make it required to cover the whole day and not to have overlap somehow so it'll be possible for a validator to spot things like there's a 1 minute overlap between 08:00-10:00 and 10:00-16:00

[davidlewis-ito]

Just to highlight a couple of areas we think could do with some more consideration

Determinant of peakness

@isabelle-dr Thanks for highlighting the use-case where the peakness might be related to the trip rather than the time at which the rider joins a trip.

The other case we have raised previously is where the peakness depends upon the time at which the rider enters the fare area of a station - for example the time the rider "touches in" at their origin station on the London Tube.

So we see three cases;

1. Peakness depends upon time the rider joins a trip - covered by timeframes in this PR
2. Peakness depends upon time the rider enters a station - would require amendment to this PR
3. Peakness depends upon time the trip starts or ends - requires either timeframes that apply to trips or (more preferably we believe) the ability to mark an individual trip as having a "fare class" that would then play into fare_leg_rules

Fare_leg_rules or Fare products

Should these time discriminating factors be used to determine the correct fare via the fare_leg_rules or the fare_products table ? One approach may be to constrain the fare_leg_rules table to be concerned only with the physical journey taken (stops, distance , routes etc ). The fare_products table would then combine this with the peakness, the containers, the rider_category etc.

This is a big topic I suspect that would be best discussed at the upcoming round-table

[flocsy]

Isn't case 1 (joins a trip) and 2 (enters a station) basically the same thing, just with different timefarme_type_id?

[davidlewis-ito]

@flocsy Yes !

Previous discussions on this have debated whether the timeframe_type_id sits in timeframes.txt or in fare_leg_rules

[flocsy]

Which is a hard question. I can imagine that in most if not all places in the world the situation is something like: 8:00-09:59 peak hour. However depending on the vehicle type (bus, tram vs subway,train for example) the point of validation might be different (bus, tram: at boarding time vs subway,train: at station entrance) So the question is what we prefer to "duplicate" in this case. If we add timeframe_type_id to timeframes, then you duplicate lines BOTH in timeframes.txt AND in fare_leg_rules.txt IMHO. If we add timeframe_type_id to fare_leg_rules.txt then we only duplicate lines there. And I think this also makes more sense because "timeframes" is about the time frames, and not where you measure it...

However, maybe, there's an even better place where we could add timeframe_type_id. In my above example it could go to the networks.txt or something like that, and then we wouldn't need to duplicate lines anywhere...

[bdferris-v2]

Hey all, trying to bring myself up to speed on this proposal. I wanted to re-open a different potentially fraught subject: empty-entry semantics.

The current language:

An empty entry in `from|to_timeframe_id` corresponds to all timeframes defined in `timeframes.timeframe_id`
excluding the ones listed under `fare_leg_rules.from|to_timeframe_id`.

seems particularly fraught, if I understand it correctly? I know there was a bit of discussion around this in original PR, but I believe it was highlighted that the second you have two different fare networks in the same feed who both want to use "empty" semantics for their "default" rule and then use different timeframes for their "specific" rules, you'll have a conflict and the fares won't compute as expected. As discussed, this is generally a problem with "empty" matching in fare_leg_rules.txt.

Curious if you have specific thoughts on the timeframe proposal regarding this issue.

[flocsy]

@bdferris-v2 I agree with you. This also bothers me every time I see it. It's confusing that usually empty matches anything, but not in this case. If you have a better (less confusing) idea, I'll most probably vote fore it.

[isabelle-dr]

Hello everyone,

MobilityData will host a roundtable discussion this week on Thursday 1 December 11:00 AM ET to discuss the open items in this PR.
We are also hosting a discussion around higher-level GTFS Fares-v2 discriminating factors and design principles, which will happen the day before on Wednesday 30 November 11:00 AM ET and it is intended to inform the decisions for this PR.

You can access both events via this calendar, and you can reach out to [email protected] to get an invite on your e-mail.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[isabelle-dr]

not stale

[flocsy]

Some of you discussed whether it's less confusing for consumers if the times are agency or stop timezone. I think there's another side of this question: on 363 days of the year probably either would work. But on the 2 days when there's the daylight saving time change it might be "crucial" to choose the "better" one and also because it only happens twice a year it's harder to test these edge cases. I'm not sure how much this happens in real world, but in theory there are many things could go wrong during the DST change night(mare): the 2 neighbouring countries might change DST on a different date (which would make 4 though days per year) but even if they are on the same day there is probably a 1 hour gap between when it occurs in the 2 countries. So I'm not sure which timezone is better to be used, but when discussing this we should consider the edge cases and the possibility to make implementation bugs.

[isabelle-dr]

Hello everyone, the working group had a final discussion on this proposal and reached consensus (see meeting notes).

• We have at least one producer: ITO World. They have shared their dataset in this file, which is part of a private feed.
• We have at least one consumer: Apple Maps. See the screenshots below for the same dataset.

As per the GTFS Amendment process, the requirements to open a vote are met.
We are opening a vote for adding variable fares by time or day, which is part of the GTFS-fares v2 proposal.

Voting ends on 2023-07-24 at 23:59:59 UTC.

weekday peak

weekday off-peak

weekend

[flocsy]

If possible please amend the descriptions for the sake of clarity (see my comments), but you get my vote
+1 Moovit

[skinkie]

+1 OpenGeo (not a producer)

[saraemarcus]

+1 Apple

[halbertram]

+1 Ito World

Would it be worth changing the empty wordings from
An empty `fare_leg_rules.from_timeframe_group_id` indicates that the start time of the leg does not affect the fare
to something like
An empty `fare_leg_rules.from_timeframe_group_id` indicates that the start time of the leg does not affect the matching of this rule
(or record?) as the start time may still affect the fare, just not via this rule?

[westontrillium]

+1 Trillium

[bdferris-v2]

+1 Google

[jfabi]

+1 from the @mbta in Boston.

Excited to see consensus built on this important addition to Fares v2—the MBTA has a couple of near-term use cases where we may implement timeframes.

[jlgauducheau]

+1 for Cityway

[npaun]

+1 Transit

[isabelle-dr]

The voting period ended on 2023-07-24 at 23:59:59 UTC.

With 9 votes in favour and no votes against, the vote to extend GTFS with Variable fares by time or day passes!
The votes came from:
Moovit (@flocsy)
OpenGeo (@skinkie)
Apple Maps (@saraemarcus)
ITO World (@halbertram )
Trillium (@westontrillium)
Google (@bdferris-v2)
MBTA (@jfabi)
Transit (@npaun)
Cityway (@jlgauducheau)

• The GTFS Fares-v2 Google Doc has been updated to reflect this change.
• The Canonical GTFS Schedule Validator has been updated to validate this data, it will be part of the 4.2.0 release, a contribution by @bdferris-v2 🙏.
• We are in the process of updating the fares v2 extension page and the data examples on gtfs.org.

Many thanks to everyone who participated in this Pull Request and in the working group meetings, and we would like to extend a special recognition to ITO World and Apple Maps for their commitment to implementing the changes proposed in this PR, which is required for every GTFS addition.