fix: date-picker bug for negative UTC timezones

[LinHuiqing]

Problem

(taken from #6025)
Date picker was using the midnight of the date picked and taking into account the timezone where the user is filling in the form. However, it does not make sense to do so as users will be picking particular dates rather than timings, and the understanding of the different timezones should be established as part of the question.

However, bug fix in PR #6025 almost caused an incident as the fix removed converting to UTC on the date picker itself, which was causing the selection offset bug.

Solution

Puts back the local time to UTC conversion right before submission so that dates saved in the DB are normalised to be in UTC time.

Note: local time to UTC conversion does not have to be restored for DateField as the responses for DateField are in string when encrypted anyway.

Breaking Changes

• No - this PR is backwards compatible

Improvements:

• Cast time to UTC only right before submitting custom date so the conversion will be completely invisible to users
• Cast time to local timezone before being shown on the frontend so the frontend timezone will be consistent

Bug Fixes:

• Users with negative UTC timezones will now have their date fields correctly reflecting the dates selected on their date pickers
• zonedTimeToUTC was converting the timings to UTC and thus the actual date being referred to will not be preserved for respondents in negative UTC timezones. Implemented normalizeDateToUtc to ensure the actual date numbers are are preserved.
  • We now cast time from local timezone to UTC timezone using Date.UTC instead of converting time using zonedTimeToUTC (which we were using wrong anyway, as pointed out by @darrelhong in fix: date picker calendar select correct date in negative UTC offset timezones #5731)

Before & After Screenshots

BEFORE:

Screen.Recording.2023-03-31.at.3.18.31.AM.mov

AFTER:

Disallow past dates:

Screen.Recording.2023-04-13.at.2.48.49.PM.mov

Disallow future dates:

Screen.Recording.2023-04-13.at.5.08.17.PM.mov

Custom date range:

Screen.Recording.2023-04-13.at.5.11.57.PM.mov

Tests

To ensure that all our bases are covered, try to test the most extreme timezones. Here are the suggested timezones:

• Most common use case: Singapore - Singapore (UTC +08)
• Negative: Alofi - Niue (UTC -11)
• Positive: Wellington - New Zealand (UTC +12)

Preparation:

• Create a form with a date field and open the form.

For the tests below, timezones A and B refer to the positive and negative timezones mentioned above, in any order.

Ensure that date picker for date field with "Disallow past dates" option exhibits expected behaviour:

1. • Set your timezone as A.
2. • On the admin panel, refresh, select and save the "Disallow past dates" option for the date field.
3. • On the respondent form, refresh and open the date picker. Only the current and future dates of the timezone should be shown.
4. • Select a date. The appropriate date should be reflected in the date field.
5. • Change your device's timezone to B.
6. • Repeat tasks 3-4.
7. • Repeat tasks 2-4.
8. • Change your device's timezone to A.
9. • Repeat tasks 3-4.

Ensure that date picker for date field with "Disallow future dates" option exhibits expected behaviour:

1. • Set your timezone as A.
2. • On the admin panel, refresh, select and save the "Disallow future dates" option for the date field.
3. • On the respondent form, refresh and open the date picker. Only the current and future dates of the timezone should be shown.
4. • Select a date. The appropriate date should be reflected in the date field.
5. • Change your device's timezone to B.
6. • Repeat tasks 3-4.
7. • Repeat tasks 2-4.
8. • Change your device's timezone to A.
9. • Repeat tasks 3-4.

Ensure that date picker for date field with "Custom date range" option exhibits expected behaviour:

1. • Set your timezone as A.
2. • On the admin panel, refresh and select the "Custom date range" option for the date field.
3. • Select a start date.
4. • Open the end date field date picker. Only the dates on or after the selected start date should be selectable.
5. • Select an end date.
6. • Save the selected date range. Optionally, check that the dates saved in the DB are in UTC.
7. • On the respondent form, refresh and open the date picker. Only the dates in the selected date range should be selectable.
8. • Select a date. The respective date should be reflected in the date field.
9. • Change your device's timezone to B.
10. • Repeat tasks 7-8.
11. • Repeat task 2. Ensure that the start and end dates remain the same.
12. • Update the start and end dates for the date field. Save it.
13. • Repeat tasks 7-8.
14. • Change your device's timezone to A.
15. • Repeat tasks 7-8.

Note: Here's how to set your Mac's timezone.

Design/ product question: Should "Disallow past dates" and "Disallow future dates" be taking the past / future based on the admins' timezone, specified timezone or the users' timezone? Also, even when we have decided on the previous question, does it even make sense if the concept of time is not considered? E.g. if the recurring event I want to organise is at 10am, our date picker won't be able to know that so if we have a timezone in mind, it won't "filter" properly. Currently, it's simply implemented based on the users' timezone so its not disorienting.

[foochifa]

A bit premature (as its still a draft), but just wanted to highlight the date stuffs

[LinHuiqing]

Will be adding the tests in a bit! Will try to be as extensive as possible so will take a while. But the videos kinda show what I think should be in the manual tests!

[LinHuiqing]

Okays tests have been updated! 👍

[foochifa]

Might be related to Shuli's comments

To replicate -> Create a date field -> set disallow past/future -> save
Go back into field -> unset disallow past/future -> validation failure

[LinHuiqing]

Might be related to Shuli's comments
<image>
To replicate -> Create a date field -> set disallow past/future -> save Go back into field -> unset disallow past/future -> validation failure

@foochifa should be fixed in b30c58c

[foochifa]

Yikes 😭

Is this caused by server side validation? My timezone is set in the US, hence 17/4/23 is a valid option.

Sorry mb! not a prob

[LinHuiqing]

<img alt="image" width="1213" src="https://user-images.githubusercontent.com/59867455/232657986-580a76fa-d316-4d1e-8df4-ac787291a6e7.png">

Yikes 😭

Is this caused by server side validation? My timezone is set in the US, hence 17/4/23 is a valid option.

I'm assuming you selected disallow past dates, then selected 17/4/23? I can't seem to replicate it though?

[foochifa]

Lgtm Thanks HQ!!