CRM-21079, CRM-9683 - Display advisories about DATETIME/TIMESTAMP columns

[totten]

Overview

Some database columns were originally flagged as DATETIME when TIMESTAMP
would behave more appropriately. CRM-9683 changed the default type of
several CiviMail columns but did not provide an upgrade path for existing sites.

Before

Existing sites may be blissfully unaware of their problematic CiviMail
schema.

After

If a site has outdated CiviMail schema, then the system-status will display
an alert about "Timestamp Schema".

Comments

• This PR is a subset/pre-requisite for CRM-20958 - Track creation+modification times for activities+cases #10754. (Not anymore. Updated to make it make it separate.)
• If you want to understand why this is presented as an optional change
  (rather than an automatic upgrade), read the "Risk Profile"
  in CRM-21079.

Old Screenshot (A)

The screenshot is preserved for historical purposes. It's no longer current with the PR.

---

• CRM-21079: Datetime logs are wrong (mostly, a little bit)
• CRM-9683: Implement timezone support for CiviMail

[jmcclelland]

Beautiful! I love it.

[mfb]

anyone know if MySQL plans to fix the year 2038 problem for TIMESTAMP anytime soon?

[davejenx]

Thanks @totten for the detailed description and proposal. I think it looks good in general. Would be good to work out how best to present this to end users, who mostly won't understand the intricacies of timezone handling.

Timezones for Activities and Cases
A non-technical user reading the above message may well think, "sounds fine, why didn't you just do it; if it should be done, what am I supposed to do?"
Is the reason for not automatically back-filling these two timestamps during the upgrade, that this could take too long on large installations?
If there's a good reason for not doing this automatically, it would be good to guide the user towards the appropriate action. If the reason is duration, we could explain that, provide a Fix It link and advise doing this at a quiet time. If there's a judgement call to be made about whether to do it, it's good to minimise the number of cases where the user is left perplexed. So if there are scenarios where it's pretty clear-cut one way or the other, then describe those, so we're in effect giving a flowchart where most users end up at a clear terminus (Fix It link or Hide) and few are left in need of external support.

Timestamp Schema

The message shown in the screenshot above will make little sense to non-technical end users and few will know what they ought to do, so this is likely to generate a lot of support requests, for an issue that will make little difference to the majority. Again, a flowchart-like approach seems sensible, minimising the number of cases where the user is left perplexed.

[eileenmcnaughton]

Looking at the code only the Timestamp Schema portion is included in this update so we should focus on getting that right to get it merged. I feel like it might be enough to say that it is an optional update and sites with no issues do not need to take any action.

[totten]

Yes, @eileenmcnaughton is correct - the PR only touches the "Timestamp Schema" section.

Is the reason for not automatically back-filling these two timestamps during the upgrade, that this could take too long on large installations?

@davejenx - ah, no. The reason is that we're worried because (a) time handling has multiple moving parts and (b) downstream folks have diverse configurations/customizations/use-cases/workflows. Of course, we believe most folks would be well-served by a conversion. (IIRC, Fuzion did this for all their customers years ago... but that has a selection-bias.) We're not in a position to provide strong, clear statements or flowcharts for every possible user.

Producing a clearer flow-chart sounds quite nice... we just don't have the means to do that a-priori. (If someone disagrees, please prove me wrong by writing one!) The compromise in this PR is to soft-pedal: the changes are optional, the tooling is add-on/experimental, and feedback/revisions are welcome.

I do appreciate the critique, esp. since my original prose was really thin, and we can do a bit better. Here's a revision:

(Note: To improve beyond that... I don't think discussing principles will help me. We'd need concrete prose/revisions/alternatives -- eg in comments here or in follow-up PRs. There's a lot more background in the JIRA and DoctorWhen links.)

[eileenmcnaughton]

I'm giving this merge on pass because I do think the points have been addressed & because I agree that it would be better to make further improvements as follow-up PRs