Contact tokens - use metadata & standardised rendering

[eileenmcnaughton]

Overview

Contact tokens - use metadata & standardised rendering.

This fixes a temporary regression (master only during refactor) where the contact tokens that were being advertised were not getting their labels translated.

It also gets us to the point where the contact tokens are being rendered based on metadata for all fields - meaning that expectations about formatting money & dates is consistent with other token entities and it uses the parent functions more.

The underlying issue is that the contact token class is using the bao query object mechanism to retrieve the contact & just treating that as the token contract - which means its a bit random. The goal is that we would have contact tokens that
reflect the token names elsewhere - which closely reflect apiv4.

However, since there are still some persisiting code paths that use legacy mechanisms to replace contact tokens I didn't want to make significant changes to the tokens we advertise in case they are being presented on a screen that does not support them.

Before

The mapping between apiv3 & v4 fields is opaque

After

The mapping is explicitly declared.

Minor changes are made to the advertised fields
https://github.com/civicrm/civicrm-core/pull/21761/files#diff-c3c60600f080b49f2a9029b82865098c2a2742fbcedce0ecd731376917d80e1bL513-L597

Of those

• the ones where 'label' is added will still be handled fine through the legacy replaceContactTokens method (which strips the 'label' before resolving)
• {contact.current_employer_id}' is now advertised as {contact.employer_id}. The old version still works but is not advertised.
  The new version may not work for flows using the old code path - but note that it would be an unusual token to use with 'current_employer` being more prevalent
• I hated 'manual_geocode' as a token - it was too irrational even for me - I removed it

But the old fields still work with very minor output changes (ie 'on_hold' now renders to 0 not FALSE and birth_date no longer shows the time as 'midnight')

https://github.com/civicrm/civicrm-core/pull/21761/files#diff-c3c60600f080b49f2a9029b82865098c2a2742fbcedce0ecd731376917d80e1bR790-R848

Technical Details

While I wouldn't call the code 'elegant' this concludes the cleanup with consistent rendering of tokens, no calls to the BAO query object and the code being generally 'in the right place' - ie contacts have their own tokens class rather than being in 'token compat subscriber'

Comments

[civibot]

(Standard links)

• If this is your first pull-request for CiviCRM, please browse CONTRIBUTING.md for information about the development and testing processes.
• If you are reviewing this pull-request, you may wish to consult the test sites and the Review Standards (long template, short template).

[eileenmcnaughton]

test this please

[eileenmcnaughton]

test this please

[eileenmcnaughton]

early return is just for clarity / marginal performance

[eileenmcnaughton]

@colemanw @totten so this is the last piece for this round - passing now

[totten]

I like that this splits getOldContactTokens() and getAllContactTokens(). To my eye, this test-coverage is the main thing demonstrating the safety of the change.

The only thing to point out is the list of changes from the description:

Of those

• the ones where 'label' is added will still be handled fine through the legacy replaceContactTokens method (which strips the 'label' before resolving)
• {contact.current_employer_id}' is now advertised as {contact.employer_id}. The old version still works but is not advertised.
  The new version may not work for flows using the old code path - but note that it would be an unusual token to use with 'current_employer` being more prevalent
• I hated 'manual_geocode' as a token - it was too irrational even for me - I removed it

This needs to be propagated to the documentation where it summarizes token changes, right? That can be done async, of course.

[magnolia61]

Hi @eileenmcnaughton
I found a problem in the 5.43 RC with contact core tokens not rendering in the body but most do render in the subject.
https://lab.civicrm.org/dev/core/-/issues/2907

[magnolia61]

It seems the changes in this pr cause the issue

[eileenmcnaughton]

@magnolia61 I just tried again to replicate - very simple attempt where I did a search for 50 participants & had a body of just {contact.first_name} and used send email. I understood that wouldn't work from your report but it did

[magnolia61]

I think I found it: https://lab.civicrm.org/dev/core/-/issues/2907

[eileenmcnaughton]

See #21812