Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(dev/core#174) Cache-keys for CRM_Utils_Cache_* should avoid reserved chars #12348

Merged
merged 10 commits into from
Jun 27, 2018

Conversation

totten
Copy link
Member

@totten totten commented Jun 21, 2018

Overview

PSR-16 defines some characters as supported (A-Za-z0-9_.), and some as reserved/forbidden ({}()/\@:), and the leaves the remainder to the implimenter.

We have a few scenarios in which reserved characters are passed to an instance of CRM_Utils_Cache_Interface. Change them.

"A string of at least one character that uniquely identifies
a cached item. Implementing libraries MUST support keys consisting of the
characters A-Z, a-z, 0-9, _, and . in any order in UTF-8 encoding and a
length of up to 64 characters. Implementing libraries MAY support
additional characters and encodings or longer lengths, but must support at
least that minimum. Libraries are responsible for their own escaping of key
strings as appropriate, but MUST be able to return the original unmodified
key string. The following characters are reserved for future extensions and
MUST NOT be supported by implementing libraries: {}()/\@:"

Before

  • CRM_Core_BAO_Cache, SettingsManager, CRM_Core_OptionGroup, CRM_Extension_Mapper pass keys to CRM_Utils_Cache_* that involve reserved characters.

After

  • CRM_Core_BAO_Cache, SettingsManager, CRM_Core_OptionGroup, CRM_Extension_Mapper pass keys to CRM_Utils_Cache_* that don't use any reserved characters.

Technical Details

  • To improve reading/debugging, it's ideal to change delimiters.
  • The formulas in CRM_Core_BAO_Cache and CRM_Core_OptionGroup are a bit more radical (using several reserved characters). Passing these through md5 makes valid keys, but they would be hard to read, so the formula is a little more complicated.

@civibot
Copy link

civibot bot commented Jun 21, 2018

(Standard links)

@totten totten force-pushed the master-clean-keys branch 2 times, most recently from 149704e to 3c0dbb7 Compare June 21, 2018 20:57
protected static function createCacheKey() {
$cacheKey = "CRM_OG_" . serialize(func_get_args());
protected static function createCacheKey($id) {
$cacheKey = "CRM_OG_" . preg_replace('/[^a-zA-Z0-9]/', '', $id) . '_' . md5(serialize(func_get_args()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use cleanKey here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could do it -- it'll work. The reason I didn't was cosmetic...

Anecdotally, cleanKey(...serialize(func_get_args())...) degraded to md5 form for every example I observed. (Which makes sense -- there's a lot of inputs, and serialize format uses a lot of meta chars.) This formula provides an intelligible prefix.

* Ex: '_abcd1234abcd1234' or 'ab_xx/cd_xxef'.
* A similar key, but suitable for use with PSR-16-compliant cache providers.
*/
public static function cleanKey($key) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test that throws a few strings at this function?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Done.

@totten
Copy link
Member Author

totten commented Jun 23, 2018

jenkins, test this please

@@ -385,8 +385,9 @@ public static function getSelectElements(
}

$argString = $all ? 'CRM_CT_GSE_1' : 'CRM_CT_GSE_0';
$argString .= $isSeparator ? '_1' : '_0';
$argString .= $isSeparator ? '_1' : ' _0';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why add a space here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! Don't think that should be there. I'll squash it.

@eileenmcnaughton
Copy link
Contributor

@totten this seems mergeable - and I endorse you to merge it - but there was just one line I queried above where a space was added that seemed like a possible mistake

@totten totten force-pushed the master-clean-keys branch from 405a038 to 8b25b1f Compare June 27, 2018 18:14
@totten
Copy link
Member Author

totten commented Jun 27, 2018

Thanks, @eileenmcnaughton.

I've rebased and sqaushed the mistaken character. Adding "merge on pass" based on that feedback.

@seamuslee001
Copy link
Contributor

Merging as per the tag

@seamuslee001 seamuslee001 merged commit 1b7de4f into civicrm:master Jun 27, 2018
@totten totten deleted the master-clean-keys branch June 27, 2018 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants