Clean up code to add custom data to forms, implement on back office participant form

[eileenmcnaughton]

Overview

The existing function is crazy complicated & interacts with lots of undefined properties, per tests - but on digging the actual code used by non-contact, non multiple record form is actually quite simple - this adds a simple function that does the things that actually need to be done for processing custom data & implements it on the participant form

Before

Incredibly confusing code with lots of notices & undefined properties, causing php 8.x tests to fail

After

I managed to figure out what the end result of all that building group trees & passing group trees around & reformatting them and the part that is needed for forms that are not doing the actual rendering is https://github.com/civicrm/civicrm-core/pull/28733/files#diff-82b0e2eac7cfc3882a5e3cf5db3be5f7a8b42257f64366e8db74f2ddafb1f18cR45-R91

Which compares favourably to

civicrm-core/CRM/Custom/Form/CustomData.php

Lines 82 to 199 in 7d4bb5e

	public static function preProcess(
	&$form, $extendsEntityColumn = NULL, $subType = NULL,
	$groupCount = NULL, $type = NULL, $entityID = NULL, $onlySubType = NULL, $isLoadFromCache = TRUE
	) {
	if (!$type) {
	CRM_Core_Error::deprecatedWarning('type should be passed in');
	$type = CRM_Utils_Request::retrieve('type', 'String', $form);
	}
	if (!isset($subType)) {
	$subType = CRM_Utils_Request::retrieve('subType', 'String', $form);
	}
	if ($subType === 'null') {
	// Is this reachable?
	$subType = NULL;
	}
	$extendsEntityColumn = $extendsEntityColumn ?: CRM_Utils_Request::retrieve('subName', 'String', $form);
	if ($extendsEntityColumn === 'null') {
	// Is this reachable?
	$extendsEntityColumn = NULL;
	}
	if ($groupCount) {
	$form->_groupCount = $groupCount;
	}
	else {
	$form->_groupCount = CRM_Utils_Request::retrieve('cgcount', 'Positive', $form);
	}
	$form->assign('cgCount', $form->_groupCount);
	//carry qf key, since this form is not inhereting core form.
	if ($qfKey = CRM_Utils_Request::retrieve('qfKey', 'String')) {
	$form->assign('qfKey', $qfKey);
	}
	if ($entityID) {
	$form->_entityId = $entityID;
	}
	else {
	$form->_entityId = CRM_Utils_Request::retrieve('entityID', 'Positive', $form);
	}
	$typeCheck = CRM_Utils_Request::retrieve('type', 'String');
	$urlGroupId = CRM_Utils_Request::retrieve('groupID', 'Positive');
	if (isset($typeCheck) && $urlGroupId) {
	$form->_groupID = $urlGroupId;
	}
	else {
	$form->_groupID = CRM_Utils_Request::retrieve('groupID', 'Positive', $form);
	}
	$gid = (isset($form->_groupID)) ? $form->_groupID : NULL;
	if (!is_array($subType) && str_contains(($subType ?? ''), CRM_Core_DAO::VALUE_SEPARATOR)) {
	CRM_Core_Error::deprecatedWarning('Using a CRM_Core_DAO::VALUE_SEPARATOR separated subType deprecated, use a comma-separated string instead.');
	$subType = str_replace(CRM_Core_DAO::VALUE_SEPARATOR, ',', trim($subType, CRM_Core_DAO::VALUE_SEPARATOR));
	}
	$singleRecord = NULL;
	if (!empty($form->_groupCount) && !empty($form->_multiRecordDisplay) && $form->_multiRecordDisplay == 'single') {
	$singleRecord = $form->_groupCount;
	}
	$mode = CRM_Utils_Request::retrieve('mode', 'String', $form);
	// when a new record is being added for multivalued custom fields.
	if (isset($form->_groupCount) && $form->_groupCount == 0 && $mode == 'add' &&
	!empty($form->_multiRecordDisplay) && $form->_multiRecordDisplay == 'single') {
	$singleRecord = 'new';
	}
	$groupTree = CRM_Core_BAO_CustomGroup::getTree($type,
	NULL,
	$form->_entityId,
	$gid,
	$subType,
	$extendsEntityColumn,
	$isLoadFromCache,
	$onlySubType,
	FALSE,
	CRM_Core_Permission::EDIT,
	$singleRecord
	);
	if (property_exists($form, '_customValueCount') && !empty($groupTree)) {
	$form->_customValueCount = CRM_Core_BAO_CustomGroup::buildCustomDataView($form, $groupTree, TRUE, NULL, NULL, NULL, $form->_entityId);
	}
	// we should use simplified formatted groupTree
	$groupTree = CRM_Core_BAO_CustomGroup::formatGroupTree($groupTree, $form->_groupCount, $form);
	if (isset($form->_groupTree) && is_array($form->_groupTree)) {
	$keys = array_keys($groupTree);
	foreach ($keys as $key) {
	$form->_groupTree[$key] = $groupTree[$key];
	}
	}
	else {
	$form->_groupTree = $groupTree;
	}
	}
	/**
	* @param CRM_Core_Form $form
	*
	* @return array
	*/
	public static function setDefaultValues(&$form) {
	$defaults = [];
	CRM_Core_BAO_CustomGroup::setDefaults($form->_groupTree, $defaults, FALSE, FALSE, $form->get('action'));
	return $defaults;
	}
	/**
	* @param CRM_Core_Form $form
	*/
	public static function buildQuickForm(&$form) {
	$form->addElement('hidden', 'hidden_custom', 1);
	$form->addElement('hidden', "hidden_custom_group_count[{$form->_groupID}]", $form->_groupCount);
	CRM_Core_BAO_CustomGroup::buildQuickForm($form, $form->_groupTree);
	}

I suspect there might be a better way to load the custom data entity IDs that @colemanw will suggest - although I would prefer to do that as a follow up at this point (since it has been a slog & what is there is not that bad)

Technical Details

I identified that required fields are enforced by ajax in popup mode but nothing enforces them in non popup mode - that was pre-existing.

I tested with money, number, text, radio and file fields and with custom groups extending role, event id, and event type on create new and on edit

Comments

[civibot]

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers...

• ➡️ PR Demo site (can be rebuilt with the command test this please).
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

[eileenmcnaughton]

error directly relates (which is helpful)
CRM_Event_Form_ParticipantTest.testSubmitWithCustomData

[eileenmcnaughton]

We should also merge #28735 & then hopefully not touch that form again for a while

[colemanw]

non-contact, non multiple record form

It would be really great to bear in mind that the "for contacts only" rule of multi-value custom groups is being lifted. In theory now any entity can have multi-record custom data attached to it... the only obstacle is the quickforms.

So if this new refactor is flexible enough to handle those for any entity, that would be a game-changer.

[eileenmcnaughton]

@colemanw I'm not sure it gets us there with the multiple fields - that's probably out of scope here - although cleaner code is easier to fix up...

[colemanw]

Ok I ran this by making various sets of participant custom fields & registering someone on the backoffice form. All fields showed up and saved correctly.

[eileenmcnaughton]

thanks @colemanw