Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CiviGrant - Fix installation of dependencies during upgrade #22881

Merged
merged 6 commits into from
Mar 4, 2022

Conversation

colemanw
Copy link
Member

@colemanw colemanw commented Mar 3, 2022

Overview

During the upgrade process the CiviGrant extension may be installed. This ensures its dependencies are also enabled.

Fixes https://lab.civicrm.org/dev/core/-/issues/3093. Alternative to #22861 and #22880.

(Note: PR description filled-in after a couple rounds of editing. Reflects final form.)

Before

The FiveFortySeven upgrade migrates CiviGrant data and partially installs the civigrant extension. It does not activate dependencies like afform and search_kit.

This leaves the system in an inconsistent state and leads to errors.

After

The FiveFortySeven upgrade migrates CiviGrant and fully installs the CiviGrant extension along with its dependencies (whatever they may be at time of execution; eg afform and search_kit).

Technical Details

The primary change is to split/move/expand the installation step:

  • During the main body of the core schema updates, it migrates some core records and inserts a placeholder for civicrm_extension. This placeholder is inactive (is_active=0).
  • After the core schema updates, it performs a full/standard installation. This enables dependencies and fires relevant hooks.

This requires some revisions in CRM/Upgrade. The CRM/Upgrade system builds a queue of upgrade-tasks (specifically, a "priority queue" sorted on (weight,id)). Moving the installation step requires setting the weight explicitly.

@civibot
Copy link

civibot bot commented Mar 3, 2022

(Standard links)

@civibot civibot bot added the master label Mar 3, 2022
@demeritcowboy
Copy link
Contributor

Should be 5.47 right? But I can test in the meantime.

@colemanw colemanw changed the base branch from master to 5.47 March 3, 2022 15:11
@civibot civibot bot added 5.47 and removed master labels Mar 3, 2022
@demeritcowboy
Copy link
Contributor

I'm still getting that type error at

return \Civi\Api4\SearchDisplay::getFields(FALSE)
. $props is ['name'] at that point, and the return value (the value of the 'options' element) is FALSE. Sample stack trace:

1 ...\sites\all\modules\civicrm\ext\search_kit\Civi\Search\AfformSearchMetadataInjector.php(29): Civi\Search\Display::getDisplayTypes((Array:1))
2 [internal function](): Civi\Search\AfformSearchMetadataInjector::Civi\Search\{closure}(Object(phpQueryObject), "~/afsearchGrants/afsearchGrants.aff.html")
3 ...\sites\all\modules\civicrm\Civi\Angular\ChangeSet.php(59): call_user_func(Object(Closure), Object(phpQueryObject), "~/afsearchGrants/afsearchGrants.aff.html")
4 ...\sites\all\modules\civicrm\Civi\Angular\ChangeSet.php(19): Civi\Angular\ChangeSet::applyHtmlFilters((Array:2), (Array:1))
5 ...\sites\all\modules\civicrm\Civi\Angular\Manager.php(295): Civi\Angular\ChangeSet::applyResourceFilters((Array:2), "partials", (Array:1))
6 ...\sites\all\modules\civicrm\Civi\Angular\Page\Modules.php(160): Civi\Angular\Manager->getPartials("afsearchGrants")
7 ...\sites\all\modules\civicrm\Civi\Angular\Page\Modules.php(82): Civi\Angular\Page\Modules->getMetadata((Array:18), Object(Civi\Angular\Manager))
8 ...\sites\all\modules\civicrm\vendor\symfony\event-dispatcher\EventDispatcher.php(214): Civi\Angular\Page\Modules::buildAngularModules(Object(Civi\Core\Event\GenericHookEvent), "hook_civicrm_buildAsset", Object(Civi\Core\CiviEventDispatcher))
9 ...\sites\all\modules\civicrm\vendor\symfony\event-dispatcher\EventDispatcher.php(44): Symfony\Component\EventDispatcher\EventDispatcher->doDispatch((Array:6), "hook_civicrm_buildAsset", Object(Civi\Core\Event\GenericHookEvent))
10 ...\sites\all\modules\civicrm\Civi\Core\CiviEventDispatcher.php(198): Symfony\Component\EventDispatcher\EventDispatcher->dispatch("hook_civicrm_buildAsset", Object(Civi\Core\Event\GenericHookEvent))
11 ...\sites\all\modules\civicrm\CRM\Utils\Hook.php(167): Civi\Core\CiviEventDispatcher->dispatch("hook_civicrm_buildAsset", Object(Civi\Core\Event\GenericHookEvent))
12 ...\sites\all\modules\civicrm\CRM\Utils\Hook.php(2500): CRM_Utils_Hook->invoke((Array:4), "angular-modules.json", (Array:1), "application/json", NULL, NULL, NULL, "civicrm_buildAsset")
13 ...\sites\all\modules\civicrm\Civi\Core\AssetBuilder.php(220): CRM_Utils_Hook::buildAsset("angular-modules.json", (Array:1), "application/json", NULL)
14 ...\sites\all\modules\civicrm\Civi\Core\AssetBuilder.php(192): Civi\Core\AssetBuilder->render("angular-modules.json", (Array:1))
15 ...\sites\all\modules\civicrm\Civi\Core\AssetBuilder.php(135): Civi\Core\AssetBuilder->build("angular-modules.json", (Array:1))
16 ...\sites\all\modules\civicrm\Civi\Angular\AngularLoader.php(150): Civi\Core\AssetBuilder->getUrl("angular-modules.json", (Array:1))
17 ...\sites\all\modules\civicrm\CRM\Core\Resources\CollectionTrait.php(389): Civi\Angular\AngularLoader->Civi\Angular\{closure}()
18 ...\sites\all\modules\civicrm\CRM\Core\Region.php(134): CRM_Core_Region->getSettings()
19 ...\sites\all\modules\civicrm\CRM\Core\Region.php(149): CRM_Core_Region->{closure}((Array:9))
20 ...\sites\all\modules\civicrm\drupal\civicrm.module(156): CRM_Core_Region->render("")
21 ...\includes\common.inc(5993): civicrm_page_build((Array:8))
22 ...\includes\common.inc(2795): drupal_render_page((Array:8))
23 ...\includes\common.inc(2655): drupal_deliver_html_page("\n<div id=\"crm-container\" class=\"crm-container\" lang=\"en\" xml:lang=\"en...")
24 ...\includes\menu.inc(542): drupal_deliver_page("\n<div id=\"crm-container\" class=\"crm-container\" lang=\"en\" xml:lang=\"en...", "")
25 ...\index.php(21): menu_execute_active_handler()
26 {main}

@demeritcowboy
Copy link
Contributor

If I disable civigrant, uninstall search kit, reinstall search kit, then enable civigrant, then it's ok and I can see the search_display_type option group is now there, whereas it wasn't before.

@colemanw
Copy link
Member Author

colemanw commented Mar 3, 2022

So the problem is that the managed entity provided by SearchKit isn't getting inserted? Do you have to do all of those steps to get it to work, or is a cache clear enough?

@demeritcowboy
Copy link
Contributor

Cache clear isn't enough.

totten and others added 4 commits March 3, 2022 17:55
This guard was added by 912511a as part of
a previous approach to managing hooks during upgrades.  This general
approach changed with civicrm#17126; so
17126 partially undid this... but it inadvertently had the effect of
completely disabling `reconcile()` (because this guard was left).
Before
------

`migrateCiviGrant()` migrates some metadata from core-ownership to
core-extension-ownership...  and it ALSO activates the extension.  However,
the extension depends on other (possibly-inactive) extensions.  This
creates an inconsistent state (where active parts of `civigrant`
depend on inactive parts of `search_kit`).

In this inconsistent state, `ManagedEntities::reconcile()` fails.

After
-----

`migrateCiviGrant()` still migrates metadata.  However, it initially leaves
the extension inactive.  So `ManagedEntities::reconcile()` won't try to
setup these records.

After core schema is fully resolved, then it installs all necessary
extensions using normal mechanisms (with normal ordering).
@totten
Copy link
Member

totten commented Mar 4, 2022

Pushed up more fixes. Briefly:

  1. ManagedEntities::reconcile() did not reconcile during upgrades... because... it had a rule that says "Don't reconcile during upgrades".
  2. migrateCiviGrant(): Split apart the task of migrating to mgd format and the task of activating the extension.

There are longer descriptions in the commits.

@totten totten force-pushed the civiGrantFixSecondAttempt branch from 72a5313 to 458e2d8 Compare March 4, 2022 02:00
@demeritcowboy
Copy link
Contributor

So far so good.

I'll try a couple more, and make a separate ticket for the civireport menu issue, which I wouldn't call a blocker.

@totten
Copy link
Member

totten commented Mar 4, 2022

Hmm... it doesn't like upgrading from 5.39.1... but there are higher+lower versions which are OK...

@demeritcowboy
Copy link
Contributor

5.39.1 is one of the ones I'm working with. But I went straight up, not in steps like that test. I think that acl_bypass has come up elsewhere and I thought it was dealt with.

Note: This step was added to the codebase circa 5.47. The underlying schema
change actually originated circa 5.46. The step should be idempotent.
@totten totten force-pushed the civiGrantFixSecondAttempt branch from 1386dd2 to 448a357 Compare March 4, 2022 05:00
@demeritcowboy
Copy link
Contributor

Ok I did another little round on the latest here and I think I've done about as much testing as I'm going to on this so 👍 .

@totten totten merged commit 26eefaf into civicrm:5.47 Mar 4, 2022
@seamuslee001 seamuslee001 deleted the civiGrantFixSecondAttempt branch March 4, 2022 22:07
@totten totten changed the title CiviGrant install fix second attempt CiviGrant - Fix installation of dependencies during upgrade Mar 4, 2022
@eileenmcnaughton
Copy link
Contributor

@totten FYI - this has caused a regression - analysis in https://lab.civicrm.org/dev/core/-/issues/3119

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants