Fixes for running provision on an existing site. #210
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#2070: Updating a site via the Playbook fails due to field_abstract type mismatch Grok clone fails with error "Would clobber existing tag."
|
I've tried testing by doing the following, seeing a new error: On a fresh install, check out the last commit before Alpaca changes: and run Then checkout 2070-update-site-fixes branch and run again. Currently seeing this error: TASK [Islandora-Devops.crayfish : Build crayfish code including dependencies] ***
Monday 04 April 2022 10:38:41 -0300 (0:00:01.906) 0:05:59.160 **********
failed: [default] (item=Houdini) => {"ansible_loop_var": "item", "changed": false, "item": "Houdini", "msg": "Do not run Composer as root/super user! See https://getcomposer.org/root for details For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins. Installing dependencies from lock file Verifying lock file contents can be installed on current platform. In PackageFilter.php line 125: Undefined index: splits install [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress] [--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>...]", "stdout": "Do not run Composer as root/super user! See https://getcomposer.org/root for details\nFor additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins\nYou have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins.\nInstalling dependencies from lock file\nVerifying lock file contents can be installed on current platform.\n\nIn PackageFilter.php line 125:\n \n Undefined index: splits \n \n\ninstall [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress] [--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>...]\n\n", "stdout_lines": ["Do not run Composer as root/super user! See https://getcomposer.org/root for details", "For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins", "You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins.", "Installing dependencies from lock file", "Verifying lock file contents can be installed on current platform.", "", "In PackageFilter.php line 125:", " ", " Undefined index: splits ", " ", "", "install [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--dry-run] [--dev] [--no-suggest] [--no-dev] [--no-autoloader] [--no-progress] [--no-install] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--] [<packages>...]", ""]}
ok: [default] => (item=Milliner)
ok: [default] => (item=Hypercube)
ok: [default] => (item=Homarus)
ok: [default] => (item=Recast)
PLAY RECAP *********************************************************************
default : ok=321 changed=21 unreachable=0 failed=1 skipped=248 rescued=0 ignored=0
Monday 04 April 2022 10:38:58 -0300 (0:00:16.169) 0:06:15.330 **********
===============================================================================
geerlingguy.drupal : Install dependencies with composer require (this may take a while). - 122.42s
Islandora-Devops.crayfish : Build crayfish code including dependencies -- 16.17s
geerlingguy.solr : restart solr ---------------------------------------- 12.03s
install extra packages ubuntu ------------------------------------------ 11.66s
Islandora-Devops.crayfish : Install requisite packages ----------------- 10.56s
Islandora-Devops.fits : Downloading FITS executables -------------------- 9.28s
Islandora-Devops.drupal-openseadragon : Install Openseadragon with composer --- 6.52s
install python ---------------------------------------------------------- 5.50s
Copy solr config files -------------------------------------------------- 5.44s
install aptitude -------------------------------------------------------- 3.31s
Gathering Facts --------------------------------------------------------- 2.99s
geerlingguy.git : Ensure git is installed (Debian). --------------------- 2.40s
geerlingguy.apache : Ensure Apache is installed on Debian. -------------- 2.38s
geerlingguy.apache : Ensure Apache is installed on Debian. -------------- 2.37s
geerlingguy.git : Ensure git is installed (Debian). --------------------- 2.20s
Islandora-Devops.grok : Clone Grok -------------------------------------- 2.19s
Islandora-Devops.fcrepo-syn : Download syn jar file --------------------- 2.13s
gather facts ------------------------------------------------------------ 2.08s
geerlingguy.php : Ensure configuration directories exist. --------------- 2.01s
Islandora-Devops.crayfish : Install crayfish code ----------------------- 1.91s
Ansible failed to complete successfully. Any error output should be
visible above. Please fix tUnsure why this hasn't been a problem before. |
|
This is opening a few cans of worms it looks like. After fixing it so that the Crayfish application's file tree is not owned by root, composer install on each of the Crayfish apps is failing with an error in Houdini: Looks like an outdated dependent according to this: |
|
Seems like maybe we need to upgrade Crayfish from Flex to Symfony |
|
I'm hitting the drush conflict on the migrations piece when I go back to 3e502a3 |
Workaround for failing symfony/flex update.
|
@whikloj Got past the migrate now that #211 is merged. I'm using Vagrant snapshot to make this testing faster. After checking out 3e502a3 I ran $ vagrant snapshot save pre-alpacThen checking out this PR's branch, and running $ vagrant provisionThen if something fails to re-test I run $ vagrant snapshot restore pre-alpaca |
|
Finally got a clean Ansible playbook run to update a pre-Alpaca site. new error:This seems to be related to filehash, but if I now go to the Add Media page and choose a file, I get this error when hitting Save:
DBLog shows the following error: This is still occurring after running drush updatedb which has some file file hash related changes. Investigating further. |
|
@alxp hope you don't mind me putting my findings in here. You mentioned this PR in slack fixing some crawfish things. This fixes some steps in the new install but as is, it errors out at below because of permission issues. I am pretty sure this is because of the removal of the become. I am going to try removing that commit and running again. Update Same error even putting back in the Update: Put back the commit to remove - name: Fix Crayfish directory ownership
file:
state: directory
path: "{{ crayfish_install_dir }}"
owner: "{{ crayfish_user }}"
group: "{{ crayfish_user }}"
mode: "g+rw"
recurse: yes
force: yes
become: truealso become: truethe following tasks (since no longer running as root) and
|
|
I created a PR with the changes I had to make for this PR to install a new site. Feel free to merge/rebase later as needed. Thanks! |
|
Glad you got it working, I'll take a look at the changes tomorrow. Thanks! |
User/Permission changes needed for crayfish
|
Looks like another problem with Filehash and default configs. This function call in filehash.module is passing NULL in argument 3 which results in a type mismatch: function filehash_file_create(EntityInterface $file): void {
Drupal::service('filehash')->hash($file, NULL, Drupal::config('filehash.settings')->get('original'));
} |
|
Opened issue on Drupal dot org: Can't upload files after Installing File Hash 1.x, then updating to 2.x |
|
Resolved a file ownership problem with Alpaca and its log folder. This now works with the testing instructions above. Can verify that new data media get their derivatives generated, and it is Alpaca that is doing it. Other problems identified with pinning a Drupal stable version and importing features on a fresh site I will deal with in separate tickets.
|
|
Ok I pulled an old version of islandora-playbook and when upgrading it to this PR I hit this one. I didn't hit this last time, so I'm not sure if I did something different. I'm going to try again, but thought I would note this. |
|
This is brand new for a vulnerability in git. Let me find what other projects have done and share it
… On May 4, 2022, at 3:41 PM, Jared Whiklo ***@***.***> wrote:
Ok I pulled an old version of islandora-playbook and when upgrading it to this PR I hit this one.
TASK [Islandora-Devops.fits : Install FITS Microservice from Github] ***********
Wednesday 04 May 2022 15:29:07 -0500 (0:00:00.314) 0:05:54.471 *********
fatal: [default]: FAILED! => {"changed": false, "msg": "Failed to set a new url https://github.com/roblib/CrayFits.git for origin: fatal: unsafe repository ('/var/www/html/CrayFits' is owned by someone else)\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /var/www/html/CrayFits\n"}
I didn't hit this last time, so I'm not sure if I did something different. I'm going to try again, but thought I would note this.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.
|
|
You will probably need to add something like the following - name: "Get git config global safe directories dir"
shell: "git config --global --get-all safe.directory || echo ''"
register: "__git_config_global_safe_dir"
- name: "Set Fixity source directory as git safe dir"
command: "git config --global --add safe.directory {{ crayfits_home }}/CrayFits"
when:
- crayfits_home + '/CrayFits' not in __git_config_global_safe_dir.stdout_linearound this line https://github.com/Islandora-Devops/islandora-playbook/blob/dev/roles/internal/Islandora-Devops.fits/tasks/build-fits-site.yml#L26 |
|
Thanks @misilot , the announcement of the vulnurability is here: https://github.blog/2022-04-12-git-security-vulnerability-announced/ CrayFits is owned by www-data which is what we want I think, but the command will be being run by Vagrant, so adding the safe directory config option is probably the correct one. I'll put it in and push it up. |
|
This is coming up further along the build process also, ironically in Alpaca is where it appears next: TASK [Islandora-Devops.alpaca : Clone CLAW Alpaca] *****************************
Thursday 05 May 2022 09:53:14 -0300 (0:00:01.254) 0:07:05.792 **********
fatal: [default]: FAILED! => {"changed": false, "msg": "Failed to set a new url https://github.com/Islandora/Alpaca.git for origin: fatal: unsafe repository ('/opt/alpaca' is owned by someone else)\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /opt/alpaca\n"} |
|
Successful build on existing site. We should probably look at a more correct way to deal with checking out a repository to be one by one user in a folder owned by root but we can deal with that later. |
|
Ok, no more commits on this PR. I have built a site waay back at 3e502a3 and added some resources. Now "upgrading" the site. |
whikloj
approved these changes
May 31, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
GitHub Issue: (link)
#2070: Updating a site via the Playbook fails due to field_abstract type mismatch
Also fixes error where Grok clone fails with error "Would clobber existing tag."
Release pull requests, etc.)
What does this Pull Request do?
Fixes so that the playbook will run without error on an existing site.
Fixes a number of errors encountered when running the playbook on an existing Islandora installation.
What's new?
Does this change require documentation to be updated?
no (No where is the behaviour of re-importing the features documented)
Does this change add any new dependencies?
no
Does this change require any other modifications to be made to the repository
no
Could this change impact execution of existing code?
No
How should this be tested?
If available, run the playbook on an existing Islandora site with the standard install profile.
Also run the playbook as a fresh installation.
Make sure derivative generation works as expected.
Additional Notes:
Note that this PR includes the recent change to move from Karaf to the one-jar Alpaca, so this testing may uncover bugs relating to that update.
Interested parties
Tag (@ mention) interested parties or, if unsure, @Islandora-Devops/committers