Distmaker: add CiviCRM Standalone support

[mlutfy]

Overview

Adds support for building CiviCRM Standalone tar.gz archives for nightly/rc/stable releases.

Before

No CiviCRM Standalone builds.

After

civicrm-standalone-5.66-standalone.tar.gz 🌄

Technical Details

I wasn't too sure what directory structure to follow.

Based on civicrm standalone installation docs, I figured it would be best to not use the composer template, and use the more minimal method. It still supports hosting civicrm and private files outside the webroot.

Comments

This may be of a surprise, but I did test locally:

./src/distmaker/distmaker.sh standalone

and then I checked the resulting tar, but I did not yet test using the tar to install CiviCRM from scratch.

[civibot]

Thank you for contributing to CiviCRM! ❤️ We will need to test and review the PR. 👷

Introduction for new contributors

• If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
• A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
• A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
• If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
• Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
• A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
• To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
• For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers

• ➡️ Demo site for this PR
• 📖 Review standards
• 🗒️ Review template (brief or verbose)

[totten]

Based on civicrm standalone installation docs, I figured it would be best to not use the composer template, and use the more minimal method. It still supports hosting civicrm and private files outside the webroot....

Yeah, I can see the rationale for skipping the composer template on this build. (There's kind of a chicken/egg problem with having distmaker do a release that depends on composer to fetch the release that distmaker is building... The actual dependency-loop is a little bigger than that... It's not impossible to address, but it does twist the brain a little.)

FWIW, in the layout defined by "Download CiviCRM directly (Standard web-server)", it actually does store civicrm in the public web folder (i.e. web/core and web/core/packages). A working system needs some way to serve static assets (CSS, JS, PNG, etc). Major options are:

1. Putcivicrm in the public folder.
   • Example: "Download CiviCRM directly (Standard web-server)"
2. Put civicrm in a private folder and define HTTPD rules for static assets.
   • Example: "Download CiviCRM directly (PHP built-in web-server)"
3. Put civicrm in a private folder and define sync rules for static assets.
   • Example: "Download the starter template" with civicrm-asset-plugin

IMHO, it's OK to do the "easy thing" and use web/core/ and web/core/packages/. A huge portion of Civi sites (D7/BD/WP/J) do use a public folder, and (IIRC) it's been a number of years since we actually had an issue stemming from that. It's not ideal, but it's not terrible.

If you do want a way to use a private folder without the composer plugin, then... you could try to sort out .htaccess and nginx options. I'm not sure how easy/hard that would be. Here's an example configuration from PHP 's built-in server. It could be locked down a bit further. (Allow folders js/ ang/ ext/; disallow folders CRM/ Civi/. Allow extensions *.js *.css; disallow extensions *.php *.sh *.conf.)

[mlutfy]

@totten thanks for clarifying - personally I'm fine with web/core being public (risks are minimal as you say). I was thinking more about ConfigAndLog and private uploads, that they would be nice to have outside the webroot.

To be honest I'm even on the fence about that, because many shared webhosts just have a public_html DocumentRoot that cannot be changed, and they would have to fiddle to move files up a directory, but I think it's an OK compromise, the absolutely lowest tier of hosting probably will struggle too much with CiviCRM anyway.

I did some r-run and found a few issues:

• core was not inside web
• the installer looks for files such as /assets/civicrm/core/setup/res/updated-logo.jpg - I noticed that setup/plugins/init/Standalone.civi-setup.php sets the path, but it's not detecting civicrm/civicrm-asset-plugin and going in the "else", so I'm not sure where it's being set.

Then had these installer warnings, but I don't think we can do anything about them, except the last one about cmsDb.password:

• The settings file "/home/mathieu/buildkit/build/xx.dev501.symbiodev.xyz/data/civicrm.settings.php" cannot be created. Ensure the parent folder is writable.
• The template compile dir "/home/mathieu/buildkit/build/xx.dev501.symbiodev.xyz/data/templates_c" cannot be created. Ensure the parent folder is writable.
• The property "cmsDb.password" is blank. This may be correct in some controlled environments; it could also be a mistake or a symptom of an insecure configuration.

I fixed the first two points by doing a chgrp www-data data, which is documented already I think in the normal cms installs.

[mlutfy]

Forced-push a fix for web/core, but I will circle back for the /assets URL bug

[totten]

the installer looks for files such as /assets/civicrm/core/setup/res/updated-logo.jpg - I noticed that setup/plugins/init/Standalone.civi-setup.php sets the path, but it's not detecting civicrm/civicrm-asset-plugin and going in the "else", so I'm not sure where it's being set.

I think the problem is higher up -- when index.php calls the setup APIs to load the GUI, it sets /assets/civicrm/core.

Maybe there's a way for GUI loader to use the output from Standalone.civi-setup.php? e.g.

// $coreUrl = '/assets/civicrm/core';

\Civi\Setup::init([ ... ]); // Fires `civi.setup.init` and thus `Standalone.civi-setup.php`

$coreUrl = \Civi\Setup::instance()->getModel()->mandatorySettings['userFrameworkResourceURL'];

[totten]

FWIW, chmod step is currently mentioned in https://docs.civicrm.org/installation/en/latest/standalone/#webserver. I don't actually see it in the other guides... not certain why... maybe, if the GUI gives advice, then the Installation Guide feels redundant? But then maybe it should be a nicer warning?

Like maybe:

1. Mention the name of the PHP web-user (www-data or www or similar) so that the admin doesn't have to guess
2. Show a hyperlink to a page with more discussion

I see a placeholder page at https://docs.civicrm.org/installation/en/latest/general/permissions/.

(Re: cmsDb -- I guess all we can do is suppress the check when uf==Standalone.)

[mlutfy]

@totten Ignore my last comment, sorry I need more coffee ;)

• Fixed the assets bug based on your example (I didn't test completely, but tested that the tar.gz output works)
• Removed the cmsDb warning for Standalone

I would circle back for the file permission warnings in a separate PR. There may be other issues, but I would prefer treating them separately.

One thing that seems to be missing is the web-accessible web/upload directory (which must be chowned/chmodded too), and oddly the installer did not complain about it.

[totten]

Cool. I did a full test-run (build civicrm-5.66.alpha1-standalone.tar.gz on one project; and use that to spin-up another local project on Apache VDR). The site was installed and active. 😃

[mlutfy]

@totten thanks! Now that nightly archives are working (and on gcloud), I added it to download.civicrm.org: civicrm/civicrm-dist-manager#12