Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mod_Security Rule Issue with Comet Cache Pro Updater #416

Closed
seozones opened this issue Feb 18, 2015 · 14 comments
Closed

Mod_Security Rule Issue with Comet Cache Pro Updater #416

seozones opened this issue Feb 18, 2015 · 14 comments

Comments

@seozones
Copy link

Hi there. I put in a request regarding this issue awhile back and it seems the issue is still unresolved. When I use the Pro Updater, it redirects to a 404 error page (page cannot be found). My techs have to disable a Mod_Security rule in order for me to update the plugin automatically using the Pro Updater which is a pain because I have many sites across many servers so it gets old asking them to disable the security rule for each of my sites. My tech has told me this (back on Nov 18):

It was mod_security rule #340163, here's the full line from our logs:

[Tue Nov 18 18:30:18 2014] [error] ModSecurity: Access denied with code 403 (phase 2). Match of "rx ://%{SERVER_NAME}/" against "ARGS:quick_cache__update_zip" required. [file "/usr/local/apache/conf/mod_sec/mod_sec.asl.conf"] [line "547"] [id "340163"] [rev "289"] [msg "Atomicorp.com UNSUPPORTED DELAYED Rules: Remote File Injection attempt in ARGS (MM)"] [data "http://wsharks.com/1vlhatG?file=quick-cache-pro-v141110.zip [1]";] [severity "CRITICAL"] [hostname "www.goldleafcare.com [2]";] [uri "/wp-admin/update.php"] [unique_id "VGvWisYL08cAAAOvAOkAAAAN"]

However, I was told that the issue would be resolved in a near future release. Any update on that fix?

@raamdev
Copy link
Contributor

raamdev commented Feb 19, 2015

@seozones Thank you for the report.

ModSecurity has been known to cause issues like this with many different plugins; ModSecurity is designed to be overly cautious and often ends up blocking valid requests. We generally recommend requesting that your web host whitelist the specific rule causing problems (in this case #340163) so that the Pro Updater can run properly or, if you'd rather not have any rules disabled, to manually download and update Quick Cache Pro whenever an update is available.

That said, I'm going to leave this GitHub issue open so that we can do some more research and see if it's possible to get around this specific ModSecurity rule by changing how the Pro Updater works.

@seozones
Copy link
Author

That would be great if you can find a workaround. Thanks raamdev!

@jaswrks
Copy link

jaswrks commented Mar 2, 2015

Remote File Injection attempt in ARGS (MM)"] [data "http://wsharks.com/1vlhatG?file=quick-cache-pro-v141110.zip [1]";]

@raamdev This URL is actually base64 encoded by Zen/Quick Cache. I see in this report that it is raw. Do you know if ModSecurity is capable of decoding the value? I'm guessing that is what happened here.

@jaswrks
Copy link

jaswrks commented Mar 2, 2015

Referencing this line in a private repo.

@raamdev
Copy link
Contributor

raamdev commented Mar 2, 2015

Do you know if ModSecurity is capable of decoding the value?

Yes, it is. The Base64 decoder in ModSecurity is based off the RFC-4648 implementation of Base64. It most likely decodes Base64-encoded strings because obfuscating malicious code by Base64-encoding it is a very common thing, especially in PHP scripts. My guess is ModSecurity decodes the string for logging and/or comparison against known attack vectors.

@jaswrks
Copy link

jaswrks commented Mar 2, 2015

What Needs to Happen Here?

  • We need to use proprietary encryption on this string, or store it in a transient and reference it by ID.

@jaswrks
Copy link

jaswrks commented Mar 26, 2015

Adding a list of items (↑ above) that need to occur.

@raamdev raamdev added this to the Next Release (Pro) milestone Oct 8, 2015
@raamdev raamdev modified the milestones: Next Release (Pro), Future Release (Pro) Oct 29, 2015
@raamdev raamdev modified the milestones: Next Release (Pro), Future Release (Pro) Dec 2, 2015
@raamdev raamdev modified the milestones: Next Release (Pro), Future Release (Pro) Dec 28, 2015
@raamdev raamdev modified the milestones: Future Release (Pro), Next Release (Pro); 1st CC Notice Feb 13, 2016
@raamdev raamdev modified the milestones: Next Release, Future Release Feb 26, 2016
@raamdev raamdev removed this from the Next Release milestone Apr 4, 2016
@raamdev
Copy link
Contributor

raamdev commented May 21, 2016

Noting that we came up for a workaround for this issue (in the private team repo) that utilizes WP Transients. This issue is now ready for work:


In proUpdate() we could modify the existing behavior to store the zip file URL in a WordPress Transient:

// [...]
GLOBAL_NS.'_update_pro_version' => $product_api_response->pro_version,
// GLOBAL_NS.'_update_pro_zip'     => base64_encode($product_api_response->pro_zip),

$is_multisite  = is_multisite();
$get_transient = $is_multisite ? 'get_site_transient' : 'get_transient';
$set_transient = $is_multisite ? 'set_site_transient' : 'set_transient';

// Store the Pro Zip URL in Transient with a 5 minute expiration
if (false === ($get_transient(GLOBAL_NS.'_update_pro_zip_'.$product_api_response->pro_version))) {
                $set_transient(GLOBAL_NS.'_update_pro_zip_'.$product_api_response->pro_version,  $product_api_response->pro_zip, 60*5);
}

Then on the other side (in preSiteTransientUpdatePlugins()), we could do the following:

if (empty($_r[GLOBAL_NS.'_update_pro_version'])) {
        return $transient; // Nothing to do here.
    }

$is_multisite  = is_multisite();
$get_transient = $is_multisite ? 'get_site_transient' : 'get_transient';
$set_transient = $is_multisite ? 'set_site_transient' : 'set_transient';
$delete_transient = $is_multisite ? 'delete_site_transient' : 'delete_transient';

if (false === ($update_pro_zip = $get_transient(GLOBAL_NS.'_update_pro_zip_'.$_r[GLOBAL_NS.'_update_pro_version']))) {
        return $transient; // Nothing to do here.
    }

// Clean up; transient value has been retrieved above if we got this far, so let's delete it.
$delete_transient(GLOBAL_NS.'_update_pro_zip_'.$product_api_response->pro_version);

$update_pro_version = (string) $_r[GLOBAL_NS.'_update_pro_version'];
// $update_pro_zip     = base64_decode((string) $_r[GLOBAL_NS.'_update_pro_zip'], true);

@raamdev raamdev added this to the Next Release milestone May 21, 2016
@raamdev raamdev changed the title Mod_Security Rule Issue with Quick Cache Pro Updater Mod_Security Rule Issue with Comet Cache Pro Updater May 21, 2016
@jaswrks
Copy link

jaswrks commented May 21, 2016

Noting that we can avoid the complication of _site_ or not in those transient calls, because it's safe to use get_site_transient() and get_site_option() in Comet Cache; i.e., CC is a network-only plugin on a Multisite installation, so it simplifies that for us automatically.

@raamdev
Copy link
Contributor

raamdev commented May 21, 2016

because it's safe to use get_site_transient() and get_site_option() in Comet Cache; i.e., CC is a network-only plugin on a Multisite installation, so it simplifies that for us automatically.

I'm sorry, that's not clear to me. Are you saying that we can use get_site_transient() and drop the usage of get_transient() even if we're not on a multisite network? (And if that's not correct, then how can we avoid the complication? Don't we need to consider non-Multisite and Multisite here?)

@raamdev
Copy link
Contributor

raamdev commented May 21, 2016

@jaswsinc Also, while testing the above code, I'm getting "The plugin is at the latest version." whenever I try to run the Pro Plugin Updater (upgrading from v160417 → v160521). That message is coming from wp-admin/includes/class-wp-upgrader.php. I'm still trying to figure out what's going on, but I know you've spent more time with this area of the codebase so I thought I'd see if you have any ideas. :-)

@raamdev raamdev self-assigned this May 21, 2016
@jaswrks
Copy link

jaswrks commented May 21, 2016

Are you saying that we can use get_site_transient() and drop the usage of get_transient() even if we're not on a multisite network?

That is correct, yes. Those functions work on both types of installs with graceful/logical fallbacks. So you can just use get_site_option() and get_site_transient() explicitly without any further consideration in the CC plugin.

@raamdev
Copy link
Contributor

raamdev commented May 21, 2016

Next Release Changelog:

  • Enhancement: The Pro Plugin Updater has been improved to allow for better compatibility with hosting platforms that use Apache's ModSecurity. In some cases, site owners were seeing a 404 error when attempting to update the Pro version using the Pro Plugin updater because certain ModSecurity rules were blocking the Pro Updater requests. The Pro Plugin Updater now uses WP Transients to store the necessary metadata, which works around the issue with ModSecurity. Props to @seozones for reporting and @jaswsinc for help fixing this. Issue #416.

@raamdev raamdev closed this as completed May 21, 2016
raamdev added a commit that referenced this issue Jul 7, 2016
- **New Feature! Apache Optimizations.** This release includes a completely new option panel for Apache Performance Tuning. Current options for Apache tuning include GZIP Compression, Leverage Browser Caching, Enforce Canonical URLs, and Send Access-Control-Allow-Origin Header (for Static CDN Filters). These options automatically add or remove from your `.htaccess` file the appropriate configuration based on the options you enable or disable (all options are disabled by default, so your `.htaccess` file is not modified unless you say so). If you prefer to update your `.htaccess` file manually, the necessary configuration can be viewed beneath each option. Props @jaswsinc, @renzms. See [Issue #789](#789).
- **New Feature!** A new "Enable GZIP Compression" option has been added to the new Apache Optimizations panel. This option will automatically add the appropriate configuration to your `.htaccess` file to enable GZIP compression. This option is disabled by default. The old "GZIP Compression" panel has been removed in favor of the new option inside Apache Optimizations. Props @renzms, @jaswsinc. See [Issue #764](#764).
- **New Feature!** Multisite Host Exclusion Patterns. It's now possible to exclude entire sites from the cache in a Multisite Network environment. Domain mapping is also supported! See _Comet Cache → Plugin Options → Host Exclusion Patterns_. If you're running a Multisite Network with Sub-Directories, you can exclude sites using the existing URI Exclusion Patterns feature. Props @kristineds. See [Issue #754](#754).
- **New Feature (Pro)!** A new "Leverage Browser Caching" option has been added to the new Apache Optimizations panel. This option will automatically add the appropriate configuration to your `.htaccess` file to enable Browser Caching. This option is disabled by default. Props @renzms, @jaswsinc. See [Issue #764](#764).
- **New Feature (Pro)!** A new "Enforce Canonical URLs" option has been added to the new Apache Optimizations panel. This options adds the appropriate `.htaccess` code to enforce the correct canonical URLs according to your WordPress Permalink settings (Comet Cache detects if the Permalink Structure ends with a trailing slash, or without a trailing slash). Props @renzms, @jaswsinc. See [Issue #554](#554).
- **Bug Fix**: In some scenarios the Cron Event that cleans up expired cache files (`_cron_comet_cache_cleanup`) would never run, or the Next Run time would constantly reset to 1 minute away from running every time a page was reloaded. We suspect this is a race condition and in attempt to work around this issue we now skip all of our Cron-related checks if Cron is currently in the middle of running a process. Props @xberg and @lkraav for help reporting. See [Issue #653](#653).
- **Bug Fix**: If your site uses aliased domains, Comet Cache now properly considers all possible domain variations when it clears the cache on WP Standard installations. Props @kristineds, @jaswsinc, @yoffe, and @VR51. See [Issue #608](#608).
- **Bug Fix** (Pro): Fixed a bug where Comet Cache would appear to prevent WordPress from redirecting Permalinks that don't include a trailing slash, to the URL that does include a trailing slash. This was due to the fact that Comet Cache loads very early on (for caching purposes) and as a result the WordPress `redirect_canonical()` function never gets run. This was fixed by adding an option to the new Apache Optimizations panel that allows you to Enforce Canonical URLs. Props @renzms, @jaswsinc. See [Issue #554](#554).
- **UX Bug Fix** (Pro): If you had your WordPress Dashboard login details saved by your browser, the browser autofill would automatically fill in the Pro Plugin Updater fields with those details, which then needed to be replaced with your actual Pro license details. The browser autofill has been disabled for those fields (tested in Chrome, Firefox, and Safari). Props @renzms. See [Issue #741](#741).
- **Enhancement**: Added links the Options Page for the Comet Cache [Twitter](http://twitter.com/cometcache) and [Facebook](http://facebook.com/cometcache) accounts. Props @renzms. [Issue #771](#771).
- **Enhancement:** Added full support for UTF-8 (multibyte strings). This release adds full support for UTF-8 throughout the Comet Cache codebase, greatly enhancing Comet Cache's ability to deal with file paths and URLs that may contain UTF-8 characters. Props @jaswsinc. [Issue #703](#703).
- **UI Enhancements**: Improved the Logged-In Users and the Client-Side Caching options panels to dim additional options when the feature is disabled. Additionally, the "Enable HTML Compression for Logged-In Users?" option has been relocated from the HTML Compressor option panel to the more appropriate Logged-In Users option panel. See [Issue #768](#768).
- **UX Enhancement**: Improved the inline docs for Auto-Clear List of Custom URLs to clarify that full URLs must be provided. Props @renzms. See [Issue #781](#781).
- **Enhancement** (Pro): The Pro Plugin Updater has been improved to allow for better compatibility with hosting platforms that use Apache's ModSecurity. In some cases, site owners were seeing a 404 error when attempting to update the Pro version using the Pro Plugin updater because certain ModSecurity rules were blocking the Pro Updater requests. The Pro Plugin Updater now uses WP Transients to store the necessary metadata, which works around the issue with ModSecurity. Props to @seozones for reporting and @jaswsinc for help fixing this. [Issue #416](#416).
- **Enhancement** (Pro): When Static CDN Filters are enabled, it's now possible to disable the automatic insertion of rules into your `.htaccess` file that are designed to prevent issues with [CORS](https://cometcache.com/kb-article/what-are-cross-origin-request-blocked-errors/). See _Apache Optimizations → Send Access-Control-Allow-Origin Header?_ See [Issue #787](#787).
- **Enhancement** (Pro): The HTML Notes added to the bottom of a cached page now specify if the page was cached as the result of an HTTP Request or if it was cached by the Auto-Cache Engine. Props @kristineds. See [Issue #292](#292).
- **Enhancement** (Pro): The Auto-Cache Engine now supports a fallback to cURL using the WP HTTP API. If your PHP configuration has `allow_fopen_url=0`, the Auto-Cache Engine will use the fallback to download the XML Sitemap and parse it from a temporary file. If you want to force the use of this fallback even when `allow_fopen_url=1`, you can use [a filter](#440 (comment)). See [Issue #440](#440).
- **UI Enhancement** (Pro): A second button has been added to the bottom of the Pro Plugin Updater page that allows you to "Save and Update Comet Cache Pro" in one step. Props @renzms. See [Issue #741](#741).
- **UI Enhancement** (Pro): The "Cache Stats" button in Admin Bar is now linked to the Cache Stats page. Instead of hovering over the button and then clicking "More Info" inside the popup panel, you can now just click the "Cache Stats" button to go directly to the Cache Stats page. Props @Presskopp, @renzms. See [Issue #780](#780).
- **Comment Mail Compatibility:** Improved compatibility with the Comment Mail plugin by automatically clearing the cache whenever Comment Mail options are changed. Many of the Comment Mail options affect front-end portions of the site, so it's important that the cache is cleared whenever Comment Mail options change. See [Comment Mail Issue #278](wpsharks/comment-mail#278 (comment)).
- **PHP Compatibility:** Improved compatibility back to PHP 5.2 (the lowest version allowed by WordPress). Comet Cache still requires PHP 5.4+, but if you install Comet Cache on a site running PHP 5.2, it will now fail gracefully with a Dashboard notice indicating PHP 5.4+ is required, instead of producing a fatal error. See [Issue #784](#784).
- **WP-CLI Compatibility**: Fixed a bug with deactivating Comet Cache using WP-CLI. Doing so was producing a "Invalid argument; host token empty!" error message. This has been resolved. Props @MarioKnight @jaswsinc @renzms. See [Issue #728](#728).
- Renamed `COMET_CACHE_ALLOW_BROWSER_CACHE` constant to `COMET_CACHE_ALLOW_CLIENT_SIDE_CACHE`. Backwards compatibility has been maintained.
- Renamed `allow_browser_cache` plugin option to `allow_client_side_cache`.
@raamdev
Copy link
Contributor

raamdev commented Jul 7, 2016

Comet Cache v160706 has been released and includes changes from this GitHub Issue. See the v160706 announcement for further details.


This issue will now be locked to further updates. If you have something to add related to this GitHub Issue, please open a new GitHub Issue and reference this one (#416).

@wpsharks wpsharks locked and limited conversation to collaborators Jul 7, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants