Comet Cache intervening with WordPress trailing slashes in Permalinks

[raamdev]

We're investigating a report of ZenCache preventing a WordPress site from redirecting example.com/test to example.com/test/ (with the slash).

I'm having varying success reproducing this issue. I can reproduce it on my live test site, but I cannot reproduce it on my local test site. At first I thought this had only to do with the .htaccess rewrite rules, but then I remembered seeing code in WordPress Core somewhere that looks at the permalink settings and redirects to the /-version if the trailing slash exists in the Permalink settings.

@jaswsinc writes...

Right. There is the user_trailingslashit() function that handles this in permalink generation. https://codex.wordpress.org/Function_Reference/user_trailingslashit
See also: https://github.com/WordPress/WordPress/blob/ce557062f4123d8513378cf415b4e8b612c33ccc/wp-includes/canonical.php#L41
See also: http://httpd.apache.org/docs/current/mod/mod_dir.html#directoryslash

That does sound like it could be a bug. I will run tests against this later and see what I can do.

---

Support tickets referencing this issue

• https://websharks.zendesk.com/agent/tickets/7861

[jaswrks]

My Thoughts

There's no reason for ZenCache to differentiate a trailing slash from no trailing slash, in terms of how we generate cache paths. It's really the exact same location. The only difference is in preference; i.e., what a site owner prefers on the end of their permalinks.

I do agree it's a bug though, because whatever you configure in your WordPress Permalink structure should be adhered to. If you configure your Permalink structure with a trailing /, WordPress generates all permalinks that way. And, by default, the redirect_canonical() function in WordPress enforces that behavior.

The issue here is that redirect_canonical() is bypassed whenever caching is enabled; i.e., whenever ZenCache is running. The cache file is detected early in the request, then served, long before redirect_canonical() gets a chance to run. No way around this either. In fact, that's intentional. So to resolve this we need to get creative.

---

Suggested Next Actions

I would say this issue is blocked by the fact that we don't yet have a system in place that deals with .htaccess tweaks. Once work on this outline I posted a few days ago is underway, I think that a solution will present itself here.

What I suggest is that we add a couple of lines into the /.htaccess file whenever ZenCache is enabled that take care of this issue; based on the configured permalink structure. WordPress itself relies upon redirect_canonical(), but that's not feasible w/ caching enabled. Thus, we need to deal with this at the Apache level via .htaccess in my view.

[raamdev]

I would say this issue is blocked by the fact that we don't yet have a system in place that deals with .htaccess tweaks. Once work on this outline I posted a few days ago is underway, I think that a solution will present itself here.

Got it. Thanks for the update!

Blocked by #427

[raamdev]

@jaswsinc I'm updating this GitHub issue to note that it's no longer blocked by #427. Do you have any examples for what .htaccess rule(s) will need to be added to fix this issue?

Also, I'm a bit worried that such rules would have such a wide-reaching effect on a site that it might be cause for trouble. For example, if a site owner has their own custom rewrite rules (in addition to those created by WordPress), the rules we add to fix this GitHub issue might create an unexpected conflict. Thoughts on this?

[jaswrks]

Force a trailing slash:

# BEGIN Force Trailing Slash
<IfModule rewrite_module>
  RewriteEngine On
  RewriteBase /

  # If not a real file or directory.
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d

  # Not a part of the WP admin area.
  RewriteCond %{REQUEST_URI} !(?:^|/)wp\-admin(?:/|$)

  # If there is no trailing slash.
  RewriteCond %{REQUEST_URI} !(?:.*)/$

  # Force a trailing slash on all virtual requests.
  RewriteRule ^(.*)$ /$1/ [QSA,L,R=301]
</IfModule>
# END Force Trailing Slash

Force NO trailing slash:

# BEGIN Force NO Trailing Slash
<IfModule rewrite_module>
  RewriteEngine On
  RewriteBase /

  # If not a real file or directory.
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d

  # Not a part of the WP admin area.
  RewriteCond %{REQUEST_URI} !(?:^|/)wp\-admin(?:/|$)

  # If there is a trailing slash.
  RewriteCond %{REQUEST_URI} (?:.*)/$

  # Force NO trailing slash on all virtual requests.
  RewriteRule ^(.*)/$ /$1 [QSA,L,R=301]
</IfModule>
# END Force NO Trailing Slash

---

Also, I'm a bit worried that such rules would have such a wide-reaching effect on a site that it might be cause for trouble. For example, if a site owner has their own custom rewrite rules (in addition to those created by WordPress), the rules we add to fix this GitHub issue might create an unexpected conflict. Thoughts on this?

I think if we detect permalink settings that either require or do not require a trailing slash and add the appropriate snippet to the very top of the .htaccess file there is very little chance of there being an issue. Really, one of these snippets just needs to come somewhere before the WordPress section.

However, since Permalink settings in WordPress can be changed, and because of the unknowns that you mentioned, I would suggest that this is not injected automatically, but presented as an option for those who would like to implement it. Upon enabling the option, only then do we automatically inject it.

[raamdev]

I would suggest that this is not injected automatically, but presented as an option for those who would like to implement it. Upon enabling the option, only then do we automatically inject it.

Great idea! Where do you think such an option should go? I'm thinking maybe a new .htaccess Tweaks options panel. We could then use that new panel in the future for any other .htaccess tweaks like this that we might want to make easy for a site owner to add/remove from their .htaccess file.

[jaswrks]

Where do you think such an option should go? I'm thinking maybe a new .htaccess Tweaks

After having worked with some additional tweaks as a part of the work in this PR, I think a good title for this would be, "Apache .htaccess Tuner". Like you said, we can nest all of the tweaks that we are going to introduce in this panel. The one mentioned in this issue, and the others that I'm working on in that PR too.

[raamdev]

"Apache .htaccess Tuner"

That means we'd probably want to get rid of the GZIP Compression menu option and move that into the new Apache Optimization (?) panel, correct? That probably makes sense considering the whole GZIP Compression panel won't be applicable to web servers not running Apache. (Sure, we could give advice for enabling GZIP Compression on other web servers inside the GZIP Compression options panel, but I feel like that's not an organized way to approach this now that we're adding several more Apache-related optimizations.)

If we go the route of condensing Apache-related stuff into a single Apache Optimization panel, then we could eventually have other panels for other web servers, e.g., NGINX Optimization, where we give advice for optimizing other web servers (assuming ZenCache cannot optimize them itself). Thoughts?

[jaswrks]

Apache Optimization

Sounds perfect. I wanted to go with "Apache Performance Tuning", but that seems a little misleading. Apache Optimization is better.

If we go the route of condensing Apache-related stuff into a single Apache Optimization panel, then we could eventually have other panels for other web servers, e.g., NGINX Optimization, where we give advice for optimizing other web servers (assuming ZenCache cannot optimize them itself). Thoughts?

I think that's a great idea!

[raamdev]

I think that's a great idea!

Cool! If you don't want to work the consolidation and creation of a new Apache Optimization panel into wpsharks/comet-cache-pro#165, let me know when you're ready for me to dive in and I can take care of that.

[jaswrks]

Go for it! I still have some work to do in other areas before I get to the UI, so if you'd like to work on that panel then I can merge that into feature/sonic-boom when I'm ready to start testing. That would make this go faster I think.

[raamdev]

then I can merge that into feature/sonic-boom when I'm ready to start testing. That would make this go faster I think.

Roger that.

[raamdev]

Punting this to a Future Release. We need to build an Apache Optimizations panel so that we have a place to put the option required to close this issue. The Apache Optimizations panel itself depends on work discussed in https://github.com/websharks/team/issues/36.

[raamdev]

Noting that this feature should be disabled by default. We should not be updating the .htaccess unless the site owner tells Comet Cache it should be doing so.

Apache Optimization should be a Pro-only feature, yes.

Otherwise the outline looks good to me.

[raamdev]

@raamdev writes in #554 (comment)...

That means we'd probably want to get rid of the GZIP Compression menu option and move that into the new Apache Optimization (?) panel, correct? That probably makes sense considering the whole GZIP Compression panel won't be applicable to web servers not running Apache. (Sure, we could give advice for enabling GZIP Compression on other web servers inside the GZIP Compression options panel, but I feel like that's not an organized way to approach this now that we're adding several more Apache-related optimizations.)

@renzms As I mentioned on Slack, let's go ahead and move GZIP Compression into the Apache Optimizations panel. Also, we'll make the Apache Optimizations panel available in the Lite version (so that Lite users still gain access to GZIP Compression), but all options, except for the GZIP Compression, should be a Pro-only features.

With the Apache Optimizations panel available in the Lite version, we could add an extra Pro Preview teaser right inside that panel, so that site owners using the Lite version and looking at the Apache Optimizations panel see that there are additional optimizations available in the Pro version. (We'll save that work for a later date and a separate GitHub issue—I just wanted to make note of it for future reference.)

[raamdev]

Next Release Changelog:

• Bug Fix (Pro): Fixed a bug where Comet Cache would appear to prevent WordPress from redirecting Permalinks that don't include a trailing slash, to the URL that does include a trailing slash. This was due to the fact that Comet Cache loads very early on (for caching purposes) and as a result the WordPress redirect_canonical() function never gets run. This was fixed by adding an option to the new Apache Optimizations panel that allows you to Enforce Canonical URLs. Props @renzms, @jaswsinc. See Issue #554.
• New Pro Feature! A new "Enforce Canonical URLs" option has been added to the new Apache Optimizations panel. This options adds the appropriate .htaccess code to enforce the correct canonical URLs according to your WordPress Permalink settings (Comet Cache detects if the Permalink Structure ends with a trailing slash, or without a trailing slash). Props @renzms, @jaswsinc. See Issue #554.

[raamdev]

Comet Cache v160706 has been released and includes changes from this GitHub Issue. See the v160706 announcement for further details.

---

This issue will now be locked to further updates. If you have something to add related to this GitHub Issue, please open a new GitHub Issue and reference this one (#554).