Assets not loaded after upgrading from 4.0 to 4.1

[joeforjoomla]

Steps to reproduce the issue

1. Install 4.0.5

2. Upgrade to 4.1 package

3. Update the database:
   

Expected result

All is working

Actual result

CSS and assets are lost no longer loaded. It seems that they can't be found in the new 'media' folder but only under 'classic' templates folder.

[brianteeman]

confirmed and why I raised #36423

[dgrammatiko]

I have no clue how you tried to upgrade to 4.1 but the 2 methods that I used worked flawlessly:

• providing the 4.1 update zip file
• using a custom update channel:
  

[brianteeman]

1. Thats not the beta3
2. did you try database structure fix afterwards

[dgrammatiko]

1. Yes, it the nightly, which is newer than the beta 3
2. No because there wasn't any reason to do something manually. (BTW the db structure will ONLY fix the structure, will not apply the table content transforms as they are not structure)
   

[joeforjoomla]

@dgrammatiko

1. There are no updates for custom update channel
2. To upgrade, i've used the manual zip file and AFTERWARDS updated the database. And assets are not found in the media folder, but STILL searched under the templates folder
   

[dgrammatiko]

The static files exist in the update package:

But I think both of you are on Windows and the finilisation part of the update is failing silently thus you have the db structure warnings. I think I know what's failing here but I need to coordinate the fix with @richard67 (remember that we skipped/patched the Path::clean or something similar in the script.php?).

[richard67]

To upgrade, i've used the manual zip file and AFTERWARDS updated the database

@joeforjoomla Does that mean you just have uploaded and unpacked the zip file e.g. via FTP or SSH access and then run the database fixer? Or does that mean you have used the "Upload & Update" method of the Joomla Update Component?

The first way, upload manually and then use the DB fixer, is not a valid update method anymore since Joomla 3.5. You HAVE TO use the update component.

[richard67]

@dgrammatiko What we have done in script.php was only for the copying of any custom files in the template folder. It should not cause such a problem with missing assets because the stuff in script.php runs AFTER the zip package has been unpacked and AFTER the old obsolete core files have been deleted and BEFORE the old obsolete core folders are deleted..

[dgrammatiko]

It should not cause such a problem with missing assets because the stuff in script.php runs AFTER the zip package has been unpacked and AFTER the old obsolete core files have been deleted and BEFORE the old obsolete core folders are deleted..

Nope, the file part comes before the db changes, so if it fails the script won't get the db updates. The error here is obvious as the templates should have their styles table updated (column inheritable needs to be 1 not 0). This is obviously what's missing here, either @brianteeman or @joeforjoomla could verify it.

[brianteeman]

I have already confirmed in another issue that if an SQL silently fails then the remove files doesn't happen

[brianteeman]

#36485 #36482 #36492

but hey what do I know

[joeforjoomla]

@richard67 @dgrammatiko installing the package with 'Upload and update' does not work.

Afterwards all assets are no longer loaded:

I have to manually COPY BACK assets files from the /media folder to the templates folder, both backend and frontend

[richard67]

@joeforjoomla I see. Thanks for clarifying.

[dgrammatiko]

I will ask again: is this a windows based server?

[brianteeman]

yes but it doeznt matter. If the sql fails (silently) then the script.php step is missed ou and it goes straight to telling you the update was a success

[dgrammatiko]

If the sql fails (silently) then the script.php step is missed ou and it goes straight to telling you the update was a success

It's not the db part but some filesystem related operation that fails here. The comments following #35874 (comment) should give us a hint of what's failing. The bad news here is that I cannot patch this blindly (I don't have a windows based PC around) so someone else needs to debug it.

[joeforjoomla]

@dgrammatiko it's a standard Apache server on a Windows PC

[dgrammatiko]

@joeforjoomla @brianteeman could you test upgrading to 4.1 using the package from #36574

[joeforjoomla]

@dgrammatiko just tested #36574 but still same issue.

[richard67]

@joeforjoomla If you hover over the "2 Problems" badge, which details are shown in the tool tip?

[joeforjoomla]

@richard67 at the moment database problems are no longer displayed, the schema structure has been updated.

[dgrammatiko]

@joeforjoomla do you

• have the media/templates/site/... folders
• have the js, css, images folders in the templates/cassiopeia and administrator/templates/atum

[brianteeman]

testing your update link on linux.
update successful but reports database schema problems

clicking on update structure resolves the above but obv should not be necessary.

now to test on a 100% clean windows

[richard67]

@brianteeman The one problem about update version not matching the CMS version can be ignored. You always get that when using (i.e, installing a or updating to a) patched package built by drone for a PR.

[brianteeman]

@richard67 ok - was just reporting for completeness

[dgrammatiko]

@brianteeman @joeforjoomla could you test #36574 once the Drone is done?

The change there now will not run the code for the templates files sync so if it works then it's an obvious indication that the code that syncs the assets is failing. Thanks

[brianteeman]

I can confirm that I can update on windows with a completely clean install using BOTH nightly and the build packages from the PR

I can also tell you how to replicate the screenshot in #36567 (comment).

To Replicate you need to force an sql error in the update which is easy to happen when you've been doing a lot of testing but if you need to force it then just change the template_id of any of the mail_templates in the database to plg_system_tasknotification.failure_mail

Now when you do the joomla update it will silently fail on the sql and you will end up with the same screen as @joeforjoomla

[dgrammatiko]

@brianteeman the db failure might as well be a valid reason why the update failed but in this case the problem comes from my code. Here's the order of execution:

So the break should be in the moveRemainingTemplateFiles()

[brianteeman]

Which is exactly what i said before in my first issue

[joeforjoomla]

@joeforjoomla do you

• have the media/templates/site/... folders
• have the js, css, images folders in the templates/cassiopeia and administrator/templates/atum

@dgrammatiko after the upgrade to 4.1 i have * have the media/templates/site/... folders but i've lost js, css, images folders in the templates/cassiopeia and administrator/templates/atum

[dgrammatiko]

One solution is to wrap every db update part in script.php in a try/catch so even if something throws the rest of the code will run. Actually everything there should be wrapped in a try/catch

[richard67]

One solution is to wrap every db update part in script.php in a try/catch so even if something throws the rest of the code will run. Actually everything there should be wrapped in a try/catch

We should not do that in a way so that if one SQL statement fails then the next one runs. This would cause a mess. We have to end SQL scripts execution if an error happens with an SQL statement.

[dgrammatiko]

There's an obvious easy test here:

• Download the update zip file, extract it and remove the file administrator/components/com_admin/sql/updates/mysql/4.1.0-2021-11-20.sql and then re zip it
• If the update works the problem is in the Task Scheduler if not it's on the child templates SQL file (the next one in that folder)

[joeforjoomla]

There's an obvious easy test here:

• Download the update zip file, extract it and remove the file administrator/components/com_admin/sql/updates/mysql/4.1.0-2021-11-20.sql and then re zip it
• If the update works the problem is in the Task Scheduler if not it's on the child templates SQL file (the next one in that folder)

@dgrammatiko i've just done this test but nothing changes. Still the same issue.

[dgrammatiko]

Out of curiosity @brianteeman @joeforjoomla what's your MySQL version ?

[brianteeman]

8.0.19 MySQL Community Server - GPL

[joeforjoomla]

My version is MySQL 5.7

[richard67]

To me the screenshot in the comment above doesn't look as if there was an SQL error. It shows the finalization step, and that would not be the case when an SQL error happened.

To be sure: Which SQL statement is the last one logged in administrator/logs/joomla_update.php?

[richard67]

@joeforjoomla Just to be sure: Do you use an .htaccess file in the Joomla root? If so, does it contain more things than the htaccess.txt shipped with Joomla? If so, does it contain a line like the following?

RewriteRule ^administrator/components/com_joomlaupdate/restore\.php$ - [L]

And it does not contain a line like the following?

RewriteRule ^administrator\/components\/com_joomlaupdate\/extract\.php$ - [L]

If this is the case, then that's your problem, it cannot run the finalization step at all.

Check https://docs.joomla.org/Htaccess_changes_after_joomla4.0.4 and https://www.joomla.org/announcements/release-news/5850-changes-to-update-process-that-you-need-to-be-aware-of.html for the details.

It's just an idea, maybe I'm completely wrong here.

[richard67]

@joeforjoomla Could you run the following SQL statement (replace #__ with the right prefix) and report back if the assets are loaded right after that so the backend looks better?

UPDATE `#__template_styles` SET `inheritable` = 1 WHERE `template` = 'atum' AND `client_id` = 1 OR `template` = 'cassiopeia' AND `client_id` = 0;

This statement should have been run during the update, but it seems that has not happened for some reason.

To find out the reason I still need to know which was the last SQL statement reported in the update log of the failed update attempt.

[dgrammatiko]

Potential fix: #36585
Please test

[joeforjoomla]

@joeforjoomla Just to be sure: Do you use an .htaccess file in the Joomla root? If so, does it contain more things than the htaccess.txt shipped with Joomla? If so, does it contain a line like the following?

RewriteRule ^administrator/components/com_joomlaupdate/restore\.php$ - [L]

And it does not contain a line like the following?

RewriteRule ^administrator\/components\/com_joomlaupdate\/extract\.php$ - [L]

If this is the case, then that's your problem, it cannot run the finalization step at all.

Check https://docs.joomla.org/Htaccess_changes_after_joomla4.0.4 and https://www.joomla.org/announcements/release-news/5850-changes-to-update-process-that-you-need-to-be-aware-of.html for the details.

It's just an idea, maybe I'm completely wrong here.

@richard67 not really the htaccess is not enabled at all

[joeforjoomla]

Potential fix: #36585 Please test

@dgrammatiko @richard67 Yes that was the problem. Confirmed.

[dgrammatiko]

@joeforjoomla please mark your test

[richard67]

Solved with #36585 .