[4.0] Converting several mails to mail templates #28722

Hackwar · 2020-04-18T12:58:45Z

This converts several of the core mails to the mail template system.

List of mails send in Joomla

If someone else wants to take over the remaining ones, be my guest.

Testinstructions

Basically, you have to trigger sending the mails in the cases marked as done above and see if they still arrive identically to the way they did before. Then create a template and see if your modifications are reflected.

Comments

I had to extend the MailTemplate class with a few modifications and I guess this wont be the last time this is necessary.

…4mails

toivo · 2020-04-19T14:11:38Z

Results from selected initial tests to see if the marked emails work as before:
Mail: "Testmail in global configuration"
Result: works as before

Mail: "Mail to the new user when creating a new user in the backend”
Result: failure
"Warning
Empty message body
Email could not be sent"

Mail: "Password reset for users in frontend"
Result: failure
"Warning
Empty message body
Notice
Reminder failed: Failed sending email."

My test environment is Windows 10, Wampserver 3.20, MySQL 5.7.23 and PHP 7.4.2.

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

toivo · 2020-04-19T14:38:54Z

Changed the Mail Templates – Options: Per Template Mail Settings No --> Yes
The result from requesting username reminder and password reset emails is error "0 Call to a member function get() on null"

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

toivo · 2020-04-19T16:05:34Z

params of com_mails: {"mail_style":"plaintext","alternative_mailconfig":"1","copy_mails":"0","attachment_folder":""}
Call stack

Function Location

1 () JROOT\libraries\src\Mail\MailTemplate.php:198
2 Joomla\CMS\Mail\MailTemplate->send() JROOT\components\com_users\src\Model\RemindModel.php:191
3 Joomla\Component\Users\Site\Model\RemindModel->processRemindRequest() JROOT\components\com_users\src\Controller\RemindController.php:42
4 Joomla\Component\Users\Site\Controller\RemindController->remind() JROOT\libraries\src\MVC\Controller\BaseController.php:729
5 Joomla\CMS\MVC\Controller\BaseController->execute() JROOT\libraries\src\Dispatcher\ComponentDispatcher.php:146
6 Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch() JROOT\libraries\src\Component\ComponentHelper.php:384
7 Joomla\CMS\Component\ComponentHelper::renderComponent() JROOT\libraries\src\Application\SiteApplication.php:196
8 Joomla\CMS\Application\SiteApplication->dispatch() JROOT\libraries\src\Application\SiteApplication.php:235
9 Joomla\CMS\Application\SiteApplication->doExecute() JROOT\libraries\src\Application\CMSApplication.php:231
10 Joomla\CMS\Application\CMSApplication->execute() JROOT\includes\app.php:63
11 require_once() JROOT\index.php:36

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

Hackwar · 2020-04-20T19:06:02Z

Have you applied the database changes as well?

toivo · 2020-04-20T23:35:16Z

Which database changes? I tested #28722 with J4 Patch Tester RC2 and the latest J4 Nightly Build. Yesterday I noticed that the latest Nightly Build was in fact from April 14, which may explain the issue. Asked about it in #27666 and richard67 was going to inform the CMS maintenance team. April 14 is still the latest Nightly Build.

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

Quy · 2020-04-20T23:39:26Z

You can also try the download package in this PR.
Scroll to the bottom.
Click on Show all checks.
In the Download row, click Details.

https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/28722/downloads/31378/

toivo · 2020-04-21T01:43:04Z

@Quy Sorry, I tried but your instructions are not detailed enough. What am I supposed to download from https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/28722/downloads/31378/ ? Links to both .zip files are broken.

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

richard67 · 2020-04-21T07:03:38Z

@toivo When testing a Pull Request (PR) it is always good to look first which files it changes. You can see that for this PR here on GitHub: https://github.com/joomla/joomla-cms/pull/28722/files. If a PR adds a new update sql script or changes an existing one, it means the PR also contains database changes. In case of this PR here these are the files 4.0.0-2020-04-16.sql added by this PR. To apply such database changes you can use PhpMyAdmin or whatever and run the SQL from the file for your database type (mysql I guess) in that tool. You have to replace the "#__" with your actual database prefix in table names before executing the SQL. You should know that there is no way tro roll back these database changes, i.e. you should do such tests only on a testing environment which you later can throw away or reset to a standard installation, never do that on a real live site. The Patchtester can't apply and revert database changes.

I hope these explanations help you a bit to understand the previous comment about database changes of this PR.

toivo · 2020-04-21T07:51:47Z

@richard67, that was helpful, thanks. I will apply the two .sql files against the database created by the Nightly Build from April 14 and run the J4 Patch Tester RC2. - Any news when the Nightly Builds might continue with new versions?

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

richard67 · 2020-04-21T07:56:51Z

@toivo They are still working on it as far as I know. Regarding SQL, don't forget to replace the "#__" with the table prefix (including the onderscore at the end) before running the statements.

toivo · 2020-04-21T09:51:49Z

@Richar67 Just installed the latest Nightly Build from April 14, executed com_admin/sql/updates/mysql/4.0.0-2020-04-16.sql and applied PR #28722 using J4 Patch Tester RC2, then found that the language files are out of date because com_mails is displaying raw language strings, for example com_contact_MAIL_mail_TITLE. I would like to test this PR but where are the updates to the language files?

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

richard67 · 2020-04-21T10:07:06Z

@toivo I can't see any language sting with name like com_contact_MAIL_mail_TITLE (case insensitive) is being used. So either there is something wrong with your testing environment, or the name of that language string is different.

richard67 · 2020-04-21T10:19:09Z

@Hackwar It seems there is something wrong with the update sql script. First of all there is a warning or error (depending on mysql strict mode settings) telling that columns htmlbody and attachments don't have a default value. I think you should include these columns into your insert statement and insert an emptry string for them. This should be done for both the mysql and the postgresql file, like it is already done in the supports.sql for the corresponding database type.

In addition, @toivo is right: There are strange language sting names instead of real texts shown in the mail templates list view after having run the update SQL. Here just an example with 2 of them, but it's true for all the new records.

toivo · 2020-04-26T11:41:17Z

Localhost: Windows 10, Wampserver 3.2.1, Apache 2.4.1, MySQL 8.0.19, PHP 7.4.5
J4 version: Nightly Build of April 26 - https://developer.joomla.org/nightlies/Joomla_4.0.0-beta1-dev-Development-Full_Package.zip

Part 1 "Test if emails still arrive identically to the way they did before"
• Testmail in global configuration - OK
• Mail to the new user when creating a new user in the backend - OK
• Password reset for users in frontend - OK
• Username reminder for users in frontend - OK
• Contact form in com_contact - OK

• not tested: com_contact API controller to send a mail
• not tested: Joomla Updatenotification

• Massmailer mail - error - details below:

An error has occurred.
0 Too few arguments to function Joomla\CMS\Mail\MailTemplate::__construct(), 1 passed in C:\www\joomla4\administrator\components\com_users\src\Model\MailModel.php on line 179 and at least 2 expected
Call stack

Function Location

1 () JROOT\libraries\src\Mail\MailTemplate.php:90
2 Joomla\CMS\Mail\MailTemplate->__construct() JROOT\administrator\components\com_users\src\Model\MailModel.php:179
3 Joomla\Component\Users\Administrator\Model\MailModel->send() JROOT\administrator\components\com_users\src\Controller\MailController.php:44
4 Joomla\Component\Users\Administrator\Controller\MailController->send() JROOT\libraries\src\MVC\Controller\BaseController.php:729
5 Joomla\CMS\MVC\Controller\BaseController->execute() JROOT\libraries\src\Dispatcher\ComponentDispatcher.php:146
6 Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch() JROOT\libraries\src\Component\ComponentHelper.php:384
7 Joomla\CMS\Component\ComponentHelper::renderComponent() JROOT\libraries\src\Application\AdministratorApplication.php:131
8 Joomla\CMS\Application\AdministratorApplication->dispatch() JROOT\libraries\src\Application\AdministratorApplication.php:174
9 Joomla\CMS\Application\AdministratorApplication->doExecute() JROOT\libraries\src\Application\CMSApplication.php:231
10 Joomla\CMS\Application\CMSApplication->execute() JROOT\administrator\includes\app.php:63
11 require_once() JROOT\administrator\index.php:36

MySQL reported two warnings when the SQL update file 4.0.0-2020-04-16.sql was executed:
1364 Field 'htmlbody' doesn't have a default value
1364 Field 'attachments' doesn't have a default value

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/28722.}

richard67 · 2020-04-26T11:43:54Z

MySQL reported two warnings when the SQL update file 4.0.0-2020-04-16.sql was executed:
1364 Field 'htmlbody' doesn't have a default value
1364 Field 'attachments' doesn't have a default value

I've reported this already with my comment above.

richard67 · 2020-04-26T12:44:44Z

@toivo Thanks for reporting back. Please wait with further tests until the necessary corrections have been made in this Pull Request (PR). I'll give you a notice when the PR can be tested again.

richard67 · 2020-04-26T12:57:36Z

@Hackwar For the SQL error mentioned in the comments above for updating see Hackwar#40.

richard67 · 2020-05-25T10:02:20Z

Beside the HTML mode scheck box of the Mass Mail form being ignored, there are other, already known issues with HTML mode and mail templates, see #28524 and #27619 .

@Hackwar But the general question remains what to do with the check box options in the Mass Mail form. Maybe they should be removed from this form if these options have to be configured in the mail template?

richard67 · 2020-05-25T10:27:37Z

@toivo Regarding the update notification email I suggest you do not use any hack. Use a clean 4.0-dev or latest J4 nightly build. Then in the Joomla Update Component's option, change update channel to "Custom URL", set minimum stability to "Development" and enter following custom URL: https://test5.richard-fath.de/j4-nightlies-update-url-patched-for-test.xml, then save.

This should cause an update to 4.0.0-beta2-dev be found (which is fake because it just links to the nightly of tonight). The go to the Extension Installer's options and set the cache time to zero and minimum stability to "Development" and save. Then you should get some email, if everything else is set up right.

Please test if this works for you and report back the result.

richard67 · 2020-05-25T10:51:49Z

@toivo Maybe the internal server error in my case comes from my mail using smtp, but in the API's ContactController.php there seems the PHP mailer to be used in any case. So that could be another issue not related to this PR. Update: Forget about it. I see with debug on the same as you reported, the RuntimeException: Display email form disabled. in \/joomla-cms-4.0-dev\/api\/components\/com_contact\/src\/Controller\/ContactController.php:125. It seems to come from here:

joomla-cms/api/components/com_contact/src/Controller/ContactController.php

Lines 123 to 126 in 944ffd5

    
           if (!$contactParams->get('show_email_form')) 
        
           { 
        
           	throw new \RuntimeException(Text::_('JLIB_APPLICATION_ERROR_DISPLAY_EMAIL_FORM')); 
        
           }

This part was not touched by this PR, so I think it's really unrelated.

toivo · 2020-05-25T11:17:43Z

@richard67 The configuration of the workaround to get the Updatenotification email without any tweaks worked all right, but my Super User received four (4) identical emails and they keep coming because of the cache time. I can now apply the patch and continue testing, unless some changes are going to be made overnight.

richard67 · 2020-05-25T11:18:54Z

@toivo I found the problem: I have to explicitely switch the "Show email form" to "Yes" for the particular contact, then it works.

The reason is that the code I've linked in my previous comment checks only the options of the particular contact, and only if Yes, not if any inherited setting.

That's definitely an issue not related to this PR.

But when I got it work, the email contained the language sting constants and nothing else. I have to check if this happens also without this PR.

toivo · 2020-05-25T11:22:10Z

@richard67 Ok, setting the option 'Contact Form' to 'Show' makes sense. The option 'Session Check' is probably not going to be used in the API.

richard67 · 2020-05-25T11:25:05Z

@toivo When I send the email now with the API to the contact, I get the language constants in the email when having no template for en-GB or when having a template with all buttons to "No" so it uses the language constants. Only when switching the buttons to "Yes" I get the translated text. This could be related to this PR. I will continue to investigate and provide a patch for @Hackwar like before if I find the reason and a valid fix.

Update: If I add the language strings COM_CONTACT_COPYSUBJECT_OF, COM_CONTACT_COPYTEXT_OF, COM_CONTACT_ENQUIRY_SUBJECT and COM_CONTACT_ENQUIRY_TEXT to file api/language/en-GB/joomla.ini, the emails contain the translated texts. But I don't think that's the way it should be solved, or is it? How is the api supposed to load language files? The one it should load here is language/en-GB/com_contact.ini, but it isn't loaded so it needs that hack to add the strings to api/language/en-GB/joomla.ini.

richard67 · 2020-05-25T12:16:36Z

Update: If I add a Factory::getLanguage()->load('com_contact', JPATH_SITE); to function _sendEmail of file api/components/com_contact/src/Controller/ContactController.php, the strings get translated in any case. @Hackwar What would be the roight place for that? Just at the top, or just after $app = $this->app;, or elsewhere?

wilsonge · 2020-05-25T13:15:22Z

_sendEmail of file api/components/com_contact/src/Controller/ContactController.php

Sounds an ok fix to me. Honestly moving that sendEmail part to a helper function shared by API and site rather than having duplicates in the controller of api and site might be best to avoid the duplication (and that would also have to load the lang file)

richard67 · 2020-05-25T13:18:44Z

Honestly moving that sendEmail part to a helper function shared by API and site rather than having duplicates in the controller of api and site might be best to avoid the duplication (and that would also have to load the lang file)

Agree ... but there is anyway potential regarding future PRs for improvement of the Mail Templates component, e.g. regarding HTML mode. Most of the issues are already covered by issues #28524 and #27619 .

But this PR should be ok now, I think. Will do a complete test again later.

P.S. And also potential of API improvements and simplifications, of course.

wilsonge · 2020-05-25T18:35:22Z

Merging this to get it through in time for beta this weekend. @richard67 if you still have time to report the results of your tests would be good

richard67 · 2020-05-25T18:57:27Z

@wilsonge @Hackwar Test were all ok with text emails. I've tested for each template without a template created for en-GB, then with template but default using language strings, and finally with own content. HTML templates don't work yet, as stated in issue #28524 , and so the "HTML mode" of the Mass Mail doesn't work, it always sends text mails (content type "text/plain"). Another issue #27619 also discusses the HTML templates.

richard67 · 2020-05-25T18:58:21Z

@wilsonge @Hackwar So it needs a decision about the check box for HTML mode in the Mass Mail form: Make it work or remove it from UI.

richard67 · 2020-05-25T19:13:21Z

@wilsonge @Hackwar I've created a new issue so the remaining mails to be done will not be forgotten: #29215 .

infograf768 · 2020-05-26T08:21:59Z

Hmm
+COM_USERS_MAIL_REMINDER_DESC="This mail is sent to a user when by the \"Forgot your username?\" link e.g. in a login form."
when by ?

richard67 · 2020-05-26T08:36:39Z

@infograf768 Clearly a mistake, should be "by" only, without the "when". Could you make a PR?

infograf768 · 2020-05-26T09:00:26Z

will do as well as another issue concerning com_mails

richard67 · 2020-05-26T09:02:27Z

Which other issue? If it is the mass mail html mode not working: I wanted to open an issue for it later.

infograf768 · 2020-05-26T09:12:41Z

see #29224

richard67 · 2020-05-26T11:46:00Z

For issue #28524 with HTML body see my new PR #29229 .

Hackwar added 7 commits April 16, 2020 23:21

Converting mail from updatenotification to mail template

9cea6cd

Converting contact form to mail templates

4c66170

Converting com_contact API controller to Mailtemplates

2038846

Converting Massmailer mail to mail templates

c5d168b

Converting new user mail from plg_user_joomla to mail templates

84c9463

Converting password reset mail to mail templates

7ad2a42

Converting com_users remind mail to mail templates

f4df2b1

joomla-cms-bot added Language Change This is for Translators PR-4.0-dev labels Apr 18, 2020

Hackwar added 2 commits April 18, 2020 16:27

Codestyle

7d56d8f

Merge branch '4.0-dev' of https://github.com/joomla/joomla-cms into j…

d10fe21

…4mails

Fix update sql for new mail templates

db33d18

richard67 mentioned this pull request Apr 26, 2020

[Joomla CMS PR #28722] Fix update sql for new mail templates Hackwar/joomla-cms#40

Merged

Load client language file for com_contact in API submitt contact form

1110ca1

richard67 mentioned this pull request May 25, 2020

[4.0] [PR 28722] Load client language file for com_contact in API submitt contact form Hackwar/joomla-cms#45

Merged

Merge pull request #45 from richard67/4.0-dev-Hackwar-j4mails-mod-3

8cf2608

wilsonge merged commit e47d3ec into joomla:4.0-dev May 25, 2020

wilsonge added this to the Joomla 4.0 milestone May 25, 2020

richard67 mentioned this pull request May 25, 2020

[4.0] Remaining mails to be converted to mail templates #29215

Closed

15 tasks

jgerman-bot mentioned this pull request May 25, 2020

[4.0] Converting several mails to mail templates joomlagerman/joomla#1079

Closed

infograf768 mentioned this pull request May 26, 2020

[4.0] com_mails: string typo & template fields labels display missing #29224

Merged

Hackwar deleted the j4mails branch May 26, 2020 09:55

sakiss pushed a commit to sakiss/joomla-cms that referenced this pull request Oct 16, 2020

[4.0] Converting several mails to mail templates (joomla#28722)

495f804

[4.0] Converting several mails to mail templates #28722

[4.0] Converting several mails to mail templates #28722

Conversation

Hackwar commented Apr 18, 2020 • edited Loading

List of mails send in Joomla

Testinstructions

Comments

toivo commented Apr 19, 2020

toivo commented Apr 19, 2020

toivo commented Apr 19, 2020

Function Location

Hackwar commented Apr 20, 2020

toivo commented Apr 20, 2020

Quy commented Apr 20, 2020

toivo commented Apr 21, 2020

richard67 commented Apr 21, 2020

toivo commented Apr 21, 2020

richard67 commented Apr 21, 2020

toivo commented Apr 21, 2020

richard67 commented Apr 21, 2020

richard67 commented Apr 21, 2020 • edited Loading

toivo commented Apr 26, 2020

Function Location

richard67 commented Apr 26, 2020 • edited Loading

richard67 commented Apr 26, 2020

richard67 commented Apr 26, 2020

richard67 commented May 25, 2020

richard67 commented May 25, 2020 • edited Loading

richard67 commented May 25, 2020 • edited Loading

toivo commented May 25, 2020 • edited Loading

richard67 commented May 25, 2020

toivo commented May 25, 2020 • edited Loading

richard67 commented May 25, 2020 • edited Loading

richard67 commented May 25, 2020

wilsonge commented May 25, 2020

richard67 commented May 25, 2020 • edited Loading

wilsonge commented May 25, 2020

richard67 commented May 25, 2020

richard67 commented May 25, 2020 • edited Loading

richard67 commented May 25, 2020

infograf768 commented May 26, 2020

richard67 commented May 26, 2020

infograf768 commented May 26, 2020

richard67 commented May 26, 2020

infograf768 commented May 26, 2020

richard67 commented May 26, 2020

Hackwar commented Apr 18, 2020 •

edited

Loading

richard67 commented Apr 21, 2020 •

edited

Loading

richard67 commented Apr 26, 2020 •

edited

Loading

richard67 commented May 25, 2020 •

edited

Loading

richard67 commented May 25, 2020 •

edited

Loading

toivo commented May 25, 2020 •

edited

Loading

toivo commented May 25, 2020 •

edited

Loading

richard67 commented May 25, 2020 •

edited

Loading

richard67 commented May 25, 2020 •

edited

Loading

richard67 commented May 25, 2020 •

edited

Loading