Support for testing emails sent from Drupal

[jonathanjfshaw]

Following on from https://www.drupal.org/node/1935598, planning for adding the ability to test emails sent by Drupal.

jhedstrom commented there: "The new steps should go into a dedicated context so it can be enabled as needed. I think some changes will be necessary to the Drupal Drivers too."

Currently I'm aware of 5 fleshed out available code samples on this subject, which show different approaches:

• @Berdir kindly shared his approach with me (which I've put in a google doc here https://docs.google.com/document/d/13Mw4Bc0t967bdqlCkenluCVXLKGLIa4YzoRY... for easy reference). He uses core's test_mail_collector.
• @larowlan blogs in detail here https://www.previousnext.com.au/blog/advanced-testing-drupal-emails-beha... about his approach, also using test_mail_collector.
• @douggreen (correct link is https://github.com/tag1consulting/behat-drupalextension/blob/master/src/..., lines 530-686 and 1053-1074) uses imap to check an email is actually send and received.
• @jamessharv and @jurgenhaas created a subcontext for the maillog module in #1932698
• https://github.com/NuCivic/dkanextension src/Drupal/DKANExtension/Context/MailContext.php I have not reviewed
• some StackExchange discussion here: http://drupal.stackexchange.com/questions/187643/how-do-i-inspect-mail-messages-in-behat-tests (using test_mail_collector)

I want to suggest the following approach:

• The Drupal Extension's out-of-the-box email system would use test_mail_collector. It's core not contrib (unlike maillog), and requires no setup (unlike IMAP).
• By default, testing never sends live emails. The Drupal Extension should record the original mail_system on load, swap it out for the test_mail_collector, and restore the original at the end of testing.
• A tag @live-email could be added to any feature or scenario, which would temporarily swap in the original mail_system just for that scenario; this would facilitate IMAP type solutions or manual checking, for anyone wants that
• The test_mail_collector's email cache would be flushed a) at the beginning of every feature or scenario tagged @email, and b) by the step "Given the email cache is empty"
• It's important that steps behave well even when multiple emails are sent to the same address, they don't just look at the first one

Currently used step definitions are ...

douggreen:
@then /^I (?:(?:do|should) )?receive mail (titled )?"(?P[^"])"$/
@then /^I (?:do|should) not receive mail (titled )?"(?P[^"])"$/
@when /^I click "(?P[^"])" in mail(?: (?:titled )?"(?P[^"])")?$/

Berdir:
@then /^an email was sent to "([^"])"$/
@then /^no email has been sent$/
@given /^the mail subject is "([^"])"$/

larowlan:
@then /^the email to "([^"])" should contain
@given /^the email should contain "([^"])"$/

jamesharv:
@then /^(\d+) emails? should be sent:$/i
(which allows for a table with columns from, to, reply-to, subject and body

I propose (roughly):

1. (e)(-)mail(s)
2. Then (I) should be / am / was / have been sent //// Then a/an/this/these email(s) should be / are / were / have been sent
3. or not sent
4. to me / to [email protected]
5. containing / with (a/the) subject/body (containing) "example words"
6. allow for jamesharv's table format with all email fields supported by test_mail_collector
7. Allow for further steps that reuse the email(s) that has/have been identified from previous step and then require more:

• And the email(s) has/have/contains/contain (the subject/body/text) "example words"
• And the email(s) (have the) from address / from name / reply-to address / to address / to name (is/are) "blah"

8. When I click / follow (the link) "some link" in the email [allow for plain text URLs also]

[jonathanjfshaw]

As discussed here: https://forum.civicrm.org/index.php?topic=36633.0
I'd like to be able to use the same steps and basic logic with testing emails sent through civiCRM. To do this, I propose:
a) to abstract the access to the stored test emails
b) have an @civiMail tag (provided by a subcontext in the civiCRM drupal module) that switches a scenario over from inspecting the Drupal test email storage (its default behavior) and instead inspects civiCRMs test email storage.

[jonathanjfshaw]

I have forked the drupal extension, created an email Branch, and written an "emails.feature" that is a test defining by examples all the steps/syntax I'd like to have:
https://github.com/shrimala/drupalextension/blob/email/features/emails.feature

The steps to implement are:

1. edit the drupalextension so that Drupal's "testEmailSystem" is loaded by default; i.e. without explicit instruction it never sends email. But store the prior email system, so it can be swapped back in if desired by a specific scenario.

2. create an @emails-live tag that swaps the original mail system back in before a scenario with that tag, and takes it out again after the scenario.

3. create (in the drupaldriver) an abstract test mail querying system that by default queries Drupal's testMailSystem variable, BUT can be over-ridden in BeforeScenario hooks by @tags from contrib feature contexts that specify a different test mail catcher. For example, an @civiMail tag created in a seperate contrib Behat feature context for civiCRM, would provide a mechanism to query civiCRM's mail log and tell this EmailContext to use that mechanism instead. This means we can use the same test steps we're going to define here, but with a completely different way of sending/storing test emails.

4. create an @email or @emails AfterScenario hook that empties Drupal's testMailSystem variable after every scenario with that tag.

5. create a "Drupal sends mail" steps in drupalextension/features/bootstrap/FeatureContext.php in order to enable the "emails.feature" automated tests of the drupal extension module itself. "Drupal sends mail" is not a step actually used to test live sites, it's just necessary to test the test steps we're creating in EmailContext. It's for testing the tests.

6. create in EmailContext.php all the necessary step definitions to get the tests in emails.feature to pass. I have specified a lot of syntactical variations to make it easier for test writers. But these are not important, and if any of them are particularly difficult, it's OK to ignore them. I'm hoping that they will be trivially easy to create using Behat's turnip syntax (replaces Regex in Behat version 3). All functions that query the Drupal API should be in the drupaldriver not the drupalextension.

[jhedstrom]

This has been merged. Thanks all!