Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CRM-21521: fix nested multipart handling in bounce processing #11390

Merged
merged 1 commit into from
Jan 15, 2018

Conversation

ejegg
Copy link
Contributor

@ejegg ejegg commented Dec 7, 2017

When for example a multipart-report email body's first part is
multipart-related, extract the text correctly instead of crashing.

This logic should handle all kinds of weird multipart nesting, up
to MIME_MAX_RECURSION levels deep.

Overview

Makes email bounce processing more robust

Before

Bounce processing crashes when a multipart-report email body's first part is multipart-related

After

Bounce processing correctly extracts arbitrarily nested multiparts

Technical Details

Pulls various kinds of multipart extraction into their own functions that can be called recursively. Gives up after a set recursion depth.


copy(__DIR__ . '/data/bounces/' . $mail, __DIR__ . '/data/mail/' . $mail);
$this->callAPISuccess('job', 'fetch_bounces', array());
$this->assertFalse(file_exists(__DIR__ . '/data/mail/' . $mail));
$this->checkMailingBounces(1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ejegg Just wondering if its worth while checking for a specific type and or message stored in the db associated with the bounce also?

@ejegg
Copy link
Contributor Author

ejegg commented Dec 7, 2017

Any thoughts on the 'style error'? I can't figure out what it's complaining about.

}
elseif ($multipart instanceof ezcMailMultipartRelated) {
$text = self::getTextFromMultipartRelated($multipart, $recursionLevel);
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

else needs to be on a new line

foreach ($multipart->getParts() as $part) {
if (isset($part->subType) and $part->subType === 'plain') {
$text = $part->text;
} elseif ($part instanceof ezcMailMultipart) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elseif needs to be on a new line

foreach ($related->getRelatedParts() as $part) {
if (isset($part->subType) and $part->subType === 'plain') {
$text = $part->text;
} elseif ($part instanceof ezcMailMultipart) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elseif needs to be on a new line

}
}
return $text;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a blank line between this closing brace and the class closing brace

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ejegg you need 1 complete blank line after this brace but before the } for the class

@seamuslee001
Copy link
Contributor

@ejegg see my comments just now

@ejegg
Copy link
Contributor Author

ejegg commented Dec 7, 2017

Thanks seamuslee001! I can squash all those changes together to make the commit history cleaner

@ejegg ejegg changed the title CRM-21521: read multipart-related inside report CRM-21521: fix nested multipart handling in bounce processing Dec 8, 2017
@ejegg
Copy link
Contributor Author

ejegg commented Dec 12, 2017

@seamuslee001 this is all squashed and passing tests!

@seamuslee001
Copy link
Contributor

nice work @ejegg :-) I'll see if i can get a colleague of mine to review it

$text = $part->text;
}
elseif ($part instanceof ezcMailMultipart) {
$text = self::getTextFromMultipart($part);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need to be passing through the $recussionLevel here?

@ejegg ejegg force-pushed the bounceProcessing branch 2 times, most recently from cda8193 to cfa8dbd Compare December 13, 2017 15:38
@ejegg
Copy link
Contributor Author

ejegg commented Dec 13, 2017

Good catch @seamuslee001, fixed!

@eileenmcnaughton
Copy link
Contributor

I can't merge this as @ejegg is a colleague but I merged the same patch into our wmf & want to add some notes as to the review I did there

  1. I found a less complex multipart email and introduced a unit test for that sample into the main CiviCRM test suite (already merged)
  2. I went through the extraction process locally, noting that the only significant changes related to the recursion issue on the email being processed
  3. checked that running the unit test per this PR on the anonymised version of the email did fail just as it did on our production & that with the changes it now passes.

This code will only be reached when processing a multipart email & we have assurances from the added unit test that we have not broken that.

@colemanw
Copy link
Member

I'm a little wary of adding global constants because of potential namespace conflicts. Any reason MAIL_MAX_RECURSION can't be added to the class instead of globally?

@ejegg ejegg force-pushed the bounceProcessing branch from cda8193 to ab48be7 Compare January 8, 2018 17:24
When for example a multipart-report email body's first part is
multipart-related, extract the text correctly instead of crashing.

This logic should handle all kinds of weird multipart nesting, up
to MIME_MAX_RECURSION levels deep.
@ejegg ejegg force-pushed the bounceProcessing branch from ab48be7 to 01467ae Compare January 8, 2018 17:27
@ejegg
Copy link
Contributor Author

ejegg commented Jan 8, 2018

Thanks for taking a look @colemanw! I've changed the MAX_RECURSION bit to a class constant.

@ejegg
Copy link
Contributor Author

ejegg commented Jan 8, 2018

Looks like some unrelated DB errors in the testing setup

@ejegg
Copy link
Contributor Author

ejegg commented Jan 10, 2018

jenkins re-test this please

1 similar comment
@ejegg
Copy link
Contributor Author

ejegg commented Jan 10, 2018

jenkins re-test this please

@eileenmcnaughton
Copy link
Contributor

test this please

@colemanw colemanw merged commit 6afa43d into civicrm:master Jan 15, 2018
@colemanw
Copy link
Member

I'm satisfied that this has had enough unit testing and code review to merge. Good work @ejegg

kenwest added a commit to kenwest/cbf-drupal that referenced this pull request Aug 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants