Add Dynamic Template Support

[thinkingserious]

Issue Summary

On 7/24/2018, our team publicly launched dynamic content for transactional templates. It is now available for all customers sending over v3 of our Mail Send API. Iterate over lists, handle conditionals and more, thanks to native support for a subset of Handlebars syntax!

More information can be found in our blog post announcement.

You can currently use this feature by manually creating the request body as shown here.

Now, we need to create helper code and examples for this SDK.

Acceptance Criteria

• Implement a helper similar to what we have for the legacy templates
• Update the USE_CASES.md example to demonstrate the new Dynamic Templates using the helper and re-name the current example to Legacy

Documentation

• Reference Implementation
• How to Send an Email With Dynamic Transactional Templates
• Using Handlebars
• Interactive API Docs

[Jericho]

To everyone waiting for dynamic templates to be supported by SendGrid's c# client: please note that I released StrongGrid v0.46.0 with support for dynamic templates. You can create and update dynamic templates and, of course, send emails using a dynamic template. I also wrote down a few thoughts and observations and I invite everybody to contribute their own observations as they figure out the peculiarities of these dynamic templates.

Also, I will be adding a way to specify test data when creating a dynamic template and a way to include iterable items and deep objects in your dynamic data. Comments, ideas and suggestions are welcome.

[Jericho]

Quick update: StrongGrid 0.47.0 was released earlier today with full dynamic templates support including test data, deep objects and iterable items. Please see the dynamic templates section in the readme for code samples.

[RoelVerhees]

I will try to add this feature, if i have some time left. I guess this is gonna be added in the new version Nuget version of SendGrid?

[thinkingserious]

Hello @RoelVerhees,

That's awesome! Yes, we would add your PR to the next Nuget version assuming it is merged.

With Best Regards,

Elmer

[RoelVerhees]

All the unittests that sent something to SendGrid fail, what am i doing wrong?

[thinkingserious]

Hi @RoelVerhees,

I'm guessing there is an issue with your Prism setup. Did you follow the instructions here? Alternatively, you can push up your branch to GitHub and let Travis handle the tests.

With Best Regards,

Elmer

[andrzej-rojek]

For the time being I'm using the following workaround to pass dynamic template data. Instead of client.SendEmailAsync(msg) you can get message JSON string, insert dynamic template data, serialize back to JSON and then send it over using client.RequestAsync() method:

var json = msg.Serialize();
var obj = (JObject)JsonConvert.DeserializeObject(json);
var personalizations = (JArray)obj["personalizations"];
var personalization = (JObject)personalizations[0];
personalization.Property("to").AddAfterSelf(new JProperty("dynamic_template_data", substitutions));
json = JsonConvert.SerializeObject(obj);
client.RequestAsync(SendGridClient.Method.POST, json, null, "mail/send");

[shdnx]

Based on @andrzej-rojek 's workaround, here is mine, slightly more general and fluffed out:

Task<Response> _SendMail(SendGridMessage message)
{
    string messageJSON = message.Serialize();

    var obj = (JObject)JsonConvert.DeserializeObject(messageJSON);
    var personalizations = (JArray)obj["personalizations"];
    foreach (var personalization in personalizations.Cast<JObject>())
    {
        var substitutions = personalization.Property("substitutions");
        substitutions.Replace(new JProperty("dynamic_template_data", substitutions.Value));
    }

    messageJSON = JsonConvert.SerializeObject(obj);
    return Client.RequestAsync(SendGridClient.Method.POST, messageJSON, null, "mail/send");
}

Using this, you're able to add your dynamic template data as substitutions, by building up Personalization.Substitutions, and then this code will rename it to dynamic_template_data. This will work for dynamic template data that's just primitives (strings, integers, etc.), but not with things like objects or arrays.

[dmbaker90]

I'm new to SendGrid and don't really understand @andrewsc3 or @shdnx workarounds. I am getting the Substitutions may not be used with dynamic templating error. I have a Dictionary<string, string> of substitutions. What do you do with that in the workaround?

For anyone else directed here, Legacy Transactional Templates appear to work fine for me.

[thinkingserious]

Do you mind sharing the code that is giving you that error @dmbaker90?

[dmbaker90]

Same issue and code as #712 which was closed and directed here.

[ghost]

Hi Elmer,

Firstly, hang in there; I don't envy you at this particular moment; we all appreciate the effort that you and your team are putting in. Secondly, what's the ETA on delivery for this item? It's a blocking issue for my team and we're going to make a decision to drop SendGrid as a service if the API clients aren't kept in sync--this isn't a reflection on you or your team but on SendGrid's product management activities in my opinion.

While I like the idea that SendGrid leverages the open source community to get more technologies supported, in my experience going live with a new API but with poor docs and key functionality gaps in the AP clients is a silly mistake that erodes confidence and creates a terribly negative experience.

Warm regards,
Seb

[Jericho]

@sebgx Rather than dropping SendGrid as a service because their C# client lags behind their API, I suggest you look at an alternative C# client: StrongGrid (disclaimer: I'm the author). StrongGrid is a client that covers 100% of SendGrid's API including sending emails using dynamic templates.

[thinkingserious]

Hi @sebgx,

The failing is not on the product team, they would have certainly have liked our SDKs to be ready on day one of the release and I believe they made the correct decision to not delay the release on account of the SDK helpers not being available. The simple fact is that I just could not get it completed in time for launch. This issue was a lower priority item on my backlog given there is a workaround, but you all have spoken (and we take all of your feedback to heart) and this functionality has now risen significantly on my backlog.

The current status is that we just released the helpers for PHP yesterday, Python is in progress and C# is next. Please hang in there and utilize one of the workarounds, or please take up @Jericho's suggestion of utilizing the excellent StrongGrid client.

Thanks!

With Best Regards,

Elmer

[ghost]

Hi Elmer,

While I think your heart is in the right place, let's unpack what this means for my business. In selecting SendGrid, dynamic templates and the WYSIWYG designer were key tech differentiators because both meant time savings for a team that is under pressure. We sign-up, configure and then follow the docs precisely as instructed, but nothing works. Further, the errors provided are difficult to consume and send us to docs that provide no solutions. We check our code, the docs, and then search for quite some time to eventually find out that the API is out of sync with its clients. We then implement the workaround suggested knowing that we're picking up tech debt and creating a code smell with guaranteed rework that will be required. We test, no errors but equally no e-mails delivered.

We e-mail support. When someone does eventually get back to us many hours later, the solution is to buy a subscription and a dedicated IP. So, what's the point of the trial and why on earth should I trust that after failing miserably on everything I care about, that rewarding those failures with money will make things better?

Simple solution to all of the problems... SparkPost. We actually get a trial. We actually get e-mails delivered. No tech debt or code smells. Good docs. Great support. API and client are super simple. Error consumption is straightforward. Very competitive pricing. Great IP reputation management even on shared IPs and trial accounts.

It's one thing to lose a customer to a competitor, it's quite something else to drive the customer away and guarantee that the customer is never coming back.

Warm regards,
Seb

[ChrisASearles]

@thinkingserious I think the primary issue is that the docs weren't updated to reflect the fact that the new templates weren't yet supported. At the same time, the old templates were moved to "Legacy" status, indicating to all users that those templates should be deprecated in favor of the new version.

Had there at least been some documentation to that fact prior to the release of the new templates, it would have saved everyone here a significant amount of time trying to work with what is now the recommended solution from SendGrid. With an SDK that is supported by the organization, one would expect at least this much.

I've been happy with SendGrid and don't plan on going anywhere else. However, as I initially came to you because of my relationship with Azure and your prominence within that environment, one would think your C# SDK would warrant at least enough attention to garner some updated documentation with such a large core platform change.

Glad you're working on this now. As @sebgx mentioned, each day that passes creates more tech debt as we create new legacy templates and continue to maintain old code, so we appreciate that this has moved to the top of your queue.

Thank you for your engagement with the community and hopefully this will help improve the experience for us .NETters in the future.

[thinkingserious]

Hello Everyone!

Python support was just released today, so now this SDK is at the top of our list. Thanks for your patience, this one should be launched soon :)

@sebgx @ChrisASearles,

Thanks for the continued conversation, your feedback matters to us and we read and consider it all.

Today we have expanded our DX (Developer Experience - we are responsible for these SDKs) team (w00t!) and we are committed to serving you to the best of our abilities :) So please, always feel free to make your voice heard here. This is your/our community.

With Best Regards,

Elmer

[scgough]

Hi all - based on @shdnx 's post above I'm doing the following but it's not working :/
The mail sends but substitution tags aren't replaced in the dynamic template.

For example, in my mail content I have the tag -testsub-

I run the following over my SendGrid message object:

string messageJSON = mail.Serialize();

var obj = (JObject)JsonConvert.DeserializeObject(messageJSON);
var personalizations = (JArray)obj["personalizations"];
foreach (var personalization in personalizations.Cast<JObject>())
{
        var substitutions = personalization.Property("substitutions");
        substitutions.Replace(new JProperty("dynamic_template_data", substitutions.Value));
}

This gives me messageJSON value of:

{
    "from":{
        "email":"[email protected]"
    },
    "subject":"(Test)",
    "personalizations":[{
        "to":[{"email":"[email protected]"}],
        "dynamic_template_data":{"-testsub-":"TESTING 1","testsub":"TESTING 2"}
     }],
     "content":[{"type":"text/html","value":" "}],
     "template_id":"d-templateidhere"}

I send using: var response = await sendGridClient.RequestAsync(SendGridClient.Method.POST, messageJSON, null, "mail/send");

As you can see, i'm trying to replace with -testsub- and testsub

Any ideas?!

[shdnx]

@scgough dynamic_template_data is only applicable when you use the new dynamic transactional SendGrid templates, which work with Handlebars.js. Your data -testsub- is invalid because of that, I imagine. testsub is valid, but in the template you have to have {{testsub}} to get that replaced. Read up on Handlebars.js and the SendGrid documentation.

Also, saying "it's not working" without providing an error message is generally not helpful, and chances are nobody will be able to help you. Any time you're asking for help, always state the error message and all other error-related information you're getting (e.g. stack trace).

[scgough]

@shdnx - apologies but there is no error message. My mail comes through but the substitutions aren't made. From an API POV the 'send' is a success.

I will look at the handlebar syntax as this could well be a key factor in to why I'm having issues.

[bhanodai]

Any updates on the this??

[thinkingserious]

I'm currently working on this.

[Pieter-1337]

Any updates on this matter. I am need of using Dynamic Template data on objects etc.. and not just on strings, integers etc... Any timeframe to where we can expect support for this matter?

[matheusdelre]

Any news on the subject?

[bhanodai]

Thanks for the fix. Do we have a date when you are planning to release the next version?

[shervinw]

Hours lost following docs! What is the recommendation?

[deivydas321]

Following. Is there any date for the fix?

[thinkingserious]

Hello @Deivid321,

The update is live in release 9.10.0.

Thanks!

With Best Regards,

Elmer

[deivydas321]

Thanks @thinkingserious , are there any examples how to use substitutions on c#? Currently my tests fail when I try to use it like this:

var msg = new SendGridMessage();
            msg.SetSandBoxMode(_testMode);
            msg.SetFrom(email.Sender);

            var sendGridRecipients = CreateSendGridRecipients(email.Recipients);
            msg.AddTos(sendGridRecipients.ToList());
            msg.SetTemplateId(email.TemplateId);
            msg.AddSubstitution("-name-", "Deivydas");
            
            return msg;

SenGrid says Bad Request when I am adding substitution.

[thinkingserious]

Hi @Deivid321,

Here you go. I hope that helps!

With Best Regards,

Elmer

[deivydas321]

Hi @thinkingserious ,

Thanks! That helped a lot.

Regards,
Deivydas

[abhilashabhatia]

Any news on the subject?

I can't get the Subject to work. Does it has to be part of the dynamicTemplateData key in Node?

[thinkingserious]

Hello @abhilashabhatia,

Please see this example. I hope that helps!

With Best Regards,

Elmer

[SamuelNF]

I'm having the same issues talked about here: #727

I'm using the latest version of the library - is the X-No-CORS-Reason bug still an issue in the latest release?

[c4c1us]

Hello @abhilashabhatia,

Please see this example. I hope that helps!

With Best Regards,

Elmer

Same issue on sendgrid-php, I used Legacy Template and it worked.