alerts failed: context canceled & context deadline exceeded

[wkshare]

time="2016-05-10T14:06:36+08:00" level=error msg="Error on notify: context canceled" source="notify.go:152" 
time="2016-05-10T14:06:36+08:00" level=error msg="Notify for 3336 alerts failed: context canceled" source="dispatch.go:238" 
time="2016-05-10T14:09:22+08:00" level=error msg="Error on notify: context deadline exceeded" source="notify.go:152" 
time="2016-05-10T14:09:22+08:00" level=error msg="Notify for 3331 alerts failed: context deadline exceeded" source="dispatch.go:238"

rate(prometheus_notifications_sent_total[1m])
= 3 alerts / sec

What is the reason???

[brian-brazil]

Which receiver are you trying to use? It seems to be timing out on the other end.

[wkshare]

Configuration of the SMTP mail, using the mailgun.

May be the SMTP timeout?

@brian-brazil

[brian-brazil]

That sounds like a SMTP timeout. With so many alerts the notification may be big enough to be causing issues.

[wkshare]

i got. @brian-brazil

smtp timeout