Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while trying to send e-mail #462

Closed
jagbarcelo opened this issue Dec 6, 2023 · 7 comments
Closed

Error while trying to send e-mail #462

jagbarcelo opened this issue Dec 6, 2023 · 7 comments

Comments

@jagbarcelo
Copy link

jagbarcelo commented Dec 6, 2023

We have a docker-compose install running but whenever a users tries to reset his password throght the lost password/reset password feature we can see the following lines in the logs of grampsweb_celery container:

[2023-12-06 10:44:48,556: INFO/MainProcess] mingle: searching for neighbors
[2023-12-06 10:44:49,564: INFO/MainProcess] mingle: all alone
[2023-12-06 10:44:49,576: INFO/MainProcess] celery@6ae888b51f67 ready.
[2023-12-06 10:45:17,037: INFO/MainProcess] Task gramps_webapi.api.tasks.send_email_reset_password[8c7c35da-dd50-4ed5-8345-93f7d9d44ad5] received
[2023-12-06 10:45:17,396: WARNING/ForkPoolWorker-4] [2023-12-06 10:45:17,396] ERROR in util: Error while trying to send e-mail.
[2023-12-06 10:45:17,396: ERROR/ForkPoolWorker-4] Error while trying to send e-mail.
[2023-12-06 10:45:17,411: ERROR/ForkPoolWorker-4] Task gramps_webapi.api.tasks.send_email_reset_password[8c7c35da-dd50-4ed5-8345-93f7d9d44ad5] raised unexpected: ValueError('Error while trying to send e-mail.')
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 399, in send_email
    smtp = smtplib.SMTP_SSL(host=host, port=port, timeout=10)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 1050, in __init__
    SMTP.__init__(self, host, port, local_hostname, timeout,
  File "/usr/lib/python3.11/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 1057, in _get_socket
    new_socket = self.context.wrap_socket(new_socket,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/usr/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/util/celery.py", line 20, in __call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 68, in send_email_reset_password
    send_email(subject=subject, body=body, to=[email])
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 418, in send_email
    raise ValueError("Error while trying to send e-mail.")
ValueError: Error while trying to send e-mail.

We are using the following docker-compose environment variables:

    environment:
      GRAMPSWEB_BASE_URL: "https://gramps.mydomain.com"
      GRAMPSWEB_TREE: "Gramps Web"
      GRAMPSWEB_CELERY_CONFIG__broker_url: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_CELERY_CONFIG__result_backend: "redis://grampsweb_redis:6379/0"
      GRAMPSWEB_RATELIMIT_STORAGE_URI: redis://grampsweb_redis:6379/1
      GRAMPSWEB_EMAIL_HOST: "smtp.office365.com" # SMTP server host (e.g. for sending password reset e-mails)
      GRAMPSWEB_EMAIL_PORT: 587 # SMTP server port. defaults to 465
      GRAMPSWEB_EMAIL_USE_TLS: "True" # The same errors appear when using "False"
      GRAMPSWEB_EMAIL_HOST_USER: "[email protected]"
      GRAMPSWEB_EMAIL_HOST_PASSWORD: "my-app-password" # AppPasswords are created via https://account.live.com/proofs/Manage/additional in order to bypass MFA
      GRAMPSWEB_DEFAULT_FROM_EMAIL: "[email protected]"

As you can see we are sending the emails from an @outlook.com address via smtp.office365.com host using an application password created through the page https://account.live.com/proofs/Manage/additional (for more information about app passwords, see https://go.microsoft.com/fwlink/?LinkId=277093 )

We have the email feature set up this way in other devices/services (because the sending email address has Multifactor Authenticator enabled) and it works perfectly fine. It does not matter if we try with TLS: true or false when using PORT: 587. Both cases yield the same results.

However, when using PORT 465, the errors are slightly different (with this port it fails a line earlier, "/usr/lib/python3.11/smtplib.py", line 1056):

  File "/usr/lib/python3.11/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 1056, in _get_socket
    new_socket = super()._get_socket(host, port, timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 851, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.11/socket.py", line 836, in create_connection
    sock.connect(sa)
OSError: [Errno 99] Cannot assign requested address
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/util/celery.py", line 20, in __call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/tasks.py", line 68, in send_email_reset_password
    send_email(subject=subject, body=body, to=[email])
  File "/usr/local/lib/python3.11/dist-packages/gramps_webapi/api/util.py", line 418, in send_email
    raise ValueError("Error while trying to send e-mail.")
ValueError: Error while trying to send e-mail.

The error when using port 587 is related to [SSL: WRONG_VERSION_NUMBER]. Maybe it is something related to the version of TLS being used (TLS 1.1 is now deprecated).

@DavidMStraub
Copy link
Member

Hi,

not sure what the problem is. Here is a minimal example, can you please run that directly in Python and try out which configuration works for you?

import smtplib
from email.message import EmailMessage
from email.utils import make_msgid


msg = EmailMessage()
msg.set_content("Hello World.")
msg["Subject"] = "Test"
msg["From"] = "[email protected]"
msg["To"] = "[email protected]"
msg["Message-ID"] = make_msgid()
host = "smtp.office365.com"
port = 587
user = "[email protected]"
password = "my-app-password"
use_tls = True  # try with False as well
if use_tls:
    smtp = smtplib.SMTP_SSL(host=host, port=port, timeout=10)
else:
    smtp = smtplib.SMTP(host=host, port=port, timeout=10)
    smtp.ehlo()
    smtp.starttls()
    smtp.ehlo()
smtp.login(user, password)
smtp.send_message(msg)
smtp.quit()

@jagbarcelo
Copy link
Author

jagbarcelo commented Dec 6, 2023

@DavidMStraub first of all, thanks for quick reply.

I just created a quick and dirty /users/temp/test.py file with your contents (updating my email and password, but respecting exact same line-numbering) and this is the output (with use_tls = True):

root@8f6df84b2c5f:/app/users/test# python3 test.py
Traceback (most recent call last):
  File "/app/users/test/test.py", line 18, in <module>
    smtp = smtplib.SMTP_SSL(host=host, port=port, timeout=10)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 1050, in __init__
    SMTP.__init__(self, host, port, local_hostname, timeout,
  File "/usr/lib/python3.11/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/smtplib.py", line 1057, in _get_socket
    new_socket = self.context.wrap_socket(new_socket,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/usr/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)
root@8f6df84b2c5f:/app/users/test#

With use_tls = false, the email is sent and received just fine, which is weird because I tested that option in Gramps.

Just in case, I have double checked it now. I have (again) tested Gramps using EMAIL_USE_TLS: "False" and also GRAMPSWEB_EMAIL_USE_TLS: "False": It fails in both cases.

@jagbarcelo
Copy link
Author

Viewin the source at: https://github.com/gramps-project/gramps-webapi/blob/ad17934c976bd8421d3ef601ac2cfbb9bc4fe605/gramps_webapi/api/util.py#L392-L396
you can see that we are retrieving the environment variables without the prefix GRAMPSWEB_

However, when we used those environment variable names (without prefix) we received a bunch of messages like these in grampsweb container (and that's why we added the prefix).

  warnings.warn(
/usr/local/lib/python3.11/dist-packages/gramps_webapi/app.py:69: UserWarning: Setting the `EMAIL_HOST` config option via the `EMAIL_HOST` environment variable is deprecated and will stop working in the future. Please use `GRAMPSWEB_EMAIL_HOST` instead.
  warnings.warn(
/usr/local/lib/python3.11/dist-packages/gramps_webapi/app.py:69: UserWarning: Setting the `EMAIL_PORT` config option via the `EMAIL_PORT` environment variable is deprecated and will stop working in the future. Please use `GRAMPSWEB_EMAIL_PORT` instead.

Maybe by fixing those warnings in grampsweb we broke something else in webapi because it is not ready to retrive these new variables that are suggested instead.

@DavidMStraub
Copy link
Member

No, that's fine. get_config gets it from the flask config, and the flask config used to get it from the env var with the same name (deprecated) or from the env var with GRAMPSWEB_ prefixed. So that's not the issue.

But indeed it's very puzzling that it works for you using the above script, but not in Gramps Web.

You are using the default docker images, right?

@DavidMStraub
Copy link
Member

DavidMStraub commented Dec 6, 2023

I think the problem is that you are using GRAMPSWEB_EMAIL_USE_TLS="False". Please try GRAMPSWEB_EMAIL_USE_TLS="false". If that still doesn't work, please share the error.

@jagbarcelo
Copy link
Author

Yes, that was it!

Using GRAMPSWEB_EMAIL_USE_TLS: "false" in lowercase did the trick.

I started using the docker-compose.yaml from https://github.com/gramps-project/web/blob/main/examples/docker-compose-base/docker-compose.yml and ended customising it a bit with some extra environment variables for the email feature. I thought that Python would do that trivial conversions from string to boolean values. It seems I was wrong. Maybe updating that file with these environment variables, and some comments would avoid others this very same issue in the future.

Thanks a lot for you help.

@DavidMStraub
Copy link
Member

Thanks!

I added a warning to the docs: gramps-project/gramps-web-docs@efc54b1

Not allowing anything but true/false when using environment variables for boolean config options is a Flask design choice (https://flask.palletsprojects.com/en/2.3.x/config/#configuring-from-environment-variables).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants