From 910c393f400ffc35202f89e38284f0199130cdf3 Mon Sep 17 00:00:00 2001 From: Simon Robinson Date: Mon, 11 Nov 2024 11:22:52 +0000 Subject: [PATCH] Improve documentation and example configurations --- README.md | 54 +++++++++++----------- emailproxy.config | 114 +++++++++++++++++++++++++++------------------- 2 files changed, 93 insertions(+), 75 deletions(-) diff --git a/README.md b/README.md index af830c3..6271ed3 100644 --- a/README.md +++ b/README.md @@ -10,15 +10,15 @@ The proxy works in the background with a menu bar/taskbar helper or as a headles ### Example use-cases - You need to use an Office 365 email account, but don't get on with Outlook. -The email client you like doesn't support OAuth 2.0, which became mandatory [in January 2023](https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-september/ba-p/3609437) ([September 2024 for free Hotmail/Outlook accounts](https://support.microsoft.com/en-us/office/modern-authentication-methods-now-needed-to-continue-syncing-outlook-email-in-non-microsoft-email-apps-c5d65390-9676-4763-b41f-d7986499a90d); [September 2025 for O365 SMTP](https://techcommunity.microsoft.com/t5/exchange-team-blog/exchange-online-to-retire-basic-auth-for-client-submission-smtp/ba-p/4114750)). +The email client you like doesn't support OAuth 2.0, which became mandatory [in January 2023](https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-deprecation-in-exchange-online-september/ba-p/3609437) ([September 2024 for personal Hotmail/Outlook accounts](https://support.microsoft.com/en-us/office/modern-authentication-methods-now-needed-to-continue-syncing-outlook-email-in-non-microsoft-email-apps-c5d65390-9676-4763-b41f-d7986499a90d); [September 2025 for O365 SMTP](https://techcommunity.microsoft.com/t5/exchange-team-blog/exchange-online-to-retire-basic-auth-for-client-submission-smtp/ba-p/4114750)). - You used to use Gmail via IMAP/POP/SMTP with your raw account credentials (i.e., your real password), but cannot do this now that Google has disabled this method, and don't want to use an [App Password](https://support.google.com/accounts/answer/185833) (or cannot enable this option). - You have an account already set up in an email client, and you need to switch it to OAuth 2.0 authentication. You can edit the server details, but the client forces you to delete and re-add the account to enable OAuth 2.0, and you don't want to do this. - You have made your own script or application that sends or receives email, but it doesn't support OAuth 2.0, and you don't want to have to modify it to implement this. - You work with multiple services or applications that use IMAP/POP/SMTP, and you don't want to have to set up OAuth 2.0 independently for each one. -In all of these cases and more, this proxy can help. -Follow the instructions here to get started, and please [open an issue](https://github.com/simonrob/email-oauth2-proxy/issues) with any problems or suggestions. +In all of these cases and more, this proxy can help – just follow the instructions below to get started. +Visit the [Discussions pages](https://github.com/simonrob/email-oauth2-proxy/discussions) for help with any configuration or setup problems, or [open an issue](https://github.com/simonrob/email-oauth2-proxy/issues) to report bugs or make suggestions. For commercial support or feature requests, please also consider [sponsoring this project](https://github.com/sponsors/simonrob?frequency=one-time). @@ -32,7 +32,7 @@ Begin by downloading the proxy via one of the following methods: Next, edit the sample `emailproxy.config` file to add configuration details for each email server and account that you want to use with the proxy. -[Guidance and example account configurations](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) are provided for Office 365, Gmail and several other providers, though you will need to insert your own client credentials for each one (see the [client credentials documentation](#oauth-20-client-credentials) below for guidance). +[Guidance and example account configurations](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) are provided for Office 365, Gmail and several other providers, though you will need to insert your own client credentials for each one (see the [client credentials documentation](#oauth-20-client-credentials) below for help doing this). You can remove details from the sample configuration file for services you don't use, or add additional ones for any other OAuth 2.0-authenticated IMAP/POP/SMTP servers you would like to use with the proxy. You can now start the proxy: depending on which installation option you chose, either launch the application or use the appropriate run command listed above. @@ -58,25 +58,26 @@ After your accounts are fully set-up and authorised, no further proxy interactio It will notify you if this is the case. ### OAuth 2.0 client credentials -As part of the proxy setup process you need to provide an OAuth 2.0 `client_id` and `client_secret` to allow it to authenticate with email servers on your behalf. +As part of the proxy setup process you need to provide an OAuth 2.0 `client_id` and (in many cases) a `client_secret` to allow it to authenticate with email servers on your behalf. If you have an existing client ID and secret for a desktop app, you can use these directly in the proxy. If this is not possible, you can also reuse the client ID and secret from any email client that supports IMAP/POP/SMTP OAuth 2.0 authentication with the email server you would like to connect to (such as [the](https://github.com/mozilla/releases-comm-central/blob/812b7c9068ca5cac0580b0ddbea8e34c141cd441/mailnews/base/src/OAuth2Providers.jsm) [many](https://github.com/mozilla/releases-comm-central/blob/master/mailnews/base/src/OAuth2Providers.sys.mjs) [existing](https://github.com/Foundry376/Mailspring/blob/master/app/internal_packages/onboarding/lib/onboarding-constants.ts) [open](https://gitlab.gnome.org/GNOME/evolution-data-server/-/blob/master/CMakeLists.txt) [source](https://gitlab.gnome.org/GNOME/gnome-online-accounts/-/blob/master/meson_options.txt) [clients](https://github.com/M66B/FairEmail/blob/master/app/src/main/res/xml/providers.xml) with OAuth 2.0 support), but please do this with care and restraint as access through reused tokens will be associated with the token owner rather than your own client. -If you do not have access to credentials for an existing client you will need to register your own. +If you do not want to use credentials from an existing client you will need to register your own. The process to do this is different for each provider, but the registration guides for several common ones are linked here. In all cases, when registering, make sure your client is set up to use an OAuth scope that will give it permission to access IMAP/POP/SMTP as desired. It is also highly recommended to use a scope that will grant "offline" access (i.e., a way to [refresh the OAuth 2.0 authentication token](https://oauth.net/2/refresh-tokens/) without user intervention). The [sample configuration file](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) provides example scope values for several common providers. -- Office 365: register a new [Microsoft identity application](https://learn.microsoft.com/entra/identity-platform/quickstart-register-app) -- Gmail / Google Workspace: register a [Google API desktop app client](https://developers.google.com/identity/protocols/oauth2/native-app) -- Outlook / Hotmail (free accounts): because you are not the administrator for these Microsoft-operated domains, the only option is to reuse an existing client ID – see, for example, [Thunderbird](https://blog.thunderbird.net/2023/01/important-message-for-microsoft-office-365-enterprise-users/), or the links above -- AOL and Yahoo Mail (and subproviders such as AT&T) are not currently allowing new client registrations with the OAuth email scope – the only option here is to reuse the credentials from an existing client that does have this permission +- Office 365: register a new [Microsoft identity application](https://learn.microsoft.com/entra/identity-platform/quickstart-register-app). +- Gmail / Google Workspace: register a [Google API desktop app client](https://developers.google.com/identity/protocols/oauth2/native-app). +- Outlook / Hotmail (personal accounts): If you are part of the Microsoft 365 Developer Programme or have an Azure account (including free accounts), you can create your own app registration in the Entra admin centre – see [this discussion](https://github.com/simonrob/email-oauth2-proxy/discussions/301) for a guide. +If not, you will need to reuse an existing client ID – see, for example, [this sample configuration](https://github.com/simonrob/email-oauth2-proxy/issues/297#issuecomment-2424200404). +- AOL and Yahoo Mail (and subproviders such as AT&T) are not currently allowing new client registrations with the OAuth email scope – the only option here is to reuse the credentials from an existing client that does have this permission. The proxy supports [Google Cloud service accounts](https://cloud.google.com/iam/docs/service-account-overview) for access to Google Workspace Gmail. -It also supports the [client credentials grant (CCG)](https://learn.microsoft.com/entra/identity-platform/v2-oauth2-client-creds-grant-flow) and [resource owner password credentials grant (ROPCG)](https://learn.microsoft.com/entra/identity-platform/v2-oauth-ropc) OAuth 2.0 flows, and [certificate credentials (JWT)](https://learn.microsoft.com/entra/identity-platform/certificate-credentials). -Please note that currently only Office 365 is known to support the CCG, ROPCG and certificate credentials methods. +It also supports the [client credentials grant (CCG)](https://learn.microsoft.com/entra/identity-platform/v2-oauth2-client-creds-grant-flow), [resource owner password credentials grant (ROPCG)](https://learn.microsoft.com/entra/identity-platform/v2-oauth-ropc) and [device authorisation grant (DAG)](https://tools.ietf.org/html/rfc8628) OAuth 2.0 flows, and [certificate credentials (JWT)](https://learn.microsoft.com/entra/identity-platform/certificate-credentials). +Please note that currently only Office 365 / Outlook is known to support the CCG, ROPCG, DAG and certificate credentials methods. See the [sample configuration file](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) for further details. @@ -84,7 +85,7 @@ See the [sample configuration file](https://github.com/simonrob/email-oauth2-pro When starting the proxy there are several optional arguments that can be set to customise its behaviour. - `--no-gui` will launch the proxy without an icon, which allows it to be run as a `systemctl` service as demonstrated in [this example](https://github.com/simonrob/email-oauth2-proxy/issues/2#issuecomment-839713677), or fully headless as demonstrated in [various](https://github.com/michaelstepner/email-oauth2-proxy-aws) [other](https://github.com/blacktirion/email-oauth2-proxy-docker) subprojects. -Please note that unless you also specify one of the authorisation options below, this mode is only of use if you have already authorised your accounts through the proxy in GUI mode, or are importing a pre-authorised proxy configuration file from elsewhere. +Please note that unless you also specify one of the authorisation options below, or are using an OAuth 2.0 flow that does not require user authorisation, this mode is only of use if you have already authorised your accounts through the proxy in GUI mode, or are loading a proxy configuration file that already contains the cached authorisation tokens. If you do not set `--external-auth` or `--local-server-auth`, accounts that have not yet been authorised (or for whatever reason require re-authorisation) will time out when authenticating, and an error will be printed to the log. - `--external-auth` configures the proxy to present an account authorisation URL to be opened in an external browser and wait for you to copy+paste the post-authorisation result. @@ -104,7 +105,7 @@ See the [sample configuration file](https://github.com/simonrob/email-oauth2-pro - `--config-file` allows you to specify the location of a [configuration file](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) that the proxy should load. If this argument is not provided, the proxy will look for `emailproxy.config` in its working directory. By default, the proxy also saves cached OAuth 2.0 tokens back to this file, so it must be writable. -See the `--cache-store` option, if you would rather store configuration and cached values separately. +See the `--cache-store` option if you would rather store configuration and cached values separately. - `--cache-store` is used to specify a separate location in which to cache authorised OAuth 2.0 tokens and associated metadata. The value of this argument can either be the full path to a local file (which must be writable), or an identifier for an external store such as a secrets manager (see the [advanced configuration](#advanced-configuration) section). @@ -119,24 +120,17 @@ This argument is identical to enabling debug mode from the proxy's menu bar icon If needed, debug mode can also be toggled at runtime by sending the signal `SIGUSR1` (e.g.: `pkill -SIGUSR1 -f emailproxy`). ### Advanced configuration -The [example configuration file](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) contains further documentation for various additional features of the proxy, including catch-all (wildcard) accounts, locally-encrypted connections and advanced Office 365 OAuth 2.0 flows. - -The proxy caches authenticated OAuth 2.0 tokens and associated metadata back to its own configuration file by default, but can alternatively be configured to use either a separate local file or a secrets manager service for this purpose. -Currently only [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) is supported for remote token storage. -To use this feature, set the [`--cache-store`](#optional-arguments-and-configuration) parameter to either a full ARN or a secret name, prefixing the value with `aws:` to identify its type to the proxy. -You must also install the AWS SDK for Python: `python -m pip install boto3` and [set up authentication credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#configuration) (including a region). -The minimum required permissions for the associated AWS IAM user are `secretsmanager:GetSecretValue` and `secretsmanager:PutSecretValue`. -If the named AWS Secret does not yet exist, the proxy will attempt to create it; here, the `secretsmanager:CreateSecret` permission is also required. +The [example configuration file](https://github.com/simonrob/email-oauth2-proxy/blob/main/emailproxy.config) contains further documentation for many additional features of the proxy, including catch-all (wildcard) accounts, locally-encrypted connections, advanced OAuth 2.0 flows, integration with a secrets manager and more. If you are using the proxy in a non-GUI environment it is possible to skip installation of dependencies that apply only to the interactive version. To do this, install via `python -m pip install emailproxy` (i.e., without the `[gui]` variant option), and pass the [`--no-gui`](#optional-arguments-and-configuration) argument when starting the proxy. -Please note that the proxy was designed as a GUI-based tool from the outset due to the inherently interactive nature of OAuth 2.0 authorisation, and there are limits to its ability to support fully no-GUI operation. +Please note that the proxy was designed as a GUI-based tool from the outset due to the inherently interactive nature of the most common OAuth 2.0 authorisation flows, and there are limits to its ability to support fully no-GUI operation. See the [optional arguments and configuration](#optional-arguments-and-configuration) section of this file for further details. If your network requires connections to use an existing proxy, you can instruct the script to use this by setting the [proxy handler](https://docs.python.org/3/library/urllib.request.html#urllib.request.ProxyHandler) environment variable `https_proxy` (and/or `http_proxy`) – for example, `https_proxy=localhost python -m emailproxy`. -After installing its requirements, the proxy script can be packaged as a single self-contained executable using [Nuitka](https://nuitka.net/) (`nuitka --standalone --macos-create-app-bundle emailproxy.py`) or [pyinstaller](https://pyinstaller.org/) (`pyinstaller --onefile emailproxy.py`). -If you are using pyinstaller and the GUI version of the proxy, you may need to add `--hidden-import timeago.locales.en_short` until [this `timeago` issue](https://github.com/hustcc/timeago/issues/40) is resolved. +After installing its requirements, the proxy script can be packaged as a single self-contained executable using [Nuitka](https://nuitka.net/) (`nuitka --standalone --macos-create-app-bundle emailproxy.py`) or [pyinstaller](https://pyinstaller.org/) (`pyinstaller --onefile emailproxy.py`[[1]](#f1)). +A pyinstaller-packaged version is provided automatically for each [release](https://github.com/simonrob/email-oauth2-proxy/releases). Python 3.7 or later is required to run the proxy. The [python2 branch](https://github.com/simonrob/email-oauth2-proxy/tree/python2) provides minimal compatibility with python 2.7, but with a limited feature set, and no ongoing maintenance. @@ -212,7 +206,7 @@ On Windows this is normally limited to keyboard shortcuts (i.e., copy/paste), bu As a workaround, the proxy will enable pywebview's debug mode when you run the proxy itself in debug mode, which should allow you to use the right-click context menu to copy/paste to enter text. If you are unable to proceed with popup-based authentication even with this workaround, it is worth trying the proxy's `--external-auth` or `--local-server-auth` options. -- On more recent macOS versions (10.14 and later), you may find that when first running the proxy as a service you need to manually load its launch agent in order to trigger a file access permission prompt. +- On macOS (10.14 and later), you may find that when first running the proxy as a service you need to manually load its launch agent in order to trigger a file access permission prompt. You will know intervention is necessary if the proxy exits (rather than restarts) the first time you click `Start at login` from its menu bar icon. To resolve this, exit the proxy and then run `launchctl load ~/Library/LaunchAgents/ac.robinson.email-oauth2-proxy.plist` from a terminal. A permission pop-up should appear requesting file access for python. @@ -241,13 +235,13 @@ See the [documentation and examples](https://github.com/simonrob/email-oauth2-pr ## Potential improvements (pull requests welcome) - Full feature parity on different platforms (e.g., live menu updating; monitoring network status; clickable notifications) - Switch to asyncio? (with Python 3.12, [PEP 594](https://peps.python.org/pep-0594/) removed the asyncore package that the proxy is built upon – currently mitigated by the use of [pyasyncore](https://pypi.org/project/pyasyncore/)) -- STARTTLS for IMAP/POP? +- Remote STARTTLS for IMAP/POP? ## Related projects and alternatives Michael Stepner has created a [Terraform configuration](https://github.com/michaelstepner/email-oauth2-proxy-aws) that helps run this proxy on a lightweight cloud server (AWS EC2). Thiago Macieira has provided a [makefile and systemd configuration files](https://github.com/thiagomacieira/email-oauth2-proxy/tree/Add_a_Makefile_and_systemd_configuration_files_to_install_system_wide). -For Docker, blacktirion has an [example configuration](https://github.com/blacktirion/email-oauth2-proxy-docker). +For Docker, Moriah Morgan has an [example configuration](https://github.com/blacktirion/email-oauth2-proxy-docker). If you already use postfix, the [sasl-xoauth2](https://github.com/tarickb/sasl-xoauth2) plugin is probably a better solution than running this proxy. Similarly, if you use an application that is able to handle OAuth 2.0 tokens but just cannot retrieve them itself, then [pizauth](https://github.com/ltratt/pizauth), [mailctl](https://github.com/pdobsan/mailctl) or [oauth-helper-office-365](https://github.com/ahrex/oauth-helper-office-365) may be more appropriate. @@ -260,3 +254,7 @@ This proxy was developed to work around these limitations for providers that do ## License [Apache 2.0](https://github.com/simonrob/email-oauth2-proxy/blob/main/LICENSE) + + +--- +1. If you are packaging the GUI version of the proxy using pyinstaller, you may need to add `--hidden-import timeago.locales.en_short` until [this `timeago` issue](https://github.com/hustcc/timeago/issues/40) is resolved. diff --git a/emailproxy.config b/emailproxy.config index dde604f..71d1d28 100644 --- a/emailproxy.config +++ b/emailproxy.config @@ -18,7 +18,7 @@ documentation = Local servers are specified as demonstrated below where, for exa port must be above 1023 (unless the proxy script is run via sudo), below 65536, and unique across local servers. Multiple accounts can share the same server, however. Each server section must specify the `server_address` and `server_port` of the remote server that it will be proxying - you can obtain these values from your email provider, - or use the details below for Office 365 and/or Gmail. + or use the details below (examples are given for Office 365 / Outlook and Gmail). To allow the proxy to operate, your email client must be set up to use an unencrypted connection for IMAP/SMTP/POP (i.e., no STARTTLS or SSL/TLS, just plain login credentials). The proxy will create a secure connection on your @@ -48,17 +48,26 @@ documentation = Local servers are specified as demonstrated below where, for exa /etc/letsencrypt/live/mail.example.net/privkey.pem) for the server you are using the proxy with, and it will use these to set up a secure connection between itself and your email client. + Advanced feature - proxy plugins: + - Plugins are an advanced feature that enable the use of separate scripts to modify IMAP/POP/SMTP commands when they + are received from the client or server before passing through to the other side of the connection. For more details + about how to install and enable this feature, see the additional documentation and range of sample plugins explained + at https://github.com/simonrob/email-oauth2-proxy/tree/plugins/plugins#email-oauth-20-proxy-plugins + [IMAP-1993] +documentation = *** note: this server will work for both Office 365 and personal Outlook/Hotmail accounts *** server_address = outlook.office365.com server_port = 993 local_address = 127.0.0.1 [POP-1995] +documentation = *** note: this server will work for both Office 365 and personal Outlook/Hotmail accounts *** server_address = outlook.office365.com server_port = 995 local_address = 127.0.0.1 [SMTP-1587] +documentation = *** note: this server will work for both Office 365 and personal Outlook/Hotmail accounts *** server_address = smtp-mail.outlook.com server_port = 587 server_starttls = True @@ -87,9 +96,9 @@ documentation = Accounts are specified using your email address as the section h OAuth 2.0 flow you are using, other values may also be required (see examples below). If you are adding an account for a service other than the examples shown below then the provider's documentation should provide these details. - You will also need to add your own `client_id` and `client_secret` values as indicated below. These can either be - reused from an existing source (such as another email client that supports OAuth 2.0), or you can register and use - your own desktop app API client credentials. See https://developers.google.com/identity/protocols/oauth2/native-app + You will also need to add your own `client_id` and (in many cases) `client_secret` values as indicated below. These + can either be reused from an existing source (such as an email client that supports OAuth 2.0), or you can register + your own desktop app client credentials. See https://developers.google.com/identity/protocols/oauth2/native-app and the Microsoft link below for details. Multiple accounts on the same server can use the same values for the `client_id` and `client_secret` properties; just duplicate these in each account's entry below, or see the advanced `allow_catch_all_accounts` option. Note that while there are example account configurations for AOL and Yahoo Mail @@ -99,8 +108,9 @@ documentation = Accounts are specified using your email address as the section h Once the proxy is correctly configured, after the first successful use of an account its access token details will be cached for future use, encrypted with the IMAP/POP/SMTP password you used in your email client. By default this configuration file is reused for caching (so it must be writable), but you can specify a different location or - method using the proxy's `--cache-store` parameter. You should not add or edit cached values manually (i.e., - `token_salt`, `access_token`, `access_token_expiry`, `refresh_token` and `last_activity`); the proxy handles this. + method using the proxy's `--cache-store` parameter. See below for advanced use of this option to integrate with a + secrets manager service. You should not add or edit cached values manually (i.e.,`token_salt`, `token_iterations`, + `access_token`, `access_token_expiry`, `refresh_token` and `last_activity`); the proxy handles this. The password used in your email client is not used for authentication with the actual email server (this is done via OAuth 2.0 in a web browser), so it can be different to your real account password, which is helpful for debugging. @@ -108,33 +118,34 @@ documentation = Accounts are specified using your email address as the section h password to avoid repeated re-authentication requests (which is the proxy's default behaviour when credential decryption fails). See the proxy's README.md file for more information and the end of this file for further options. - Office 365 customisation: - - Unlike other providers, Office 365 requires an OAuth 2.0 scope that explicitly specifies `offline_access` (shown - in the example below) in order to allow the proxy to refresh its access token on your behalf. The proxy will still - work if this parameter is not included, but you will need to re-authenticate extremely often (about once per hour). + Office 365 / Outlook customisation: + - Unlike other providers, Office 365 / Outlook requires an OAuth 2.0 scope that explicitly grants `offline_access` + (shown in the examples below) in order to allow the proxy to refresh its access token on your behalf. The proxy will + still without this parameter, but you will need to re-authenticate extremely often (about once per hour). - - The example Office 365 configuration entries below use an OAuth 2.0 scope that clearly specifies IMAP, POP and - SMTP permission. If you do not require one or more of these protocols, you may remove the relevant values to ensure - the access tokens obtained on your behalf are as precisely-targeted as possible. Conversely, it is also possible to - replace these specific scopes with the more generic `https://outlook.office365.com/.default`. Switching to a broader - scope value may also be needed if you are using a version of O365 delivered by a regional provider (e.g., 21Vianet). - See: https://github.com/simonrob/email-oauth2-proxy/issues/255 for more details and discussion. + - The example Office 365 / Outlook configuration entries below use an OAuth 2.0 scope that clearly specifies IMAP, + POP and SMTP permission. If you do not require one or more of these protocols, you may remove the relevant values to + ensure access tokens obtained on your behalf are as precisely-targeted as possible. Conversely, it is also possible + to replace these specific scopes with the more generic `https://outlook.office365.com/.default`. Switching to a + broader scope value may also be needed if you are using Microsoft services delivered by a regional provider (e.g., + 21Vianet). See: https://github.com/simonrob/email-oauth2-proxy/issues/255 for more details and discussion. - By default, new Entra (Azure AD) clients are accessible only within your own tenant. If you are registering a new client to use with the proxy (and do not want to make it available outside your own organisation) you will need to - replace `common` with your tenant ID in the Office 365 `permission_url` and `token_url` values below. Alternatively, - you can reuse credentials from an existing client registration (see the proxy's README.md file), or configure your - client as a multi-tenant application. For more detail about this, and guides for setting up your desktop app client, - see the documentation at https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app. + replace `common` with your tenant ID in the `permission_url` and `token_url` values below. Alternatively, you can + reuse credentials from an existing client registration (see the proxy's README.md file), or configure your client as + a multi-tenant application. For more detail about this, and guides for setting up your desktop app client, see the + documentation at https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app. - Office 365 shared mailboxes are supported: add an account entry here using the email address of the shared mailbox as the account name. When asked to authenticate, log in as the user that access has been delegated to. - Note that Office 365 no-longer supports the `authorised.user@example.com/delegated.mailbox` username syntax here. + Note that Office 365 no-longer supports the `authorised.user@example.com/delegated.mailbox` username syntax. - - It is possible to create Office 365 clients that do not require a secret to be sent. If this is the case for your - setup, delete the `client_secret` line from your account's configuration entry (do not leave the default value). + - It is possible to create Office 365 / Outlook OAuth 2.0 clients that do not require a secret to be sent. If this + is the case for your setup, delete the `client_secret` line from your account's configuration entry (do not leave + the default value). - - To use O365 certificate credentials instead of a client secret, delete the `client_secret` line and instead + - To use Office 365 certificate credentials instead of a client secret, delete the `client_secret` line and instead provide a `jwt_certificate_path` (e.g., /path/to/certificate.pem) and `jwt_key_path` (e.g., /path/to/key.pem). Further documentation and examples can be found at https://github.com/simonrob/email-oauth2-proxy/pull/247. @@ -147,17 +158,18 @@ documentation = Accounts are specified using your email address as the section h - WARNING: Please note that by default the CCG flow has essentially no local access control when creating new accounts (no user consent is involved, so the proxy cannot validate login attempts unless an account entry - already exists its configuration file). Using the CCG flow with the proxy in a publicly-accessible context is - not advised. This is especially important when using the proxy's catch-all feature (which is likely to be the - case given the typical use-cases for the CCG flow). Because of this, you are highly encouraged to enable the - proxy's secret encryption option - see `encrypt_client_secret_on_first_use` at the end of this file. In + already exists in its configuration file). Using the CCG flow with the proxy in a publicly-accessible context + is not advised. This is especially important when using the proxy's catch-all feature (which is likely to be + the case given the typical use-cases for the CCG flow). Because of this, you are highly encouraged to enable + the proxy's secret encryption option - see `encrypt_client_secret_on_first_use` at the end of this file. In addition, if you are using the proxy in an environment where there is any possibility of malicious access attempts before the first valid login, pre-encrypting account entries is highly recommended. See the example script at https://github.com/simonrob/email-oauth2-proxy/issues/61#issuecomment-1259110336. - - The proxy supports the device authorisation grant (DAG) OAuth 2.0 flow (RFC 8628), which may better suit headless - systems. To use this flow, set `oauth2_flow = device`. With this flow, the proxy receives authorisation responses - directly from the service provider, so no `redirect_uri` is needed. An example account configuration is given below. + - The proxy supports the device authorisation grant (DAG) OAuth 2.0 flow, which may better suit headless systems + (currently only known to be available for Office 365 / Outlook). To use this flow, set `oauth2_flow = device`. With + this flow, the proxy receives authorisation responses directly from the service provider, so no `redirect_uri` is + needed. An example account configuration is given below. Gmail customisation: - The proxy supports the use of service accounts with Gmail for Google Workspace (note: normal Gmail accounts do not @@ -168,11 +180,13 @@ documentation = Accounts are specified using your email address as the section h full path to the JSON key file. To include the key directly, set `client_id = key`, then paste the full contents of your service account's JSON key as the value for `client_secret`, making sure all lines are indented by at least one space (so that the proxy can tell they are all part of one value). An example is given for both methods towards the - end of the sample account entries below. + end of the sample account entries below. Note that when creating the account entry here and when logging in from an + email client, the username you should use is the account you are trying to access, not the service account user + (i.e., do not use the auto-generated address that is similar to `your-user@your-project.iam.gserviceaccount.com`). - - WARNING: Please note that the same potential security issues outlined above with O365's CCG flow also apply to - the service account method: there is essentially no local access control when creating new accounts. Using a - service account with the proxy in a publicly-accessible context is not advised. You are highly encouraged to + - WARNING: Please note that the same potential security issues outlined above with O365's CCG flow also apply + to the service account method: there is essentially no local access control when creating new accounts. Using + a service account with the proxy in a publicly-accessible context is not advised. You are highly encouraged to enable the proxy's secret encryption option (see `encrypt_client_secret_on_first_use` at the end of this file) and consider pre-encrypting account entries. A sample pre-encryption script is provided for reference at https://github.com/simonrob/email-oauth2-proxy/issues/212#issuecomment-1867557029 @@ -181,7 +195,7 @@ documentation = Accounts are specified using your email address as the section h - For most configurations the default `redirect_uri` value of `http://localhost` is correct, unless you have explicitly set the OAuth 2.0 client configuration with your provider to use a different address for this purpose (e.g., redirecting via an external domain). If this is the case, you will need to manually redirect this to the - proxy. Please note that in most cases the address is indeed `http://localhost`, not `https`. + proxy. Please note that in most cases the address is indeed `http://`, not `https://`. - When using the `--local-server-auth` option you will need to either run the proxy with additional privileges to use the implicit default port 80 (e.g., via sudo) or specify a different port (and/or a different host if needed) - @@ -195,20 +209,25 @@ documentation = Accounts are specified using your email address as the section h the same address. To avoid clashes, it is recommended that each account has a unique `redirect_uri` (or `redirect_listen_address`) value, for example by using a different port for each account. -[your.office365.address@example.com] + Integration with a secrets manager: + - The proxy caches authenticated OAuth 2.0 tokens and associated metadata back into this configuration file by + default, but can alternatively be configured to use either a separate local file (via `--cache-store /path/to/file`) + or a secrets manager service for remote token storage. Currently only AWS Secrets Manager is supported. To use this + feature, set the proxy's `--cache-store` parameter to either a full AWS ARN or a secret name, prefixing the value + with `aws:` to identify its type to the proxy. If not already present, you must also install the AWS SDK for Python + (`python -m pip install boto3`) and set up authentication credentials (including a region) - see the documentation + at https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#configuration. The minimum required + permissions for the associated AWS IAM user are `secretsmanager:GetSecretValue` and `secretsmanager:PutSecretValue`. + If the named AWS Secret does not yet exist, the proxy will attempt to create it; in this case, the permission + `secretsmanager:CreateSecret` is also required. + +[your.office365.or.outlook.address@example.com] permission_url = https://login.microsoftonline.com/common/oauth2/v2.0/authorize token_url = https://login.microsoftonline.com/common/oauth2/v2.0/token oauth2_scope = https://outlook.office.com/IMAP.AccessAsUser.All https://outlook.office.com/POP.AccessAsUser.All https://outlook.office.com/SMTP.Send offline_access redirect_uri = http://localhost client_id = *** your client id here *** -client_secret = *** your client secret here *** - -[your.free.outlook.or.hotmail.address@outlook.com] -permission_url = https://login.microsoftonline.com/common/oauth2/v2.0/authorize -token_url = https://login.microsoftonline.com/common/oauth2/v2.0/token -oauth2_scope = https://outlook.office.com/IMAP.AccessAsUser.All https://outlook.office.com/POP.AccessAsUser.All https://outlook.office.com/SMTP.Send offline_access -client_id = *** your client id here - note that as you are not the administrator of Hotmail.com / Outlook.com, you will likely need to reuse an existing client ID (see the proxy's readme) *** -redirect_uri = https://localhost +client_secret = *** your client secret here (remove this entire line if a secret is not required) *** [your.email@gmail.com] permission_url = https://accounts.google.com/o/oauth2/auth @@ -240,6 +259,7 @@ token_url = https://login.microsoftonline.com/common/oauth2/v2.0/token oauth2_scope = https://outlook.office.com/IMAP.AccessAsUser.All https://outlook.office.com/POP.AccessAsUser.All https://outlook.office.com/SMTP.Send offline_access oauth2_flow = device client_id = *** your client id here *** +client_secret = *** your client secret here (remove this entire line if a secret is not required) *** [ccg.flow.configured.address@your-tenant.com] documentation = *** note: this is an advanced O365 account example; in most cases you want the version above instead *** @@ -247,7 +267,7 @@ token_url = https://login.microsoftonline.com/*** your tenant id here ***/oauth2 oauth2_scope = https://outlook.office365.com/.default oauth2_flow = client_credentials client_id = *** your client id here *** -client_secret = *** your client secret here *** +client_secret = *** your client secret here (remove this entire line if a secret is not required) *** [ropcg.flow.configured.address@your-tenant.com] documentation = *** note: this is an advanced O365 account example; in most cases you want the version above instead *** @@ -255,7 +275,7 @@ token_url = https://login.microsoftonline.com/*** your tenant id here ***/oauth2 oauth2_scope = https://outlook.office365.com/IMAP.AccessAsUser.All https://outlook.office365.com/POP.AccessAsUser.All https://outlook.office365.com/SMTP.Send offline_access oauth2_flow = password client_id = *** your client id here *** -client_secret = *** your client secret here *** +client_secret = *** your client secret here (remove this entire line if a secret is not required) *** [service.account.accessible.address@your-domain.com] documentation = *** note: this is an advanced Google account example; in most cases you want the version above instead ***