Merge branch 'terminal-auth' into main

README.md

@@ -73,21 +73,21 @@ See the [sample configuration file](emailproxy.config) for further details.
 ## Optional arguments and configuration
 When starting the proxy there are several optional arguments that can be set to customise its behaviour.
 
-- `--external-auth` configures the proxy to present a clickable account authorisation link that opens in an external browser, rather than using its own built-in browser popup.
-This can be useful in situations where the script's browser window does not have access to some required authentication attribute of your typical setup.
-Once you have authorised account access using this method, paste the URL from the browser's address bar back into the script's popup window to give it access to transparently proxy your login.
+- `--no-gui` will launch the proxy without an icon, which allows it to be run as a `systemctl` service as demonstrated in [issue 2](https://github.com/simonrob/email-oauth2-proxy/issues/2#issuecomment-839713677), or fully headless as demonstrated in [various](https://github.com/michaelstepner/email-oauth2-proxy-aws) [other](https://github.com/interone-ms/email-oauth2-proxy/commits/feature/docker-build) subprojects.
+Please note that on its own this mode is only of use if you have already authorised your accounts through the proxy in GUI mode, or are importing a pre-authorised proxy configuration file from elsewhere.
+Unless this option is used in conjunction with `--external-auth` or `--local-server-auth`, accounts that have not yet been authorised (or for whatever reason require reauthorisation) will time out when authenticating, and an error will be printed to the log.
+
+- `--external-auth` configures the proxy to present an account authorisation URL to be opened in an external browser and wait for you to copy+paste the post-authorisation result.
+In GUI mode this can be useful in situations where the script's own browser window does not have access to some required authentication attribute of your typical setup.
+In no-GUI mode this option allows you to authenticate accounts without needing to start a local web server.
+Once you have authorised account access using this method, paste the final URL from your browser's address bar back into the script's popup window (GUI mode) or the terminal (no-GUI mode) to give it access to transparently proxy your login.
 You should ignore any browser error message that is shown (e.g., `localhost refused to connect`); the important part is the URL itself.
 This argument is identical to enabling external authorisation mode from the `Authorise account` submenu of the menu bar icon.
 
-- `--no-gui` will launch the proxy without an icon, which allows it to be run as a `systemctl` service as demonstrated in [issue 2](https://github.com/simonrob/email-oauth2-proxy/issues/2#issuecomment-839713677), or fully headless as demonstrated in [various](https://github.com/michaelstepner/email-oauth2-proxy-aws) [other](https://github.com/alexpdp7/email-oauth2-proxy/commit/f907e85774e8959fe4a1e5c8deaa163dfc3c573d) [subprojects](https://github.com/linka-cloud/email-oauth2-proxy/commit/67ca6b8fd0709d85480de2e3ea0af79439e6ba22).
-Please note that on its own this mode is only of use if you have already authorised your accounts through the proxy in GUI mode, or are importing a pre-authorised proxy configuration file from elsewhere.
-Unless this option is used in conjunction with `--local-server-auth`, accounts that have not yet been authorised (or for whatever reason require reauthorisation) will time out when authenticating, and an error will be printed to the log.
-
 - `--local-server-auth` instructs the proxy to print account authorisation links to its log and temporarily start an internal web server to receive responses, rather than displaying a browser popup window or relying on any GUI interaction.
 This argument is useful primarily in conjunction with the `--no-gui` option and some form of log monitoring.
 The `--external-auth` option is ignored in this mode.
-Please note also that while authentication links can be processed from anywhere, the final redirection target (i.e., a link starting with your account's `redirect_uri` value) must be accessed from the machine hosting the proxy itself, rather than any remote client.
-See [various](https://github.com/simonrob/email-oauth2-proxy/issues/33) [issue](https://github.com/simonrob/email-oauth2-proxy/issues/42) [discussions](https://github.com/simonrob/email-oauth2-proxy/issues/59) for why this is the case.
+Please note also that while authentication links can be processed from anywhere, the final redirection target (i.e., a link starting with your account's `redirect_uri` value) must be accessed from the machine hosting the proxy itself, so that the local server can actually receive the authorisation result.
 
 - `--config-file` allows you to specify the location of a [configuration file](emailproxy.config) that the proxy should load.
 If this argument is not provided, the proxy will look for `emailproxy.config` in the same directory as the script itself.

emailproxy.py

@@ -6,7 +6,7 @@
 __author__ = 'Simon Robinson'
 __copyright__ = 'Copyright (c) 2022 Simon Robinson'
 __license__ = 'Apache 2.0'
-__version__ = '2022-12-14'  # ISO 8601 (YYYY-MM-DD)
+__version__ = '2022-12-23'  # ISO 8601 (YYYY-MM-DD)
 
 import argparse
 import base64
@@ -60,16 +60,18 @@
 
 # by default the proxy is a GUI application with a menu bar/taskbar icon, but it is also useful in 'headless' contexts
 # where not having to install GUI-only requirements can be helpful - see the proxy's readme and requirements-no-gui.txt
-no_gui_parser = argparse.ArgumentParser()
+no_gui_parser = argparse.ArgumentParser(add_help=False)
 no_gui_parser.add_argument('--no-gui', action='store_true')
-if not no_gui_parser.parse_known_args()[0].no_gui:
+no_gui_parser.add_argument('--external-auth', action='store_true')
+no_gui_args = no_gui_parser.parse_known_args()[0]
+if not no_gui_args.no_gui:
     import pkg_resources  # from setuptools - used to check package versions and choose compatible methods
     import pystray  # the menu bar/taskbar GUI
     import timeago  # the last authenticated activity hint
     from PIL import Image, ImageDraw, ImageFont  # draw the menu bar icon from the TTF font stored in APP_ICON
 
     # noinspection PyPackageRequirements
-    import webview  # the popup authentication window (in default and `--external-auth` modes only)
+    import webview  # the popup authentication window (in default and GUI `--external-auth` modes only)
 
     # for macOS-specific functionality
     if sys.platform == 'darwin':
@@ -89,7 +91,18 @@ class Icon:
     class AppKit:
         class NSObject:
             pass
+
+
+    if no_gui_args.external_auth:
+        try:
+            # prompt_toolkit is a recent dependency addition that is only required in no-GUI external authorisation
+            # mode, but may not be present if only the proxy script itself has been updated
+            import prompt_toolkit
+        except ModuleNotFoundError:
+            sys.exit('Unable to load prompt_toolkit, which is a requirement when using `--external-auth` in `--no-gui` '
+                     'mode. Please run `python -m pip install -r requirements-no-gui.txt`')
 del no_gui_parser
+del no_gui_args
 
 APP_NAME = 'Email OAuth 2.0 Proxy'
 APP_SHORT_NAME = 'emailproxy'
@@ -625,10 +638,10 @@ def get_oauth2_authorisation_code(permission_url, redirect_uri, redirect_listen_
                 return False, '%s is shutting down' % APP_NAME
 
             if data['permission_url'] == permission_url and data['username'] == username:  # a response meant for us
-                # to improve no-GUI mode we also support the use of a local server to receive the OAuth redirection
-                # (note: not enabled by default because no-GUI mode is typically unattended, but useful in some cases)
-                if 'expired' in data and data['expired']:  # local server auth wsgi request error or failure
-                    return False, 'Local server auth request failed'
+                # to improve no-GUI mode we also support the use of a local redirection receiver server or terminal
+                # entry to authenticate; this result is a timeout, wsgi request error/failure, or terminal auth ctrl+c
+                if 'expired' in data and data['expired']:
+                    return False, 'No-GUI authorisation request failed or timed out'
 
                 if 'local_server_auth' in data:
                     threading.Thread(target=OAuth2Helper.start_redirection_receiver_server, args=(data,),
@@ -1885,12 +1898,16 @@ class App:
     def __init__(self):
         global CONFIG_FILE_PATH
         parser = argparse.ArgumentParser(description=APP_NAME)
-        parser.add_argument('--external-auth', action='store_true', help='handle authorisation via an external browser '
-                                                                         'rather than this script\'s own popup window')
         parser.add_argument('--no-gui', action='store_true', help='start the proxy without a menu bar icon (note: '
                                                                   'account authorisation requests will fail unless a '
                                                                   'pre-authorised configuration file is used, or you '
-                                                                  'enable `--local-server-auth` and monitor output)')
+                                                                  'enable `--external-auth` or `--local-server-auth` '
+                                                                  'and monitor log output)')
+        parser.add_argument('--external-auth', action='store_true', help='handle authorisation externally: rather than '
+                                                                         'intercepting `redirect_uri`, the proxy will '
+                                                                         'wait for you to paste the result into either '
+                                                                         'its popup window (GUI-mode) or the terminal '
+                                                                         '(no-GUI mode; requires `prompt_toolkit`)')
         parser.add_argument('--local-server-auth', action='store_true', help='handle authorisation by printing request '
                                                                              'URLs to the log and starting a local web '
                                                                              'server on demand to receive responses')
@@ -2096,7 +2113,7 @@ def create_config_menu(self):
                 items.append(pystray.MenuItem('    Refresh activity data', self.icon.update_menu))
         items.append(pystray.Menu.SEPARATOR)
 
-        items.append(pystray.MenuItem('Edit configuration file...', self.edit_config))
+        items.append(pystray.MenuItem('Edit configuration file...', lambda: self.system_open(CONFIG_FILE_PATH)))
 
         # asyncore sockets on Linux have a shutdown delay (the time.sleep() call in asyncore.poll), which means we can't
         # easily reload the server configuration without exiting the script and relying on daemon threads to be stopped
@@ -2132,16 +2149,16 @@ def get_last_activity(account):
         return '    %s (%s)' % (account, formatted_sync_time)
 
     @staticmethod
-    def edit_config():
+    def system_open(path):
         AppConfig.save()  # so we are always editing the most recent version of the file
         if sys.platform == 'darwin':
-            result = os.system('open %s' % CONFIG_FILE_PATH)
+            result = subprocess.call(['open', path])
             if result != 0:  # no default editor found for this file type; open as a text file
-                os.system('open -t %s' % CONFIG_FILE_PATH)
+                subprocess.call(['open', '-t', path])
         elif sys.platform == 'win32':
-            os.startfile(CONFIG_FILE_PATH)
+            os.startfile(path)
         elif sys.platform.startswith('linux'):
-            os.system('xdg-open %s' % CONFIG_FILE_PATH)
+            subprocess.call(['xdg-open', path])
         else:
             pass  # nothing we can do
 
@@ -2247,7 +2264,7 @@ def authorisation_window_loaded(self):
             # respond to both the original request and any duplicates in the list
             completed_request = None
             for request in self.authorisation_requests[:]:  # iterate over a copy; remove from original
-                if OAuth2Helper.match_redirect_uri(request['redirect_uri'], url) and request['username'] == username:
+                if request['username'] == username and OAuth2Helper.match_redirect_uri(request['redirect_uri'], url):
                     Log.info('Returning authorisation request result for', request['username'])
                     RESPONSE_QUEUE.put(
                         {'permission_url': request['permission_url'], 'response_url': url, 'username': username})
@@ -2433,7 +2450,7 @@ def notify(self, title, text):
                     for replacement in (('\\', '\\\\'), ('"', '\\"')):  # osascript approach requires sanitisation
                         text = text.replace(*replacement)
                         title = title.replace(*replacement)
-                    os.system('osascript -e \'display notification "%s" with title "%s"\'' % (text, title))
+                    subprocess.call(['osascript', '-e', 'display notification "%s" with title "%s"' % (text, title)])
 
             elif self.icon.HAS_NOTIFICATION:
                 self.icon.remove_notification()
@@ -2528,6 +2545,67 @@ def load_and_start_servers(self, icon=None, reload=True):
         Log.info('Initialised', APP_NAME, '- listening for authentication requests. Connect your email client to begin')
         return True
 
+    @staticmethod
+    def terminal_external_auth_input(prompt_session, prompt_stop_event, data):
+        with contextlib.suppress(Exception):  # cancel any other prompts; thrown if there are none to cancel
+            prompt_toolkit.application.current.get_app().exit(exception=EOFError)
+            time.sleep(1)  # seems to be needed to allow prompt_toolkit to clean up between prompts
+
+        # noinspection PyUnresolvedReferences
+        with prompt_toolkit.patch_stdout.patch_stdout():
+            open_time = 0
+            response_url = None
+            Log.info('Please visit the following URL to authenticate account %s: %s' % (
+                data['username'], data['permission_url']))
+            # noinspection PyUnresolvedReferences
+            style = prompt_toolkit.styles.Style.from_dict({'url': 'underline'})
+            prompt = [('', '\nCopy+paste or press [↵ Return] to visit the following URL and authenticate account %s: ' %
+                       data['username']), ('class:url', data['permission_url']), ('', ' then paste here the full '),
+                      ('', 'post-authentication URL from the browser\'s address bar (it should start with %s): ' %
+                       data['redirect_uri'])]
+            while True:
+                try:
+                    response_url = prompt_session.prompt(prompt, style=style)
+                except (KeyboardInterrupt, EOFError):
+                    break
+                if not response_url:
+                    if time.time() - open_time > 1:  # don't open many windows on key repeats
+                        App.system_open(data['permission_url'])
+                        open_time = time.time()
+                else:
+                    break
+
+            prompt_stop_event.set()  # cancel the timeout thread
+
+            result = {'permission_url': data['permission_url'], 'username': data['username']}
+            if response_url:
+                Log.debug('No-GUI external auth mode: returning response', response_url)
+                result['response_url'] = response_url
+            else:
+                Log.debug('No-GUI external auth mode: no response provided; cancelling authorisation request')
+                result['expired'] = True
+            RESPONSE_QUEUE.put(result)
+
+    @staticmethod
+    def terminal_external_auth_timeout(prompt_session, prompt_stop_event):
+        prompt_time = 0
+        while prompt_time < AUTHENTICATION_TIMEOUT and not prompt_stop_event.is_set():
+            time.sleep(1)
+            prompt_time += 1
+
+        if not prompt_stop_event.is_set():
+            with contextlib.suppress(Exception):  # thrown if the prompt session has already exited
+                prompt_session.app.exit(exception=EOFError)
+                time.sleep(1)  # seems to be needed to allow prompt_toolkit to clean up between prompts
+
+    def terminal_external_auth_prompt(self, data):
+        prompt_session = prompt_toolkit.PromptSession()
+        prompt_stop_event = threading.Event()
+        threading.Thread(target=self.terminal_external_auth_input, args=(prompt_session, prompt_stop_event, data),
+                         daemon=True).start()
+        threading.Thread(target=self.terminal_external_auth_timeout, args=(prompt_session, prompt_stop_event),
+                         daemon=True).start()
+
     def post_create(self, icon):
         if EXITING:
             return  # to handle launch in pystray 'dummy' mode without --no-gui option (partial initialisation failure)
@@ -2544,12 +2622,20 @@ def post_create(self, icon):
                 break
             if not data['expired']:
                 Log.info('Authorisation request received for', data['username'],
-                         '(local server auth mode)' if self.args.local_server_auth else '(interactive mode)')
+                         '(local server auth mode)' if self.args.local_server_auth else '(external auth mode)' if
+                         self.args.external_auth else '(interactive mode)')
                 if self.args.local_server_auth:
+                    self.notify(APP_NAME, 'Local server auth mode: please authorise a request for account %s' %
+                                data['username'])
                     data['local_server_auth'] = True
                     RESPONSE_QUEUE.put(data)  # local server auth is handled by the client/server connections
-                    self.notify(APP_NAME,
-                                'Local server auth mode: please authorise a request for account %s' % data['username'])
+                elif self.args.external_auth and self.args.no_gui:
+                    if sys.stdin.isatty():
+                        self.notify(APP_NAME, 'No-GUI external auth mode: please authorise a request for account '
+                                              '%s' % data['username'])
+                        self.terminal_external_auth_prompt(data)
+                    else:
+                        Log.error('Not running interactively; unable to handle no-GUI external auth request')
                 elif icon:
                     self.authorisation_requests.append(data)
                     icon.update_menu()  # force refresh the menu

requirements-no-gui.txt

@@ -8,3 +8,6 @@ pyasyncore; python_version >= '3.12'
 
 # macOS only: output to unified logging
 pyoslog>=0.3.0; sys_platform == 'darwin'
+
+# required only if using the --external-auth option in --no-gui mode
+prompt_toolkit