Proxy python app frameworks

[sharon-wang]

Description

• Addresses Integrate Python web apps with Workbench #4769 (as much as possible for now)

Framework Support Summary (based on limited testing)

Framework	Positron Server Web	Positron Desktop	Positron on Workbench	Notes
Dash	✅ yes	✅ yes	✅ yes-ish	Workbench extension issue to skip Dash framework handling to avoid conflicting with Positron's Dash framework handling
Fastapi	✅ yes	✅ yes	🛑 no	Seems to be a conflict between the way we run a fastapi app and that Workbench is setting UVICORN_ROOT_PATH
Flask	✅ yes	✅ yes	✅ yes	Workbench: works when including the code snippet from Workbench docs.
Gradio	✅ yes-ish	✅ yes-ish	✅ yes-ish	Working when the following dependency versions are used: gradio==3.3.1 fastapi==0.85.2 httpx==0.24.1. See gradio-app/gradio#9529 for more information on why more recent versions don't work.
Streamlit	✅ yes	✅ yes	🟨 partially	Not working on Workbench when SSL is enabled
Shiny	✅ yes	✅ yes	✅ yes

Implementation Notes

Positron Proxy

CC @softwarenerd & @jmcphers

• Add new command positronProxy.startPendingProxyServer which starts a new proxy server, but doesn't set up the middleware right away. The command will return the externalUri of the proxy server (this is the same as the url shown in the Viewer), the proxyPath (which some app frameworks use as the urlPrefix) and finishProxySetup() which the command-caller will invoke, passing the targetOrigin (the actual app url) so that we can finish setting up the middleware to proxy requests from the externalUri (app proxy uri) to the targetOrigin (actual app uri).
• Refactor the proxy code so we can set up a proxy in multiple steps, not only all-at-once
  • existing callers of startProxyServer() will continue to use that method which still does the all-at-once proxy setup, by calling startNewProxyServer() and finishProxySetup() in succession
  • new option to call startNewProxyServer() standalone, which will only start up the server and then call finishProxySetup() later, once the targetOrigin is known
• Move the HTML-rewriting code to the util file

Positron Run App

CC @seeM

• expand the localUrlRegex to include the path after the url port <HOST>:<PORT>/<PATH>
• execute the command positronProxy.startPendingProxyServer before setting up terminal options or the debug configuration, so we can start the proxy server and get the urlPrefix
• before previewing the url in the Viewer, finish the proxy setup by passing the localUri to the positron proxy via proxyInfo.finishProxySetup(localUri.toString()), so that the proxy middleware is set up
• remove port from types and test files as it is unused
• increase timeout for waiting for the app url to show in the terminal (in particular, Gradio would take a bit longer on Workbench and we would timeout)

Python Extension

CC @seeM

• remove port from the webAppCommands and related test files which is unused
• update the framework-specific app arguments and environment variables to work with our proxy server situation

Custom resolveExternalUri

CC @melissa-barca

• update the resolvedUri to inherit the protocol used by the main window as the uri's scheme, so we can upgrade to https if the main window is running in a secure context
• NOTE: this will need to be upstreamed

QA Notes

This PR involves refactoring the positron-proxy, which is used by Help, Interactive Plots and the Viewer. We should not see any regression in these types of proxied content.

Positron Server Web and Desktop should be working across all app frameworks in:

• https://github.com/posit-dev/qa-example-content/tree/main/workspaces/python_apps
  • When running the Gradio app, you will need to install these versions gradio==3.3.1 fastapi==0.85.2 httpx==0.24.1. If you can get it working with a more recent combination of versions, please let me know!
• https://github.com/posit-dev/qa-example-content/tree/main/workspaces/shiny-py-example
• https://github.com/posit-dev/qa-example-content/tree/main/workspaces/streamlit-py-example

Positron on Workbench:

• dash works if the generated .env file is deleted before running. Once this issue is complete, this extra step won't be needed.
• fastapi does not work. The current hunch is that Workbench setting UVICORN_ROOT_PATH at session start is interfering with how we run the fastapi app with uvicorn. The UVICORN_ROOT_PATH is set based on the default fastapi port 8000, however I think we want the proxied app url as the root path, which is provided by the Positron Proxy. But, when setting --root-path in the fastapi app command to the proxied root path, it does not work in Server Web or Desktop. TBD if this resolves the issue on Workbench. More investigation needed.
• flask works as long as you include the code snippet from Workbench docs.
• gradio works when these versions are installed gradio==3.3.1 fastapi==0.85.2 httpx==0.24.1. There's an issue in newer versions of Gradio when the app is run via a proxy.
• streamlit does not work when SSL is set up. More investigation needed.
• shiny no notes!

[seeM]

This is great @sharon-wang! positron-python and positron-run-app code changes look good. I don't know how the ui.py type error snuck through to main, but I think it is fixed in #4966.

EDIT: Oh it looks like the positron-python unit tests also need to be updated.

[sharon-wang]

Thanks @seeM! We're seeing all green checkmarks now :) And I've copied over the types from positron-run-app to positron-python 👍

[jmcphers]

I think someone with more experience in these frameworks should test the changes in detail, but code LGTM modulo a couple of exception handlers. Splitting the proxy creation into two pieces is a cool way to deal with the chicken-and-egg problem.

[sharon-wang]

Thanks for the review @jmcphers!

@softwarenerd and I did a live review and we opted to "modernize" the promise handling and I've added some corresponding exception handling.

[softwarenerd]

LG!

[jonvanausdeln]

On Windows Desktop, a simple dash app (found here) gets stuck on the "Loading".

Here is some out put in the dev console

It should just be "Hello world"

[sharon-wang]

Integration tests are now passing ✅
Windows full suite now passing ✅

The last full run failed on one test, but it is another instance of #4863 based on the screenshot in the run artifacts.

---

Thanks everyone for the reviews and testing! 🙌