Add a "live test" to template CI

[freakboy3742]

What is the problem or limitation you are having?

The CI configuration for the briefcase template currently validates the create and build commands, but doesn't do any validation that the app actually runs. It's possible to successfully create an app, but be unable to run that app.

Describe the solution you'd like

As a final step in CI, we should actually run the app.

This will require modifying the test apps so that they will stop (or that logging will stop streaming). Adding a background task that sleeps a second, then calls app.exit() (or equivalent) and outputs a logging sentinel (the same format used for test termination) should be sufficient.

Describe alternatives you've considered

We could just run the app in --test mode; however, that will require rebuilding the app in test mode, which will take a while.

Additional context

No response

[rmartin16]

I've been trying to flesh out an implementation for this.

Unlike the app template repos that contain Briefcase projects already, CI must generate the Briefcase projects for this repo to actually test it. Given this, I'm trying to figure out the best way to inject one of the methods you've mentioned to stop the app once it starts running with briefcase run.

To do with while still rolling out this template, I'm imagining a new conditional flag in the input dict for cookiecutter that will include a bit of code to stop the app once it has successfully started. Additionally, this means that the briefcase new command will need a new switch (maybe something like --auto-exit) that will pass this new flag to cookiecutter when rolling the project out.

I think this seems less than ideal to me, though....am I missing something perhaps?

[freakboy3742]

I've been trying to flesh out an implementation for this.
...
I think this seems less than ideal to me, though....am I missing something perhaps?

You're not missing anything; I might have been a little enthusiastic flagging this as "good first issue".

I'm not wild about adding an --auto-exit flag - that seems very "niche purpose".

However, I do have another idea (or, at least, the start of an idea). The requirements for this are actually pretty close to the requirements for beeware/briefcase#1288. In that case, what we have is a need for a "base Toga template, but with different app content". In the Positron case, we need the body logic for a "launch a web server and a web view" app. In the testing case we need "hit exit after 2 seconds" body logic. However, both are examples of "inject body logic into an existing app template".

So - Toga would provide a "vanilla" and "positron" option; we could provide a Pyside/PPB/PyGame plugin (with the hope those communities would eventually take ownership of their respective plugins), and Briefcase would provide an "app test" plugin. We'd only install the Briefcase app test plugin for our own testing purposes; we probably wouldn't even publish it to PyPI. Any other GUI toolkit that wanted to plug into Briefcase could provide its own plugin.

Does that approach make any sense to you?

[rmartin16]

I think it makes sense....but want to make sure I understand the scope of the solution you're imagining.

Right now, the list of "GUI toolkits" to choose from while running briefcase new is defined as a part of Briefcase itself. Your solution would instead enumerate the available GUI toolkits based on well-defined entry points registered by other installed packages (aka plugins).

These packages would effectively be responsible for defining a template to roll out as the app project directory (and effectively supplanting this repo entirely); a simple implementation would be to just expose a URL briefcase could use to pass to cookiecutter. Going further, I suppose, Briefcase could even leverage such a plugin package to control the template for the Create command.

Am I on track with what you're imagining?

I haven't done much with entry points so I'm curious to experiment with them. A cursory review of their docs certainly suggests this was an intended use-case in their design and a widely-used strategy for plugin support.

[freakboy3742]

I think it makes sense....but want to make sure I understand the scope of the solution you're imagining.

Right now, the list of "GUI toolkits" to choose from while running briefcase new is defined as a part of Briefcase itself. Your solution would instead enumerate the available GUI toolkits based on well-defined entry points registered by other installed packages (aka plugins).

These packages would effectively be responsible for defining a template to roll out as the app project directory (and effectively supplanting this repo entirely); a simple implementation would be to just expose a URL briefcase could use to pass to cookiecutter. Going further, I suppose, Briefcase could even leverage such a plugin package to control the template for the Create command.

Am I on track with what you're imagining?

Details notwithstanding - yes. Exactly where the line is between "cookiecutter template" and "content we can inject into that template" isn't entirely clear to me at this point in the design process; but broadly speaking, I think we're on the same page.

I haven't done much with entry points so I'm curious to experiment with them. A cursory review of their docs certainly suggests this was an intended use-case in their design and a widely-used strategy for plugin support.

Yeah - we're already using entry points internally to Briefcase - each platform and backend is an entry point. Nobody has taken us up on the offer yet, but at least in theory, you could define an entirely different platform for Briefcase without our involvement. It's a really powerful mechansim, and the implementaiton is disturbingly straightforward from a developer's point of view.

[rmartin16]

"good first issue"

• Package Automation with Briefcase in CI briefcase#1562
• Add support for specifying overrides in the command line for new projects briefcase#1552
• Introduce package for Briefcase Automation briefcase#1549
• Allow access to GPU and sound briefcase-linux-flatpak-template#31
• Use a platform-specific cache key that rotates each month .github#71
• Run apps created in app-build-verify .github#69
• Remove testing apps since CI will generate them now briefcase-windows-VisualStudio-template#26
• Remove testing apps since CI will generate them now briefcase-windows-app-template#26
• Remove testing apps since CI will generate them now briefcase-web-static-template#13
• Remove testing apps since CI will generate them now briefcase-macOS-Xcode-template#34
• Remove testing apps since CI will generate them now briefcase-macOS-app-template#29
• Remove testing apps since CI will generate them now briefcase-linux-system-template#17
• Remove testing apps since CI will generate them now briefcase-linux-flatpak-template#30
• Remove testing apps since CI will generate them now; disable PySide2 testing briefcase-linux-appimage-template#45
• Remove testing apps since CI will generate them now briefcase-iOS-Xcode-template#29
• Remove testing apps since CI will generate them now briefcase-android-gradle-template#73
• Add reusable actions to create and/or verify a new project  .github#68
• Wait for app to exit normally after log streaming briefcase#1541
• Do not strip libraries from PyPI packages briefcase-linux-flatpak-template#29
• Support template branch input app-build-verify workflow .github#64
• Generalize template for GUI framework plugins #82
• Add support for third-party GUI framework plugins briefcase#1524

😉 jk...glad I made it out the other side of this rabbit hole 😎

[freakboy3742]

Yeah... might have misjudged that one a bit... 🤣