Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loom does not perform any long-term testing #65

Open
prototypicalpro opened this issue Jan 30, 2020 · 0 comments
Open

Loom does not perform any long-term testing #65

prototypicalpro opened this issue Jan 30, 2020 · 0 comments
Labels
enhancement New feature or request

Comments

@prototypicalpro
Copy link
Member

Is your feature request related to a problem? Please describe.
When using Loom in a research context, it is assumed that if a feature is integrated into Loom, that same feature can be deployed and will continue to function for an indefinite period of time. Currently Loom does not perform any long-term testing of integrated features, and as a result cannot provide any such guarantee. While this lack of verification may have aligned with the original goals of Loom (education about sensors), it has become apparent that a long-term guarantee will be necessary if Loom continues to move towards being a pure-research platform.

When speaking with researchers, we found a reasonable time-frame for a remote deployment to be ~6 months. Not all functionality in Loom is worth verifying, but at least a minimum viable products worth should be verified to function without human interaction for 6 months.

Describe the solution you'd like
I propose that every month or major feature change, a known subset of Loom functionality (such as the SmartRock project with a battery) be built and flashed (and labeled) with the current stable Loom, and placed into a cabinet. Each week (or automatically) this build of Loom should be checked to verify it is behaving correctly. The next month/major feature, this process should be repeated, placing the new build next to the one from the previous month, and so on. Once a build of Loom passes the six month mark, that build of Loom is considered "proven" and can be removed from the cabinet. Once a build of Loom is placed into the cabinet, it's code in source control and it's attached peripherals must not be modified, otherwise jeopardizing the test results.

With this system, after the initial six months a build of Loom will be considered "proven" once every month, allowing projects in active deployments to still receive features on a longer timescale. If a project uses a feature from Loom that has not been "proven", we will make no guarantee of the feature working beyond a few days.

@prototypicalpro prototypicalpro added the enhancement New feature or request label Jan 30, 2020
@Cogswatch Cogswatch added this to the Refactor the third milestone Apr 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants