Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stricter pre-release test plan #660

Closed
totaam opened this issue Aug 23, 2014 · 13 comments
Closed

stricter pre-release test plan #660

totaam opened this issue Aug 23, 2014 · 13 comments

Comments

@totaam
Copy link
Collaborator

totaam commented Aug 23, 2014

Issue migrated from trac ticket # 660

component: core | priority: major | resolution: fixed

2014-08-23 04:21:09: totaam created the issue


What we have right now does not work, we have too many tickets causing regressions (#614), regressions that don't get tested (#658, #655 / #656), then I ended up messing up #651.

We need to find all these issues before the release, not after.

I think I will cut the version from trunk roughly 2 weeks before the actual release date, make a release-candidate build, raise all outstanding tickets to release-blockers and create a tracker ticket to ensure a number of test scenarios are tested properly (platforms, OS / Python / toolkit versions, backwards compatibility, encodings, etc..) - see Testing.

@totaam
Copy link
Collaborator Author

totaam commented Aug 23, 2014

2014-08-23 06:41:47: onlyjob commented


I'd like to suggest introducing some post-build unit tests. Long time ago I expressed some concerns about automated testing in #177 so perhaps it could be useful to run at least some tests on every build. We can use xvfb-run to run tests that require X.

@totaam
Copy link
Collaborator Author

totaam commented Aug 24, 2014

2014-08-24 05:42:25: totaam commented


The problem with the tests is that very few of the problems we encountered recently would have been found using automated tests or unit tests.

That said, it can't hurt. So I am going to try to deal with #177.

@totaam
Copy link
Collaborator Author

totaam commented Mar 14, 2015

2015-03-14 03:38:01: totaam changed status from new to assigned

@totaam
Copy link
Collaborator Author

totaam commented Mar 14, 2015

2015-03-14 03:38:01: totaam changed owner from antoine to totaam

@totaam
Copy link
Collaborator Author

totaam commented Mar 14, 2015

2015-03-14 03:38:01: totaam commented


This one should have been caught: #822 (just start shadow and attach)

@totaam
Copy link
Collaborator Author

totaam commented Apr 30, 2015

2015-04-30 16:15:42: antoine changed status from assigned to new

@totaam
Copy link
Collaborator Author

totaam commented Apr 30, 2015

2015-04-30 16:15:42: antoine changed owner from totaam to afarr

@totaam
Copy link
Collaborator Author

totaam commented Apr 30, 2015

2015-04-30 16:15:42: antoine commented


The more complete integration tests will be dealt with in #847, #177 is now closed.

I have tagged 0.15.x in 9202 and I am building updated beta packages. Unless something major shows up, I will release 0.15.0 in about 5 days.

Of all the [/query?status=assigned&status=new&status=reopened&group=status&milestone=0.15 remaining 0.15 tickets], there is only one "blocker" (#770) and even this one will not hold up the release.

@afarr / testers: please take a look at [/milestone/0.15] and add to this ticket with the various end-to-end tests that you've run to verify that the new features do work, and that we haven't borken any platforms or existing functionality.
One tool that is worth using during testing is the "box painting": #760

I've done quite a bit of testing today between total power cuts.. including:

@totaam
Copy link
Collaborator Author

totaam commented May 1, 2015

2015-05-01 08:52:04: onlyjob commented


I don't want to hold release but 5 days (not even 5 working days) is too little even to schedule testing... I have most of the next week already booked so it looks like I will be testing 0.15 only after release (which is OK I just wish the notice wouldn't be so short)... Congratulations for new milestone and good luck with release, Antoine. Thanks for your much appreciated hard work on Xpra.

@totaam
Copy link
Collaborator Author

totaam commented May 5, 2015

2015-05-05 13:28:52: antoine commented


Actually, I am quite happy to delay things a bit more to ensure more tickets get closed.
Tagging 0.15 has allowed me to look at new things in the meantime, which is also very helpful in identifying bugs. And we even have one ticket re-opened: #626

@totaam
Copy link
Collaborator Author

totaam commented Jul 28, 2015

2015-07-28 07:53:53: antoine changed status from new to closed

@totaam
Copy link
Collaborator Author

totaam commented Jul 28, 2015

2015-07-28 07:53:53: antoine set resolution to fixed

@totaam
Copy link
Collaborator Author

totaam commented Jul 28, 2015

2015-07-28 07:53:53: antoine commented


Not heard back, closing - will follow up in #847.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant