Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Containerize Playwright end-to-end tests to resolve screenshot comparison issues caused by OS differences #1114

Open
ljmotta opened this issue Apr 23, 2024 · 0 comments · May be fixed by apache/incubator-kie-tools#2866
Labels
area:tests Related to automated tests of any kind area:tools Issues affecting Apache KIE tooling projects

Comments

@ljmotta
Copy link

ljmotta commented Apr 23, 2024

Currently, some Playwright tests use screenshot comparisons, which are prone to failure due to operating system differences, such as variations in fonts and UI rendering [1][2][3]. These inconsistencies lead to unreliable test results across different environments.

To address this issue, we need to standardize how tests are executed within the kie-tools monorepo. By leveraging containers, we can create a stable, consistent environment for running tests, independent of the host OS. This approach ensures reproducibility and resolves screenshot comparison failures.

[1] microsoft/playwright#8161
[2] microsoft/playwright#16104
[3] microsoft/playwright#6907

@ljmotta ljmotta added area:tools Issues affecting Apache KIE tooling projects area:tests Related to automated tests of any kind labels Apr 23, 2024
@ljmotta ljmotta changed the title Screenshots comparison from end-to-end tests don't work on OSs different than Ubuntu Server 22.04 Containerize Playwright end-to-end tests to resolve screenshot comparison issues caused by OS differences Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:tests Related to automated tests of any kind area:tools Issues affecting Apache KIE tooling projects
Projects
Status: 📋 Backlog
1 participant