Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testfailure fix for MacOS (DirtyStateEditorSupportIntegrationTest) #3275

Conversation

mehmet-karaman
Copy link
Contributor

This patch adds a delay (waitForReconciler-method call before checking the document content) and it should fix the synchronization problem which causes flaky tests under mac os.

@mehmet-karaman
Copy link
Contributor Author

mehmet-karaman commented Dec 2, 2024

@LorenzoBettini I hope this helps for fixing the test problem. Do you have time to check if its ok now?

Copy link

github-actions bot commented Dec 2, 2024

Test Results

  6 460 files  ±0    6 460 suites  ±0   3h 20m 33s ⏱️ + 10m 2s
 43 238 tests ±0   42 654 ✅ ±0    584 💤 ±0  0 ❌ ±0 
170 249 runs  +5  167 904 ✅ ±0  2 336 💤 ±0  7 ❌ +3  2 🔥 +2 

Results for commit c2a7058. ± Comparison against base commit 5efcdea.

@LorenzoBettini LorenzoBettini added this to the Release_2.38 milestone Dec 4, 2024
@LorenzoBettini
Copy link
Contributor

@mehmet-karaman looks like it fixes the problem, and for sure fixes the arguments passed to assert methods.
Let's merge it and see whether that test has been fixed for good.

@LorenzoBettini LorenzoBettini merged commit 8190316 into eclipse-xtext:main Dec 4, 2024
9 checks passed
@mehmet-karaman mehmet-karaman deleted the Fix_flaky_test_macos_for_refresh_editor_on_sync branch December 4, 2024 09:07
@LorenzoBettini
Copy link
Contributor

https://github.com/eclipse-xtext/xtext/runs/33937371956

@mehmet-karaman it falls again..

@mehmet-karaman
Copy link
Contributor Author

Need to reproduce this myself and fix this instead of trial and error approach..

@memin20
Copy link

memin20 commented Dec 6, 2024

image

Yesterday I've spend the whole day with experimenting on mac. It runs successful in a junit plugin test, but fails on maven build. I've added several debug outputs tried to understand the state of the object and when the change happens, but it seems that there is no way without trying to understand with debugging in tycho surefire.

@LorenzoBettini
Copy link
Contributor

If it fails in the Maven build constantly, maybe an explicit refresh?

@memin20
Copy link

memin20 commented Dec 6, 2024

explicit Refresh of the IFile didn't helped.. It still fails.. There must be an Eclipse Job, which is scheduled but not executed..

@LorenzoBettini
Copy link
Contributor

Does it systematically fail?

@memin20
Copy link

memin20 commented Dec 6, 2024

Mostly they fail, but sometimes they are succesful..

image

The Debug output:
Checking Eclipse Jobs
Flushing ui.
more work todo...
Animation start
Flushing ui.
Decoration Calculation
Flushing ui.
Reporting encoding changes.
Flushing ui.
Initializing Quick Diff
Flushing ui.
XtextReconcilerJob
Flushing ui.
Open Blocked Dialog
Flushing ui.
Building
Flushing ui.
Flushing content description cache.
Flushing ui.
Update Job
Flushing ui.
Compacting resource model
Flushing ui.
Workbench Auto-Save Job
Flushing ui.
Periodic workspace save.
Ending of Eclipse Jobs checks

This is the constellation where it is succesfull.. I will compare it with the failing test..

@LorenzoBettini
Copy link
Contributor

Just a few thoughts:

  • you don't wait for each job, if I understand correctly
  • maybe that loop should be done right before the last assert

@LorenzoBettini
Copy link
Contributor

Maybe unrelated, but please when posting snippets use the markdown syntax instead of posting screenshots of code ;)

@LorenzoBettini
Copy link
Contributor

  • you don't wait for each job

By "wait" I mean "join".

@memin20
Copy link

memin20 commented Dec 6, 2024

It causes a deadlock, because some of these jobs are waiting for the ui thread.. So I have to do a kind of polling. I will remove that break check that again.

@memin20
Copy link

memin20 commented Dec 6, 2024

I've executed the maven build several times and it seems the polling helped, the test is always successful and not flaky anymore.

The asserts have to wait until all scheduled jobs are done (except the periodic jobs). I am going to clean up the code, execute that build several time and if its still ok.. i will create a new PR :)

Thank your for your patience @LorenzoBettini

@memin20
Copy link

memin20 commented Dec 6, 2024

The solution still fails sometimes, depending on the state of my mac.. I will try to run the junit plugin test with the same arguments as the maven build is using.. i must be able to debug this..

@LorenzoBettini
Copy link
Contributor

@memin20 please don't waste too much time on that. Given all your experiments it might also be a lost cause: there must be severe synchronization issues in the platform itself and we don't seem to be able to do anything about that..

@mehmet-karaman
Copy link
Contributor Author

Should we disable this test for macos?

@LorenzoBettini
Copy link
Contributor

I'd agree. @szarnekow wdyt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants