Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling of compressed timeline files on startup [BF-1391] #552

Merged
merged 4 commits into from
Aug 19, 2022

Conversation

alexole
Copy link
Contributor

@alexole alexole commented Aug 18, 2022

  • Fix hardcoded WAL file type in the upload event
  • Fix for flaky test_pause_on_disk_full
  • Fix for flaky test_surviving_pg_receivewal_hickup

@alexole alexole requested review from a team as code owners August 18, 2022 08:52
Removed hardcoded WAL file type from upload event.
@alexole alexole force-pushed the alex-fix-hardcoded-wal-type branch from c794f30 to d8a0894 Compare August 18, 2022 08:57
@alexole alexole changed the title Fix handling of compressed timeline files on startup Fix handling of compressed timeline files on startup [BF-1391] Aug 18, 2022
@codecov
Copy link

codecov bot commented Aug 18, 2022

Codecov Report

Merging #552 (7e713c6) into release-2.2.2 (f0d3221) will increase coverage by 0.29%.
The diff coverage is 100.00%.

@@                Coverage Diff                @@
##           release-2.2.2     #552      +/-   ##
=================================================
+ Coverage          89.31%   89.60%   +0.29%     
=================================================
  Files                 29       29              
  Lines               4493     4494       +1     
=================================================
+ Hits                4013     4027      +14     
+ Misses               480      467      -13     
Impacted Files Coverage Δ
pghoard/pghoard.py 84.00% <100.00%> (+1.37%) ⬆️
pghoard/common.py 93.04% <0.00%> (-1.31%) ⬇️
pghoard/fetcher.py 97.87% <0.00%> (+2.12%) ⬆️
pghoard/walreceiver.py 97.85% <0.00%> (+4.28%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@alexole
Copy link
Contributor Author

alexole commented Aug 18, 2022

I've restarted the pipeline to check if there is any correlation with this fix and flaky test.

@alexole
Copy link
Contributor Author

alexole commented Aug 18, 2022

Feel free to merge it after the run, unfortunately this does not fix a flaky test :)

@alexole
Copy link
Contributor Author

alexole commented Aug 18, 2022

I've added one workaround for flaky receivewal hickup test, it should be OK for now, we don't usually recover failed WAL receiver anyway, it requires manual intervention if WAL receiver failed for some reason, so the restart of pghoard will fix the problem with non complete partial files. For the test I just call a function, which is usually called after pghoard start, so it truncates the incomplete files.

@alexole alexole marked this pull request as draft August 18, 2022 15:12
When test stops receivexlogs thread, the subprocess of receivexlog
after being killed might leave incomplete .partial files, which make
the restarted receivexlog stop working. Usually those incomplete files
are truncated by pghoard after restart, so this workaround should just
make test less flaky. Ideally the receivexlogs thread should do that
cleanup in case of this kind of problems.
@alexole alexole force-pushed the alex-fix-hardcoded-wal-type branch 2 times, most recently from dd9fc09 to d4adeff Compare August 19, 2022 09:45
@kmichel-aiven kmichel-aiven merged commit 78ce24f into release-2.2.2 Aug 19, 2022
@kmichel-aiven kmichel-aiven deleted the alex-fix-hardcoded-wal-type branch August 19, 2022 11:48
@alexole alexole mentioned this pull request Aug 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants