Fix alignment in chunkedContentCoder #147
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
An unaligned atomic bug was unfortunately introduced in #119 because the `bytesWritten` field was placed at the end of the `chunkedContentCoder` struct. This places this after the bytes.Buffers and the bool causing it to be misaligned. The ideal placement of this variable is not entirely clear but placing it before the progresiveWrite bool should help. An alternative would be to just place this atomic field at the top of the struct then there would be no risk of it becoming misaligned in future. I moved a few things around to reduce the size of the struct too but it could be possible to adjust things a little more to make the struct a little smaller. Signed-off-by: Andrew Thornton <[email protected]>
|
Thanks you very much @zeripath for raising this bug. I'm not entirely sure about the specifics of the bug over here. However, it looks like the panic message that the user has posted in the issue go-gitea/gitea#21869 shows that the atomic.AddUint64() is the cause of panic. Now, I also observe that the version of the zapx and bleve used is 15.3.5 and 2.3.4 respectively. These releases had the atomic operations in incrementBytesWritten method (which is why I'm assuming its failing). The upcoming release of bleve (2.3.6) will have the zapx tag 15.3.7 which avoids the atomic operations altogether. My thinking is that this issue won't occur when you use 2.3.6. However, if the issue still persists in that release, please go ahead create an issue or a PR (if you've verified a fix for the same) and add me as a reviewer for it. Thank you! |
abhinavdangeti
requested review from
abhinavdangeti,
metonymic-smokey and
moshaad7
abhinavdangeti
approved these changes
Nov 29, 2022
Thejas-bhat
approved these changes
Nov 30, 2022
zeripath
added a commit
to zeripath/gitea
that referenced
this pull request
Dec 4, 2022
There is an unaligned atomic field in zapx 15.3.5 which should have been fixed in blevesearch/zapx#147 This bug causes issues on 32bit builds. Update bleve and zapx to account for this. Fix go-gitea#21957 Signed-off-by: Andrew Thornton <[email protected]>
cj123
added a commit
to cj123/zapx
that referenced
this pull request
Feb 17, 2023
Following the fix in blevesearch#147, a 'panic: unaligned 64-bit atomic operation' is still occurring on some platforms. As noted in blevesearch#147, this commit implements the alternative fix by placing the bytesWritten field at the top of the struct. Per the Golang sync/atomic documentation, on ARM, 386 and 32-bit MIPS systems, this is the reliable way to prevent these issues. Signed-off-by: Callum Jones <[email protected]>
cj123
added a commit
to cj123/zapx
that referenced
this pull request
Feb 17, 2023
Following the fix in blevesearch#147, a 'panic: unaligned 64-bit atomic operation' is still occurring on some platforms. As noted in blevesearch#147, this commit implements the alternative fix by placing the bytesWritten field at the top of the struct. Per the Golang sync/atomic documentation, on ARM, 386 and 32-bit MIPS systems, this is the reliable way to prevent these issues. Signed-off-by: Callum Jones <[email protected]>
abhinavdangeti
pushed a commit
that referenced
this pull request
Feb 21, 2023
…ms. (#148) Following the fix in #147, a 'panic: unaligned 64-bit atomic operation' is still occurring on some platforms. As noted in #147, this commit implements the alternative fix by placing the bytesWritten field at the top of the struct. Per the Golang sync/atomic documentation, on ARM, 386 and 32-bit MIPS systems, this is the reliable way to prevent these issues. Signed-off-by: Callum Jones <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An unaligned atomic bug was unfortunately introduced in #119 because the
bytesWrittenfield was placed at the end of thechunkedContentCoderstruct.This places this after the bytes.Buffers and the bool causing it to be misaligned.
The ideal placement of this variable is not entirely clear but placing it before the progresiveWrite bool should help.
An alternative would be to just place this atomic field at the top of the struct then there would be no risk of it becoming misaligned in future.
I moved a few things around to reduce the size of the struct too but it could be possible to adjust things a little more to make the struct a little smaller.
Signed-off-by: Andrew Thornton [email protected]