Duplicate key violations #2429
Assignees
Labels
Comments
|
So I did some digging into this. It looks like a classic race-condition. In alchemy-dragonfly-s3: module Alchemy
module Dragonfly
module S3
class CreatePictureThumb
def self.call(variant, signature, uid)
# create the thumb before uploading
# to prevent db race conditions
thumb = nil
if variant.picture.valid?
thumb = Alchemy::PictureThumb.create!(
picture: variant.picture,
signature: signature,
uid: uid,
)
end
begin
# fetch and process the image
image = variant.image
# upload the processed image
image.store(path: uid)
rescue RuntimeError, Excon::Error => e
ErrorTracking.notification_handler.call(e)
# destroy the thumb if processing or upload fails
thumb&.destroy
end
end
end
end
end
endShouldn't this use something like https://apidock.com/rails/ActiveRecord/Relation/create_or_find_by ? |
|
We only write if the thumbnail does not exist, as you can see here Maybe there is a more thorough way of doing it? Want to try finding a fix? |
tvdeyen
added a commit
to tvdeyen/alchemy_cms
that referenced
this issue
Feb 26, 2023
In a multi concurrent application server (like Puma) it happens that a thumb might already exist for a given signature during the very short timeframe of finding it vs creating it. Using ARs [create_or_find_by!](https://github.com/rails/rails/blob/8015c2c2cf5c8718449677570f372ceb01318a32/activerecord/lib/active_record/relation.rb#L218) do avoid ActiveRecord::RecordNotUnique errors. ActiveStorage does the exact same thing to avoid concurrency issues. Closes AlchemyCMS#2429
3 tasks
tvdeyen
added a commit
to tvdeyen/alchemy_cms
that referenced
this issue
Feb 26, 2023
In a multi concurrent application server (like Puma) it happens that a thumb might already exist for a given signature during the very short timeframe of finding it vs creating it. Using ARs [create_or_find_by!](https://github.com/rails/rails/blob/8015c2c2cf5c8718449677570f372ceb01318a32/activerecord/lib/active_record/relation.rb#L218) do avoid ActiveRecord::RecordNotUnique errors. ActiveStorage does the exact same thing to avoid concurrency issues. Closes AlchemyCMS#2429
|
@dssjoblom would please give # Gemfile
gem "alchemy_cms", git: "https://github.com/tvdeyen/alchemy_cms", branch: "6.1-write-thumbs-on-writing-db"a try? |
tvdeyen
added a commit
to AlchemyCMS/alchemy-dragonfly-s3
that referenced
this issue
Feb 26, 2023
In a multi concurrent application server (like Puma) it happens that a thumb might already exist for a given signature during the very short timeframe of finding it vs creating it. Using ARs [create_or_find_by!](https://github.com/rails/rails/blob/8015c2c2cf5c8718449677570f372ceb01318a32/activerecord/lib/active_record/relation.rb#L218) do avoid ActiveRecord::RecordNotUnique errors. ActiveStorage does the exact same thing to avoid concurrency issues. Refs AlchemyCMS/alchemy_cms#2429
|
@dssjoblom and for # Gemfile
gem "alchemy-dragonfly-s3", git: "https://github.com/AlchemyCMS/alchemy-dragonfly-s3", branch: "fix-concurrency-issues" |
tvdeyen
added a commit
to AlchemyCMS/alchemy-dragonfly-s3
that referenced
this issue
Feb 27, 2023
In a multi concurrent application server (like Puma) it happens that a thumb might already exist for a given signature during the very short timeframe of finding it vs creating it. Using ARs [create_or_find_by!](https://github.com/rails/rails/blob/8015c2c2cf5c8718449677570f372ceb01318a32/activerecord/lib/active_record/relation.rb#L218) do avoid ActiveRecord::RecordNotUnique errors. ActiveStorage does the exact same thing to avoid concurrency issues. Refs AlchemyCMS/alchemy_cms#2429
dbwinger
pushed a commit
to dbwinger/alchemy_cms
that referenced
this issue
Mar 20, 2023
In a multi concurrent application server (like Puma) it happens that a thumb might already exist for a given signature during the very short timeframe of finding it vs creating it. Using ARs [create_or_find_by!](https://github.com/rails/rails/blob/8015c2c2cf5c8718449677570f372ceb01318a32/activerecord/lib/active_record/relation.rb#L218) do avoid ActiveRecord::RecordNotUnique errors. ActiveStorage does the exact same thing to avoid concurrency issues. Closes AlchemyCMS#2429
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps to reproduce
When upgrading from an older 6.0.0-pre version to 6.1.1, it seems that there are duplicate key violations while rendering the templates. Strangely enough, reloading the page makes the error go away.
Update: I get this also in the admin portion. If I upload an image and use it in an element, the preview always generates the same error. When I reload the preview, the image is there and everything works normally.
Stacktrace:
Expected behavior
There should be no duplicate key violations. I'd also not expect a GET request to cause database writes (like thumbnail creation), see #2428
Actual behavior
I get a duplicate key violation. It's also mysterious that once the same page is loaded again the error disappears.
System configuration
The text was updated successfully, but these errors were encountered: