RDF4j with multiple readers, 1 writer getting "SAIL is already locked by: [some other ip]"

[cgokey]

I'm trying to setup RDF4J and using version 5 under tomcat with a native store in AWS. I'm using a fargate service where I have multiple readers and 1 writer. I'm obviously not understanding something about the transactional model because as soon as I issue a sparql update and the writer updates the database, none of my readers can access it, even after the transaction has finished -- I'm getting a "SAIL is already locked by: [some other ip]" error message. Is this not supported through the normal /rdf4j-server/server endpoint provided by rdf4j?

I was hoping that as long as I ONLY had 1 writer making changes to the EFS file system (thus not making concurrent updates) , I could still read with the other processes.

[hmottestad]

Is Fargate running up multiple RDF4J instances that interact with the files on disk? And you've set a rule that only allows Fargate to run up one RDF4J instance for writing, but multiple for reading?

[cgokey]

Yes, that is correct. Only one dedicated instance will write to the disk, the other instance will ONLY read.

[hmottestad]

There is a special file used as a lock that is created when you initialize the sail, which will likely happen once you try to perform an operation on the sail. The files for the database were never intended to be shared between multiple instances of RDF4J. Even if you were to bypass the lock you would get inconsistent data when your other sails read while the writer is modifying the files on disk.

[kenwenzel]

It may work with the LmdbStore as long as autoGrow is not used. But this mode of operation is completely untested.

[cgokey]

Thanks, yeah I believe back in the days of working in lucene, a delta file was created for incremental additions/updates, so could have multiple readers, so was hoping it worked similar… But I might be remembering things wrong. Looking at some other rdf dbs your page links to, might look into halyard next then… Chris

…

On Sun, Feb 2, 2025 at 1:49 PM kenwenzel ***@***.***> wrote: It may work with the LmdbStore as long as autoGrow is not used. But this mode of operation is completely untested. — Reply to this email directly, view it on GitHub <#5244 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFIEXTSPEXCUA5AQL7LUPVD2NZSFHAVCNFSM6AAAAABWJYGHVKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMBTGM3TMOA> . You are receiving this because you authored the thread.Message ID: ***@***.***>