How can I verify is the document is already loaded in SQL server using WithSqlServerMemoryDb

[peopleworks]

Hi,

How can I verify is the document is already loaded in SQL server WithSqlServerMemoryDb, because every time I run is adding again and again the same document.

Thank you

[kbeaugrand]

Hi,

Did you correctly configured the document storage?

IsDocumentReadyAsync verifies the pipeline status within the DocumentStorage, which, based on your configuration.

You can use some packages like to configure the document storage :

        memoryBuilder
            .WithAzureBlobsDocumentStorage(new AzureBlobsConfig()
            {
                ConnectionString = "...",
                Container = "..."
            })

[peopleworks]

Hi @kbeaugrand

I thought that with SQL server I could verify the status of the document, without having to save it somewhere else.

Thank you
Pedro

[dluc]

hi @peopleworks KM uses 2 storages:

1. Document Store: this is where files are uploaded at the start. This is also where the "status" of the ingestion is stored. By default KM uses a volatile storage. If you want to check if a file has already been uploaded, you should configure your solution to use a persistent storage, such as Azure Blobs, AWS S3, or local Disk.
2. Memory Store: this is where chunks and vectors are stored, when the ingestion is complete. For each upload multiple records are created or updated. In your case this is SQL Server.

Looking at the API, here's what's used:

Method	Doc Store	Mem Store
ImportDocument	x	x
ImportText	x	x
ImportWebPage	x	x
ListIndexes		x
DeleteIndex	x	x
DeleteDocument	x	x
IsDocumentReady	x
GetDocumentStatus	x
ExportFile	x
Search		x
Ask		x

[peopleworks]

Hi @dluc

Thank you so much for the explanation. ✨✨✨✨✨