Backup restore is slow.

[martinmr]

What version of Dgraph are you using?

master

Have you tried reproducing the issue with the latest release?

yes

Steps to reproduce the issue (command/config used to run Dgraph).

Daniel was trying to restore a backup and it took a while. I also thought restoring was a bit slow while doing some testing for online restores (but not as bad). But Santo was able to restore in a reasonable time so I am not 100% if this is a problem

While investigating ,I noticed that restores are being written to the database with a class called KVLoader. The comments in the code says this struct was specifically created to deal with backups. This was done before I started working on backups so I don't know the motivation for using KVLoader instead of the write batcher or the stream writer.

Assigning to @jarifibrahim for preliminary investigation. Do you have any idea why this class is there? is it necessary or can it be replace with another loader that is hopefully more performant? If something else can replace it, I can make the change.

[jarifibrahim]

@martinmr, the KvLoader struct is used to load up unsorted data in badger. It is the fastest way to load unsorted data. Even if the data was sorted, we couldn't use the stream writer because stream writer cannot work with a DB with existing data (stream writer clears the DB before loading data).

So the batch set API (kv loader) is the fastest way to load data into badger right now. The slow writes we're seeing during restore might be related to hypermodeinc/badger#1283

[martinmr]

Ok. This could be causes by that issue since Santo didn't have any issues restoring the data. Thanks for the reply. I'll close the issue now since we are already tracking the deadlocks in hypermodeinc/badger#1283