Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement document deletion #464

Closed
1 of 3 tasks
cozitive opened this issue Feb 10, 2023 · 0 comments · Fixed by #540 or #558
Closed
1 of 3 tasks

Implement document deletion #464

cozitive opened this issue Feb 10, 2023 · 0 comments · Fixed by #540 or #558
Labels
enhancement 🌟 New feature or request sdk ⚒️

Comments

@cozitive
Copy link
Contributor

cozitive commented Feb 10, 2023

What would you like to be added:
Document deletion is one of the essential features. It should be implemented to save resources.

Why is this needed:
Without deletion, accumulated documents may waste storage resources.


Delivering delete request as a form of Operation
Existing Operation struct can be reused for sending a delete request. If a client send a delete operation, RPC server will deliver it to service server like other normal operations, and Clients/Documents/Packs server in service level will communicate with backend server for deletion.

Broadcasting document deletion to attached clients
Clients attached to a deleted document should know that the document is to be deleted. Packs server must notify it via PushPull process.

Deletion policy
Data cannot be recovered after deleting a document in DB, so deferred deletion is necessary for safety. When a user removes a document, all related data will be pending for deletion. To-be-deleted objects will just be indicated by removed flag, and will be permanently deleted after a pre-defined waiting period by housekeeping service. The waiting period would be determined by an admin.

Data to be deleted:
To completely delete a document, all data related to the document in the server and DB should be eliminated.

struct DB column DB related key
DocInfo documents
ClientDocInfo (in ClientInfo) clients documents.${docID}.${*}
ChangeInfo changes doc_id
SnapshotInfo snapshots doc_id
SyncedSeqInfo syncedseqs doc_id

Tasks
To successfully implement this API, we need to seperate our tasks and implement them step by step.

  • 1. Implement basic document deletion API with simply setting RemovedAt date in docInfo
  • 3. Implement housekeeping background process to physically remove document(is_removed = true) and its relevant data.
  • 2. Determine whether to include removed documents in ListDocuments API used by Dashboard and CLI(Admin)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment