Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performing batch transactions #922

Closed
JeromeDeLeon opened this issue Sep 13, 2019 · 21 comments · Fixed by #2265
Closed

Performing batch transactions #922

JeromeDeLeon opened this issue Sep 13, 2019 · 21 comments · Fixed by #2265
Labels
bounty:$100 Bounty applies for fixing this issue (Parse Bounty Program) state:released-alpha Released as alpha version type:feature New feature or improvement of existing feature

Comments

@JeromeDeLeon
Copy link
Contributor

Is your feature request related to a problem? Please describe.
This was already supported in the PServer, but in JS-SDK, it limits us to only perform either save or delete but not both.

Describe the solution you'd like
It would be better if we could constructr a batch operation like:

await Parse.Object.performTransaction(
  [
    { method: "PUT", objects: [PObject] },
    { method: "DELETE, object: PObject },
    { method: "POST", object: [PObject, PObject] }
  ],
  { transaction: true, useMasterKey: true }
);

and performs that sequentially.

Describe alternatives you've considered

Additional context
Also, using relations when doing batch operation is not allowed because you cannot add something to relation when it is new. I don't know if there's already a solution to this but my current implementation is to use forkJoin([DeleteObjects(), SaveObjects()]) using RxJS.

@davimacedo davimacedo added type:feature New feature or improvement of existing feature discussion labels Sep 13, 2019
@davimacedo
Copy link
Member

davimacedo commented Sep 13, 2019

I like the idea of having some way to send a batch from the JS SDK but I am not sure if the proposed API is the best one. I'd probably go with something like this:

const transaction = Parse.Object.createTransaction();
someObject.destroy({ useMasterKey: true, transaction });
someObject.save({ transaction });
try {
  transaction.commit();
} catch (e) {
  transaction.abort();
}

@JeromeDeLeon is this something that you'd be willed to tackle?

@JeromeDeLeon
Copy link
Contributor Author

That would be nice too. The reason I suggested that is because I want to use what is already existed in the codebase of PS. We could make use of createTransactionalSession from the PS to await on it for session request but how could we incorporate session AKA transactionToken for every write or even read operations?

@JeromeDeLeon
Copy link
Contributor Author

Wait. I just saw that PS already incorporated the transactionalSession. Is there a way for PS to send transactionalSession to SDK? same with commit and abort? something like:
GET "/transaction", POST "/transaction/(commit | abort)

@davimacedo
Copy link
Member

No.. there is no way to do it. Since the Parse Server can run in multiple processes at the same time, there is no warranty that two requests will go to the same process. In the case they go to different processes, you will not be able to use the same MongoDB session.

So, the solution is actually use the /batch endpoint. I mean... when the JS SDK call transaction.commit();, we will have to put all requests together and send in a single request through the /batch endpoint. Got it?

@JeromeDeLeon
Copy link
Contributor Author

JeromeDeLeon commented Sep 14, 2019

// create unique ID or something
const transaction = Parse.Obejct.createTransaction();

// These operations will not be run until commit() is called
// maybe save it to array of operations
someObject.destroy({ useMasterKey: true, transaction });
someObject.save({ transaction });
 
try {
// since /batch already have abort operations no need to manually abort
transaction.commit();
} catch(e) {
console.log(e);
}

What I have in mind is that when I call createTransaction, transaction will receive an object containing the token and commit and for each operations, we need to associate that with the transaction and inside, it will not be save but instead put in an array of requests to be collect later when commit is called?

@davimacedo
Copy link
Member

Yes. That's the idea. We will probably need the abort in the client side as well to revert the object states.

@mfkenson
Copy link

Currently with the JS SDK (master, 2.11.0), there is no way to pass parameter "transaction" to the Parse Server. (Correct me if I am wrong) so the transactional stuff (which have been done on Parse Server) does not work with the JS SDK.

My plan of attack is to make a PR which includes:

  1. Adding an option in saveAll, destroyAll and pass it to Parse Server

Does this make sense? @davimacedo

@mfkenson
Copy link

// create unique ID or something
const transaction = Parse.Obejct.createTransaction();

// These operations will not be run until commit() is called
// maybe save it to array of operations
someObject.destroy({ useMasterKey: true, transaction });
someObject.save({ transaction });
 
try {
// since /batch already have abort operations no need to manually abort
transaction.commit();
} catch(e) {
console.log(e);
}

What I have in mind is that when I call createTransaction, transaction will receive an object containing the token and commit and for each operations, we need to associate that with the transaction and inside, it will not be save but instead put in an array of requests to be collect later when commit is called?

I am trying to use a similar approach on server-side Parse.Cloud function! Getting the database object in cloud function and manually invoking createTransactionalSession at the beginning of my cloud function. And then invoking

  • commitTransactionalSession
  • abortTransactionalSession
    after some CRUD operations.

I am stuck now and I have no idea why this does not work as expected...Perhaps the cloud function and the CRUD handlers they are not pointing to the same database object..?

@davimacedo
Copy link
Member

I think that your original idea of changing saveAll, destroyAll makes sense, will be a lot easier and a very good first step. Why don't you start with them?

@mfkenson
Copy link

@davimacedo Yes I did give a try (#1090 ). Hope this is a good start to contribute.

'changing saveAll, destroyAll' in the JS SDK should be the top priority to support what have been done on Parse Server batch endpoint.

I might get back to the cloud function approach later.

@davimacedo
Copy link
Member

Great job. It is a good start for sure. I've just left a comment over there.

@itsnitigya
Copy link

Any updates on this issue?

@jonas-db
Copy link

jonas-db commented May 1, 2021

Is there any update on this issue or a workaround? We have several use cases where transactions would be necessary (eg update 2 objects at the same time or rollback)

@danielmalmros
Copy link

I am also interested in an update🙂

@Kingtous
Copy link

interested.

@mtrezza
Copy link
Member

mtrezza commented Aug 19, 2021

If anyone interested in picking up @mfkenson's #1090?

@JoeProgram
Copy link

I'm interested in this too.

@gtzinos
Copy link

gtzinos commented May 2, 2023

Hello everyone, do you have any updates on this? By the way, it sounds like an amazing feature!

@alljinx
Copy link

alljinx commented May 5, 2023

I'm interested in this too.

@mtrezza mtrezza added bounty:$50 Bounty applies for fixing this issue (Parse Bounty Program) bounty:$100 Bounty applies for fixing this issue (Parse Bounty Program) and removed bounty:$50 Bounty applies for fixing this issue (Parse Bounty Program) labels May 7, 2023
@mtrezza
Copy link
Member

mtrezza commented May 7, 2023

Added a bounty to this feature due to high demand.

@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 5.3.0-alpha.4

@parseplatformorg parseplatformorg added the state:released-alpha Released as alpha version label Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bounty:$100 Bounty applies for fixing this issue (Parse Bounty Program) state:released-alpha Released as alpha version type:feature New feature or improvement of existing feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.