Attribute update operators (inc, push, etc.)

[fgalan]

This issue is about implementing advanced update operators for attributes, beyond the current "set value". For instance "increment attribute value by 1" or "remove the last element of an attribute array".

Taking into account we use MongoDB as DB, a good strategy will be to support all the update operators supported by MongoDB, described at https://docs.mongodb.com/manual/reference/operator/update/. We should do it in a smart way, so if MongoDB adds a new operator in a new release, it would be easy to support it also in the CB.

[fgalan]

The following operators will be taken into account:

• $inc: increase numeric atrribute
• $max: max between the current value and the intended update for numeric attributes
• $min: min between the current value and the intended update for numeric attributes
• $mul: multiplies by number attribute
• $push: add item to array attribute
• $addToSet: add item to array attribute if not already in the array
• $pull: remove elements from array attribute (by particular values or filter)
• $pullAll: remove a list of elements from array attribute
• $bit: bitwise operations on numeric attributes

---

Examples follow:

$inc:

POST /v2/entities/E/attrs
{
  "A": {
    "value": { "$inc": 1 },
    "type": "Number"
  }
}

($min, $max and $mul are pretty similar to $inc)

$push:

POST /v2/entities/E/attrs
{
  "A": {
    "value": { "$push": "foo" },
    "type": "StructuredValue"
  }
}

($addToSet is pretty similar to $push)

$pull with a particular value:

POST /v2/entities/E/attrs
{
  "A": {
    "value": { "$pull": 24 },
    "type": "StructuredValue"
  }
}

$pull with a filter:

POST /v2/entities/E/attrs
{
  "A": {
    "value": { "$pull": {"$gt": 20} },
    "type": "StructuredValue"
  }
}

$pullAll:

POST /v2/entities/E/attrs
{
  "A": {
    "value": { "$pullAll": [4, 8, 17] },
    "type": "StructuredValue"
  }
}

$bit:

POST /v2/entities/E/attrs
{
  "A": {
    "value": { "$bit": { "xor": 5 } },
    "type": "StructuredValue"
  }
}

[fgalan]

Implementation notes:

From the point of view of the update, it seems pretty simple: just to "pass through" the operation to the corresponding MongoDB operator at update() time.

Things get tricky when notification enter into play, as with the current approach we would need to do the operation in memory in parallel to the modification done at MongoDB. This is easy for some operations (eg. $inc) but not for others (eg. $addToSet). Thus, if update operators are used then, instead of plain update() we should use findAndModify() with new: true and use the returned value as entity in memory for notifications. There is a small penalty in performance, as update() is more efficient than findAndModify(), but I think is feasible.

However, weird things may occur if several CB are in parallel. For instance, CB A and B, attribute being X=1 in MongoDB. Two updates occur, one with $inc:1 at CB A and other $inc:2 in CB B. Depending on timing several things may occur:

• CB A executes update first
  • The notification sent by CB A arrives first to receiver: the receiver gets the data in the following order: X=2, X=4 (correct, as the last notification contains the final state)
  • The notification sent by CB B arrives first to receiver: the receiver gets the data in the following order: X=4, X=2 (weird)
• CB B executes update first
  • The notification sent by CB A arrives first to receiver: X=4, X=3 (weird)
  • The notification sent by CB B arrives first to receiver: X=3, X=4 (correct, as the last notification contains the final state)

This should be warmed at documentation.

As alternative we can disable notification of these attributes or, at least, allow disabling them by parametrization (?options=skipComputedNotifications).

[fgalan]

Implementation would be split in (at least) two PRs:

• First PR without taking into account notifications
• Second PR with the notifications part

[fgalan]

First PR #3814

[fgalan]

The $bit operator is not going to be implemented in the context of this issue due to it would be hard with the current implementation. A new specific issue has been created about it: #3938, providing specific details.

[fgalan]

Second PR #3954

[fgalan]

Also related PR #4009 with two new operators ($set and $unset)