Replacing the operator $where with aggregation pipeline #4148

anupamme · 2022-06-03T07:51:55Z

Is your feature request related to a problem / use case? Please describe.
I'm part of Azure Cosmos DB team and we are working to make our Mongo API work with Fiware-Orion without any issues. One of these issues is: the use of Mongo DB operator $where [1] in the codebase [2]. Since $where has been deprecated by Mongo DB and would not be supported in future releases (it is presently supported though), we recommend to replace $where with the aggregation pipeline [3].

https://www.mongodb.com/docs/manual/reference/operator/query/where/
fiware-orion/src/lib/mongoBackend/MongoCommonUpdate.cpp

Line 1257 in e1b079c

static void fill_idPtypeNP
https://www.mongodb.com/docs/manual/core/aggregation-pipeline/

Describe the solution you'd like
We recommend moving away from $where operator and use aggregation pipeline instead. The codebase at [2] to be replaced with using aggregation pipeline.

Describe alternatives you've considered
$where is a deprecated feature in MongoDB so considering an alternate is going to be a lot more work as we will supporting a deprecated feature in the future. So we did not consider any other alternative.

Describe why you need this feature

To be able to offer Azure Cosmos DB API as an fully-functional option to the users of Fiware-Orion.
Customers of Fiware-Orion are coming to us saying that Cosmos DB Mongo API does not support $where operator. We would like to resolve this issue.

Additional information
Add any other information, diagrams or screenshots about the feature request here.

Do you have the intention to implement the solution

Yes, I have the knowledge to implement this new feature.
Yes, but I will need help.
No, I do not have the skills.

fgalan · 2022-06-03T08:17:58Z

Thank you very much for your feedback! We started to use $where operation in a far away past (when MongoDB 2.4/2.6 was the newest MongoDB version :) and it was not possible to solve the problem with the aggregation pipeline stages at that moment. Now the proper aggregation stages exists for that, so I think is a good idea what you propose.

Please, go ahead with the pull request fixing this. I can provide some degree of support (best effort mode) if you need it.

By the way, your issue report is pretty comprehensive. However, it would be great if you could cite (from MongoDB official documentation) the place in which they say $where operator is deprecated.

anupamme · 2022-06-20T04:34:08Z

Here are some of the links which mention that $where is either deprecated, not usable or poses security threat:

"Starting in MongoDB 4.4, $where no longer supports the deprecated BSON type JavaScript code with scope (BSON type 15). The $where operator only supports BSON type String (BSON type 2) or BSON type JavaScript (BSON type 13). The use of BSON type JavaScript with scope for $where has been deprecated since MongoDB 4.2.1"
Javascript is not enabled in some of the Atlas instances e.g. M0 free clusters and M2/M5 shared clusters do not support server-side JavaScript.

EXAMPLE

$where and map-reduce are not supported.

For security reasons users prefer to disable Javascript (server side scripting), also recommended by MongoDB (https://www.mongodb.com/docs/manual/reference/operator/query/where/#javascript-enablement)

In contrast, aggregation pipeline is supported in all instances, has no security issues in using it and also is not deprecated. Hope this answers your query.

nsrivastava9 · 2022-07-26T10:17:17Z

Hi @fgalan , Can you please provide the sample document /document structure and the query that is being executed to fetch the document as per fiware-orion/src/lib/mongoBackend/MongoCommonUpdate.cpp

anupamme · 2022-07-27T09:36:54Z

Hello @fgalan

Reaching out as we (@nsrivastava9 and I) need some high level guidance for making this change.

What we want to do is replace $where with aggregation pipeline [https://www.mongodb.com/docs/manual/core/aggregation-pipeline/]

So effectively we would like to make $where an alias for aggregation pipeline, meaning whenever the user calls @where we end up calling command aggregate with appropriate arguments.

In code we are looking into MongoCommonUpdate.cpp [https://github.com/telefonicaid/fiware-orion/blob/e1b079c8843a759b3d53d458eb2034dc8a9793e3/src/lib/mongoBackend/MongoCommonUpdate.cpp#L1257]

But could you give some guidance in terms what changes in the code need to be made for this?

fgalan · 2022-08-05T10:59:57Z

Hi @fgalan , Can you please provide the sample document /document structure and the query that is being executed to fetch the document as per fiware-orion/src/lib/mongoBackend/MongoCommonUpdate.cpp

For Orion logs (contextBroker -fg -logLevel DEBUG -t 100 -noCache -logForHumans) I think this is the query you are asking for:

DEBUG@2022-08-05T10:56:17.271Z  MongoCommonUpdate.cpp[1538]: query() in 'orion.csubs' collection: '{ "$or" : [ { "entities.id" : "Room1", "$or" : [ { "entities.type" : "Room" }, { "entities.type" : { "$exists" : false } } ], "entities.isPattern" : "false", "entities.isTypePattern" : { "$ne" : true }, "expiration" : { "$gt" : 1659696977 }, "status" : { "$ne" : "inactive" } }, { "entities.isPattern" : "true", "entities.isTypePattern" : { "$ne" : true }, "expiration" : { "$gt" : 1659696977 }, "status" : { "$ne" : "inactive" }, "$where" : { "$code" : "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"true\" && !this.entities[i].isTypePattern && (this.entities[i].type == \"Room\" || this.entities[i].type == \"\" || !(\"type\" in this.entities[i])) && \"Room1\".match(this.entities[i].id)) {return true; }}return false; }" } }, { "entities.isPattern" : "false", "entities.isTypePattern" : true, "expiration" : { "$gt" : 1659696977 }, "status" : { "$ne" : "inactive" }, "$where" : { "$code" : "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"false\" && this.entities[i].isTypePattern && this.entities[i].id == \"Room1\" && \"Room\".match(this.entities[i].type)) {return true; }}return false; }" } }, { "entities.isPattern" : "true", "entities.isTypePattern" : true, "expiration" : { "$gt" : 1659696977 }, "status" : { "$ne" : "inactive" }, "$where" : { "$code" : "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"true\" && this.entities[i].isTypePattern && \"Room1\".match(this.entities[i].id) && \"Room\".match(this.entities[i].type)) {return true; }}return false; }" } } ], "servicePath" : { "$in" : [ "/", "/#" ] } }

After a JSON pretty print ;) looks this way:

{
	"$or": [{
		"entities.id": "Room1",
		"$or": [{
			"entities.type": "Room"
		}, {
			"entities.type": {
				"$exists": false
			}
		}],
		"entities.isPattern": "false",
		"entities.isTypePattern": {
			"$ne": true
		},
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		}
	}, {
		"entities.isPattern": "true",
		"entities.isTypePattern": {
			"$ne": true
		},
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		},
		"$where": {
			"$code": "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"true\" && !this.entities[i].isTypePattern && (this.entities[i].type == \"Room\" || this.entities[i].type == \"\" || !(\"type\" in this.entities[i])) && \"Room1\".match(this.entities[i].id)) {return true; }}return false; }"
		}
	}, {
		"entities.isPattern": "false",
		"entities.isTypePattern": true,
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		},
		"$where": {
			"$code": "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"false\" && this.entities[i].isTypePattern && this.entities[i].id == \"Room1\" && \"Room\".match(this.entities[i].type)) {return true; }}return false; }"
		}
	}, {
		"entities.isPattern": "true",
		"entities.isTypePattern": true,
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		},
		"$where": {
			"$code": "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"true\" && this.entities[i].isTypePattern && \"Room1\".match(this.entities[i].id) && \"Room\".match(this.entities[i].type)) {return true; }}return false; }"
		}
	}],
	"servicePath": {
		"$in": ["/", "/#"]
	}
}

fgalan · 2022-08-05T11:06:33Z

Especifically regarding idTtypeNP we have:

DEBUG@2022-08-05T10:56:17.270Z  MongoCommonUpdate.cpp[1282]: idTtypeNP function: function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == "true" && !this.entities[i].isTypePattern && (this.entities[i].type == "Room" || this.entities[i].type == "" || !("type" in this.entities[i])) && "Room1".match(this.entities[i].id)) {return true; }}return false; }

fgalan · 2022-08-05T11:31:56Z

What we want to do is replace $where with aggregation pipeline [https://www.mongodb.com/docs/manual/core/aggregation-pipeline/]

So effectively we would like to make $where an alias for aggregation pipeline, meaning whenever the user calls @where we end up calling command aggregate with appropriate arguments.

Not sure of fully understand what you mean by alias... in general in MongoDB, you cannot "blindly" replace usage of "$where in find()" by "$where in aggregation pipeline" as they are different operations (i.e. find() is a operation to query a database collection while aggregate() is a wrapper for a more complex command in MongoDB).

Having said that, note that "$where" is currently used in only one place in the Orion code: addTriggeredSubscriptions_noCache() and its sub-functions (i.e. fill_idNPtypeNP(), fill_idPtypeNP(), fill_idNPtypeP()andfill_idPtypeP()`)

Thus, under my understanding, I think it is just a matter of replace addTriggeredSubscriptions_noCache() by an equivalent function which uses the aggregation framework instead of find() with $where. In other words, to use orion::collectionAggregate() instead of orion::collectionQuery() in addTriggeredSubscriptions_noCache().

The structured of the current query has been posted in a previous comment. I hope it helps in the re-implementation of addTriggeredSubscriptions_noCache().

anupamme · 2022-08-10T11:15:09Z

Hi @fgalan

Thanks for your detailed message. It helps clarify how we can re-implement addTriggeredSubscriptions_noCache(). I have a couple of follow up questions.

The way I understand is this that in its current implementation, it builds bgP->query by calling 4 sub-functions (i.e. fill_idNPtypeNP(), fill_idPtypeNP(), fill_idNPtypeP() and fill_idPtypeP()) and then uses this to call orion::collectionQuery .

Instead now we will have to build a pipeline just like it is built in mongoQueryTypes.cpp and then call orion::collectionAggregate with this pipeline object. Is my understanding correct?

My follow up question is that after using orion::collectionAggregate, how will the structure of the current query (JSON structure) change? e.g. this snippet

"$where": {"$code": "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"true\" && !this.entities[i].isTypePattern && (this.entities[i].type == \"Room\" || this.entities[i].type == \"\" || !(\"type\" in this.entities[i])) && \"Room1\".match(this.entities[i].id)) {return true; }}return false; }"}

Now the key $where be replaced by what?
The value part will be replaced by the contents of the pipeline we will construct, is that correct?

Also, how will the structure of the rest of the query (or json) will change?

I was looking for a couple of examples on how to construct the pipeline object (which can be passed as an argument to orion::collectionAggregate), are there any examples in the codebase?

fgalan · 2022-08-11T08:34:42Z

Hi @fgalan

Thanks for your detailed message. It helps clarify how we can re-implement addTriggeredSubscriptions_noCache(). I have a couple of follow up questions.

The way I understand is this that in its current implementation, it builds bgP->query by calling 4 sub-functions (i.e. fill_idNPtypeNP(), fill_idPtypeNP(), fill_idNPtypeP() and fill_idPtypeP()) and then uses this to call orion::collectionQuery .

Instead now we will have to build a pipeline just like it is built in mongoQueryTypes.cpp and then call orion::collectionAggregate with this pipeline object. Is my understanding correct?

I think your understanding is correct.

My follow up question is that after using orion::collectionAggregate, how will the structure of the current query (JSON structure) change? e.g. this snippet

"$where": {"$code": "function(){for (var i=0; i < this.entities.length; i++) {if (this.entities[i].isPattern == \"true\" && !this.entities[i].isTypePattern && (this.entities[i].type == \"Room\" || this.entities[i].type == \"\" || !(\"type\" in this.entities[i])) && \"Room1\".match(this.entities[i].id)) {return true; }}return false; }"}

Now the key $where be replaced by what?

The value part will be replaced by the contents of the pipeline we will construct, is that correct?

Also, how will the structure of the rest of the query (or json) will change?

Not sure what you are asking here... :) Maybe you are asking for and explanation on what the function() in the $where clause does? The purpose of that function() is to to a "reverse regex" operation. That it, in the query we have the entity (id "Room1" and type "Room" in the example above) and we want to retrieve the documents in the csubs collection which match it.

This way is more readable:

function(){
    for (var i=0; i < this.entities.length; i++) {
	if (this.entities[i].isPattern == \"true\" && 
	    !this.entities[i].isTypePattern        && 
		(this.entities[i].type == \"Room\" || this.entities[i].type == \"\" || !(\"type\" in this.entities[i])) && 
		\"Room1\".match(this.entities[i].id)) {
	  return true; }
	}
	return false; 
  }

It works in the following way. this represents a document in the csubs collection (described here: https://fiware-orion.readthedocs.io/en/master/admin/database_model.html#csubs-collection)

A loop (for) iterates on the entities field. A positive match (return true) happens if at least one element in the array accomplish the following:
- isPattern is true
- it doesn't have a isTypePattern field (not sure why this is needed in this particular example, but probably the entities with isTypePattern are taken into account in other of the $where clauses of the complete query)
- type is "Room" or the type is empty or element doesn't have type field (the second and third possibilities are "legacy" cases from NGSIv1, now in NGSIv2 all entities have a type)
- id regex matches "Room1"

More on this approach to do "reverse regex" in this post at SOF

I was looking for a couple of examples on how to construct the pipeline object (which can be passed as an argument to orion::collectionAggregate), are there any examples in the codebase?

If you look for orion::collectionAggregate() in the code base, we can find 3 examples. The way in which the pipeline is build could be a bit "obscure" (due to the "nested append" approach) but you will find description comments with the pipeline structure. In particular:

fiware-orion/src/lib/mongoBackend/mongoQueryTypes.cpp

Line 295 in 9b066b5

* db.entities.aggregate([{$match: { "_id.servicePath": /.../ } },
fiware-orion/src/lib/mongoBackend/mongoQueryTypes.cpp

Line 451 in 9b066b5

* db.entities.aggregate([ {$match: { "_id.servicePath": /.../ } },
fiware-orion/src/lib/mongoBackend/mongoQueryTypes.cpp

Line 295 in 9b066b5

* db.entities.aggregate([{$match: { "_id.servicePath": /.../ } },

fgalan · 2022-08-11T08:46:23Z

Having a look to the cited post at SOF maybe a simpler solution to aggregation pipeline is possible :)

In particular, to follow this advice and use $expr with $regexMatch to implement the "reverse regex".

This example is closer to what we need: https://mongoplayground.net/p/ibWLonDkxFT

fgalan · 2022-08-11T08:56:22Z

This example is closer to what we need: https://mongoplayground.net/p/ibWLonDkxFT

Trying to get even more closer, maybe there are some problems... see https://stackoverflow.com/questions/73317911/expr-with-regexmatch-doesnt-work-when-the-pattern-is-inside-an-array

fgalan · 2022-08-11T12:51:12Z

This example is closer to what we need: https://mongoplayground.net/p/ibWLonDkxFT

Trying to get even more closer, maybe there are some problems... see https://stackoverflow.com/questions/73317911/expr-with-regexmatch-doesnt-work-when-the-pattern-is-inside-an-array

A solution to the problem was provided (https://stackoverflow.com/a/73318486/1485926):

db.collection.find({
  "$expr": {
    "$anyElementTrue": {
      "$map": {
        "input": "$patterns.pattern",
        "in": {
          "$regexMatch": {
            "input": "Room1",
            "regex": "$$this",
            "options": "i"
          }
        }
      }
    }
  }
})

Example: https://mongoplayground.net/p/2bb8aC45kJS

anupamme · 2022-08-16T10:46:39Z

Hi @fgalan

Thanks for these pointers. So now what we'd be doing is replacing the current "reverse-regex" implementation which uses $where operator to $expr as you pointed out in your last comment.

In your earlier comment (#4148 (comment)) you mentioned the compiled query looks like this (after json pretty print)

{
	"$or": [{
		"entities.id": "Room1",
		"$or": [{
			"entities.type": "Room"
		}, {
			"entities.type": {
				"$exists": false
			}
		}],
		"entities.isPattern": "false",
		"entities.isTypePattern": {
			"$ne": true
		},
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		}
	}, {
		"entities.isPattern": "true",
		"entities.isTypePattern": {
			"$ne": true
		},
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		},
		"$expr": {
            "$anyElementTrue": {
              "$map": {
                "input": "",
                "in": {
                  "$regexMatch": {
                    "input": "",
                    "regex": "$$this",
                    "options": "i"
                  }
                }
              }
            }
          }
	}, {
		"entities.isPattern": "false",
		"entities.isTypePattern": true,
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		},
		"$expr": {
            "$anyElementTrue": {
              "$map": {
                "input": "",
                "in": {
                  "$regexMatch": {
                    "input": "",
                    "regex": "$$this",
                    "options": "i"
                  }
                }
              }
            }
          }
	}, {
		"entities.isPattern": "true",
		"entities.isTypePattern": true,
		"expiration": {
			"$gt": 1659696977
		},
		"status": {
			"$ne": "inactive"
		},
		"$expr": {
            "$anyElementTrue": {
              "$map": {
                "input": "",
                "in": {
                  "$regexMatch": {
                    "input": "",
                    "regex": "$$this",
                    "options": "i"
                  }
                }
              }
            }
          }
	}],
	"servicePath": {
		"$in": ["/", "/#"]
	}
}

Notice that I have replaced all the 3 occurrences of $where with $expr. The value of $expr is something we are working on currently so we will plug in those values when we are ready.

So, wanted to check is this JSON structure correct?

If yes, then follow up question is how do we replace the line (

fiware-orion/src/lib/mongoBackend/MongoCommonUpdate.cpp

Line 1294 in e1b079c

bgP->boPNP.appendCode("$where", bgP->functionIdPtypeNP);

)

bgP->boPNP.appendCode("$where", bgP->functionIdPtypeNP);

We are not doing appendCode instead only an expression so should we do something like this:

bgP->boPNP.append("$expr", <reverse_regex_expression_json_object>);

where reverse_regex_expression_json_object is correct implementation which has structure like this:

"$anyElementTrue": {
              "$map": {
                "input": "",
                "in": {
                  "$regexMatch": {
                    "input": "",
                    "regex": "$$this",
                    "options": "i"
                  }
                }
              }
            }

Just wanted to make sure my current understanding is correct, so let me know your comments.

fgalan · 2022-08-17T09:48:42Z

Notice that I have replaced all the 3 occurrences of $where with $expr. The value of $expr is something we are working on currently so we will plug in those values when we are ready.

So, wanted to check is this JSON structure correct?

The structure seems to be correct in the sense it replaces $where by $expr. I have noticed that input fields are empty string "" which wouldn't be correct, but I guess this is due you are still working on it.

To see if your implementation is correct, create a pull request with the change. We have a lot of regression tests for subscriptions in "noCache" way (when this functionality is used) so if your pull requests pass the tests that would be ok.

If yes, then follow up question is how do we replace the line (

fiware-orion/src/lib/mongoBackend/MongoCommonUpdate.cpp

Line 1294 in e1b079c

bgP->boPNP.appendCode("$where", bgP->functionIdPtypeNP);

)

bgP->boPNP.appendCode("$where", bgP->functionIdPtypeNP);

We are not doing appendCode instead only an expression so should we do something like this:

bgP->boPNP.append("$expr", <reverse_regex_expression_json_object>);

where reverse_regex_expression_json_object is correct implementation which has structure like this:
"$anyElementTrue": {
              "$map": {
                "input": "",
                "in": {
                  "$regexMatch": {
                    "input": "",
                    "regex": "$$this",
                    "options": "i"
                  }
                }
              }
            }
Just wanted to make sure my current understanding is correct, so let me know your comments.

It makes sense to change appendCode() by just append() in this case, so I understand it's ok.

anupamme · 2022-09-05T08:39:56Z

Hi @fgalan

We have written the expression for the 4 conditions in the current code and the new expression is in the playground: Playground Link.

I am pasting it here as well:

db.collection.find({
  "$expr": {
    "$anyElementTrue": {
      "$map": {
        "input": "$entities",
        "in": {
          $and: [
            {
              "$regexMatch": {
                "input": "Room1",
                "regex": "$$this.id",
                "options": "i"
              },
              
            },
            {
              "$eq": [
                "$$this.isPattern",
                "true"
              ]
            },
            {
              $or: [
                {
                  "$eq": [
                    "$$this.type",
                    "Room"
                  ],
                  
                },
                {
                  "$eq": [
                    "$$this.type",
                    ""
                  ]
                }
              ]
            },
            {
              $eq: [
                "$$this.isTypePattern",
                null
              ]
            }
          ]
        }
      }
    }
  }
})

There is still one condition not correct: it requires a check that isTypePattern field is not present. I couldn't find an operator to perform this check so I am currently checking if it is null, which of course is not the same.

I wanted to get feedback before I commit it and raise a PR. So kindly let me know.

fgalan · 2022-09-05T18:20:34Z

There is still one condition not correct: it requires a check that isTypePattern field is not present. I couldn't find an operator to perform this check so I am currently checking if it is null, which of course is not the same.

Maybe the $exits operator could help?

anupamme · 2022-09-06T12:04:52Z

Hi @fgalan , I played around with $exists operator, it works when it is part of boolean expression but when I embed it within "in" of $map then the compiler throws an error. Here is the link playground

So I'm not sure how to use $exists within $map. If you can suggest me how to go about doing it, that would be great.

fgalan · 2022-09-06T14:47:54Z

Hi @fgalan , I played around with $exists operator, it works when it is part of boolean expression but when I embed it within "in" of $map then the compiler throws an error. Here is the link playground

So I'm not sure how to use $exists within $map. If you can suggest me how to go about doing it, that would be great.

I'd suggest to ask at StackOverflow using the "mongodb" and "mondob-query" labels, in the case it helps.

anupamme · 2022-09-06T15:30:10Z

Done

anupamme · 2022-09-07T14:12:06Z

Hey @fgalan

So someone responded at the Stackoverflow.

I will create a PR with this solution. If you have comments let me know.

fgalan · 2022-09-07T14:24:16Z

I will create a PR with this solution. If you have comments let me know.

Great! I'll have a look to the PR and provide comments on it. Thanks!

anupamme · 2022-09-14T10:27:40Z

Hi @fgalan

I created a pull request.

I have made the change only for function fill_idPtypeNP. Once it is approved I will make the similar changes for the other three functions:

fill_idNPtypeNP(), fill_idNPtypeP()and fill_idPtypeP()

Looking forward to your comments. Thanks!

anupamme · 2022-09-14T11:12:24Z

I see there are three failing tests:

on style issue
mismatch in line count.

Can you give me some guidance on how to fix?

Also, is there a way I can run these tests locally and know the results?

fgalan · 2022-09-14T12:03:36Z

With regars to style issue (https://github.com/telefonicaid/fiware-orion/actions/runs/3052031682/jobs/4921123995) I think this is the cause:

0002 errors of category 'Line ends in whitespace'
...
src/lib/mongoBackend/MongoCommonUpdate.cpp:1306:  Line ends in whitespace.  Consider deleting these extra spaces.  [whitespace/end_of_line] [4]
src/lib/mongoBackend/MongoCommonUpdate.cpp:1312:  Line ends in whitespace.  Consider deleting these extra spaces.  [whitespace/end_of_line] [4]

With regards to functional test failing it's a bit weird... I could have expected that "no cache" test were failing (due to some problem with the new query) but the fail in "cache" tests is weird, as they are not using that part of the code at all.

The procedure to run tests locally is described in this section: https://fiware-orion.readthedocs.io/en/master/admin/build_source.html#testing-and-coverage (the make functional_test INSTALL_DIR=~ part, specially) and here https://github.com/telefonicaid/fiware-orion/tree/master/test/functionalTest.

I'll try to have a look myself but it could take some time, so maybe you prefer to do progress on your own...

anupamme · 2022-09-27T07:14:33Z

Hi @fgalan

I tried installing functional Tests by going to the link. One of the steps is:

make install_scripts

However, there is no Makefile in the repository. Am I missing something here?

fgalan · 2022-09-27T14:46:37Z

This is the makefile: https://github.com/telefonicaid/fiware-orion/blob/master/makefile

fgalan · 2022-09-27T14:48:34Z

Maybe it is useful to have a look to CI Docker at https://github.com/telefonicaid/fiware-orion/tree/master/ci. This Docker container is built to run unit and functional test, so if you mimics the same steps in your environment you will end with a functional environment to run Orion tests.

fgalan · 2022-10-21T14:24:07Z

Now that the PR is merged, the next question I have is when will this code go live?

This will be part of version 3.8.0, to be released soon. There is only one issue pending to freeze the release (this one: #4085) but it is a complex functionality, so it could take some time.

In the meanwhile, the :latest version in telefonicaiot dockerhub should provide that functionality. You can use it while 3.8.0 gets released.

fgalan · 2022-10-24T14:05:52Z

PR #4226 with additional code comments and documentation fixes.

anupamme added the backlog label Jun 3, 2022

fgalan mentioned this issue Sep 14, 2022

Replacing the $where operator by $expr #4213

Merged

anupamme added a commit to anupamme/fiware-orion that referenced this issue Oct 20, 2022

adding entry for (telefonicaid#4148) replace $where by $expr

5d59b85

fgalan added this to the 3.8.0 milestone Oct 20, 2022

fgalan mentioned this issue Oct 24, 2022

FIX add code comments showing query structure in csubs no cache logic #4226

Merged

fgalan closed this as completed Nov 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replacing the operator $where with aggregation pipeline #4148

Replacing the operator $where with aggregation pipeline #4148

anupamme commented Jun 3, 2022 •

edited

Loading

fgalan commented Jun 3, 2022 •

edited

Loading

anupamme commented Jun 20, 2022

nsrivastava9 commented Jul 26, 2022

anupamme commented Jul 27, 2022

fgalan commented Aug 5, 2022 •

edited

Loading

fgalan commented Aug 5, 2022

fgalan commented Aug 5, 2022

anupamme commented Aug 10, 2022

fgalan commented Aug 11, 2022 •

edited

Loading

fgalan commented Aug 11, 2022

fgalan commented Aug 11, 2022

fgalan commented Aug 11, 2022

anupamme commented Aug 16, 2022 •

edited

Loading

fgalan commented Aug 17, 2022

anupamme commented Sep 5, 2022 •

edited

Loading

fgalan commented Sep 5, 2022

anupamme commented Sep 6, 2022 •

edited

Loading

fgalan commented Sep 6, 2022

anupamme commented Sep 6, 2022

anupamme commented Sep 7, 2022

fgalan commented Sep 7, 2022 •

edited

Loading

anupamme commented Sep 14, 2022

anupamme commented Sep 14, 2022

fgalan commented Sep 14, 2022

anupamme commented Sep 27, 2022

fgalan commented Sep 27, 2022

fgalan commented Sep 27, 2022

fgalan commented Oct 21, 2022

fgalan commented Oct 24, 2022

Replacing the operator $where with aggregation pipeline #4148

Replacing the operator $where with aggregation pipeline #4148

Comments

anupamme commented Jun 3, 2022 • edited Loading

fgalan commented Jun 3, 2022 • edited Loading

anupamme commented Jun 20, 2022

nsrivastava9 commented Jul 26, 2022

anupamme commented Jul 27, 2022

fgalan commented Aug 5, 2022 • edited Loading

fgalan commented Aug 5, 2022

fgalan commented Aug 5, 2022

anupamme commented Aug 10, 2022

fgalan commented Aug 11, 2022 • edited Loading

fgalan commented Aug 11, 2022

fgalan commented Aug 11, 2022

fgalan commented Aug 11, 2022

anupamme commented Aug 16, 2022 • edited Loading

fgalan commented Aug 17, 2022

anupamme commented Sep 5, 2022 • edited Loading

fgalan commented Sep 5, 2022

anupamme commented Sep 6, 2022 • edited Loading

fgalan commented Sep 6, 2022

anupamme commented Sep 6, 2022

anupamme commented Sep 7, 2022

fgalan commented Sep 7, 2022 • edited Loading

anupamme commented Sep 14, 2022

anupamme commented Sep 14, 2022

fgalan commented Sep 14, 2022

anupamme commented Sep 27, 2022

fgalan commented Sep 27, 2022

fgalan commented Sep 27, 2022

fgalan commented Oct 21, 2022

fgalan commented Oct 24, 2022

anupamme commented Jun 3, 2022 •

edited

Loading

fgalan commented Jun 3, 2022 •

edited

Loading

fgalan commented Aug 5, 2022 •

edited

Loading

fgalan commented Aug 11, 2022 •

edited

Loading

anupamme commented Aug 16, 2022 •

edited

Loading

anupamme commented Sep 5, 2022 •

edited

Loading

anupamme commented Sep 6, 2022 •

edited

Loading

fgalan commented Sep 7, 2022 •

edited

Loading