Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add explain support #2626

Merged
merged 6 commits into from
Nov 30, 2020
Merged

feat: add explain support #2626

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2764f4e
add explain functionality to ops
HanaPearlman Nov 18, 2020
4cad28b
test changes
HanaPearlman Nov 18, 2020
8ef85ba
documentation cleanup
HanaPearlman Nov 18, 2020
f976582
fix retryWrite bug
HanaPearlman Nov 18, 2020
4b7d9de
use enum to check verbosity strings
HanaPearlman Nov 19, 2020
3a2df97
list valid verbosity strings in param types
HanaPearlman Nov 19, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions lib/aggregation_cursor.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,13 @@ AggregationCursor.prototype.get = AggregationCursor.prototype.toArray;

/**
* Execute the explain for the cursor
*
* For backwards compatibility, a verbosity of true is interpreted as "allPlansExecution"
* and false as "queryPlanner". Prior to server version 3.6, aggregate()
* ignores the verbosity parameter and executes in "queryPlanner".
*
* @method AggregationCursor.prototype.explain
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [verbosity=true] - An optional mode in which to run the explain.
* @param {AggregationCursor~resultCallback} [callback] The result callback.
* @return {Promise} returns Promise if no callback passed
*/
Expand Down
15 changes: 12 additions & 3 deletions lib/collection.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,6 @@ const DEPRECATED_FIND_OPTIONS = ['maxScan', 'fields', 'snapshot', 'oplogReplay']
* @param {object} [options.fields] **Deprecated** Use `options.projection` instead
* @param {number} [options.skip=0] Set to skip N documents ahead in your query (useful for pagination).
* @param {Object} [options.hint] Tell the query to use specific indexes in the query. Object of indexes to use, {'_id':1}
* @param {boolean} [options.explain=false] Explain the query instead of returning the data.
* @param {boolean} [options.snapshot=false] DEPRECATED: Snapshot query.
* @param {boolean} [options.timeout=false] Specify if the cursor can timeout.
* @param {boolean} [options.tailable=false] Specify if the cursor is tailable.
Expand All @@ -310,6 +309,7 @@ const DEPRECATED_FIND_OPTIONS = ['maxScan', 'fields', 'snapshot', 'oplogReplay']
* @param {boolean} [options.noCursorTimeout] The server normally times out idle cursors after an inactivity period (10 minutes) to prevent excess memory use. Set this option to prevent that.
* @param {object} [options.collation] Specify collation (MongoDB 3.4 or higher) settings for update operation (see 3.4 documentation for available fields).
* @param {boolean} [options.allowDiskUse] Enables writing to temporary files on the server.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not sure the best way to communicate what the valid values of options.explain are. We could list the valid strings and how the boolean values will be interpreted, which is what we do for readPreference, or we could link to some server documentation, which is what we do for hint.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did suggest lower to put the strings in, if we put the strings in VSCode can suggest the strings to devs 🤷 but then it might error if there's a new string later.. Maybe the link is best

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we go ahead with using the enum from 4.0 to check that the string passed in is a valid verbosity, then it will error if there's a new verbosity string added later anyway.

Since we went ahead with doing the valid verbosity check in 4.0, my thought is to go ahead and do that here as well, and list the strings within the type. More work for us later if a new string is added, though.

* @throws {MongoError}
* @return {Cursor}
Expand Down Expand Up @@ -739,6 +739,7 @@ Collection.prototype.insert = deprecate(function(docs, options, callback) {
* @param {boolean} [options.checkKeys=false] If true, will throw if bson documents start with `$` or include a `.` in any key value
* @param {boolean} [options.serializeFunctions=false] Serialize functions on any object.
* @param {boolean} [options.ignoreUndefined=false] Specify if the BSON serializer should ignore undefined fields.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~updateWriteOpCallback} [callback] The command result callback
* @return {Promise} returns Promise if no callback passed
Expand Down Expand Up @@ -814,6 +815,7 @@ Collection.prototype.replaceOne = function(filter, doc, options, callback) {
* @param {boolean} [options.checkKeys=false] If true, will throw if bson documents start with `$` or include a `.` in any key value
* @param {boolean} [options.serializeFunctions=false] Serialize functions on any object.
* @param {boolean} [options.ignoreUndefined=false] Specify if the BSON serializer should ignore undefined fields.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~updateWriteOpCallback} [callback] The command result callback
* @return {Promise<Collection~updateWriteOpResult>} returns Promise if no callback passed
Expand Down Expand Up @@ -903,6 +905,7 @@ Collection.prototype.update = deprecate(function(selector, update, options, call
* @param {boolean} [options.checkKeys=false] If true, will throw if bson documents start with `$` or include a `.` in any key value
* @param {boolean} [options.serializeFunctions=false] Serialize functions on any object.
* @param {boolean} [options.ignoreUndefined=false] Specify if the BSON serializer should ignore undefined fields.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {string|object} [options.hint] optional index hint for optimizing the filter query
* @param {Collection~deleteWriteOpCallback} [callback] The command result callback
Expand Down Expand Up @@ -937,6 +940,7 @@ Collection.prototype.removeOne = Collection.prototype.deleteOne;
* @param {boolean} [options.checkKeys=false] If true, will throw if bson documents start with `$` or include a `.` in any key value
* @param {boolean} [options.serializeFunctions=false] Serialize functions on any object.
* @param {boolean} [options.ignoreUndefined=false] Specify if the BSON serializer should ignore undefined fields.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {string|object} [options.hint] optional index hint for optimizing the filter query
* @param {Collection~deleteWriteOpCallback} [callback] The command result callback
Expand Down Expand Up @@ -1044,7 +1048,6 @@ Collection.prototype.save = deprecate(function(doc, options, callback) {
* @param {object} [options.fields] **Deprecated** Use `options.projection` instead
* @param {number} [options.skip=0] Set to skip N documents ahead in your query (useful for pagination).
* @param {Object} [options.hint] Tell the query to use specific indexes in the query. Object of indexes to use, {'_id':1}
* @param {boolean} [options.explain=false] Explain the query instead of returning the data.
* @param {boolean} [options.snapshot=false] DEPRECATED: Snapshot query.
* @param {boolean} [options.timeout=false] Specify if the cursor can timeout.
* @param {boolean} [options.tailable=false] Specify if the cursor is tailable.
Expand All @@ -1063,6 +1066,7 @@ Collection.prototype.save = deprecate(function(doc, options, callback) {
* @param {boolean} [options.partial=false] Specify if the cursor should return partial results when querying against a sharded system
* @param {number} [options.maxTimeMS] Number of milliseconds to wait before aborting the query.
* @param {object} [options.collation] Specify collation (MongoDB 3.4 or higher) settings for update operation (see 3.4 documentation for available fields).
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~resultCallback} [callback] The command result callback
* @return {Promise} returns Promise if no callback passed
Expand Down Expand Up @@ -1580,6 +1584,7 @@ Collection.prototype.countDocuments = function(query, options, callback) {
* @param {(ReadPreference|string)} [options.readPreference] The preferred read preference (ReadPreference.PRIMARY, ReadPreference.PRIMARY_PREFERRED, ReadPreference.SECONDARY, ReadPreference.SECONDARY_PREFERRED, ReadPreference.NEAREST).
* @param {number} [options.maxTimeMS] Number of milliseconds to wait before aborting the query.
* @param {object} [options.collation] Specify collation settings for operation. See {@link https://docs.mongodb.com/manual/reference/command/aggregate|aggregation documentation}.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~resultCallback} [callback] The command result callback
* @return {Promise} returns Promise if no callback passed
Expand Down Expand Up @@ -1659,6 +1664,7 @@ Collection.prototype.stats = function(options, callback) {
* @param {boolean} [options.checkKeys=false] If true, will throw if bson documents start with `$` or include a `.` in any key value
* @param {boolean} [options.serializeFunctions=false] Serialize functions on any object.
* @param {boolean} [options.ignoreUndefined=false] Specify if the BSON serializer should ignore undefined fields.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~findAndModifyCallback} [callback] The collection result callback
* @return {Promise<Collection~findAndModifyWriteOpResultObject>} returns Promise if no callback passed
Expand Down Expand Up @@ -1692,6 +1698,7 @@ Collection.prototype.findOneAndDelete = function(filter, options, callback) {
* @param {boolean} [options.checkKeys=false] If true, will throw if bson documents start with `$` or include a `.` in any key value
* @param {boolean} [options.serializeFunctions=false] Serialize functions on any object.
* @param {boolean} [options.ignoreUndefined=false] Specify if the BSON serializer should ignore undefined fields.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~findAndModifyCallback} [callback] The collection result callback
* @return {Promise<Collection~findAndModifyWriteOpResultObject>} returns Promise if no callback passed
Expand Down Expand Up @@ -1726,6 +1733,7 @@ Collection.prototype.findOneAndReplace = function(filter, replacement, options,
* @param {boolean} [options.checkKeys=false] If true, will throw if bson documents start with `$` or include a `.` in any key value
* @param {boolean} [options.serializeFunctions=false] Serialize functions on any object.
* @param {boolean} [options.ignoreUndefined=false] Specify if the BSON serializer should ignore undefined fields.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] An ptional session to use for this operation
* @param {Collection~findAndModifyCallback} [callback] The collection result callback
* @return {Promise<Collection~findAndModifyWriteOpResultObject>} returns Promise if no callback passed
Expand Down Expand Up @@ -1831,7 +1839,6 @@ Collection.prototype.findAndRemove = deprecate(function(query, sort, options, ca
* @param {number} [options.batchSize=1000] The number of documents to return per batch. See {@link https://docs.mongodb.com/manual/reference/command/aggregate|aggregation documentation}.
* @param {object} [options.cursor] Return the query as cursor, on 2.6 > it returns as a real cursor on pre 2.6 it returns as an emulated cursor.
* @param {number} [options.cursor.batchSize=1000] Deprecated. Use `options.batchSize`
* @param {boolean} [options.explain=false] Explain returns the aggregation execution plan (requires mongodb 2.6 >).
* @param {boolean} [options.allowDiskUse=false] allowDiskUse lets the server know if it can use disk to store temporary results for the aggregation (requires mongodb 2.6 >).
* @param {number} [options.maxTimeMS] maxTimeMS specifies a cumulative time limit in milliseconds for processing operations on the cursor. MongoDB interrupts the operation at the earliest following interrupt point.
* @param {number} [options.maxAwaitTimeMS] The maximum amount of time for the server to wait on new documents to satisfy a tailable cursor query.
Expand All @@ -1843,6 +1850,7 @@ Collection.prototype.findAndRemove = deprecate(function(query, sort, options, ca
* @param {object} [options.collation] Specify collation settings for operation. See {@link https://docs.mongodb.com/manual/reference/command/aggregate|aggregation documentation}.
* @param {string} [options.comment] Add a comment to an aggregation command
* @param {string|object} [options.hint] Add an index selection hint to an aggregation command
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~aggregationCallback} callback The command result callback
* @return {(null|AggregationCursor)}
Expand Down Expand Up @@ -2089,6 +2097,7 @@ Collection.prototype.group = deprecate(function(
* @param {boolean} [options.jsMode=false] It is possible to make the execution stay in JS. Provided in MongoDB > 2.0.X.
* @param {boolean} [options.verbose=false] Provide statistics on job execution time.
* @param {boolean} [options.bypassDocumentValidation=false] Allow driver to bypass schema validation in MongoDB 3.2 or higher.
* @param {'queryPlanner'|'queryPlannerExtended'|'executionStats'|'allPlansExecution'|boolean} [options.explain] The verbosity mode for the explain output.
* @param {ClientSession} [options.session] optional session to use for this operation
* @param {Collection~resultCallback} [callback] The command result callback
* @throws {MongoError}
Expand Down
3 changes: 2 additions & 1 deletion lib/core/sdam/topology.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -930,7 +930,8 @@ function executeWriteOperation(args, options, callback) {
!!options.retryWrites &&
options.session &&
isRetryableWritesSupported(topology) &&
!options.session.inTransaction();
!options.session.inTransaction() &&
options.explain === undefined;

topology.selectServer(writableServerSelector(), options, (err, server) => {
if (err) {
Expand Down
3 changes: 2 additions & 1 deletion lib/core/topologies/mongos.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -919,7 +919,8 @@ function executeWriteOperation(args, options, callback) {
!!options.retryWrites &&
options.session &&
isRetryableWritesSupported(self) &&
!options.session.inTransaction();
!options.session.inTransaction() &&
options.explain === undefined;

const handler = (err, result) => {
if (!err) return callback(null, result);
Expand Down
3 changes: 2 additions & 1 deletion lib/core/topologies/replset.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1193,7 +1193,8 @@ function executeWriteOperation(args, options, callback) {
!!options.retryWrites &&
options.session &&
isRetryableWritesSupported(self) &&
!options.session.inTransaction();
!options.session.inTransaction() &&
options.explain === undefined;

if (!self.s.replicaSetState.hasPrimary()) {
if (self.s.disconnectHandler) {
Expand Down
23 changes: 12 additions & 11 deletions lib/core/wireprotocol/query.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ const isSharded = require('./shared').isSharded;
const maxWireVersion = require('../utils').maxWireVersion;
const applyCommonQueryOptions = require('./shared').applyCommonQueryOptions;
const command = require('./command');
const decorateWithExplain = require('../../utils').decorateWithExplain;
const Explain = require('../../explain').Explain;

function query(server, ns, cmd, cursorState, options, callback) {
options = options || {};
Expand All @@ -31,7 +33,14 @@ function query(server, ns, cmd, cursorState, options, callback) {
}

const readPreference = getReadPreference(cmd, options);
const findCmd = prepareFindCommand(server, ns, cmd, cursorState, options);
let findCmd = prepareFindCommand(server, ns, cmd, cursorState, options);

// If we have explain, we need to rewrite the find command
// to wrap it in the explain command
const explain = Explain.fromOptions(options);
if (explain) {
findCmd = decorateWithExplain(findCmd, explain);
}

// NOTE: This actually modifies the passed in cmd, and our code _depends_ on this
// side-effect. Change this ASAP
Expand Down Expand Up @@ -59,7 +68,7 @@ function query(server, ns, cmd, cursorState, options, callback) {

function prepareFindCommand(server, ns, cmd, cursorState) {
cursorState.batchSize = cmd.batchSize || cursorState.batchSize;
let findCmd = {
const findCmd = {
find: collectionNamespace(ns)
};

Expand Down Expand Up @@ -143,14 +152,6 @@ function prepareFindCommand(server, ns, cmd, cursorState) {
if (cmd.collation) findCmd.collation = cmd.collation;
if (cmd.readConcern) findCmd.readConcern = cmd.readConcern;

// If we have explain, we need to rewrite the find command
// to wrap it in the explain command
if (cmd.explain) {
findCmd = {
explain: findCmd
};
}

return findCmd;
}

Expand Down Expand Up @@ -188,7 +189,7 @@ function prepareLegacyFindQuery(server, ns, cmd, cursorState, options) {
if (typeof cmd.showDiskLoc !== 'undefined') findCmd['$showDiskLoc'] = cmd.showDiskLoc;
if (cmd.comment) findCmd['$comment'] = cmd.comment;
if (cmd.maxTimeMS) findCmd['$maxTimeMS'] = cmd.maxTimeMS;
if (cmd.explain) {
if (options.explain !== undefined) {
// nToReturn must be 0 (match all) or negative (match N and close cursor)
// nToReturn > 0 will give explain results equivalent to limit(0)
numberToReturn = -Math.abs(cmd.limit || 0);
Expand Down
11 changes: 10 additions & 1 deletion lib/core/wireprotocol/write_command.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
const MongoError = require('../error').MongoError;
const collectionNamespace = require('./shared').collectionNamespace;
const command = require('./command');
const decorateWithExplain = require('../../utils').decorateWithExplain;
const Explain = require('../../explain').Explain;

function writeCommand(server, type, opsField, ns, ops, options, callback) {
if (ops.length === 0) throw new MongoError(`${type} must contain at least one document`);
Expand All @@ -15,7 +17,7 @@ function writeCommand(server, type, opsField, ns, ops, options, callback) {
const ordered = typeof options.ordered === 'boolean' ? options.ordered : true;
const writeConcern = options.writeConcern;

const writeCommand = {};
let writeCommand = {};
writeCommand[type] = collectionNamespace(ns);
writeCommand[opsField] = ops;
writeCommand.ordered = ordered;
Expand All @@ -36,6 +38,13 @@ function writeCommand(server, type, opsField, ns, ops, options, callback) {
writeCommand.bypassDocumentValidation = options.bypassDocumentValidation;
}

// If a command is to be explained, we need to reformat the command after
// the other command properties are specified.
const explain = Explain.fromOptions(options);
if (explain) {
writeCommand = decorateWithExplain(writeCommand, explain);
}

const commandOptions = Object.assign(
{
checkKeys: type === 'insert',
Expand Down
Loading