Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update mongosh to 2.0.0 and driver to 6.0.0 VSCODE-453 #592

Merged
merged 1 commit into from
Sep 13, 2023
Merged

feat: update mongosh to 2.0.0 and driver to 6.0.0 VSCODE-453 #592

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .depcheckrc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,4 @@ ignores:
- postcss-loader
- style-loader
- ts-loader
- mongodb-runner
4,231 changes: 960 additions & 3,271 deletions package-lock.json
View file
Open in desktop

Large diffs are not rendered by default.

36 changes: 21 additions & 15 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,11 @@
"watch": "npm run compile && npm-run-all -p watch:*",
"watch:extension": "npm run compile:extension -- -watch",
"watch:extension-bundles": "webpack --mode development --watch",
"pretest": "npm run compile && mongodb-runner start --port=27018",
"pretest": "npm run compile && mongodb-runner start --id=vscode -- --port 27018",
"test": "npm run test-webview && npm run test-extension",
"test-extension": "cross-env NODE_OPTIONS=--no-force-async-hooks-checks xvfb-maybe node ./out/test/runTest.js",
"test-webview": "jest",
"posttest": "mongodb-runner stop --port=27018",
"posttest": "mongodb-runner stop --id=vscode",
"analyze-bundle": "webpack --mode production --analyze",
"vscode:prepublish": "npm run clean && npm run compile:keyfile && npm run compile:resources && webpack --mode production",
"check": "npm run lint && npm run depcheck",
Expand Down Expand Up @@ -980,24 +980,25 @@
"@iconify-icons/codicon": "^1.2.25",
"@iconify/react": "^1.1.4",
"@leafygreen-ui/logo": "^8.0.4",
"@mongodb-js/mongodb-constants": "^0.6.5",
"@mongosh/browser-runtime-electron": "^1.10.4",
"@mongosh/i18n": "^1.10.4",
"@mongosh/service-provider-server": "^1.10.4",
"@mongosh/shell-api": "^1.10.4",
"@mongodb-js/mongodb-constants": "^0.7.0",
"@mongosh/browser-runtime-electron": "^2.0.0",
"@mongosh/i18n": "^2.0.0",
"@mongosh/service-provider-server": "^2.0.0",
"@mongosh/shell-api": "^2.0.0",
"analytics-node": "^6.2.0",
"bson": "^5.3.0",
"bson-transpilers": "^2.0.4",
"classnames": "^2.3.2",
"debug": "^4.3.4",
"dotenv": "^16.3.1",
"micromatch": "^4.0.5",
"mongodb": "^5.8.1",
"mongodb-build-info": "^1.5.0",
"mongodb": "^6.0.0",
"mongodb-build-info": "^1.6.2",
"mongodb-cloud-info": "^2.1.0",
"mongodb-connection-string-url": "^2.6.0",
"mongodb-data-service": "^22.8.0",
"mongodb-query-parser": "^2.5.0",
"mongodb-data-service": "^22.10.0",
"mongodb-data-service-legacy": "npm:[email protected]",
"mongodb-query-parser": "^3.1.3",
"mongodb-schema": "^11.2.2",
"numeral": "^2.0.6",
"react": "^17.0.2",
Expand All @@ -1012,10 +1013,10 @@
},
"devDependencies": {
"@babel/preset-typescript": "^7.22.5",
"@mongodb-js/oidc-plugin": "^0.2.4",
"@mongodb-js/oidc-plugin": "^0.3.0",
"@mongodb-js/prettier-config-compass": "^1.0.0",
"@mongodb-js/sbom-tools": "^0.5.4",
"@mongosh/service-provider-core": "^1.10.4",
"@mongosh/service-provider-core": "^2.0.0",
"@types/analytics-node": "^3.1.11",
"@types/babel__core": "^7.20.1",
"@types/babel__traverse": "^7.20.1",
Expand Down Expand Up @@ -1064,8 +1065,8 @@
"mocha": "^8.4.0",
"mocha-junit-reporter": "^2.2.0",
"mocha-multi": "^1.1.7",
"mongodb-client-encryption": "^2.8.0",
"mongodb-runner": "^4.10.0",
"mongodb-client-encryption": "^6.0.0",
"mongodb-runner": "^5.4.4",
"node-loader": "^0.6.0",
"npm-run-all": "^4.1.5",
"ora": "^5.4.1",
Expand All @@ -1089,6 +1090,11 @@
"xvfb-maybe": "^0.2.1",
"yargs-parser": "^20.2.9"
},
"overrides": {
"mongodb-connection-model": {
"@mongodb-js/compass-utils": "0.3.4"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mongodb-js/compass-utils doesn’t export getStoragePaths() in the most recent version anymore which makes require()ing mongodb-connection-model fail, technically a breaking change and maybe we should have actually labelled mongodb-js/compass@2c91060 as feat!: rather than chore(fs): …

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know that vscode is so coupled with compass. Actually I didnt know that we are using legacy compass connections in vscode at all.

}
},
"precommit": [
"check"
]
Expand Down
44 changes: 29 additions & 15 deletions src/connectionController.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,8 @@
import * as vscode from 'vscode';
import {
convertConnectionModelToInfo,
getConnectionTitle,
extractSecrets,
mergeSecrets,
connect,
} from 'mongodb-data-service';
import { connect } from 'mongodb-data-service';
import type {
DataService,
ConnectionInfo,
ConnectionOptions,
ConnectionOptions as ConnectionOptionsFromCurrentDS,
} from 'mongodb-data-service';
import ConnectionString from 'mongodb-connection-string-url';
import { EventEmitter } from 'events';
Expand All @@ -31,6 +24,24 @@ import { StorageVariables } from './storage';
import type { StatusView } from './views';
import type TelemetryService from './telemetry/telemetryService';
import LINKS from './utils/links';
import type {
ConnectionInfo as ConnectionInfoFromLegacyDS,
ConnectionOptions as ConnectionOptionsFromLegacyDS,
} from 'mongodb-data-service-legacy';
import {
getConnectionTitle,
extractSecrets,
mergeSecrets,
Comment on lines +32 to +34
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are now exported by @mongodb-js/connection-storage, and we may be able to use it, but that has a hard, non-optional/non-lazy dependency on keytar, so I don’t think we can use it here. Together with the item below, that made it seem very reasonable to just use a prior version of mongodb-data-service and take care of this issue when we actually do the VSCode connection improvements project.

convertConnectionModelToInfo,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could copy the source code for this one, but then we’d still have a dependency on mongodb-connection-model and would need to use the outdated version of that (or, we could, recursively, also add the source code for that entire package).

} from 'mongodb-data-service-legacy';

export function launderConnectionOptionTypeFromLegacyToCurrent(
opts: ConnectionOptionsFromLegacyDS
): ConnectionOptionsFromCurrentDS {
// Ensure that, at most, the types for OIDC mismatch here.
return opts as Omit<typeof opts, 'oidc'>;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don’t have proper OIDC support yet anyway

}

// eslint-disable-next-line @typescript-eslint/no-var-requires
const packageJSON = require('../package.json');

Expand All @@ -54,7 +65,7 @@ export interface StoreConnectionInfo {
name: string; // Possibly user given name, not unique.
storageLocation: StorageLocation;
secretStorageLocation?: SecretStorageLocationType;
connectionOptions?: ConnectionOptions;
connectionOptions?: ConnectionOptionsFromLegacyDS;
connectionModel?: LegacyConnectionModel;
}

Expand Down Expand Up @@ -471,7 +482,7 @@ export default class ConnectionController {

parseNewConnection(
rawConnectionModel: LegacyConnectionModel
): ConnectionInfo {
): ConnectionInfoFromLegacyDS {
return convertConnectionModelToInfo({
...rawConnectionModel,
appname: `${packageJSON.name} ${packageJSON.version}`, // Override the default connection appname.
Expand All @@ -483,7 +494,7 @@ export default class ConnectionController {
): Promise<MigratedStoreConnectionInfoWithConnectionOptions> {
// We don't want to store secrets to disc.
const { connectionInfo: safeConnectionInfo, secrets } = extractSecrets(
newStoreConnectionInfoWithSecrets as ConnectionInfo
newStoreConnectionInfoWithSecrets as ConnectionInfoFromLegacyDS
);
const savedConnectionInfo = await this._storageController.saveConnection({
...newStoreConnectionInfoWithSecrets,
Expand All @@ -498,7 +509,7 @@ export default class ConnectionController {
}

async saveNewConnectionFromFormAndConnect(
originalConnectionInfo: ConnectionInfo,
originalConnectionInfo: ConnectionInfoFromLegacyDS,
connectionType: ConnectionTypes
): Promise<ConnectionAttemptResult> {
const name = getConnectionTitle(originalConnectionInfo);
Expand Down Expand Up @@ -526,9 +537,12 @@ export default class ConnectionController {
return this._connect(savedConnectionInfo.id, connectionType);
}

async _connectWithDataService(connectionOptions: ConnectionOptions) {
async _connectWithDataService(
connectionOptions: ConnectionOptionsFromLegacyDS
) {
return connect({
connectionOptions,
connectionOptions:
launderConnectionOptionTypeFromLegacyToCurrent(connectionOptions),
productName: packageJSON.name,
productDocsLink: LINKS.extensionDocs(),
});
Expand Down
24 changes: 13 additions & 11 deletions src/language/mongoDBService.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,17 +352,19 @@ export default class MongoDBService {
}

try {
const documents = this._serviceProvider.find(
databaseName,
collectionName,
{},
// TODO: figure out if we need parseSchema for one field at all.
// For one document we can simply get deep object keys,
// or we could analyze the schema for at least 5-10 documents.
// The current behaviour came from Compass when we do the same:
// https://github.com/mongodb-js/compass/blob/main/packages/compass-field-store/src/stores/store.js#L193
{ limit: 1 }
);
const documents = await this._serviceProvider
.find(
databaseName,
collectionName,
{},
// TODO: figure out if we need parseSchema for one field at all.
// For one document we can simply get deep object keys,
// or we could analyze the schema for at least 5-10 documents.
// The current behaviour came from Compass when we do the same:
// https://github.com/mongodb-js/compass/blob/main/packages/compass-field-store/src/stores/store.js#L193
{ limit: 1 }
)
.toArray();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseSchema would want a cursor from a 5.x driver right now, should probably update this as well at some point but since we only use a single document anyway .toArray() is a nice way to decouple the types here


const schema = await parseSchema(documents);
result = schema?.fields ? schema.fields.map((item) => item.name) : [];
Expand Down
8 changes: 4 additions & 4 deletions src/test/suite/connectionController.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import AUTH_STRATEGY_VALUES from '../../views/webview-app/connection-model/const
import ConnectionController, {
DataServiceEventTypes,
keytarMigrationFailedMessage,
launderConnectionOptionTypeFromLegacyToCurrent,
} from '../../connectionController';
import formatError from '../../utils/formatError';
import { StorageController, StorageVariables } from '../../storage';
Expand Down Expand Up @@ -836,7 +837,8 @@ suite('Connection Controller Test Suite', function () {
await sleep(50);

return connect({
connectionOptions,
connectionOptions:
launderConnectionOptionTypeFromLegacyToCurrent(connectionOptions),
});
}
);
Expand Down Expand Up @@ -1199,9 +1201,7 @@ suite('Connection Controller Test Suite', function () {
monitorCommands: true,
useSystemCA: undefined,
authMechanismProperties: {},
oidc: {
allowedFlows: ['auth-code'],
},
oidc: {},
productDocsLink:
'https://docs.mongodb.com/mongodb-vscode/?utm_source=vscode&utm_medium=product',
productName: 'mongodb-vscode',
Expand Down
9 changes: 9 additions & 0 deletions syntaxes/mongodbInjection.tmLanguage.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1935,6 +1935,15 @@
}
}
},
{
"name": "meta.object.member.mongodb",
"match": "\\$vectorSearch\\b",
"captures": {
"0": {
"name": "keyword.other.$vectorSearch.mongodb"
}
}
},
{
"name": "meta.object.member.mongodb",
"match": "\\$set\\b",
Expand Down