fix(mongodb): update ssl connection settings #1809

lirantal · 2017-07-08T09:37:42Z

Updating SSL connection settings to be in-par with new versions of Mongoose.
Related issue: c0f6cb3#commitcomment-22974350

…icates (#1472) Fixes #1349

staminna · 2017-07-08T14:22:32Z

I have updated the secure.js file and created the certificate as per the official mongodb documentation, but they provide mongodb-cert.key, mongodb-cert.crt and only one mongodb.pem file. I tried to match but it gives one error though. I am use having updated mongod.conf with:

net:
   ssl:
      mode: requireSSL
      PEMKeyFile: /Path/to/app/website/config/sslcerts/mongodb.pem

Here is my shell script I used for mongod:

mongod --fork --logpath logs/mongodb.log --dbpath mongodb --sslMode requireSSL --sslPEMKeyFile /path/to/app/website/config/sslcerts/mongodb.pem --sslCAFile /path/to/app/website/config/sslcerts/mongodb-cert.crt

child process started successfully, parent exiting

Running the app

Application loaded using the "secure" environment configuration
MEAN.JS application started on port 8443
Could not connect to MongoDB!
MongoError: connection 0 to localhost:27017 closed

lirantal · 2017-07-08T14:55:20Z

Before jumping at the problem and concluding that it's a MEAN.JS configuration issue, can you connect to the MongoDB server with the mongo client using the ssl connection strings?

Make sure that works first, and then we'll figure out the correct config items to put for MEAN.JS

staminna · 2017-07-08T15:26:08Z

I can connect mongod to mongo server without any errors or warnings with the certificates generated for 127.0.0.1

$ mongo --ssl --sslPEMKeyFile=/path/to/app/website/config/sslcerts/mongodb.pem  --sslCAFile=/path/to/app/website/config/sslcerts/mongodb-cert.crt
MongoDB shell version: 3.2.11
connecting to: test

Launching Mean now outputs the following error:

Server Error
MongoError: connection 13 to localhost:27017 closed
at Function.MongoError.create (/path/to/app/website/node_modules/mongodb-core/lib/error.js:29:11)
at Socket. (/Users/jorge/webapp/advgenie/app/website/node_modules/mongodb-core/lib/connection/connection.js:202:22)
at Socket.g (events.js:260:16)
at emitOne (events.js:77:13)
at Socket.emit (events.js:169:7)
at TCP._onclose (net.js:487:12)

then the connection number iterates until 68th

Edit: Now for some reason doesn't Iterate from connection 0. I even reverted my git repository.

Here is my log file:

[initandlisten] waiting for connections on port 27017 ssl
I NETWORK [initandlisten] connection accepted from 127.0.0.1:57974 #1 (1 connection now open)
I NETWORK [conn1] AssertionException handling request, closing client connection: 17189 The server is configured to only allow SSL connections

lirantal · 2017-07-14T07:08:59Z

Which version do you have installed of mongoose now?

Edit the lib/mongoose.js file and add mongoose.set('debug', true) to see if it will be more verbose on errors.

The official docs actually still refer to a server object inside options: http://mongoosejs.com/docs/connections.html but rather this open issue is the only source of truth it seems: Automattic/mongoose#5442

staminna · 2017-07-14T14:28:26Z

My version of mongoose is

[email protected]

I don't have mongoose.js in my project, only minified and I don't really know where to set the debug instruction to true in 10426 lines of code ( I have unminified the file online).

Where is this file supposed to be in regards to the project folder? website/node_modules/mongoose/bin/mongoose.js ?
My mongoose.min.js is outside website directory but it should work nevertheless.

lirantal · 2017-07-14T14:30:59Z

which version of MEAN.js are you using? and the file I'm talking about should be found in config/lib/mongoose.js on the project's root directory

staminna · 2017-07-14T14:42:16Z

It's from a stable 3.x MEAN.JS fork. Perhaps you can check it out?
I believe if I could have mongoose.js file with the instruction I can minify it and then it would run test and debug more verbosely.

I've found occurence of mongoose.set in mongoose's folder index.js file and removed the comment though:

Application loaded using the "secure" environment configuration
/app/website/node_modules/mongoose/lib/index.js:77
   mongoose.set('debug', true); // enable logging collection methods + arguments to the console
           ^

TypeError: Cannot read property 'set' of undefined

 at Object.<anonymous> (/app/website/node_modules/mongoose/lib/index.js:77:12)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/app/website/node_modules/mongoose/index.js:7:18)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/app/website/server.js:8:13)
    at Module._compile (module.js:409:26)

lirantal · 2017-07-17T05:32:54Z

@staminna that link is broken and 3 is considerably old, moreover I don't understand why you have a minified version of... anything on the backend basically.

How about you use the latest master branch version to test it out?

staminna · 2017-07-17T15:25:44Z

I noticed the link has a dot in the end of the URL. I'll update as soon as to a computer. It's very Interesting though. It should work, but I can't abandon a MEAN. JS I have been working on for a year. It's the latest functionality I need to finish the project and security is crucial for the app.

simison · 2017-07-17T15:38:10Z

So it's a fork of old version of Mean.js: https://github.com/dreamerkumar/bizsitegenie

@staminna so you should be looking at this:

https://github.com/dreamerkumar/bizsitegenie/blob/master/website/server.js#L15-L21

I guess that instead of this:

// Bootstrap db connection
var db = mongoose.connect(config.db, function(err) {
	if (err) {
		console.error(chalk.red('Could not connect to MongoDB!'));
		console.log(chalk.red(err));
	}
});

you'll need something like:

// Bootstrap db connection
var db = mongoose.connect(config.db, {
 // ssl options for Mongoose v4.10.x
}, function(err) {
	if (err) {
		console.error(chalk.red('Could not connect to MongoDB!'));
		console.log(chalk.red(err));
	}
});

To be honest this discussions shouldn't be under this pull request but somewhere else instead.

mleanos · 2017-07-18T02:24:32Z

but rather this open issue is the only source of truth it seems

Yes, I think we can safely say the referenced issue in Mongoose's project is correct. I'm sure these changes can be merged in without issue.

However, it would be great if we can get confirmation..

@staminna As Mikael mentioned before, you will need to pass your config settings to the Mongoose connect method here https://github.com/dreamerkumar/bizsitegenie/blob/0c4fb1afafa07a9d751922232db000a16515c1f1/website/server.js#L16, or Mongoose won't be able to connect using SSL.

staminna · 2017-07-18T14:46:33Z

I've added

var fs = require('fs'),
var mongoose = require('mongoose');

and the secure options array to the second parameter of mongoose.connect() function. as suggested, then I lunched mongod with SSL certifications but even if I comment out the cert files with minimum amount of options still returns

Could not connect to MongoDB!
MongoError: connection 0 to localhost:27017 closed

I changed the path in the fs.readFileSync to check for errors to see if it would show errors and it did so I am sure they are being loaded as expected. Should I open a new issue and reference this pull?

simison · 2017-07-18T14:51:43Z

@staminna I don't understand what fs is doing here... edit: reading cert files? I guess I'm not following this closely enough. ;-)

Try removing as much complications (like loading config objects) away from your app and try adding correct config objects directly to mongoose.connect(). When you get that working, you can start looking at loading configs etc.

Actually even just running plain one file test which does nothing but connects to db with mongoose would probably help you to debug this.

staminna · 2017-07-18T18:42:47Z

Yes fs @simison reading the cert files.
I did the most minimal app I could with express and mongoose but I can't pass through

MongoError: connection 0 to localhost:27017 closed
and sometimes with the same code:

MongoError: read ECONNRESET

even after reading everything in stackoverflow.
I am using mac os.

mleanos · 2017-07-19T06:24:31Z

@staminna In your screen shot, the Mongoose SSL options are commented out. Was this intentional for just posting here? Otherwise, you'd need to un-comment the options for it to work.

staminna · 2017-07-19T14:17:16Z

It's intentional only to make a point. There is no difference, the error and issue is the same either commenting or not. Something is wrong. This is the most minimalist piece of code I could do and even without launching mongod without authentication and SSL options on the command line the error is "undefined" Could not connect to MongoDB! Undefined I also have updated my mongod.conf file: net: security: enableEncryption: true ssl: mode: requireSSL PEMKeyFile: /app/website/config/sslcerts/mongodb.pem sslCAFile: /app/website/config/sslcerts/mongodb-cert.crt bindIp: 127.0.0.1 port: 27017 allowInvalidCertificates: true allowInvalidHostnames: true I will reverse the invalid certificates and hostnanes to false once I am sure it works.

lirantal · 2017-07-19T21:47:47Z

@staminna can you make sure you enable mongoose debugging (explained before with mongoose.set('debug', true)) and make sure the options are set?

staminna · 2017-07-20T02:18:19Z

I think I finally found something in the logs.
Sorry I missed that before.

2017-07-20T03:12:13.431+0100 I NETWORK [initandlisten] connection accepted from 127.0.0.1:58265 #3 (1 connection now open)
2017-07-20T03:12:13.439+0100 D - [conn3] User Assertion: 17189:The server is configured to only allow SSL connections
2017-07-20T03:12:13.439+0100 I NETWORK [conn3] AssertionException handling request, closing client connection: 17189 The server is configured to only allow SSL connections
2017-07-20T03:12:25.033+0100 D INDEX [TTLMonitor] TTL -- ns: webapplication.sessions key: { expires: 1 }
2017-07-20T03:12:25.033+0100 D INDEX [TTLMonitor] TTL deleted: 0
2017-07-20T03:12:26.130+0100 I NETWORK [initandlisten] connection accepted from 127.0.0.1:58318 #4 (1 connection now open)
2017-07-20T03:12:26.138+0100 D - [conn4] User Assertion: 17189:The server is configured to only allow SSL connections
2017-07-20T03:13:25.036+0100 D INDEX [TTLMonitor] TTL -- ns: webapplication.sessions key: { expires: 1 }
2017-07-20T03:13:25.037+0100 D INDEX [TTLMonitor] TTL deleted: 0
2017-07-20T03:14:25.038+0100 D INDEX [TTLMonitor] TTL -- ns: webapplication.sessions key: { expires: 1 }
2017-07-20T03:14:25.039+0100 D INDEX [TTLMonitor] TTL deleted: 0

lirantal · 2017-07-20T05:50:29Z

those logs though are hardly helpful as they are the mongo server logs, not the mongo client.

staminna · 2017-07-20T16:30:08Z

I looked at /usr/local/var/log/mongodb/ and there is nothing there. I set the logs up on mongodb.conf on another directory, restarted mongod and run the minimal amount of code required to connect mongoose to the mongodb database and nothing got logged either.
systemLog: destination: file path: "/app/mongodblogs/syslog.log" logAppend: true
I set mongoose.set('debug', true) above mongoose.connect and it was ok with node.js
Where are the client side logs on mac os? I

The error message are still the same.
I also tested to open the database with the SSL certificates and passphrase with Robomongo and It's ok.

My production environment keeps getting hacked and I am losing data records on the db on a weekly basis.

mleanos · 2017-07-27T00:18:18Z

My production environment keeps getting hacked and I am losing data records on the db on a weekly basis.

@staminna As this relates to MEANJS, it's merely a configuration change. Beyond that, if you're having issues with these settings not working, you'll need to confirm your setup is correct to be used with these settings. If you think this is an issue with Mongoose/MongoDB you should open an issue on either of those projects.

This discussion, at the very least, should be moved to a new issue here in this repo. This PR is getting polluted with this back & forth.

staminna · 2017-07-27T15:17:13Z

Sorry guys, I'm not a software Engineer.
As @mleanos explained, can someone help on a new issue somewhere in relation to this repo? I'm not sure what that means. I have created a new stackoverflow regarding this minimum required code for SSL connections.

simison · 2017-08-14T08:11:20Z

@lirantal any changes to get this in for 0.6.0 or should we just postpone?

lirantal · 2017-08-14T08:20:00Z

while we haven't tested the certificate use-case I think it's aligned with documentation for new mongodb.

simison · 2017-08-14T08:21:57Z

Ah right it's just a comment change! Just didn't expect that seeing coverage+Travis failed.

Let's merge.

mleanos · 2017-08-14T20:38:38Z

I agree that we should merge. The production env settings are commented out anyways.

I've re-ran the build a few times over the past weeks, and still it fails on the E2E tests. Seems odd to me, but it's not relevant to these changes.

simison · 2017-08-14T20:47:59Z

👍

simison · 2017-08-14T20:51:57Z

@mleanos

I've re-ran the build a few times over the past weeks, and still it fails on the E2E tests. Seems odd to me, but it's not relevant to these changes.

This might help: #1845

lirantal · 2017-08-15T07:31:04Z

thanks!

fix(mongodb): update ssl connection settings

6a754d5

lirantal self-assigned this Jul 8, 2017

lirantal referenced this pull request Jul 8, 2017

feat(mongodb): support for mongodb secure connections with ssl certif…

c0f6cb3

…icates (#1472) Fixes #1349

simison added this to the 0.6.0 milestone Aug 11, 2017

simison mentioned this pull request Aug 11, 2017

MEAN.JS 1.0 Roadmap #1666

Open

5 tasks

simison added the issue:bug label Aug 13, 2017

simison merged commit 4fcf240 into meanjs:master Aug 14, 2017

cicorias pushed a commit to JavaScriptExpert/mean that referenced this pull request Sep 12, 2017

fix(mongodb): update ssl connection settings (meanjs#1809)

32d4fec

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(mongodb): update ssl connection settings #1809

fix(mongodb): update ssl connection settings #1809

lirantal commented Jul 8, 2017 •

edited

Loading

staminna commented Jul 8, 2017

lirantal commented Jul 8, 2017

staminna commented Jul 8, 2017 •

edited

Loading

lirantal commented Jul 14, 2017

staminna commented Jul 14, 2017

lirantal commented Jul 14, 2017

staminna commented Jul 14, 2017 •

edited

Loading

lirantal commented Jul 17, 2017

staminna commented Jul 17, 2017

simison commented Jul 17, 2017 •

edited

Loading

mleanos commented Jul 18, 2017 •

edited

Loading

staminna commented Jul 18, 2017

simison commented Jul 18, 2017 •

edited

Loading

staminna commented Jul 18, 2017 •

edited

Loading

mleanos commented Jul 19, 2017

staminna commented Jul 19, 2017 via email •

edited

Loading

lirantal commented Jul 19, 2017

staminna commented Jul 20, 2017 •

edited

Loading

lirantal commented Jul 20, 2017

staminna commented Jul 20, 2017 •

edited

Loading

mleanos commented Jul 27, 2017

staminna commented Jul 27, 2017 •

edited

Loading

simison commented Aug 14, 2017

lirantal commented Aug 14, 2017

simison commented Aug 14, 2017

mleanos commented Aug 14, 2017 •

edited

Loading

simison commented Aug 14, 2017

simison commented Aug 14, 2017 •

edited

Loading

lirantal commented Aug 15, 2017

fix(mongodb): update ssl connection settings #1809

fix(mongodb): update ssl connection settings #1809

Conversation

lirantal commented Jul 8, 2017 • edited Loading

staminna commented Jul 8, 2017

lirantal commented Jul 8, 2017

staminna commented Jul 8, 2017 • edited Loading

lirantal commented Jul 14, 2017

staminna commented Jul 14, 2017

lirantal commented Jul 14, 2017

staminna commented Jul 14, 2017 • edited Loading

lirantal commented Jul 17, 2017

staminna commented Jul 17, 2017

simison commented Jul 17, 2017 • edited Loading

mleanos commented Jul 18, 2017 • edited Loading

staminna commented Jul 18, 2017

simison commented Jul 18, 2017 • edited Loading

staminna commented Jul 18, 2017 • edited Loading

mleanos commented Jul 19, 2017

staminna commented Jul 19, 2017 via email • edited Loading

lirantal commented Jul 19, 2017

staminna commented Jul 20, 2017 • edited Loading

lirantal commented Jul 20, 2017

staminna commented Jul 20, 2017 • edited Loading

mleanos commented Jul 27, 2017

staminna commented Jul 27, 2017 • edited Loading

simison commented Aug 14, 2017

lirantal commented Aug 14, 2017

simison commented Aug 14, 2017

mleanos commented Aug 14, 2017 • edited Loading

simison commented Aug 14, 2017

simison commented Aug 14, 2017 • edited Loading

lirantal commented Aug 15, 2017

lirantal commented Jul 8, 2017 •

edited

Loading

staminna commented Jul 8, 2017 •

edited

Loading

staminna commented Jul 14, 2017 •

edited

Loading

simison commented Jul 17, 2017 •

edited

Loading

mleanos commented Jul 18, 2017 •

edited

Loading

simison commented Jul 18, 2017 •

edited

Loading

staminna commented Jul 18, 2017 •

edited

Loading

staminna commented Jul 19, 2017 via email •

edited

Loading

staminna commented Jul 20, 2017 •

edited

Loading

staminna commented Jul 20, 2017 •

edited

Loading

staminna commented Jul 27, 2017 •

edited

Loading

mleanos commented Aug 14, 2017 •

edited

Loading

simison commented Aug 14, 2017 •

edited

Loading