Multi-app tenancy?

[iOSDev33]

Super sad Parse is closing down. Got a question do we need to have separate parse servers / mongoDB's for each app? How do we migrate multiple Parse hosted apps? Thanks!

[gfosco]

Right now it is intended to run a single app, and in a single database, yes. The capability to run more than one exists (see cache.js and testing-routes.js) but that doesn't allow for different cloud code per app. I'm open to making it multi-app capable.

[kingmatusevich]

This should be done, though notifications and webui are first priority.

[duergner]

I've been thinking about adding provisioning tools that can be used to spin up new instances of parse-server for a new app.

Something like using nginx for loadbalancing to different NodeJS instances. Would make it easier to scale out and wouldn't introduce additional code on the main code base.

Furthermore it'll be easier to move specific apps over to dedicated hardware if they require it.

[lacker]

IMO I would advise people to have separate servers for separate apps. I think it is much cleaner. I think people will disagree on this aesthetically though - it probably depends on your use case.

[prkeshri]

Hello @gfosco , @lacker ,
In the original parse website, we have options to add, and modify multiple app entries. So, is the open source a mini version of that? If that is the case, migrating a large number apps would require to have a large amount of servers.

[DanielsCode]

+1 multi app support would be great

[takusen]

Can't understand why this ticket is closed, as nothing has been decided so far.
IMHO this really is a must-have, I'm sure many of former Parse users were managing more than one app on their account. @lacker, could you explain why you're thinking that one server per app would be the cleanest approach?

[flovilmart]

Due to some restriction in the Parse node.js SDK we're unable to properly manage multiple apps within the same node process. Cloud Code can't support multiple apps as the JS SDK is a singleton and node js is event based. Race conditions occur during asynchronous calls, and the state of the Parse global may change.

You can run multiple parse servers in different node processes with different ports. Depending your configuration, you can achieve routing the traffic with nginx from a single endpoint or from a front express server.

[takusen]

Very clear explanation @flovilmart, now I get it, thanks a lot.

[ericraio]

@flovilmart Sorry to bring up an old thread.

How would you achieve the ability to have the same users across multiple apps with parse?

[flovilmart]

That's not possible across multiple appIds

[ronnno]

@flovilmart Will there by any problem if one of the ParseServer instances does not have any cloud code, and they both share the same appId and Master Key?

The configuration I need is for the dashboard to use readPreference. All I need is to change the mongoDb connection string, so that heavy Dashboard queries will go to my secondary rather than stall the primary. currently I've set this up and it seems to be working. But if it's not recommended or if the databaseUrl can leak between the two servers that is a problem because I assume strict consistency in my cloud code logic..

[flovilmart]

I won't comment as I don't recommend running that setup at all

[ronnno]

Well, after looking in the database logs it looks like most if not all of my queries went to the secondary since I deployed this "multi-server" setup so I got my answer - the connection string leaks as well.

[grosscorporation]

I managed to create a multi-tenant server and using the cloud code that pulls and initialize multiple (different) config json files with unique ports and app credentials and noding the index.js to run sitesObject.forEach(function (sitename) { works well so far, I have 12 apps running with ease, also managed to configure the /dashboard it.
`
const tools = require ( './gross_library/tools' );

// read 'sites' directory
const sitesObject = tools.readDirectories ( './sites' );

// Fire up all 'sites'
sitesObject.forEach ( function ( sitename ) {

const Promise           = require ( "bluebird" );
const express           = require ( 'express' );
const cors              = require ( 'cors' );
const ParseServer       = require ( 'parse-server' ).ParseServer;
const ParseDashboard    = require ( 'parse-dashboard' );
const path              = require ( 'path' );
const nconf             = require ( 'nconf' );
const fs                = require ( 'fs-extra' );
const http              = require ( 'http' );
const https             = require ( 'https' );
const fileUpload        = require ( 'express-fileupload' );
const randomstring      = require ( "randomstring" );
const allowInsecureHTTP = false;

const app = express ();

const sslPath    = '/etc/letsencrypt/live/invoice.api.gogross.com/';
const sslOptions = {
	key : fs.readFileSync ( sslPath + 'privkey.pem', 'utf8' ),
	cert : fs.readFileSync ( sslPath + 'fullchain.pem', 'utf8' )
};

nconf.argv ().env ().file ( { file : './sites/' + sitename + '/config.json' } );

const api = new ParseServer ( {
	databaseURI : 'mongodb://localhost:27017/' + nconf.get ( 'DATABASE_NAME' ),
	cloud : './cloud/main.js',
	appId : nconf.get ( 'APP_ID' ),
	masterKey : nconf.get ( 'MASTER_KEY' ),
	serverURL : nconf.get ( 'SERVER_URL' ),
	fileKey : nconf.get ( 'FILES_ADAPTER' ),
	publicServerURL : nconf.get ( 'SERVER_URL' ),
	appName : 'Gross™ | ' + nconf.get ( 'CLIENT_NAME' ),
	liveQuery : {
		classNames : [ 'customers', 'contacts' ]
	},
	emailAdapter : {
		module : 'parse-server-simple-mailgun-adapter'
	}
} );

const dashboard = new ParseDashboard ( {
	apps : [
		{
			serverURL : nconf.get ( 'SERVER_URL' ),
			appId : nconf.get ( 'APP_ID' ),
			masterKey : nconf.get ( 'MASTER_KEY' ),
			appName : 'Gross™ | ' + nconf.get ( 'CLIENT_NAME' )
		}
	], users : [
		{
			"user" : "user",
			"pass" : "admin"
		}
	]
}, allowInsecureHTTP );

app.use ( cors () );

app.set ( 'view engine', 'html' );
app.set ( 'view engine', 'ejs' );

app.use ( '/app', express.static ( path.join ( __dirname, '/app' ) ) );

// app.use('/install', express.static(path.join(__dirname, '/install')));
app.use ( '/public', express.static ( path.join ( __dirname, '/public' ) ) );

// make the Parse Dashboard available at /dashboard
app.use ( '/dashboard', dashboard );

app.use ( '/sites/' + sitename + '/_temp-reports', express.static ( path.join ( __dirname, '/sites/' + sitename + '/_temp-reports' ) ) );

app.get ( '/invoice-print/:objectId', function ( req, res, next ) {
	Parse.Cloud.run ( 'pdfInvoice', {
		invoiceNumber : req.params.objectId,
		invoiceSettings : {}
	} ).then ( function ( invoice ) {
		var tempFile = invoice;
		fs.readFile ( tempFile, function ( err, data ) {
			res.contentType ( "application/pdf" );
			res.send ( data );
		} );
	} );
} );

app.get ( '/quotation-print/:objectId', function ( req, res, next ) {
	Parse.Cloud.run ( 'pdfQuote', {
		invoiceNumber : req.params.objectId,
		invoiceSettings : {}
	} ).then ( function ( invoice ) {
		var tempFile = invoice;
		fs.readFile ( tempFile, function ( err, data ) {
			res.contentType ( "application/pdf" );
			res.send ( data );
		} );
	} );
} );

app.get ( '/purchase-order-print/:objectId', function ( req, res, next ) {
	Parse.Cloud.run ( 'pdfOrder', {
		invoiceNumber : req.params.objectId,
		invoiceSettings : {}
	} ).then ( function ( invoice ) {
		var tempFile = invoice;
		fs.readFile ( tempFile, function ( err, data ) {
			res.contentType ( "application/pdf" );
			res.send ( data );
		} );
	} );
} );

app.get ( '/print-payment/:paymentPrint', function ( req, res, next ) {
	const filePath = "/_temp-reports/" + req.params.paymentPrint;
	fs.readFile ( __dirname + filePath, function ( err, data ) {
		res.contentType ( "application/pdf" );
		
		res.send ( data );
	} );
} );

const mountPath = nconf.get ( 'PARSE_MOUNT_PATH' );
app.use ( mountPath, api );

app.get ( '/', function ( req, res, next ) {
	res.status ( 200 ).send ( 'Gross™ Systems INC' );
} );

const port       = nconf.get ( 'SERVER_PORT' );
const clientName = nconf.get ( 'CLIENT_NAME' );

const server = https.createServer ( sslOptions, app ).listen ( port, function () {
	console.log ( 'Gross™ https://invoice.api.gogross.com:' + port + ' / Client: ' + clientName + '.' );
} );

ParseServer.createLiveQueryServer ( server );

} );

// create a site
// required: clientName, clientURL
Parse.Cloud.define ( 'createNewSite', function ( request, response ) {

const fq = request.params;

if ( fq.adminKey === "tRAAsoO5jqjr4KL2J8mS1dbICDR78OHw" && fq.password === "@An716288" ) {
	
	const captureDatabaseName = new Promise ( function ( resolve, reject ) {
		const databaseName = randomstring.generate ();
		resolve ( databaseName );
		
	} );
	const createDatabase      = new Promise ( function ( resolve, reject ) {
		const MongoClient = require ( 'mongodb' ).MongoClient;
		const url         = "mongodb://localhost:27017/" + databaseName; // mydatabase is the name of db
		MongoClient.connect ( url, function ( err, db ) {
			if ( err ) {
				throw err;
			} else {
				
				const siteSettings = {
					"DATABASE_NAME" : databaseName,
					"PARSE_MOUNT_PATH" : "/parse",
					"SERVER_URL" : "https://invoice.api.gogross.com:10006/parse",
					"APP_ID" : randomstring.generate (),
					"MASTER_KEY" : randomstring.generate (),
					"FILE_KEY" : randomstring.generate (),
					"REST_KEY" : randomstring.generate (),
					"JS_KEY" : randomstring.generate (),
					"FILES_ADAPTER" : randomstring.generate (),
					"CLIENT_NAME" : fq.clientName,
					"FOLDER_ID" : "_default",
					"SERVER_PORT" : 10006
				};
				
				tools.createSite ( fq.clientURL, siteSettings );
				
				console.log ( "Database created!" );
				
				db.close ();
			}
		} );
	} );
	const installDemo         = new Promise ( function ( resolve, reject ) {
		setTimeout ( resolve, 1000, 'done three' );
	} );
	Promise.all ( [
		captureDatabaseName,
		createDatabase,
		installDemo
	] ).then ( function ( values ) {
		
		console.log ( values ); // [3, 1337, "foo"]
		
	} );
	
} else {
	response.error ( 'Site not created, check privileges' )
}

} );

`

[flovilmart]

@tinocosta84, I strongly suggest that you cease running that code in production. I probably works because you don’t have many concurrent requests nor async code in CloudCode that takes long enough so the Parse SDK get re-initialized by a request coming in to another app.

[grosscorporation]

@flovilmart Running pretty well for now with no isolation as I deployed pm2 ecosystems, I had as much doubt but even all the modules for all the apps are fired up. Its perhaps best to suggest a fewer number of apps as I am testing it on a 2GB RAM

[flovilmart]

It’s not about the RAM! It’s about concurrent requests, Cloud Code and calls against multiple apps. Again, this is explicitly discouraged and yes the side effects are not obvious, instead of going butt-head and claiming this is a valid method, I can guarantee that it isn’t.

[benishak]

@flovilmart

Do you know how Parse.com managed multiple apps? did they use AWS Lambda or something else? I'm talking from generally from a technical perspective not referring to enable this in Parse Server

[natanrolnik]

@benishak Parse.com backend initially started with Ruby on Rails, and later on they changed it to a Go backend. You can read about it here.

[benishak]

Thanks a lot that sounds good but Cloud Code was always JavaScript. But how did they manage all these cloud code, I don't think they were loading all the cloud codes in memory at the startup, that's not going to scale whether they used Go or NodeJS, right?

[ronnno]

i think they launched a dedicated node process for each incoming request. i read this somewhere, and it's supported by the fact that user information is useMasterKey were global/static variables in node - the entire process isolated for each request. that is very aggressive segregation at the request level...

…

__________________________ Ron Bresler | Milestone Sports Director of Product Management +972 54.815.1274

On Mon, Jul 3, 2017 at 12:28 PM, Ben Ishak ***@***.***> wrote: Thanks a lot that sounds good but Cloud Code was always JavaScript. But how did they manage all these cloud code, I don't think they were loading all the cloud codes in memory at the startup, that's not going to scale whether they used Go or NodeJS, right? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#15 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APEWUK74j7UY2XBgPJ7XjULt5BZOaQW7ks5sKLRPgaJpZM4HO9hN> .

[flovilmart]

Yes, an isolated custom node runtime was launched for every hook, which was slow AF. This works well if you run everything on beefy machines, as the process startup time and memory consumption isn’t a big dead, but in the scenario of parse-server, this doesn’t make sense.

Initially when I was working on it, I made Cloud Code run in a separate process (started at the beginning), and communicate through HTTP to it (so I didn’t have to rewrite an IPC protocol). This made multitenancy possible but ultimately rejected as a PR. If you wanna read the discussion, #263

[flovilmart]

@zealmurapa I'm not sure what you mean or where you wanna go with it. Multi-tenancy is likely to never land, and is not a priority.

[richjing]

I developed multiple apps parse server in the safe way(not that efficiency but should not having the concurrent issue in cloud code), and make it easy to setup, take a look on this repo

[kerbymart]

@richjing did you manage to run multiple apps on 1 Parse server without issues with Cloud Code?

[richjing]

@kerbymart I use PM2 to start multiple parse server apps. Therefore, there is no cloud code concurrent issue. Each app run separately.
I create the repo named multiple-apps-parse-server, you can take a look on it.
Here show some features:

• run and manage multiple parse apps (instances) in a server and using a single port.
• one code, one database, create a parse app in one second.
• parse dashboard integrated, each app's manager can log into parse dashboard to manage their app.
• one admin account in parse dashboard to manage all apps

[chungminhtu]

@kerbymart I use PM2 to start multiple parse server apps. Therefore, there is no cloud code concurrent issue. Each app run separately. I create the repo named multiple-apps-parse-server, you can take a look on it. Here show some features:

• run and manage multiple parse apps (instances) in a server and using a single port.
• one code, one database, create a parse app in one second.
• parse dashboard integrated, each app's manager can log into parse dashboard to manage their app.
• one admin account in parse dashboard to manage all apps

This is really great and it works for me.