[MongoDB Atlas] Identify a working solution for the MongoDB Atlas integration

[milan-elastic]

Identify a way to implement MongoDB Atlas integration using available beats inputs/modules and analysis of what kind of data will be generated/collected using it.
for more information please find meta here.

[milan-elastic]

For the MongoDB Atlas integration, we have a total of 3 approaches:
1 . MongoDB Atlas metrics can be collected using Prometheus, But we need M10+ cluster license for this approach.
2. MongoDB Atlas data can be collected using REST API, but Digest Authentication is not supported in Http metricbeat module hence we won't be able to make REST API calls using it.
3. We explored CEL input in order to collect MongoDB metrics from the REST API, but in CEL also Digest Authentication is not supported. We have raised issue related to it. So currently we can't use CEL as well

Note: If CEL/Http metricbeat module will support Digest Authentication then we won't need an M10+ cluster license.

[milan-elastic]

Following data can be collected using the prometheus input and filestream input, please find the details below:

No.	Data stream name	Type of data streams (logs/metrics)	Input Type	Description
1	connection	Metrics	Prometheus(Metricbeat module)	A datastream that reports on the status of the connections. Use these values to assess the current load and capacity requirements of the server.
2	network	Metrics	Prometheus(Metricbeat module)	A datastream that reports data on MongoDB's network use.
4	operation	Metrics	Prometheus(Metricbeat module)	A datastream that reports on database operations and database replication operations by type since the mongod instance last started.
5	global_lock	Metrics	Prometheus(Metricbeat module)	A document that reports on the database's lock state.
6	replication	Metrics	Prometheus(Metricbeat module)	A datastream that reports metrics related to the replication process.
7	assert	Metrics	Prometheus(Metricbeat module)	A datastream that reports on the number of assertions raised since the MongoDB process started.
8	wired_tiger	Metrics	Prometheus(Metricbeat module)	wiredTiger information only appears if using the WiredTiger storage engine. Some of the statistics roll up for the server.
9	hardware	Metrics	Prometheus(Metricbeat module)	This datastream collects metrics related to hardware such as disk process etc.
10	activity_log	Logs	Filestream(Filebeat Input)	This datastream collects activity logs of MongoDB atlas.
11	audit_log	Logs	Filestream(Filebeat Input)	Database auditing lets administrators track system activity for deployments with multiple users. Atlas administrators can select the actions, database users, Atlas roles, and LDAP groups that they want to audit.
12	miscellaneous	Metrics	Prometheus(Metricbeat module)	This datastream collects data of the query execution system, memory, document, command.