Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README #62

Merged
merged 1 commit into from
Mar 26, 2024
Merged

Update README #62

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions architecture/logging/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,33 @@
# Logging System

For better understanding of the development, test, deploy and operation, the following diagram identified several parts of the system that composed the structure of the logging system in NERC.
The following diagram identifies several parts of the system that compose the structure of the NERC logging system, helping better explain development, testing, deployment, and operation of the logging system in NERC.


![plot](./images/Logging_Logical_Diagram.jpg)


## Schematic Diagram

Logs are written by the clusters managed by [Advanced Cluster Management (ACM)](https://www.redhat.com/en/technologies/management/advanced-cluster-management) and read by the Cluster admins with read permission only.
Logs are written by [Advanced Cluster Management (ACM)](https://www.redhat.com/en/technologies/management/advanced-cluster-management) clusters and read by the Cluster admins with read permission only.

The below diagram depicts the read and write flow. However, please bear in mind that only essential components are identified for the sake of brevity as lots are happening behind the scene.
The diagram below depicts the read-and-write flow. However, please remember that only essential components are identified for brevity.

![plot](./images/Logging_Schematic_Diagram.jpg)


## Read path

1. The read request comes from the admin users who have the permission (by adding into [nerc-logs-metrics](https://github.com/orgs/OCP-on-NERC/teams/nerc-logs-metrics) group in GitHub) to read the logs from the OpenShift Production (as an example) Cluster Console.
1. The read request comes from the admin users who have permission (by adding them to the [nerc-logs-metrics](https://github.com/orgs/OCP-on-NERC/teams/nerc-logs-metrics) group in GitHub) to read the logs from the OpenShift Production (for example) Cluster Console.

2. The query request is sent to the LokiStack in the Observability cluster and is handled by the Querier service. This service processes queries using [LogQL](https://grafana.com/docs/loki/latest/query/) query language, fetching logs both from the ingesters and from the long-term storage.
2. The query request is sent to LokiStack in the Observability cluster and handled by the Querier service. This service processes queries using the [LogQL](https://grafana.com/docs/loki/latest/query/) query language, fetching logs from both the ingesters and the long-term storage.

3. To fetch the log data from the long-term storage, the request is forwarded to the NooBaa operator, one of the operators in the [OpenShift Data Foundation](https://www.redhat.com/en/technologies/cloud-computing/openshift-data-foundation) installed in Observability cluster. This operator is responsible to manage [Multicloud Object Gateway](https://www.redhat.com/en/blog/introducing-multi-cloud-object-gateway-for-openshift) components, such as object bucket claim, bucket classes, and backing store in the OpenShift Data Foundation.

4. The backing store handles the read and write request from the customer, as it provides MCG the ability to save the data of the provisioned bucket on top of it. In this architecture, the backing store is [AWS:S3 bucket](https://aws.amazon.com/pm/serv-s3/?gclid=EAIaIQobChMIodfHoK2egwMVwuXVCh2cWw-1EAAYASAAEgLCNvD_BwE&trk=518a7bef-5b4f-4462-ad55-80e5c177f12b&sc_channel=ps&ef_id=EAIaIQobChMIodfHoK2egwMVwuXVCh2cWw-1EAAYASAAEgLCNvD_BwE:G:s&s_kwcid=AL!4422!3!645186213484!e!!g!!aws%20s3!19579892800!143689755565).
4. The backing store handles the read and write request from the customer, as it allows MCG to save the data of the provisioned bucket on top of it. In this architecture, the backing store is [AWS:S3 bucket](https://aws.amazon.com/pm/serv-s3/?gclid=EAIaIQobChMIodfHoK2egwMVwuXVCh2cWw-1EAAYASAAEgLCNvD_BwE&trk=518a7bef-5b4f-4462-ad55-80e5c177f12b&sc_channel=ps&ef_id=EAIaIQobChMIodfHoK2egwMVwuXVCh2cWw-1EAAYASAAEgLCNvD_BwE:G:s&s_kwcid=AL!4422!3!645186213484!e!!g!!aws%20s3!19579892800!143689755565).

6. A S3 bucket is provisioned by a [Object Bucket Claim](https://access.redhat.com/documentation/en-us/red_hat_openshift_container_storage/4.8/html/managing_hybrid_and_multicloud_resources/object-bucket-claim˘) for storing data. And the route for the S3 access is provided by the internal CRD called NooBaa.

> NERC logs are stored inside Log Bucket. Since the logs are purged to keep log size under control, there is also backup of the logs and its corresponding ObjectBucketClaim. The read and write request of logs only involve this Log bucket from any ACM cluster, however Observability cluster can make read request to the Log Backup bucket as well.<br/><br/>The data sync between these two buckets is provisioned by mirroring feature in OpenShift Data Foundation.
> NERC logs are stored inside Log Bucket. Since the logs are purged to keep log size under control, there is also backup of the logs and its corresponding ObjectBucketClaim. The read and write request of logs only involve this Log bucket from any ACM cluster, however Observability cluster can make read request to the Log Backup bucket as well.<br/><br/>The data sync between these two buckets is provisioned by the mirroring feature in OpenShift Data Foundation.

7. After getting access to the logs from the bucket, the query returns the data following the reverse path. Query result is sent to the Query frontend instead of Querier to display to the admin users who requested for the log data.

Expand All @@ -43,8 +43,8 @@ To aggregate the logs (application, infrastructure, and audit) the [Cluster Logg

3. Logs are then forwarded to the observability cluster where LokiStack is deployed and is used as external log store for the logging system in NERC.

> LokiStack refers to the logging subsystem supported combination of Loki and web proxy with OpenShift Container Platform authentication integration.
> LokiStack is the logging subsystem supported by a combination of Loki and web proxy with OpenShift Container Platform authentication integration.

4. The forwarded logs are ingested by the Ingester pods of Loki from the Distributor which does validation of logs. Ingesters are responsible for writing the logs to backup store.
4. The forwarded logs are ingested by Loki's ingester pods from the Distributor, which validates the logs. Ingesters are responsible for writing the logs to the backup store.

5. The LOKI ingesters send the data to NooBaa operator to access the log bucket to write the data which is already setup earlier by OpenShift Data Foundation components.
5. The LOKI ingesters send the data to the NooBaa operator, who accesses the log bucket and writes the data, which is already set up by OpenShift Data Foundation components.