Skip to content

Commit

Permalink
Start generation
Browse files Browse the repository at this point in the history 
Signed-off-by: Jack Baldry <[email protected]>
  • Loading branch information
@jdbaldry
jdbaldry committed Jun 17, 2024
1 parent 9748e07 commit 01904d0
Show file tree
Hide file tree
Showing 4 changed files with 195 additions and 112 deletions.
39 changes: 18 additions & 21 deletions loki/loki-quickstart/index.json
View file
Original file line number Diff line number Diff line change
@@ -1,24 +1,21 @@
{
"title": "Loki Quickstart Demo",
"description": "This sandbox provides an online enviroment for testing the Loki quickstart demo.",
"details": {
"intro": {
"text": "intro.md"
"title": "Loki Quickstart Demo",
"description": "This sandbox provides an online enviroment for testing the Loki quickstart demo.",
"details": {
"intro": {
"text": "intro.md"
},
"steps": [
{
"text": "step1.md"
},
"steps": [
{
"text": "step1.md"
},
{
"text": "step2.md"
}
],
"finish": {
"text": "finished.md"
{
"text": "step2.md"
}
}
,
"backend": {
"imageid": "ubuntu"
}
}
],
"finished": {}
},
"backend": {
"imageid": "ubuntu"
}
}
11 changes: 6 additions & 5 deletions loki/loki-quickstart/intro.md
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
![Loki Quickstart](../../assets/loki-ile.png)


# Quickstart to run Loki locally

If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki/latest/get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs.
If you want to experiment with Loki, you can run Loki locally using the Docker Compose file that ships with Loki. It runs Loki in a [monolithic deployment](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/deployment-modes/#monolithic-mode) mode and includes a sample application to generate logs.

The Docker Compose configuration instantiates the following components, each in its own container:

- **flog** a sample application which generates log lines. [flog](https://github.com/mingrammer/flog) is a log generator for common log formats.
- **flog** a sample application which generates log lines.
[flog](https://github.com/mingrammer/flog) is a log generator for common log formats.
- **Grafana Alloy** which scrapes the log lines from flog, and pushes them to Loki through the gateway.
- **Gateway** (NGINX) which receives requests and redirects them to the appropriate container based on the request's URL.
- **Gateway** (nginx) which receives requests and redirects them to the appropriate container based on the request's URL.
- One Loki **read** component (Query Frontend, Querier).
- One Loki **write** component (Distributor, Ingester).
- One Loki **backend** component (Index Gateway, Compactor, Ruler, Bloom Compactor (Experimental), Bloom Gateway (Experimental)).
- **Minio** an S3-compatible object store which Loki uses to store its index and chunks.
- **Grafana** which provides visualization of the log lines captured within Loki.

![Loki Quickstart](https://grafana.com/media/docs/loki/get-started-flog-v3.png)
{{< figure max-width="75%" src="/media/docs/loki/get-started-flog-v3.png" caption="Getting started sample application" alt="Getting started sample application">}}

87 changes: 53 additions & 34 deletions loki/loki-quickstart/step1.md
View file
Original file line number Diff line number Diff line change
@@ -1,49 +1,68 @@

# Step 1: Installing Loki and collecting sample logs

## Install Loki and collecting sample logs



**To install Loki locally, follow these steps:**

1. Create a directory called `evaluate-loki` for the demo environment. Make `evaluate-loki` your current working directory:
1. Create a directory called `evaluate-loki` for the demo environment.
Make `evaluate-loki` your current working directory:

<!-- Killercoda execute START -->

```bash
mkdir evaluate-loki
cd evaluate-loki
```execute
<!-- Killercoda execute END -->
1. Download `loki-config.yaml`, `alloy-local-config.yaml`, and `docker-compose.yaml`:
<!-- Killercoda execute START -->
```bash
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml
```execute
<!-- Killercoda execute END -->
```bash
mkdir evaluate-loki
cd evaluate-loki
```{{execute}}
1. Deploy the sample Docker image.
2. Download `loki-config.yaml`, `alloy-local-config.yaml`, and `docker-compose.yaml`:
With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`:
```bash
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/loki-config.yaml -O loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/alloy-local-config.yaml -O alloy-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/examples/getting-started/docker-compose.yaml -O docker-compose.yaml
```{{execute}}
<!-- Killercoda execute START -->
3. Deploy the sample Docker image.
```bash
docker compose up -d
```execute
With `evaluate-loki` as the current working directory, start the demo environment using `docker compose`:
<!-- Killercoda execute END -->
```bash
docker-compose up -d
```{{execute}}
You should see something similar to the following:
You should see something similar to the following:
```bash
✔ Network evaluate-loki_loki Created 0.1s
✔ Container evaluate-loki-minio-1 Started 0.6s
✔ Container evaluate-loki-flog-1 Started 0.6s
✔ Container evaluate-loki-backend-1 Started 0.8s
✔ Container evaluate-loki-write-1 Started 0.8s
✔ Container evaluate-loki-read-1 Started 0.8s
✔ Container evaluate-loki-gateway-1 Started 1.1s
✔ Container evaluate-loki-grafana-1 Started 1.4s
✔ Container evaluate-loki-alloy-1 Started 1.4s
```

```bash
✔ Network evaluate-loki_loki Created 0.1s
✔ Container evaluate-loki-minio-1 Started 0.6s
✔ Container evaluate-loki-flog-1 Started 0.6s
✔ Container evaluate-loki-backend-1 Started 0.8s
✔ Container evaluate-loki-write-1 Started 0.8s
✔ Container evaluate-loki-read-1 Started 0.8s
✔ Container evaluate-loki-gateway-1 Started 1.1s
✔ Container evaluate-loki-grafana-1 Started 1.4s
✔ Container evaluate-loki-alloy-1 Started 1.4s
```
1. (Optional) Verify that the Loki cluster is up and running.

4. (Optional) Verify that the Loki cluster is up and running.
- The read component returns `ready` when you point a web browser at [http://localhost:3101/ready]({{TRAFFIC_HOST1_3101}}/ready). The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` will show prior to the read component being ready.
- The write component returns `ready` when you point a web browser at [http://localhost:3102/ready]({{TRAFFIC_HOST1_3102}}/ready). The message `Ingester not ready: waiting for 15s after being ready` will show prior to the write component being ready.
- The read component returns `ready` when you browse to [http://localhost:3101/ready](http://localhost:3101/ready).
The message `Query Frontend not ready: not ready: number of schedulers this worker is connected to is 0` shows until the read component is ready.
- The write component returns `ready` when you browse to [http://localhost:3102/ready](http://localhost:3102/ready).
The message `Ingester not ready: waiting for 15s after being ready` shows until the write component is ready.

5. (Optional) Verify that Grafana Alloy is running.
- Grafana Alloy's UI can be accessed at [http://localhost:12345]({{TRAFFIC_HOST1_12345}}).
1. (Optional) Verify that Grafana Alloy is running.
- You can access the Grafana Alloy UI at [http://localhost:12345](http://localhost:12345).

170 changes: 118 additions & 52 deletions loki/loki-quickstart/step2.md
View file
Original file line number Diff line number Diff line change
@@ -1,105 +1,171 @@
# Viewing your logs in Grafana

Once you have collected logs, you will want to view them. You can view your logs using the command line interface, [LogCLI](/docs/loki/latest/query/logcli/), but the easiest way to view your logs is with Grafana.

1. Use Grafana to query the Loki data source.
## View your logs in Grafana

The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application. You can access the Grafana cluster by navigating to [http://localhost:3000]({{TRAFFIC_HOST1_3000}}). The Grafana instance provided with this demo has a Loki [datasource](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured.
After you have collected logs, you will want to view them.
You can view your logs using the command line interface, [LogCLI](/docs/loki/<LOKI_VERSION>/query/logcli/), but the easiest way to view your logs is with Grafana.

![Grafana Query Builder](https://grafana.com/media/docs/loki/grafana-query-builder-v2.png)
1. Use Grafana to query the Loki data source.

2. From the Grafana main menu, click the **Explore** icon (1) to launch the Explore tab. To learn more about Explore, refer the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation.
The test environment includes [Grafana](https://grafana.com/docs/grafana/latest/), which you can use to query and observe the sample logs generated by the flog application.

3. From the menu in the dashboard header, select the Loki data source (2). This displays the Loki query editor. In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki/latest/query/), to query your logs.
To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/).
You can access the Grafana cluster by browsing to [http://localhost:3000](http://localhost:3000).

4. The Loki query editor has two modes (3):
The Grafana instance in this demonstration has a Loki [data source](https://grafana.com/docs/grafana/latest/datasources/loki/) already configured.

{{< figure src="/media/docs/loki/grafana-query-builder-v2.png" caption="Grafana Explore" alt="Grafana Explore">}}

1. From the Grafana main menu, click the **Explore** icon (1) to open the Explore tab.

To learn more about Explore, refer to the [Explore](https://grafana.com/docs/grafana/latest/explore/) documentation.

1. From the menu in the dashboard header, select the Loki data source (2).

This displays the Loki query editor.

In the query editor you use the Loki query language, [LogQL](https://grafana.com/docs/loki/<LOKI_VERSION>/query/), to query your logs.
To learn more about the query editor, refer to the [query editor documentation](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/).

1. The Loki query editor has two modes (3):

- [Builder mode](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/#builder-mode), which provides a visual query designer.
- [Code mode](https://grafana.com/docs/grafana/latest/datasources/loki/query-editor/#code-mode), which provides a feature-rich editor for writing LogQL queries.

Next we’ll walk through a few simple queries using both the builder and code views.
1. Click **Code** (3) to work in Code mode in the query editor.

Here are some sample queries to get you started using LogQL.
These queries assume that you followed the instructions to create a directory called `evaluate-loki`.

If you installed in a different directory, you’ll need to modify these queries to match your installation directory.

After copying any of these queries into the query editor, click **Run Query** (4) to execute the query.

1. View all the log lines which have the container label `evaluate-loki-flog-1`:

<!-- Killercoda copy START -->

```bash
{container="evaluate-loki-flog-1"}
```copy
<!-- Killercoda copy END -->
In Loki, this is a log stream.
Loki uses [labels](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/labels/) as metadata to describe log streams.
5. Click **Code** (3) to work in Code mode in the query editor.
Loki queries always start with a label selector.
In the previous query, the label selector is `{container="evaluate-loki-flog-1"}`.
Here are some basic sample queries to get you started using LogQL. Note that these queries assume that you followed the instructions to create a directory called `evaluate-loki`. If you installed in a different directory, you’ll need to modify these queries to match your installation directory. After copying any of these queries into the query editor, click **Run Query** (4) to execute the query.
1. To view all the log lines which have the container label `evaluate-loki-grafana-1`:
1. View all the log lines which have the container label "flog":
<!-- Killercoda copy START -->
```bash
{container="evaluate-loki_flog_1"}
```{{copy}}
```bash
{container="evaluate-loki-grafana-1"}
```copy
In Loki, this is called a log stream. Loki uses [labels](https://grafana.com/docs/loki/latest/get-started/labels/) as metadata to describe log streams. Loki queries always start with a label selector. In the query above, the label selector is `container`.
<!-- Killercoda copy END -->
2. To view all the log lines which have the container label "grafana":
1. Find all the log lines in the `{container="evaluate-loki-flog-1}` stream that contain the string `status`:
```bash
{container="evaluate-loki_grafana_1"}
```{{copy}}
<!-- Killercoda copy START -->
3. Find all the log lines in the container=flog stream that contain the string "status":
```bash
{container="evaluate-loki-flog-1"} |= `status`
```copy
```bash
{container="evaluate-loki_flog_1"} |= "status"
```{{copy}}
<!-- Killercoda copy END -->
4. Find all the log lines in the container=flog stream where the JSON field "status" is "404":
1. Find all the log lines in the `{container="evaluate-loki-flog-1}` stream where the JSON field `status` has the value `404`:
```bash
{container="evaluate-loki_flog_1"} | json | status="404"
```{{copy}}
<!-- Killercoda copy START -->
5. Calculate the number of logs per second where the JSON field "status" is "404":
```bash
{container="evaluate-loki-flog-1"} | json | status=`404`
```copy
```bash
sum by(container) (rate({container="evaluate-loki_flog_1"} | json | status="404" [$__auto]))
```{{copy}}
<!-- Killercoda copy END -->
The final query above is a metric query which returns a time series. This will trigger Grafana to draw a graph of the results. You can change the type of graph for a different view of the data. Click **Bars** to view a bar graph of the data.
1. Calculate the number of logs per second where the JSON field `status` has the value `404`:
6. Click the **Builder** tab (3) to return to Builder mode in the query editor.
1. In Builder view, click **Kick start your query**(5).
2. Expand the **Log query starters** section.
3. Select the first choice, **Parse log lines with logfmt parser**, by clicking **Use this query**.
4. On the Explore tab, click **Label browser**, in the dialog select a container and click **Show logs**.
<!-- Killercoda copy START -->
For a thorough introduction to LogQL, refer to the [LogQL reference](https://grafana.com/docs/loki/latest/query/).
```bash
sum by(container) (rate({container="evaluate-loki-flog-1"} | json | status=`404` [$__auto]))
```copy
<!-- Killercoda copy END -->
The final query is a metric query which returns a time series.
This makes Grafana draw a graph of the results.
You can change the type of graph for a different view of the data.
Click **Bars** to view a bar graph of the data.
1. Click the **Builder** tab (3) to return to builder mode in the query editor.
1. In builder mode, click **Kick start your query** (5).
1. Expand the **Log query starters** section.
1. Select the first choice, **Parse log lines with logfmt parser**, by clicking **Use this query**.
1. On the Explore tab, click **Label browser**, in the dialog select a container and click **Show logs**.
For a thorough introduction to LogQL, refer to the [LogQL reference](https://grafana.com/docs/loki/<LOKI_VERSION>/query/).
## Sample queries (code view)
Here are some more sample queries that you can run using the Flog sample data.
To see all the log lines that flog has generated, enter the LogQL query:
<!-- Killercoda copy START -->
```bash
{container="evaluate-loki_flog_1"}|= ""
```{{copy}}
{container="evaluate-loki-flog-1"}
```copy
<!-- Killercoda copy END -->
The flog app generates log lines for simulated HTTP requests.
To see all `GET` log lines, enter the LogQL query:
<!-- Killercoda copy START -->
```bash
{container="evaluate-loki_flog_1"} |= "GET"
```{{copy}}
{container="evaluate-loki-flog-1"} |= "GET"
```copy
<!-- Killercoda copy END -->
To see all `POST` methods, enter the LogQL query:
<!-- Killercoda copy START -->
```bash
{container="evaluate-loki_flog_1"} |= "POST"
```{{copy}}
{container="evaluate-loki-flog-1"} |= "POST"
```copy
<!-- Killercoda copy END -->
To see every log line with a 401 status (unauthorized error), enter the LogQL query:
<!-- Killercoda copy START -->
```bash
{container="evaluate-loki_flog_1"} | json | status="401"
```{{copy}}
{container="evaluate-loki-flog-1"} | json | status="401"
```copy
To see every log line that does not contain the value 401:
<!-- Killercoda copy END -->
To see every log line that doesn't contain the text `401`:
<!-- Killercoda copy START -->
```bash
{container="evaluate-loki_flog_1"} != "401"
```{{copy}}
{container="evaluate-loki-flog-1"} != "401"
```copy
<!-- Killercoda copy END -->
For more examples, refer to the [query documentation](https://grafana.com/docs/loki/<LOKI_VERSION>/query/query_examples/).
For more examples, refer to the [query documentation](https://grafana.com/docs/loki/latest/query/query_examples/).

0 comments on commit 01904d0

Please sign in to comment.