Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement vector() function #6946

Closed
tonychoe opened this issue Aug 19, 2022 · 2 comments · Fixed by #7007, #7023 or #7044
Closed

Implement vector() function #6946

tonychoe opened this issue Aug 19, 2022 · 2 comments · Fixed by #7007, #7023 or #7044

Comments

@tonychoe
Copy link
Contributor

tonychoe commented Aug 19, 2022 •
edited
Loading

Is your feature request related to a problem? Please describe.
I want to turn null (no data) to zero in query result in easy and reliable way.

There are other Loki users who want the same.
https://community.grafana.com/t/how-can-i-turn-no-data-to-zero-in-loki/40694

Describe the solution you'd like
Implement a function like vector() in PromQL.

vector(s scalar) returns the scalar s as a vector with no labels.
Then users can join their query with vector() to turn NULL to zero, as below.

vector1
or
vector(0)

Describe alternatives you've considered
(1) Grafana's transform feature.
However, this feature only works for a dashboard panel with single Loki query. It doesn't work for multiple queries for a panel.

(2) Tried to generate zeros reliably, but not perfect

sum(count_over_time({namespace="traefik"}|="foo"[1m])) 
or
sum(count_over_time({namespace="traefik"}[1m])) < bool 0

The 2nd vector can still return NULL if there is no log data matching labels sets. Not useful for log streams which has fluctuating log ingestion.

(3) Tried absent_over_time() as 2nd vector instead, it didn't work for me.

(4) First, stored the log-metrics in Prometheus. Second, use vector() in Prometheus.
This works of course. But I wish I can do the same using LokiQL which is upstream.

Additional context
@liguozhong liguozhong mentioned this issue Aug 31, 2022
Merged
4 tasks
dannykopping pushed a commit that referenced this issue Sep 1, 2022
@liguozhong
[new feature] syntax: support "vector" grama (#7007)
6f4671a 
<!--  Thanks for sending a pull request!  Before submitting:

1. Read our CONTRIBUTING.md guide
2. Name your PR as `<Feature Area>: Describe your change`.
  a. Do not end the title with punctuation. It will be added in the changelog.
  b. Start with an imperative verb. Example: Fix the latency between System A and System B.
  c. Use sentence case, not title case.
  d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be.
3. Rebase your PR if it gets out of sync with main
-->

**What this PR does / why we need it**:
Implement vector() function.

The "vector function" feature is divided into multiple PRs to avoid excessively large PRs resulting in too long review time.

**Which issue(s) this PR fixes**:
Fixes #6946

**Special notes for your reviewer**:
The test for syntax has passed
![image](https://user-images.githubusercontent.com/9583245/187588436-7c2c401f-1d88-4bff-a41a-965d36454ebe.png)

<!--
Note about CHANGELOG entries, if a change adds:
* an important feature
* fixes an issue present in a previous release, 
* causes a change in operation that would be useful for an operator of Loki to know
then please add a CHANGELOG entry.

For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki.

Note about the upgrade guide, if this changes:
* default configuration values
* metric names or label names
* changes existing log lines such as the metrics.go query output line
* configuration parameters 
* anything to do with any API
* any other change that would require special attention or extra steps to upgrade
Please document clearly what changed AND what needs to be done in the upgrade guide.
-->
**Checklist**
- [ ] Documentation added
- [ ] Tests updated
- [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
@dannykopping dannykopping reopened this Sep 1, 2022
@dannykopping
Copy link
Contributor

@liguozhong I'm reopening this since your PR doesn't fully implement this yet, since you're kindly splitting up the work into multiple PRs.

@liguozhong liguozhong mentioned this issue Sep 1, 2022
Merged
4 tasks
owen-d pushed a commit that referenced this issue Sep 2, 2022
@liguozhong
[new feature] logql engine support exec vector(0) grama (#7023)
3296c6d 
**What this PR does / why we need it**:
logql engine support exec `vector(0)` grama.
new PR of :#7007
**Which issue(s) this PR fixes**:
Fixes #6946

**Special notes for your reviewer**:
test passed.
test1: vectorExpr only
![image](https://user-images.githubusercontent.com/9583245/187871060-2ce63f01-5613-4a1e-9a52-59712f32cc7c.png)

test2: binExpr and vectorExpr
![image](https://user-images.githubusercontent.com/9583245/188090201-702c1ec4-7c37-41cb-9f2f-8f137a39dadf.png)

**Checklist**
- [ ] Documentation added
- [x] Tests updated
- [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
@liguozhong liguozhong mentioned this issue Sep 5, 2022
Merged
4 tasks
dannykopping added a commit that referenced this issue Sep 5, 2022
@liguozhong @dannykopping
[doc] logql: logql engine support exec vector(0) grama (#7044)
93a5a71 
<!--  Thanks for sending a pull request!  Before submitting:

1. Read our CONTRIBUTING.md guide
2. Name your PR as `<Feature Area>: Describe your change`.
  a. Do not end the title with punctuation. It will be added in the changelog.
  b. Start with an imperative verb. Example: Fix the latency between System A and System B.
  c. Use sentence case, not title case.
  d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be.
3. Rebase your PR if it gets out of sync with main
-->

**What this PR does / why we need it**:
logql engine support exec vector(0) grama.
new PR of :#7023

**Which issue(s) this PR fixes**:
Fixes #6946

**Special notes for your reviewer**:
preview
![image](https://user-images.githubusercontent.com/9583245/188355186-9a74d1ce-f062-45e8-8516-52c89383eeec.png)

<!--
Note about CHANGELOG entries, if a change adds:
* an important feature
* fixes an issue present in a previous release, 
* causes a change in operation that would be useful for an operator of Loki to know
then please add a CHANGELOG entry.

For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki.

Note about the upgrade guide, if this changes:
* default configuration values
* metric names or label names
* changes existing log lines such as the metrics.go query output line
* configuration parameters 
* anything to do with any API
* any other change that would require special attention or extra steps to upgrade
Please document clearly what changed AND what needs to be done in the upgrade guide.
-->
**Checklist**
- [ ] Documentation added
- [ ] Tests updated
- [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`

Co-authored-by: Danny Kopping <[email protected]>
lxwzy pushed a commit to lxwzy/loki that referenced this issue Nov 7, 2022
@liguozhong @lxwzy
[new feature] syntax: support "vector" grama (grafana#7007)
1e5b35f 
<!--  Thanks for sending a pull request!  Before submitting:

1. Read our CONTRIBUTING.md guide
2. Name your PR as `<Feature Area>: Describe your change`.
  a. Do not end the title with punctuation. It will be added in the changelog.
  b. Start with an imperative verb. Example: Fix the latency between System A and System B.
  c. Use sentence case, not title case.
  d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be.
3. Rebase your PR if it gets out of sync with main
-->

**What this PR does / why we need it**:
Implement vector() function.

The "vector function" feature is divided into multiple PRs to avoid excessively large PRs resulting in too long review time.

**Which issue(s) this PR fixes**:
Fixes grafana#6946

**Special notes for your reviewer**:
The test for syntax has passed
![image](https://user-images.githubusercontent.com/9583245/187588436-7c2c401f-1d88-4bff-a41a-965d36454ebe.png)

<!--
Note about CHANGELOG entries, if a change adds:
* an important feature
* fixes an issue present in a previous release, 
* causes a change in operation that would be useful for an operator of Loki to know
then please add a CHANGELOG entry.

For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki.

Note about the upgrade guide, if this changes:
* default configuration values
* metric names or label names
* changes existing log lines such as the metrics.go query output line
* configuration parameters 
* anything to do with any API
* any other change that would require special attention or extra steps to upgrade
Please document clearly what changed AND what needs to be done in the upgrade guide.
-->
**Checklist**
- [ ] Documentation added
- [ ] Tests updated
- [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
lxwzy pushed a commit to lxwzy/loki that referenced this issue Nov 7, 2022
@liguozhong @lxwzy
[new feature] logql engine support exec vector(0) grama (grafana#7023)
331ef22 
**What this PR does / why we need it**:
logql engine support exec `vector(0)` grama.
new PR of :grafana#7007
**Which issue(s) this PR fixes**:
Fixes grafana#6946

**Special notes for your reviewer**:
test passed.
test1: vectorExpr only
![image](https://user-images.githubusercontent.com/9583245/187871060-2ce63f01-5613-4a1e-9a52-59712f32cc7c.png)

test2: binExpr and vectorExpr
![image](https://user-images.githubusercontent.com/9583245/188090201-702c1ec4-7c37-41cb-9f2f-8f137a39dadf.png)

**Checklist**
- [ ] Documentation added
- [x] Tests updated
- [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
lxwzy pushed a commit to lxwzy/loki that referenced this issue Nov 7, 2022
@liguozhong @dannykopping @lxwzy
[doc] logql: logql engine support exec vector(0) grama (grafana#7044)
f5910bb 
<!--  Thanks for sending a pull request!  Before submitting:

1. Read our CONTRIBUTING.md guide
2. Name your PR as `<Feature Area>: Describe your change`.
  a. Do not end the title with punctuation. It will be added in the changelog.
  b. Start with an imperative verb. Example: Fix the latency between System A and System B.
  c. Use sentence case, not title case.
  d. Use a complete phrase or sentence. The PR title will appear in a changelog, so help other people understand what your change will be.
3. Rebase your PR if it gets out of sync with main
-->

**What this PR does / why we need it**:
logql engine support exec vector(0) grama.
new PR of :grafana#7023

**Which issue(s) this PR fixes**:
Fixes grafana#6946

**Special notes for your reviewer**:
preview
![image](https://user-images.githubusercontent.com/9583245/188355186-9a74d1ce-f062-45e8-8516-52c89383eeec.png)

<!--
Note about CHANGELOG entries, if a change adds:
* an important feature
* fixes an issue present in a previous release, 
* causes a change in operation that would be useful for an operator of Loki to know
then please add a CHANGELOG entry.

For documentation changes, build changes, simple fixes etc please skip this step. We are attempting to curate a changelog of the most relevant and important changes to be easier to ingest by end users of Loki.

Note about the upgrade guide, if this changes:
* default configuration values
* metric names or label names
* changes existing log lines such as the metrics.go query output line
* configuration parameters 
* anything to do with any API
* any other change that would require special attention or extra steps to upgrade
Please document clearly what changed AND what needs to be done in the upgrade guide.
-->
**Checklist**
- [ ] Documentation added
- [ ] Tests updated
- [ ] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`

Co-authored-by: Danny Kopping <[email protected]>
@mprokopov mprokopov mentioned this issue Jan 23, 2023
Closed
@ringerc
Copy link

ringerc commented Oct 28, 2024

For others who found this and who want to create a vector with labels from a scalar, the only option at the moment appears to be label_replace(vector(1), "newlabel", "newvalue", "", ""))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@ringerc @dannykopping @tonychoe