Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sqlqueryreceiver] - add optional static_attributes to metrics. #11868

Merged
merged 9 commits into from
Jul 27, 2022

Conversation

Konig-Corey
Copy link
Contributor

@Konig-Corey Konig-Corey commented Jul 1, 2022

Description:

Adding a feature:

  • Add ability to set static tags for metrics returned from queries

Testing:

  • Added config test to validate config is loading a map of strings for tags.

Documentation:

  • Entry added to README.md to specify static tag usage.
    FYI @pmcollins

@Konig-Corey Konig-Corey requested a review from a team July 1, 2022 16:35
@Konig-Corey Konig-Corey requested a review from dmitryax as a code owner July 1, 2022 16:35
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jul 1, 2022

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: Konig-Corey (88dccbc2fd16bcc3166f8c31022513c97e638373, d92385ce68935a199fb37f07010724d412003681, e8bc25c8b9df6dd533938da6c0c7fcbfe4af8881)

@dmitryax
Copy link
Member

dmitryax commented Jul 1, 2022

Also it would be nice to have this PR broken into several ones since the fixes are unrelated to each other. And don't forget to add change log entries :)

@Konig-Corey
Copy link
Contributor Author

Konig-Corey commented Jul 1, 2022

I can close this and split it into two. I agree it makes more sense to have two PRs one for the addition of static tags, and another for the timestamp addition.

@Konig-Corey
Copy link
Contributor Author

@dmitryax removed the timestamp fix and will move to a different PR. Leaving this open for addition of static tag feature.

@dmitryax
Copy link
Member

dmitryax commented Jul 1, 2022

@Konig-Corey please update PR description

@dmitryax
Copy link
Member

dmitryax commented Jul 1, 2022

and a changelog entry is still required

CHANGELOG.md Outdated Show resolved Hide resolved
@dmitryax
Copy link
Member

dmitryax commented Jul 1, 2022

We just recently introduced another way to handle changelog. Please follow the updated process https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#adding-a-changelog-entry

@Konig-Corey
Copy link
Contributor Author

@dmitryax Added new change log.

@dmitryax
Copy link
Member

dmitryax commented Jul 5, 2022

@Konig-Corey please rebase

@Konig-Corey
Copy link
Contributor Author

@Konig-Corey please rebase

rebased from upstream main branch.

@Konig-Corey Konig-Corey changed the title Sqlqueryreceiver tags [sqlqueryreceiver] - add optional static_attributes to metrics. Jul 5, 2022
go.mod Outdated Show resolved Hide resolved
@dmitryax
Copy link
Member

dmitryax commented Jul 5, 2022

please take a look at CI failures

@@ -44,6 +45,8 @@ receivers:
- metric_name: movie.genres
value_column: "count"
attribute_columns: [ "genre" ]
static_attributes:
dbinstance: mydbinstance
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I want to set an attribute to true? How will we add this if we need it in a backwards compatible way?

Copy link
Contributor Author

@Konig-Corey Konig-Corey Jul 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im not sure what you are asking. It only takes string values. Any non string values would likely cause an issue or be converted to a string.

Copy link
Member

@dmitryax dmitryax Jul 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The @mx-psi's point is that if we want to add attributes with different value types, we won't be able to do that without breaking existing configuration interface. So it's better to design it better in advance. Maybe something like:

static_attributes: 
  - key: dbinstance
    string_value: mydbinstance

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we still applying this configuration interface ^ ? @mx-psi what do you think bout it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think changing StaticAttributes to map[string]interface{} and then doing:

for k, v := range cfg.StaticAttributes {
     switch v.(type) {
         case string:
	     attrs.InsertString(k, v)
          // ... other cases here, or even just a default that returns an error saying non-string is not supported
     }

Should work, and we don't have to complicate the configuration schema. Does that make sense?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our config provider definitely allows doing that: https://go.dev/play/p/CsyC3KuTer5

The yes/no case is an interesting edge case. The YAML package we use documents this explicitly here, in this case it will be a string since the field we unmarshal into is not a bool (see playground above). I think that's an acceptable behavior for me (but I understand if you feel otherwise).

Copy link
Member

@dmitryax dmitryax Jul 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this approach. @Konig-Corey can you please keep existing interface but make sure that only string values are accepted for now. Later we can add support for other types if needed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After giving it a second thought, I would say I'd rather prefer explicit type definition in yaml config rather than relying on implying type from the value. As a user, I would rather be explicit about what type of values I will get, instead of figuring out what's the parsing logic to make sure that attribute has a type that I want. I don't think users will be setting many static attributes, usually one or two per metric, having another line in the settings per attribute is not a big overhead IMO.

Specifying attributes in a list also makes the emitted metric with attributes in predictable order.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess not everyone has the same familiarity with YAML's quirkiness so that's a good argument.

Another reason I did not mention before to favor the map approach is that we sort-of have that on core here. I don't feel very strongly in favor of either implementation, but we should probably be consistent on both places. I don't want to drag this on forever, so I am fine with trying the explicit type here and then deciding which one we like the most.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, let's keep the same approach in this processor. I believe we can keep existing config behavior, given that the component is in alpha state. We can add restriction to string only attribute later.

receiver/sqlqueryreceiver/README.md Show resolved Hide resolved
@Konig-Corey Konig-Corey requested a review from a team July 6, 2022 13:39
@Konig-Corey Konig-Corey requested a review from jpkrohling as a code owner July 6, 2022 13:39
@mx-psi
Copy link
Member

mx-psi commented Jul 6, 2022

Looks like a rebase went wrong?

@Konig-Corey
Copy link
Contributor Author

Working to fix it. Sorry about that.

@Konig-Corey Konig-Corey force-pushed the sqlqueryreceiver-tags branch from 4138f90 to baa3bf1 Compare July 6, 2022 21:50
@Konig-Corey Konig-Corey force-pushed the sqlqueryreceiver-tags branch from baa3bf1 to 3e0e7e3 Compare July 6, 2022 21:52
Copy link
Member

@dmitryax dmitryax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Sorry for the delay

@dmitryax
Copy link
Member

@mx-psi PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants