Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support mutating tags on write requests based on headers #2254

Closed
schallert opened this issue Apr 7, 2020 · 1 comment · Fixed by #2255
Closed

Support mutating tags on write requests based on headers #2254

schallert opened this issue Apr 7, 2020 · 1 comment · Fixed by #2255

Comments

@schallert
Copy link
Collaborator

schallert commented Apr 7, 2020

Background

We currently have support for restricting query results based on the header M3-Restrict-By-Tags-JSON. This enables
users who have pre-partitioned their metrics before writing them to partition their reads without modifying every
incoming query. The supported operators are EQUAL, NOTEQUAL, REGEXP, NOTREGEXP, EXISTS, and NOTEXISTS.

Similar behavior would be useful on the write path.

Proposal

The proposed fix for this is to add a new header, M3-Map-Tags-By-JSON.

It would have a similar structure to the query Restrict tags but support the following operators: APPEND, DROP, and
REPLACE. More operators could be added over time.

The format of a parsed map header would be:

type MapTagsOptions struct {
	TagMappers []TagMapper `json:"tagMappers"`
}

type TagMapper struct {
	Append  AppendOp  `json:"append"`
	Drop    DropOp    `json:"drop"`
	Replace ReplaceOp `json:"replace"`
}

type AppendOp struct {
	Tag   string `json:"tag"`
	Value string `json:"value"`
}

type DropOp struct {
	Tag   string `json:"tag"`
	Value string `json:"value"`
}

type ReplaceOp struct {
	Tag      string `json:"tag"`
	OldValue string `json:"old"`
	NewValue string `json:"new"`
}

APPEND

APPEND with value tag="foo" and value="bar" would unconditionally add tag-value pair "foo":"bar" to all
timeseries included in the write request. Any timeseries with a non-empty "foo" tag would have its value for that tag
replaced.

DROP

DROP with tag="foo" and an empty value would remove all tag-value pairs in all timeseries in the write request where
the tag was "foo". If value is non-empty, a tag-value pair would only be removed if the value was equal to value.

REPLACE

REPLACE with tag="foo", an empty old field, and a non-empty new field would unconditionally replace the value of
any tag-value pair of any timeseries in the write request where the tag is "foo" with the value of new. If old is
non-empty, a value would only be replaced if the value was equal to old.

Future Operators

The plan is to implement APPEND first and validate it works correctly.

More operators could be added over time, such as operators similar to existing ones but that perform matching or
replaces based on regexes.

Updates:

  • Mutate -> Map
  • MapTagOptions is an array of mappers each containing an op rather than a bunch of op arrays to make order of operations more clear.
@robskillington
Copy link
Collaborator

robskillington commented Apr 7, 2020

SGTM, my vote would be for M3-Map-Tags-By-JSON as a name for the header since we're mapping a set to new values, mutate works as well but seems long and not as commonly used for this type of dimension mapping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants