Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flagext package #16

Merged
merged 25 commits into from
Aug 19, 2021
Merged

Add flagext package #16

merged 25 commits into from
Aug 19, 2021

Conversation

aknuds1
Copy link
Collaborator

@aknuds1 aknuds1 commented Aug 19, 2021

Add flagext package from Cortex. Will be needed by the kv package.

tomwilkie and others added 23 commits August 19, 2021 09:01
This allows us to start Cortex on the Mac.

Signed-off-by: Tom Wilkie <[email protected]>
Signed-off-by: Tom Wilkie <[email protected]>
Many ingester flags were recently broken. This fixes the flags that were
broken, but if anyone is running the new HA Ruler they will need to
update their consul flags.

Signed-off-by: Chris Marchbanks <[email protected]>
This reverts commit 928ae26d81d18b430786dbebd7e5ee8b7f5059ac.

Signed-off-by: Chris Marchbanks <[email protected]>
* Calling flag.Parse() twice confuses some flags.

flag.Parse() was called twice previously, first to get -config.file
value to parse the config file, and second to parse remaining command
line parameters – overwriting values from config file.

Unfortunately that confuses command line parameters that accept multiple
values. Here we get -config.file via a separate FlagSet to avoid calling
flag.Parse() (on default FlagSet) twice.

Signed-off-by: Peter Štibraný <[email protected]>

* Reverted unintended import order change.

Signed-off-by: Peter Štibraný <[email protected]>

* Call RegisterFlags(&cfg) to set defaults before parsing config file.

Signed-off-by: Peter Štibraný <[email protected]>

* Added some unit tests for parsing config.

Signed-off-by: Peter Štibraný <[email protected]>

* Test mode in main simply dumps the config YAML if no error occurs

Added tests for happy path.

Signed-off-by: Peter Štibraný <[email protected]>

* Separate stdout/stderr expected messages and checks.

Signed-off-by: Peter Štibraný <[email protected]>

* Use mutexProfileFraction only after flags are parsed.

Signed-off-by: Peter Štibraný <[email protected]>

* Renamed default test name.

Signed-off-by: Peter Štibraný <[email protected]>

* Reset testMode to original value at the end of test.

Signed-off-by: Peter Štibraný <[email protected]>
Signed-off-by: yeya24 <[email protected]>
…(#2106)

* Verify that files have correct order of imports.

goimports -local github.com/cortexproject/cortex does the trick.

Signed-off-by: Peter Štibraný <[email protected]>

* goimports -local github.com/cortexproject/cortex -w pkg/ cmd/ tools/

Signed-off-by: Peter Štibraný <[email protected]>

* Rerun make protos.

Signed-off-by: Peter Štibraný <[email protected]>

* Fixed import
Signed-off-by: Peter Štibraný <[email protected]>

* Rerun make protos for ruler
Signed-off-by: Peter Štibraný <[email protected]>
…(#2165)

* Add /config handler which exposes the current YAML config of cortex.

Signed-off-by: Tom Wilkie <[email protected]>
Signed-off-by: Tom Wilkie <[email protected]>

* Add changelog and line to docs.

Signed-off-by: Tom Wilkie <[email protected]>
Signed-off-by: Tom Wilkie <[email protected]>

* Fix lint.

(Also regen docs)

Signed-off-by: Tom Wilkie <[email protected]>
Signed-off-by: Tom Wilkie <[email protected]>

* Fix marshalling / unmarshalling of flagext values and add test.

Signed-off-by: Tom Wilkie <[email protected]>
Signed-off-by: Tom Wilkie <[email protected]>

* Add PR number to changelog.

Signed-off-by: Tom Wilkie <[email protected]>
Signed-off-by: Tom Wilkie <[email protected]>

* Fixed linter

Signed-off-by: Marco Pracucci <[email protected]>

* Fixed URLValue zero value marshalling

Signed-off-by: Marco Pracucci <[email protected]>

* Removed omitempty from LifecyclerConfig

Signed-off-by: Marco Pracucci <[email protected]>

* Added integration test on GET /config endpoint

Signed-off-by: Marco Pracucci <[email protected]>

Co-authored-by: Marco Pracucci <[email protected]>
* Cleanup linting errors around deadcode

To be specific ineffassign, deadcode and unused errors

Signed-off-by: Nathan Zender <[email protected]>

* Replaced by keysFn and if the config defines it then it will distribute
keys using hashing

Signed-off-by: Nathan Zender <[email protected]>

* Hold over from when readBatch as also the iterator

Now that we have an iterator there is no need to also have a consumed
bool on the underlying object.

Signed-off-by: Nathan Zender <[email protected]>

* Necessary to fix the false sharing problem

Will never actually be used. Only necessary to pad out CPU cache lines.

Signed-off-by: Nathan Zender <[email protected]>

* Removing unused code

Signed-off-by: Nathan Zender <[email protected]>

* Cleanup all gosimple suggestions

Signed-off-by: Nathan Zender <[email protected]>

* Fixing all errcheck

Attempted not to change any existing logic so if an error was ignored we
will now either explicitly ignore it or in the case of a goroutine being
called with a func that is ignoring the error we will just put a
//noling:errcheck on that line.

If it was in a test case we went ahead and did the extra assertion
checks b/c its always good to know where something might have errored in
your test cases.

Signed-off-by: Nathan Zender <[email protected]>

* Fix most staticcheck lint issues

Signed-off-by: Nathan Zender <[email protected]>

* Cleanup deprecated call to snappy.NewWriter

Signed-off-by: Nathan Zender <[email protected]>

* Remove rev from Makefile

Signed-off-by: Nathan Zender <[email protected]>

* Reorder imports

Signed-off-by: Nathan Zender <[email protected]>

* Ignoring this for now

We have opened up issue
cortexproject/cortex#2015 to address the
deprecation of this type.

Signed-off-by: Nathan Zender <[email protected]>

* Explicitly ignoring this error

The test is currently failing due to a data race. I believe it is due to
this bug in golang.

golang/go#30597

As far as this test cares it does not really matter that this happens so
removing the need to check for NoError "fixes" it.

Signed-off-by: Nathan Zender <[email protected]>

* Require noerror since this is a test

Signed-off-by: Nathan Zender <[email protected]>

* Switch over to use require.NoError

Signed-off-by: Nathan Zender <[email protected]>

* Move func to test class since that is only place it was used

Signed-off-by: Nathan Zender <[email protected]>

* Log warning if save to cache errors

Signed-off-by: Nathan Zender <[email protected]>

* Condense a little

Signed-off-by: Nathan Zender <[email protected]>

* Use returned error instead of capturing it

Signed-off-by: Nathan Zender <[email protected]>

* Bringing back ctx and adding comment

Signed-off-by: Nathan Zender <[email protected]>

* Log error if changing ring state fails when Leaving

Signed-off-by: Nathan Zender <[email protected]>

* If context deadline exceeded return the error

Signed-off-by: Nathan Zender <[email protected]>

* Can't defer this otherwise we will have no data

Signed-off-by: Nathan Zender <[email protected]>

* Comment to make it clear why this nolint was added

Signed-off-by: Nathan Zender <[email protected]>

* Refactor method out

Since Fixture is already in testutils and it is being used in both
places pulled it out into a common helper method in the testutils
package.

Signed-off-by: Nathan Zender <[email protected]>

* io.Copy added to global errcheck exclude

Signed-off-by: Nathan Zender <[email protected]>

* If error dont do anything else

Signed-off-by: Nathan Zender <[email protected]>

* Cleanup unused function

Signed-off-by: Nathan Zender <[email protected]>

* Adding tracer to global excludes

Signed-off-by: Nathan Zender <[email protected]>

* Cleanup post rebase

Formatting and import issues that got missed when merging.

Signed-off-by: Nathan Zender <[email protected]>

* Ratelimiter returns resource exhausted error

This is necessary so that when it is used with the backoff retry it will
allow for the backoff to continue to work as expected.

Signed-off-by: Nathan Zender <[email protected]>
* Hide passwords in URLs from flags.
* Add secret flag handler that hides the value of the secret when marshalled to YAML.
* Use flagext.Secret for Azure, GCP, Redis, Cassandra secrets and block storage config.
* Teach doc generator how to handle flagext.Secret.

Signed-off-by: Tom Wilkie <[email protected]>
Also, I couldn't find a better way to do this.

Fixes: cortexproject/cortex#1425

Signed-off-by: Goutham Veeramachaneni <[email protected]>
* Added support to querier to work with multiple stores.
Store queryables now have filtering function.

Signed-off-by: Peter Štibraný <[email protected]>

* Querier can now use second store for querying.

Signed-off-by: Peter Štibraný <[email protected]>

* Fixes for using chunks and blocks at the same time.

Signed-off-by: Peter Štibraný <[email protected]>

* Added tests for new filtering queryables.

Signed-off-by: Peter Štibraný <[email protected]>

* Added test for distributor querier filter.

Signed-off-by: Peter Štibraný <[email protected]>

* Added test for using multiple store queryables.

Signed-off-by: Peter Štibraný <[email protected]>

* Ignore linter trying to get rid of else.

This form allows q to have smaller scope,
which reduces possibility of incorrect reuse.

Signed-off-by: Peter Štibraný <[email protected]>

* Allow "0" value.

Signed-off-by: Peter Štibraný <[email protected]>

* Mention available formats.

Signed-off-by: Peter Štibraný <[email protected]>

* Support for flagext.Time.

Signed-off-by: Peter Štibraný <[email protected]>

* Extend help for querier.use-second-store-before-time.

Signed-off-by: Peter Štibraný <[email protected]>

* Added CHANGELOG entry, and check to avoid using same primary and secondary engines.

Signed-off-by: Peter Štibraný <[email protected]>

* Put comment why formatMatcher is used.

Signed-off-by: Peter Štibraný <[email protected]>

* Mention that querying feature is experimental.

Signed-off-by: Peter Štibraný <[email protected]>

* Enhance CHANGELOG.md

Signed-off-by: Peter Štibraný <[email protected]>

* Explain default value.

Signed-off-by: Peter Štibraný <[email protected]>

* Added <time> placeholder, with example better examples.

Signed-off-by: Peter Štibraný <[email protected]>

* Removed `buildService` function.

Signed-off-by: Peter Štibraný <[email protected]>

* Mention that last two formats are specified by RFC 3339.

Signed-off-by: Peter Štibraný <[email protected]>

* Fix docs.

Signed-off-by: Peter Štibraný <[email protected]>
* fix: `-ruler.alertmanager-url` should be a string

This was never space separated, you needed to register the flag multiple times for it to append to the list of strings.

On top of that, the previous change implied that when using a config file you would need to provide a a list instead of a string thus breaking exisiting configuration.

Signed-off-by: gotjosh <[email protected]>

* Unify the use of StringSlice flags

We had two very similar flag extensions: `Strings` and `StringSlice`.

With this, we unify its use by removing `Strings` in favour of `StringSlice` but keeping the string representation favoured in `Strings`.

The difference here is that by using `Sprintf` we would get the brackets representation as part of the string.

Signed-off-by: gotjosh <[email protected]>

* Rename `api_ruler_test` to `ruler_test`

This is no longer only about the API.

Signed-off-by: gotjosh <[email protected]>

* Create an integration test for Alertmanager discovery

Signed-off-by: gotjosh <[email protected]>

* Fix existing test

Signed-off-by: gotjosh <[email protected]>
* Add an ability to load an arbitrary list of Cortex modules

Signed-off-by: Igor Novgorodov <[email protected]>

* Fix module loading logic

Signed-off-by: Igor Novgorodov <[email protected]>

* Improve multiple module loading

Signed-off-by: Igor Novgorodov <[email protected]>

* Add forgotten files

Signed-off-by: Igor Novgorodov <[email protected]>

* Fix linter errors

Signed-off-by: Igor Novgorodov <[email protected]>

* Reduce state in ModuleManager

Signed-off-by: Igor Novgorodov <[email protected]>

* Fix pkg/cortex & pkg/util/modules tests, make initModules() stateless, update docs

Signed-off-by: Igor Novgorodov <[email protected]>

* Fix missed mess with &c.Target pointer cast

Signed-off-by: Igor Novgorodov <[email protected]>

* Cosmetic changes to minimize diff

Signed-off-by: Igor Novgorodov <[email protected]>

* Fix problems with PR #3287

Signed-off-by: Igor Novgorodov <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Marco Pracucci <[email protected]>

* Update pkg/cortex/cortex_test.go

Signed-off-by: Marco Pracucci <[email protected]>

* Fixed linter

Signed-off-by: Marco Pracucci <[email protected]>

Co-authored-by: Marco Pracucci <[email protected]>
… (#3734)

* separate out some util packages

Signed-off-by: Kevin Minehart <[email protected]>

* use goimports with -local

Signed-off-by: Kevin Minehart <[email protected]>

* refactor: reduce the number of transitive imports when using math and logging functions

Signed-off-by: Kevin Minehart <[email protected]>

* add deprecation warning to pkg/log/log.go

Signed-off-by: Kevin Minehart <[email protected]>
… (#4085)

* Introduced firewall in the Alertmanager to block specific addresses in receiver integrations

Signed-off-by: Marco Pracucci <[email protected]>

* Adapted implementation based on new design in prometheus/common

Signed-off-by: Marco Pracucci <[email protected]>

* Updated doc

Signed-off-by: Marco Pracucci <[email protected]>

* Fixed doc

Signed-off-by: Marco Pracucci <[email protected]>

* Improved doc

Signed-off-by: Marco Pracucci <[email protected]>

* Improved config description

Signed-off-by: Marco Pracucci <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
@aknuds1 aknuds1 requested review from treid314 and a team August 19, 2021 07:06
@aknuds1 aknuds1 added the enhancement New feature or request label Aug 19, 2021
@aknuds1 aknuds1 mentioned this pull request Aug 19, 2021
1 task
Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

flagext/deprecated.go Outdated Show resolved Hide resolved
Signed-off-by: Arve Knudsen <[email protected]>
@aknuds1 aknuds1 merged commit 94b98c2 into main Aug 19, 2021
@aknuds1 aknuds1 deleted the feat/flagext branch August 19, 2021 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.