Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Watcher accounts constructed lazily #36656

Merged
merged 4 commits into from
Dec 17, 2018

Conversation

albertzaharovits
Copy link
Contributor

This fixes two bugs about watcher notifications:

  • registering accounts that had only secure settings was not possible before; these accounts are very much practical for Slack and PagerDuty integrations.

  • removes the limitation that, for an account with both secure and cluster settings, the admin had to first change/add the secure settings and only then the dependent dynamic cluster settings. The reverse order would trigger a SettingsException for an incomplete account. This had been partially discussed in: Watcher deprecate notification service settings #36403 (comment)

The workaround is to lazily instantiate account objects. Previously the approach was to greedily validate all the account settings by constructing the account objects even if they would not ever be used by actions. This made sense in a world where all the settings were set by a single API. But given that accounts have dependent settings (that must be used together) that have to be changed using different APIs (POST _nodes/reload_secure_settings and PUT _cluster/settings), the settings could be in an invalid state in between the calls.
This fix only builds account objects (and validates the settings) when they are needed by actions.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

// update both
if (randomBoolean()) {
// update secure first
secureSettingValue.set(randomAlphaOfLength(4));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

putting these three line calls in dedicated methods allows you to get rid of the "first/second" comments and its easy to figure out the order when reading

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did the replacement but parameters have side effects and the functions look super ugly.

@spinscale
Copy link
Contributor

does this PR also justify a backport into 6.5?

@albertzaharovits
Copy link
Contributor Author

does this PR also justify a backport into 6.5?

I think so yes...

@albertzaharovits
Copy link
Contributor Author

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request-2/2425/console

19:50:22 > Task :x-pack:plugin:watcher:test FAILED
19:50:22 Tests with failures:
19:50:22   - org.elasticsearch.xpack.watcher.notification.hipchat.HipChatServiceTests.testSingleAccountIntegration
19:50:22   - org.elasticsearch.xpack.watcher.notification.hipchat.HipChatServiceTests.testMultipleAccounts
19:50:22   - org.elasticsearch.xpack.watcher.notification.hipchat.HipChatServiceTests.testSingleAccountV1
19:50:22   - org.elasticsearch.xpack.watcher.notification.hipchat.HipChatServiceTests.testSingleAccountIntegrationNoRoomSetting
19:50:22   - org.elasticsearch.xpack.watcher.notification.hipchat.HipChatServiceTests.testSingleAccountUser

Was a legit failure!

The problem is that Accounts were not cached, every request for an account rebuilded it. I have swapped Supplier for LazyInitializable to remediate, and added a test. I think it's good now!

Copy link
Contributor

@spinscale spinscale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for picking this up!

@albertzaharovits albertzaharovits merged commit 6f03899 into elastic:master Dec 17, 2018
@albertzaharovits albertzaharovits deleted the lazy_reload branch December 17, 2018 12:46
albertzaharovits added a commit that referenced this pull request Dec 17, 2018
This fixes two bugs about watcher notifications:
* registering accounts that had only secure settings was not possible before;
these accounts are very much practical for Slack and PagerDuty integrations.
* removes the limitation that, for an account with both secure and cluster settings,
the admin had to first change/add the secure settings and only then add the
dependent dynamic cluster settings. The reverse order would trigger a
SettingsException for an incomplete account.

The workaround is to lazily instantiate account objects, hoping that when accounts
are instantiated all the required settings are in place. Previously, the approach
was to greedily validate all the account settings by constructing the account objects,
even if they would not ever be used by actions. This made sense in a world where
all the settings were set by a single API. But given that accounts have dependent
settings (that must be used together) that have to be changed using different APIs
(POST _nodes/reload_secure_settings and PUT _cluster/settings), the settings group
would technically be in an invalid state in between the calls.
This fix builds account objects, and validates the settings, when they are
needed by actions.
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Dec 18, 2018
* master: (30 commits)
  Revert "[Geo] Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach (elastic#35320)"
  Deprecate types in get_source and exist_source (elastic#36426)
  Fix duplicate phrase in shrink/split error message (elastic#36734)
  ingest: support default pipelines + bulk upserts (elastic#36618)
  TESTS:Debug Log. IndexStatsIT#testFilterCacheStats
  [Geo] Integrate Lucene's LatLonShape (BKD Backed GeoShapes) as default `geo_shape` indexing approach (elastic#35320)
  [TEST] fix float comparison in RandomObjects#getExpectedParsedValue
  Initialize startup `CcrRepositories` (elastic#36730)
  ingest: fix on_failure with Drop processor (elastic#36686)
  SNAPSHOTS: Adjust BwC Versions in Restore Logic (elastic#36718)
  [Painless] Add boxed type to boxed type casts for method/return (elastic#36571)
  Do not resolve addresses in remote connection info (elastic#36671)
  Add back one line removed by mistake regarding java version check and COMPAT jvm parameter existence
  Fixing line length for EnvironmentTests and RecoveryTests (elastic#36657)
  SQL: Fix translation of LIKE/RLIKE keywords (elastic#36672)
  [DOCS] Adds monitoring requirement for ingest node (elastic#36665)
  SNAPSHOTS: Disable BwC Tests Until elastic#36659 Landed (elastic#36709)
  Add doc's sequence number + primary term to GetResult and use it for updates (elastic#36680)
  [CCR] Add time since last auto follow fetch to auto follow stats (elastic#36542)
  Watcher accounts constructed lazily (elastic#36656)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants