Kibana api key #372

andrianjardan · 2023-06-30T09:54:22Z

Fixes #364

cla-checker-service · 2023-06-30T09:54:26Z

💚 CLA has been signed

andrianjardan · 2023-07-05T06:19:59Z

I guess disaster37/go-kibana-rest#5 has to be merged first for this to work.

andrianjardan · 2023-07-21T06:55:11Z

Seems like there's no activity in disaster37/go-kibana-rest#5, and the project looks abandoned

andrianjardan · 2023-08-09T09:10:17Z

https://github.com/disaster37/go-kibana-rest is abandoned since 2022, @tobio since this is a library crucial for the plugin functionality, does it make sense to fork it under Elastic orga ?

tobio · 2023-08-15T12:31:18Z

does it make sense to fork it under Elastic orga

Yes it does, there's been some discussions about this previously. I'm not clear where that stands right now. @Kushmaro FYI on having official client support for Kibana.

@andrianjardan IIUC Kibana API's should come with an OpenAPI spec in the future which reduces the need for a hand-written client library. I wonder if the interim solution here is to fork the client inside this repo rather than by itself.

andrianjardan · 2023-08-17T06:28:40Z

@tobio I could do so, should I ?

tobio · 2023-08-17T06:55:52Z

That would be amazing if you've got an opportunity to do so!

andrianjardan · 2023-08-18T07:33:06Z

@tobio aaand it's done ;)

tobio

Can you update the existing provider test to verify that Kibana API keys are working too.

tobio · 2023-08-20T22:56:07Z

internal/clients/api_client.go

@@ -508,6 +514,9 @@ func buildKibanaConfig(d *schema.ResourceData, baseConfig BaseConfig) (kibana.Co
 		if password := os.Getenv("KIBANA_PASSWORD"); password != "" {
 			config.Password = strings.TrimSpace(password)
 		}
+		if api_key := os.Getenv("KIBANA_API_KEY"); api_key != "" {


Can we make sure this is added to the provider docs as well.

tobio · 2023-08-20T23:05:14Z

internal/clients/api_client.go

API key support also needs to be added to connectors, as defaults for the fleet client and the SLO auth context

That's too many places :( There must be a better way.

I agree, it would be nice refactor some of this functionality. I'm hopeful we'll have a single client for the entire Kibana surface area soon so that we don't have to implement glue code around all the existing clients.

tobio · 2023-08-20T23:07:08Z

generated/alerting/client.go

@@ -414,6 +414,11 @@ func (c *APIClient) prepareRequest(
 			localVarRequest.SetBasicAuth(auth.UserName, auth.Password)
 		}

+		// APIKey Authentication


This has been manually added to the generated client? I think that's fine for now, these openapi clients are a bit of a mess right now. We need to tidy this up.

Can we add a comment calling out that this is inserted after generation.

jloleysens · 2023-09-07T11:04:28Z

Just wanted to ^bump this as adding support for Kibana API keys in the terraform provider would be very useful!

andrianjardan · 2023-09-07T11:15:24Z

@jloleysens I'll try to come back to it next week

electra06 · 2023-10-31T04:08:00Z

@andrianjardan - any update on this?

andrianjardan · 2023-10-31T11:49:12Z

@electra06 I am unfortunately busy with other tasks right now and the progress here is on hold, but I will do my best to continue working on it next week.

electra06 · 2023-11-02T02:57:44Z

@electra06 I am unfortunately busy with other tasks right now and the progress here is on hold, but I will do my best to continue working on it next week.

Thank you @andrianjardan - one of our developers has this requirement and it would be helpful once this has been pushed thru then.

electra06 · 2023-11-08T02:34:50Z

@andrianjardan - bumping on this once again, hope you got the time to work on this one this week :-)

electra06 · 2023-11-15T11:36:11Z

@andrianjardan - sorry to bump this again, our developer is following up as this one is very useful for them. Appreciate your time and effort.

andrianjardan · 2023-11-15T12:33:45Z

@electra06 absolutely no worries, I understand this is useful, and we also need it, but I got sick last week and the amount of work is not getting less...

Sorry for dragging it for too long, but it was intended as a small and quick fix, and turned into a longer journey, as my experience with Golang is quite limited.

I am happy to hand it over to someone else who has more time and expertise, if needed.

jloleysens · 2023-11-20T12:45:06Z

@andrianjardan , I've attempted to address outstanding items, hopefully we can get this over the line!

jloleysens

self-review

jloleysens · 2023-11-20T12:46:48Z

generated/alerting/api_alerting.go

@@ -583,7 +583,7 @@ func (a *AlertingApiService) CreateRuleExecute(r ApiCreateRuleRequest) (*RuleRes
 				} else {
 					key = apiKey.Key
 				}
-				localVarHeaderParams["ApiKey"] = key
+				localVarHeaderParams["Authorization"] = key


This generated Kibana client code is buggy :(

We will have to revisit this and ensure we can generate functioning HTTP requests from the OpenAPI specs. I think this task is beyond this PR so I've modified the generated code directly. We should open an issue for fixing this!

generated/alerting/api/openapi.yaml

andrianjardan · 2023-11-20T14:02:37Z

@jloleysens thanks a lot for jumping on this, I am sure it is going to be appreciated by everyone waiting :)!

.github/workflows/test.yml

tobio

Thanks for moving this forward, couple of comments on the code.

generated/alerting/api/openapi.yaml

internal/kibana/alerting_test.go

internal/schema/connection.go

internal/clients/config/kibana.go

internal/schema/connection.go

jloleysens · 2023-11-22T15:49:50Z

@tobio I've updated the docs in 0885614, would you mind doing another review?

tobio

LGTM. I see a weird issue in running the acceptance tests locally with an API key here, but it looks unrelated, I'll open an issue for it. The tests pass if run individually which is the main thing, but one of the tests (probably the API key test but I haven't looked at all) is cleaning up existing keys so when the entire suite is run, the key based tests fail since the key is removed in an earlier test.

tobio · 2023-11-23T04:05:53Z

FYI #490

electra06 · 2023-11-24T04:12:44Z

Hi All,
Thank you for moving this forward :-), looks like there is still an issue?

tobio · 2023-11-24T04:13:50Z

looks like there is still an issue?

What's the issue?

electra06 · 2023-11-24T05:43:09Z

What's the issue?
@tobio just confirming :-), can we share this now to the customer?

tobio · 2023-11-24T05:59:46Z

We'll need a new provider release including this PR first. I don't have a timeframe to share right now, we're discussing that internally at the moment.

* Adding api key functionality for Kibana * Adding support for api_key for Kibana * Adding go-kibana-rest lib locally as it is abandoned by authors * update docs * added support for SLO client to use API keys * add auth interceptor to connectors client * added test case for Kibana API keys via provider * slight fix to table test * added fwschema * fix copy-pasta * some typos, generate docs * manually patch generated code for now * update specs * update new kb config from framework * added api key to provider tests * use built in jq * remove kibana api key env * API keys should cascade from ES->Kibana->Fleet * update rules doc --------- Co-authored-by: Jean-Louis Leysens <[email protected]> Co-authored-by: Toby Brain <[email protected]>

andrianjardan added 2 commits June 30, 2023 11:59

Adding api key functionality for Kibana

1724ff5

Adding support for api_key for Kibana

0f738b7

andrianjardan force-pushed the kibana_api_key branch from 7ea511e to 0f738b7 Compare June 30, 2023 09:59

Adding go-kibana-rest lib locally as it is abandoned by authors

21c74a8

tobio reviewed Aug 20, 2023

View reviewed changes

tobio mentioned this pull request Oct 1, 2023

Add plugin framework #343

Merged

andrianjardan closed this Nov 9, 2023

andrianjardan reopened this Nov 9, 2023

Merge branch 'main' into kibana_api_key

bd9d226

jloleysens added 5 commits November 20, 2023 10:39

update docs

c028819

added support for SLO client to use API keys

08611b0

add auth interceptor to connectors client

c57830b

added test case for Kibana API keys via provider

e1b5af8

slight fix to table test

446edfb

jloleysens added 7 commits November 20, 2023 11:41

Merge branch 'main' into kibana_api_key

933da1b

added fwschema

dda24a6

fix copy-pasta

39587a8

some typos, generate docs

28a7fd9

manually patch generated code for now

3959bac

update specs

a95fd7c

update new kb config from framework

ce4c399

jloleysens reviewed Nov 20, 2023

View reviewed changes

jloleysens added 2 commits November 20, 2023 14:34

added api key to provider tests

068acdd

use built in jq

c5123fa

remove kibana api key env

1f0608b

jloleysens reviewed Nov 20, 2023

View reviewed changes

.github/workflows/test.yml Show resolved Hide resolved

tobio reviewed Nov 20, 2023

View reviewed changes

tobio and others added 3 commits November 21, 2023 09:23

API keys should cascade from ES->Kibana->Fleet

bddddbf

Merge branch 'main' into kibana_api_key

e8a3391

update rules doc

0885614

tobio mentioned this pull request Nov 23, 2023

[Bug] Kibana action connector can not be created while reusing the api_key from elasticsearch due to Unauthorized error. #489

Closed

tobio approved these changes Nov 23, 2023

View reviewed changes

jloleysens merged commit 6fcbbaa into elastic:main Nov 23, 2023
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kibana api key #372

Kibana api key #372

andrianjardan commented Jun 30, 2023

cla-checker-service bot commented Jun 30, 2023 •

edited

Loading

andrianjardan commented Jul 5, 2023

andrianjardan commented Jul 21, 2023

andrianjardan commented Aug 9, 2023 •

edited

Loading

tobio commented Aug 15, 2023

andrianjardan commented Aug 17, 2023

tobio commented Aug 17, 2023

andrianjardan commented Aug 18, 2023

tobio left a comment

tobio Aug 20, 2023

tobio Aug 20, 2023

jloleysens Nov 20, 2023 •

edited

Loading

tobio Aug 20, 2023

jloleysens commented Sep 7, 2023

andrianjardan commented Sep 7, 2023

electra06 commented Oct 31, 2023

andrianjardan commented Oct 31, 2023

electra06 commented Nov 2, 2023

electra06 commented Nov 8, 2023

electra06 commented Nov 15, 2023

andrianjardan commented Nov 15, 2023

jloleysens commented Nov 20, 2023

jloleysens left a comment

jloleysens Nov 20, 2023

andrianjardan commented Nov 20, 2023

tobio left a comment

jloleysens commented Nov 22, 2023 •

edited

Loading

tobio left a comment

tobio commented Nov 23, 2023

electra06 commented Nov 24, 2023

tobio commented Nov 24, 2023

electra06 commented Nov 24, 2023

tobio commented Nov 24, 2023

Kibana api key #372

Kibana api key #372

Conversation

andrianjardan commented Jun 30, 2023

cla-checker-service bot commented Jun 30, 2023 • edited Loading

andrianjardan commented Jul 5, 2023

andrianjardan commented Jul 21, 2023

andrianjardan commented Aug 9, 2023 • edited Loading

tobio commented Aug 15, 2023

andrianjardan commented Aug 17, 2023

tobio commented Aug 17, 2023

andrianjardan commented Aug 18, 2023

tobio left a comment

Choose a reason for hiding this comment

tobio Aug 20, 2023

Choose a reason for hiding this comment

tobio Aug 20, 2023

Choose a reason for hiding this comment

jloleysens Nov 20, 2023 • edited Loading

Choose a reason for hiding this comment

tobio Aug 20, 2023

Choose a reason for hiding this comment

jloleysens commented Sep 7, 2023

andrianjardan commented Sep 7, 2023

electra06 commented Oct 31, 2023

andrianjardan commented Oct 31, 2023

electra06 commented Nov 2, 2023

electra06 commented Nov 8, 2023

electra06 commented Nov 15, 2023

andrianjardan commented Nov 15, 2023

jloleysens commented Nov 20, 2023

jloleysens left a comment

Choose a reason for hiding this comment

jloleysens Nov 20, 2023

Choose a reason for hiding this comment

andrianjardan commented Nov 20, 2023

tobio left a comment

Choose a reason for hiding this comment

jloleysens commented Nov 22, 2023 • edited Loading

tobio left a comment

Choose a reason for hiding this comment

tobio commented Nov 23, 2023

electra06 commented Nov 24, 2023

tobio commented Nov 24, 2023

electra06 commented Nov 24, 2023

tobio commented Nov 24, 2023

cla-checker-service bot commented Jun 30, 2023 •

edited

Loading

andrianjardan commented Aug 9, 2023 •

edited

Loading

jloleysens Nov 20, 2023 •

edited

Loading

jloleysens commented Nov 22, 2023 •

edited

Loading