Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying to add APM integration returns 404 #146312

Closed
simitt opened this issue Nov 24, 2022 · 14 comments · Fixed by #147257
Closed

Trying to add APM integration returns 404 #146312

simitt opened this issue Nov 24, 2022 · 14 comments · Fixed by #147257
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Team:Fleet Team label for Observability Data Collection Fleet team v8.6.0

Comments

@simitt
Copy link
Contributor

simitt commented Nov 24, 2022

Kibana version: 8.6.0-SNAPSHOT

Elasticsearch version: 8.6.0-SNAPSHOT

Original install method (e.g. download page, yum, from source, etc.): Docker

Describe the bug:
On a fresh cluster, when trying to add an APM Integration a 404 is reported.
Installing directly via Settings works, while it indicates that 0 assets would be installed.

Steps to reproduce:
@maryam-saeidi shared a screencast describing the problem:
https://user-images.githubusercontent.com/5555349/203823955-c9039f81-6e1c-46a4-8f83-4ce064d86130.mov

Workaround via installing from Settings shows:

Screenshot 2022-11-24 at 16 43 15

Expected behavior:
The APM integration is already packaged with Kibana, it should be loaded from there.

Additional notes:
Could this be related to the version rewrite that happens on pre-release builds when packaging the APM integration with Kibana?

@simitt simitt added bug Fixes for quality problems that affect the customer experience Team:Fleet Team label for Observability Data Collection Fleet team v8.6.0 labels Nov 24, 2022
@simitt simitt added this to the 8.6. milestone Nov 24, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@kpollich
Copy link
Member

There is no apm-8.6.0 in EPR: https://epr.elastic.co/package/apm/8.6.0/

The bundled version of APM in this environment would be available at apm-8.6.0-SNAPSHOT. Are we hardcoding a link to apm-8.6.0 from the "APM Integration in Fleet" button in the APM UI?

@simitt
Copy link
Contributor Author

simitt commented Nov 29, 2022

@sqren could you help answer this? Now that the apmpackage is only bundled in Kibana, and not pushed to epr any longer. What would the correct link be?

@jen-huang
Copy link
Contributor

The APM button in the tutorial is non-versioned, it links to /app/integrations#/detail/apm/overview so it should be fine.

I just tried this on 8.6.0 BC3 on Cloud and everything worked fine. I'm not sure what the difference via Docker would be.

@sorenlouv
Copy link
Member

sorenlouv commented Nov 30, 2022

As @jen-huang says, there's no hardcoded version in the link from APM (possibly elsewhere):

const apmIntegrationHref = shouldLinkToMigration
? `${basePath}/app/apm/settings/schema`
: `${basePath}/app/integrations#/detail/apm/overview`;

@sorenlouv
Copy link
Member

@maryam-saeidi shared a screencast describing the problem:

It's a little hard to debug as I can't see the url bar, and can't determine where this goes wrong, and whether it's something APM is doing wrong.

@jlind23
Copy link
Contributor

jlind23 commented Dec 7, 2022

@simitt @sqren is this issue still relevant and if yes how can we make it move forward?

@sorenlouv
Copy link
Member

sorenlouv commented Dec 7, 2022

I'd need to know the url of the page where the integration is installed, and the url of the page that shows a 404 to determine if this is an issue on the APM side.

@jlind23
Copy link
Contributor

jlind23 commented Dec 7, 2022

@maryam-saeidi would you be able to share the URL you got please?

@maryam-saeidi
Copy link
Member

maryam-saeidi commented Dec 7, 2022

@jlind23 @sqren I faced the issue during testing BC2(8.6.0) according to these steps, so I was testing it locally.

Is this what you are looking for or do you mean the exact path? I can also try again with the latest BC to see if the issue still exists. I will share the exact URL after testing the lastest BC.

@maryam-saeidi
Copy link
Member

@sqren This is the URL: http://localhost:5601/app/fleet/integrations/apm/add-integration (BC5 has the same error)

@sorenlouv
Copy link
Member

sorenlouv commented Dec 7, 2022

Hmm.. I can't repro on 8.7

fleet-apm

I'm not sure this is an APM issue. Could it be a fleet issue?

@jlind23
Copy link
Contributor

jlind23 commented Dec 7, 2022

@jen-huang could you please take a look again? @sqren 8.6 and 8.7 are not different so I would be surprised if this is coming from fleet.

@juliaElastic
Copy link
Contributor

juliaElastic commented Dec 8, 2022

I could reproduce with 8.6 BC5 by following the steps as Mary mentioned here.

I noticed that the API call http://localhost:5601/api/fleet/epm/packages/apm/8.6.0 returns a response, but with full=true parameter, it doesn't:

image

image

I think this bug might be caused by #144343 which introduced the full=true flag, which tries to fetch the package archive from EPR.
The issue doesn't happen on cloud because APM is already installed so the details are coming from the installed package.

image

There is a workaround to install the package with API (that installs from bundled packages) and then open the add-integration page.

POST http://elastic:changeme@localhost:5601/api/fleet/epm/packages/apm/8.6.0
kbn-xsrf: kibana

I also observed that locally the apm version returned is 8.6.0-preview or 8.7.0-preview, depending on the branch. I think this might be a specific behaviour in a SNAPSHOT version. These preview versions are not returned in a BC build downloaded locally.
This functionality is caused by loading the package caused by using prerelease=true flag - in case of APM we don't have a concrete version in the url. We should load the prerelease setting instead. https://github.com/elastic/kibana/pull/144431/files#diff-65c6709304b479e2fbd8afe2018c657ffd50f900949c19c8382a37937a3c816a

image

cc @hop-dev

@juliaElastic juliaElastic self-assigned this Dec 8, 2022
juliaElastic added a commit that referenced this issue Dec 12, 2022
## Summary

Fixes #146312

Fall back to bundled packages when loading package archive.

Also fixed an issue with prerelease flag caused by
#144431

Loading prerelease setting on Add/Edit integration page, so that it
works well for cases where the url doesn't contain a concrete version
like APM

Steps to verify:
- Start kibana locally, make sure that APM bundled package is there in
`x-pack/plugins/fleet/target/bundled_packages`
- Load http://localhost:5601/julia/app/integrations/detail/apm/overview
(with default "beta integrations" off flag)
- Expect that 8.6.0 version is showing up (or 8.7.0 depending on the
build)
- When clicking on Add Elastic APM, expect 8.6.0 to be loaded
successfully

<img width="1773" alt="image"
src="https://user-images.githubusercontent.com/90178898/206482128-634ca823-bb21-46a9-a8b9-fc81d5dc126d.png">

Note: there is still a UI issue with loading assets on Overview page if
the package is not in EPR.

<img width="1747" alt="image"
src="https://user-images.githubusercontent.com/90178898/206478275-ee8e9494-234f-44c8-88d9-a91c791b0196.png">

- switch "beta integrations" on on Integrations page
- Open APM overview page again
- expect `8.7.0-preview` version with the selector to change to last GA

<img width="1100" alt="image"
src="https://user-images.githubusercontent.com/90178898/206480986-2ed05b12-49e4-4c88-97c6-6cdf12f819da.png">


### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Dec 12, 2022
## Summary

Fixes elastic#146312

Fall back to bundled packages when loading package archive.

Also fixed an issue with prerelease flag caused by
elastic#144431

Loading prerelease setting on Add/Edit integration page, so that it
works well for cases where the url doesn't contain a concrete version
like APM

Steps to verify:
- Start kibana locally, make sure that APM bundled package is there in
`x-pack/plugins/fleet/target/bundled_packages`
- Load http://localhost:5601/julia/app/integrations/detail/apm/overview
(with default "beta integrations" off flag)
- Expect that 8.6.0 version is showing up (or 8.7.0 depending on the
build)
- When clicking on Add Elastic APM, expect 8.6.0 to be loaded
successfully

<img width="1773" alt="image"
src="https://user-images.githubusercontent.com/90178898/206482128-634ca823-bb21-46a9-a8b9-fc81d5dc126d.png">

Note: there is still a UI issue with loading assets on Overview page if
the package is not in EPR.

<img width="1747" alt="image"
src="https://user-images.githubusercontent.com/90178898/206478275-ee8e9494-234f-44c8-88d9-a91c791b0196.png">

- switch "beta integrations" on on Integrations page
- Open APM overview page again
- expect `8.7.0-preview` version with the selector to change to last GA

<img width="1100" alt="image"
src="https://user-images.githubusercontent.com/90178898/206480986-2ed05b12-49e4-4c88-97c6-6cdf12f819da.png">

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

(cherry picked from commit 24e2fcf)
kibanamachine referenced this issue Dec 12, 2022
#147335)

# Backport

This will backport the following commits from `main` to `8.6`:
- [[Fleet] fix for apm not found in epr + prerelease flag
(#147257)](#147257)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Julia
Bardi","email":"[email protected]"},"sourceCommit":{"committedDate":"2022-12-12T10:06:50Z","message":"[Fleet]
fix for apm not found in epr + prerelease flag (#147257)\n\n##
Summary\r\n\r\nFixes
https://github.com/elastic/kibana/issues/146312\r\n\r\nFall back to
bundled packages when loading package archive.\r\n\r\nAlso fixed an
issue with prerelease flag caused
by\r\nhttps://github.com//pull/144431\r\n\r\nLoading
prerelease setting on Add/Edit integration page, so that it\r\nworks
well for cases where the url doesn't contain a concrete version\r\nlike
APM\r\n\r\nSteps to verify:\r\n- Start kibana locally, make sure that
APM bundled package is there
in\r\n`x-pack/plugins/fleet/target/bundled_packages`\r\n- Load
http://localhost:5601/julia/app/integrations/detail/apm/overview\r\n(with
default \"beta integrations\" off flag)\r\n- Expect that 8.6.0 version
is showing up (or 8.7.0 depending on the\r\nbuild)\r\n- When clicking on
Add Elastic APM, expect 8.6.0 to be loaded\r\nsuccessfully\r\n\r\n<img
width=\"1773\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206482128-634ca823-bb21-46a9-a8b9-fc81d5dc126d.png\">\r\n\r\nNote:
there is still a UI issue with loading assets on Overview page if\r\nthe
package is not in EPR.\r\n\r\n<img width=\"1747\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206478275-ee8e9494-234f-44c8-88d9-a91c791b0196.png\">\r\n\r\n-
switch \"beta integrations\" on on Integrations page\r\n- Open APM
overview page again\r\n- expect `8.7.0-preview` version with the
selector to change to last GA\r\n\r\n<img width=\"1100\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206480986-2ed05b12-49e4-4c88-97c6-6cdf12f819da.png\">\r\n\r\n\r\n###
Checklist\r\n\r\n- [ ] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"24e2fcf47014d937a5019d83b95565a010ae4e22","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v8.6.0","v8.7.0"],"number":147257,"url":"https://github.com/elastic/kibana/pull/147257","mergeCommit":{"message":"[Fleet]
fix for apm not found in epr + prerelease flag (#147257)\n\n##
Summary\r\n\r\nFixes
https://github.com/elastic/kibana/issues/146312\r\n\r\nFall back to
bundled packages when loading package archive.\r\n\r\nAlso fixed an
issue with prerelease flag caused
by\r\nhttps://github.com//pull/144431\r\n\r\nLoading
prerelease setting on Add/Edit integration page, so that it\r\nworks
well for cases where the url doesn't contain a concrete version\r\nlike
APM\r\n\r\nSteps to verify:\r\n- Start kibana locally, make sure that
APM bundled package is there
in\r\n`x-pack/plugins/fleet/target/bundled_packages`\r\n- Load
http://localhost:5601/julia/app/integrations/detail/apm/overview\r\n(with
default \"beta integrations\" off flag)\r\n- Expect that 8.6.0 version
is showing up (or 8.7.0 depending on the\r\nbuild)\r\n- When clicking on
Add Elastic APM, expect 8.6.0 to be loaded\r\nsuccessfully\r\n\r\n<img
width=\"1773\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206482128-634ca823-bb21-46a9-a8b9-fc81d5dc126d.png\">\r\n\r\nNote:
there is still a UI issue with loading assets on Overview page if\r\nthe
package is not in EPR.\r\n\r\n<img width=\"1747\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206478275-ee8e9494-234f-44c8-88d9-a91c791b0196.png\">\r\n\r\n-
switch \"beta integrations\" on on Integrations page\r\n- Open APM
overview page again\r\n- expect `8.7.0-preview` version with the
selector to change to last GA\r\n\r\n<img width=\"1100\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206480986-2ed05b12-49e4-4c88-97c6-6cdf12f819da.png\">\r\n\r\n\r\n###
Checklist\r\n\r\n- [ ] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"24e2fcf47014d937a5019d83b95565a010ae4e22"}},"sourceBranch":"main","suggestedTargetBranches":["8.6"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/147257","number":147257,"mergeCommit":{"message":"[Fleet]
fix for apm not found in epr + prerelease flag (#147257)\n\n##
Summary\r\n\r\nFixes
https://github.com/elastic/kibana/issues/146312\r\n\r\nFall back to
bundled packages when loading package archive.\r\n\r\nAlso fixed an
issue with prerelease flag caused
by\r\nhttps://github.com//pull/144431\r\n\r\nLoading
prerelease setting on Add/Edit integration page, so that it\r\nworks
well for cases where the url doesn't contain a concrete version\r\nlike
APM\r\n\r\nSteps to verify:\r\n- Start kibana locally, make sure that
APM bundled package is there
in\r\n`x-pack/plugins/fleet/target/bundled_packages`\r\n- Load
http://localhost:5601/julia/app/integrations/detail/apm/overview\r\n(with
default \"beta integrations\" off flag)\r\n- Expect that 8.6.0 version
is showing up (or 8.7.0 depending on the\r\nbuild)\r\n- When clicking on
Add Elastic APM, expect 8.6.0 to be loaded\r\nsuccessfully\r\n\r\n<img
width=\"1773\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206482128-634ca823-bb21-46a9-a8b9-fc81d5dc126d.png\">\r\n\r\nNote:
there is still a UI issue with loading assets on Overview page if\r\nthe
package is not in EPR.\r\n\r\n<img width=\"1747\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206478275-ee8e9494-234f-44c8-88d9-a91c791b0196.png\">\r\n\r\n-
switch \"beta integrations\" on on Integrations page\r\n- Open APM
overview page again\r\n- expect `8.7.0-preview` version with the
selector to change to last GA\r\n\r\n<img width=\"1100\"
alt=\"image\"\r\nsrc=\"https://user-images.githubusercontent.com/90178898/206480986-2ed05b12-49e4-4c88-97c6-6cdf12f819da.png\">\r\n\r\n\r\n###
Checklist\r\n\r\n- [ ] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"24e2fcf47014d937a5019d83b95565a010ae4e22"}}]}]
BACKPORT-->

Co-authored-by: Julia Bardi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Team:Fleet Team label for Observability Data Collection Fleet team v8.6.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants