Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Functionbeat 7.9.0 deploy failure on GCP #20830

Closed
aschifman opened this issue Aug 27, 2020 · 6 comments · Fixed by #21070
Closed

Functionbeat 7.9.0 deploy failure on GCP #20830

aschifman opened this issue Aug 27, 2020 · 6 comments · Fixed by #21070
Assignees
@kvch
Labels
Functionbeat Team:Services (Deprecated) Label for the former Integrations-Services team

Comments

@aschifman
Copy link
Contributor

Deploying functionbeat 7.9.0 fails after the following command:

./functionbeat -v -e -d "*" deploy storage

Error output:

Function: storage, could not deploy, error: error while creating function (code: 3):
Build failed: src/storage/storage.go:11:2: cannot find package "github.com/elastic/beats/v7/libbeat/beat" in any of:
        /usr/local/go/src/github.com/elastic/beats/v7/libbeat/beat (from $GOROOT)
        /workspace/src/github.com/elastic/beats/v7/libbeat/beat (from $GOPATH)"

Many additional lines similar to above and finishes with...

Fail to deploy 1 function(s)"

For confirmed bugs, please report:

  • Version: functionbeat-7.9.0-linux-x86_64
  • Operating System: CentOS-6-1
  • Discuss Forum URL: spoke with Noémi Ványi through slack #beats
  • Steps to Reproduce: I can see same results with MacOS laptop and GCP provisioned CentOS VM. Previous 7.8.1 version using the same credentials file and functionbeat.yml file succeeded with following output:
2020-08-27T14:43:11.850Z        DEBUG   [gcp.executor]  executor/executor.go:68 All operations successful
2020-08-27T14:43:11.850Z        DEBUG   [gcp]   gcp/cli_manager.go:50   Successfully created function 'storage'
2020-08-27T14:43:11.850Z        DEBUG   [gcp]   gcp/cli_manager.go:51   Deploy finish for function 'storage'
Function: storage, deploy successful
2020-08-27T14:43:11.850Z        DEBUG   [cli-handler]   cmd/cli_handler.go:64   Deploy execution ended"
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 27, 2020
@kvch kvch added Functionbeat Team:Services (Deprecated) Label for the former Integrations-Services team labels Aug 27, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 27, 2020
@rayward
Copy link

rayward commented Sep 10, 2020 •
edited
Loading

I'm getting a whole bunch of package missing errors as well (though trying to deploy for pubsub).

Is this because of the cut over to go modules @kvch?

Should ./functionbeat be generating a go.mod file to be bundled into the package?

I tried

go mod init github.com/elastic/beats/v7
go mod tidy

but the deploy command won't include go.mod and go.sum

@kvch
Copy link
Contributor

kvch commented Sep 10, 2020

@rayward Yes, the issue exists because of go modules. However, we need to go back to vendoring because the Beats framework supports only Golang 1.14. This seems to be incompatible with the Go 1.13 environment on GCP.

@rayward
Copy link

rayward commented Sep 10, 2020 •
edited
Loading

@aschifman I was able to deploy with 7.7.1 of functionbeat

edit: though i now just get Exiting: error loading config file: open functionbeat.yml: no such file or directory :(

@rayward
Copy link

rayward commented Sep 11, 2020

@kvch I thought 1.13 supported go modules, and GCP supports this - https://cloud.google.com/functions/docs/writing/specifying-dependencies-go

@aschifman
Copy link
Contributor Author

Thanks @rayward. Please try with 7.8.1 if you can, it had worked for me.

@kvch kvch mentioned this issue Sep 14, 2020
Merged
3 tasks
kvch added a commit that referenced this issue Sep 14, 2020
@kvch
Add vendoring to Google Cloud Functions again (#21070)
50ea11a 
## What does this PR do?

This PR adds the vendor folder back the Cloud Functions of Functionbeat, so they can be used on GCP. This PR basically reverts the removal of the vendor builder functionality.

## Why is it important?

The functions are currently broken, as there is no go.mod file and vendor folder. However, we cannot use go modules, because the Golang environment on GCP runs go version 1.13. Thus, we have to fall back to vendoring.

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
- [x] I have added an entry in `CHANGELOG.next.asciidoc` or `CHANGELOG-developer.next.asciidoc`.

## Related issues

Closes #20830
kvch added a commit to kvch/beats that referenced this issue Sep 14, 2020
@kvch
Add vendoring to Google Cloud Functions again (elastic#21070)
b60cd4b 
## What does this PR do?

This PR adds the vendor folder back the Cloud Functions of Functionbeat, so they can be used on GCP. This PR basically reverts the removal of the vendor builder functionality.

## Why is it important?

The functions are currently broken, as there is no go.mod file and vendor folder. However, we cannot use go modules, because the Golang environment on GCP runs go version 1.13. Thus, we have to fall back to vendoring.

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
- [x] I have added an entry in `CHANGELOG.next.asciidoc` or `CHANGELOG-developer.next.asciidoc`.

## Related issues

Closes elastic#20830

(cherry picked from commit 50ea11a)
@kvch kvch mentioned this issue Sep 14, 2020
Merged
3 tasks
kvch added a commit to kvch/beats that referenced this issue Sep 14, 2020
@kvch
Add vendoring to Google Cloud Functions again (elastic#21070)
b4d55e6 
## What does this PR do?

This PR adds the vendor folder back the Cloud Functions of Functionbeat, so they can be used on GCP. This PR basically reverts the removal of the vendor builder functionality.

## Why is it important?

The functions are currently broken, as there is no go.mod file and vendor folder. However, we cannot use go modules, because the Golang environment on GCP runs go version 1.13. Thus, we have to fall back to vendoring.

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
- [x] I have added an entry in `CHANGELOG.next.asciidoc` or `CHANGELOG-developer.next.asciidoc`.

## Related issues

Closes elastic#20830

(cherry picked from commit 50ea11a)
@kvch kvch mentioned this issue Sep 14, 2020
Merged
3 tasks
kvch added a commit that referenced this issue Sep 14, 2020
@kvch
Add vendoring to Google Cloud Functions again (#21070) (#21078)
efd1f3b 
## What does this PR do?

This PR adds the vendor folder back the Cloud Functions of Functionbeat, so they can be used on GCP. This PR basically reverts the removal of the vendor builder functionality.

## Why is it important?

The functions are currently broken, as there is no go.mod file and vendor folder. However, we cannot use go modules, because the Golang environment on GCP runs go version 1.13. Thus, we have to fall back to vendoring.

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
- [x] I have added an entry in `CHANGELOG.next.asciidoc` or `CHANGELOG-developer.next.asciidoc`.

## Related issues

Closes #20830

(cherry picked from commit 50ea11a)
kvch added a commit that referenced this issue Sep 14, 2020
@kvch
Cherry-pick #21070 to 7.9: Add vendoring to Google Cloud Functions ag…
6148cbf 
…ain (#21079)

* Add Known issues section with an issue about Functionbeat

* Add vendoring to Google Cloud Functions again (#21070)

## What does this PR do?

This PR adds the vendor folder back the Cloud Functions of Functionbeat, so they can be used on GCP. This PR basically reverts the removal of the vendor builder functionality.

## Why is it important?

The functions are currently broken, as there is no go.mod file and vendor folder. However, we cannot use go modules, because the Golang environment on GCP runs go version 1.13. Thus, we have to fall back to vendoring.

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
- [x] I have added an entry in `CHANGELOG.next.asciidoc` or `CHANGELOG-developer.next.asciidoc`.

## Related issues

Closes #20830

(cherry picked from commit 50ea11a)
leweafan pushed a commit to leweafan/beats that referenced this issue Apr 28, 2023
@kvch
Cherry-pick elastic#21070 to 7.9: Add vendoring to Google Cloud Funct…
f18890e 
…ions again (elastic#21079)

* Add Known issues section with an issue about Functionbeat

* Add vendoring to Google Cloud Functions again (elastic#21070)

## What does this PR do?

This PR adds the vendor folder back the Cloud Functions of Functionbeat, so they can be used on GCP. This PR basically reverts the removal of the vendor builder functionality.

## Why is it important?

The functions are currently broken, as there is no go.mod file and vendor folder. However, we cannot use go modules, because the Golang environment on GCP runs go version 1.13. Thus, we have to fall back to vendoring.

## Checklist

- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
- [x] I have added an entry in `CHANGELOG.next.asciidoc` or `CHANGELOG-developer.next.asciidoc`.

## Related issues

Closes elastic#20830

(cherry picked from commit 7c6df99)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kvch kvch

Labels
Functionbeat Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add vendoring to Google Cloud Functions again kvch/beats
4 participants
@rayward @kvch @elasticmachine @aschifman