Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V1 controller with agent #561

Merged
merged 10 commits into from
Jun 23, 2022
Merged

V1 controller with agent #561

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
81aec2c
modify specs to get shipper running
fearful-symmetry Jun 14, 2022
d369884
Merge remote-tracking branch 'upstream/main' into v1-controller-with-…
fearful-symmetry Jun 14, 2022
fb5beb1
fix specfiles for shipper
fearful-symmetry Jun 15, 2022
096f5c4
remove debug statements
fearful-symmetry Jun 15, 2022
144744f
Merge remote-tracking branch 'upstream/main' into v1-controller-with-…
fearful-symmetry Jun 16, 2022
d2366fb
Merge remote-tracking branch 'upstream/main' into v1-controller-with-…
fearful-symmetry Jun 20, 2022
45e0e8b
Merge remote-tracking branch 'upstream/main' into v1-controller-with-…
fearful-symmetry Jun 22, 2022
e84d34a
attept at fixing CI issues
fearful-symmetry Jun 22, 2022
a0b2542
cleanup, add comment
fearful-symmetry Jun 22, 2022
cc808f1
update magefile, specfile name
fearful-symmetry Jun 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions dev-tools/packaging/packages.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,14 @@ shared:
source: '{{.AgentDropPath}}/fleet-server-{{ beat_version }}{{if .Snapshot}}-SNAPSHOT{{end}}-{{.GOOS}}-{{.AgentArchName}}.tar.gz.asc'
mode: 0644
skip_on_missing: true
'data/{{.BeatName}}-{{ commit_short }}/downloads/elastic-agent-shipper-{{ beat_version }}{{if .Snapshot}}-SNAPSHOT{{end}}-{{.GOOS}}-{{.AgentArchName}}.tar.gz.sha512':
source: '{{.AgentDropPath}}/elastic-agent-shipper-{{ beat_version }}{{if .Snapshot}}-SNAPSHOT{{end}}-{{.GOOS}}-{{.AgentArchName}}.tar.gz.sha512'
mode: 0644
skip_on_missing: true
'data/{{.BeatName}}-{{ commit_short }}/downloads/elastic-agent-shipper-{{ beat_version }}{{if .Snapshot}}-SNAPSHOT{{end}}-{{.GOOS}}-{{.AgentArchName}}.tar.gz':
source: '{{.AgentDropPath}}/elastic-agent-shipper-{{ beat_version }}{{if .Snapshot}}-SNAPSHOT{{end}}-{{.GOOS}}-{{.AgentArchName}}.tar.gz'
mode: 0644
skip_on_missing: true
'data/{{.BeatName}}-{{ commit_short }}/downloads/cloudbeat-{{ beat_version }}{{if .Snapshot}}-SNAPSHOT{{end}}-{{.GOOS}}-{{.AgentArchName}}.tar.gz':
source: '{{.AgentDropPath}}/cloudbeat-{{ beat_version }}{{if .Snapshot}}-SNAPSHOT{{end}}-{{.GOOS}}-{{.AgentArchName}}.tar.gz'
mode: 0644
Expand Down Expand Up @@ -669,12 +677,49 @@ shared:
source: _meta/kibana.generated
mode: 0644

- &binary_component_files
'{{.BeatName}}{{.BinaryExt}}':
source: build/golang-crossbuild/{{.BeatName}}-{{.GOOS}}-{{.Platform.Arch}}{{.BinaryExt}}
mode: 0755
LICENSE.txt:
source: '{{ repo.RootDir }}/LICENSE.txt'
mode: 0644
NOTICE.txt:
source: '{{ repo.RootDir }}/NOTICE.txt'
mode: 0644
.build_hash.txt:
content: >
{{ commit }}
mode: 0644
'{{.BeatName}}.yml':
source: '{{.BeatName}}.yml'
mode: 0600
config: true

# Binary package spec (tar.gz for linux/darwin) for community beats.
- &binary_spec
<<: *common
files:
<<: *binary_files

# Binary spec for "components" such as the shipper
- &binary_component_spec
<<: *common
files:
<<: *binary_component_files

# Binary package spec (zip for windows) for community beats.
- &windows_binary_component_spec
<<: *common
files:
<<: *binary_component_files
install-service-{{.BeatName}}.ps1:
template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/windows/install-service.ps1.tmpl'
mode: 0755
uninstall-service-{{.BeatName}}.ps1:
template: '{{ elastic_beats_dir }}/dev-tools/packaging/templates/windows/uninstall-service.ps1.tmpl'
mode: 0755

# Binary package spec (zip for windows) for community beats.
- &windows_binary_spec
<<: *common
Expand Down Expand Up @@ -917,6 +962,24 @@ specs:
<<: *binary_spec
<<: *elastic_license_for_binaries

#elastic-agent-shipper packaged for the agent
elastic_shipper_agent:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the packaging spec that's called from the shipper itself during mage package.

- os: windows
types: [zip]
spec:
<<: *windows_binary_component_spec

- os: darwin
types: [tgz]
spec:
<<: *binary_component_spec

- os: linux
types: [tgz]
spec:
<<: *binary_component_spec


# Elastic Beat with Elastic License and binary taken the current directory.
elastic_beat_xpack_reduced:
###
Expand Down
3 changes: 2 additions & 1 deletion internal/pkg/agent/program/supported.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions internal/spec/elastic-agent-shipper.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Shipper
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please rename to elastic-agent-shipper.spec.yml we added spec so we dont have a conflict with config files

cmd: elastic-agent-shipper
args: []
artifact: elastic-agent-shipper
restart_on_output_change: true

rules:
- fix_stream: {}
- filter:
selectors:
- shipper

when: length(${shipper}) > 0
13 changes: 12 additions & 1 deletion magefile.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -689,9 +689,19 @@ func packageAgent(requiredPackages []string, packagingFn func()) {
}
}
} else {
// This shouldn't be needed as a special append() here, but until our artiface APIs have released shipper packages,
// we need to make sure the package command doesn't try to download a shipper tarball and blow up
packedBeats = append(packedBeats, "elastic-agent-shipper")
// build from local repo, will assume beats repo is located on the same root level
fmt.Println(">>> Building from local repo")
for _, b := range packedBeats {
pwd, err := filepath.Abs(filepath.Join("../beats/x-pack", b))
var pwd string
var err error
if !strings.Contains(b, "beat") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'd rather have this explicitely configured e.g changing packedBeats to a struct or a map where key is a beat name and value is IsXpack

pwd, err = filepath.Abs(filepath.Join("../", b))
} else {
pwd, err = filepath.Abs(filepath.Join("../beats/x-pack", b))
}
if err != nil {
panic(err)
}
Expand Down Expand Up @@ -729,6 +739,7 @@ func packageAgent(requiredPackages []string, packagingFn func()) {
}

func fetchBinaryFromArtifactsApi(ctx context.Context, packageName, artifact, version, downloadPath string) error {
fmt.Printf(">>> About to download: %s\n", packageName)
location, err := downloads.FetchBeatsBinary(
ctx,
packageName,
Expand Down