Vagrant beats building and testing plattform

[philippkahr]

Describe the enhancement:
Hi,

I hope it is ok that I ping you on this issue @andrewkroh as I saw that you worked on this Vagrantfile.

I am currently trying to develop modules for beats and add more dashboards. However, my MacOS is cluttered with multiple python and go environments from other OSS projects and scientific research. Thus I am facing issues with beats not honoring python environments setup by anaconda, not properly resolving dependencies for python libs like import yaml and many more weird issues.

I started using the provided Vagrantfiles to bootstrap me a ubuntu1804, which works sort of fine. When using make import-dashboards it tries to resolve the two environment variables ES_URL and KIBANA_URL if they are not set, it will set them to http://localhost:9200 and http://localhost:5601. The variables are set inside the https://github.com/elastic/beats/blob/master/libbeat/scripts/Makefile. This is a problem as the vagrant box cannot contact my elastic search and kibana via localhost. When using VirtualBox as a provider, the localhost is always reachable under 10.0.2.2 while using NAT as a network mode. Per default, Vagrant creates such a NAT network adapter. I solved this issue by adding the following code and adding it as a shell provisioner to the ubuntu1804 box. The script will check if it is virtualized by Oracle (VirtualBox) and set the ES_URL and KIBANA_URL to the localhost 10.0.2.2.

def linuxEnvironmentVariables()
  return <<SCRIPT
#!/usr/bin/env bash
if hostnamectl status | grep -q "Virtualization: oracle"; then
  echo 'export ES_URL=http://10.0.2.2:9200' >> ~/.bash_profile
  echo 'export KIBANA_URL=http://10.0.2.2:5601' >> ~/.bash_profile
fi
SCRIPT
end

Additionally, the VirtualBox image created by Vagrant defaults to 1024mb of RAM which is not enough to even build a single beat. Therefore, I propose to add this section the boxes you are using. I found it failing with 2048MB sometimes, I could not find the sweet spot yet, at least with 4096MB it is working without any issue. The default Vagrant box uses only 2 cores which makes the build slower, changing the VirtualBox CPUs to 4 adds more power.

    #c.vm.provider :virtualbox do |vbox|
    #  vbox.memory = 4096
    #  vbox.cpus = 4
    #end

Error logs

Not enough RAM error

vagrant@ubuntu-bionic:~/go/src/github.com/elastic/beats/metricbeat$ make
go build -i -ldflags "-X github.com/elastic/beats/libbeat/version.buildTime=2019-10-19T11:53:36Z -X github.com/elastic/beats/libbeat/version.commit=363e3494bdafc50c3447e7a73d94e4caf9fc3c8c"
# github.com/elastic/beats/metricbeat/module/vsphere/vendor/github.com/vmware/govmomi/vim25/types
fatal error: runtime: out of memory

runtime stack:
runtime.throw(0xe60e57, 0x16)
        /usr/local/go/src/runtime/panic.go:617 +0x72
runtime.sysMap(0xc044000000, 0x4000000, 0x16cd178)
        /usr/local/go/src/runtime/mem_linux.go:170 +0xc7
runtime.(*mheap).sysAlloc(0x16a57c0, 0x2000, 0x16a57d0, 0x1)
        /usr/local/go/src/runtime/malloc.go:633 +0x1cd
runtime.(*mheap).grow(0x16a57c0, 0x1, 0x0)
        /usr/local/go/src/runtime/mheap.go:1222 +0x42
runtime.(*mheap).allocSpanLocked(0x16a57c0, 0x1, 0x16cd188, 0x7f580587cac8)
        /usr/local/go/src/runtime/mheap.go:1150 +0x37f
runtime.(*mheap).alloc_m(0x16a57c0, 0x1, 0x16c0012, 0x7f580587cac8)
        /usr/local/go/src/runtime/mheap.go:977 +0xc2
runtime.(*mheap).alloc.func1()
        /usr/local/go/src/runtime/mheap.go:1048 +0x4c
runtime.systemstack(0x0)
        /usr/local/go/src/runtime/asm_amd64.s:351 +0x66
runtime.mstart()
        /usr/local/go/src/runtime/proc.go:1153

goroutine 1 [running]:
runtime.systemstack_switch()
        /usr/local/go/src/runtime/asm_amd64.s:311 fp=0xc034b22080 sp=0xc034b22078 pc=0x458f30
runtime.(*mheap).alloc(0x16a57c0, 0x1, 0xc043010012, 0xc000330480)
        /usr/local/go/src/runtime/mheap.go:1047 +0x8a fp=0xc034b220d0 sp=0xc034b22080 pc=0x423f9a
runtime.(*mcentral).grow(0x16a5fc0, 0x0)
        /usr/local/go/src/runtime/mcentral.go:256 +0x95 fp=0xc034b22118 sp=0xc034b220d0 pc=0x416ee5
runtime.(*mcentral).cacheSpan(0x16a5fc0, 0x0)
        /usr/local/go/src/runtime/mcentral.go:106 +0x2ff fp=0xc034b22178 sp=0xc034b22118 pc=0x4169ef
runtime.(*mcache).refill(0x7f580abf46d0, 0x12)
        /usr/local/go/src/runtime/mcache.go:135 +0x86 fp=0xc034b22198 sp=0xc034b22178 pc=0x416486
runtime.(*mcache).nextFree(0x7f580abf46d0, 0x12, 0x80, 0xe3d440, 0xc043feb3b0)
        /usr/local/go/src/runtime/malloc.go:786 +0x88 fp=0xc034b221d0 sp=0xc034b22198 pc=0x40b3c8
runtime.mallocgc(0x80, 0xe3d440, 0xc043feb301, 0xc043ff7f80)
        /usr/local/go/src/runtime/malloc.go:939 +0x76e fp=0xc034b22270 sp=0xc034b221d0 pc=0x40bcde
runtime.newobject(0xe3d440, 0xc043ff7f80)
        /usr/local/go/src/runtime/malloc.go:1068 +0x38 fp=0xc034b222a0 sp=0xc034b22270 pc=0x40c0e8
cmd/compile/internal/gc.nodl(0x100000000001, 0xc043fe6814, 0xc043feb3b0, 0xc043ff7f80, 0xc043ff7f80)
        /usr/local/go/src/cmd/compile/internal/gc/subr.go:324 +0x11a fp=0xc034b222d8 sp=0xc034b222a0 pc=0xcde47a
cmd/compile/internal/gc.nod(...)
        /usr/local/go/src/cmd/compile/internal/gc/subr.go:301
cmd/compile/internal/gc.walkcompare(0xc043ff7e80, 0xc043ff7f10, 0xc00015963a)
        /usr/local/go/src/cmd/compile/internal/gc/walk.go:3198 +0x119f fp=0xc034b22538 sp=0xc034b222d8 pc=0xd3290f
cmd/compile/internal/gc.walkexpr(0xc043ff7e80, 0xc043ff7f10, 0x100000000001)
        /usr/local/go/src/cmd/compile/internal/gc/walk.go:554 +0x69ad fp=0xc034b22bf8 sp=0xc034b22538 pc=0xd1e0dd
cmd/compile/internal/gc.walkexprlist(0xc043f7bbe0, 0x1, 0x1, 0xc043ff7f10)
        /usr/local/go/src/cmd/compile/internal/gc/walk.go:365 +0x50 fp=0xc034b22c28 sp=0xc034b22bf8 pc=0xd16ff0
cmd/compile/internal/gc.walkstmt(0xc043ff7f00, 0xc043f7bb01)
        /usr/local/go/src/cmd/compile/internal/gc/walk.go:306 +0x7f3 fp=0xc034b22e08 sp=0xc034b22c28 pc=0xd15bb3
cmd/compile/internal/gc.walkstmtlist(0xc043f7bbf0, 0x1, 0x1)
        /usr/local/go/src/cmd/compile/internal/gc/walk.go:79 +0x46 fp=0xc034b22e30 sp=0xc034b22e08 pc=0xd152a6
cmd/compile/internal/gc.walk(0xc043fe6840)
        /usr/local/go/src/cmd/compile/internal/gc/walk.go:63 +0x3e4 fp=0xc034b22f38 sp=0xc034b22e30 pc=0xd14df4
cmd/compile/internal/gc.compile(0xc043fe6840)
        /usr/local/go/src/cmd/compile/internal/gc/pgen.go:235 +0x6c fp=0xc034b22fb0 sp=0xc034b22f38 pc=0xc7485c
cmd/compile/internal/gc.funccompile(0xc043fe6840)
        /usr/local/go/src/cmd/compile/internal/gc/pgen.go:221 +0xc2 fp=0xc034b23008 sp=0xc034b22fb0 pc=0xc746e2
cmd/compile/internal/gc.geneq(0xc043ff9a40, 0xc002f5c240)
        /usr/local/go/src/cmd/compile/internal/gc/alg.go:483 +0x96e fp=0xc034b23230 sp=0xc034b23008 pc=0xbe520e
cmd/compile/internal/gc.dalgsym(0xc002f5c240, 0xffffffffffffffff)
        /usr/local/go/src/cmd/compile/internal/gc/reflect.go:1684 +0x5cc fp=0xc034b232c8 sp=0xc034b23230 pc=0xc9735c
cmd/compile/internal/gc.dcommontype(0xc043ff0af0, 0xc002f5c240, 0xc002f5c001)
        /usr/local/go/src/cmd/compile/internal/gc/reflect.go:810 +0x96 fp=0xc034b233d0 sp=0xc034b232c8 pc=0xc91296
cmd/compile/internal/gc.dtypesym(0xc002f5c240, 0xc00da21880)
        /usr/local/go/src/cmd/compile/internal/gc/reflect.go:1348 +0xd19 fp=0xc034b235a8 sp=0xc034b233d0 pc=0xc940b9
cmd/compile/internal/gc.dumpsignats()
        /usr/local/go/src/cmd/compile/internal/gc/reflect.go:1510 +0x38c fp=0xc034b236d0 sp=0xc034b235a8 pc=0xc95e9c
cmd/compile/internal/gc.dumpLinkerObj(0xc012c9ba20)
        /usr/local/go/src/cmd/compile/internal/gc/obj.go:133 +0xd4 fp=0xc034b237c0 sp=0xc034b236d0 pc=0xc66fb4
cmd/compile/internal/gc.dumpobj1(0x7ffc30cb253b, 0x23, 0x3)
        /usr/local/go/src/cmd/compile/internal/gc/obj.go:71 +0x106 fp=0xc034b23868 sp=0xc034b237c0 pc=0xc666b6
cmd/compile/internal/gc.dumpobj()
        /usr/local/go/src/cmd/compile/internal/gc/obj.go:47 +0x50 fp=0xc034b23890 sp=0xc034b23868 pc=0xc66540
cmd/compile/internal/gc.Main(0xe797e0)
        /usr/local/go/src/cmd/compile/internal/gc/main.go:710 +0x3371 fp=0xc034b23f20 sp=0xc034b23890 pc=0xc4c5f1
main.main()
        /usr/local/go/src/cmd/compile/main.go:51 +0xad fp=0xc034b23f98 sp=0xc034b23f20 pc=0xd8721d
runtime.main()
        /usr/local/go/src/runtime/proc.go:200 +0x20c fp=0xc034b23fe0 sp=0xc034b23f98 pc=0x42e54c
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc034b23fe8 sp=0xc034b23fe0 pc=0x45ae81
../libbeat/scripts/Makefile:107: recipe for target 'metricbeat' failed
make: *** [metricbeat] Error 2

Ressources
https://docs.oracle.com/cd/E97728_01/F12469/html/changenat.html

• I have found a lot of issues regarding build time, python2 and many more. I could not really find something related to Vagrant.

[jsoriano]

@philippkahr could you check if something has improved now after migrating to Python 3 and venv (#14798) so you don't need the virtual machine for development?
Suggestions for the documentation added are also welcome 🙂

[philippkahr]

@jsoriano thanks for the heads up! everything seems to work now with python 3. I still think that the vagrant boxes might be useful for some. I will still try to build it and run it once in the vagrant box before committing it, to see if it really builds on a proper new platform all the time :)

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.