Getting Started build-from-source findings

[johnkerl]

Following https://opentelemetry.io/docs/collector/getting-started/ to build from source, in particular this part

$ git clone [email protected]:open-telemetry/opentelemetry-collector.git; \
    cd opentelemetry-collector; make otelcol; \
    go build examples/demo/app/main.go; ./main & pid1="$!"; \
    ./bin/otelcol_$(go env GOOS)_$(go env GOARCH) --config ./examples/local/otel-config.yaml; kill $pid1

I ran into several issues along the way.

I'm new to Opentelemetry and maybe some of these issues are RTM or PEBKAC. Nonetheless perhaps others would benefit from some of these missing steps being written down?

I feel like a "getting started" should be shorter than longer, and too many details would crowd the page ... maybe we could create and link to a separate "do these things first" section?

Missing addlicense

$ make
addlicense FAILED => add License errors:
/bin/sh: addlicense: command not found

$ go get -u github.com/google/addlicense
$ make
addlicense FAILED => add License errors:
/bin/sh: addlicense: command not found

# ~/.bashrc:
export GOPATH=$HOME/go # package cache
export PATH=${PATH}:$GOPATH/bin # has addlicense in it

$ make
running make checklicense in root
Check License finished successfully
running make checklicense in ./cmd/issuegenerator
Check License finished successfully
running make checklicense in ./cmd/mdatagen
Check License finished successfully
running make checklicense in ./examples/demo/app
Check License finished successfully
running make checklicense in ./internal/tools
Check License finished successfully
running make impi in root
make[2]: impi: No such file or directory
make[2]: *** [impi] Error 1
make[1]: *** [for-all] Error 2
make: *** [goimpi] Error 2

Missing mdatagen

$ make otelcol
go generate ./...
receiver/hostmetricsreceiver/codegen.go:17: running "mdatagen": exec: "mdatagen": executable file not found in $PATH
service/internal/gen.go:17: running "esc": exec: "esc": executable file not found in $PATH
make: *** [otelcol] Error 1

$ cd cmd/mdatagen
$ make
./lint.go
./lint_test.go
./loader.go
./loader_test.go
./main.go
./main_test.go
./metricdata.go
./metricdata_test.go

$ ls -lrt
... no executable ...

$ cat README.md
# Metadata Generator

Tool that generates metadata.

No build info here.

$ go build main.go
# command-line-arguments
./main.go:50:13: undefined: loadMetadata
./main.go:67:13: undefined: formatIdentifier
./main.go:72:30: undefined: templateContext

$ export GOPATH=$(pwd)
$ go build main.go
$GOPATH/go.mod exists but should not

$ mv go.mod go.mod-
$ go build main.go
# command-line-arguments
./main.go:50:13: undefined: loadMetadata
./main.go:67:13: undefined: formatIdentifier
./main.go:72:30: undefined: templateContext

$ go build
...

$ ls -l mdatagen
-rwxr-xr-x  1 kerl  staff  6734520 Dec  4 17:05 mdatagen

$ file mdatagen
mdatagen: Mach-O 64-bit executable x86_64

Missing esc

$ cd ../..

$ pwd
/Users/kerl/git/fork/opentelemetry-collector

$ export PATH=$PATH:$(pwd)/cmd/mdatagen/

$ make otelcol
pattern ./...: directory cmd/mdatagen/pkg/mod/github.com/go-playground/[email protected]+incompatible outside available modules
go generate ./...
service/internal/gen.go:17: running "esc": exec: "esc": executable file not found in $PATH
make: *** [otelcol] Error 1

$ find . -name esc
./cmd/mdatagen/pkg/mod/cache/download/github.com/mjibson/esc
Visit mjibson/esc 

$ go get -u github.com/mjibson/esc
go: downloading github.com/mjibson/esc v0.2.0
...

$ which esc
/Users/kerl/go/bin/esc

$ file $(which esc)
/Users/kerl/go/bin/esc: Mach-O 64-bit executable x86_64

Successful build

$ make otelcol
pattern ./...: directory cmd/mdatagen/pkg/mod/github.com/go-playground/[email protected]+incompatible outside available modules
go generate ./...
GO111MODULE=on \
CGO_ENABLED=0 \
go build -o ./bin/otelcol_darwin_amd64 \
-ldflags "-X go.opentelemetry.io/collector/internal/version.GitHash=38c7aa35 \
-X go.opentelemetry.io/collector/internal/version.BuildType=release" \
./cmd/otelcol

$ ll bin/otelcol_darwin_amd64
-rwxr-xr-x  1 kerl  staff  92940904 Dec  4 17:18 bin/otelcol_darwin_amd64

$ file bin/otelcol_darwin_amd64
bin/otelcol_darwin_amd64: Mach-O 64-bit executable x86_64

$ go build examples/demo/app/main.go

$ ll main
-rwxr-xr-x  1 kerl  staff  14170036 Dec  4 17:20 main

$ file main
main: Mach-O 64-bit executable x86_64

$ ./bin/otelcol_$(go env GOOS)_$(go env GOARCH) \
  --config ./examples/local/otel-config.yaml
2020-12-04T17:21:12.245Z	info	service/service.go:409	Starting OpenTelemetry Collector...{"Version": "latest", "GitHash": "38c7aa35", "NumCPU": 12}
2020-12-04T17:21:12.245Z	info	service/service.go:253	Setting up own telemetry...
2020-12-04T17:21:12.248Z	info	service/telemetry.go:101	Serving Prometheus metrics	{"address": ":8888", "level": 0, "service.instance.id": "6e73ab1e-2fd9-4892-b2c9-3ff9a256d40c"}
2020-12-04T17:21:12.248Z	info	service/service.go:290	Loading config
[lots and lots of output, yay!]

Setup for subsequent runs

# Good idea for ~/.bashrc:
export GOPATH=$HOME/go # package cache
export PATH=${PATH}:$GOPATH/bin # has addlicense in it

Either in local script, or, maybe ~/.bashrc:

export PATH=$PATH:$(pwd)/cmd/mdatagen

[johnkerl]

(I'm happy to put up a PR, but, I wanted to first get a feel for whether I'm going in the right direction or not.)

[austinlparker]

I think there's some steps missing from the docs that are in https://github.com/open-telemetry/opentelemetry-collector/blob/master/CONTRIBUTING.md, yeah. I think make install-tools fixes some of this though.

[flands]

Yeah, make install-tools is the missing step and then this works.