Logs: add stream input vars

[mtojek]

This PR adds variables to logs stream input.

Blocker - I'm opening this as draft, because there is a design issue to be solved. If both, metricbeat and filebeat, have a dataset with the same name (e.g. aws - elb), the dataset/manifest.yml can't define two different types.

see cloudtrail/manifest.yml:

title: Aws Cloudtrail Logs
release: beta
type: logs
streams:
- input: logs
  vars:
  - default: s3
    name: input
  - name: queue_url
  - name: shared_credential_file
  - name: credential_profile_name
  - name: visibility_timeout
  - name: api_timeout
  - name: endpoint
  title: Aws cloudtrail logs
  description: Collect Aws cloudtrail logs

As there is no metricbeat module for cloudtrail, this file hasn't been overridden (opposite to some other aws datasets).

Fixed by creating two directories for datasets having the same name (e.g. cloudwatch-metrics and cloudwatch-logs).

BTW. Package validation passed:

$ go run ./dev/generator/ -sourceDir=./dev/packages/beats/ -publicDir=./public -tarGz=true

BTW2. Packages are properly pulled and presented by Kibana's ingestManager

$ yarn es snapshot -E xpack.security.authc.api_key.enabled=true

$ yarn start --xpack.ingestManager.enabled=true --xpack.ingestManager.epm.enabled=true --xpack.ingestManager.fleet.enabled=true --xpack.ingestManager.epm.registryUrl=http://localhost:8080/

Issue: #221

[ruflin]

The syntax in the package is slightly different because of the UI. It has also a default part inside: https://github.com/elastic/package-registry/blob/master/dev/packages/example/nginx-1.2.0/dataset/access/manifest.yml#L29

[mtojek]

Sorry about that. Fixed.

[ruflin]

Interesting that this will pass :-)

[ruflin]

I'm thinking if we should add it here already or not. I like that it does validate the packages. But I'm wondering if they should already served. The good news is I assume things like nginx and system package keep working, as they have a never version number.

[ruflin]

@mtojek To get this PR in, can you remove this and merge it without it?

[mtojek]

Sure, not a problem. Reverted.