Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic merge from master to 6.x branch #5217

Merged
merged 27 commits into from
Sep 21, 2017

Conversation

ruflin
Copy link
Contributor

@ruflin ruflin commented Sep 21, 2017

No description provided.

Steffen Siering and others added 27 commits September 11, 2017 20:39
* [Docs] Add option to log messages in JSON

* Add logging.files.permissions to full reference yml file

Edit description of file permissions
* Reorder processors pipeline order

- reorder processors, such that all client processors run before the
  global processors
- remove support for EventsMetadataKey

* Combine fields and tags if possible

- combine client internal field updates and configured field updates
  into one dictionary
- combine proessors for adding client and global fields+tags if no
  client processors are configured

* Do not copy if no processors are defined

Do not Clone (deep copy) fields being added in the pipeline if no
processors are configured.

As processors might add/remove fields and potentially modify shared
field objects, these must be copied if there is a chance global shared
structured being overwritten by processors. Especially if processors are
guarded by conditions.

* Fix processors order once again

Problem is local fields+tags must be applied after the globaly
configured fields and tests, while client processors must be run before
the global ones.

* review

- use 'disabled' flag
- simplify pipeline fields init
* memqueue cleanup

Move the buffer and message handling from memqueue broker type
to the actual go routines handling the messaging.

* Add specialized buffers to memqueue

Use specialized buffer types and eventloops depending on the memqueue
configuration.

If flushing is disabled, a region based ring buffer is used, as loads of
small batches can be generated and we want to minimize additional
allocations of small objects.

If flushing is enabled a list of active and flushed buffers is managed
by the queue. If a buffer is flushed by timeout, but not yet processed
(and not full), additional events being published are added to the
already flushed buffer.

* ensure flush list does not contain empty buffers
This command can be used in the beats to be run after `make update` is executed. This allows for example to modify the libbeat specific parts of the config file if needed.

The command is optional in a beat, meaning if it does not exist it won't return an error. Also if it exist and it returns an error, make update will still skip the error.
* Add @metadata.version to json events

- pass beat.Info into the codecs
- update json encoder to include @metadata.version (for Console, File, Logstash, Redis, Kafka)

* Update Ls docs to use the version field

* Fix test build

* s/second/third/

* Fix metricbeat unit tests
* Update queue.mem defaults

- Enable flush based memory queue by default:
  Increases chance of pushing bigger batches of events to the outputs.
  Especially if number of CPUs is limited to 1 or 2 this helps with
  throughput, as outputs and producers go-routines will interleave less.

- Set flush.events=2048 and flush.timeout=1s

* set queue flush parameters in system tests

update queue flush parameters to min_events of 8 and a timeout of 0.1s.
This hopefully stabilizes some time sensitive tests.

* Fix typo in CHANGELOG
This is a convenience script for opening PRs without messing as much with
the Github web UI. For example, it allows to quickly create "close changelog"
PRs against a particular branch.

* Add --labels flag
- Change defaults in reference.yml.
- Set index-prefix to name by default.
[DOCS] Update link to certgen command
Moved the shutdown timeout after stopping the flows, but before stopping
the transaction publishing. It needs to be after stopping the flows because
the tests rely on the shutdown sequence to create flow reports.
We had it in the repository, but wasn't deployed in the packages. This
corrects this. It also renamed LICENSE and NOTICE to LICENSE.txt/NOTICE.txt
to align with Elasticsearch and Kibana.
The merge_pr script did also remove all git ignored files. This lead to issues with the IDE and some of my test files like `filebeat.dev.yml` were also removed. We removed this in the past from cherrypick_pr so I assume its ok to also remove it here.
@exekias exekias merged commit a2f5702 into elastic:6.x Sep 21, 2017
@ruflin ruflin deleted the automatic_merge_from_master_to_6.x_branch branch September 21, 2017 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.