-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
batch: allow to pass archive policy #655
Conversation
d7b488e
to
e13a53c
Compare
Pull-request updated, HEAD is now e13a53c |
Pull-request updated, HEAD is now 3192bd0 |
3192bd0
to
17b94ac
Compare
Pull-request updated, HEAD is now 17b94ac |
Tests SUCCESS for HEAD 17b94ac
|
17b94ac
to
9a4efb4
Compare
Pull-request updated, HEAD is now 9a4efb4 |
9a4efb4
to
7420000
Compare
Pull-request updated, HEAD is now 7420000 |
Tests FAILURE for HEAD 7420000
|
Pull-request updated, HEAD is now fd5fa40 |
fd5fa40
to
708aa13
Compare
Pull-request updated, HEAD is now 708aa13 |
Tests FAILURE for HEAD 708aa13
|
708aa13
to
459736f
Compare
Pull-request updated, HEAD is now 459736f |
459736f
to
875b891
Compare
Pull-request updated, HEAD is now 875b891 |
Tests FAILURE for HEAD 875b891
|
1 similar comment
Tests FAILURE for HEAD 875b891
|
gnocchi/rest/api.py
Outdated
try: | ||
measures = MeasuresListSchema(params) | ||
except voluptuous.Error as e: | ||
abort(400, "Invalid input: %s" % e) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're changing the sentence, can we go ahead and change the format to be JSON friendly?
Is Ceilometer using this endpoint and parses this error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No their are not used. I changed that.
Tests SUCCESS for HEAD 875b891
|
875b891
to
0871564
Compare
Pull-request updated, HEAD is now 0871564 |
Tests FAILURE for HEAD 0871564
|
gnocchi/rest/api.py
Outdated
try: | ||
return voluptuous.Schema(schema, required=required)(data) | ||
except voluptuous.Error as e: | ||
abort(400, "Invalid input: %s" % e) | ||
if detailed_exc: | ||
abort(400, e) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But this is gonna return a string right, because voluptuous.Error
has no jsonify()? This does not seem like the final goal we want to achieve.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
abort() method build the json for any voluptuous error now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, I will move the json build here, it make more sense
Pull-request updated, HEAD is now de28bec |
0871564
to
de28bec
Compare
Tests SUCCESS for HEAD de28bec
|
voluptuous.Any(MeasuresListSchema, | ||
{voluptuous.Optional("archive_policy_name"): | ||
six.text_type, | ||
voluptuous.Optional("unit"): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is unit here because in case it needs to be created?
i'll be honest, i don't really like the unit attribute... to me it's equivalent to having a description attribute... but oh well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really get the use case behind unit too. I just need to be able to fill any metric attributes here for Ceilometer and unit is just one of them.
gnocchi/rest/api.py
Outdated
@@ -148,18 +148,24 @@ def deserialize(expected_content_types=None): | |||
return params | |||
|
|||
|
|||
def validate(schema, data, required=True): | |||
def validate(schema, data, required=True, detailed_exc=False): | |||
try: | |||
return voluptuous.Schema(schema, required=required)(data) | |||
except voluptuous.Error as e: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should only catch voluptuous.Invalid
so then…
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
gnocchi/rest/api.py
Outdated
abort(400, "Invalid input: %s" % e) | ||
if detailed_exc: | ||
abort(400, {"cause": "Invalid input", | ||
"detail": str(e)}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
… here you can use e.path
to have details being not a string but a computer understandable path to the key that is wrong.
A string could be constructed for the "reason" field.
Also cause
has been set usually to "Attribute value error"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sileht let me know if you want me to do a different PR for the voluptuous change.
I don't want to merge this patch with that if we need to change it just after
voluptous schema must not abort() but raise an exception otherwise, we can't mix them in voluptous.Any/voluptous.All/...
This change allow to pass the archive policy/unit when we batch measures for metrics.
de28bec
to
4fe78cb
Compare
Pull-request updated, HEAD is now 4fe78cb |
Tests ERROR for HEAD 4fe78cb
|
- ["2015-03-06T14:35:12+00:00", 1.0, 24.0] | ||
|
||
- name: get created metric to check creation with newformat | ||
GET: /v1/resource/generic/46c9418d-d63b-4cdd-be89-8f57ffc5952e/metric/newformat |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should you check that this newformat uses the defined policy and not a default policy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is checked line 228
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
word :)
- ["2015-03-06T14:35:12+00:00", 1.0, 24.0] | ||
|
||
- name: get created metric to check creation with newformat | ||
GET: /v1/resource/generic/46c9418d-d63b-4cdd-be89-8f57ffc5952e/metric/newformat |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
word :)
This creates two archive policies "ceilometer-low" and "ceilometer-low-rate" in Gnocchi. And use them to create metrics in Gnocchi. The archive policies settings can now be configured in gnocchi_resources.yaml. This requires upcoming release of coming (gnocchixyz/gnocchi#655) Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636
This creates two archive policies "ceilometer-low" and "ceilometer-low-rate" in Gnocchi. And use them to create metrics in Gnocchi. The archive policies settings can now be configured in gnocchi_resources.yaml. This requires upcoming release of coming (gnocchixyz/gnocchi#655) Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636 Conflicts: ceilometer/publisher/data/gnocchi_resources.yaml doc/source/admin/telemetry-measurements.rst
This creates two archive policies "ceilometer-low" and "ceilometer-low-rate" in Gnocchi. And use them to create metrics in Gnocchi. The archive policies settings can now be configured in gnocchi_resources.yaml. This requires upcoming release of coming (gnocchixyz/gnocchi#655) Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636 Conflicts: ceilometer/publisher/data/gnocchi_resources.yaml doc/source/admin/telemetry-measurements.rst
This creates two archive policies "ceilometer-low" and "ceilometer-low-rate" in Gnocchi. And use them to create metrics in Gnocchi. The archive policies settings can now be configured in gnocchi_resources.yaml. This requires upcoming release of coming (gnocchixyz/gnocchi#655) Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636 Conflicts: ceilometer/publisher/data/gnocchi_resources.yaml doc/source/admin/telemetry-measurements.rst (cherry picked from commit bb4692d) original commit b96b16928ca84bd1c4025c3f996a65e447a8920f StoryBoard: “Various backports to core openstack components to pick up fixes” https://storyboard.openstack.org/#!/story/2003407
This creates two archive policies "ceilometer-low" and "ceilometer-low-rate" in Gnocchi. And use them to create metrics in Gnocchi. The archive policies settings can now be configured in gnocchi_resources.yaml. This requires upcoming release of coming (gnocchixyz/gnocchi#655) Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636 Conflicts: ceilometer/publisher/data/gnocchi_resources.yaml doc/source/admin/telemetry-measurements.rst (cherry picked from commit bb4692d) original commit b96b16928ca84bd1c4025c3f996a65e447a8920f StoryBoard: “Various backports to core openstack components to pick up fixes” https://storyboard.openstack.org/#!/story/2003407
This creates two archive policies "ceilometer-low" and "ceilometer-low-rate" in Gnocchi. And use them to create metrics in Gnocchi. The archive policies settings can now be configured in gnocchi_resources.yaml. This requires upcoming release of coming (gnocchixyz/gnocchi#655) Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636 Conflicts: ceilometer/publisher/data/gnocchi_resources.yaml doc/source/admin/telemetry-measurements.rst (cherry picked from commit bb4692d) original commit b96b16928ca84bd1c4025c3f996a65e447a8920f StoryBoard: “Various backports to core openstack components to pick up fixes” https://storyboard.openstack.org/#!/story/2003407
* Update ceilometer from branch 'master' - Replace tripleo-scenario002-multinode with scenario002-standalone Change-Id: I09b85238c73c841366ecb08183289ff94e5beb09 Story: https://tree.taiga.io/project/tripleo-ci-board/us/532 - Merge "Replace tripleo-scenario001-multinode with scenario001-standalone" - Use kafka extras for oslo.messaging oslo.messaging replaced kafka-python to confluent-kafka [1], so instead of adding the kafka-python to test-requirements the oslo-messaging[kafka] should be used. [1] https://github.com/openstack/oslo.messaging/blob/master/releasenotes/notes/kafka-client-library-change-fe16d5a34550db7f.yaml#L11 Closes-Bug: #1808350 Change-Id: If56890b35bbe40141c01467e47bccdf4d5584e86 - Replace tripleo-scenario001-multinode with scenario001-standalone The scenario001-standalone job is added in the depends-on below. Tracked by tripleo ci squad at [1] [1] https://tree.taiga.io/project/tripleo-ci-board/us/191 Depends-On: https://review.openstack.org/619508 Change-Id: I4fe2a8a6b7e879a5cb256b68e5ed91ffeeabf79a - Merge "Replace deprecated snmp metrics" - Fix the section of notify_on_state_change to [notifications] https://github.com/openstack/nova/blob/042f08ee968fc83b80e554efe58270c8dfd6265c/nova/conf/notifications.py#L41 Change-Id: Ic09b05e3b4603f5967a2ee67191b931661ef8fe0 Closes-Bug: #1800674 - Replace deprecated snmp metrics Change-Id: I9306afe72601baf87ca4767c15e719ec6d23dfe7 - Merge "Don't quote {posargs} in tox.ini" - Merge "Volume discovery minimum 3.43 api version" - Skip oslo.messaging 9.0.0 release With oslo.messaging 9.0.0 the ceilometer tests are failing. A fix [1] in oslo.messaging already proposed and merged, but a release is needed for the ceilometer tests to pass. [1] https://review.openstack.org/#/c/608196/ Change-Id: Ia90e4fb28c42467a9090d53742b5acc9d4841aaa - Don't quote {posargs} in tox.ini Quotes around {posargs} cause the entire string to be combined into one arg that gets passed to stestr. This prevents passing multiple args (e.g. '--concurrency=16 some-regex') Change-Id: Ieeaf4f9abcb06fdb38fc91fb58d2a684acba7082 - Volume discovery minimum 3.43 api version Set the volume api version to 3.43 to make sure we get 3.41 change where it exposes the user_id for snapshots. This will fix the volume.snapshot.size metric that is not working when trying to access the user_id throws a AttributeError. Change-Id: I2414717e7c9e64b0092fe93c97a062a48dbf09a8 Closes-Bug: 1794475 - Merge "Imported Translations from Zanata" - Merge "event: remove deprecated method" - Merge "Deprecate event subsystem" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I42d4284dc44e9201f073dcb59dce18feaed750a0 - Merge "Use openstack-tox-cover template" - Merge "opts: remove unused import" - opts: remove unused import Change-Id: I24c90f5331897a0f9afc2ce4b3a9a5be42359ae6 - gnocchi: Don't fail if archive policy exists Change-Id: I19abcd590fee728ee1ccd6cd0454505a2d503bd8 - Deprecate event subsystem This subsystem has never been finished and is not maintained. Deprecate it for future removal. Change-Id: I934696b8fc066adc2d9085c64dffb3759fa4489c - event: remove deprecated method Change-Id: Iadda5e51d61c44c4dd11d7fc82bc7938f74baa5c - gnocchi: handle domain for filtering swift Change-Id: I41015ef4ff8ef53e5d40f6ce6e56f3098e327c43 - Fix logic when selecting configuration file if the configuration have been found in /etc/ceilometer we currently raise a RuntimeError. This change fixes that. Change-Id: I121bd9c387b677c1001abeddffc48f9ad3494e5c - agent: Remove unused parameter in load_config This change removes an unused parameter. Change-Id: I376e2823f2af5907f1be2e254c0faddff8116ba7 - Merge "agent: use any to filter source objects" - agent: remove unused config change detection code Change-Id: I0c1a42cfd78cae42289e5d2b35df33f289509461 - agent: use any to filter source objects Change-Id: Ic9105eae9fa72ea34ab336ec8439909553fc4d39 - Merge "pipeline: remove confusing log message" - Merge "notification: remove unused constant" - Merge "notification: declare logging method as static" - Merge "notification: do not store temporary transport" - Merge "notification: remove unused pipeline listener" - Merge "pipeline: simplfy classes" - Merge "compute: remove deprecated disk meters" - Merge "pipeline: remove transformer support" - Merge "Fix ceilometer polling process not able to coordinate issue" - Merge "publisher: mark messaging topic options deprecated" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Iaedfee71dbdb2af920505cfd24e3310c6126d5ac - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Id6300ac46b383be199af9968d48374e23bc59c4b - Merge "Fix the default filter_project name" - Fix the default filter_project name Change-Id: I64a86c80e919b0306b4edbcea5daa03cb1197937 - Use openstack-tox-cover template Use openstack-tox-cover template, this runs the cover job in the check queue only. Remove jobs and use template instead. Change-Id: Icbe18f9f5c2249c878c54621aafc3dd10106f20d - pipeline: remove confusing log message This made sense when Ceilometer supported non-decoupled pipeline configuration, but it does not anymore. Change-Id: I9cfb186e41a65ed16717bb0a66ac7230230692e0 - notification: remove unused constant Change-Id: I1d1dc06e967bd4c91161eb119a37613a7b04f24b - notification: declare logging method as static Change-Id: Ie882ff258d5a7b88caf29e64cd13f01cb5261326 - notification: do not store temporary transport Change-Id: I6ef8a3f7cd3ac2e824bd2a64a64953d13413f0f7 - notification: remove unused pipeline listener Change-Id: If004104d8920f33ab89c7584e685c67208c59675 - pipeline: simplfy classes Change-Id: I40a54a3cf94f8171db17f2b9ae2def4d0e9d1ab1 - compute: remove deprecated disk meters The equivalent disk.device meters are available for a while now. Change-Id: I6f1af3b8d0a1ec32b2722db62ab9cafe6309532f - pipeline: remove transformer support Transformers cannot work correctly on multiple nodes without workload partitioning, which has been removed. The transformation of data is no more the responsability of Ceilometer. The data storage used by default (Gnocchi), is able to handle the scenario that Ceilometer covered by default such as computing the rate of a metric. Change-Id: If3683318b998a37c40bc09314dd8ab3eef326ba7 Depends-On: Ifd1d04ce813028f115c19bc983e7dd1e63c6f8a5 Depends-On: I0330c09d72c20d63d08770b52d3071512a418260 - Follow the new PTI for document build For compliance with the Project Testing Interface as described in: https://governance.openstack.org/tc/reference/project-testing-interface.html For more details information, please refer to: http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html Change-Id: Ibdd264f0ef0c27762d47a70da192a858e62074a9 - Merge "notification: remove workload partitioning" - Merge "gnocchi: add ceilometer-high{,-rate} archive policies" - Merge "gnocchi: use ceilometer-low as default archive policy" - add python 3.6 unit test job This is a mechanically generated patch to add a unit test job running under Python 3.6 as part of the python3-first goal. See the python3-first goal document for details: https://governance.openstack.org/tc/goals/stein/python3-first.html Change-Id: Ife99d6a9620152deb8b991eb56730796fe304afd Story: #2002586 Task: #24339 - switch documentation job to new PTI This is a mechanically generated patch to switch the documentation jobs to use the new PTI versions of the jobs as part of the python3-first goal. See the python3-first goal document for details: https://governance.openstack.org/tc/goals/stein/python3-first.html Change-Id: Ie153246b718d80b1ddf179a9e064cfc2dafbd178 Story: #2002586 Task: #24339 - import zuul job settings from project-config This is a mechanically generated patch to complete step 1 of moving the zuul job settings out of project-config and into each project repository. Because there will be a separate patch on each branch, the branch specifiers for branch-specific jobs have been removed. Because this patch is generated by a script, there may be some cosmetic changes to the layout of the YAML file(s) as the contents are normalized. See the python3-first goal document for details: https://governance.openstack.org/tc/goals/stein/python3-first.html Change-Id: I789ea2f371239239fbe6ec7edcd173a2caf23a65 Story: #2002586 Task: #24339 - notification: remove workload partitioning Workload partitioning has been quite fragile and poorly performing so it's not advised to use it. It was useful for transformers: since transformers are going away too, let's simplify the code base and remove it Change-Id: Ief2f0e00d3c091f978084da153b0c76377772f28 - gnocchi: add ceilometer-high{,-rate} archive policies Change-Id: Id2fde1786fbec012781c1794f78c948fa4e8a24e - gnocchi: use ceilometer-low as default archive policy Otherwise it's never used, at all. Change-Id: I68c29a9f9312300474d5e5e9c6944ce617bc1e3a - Merge "polling: remove deprecated option batch_polled_samples" - polling: remove deprecated option batch_polled_samples Change-Id: Id854b788c3dbc34b9fb944b6dc72d7f3b48d5495 - notifications: remove deprecated meter_definitions_cfg_file option Change-Id: Ia9393b0fc3cf6a68f303f66dbf4aae32d9edba90 - publisher: mark messaging topic options deprecated Change-Id: Id652abb5b059029eff2f6fa91bc966cbe6b9a546 - Fix ceilometer polling process not able to coordinate issue Add option start_heart=True when start partition_coordinator to prevent old member be overwritten when new member joined group Besides upadte member information each time before doing poll Change-Id: I3d0acda90a57b4cb32468433f6bd593a038e76ec Closes-Bug: #1787125 Signed-off-by: Leehom Li (feli5) <[email protected]> - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Ib96c9286d980a835294d0355c3f5f912d0cef772 - Merge "Add option to file publisher to write json" - Merge "objectstore/rgw: Add config option to support RGW implicit tenants" - Merge "Ability to define batch size off polled samples." - Migrate to stestr for running tests stestr is recommended by OpenStack Project Testing Interface for running tests [1]. This change migrates to stestr for running unit tests and coverage job. Modified `cover` env allows running it as Zuul job and visualising its results in Gerrit. [1] https://governance.openstack.org/tc/reference/pti/python.html#python-test-running Change-Id: I9e8b79d7a18cd36946daff10abb1a2f43b1aa3a1 - Add option to file publisher to write json Change-Id: I21f6f8fca8de1d6b8784e140382d1a1a99398279 Closes-Bug: #1783985 - Remove unused tox target Change-Id: Ia2079aa316aa23f7577c94f009fc1af727dac3ab - Add gating on py37 Change-Id: I248c7ff3e306fa1086920c24baa73312e5e31b91 - udp: remove wrong and useless tests Those tests do not work on systems that are able to create AF_INET6 socket for IPv4 addresses (e.g. Darwin). They also don't test anything since they're just mocking stuff to check the logic. Change-Id: I2205b969db09760dd96cfbc0fd701896ef944cf4 - Add support for Python 3.7 Change-Id: I3b880cbf1de13d61d557cde2be70a1efe07761ba - publisher: allow to not inherit from ConfigPublisherBase Change-Id: I9bb2f392bcdd5d920a8f0db883766e5439d7de73 - Merge "Remove deprecated gnocchi_dispatcher option group" - Remove deprecated gnocchi_dispatcher option group Change-Id: Ic8d1a0c56e4bfd04d468c9f4a38236d29f4fe4f8 - Merge "polling: remove useless base class" - Merge "gnocchi: don't create metrics with resource" - gnocchi: don't create metrics with resource Ceilometer creates metrics that will never get measures (service not polled, notification not configured). This is because we create all metrics during resource creation. That was done sometimes ago because Gnocchi was not provide metrics creating during batching process. But since a while now, we also create metrics (if missing) during the batch processing. This change removes the creation of metrics during resource creation. All metrics with measures got created during the batch call. This stops to have metric without measures in Gnocchi. Change-Id: I3f3b805722b11029aefdb76bae93413cb32ecff6 - polling: remove useless base class Change-Id: I7c5a248089b9f6dad8952f302ac99e6525054d82 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I95df9f07b61d2e3cda69717ce692b0d39f1ef936 - Remove dead link in doc file doc/source/contributor/install/manual.rst was removed by https://review.openstack.org/#/c/530930/. Change-Id: I64b89e51d051c30e6e1a5e43b4549b02445e4e30 - objectstore/rgw: Add config option to support RGW implicit tenants If "rgw keystone implicit tenants" is enabled for Rados Gateway to enable Multi-Tenacy [1], the created RGW user IDs are in the format "keystone_project_id$keystone_project_id", i.e. the RGW tenant ID is added to the user ID. This breaks metering via Ceilometer as the usage is always queried for uid=keystone_project_id. If implicit tenants are enabled in RGW, Ceilometer has to query based on the adapted used IDs. This commit introduces support for querying the correct user accounts. [1] http://docs.ceph.com/docs/mimic/radosgw/multitenancy/ Change-Id: I6de4c5ce0e4f59c1d952f8fd39de64937e781280 Signed-off-by: Felix Walter <[email protected]> - Fix broken link to ceph radosgw usage logging docs Change-Id: Ib94057b83c658f1b434f85f22e312b05369979be Closes-Bug: 1776149 - fix tox python3 overrides We want to default to running all tox environments under python 3, so set the basepython value in each environment. We do not want to specify a minor version number, because we do not want to have to update the file every time we upgrade python. We do not want to set the override once in testenv, because that breaks the more specific versions used in default environments like py35 and py36. Change-Id: I6f1df527590d7b9ffdf7dd97c0fedf27f11b9e21 Signed-off-by: Doug Hellmann <[email protected]> - Wrong block format in rst file This bug causes the following code-blocks cannot be rendered. Change-Id: I591ee2e557b0824e4ffb9f9ded6f11dcee0f5e92 - del unused module Change-Id: I1bb4ac082537fe0fd40542fee2c73deed208e5f5 - Ability to define batch size off polled samples. Currently if batch_polled_samples=True (default) then it will batch up all samples of a poller into one message. This can be problematic if the poller returns a large number of samples. Change-Id: Ifc60a0cb81876f6d9054cae82ba6579707a02f67 Closes-bug: #1770310 - Adjust the controller installation position As usual, we install the controller node before compute node, the docs should be optimise Change-Id: Icc6e6976872d59b7a50436a17ed0e01ed50311dc - Merge "fix typos" - fix typos fix misspelling of word 'available' Change-Id: I7b4de4a4eecc587cfe372caeeb02a6dc61cef865 - inspector: memory: use usable of memoryStats if available Since kernel v4.6, virtio balloon driver commit 5057dcd0f introduced metric VIRTIO_BALLOON_S_AVAIL, corresponding to 'Available' in /proc/meminfo. Libvirt exposed this metric as 'usable'. As 'Available' of meminfo is an estimate of how much memory is available for starting new applications, without swapping. It's a better metric for calculating memory_usage. Change-Id: I3b935f1fc2ed74ca45b26990c4f2bd5996e1dfea Signed-off-by: Chen Hanxiao <[email protected]> - snmp: ignore rfc1905.NoSuchInstance result If the SNMP polling return such a value, it should be ignored. Change-Id: I275f43f8800d9477fedd100db68b48c93e7ad75d - snmp: make oid value retrieval more solid There's no need to try to convert before ignoring. Change-Id: Ie9dcb63c43e7d69ce25aff62a2575b89f26ffca5 - Merge "Replace Chinese punctuation with English punctuation" - Replace Chinese punctuation with English punctuation Curly quotes(Chinese punctuation) usually input from Chinese input method. When read from english context, it makes some confusion. Change-Id: I76f9d6102dbf72f234e10ba2ff426682011a5e79 - Remove restiction on allowable namespaces in polling. Change-Id: Icc972d0880283d46ff97203c67816e1aa103d504 Closes-Bug: #1767939 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I55782951a4db49e544bfcf2d5db15c495e8bbf0b - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Iba394f9cceae0f7e27fac06a90f7157476ac93fc - Merge "Stop gnocchi during unstack" - Merge "Unlist non existent storage.api.request" - Merge "Remove liusheng and llu from the maintainers" - Merge "Deprecating transformers and pipeline partitioning" - Deprecating transformers and pipeline partitioning Theses features doesn't work well, rate-of-change metrics can still wrongly be computed even with Pipeline partioning enabled. Also backend like Gnocchi offers a better alternative to compute them. This deprecates these two features, to be able to remove them in a couple of releases. Change-Id: I52362c69b7d500bfe6dba76f78403a9d376deb80 - Unlist non existent storage.api.request This metric has been removed in 2015 when the Ceilometer WSGI middleware has been removed from this repository. The new ceilometermiddleware package does not provide it. Change-Id: Ib0aa454f3f14efa034491a0f113998cb598f9628 - Stop gnocchi during unstack since it is ceilometer's devstack plugin who enables/starts gnocchi services, it is appropriate to also stop them during unstack phase. Change-Id: I97747a98ddb139a6f89fa54b4d17e0ca1d1bdf62 - Merge "publisher: add a Prometheus Pushgateway publisher" - publisher: add a Prometheus Pushgateway publisher This change adds a publisher to push metrics to Prometheus Pushgateway. Change-Id: I7d5f39f036714ac1e36d3297ad58dd8498908a53 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I0a38e3eb0bf211c10510d78b9aae2eb99dbcfe57 - Remove liusheng and llu from the maintainers They have not contributed for >1 year. Change-Id: Ide4223884c80177bf7471d61d3342e9ef7a12b00 - Merge "supplement "zakar" and "https" description" - supplement "zakar" and "https" description It was mentioned "7 different transports", but there just shows 5 transports. I supplement 2 missing transports "zakar" and "https". Change-Id: If5db5e31f7c7d1101ccf4d285cfb67b5af1944cb - fix errors about words spelling. There are some errors about words spelling. Others is about wrong referend. Change-Id: Id04788f2736e2782d888f49bb93751b95eab0a41 - Merge "Fix doc title format error" - Fix doc title format error Change-Id: I58c7fddc0d76b98471b4675e80a16a793c1abd24 Signed-off-by: Yuanbin.Chen <[email protected]> - Modify the empty list ensure method. Change-Id: I1634d0df96578eecba47e990c1228df1170bf91a Signed-off-by: Yuanbin.Chen <[email protected]> - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I745b560b2856ee1991780d23b754f4fd863766e3 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I65e9f83213afe480e112b0c562908403a4bddeeb - Merge "fix a typo in documentation" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Ia9031712c5483e01700dd4449682b52457179676 - Merge "Allow requeues in when there is a failure in EventsNotificationEndpoint" - fix a typo in documentation Change-Id: I18b3af31424fbe3979f3ab3324b8c6ca89498cf4 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Ie26a888c33d61cac88788c54ca99d39dc960a3fd - Merge "Remove run-tests.sh" - Merge "Imported Translations from Zanata" - Merge "avoid verbose tracebacks on known errors" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Iffb35db056786e08abdd9c6d2e7fdf644aa08d25 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Ib2d9fbbe0dc36471de729a192191f356bb5ce0a3 - Don't use gnocchiclient during publisher init We can't create archive policies in publisher init because if it fail because Gnocchi is not yet ready the publisher fail to load. Ceilometer will work, but samples will go to nowhere. This change creates/checks archive policies when we publish a sample for the very first time. If that fail because Gnocchi is not ready, this will just retry next time Ceilometer will publish samples. Closes-Bug: #1752420 Change-Id: Ib6b4da54592ad99a4e6561a73473b6c7ec73a21f - Merge "set cache with attributes only" - Merge "Cached should not be used when creating gnocchi resources" - Merge "gnocchi: configure archive policies on Ceilo side" - Merge "cinder: link volume to image and instance" - gnocchi: configure archive policies on Ceilo side This creates two archive policies "ceilometer-low" and "ceilometer-low-rate" in Gnocchi. And use them to create metrics in Gnocchi. The archive policies settings can now be configured in gnocchi_resources.yaml. This requires upcoming release of coming (https://github.com/gnocchixyz/gnocchi/pull/655) Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636 - Merge "Add new attributes to instance" - Merge "update ipmi docs" - Merge "remove integration playbooks" - cinder: link volume to image and instance This change tracks the image_id and the instance_id of volume. Change-Id: I5e15080fad6cabece132d4609ed473494a64495e - Add new attributes to instance This change adds launched_at/started_at/deleted_at attribute to instance. Rely on started_at/ended_at of Gnocchi to known when the operation have done is not precise. Nova records the exact time of this event, just copy them to Gnocchi. Change-Id: If7870fd525d1ef58e4c2dbbbf93525bab3623b0f - Merge "capture cell name" - Merge "add ipmi sensor data to gnocchi" - Merge "hyper-v: Converts all os-win exceptions" - Merge "fix meter formating" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I927c46ef4d937300c18375a5429a66d9db37ffe4 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I9603f587f4e19bb9775a0702709dc59d9360655a - Merge "add disk.device.*.latency meters" - Merge "remove ceilometerclient references" - hyper-v: Converts all os-win exceptions When os-win was first introduced, a decorator which converts all os-win exceptions to virt inspector exceptions has been added to the HyperVInspector. However, the decorator works as intended for methods that return values, and not for the ones that yield (they return a generator). This patch makes sure that exceptions are converted properly for yielding methods as well. Closes-Bug #1751088 Change-Id: I7d09e1860c6940758f0d0965fedfe4dd285e0cae - Merge "Remove deprecated rgw meters" - remove integration playbooks this is defined in telemetry-tempest-plugin Change-Id: Idc5ceee2c1cf1fec6c13bcabfbe52d926d6a755a - fix meter formating Change-Id: I81b2c275f3173f70f170e875e6e348666c0f684b - the previous patch was missing a 'continue' This commit puts a try/except AttributeError block around all the code that is fetching attributes on the result of metadata_xml.find(...). Change-Id: I41aa76cf9def3e8c4bceef0280d15c1fd7c48e3d Closes-Bug: #1749960 - Gracefully handle missing metadata in libvirt xml Missing metadata in the libvirt domain xml for a nova instance would cause ceilometer-compute to abort, leading to missing metrics for the current and any subequent libvirt guests. This commit puts a try/except AttributeError block around all the code that is fetching attributes on the result of metadata_xml.find(...). Change-Id: I8adc609cc21c86de2daba326d24b73a80d6eb61f Closes-Bug: #1749960 - Merge "ignore compute.instance.update for metrics" - add disk.device.*.latency meters it's missing from docs and gnocchi see: I0235087af459278b9ad0a66f95c4e4c4ac72e112 Change-Id: If2bad05c52c636c728c95ab6e9520cb6467a99e1 - remove ceilometerclient references it's gone. Change-Id: Ibbe3f62714fa76e9b4338eeb507e35ad620208c8 - avoid verbose tracebacks on known errors - log that no pollsters loaded rather than throwing error - ipmi are expected to be missing as note says, just debug logs with details rather than massive traceback. Closes-Bug: #1747536 Change-Id: Ib592c84dca017e94c66aad0170f0317e514fd747 - update ipmi docs - add install instructions - fix docs to better show ipmi meters from notifications(ironic) and pollsters(ipmitool/node manager) Closes-Bug: #1746736 Change-Id: Ia83b56006e201bb0f8681ac1299387fb2ee6bdb6 - add ipmi sensor data to gnocchi we've been missing this data for a while. Change-Id: I0df15c3e2f4ce98a41320a711e1f18d2c5d7c34d Related-Bug: #1746736 - Merge "Fix glossary.rst" - Merge "Update reno for stable/queens" - Fix glossary.rst Fix indentation: Wrong indentation formats text as block quote. Remove unrelated ceilometer definition. Change-Id: I92a661b4c9b74a680504cc824a3b958c71f2570d - Remove deprecated rgw meters Change-Id: Ia1e8466d613acfc5ccfa29cc44bf101afb0e5e88 - Use msgpack instead of msgpack-python msgpack-python got renamed to msgpack (see https://pypi.python.org/pypi/msgpack-python/0.5.1) and msgpack-python is deprecated now. This is important because other requirements already switched to msgpack (eg. oslo.serialization) and installing both in parallel is not possible for Distros. Change-Id: I603eadbc2a5e1705375050920d5f1631c51a5a02 Closes-Bug: #1743445 - Update reno for stable/queens Change-Id: I83009738c0f013580106dc6dcc05a8391919dc14 - Merge "add hardware.disk.read|write.* stats" - Merge "utils: move hash_of_set where it's solely used" - Merge "Imported Translations from Zanata" - Merge "utils: move kill_listeners to ceilometer.notification" - Merge "add volume.provider.* meters to docs" - Merge "Remove bundled intree ceileometer tempest plugin" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I173ccdf966a78697fde223b930c112cdbe0b3b3b - add hardware.disk.read|write.* stats enable read|write hardware stats via snmp see: http://www.circitor.fr/Mibs/Html/U/UCD-DISKIO-MIB.php Change-Id: I98bad10df94539a711382bef6a48163660a79bba - Merge "add volume provider resource types to gnocchi" - Merge "support cinder capacity statistics" - Merge "add note explaining gnocchi coordination_url" - Merge "Deprecate aggregated disk.* metrics on instance" - add volume.provider.* meters to docs Change-Id: I3f562bc5b6e5690a74ab22af82be720c605fb08b Implements: cinder-capacity-notifications - add note explaining gnocchi coordination_url Change-Id: I632663679d0649e1d5912aef98a65c610fdeaac2 Partial-Bug: #1745806 - capture cell name see: https://github.com/openstack/nova/blob/24152a8f8fb60ae0f306bbf21fe775a2c5bafbc8/nova/notifications/base.py#L434 i'm not adding meter because cell_name doesn't exist in libvirt metadata so libvirt_metadata polling will never find it. Change-Id: I58e35261e23041f640b73fecc1e4082db067abe9 - Merge "Zuul: Remove project name" - Deprecate aggregated disk.* metrics on instance disk.* are just aggregates of disk.device.*. We basically build the same think twice. It's up to the backend (ie: Gnocchi) to aggregate them if someone want the aggregate. Change-Id: I612b575004f65665f8630f19f56c2fb3637448fd - ignore compute.instance.update for metrics nova sends *.update events in between create.start and create.end events and it might not have a host name (which we need). in reality to any changes to vcpus/memory/root_gb/ephemeral_gb metrics will either send a *.resize notification and/or require a reboot which sends it's own notification. because of that, just ignore *.update notifications Change-Id: I1f975c2d73adfc19126d70bdb1ab452e37b278fd closes-bug: #1718290 - utils: move hash_of_set where it's solely used Change-Id: I5a842da34c1c2cf96eb24b2d086f5992c4990af1 - utils: move kill_listeners to ceilometer.notification Change-Id: I9da63dcf30c11b58298c6db89090fe9e27a8065a - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I0cd3f7a4b19b1b447626f7190e9a42ba19865a96 - set cache with attributes only only the attributes of a resource may change so there's no reason to capture the entire resource in cache. this means we don't need to cache if no attributes. this also changes code so we only compute attribute_hash once. Change-Id: I1363e0b0dc6e0957a7e933536b67737bd3496cbd - Cached should not be used when creating gnocchi resources Once I delete resource manually by gnocchi client, the resource won't be created by the ceilometer until 600s passed, because the resource has cached in the memcached. Change-Id: I601e39b4c4782276daba3f9d1b7be92f09efabfc Closes-Bug: #1718570 - Merge "utils: move publisher-only utils functions in publisher" - Zuul: Remove project name Zuul no longer requires the project-name for in-repo configuration. Omitting it makes forking or renaming projects easier. Change-Id: Ief4619ab4970bfea8dfcbac47ab3cd5aa951d7a1 - Merge "Replace curly quotes with straight quotes" - Merge "Imported Translations from Zanata" - Remove run-tests.sh This is barely useful now, simplify test running. Change-Id: I21ab76248797cdc05ac64ec4018c4e472c5b7317 - utils: move publisher-only utils functions in publisher Change-Id: Ia110b6ee4e087a3b987d160594d5d6e7958228b2 - remove repeated host It maybe remove repeated host in memory meter. Change-Id: I8972f5efab0c7a28af6b440883f394d35d897eef - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I214c7193e267a6d69e53685bdee95f1995d258b2 - Merge "Delete not applicable definition." - Merge "Remove use of unsupported TEMPEST_SERVICES variable" - Replace curly quotes with straight quotes Curly quotes usually input from Chinese input method. When read from english context, it makes some confusion. Change-Id: Iac767651ec7afd1ef8a656c96a5735ed00310319 - Delete not applicable definition. manager module polling_task definition value None not applicable. Change-Id: If087fb490a3381531fbc20cf12181bab658aa5ae Signed-off-by: Yuanbin.Chen <[email protected]> - Merge "remove sample sorting" - add volume provider resource types to gnocchi Implements: cinder-capacity-notifications Change-Id: I23429467e61fea701949b99a2ac53eb988a6cf64 - support cinder capacity statistics cinder sends a notification about its capacity: https://review.openstack.org/#/c/206923 for provider pool, the name_to_id is an concatenation of host#pool_name. Change-Id: Ica0b55658ae9ac9ec1ed4ef5ab8cda450a139c7d Implements: cinder-capacity-notifications - Merge "Remove state_description for trove" - Remove use of unsupported TEMPEST_SERVICES variable TEMPEST_SERVICES global variable is not supported by devstack since long back. - I380dd20e5ed716a0bdf92aa02c3730359b8136e4 - I9c24705e494689f09a885eb0a640efd50db33fcf Service availability of tempest known services will be set by devstack with local check. - I02be777bf93143d946ccbb8e9eff637bfd1928d4 This commit removes the unused TEMPEST_SERVICES setting Related-Bug: #1743688 Change-Id: I50bd32fc2103af0170ee8d350f31cf589d85422f - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Id39c8faf434f9dd0c7d89342b78cef35f78b48df - Merge "polling: iter randomly over sources and pollsters when polling" - Merge "Remove the redundant conf.py file" - Merge "utils: remove unused decimal/dt conversion functions" - Merge "polling: simplify manager method" - Merge "Remove shuffle_time_before_polling_task option" - Remove state_description for trove State description and state are always with same value, see https://github.com/openstack/trove/blob/master/trove/extensions/mgmt/instances/models.py#L217-L218 Change-Id: Id86f513c329946f3816a852e478f57a5dec21779 - Merge "Add README.rst record more project message" - polling: iter randomly over sources and pollsters when polling By polling in a random order sources and pollster, it is more likely than different Ceilometer agents will not hit the same e.g. API endpoint at the same time. Change-Id: I754a67a8adfb97f8950c666f9aab3bc3d435e2ac Closes-Bug: #1734898 - polling: simplify manager method I know it was a trend to mock everything to see we call code, but that actually does not check anything useful… Change-Id: I0bff9e01c05b58f44bb994992b143e84a1f723c0 - Remove shuffle_time_before_polling_task option The problem that shuffle_time_before_polling_task tries to solve is the startup of a horde of Ceilometer instances that would start polling the same thing at the same time. It's actually unlikely they would all start at the same right second, and the correct fix would be to do that each time. Related-Bug: #1734898 Change-Id: If8141f6b48657c06e8e782eeef9b209dabb2097c - utils: remove unused decimal/dt conversion functions Change-Id: I87f02511cb63a321a964583f9ee71e6ac2259d06 - devstack: fix gnocchi database setup With postgresql, we must wait at least post-config to create the database. Change-Id: I72ce33828febb48136e239d0e70f07674aef9d4a - gate: move tripleo job to experimental Change-Id: I1791bb7b28c8717ad3886e970427e3c48a4fa01e - Merge "tempest: reuse zuul conf of the plugin" - Merge "tempest: use new plugin" - Merge "cleanup measurements page" - Remove bundled intree ceileometer tempest plugin * All the telemetry related projects intree tempest plugins are now moved to telemetry-tempest-plugin Change-Id: I57738964f65495364dcc74821634d54aea1769fd - tempest: reuse zuul conf of the plugin Change-Id: I8af04b3739551ce3002b4e4f46bd767d28fb58cd Depends-On: If5bd38b9cf831fc00da35cca61021779db1b081d - Merge "remove gnocchi ignore option" - Merge "simplify gnocchi batch push setup" - Merge "compute sample as dictionary once" - Merge "fix gnocchi stats logging" - tempest: use new plugin Change-Id: I870081a44d7b0bad3de688f4ba692ee9240bd799 - Merge "use hashmap to quickly find matching resource def" - Merge "tempest: remove ceilometer-api alarm" - Remove the redundant conf.py file This conf.py is redundant, all project guides only use the "doc/source/conf.py" file. We should remove this one. Change-Id: I820688d3ffbf01cc057c1a44bfd69f5291841159 - ignore compute.instance.create.start for metrics do not build metrics off compute.instance.create.start as it doesn't have enough information in it to push to Gnocchi. there is an .end event that happens seconds later so .start event has little value outside context of event. we change fnmatch to regex for more useful regex comparison and it's faster not a complete fix as compute.instance.update also is missing host value occasionally Partial-Bug: #1718290 Change-Id: I2aa7045873fd740255a3be5f64b9e9f5fd94e2d4 - tempest: remove ceilometer-api alarm Change-Id: Id89130fd8a782dea863c85b37a919fbf0f0897dd - Remove extra space between method parameters Change-Id: I8fbcb516febd5c9a0008c9bb727031015b3759de - Merge "change doc ceilometer bug tracker url" - change doc ceilometer bug tracker url Change-Id: Iffccf8aff136cfd1a27aa98e18c07272ef18034a Signed-off-by: Yuanbin.Chen <[email protected]> - remove gnocchi ignore option i'm not sure what this does but it is: not used in gnocchi_resources.yaml, not tested, and not ever explained in original patch[1] [1] https://review.openstack.org/#/c/289807 Change-Id: I20c158747c79a05778f27bf4b29319a6262db640 - simplify gnocchi batch push setup we run through samples and format samples and resources to what gnocchi expects. this simplifies that process a little: - don't needlessly sort and group by metric name - build a full resource model iff resource needs to be created Change-Id: I2ac25b3b0978eed664c500e645bae2d1b4ae6781 - compute sample as dictionary once we shouldn't be wasting time doing the same thing over and over in loop Change-Id: Iaa5aeb5b91004945a24fab105c8b4884918c1644 - fix gnocchi stats logging don't bother with a running count to build stats Change-Id: I6c40e49046115512d5858a6e1348629e10d6a42f - use hashmap to quickly find matching resource def we shouldn't waste our time looping through resources and searching metrics to find a match. just build hashmap and find the resource based on metric. this removes fnmatch functionality because we don't match on wildcards and the code itself actually requires metric names to be explicit to create_metrics Change-Id: I2398247270217759c876ab5a9b60038dad79a9d3 - cleanup measurements page - remove note about previously storing events as meters. - docs are not branchless anymore, remove anything that has been removed - minor grammar changes Change-Id: If8486f5edb3d83c2d520345c028bc3f23609b376 - Add README.rst record more project message Change-Id: I67a77ba02f503c454eeec857f65575a62ceb7170 Signed-off-by: Yuanbin.Chen <[email protected]> - Merge "Add user_id for sample volume.snapshot.size" - Merge "Imported Translations from Zanata" - Merge "Check required Gnocchi version is installed" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I10171d58e504f5d95db02ec366e6f451ddf9d28a - Merge "Replace ujson with json" - Merge "update best practices" - Merge "cleanup data-pipelines admin guide" - Merge "cleanup data collection admin guide" - Check required Gnocchi version is installed Change-Id: I301511279c1d49736027e51274c8740935e154b0 - Replace ujson with json ujson has not had any active maintenance for the last 12 months; switch to using json module instead. Change-Id: I39027b534e94b3f877d881647a7c843183f60f92 Closes-Bug: 1737989 - Merge "Fix maximum recursion depth exceeded bug when property referencing itself" - Merge "Run all telemetry tests in integration jobs" - Fix maximum recursion depth exceeded bug when property referencing itself Change-Id: Ifa5c3a03bfb05f8c69cb7a52f4a09d5d0f7855bd - Run all telemetry tests in integration jobs Change-Id: I551e218a3833bb3b55db6778b871e7994a4873b8 - Merge "cleanup admin-guide architecture" - Merge "Do not check iterable objects before for loop" - Merge "Add missing name traits" - Merge "reorder admin and contributor topics" - Merge "remove configuration from contributor guide" - Merge "remove install section from contributor guide" - Merge "partial clean up of contributor install info" - Merge "ensure pipeline_listener set up" - update best practices - remove 'prior to ocata' conditions. - remove shuffle_time_before_polling_task as it doesn't work in reality and is being removed. - add a note when to enable workload_partitioning of notification agent Change-Id: I44c030835de1517a3c067ab0632c09f4a5fe2f15 - cleanup data-pipelines admin guide - add a note mentioning transformers are not durable and you might be able to use gnocchi - explain how to enabled/disable pipelines. Change-Id: I3f50d5b5a1e22a114ef2c8e17c1e659f1c0eb761 - cleanup data collection admin guide - remove list of notifications used for meters, it's not accurate and is arguably of no value to an administrator even if it's accurate. just list where the endpoints are and how to configure. - remove comments about database connection, this means nothing - remove libvirt comment, listing all services polled as these are stated elsewhere and are just repeated. Change-Id: If67ecf7e507da70abae3734618358c2458f49037 - cleanup admin-guide architecture - remove reference to api - remove dup details on compute, central, ipmi already in data collection - remove hbase from panko support - remove listing libvirt hypervisors, we only test kvm/qemu. - remove generic listing of 'OpenStack Networking' meters. we already have a measurements page for this - remove user, roles, projects notes. we don't have api and it means very little as is Change-Id: I292cdd9297549ab8cb78caf6b15941b3986ca898 - reorder admin and contributor topics - create logical sections - move troubleshooting-guide to bottom as it's basically useless (but we should add to it) - i've no idea what gmr is so i just put it where it is. Change-Id: I647a84e2d29f00a8c64c42914939d60cc8aa6ebe - remove configuration from contributor guide - move polling configuration to admin data-collection docs - move pipeline partitioning to admin data-pipeline docs - add a note that it's only required for transformations - publisher section is already in admin docs - pipeline configuration is already in admin docs Change-Id: Ib29ec71d835b8625b518ede98e00078e7de3d282 - remove install section from contributor guide this is all irrelevant or duplicated: - dbreco.rst - duplicates supported databases in admin-guide - ceilometer to gnocchi section is old and deck doesn't really reflect current gnocchi. also, all ocata+ docs install gnocchi by default. - custom.rst - duplicates telemetry-data-pipelines in admin-guide - dups best practices in admin-guide with shuffle option - upgrade.rst - upgrade is just stop, upgrade, restart for each service - nothing is unique except notification agent with partitioning[1] and existing stuff doesn't address it. [1] https://bugs.launchpad.net/ceilometer/+bug/1729446 Change-Id: I2de2e7ba8789d896b19320c798150d0c4c6efe0d - partial clean up of contributor install info - remove duplication of install-guide steps - create a page for installing with devstack and integrating with other telemetry services. Change-Id: I697ea46bcb0db02fdb561f0161e22411e23679c7 - cleanup contributor plugins details - remove section about tests. this is obvious and useless. - move note about stevedore usage to top - add note about how notification agents work - remove most of polling details as new plugins are mostly copy/paste. Change-Id: I2962aa9815e821c8fc79a8658ddd999daf20b40d - Merge "Remove filter_service_activity option in doc" - Do not check iterable objects before for loop get_VIFs, get_VBDs will return set For more details see the XenAPI docs: https://docs.citrix.com/content/dam/docs/en-us/xenserver/xenserver-62/xenenterpriseapi.pdf Change-Id: Ic3f0e2eb18d5d6408c60979383465575e0a99d05 - Merge "Add doc8 to pep8 check for ceilometer project" - Merge "Do not check keystone domains length in TenantDiscovery" - Do not check keystone domains length in TenantDiscovery For keystone v2 and v3, it always return one domain at least Change-Id: I1586bb07fe135eadc968fec1c675de48b88cf13c - Merge "Imported Translations from Zanata" - Add doc8 to pep8 check for ceilometer project This patch adds a doc8 check of .rst files to the current pep8 check. It includes fixes to the .rst files that didn't pass the check. Change-Id: If159ab37e2f59d7fe9ee1d7c3ebf0f62f030c87f Co-authored-By: Hoang Trung Hieu <[email protected]> - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I8a2a76cd707290cb1923e3bf324006dc38e135be - Remove filter_service_activity option in doc Change-Id: I2540309359f1f309e2329fdc35f1b78df7b0e28b - Minor update comment of devstack settings Change-Id: I273b4bcf3897416ffa61cfc92369b475d29ea8fb - Add user_id for sample volume.snapshot.size Volume Snapshot details include user_id since version 3.41 https://developer.openstack.org/api-ref/block-storage/v3/#list-snapshots-and-details Change-Id: I13ab9733d79a19bc191792ee8c7d11ab477df52e - Remove useless trailing newlines Change-Id: Id8ed40014679d5b999e3a43ef8f957eea90d28a5 - Add missing name traits Change-Id: I4515c7d34be0b3636ad071201c19fe9e3ad83f1e Closes-Bug: #1738361 - Merge "clean up non-install parts of contributor docs" - Merge "devstack: redis on opensuse needs to have default config" - devstack: redis on opensuse needs to have default config this patch adds a default config and uses template version of unit-file to restart redis Change-Id: I3c3498f408ac832c50d8f0e1834247492570fd4c - Merge "minor update to gnocchi contributor docs" - Merge "Update NotifierPublisher class doc" - Merge "Update http publisher options in doc" - Merge "[doc] frequency of polling should be controlled via the polling configuration" - ensure pipeline_listener set up unit tests occasionally fail because it seems like the pipeline_listener has yet to be configured yet. this waits to ensure both the group_state is set and the pipeline_listener is set before doing anything. Change-Id: I5af31d34e2f1f8fa3d198f9d58be5beca7984d65 - remove sample sorting this does nothing. the listener currently consumes events across all queues, drops them into an 'incoming' queue. from here, a thread batches them based on batch options. the batched messages themselves are still in order and therefore sorting it does nothing. the only reason it'd be out of order is if a message was requeue'd. in this case, sorting will probably do nothing since the requeue'd location is probably not remotely close to original ordering and will probably never be batched with same messages to sort correctly. Change-Id: I6faa97bb90bee00ec94f4faf622822ea93655efb - Update http publisher options in doc Change-Id: Ie8cd071c02e4413af41afd1bf317eecec51fc297 - clean up non-install parts of contributor docs - remove specific comments about preferring notifications v. polling just say, polling can add load - remove kafka publisher reference - remove database publisher reference - move key links (wiki, launchpad, ml, etc...) to index - put all meter details in one area - clean up testing Change-Id: I1993a9945334ffe631f2ddb3dcd0316d0e71f258 - Update NotifierPublisher class doc Change-Id: Ib41ba3e53a1705c9fac61a36fca67ebb85134c1b - Fix incorrect yaml code block in pipeline definition Change-Id: I0181cb4bfce5bf3b486938ef64532df77cb0a764 - [doc] frequency of polling should be controlled via the polling configuration In the architecture doc: 'The frequency of polling is controlled via the pipeline configuration. See Pipelines for details' should be changed to: 'The frequency of polling is controlled via the polling configuration. See Pollings for details' Change-Id: I34cfac4555801139700e55aaec24fcf98ee432c1 - remove kafka publisher it's been removed. Change-Id: I8e88573e5526b663d3f25055d6cfad534e3fef5c - Fix typo in utils.py Change-Id: I3722243a8cd31a16bfc60fda45acb7a9d0a0dc47 - Merge "Move utils.dict_to_keyval to opendaylight" - Merge "split partitioning polling tests" - Merge "drop base polling test separation" - Merge "static resources not picked up" - Merge "Move delayed out of utils" - split partitioning polling tests only setup partitioning when the test requires it Closes-Bug: #1732228 Change-Id: I80999b077f867f4c60a918e1c9b80956daebe3de - drop base polling test separation we only have one polling agent and it's difficult to make edits having half the code in one place and the other half in another Change-Id: I8b1a4e840e32e1a4052351569aec12f365d39710 - static resources not picked up polling is still broken if backend_url is not set. switch to tooz hashrings caused it to be ignored. Change-Id: Iace637dd5ed8a852db79b23e99918ca3e5c6605e Closes-Bug: #1732147 - Merge "Merge aodh tempest tests and configuration in ceilometer" - Merge aodh tempest tests and configuration in ceilometer * In order to achieve the tempest plugin split goal, we are merging the aodh tempest tests and config in to ceilometer, then we can move all the telemetry tests in a single repo. Change-Id: I4e0952487d0fe73992e8fe9d6ddda3f98054ec60 - Move delayed out of utils This is only used in one place. Change-Id: If69e8f358be774783471e5a0a6a44bcaa9f80008 - doc: remove useless Ceilometer service creation Change-Id: I061e2b2d55482594cee5671129bd549627b3ee1c - Move utils.dict_to_keyval to opendaylight This seems to be a data transformation specific to this driver, so let's move it there. Change-Id: I3a96fdafbc966732096f4db03cd7f035d169a978 - Merge "ignore api docs" - Merge "cleanup collector references" - Merge "simplify cache generation" - minor update to gnocchi contributor docs Change-Id: If6aae7a3ed5a677bafb145ac5c2ac16807cdcab5 - ignore api docs doc job autogens code docs Change-Id: I8a4f67b4c8addadb6b0e173b0517ec01abadd47b - cleanup collector references we don't have collector anymore. R.I.P. Change-Id: Ie1ea64622f92571373f842668b7573fd45953680 - Merge "Merge panko tempest config into ceilometer repo" - Merge panko tempest config into ceilometer repo * https://review.openstack.org/525072 removes the dummy panko tempest plugin from panko repo but panko tempest config is used in ceilometer integration tests. It merges the required config in order to avoid turbulence in integration tests. Change-Id: I97a5abed3486f63363782f52e7746e87bd88ed4a - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I683f9137cbaf210dd3e6b272947e5bb5367fb3c3 - Merge "remove ceilometerclient" - Merge "Imported Translations from Zanata" - remove ceilometerclient ceilometerclient is no longer maintained Change-Id: I73587e194341670951b7edb321d86af4f256cb27 - Merge "remove unused pollster exception resources" - Merge "simplify test data generation" - Merge "remove duplicate polling.yaml setup" - Merge "remove test_manager_exception_persistency" - Merge "minor polling cleaning" - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I20d9a1c4f485115dff6c077e562d1fade43e5104 - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: I2770cf36fbcf42fbbfec685fa0375bfa1b4cac47 - remove unused pollster exception resources this stuff isn't used in any of the tests. there is an existing pollster exception test here in test_manager.TestRunTasks.test_polling_exception Change-Id: I868698dcad765880d30d5b2703250aad72a33338 - simplify test data generation Change-Id: I611ce1974b62719030b5ff41e694eade5fda8abc - remove duplicate polling.yaml setup Change-Id: Id4640cd5054140e3f175b71a3a0339dba7b9011d - remove test_manager_exception_persistency i'm not sure exactly what this tests... but the original test[1] does not exist anymore [1] I808dcfae18d23240f8e095d6c97c8dede7dede8f Change-Id: I86a96557f7e316850adae32b9976cb1d7c7b12b3 - minor polling cleaning - remove custom Sample object from test - remove mock of threadgroup that does not exist. Change-Id: I1a55df5533b978c4c56dee37502fdb9f961cf91d - Merge "Change oslo.messaging prefetch default" - Allow requeues in when there is a failure in EventsNotificationEndpoint When `ack_on_event_error` is set to False, it ack's the notification anyway. The reason behind this is the configuration of the batch notification listener which doesn't allow requeues Change-Id: I5511272c4dc2d5759cab8b9e695bbd9ed6a1bf6a Closes-Bug: #1720329 - Merge "Remove unused ceilometer.utils.update_nested" - Merge "utils: remove unused create_periodic function" - Add cpu_l3_cache to polling yaml We need this for cache monitoring features to work out of the box. Change-Id: I1d145a7e869416e6c0fe0133d94d126f740db016 - Merge "polling: run polling tasks immediately on start" - Merge "Set shuffle_time_before_polling_task to float and set a minimum" - simplify cache generation don't use lambda to filter. this is 25% faster but really negligible. Change-Id: Ib76470fc3a5cdfc79e497d568147b66c08e6dba1 - Put configurations to appropriate part One for pipeline.yaml, anthter for ceilometer.conf. Change-Id: I19e6e2b0b406f51c15aa8fe91b3f3f61023cd40b - polling: run polling tasks immediately on start When Ceilometer polling agent start, one has to wait N seconds for the first polling to happen. This makes testing extremely difficult. I can't see any good reason to not poll at (re)start. Since the last run time is lost anyway, the interval will never be perfect. So at least let's make it convenient by polling on startup. Also set a default random 0-10 seconds delay before the first poll so if a lot of daemons are started at the same time they don't all hit the same endpoint at the same time. Change-Id: I0741a586cec499c259f0e90977f185c4e68a99d3 - Set shuffle_time_before_polling_task to float and set a minimum Change-Id: I34eb3cc54680f4fe6c3a79c288b749b9dfa6e968 - Merge "Imported Translations from Zanata" - Merge "fix ceilometer-compute invoke libvirt exception error" - fix ceilometer-compute invoke libvirt exception error when nova compute start vm not create by nova-compute, the ceilometer-compute will occur libvirtError: "metadata not found: Requested metadata element is not present". and cause all vm meter not report. Change-Id: Id71788606bc0da9a7959831fb90d13c25c0b8dcb - Imported Translations from Zanata For more information about this automatic import see: https://docs.openstack.org/i18n/latest/reviewing-translation-import.html Change-Id: Ibb389c83463accdb847f3fbd651541901c5e4dbd - Merge "always declare partitioning variables" - Merge "queues per manager rather than per pipeline" - Merge "pluggable notification agent" - always declare partitioning variables also, just use partition_coordinator to figure out if we need to handle stuff as that's what we use everywhere else. Change-Id: I8724a41408b89f29b600a03fbf1c7febb55fb5e5 - Remove setting of version/release from releasenotes Release notes are version independent, so remove version/release values. We've found that projects now require the service package to be installed in order to build release notes, and this is entirely due to the current convention of pulling in the version information. Release notes should not need installation in order to build, so this unnecessary version setting needs to be removed. This is needed for new release notes publishing, see I56909152975f731a9d2c21b2825b972195e48ee8 and the discussion starting at http://lists.openstack.org/pipermail/openstack-dev/2017-November/124480.html . Fix two releasenotes that used broken RST, `X' is invalid, use instead ``X``. The release notes build failed due to the broken RST. Change-Id: I5f526f39bdc9bb76266b5a86a4b96854db21d0de - treat warnings as errors in release notes build Change-Id: Ia440af605a39b8cb765590f8f080fd2cdfbf2249 Signed-off-by: Doug Hellmann <[email protected]> - place release notes inline Replace the instructions for querying for release notes with the results of a reno report because some of the broken release notes are on branches that have been closed and we have no way to edit them. Fix the markup by replacing LaTeX style quotes with RST quotes. Change-Id: I4a42c099280a8cd16e9efa3756058b1ed905926c Signed-off-by: Doug Hellmann <[email protected]> - Merge "Remove unused ceilometer.utils.EPOCH_TIME" - Merge "Remove unused ceilometer.utils.sanitize_timestamp" - Merge "zuul: run TripleO jobs with new zuulv3 layout" - zuul: run TripleO jobs with new zuulv3 layout Configure the layout to execute the scenario001 and 002 jobs. Note that baremetal job will be run on stable/pike and container on Queens and beyond. A note has been added in comment explaining more details. Change-Id: Ic9fc3bd769a5ccafb3f81fe55513c96d611dbe7d - Remove unused ceilometer.utils.update_nested Change-Id: I2fd6b649cf6faf4089d01eb410dfb44f1587ae42 - Remove unused ceilometer.utils.EPOCH_TIME Change-Id: I13f9787e7ce289c428ce87771f432902ec305e01 - Remove unused ceilometer.utils.sanitize_timestamp Change-Id: I471ff71aae9644ea38830b99eafbca22c48adcab - utils: remove unused create_periodic function Change-Id: Ia1e6e45466a4237f81d67885d1a59bed6b06d9d1 - queues per manager rather than per pipeline currently we create queue per pipeline which is not necessary. it creates more memory usage and doesn't necessarily distribute work more effectively. this hashes data to queues based on manager but still internally, the data is destined to specific pipeline based on event_type. this will minimise queue usage while keeping internal code path the same. Change-Id: I0ccd51f13457f208fe2ccedb6e680c91e132f78f - pluggable notification agent event, meter (and any other custom pipeline) can be enabled/disabled by setting `pipelines` option under [notification] agent Change-Id: Ia21256d0308457d077836e27b45d2acb8bb697e4 Closes-Bug: #1720021 - remove redundant filter check Change-Id: I2d1b476b2d867cda942ae095380c836c91872286 - move pipeline out of __init__ see title Change-Id: I9ff4403bffc39dafccc7d2fe407a4498e2f4cae0 - nearly pluggable notification agent notification agent now just asks for pipelinemanagers and gets endpoints it should broadcast to from there. it only sets up a listener for main queue and a listener for internal queue (if applicable) - pass in publishing/processing context into endpoints instead of manager. context is based on partitioning or not - move all endpoint/notifier setup to respective pipeline managers - change interim broadcast filtering to use event_type rather than publisher_id so all filtering uses event_type. - add namespace to load supported pipeline managers - remove some notification tests as they are redundant and only different that it …
This change allow to pass the archive policy when we batch
measures for metrics.