-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Gather BIT metrics [implementation] #3122
Conversation
Quick question: this is supposed to be used during development and not as a publicly available feature flag, right? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work, this will help us improve performance and provide valuable insights 👍 I do have a few questions and remarks to improve this even more.
Also, I still think we should eventually use OpenTelemetry for these kind of benchmarks or metrics gathering. It gives a more standardize way to deal with this and possibly save us a lot of code and effort in developing our own tools. This metric isn't the best example maybe, but it's close to what is possible with OpenTelemetry.
octopoes/octopoes/config/settings.py
Outdated
@@ -89,3 +89,4 @@ def settings_customise_sources( | |||
DEFAULT_LIMIT = 50 | |||
DEFAULT_OFFSET = 0 | |||
QUEUE_NAME_OCTOPOES: str = "octopoes" | |||
GATHER_BIT_METRICS: bool = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure what the intention is, but usually feature flags are configurable. Perhaps this fits better in the Settings
class and documented. This could work if it's only a private debug variable for development, although the location is bit misleading since it's placed next to the constants.
The bool
hint is unnecessary here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The intention is to have a developer flag to toggle gathering [bit] metrics (without exposing it to the public). Please let me know what the best way of defining this would be.
Thanks @ammar92 for your review. I will address your style and efficiency concerns (whilst resolving the merge conflicts). Regarding OpenTelemetry, can you elaborate on what you would like to see; where you would like this to go? |
Thanks and welcome back!
Let's meet soon to discuss this. The main message I'd like to convey is that it could provide us with valuable insights into performance, but also how the application functions over time or the scoped timeline of a specific action within the application (e.g. what requests were made, in what order and consumed time). |
@originalsouth Could you also rename the branch from 'feature/3116' to something more descriptive? This will make typing release notes much easier. Thanks! |
We had this discussion before, branch names do not matter whatsoever and make a whole mess when changing. I can rename it if you insist but I do not see the point. |
3851bf7
to
90fbda2
Compare
Lets keep our branch-names somewhat descriptive. They do matter in the creation of release notes, and when looking at the lists of branches. having said that, there's no need to rename this branch, as I agree that will just make things messy. |
Checklist for QA:
What works:Ran into some issues while running the scripts, which resulted in a few changes. Those are now fixed and it looks great. Only thing that remains is writing documentation on how to use the scripts. This will be picked up in #3228 What doesn't work:n/a Bug or feature?:n/a |
* main: (31 commits) Refactor Task List and filters with error handlers for Scheduler (#1957) Fix filtering on plugin_id for normalizers (#3226) Implement `structlog` (#3175) Gather BIT metrics [implementation] (#3122) Add observation data to observation table in OOI detail page (#3186) cve-2024-6387 from RickGeex (#3194) Recalculate bit when a config object changes (#3206) Use more concise regexes (#3181) Updated Django (#3217) Updated `zipp` (#3215) Feature/boefje normalizer config models (#3118) Updated `certifi` (#3209) Add pluginToggler.js to Aggregate Report (#3202) Update to Django 5.0 (#2939) Update Dockerfile, fix Sonarcloud issue (#3180) Better default list of world writable domains in CSP checker (#3165) Update 1.16 release notes (#3195) Remove non standard header findings and add deprecated headers findings (#3127) Fix/sonarcloud https redirect dockerfiles (#3185) Bump docker/build-push-action from 5 to 6 (#3164) ...
Changes
See #3116
Issue link
See #3116
Demo
See #3116
QA notes
Please add some information for QA on how to test the newly created code.
Code Checklist
.env
changes files if required and changed the.env-dist
accordingly.Checklist for code reviewers:
Copy-paste the checklist from the docs/source/templates folder into your comment.
Checklist for QA:
Copy-paste the checklist from the docs/source/templates folder into your comment.