Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: active orders observability TODOs #469

Merged
merged 6 commits into from
Aug 26, 2024

Conversation

p0mvn
Copy link
Member

@p0mvn p0mvn commented Aug 23, 2024

We need to add these. Making this PR for tracking. Please

Summary by CodeRabbit

  • New Features

    • Enhanced error handling for order book processing with added alert suggestions.
    • Improved logging visibility for error conditions in active orders and limit order creation.
    • Introduced new telemetry metrics for monitoring errors in order management.
  • Bug Fixes

    • Adjusted normalization factor calculation for price determination, ensuring more accurate values.
  • Chores

    • Added comments indicating the need for future monitoring improvements via Prometheus metrics.

Copy link
Contributor

coderabbitai bot commented Aug 23, 2024

Walkthrough

The changes involve enhancements to error handling and logging in the ingestUseCase and orderbookUseCaseImpl classes. A new Prometheus metric counter has been introduced to monitor errors during order book processing. Additionally, the logging level for errors has been elevated, and new telemetry metrics have been implemented to improve visibility and monitoring of order-related operations.

Changes

Files Change Summary
ingest/usecase/ingest_usecase.go Introduced a Prometheus metric counter for tracking order book processing errors and updated error logging with a more descriptive metric name. Added a comment regarding the need for an alert mechanism.
orderbook/usecase/orderbook_usecase.go Elevated logging level from Info to Error for errors in GetActiveOrders and createLimitOrder. Integrated telemetry counters for tracking specific error occurrences during order fetching and limit order creation.
domain/telemetry.go Added a new metric for monitoring errors in order book pool processing, registered in the init function.
orderbook/telemetry/telemetry.go Implemented telemetry metrics for order management, including counters for errors in retrieving active orders, tick not found, and creating limit orders, all registered for Prometheus monitoring.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant IngestUseCase as Ingest
    participant OrderbookUseCaseImpl as Orderbook
    participant TokensUseCase as Tokens

    Client->>Ingest: Process Orderbook Pool
    Ingest->>Ingest: Increment Error Counter if fails
    Ingest->>Client: Return Status

    Client->>Orderbook: GetActiveOrders()
    Orderbook->>Orderbook: Log Error if fails
    Orderbook->>Client: Return Active Orders

    Client->>Orderbook: createLimitOrder()
    Orderbook->>Tokens: GetSpotPriceScalingFactorByDenom()
    Tokens-->>Orderbook: Return Scaling Factor
    Orderbook->>Orderbook: Calculate Normalized Price
    Orderbook->>Orderbook: Log Error if fails
    Orderbook->>Client: Return Limit Order Status
Loading

Poem

🐇 In the code, we hop and play,
Where order books lead the way.
With alerts and logs, we’ll find our goal,
Normalized prices, making us whole!
So raise a cheer, for changes bright,
In the world of code, we take flight! 🎉


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@p0mvn p0mvn marked this pull request as ready for review August 23, 2024 21:27
Base automatically changed from roman/overflow-price to v25.x August 26, 2024 10:34
@PaddyMc PaddyMc added the A:backport/v26.x backport patches to v26.x branch label Aug 26, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between b63ca05 and 490f4c7.

Files selected for processing (2)
  • ingest/usecase/ingest_usecase.go (1 hunks)
  • orderbook/usecase/orderbook_usecase.go (5 hunks)
Additional comments not posted (4)
orderbook/usecase/orderbook_usecase.go (4)

134-138: LGTM! Consider implementing the alert mechanism.

The elevated error logging enhances visibility. Consider implementing the suggested alert mechanism for better monitoring.


185-189: LGTM! Consider implementing the alert mechanism.

The elevated error logging enhances visibility. Consider implementing the suggested alert mechanism for better monitoring.


232-235: LGTM!

The updated normalization factor calculation and enhanced error handling improve the accuracy and robustness of the method.


302-302: LGTM! Verify the business logic.

The updated calculation method for the normalized price may reflect a change in the intended business logic. Ensure this change aligns with the business requirements.

Please verify that the change in the calculation method aligns with the intended business logic.

@@ -159,7 +163,7 @@ func (o *orderbookUseCaseImpl) GetActiveOrders(ctx context.Context, address stri
if !ok {
o.logger.Info("tick not found", zap.Any("contract", orderbook.ContractAddress), zap.Any("ticks", order.TickId), zap.Any("ok", ok))

// TODO: if tick not found, add an alert
// TODO: (alert) if tick not found, add an alert
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider implementing the alert mechanism.

The TODO comment suggests adding an alert if a tick is not found. Implementing this will enhance monitoring.

Do you want me to generate the alert mechanism code or open a GitHub issue to track this task?

Comment on lines 306 to 308
// TODO: (alert) if failed to process orderbook pool, add an alert
// Prometheus metric counter and alert

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider implementing the alert mechanism.

The TODO comment suggests adding an alert if processing the orderbook pool fails. Implementing this will enhance monitoring.

Do you want me to generate the alert mechanism code or open a GitHub issue to track this task?

@@ -0,0 +1,56 @@
package telemetry
Copy link
Collaborator

@deividaspetraitis deividaspetraitis Aug 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would introducing Orderbook module level telemetry package help to lower conginitive load comparing of global one? I see following benefits:

  • All gauges are scoped for specific module, easier navigate and to maintain
  • Readability: telemetry.GetTickByIDNotFoundCounter.Inc(), no need long names, for example SQSIngestHandlerProcessOrderbookPoolErrorCounter

cc @p0mvn

Introduce Prometheus error counters
@deividaspetraitis deividaspetraitis force-pushed the roman/add-observability-todos branch from 60da066 to 346d95f Compare August 26, 2024 13:25
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
8.3% Coverage on New Code (required ≥ 50%)

See analysis details on SonarCloud

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 490f4c7 and 346d95f.

Files selected for processing (4)
  • domain/telemetry.go (3 hunks)
  • ingest/usecase/ingest_usecase.go (1 hunks)
  • orderbook/telemetry/telemetry.go (1 hunks)
  • orderbook/usecase/orderbook_usecase.go (4 hunks)
Files skipped from review as they are similar to previous changes (1)
  • ingest/usecase/ingest_usecase.go
Additional comments not posted (13)
orderbook/telemetry/telemetry.go (7)

14-14: LGTM!

The metric name is descriptive and follows a consistent naming convention.

The code changes are approved.


19-19: LGTM!

The metric name is descriptive and follows a consistent naming convention.

The code changes are approved.


28-28: LGTM!

The metric name is descriptive and follows a consistent naming convention.

The code changes are approved.


30-35: LGTM!

The counter is correctly defined and initialized.

The code changes are approved.


37-42: LGTM!

The counter is correctly defined and initialized.

The code changes are approved.


44-49: LGTM!

The counter is correctly defined and initialized.

The code changes are approved.


52-55: LGTM!

The init function is correctly registering the counters with Prometheus.

The code changes are approved.

domain/telemetry.go (3)

21-21: LGTM!

The metric name is descriptive and follows a consistent naming convention.

The code changes are approved.


160-165: LGTM!

The counter is correctly defined and initialized.

The code changes are approved.


298-298: LGTM!

The init function is correctly registering the new counter with Prometheus.

The code changes are approved.

orderbook/usecase/orderbook_usecase.go (3)

135-136: LGTM!

The telemetry counter for tracking errors during fetching active orders is correctly integrated. Elevating the logging level to Error enhances visibility.

The code changes are approved.


162-163: LGTM!

The telemetry counter for tracking the number of times a tick is not found by ID is correctly integrated.

The code changes are approved.


181-182: LGTM!

The telemetry counter for tracking errors during creating limit orders is correctly integrated. Elevating the logging level to Error enhances visibility.

The code changes are approved.

@deividaspetraitis deividaspetraitis self-requested a review August 26, 2024 13:51
Copy link
Collaborator

@deividaspetraitis deividaspetraitis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added prometheus gauges.

@deividaspetraitis deividaspetraitis merged commit 4f5b8b8 into v25.x Aug 26, 2024
8 of 9 checks passed
@deividaspetraitis deividaspetraitis deleted the roman/add-observability-todos branch August 26, 2024 15:16
mergify bot pushed a commit that referenced this pull request Aug 26, 2024
* fix: active orders overflow bug

* chore: active orders observability TODOs

* updates

* lint

* chore: active orders observability TODOs

Introduce Prometheus error counters

---------

Co-authored-by: Deividas Petraitis <[email protected]>
(cherry picked from commit 4f5b8b8)
deividaspetraitis pushed a commit that referenced this pull request Aug 27, 2024
* fix: active orders overflow bug

* chore: active orders observability TODOs

* updates

* lint

* chore: active orders observability TODOs

Introduce Prometheus error counters

---------

Co-authored-by: Deividas Petraitis <[email protected]>
(cherry picked from commit 4f5b8b8)

Co-authored-by: Roman <[email protected]>
cryptomatictrader added a commit that referenced this pull request Aug 27, 2024
v25.17.0 tag contains:
f1b7756 fix: no routes for this trade (#480)
e6d44b6 perf: add load test for active orders (#472)
bc570d6 active order fault tolerance via "best-effort" boolean return  (#471)
37f4424 add concurrency to the order book processing in active orders (#470)
4f5b8b8 chore: active orders observability TODOs (#469)
7a28502 chore: add mergify backport on tag action (#473)
b63ca05 fix: active orders overflow bug (#468)
cryptomatictrader added a commit that referenced this pull request Aug 27, 2024
* Update CHANGELOG.md

v25.17.0 tag contains:
f1b7756 fix: no routes for this trade (#480)
e6d44b6 perf: add load test for active orders (#472)
bc570d6 active order fault tolerance via "best-effort" boolean return  (#471)
37f4424 add concurrency to the order book processing in active orders (#470)
4f5b8b8 chore: active orders observability TODOs (#469)
7a28502 chore: add mergify backport on tag action (#473)
b63ca05 fix: active orders overflow bug (#468)

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

---------

Co-authored-by: Roman <[email protected]>
mergify bot pushed a commit that referenced this pull request Aug 28, 2024
* Update CHANGELOG.md

v25.17.0 tag contains:
f1b7756 fix: no routes for this trade (#480)
e6d44b6 perf: add load test for active orders (#472)
bc570d6 active order fault tolerance via "best-effort" boolean return  (#471)
37f4424 add concurrency to the order book processing in active orders (#470)
4f5b8b8 chore: active orders observability TODOs (#469)
7a28502 chore: add mergify backport on tag action (#473)
b63ca05 fix: active orders overflow bug (#468)

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

---------

Co-authored-by: Roman <[email protected]>
(cherry picked from commit 1f6db39)
cryptomatictrader added a commit that referenced this pull request Aug 28, 2024
* Update CHANGELOG.md

v25.17.0 tag contains:
f1b7756 fix: no routes for this trade (#480)
e6d44b6 perf: add load test for active orders (#472)
bc570d6 active order fault tolerance via "best-effort" boolean return  (#471)
37f4424 add concurrency to the order book processing in active orders (#470)
4f5b8b8 chore: active orders observability TODOs (#469)
7a28502 chore: add mergify backport on tag action (#473)
b63ca05 fix: active orders overflow bug (#468)

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

* Update CHANGELOG.md

Co-authored-by: Roman <[email protected]>

---------

Co-authored-by: Roman <[email protected]>
(cherry picked from commit 1f6db39)

Co-authored-by: Calvin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A:backport/v26.x backport patches to v26.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants