Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(plugin): rework broadcast plugin #732

Merged
merged 9 commits into from
Jun 21, 2021

Conversation

rustatian
Copy link
Member

@rustatian rustatian commented Jun 17, 2021

Reason for This PR

closes: #729
closes: #733
ref: #704

This PR introduces a reworked broadcast plugin, which is now more user friendly. In our previous approach, websockets plugin was an implementation of the broadcast. Now, broadcast features excluded from the websockets plugin and moved into a separate broadcast plugin which any plugin can import (in the plugin's Init func) to receive broadcast messages.

Description of Changes

  • Create a new plugin -> broadcast which can be imported to any plugin to receive broadcast messages.

Beta architecture:
image


Driver registration schema:

image


Configuration sample:

websockets: # <----- one of possible subscribers
  path: /ws
  broker: default # <------ broadcast broker to use --------------- |
                                                                    |  match
broadcast: # <-------- broadcast entry point plugin                 |
  default: # <----------------------------------------------------- |
     driver: redis
     addrs:  # <---------------------------- Local redis section
         - "localhost:6379"
      db: 3 # <-- for example, special ws DB
  test:
     driver: memory

New proto schema (still in BETA):

syntax = "proto3";

package websockets.v1beta;
option go_package = "./;websocketsv1beta";

message Message {
    string command = 1;
    repeated string topics = 2;
    bytes payload = 3;
}

// RPC request with messages
message Request {
    repeated Message messages = 1;
}

// RPC response (false in case of error)
message Response {
    bool ok = 1;
}

LIMITATIONS

When you develop your own plugin, make sure to NOT duplicate Redis connections within 1 broadcast section:
For example, image you have such broadcast config:

broadcast:
  test:
    driver: redis
    addrs:
      - "localhost:6379" # <----------
  test2:
    driver: redis
    addrs:
      - "localhost:6379" # <----------
  test3:
    driver: memory
  test4:
    driver: memory

If you tried to push a message into the test and test2 broadcast sections, the message will be duplicated, because you used the same Redis connection. This is equal to publish twice into the test or test2 connections.
You either should use 1 global Redis section OR local Redis sections connected to different clusters.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the MIT license.

PR Checklist

[Author TODO: Meet these criteria.]
[Reviewer TODO: Verify that these criteria are met. Request changes if not]

  • All commits in this PR are signed (git commit -s).
  • The reason for this PR is clearly provided (issue no. or explanation).
  • The description of changes is clear and encompassing.
  • Any required documentation changes (code and docs) are included in this PR.
  • Any user-facing changes are mentioned in CHANGELOG.md.
  • All added/changed functionality is tested.

Signed-off-by: Valery Piashchynski <[email protected]>
@rustatian rustatian added C-enhancement Category: enhancement. Meaning improvements of current module, transport, etc.. A-plugin Area: module beta-nominated Nominated for backporting to the RR in the beta channel. labels Jun 17, 2021
@rustatian rustatian added this to the 2.3.1 milestone Jun 17, 2021
@rustatian rustatian requested a review from wolfy-j June 17, 2021 16:26
@rustatian rustatian self-assigned this Jun 17, 2021
Signed-off-by: Valery Piashchynski <[email protected]>
Signed-off-by: Valery Piashchynski <[email protected]>
@codecov
Copy link

codecov bot commented Jun 18, 2021

Codecov Report

Merging #732 (2ab22ac) into master (25e0841) will decrease coverage by 0.03%.
The diff coverage is 56.92%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #732      +/-   ##
==========================================
- Coverage   67.61%   67.57%   -0.04%     
==========================================
  Files          93       94       +1     
  Lines        4817     4824       +7     
==========================================
+ Hits         3257     3260       +3     
- Misses       1146     1151       +5     
+ Partials      414      413       -1     
Impacted Files Coverage Δ
plugins/kv/drivers/boltdb/driver.go 52.45% <ø> (ø)
plugins/kv/drivers/boltdb/plugin.go 72.22% <ø> (ø)
plugins/kv/drivers/memcached/driver.go 50.00% <ø> (ø)
plugins/kv/drivers/memcached/plugin.go 58.33% <ø> (ø)
plugins/kv/rpc.go 72.72% <ø> (ø)
plugins/memory/kv.go 60.60% <ø> (ø)
plugins/memory/plugin.go 88.23% <ø> (ø)
plugins/redis/kv.go 60.71% <ø> (ø)
plugins/redis/plugin.go 65.00% <ø> (ø)
proto/kv/v1beta/kv.pb.go 42.60% <ø> (ø)
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 25e0841...2ab22ac. Read the comment docs.

Signed-off-by: Valery Piashchynski <[email protected]>
Signed-off-by: Valery Piashchynski <[email protected]>
Signed-off-by: Valery Piashchynski <[email protected]>
- Rework internal message bus
- Add new tests for the broadcast plugin and include them into the GA

Signed-off-by: Valery Piashchynski <[email protected]>
Signed-off-by: Valery Piashchynski <[email protected]>
@rustatian rustatian marked this pull request as ready for review June 20, 2021 16:35
Signed-off-by: Valery Piashchynski <[email protected]>
@rustatian rustatian merged commit 591b69b into master Jun 21, 2021
@rustatian rustatian deleted the refactor/real_broadcast_plugin branch June 21, 2021 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-plugin Area: module beta-nominated Nominated for backporting to the RR in the beta channel. C-enhancement Category: enhancement. Meaning improvements of current module, transport, etc..
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG, WEBSOCKETS] Messages duplication [WEBSOCKETS] Rework broadcast approach
2 participants