exchangeSplit option unclear and undocumented.

[petersilva]

I don't remember what exchangeSplit does. I think the idea is that you have, say, a winnow with post_exchangeSplit making a bunch of exchanges and sending to them with sorting based on modulo of check sums. the complementary option would be something where you declare a subscriber, where each instance subscribes to a corresponding exchange

in winnow config:

winnow
    post_exchange xs_foo
    post_exchange_split 4

  ---> declares exchanges  xs_foo00, xs_foo01, xs_foo02, xs_foo03

in subscriber config

  instances 4
   exchange xs_foo
   exchangeSplit yes

  subtopic A

should result in different bindings for each instance:

instance 1 should bind it's queue to topic A of exchange xs_foo00

instance 2 should bind its queue to topic A of exchange xs_foo01

instance 3 should bind it's queue to topic A of exchange xs_foo02

...

So... I think that is what it does and is for. It makes it easier to declare subscribers to a queue.

Looking at the repository:

• it looks, at least partially maybe fully implemented.
• there is no documentation of it.

The goal of this issue is to test it out, and make a decision about whether to

a) finish it, and write some documentation, or
b) remove it.

[reidsunderland]

I tried to use exchangeSplit:

1. exchangeSplit is defined in both count_options and a flag_options in config.py. sr3 show shows True/False, so it's treating it as a flag option.

   If we keep it as a flag option, then the user needs to define the number of instances.

2. instances > 1 have issues with writing/setting/reading the qname:

   2024-05-13 20:24:18,972 [CRITICAL] 1600448 sarracenia.config _resolveQueueName failed to read queue name from /net/local/home/sunderlandr/.cache/sr3/subscribe/configname/subscribe.configname.bunnymaster02.qname
   

   The subscribe.configname.bunnymaster01.qname gets created, but the others don't.

3. It doesn't seem like the exchange name is getting set correctly:

   # From instance 1, the exchange in the config is xs_TESTEXCH_, so it should be using xs_TESTEXCH_00
   
   2024-05-13 20:24:20,885 [ERROR] sarracenia.moth.amqp getSetup AMQP getSetup failed to localhost with Queue.bind: (404)     NOT_FOUND - no exchange 'xs_TESTEXCH_' in vhost '/'
   

   sr3 show does show all the exchanges though:

    'exchange': ['xs_TESTEXCH_00', 'xs_TESTEXCH_01', 'xs_TESTEXCH_02',
                 'xs_TESTEXCH_03', 'xs_TESTEXCH_04'],
   

[petersilva]

Right... so if each instance is subscribing to e different exchange, then each instance needs a different queue. so we need to add an instance number to the queue_name somehow.
then each instance's binding swill be between one queue and one exchange.

[petersilva]

Is it creating multiple queues? or maybe that's the part that is missing?

[petersilva]

Another thing... instances modulo exchangeSplit...

instances 10
exchangeSplit 20

means create 10 queues, each one subscribed to two exchanges.

[petersilva]

There is also the question of cluster awareness. if you have multiple nodes sharing a load, then you need one instance per exchange... regardless of the number of nodes. So this introduces the need for cluster topology awareness.

see #1297 (comment)