Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add queue subscriber and unit test #388

Merged
merged 1 commit into from
Apr 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion async-nats/examples/sub.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
use bytes::Bytes;
use futures_util::StreamExt;
use std::error::Error;
use std::time::Instant;
Expand Down
41 changes: 37 additions & 4 deletions async-nats/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ pub enum ClientOp {
Subscribe {
sid: u64,
subject: String,
queue_group: Option<String>,
},
Unsubscribe {
id: u64,
Expand Down Expand Up @@ -455,9 +456,18 @@ impl Connection {
self.stream.write_all(b"\r\n").await?;
}

ClientOp::Subscribe { sid, subject } => {
ClientOp::Subscribe {
sid,
subject,
queue_group,
} => {
self.stream.write_all(b"SUB ").await?;
self.stream.write_all(subject.as_bytes()).await?;
if let Some(queue_group) = queue_group {
self.stream
.write_all(format!(" {}", queue_group).as_bytes())
.await?;
}
self.stream
.write_all(format!(" {}\r\n", sid).as_bytes())
.await?;
Expand Down Expand Up @@ -725,15 +735,38 @@ impl Client {
format!("_INBOX.{}", nuid::next())
}

pub async fn queue_subscribe(
&mut self,
subject: String,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to stray from your existing conventions, but it would be slightly more ergonomic for the signature of subscribe to be
fn subscribe<T:ToString>(&self, subject: T)
and for queue_subscribe to be
fn queue_subscribe<T1:ToString,T2:ToString>(&self, subject: T1, queue_group: T2)

Or, if they are to be unified into a single subscribe function, then
fn subscribe<T1:ToString,T2:ToString>(&self, subject: T1, queue_group: Option<T2>)

If you're ok with either of those, I can also implement it on the next commit.

Copy link
Member

@Jarema Jarema Apr 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ToString allocates, even if you pass String.

Into<String> is a better option here, though we decided to NOT have that right now, as we might add Subject type later, which would break API.

We refrained from having those helpers to allow ourselves to decide later.
Both @caspervonb and I want to be very clear in the codebase - especially in publish which can happen thousands of times per second - what is allocated etc.

Copy link
Collaborator

@caspervonb caspervonb Apr 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hesitant towards doing this because the ToString implementation for String is a deep clone, it could be Into<String> which should be a move but personally I'm more for explicit conversions than implicit ones as the caller is in control of how the conversion happens.

We're considering if we want a (potentially interned) type for Subjects, header names will likely get one in the form of HeaderNames from the http crate's API.

I'd say hold off on this for now, at-least until the subject-type matter is resolved.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Short notice: So far #222 is ready for review and completely integrated the Subject type and a AsSubject trait into the old code base. It may went unnoticed with all the async rewriting process.

queue_group: String,
) -> Result<Subscriber, io::Error> {
self._subscribe(subject, Some(queue_group)).await
}

pub async fn subscribe(&mut self, subject: String) -> Result<Subscriber, io::Error> {
self._subscribe(subject, None).await
}

// TODO: options/questions for nats team:
// - should there just be a single subscribe() function (would be breaking api against 0.11.0)
// - if queue_subscribe is a separate function, how do you want to name the private function here?
async fn _subscribe(
&mut self,
subject: String,
queue_group: Option<String>,
) -> Result<Subscriber, io::Error> {
let (sender, receiver) = mpsc::channel(16);

// Aiming to make this the only lock (aside from internal locks in channels).
let mut context = self.subscription_context.lock().await;
let sid = context.insert(Subscription { sender });

self.sender
.send(ClientOp::Subscribe { sid, subject })
.send(ClientOp::Subscribe {
sid,
subject,
queue_group,
})
.await
.unwrap();

Expand All @@ -749,9 +782,9 @@ impl Client {
}
}

/// Connets to the NATS with specified options.
/// Connects to NATS with specified options.
///
/// It is generally advised to use [ConnectOptions] instead, as it provides builder for whole
/// It is generally advised to use [ConnectOptions] instead, as it provides a builder for whole
/// configuration.
///
/// # Examples
Expand Down
49 changes: 48 additions & 1 deletion async-nats/tests/client_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ mod client {

use super::nats_server;
use bytes::Bytes;
use futures_util::StreamExt;
use futures_util::{future::join_all, StreamExt};

#[tokio::test]
async fn basic_pub_sub() {
Expand Down Expand Up @@ -46,6 +46,53 @@ mod client {
assert_eq!(i, 10);
}

#[tokio::test]
async fn queue_sub() {
let server = nats_server::run_basic_server();
const NUM_SUBSCRIBERS: usize = 3;
const NUM_ITEMS: usize = 20;

let mut subscribers = Vec::new();
let mut client = async_nats::connect(server.client_url()).await.unwrap();
for _i in 0..NUM_SUBSCRIBERS {
subscribers.push(
client
.queue_subscribe("qfoo".into(), "group".into())
.await
.unwrap(),
);
}

for _ in 0..NUM_ITEMS {
client.publish("qfoo".into(), "data".into()).await.unwrap();
}
client.flush().await.unwrap();
let mut results = Vec::new();
for mut subscriber in subscribers.into_iter() {
results.push(tokio::spawn(async move {
let mut count = 0u32;
while let Ok(Some(item)) = tokio::time::timeout(
tokio::time::Duration::from_millis(1000),
subscriber.next(),
)
.await
{
count += 1;
}
count
}));
}
let counts = join_all(results.iter_mut())
.await
.into_iter()
.filter_map(|n| n.ok())
.collect::<Vec<u32>>();
let total: u32 = counts.iter().sum();
assert_eq!(total, NUM_ITEMS as u32, "all items received");
let num_receivers = counts.into_iter().filter(|n| *n > 0u32).count();
assert!(num_receivers > 1, "should not all go to single subscriber");
}

#[tokio::test]
async fn cloned_client() {
let server = nats_server::run_basic_server();
Expand Down