Added draft of an article about QoS #36

esteve · 2015-06-27T00:06:30Z

Draft of an article about QoS

Comments and questions are more than welcome.

tfoote · 2015-06-29T20:19:10Z

I think this might be a good place to define our standard set of QoS profiles.

esteve · 2015-06-29T20:23:45Z

@tfoote agreed. I'm adding a few to the proposed implementation and experimenting with them to have a sense of what they may look like. So far I have:

sensor data (last history, depth size 5, best effort)
parameters (last history, depth size 10, reliable)
client/service (last history, depth 10, reliable)
default (all history, depth 10, reliable)

wjwwood · 2015-06-29T20:25:03Z

articles/qos.md

+
+DDS provides fine-grained control over the Quality of Service (QoS) setting for each of the entities involved in the system.
+Common entities whose behavior can be modified via QoS settings include: Topic, DataReader, DataWriter, Publisher and Subscriber.
+QoS is enforced based on a Request vs Offerent Model, however Publications and Subscriptions will only match if the QoS settings are compatible.


Unless I'm mistaken, in English I believe this is Offerer rather than Offerent.

Good catch, thanks!

wjwwood · 2015-06-29T20:31:07Z

What's there looks good to me, but without more it's hard for me to see where this article is going. Maybe it would help me if you could add some more skeleton structure to the document with TODO's to fill it with details like the different proposed built-in QoS defaults.

esteve · 2015-06-29T20:40:54Z

@wjwwood yeah, I just used this PR to structure my thoughts a bit. I'm making the API changes in the following PRs and will reflect them in this document:

ros2/rmw_opensplice#42
ros2/rmw#15
ros2/rmw_connext#42

jacquelinekay · 2015-06-30T18:47:24Z

articles/qos.md

+Both PrismTech OpenSplice and RTI Connext support loading of QoS policies via an external XML file.
+In environments where DDS is already deployed and also to enable more extensibility other than the offered by the ROS 2.0 and the predefined profiles, ROS 2.0 may provide loading of the QoS settings via the same mechanisms the underlying DDS implementations use.
+
+## References


+1 for references!

While reviewing the realtime article, Adolfo encouraged me to cite sources inline. I like the idea of mapping information directly to sources, and I think we should come up with a unified way of doing it (if you want to do it in this article).

I see two options for this:

Provide a link directly in the text

Footnotes, for example: "DDS provides fine-grained control over QoS (1)", with a list of numbers at the bottom of the document, e.g. "1. Gordon A. Hunt, DDS - Advanced Tutorial, Using QoS to Solve Real-World Problems" and maybe with a page number for PDFs, etc.

gbiggs · 2015-08-06T01:31:28Z

articles/qos.md

+
+## ROS 2.0 proposal
+
+Given the complexity of choosing the correct QoS settings for a given scenario, it may make sense for ROS 2.0 to provide a set of predefined QoS profiles for common usecases (e.g. sensor data, real time, etc.), while at the same time give the flexibility to control specific features of the QoS policies for the most common entities.


I like the idea of providing good defaults for different use cases, rather than a global set of defaults expected to be used everywhere. An integrator will be able to say "This topic is carrying range data; I shall use the sensor data QoS set."

BobDean · 2015-08-06T03:07:45Z

What us the reasoning for the chosen depth sizes? And given the history settings, does this mean the dds layer us handling all message queuing, that there will not be a node level "callback queue"?

wjwwood · 2015-08-11T04:15:54Z

@BobDean I think the current defaults are pretty arbitrary, and @esteve is looking into the best defaults for a few cases. DDS is handling all of the queueing, though we might have a ring buffer, whose size is driven by the depth setting, which is used for intra process communications. That's all still up in the air at the moment though.

jacquelinekay · 2015-09-15T18:26:01Z

Now that the QoS API has been designed and iterated over, it seems like a good time to make this design document more specific so that future changes we make are consistent with each other.

Here are a few questions that have been raised about the current prototype during the review process:

What are the default QoS profiles for pub/sub and services and what is the justification for choosing these values?
What is the behavior for overriding system defaults for certain QoS profiles? Should it be uniform for all entities? For example, in rmw_connext, the behavior for the depth setting for Publishers and Subscriptions is confusing.
Should the QoS settings passed to DDS also affect the client library layer (e.g. depth determines the size of ring buffer for intra-process communication)?

esteve · 2015-09-15T18:28:13Z

@jacquelinekay thanks for summarizing the discussion, I'll update the document to address these questions.

esteve · 2015-09-25T18:27:07Z

I've added a section about QoS profiles. I don't know how intraprocess should honor (or if it should) the QoS settings, so I added that as an open question.

Does anyone think it'd worth adding a terminology section explaining every QoS setting implemented in the current API? I think it'd be useful, but I worried this would make the article too long.

tfoote · 2015-09-25T18:34:28Z

Does anyone think it'd worth adding a terminology section explaining every QoS setting implemented in the current API? I think it'd be useful, but I worried this would make the article too long.

Terminology might be good, but I'd suggest linking to a DDS documentation of QoS definitions, just clearly state which ones we're supporting maybe?

wjwwood · 2015-09-25T18:36:41Z

I actually think this article should stand on it's own and that readers should not have to go to a set of DDS documentation to understand what QoS settings ROS is providing and what they do.

esteve · 2015-09-25T18:41:32Z

I agree with @wjwwood , I prefer documentation that doesn't force me to jump to external resource for core topics. I can add links to the QoS section of the DDS spec in case the readers want to know more.

esteve · 2015-09-25T20:25:43Z

I updated the document removing the QoS profile and instead explained it can store and how the values affect its behavior.

dirk-thomas · 2015-09-25T22:03:09Z

articles/qos.md

+  * Reliable: guarantee that samples are delivered, may retry multiple times.
+* Durability.
+  * Transient local: *only applies to `DataWriter`s*. `DataWriter` becomes responsible of persisting samples until a corresponding `DataReader` becomes available.
+  * Volatile: no attempt is made to persist samples.


For each of the main bullets there is also the option of "system default".

Added note about this. Thanks.

esteve · 2015-09-28T20:16:03Z

I've rebased this branch and squashed the commits. Is there anything that needs improving?

dirk-thomas · 2015-09-28T20:18:17Z

articles/qos.md

+
+* Default QoS settings for publishers and subscriptions.
+In order to make the transition from ROS1 to ROS2, exercising a similar network behavior is desirable.
+By default, publishers and subscriptions are reliable in ROS2.


Should we add note here that this default value won't work for larger messages for all rmw implementations atm?

Good point, will update the document to reflect this.

esteve · 2015-09-29T16:54:48Z

Addressed @dirk-thomas 's latest feedback, squashed again and pushed.

dirk-thomas · 2015-09-29T17:09:11Z

articles/qos.md

+* Default QoS settings for publishers and subscriptions.
+In order to make the transition from ROS1 to ROS2, exercising a similar network behavior is desirable.
+By default, publishers and subscriptions are reliable in ROS2.
+However, many DDS vendors don't support publishing large messages (e.g. images) with reliable connections.


The sentence is a bit misleading. Connext does support publishing large messages with reliable connections. But we have to use a different API for that.

wjwwood · 2015-10-07T21:29:58Z

Todo:

@esteve to address one more comment
Link to each current profile with a justification and a note that these are subject to suggestion and feedback.

esteve · 2015-10-08T00:55:25Z

Updated the article with the most recent feedback. I'm going to squash and merge this PR soon unless there are more comments.

Added article about QoS

dirk-thomas · 2015-10-08T01:04:10Z

Currently the article does not mention the parameter related profiles. Is that intentional? If not, I think especially the depth of 1000 could use some rational.

dirk-thomas · 2015-10-22T14:06:32Z

@esteve Can you please comment on the previous question.

esteve · 2015-10-22T20:44:53Z

The parameters profile is not mentioned in the article because we still haven't officially released the support for parameters. In any case, the value in there is purely arbitrary, the parameters profile has a priori the same rationale as the services one, so it may make sense to use the same values.

dirk-thomas · 2015-10-25T02:12:33Z

Since the parameters are scheduled for the upcoming alpha next week I think the article should be updated to cover the QoS setting for services (and parameters).

esteve added in progress Actively being worked on (Kanban column) in review Waiting for review (Kanban column) and removed in progress Actively being worked on (Kanban column) labels Jun 27, 2015

esteve self-assigned this Jun 29, 2015

wjwwood reviewed Jun 29, 2015
View reviewed changes

jacquelinekay reviewed Jun 30, 2015
View reviewed changes

jacquelinekay mentioned this pull request Jun 30, 2015

The specified queue size/depth in rmw interface... ros2/rmw#13

Open

jacquelinekay mentioned this pull request Jul 9, 2015

Realtime testing RTI Connext example osrf/ros_dds#28

Merged

dirk-thomas added in progress Actively being worked on (Kanban column) ready Work is about to start (Kanban column) and removed in review Waiting for review (Kanban column) in progress Actively being worked on (Kanban column) labels Jul 28, 2015

dirk-thomas mentioned this pull request Aug 4, 2015

Add additional QoS settings to Pub/Sub API ros2/ros2#66

Closed

gbiggs reviewed Aug 6, 2015
View reviewed changes

dirk-thomas mentioned this pull request Aug 19, 2015

Add system default QoS profile ros2/ros2#93

Closed

wjwwood closed this Sep 1, 2015

wjwwood removed the ready Work is about to start (Kanban column) label Sep 1, 2015

wjwwood reopened this Sep 1, 2015

wjwwood added in progress Actively being worked on (Kanban column) ready Work is about to start (Kanban column) and removed in progress Actively being worked on (Kanban column) labels Sep 1, 2015

esteve removed the ready Work is about to start (Kanban column) label Sep 22, 2015

esteve added the in progress Actively being worked on (Kanban column) label Sep 22, 2015

dirk-thomas reviewed Sep 25, 2015
View reviewed changes

esteve added in review Waiting for review (Kanban column) and removed in progress Actively being worked on (Kanban column) labels Sep 28, 2015

esteve force-pushed the qos branch 2 times, most recently from a7fd04f to ce36d2f Compare September 28, 2015 20:15

dirk-thomas reviewed Sep 28, 2015
View reviewed changes

esteve force-pushed the qos branch from 8de0cc3 to 55786d6 Compare September 29, 2015 16:53

dirk-thomas reviewed Sep 29, 2015
View reviewed changes

esteve force-pushed the qos branch from 55786d6 to e228c20 Compare October 7, 2015 23:27

Added article about QoS

acf480c

esteve force-pushed the qos branch from 2874011 to acf480c Compare October 8, 2015 01:01

esteve added a commit that referenced this pull request Oct 8, 2015

Merge pull request #36 from ros2/qos

f0119e0

Added article about QoS

esteve merged commit f0119e0 into gh-pages Oct 8, 2015

esteve removed the in review Waiting for review (Kanban column) label Oct 8, 2015

esteve deleted the qos branch October 8, 2015 01:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added draft of an article about QoS #36

Added draft of an article about QoS #36

esteve commented Jun 27, 2015

tfoote commented Jun 29, 2015

esteve commented Jun 29, 2015

wjwwood Jun 29, 2015

esteve Jun 29, 2015

wjwwood commented Jun 29, 2015

esteve commented Jun 29, 2015

jacquelinekay Jun 30, 2015

gbiggs Aug 6, 2015

BobDean commented Aug 6, 2015

wjwwood commented Aug 11, 2015

jacquelinekay commented Sep 15, 2015

esteve commented Sep 15, 2015

esteve commented Sep 25, 2015

tfoote commented Sep 25, 2015

wjwwood commented Sep 25, 2015

esteve commented Sep 25, 2015

esteve commented Sep 25, 2015

dirk-thomas Sep 25, 2015

esteve Sep 25, 2015

esteve commented Sep 28, 2015

dirk-thomas Sep 28, 2015

esteve Sep 28, 2015

esteve commented Sep 29, 2015

dirk-thomas Sep 29, 2015

wjwwood commented Oct 7, 2015

esteve commented Oct 8, 2015

dirk-thomas commented Oct 8, 2015

dirk-thomas commented Oct 22, 2015

esteve commented Oct 22, 2015

dirk-thomas commented Oct 25, 2015


		## ROS 2.0 proposal

		Given the complexity of choosing the correct QoS settings for a given scenario, it may make sense for ROS 2.0 to provide a set of predefined QoS profiles for common usecases (e.g. sensor data, real time, etc.), while at the same time give the flexibility to control specific features of the QoS policies for the most common entities.

Added draft of an article about QoS #36

Added draft of an article about QoS #36

Conversation

esteve commented Jun 27, 2015

tfoote commented Jun 29, 2015

esteve commented Jun 29, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wjwwood commented Jun 29, 2015

esteve commented Jun 29, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BobDean commented Aug 6, 2015

wjwwood commented Aug 11, 2015

jacquelinekay commented Sep 15, 2015

esteve commented Sep 15, 2015

esteve commented Sep 25, 2015

tfoote commented Sep 25, 2015

wjwwood commented Sep 25, 2015

esteve commented Sep 25, 2015

esteve commented Sep 25, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

esteve commented Sep 28, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

esteve commented Sep 29, 2015

Choose a reason for hiding this comment

wjwwood commented Oct 7, 2015

esteve commented Oct 8, 2015

dirk-thomas commented Oct 8, 2015

dirk-thomas commented Oct 22, 2015

esteve commented Oct 22, 2015

dirk-thomas commented Oct 25, 2015