Sending dynamic data types

[elfenpiff]

Brief feature description

Dynamic containers like a std::vector, std::map, and std::list should be transmittable via shared memory without data copy.

Data type wish list

• std::vector
• std::map

Requirements

• The data type should be able to grow dynamically and its size should not be determined on creation.

A partial solution for this issue will come with issue #911

At the moment the conclusion is that dynamic STL containers will most likely never be supported by iceoryx in a zero copy way.

[MatthiasKillat]

An important consideration for data types to be send need to satisfy certain conditions:

1. They need to be entirely contained in Shared Memory - no pointers to outside memory, no dynamic allocators.
2. The lifetime of the Shared Memory needs to be ensured (the publish subscribe API does this for example).
3. The data structure cannot use pointers internally to reference its data (i.e. like naively implemented lists or trees do).

Data structures need to satisfy some kind of relocatability property, meaning that it could theoretically be copied bitwise to a new location and still work properly (like the original). As an alternative view, if all memory addresses are shifted by some constant offset it needs to also work properly. Trivially-copyable objects satisfy this property but the class of relocatable types is actually larger.

Note: the untyped C++ or C API can already be seen like some Shared Memory allocator. This can be used to create custom allocators. Furthermore custom allocators in template parameters have the unfortunate problem that they lead to different types for different allocators.

1. cxx::vector does not have this property yet, but it should be rather easy to create a version which uses a relocatable data pointer which could be send via iceoryx.

2. A proper map implementation which gives performance guarantees (essentially O(log(n)) complexity for most operations) is rather involved. It would also have to work with relocatable pointers or indices internally (instead of regular pointers).

[Indra5196]

@MatthiasKillat I guess the problem mentioned in "Note" can be resolved using std::pmr containers of C++17

[Indra5196]

@elfenpiff @MatthiasKillat Had a doubt, will the size of a dynamic vector implementation be confined to the allocated chunk size? or can it be like allocating a larger chunk in case the vector grows more than chunk size?

[elfenpiff]

@elfenpiff @MatthiasKillat Had a doubt, will the size of a dynamic vector implementation be confined to the allocated chunk size? or can it be like allocating a larger chunk in case the vector grows more than chunk size?

@Indra5196
We could implement the shared memory allocator in a way that we acquire a new chunk whenever the container requires more space. The chunk size would be then the smallest amount the vector grows.

I think that the STL containers are using a strategy in which the size increases with a power of two and we would then have a static size - but this could also become dynamic.

In the end, we have a lot of options here and we would not stick to one chunk size. But there are still hurdles on the way since containers like the vector are requiring consecutive memory. When you acquire an additional chunk you do not have the guarantee that it proceeds from the previous one - but this is also the case when you call realloc in C. Whenever this happens you have to copy the whole container to a new bigger chunk or to two consecutive chunks like in the STL implementation.

[elfenpiff]

@Indra5196 I added a requirement in this issue for the datatypes so that your request/question is on record.

[Indra5196]

@elfenpiff Storing vector in consecutive chunks..........can't even imagine how difficult/expensive it would be to manage such a vector

[elfenpiff]

@Indra5196 it is also possible that we follow a different approach and write a shared memory allocator. The consecutive chunks thing was just one quick idea.
But you are right, this will be a difficult challenge but in the end, it shouldn't be more expensive than an STL vector.

[Indra5196]

@elfenpiff So in order to do that, we might need to modify the ChunkHeader or make the SharedChunk take multiple resources

[elfenpiff]

@Indra5196 would you like to work on that feature?

Maybe the cleanest way for handling this would be to write a shared memory allocator to which the publisher has access to. Then you would create your std::vector with this allocator and send it.

static SharedMemoryManager g_sharedMemoryManager;

template<typename T>
struct SharedMemoryAllocator {
  T * allocate(std::size_t n) {
     return g_sharedMemory.allocate(n);
  }

  void deallocate(T * ptr, std::size_t n) {
      g_sharedMemory.free(ptr, n);
  }
}

struct TopicData {
  int blubb;
  std::vector<int, SharedMemoryAllocator<int>> intVector;
};

int main() {
    iox::runtime::PoshRuntime::initRuntime("myApp");
    g_sharedMemoryManager = iox::runtime::PoshRuntime::getRuntime().getSharedMemoryManager();

    iox::popo::Publisher<TopicData> publisher({"Service", "Description", "Foo"});
    auto sample = publisher.loan();
    if ( sample ) {
        sample.intVector.emplace_back(123);
        sample.publish();
    }
}

This is how an example could look like. The SharedMemoryAllocator would be defined in a separate header and the line
g_sharedMemoryManager = iox::runtime::PoshRuntime::getRuntime().getSharedMemoryManager(); would be performed in initRuntime.

But the real challenge is then to implement the SharedMemoryManager (I couldn't find a better name) which would require some kind of heap like memory implementation. This class would be constructed directly into the shared memory and the user has to provide only the size of this whole class. For instance 4GB and then you can use those 4GB with the custom allocator.

At the moment we are all working on the 1.0 release but after that is done we can talk about the concept on a blackboard, exchange ideas, create a design draft and after it is merged we could start implementing it.

@Indra5196 if you had a free wish, till when would you like to have this feature? And would the pseudo code draft fit your use case?
@budrus When would we have time to realize something like this?

[elBoberido]

@elfenpiff it's me, the party pooper ;). You cannot reliably send STL container by the shared memory. Even if you have this custom allocator the usage is quite restricted and the only possible use case would be a publisher specialized for a given container like Publisher<std::vector> but not a struct TopicData { std::vector }; as data type since the container might use pointer internally which horribly breaks if the shared memory is not mapped to the same address, which we do not enforce. Even with that, you reliably need to get to the underlying storage in order to just send the memory chunk.

Furthermore, the memory chunks need to be taken from the MemPools since that is the only memory which all subscriber know about. Creating a new shared memory would make it necessary to inform all the subscriber that they need to map an additional shared memory into their address space.

One viable option might be to have an allocator based on a publisher and each time the container tries to allocate new memory, the publisher would loan a new memory chunk. There is still the problem to reliably access the loaned memory.

[elfenpiff]

@elBoberido bad party pooper! But you are right I totally forgot the relocatability! So we can throw the previous idea right out of the window.

[Indra5196]

@Indra5196 would you like to work on that feature?

Maybe the cleanest way for handling this would be to write a shared memory allocator to which the publisher has access to. Then you would create your std::vector with this allocator and send it.

static SharedMemoryManager g_sharedMemoryManager;

template<typename T>
struct SharedMemoryAllocator {
  T * allocate(std::size_t n) {
     return g_sharedMemory.allocate(n);
  }

  void deallocate(T * ptr, std::size_t n) {
      g_sharedMemory.free(ptr, n);
  }
}

struct TopicData {
  int blubb;
  std::vector<int, SharedMemoryAllocator<int>> intVector;
};

int main() {
    iox::runtime::PoshRuntime::initRuntime("myApp");
    g_sharedMemoryManager = iox::runtime::PoshRuntime::getRuntime().getSharedMemoryManager();

    iox::popo::Publisher<TopicData> publisher({"Service", "Description", "Foo"});
    auto sample = publisher.loan();
    if ( sample ) {
        sample.intVector.emplace_back(123);
        sample.publish();
    }
}

This is how an example could look like. The SharedMemoryAllocator would be defined in a separate header and the line
g_sharedMemoryManager = iox::runtime::PoshRuntime::getRuntime().getSharedMemoryManager(); would be performed in initRuntime.

But the real challenge is then to implement the SharedMemoryManager (I couldn't find a better name) which would require some kind of heap like memory implementation. This class would be constructed directly into the shared memory and the user has to provide only the size of this whole class. For instance 4GB and then you can use those 4GB with the custom allocator.

At the moment we are all working on the 1.0 release but after that is done we can talk about the concept on a blackboard, exchange ideas, create a design draft and after it is merged we could start implementing it.

@Indra5196 if you had a free wish, till when would you like to have this feature? And would the pseudo code draft fit your use case?
@budrus When would we have time to realize something like this?

@elfenpiff I would love to contribute to this :-). We can surely have a discussion on this whenever feasable. If you are asking for a free will, I wish for today ;-), but realistically, it will be great if we are able to do it within April.

[elfenpiff]

@elfenpiff I would love to contribute to this :-).

@Indra5196 this sounds great and we should coordinate this on this week's developer meet-up. But implementing this in April seems unrealistic I think we need a lot more time for this since there is a lot of infrastructure to build behind the scenes.

But as a short term solution, you could use the iceoryx alternative cxx::vector, cxx::string, cxx::list and to ease the handling we could provide conversation operations so that you can work with them easily (not yet available).

Then code like this would be possible:

struct TopicData {
  cxx::vector<int, 100> vec;
  cxx::string<100> str;
};
void send(std::vector<int> myVector, std::string myString) {
   myPublisher.loan().and_then([&](auto & sample) {
      sample.vec = myVector; // converted to cxx::vector by copy
      sample.str = myString; // converted to cxx::string by copy
   });
}
void receive() {
  std::vector<int> myVector;
  std::string myString;

  mySubscriber.take().and_then([&](auto & sample) {
      myVector = sample.vec; // convert back to std::vector by another copy
      myString = sample.str; // convert back to std::string by another copy
  });
}

Those conversation operators could be implemented in April!

[mossmaurice]

@elfenpiff Can we close this in favour of #911?

[elfenpiff]

@mossmaurice

Can we close this in favour of #911?

The dynamic type issue will not solve this issue completely since here we explicitly would like to send dynamic STL types which are using the heap.

[budrus]

Closing this as #911 will be a more generic solution