Preload libraries and use copy method to provide LibraryStates and respective models

[franzlst]

The library tree should pre-load all state machine in the library tree as LibraryStates and provide copies of those instate that those are loaded from the file-system. Therefore the LibraryState.init method needs to be changed changed into the manner to request a library state (model) from the library manager (model) instate of load the state machine from file and picking the root state from it.

This issue is also dependent to issue #370 because most of the performance of this issue will come from it.

The issue is not simple because the meta data of the root state of a insert library state is adapted to fit into the library state size. So take care check that this adaptation is still performed after the feature implementation.

Further the feature needs threading to not conflict with the user which is trying to use the GUI. Therefore a "worker thread" has to hold the library manager (tree of Library States) up to date.

After talking with @sebastian-brunner we got some kind of a road-map to implement this feature.

• The central element of all LibraryState generation becomes the library manager (or model)
  • all LibraryState (or model) is requesting the manager for a copy of the root state (state_copy) or meta data to load the core- and meta-data only once
• A first implementation of pre-load can be done by first hold a copy of all already loaded libraries
  whereby the complex "worker thread" is not needed
  • this will be the first version that can be merged into the developer and master branch
• the "worker thread" implementation
  • The thread worker has to have a flag to be controlled to be active and non-active (those is connected to a button in the widget)
    • the flag will have a default value taken from the config.yaml file whereby it can be directly deactivated
    • the thread worker also should give feedback on its progress
  • It has to be clear which library the "worker thread" is actually loading
  • every load or copy of a library performed by the worker thread has to performed via the gtk idle add queue and maybe with low priority -- that means recursive loads of libraries are separately insert in the queue
  • the users request of opening libraries while those are still pre-loading has to be prior and be added to the list without creating a conflict (some kind of locking mechanism or the knowledge which element the worker is doing has to be used)

Originally created by @Rbelder at 2017-07-12 12:57:44+00:00 (moved from RMC internal repository)

[franzlst]

I updated the description of the issue with results from a discussion with @sebastian-brunner and created sub-tasks and a queue of a possible implementation order. Please comment, if this can be done better or it is not clear what I meant. Or simply update the description. :)

Originally created by @Rbelder at 2017-07-13 15:46:54+00:00 (moved from RMC internal repository)

[franzlst]

A first implementation of pre-load can be done by first hold a copy of all already loaded libraries
whereby the complex "worker thread" is not needed

I think this is generally the better approach. Why should all libraries be loaded when opening the GUI? Most of them will probably not be needed. In my opinion, a library should be loaded from the file system when it is required the first time. This library is added to a list of loaded libraries, from where copies are taken (already for the first request a copy is taken from the version loaded from the file system).

Originally created by @franzlst ([email protected]) at 2017-07-13 15:57:31+00:00 (moved from RMC internal repository)

[franzlst]

... Why should all libraries be loaded when opening the GUI? Most of them will probably not be needed...

I don't agree here. In ROBEX we had a quite exhaustive coverage of all libraries we ever wrote for the LRU (at least of those which were not purely demo related). Another point is that it would be awesome to have RAFCON loading all libraries directly after launching in order that the programmer does not have to wait when he opens a bigger state machine for the first time.
On the other hand we have to watch RAM usage ...
We should probably discuss that in the next RAFCON meeting.

Originally created by @sebastian-brunner ([email protected]) at 2017-07-13 16:11:21+00:00 (moved from RMC internal repository)

[franzlst]

If we implement it that way, there should at least be an option to deactivate that behavior, as the use cases are quite different (e.g. I have a lot of libraries in the manager, which I don't need most of the time).

Originally created by @franzlst ([email protected]) at 2017-07-13 16:13:48+00:00 (moved from RMC internal repository)

[franzlst]

Yes this would definitely be a good idea.

Originally created by @sebastian-brunner ([email protected]) at 2017-07-13 16:14:53+00:00 (moved from RMC internal repository)

[franzlst]

That it will be optional that is already in mind. When I was with Sebastian we also had in mind to put something like a play and pause button in the library tree widget to give full control to the user. I have updated this in the description, now.

... (e.g. I have a lot of libraries in the manager, which I don't need most of the time) ...

I also do not think it is the best in the world to have by default all sets of libraries in your config.yaml. I think config.yaml files have to be seen as project-files and ROBEX is a huge project but maybe you mount all because of a missing project structure. Would be interesting if the reason of this is connected to missing features in RAFCON.

... Why should all libraries be loaded when opening the GUI? Most of them will probably not be needed...

First of all this is maybe connected to the paragraph above (project oriented config.yaml files).
On the other hand the preloaded libraries will accelerate renaming and propagation of changes in the library tree and give freedom to the user. Most important it is that the RAM is not polluted and the gtk main loop is not disturbed. The main loop is maybe a challenge. In case of RAM I do not have concerns because I think major storage consumption is caused by the widget and graphical viewer but this has to be tested. On the other hand it also could help to only pre-load the core to have the needed overview and option to edit only needed element in case.

Originally created by @Rbelder at 2017-07-13 17:23:47+00:00 (moved from RMC internal repository)