Custom ISerializerExtension is throwing "Recursive dependency detected" exceptions

[gmkado]

I'm trying to resolve my own references to get more flexibility on how I resolve them (i.e. some of the references come from other documents).

I'm having a hard time getting a custom ISerializerExtension working. I was able to get the example case to work but when I tried replacing ISerializer with ISerializer I got a LightInject "Recursive dependency" error.

I thought it might be due to using IContents vs ISerializers (not really sure what the difference is there) but both attempts throw the same error. I'm showing the test with ISerializers because that's what I'm working with now. Any ideas?

sealed class Extension : ISerializerExtension
    {
        public static Extension Default { get; } = new Extension();

        Extension() { }

        public IServiceRepository Get(IServiceRepository parameter) =>
                parameter.Decorate<ISerializers, MySerializerDecorator>();

        void ICommand<IServices>.Execute(IServices parameter) { }

        public class MySerializerDecorator : ISerializers
        {
            private ISerializers _serializers;
            private ISerializer<Subject> _myInjectedSerializer;

            public MySerializerDecorator(ISerializers serializers) {
                _serializers = serializers;
                _myInjectedSerializer = new ResolveReferenceSerializer(serializers.Get(typeof(Subject)).For<Subject>());
            }

            public ISerializer Get(TypeInfo parameter)
            {
                return parameter == typeof(Subject) ? _myInjectedSerializer.Adapt() : _serializers.Get(parameter);
            }
        }

        sealed class ResolveReferenceSerializer : ISerializer<Subject>
        {
            readonly ISerializer<Subject> _previous;
            readonly Dictionary<string, Subject> _store = new Dictionary<string, Subject>();

            public ResolveReferenceSerializer(ISerializer<Subject> previous) => _previous = previous;

            public void Write(IFormatWriter writer, Subject instance)
            {
                _previous.Write(writer, instance);
            }

            public Subject Get(IFormatReader parameter)
            {
                var instance = _previous.Get(parameter);
                if (_store.ContainsKey(instance.Id))
                {
                    instance = _store[instance.Id];
                }
                else
                {
                    _store[instance.Id] = instance;
                }

                return instance;
            }
        }

Heres the test:

        [Test]
        public void ExtensionTest()
        {
            IExtendedXmlSerializer serializer = new ConfigurationContainer()
                                                                    .EnableImplicitTyping(typeof(Subject))
                                                                    .Extend(Extension.Default)
                                                                    .Create();
            const string message = "Hello World!";
            var mysubject = new Subject { Message = message };
            string document = serializer.Serialize(
                new List<Subject>() {
                    mysubject,mysubject
                });

            var subject = serializer.Deserialize<List<Subject>>(document);

            Assert.That(subject[0], Is.EqualTo(subject[1]));
        }

And here's the error:

Message: System.InvalidOperationException : Unable to resolve type: ExtendedXmlSerializer.IExtendedXmlSerializer, service name: 
  ----> System.InvalidOperationException : Unresolved dependency [Target Type: ExtendedXmlSerializer.ExtensionModel.Xml.ExtendedXmlSerializer], [Parameter: serializer(ExtendedXmlSerializer.ExtensionModel.Xml.ISerializer)], [Requested dependency: ServiceType:ExtendedXmlSerializer.ExtensionModel.Xml.ISerializer, ServiceName:]
  ----> System.InvalidOperationException : Unresolved dependency [Target Type: ExtendedXmlSerializer.ExtensionModel.Xml.Serializer], [Parameter: read(ExtendedXmlSerializer.ExtensionModel.Xml.IRead)], [Requested dependency: ServiceType:ExtendedXmlSerializer.ExtensionModel.Xml.IRead, ServiceName:]
  ----> System.InvalidOperationException : Unresolved dependency [Target Type: ExtendedXmlSerializer.ExtensionModel.Xml.Read], [Parameter: serializers(ExtendedXmlSerializer.ContentModel.Content.ISerializers)], [Requested dependency: ServiceType:ExtendedXmlSerializer.ContentModel.Content.ISerializers, ServiceName:]
  ----> System.InvalidOperationException : Unable to resolve type: ExtendedXmlSerializer.ContentModel.Content.IContents, service name: 
  ----> System.InvalidOperationException : Unable to resolve type: ExtendedXmlSerializer.ContentModel.Collections.DefaultCollections, service name: 
  ----> System.InvalidOperationException : Unresolved dependency [Target Type: ExtendedXmlSerializer.ContentModel.Collections.DefaultCollections], [Parameter: serializers(ExtendedXmlSerializer.ContentModel.RuntimeSerializers)], [Requested dependency: ServiceType:ExtendedXmlSerializer.ContentModel.RuntimeSerializers, ServiceName:]
  ----> System.InvalidOperationException : Unresolved dependency [Target Type: ExtendedXmlSerializer.ContentModel.RuntimeSerializers], [Parameter: serializers(ExtendedXmlSerializer.ContentModel.Content.ISerializers)], [Requested dependency: ServiceType:ExtendedXmlSerializer.ContentModel.Content.ISerializers, ServiceName:]
  ----> System.InvalidOperationException : Recursive dependency detected: ServiceType:ExtendedXmlSerializer.ContentModel.Content.ISerializers, ServiceName:]

[issue-label-bot]

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

[Mike-E-angelo]

Ah, yes indeed @gmkado the call to ISerializers.Get while in the constructor will result in a recursion. You want to defer that into the body of the Get as such:

public class MySerializerDecorator : ISerializers
{
	private ISerializers _serializers;

	public MySerializerDecorator(ISerializers serializers) => _serializers = serializers;

	public ISerializer Get(TypeInfo parameter)
	{
		return parameter == typeof(Subject)
			       ? new ResolveReferenceSerializer(_serializers.Get(typeof(Subject)).For<Subject>())
				       .Adapt()
			       : _serializers.Get(parameter);
	}
}

Note that the result of this operation is cached so instantiating a new instance of ResolveReferenceSerializer is a one-time allocation per type which is then stored in a dictionary.

As for ISerializers vs. IContents:

ISerializers is intended for providing serializers that deal with XML document root nodes and list elements, and IContents is used for providing serializers that deal with only the contents of a object.

For some additional context, every object has a "container" context. This could be the root document or the property/field of a parent serializer. ISerializers returns the serializer that emits the entire graph (container + content) whereas IContents returns the serializer that only emits the content.

Note that a "container" is the start + end tags. ISerializers is aware of these whereas IContents is not.

Please let me know if you have any further questions around this and I will do my best to answer them for you. 👍

[gmkado]

Thanks for the clarification, that makes sense regarding the dependency injection, I didn't realize it was cached.

Regarding ISerializers vs IContents, can you give an example of when to use one vs the other? As far as I can tell, I can use DecorateContentsWith + IContents or Decorate + ISerializers and get the same outcome.

[Mike-E-angelo]

Sure @gmkado let me think about this and see if I can get some sample code for you here to delineate the two scenarios.

[Mike-E-angelo]

Alright @gmkado I looked into this some more. You are correct in that the approaches you mention are similar. I would say the following about the two:

• ISerializers returns the "external" or "container" serializer of a type.
• IContents returns the "internal" or "content" serializer of a type.

Worth noting that the serializer returned from ISerializers has a reference to the serializer from IContents. The serializer processes the "container" (or document/element node) of the object, then passes control to the IContents to do its own processing.

So that stated, the ISerializers serializer knows about the IContents serializer, but the IContents serializer does not know about the ISerializers serializer.

For reference, there are about a dozen implementations of ISerializers and nearly 3x more implementations of IContents in the codebase, making IContents the more utilized/popular of the two.

When I think of ISerializers, I think of the primary entry of an object, before anything has been processed. As mentioned in the previous post this is typically -- if not exclusively -- for object/document roots and list elements.

As for examples, I think of RootInstanceExtension.Serializers (link) and CachedSerializers (link). Root instances especially as it is basically capturing a root instance whenever it is passed it for serialization, for use in additional processing down the line.

So to sum up for ISerializers I think it's best to think of "root element" and "list elements". It handles the external "container" of an object and then passes control to the IContents serializer. IContents is for the contents of an object/document/element and is primarily where the magic happens. Most likely, you will want to be working with IContents, but in some cases you will be wanting to work with before that (the "container") so ISerializers is where you would go for that.

Hope that adds some clarification for you. Please let me know if you have any further questions or points that need additional clarification. FWIW I am tagging this post now as documentation for others who are searching for the same thing, so continue to ask away if you feel something needs further clarification. :)

[Mike-E-angelo]

Doh, I guess I said I was going to provide some code, but got distracted with traveling through the codebase and looking at existing implementations. I figured I would use them instead as they do a better job anyways as, you know, working code. 😅 Hopefully those links provided will help assist in providing some further context and meet the intended effect. Please let me know if that's not the case. 👍

[gmkado]

@Mike-E-wins Thanks as always for the quick and detailed response. I'm new to coding in general so digging through the codebase has been very helpful in learning how to write decoupled code.

That being said, I'm finding it a bit tricky to follow whats going on behind the scenes (I guess those go hand in hand). So this bit was particularly helpful for me:

The serializer processes the "container" (or document/element node) of the object, then passes control to the IContents to do its own processing.

Is the process of how IContents and ISerializers pass and execute control documented anywhere? I'm trying to look through the code to see how this works but it would be helpful to have a framework in my mind as I go through it.

[Mike-E-angelo]

Ah so we're actually really in uncharted territory I'm afraid, @gmkado. Earlier in the year (or late last year) we took a pass at improving the documentation and even though we (I) spent a ton of time addressing conceptual concerns I wasn't able to make it into the customization/extension model.

The thinking -- besides there being limited time to spend on such matters -- is that the extension model is considered "advanced" so if someone has any questions around this they can ask away and I will do my best to assist. I guess what I should do is have a better strategy/plan around sharing existing code with good examples to get interested developers better on their way.

Along such lines, one thing I did want to mention is that, to add to the above, the "root" ISerializers is a good place to start to see where control is passed off to IContents:

https://github.com/ExtendedXmlSerializer/home/blob/master/src/ExtendedXmlSerializer/ContentModel/Content/Serializers.cs

You can see that on line #19 the IContents.Get is called. You can also see the Container in action there as well. I think I should have led with that. 😆 But you can see it's very simple and is the "kernel" implementation that is then subsequently wrapped with further decorations.

GitHub

ExtendedXmlSerializer/home

A highly configurable and eXtensible Xml serializer for .NET. - ExtendedXmlSerializer/home

[gmkado]

Thanks @Mike-E-wins . I do feel like the true power of this library comes from the extensibility, so it would be nice if that were a little more accessible. I wonder if you could have a "dumbed down" version of the ISerializerExtension that could decorate a type's serializer without the dependency injection boilerplate. Ideally I just want to write the ISerializer and register it with its type (similar to IConverter and the v1 IExtendedXmlCustomSerializer).

Also, its not clear to me how to use IFormatReader and IFormatWriter, so a section in the documentation about those would be useful. As an example, say I have the following class:

       public class CommandList : ICommand
        {
            public List<ICommand> Commands { get; set; } = new List<ICommand>();
        }

These can be nested so my xml ends up looking something like

<?xml version="1.0" encoding="utf-8"?>
<CommandList xmlns:exs="https://extendedxmlserializer.github.io/v2" xmlns:ns1="clr-namespace:System.Collections.ObjectModel;assembly=System" >  
  <Commands>
    <CommandList >
      <Commands>
        <CommandA />
        <CommandB />
...
      </Commands>
    </CommandList >    
  </Commands>
</CommandList >

Now this is a very simplified model of what I actually have. I have some command lists that are purely lists and some that have other elements. For the commands that are purely lists, I'd like to intercept the serialization and have it serialize something like:

<?xml version="1.0" encoding="utf-8"?>
<CommandList xmlns:exs="https://extendedxmlserializer.github.io/v2" xmlns:ns1="clr-namespace:System.Collections.ObjectModel;assembly=System" >  
    <CommandList >
        <CommandA />
        <CommandB />
...
    </CommandList >    
</CommandList >

I.e. clean it up by taking out the superfluous Commands tags. My first pass in a unit test was to do something like this:

sealed class CommandListLifter : ISerializer<CommandList >
            {
                readonly ISerializer<CommandList > _previous;

                public CommandListLifter(ISerializer<CommandList > previous) => _previous = previous;

                public void Write(IFormatWriter writer, CommandList instance)
                {
                    instance.Commands.ForEach(cmd =>
                    {
                        writer.Start(new MyIdentity()); // <-- how to correctly use IIdentity?
                        // how to get cmd's type serializer and use it here?
                        writer.EndCurrent();
                    });
                }

                public CommandList Get(IFormatReader parameter)
                {        
                    var instance = _previous.Get(parameter);           // <-- how to read it back?
                    return instance; 
                }
            }

As you can see from my comments, its not clear to me how to correctly use IFormatWriter and IFormatReader to get what I need. Thanks again for your help!

[Mike-E-angelo]

Oh man, what you are after here @gmkado is default content, which is something of a sore topic with me, mainly because I have not been able to figure out a way to do it myself. This has been captured in #240. I have given it a go several times now and it's failed due to challenges from the design of the reader. The reader needs a rewrite, pure and simple.

As for IFormatReader and IFormatWriter, you're right that those need some love. I'm honestly not very happy with the documentation, it's very v1 -- but at least there's "something" which was pretty much the intention when going through it at the start of v3. The problem is that it is really like its own project of its own, and takes a lot of time and energy, even when done poorly. 😆

The best I can do in light of #383 is try to help out with some code for you. I will see if I can put aside some time here in the next day or so to get you something that can provide you some better context/knowledge on how the pieces are put together.

[Mike-E-angelo]

Looking into this now @gmkado ... one thing I did notice that I wanted to share is that there's this notion of composing serializer, which you can use to essentially intercept the serializer creation pipeline and then further use to decorate created serializers. Here's some code demonstrating that for you:

https://github.com/ExtendedXmlSerializer/home/blob/master/test/ExtendedXmlSerializer.Tests.ReportedIssues/Issue349Tests.cs

I also like this as it makes use of the IIdentityStore so you can see how identities are defined/utilized.

GitHub

ExtendedXmlSerializer/home

A highly configurable and eXtensible Xml serializer for .NET. - ExtendedXmlSerializer/home

[Mike-E-angelo]

I believe we have addressed this issue as much as possible. Ideally we should have some advanced documentation but my time is fully spent in fixing bugs with this project. If, however, you manage to get comfortable around the topic and have something to add, I would be more than happy to accept a pull request. :)

In any case, please do let me know if there is any further assistance you require on this issue (or any other) and I will do my best to assist.

Closing for now.