Add Mtom support in WCF runtime

[chris5287]

I am getting the following error when using the Visual Studio WCF Connected Service extension for Visual Studio 2017:

Scaffolding Code ...
Error:Warning: The following Policy Assertions were not Imported:   XPath://wsdl:definitions[@targetNamespace='http://imaging.oracle/']/wsdl:binding[@name='DocumentContentServicePortBinding']
  Assertions:
    <ns1:OptimizedMimeSerialization xmlns:ns1='http://schemas.xmlsoap.org/ws/2004/09/policy/optimizedmimeserialization'>..</ns1:OptimizedMimeSerialization>
Warning: Unsupported binding element type: 'System.ServiceModel.Channels.UnrecognizedAssertionsBindingElement'.
Warning: Unsupported message encoding element type: 'System.ServiceModel.Channels.MtomMessageEncodingBindingElement'. It must be one of the following types: 'System.ServiceModel.Channels.BinaryMessageEncodingBindingElement', 'System.ServiceModel.Channels.TextMessageEncodingBindingElement.'
Warning: Endpoint 'DocumentContentServicePort' at address 'http://localhost/imaging/ws/DocumentContentService' contains one or more bindings not compatible with .Net Core apps, skipping...
Error: No endpoints compatible with .Net Core apps were found.
Failed to generate service reference.

The WSDL/XSD file causing the issue:
DocumentContentService.xsd.txt
DocumentContentService.wsdl.txt

dotnet --info

.NET Command Line Tools (1.0.0)

Product Information:
 Version:            1.0.0
 Commit SHA-1 hash:  e53429feb4

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 RID:         win7-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.0

[hongdai]

Hi Chris, this is because WCF runtime currently does not support Mtom. Potential workaround is try to disable Mtom on the service side if you have access to the service.

If your scenario needs Mtom support, please go ahead open a issue for WCF runtime to consider adding the support.

Thanks,
Hong

[chris5287]

Hi Hong, I'm afraid I do not have control over the server side :(

Where do I need to raise the issue to add Mtom support, or does this issue already cover it?

[hongdai]

I think you can rename this issue to add Mtom support in WCF runtime. We share the same git Repro.

[eser]

Same issue for me.

Warning: Unsupported binding element type: 'System.ServiceModel.Channels.UnrecognizedAssertionsBindingElement'.
Warning: Unsupported message encoding element type: 'System.ServiceModel.Channels.MtomMessageEncodingBindingElement'. It must be one of the following types: 'System.ServiceModel.Channels.BinaryMessageEncodingBindingElement', 'System.ServiceModel.Channels.TextMessageEncodingBindingElement.'
Warning: Endpoint 'ZPM_ITS_ORDER_soap12' at address 'http://tpsappr1.customer.sap.local:8000/sap/bc/srt/rfc/sap/zpm_its_order_detail_ws/400/zpm_its_order/zpm_its_order' contains one or more bindings not compatible with .Net Core apps, skipping...
Warning: Endpoint 'ZPM_ITS_ORDER' at address 'http://tpsappr1.customer.sap.local:8000/sap/bc/srt/rfc/sap/zpm_its_order_detail_ws/400/zpm_its_order/zpm_its_order' contains one or more bindings not compatible with .Net Core apps, skipping...
Error: No endpoints compatible with .Net Core apps were found.

Failed to generate service reference.

[OkuraSt]

same here @eserozvataf

Do you have any workarounds for consuming this type of SAP WS in net core?

Warning: Endpoint 'ZBWS_MAT_STOCKALMACEN' at address 'http://lcs03..local:8000/sap/bc/srt/rfc/sap/zws_mat_stockalmacen/200/zws_mat_stockalmacen/zbws_mat_stockalmacen' contains one or more bindings not compatible with .Net Core apps, skipping...
Error: No endpoints compatible with .Net Core apps were found.

Failed to generate service reference.

[suadev]

Same here :(

[felipecaputo]

Hello folks! Is there any chance to get this implemented?

[zhenlan]

Not in a plan at this point. Please upvote if this is important. It will be helpful for future planning.

[felipecaputo]

I've tried to consume a java SOAP without mtom enabled, and it accepted my request, but the response came with xop/xml.

The content type multipart/related; type="application/xop+xml" ... start-info="text/xml" of the response message does not match the content type of the binding (text/xml; charset=utf-8). If using a custom encoder, be sure that the IsContentTypeSupported method is implemented properly

There is any way I can add a custom decoder to receive this message, while it isn't supported yet?

[Kantuz001]

Same problem(

[NapalmCodes]

Has anyone figured out a work around? I can get the action to occur via the SOAP call but the response is what is failing. Even if I could get the string of the response (which is in the exception message) I could parse what I need out of it as a short term fix.

[felipecaputo]

not yet :(

[dnbr2002]

We could really use some support for this and were hoping with dotnet core 2.0 it might get included. Any chance it's being added in a future release?

[rezvanf]

I have the same problem

I've tried to consume a java SOAP without mtom enabled, and it accepted my request, but the response came with xop/xml.

What i don't understand is at the supported features for .net core 1.0

https://github.com/dotnet/wcf/blob/master/release-notes/SupportedFeatures-v1.0.0.md

It shows message encoding for both text and binary supported.

So how do we do this please if we are consuming WCF services in our .net core apps and need message encoding of WSEncoding.Mtom

[mconnew]

I thought I would give some background on why this isn't a trivial task. This isn't a case of simply copying the code over. On the full framework we use HttpWebRequest for sending an HTTP request. We get the request stream from HttpWebRequest and then write the request message to it.

In .Net Core, we don't have HttpWebRequest available and so need to use HttpClient instead. This has a completely different model for sending a request, you pass it a class derived from HttpContent and HttpClient will request a stream from the HttpContent which it can read from and will subsequently write out to it's internal request stream. This meant that about 75% of the Http transport was basically a rewrite. We encapsulate the request message inside our own HttpContent class and when the request bytes are needed, we use the MessageEncoder to encode the message and then return that. This works well because the channel layer abstraction boundaries are cleanly observed.

The problem with MTOM is that how it's implemented on the full framework the clean api boundaries between the transport and MessageEncoder were trampled on by a herd of angry elk. With the reversal of data flow (HttpClient pulls from HttpContent instead of WCF pushing to HttpWebRequest's request stream), we will need to do a complete rewrite of the MTOM feature and probably refactor the current code so that HttpRequestMessage/HttpResponseMessage are the encapsulation boundary instead of HttpContent.

If I was working on this full time with no other tasks to work on, I would estimate it would take me about 2 months to get it to a preview state. We're aware there are some shortcomings in the full framework implementation such as the inability to precisely control which fields are placed into their own multi-part section and which are serialized inline so any rewrite would also have an eye to making sure we have a path to exposing a clean way to control this as it has been a major pain point for some.

Hopefully this explanation helps you understand why we haven't simply ported the feature at this point.

[felipecaputo]

@mconnew, first of all, thanks for your feedback about this topic. I don't know the framework too well, but with the right direction I would really like to do it, having the right direction to do it.

It seems really a complex implementation but, may more people would like to help if you can spare some time in tutoring or breaking it into small implementations we could do to take advantage of the OpenSource community?

Do you think it could be possible?

[malsoft-pl]

Same problem(

Does anyone know how to write it?

var myBinding = new WSHttpBinding();
myBinding.Security.Mode = SecurityMode.Transport;
myBinding.Security.Transport.ClientCredentialType = HttpClientCredentialType.None;
myBinding.MessageEncoding = WSMessageEncoding.Mtom;

[ghost]

I have this response from a service:

--MIMEBoundary_59f98a3b863640e46b622e5739083846dab4b1825abc60d0
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID:
[email protected]

--MIMEBoundary_59f98a3b863640e46b622e5739083846dab4b1825abc60d0--

<ns2:res xmlns:ns2="http:/xxxxx/v1.0/types">
1162898
2018-03-13T15:08:30.119+01:00
</ns2:res>

How can I convert it easily to an xml? I have the class created from the xsd, but the MTOM header is annoying me.....

[mananchoksi007]

Receives Below Error while calling the post method on WCF service passing xml parameter

0202Unable to unmarshal XML stream. Please verify and resubmit.::Premature end of file.Invalid input

Its successfully working with .Net Framework. Is there different way to post the xml content then .Net framework soap service call?

[lalenzos]

Are there any news regarding ASP.NET Core and SOAP MTOM?

[nalincb]

@andersonaap , yes that was a mistake and I updated code. Cheers.

[ghost]

Is there ETA on this one?
We are trying to do some long term planning and are keen to move to .NET 5, but this is looking like the only impediment. Changing the server is not an option since its a government service that we consume and have no control over.
Issue is three years old and made 'Priority 1' nine months ago, but it looks like there has be no traction in that time. Is Microsoft committing to this piece of work or still undecided?

[RealHabix]

Is there ETA on this one?
We are trying to do some long term planning and are keen to move to .NET 5, but this is looking like the only impediment. Changing the server is not an option since its a government service that we consume and have no control over.
Issue is three years old and made 'Priority 1' nine months ago, but it looks like there has be no traction in that time. Is Microsoft committing to this piece of work or still undecided?

Milestone
6.0

[mconnew]

As @RealHabix pointed out, the plan is to have this work completed for the release which aligns with the .NET 6 release. Note that we still only depend on netstandard 2.0 so we won't have a requirement of .NET 6 to use it. All currently supported versions of .NET/.NET Core support netstandard 2.0 so there shouldn't be forced to update your application to use it. This will continue to be our plan until there's a feature that requires .NET 6.

[NicholasNoise]

6.0 Preview 2 is already here, but XmlDictionaryReader.CreateMtomReader still throws exception.
Any news?

[tclopes]

Any news about supporting MTOM?

[mconnew]

Good news, Mtom support has now been posted as a PR!!

[chempro]

This is great, nice wok
Is there a rough ETA on release date?

[mconnew]

It will be available in the upcoming RC release for .NET 6. Just a reminder, WCF doesn't require .NET 6 to use our latest packages and until we have a reason to do so, our latest packages will continue to work on all supported version of .NET. We don't have a lot of code churn in the WCF client code, it's mostly adding stuff so you can have a high level of confidence in the stability of pre-release versions.

[bcallaghan-et]

I have installed System.ServiceModel.Http version 4.9.0-rc1.21431.2 to experiment with the new MTOM feature, but I cannot instantiate the MtomMessageEncodingBindingElement to use with my CustomBinding. Instead, I get the following compiler error

The type of namespace name 'MtomMessageEncodingBindingElement' could not be found (are you missing a using directive or an assembly reference?)

I have ensured that all of my NuGet packages are up-to-date (all ServiceModel packages are 4.9.0-rc1, clean, rebuild). Using dotPeek, I can open the System.Private.ServiceModel assembly and see the binding element there, in the namespace I expected. I've double-checked the source code to verify that the class is public. What am I missing?

[mconnew]

@bcallaghan-et, apparently I forgot to add the MtomMessageEncodingBindingElement to the reference assembly itself. You can use it by specifying it's usage with any of the various bindings. For example, BasicHttpBinding has a MessageEncoding property that allows you to specify MTOM. If you need a standalone binding element, you should be able to extract an instance from a binding and set the MessageVersion to be what you need. So for example:

var binding = new BasicHttpBinding { MessageEncoding = WSMessageEncoding.Mtom };
MessageEncodingBindingElement mebe = binding.CreateBindingElements().Find<MessageEncodingBindingElement>();
mebe.MessageVersion = MessageVersion.Soap12;

We'll get it added to the reference assembly before the final release.

[bcallaghan-et]

@mconnew Thank you for the workaround. I need to use MTOM with a custom binding, as the remote service requires both Basic auth and client certificates, and I couldn't find a way to do that with the BasicHttpsBinding. This issue isn't as urgent as I thought, so I can wait for the final release.

[NicholasNoise]

But still:

System.PlatformNotSupportedException
The Message Transmission Optimization Mechanism (MTOM) message encoding is not supported on this platform.
   at System.Xml.XmlDictionaryReader.CreateMtomReader(Stream stream, Encoding[] encodings, String contentType, XmlDictionaryReaderQuotas quotas, Int32 maxBufferSize, OnXmlDictionaryReaderClose onClose)
   at System.Xml.XmlDictionaryReader.CreateMtomReader(Stream stream, Encoding encoding, XmlDictionaryReaderQuotas quotas)

How do I enable mtom encoding?

[mconnew]

@NicholasNoise, because the WCF packages work on any supported .NET Core/.NET release and we don't want to force upgrading to .NET 6, the MtomReader implementation has been built in to the WCF packages themselves instead of calling XmlDictionary[Reader|Writer].CreateMtomReader. This also made sense as MTOM is a SOAP only protocol so including it in the runtime and having everyone pay the cost didn't seem like the correct way to do things. This means XmlDictionaryReader.CreateMtomReader will still continue to throw an exception, but you can configure WCF to use MTOM just fine.
What are you doing to trigger that call stack?

[NicholasNoise]

@mconnew I do not use WCF itself, but to parse MTOM responses. So code that has been valid using netfx is no longer the correct way to do things. Is a workaround here for me?

        public static XmlDocument MtomToXml(Stream stream)
        {
            stream.Seek(0, SeekOrigin.Begin);
            using var reader = XmlDictionaryReader.CreateMtomReader(stream, Encoding.UTF8, XmlDictionaryReaderQuotas.Max);
            var doc = new XmlDocument();
            doc.Load(reader);
            return doc;
        }

[NicholasNoise]

@mconnew any news mate?

[mconnew]

Sorry I forgot to respond to your last comment. The easiest way to do this I believe is to use the MtomMessageEncodingBindingElement. This will output a Message object. Depending on how you want to parse things, I don't know how appropriate this is to your use case. Presuming you want to minimize changes elsewhere in the code, this can be converted to an XmlDocument. Here's some example code on how to do this. You might need to tweak it for your scenario a bit if the SOAP/Envelope version is different than the default. Something like this should work for you:

            var mtomBE = new MtomMessageEncodingBindingElement(MessageVersion.Default, Encoding.UTF8);
            var encoderFactory = mtomBE.CreateMessageEncoderFactory();
            var encoder = encoderFactory.Encoder;
            var message = encoder.ReadMessage(stream, int.MaxValue);
            var messageBufferedCopy = message.CreateBufferedCopy(int.MaxValue);
            var writableMessage = messageBufferedCopy.CreateMessage();
            XmlDocument doc = new XmlDocument();
            using (XmlWriter xmlDocWriter = doc.CreateNavigator().AppendChild())
            {
                using (XmlDictionaryWriter writer = XmlDictionaryWriter.CreateDictionaryWriter(xmlDocWriter))
                {
                    writableMessage.WriteMessage(writer);
                }
            }

Let me know if that works for you or if you hit any problems. I couldn't test this code as I don't have an Mtom stream lying around handily to test this so it might need some tweaks.

I'm curious why you are using the MtomReader directly instead of using WCF to do the work? You can use WCF in a contractless way using the IRequestChannel interface to make a request and get a reply using Message objects. Typically you would do this if the request or reply message if a little too free form for a contract definition. Going low level like this is usually a result of not knowing how to get WCF to go a little lower level. XmlDocument isn't as performant as XmlReader (which is what WCF uses to expose the body) and I wonder if there might be a better way to achieve your end goal. I often find taking a step back and looking at the original problem which is needing to be solved can lead to a simpler/cleaner solution.

[JoaoSaDevdotNet]

Someone Fix it? I got the issue Message Transmission Optimization Mechanism (MTOM) message encoding is not supported on this platform every time I try.

[spodskubka]

@JoaoSaDevdotNet what do you need fixed? It's very hard to help you based on that kind of error description.

Do you have an actual code sample? Maybe with a list of referenced NuGet packages and their versions?

Then somebody may be able to tell you what the problem is. 👍

[JoaoSaDevdotNet]

stream.Seek(0, SeekOrigin.Begin);
using var reader = XmlDictionaryReader.CreateMtomReader(stream, Encoding.UTF8, XmlDictionaryReaderQuotas.Max);
var doc = new XmlDocument();
doc.Load(reader);
return doc;

I tried to use your solution but I am getting the error The expected encoding 'utf-16LE' does not match the actual encoding 'utf-16BE'. Do you know what means? I can't be able to found nothing about it.

[JoaoSaDevdotNet]

@JoaoSaDevdotNet what do you need fixed? It's very hard to help you based on that kind of error description.

Do you have an actual code sample? Maybe with a list of referenced NuGet packages and their versions?

Then somebody may be able to tell you what the problem is. 👍

Okay follow the code

using var mtomMessageInMemory = new MemoryStream(xml);
var dataSerializer = new DataContractSerializer(typeof(ReturningType));

Here I get the error I said
var reader = XmlDictionaryReader.CreateMtomReader(mtomMessageInMemory, Encoding.UTF8, lDictionaryReaderQuotas.Max);

return (ReturningType)dataSerializer.ReadObject(reader);

[mconnew]

@JoaoSaDevdotNet , try replacing Encoding.UTF8 with Encoding.BigEndianUnicode

[jhowfaithluby]

Hello @mconnew ,
I tried this solution proposed in the answers: #1810 (comment)
but without success, the same error reading not supported.
I'm using webapi .netcore 6 is there any way to do it without needing a wcf?

[Homaks]

Hello, @mconnew
Simple answer to your question (#1810 (comment)): legacy or some not standard soap services ((
Please make a class XmlMtomReader public. You will make a lot of people happy )