fix order of cxf handlers to enable symmetric tracing around jaxws handler chain

[pellmont]

the current implementation of Start and End around the invocation of a Jax WS is asymmetric around the JAX-WS Handler Chain.

Current behavior:
(execution of incoming MessageHandlers) -> (TracingStartInInterceptor) -> (WebService Invocation) -> (execution of outgoing MessageHandlers) -> (TracingEndInInterceptor)

if I understood the code of this cxf instrumentation correctly, the intent was to build the span close around the WebService Invocation (without Handler Chains).

So the desired behavior would look like this:
(execution of incoming MessageHandlers) -> (TracingStartInInterceptor) -> (WebService Invocation) -> (TracingEndInInterceptor) -> (execution of outgoing MessageHandlers)

Unfortunately CXF is calling the Outgoing Chain inside the POST_INVOKE Phase of Cxf (so the outgoing chain is technically a sub-chain in the incoming chain... which is documented but quite surprising...).

So the solution in the fix at least guarantees the the outgoing chain is invoked AFTER end of tracing. For any extra Interceptors in the POST_INVOKE Phase there is still no guarantee of ordering, but I think this is not a opentelemetry issue but a design-flaw of CXF...

[laurit]

I think we should also modify tests. Perhaps add

    endpoint.getOutInterceptors().add(new AbstractPhaseInterceptor<Message>(Phase.SEND) {
      @Override
      void handleMessage(Message message) throws Fault {
        Context context = Context.current()
        if (LocalRootSpan.fromContext(context) != Span.fromContext(context)) {
          throw new IllegalStateException("handler span should not be ended before outgoing interceptors")
        }
      }
    })

after

opentelemetry-java-instrumentation/instrumentation/jaxws/jaxws-2.0-cxf-3.0/javaagent/src/test/groovy/TestWsServlet.groovy

Line 20 in aceb5b2

endpoint.publish("/HelloService")

[trask]

thanks @pellmont!

can you also update the test as @laurit suggested?

[pellmont]

Thanks @trask and @laurit !