Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FISH-5636 Remote EJB Tracing Improvements #5410

Merged
merged 11 commits into from
Sep 24, 2021

Conversation

Pandrex247
Copy link
Member

@Pandrex247 Pandrex247 commented Sep 14, 2021

Description

An attempt to improve the throughput of the OpenTracing service, particularly with regards to remote EJBs.
The main change is the un-synchronising of the getTracer method - creation of tracers is still synchronised but is now split off into another method and utilises double-checked locking to prevent conflicts. While double-checked locking can be naughty, in this case it's being done on a map itself, so the issue of partially constructed references not returning null Shouldn't™ occur.

My results from performance testing were largely inconclusive since the realm of variance is pretty large - this is working at the micro and millisecond timeframe, any slight deviation outside of my control (e.g. OS runs a background process) will affect the result. In any case, I shall list the reasoning of how each change should help below.

  • getTracer split into two methods: getTracer and createTracer
    • getTracer is now unsynchronised, meaning there should be less contention waiting for locks in normal operation since only createTracer is synchronised (tracers only get created once)
  • StandardWrapper no longer does two calls for getTracer back-to-back (less of an impact now that getTracer is no longer synced but still saves performing the get operation on a ConcurrentHashMap twice)
  • Lookup of RequestTracingService moved from SpanBuilder constructor to OpenTracingService (and then passed to Tracer as a constructor parameter). The intent here is to make it so an HK2 lookup doesn't need to be performed each time a span is created, and is instead done once.

This PR also includes some cleanup of deprecated or ugly code:

  • MockTracer removed from code base - this is now handled using service loader
    • Supplemental - I noticed the export for log was missing and so I added it in
  • GlobalTracer.register(Tracer) is deprecated, so has been replaced with GlobalTracer.registerIfAbsent(Callable)
    • A lot of the null checks being done here were redundant, since they are handled by the factory. In any case, a try-catch has been added to the factory around the create method since that can (but Shouldn't™) be thrown by GlobalTracer.registerIfAbsent(Callable)
  • Removed Events from constructor of RequestTraceStoreFactory - not used.
  • DAY variable removed from RequestTracingService - not used.
  • Static ScopeManager moved from OpenTracingService to Tracer - it isn't used anywhere else, and our Tracer impl doesn't use any other kind of ScopeManager

Important Info

Dependant PRs

MicroProfile OpenTracing TCK Runner update: payara/MicroProfile-TCK-Runners#153

Blockers

None

Testing

New tests

None - with the level of sensitivity being worked on here (milliseconds) performance tests are not suitable for day-to-day testing.

Testing Performed

Ran my own test using JMeter numerous times against various points on the branch and compared against master - results largely inconclusive. Added my own logging around every call to getTracer to try and get a sense of the response time of synchronised vs. un-synchronised (inconclusive).

Ran a test created by @fturizo and inspected using JProfiler numerous times against various points on the branch and compared against master to see if any discernible difference could be gained (inconclusive).

Ran MicroProfile OpenTracing TCK to check everything still works.

Test suites executed

MicroProfile OpenTracing

Testing Environment

Windows 10, JDK8.

Documentation

N/A

Notes for Reviewers

Please keep performance in mind when reviewing, pointing out stuff I've missed or things I may have misunderstood.

…nd only get tracer once in StandardWrapper

Signed-off-by: Andrew Pielage <[email protected]>
Signed-off-by: Andrew Pielage <[email protected]>
Signed-off-by: Andrew Pielage <[email protected]>
Signed-off-by: Andrew Pielage <[email protected]>

FISH-5636 Correct try-catch

Signed-off-by: Andrew Pielage <[email protected]>
Signed-off-by: Andrew Pielage <[email protected]>
// Does this NEED to be synchronised? Tracers don't store state, and Scopes are ThreadLocal

// Double-checked locking - potentially naughty
Tracer tracer = tracers.get(applicationName);
Copy link
Contributor

@svendiedrichsen svendiedrichsen Sep 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about ConcurrentHashMap.computeIfAbsent and put the following code into the compute function?

return requestTracingService.isRequestTracingEnabled();
}
return false;
}

public RequestTracingService getRequestTracingService() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about returning an Optional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think returning an Optional really adds much here.
The method is intended as a poke to HK2 if required rather than any kind of true getter (public scope currently present is wrong - should be private).

@jGauravGupta jGauravGupta self-requested a review September 17, 2021 08:24
@Pandrex247
Copy link
Member Author

Jenkins test please

@Pandrex247 Pandrex247 merged commit 67ce52a into payara:master Sep 24, 2021
@Pandrex247 Pandrex247 deleted the FISH-5636-Community branch September 24, 2021 08:35
JamesHillyard pushed a commit to JamesHillyard/Payara that referenced this pull request Oct 28, 2021
FISH-5636 Remote EJB Tracing Improvements
JamesHillyard added a commit to JamesHillyard/Payara that referenced this pull request Dec 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants