fix race conditions on entry

[lwsanty]

closes #1046

[bconway]

This is still a bug. It can occur with any use of Entry and a hook (i.e. logrustash).

[markphelps]

Thank you @blocknonip and @lwsanty!!. Sorry for the stalebot spam. I thought it would only label/comment on issues, not PRs too.

[flimzy]

This lock is far too aggressive. It locks the entire logger, rather than a single entry as I believe it should.

[dgsb]

Agree, we should find a better solution than this one.

[markphelps]

@flimzy my apologies for merging this so quickly. would you want to work on a PR to lock only the entry? If not I can early this coming week.

[flimzy]

I looked at it briefly before posting my comment, and it wasn't immediately obvious the proper solution. It looks like we need to add a mutex to the Entry type, and lock it in WithFields and in fireHooks, but my first attempt lead to a deadlock.

While working on that, I ran into what may be other overly-aggressive locks which could be relaxed, too, but it's tricky to do that with confidence, without tests covering all possible races (which are incredibly hard to test).

I'll try again if I have time, but you may get to it faster than me.

[markphelps]

@flimzy ok I created #1122 to track. I'll also try taking a look