Akka.Persistence: IStash filters out messages with identical senders + references

[Aaronontheweb]

Version Information
Version of Akka.NET? v1.5.30
Which Akka.NET Modules? Akka.Persistence

Describe the bug

Two versions of the same test, against an actor that uses Stash.Stash() internally - the CohortActor in this case is an UntypedPersistentActor that implements IWithStash

[Fact]
public async Task ShouldAllowDuplicatePauseCommands()
    {
        // arrange
        var cohortId = TestCohortId;
        var cohortActor = await ActorRegistry.GetAsync<SubscriptionsActorRegistryKeys.CohortActorKey>();
        var pauseCmd = new CohortSendingCommands.PauseCohortSends(cohortId, DateTime.UtcNow);
        
        // act
        cohortActor.Tell(pauseCmd, TestActor);
        cohortActor.Tell(pauseCmd, TestActor);
        
        // assert
        var resp1 = await ExpectMsgAsync<CommandResponse>();
        var resp2 = await ExpectMsgAsync<CommandResponse>();
        
        Assert.Equal(CommandResponseCode.Success, resp1.Code);
        Assert.Equal(CommandResponseCode.NoOp, resp2.Code);
    }

This test fails - the second cohortActor.Tell(pauseCmd, TestActor); does not get processed, so the second ExpectMsgAsync times out. That message simply disappears - no DeadLetter notification or anything. I even observed two messages both go into the stash Stash.Count == 2 but when it comes time to UnstashAll, only a single instance makes it into the mailbox.

The second version of this test passes:

[Fact]
public async Task ShouldAllowDuplicatePauseCommands()
{
    // arrange
    var cohortId = TestCohortId;
    var cohortActor = await ActorRegistry.GetAsync<SubscriptionsActorRegistryKeys.CohortActorKey>();
    var pauseCmd1 = new CohortSendingCommands.PauseCohortSends(cohortId, DateTime.UtcNow);
    var pauseCmd2 = new CohortSendingCommands.PauseCohortSends(cohortId, DateTime.UtcNow);
    
    // act
    cohortActor.Tell(pauseCmd1, TestActor);
    cohortActor.Tell(pauseCmd2, TestActor);
    
    // assert
    var resp1 = await ExpectMsgAsync<CommandResponse>();
    var resp2 = await ExpectMsgAsync<CommandResponse>();
    
    Assert.Equal(CommandResponseCode.Success, resp1.Code);
    Assert.Equal(CommandResponseCode.NoOp, resp2.Code);
}

The message content is effectively identical, but the pauseCmd1 and pauseCmd2 reference equality checks do not pass!

To Reproduce

Runnable reproduction in LinqPad here: https://share.linqpad.net/exn9a5du.linq - this one uses Akka.Persistence actors

Expected behavior

Both stashed messages should be successfully processed.

Actual behavior

In other tests, only one of the stashed messages gets processed.

Additional context

Worth noting - if I can't reproduce this with vanilla Akka even if I do weird stuff like change the behavior to non-default in the CTOR of the UntypedActor: https://share.linqpad.net/s7urq58p.linq