Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dubbo RPC attachments java.util.ConcurrentModificationException #10078

Closed
shenjiaqing1 opened this issue May 24, 2022 · 6 comments · Fixed by #10139
Closed

dubbo RPC attachments java.util.ConcurrentModificationException #10078

shenjiaqing1 opened this issue May 24, 2022 · 6 comments · Fixed by #10139
Labels
type/bug Bugs to being fixed
Milestone

Comments

@shenjiaqing1
Copy link

截屏2022-05-24 上午11 58 32
image
image
hashmap改成ConcurrentHashMap

@shenjiaqing1 shenjiaqing1 added the type/bug Bugs to being fixed label May 24, 2022
@chickenlj
Copy link
Contributor

RpcInvocation 实例是在什么场景出现了被多线程共享的情况?

@shenjiaqing1
Copy link
Author

RpcInvocation 实例是在什么场景出现了被多线程共享的情况?

有两个条件,条件一是dubbo开启异步回调,条件二是高并发的情况下会存在非常小的概率发生这样的问题。线程1和线程2共享一个map,线程1在操作for循环的时候,线程2有一定的概率会修改与线程1共享的map。

@shenjiaqing1
Copy link
Author

RpcInvocation 实例是在什么场景出现了被多线程共享的情况?

image
我能理解为当存在同一个线程执行另外一个rpc调用的时候,就可能会发生前一个rpc异步回调的时候RpcContext被修改,是吗?

@CVnot
Copy link

CVnot commented Aug 7, 2023

就可能会发生前

@chickenlj @shenjiaqing1 我理解这个地方不应该会有并发问题的,因为一个线程发起完请求之后,AsyncRpcResult中存储了一份Threadlocal的对象,并且在ContextFilter进行了remove,所以即使这个这个线程在接收到下一个请求的同时,上一个请求返回了,也不会修改一个对象的。我看的代码版本是2.7.18
image

@CVnot
Copy link

CVnot commented Aug 8, 2023

我理解不会的,都是ThreadLocal的,上一个请求回来的对象和下一次请求的不是一个对象的,我们也遇到类似的问题,不过并不是这个地方

@giraffe-tree
Copy link
Contributor

我也遇到的这个问题
consumer / provider 用的是都是 dubbo 2.7.9

org.apache.dubbo.rpc.RpcException: Failed to invoke the method inWithResult in the service xxxx.xxxxService. Tried 1 times of the providers [xxxx:17071] (1/68) from the registry xxxx:7400 on the consumer xxxx using the dubbo version 2.7.9. Last error is: null
        at org.apache.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:113)
        at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:265)
        at org.apache.dubbo.rpc.cluster.interceptor.ClusterInterceptor.intercept(ClusterInterceptor.java:47)
        at org.apache.dubbo.rpc.cluster.support.wrapper.AbstractCluster$InterceptorInvokerNode.invoke(AbstractCluster.java:92)
        at org.apache.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:93)
        at org.apache.dubbo.registry.integration.InterfaceCompatibleRegistryProtocol$MigrationInvoker.invoke(InterfaceCompatibleRegistryProtocol.java:132)
        at org.apache.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:83)
        at xxx 业务堆栈
        at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
        at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
        at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
        at org.apache.dubbo.rpc.protocol.dubbo.FutureAdapter.lambda$new$0(FutureAdapter.java:49)
        at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
        at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
        at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
        at org.apache.dubbo.remoting.exchange.support.DefaultFuture.doReceived(DefaultFuture.java:208)
        at org.apache.dubbo.remoting.exchange.support.DefaultFuture.received(DefaultFuture.java:175)
        at org.apache.dubbo.remoting.exchange.support.DefaultFuture.received(DefaultFuture.java:163)
        at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleResponse(HeaderExchangeHandler.java:60)
        at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:181)
        at org.apache.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:51)
        at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:57)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)
        at java.util.HashMap$EntryIterator.next(HashMap.java:1479)
        at java.util.HashMap$EntryIterator.next(HashMap.java:1477)
        at java.util.HashMap.putMapEntries(HashMap.java:512)
        at java.util.HashMap.putAll(HashMap.java:785)
        at org.apache.dubbo.rpc.RpcInvocation.addObjectAttachments(RpcInvocation.java:337)
        at org.apache.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:160)
        at org.apache.dubbo.rpc.protocol.AsyncToSyncInvoker.invoke(AsyncToSyncInvoker.java:52)
        at org.apache.dubbo.rpc.listener.ListenerInvokerWrapper.invoke(ListenerInvokerWrapper.java:78)
        at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:83)
        at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:83)
        at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:89)
        at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:83)
        at org.apache.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:51)
        at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:83)
        at org.apache.dubbo.rpc.filter.ConsumerContextFilter.invoke(ConsumerContextFilter.java:69)
        at org.apache.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:83)
        at org.apache.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:56)
        at org.apache.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:82)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Bugs to being fixed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants