Skip to content
This repository has been archived by the owner on Sep 26, 2019. It is now read-only.

Switch back to Xerial Snappy Library #284

Merged
merged 5 commits into from
Nov 20, 2018
Merged

Conversation

ajsutton
Copy link
Contributor

PR description

The iq80 Snappy library has insufficient bounds checking resulting in errors like:

2018-11-20 06:57:16.308+10:00 | nioEventLoopGroup-3-1 | ERROR | DeFramer | Exception while processing incoming message
io.netty.handler.codec.DecoderException: java.lang.ArrayIndexOutOfBoundsException: 2
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459) ~[netty-codec-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-codec-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-transport-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) [netty-common-4.1.15.Final.jar:4.1.15.Final]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_25]
Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
        at org.iq80.snappy.SnappyDecompressor.readUncompressedLength(SnappyDecompressor.java:416) ~[snappy-0.4.jar:?]
        at org.iq80.snappy.SnappyDecompressor.getUncompressedLength(SnappyDecompressor.java:31) ~[snappy-0.4.jar:?]
        at org.iq80.snappy.Snappy.getUncompressedLength(Snappy.java:79) ~[snappy-0.4.jar:?]
        at tech.pegasys.pantheon.ethereum.p2p.rlpx.framing.SnappyCompressor.uncompressedLength(SnappyCompressor.java:38) ~[pantheon-p2p-0.9.0-SNAPSHOT.jar:0.9.0-dev-c8549bdc]
        at tech.pegasys.pantheon.ethereum.p2p.rlpx.framing.Framer.processFrame(Framer.java:259) ~[pantheon-p2p-0.9.0-SNAPSHOT.jar:0.9.0-dev-c8549bdc]
        at tech.pegasys.pantheon.ethereum.p2p.rlpx.framing.Framer.deframe(Framer.java:155) ~[pantheon-p2p-0.9.0-SNAPSHOT.jar:0.9.0-dev-c8549bdc]
        at tech.pegasys.pantheon.ethereum.p2p.netty.DeFramer.decode(DeFramer.java:69) ~[pantheon-p2p-0.9.0-SNAPSHOT.jar:0.9.0-dev-c8549bdc]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-codec-4.1.15.Final.jar:4.1.15.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-codec-4.1.15.Final.jar:4.1.15.Final]
        ... 16 more

The issue that previously caused us to switch libraries (#251) was because the incoming message was invalid and our error handling has been improved to reduce the noise in that case. As a result we can switch back to the more reliable xerial Snappy library.

Fixed Issue(s)

@ajsutton ajsutton added the bug Something isn't working label Nov 20, 2018
Copy link
Contributor

@CjHare CjHare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ajsutton ajsutton merged commit 33ac7e1 into PegaSysEng:master Nov 20, 2018
@ajsutton ajsutton deleted the snappy-jni branch November 20, 2018 06:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants