Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clone PerMessageDeflateExtension values correctly #1439

Merged
merged 5 commits into from
Nov 11, 2024
Merged

Clone PerMessageDeflateExtension values correctly #1439

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a566891
Correctly clone values provided by the setter
marci4 Oct 8, 2024
1e9fbbf
Reuse inflater/deflater
marci4 Oct 8, 2024
fe7635b
Provide a setter/getter to set the deflater level
marci4 Oct 13, 2024
4f4aed5
Extends tests to better test the deflater level
marci4 Oct 13, 2024
dfca00b
Add Constructor with custom compression level
marci4 Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 27 additions & 18 deletions ...ain/java/org/java_websocket/extensions/permessage_deflate/PerMessageDeflateExtension.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,23 +53,28 @@ public class PerMessageDeflateExtension extends CompressionExtension {
// For WebSocketClients, this variable holds the extension parameters that client himself has requested.
private Map<String, String> requestedParameters = new LinkedHashMap<>();

private Inflater inflater = new Inflater(true);
private Deflater deflater = new Deflater(Deflater.DEFAULT_COMPRESSION, true);

public Inflater getInflater() {
return inflater;
}
private int deflaterLevel = Deflater.DEFAULT_COMPRESSION;

public void setInflater(Inflater inflater) {
this.inflater = inflater;
}
private Inflater inflater = new Inflater(true);
private Deflater deflater = new Deflater(this.deflaterLevel, true);

public Deflater getDeflater() {
return deflater;
/**
* Get the compression level used for the compressor.
* @return the compression level (0-9)
*/
public int getDeflaterLevel() {
return this.deflaterLevel;
}

public void setDeflater(Deflater deflater) {
this.deflater = deflater;
/**
* Set the compression level used for the compressor.
* @param level the compression level (0-9)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though the Deflater Javadoc says the same, the implementation actually also accepts DEFAULT_COMPRESSION, which is -1. The getter above would then return -1 as well.

*/
public void setDeflaterLevel(int level) {
this.deflater.setLevel(level);
this.deflaterLevel = level;
//If the compression level is changed, the next invocation of deflate will compress the input available so far with the old level (and may be flushed); the new level will take effect only after that invocation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this does not work in Java 8:

While this matches the Java 8 Deflater Javadoc, the implementation does not match the documentation:
https://github.com/openjdk/jdk8u/blob/master/jdk/src/share/classes/java/util/zip/Deflater.java

The set level is never picked up as far as I can tell. The "setParams" boolean is never evaluated. It is in Java 11, though, so it works there:
https://github.com/openjdk/jdk11u/blob/master/src/java.base/share/classes/java/util/zip/Deflater.java

So, for Java 8 compatibility, the Deflater instance has to be created with the configured compression level.

How about not having setDeflaterLevel at all, but rather making it an optional constructor argument:

private final int compressionLevel;
private final Inflater inflater;
private final Deflater deflater;

public PerMessageDeflateExtension() {
this(Deflater.DEFAULT_COMPRESSION);
}

public PerMessageDeflateExtension(int compressionLevel) {
this.compressionLevel = compressionLevel;
this.inflater = new Inflater(true);
this.deflater = new Deflater(compressionLevel, true);
}

public int getCompressionLevel() {
return compressionLevel;
}

@OverRide
public IExtension copyInstance() {
PerMessageDeflateExtension clone = new PerMessageDeflateExtension(this.getCompressionLevel());
clone.setThreshold(this.getThreshold());
clone.setClientNoContextTakeover(this.isClientNoContextTakeover());
clone.setServerNoContextTakeover(this.isServerNoContextTakeover());
return clone;
}

this.deflater.deflate(new byte[0]);
}

/**
Expand Down Expand Up @@ -166,15 +171,15 @@ We can check the getRemaining() method to see whether the data we supplied has b
Note that this behavior doesn't occur if the message is "first compressed and then fragmented".
*/
if (inflater.getRemaining() > 0) {
inflater = new Inflater(true);
inflater.reset();
decompress(inputFrame.getPayloadData().array(), output);
}

if (inputFrame.isFin()) {
decompress(TAIL_BYTES, output);
// If context takeover is disabled, inflater can be reset.
if (clientNoContextTakeover) {
inflater = new Inflater(true);
inflater.reset();
}
}
} catch (DataFormatException e) {
Expand Down Expand Up @@ -244,8 +249,7 @@ public void encodeFrame(Framedata inputFrame) {
}

if (serverNoContextTakeover) {
deflater.end();
deflater = new Deflater(Deflater.DEFAULT_COMPRESSION, true);
deflater.reset();
}
}

Expand Down Expand Up @@ -330,7 +334,12 @@ public String getProvidedExtensionAsServer() {

@Override
public IExtension copyInstance() {
return new PerMessageDeflateExtension();
PerMessageDeflateExtension clone = new PerMessageDeflateExtension();
clone.setThreshold(this.getThreshold());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please insert:

  • clone.setDeflater(this.getDeflater());
  • clone.setInflater(this.getInflater());

and replace the new Inflater() calls in lines 169 and 177 with inflater.reset(), and the deflater.end() & new Deflater() calls in lines 247/248 with deflater.reset().

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not going to reuse the deflater/inflater for multiple clients

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using reset however should be done, I agree

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not going to reuse the deflater/inflater for multiple clients

Could you elaborate? If the user of this library wants to hand in a different inflater/deflater, why should that not be used?

Specific use case: I want to use BEST_COMPRESSION, not DEFAULT_COMPRESSION, to get the most out of compression (and both my servers and clients are powerful enough to put up that extra effort). That's why I hand in a new Deflater(Deflater.BEST_COMPRESSION, true) instance. I'd like that to work.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that we create a new instance of a Draft (and therefore the provided extension with the custom deflater/inflater) for every new client connecting to the server.
https://github.com/TooTallNate/Java-WebSocket/blob/master/src/main/java/org/java_websocket/WebSocketImpl.java#L272

If we simply use the single instance provided by the user, we will get in trouble for the server implementation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we simply use the single instance provided by the user, we will get in trouble for the server implementation

Ah, thanks, I see. I was just lucky that I was only using 1 server and client for my WebSocket compression tests.

Indeed, then an API change is needed to set Inflater and Deflater factories. Or, if we think that this flexibility is not needed, a setCompressionLevel(int level); API would do for me. Then this class would always instantiate the Inflater and Deflater objects, always with nowrap=true.

Copy link
Collaborator Author

@marci4 marci4 Oct 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking more into the documentation, I think that nowrap will always be true in our case

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we simply use the single instance provided by the user, we will get in trouble for the server implementation

Ah, thanks, I see. I was just lucky that I was only using 1 server and client for my WebSocket compression tests.

Indeed, then an API change is needed to set Inflater and Deflater factories. Or, if we think that this flexibility is not needed, a setCompressionLevel(int level); API would do for me. Then this class would always instantiate the Inflater and Deflater objects, always with nowrap=true.

Yeah I think so too. We only need the compression level, nowrap should be true.

@PhilipRoman you got any objections?

Copy link
Collaborator Author

@marci4 marci4 Oct 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, there is no getter for the level/strategy of a Deflater.

Therefore:
new API -> setCompressionLevel()
remove getter/setter for Deflater/Inflater

clone.setClientNoContextTakeover(this.isClientNoContextTakeover());
clone.setServerNoContextTakeover(this.isServerNoContextTakeover());
clone.setDeflaterLevel(this.getDeflaterLevel());
return clone;
}

/**
Expand Down
167 changes: 147 additions & 20 deletions src/test/java/org/java_websocket/extensions/PerMessageDeflateExtensionTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
package org.java_websocket.extensions;

import static java.util.zip.GZIPInputStream.GZIP_MAGIC;
import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;

import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.zip.Deflater;
import java.util.zip.Inflater;
import org.java_websocket.exceptions.InvalidDataException;
Expand Down Expand Up @@ -51,6 +53,111 @@ public void testDecodeFrameIfRSVIsNotSet() throws InvalidDataException {
assertFalse(frame.isRSV1());
}

@Test
public void testDecodeFrameNoCompression() throws InvalidDataException {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setDeflaterLevel(Deflater.NO_COMPRESSION);
deflateExtension.setThreshold(0);
String str = "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text";
byte[] message = str.getBytes();
TextFrame frame = new TextFrame();
frame.setPayload(ByteBuffer.wrap(message));
deflateExtension.encodeFrame(frame);
byte[] payloadArray = frame.getPayloadData().array();
assertArrayEquals(message, Arrays.copyOfRange(payloadArray, 5,payloadArray.length-5));
assertTrue(frame.isRSV1());
deflateExtension.decodeFrame(frame);
assertArrayEquals(message, frame.getPayloadData().array());
}

@Test
public void testDecodeFrameBestSpeedCompression() throws InvalidDataException {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setDeflaterLevel(Deflater.BEST_SPEED);
deflateExtension.setThreshold(0);
String str = "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text";
byte[] message = str.getBytes();
TextFrame frame = new TextFrame();
frame.setPayload(ByteBuffer.wrap(message));

Deflater localDeflater = new Deflater(Deflater.BEST_SPEED,true);
localDeflater.setInput(ByteBuffer.wrap(message).array());
byte[] buffer = new byte[1024];
int bytesCompressed = localDeflater.deflate(buffer, 0, buffer.length, Deflater.SYNC_FLUSH);

deflateExtension.encodeFrame(frame);
byte[] payloadArray = frame.getPayloadData().array();
assertArrayEquals(Arrays.copyOfRange(buffer,0, bytesCompressed), Arrays.copyOfRange(payloadArray,0,payloadArray.length));
assertTrue(frame.isRSV1());
deflateExtension.decodeFrame(frame);
assertArrayEquals(message, frame.getPayloadData().array());
}

@Test
public void testDecodeFrameBestCompression() throws InvalidDataException {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setDeflaterLevel(Deflater.BEST_COMPRESSION);
deflateExtension.setThreshold(0);
String str = "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text";
byte[] message = str.getBytes();
TextFrame frame = new TextFrame();
frame.setPayload(ByteBuffer.wrap(message));

Deflater localDeflater = new Deflater(Deflater.BEST_COMPRESSION,true);
localDeflater.setInput(ByteBuffer.wrap(message).array());
byte[] buffer = new byte[1024];
int bytesCompressed = localDeflater.deflate(buffer, 0, buffer.length, Deflater.SYNC_FLUSH);

deflateExtension.encodeFrame(frame);
byte[] payloadArray = frame.getPayloadData().array();
assertArrayEquals(Arrays.copyOfRange(buffer,0, bytesCompressed), Arrays.copyOfRange(payloadArray,0,payloadArray.length));
assertTrue(frame.isRSV1());
deflateExtension.decodeFrame(frame);
assertArrayEquals(message, frame.getPayloadData().array());
}

@Test
public void testDecodeFrameSwitchCompression() throws InvalidDataException {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setDeflaterLevel(Deflater.NO_COMPRESSION);
deflateExtension.setThreshold(0);
String str = "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text"
+ "This is a highly compressable text";
byte[] message = str.getBytes();
TextFrame frame = new TextFrame();
frame.setPayload(ByteBuffer.wrap(message));

Deflater localDeflater = new Deflater(Deflater.BEST_COMPRESSION,true);
localDeflater.setInput(ByteBuffer.wrap(message).array());
byte[] buffer = new byte[1024];
int bytesCompressed = localDeflater.deflate(buffer, 0, buffer.length, Deflater.SYNC_FLUSH);

// Change the deflater level after the creation and switch to a new deflater level
// Compression strategy should be applied instantly since we call .deflate manually
deflateExtension.setDeflaterLevel(Deflater.BEST_COMPRESSION);
deflateExtension.encodeFrame(frame);
byte[] payloadArray = frame.getPayloadData().array();
assertArrayEquals(Arrays.copyOfRange(buffer,0, bytesCompressed), Arrays.copyOfRange(payloadArray,0,payloadArray.length));
assertTrue(frame.isRSV1());
deflateExtension.decodeFrame(frame);
assertArrayEquals(message, frame.getPayloadData().array());
}

@Test
public void testEncodeFrame() {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
Expand Down Expand Up @@ -191,35 +298,55 @@ public void testSetClientNoContextTakeover() {
@Test
public void testCopyInstance() {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
IExtension newDeflateExtension = deflateExtension.copyInstance();
assertEquals(deflateExtension.toString(), newDeflateExtension.toString());
}
PerMessageDeflateExtension newDeflateExtension = (PerMessageDeflateExtension)deflateExtension.copyInstance();
assertEquals("PerMessageDeflateExtension", newDeflateExtension.toString());
// Also check the values
assertEquals(deflateExtension.getThreshold(), newDeflateExtension.getThreshold());
assertEquals(deflateExtension.isClientNoContextTakeover(), newDeflateExtension.isClientNoContextTakeover());
assertEquals(deflateExtension.isServerNoContextTakeover(), newDeflateExtension.isServerNoContextTakeover());
assertEquals(deflateExtension.getDeflaterLevel(), newDeflateExtension.getDeflaterLevel());

@Test
public void testGetInflater() {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
assertEquals(deflateExtension.getInflater().getRemaining(), new Inflater(true).getRemaining());
}

@Test
public void testSetInflater() {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setInflater(new Inflater(false));
assertEquals(deflateExtension.getInflater().getRemaining(), new Inflater(false).getRemaining());
deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setThreshold(512);
deflateExtension.setServerNoContextTakeover(false);
deflateExtension.setClientNoContextTakeover(true);
deflateExtension.setDeflaterLevel(Deflater.BEST_COMPRESSION);
newDeflateExtension = (PerMessageDeflateExtension)deflateExtension.copyInstance();

assertEquals(deflateExtension.getThreshold(), newDeflateExtension.getThreshold());
assertEquals(deflateExtension.isClientNoContextTakeover(), newDeflateExtension.isClientNoContextTakeover());
assertEquals(deflateExtension.isServerNoContextTakeover(), newDeflateExtension.isServerNoContextTakeover());
assertEquals(deflateExtension.getDeflaterLevel(), newDeflateExtension.getDeflaterLevel());


deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setThreshold(64);
deflateExtension.setServerNoContextTakeover(true);
deflateExtension.setClientNoContextTakeover(false);
deflateExtension.setDeflaterLevel(Deflater.NO_COMPRESSION);
newDeflateExtension = (PerMessageDeflateExtension)deflateExtension.copyInstance();

assertEquals(deflateExtension.getThreshold(), newDeflateExtension.getThreshold());
assertEquals(deflateExtension.isClientNoContextTakeover(), newDeflateExtension.isClientNoContextTakeover());
assertEquals(deflateExtension.isServerNoContextTakeover(), newDeflateExtension.isServerNoContextTakeover());
assertEquals(deflateExtension.getDeflaterLevel(), newDeflateExtension.getDeflaterLevel());
}

@Test
public void testGetDeflater() {
public void testDeflaterLevel() {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
assertEquals(deflateExtension.getDeflater().finished(),
new Deflater(Deflater.DEFAULT_COMPRESSION, true).finished());
assertEquals(Deflater.DEFAULT_COMPRESSION, deflateExtension.getDeflaterLevel());
deflateExtension.setDeflaterLevel(Deflater.BEST_SPEED);
assertEquals(Deflater.BEST_SPEED, deflateExtension.getDeflaterLevel());
}

@Test
public void testSetDeflater() {
public void testDefaults() {
PerMessageDeflateExtension deflateExtension = new PerMessageDeflateExtension();
deflateExtension.setDeflater(new Deflater(Deflater.DEFAULT_COMPRESSION, false));
assertEquals(deflateExtension.getDeflater().finished(),
new Deflater(Deflater.DEFAULT_COMPRESSION, false).finished());
assertFalse(deflateExtension.isClientNoContextTakeover());
assertTrue(deflateExtension.isServerNoContextTakeover());
assertEquals(1024, deflateExtension.getThreshold());
assertEquals(Deflater.DEFAULT_COMPRESSION, deflateExtension.getDeflaterLevel());
}
}
Loading