Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java MultiGet NPE with key list 70_000 , but ok with 60_000 #8039

Closed
hsnamed opened this issue Mar 7, 2021 · 1 comment
Closed

Java MultiGet NPE with key list 70_000 , but ok with 60_000 #8039

hsnamed opened this issue Mar 7, 2021 · 1 comment
Assignees
Labels

Comments

@hsnamed
Copy link

hsnamed commented Mar 7, 2021

rocksdbjni-6-5-12

What the reason for this behavior
All options by default

private byte[] generateString(int size) {
        var ba = new byte[size];
        new Random().nextBytes(ba);
        return ba;
 }
var lst = IntStream.range(0, 70_000).mapToObj(i -> generateString(4)).collect(Collectors.toList());
var res= db.multiGetAsList(lst);

Got exception

java.lang.NullPointerException
	at java.base/java.util.Objects.requireNonNull(Objects.java:221)
	at java.base/java.util.Arrays$ArrayList.<init>(Arrays.java:4323)
	at java.base/java.util.Arrays.asList(Arrays.java:4310)
	at org.rocksdb.RocksDB.multiGetAsList(RocksDB.java:2424)

But multigeting list with 60_000 keys is ok.

@adamretter
Copy link
Collaborator

adamretter commented Apr 26, 2021

@hsnamed Thanks for the report, I can reproduce this via the test:

  /**
   * Test for https://github.com/facebook/rocksdb/issues/8039
   */
  @Test
  public void multiGetAsListLarge() throws RocksDBException {
    final List<byte[]> keys = new ArrayList();
    for (int i = 0; i < 70000; i++) {
      final byte[] key = new byte[4];
      rand.nextBytes(key);
      keys.add(key);
    }

    try (final Options opt = new Options()
                 .setCreateIfMissing(true);
         final RocksDB db = RocksDB.open(opt,
                 dbFolder.getRoot().getAbsolutePath())
    ) {
      final List<byte[]> values = db.multiGetAsList(keys);
      assertThat(values.size()).isEqualTo(keys.size());
    }
  }

There is nothing obvious that strikes me as a problem. I will have to dig into this a bit on the C++ JNI side...

yoori pushed a commit to yoori/rocksdb that referenced this issue Nov 26, 2023
Summary:
closes facebook/rocksdb#8039

Unnecessary use of multiple local JNI references at the same time, 1 per key, was limiting the size of the key array. The local references don't need to be held simultaneously, so if we rearrange the code we can make it work for bigger key arrays.

Incidentally, make errors throw helpful exception messages rather than returning a null pointer.

Pull Request resolved: facebook/rocksdb#9012

Reviewed By: mrambacher

Differential Revision: D31580862

Pulled By: jay-zhuang

fbshipit-source-id: ce05831d52ede332e1b20e74d2dc621d219b9616
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants