Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic on index out of bounds #819

Closed
GregoryConrad opened this issue Jun 30, 2024 · 2 comments · Fixed by #825
Closed

panic on index out of bounds #819

GregoryConrad opened this issue Jun 30, 2024 · 2 comments · Fixed by #825

Comments

@GregoryConrad
Copy link
Contributor

I was trying to figure out the maximum key & value size redb supports since I didn't see it specifically documented anywhere (side question: what is the maximum k/v size redb supports?), so I decided to just figure it out for myself. Unfortunately, when I attempted to, I think I hit an edge case:

$ cargo run --release
   Compiling libc v0.2.155
   Compiling redb v2.1.1
   Compiling thing v0.1.0 (/Users/gconrad/Downloads/thing)
    Finished `release` profile [optimized] target(s) in 3.33s
     Running `target/release/thing`
thread 'main' panicked at /Users/gconrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/redb-2.1.1/src/tree_store/page_store/region.rs:73:28:
index out of bounds: the len is 21 but the index is 21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at /Users/gconrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/redb-2.1.1/src/tree_store/page_store/page_manager.rs:1054:43:
called `Result::unwrap()` on an `Err` value: PoisonError { .. }
stack backtrace:
   0:        0x1001f3a04 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h01b2beffade888b2
   1:        0x100209884 - core::fmt::write::hbadb443a71b75f23
   2:        0x1001f1d80 - std::io::Write::write_fmt::hc09d7755e3ead5f0
   3:        0x1001f385c - std::sys_common::backtrace::print::h28349e5c25acbac7
   4:        0x1001f4c1c - std::panicking::default_hook::{{closure}}::hd24b6196784d991e
   5:        0x1001f4900 - std::panicking::default_hook::hfcec80a2720c8c73
   6:        0x1001f5510 - std::panicking::rust_panic_with_hook::h84760468187ddc85
   7:        0x1001f4efc - std::panicking::begin_panic_handler::{{closure}}::he666a5eb600a7203
   8:        0x1001f3e88 - std::sys_common::backtrace::__rust_end_short_backtrace::h592f44d2bf9f843f
   9:        0x1001f4c74 - _rust_begin_unwind
  10:        0x100211444 - core::panicking::panic_fmt::h98bbf7bdf4994454
  11:        0x100211740 - core::result::unwrap_failed::h8e3b933261dd7fec
  12:        0x1001d8b44 - <redb::tree_store::page_store::page_manager::TransactionalMemory as core::ops::drop::Drop>::drop::hfe991fcf7f46db61
  13:        0x10018dcd0 - alloc::sync::Arc<T,A>::drop_slow::h51e838c18fe31e94
  14:        0x10018fe98 - core::ptr::drop_in_place<redb::db::Database>::h37fb11f6bf320a0a
  15:        0x100191df0 - thing::main::h4f7a964ed7cb9a42
  16:        0x10018f3a8 - std::sys_common::backtrace::__rust_begin_short_backtrace::h4b5c857910f82c09
  17:        0x10018f3c0 - std::rt::lang_start::{{closure}}::h580f3206fe5f58f9
  18:        0x1001f01c4 - std::rt::lang_start_internal::h39923ab4c3913741
  19:        0x100191e30 - _main
thread 'main' panicked at library/core/src/panicking.rs:164:5:
panic in a destructor during cleanup
thread caused non-unwinding panic. aborting.
fish: Job 1, 'cargo run --release' terminated by signal SIGABRT (Abort)

Thankfully, this is extremely easy to reproduce. Copy the below into a fresh cargo project that depends upon redb 2.1.1.

use redb::{Database, TableDefinition};

const TABLE: TableDefinition<&[u8], &[u8]> = TableDefinition::new("items");

fn main() {
    let db = Database::create("items.redb").unwrap();

    let txn = db.begin_write().unwrap();
    let mut table = txn.open_table(TABLE).unwrap();

    let key = (0..3_000_000_000usize)
        .map(|x| (x % 255usize) as u8)
        .collect::<Vec<_>>();
    let value = (0..2_000_000_000usize)
        .rev()
        .map(|x| (x % 255usize) as u8)
        .collect::<Vec<_>>();

    table
        .insert(key.as_slice(), value.as_slice())
        .expect("insert should work");

    drop(table);
    txn.commit().unwrap();

    let txn = db.begin_read().unwrap();
    let table = txn.open_table(TABLE).unwrap();

    table.get(key.as_slice()).expect("read should work");
}
@cberner
Copy link
Owner

cberner commented Jun 30, 2024

The maximum size is 3GB for a key or value:

redb/src/error.rs

Lines 13 to 14 in 19fc5b5

/// The value being inserted exceeds the maximum of 3GiB
ValueTooLarge(usize),

But ya this crash makes sense. I didn't consider the fact that someone might insert a 3GB key and a 3GB value, and that will overflow the storage limit of a page. I'll have to think a bit on the best way to fix this. The maximum for a key + value is unspecified, but is somewhere around 4GB minus some overhead

@GregoryConrad
Copy link
Contributor Author

Thanks for the info! Appreciate the quick turnaround

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants