Changed the SPI code so that the iterator usage is Rusty #4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added a SpiHeapTupleData item type for the SpiTupleTable struct and code to let it be used as a normal rust iterator, eg:
|
Much appreciated! |
yrashk
pushed a commit
to yrashk/pgx
that referenced
this pull request
Dec 7, 2022
…ouse Test for Clickhouse FDW
Sasasu
added a commit
to Sasasu/pgrx
that referenced
this pull request
Aug 17, 2023
this is because pgrx will overwrite the .so file in place. dlopen will use mmap with MAP_PRIVATE. but if other process is modifing the file. dlopen will use MAP_PRIVATE to open the file and map a read-only memory use mmap(2). usually this memory has copy-on-write. if others is modifing the file. the previous mapped memory should not change. but there is a undefined behavior, from man mmap(2): ``` MAP_PRIVATE: It is unspecified whether changes made to the file after the mmap() call are visible in the mapped region. ``` what actually happens is that the read-only memory in postgresql is modified, all pointers in the .TEXT segment is mashed up. the call stack looks like ``` #0 0x0000000000731510 in ?? () pgcentralfoundation#1 0x00007f7a94515132 in core::sync::atomic::AtomicUsize::store (self=0x7f7a95110318 <pgrx_pg_sys::submodules::thread_check::ACTIVE_THREAD::h60448dcb81097e92>, val=0, order=core::sync::atomic::Ordering::Relaxed) at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/sync/atomic.rs:2291 pgcentralfoundation#2 0x00007f7a944f574b in pgrx_pg_sys::submodules::thread_check::init_active_thread::clear_in_child () at src/submodules/thread_check.rs:39 pgcentralfoundation#3 0x00007f7a962f8a88 in __run_postfork_handlers (who=who@entry=atfork_run_child, do_locking=do_locking@entry=false, lastrun=lastrun@entry=2) at register-atfork.c:187 pgcentralfoundation#4 0x00007f7a962df773 in __libc_fork () at fork.c:109 pgcentralfoundation#5 0x00005555e66ad948 in fork_process () at fork_process.c:61 pgcentralfoundation#6 0x00005555e66a5d48 in StartAutoVacWorker () at autovacuum.c:1543 pgcentralfoundation#7 0x00005555e66c43f7 in StartAutovacuumWorker () at postmaster.c:6155 pgcentralfoundation#8 0x00005555e66c3d21 in sigusr1_handler (postgres_signal_arg=10) at postmaster.c:5820 pgcentralfoundation#9 <signal handler called> pgcentralfoundation#10 0x00007f7a9630eb84 in __GI___select (nfds=6, readfds=0x7ffc61c2fa40, writefds=0x0, exceptfds=0x0, timeout=0x7ffc61c2f9b0) at ../sysdeps/unix/sysv/linux/select.c:69 pgcentralfoundation#11 0x00005555e66be343 in ServerLoop () at postmaster.c:1950 pgcentralfoundation#12 0x00005555e66bdb0f in PostmasterMain (argc=5, argv=0x5555e8fb5490) at postmaster.c:1631 pgcentralfoundation#13 0x00005555e6560e41 in main (argc=5, argv=0x5555e8fb5490) at main.c:240 ``` the is `pgrx_pg_sys` try to run the postfork hook. but the variable `ACTIVE_THREAD` and the code binary does not in the previous place.
Sasasu
added a commit
to Sasasu/pgrx
that referenced
this pull request
Aug 17, 2023
this is because pgrx will overwrite the .so file in place. dlopen will use mmap with MAP_PRIVATE. but if other process is modifing the file. dlopen will use MAP_PRIVATE to open the file and map a read-only memory use mmap(2). usually this memory has copy-on-write. if others is modifing the file. the previous mapped memory should not change. but there is a undefined behavior, from man mmap(2): ``` MAP_PRIVATE: It is unspecified whether changes made to the file after the mmap() call are visible in the mapped region. ``` what actually happens is that the read-only memory in postgresql is modified, all pointers in the .TEXT segment is mashed up. the call stack looks like ``` #0 0x0000000000731510 in ?? () pgcentralfoundation#1 0x00007f7a94515132 in core::sync::atomic::AtomicUsize::store (self=0x7f7a95110318 <pgrx_pg_sys::submodules::thread_check::ACTIVE_THREAD::h60448dcb81097e92>, val=0, order=core::sync::atomic::Ordering::Relaxed) at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/sync/atomic.rs:2291 pgcentralfoundation#2 0x00007f7a944f574b in pgrx_pg_sys::submodules::thread_check::init_active_thread::clear_in_child () at src/submodules/thread_check.rs:39 pgcentralfoundation#3 0x00007f7a962f8a88 in __run_postfork_handlers (who=who@entry=atfork_run_child, do_locking=do_locking@entry=false, lastrun=lastrun@entry=2) at register-atfork.c:187 pgcentralfoundation#4 0x00007f7a962df773 in __libc_fork () at fork.c:109 pgcentralfoundation#5 0x00005555e66ad948 in fork_process () at fork_process.c:61 pgcentralfoundation#6 0x00005555e66a5d48 in StartAutoVacWorker () at autovacuum.c:1543 pgcentralfoundation#7 0x00005555e66c43f7 in StartAutovacuumWorker () at postmaster.c:6155 pgcentralfoundation#8 0x00005555e66c3d21 in sigusr1_handler (postgres_signal_arg=10) at postmaster.c:5820 pgcentralfoundation#9 <signal handler called> pgcentralfoundation#10 0x00007f7a9630eb84 in __GI___select (nfds=6, readfds=0x7ffc61c2fa40, writefds=0x0, exceptfds=0x0, timeout=0x7ffc61c2f9b0) at ../sysdeps/unix/sysv/linux/select.c:69 pgcentralfoundation#11 0x00005555e66be343 in ServerLoop () at postmaster.c:1950 pgcentralfoundation#12 0x00005555e66bdb0f in PostmasterMain (argc=5, argv=0x5555e8fb5490) at postmaster.c:1631 pgcentralfoundation#13 0x00005555e6560e41 in main (argc=5, argv=0x5555e8fb5490) at main.c:240 ``` the is `pgrx_pg_sys` try to run the postfork hook. but the variable `ACTIVE_THREAD` and the code binary does not in the previous place.
Sasasu
added a commit
to Sasasu/pgrx
that referenced
this pull request
Aug 17, 2023
this is because pgrx will overwrite the .so file in place. dlopen will use mmap with MAP_PRIVATE. but if other process is modifying the file. dlopen will use MAP_PRIVATE to open the file and map a read-only memory use mmap(2). usually this memory has copy-on-write. if others is modifying the file. the previously mapped memory should not change. but there is a undefined behavior, from man mmap(2): ``` MAP_PRIVATE: It is unspecified whether changes made to the file after the mmap() call are visible in the mapped region. ``` what actually happens is that the read-only memory in postgresql is modified, and all pointers in the .TEXT segment is mashed up. the call stack looks like ``` #0 0x0000000000731510 in ?? () pgcentralfoundation#1 0x00007f7a94515132 in core::sync::atomic::AtomicUsize::store (self=0x7f7a95110318 <pgrx_pg_sys::submodules::thread_check::ACTIVE_THREAD::h60448dcb81097e92>, val=0, order=core::sync::atomic::Ordering::Relaxed) at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library/core/src/sync/atomic.rs:2291 pgcentralfoundation#2 0x00007f7a944f574b in pgrx_pg_sys::submodules::thread_check::init_active_thread::clear_in_child () at src/submodules/thread_check.rs:39 pgcentralfoundation#3 0x00007f7a962f8a88 in __run_postfork_handlers (who=who@entry=atfork_run_child, do_locking=do_locking@entry=false, lastrun=lastrun@entry=2) at register-atfork.c:187 pgcentralfoundation#4 0x00007f7a962df773 in __libc_fork () at fork.c:109 pgcentralfoundation#5 0x00005555e66ad948 in fork_process () at fork_process.c:61 pgcentralfoundation#6 0x00005555e66a5d48 in StartAutoVacWorker () at autovacuum.c:1543 pgcentralfoundation#7 0x00005555e66c43f7 in StartAutovacuumWorker () at postmaster.c:6155 pgcentralfoundation#8 0x00005555e66c3d21 in sigusr1_handler (postgres_signal_arg=10) at postmaster.c:5820 pgcentralfoundation#9 <signal handler called> pgcentralfoundation#10 0x00007f7a9630eb84 in __GI___select (nfds=6, readfds=0x7ffc61c2fa40, writefds=0x0, exceptfds=0x0, timeout=0x7ffc61c2f9b0) at ../sysdeps/unix/sysv/linux/select.c:69 pgcentralfoundation#11 0x00005555e66be343 in ServerLoop () at postmaster.c:1950 pgcentralfoundation#12 0x00005555e66bdb0f in PostmasterMain (argc=5, argv=0x5555e8fb5490) at postmaster.c:1631 pgcentralfoundation#13 0x00005555e6560e41 in main (argc=5, argv=0x5555e8fb5490) at main.c:240 ``` the is `pgrx_pg_sys` try to run the postfork hook. but the variable `ACTIVE_THREAD` and the code binary does not in the previous place.
eeeebbbbrrrr
added a commit
to eeeebbbbrrrr/pgrx
that referenced
this pull request
Nov 2, 2024
I can't quite explain why, but this function was deadlocking with this backtrace: ``` * thread #1, name = 'cargo-pgrx', stop reason = signal SIGSTOP * frame #0: 0x00007fbe1eeb288d libc.so.6`syscall at syscall.S:38 frame #1: 0x000055aee526c5d3 cargo-pgrx`std::sys::sync::mutex::futex::Mutex::lock_contended::h6389e2305b0b005c [inlined] std::sys::pal::unix::futex::futex_wait::h30abf43e2d55aa33 at futex.rs:67:21 frame #2: 0x000055aee526c590 cargo-pgrx`std::sys::sync::mutex::futex::Mutex::lock_contended::h6389e2305b0b005c at futex.rs:57:13 frame #3: 0x000055aee4157835 cargo-pgrx`std::sync::mutex::Mutex$LT$T$GT$::lock::h4d2bb65800cc6fd3 at futex.rs:29:13 frame pgcentralfoundation#4: 0x000055aee41577ed cargo-pgrx`std::sync::mutex::Mutex$LT$T$GT$::lock::h4d2bb65800cc6fd3(self=0x000055aee6926c20) at mutex.rs:317:24 frame pgcentralfoundation#5: 0x000055aee406c779 cargo-pgrx`cargo_pgrx::command::install::get_git_hash::ha84d504db9d1bba8(manifest_path=0x00007ffdc43b83b0) at install.rs:507:9 frame pgcentralfoundation#6: 0x000055aee406d6e9 cargo-pgrx`cargo_pgrx::command::install::filter_contents::h8c710847129ba6be(manifest_path=0x00007ffdc43b8d10, input=String @ 0x00007ffdc43b8940) at install.rs:541:46 ```
eeeebbbbrrrr
added a commit
that referenced
this pull request
Nov 2, 2024
I can't quite explain why, but this function was deadlocking with this backtrace: ``` * thread #1, name = 'cargo-pgrx', stop reason = signal SIGSTOP * frame #0: 0x00007fbe1eeb288d libc.so.6`syscall at syscall.S:38 frame #1: 0x000055aee526c5d3 cargo-pgrx`std::sys::sync::mutex::futex::Mutex::lock_contended::h6389e2305b0b005c [inlined] std::sys::pal::unix::futex::futex_wait::h30abf43e2d55aa33 at futex.rs:67:21 frame #2: 0x000055aee526c590 cargo-pgrx`std::sys::sync::mutex::futex::Mutex::lock_contended::h6389e2305b0b005c at futex.rs:57:13 frame #3: 0x000055aee4157835 cargo-pgrx`std::sync::mutex::Mutex$LT$T$GT$::lock::h4d2bb65800cc6fd3 at futex.rs:29:13 frame #4: 0x000055aee41577ed cargo-pgrx`std::sync::mutex::Mutex$LT$T$GT$::lock::h4d2bb65800cc6fd3(self=0x000055aee6926c20) at mutex.rs:317:24 frame #5: 0x000055aee406c779 cargo-pgrx`cargo_pgrx::command::install::get_git_hash::ha84d504db9d1bba8(manifest_path=0x00007ffdc43b83b0) at install.rs:507:9 frame #6: 0x000055aee406d6e9 cargo-pgrx`cargo_pgrx::command::install::filter_contents::h8c710847129ba6be(manifest_path=0x00007ffdc43b8d10, input=String @ 0x00007ffdc43b8940) at install.rs:541:46 ```
eeeebbbbrrrr
added a commit
that referenced
this pull request
Nov 2, 2024
I can't quite explain why, but this function was deadlocking with this backtrace: ``` * thread #1, name = 'cargo-pgrx', stop reason = signal SIGSTOP * frame #0: 0x00007fbe1eeb288d libc.so.6`syscall at syscall.S:38 frame #1: 0x000055aee526c5d3 cargo-pgrx`std::sys::sync::mutex::futex::Mutex::lock_contended::h6389e2305b0b005c [inlined] std::sys::pal::unix::futex::futex_wait::h30abf43e2d55aa33 at futex.rs:67:21 frame #2: 0x000055aee526c590 cargo-pgrx`std::sys::sync::mutex::futex::Mutex::lock_contended::h6389e2305b0b005c at futex.rs:57:13 frame #3: 0x000055aee4157835 cargo-pgrx`std::sync::mutex::Mutex$LT$T$GT$::lock::h4d2bb65800cc6fd3 at futex.rs:29:13 frame #4: 0x000055aee41577ed cargo-pgrx`std::sync::mutex::Mutex$LT$T$GT$::lock::h4d2bb65800cc6fd3(self=0x000055aee6926c20) at mutex.rs:317:24 frame #5: 0x000055aee406c779 cargo-pgrx`cargo_pgrx::command::install::get_git_hash::ha84d504db9d1bba8(manifest_path=0x00007ffdc43b83b0) at install.rs:507:9 frame #6: 0x000055aee406d6e9 cargo-pgrx`cargo_pgrx::command::install::filter_contents::h8c710847129ba6be(manifest_path=0x00007ffdc43b8d10, input=String @ 0x00007ffdc43b8940) at install.rs:541:46 ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added a SpiHeapTupleData item type for the SpiTupleTable struct and code
to let it be used as a normal rust iterator.