sv_setsv_cow: only succeed if sv_setsv() would also COW #22120
Conversation
tonycoz
force-pushed
the
21877-cow-memory
branch
from
f078ccd to
b2a4d54
Compare
|
@tonycoz this p.r. was filed back in April, but is still in draft status and has acquired merge conflicts. Assuming you want to move forward with it, could you resolve those conflicts and un-draft the ticket? Thanks. |
|
I'll rebase it, I'm looking into some behaviour I didn't expect with a variant of this change. |
|
Your performance problem is related to the ticket above, and HeapReAlloc() either refuses to shrink the memory allocation as a MS perf optimization, or all "freed" memory by HeapReAlloc() from large (any alloc too large for Low Fragmention Heap), is donated to the Low Fragmentation Heap. HeapReAlloc() does a I agree with logic, is SvCUR vs SvLEN gap is too big at some threshold, do not COW. Or shrink the PV buffer in the parent SV * (maybe a very bad perf idea for PAD vars). edit: very little can be found on this bug, and im using win7, but look at this 2011 post, nobody in that thread knows why the code exists, but someone in the mid 1990s knew the secret bug/problem but it was lost to time |
https://stackoverflow.com/questions/9164837/realloc-does-not-correctly-free-memory-in-windows Above shows the bug with Windows architecture. I found the correct explanation, NT Kernel has no function called The only way to shrink the reserved address space of kernel memory handle is to VirtualAlloc a new kernel memory object, then call Side trivia |
This will actually fail in some cases in the next commit.
Previously if you had a successful match against an SV with a
SvLEN() large relative to the SvCUR() the regexp engine would
use sv_setsv_cow() to make a COW copy of the matched SV,
extending the life of the large allocation buffer.
A normal sv_setsv() normally didn't do such a COW copy, but the above
also marked the source SV as COW, so further copies of the SV
could even further extend the lifetime of the buffer, eg:
while (<>) { # readline tends to make large SvLEN()
/something/; # some sort of match
push @save, $_; # with a successful match, the large $_ buffer
# survives until @save is released
}
Fixes part of Perl#21877
This was causing leaks in a slightly different version of sv_setsv_cow(), I wasn't able to make it leak in the current version, but this may save someone else's sanity in the future, alas for my own. The rr tool was handy in tracking this down.
tonycoz
force-pushed
the
21877-cow-memory
branch
from
5717741 to
e531053
Compare
The behaviour I was concerned about was a leak if sv_setsv_cow() didn't cow copy an already COW SV, I eventually tracked this down to pp_subst, which seems to assume force_normal() will always free the buffer of a CoW SV: it doesn't, if the SV is the final owner the SV retains the buffer and CoW is turned off. Fixing that leak (freeing the memory in pp_subst) lead to a use after free in the regular expression engine that I'll be tracking down next. |
This improved performance of the test case I was using from:
to
on cygwin.
The problem that is fixed here is that sv_setsv_cow() would COW SVs even though they had a very short string in a very large buffer, which had some sort of unclear to me interaction with the Win32 virtual memory system that had a painful performance impact when the large SV was made COW.
It may also improve memory handling on non-Win32 systems, since the SV was made COW, other SV copying functions wouldn't check if it was suitable for COW, so further copies would retain a COW reference for the large buffer. In the example test case from #21877 if we pushed the SV to an array after a successful match memory use would balloon out, running even a system with a generous amount of memory, out of memory.
Fixes part of #21877