Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

heap snapshot: Count the array size on the _buffer_ not on the array. #3

Draft
wants to merge 115 commits into
base: pv-heap-snapshot
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
071bd2a
add empty files
vilterp Sep 16, 2021
381448a
Initial dummy implementation
NHDaly Sep 16, 2021
c108513
dummy print
NHDaly Sep 16, 2021
f8b347e
make dummy callable
NHDaly Sep 16, 2021
6eea92e
Add us to the makefile
NHDaly Sep 16, 2021
7fbd676
set up global heap snapshot and serializer function
vilterp Sep 16, 2021
a5b6a4f
it builds!
vilterp Sep 16, 2021
dfd5b83
fix commas
vilterp Sep 16, 2021
62a24f8
add nodes; print addresses correctly
vilterp Sep 16, 2021
d9feaf8
Plug the snapsshotter into GC fo real do
NHDaly Sep 16, 2021
05c3261
fill out metadata header
vilterp Sep 16, 2021
23ab632
factor out string table
vilterp Sep 16, 2021
75aa2b4
Remove stderr println in record_edge
vilterp Sep 16, 2021
db2361d
Merge remote-tracking branch 'vilterp/pv-heap-snapshot' into pv-heap-…
vilterp Sep 16, 2021
f2fe2cf
add todos
vilterp Sep 16, 2021
43b422b
fix commas
NHDaly Sep 16, 2021
d9db09a
direct output to a stream
vilterp Sep 16, 2021
d14bd89
Merge remote-tracking branch 'vilterp/pv-heap-snapshot' into pv-heap-…
vilterp Sep 16, 2021
27827f8
rename to take_heap_snapshot
vilterp Sep 17, 2021
d41b618
add Julia wrapper function GC.take_heap_snapshot
vilterp Sep 17, 2021
e49be9b
can't use IOStream here
vilterp Sep 17, 2021
b391f44
Add commented out code for getting size once we know the type
NHDaly Sep 17, 2021
65e726c
Make every node an "object" type for now.
NHDaly Sep 17, 2021
9a2f113
add doc comment
vilterp Sep 17, 2021
0488519
trying to get type but also seggfault
NHDaly Sep 17, 2021
5379aad
Merge remote-tracking branch 'vilterp/pv-heap-snapshot' into pv-heap-…
vilterp Sep 17, 2021
7eebc29
update hardcoded string tables
vilterp Sep 17, 2021
b188cc9
Merge remote-tracking branch 'vilterp/pv-heap-snapshot' into pv-heap-…
vilterp Sep 17, 2021
5c1fcb6
Re-enable the (Crashing) type
NHDaly Sep 17, 2021
b4a56eb
guard against nullptr types for non-objects
NHDaly Sep 17, 2021
e8cb07b
Merge branch 'pv-heap-snapshot--nhd-typeof-debugging' into pv-heap-sn…
NHDaly Sep 17, 2021
175a96a
Start to support size and name, but still crashing, so i commented out
NHDaly Sep 17, 2021
b6f863c
don't write out empty arrays which confuse chrome
vilterp Sep 17, 2021
a3fdf7b
Merge remote-tracking branch 'vilterp/pv-heap-snapshot' into pv-heap-…
vilterp Sep 17, 2021
90b9c83
Print node/edge counts & fix memory leak & build warnings
NHDaly Sep 17, 2021
0d6326d
Use node indexes instead of IDs
NHDaly Sep 17, 2021
43463ee
fixup writing indices correctly
NHDaly Sep 17, 2021
1add9a5
Added edge counts to the Node records
NHDaly Sep 17, 2021
f550708
DUH: OBJECTS NEED SIZES TO SHOW UP IN THE HEAP SNAPSHOT!
NHDaly Sep 17, 2021
c89c46e
Get the size iff it's a datatype!! that's progress!
NHDaly Sep 17, 2021
d24cd70
bugfix for DataType size
NHDaly Sep 17, 2021
a13cc11
Tried adding the object type to the snapshots
NHDaly Sep 20, 2021
4282738
get jl_value_t* from jl_taggedvalue_t*
vchuravy Sep 20, 2021
c749fec
Use jl_value_t* to get name and size
vchuravy Sep 20, 2021
5b2561c
Pretty-print type names!!! :D
NHDaly Sep 20, 2021
d8d631a
JSON-escape strings before printing them
NHDaly Sep 20, 2021
5febedc
add assert (fails) that the edges are in order
NHDaly Sep 20, 2021
0591833
add hacked fieldname to gc_debug_edge
NHDaly Sep 20, 2021
61f122b
Change to the real format now that we understand it:
NHDaly Sep 20, 2021
5feabe9
Fix node_type and edge_type string table printing
NHDaly Sep 20, 2021
d3ef567
Fix node_name and edge_name serialization
NHDaly Sep 20, 2021
add6c7f
Add todo list
NHDaly Sep 20, 2021
14bcac5
remove TODO on the bitwise or-ing, it isn't needed
NHDaly Sep 20, 2021
1bd92a1
Re-Split out the debug functions
NHDaly Sep 20, 2021
2b82345
better organize TODO
NHDaly Sep 20, 2021
c3e4a43
Change all GC callsites to use newly named debug functions
NHDaly Sep 20, 2021
93fb3f9
Woohoo~ Restructured code to support last major changes
NHDaly Sep 20, 2021
f732b55
Implement Field Names!!!!!!!!! woohooo
NHDaly Sep 20, 2021
ed1d43c
Update TODO list
NHDaly Sep 20, 2021
f16498c
Fix normal path GC code to only debug when enabled.
NHDaly Sep 21, 2021
d5629b9
Fix major bug in field name string table indexing:
NHDaly Sep 21, 2021
7bca56e
Fix some bugs with registering edges from Modules :)
NHDaly Sep 21, 2021
7375b70
Handle field names of NamedTuples without crashing
NHDaly Sep 21, 2021
2d25439
rename _record_gc_node => _record_gc_edge
vilterp Sep 21, 2021
cc021c5
capture string and symbol values; tag them for viewer
vilterp Sep 21, 2021
a2beb2e
string and symbol sizes
vilterp Sep 21, 2021
fbc740f
add gc annotations
vilterp Sep 22, 2021
6808d3f
add gc-heap-snapshot.h to gc.o dependency list
vchuravy Sep 22, 2021
8422b78
[GCHEAP] add JL_NOTSAFEPOINT
vchuravy Sep 22, 2021
1ba47ab
Disable GC Snapshotting w/ inline check for performance
NHDaly Sep 22, 2021
a108bbe
mark the gc_heap_snapshot_enabled check __unlikely
NHDaly Sep 22, 2021
96321d5
fix annotations on new inline funcs
NHDaly Sep 22, 2021
6156cde
revert to just type name
vilterp Sep 22, 2021
8d6961a
Merge branch 'pv-heap-snapshot-revert-to-typename' into pv-heap-snapshot
vilterp Sep 22, 2021
1eda121
start adding roots
vilterp Sep 22, 2021
d78510a
function to add edges from uber root to roots
vilterp Sep 22, 2021
ebe5311
add uber root and gc roots object
vilterp Sep 22, 2021
e400762
attempt to add subroots node and edges
vilterp Sep 22, 2021
477e631
add internal_root
vchuravy Sep 22, 2021
47c0c3b
Fix segfault in _gc_heap_snapshot_record_root()
NHDaly Sep 23, 2021
d7366b8
Update base/gcutils.jl
vchuravy Sep 23, 2021
53851cc
fix edge count
vilterp Sep 23, 2021
b735687
Merge branch 'pv-heap-snapshot-roots' into pv-heap-snapshot
vilterp Sep 23, 2021
5989742
remove IOStream annotation until we can get it in scope
vilterp Sep 23, 2021
91731e9
make gc root recording conditional on heap snapshotting being on
vilterp Sep 23, 2021
d4cf50f
mark gc_heap_snapshot_record_root as JL_NOTSAFEPOINT
vilterp Sep 23, 2021
e9960d5
better property names for gc roots
vilterp Sep 24, 2021
7bf6552
Apply suggestions from code review
NHDaly Sep 27, 2021
59632d3
more PR feedback
vilterp Sep 28, 2021
a14032a
comments pointing at at Node source
vilterp Sep 28, 2021
cd9a2d3
Finish implementing Jameson's PR feedback:
NHDaly Sep 28, 2021
a3d7aba
use ios_t instead of JL_STREAM
vilterp Sep 28, 2021
ed6d655
Merge remote-tracking branch 'vilterp/pv-heap-snapshot' into pv-heap-…
vilterp Sep 28, 2021
c10babe
print full types
vilterp Sep 28, 2021
542abf7
array sizes
vilterp Sep 30, 2021
4d0bcc6
try marking roots in mark_enqueued_tasks
vilterp Sep 30, 2021
ec0b082
add internal edges when scanning the stack
vilterp Sep 30, 2021
125a668
remove printing from jl_gc_mark_enqueued_tasks
vilterp Sep 30, 2021
fa30b91
remove parens
vilterp Sep 30, 2021
fd07111
represent stack frames as nodes; only root running tasks
vilterp Oct 5, 2021
ec1afc6
Halfway through draft commit to construct full fieldpath from slot
NHDaly Oct 7, 2021
64ac012
Start hooking up the field paths
NHDaly Oct 7, 2021
3236b06
tweak comment
vilterp Oct 7, 2021
af589eb
connect up the code from before (but it's still not working)
NHDaly Oct 13, 2021
ef9b34b
Add debug logging for objects in `Main` module
NHDaly Oct 13, 2021
5578921
Fix silly mistake in slot accounting :)
NHDaly Oct 13, 2021
6889aeb
Clean up debug logs
NHDaly Oct 14, 2021
34b7ed7
Merge branch 'pv-heap-snapshot' into pv-heap-snapshot-slot-fieldpath
NHDaly Oct 14, 2021
c5a20e7
Fixups, comments, todos
NHDaly Oct 14, 2021
90984d7
fix merge conflict
NHDaly Oct 14, 2021
da82125
Fix Tuple types in fieldpath_for_slot
NHDaly Oct 14, 2021
3fbd301
Add TODO about overrunning type printing buffer
NHDaly Oct 14, 2021
063b286
remove log
NHDaly Oct 14, 2021
8a560cd
Merge pull request #1 from vilterp/pv-heap-snapshot-slot-fieldpath
NHDaly Oct 14, 2021
7a104cb
Count the array size on the _buffer_ not on the array.
NHDaly Oct 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/gc-heap-snapshot.cpp
Original file line number Diff line number Diff line change
@@ -94,7 +94,7 @@ struct HeapSnapshot {
vector<Edge> edges;

StringTable names;
StringTable node_types = {"node_type1", "node_type2"};
StringTable node_types = {"object"};
StringTable edge_types = {"edge_type1", "edge_type2"};
unordered_set<size_t> seen_node_ids;
};
@@ -137,7 +137,7 @@ JL_DLLEXPORT void record_node_to_gc_snapshot(jl_value_t *a) {
g_snapshot->seen_node_ids.insert(val, (size_t)a);

Node node{
"", // string type;
"object", // string type;
"", // string name;
(size_t)a, // size_t id;
/*jl_datatype_size(type)*/ 0, // size_t self_size;