volshell: change dt() output to show where pointers lead #1028
Conversation
|
So, I'm interested to know how people would use Also might people start wanting something similar for arrays and/or arrays of pointers, etc? |
|
Thanks @ikelos! I always appericate your feedback. These are just mock ups rather any any code behind this to just hash out what it could look like. At the moment if you use We could change that to do something like this to show the actual values. Which even on it's own I think its a useful change. If we did that, it would mean we could do something like this to indent when following pointers to the struct. I think this seems better than the Also if it wasn't clear in my first explantation when the display adds the e.g. without this change: with this change: I do think people might want to do something similar for arrays too, I could try adding that after this? |
|
Oh, for some reason I thought we did already show the values by the side of a type, but I guess only if there are members? Yeah, that looks a reasonable solution, but we proabably only want to descend the tree until we hit a non-pointer (or worse, a loop), otherwise we might keep going forever. I've changed this to a draft pull so I know it's not ready to be merged yet. Should be easy to change back when we're ready... |
|
Hello again, I've made some changes so it works in the way described above. What do you think? I'm less sure my python is up to standard now however... There is now a check to stop infinite loops, it'll only deference 8 times. I'd really hope that would mean all correct use of pointers is handled with a big margin for error, and its small enough to not be a huge problem if we do get stuck in an inifite loop. I've made changes so that if an object of type with no members is given to dt(), it now outputs the value as well. e.g. When a task which has members is given, the value comes from object.vol.offset, e.g. I've also added a special case for pointers, when they are displayed it is now shown in hex, previous this came back as an int. e.g. Now when Lastly if we follow a few pointers it keeps indenting until we find the struct, find a pointer we cant follow, or we hit the limit of following 8 pointers. |
| # if we aren't able to follow the pointers anymore then there will | ||
| # be no more information to display as we've already printed the | ||
| # details of this pointer | ||
| return | ||
|
|
||
| if hasattr(volobject.vol, "members"): |
There was a problem hiding this comment.
Not sure this is a good way of telling a struct from a pointer? Can't immediately think of how to do it, but I think there is a single overarching struct type that covers the 3 different children, probably better to isinstance that?
There was a problem hiding this comment.
Great question - I'll dig into it.
There was a problem hiding this comment.
I've been exploring this issue and found that when dealing with actual objects or types with offsets, checking if it's an instance of objects.AggregateType works really well. 👍
However, I ran into difficulties when working with templates, such as when executing dt('task_struct') without any offsets. Both cases appear as ObjectTemplate types, making it challenging to distinguish between them with type checks:
(layer_name) >>> type(self.context.symbol_space.get_type('symbol_table_name1!task_struct'))
<class 'volatility3.framework.objects.templates.ObjectTemplate'>
(layer_name) >>> type(self.context.symbol_space.get_type('symbol_table_name1!int'))
<class 'volatility3.framework.objects.templates.ObjectTemplate'>I found .vol.object_class promising for getting the types:
(layer_name) >>> self.context.symbol_space.get_type('symbol_table_name1!task_struct').vol.object_class
<class 'volatility3.framework.symbols.linux.extensions.task_struct'>
(layer_name) >>> self.context.symbol_space.get_type('symbol_table_name1!int').vol.object_class
<class 'volatility3.framework.objects.Integer'>However, .vol.object_class ends up as abc.ABCMeta, making isinstance checks infeasible as far as I understand. Which is kind of understandable since they're not actual instances.
(layer_name) >>> type(self.context.symbol_space.get_type('symbol_table_name1!task_struct').vol.object_class)
<class 'abc.ABCMeta'>
(layer_name) >>> type(self.context.symbol_space.get_type('symbol_table_name1!int').vol.object_class)
<class 'abc.ABCMeta'>I've looked for solutions and found some discussions like this one on StackOverflow, but I'm still uncertain about the best approach.
It might be possible to adjust the templates to facilitate isinstance checks, but it's currently beyond my expertise. 😕 For now, I've resorted to using hasattr to differentiate between types, though it feels like a workaround:
(layer_name) >>> hasattr(self.context.symbol_space.get_type('symbol_table_name1!task_struct'), "members")
True
(layer_name) >>> hasattr(self.context.symbol_space.get_type('symbol_table_name1!int'), "members")
FalseI'm open to suggestions or guidance on how to approach this more effectively. 🙃 Let me know if you have any good ideas!
…es_pointer_upgrade
… both where the pointer is and where it is pointing to. Add extra output for null or unreadable pointers.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Hello @ikelos - I think this is ready for review again. As a recap as it's been a few months this PR is all about trying to make it easier in volshell to use It's the difference between knowing that I've updated it now to display pointers better, showing both their location and where they point to. Hopefully it's more obvious and user friendly now. It also include messages for null pointers or pointers that can't be followed. Simple things still work as expected: (layer_name) >>> dt('void')
symbol_table_name1!void
(layer_name) >>> dt('int')
symbol_table_name1!int (4 bytes, little endian, signed)Structs that include pointers show they're pointers with the (layer_name) >>> dt('task_struct')
symbol_table_name1!task_struct (1784 bytes):
0x0 : state symbol_table_name1!long int
0x8 : stack *symbol_table_name1!void
0x10 : usage symbol_table_name1!unnamed_4c3f6f38ad08303b
<snip>
0x1a8 : mm *symbol_table_name1!mm_struct
0x1b0 : active_mm *symbol_table_name1!mm_struct
<snip>With an actual object we get the values too and the header shows where the struct is located much like the repr for objects. (this is more useful later when we're following pointers more). You can see here that the (layer_name) >>> dt(task)
symbol_table_name1!task_struct (1784 bytes) @ 0x8800044a5880:
0x0 : state symbol_table_name1!long int 1
0x8 : stack *symbol_table_name1!void 0x880001ed2000
0x10 : usage symbol_table_name1!unnamed_4c3f6f38ad08303b 0x8800044a5890
0x14 : flags symbol_table_name1!unsigned int 4202752
0x18 : ptrace symbol_table_name1!unsigned int 0
0x20 : wake_entry symbol_table_name1!llist_node 0x8800044a58a0
0x28 : on_cpu symbol_table_name1!int 0
<snip>
0x688 : splice_pipe *symbol_table_name1!pipe_inode_info 0x0 (null pointer)
<snip>This is works when just giving a type and an offset as you'd expected. (layer_name) >>> dt('task_struct', task.vol.offset)
symbol_table_name1!task_struct (1784 bytes) @ 0x8800044a5880:
0x0 : state symbol_table_name1!long int 1
0x8 : stack *symbol_table_name1!void 0x880001ed2000
0x10 : usage symbol_table_name1!unnamed_4c3f6f38ad08303b 0x8800044a5890
0x14 : flags symbol_table_name1!unsigned int 4202752
0x18 : ptrace symbol_table_name1!unsigned int 0
0x20 : wake_entry symbol_table_name1!llist_node 0x8800044a58a0
0x28 : on_cpu symbol_table_name1!int 0
<snip>Here is an example where it's parsing some invalid data - you can see that the (layer_name) >>> dt('task_struct', task.vol.offset - 1)
symbol_table_name1!task_struct (1784 bytes) @ 0x8800044a587f:
0x0 : state symbol_table_name1!long int 256
0x8 : stack *symbol_table_name1!void 0x1ed200000 (unreadable pointer)
0x10 : usage symbol_table_name1!unnamed_4c3f6f38ad08303b 0x8800044a588f
0x14 : flags symbol_table_name1!unsigned int 1075904512
0x18 : ptrace symbol_table_name1!unsigned int 0
0x20 : wake_entry symbol_table_name1!llist_node 0x8800044a589f
<snip>Here is an example of following a pointer right away to get to the actual struct. It's showing that the pointer located at (layer_name) >>> dt(task.files)
symbol_table_name1!pointer (8 bytes) @ 0x8800044a5d00 -> 0x88001f5bc980
symbol_table_name1!files_struct (704 bytes) @ 0x88001f5bc980:
0x0 : count symbol_table_name1!unnamed_4c3f6f38ad08303b 0x88001f5bc980
0x8 : fdt *symbol_table_name1!fdtable 0x88001cbff400
0x10 : fdtab symbol_table_name1!fdtable 0x88001f5bc990
0x80 : file_lock symbol_table_name1!spinlock 0x88001f5bca00
0x84 : next_fd symbol_table_name1!int 3
0x88 : close_on_exec_init symbol_table_name1!embedded_fd_set 0x88001f5bca08
0x90 : open_fds_init symbol_table_name1!embedded_fd_set 0x88001f5bca10
<snip>Here is an example of following a double pointer: (layer_name) >>> dt(task.files.fdt.fd)
symbol_table_name1!pointer (8 bytes) @ 0x88001cbff408 -> 0x88001c4c2800
symbol_table_name1!pointer (8 bytes) @ 0x88001c4c2800 -> 0x8800044c6880
symbol_table_name1!file (208 bytes) @ 0x8800044c6880
0x0 : f_u symbol_table_name1!unnamed_6733425fe3c7f4c9 0x8800044c6880
0x10 : f_path symbol_table_name1!path 0x8800044c6890
0x20 : f_op *symbol_table_name1!file_operations 0xffff814378d0
0x28 : f_lock symbol_table_name1!spinlock 0x8800044c68a8
0x2c : f_sb_list_cpu symbol_table_name1!int 0
0x30 : f_count symbol_table_name1!unnamed_ab9338acf0e8cd0d 0x8800044c68b0
0x38 : f_flags symbol_table_name1!unsigned int 32770
<snip>Here is an example where trying to following the pointers to the struct fails. The "null pointer" message there hopefully making it clear why no more information is displayed. In the past you'd have used (layer_name) >>> dt(task.files.fdt.fd)
symbol_table_name1!pointer (8 bytes) @ 0xffff8162c858 -> 0xffff8162c8d8
symbol_table_name1!pointer (8 bytes) @ 0xffff8162c8d8 -> 0x0 (null pointer)To make that clear here is the raw data behind the two pointers in that last example: (layer_name) >>> dq(0xffff8162c858, 8)
0xffff8162c858 ffffffff8162c8d8 .....b..
(layer_name) >>> dq(0xffff8162c8d8, 8)
0xffff8162c8d8 0000000000000000 ........Sorry for the huge comments.... I just wanted to be as clear as I could be. If there is anything I've missed out or isn't clear please let me know. Thanks! |
| a pointer otherwise it returns just the normal type name.""" | ||
| pointer_marker = "*" * depth | ||
| try: | ||
| if member_type.vol.object_class == objects.Pointer: |
There was a problem hiding this comment.
might be better to use isinstance than equality? It's interesting this is a different test than the isinstance below used for display_type They should probably match to avoid weird discrepancies! 5:P
| else: | ||
| return member_type_name | ||
| except AttributeError: | ||
| pass # not all objects get a `object_class`, and those that don't are not pointers. |
There was a problem hiding this comment.
They should all get a type_name, but then you're having to do string comparisons (although pointer should be a built-in type, so the name shouldn't change?). Happy with either route...
|
Hey @ikelos I was looking into why I wasn't using an e.g. here the (layer_name) >>> dt(task.mm)
symbol_table_name1!pointer (8 bytes) @ 0x8800044a5a28 -> 0x88001f1bd840
symbol_table_name1!mm_struct (920 bytes) @ 0x88001f1bd840:
0x0 : mmap *symbol_table_name1!vm_area_struct 0x88001b6b4818
0x8 : mm_rb symbol_table_name1!rb_root 0x88001f1bd848
0x10 : mmap_cache *symbol_table_name1!vm_area_struct 0x88001b5d1ad8Where as here they aren't actually real pointers so the (layer_name) >>> dt('mm_struct')
symbol_table_name1!mm_struct (920 bytes):
0x0 : mmap *symbol_table_name1!vm_area_struct
0x8 : mm_rb symbol_table_name1!rb_root
0x10 : mmap_cache *symbol_table_name1!vm_area_structI cant work out a different way of doing it at the moment. So either leave it as it is with the Also - I wanted to ask if there where any version numbers to update anywhere? Does volshell count as part of the framework so need to up the minor number? (Sorry - I'm becoming a bit of a version number addict) |
|
Nope, volshell is a separate utility. I guess we could update its version number but it's far less important (it isn't expected for people to depend on volshell in its own right). I guess we could give volshell its own version number, but that might get complicated (given it kinda comes packaged with the framework). You can always jump the patch number on the framework if you like, but it really isn't an API change k particularly since it's only in the display of output, not of the actual function or operation of volshell). The version numbers shouldn't be an all consuming thing, they're just to make sure people can easily tell if their puzzle pieces can fit together properly... 5;) As to the other piece, operating on a type name rather than an object is fine, but we should try to keep the checks the same, so either both operate on the name or both operate on the type. If only operating on the type name, it should be possible to split on bang and check for "pointer" at the end, but that might make inheritance of pointer in the future more difficult (I don't immediately forsee a need for this, but as soon as I say that someone will come up with one, so again, just something to be aware of and document somewhere)... 5:) |
|
Makes sense re versions. I'll play around with those checks a bit more - i think I get it now. 😀 |
Hello,
In volshel using the
display_typecommand to show information about objects can be difficult when pointers are involved.For example the output of a struct with pointers in tells you that certain members are pointers, but not what type of struct it actually is. If you want to use see that struct you need to add
.dereference()which sometimes feels cumbersome. (I know I misspell dereference a lot! I even managed to do so while trying to make this PR... d5e0dec)These changes are to make it easier to use volshell interactively while you are exploring and understanding what there is to be found.
Without changes
Here is an example volshell session without the changes to show how this helps when using volshell interactively. Note how here when we use
dt()on atask_structobject we don't know what kind of structstack,fs,files, etc are only that they are pointers. The result just showssymbol_table_name1!pointerIf we try to access
task.fileswe need to manually add the.dereference()Again to access
task.files.fdtwe have to manually add.dereference()Now lastly to access
task.files.fdt.fdwe need to dereference twice until we find the result we were looking for.With changes
Here is the same session with these changes. Now when we use
dt()on atask_structobject we can see which members are pointers as they are marked with a*as you'd normally see in c, and we also get to see what type these members are. For example here we can see thatfilesis a pointer to afiles_structeasily.Now when we attempt to follow these pointers in volshell it is much easier to to so as we don't need to add the
.dereference()ourselves. (Although if we did add a.dereference()ourselves it would still work as normal).The output shows
dereferenced onceto show that we followed a pointer to get to thefiles_struct.Now when showing
task.files.fdtwe can see right away thatfdis actually a double pointer to afile.Finally accessing
task.files.fdt.fdis just as easy even though we need to follow a double pointer to get to the result.I know I personally find these changes very helpful, I would love to have your thoughts about these suggestions.