Think Aloud Log

[smcgregor]

I did a demonstration "think aloud" study with someone and recorded the log below. There are several observations I bolded that may warrant opening an issue.

• Direction given: You are a deepfakes researcher in politics directed to find something called the AI Incident Database and learn more about the deepfakes in politics problem.
• <Searches Google for "AI Incident Database">
• "OK there is a database that tracks AI incidents, if I am tracking deepfakes, then this sounds like what I need" Note: most people would not Google this directly, but this is the only bit that is very unrealistic in this log.
• <clicks through to the landing page>
• <sees incident 802, scrolls around page, surveys context>
• <explores the nav>
• "I am going to go through the search"
• <searches "deepfakes politics">
• "Donald trump comes up, but none of the results have the word politics" (follow up indicates curiosity on how the search index was working since "politics" was not explicitly represented. Was wondering if there was any tagging capacity.)
• <Looks at the filters>
• <Clicked on "Classifications", but does not understand what any of these are. Follow up: "maybe tags, or similar".> Sean Note: CSETv1 will make sense to almost nobody when presented by it within the UI for the first time.
• "it isn't clear how search terms relate to what is returned, since many of the results don't contain the 'politics query'"
• <clicks through to one of the incidents> "It isn't clear why it scrolled" <to the incident report, it is not clear that there is a difference between an incident and a report>
• "I am thrown off by being scrolled all the way to the bottom"
• "that's cool" <entities section, clicks through to it and explores the entities for unknown-deepfake-developer-***>
• <goes back to the incident page>
• "timeline, I like that, so I can see when this happened immediately"
• <views the jumbotron> "I don't know why this is so in your face with this background, it is like an entire screen"..."clicking on the picture doesn't even do anything, you need to click on the title to get through"
• <the reports section> "the collection or the grouping gets broken up by having the big jumbotron"
• <variants> Not sure what this has to do with the incident. "this could be a sidebar or something"..."I don't know why I would add one"