First Click Testing of Search Bar feature, using CKAN benchmarking

[Aparna1Gopal]

Dependencies

• Prioritization Workshop - Identifying the Information Architecture (IA) for Search Bar feature, using CKAN benchmarking  #274

Overview

• The Information Architecture (IA) of the 'Search Bar feature' has been built based on CKAN benchmarking & the team's collaborative prioritisation workshops

• The Search Bar features currently prioritised are documented in this Wiki

Action Items

• Review Wiki - IA for Search Bar feature
• Complete Research Plan
• Identify resources available across free plan of usability testing tools in the market (Example: Optimal Workshop, Usability Hub, Maze)
• Select Usability tool most appropriate for study from available market options
• Create Pilot Usability Test
• Review Research Plan with Team
• Pilot Test with Team
• Review & revise based on team feedback
• Create participant recruitment Slack template message & share with team for feedback
• Recruit research participants
• Close & complete usability testing once required sample size is reached
• Analyse data from the usability testing
• Identify research findings, insights, actionable recommendations
• Present the research findings to the team & discuss next steps
• Update Wiki
• Review with PMs & close issue

Resources/Instructions

• Google Drive
• Wiki
• First Testing Guide
• UX task flows vs. user flows

TEMPLATE FOR PROJECT PROGRESS

Progress: "What is the current status of your project? What have you completed and what is left to do?"
Blockers: "Difficulties or errors encountered."
Availability: "How much time will you have this week to work on this issue?"
ETA: "When do you expect this issue to be completed?"
Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Started working on the Research Plan
• Identified the tool to conduct the Tree Testing - Optimal Workshop

Blockers: "Difficulties or errors encountered."
None

Availability: "How much time will you have this week to work on this issue?"
3-4 hours

ETA: "When do you expect this issue to be completed?"
2-3 weeks including participant recruitment, data collection & data analysis

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Working on the end to end Research Plan
• Setting up study: Tree Testing in Optimal Workshop

Blockers: "Difficulties or errors encountered."
Due to free version of usability testing tool, need to work within the constraints (as number of studies/participants/study questions) are limiting. Hence prioritising accordingly.

Availability: "How much time will you have this week to work on this issue?"
3-4 hours

ETA: "When do you expect this issue to be completed?"
2-3 weeks including participant recruitment, data collection & data analysis

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Working on the end to end Research Plan
• Setting up study: First click testing in Optimal Workshop
• Pilot Test the study with the team
• Dig through the previous studies conducted regarding personas & identify the appropriate survey question to track data expert versus data novice's experience with using data i.e. repurposing an already available & researched survey question:
  • https://docs.google.com/document/d/1PTEBRiPYWQarMyXMwCxGZy_OqW4qu5MSGfqCX7kZSxM/edit
  • Benchmarking task flows of data points [by conducting a comparative analysis of CKAN websites] #192 (comment)

Blockers: "Difficulties or errors encountered."

• Due to free version of usability testing tool, need to work within the constraints (as number of studies/participants/study questions) are limited. Hence prioritising accordingly.
• Due to the free version of usability testing tool & the constraints, the usability testing method needed to be changed from tree testing to first click testing to accomodate the constants, yet yield rich impactful research results & insights.

Availability: "How much time will you have this week to work on this issue?"
3-4 hours

ETA: "When do you expect this issue to be completed?"
2-3 weeks including participant recruitment, data collection & data analysis

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[merlinsmagic]

Hi @Aparna1Gopal do you have any updates?

Progress: “What is the current status of your project? What have you completed and what is left to do?”
Blockers: “Difficulties or errors encountered.”
Availability: “How much time will you have this week to work on this issue?”
ETA: “When do you expect this issue to be completed?”
Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Reviewed required in progress items on First Click Testing of Search Bar feature, using CKAN benchmarking #282 (comment)
• Completed Research Plan
• Based on feedback & review at last team meeting, revised the Usability Test. Updated this on the Research Plan & Optimal Workshop (Usability testing tool)
• Completed Slack recruitment message. Need to review this message with Team during meeting, revise & start recruiting research participants.
• Ask for feedback on recruitment methods & specific channels that can be used for recruitment. Ask for team's help with cross-posting this recruitment message across their known networks to get a diverse pool of research participants.
• During team meeting, as for the following feedback:
  • Slack research participant recruitment message: First Click Testing of Search Bar feature, using CKAN benchmarking #282 (comment)
  • What is the difference between these image options on the usability test. Which will work best to test ATD product (Based on review with team - use scale to fit option)

Blockers: "Difficulties or errors encountered."

• Due to free version of usability testing tool, need to work within the constraints (as number of studies/participants/study questions) are limited. Hence prioritising accordingly. Discussed constraints & necessary pivots with team

Availability: "How much time will you have this week to work on this issue?"
3-4 hours

ETA: "When do you expect this issue to be completed?"
2-3 weeks including participant recruitment, data collection & data analysis

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Slack Research Participant Recruitment Message

Review content with team

• Reviewed - Decided on OPTION 2

👋Hi! We would like your help with some user testing ⬇️
This would take only 2-3 minutes of your time. Many thanks!
Project: Access the Data, a product of Hack for LA
(Provide test link)

OPTION 1

(OR)

OPTION 2

[Aparna1Gopal]

Slack Participant Recruitment Message

👋 Hi! We would like your help with some user testing ⬇️
⏰ This would take only 2-3 minutes of your time. Many thanks!
📝 Project: Access the Data, a product of Hack for LA

https://izbes7gi.optimalworkshop.com/chalkmark/1fdhmznd

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Revised the Usability Test, Research Plan & Slack Message based on feedback during team meeting
• Recruiting research participants via Slack. Currently posted on HfLA's slack channels - Communities of Practice: UI/UX, PM, Engineering, Data Science, Ops & General. To get a diverse participant group. Slack Message: First Click Testing of Search Bar feature, using CKAN benchmarking #282 (comment)
• Posted Slack recruitment message specifically in the middle of the work work, to get better traction on participants responding to the survey/test
• Will need to iterate based on response rate & number of participants recruited.

Blockers: "Difficulties or errors encountered."

• Research participant recruitment

Availability: "How much time will you have this week to work on this issue?"
2-3 hours

ETA: "When do you expect this issue to be completed?"
2-3 weeks including participant recruitment, data collection & data analysis
Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Notes

For Report Writing

https://openfieldx.com/ux-discovery-activities-user-journey-mapping/#:~:text=Task%20flows%20focus%20on%20steps,real%20decision%20points%20and%20actions.

https://www.simplilearn.com/tutorials/ui-ux-career-resources/user-flows-in-ux#:~:text=There%20are%20three%20types%20of,wire%20flows%2C%20and%20user%20flows.

https://usabilityhub.com/guides/first-click-testing

https://blog.optimalworkshop.com/first-click-testing/

https://www.userinterviews.com/ux-research-field-guide-chapter/first-click-testing

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Completed recruiting participants for the usability testing. Link to test here
• Closed test
• Need to start analysing results
• Write report & present results

Blockers: "Difficulties or errors encountered."

• None. It was surprisingly easy to recruit participants this time as compared to the extensive recruitment that the team did for personas (community activist/data novice), with zero traction.
• Reflections: The recruitment for required number of participants was complete within 24 hours. It is also a testimony of having recruited through the right networking channels [Especially, since based on past experience with participant recruitment, this was seen as a challenging task]

Availability: "How much time will you have this week to work on this issue?"
4-5 hours

ETA: "When do you expect this issue to be completed?"
2-3 weeks

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Notes

Resources/Guide to user journey mapping

https://webflow.com/blog/user-journey

https://careerfoundry.com/en/blog/ux-design/how-to-make-customer-journey-map-with-template/

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Started analysing results
• Collecting resources for project report & deliverables - First Click Testing of Search Bar feature, using CKAN benchmarking #282 (comment)

Blockers: "Difficulties or errors encountered."
None.
Identifying appropriate deliverables & time taken to complete the same

Availability: "How much time will you have this week to work on this issue?"
4-5 hours

ETA: "When do you expect this issue to be completed?"
2 weeks

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

• Analysing results, cross-referencing it with the Research Plan
• Collecting resources for project report & deliverables - First Click Testing of Search Bar feature, using CKAN benchmarking #282 (comment)

Blockers: "Difficulties or errors encountered."
None.
Identifying appropriate deliverables & time taken to complete the same

Availability: "How much time will you have this week to work on this issue?"
4-5 hours

ETA: "When do you expect this issue to be completed?"
2 weeks

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[Aparna1Gopal]

Progress: "What is the current status of your project? What have you completed and what is left to do?"

Analysing results, cross-referencing it with the Research Plan
Collecting resources for project report & deliverables - #282 (comment)
Woke simultaneously on the UXR Roadmap for Semester 2, 2023 simultaneously, so time was distributed across these two projects.

Blockers: "Difficulties or errors encountered."
None.
Identifying appropriate deliverables & time taken to complete the same

Availability: "How much time will you have this week to work on this issue?"
4-5 hours

ETA: "When do you expect this issue to be completed?"
2 weeks

Pictures: "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either:

ask for help at your next meeting
put a "Status: Help Wanted" label on your issue and pull request
put up a request for assistance on the #access-the-data channel.

[merlinsmagic]

Hi @Aparna1Gopal ,

Do we have any new updates for this issue?

Progress: “What is the current status of your project? What have you completed and what is left to do?”
Blockers: “Difficulties or errors encountered.”
Availability: “How much time will you have this week to work on this issue?”
ETA: “When do you expect this issue to be completed?”
Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”

• Completing data analysis, identifying clear trends & patterns
• Started the research report

Blockers: “Difficulties or errors encountered.”
None

Availability: “How much time will you have this week to work on this issue?”
2-3 hours

ETA: “When do you expect this issue to be completed?”
2 weeks (aiming for a research presentation of findings last week of August)

Pictures: “Add any pictures of the visual changes made to the site so far.

Data Collection-Analysis Snapshot

Draft Research Report notes Snapshot

[merlinsmagic]

Hi @Aparna1Gopal do we have new updates for this issue?

Progress: “What is the current status of your project? What have you completed and what is left to do?”
Blockers: “Difficulties or errors encountered.”
Availability: “How much time will you have this week to work on this issue?”
ETA: “When do you expect this issue to be completed?”
Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”

• Completing data analysis, identifying clear trends & patterns
• Started the research report

Blockers: “Difficulties or errors encountered.”
None

Availability: “How much time will you have this week to work on this issue?”
2-3 hours

ETA: “When do you expect this issue to be completed?”
1-2 weeks (aiming for a research presentation of preliminary findings in the last week of August)

Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”

• Completed data analysis
• Started the research report
• Left to do: Research Presentation to present to team, Research Report, complete Wiki

Blockers: “Difficulties or errors encountered.”
None

Availability: “How much time will you have this week to work on this issue?”
1 hour (as traveling this upcoming week)

ETA: “When do you expect this issue to be completed?”
2 weeks (aiming for a research presentation of preliminary findings in the second week of September), as traveling in the first week of September

Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Notes
Resource to draft a comprehensive Research Report

• How to run a quantitative usability test - And use it to continuously prove your impact
  NIKKI ANDERSON-STANIER

• Research Report on Usability Testing

• User journey mapping

• How to create high-converting landing pages in minutes, Love to Grow Newsletter by Hotjar with Hubspot

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”

• Completed: Data analysis, Started the research report
• Left to do: Research Presentation to present to team, Research Report, Complete Wiki

Blockers: “Difficulties or errors encountered.”
Traveling, thus time constraints

Availability: “How much time will you have this week to work on this issue?”
3-4 hours

ETA: “When do you expect this issue to be completed?”
1 week (aiming for a research presentation of research findings & next steps for MVP)

Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”
Completed: Data analysis, Research findings presented to team today
Left to do: Research Report + Wiki + Documentation in the google drive

Blockers: “Difficulties or errors encountered.”
Traveling, thus time constraints

Availability: “How much time will you have this week to work on this issue?”
3-4 hours

ETA: “When do you expect this issue to be completed?”
1-2 weeks

Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”
Completed: Research findings presented to team today
Left to do: Started Wiki, Updating all Documentation in the google drive

Blockers: “Difficulties or errors encountered.”
Traveling last week, thus time constraints

Availability: “How much time will you have this week to work on this issue?”
3-4 hours

ETA: “When do you expect this issue to be completed?”
1 week

Pictures: “Add any pictures of the visual changes made to the site so far.

[mxajPrice]

Hi @Aparna1Gopal, were you able to work on the wiki this week?

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”
Completed: Data collection, analysis, research findings & next steps presented to the team
Left to do:

• Updating all relevant documentation in the ATD google drive. Review all documentation. Download documentation from all relevant testing tools that were used to conduct the research.
• Complete Wiki
• Review the Wiki with PM & close issue

Blockers: “Difficulties or errors encountered.”
None

Availability: “How much time will you have this week to work on this issue?”
3-4 hours

ETA: “When do you expect this issue to be completed?”
1 week

Pictures: “Add any pictures of the visual changes made to the site so far.

[Aparna1Gopal]

Progress: “What is the current status of your project? What have you completed and what is left to do?”
Completed:

• Updating all relevant documentation in the ATD google drive. Review all documentation. Download documentation from all relevant testing tools that were used to conduct the research.
• Complete Wiki
• Review the Wiki with PM & close issue

Blockers: “Difficulties or errors encountered.”

Availability: “How much time will you have this week to work on this issue?”

ETA: “When do you expect this issue to be completed?”
The issue is complete and can be closed

Pictures: “Add any pictures of the visual changes made to the site so far.

[mxajPrice]

Wiki looks good, great job!