JupyterLab user testing for common workflows

[isabela-pf]

Summary

From the beginning, this project was meant to include user testing with disabled users. When/how we would do this user testing has been pending the completion of other work. I'm writing this issue to summarize the user testing goals for the beginning of 2023.

Because the multiple accessibility reviews that have been done on JupyterLab have been done via manual testing (and some browser automated testing) with limited engagement with users, this round of tests will be focused on common tasks completed in JupyterLab. This should include tasks that asks users to navigate the interface, read the interface, navigate documents, read documents, edit documents, and adjust settings. While it may provide more information about known issues or provide feedback on fixes that have been done since past reviews, this is not the primary focus of the tests.

Acceptance Criteria

This issue may be closed when all user testing information for this round is stored for safekeeping and recorded/linked in this issue.

(If you can think of something else, let me know!)

Logistics

Beta Give feedback

1. Timeline and team are cemented
2. Participant recruitment is complete
3. Test logistics finalized
4. All tests have been run

Supporting materials and infra

Beta Give feedback

1. User testing script is complete
2. Development environment is setup
3. Workflow is identified and added to our testing repo https://github.com/Quansight-Labs/JupyterLab-user-testing
4. GDPR and consent forms are completed and checked
5. Testing registration form complete

Analysis and Sharing

Beta Give feedback

1. Process all test data (and store where relevant)
2. All public (and non-identifiable info) is stored for the community

[isabela-pf]

Following up on some offline feedback, there's also interest in ensuring we get feedback about how participants currently use Jupyter notebooks and/or Lab if they can meet their goals there at all.

[isabela-pf]

@trallard here are potential outreach templates for finding or inviting participants. Please let me know if you have any questions or want further changes.

For finding participants

Hi {{name}},

As a part of Quansight Labs’ ongoing work to improve JupyterLab’s accessibility, we are coordinating paid user research to learn more about how usable JupyterLab and Jupyter notebooks are for people who use accessibility accommodations. The results will be made open to the public. When we last spoke, user research opportunities were an overlapping interest. We’re interested in recruiting up to ten research participants, and would be grateful to know if you had any referrals for us to follow up with.

We are looking for the following in participants:

• They use accessibility accommodations (this does not strictly need to be assistive tech).
• They have some familiarity with Jupyter notebooks or JupyterLab. Ideally they would be comfortable completing tasks in those spaces.
• They can have any perceived skill level working in Jupyter tools.

If we reach out, we’ll be asking participants to join us for a paid one-hour session where we ask them to complete tasks in JupyterLab while we observe.

Please don’t hesitate to ask if you have other questions.

Best,  
{{sender}}

For inviting participants

Hi {{name}},

I’m {{sender}}, and I’m reaching out on behalf of Quansight Labs. We’d like to invite you to join us as a paid participant in our upcoming research to learn more about how usable JupyterLab and Jupyter notebooks are for people who use accessibility accommodations. We got your contact information from {{referral}}.

During the one-hour feedback sessions we’ll be asking you to show us how you complete certain tasks with a preset version of JupyterLab. If possible, we would also be interested in seeing how you interact with JupyterLab or notebooks in your own set up. We can offer $100 per hour-long session. If you are interested, let me know and I will follow up with scheduling information.

Please don’t hesitate to ask if you have other questions.

Best,  
{{sender}}