remediated jupyter notebook v7 aria for a better annotaton object model

[tonyfast]

jupyter notebook v7 remediations

this issue provides more structure to the annotation object model in the interactive jupyter notebook v7 experience by modifying javascript in place. this effort is an extension of the notebooks for all effort to establish a minimum, semantic html5 footprint for a rendered notebook. the originial hypothesis still stands that an accessible reference implementation of the rendered notebook will extend to an assistive interactive experience.

i think this remediation is 4 primary tasks

• improved notebook level semantics for landmarks and initial APG recommendations
• introduce landmark semantics for the cells
  • establish input group semantics
  • establish preliminary group semantics

our goal is presentation an assistive experience that can be extended with feedback from users. this will drastically improve the assistive tech experience through improvements to the annotation object model.

Note: all labels will need to be internationalized.

out of scope

this documents focus on non-visual changes that improve the quality of Annotation Objective Model so the following important topics are out of scope:

• the split panel handler has an APG Window Splitter Pattern that should be implemented.
• tab traps are bad for keyboard users. tab in cell mode should not enter edit mode automatically. the assistive experience hinges on Fix tab trap notebook cells jupyterlab/jupyterlab#14115.
• modifying the native find experience without an escape hatch is super dangerous; so i'm real worried about .jp-DocumentSearch-overlay.

click "trust notebook" in the File Menu to execute this code yourself.

application level semantics

ARIA modifications to the first landmarks we encounter.

%%javascript
document.querySelectorAll("#top-panel").forEach((x,i) =>{x.setAttribute("aria-label", "Notebook Menu");});
document.querySelectorAll("#main-panel").forEach((x,i) =>{x.setAttribute("aria-labelledby", "jp-title");});
document.querySelectorAll("#menu-panel").forEach((x,i) =>{
    x.setAttribute("role", "banner"); 
    x.setAttribute("aria-label", "Notebook Menu");
});

the application can't own the first h1. we've been suggesting notebooks begin with a markdown cell and h1 as a best practice.

%%javascript
for (x in document.querySelectorAll("#top-panel #jp-title h1").children){
    // we can't set our own h1 as that will mess up custom content.
    // the css depends on the h1, but the screen reader won't find it this way.
    x.setAttribute("role", "presentation");
};

application label concerns.

%%javascript
// a user should be able to trigger the logo dialog by clicking on .jp-NotebookKernelLogo
// the trusted status button needs alt and a clearer title .jp-NotebookTrustedStatus
// the button.jp-Notebook-footer semantics are wrong. the button should be inside a footer, or role=contentinfo

i'm surprised everytime the jupyter logo sends me to the file tree. right now, the first thing you tab to tries to send you to another page. feels weird.

the changes below communicate the current semantics, but the feel of this experience should be reconsidered.

%%javascript
// it is going to make no sense to anyone that the jupyter label sends you to open files
document.querySelectorAll("#top-panel #jp-NotebookLogo").forEach((x,i) =>{
    x.setAttribute("title", "Open File Browser in a New Tab");
    x.setAttribute("alt", "Open File Browser in a New Tab");
});

label the main notebook area.

%%javascript
document.querySelectorAll("#spacer-widget-top").forEach((x,i) =>{x.setAttribute("role", "presentation");});
document.querySelectorAll(".jp-Notebook").forEach((x,i) =>{x.setAttribute("aria-label", "Notebook Cells");});

the toolbar is banner, not navigation.

%%javascript
// role navigation -> banner, this is a landmark inside of main
document.querySelectorAll(".jp-Toolbar").forEach((x,i) =>{
    x.setAttribute("role", "banner"); x.setAttribute("aria-label", "Notebook Actions");
});    

the notebook cell regions

it was found, in a single notebook mode, that some assistive tech users liked finding cells as landmarks especially in long notebooks. we can add more complicated semantics later, the feed pattern makes more sense in reading html notebooks.

the feed pattern is read mode pattern for identifying units of content is a potentially infinite scrolling element. we use this pattern because it introduces the minimal aria semantics to add order.

%%javascript
document.querySelectorAll(".jp-WindowedPanel-window").forEach((x,i) =>{
    x.setAttribute("role", "feed"); x.setAttribute("aria-label", "Cells");
});

%%javascript
var cells = document.querySelectorAll(".jp-Cell");
cells.forEach((x,i) =>{
    // lets punt on the feed pattern to start
    // x.setAttribute("role", "article"); 
    // x.setAttribute("aria-posinset", i+1); 
    // x.setAttribute("aria-setsize", cells.length); 

    // use a raw region to make sure the cells show as landmarks
    x.setAttribute("role", "region"); 
    
    var MD = x.className.includes("jp-MarkdownCell");
    var CODE = x.className.includes("jp-CodeCell");
    if (CODE){
        x.setAttribute("aria-label", `Code Cell ${i+1}`);
    } else if (MD){
        x.setAttribute("aria-label", `Markdown Cell ${i+1}`);
    } else {
        x.setAttribute("aria-label", `Cell ${i+1}`);
    };

    // i cant directly know the input number. semantically it is some kind of label.
    // this effort is concerned with the AOM, not WCAG. 
    // we do have a visible label so this should satisfy the visible label
    prompt = x.querySelectorAll(".jp-InputPrompt")[0];
    prompt.setAttribute("aria-hidden", "true");
});

input area fixes

%%javascript
var cells = document.querySelectorAll(".jp-Cell");
cells.forEach((x,i) =>{
    var MD = x.className.includes("jp-MarkdownCell");
    var CODE = x.className.includes("jp-CodeCell");

    // code, markdown, raw cells need different groupings.
    x.querySelectorAll(".jp-Cell-inputWrapper").forEach(
        (y)=>{
            y.setAttribute("role", "group");
            var label = prompt.textContent.slice(1, -2).trim();
            if (CODE){
                if (label.length){
                    y.setAttribute("aria-label", `In ${label}`);
                } else {
                    y.setAttribute("aria-label", `New Code Input`);
                }
            } else if (MD){
                y.setAttribute("aria-label", `Markdown Input`);
            } else {
                y.setAttribute("aria-label", `Input`);
            }
        }
    );
    // make the cell toolbar an APG styled toolbar. this is the most cursory modification.
    x.querySelectorAll(".jp-cell-toolbar").forEach(
        (y)=>{
            // https://www.w3.org/WAI/ARIA/apg/patterns/toolbar/examples/toolbar/
            y.setAttribute("role", "toolbar");
            y.setAttribute("aria-label", `Cell ${i+1} Toolbar`);
            y.setAttribute("aria-controls", `cell-input-${i}`);
        }
    )
});

output area fixes

i am very not confident about the output semantics. my efforts were to understand the rendered cell's landmarks. the output semantics are pure presentation in reading mode, but editting mode is several HCI PhDs.

right now, i'm only comfortable calling the outputs a group, this means they may be announced by a screen reader to provide some relative position. on disk, in the notebook format, outputs are a list of structures, as are cells, and it might make the most sense to employ a nested feed pattern. when you start to thinking about async updates then ordering is out the window.

%%javascript
var cells = document.querySelectorAll(".jp-Cell");
cells.forEach((x,i) =>{
    x.querySelectorAll(".jp-Cell-outputWrapper").forEach(
        (y)=>{
            // this will only be encountered on code cells.
            // i have no invested significant time into the semantics of the outputs
            // group is a fine semantic, but it is non specific.
            // increasingly i feel like the output is nested feed because it 
            // would allow outputs to be indexed. need more testing.
            y.setAttribute("role", "group");
            var label = prompt.textContent.slice(1, -2).trim();
            if (label.length){
                y.setAttribute("aria-label", `Output ${label}`);
            } else {
                // no need to worry about the output label when there is not execution
                // aria hidden might be appropriate without testing on a screen reader.
            };
        }
    );
    x.querySelectorAll(".jp-OutputCollapser").forEach(
        (y)=>{
            // the collapser button has no semantics or visible label.
            // this feature is not accessible, but should because for folks using screen magnification.
        }
    )
});

conclusion

• the output area is where the research stops. it is less though out than the others. announcements are part of the equation too for the proper assistive experience.
• there are significant follow up work to improve the whole experience.