Checkbox questions with "Other" should allow adding multiple others

[LeaVerou]

There is a multiple choice question template that includes multiple tools with checkboxes, plus an "Other…" option, which shows a text field. Here is a screenshot of both states, from State of CSS 2023:

Unselected	Selected

The problem is that this only allows adding one "Other…" option, which forces the survey designer to pollute the survey with a ton of options (which biases the results) to minimize the need for users to need the "Other…" option.
Respondents can always enter comma-separated values, but this is both non-obvious, and becomes increasingly tedious as the number of values increases.

I present some UI ideas below to facilitate adding multiple custom options.

General considerations

These apply across all UI ideas:

• Keyboard shortcuts to minimize homing between keyboard and pointing device:
  • Pressing Enter within a custom option should insert another custom option below it and focus it
  • Pressing Backspace within an empty custom option should delete it and focus the previous one
• The checkbox of a custom option is a distraction: when it is inconsistent with the presence or absence of text in the text field (i.e. checking it without entering text or not checking it while entering text), that tends to be a mistake. I suggest disabling it and using it entirely as output (checked if text is entered, unchecked if not). This would also improve keyboard UX by eliminating one tab stop.
• "Custom" may be better wording than "Other". It generalizes to freeform lists better, and may imply less that the custom options are outliers.

UI concept 1: "Add other" button

A potential UI for allowing multiple custom answers, without making the common case of entering one more tedious would be to convert the "Other…" option into an "Add Other…" button which inserts (and focuses) a text field option every time it’s clicked.

Before click	After click

Notes:

• Empty fields are stripped from the data upon submission.
• When clicking "Add Other…", the new option inserted should also be selected and the text box should be focused

Pros:

• UX very similar to current one

Cons:

• While keyboard navigation is similar (new: tab to button, space to press it, type; old: tab to "Other" checkbox, space to check, tab to edit box), invoking buttons and going backwards feels less smooth
• Harder to generalize to freeform lists
• UI still makes custom options feel like outliers, increasing echo chamber effects

UI concept 2: Custom by default, auto-insert another

Update: I’ve now made a high fidelity prototype for this

custom-options.mp4

In this design, a custom field is present by default (potentially with styling to make it less prominent than it currently is). Typing in it inserts another (placeholder) one underneath, so entering more items is as simple as hitting tab (or enter). Hitting tab on an empty field does not insert another item.

Pros:

• Invites more responses
• Better suited for freeform lists
• Very smooth, efficient keyboard navigation (nothing to check or press, just tabbing through like you do with regular choices)
• Less UI clutter (no need for buttons to add an option)

Cons:

• More prominent UI may not be desirable in cases where "Other" truly is an outlier
• It may be jarring to enter another placeholder item when you start typing? User testing can tell us.

[SachaG]

To clarify, the current "Other…" field accepts a freeform text string, which can contain one or multiple items. Maybe we can clarify this somehow in the current UI.

[ShaineRosewel]

I'm thinking if we can possibly add an instruction (at the same time force a rule on the input) for the respondents to separate multiple responses with a comma. If this is not too taxing on the respondent's end, it will help us also during the analysis (free form response for this item will be easier to clean).

[SachaG]

This should work.

[LeaVerou]

This is better than nothing, but still discourages multiple responses, and makes it hard to edit more than 2-3. 😕
Are your concerns with the proposal in the OP primarily around engineering effort or user experience?

If eng effort: We could join the responses behind the scenes and still store as a single string, while exposing a UI that makes it easier to insert multiple items to end-users. Then we could use any separator we want when storing the joined string, e.g. something that is even less likely to appear in organic user input.

If user experience: We could keep the exact same UI for one "Other" response, and insert a "dormant" Other 2 item under it, that is only stored if edited. Similarly, if Other 2 is edited, a dormant Other 3: is added and so on. We should probably take checkboxes out of the tabbing order, and have the checked status of the checkbox be controlled entirely by whether text has been entered in the corresponding text field (making the checkbox primarily a feedback mechanism).

[LeaVerou]

I’ve fleshed out the interaction a bit more and made a high fidelity front-end prototype here: https://lea.verou.me/stateof/mocks/custom-options/

custom-options.mp4

What may not be super clear from the video:

• We typed two custom options, then deleted the second one using backspace to remove the item
• Then we tabbed away from the question, which removed all empty fields
• Then we typed two pain points, and tabbed away from that question, which removed the empty field at the end.

While the prototype supports both single-choice and multiple-choice questions, only the latter are discussed below, since the UX for single choice questions is very similar to the current one (does not support multiple custom options, since there is no reason to).

Functionality

Both multiple and single choice questions can opt-in to allowing custom option(s). Single choice questions can only have one custom option, multiple choice questions have multiple by default. By default, any number of custom options is allowed, but this can be limited to a specific number. In that case, the number is communicated via the placeholder/aria-label. By default only one field is shown, but questions can opt-in to showing more, to encourage more question options.
Custom options have radios/checkboxes just like the predefined options, but they are disabled and get checked/unchecked automatically based on whether text has been entered.

It is possible to have questions without any predefined options, only custom ones. This gives us freeform lists for free.

If there are no predefined options, the UI adapts in a few some small ways to best serve these use cases:

• Numbers (1, 2, 3...) are shown instead of checkboxes. This is currently done in the template, but could also be done with CSS alone.
• A <textarea> is used instead of an <input> We may later want to decouple this and use a separate longform setting.
• The placeholder text / aria label is different ("Item State of CSS 2022 Questions: help us create the survey outline #1" instead of "Custom answer"). Perhaps there is some kind of placeholder text that would work for both ("freeform answer"?).

UX

• Editing the last option inserts another one after it (but does not focus it)
• Focusing away from a question removes all empty custom options except the first
• Keyboard navigation:
  • Backspace on an empty field deletes that option
  • [<input> only] Enter inserts an empty option under the current one and focuses it, unless an empty one already exists there in which case it just focuses it
  • [<input> only] Up and Down arrow keys cycle through the options

Architecture & Implementation

Changes to question spec

Three extra fields:

• allowCustom (boolean): Enable custom answers, defaults to false.
• initialCustomCount (number): Number of custom answer fields shown initially, defaults to 1.
• maxCustomCount (number): Maximum number of custom answer fields, defaults to Infinity.

To minimize options, these could be combined into one custom field that could take either a boolean value to enable custom options with the default settings, or an object to customize the behavior ({initial, max}).

Data model

Currently, each custom field has the same name, which should not change the current behavior when there is only one, backend code just needs to check whether there may also be multiple items and store an array in that case. It also means the logic works out of the box for single choice questions, with no conditional logic.

Prototype code

• Prototype files
• Relevant part of the template
• Logic
• Keyboard interaction
• CSS (practically nonexistent)
• Spec for sample questions used in prototype

MVP

Compromises we can make to reduce engineering effort, in order (compromises to make earlier are listed first):

• Replacing checkboxes with counters when there are no predefined options (we can do it in CSS)
• Max number of custom options
• Keyboard navigation (Enter, Backspace, Up/Down arrows)
• Using a <textarea> and different placeholder/label when there are no predefined options
• Initial number of custom options

Future

Better support for user cases where custom options are outliers

Not all questions need custom answers at the same frequency. For some, the custom answers are the primary means of response, and the predefined answers exist just to get the respondent in the right headspace. In some, custom answers are needed only some of the time. And lastly, there are questions, that cannot be purely multiple choice, but rarely need a custom response.

Just like we can use an initial custom count > 1 to encourage custom answers more, we could also support an initial custom count of 0 to encourage them less. What would that look like? It could show a discreet "Custom…" button instead of a placeholder option. Alternatively, we could style the existing text field to just be less prominent in these questions.

Sentiment chips on custom options

Ideally, custom options should also support sentiment chips but that's a lot of work (both implementation and UX) so let's defer it for this survey.

[LeaVerou]

Hi @SachaG,

The entire project for this survey was conceived with the previous question format

My understanding is that neither @atopal nor I conceived the project in such a way that UI changes were not an option.
Please note that State of HTML is different from State of JS or CSS, as it has a much broader scope, exactly because it actually extends beyond pure HTML to all sorts of web platform technologies (JS APIs, some CSS syntax, manifests, HTTP headers, a11y questions, etc etc). There are so many different stakeholders (several different browser teams, the developers that respond and later read the results, a variety of different standards groups and CGs etc), each with different goals and interests. The amount of content needed for all of them to get at least some useful data is simply a lot higher than in the other surveys. If we are only able to use existing UI elements, it would simply not be possible to include what is needed and maintain a reasonable length.

that was never brought up as a potential issue until now.

I identified this as top priority (second only to sentiment) on August 17th, 2 weeks ago.

I just don't understand what makes this such a high-priority item?
I also don't understand why this would have an impact on the survey outline. [...]

I explained above why we need this, but let me rephrase, in case my earlier comment buried the lede:

1. It's not just about one vs multiple answers. It's about how much the UI encourages custom answers. The current UI is designed around custom answers being outliers, and requires the questions to be designed accordingly. For some questions (e.g. tooling), it is possible to just include a ton of predefined answers so that "Other" is rarely if ever needed. I can see the argument that for those questions, this may indeed just be an enhancement. Though even for those, a) having super long questions increases survey fatigue, which is exactly what we desperately need to reduce here and b) getting to the point where a question needs pagination biases the results. However, for other questions, where we want to encourage a large % of answers to be custom (e.g. missing elements question), having a single "Other" field that requires an extra click and is not on the same "level" as other answers does not give us the kind of interaction that is necessary to collect the data we need.
2. For questions without predefined options (freeform lists), I have provided citations to studies showing how freeform lists encourage significantly more responses. Given that these are exactly the questions that are critical for browser teams, why would we not do something as easy as this to encourage responses (plus make analysis a little easier)?
3. If this were a ton of work to implement to the point it could actually delay launch (as @atopal is worried below), I would understand maybe deferring it. But at this point, there is a fully functional prototype, and all it needs is to be ported to React. It doesn't even necessarily require data model changes (if we serialize the answers as a single string and store that). In fact, I feel the time we're spending debating whether to attempt this or not in total might well be longer than what porting it would take. 😃 If that is not accurate, could you please elaborate on what is tricky?

[SachaG]

having a single "Other" field that requires an extra click and is not on the same "level" as other answers does not give us the kind of interaction that is necessary to collect the data we need.

Based on what grounds?

I feel like you're putting the burden of proof on me to prove that changing the UI is unnecessary, but that seems backwards to me. The burden of proof should be on you to support your claim that the current UI is inadequate.

Again I am not saying your proposals are not improvements, I am saying that unless we have evidence that the current UI will not let us fulfill the survey's goals, then as far as I'm concerned the bar for justifying these UI changes at this time is not cleared.

On the other hand, what we do have evidence of, is that the current UI has worked for numerous surveys in the past, and that the other stakeholders in this project have not raised any issues about it so far.

In fact, I feel the time we're spending debating whether to attempt this or not in total might well be longer than what porting it would take.

I agree that time spent debating this is not well spent, and a better use of time would be working on the outline – which unlike this is core to the survey.

[LeaVerou]

My point was not that the current UI is inadequate, just that it's designed with different use cases in mind: those where custom answers are outliers. Designing questions so that custom answers are outliers is something that definitely affects the outline, and is not always even possible.

[ShaineRosewel]

Adding multiple, separate fields for other inputs will definitely help us during the analysis phase. There are users who input multiple ideas, say, when asked about pain points. When analyzing, this means we'd have to separate different ideas if we only have a single input field. It will be a big help if these multiple ideas are already separated to begin with. Hence, I support multiple input fields for others and free form responses.

[ShaineRosewel]

In the past, I have seen lengthy answers with different items being tackled while others with a brief response in one input field. How do we extract insights from these? We might want to summarize lengthy answers through the use of existing models. However, summarizing long ones might mean we lose some information (since existing models sometimes get confused with lengthy statements with multiple ideas). We may not need to worry about losing information if multiple ideas are already separated in different input fields to begin with.

In terms of say, other magazines respondents read, if they want to input three more items in the 'others', telling them to separate entries with commas is an option but we are giving them an additional instruction to follow; also, we cannot force them to follow (eg, I used semicolon to enter multiple response and still it got recorded). This is still possible if we have multiple input fields for multiple answers. But, the idea is we took away the effort from the respondents to follow an additional instruction.

[atopal]

It looks like we're all agreed that the changes would be beneficial, and it comes down to how much benefit and how much cost. Sacha, can you comment on what implementing this would mean for the State of HTML survey? Would we risk delaying it, or would this mean a significant amount of work on your end? Lea, do we have indicators that people are more likely to add significantly more information with this UI change?