Delight: App Layout (Mobile IA)

[daniellekirkwood]

Intro

Problem we’re solving

• Set accurate first impression with new users to compete effectively and set playing field equally across platforms (iOS & Android) to deliver improvements to D1 later (Q4)

Goals

• Implement and release the proposed redesign in order to:
• Cement Element as the leading Matrix app, where we’re ceding market mindshare to rising competition due to poor usability and poor first impression
• Convert our existing early adopters into promoters

Problems we’re not solving

• Re-designing ‘second tap’ screens or flows like settings, rooms, etc

Design principles

• Can be found here

Success Metrics

• The App layout project is a strategic bet, we do not think it will have direct impact on metrics such as D1
• We intend to approach this project with a "data-validated" mindset over a "data-led" mindset; we'll seek to confirm or disprove our hypothesis at each stage to ensure success, we will not start with data to define our direction
• Each experiment will have it's own success criteria based on the hypothesis being tested
• We will use overall SUS scores and other guardrails to ensure we're doing no harm

---

Discovery

We have designs that need validating before we ship to product.
We will run a series of experiments to increase confidence in the new designs:

• App Layout: E2E Flow Experiment #225

• App Layout: Space Switching Experiment [Meta] #219

• App Layout: Edit Layout Experiment [Meta]  #220

• Mobile Layout: Tab-less layout Experiment #221

• Mobile Layout: Invites Experiment #222

• Recruiting participants

Click to see The Experimentation Plan!

The Experimentation Plan

We intend to run a series of experiments to reduce risk, remove assumptions, and gain confidence in the proposals.

• The full design proposals are here

E2E Feedback	Space Switching	Edit Layout	Tab-less Layout	Invites
Goal Show the new designs to new and existing Element users for feedback on the whole design	Goal Move Space switching to the app header. Can users still access their Spaces, is it a positive interaction?	Goal How does the design for Edit layout impact the user's home screen and does it scale well?	Goal We don't believe we need tabs at the bottom of the screen, we need to understand if (without tabs) users can navigate the app and complete the flows successfully	Goal When a user receives invites, how do they appear? We need to ensure that they're not missed but also are not blocking or disruptive to the rest of the Element experience
#225	#219	#220	#221	#222
Overview of experiment Both platforms, new and existing users	Overview of experiment Both platforms, functional prototype. Android first	Overview of experiment One platform, functional prototype. iOS	Overview of experiment Maze	Overview of experiment Maze
Status In design	Status In design	Status In design	Status In design	Status In design
Expected behaviour Users understand the new layout. Existing users highlight big impactful differences	Expected behaviour Users are able to find and switch Spaces and subspaces easily	Expected behaviour Users who take advantage of the Edit layout feature suggested have a good home experience	Expected behaviour Users are able to complete tasks in the app without getting lost	Expected behaviour Users are able to continue in their app without handling their invites first.
Actual results Pending...	Actual results Pending...	Actual results Pending...	Actual results Pending...	Actual results Pending...
Decision Pending...	Decision Pending...	Decision Pending...	Decision Pending...	Decision Pending...

---

Delivery

1. Validate designs through tests and Experiments (Est. 2 months)
2. Build new designs into app (Est. X months)
3. Launch new designs to product - all at once (Est. MM-2022)

• Delight: App Layout (Space Switching) element-android#6605
• App Layout: Edit Layout [Android] element-android#6502
• Delight: App Layout element-ios#6405

A detailed overview of spec and delivery actions can be found here

Risks and Mitigations

There are risks associated with the design, our mitigation efforts include:

• App Layout: E2E Flow Experiment #225
• vector-im/internal-planning#251

Click to Risks and Mitigation plan!

Mitigation Plan

Risk	Details	Mitigation options	Decision
Timeline	Projects of this nature can often take time or encounter a lot of unexpected road bumps/blocks	Clarity of scope on deliverables and estimated timelines	Validate designs through experimentation (Est. 2 months) Integrate designs into current app (Est. XX months)
Rolling out to new users will result negative impact to R30	A big design change is hard for users who are used to the existing product.	Understand the impact of the change Ensure a clear notification plan to existing customers (Insert GH)	TBD
Measurability	It will be hard to measure the impact of these changes	Establish a baseline metric ASAP	vector-im/internal-planning#251
...	...	...	...

Roll out details - WIP

These designs will be a big change for users and the roll out and launch must be managed effectively.
While in discussions the plan details will be here as it promotes comments.

---

Decisions

There are notable decisions that the team will make along the way, we will note them here:

1. ElementX or ElementiOS: Where and how do we build the experiments?
   * We are optimising for speed of delivery.
   * We will built Edit Layout experiment in SwiftUI and deploy using ElementiOS app to participants.
   * Read the notes here
2. TBD...
   * Lorem
   * Ipsum

[daniellekirkwood]

Decision made: We will prioritise this project, starting in Q2

It's accepted that this project may not directly impact D1 retention but is instead a strategic move to ensure that we compete visually today and have the right foundation and platform to improve D1 quicker and more effectively later.

Next Steps: April 12, 2022

• Discuss the project and scope with the team (and run a pre-mortem)
• Understand the design decisions to-date (with Designers)
• Discuss metrics and measures and undertake work to create Baseline
• Run the Maze experiments
• Discuss the technical feasibility and approach (with Engineers)
• Start to build the functional prototypes

[daniellekirkwood]

Clarifying the plan (April 29, 2022)

• We will build 2 experiments, 1 in each platform. (Building started May 2, 2022)
  • Space switching on Android
  • Edit layout on iOS
  • Each of these tests will be shared with at least 12 people, and will run for 2 weeks minimum. There will be at least 3 points that we collect feedback over this period. See here for participant and feedback collection info.
• There are 3 experiments we will run on Maze. (Amsha will start building these when back from vacation May 9, 2022)
  • Invites, tab-less, and E2E flow
• While the prototypes are with users being tested (Space switching and Edit layout) we will start to build the changes confirmed by the Maze tests

[ericdecanini]

Android space switching epic here

[gileluard]

iOS epic here

[daniellekirkwood]

#550

[daniellekirkwood]

Results: https://docs.google.com/spreadsheets/d/1d2jirxzoSbSvu0dkiaTkhm_-2r6IK1weRVBny1TmbJs/edit#gid=679956630