Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EPIC - Recordings #6077

Closed
paolodamico opened this issue Sep 23, 2021 · 15 comments
Closed

EPIC - Recordings #6077

paolodamico opened this issue Sep 23, 2021 · 15 comments
Assignees
@clarkus @paolodamico @alexkim205
Labels
epic

Comments

@paolodamico
Copy link
Contributor

paolodamico commented Sep 23, 2021 •
edited
Loading

Core Analytics focuses on enabling users to find the right recordings to watch, and in this epic we focus on extracting insights from a recording.

Goals (ordered)

  1. Be able to seamlessly find and jump into relevant parts of a recording.
  2. Be able to quickly extract qualitative insights from recordings.
  3. (Stretch; TBD) Be able to quickly save and share insights with other team members.

General activities (to discuss)

  • Provide useful relevant context from recording (e.g. pages visited, custom events triggered, ....)
  • Provide context on the seek bar about the activities performed
  • Allow fast seeking/skipping/jumping, particularly to specific events/pageviews/timestamps.
  • Fix jumpy CSS rendering / remote image loading.
  • Buffering / stream loading.
  • Clarify inactivity periods and redacted information.
  • Tagging / short links / commenting / exporting recordings.

Related issues to solve

List of related issues to address
@rcmarron
Copy link
Contributor

rcmarron commented Sep 29, 2021 •
edited
Loading

I think we should also consider this experience in conjunction with the session recording list. A couple of ways that I think they could interact:

  • By default, the events shown in the session playback experience match the events filtered for in the session recording list
  • It's obvious/easy to move to the next/previous session recording in the list.

I think both of this would really help users find the moments that they're hoping to view.

@mariusandra mariusandra mentioned this issue Sep 29, 2021
Closed
@alexkim205
Copy link
Contributor

Summarizing a few preliminary ideas brought up in a call with @rcmarron:

Session Recording Playback

  • Event Timeline
    • Timestamps in event timeline
    • Time durations for each event
    • Show event types
    • URL's not the best way to capture activity changes
    • Tree hierarchy for easier scannability
  • Fullscreen
    • What does the fullscreen experience look like?
  • Separate play mode page
    • Show session recordings list to the left, video to the right, and events metadata on the bottom
    • Optimize explorability
    • Separate playback and recording pages or singular multi-functional page?
  • Playback Video
    • Information in scrubber (divided into sections like YouTube?)
    • Heatmap highlighting parts of video with most activity
    • Informed scrubbing > blind scrubbing

Session Recordings

  • How do you get from session recordings to playback and back?
  • There're many instances in the app where we'll be displaying this Session Recordings table. What features should or shouldn't be exposed in each case?
    • Persons
    • Session Recordings
    • Potentially Session Recording Playback
  • Remove date restriction

@clarkus
Copy link
Contributor

clarkus commented Sep 30, 2021

Some early design explorations at https://www.figma.com/file/gQBj9YnNgD8YW4nBwCVLZf/PostHog-App?node-id=3753%3A28485 based on the criteria above. I'll be using this project to facilitate feedback.

@rcmarron
Copy link
Contributor

rcmarron commented Oct 1, 2021 •
edited
Loading

I've been putting more thought into this project, and I think it makes sense to consider the playback experience in conjunction with the recording list view and quality improvements. It feels like they all combine to the same overarching goal ('let people find and watch useful session recordings'), so it's hard to prioritize them in isolation. With that in mind, I put together a proposal 'scope of work' for this project in the upcoming sprint below. I tried to combine @paolodamico's list, @alexkim205's list, and other items that I've been thinking about and have seen in GH issues. It'd be great to get some feedback on this before our "Kickoff call" on Monday, so we have a good idea of what we're "kicking off" 🙂.

Goals:

  • 🟣 Quality: Make the session recordings feature feels like it works. Expectations are properly set, it doesn't crash, recordings aren't missing etc.
  • 🟠 Finding recordings: A user can find session recordings that show useful moments.
  • 🟢 Viewing recordings: A user can seamlessly find and watch relevant parts of a recording.

Proposed scope:

If we complete the work below, I think we'd have a great feature that meets the minimum goals above, and we could confidently get feedback from users to build on top of it. The 🖌️ icon means that I think it would need some design help from @clarkus.

  • 🟣 Ingestion bugs
  • 🟣 Improve documentation
    • Be clear that we don't support mobile app recording, and it only works when you're using posthog-js
    • Include 'known issues' around why recordings might be missing. (contingent on ingestion/player fixes)
  • 🟣 Fix player crashes
  • 🟣 Clarify redacted information in the player
  • 🟣 Make the player modal smoother on entry/exit
    • Right now, the player modal is jumpy as it animates in, and it just disappears on close. I'd expect it to animate in and out smoothly.
  • 🟢 🖌️ Clarify periods of inactivity and allow users to skip it
    • In the experience now, periods of inactivity are fast-forwarded, but it can still take a while to get through it. There is no indication of when the inactivity ends, so your only option is to sit through the fast-forward. Also, if you try to seek while it's fast forwarding, the fast forwarding remains on after you've gotten to a period of activity.
    • Improve session recording seekbar #5710
  • 🟢 🖌️ Provide more context in the seek bar
  • 🟢 🖌️ General UX refresh/tweaks in the playback experience
    • The player looks like three different designs merged together, can we make it more consistent?
    • There are assorted oddities of the playback experience right now:
      • Skipping back 8 seconds when on 16x speed doesn't really make sense.
      • Make it more obvious which playback speed is selected
      • We should additional user information when hovering the user's email address
      • You can seek using the event bar on the right and the bar below the video.
    • Improve session recording seekbar #5710
  • 🟠 🖌️ Filter session recording list by events/actions
    • To help users find session recordings, we should enable them to filter session recordings by the actions/events that occurred in those sessions. (e.g. show me the session where users went to this page and clicked this button). In addition, we should include session duration and time of recording
  • 🟠 Session recording list needs pagination
    • An obvious missing aspect of the basic session recording list right now.
  • 🟠 🖌️ Show which recordings the user has already viewed in the list
    • We store when a user views a session recording. We should indicate this in the session list, so that users know which ones they've already viewed.
  • 🟠 Include more interesting information about the recordings in the list
    • The session recording list view is pretty bare-bones right now. It only includes the start time, end time, person, and duration. We should add more information to help users select which recordings to watch (e.g. pages visited, custom events triggered, etc.)

(Notable omissions)

  • 🟠 🟢 🖌️ Combined recording list/player view
    • We should have a side-by-side view for the recording list and player. This would allow someone to filter the list of recordings and then quickly move from one recording to the next (no more open modal/close modal flow). This feels like a big win for letting people quickly find/navigate to interesting recordings. @alexkim205 and I we discussing how this should be the view when you first navigate to the session recordings page (instead of just the list)
    • I think this would be a big win, and it's the hardest thing for me to leave out. The reason it's left out is that we would still need to build the list view and full modal view if we want to expose session recordings multiple places in the product (e.g. the persons page), so we would still make a complete experience without doing this work.
    • Session recording playback on new tabs #2581
  • 🟠 🖌️ Saved Filters
  • 🟠 Filter by "Active time" not just duration
    • Session duration is only useful for filtering out sessions that are really short. Many times a session that is 30 min long and a session that is 1 min long both only contain 1 min of activity (because the user in the 30 min session just left the screen open). We should show active time in the list view and allow users to filter by it.
    • Allow sorting sessions by intensity #4573
  • 🟠 Filter to recordings that a user hasn't seen yet.
    • We should allow people to see the list of recordings that they haven't watched already. We allow this with sessions today, and it probably makes sense to keep it.
  • 🟢 🖌️ Events that match filters in the list view should be highlighted in the scrubber
    • If we're allowing users to filter session recordings to recordings that contain specific events, then we should highlight those events in the playback experience. It could be interesting to allow a user to 'skip to the first matching event', to make it faster to find the moment they're looking for.
  • 🟢 Links to recording timestamps

Questions:

  • Is this too much work for this sprint? Is that OK?
  • Should anything below be pulled up? Should we move anything down?
  • Are we missing things in these lists that we should be considering here?
  • There were a few items in the lists above that I didn't remember/couldn't figure out what they were.

@rcmarron rcmarron changed the title EPIC - Recordings playback experience EPIC - Session recordings Oct 1, 2021
@paolodamico paolodamico changed the title EPIC - Session recordings EPIC - Recordings Oct 4, 2021
@clarkus
Copy link
Contributor

clarkus commented Oct 4, 2021

Related - #5360 Any chance we can clean up this issue along with the other work happening here?

@posthog-contributions-bot
Copy link
Contributor

This issue has 1620 words. Issues this long are hard to read or contribute to, and tend to take very long to reach a conclusion. Instead, why not:

  1. Write some code and submit a pull request! Code wins arguments
  2. Have a sync meeting to reach a conclusion
  3. Create a Request for Comments and submit a PR with it to the meta repo or product internal repo

@clarkus
Copy link
Contributor

clarkus commented Oct 4, 2021

I think it could also be valuable to have an action at some level that essentially says "show me more recordings like this" based on common events, time lines, etc. Just thinking that once a user finds that value in a recording, validating it with other recordings might be the next step. Having a one click action that gives you that list could provide value to users depending on their use case.

@rcmarron
Copy link
Contributor

rcmarron commented Oct 4, 2021

Kick-off meeting recap

@clarkus, @alexkim205, and I just had a kick-off meeting for this project. (@paolodamico was there for the first part)

Our 'non confirmed' goals for the sprint are (will update this when we confirmed them):

  • 🟢 Playback experience: When watching a recording, 80% of the time, you can find the moment you're looking for within 3 seconds.
  • 🟣 Quality: reduce ingestion bugs X% and player crashes by x%
  • 🟠 Ship the MVP session recording page

With these goals in mind, we walked through each item and agreed upon the scope that we're tackling for this sprint. The general idea is that we should work to:

  • Make quality improvements around ingestion bugs and player crashes, but do so in a calculated manner (e.g. lets understand prevalence of a crash before going down a rabbit hole)
  • Improve the player experience specifically focused on:
    • Fix experience around periods of inactivity
    • Add event context to the player (showing events etc.)
  • Ship the session recordings list view with basic filtering

Below is how this work breaks down in more detail. It feels a bit ambitious, but we think that's ok:

Main items

  • 🟢 "Learn rr-web/rr-player" @alexkim205
    • With an eye toward understanding feasibility of:
      • Clarify periods of inactivity
      • Provide more context in the seek bar
    • The goal is to understand our limitations to better inform design of the items below.
  • 🟢 🖌️ Clarify periods of inactivity and allow users to skip it @alexkim205 + @clarkus
    • We have questions about what RR-web allows here (hence the "Learn rr-web" item above)
    • A high priority
  • 🟢 🖌️ Provide more context in the seek bar @alexkim205 + @clarkus
    • A high priority.
  • 🟢 🖌️ General UX refresh/tweaks in the playback experience @alexkim205 + @clarkus
    • A high priority
  • 🟣 Ingestion bug: undefined error @pauldambra
    • Paul is working on this one. Seems like a rabbit hole, but it's major cause of exceptions
  • 🟣 Ingestion bug: Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE. @rcmarron
    • Rick to understand the prevalence of this bug across projects. (e.g. how many projects hit it?). Depending on the answer, look at solutions to inform those users about the problem. If it's just one project, we should just send them an email, but if it's many, then let's build an alert into the product with information on ways to fix it.
  • 🟣 Fix player crashes @alexkim205
    • Look at analytics on player crashes (might have this in sentry today)
    • Time-box fixes to 2 days if there are obvious high-pri ones
  • 🟠 🖌️ Filter session recording list by events/actions @rcmarron
    • Might have existing designs from session that we can lean on
  • 🟠 Include more interesting information about the recordings in the list @rcmarron
    • Take an MVP approach here and do what's simple (e.g. user, start time, duration, start URL, end URL, and browser). We will learn more from users and can react then. Eventually, we could add much more here (e.g. user properties, device properties, what happened in the recording, etc.)
    • This will also be tightly related to what users can filter by

Smaller items

  • 🟣 Improve documentation @rcmarron
  • 🟣 Make the player modal smoother on entry/exit @alexkim205
  • 🟣 Clarify redacted information in the player @alexkim205
    • Determine how feasible this is with rr-web, but assuming it's straight forward, we should definitely do it.
  • 🟣 Look into recording deletion bug @rcmarron
  • 🟠 Session recording list needs pagination @rcmarron
    • Pagination not infinite scroll, particularly because we’re moving back and forth between player and list
  • 🟠 🖌️ Show which recordings the user has already viewed in the list @rcmarron

Notes:

  • Can potentially involve the RR-web creator if needed. He's offered to contract for us - let's keep that in mind.

@alexkim205 + @clarkus Let me know if I missed anything.

@mariusandra + @paolodamico would love your thoughts here.

@paolodamico
Copy link
Contributor Author

Thanks @rcmarron! Based on great feedback from @mariusandra & @pauldambra, goals updated on main sprint issue. Fwiw, very high level, here's what I think will have the highest impact in this sprint (feel free to change/disregard/...), which is very aligned with what were you thinking:

  1. Fix the two bugs (direct goal).
  2. Allow users to seek/skip quickly to any point in time (likely requires 4).
  3. Add relevant context to the UI (interesting/relevant events that you can jump to) [seek bar and/or sidebar].
  4. Improve recordings loading time (can we load just the first few seconds/mins instead of the entire recording before starting a playback? reduce image sizes? ...).

@rcmarron
Copy link
Contributor

rcmarron commented Oct 5, 2021

@paolodamico thanks for updating the goals and sharing this!

I hadn't put too much thought into the loading time one (I think I suffer from having pretty good internet 😁). But it makes a ton of sense to me - the payloads definitely get big.

@alexkim205 @clarkus do you have thoughts on prioritizing loading speed above some of the other work? Maybe we bump out the 'periods of inactivity' work as showing events on the timeline would help solve this pain as well.

@clarkus
Copy link
Contributor

clarkus commented Oct 5, 2021

I think we should solve for both - immediately loading the recording aligns with our goal of showing something useful as quickly as possible. I think skipping inactivity is pretty critical as well. I mostly feel this way because I don't think we have a firm idea of how we'll determine which useful events to show on the timeline. Showing some affordance that an event happens at a specific point on the seek bar is pretty straightforward, but the means in which a user defines that is still a bit nebulous. I think there's more work to do in the play back interface that allows a user to identify or promote events in some way in order to highlight them on the seekbar.

@alexkim205
Copy link
Contributor

alexkim205 commented Oct 5, 2021 •
edited
Loading

re limitations of rrweb-player:

I had some time to look into rrweb-player yesterday and found that:

  • We pretty much own the look and feel of the player experience with @posthog/react-rrweb-player. This means we can overlay indicators, tweak the interface, resize the video frame, etc. without limitations
  • We have information about "custom_events" ($pageview, $rageclick, custom actions, etc.) in this kind of schema:
...
{
    "type": 5, // int representing custom_event
    "data": {
        "tag": "$pageview",
        "payload": {
            "href": "http://localhost:8000/insights?insight=TRENDS..."
        }
    },
    "timestamp": 1633387454897,
    "delay": 2596570
}
...

With the timestamp that the event happens, we can tell rrweb to start playing at that point in time when a user clicks a corresponding seekbar indicator.

  • The rrweb api exposes these events which we can react on. The most relevant ones for our inactivity use case seems to be skip-start and skip-end which tells us when inactivity skipping starts and stops.
  • Inactivity
    • This isn't well documented anywhere but I'm assuming rrweb defines inactivity as periods of time when an event snapshot doesn't exist. An event snapshot can be any of DomContentLoaded, Load, FullSnapshot, IncrementalSnapshot, Meta, Custom, Plugin events.
    • There's no way to custom define inactivity ourselves (at least not through the rrweb api) unless we parse event snapshots ourselves and delete events that we define "inactive". This might be easy or difficult depending on how complicated our filters for inactivity get. @clarkus I think this largely depends on how we want to enable the user to define the events they want to highlight or mute.

@clarkus definitely let me know if there're any more questions you had regarding rrweb's limitations. Tagging @macobo here also so that he can fact check the above or just generally give better advice about how we can use rrweb better.

re prioritization:

Seekbar/general UX and loading time improvements to me stand out as items that I can hack together for an MVP with our current mockups. I think there's still ongoing discussion about inactivity—for now I'll use rrweb's default definition of inactivity and iterate on that once we decide what that'll look like.

  1. Learn rr-web/rr-player
  2. Improve recordings loading time (which i suspect is due to large DOM recreations)
  3. Provide more context in the seek bar and side bar
  4. Clarify periods of inactivity and allow users to skip it

@clarkus
Copy link
Contributor

clarkus commented Oct 5, 2021 •
edited
Loading

The actions I am tracking for the player:

  • Play / pause
  • Manual skip ahead by a fixed amount (7s)
  • Manual skip behind by a fixed amount (7s)
  • A toggle to automatically skip periods of inactivity
  • Playback speed controls - we currently have a lot of options and 1x isn't one of them for some reason. I'd like to revisit this to come up with a smaller set
  • Settings - not sure what all could apply here, but I am tracking placement for it for when we need it
  • Full screen / exit full screen

For the seekbar, I am tracking these capabilities

  • Summarize the elapsed duration of recording (seek bar visualizes this and it's explicitly labeled)
  • Summarize the total duration of recording (seek bar visualizes this and it's explicitly labeled)
  • Summarize the loaded (ready to be viewed) portion of the recording (seek bar visualizes this but not explicitly labeled)
  • Highlighted / annotated events - an overlay dot and a descriptive tooltip to identify the event - might only display on hover - TBD
  • Indicate periods of inactivity
  • Facilitate scrubbing forwards or backwards to a specific point in time

Most of this stuff is represented in the concepts now:
Frame 9163
Frame 9161
https://www.figma.com/file/gQBj9YnNgD8YW4nBwCVLZf/PostHog-App?node-id=4009%3A29443

Things I am tracking as possible improvement / enhancements for the future

  • Previewing the screen state for a point in time when scrubbing ahead
  • Autoplay configuration
  • Quality configuration
  • Inline actions for linking to a specific point in time in the recording
  • Annotations - being able to add commentary or tie in collaborative features
  • Ability to export a recording for replay later - what format would this take? A movie file or something else?
  • Zoom controls - it's a dom replay - why not replay in the same dimensions but at a reduced size and scale. This would allow to view the entire viewport capture area without any weird cropping because of viewport constraints.

@alexkim205 alexkim205 mentioned this issue Oct 5, 2021
Closed
2 tasks
@paolodamico
Copy link
Contributor Author

Thanks everyone, love this! Will add feedback on Figma. @rcmarron, just to clarify I've experienced issues with both lousy internet but also memory/CPU usage up to the point where at best the tab crashes, and at worst my entire computer almost freezes completely.

@clarkus clarkus mentioned this issue Oct 15, 2021
Closed
@alexkim205
Copy link
Contributor

We can close this now that most of this is implemented. Good work everyone!

@mariusa mariusa mentioned this issue Aug 29, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@clarkus clarkus

@paolodamico paolodamico

@alexkim205 alexkim205

Labels
epic
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@clarkus @rcmarron @paolodamico @alexkim205