Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: remove low value percy snapshots #26934

Merged
merged 6 commits into from
Jun 7, 2023
Merged

Conversation

jordanpowell88
Copy link
Contributor

@jordanpowell88 jordanpowell88 commented Jun 5, 2023

Additional details

This PR implements the recommends from this Percy Audit. The goal is to decrease current number of Percy snapshots. We are currently recording 50K-60K snapshots a month. This targets snapshots that are any of the following:

  • A subset of another snapshot
  • A pixel perfect duplicate of another snapshot
  • Would be better written as a text based assertion
  • Covered in an E2E test
  • Does not provide much value

Steps to test

There isn't really any - just take a look at the snapshots, you'll see a lot of duplicates were removed. I'm confident we are losing a negligible amount of coverage at best, and this number of snapshots is far more easy to review and maintain.

How has the user experience changed?

PR Tasks

@jordanpowell88 jordanpowell88 force-pushed the jordanpowell88/percy branch from c013ebb to e006730 Compare June 5, 2023 18:46
@cypress
Copy link

cypress bot commented Jun 5, 2023

4 flaky tests on run #47193 ↗︎

0 5193 77 0 Flakiness 4

Details:

fix types
Project: cypress Commit: 35ceed013d
Status: Passed Duration: 13:25 💡
Started: Jun 7, 2023 6:45 AM Ended: Jun 7, 2023 6:58 AM
Flakiness  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-electron

View Output Video

Test Artifacts
... > correctly returns currentRetry Output Video
... > correctly returns currentRetry Output Video
... > correctly returns currentRetry Output Video
Flakiness  e2e/origin/user_agent_override.cy.ts • 1 flaky test • 5x-driver-electron

View Output Video

Test Artifacts
user agent override > persists modified user agent after cy.go Output Video

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

@jordanpowell88 jordanpowell88 force-pushed the jordanpowell88/percy branch from ff0edf8 to a3314de Compare June 6, 2023 18:20
@jordanpowell88 jordanpowell88 marked this pull request as ready for review June 6, 2023 20:49
Copy link
Contributor

@lmiller1990 lmiller1990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed 267, down from 523 to 256 for a decrease of approximately 50%.

@lmiller1990 lmiller1990 requested a review from a team June 7, 2023 11:01
@lmiller1990
Copy link
Contributor

Great - pushed a couple more commits. I think we are looking good.

@lmiller1990 lmiller1990 merged commit 2263bc5 into develop Jun 7, 2023
@lmiller1990 lmiller1990 deleted the jordanpowell88/percy branch June 7, 2023 22:56
AtofStryker added a commit that referenced this pull request Jun 12, 2023
* docs: revise contributor advice for node.js setup (#26652)

* feat: initial release of cypress/vite-plugin-cypress-esm (#26663)

* chore: release @cypress/vite-plugin-cypress-esm-v1.0.0

[skip ci]

* chore: upgrade Lerna to v5 and use Nx (#26660)

* dependency(deps): update dependency engine.io to v6.4.2 [security] (#26664)

* dependency(deps): update dependency engine.io to v6.4.2 [security]

* updating changelog

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Matt Schile <[email protected]>

* fix: updated CYPRESS_DOWNLOAD_PATH_TEMPLATE regex to allow multiple replacements (#25531)

* feat: Component Testing banner (#26625)

Co-authored-by: elevatebart <[email protected]>

* chore: Update v8 snapshot cache (#26647)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>

* chore: 12.12.0 release updates (#26697)

* chore: cypress[26674] updated github workflows to use checklout@v3 and stop using set-output and move to  file use (#26696)

* fix: move types condition to the front (#26630)

* fix: move `types` condition to the front

* chore: update changelog

* optimize deps for vite

* update config

* try optimizeDeps.include

* missing dep

---------

Co-authored-by: Lachlan Miller <[email protected]>

* fix: revert #26452 (Yarn Plug n Play compat regression) (#26735)

* Revert "fix: correctly resolve dependencies for CT onboarding when using Yarn Plug n Play (#26452)"

This reverts commit 7a33f5c.

* changelog

* changelog

* chore: fixing PR link in releaseData.json (#26734)

* chore: update stalebot config (#26745)

* misc: ACI empty state slideshows (#26692)

Co-authored-by: Stokes Player <[email protected]>

* fix: do not cache module resolution during launchpad dependency detection (#26726)

Co-authored-by: Mike Plummer <[email protected]>

* fix: Vite esm plugin issues (#26714)

* chore: Update v8 snapshot cache (#26707)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>

* chore: adding in repo token to explicitly use that while running commands [skip ci] (#26746)

* chore(dependency): update dependency @percy/cypress to ^3.1.2 🌟 (#26717)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Chris Breiding <[email protected]>

* chore: Remove console.log (#26756)

* chore: put types condition first in the syncing script (#26743)

Co-authored-by: Chris Breiding <[email protected]>

* test: create lists files after folders when in same directory in specs list (#26723)

Co-authored-by: Chris Breiding <[email protected]>

* chore: update vm2 to 3.9.19 (#26772)

* chore: add telemetry to the proxy (#26695)

* chore: set up instrumentation and instrument middleware

* chore: set up console exporter

* chore: add parent span option to telemetry package

* chore: set up telemetry verbose mode

* chore: instrument the network proxy - part 1

* chore: make sure to terminate spans when request is aborted

* fix telemetry, create/end the request middle prior to sending the outbound request

* avoid telemetry ts build step, create entrypoint into packages/telemetry using TS conventions

* allow env vars to be "true" or "1"

* when creating child span, inherit their attributes directly from the parent

* create custom honeycomb exporter and span processor to log traces

* remove duplicate code that's already called in this.setRootContext

* cleanup

* more clean up

* update honeycomb network:proxy attributes, update console.log message

* yarn lock

* chore: remove performance API in middleware

* chore: end response on correct event

* recursively gather parent attributes on close

* added key and some clean up

* github action detector, move verbose into index, verbose log commands

* some tests

* clean up honeycomb exporter

* some renaming

* testing console trace link exporter

* Don't lose the top span when running in verbose.

* link to the right place for prod/dev

* changes to verbose to make sure it is read in the browser

* Apply suggestions from code review

* pass parent attributes between telemetry instances

* default to false

* 'fix' build issues

* src not dist

* add back on start span

* once more with feeling

* Fix some tests

* try this i guess

* revert auto build

* Apply suggestions from code review

Co-authored-by: Bill Glesias <[email protected]>

* support failed commands

* Address PR comments

* Address PR Comments

* error handling

* handle all the errors

---------

Co-authored-by: Bill Glesias <[email protected]>
Co-authored-by: Brian Mann <[email protected]>

* chore: update triage workflow to add comment to contributor prs (#26493)

Co-authored-by: Ben M <[email protected]>

* chore: telemetry pr cleanup (#26776)

* fix: fix UI flash when switching to debug page (#26761)

* chore: add Nx Cloud (#26712)

* chore: add empty nx.json [run ci]

* chore: add nx cloud runner [run ci]

* chore: add nx-cloud dep [run ci]

* chore: update local nx cloud accessToken to be read-only

* feat: update git related messages for runs and debug (#26758)

* feat(app): update DebugError copy

* feat: set isUsingGit project flag and consume in DebugContainer

* feat(app): update not using git icon for DebugError

* feat(app): displays alert on runs page when not using git

* feat(app): add component for when no runs for current branch

* feat(app): add warning for no runs for branch on runs container

* chore: add feat to CHANGELOG

* chore: remove logged status value

* chore: resolve import from merge conflict

* test(app): stub branch name for e2e runs spec

* chore: add line break in changelog entry

* chore: cleanup PR

* chore: fix i18n import for DebugBranchError

* chore: add utm and update Warning links to inline

* chore: capitalize Git in i18n

* ref: warning liink

* test: add i18n to tests

* test(app): change utm_source assertions

* chore: cleanup pr

* chore: remove unused prop

* test(app): remove no git warning when moving to runs page in e2e

* chore: change template logic

* chore: remove duplicate RUNS_TAB_MEDIUM const

* Changed Debug test assertion and reordered new components for Debug

---------

Co-authored-by: Stokes Player <[email protected]>

* chore: rename video processing events to capture/compress (#26800)

* chore: change processing nomenclature to compressing when printing the run.

* chore: rename 'capturing of' to 'capturing'

* chore: rename upload results to upload screenshots & videos (#26811)

* chore: rename upload results to upload screenshots & videos

* run ci

* chore: capture versions of relevant dependencies with `x-dependencies` header (#26814)

* chore: update changlelog script to handle revert pr ref (#26801)

* fix: Correct typescript scaffold dependency (#26815)

* fix: correct typescript scaffold dependency (#26204)

* add changelog

* Update change log for PR comment

Co-authored-by: Mike Plummer <[email protected]>

---------

Co-authored-by: Mike Plummer <[email protected]>
Co-authored-by: Mark Noonan <[email protected]>

* chore: 12.13.0 prep (#26833)

* chore: 12.13.0 release (#26834)

* chore: 12.13.0 changelog fix

* remove pending, bump version

* fix typo

* chore: release @cypress/vite-plugin-cypress-esm-v1.0.1

[skip ci]

* chore: Implement runSpec mutation (#26782)

* chore: replace gitter badge with discord on readme (#26771)

* chore: add GraphQL mutation for sending system notifications via Electron (#26773)

* fix: upgrade typescript from 4.7.4 to 4.9.5 (#26826)

Snyk has created this PR to upgrade typescript from 4.7.4 to 4.9.5.

See this package in npm:


See this project in Snyk:
https://app.snyk.io/org/cypress-opensource/project/d5b36925-e6ee-455d-9649-6560a9aca413?utm_source=github&utm_medium=referral&page=upgrade-pr

* test: fix 2 broken tests for Windows (#26854)

* Update stale_issues_and_pr_cleanup.yml

Upped the number of operations per run.  Have been manually doing that so this job can get through all the issues in the repo with no problems.

* chore(dep): [Snyk] Upgrade vite from 2.9.13 to 2.9.15 (#26830)

Co-authored-by: Ben M <[email protected]>

* Update triage_add_to_project.yml

* chore: fix minor background color styling in debug results component (#26887)

* Update stale_issues_and_pr_cleanup.yml

stalebot was incorrectly configured to run in debug mode.  I have updated the default to run in normal mode when running scheduled

* chore: Deprecate @cypress/xpath package (#26893)

* chore: add telemetry realworld app (#26896)

* chore: capture telemetry for realworld app maybe

* idk what i was doing

* setup record key and telemetry

* testing

* override project id

* some times we just need a little context.

* Adding tests

* Adding comment

* chore(deps): update dependency find-process to v1.4.7 🌟 (#26906)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jennifer Shehane <[email protected]>

* chore(deps): update dependency firefox-profile to v4.3.2 🌟 (#26912)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Jennifer Shehane <[email protected]>

* chore: add browser state action for debug (#26884)

* chore: add browser state action for debug (#26763)

* Address PR comments
- remove unneeded async and test context
- genericize openProject function

* Revert route change, update spec description

* chore: replace gift devDep with simple-git (#26728)

* chore: replace arg devDep with minimist; remove unused shx devDep (#26727)

* chore: enable caching for lint task (#26791)

* chore: remove old performance reporting (#26920)

* chore: remove old performance reporting

* remove libhoney dep

* try this

* build and build snapshots if deps are out of date

* foiled by a comma

* freaking comma

* no snapshots maybe ugh

* ignore engines instead

* don't need this

* remove rename support file step

* chore: update Snyk to scan all projects (#26867)

* SEC-544 chore: [Snyk] Update Snyk flag in Git actn

* Update snyk_sca_scan.yaml

Removed --debug switch from the test command

---------

Co-authored-by: brady-cypressio <[email protected]>

* chore: skip problematic component tests that fail on contributor PRs (#26924)

* chore: omit unused circle variables that cause contributor PR issues (#26935)

* chore: make git message warnings remain dismissable (#26812)

* feat: make git message warnings remain dismissable

* chore: update CHANGELOG

* chore: update CHANGELOG

* chore: remove unneeded code

* chore: update BannerId types

* chore: fix queryies

* chore: clean up PR

* chore: move TrackedWarning to frontend-shared

* chore: update import

* ref: move TrackedWarning to TrackedBanner

* chore: udpate CHANGELOG

* fix: update TrackedBanner to parse markdown message

* chore: set TrackedBanner default message prop

* chore: update RunsContainer

* chore: add missing tests and update alert type

---------

Co-authored-by: Stokes Player <[email protected]>

* chore: replace fast-glob with globby; remove unneeded getenv dep (#26730)

* fix: update angular dep min versions (#26908)

* fix: update angular dep min versions

* chore: update CHANGELOG

* chore: add line break to changelog

* chore: update changelog pending release date

* chore: remove whitespace

* fix: upgrade typescript from 4.2.3 to 4.9.5 (#26858)

Snyk has created this PR to upgrade typescript from 4.2.3 to 4.9.5.

See this package in npm:


See this project in Snyk:
https://app.snyk.io/org/cypress-opensource/project/5fdaebf8-b115-41d1-a2d9-857261769179?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: Bill Glesias <[email protected]>

* chore: Update v8 snapshot cache (#26762)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>

* feat: Implement testing type switch promos (#26894)

* feat: Implement testing type switch promos

* Add tests, changelog entry

* Add tests

* Fix button styling

* Styling fixes, add framework links

* Add missing testId

* run ci

* Fix spec

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* Fix styling issues

* Resolve code review findings

* Fix issue with yarn.lock
* Fix extra padding at bottom of promo
* Add tests for utm params
* Add test for switching testing type when both configured
* Fix changelog version

* Address review comments

* Widen promo when no image defined
* Prevent flash of promo before query resolves
* Reduce top margin

* reduce size of text box to match latest figma

* update button style to match figma

* increase width at which we collapse sidenav

* add short versions of the headings

* remove skeletons from header

* avoid extra height

* adjustments for column alignment

* fix flaky test

* update tests for responsive text changes

* update changelog

* restore spacing between header items

* avoid occasional flash of promo on page load

* update text handling

* fix types and tests

* Update packages/app/src/specs/SpecsList.vue

Co-authored-by: Stokes Player <[email protected]>

* updated final e2e bullet

* fix question mark icon flashing

* text formatting

* remove superfluous snapshot [skip ci]

---------

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: marktnoonan <[email protected]>
Co-authored-by: astone123 <[email protected]>
Co-authored-by: Stokes Player <[email protected]>
Co-authored-by: Lachlan Miller <[email protected]>

* chore: release internal-scripts-v1.0.0

[skip ci]

* chore: fix changelog links (#26948)

* chore: 12.14.0 release (#26950)

* chore: Update Chrome (stable) to 114.0.5735.106 and Chrome (beta) to 115.0.5790.13 (#26650)

* chore: bump cache version (#26952)

Co-authored-by: Ryan Manuel <[email protected]>

* chore: move release date (#26958)

* chore(deps): update dependency @antfu/utils to ^0.7.0 [security] (#26923)

* chore: fix base error styling (#26954)

* chore: remove low value percy snapshots (#26934)

* chore: remove low value percy snapshots

* chore: remove more low value percy snapshots [run ci]

* chore: remove more low value percy snapshots

* remove additional snapshots

* reduce snapshots

* fix types

---------

Co-authored-by: Lachlan Miller <[email protected]>

* chore: changelog updates (#26964)

* chore: stabilize side navigation bar during loading and switching testing types (#26953)

* fix: log video path if exists, regardless of compression (#26813)

* chore: print the video path whether or not compression is on or fails

* chore: fix video replacement regex

* chore: add bugfix entry

* feat: allow users to pass true to videoCompression config and only a… (#26810)

* chore: allow users to pass true to videoCompression config and only allow valudes 1-51 inclusively to be passed in

* run ci

* chore: allow zero to be option for CRF as we will coerve it to false and skip compression to have a lossless video, which has the same effect

* Update cli/CHANGELOG.md

* chore: update videoCompression types

* chore: update changelog

* Update cli/CHANGELOG.md

* Update cli/types/cypress.d.ts

Co-authored-by: Emily Rohrbough <[email protected]>

* Update packages/config/src/validation.ts

Co-authored-by: Emily Rohrbough <[email protected]>

* chore: update config snapshots

* Update packages/config/test/validation.spec.ts

Co-authored-by: Emily Rohrbough <[email protected]>

* chore: add system test on videoCompression=true coersion

* chore: document 0 as being false and not a valid CRF option for cypress video compression and make CRF valid values 1-51

* chore: fix types

* chore: fix types

* chore: fix change to log as feature and not bugfix

* chore: fix server side unit tests

---------

Co-authored-by: Emily Rohrbough <[email protected]>

* chore: trigger mac/linux/windows binary builds and v8 snapshot cache update tests

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: fix possible bad merge from mismatched snapshot in CLOUD_AUTO_CANCEL_MISMATCH test

* chore: fix possible bad merge from mismatched snapshot in record_spec system test

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

---------

Co-authored-by: Mike McCready <[email protected]>
Co-authored-by: Lachlan Miller <[email protected]>
Co-authored-by: semantic-release-bot <[email protected]>
Co-authored-by: Adam Stone-Lord <[email protected]>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Matt Schile <[email protected]>
Co-authored-by: Mahdi Khashan <[email protected]>
Co-authored-by: Mike Plummer <[email protected]>
Co-authored-by: elevatebart <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>
Co-authored-by: Ben M <[email protected]>
Co-authored-by: Mateusz Burzyński <[email protected]>
Co-authored-by: Stokes Player <[email protected]>
Co-authored-by: Mike Plummer <[email protected]>
Co-authored-by: Chris Breiding <[email protected]>
Co-authored-by: Luis Furtado <[email protected]>
Co-authored-by: Matt Henkes <[email protected]>
Co-authored-by: Brian Mann <[email protected]>
Co-authored-by: Emily Rohrbough <[email protected]>
Co-authored-by: Jordan <[email protected]>
Co-authored-by: Stokes Player <[email protected]>
Co-authored-by: Dave Kasper <[email protected]>
Co-authored-by: Mark Noonan <[email protected]>
Co-authored-by: Ely Lucas <[email protected]>
Co-authored-by: Snyk bot <[email protected]>
Co-authored-by: Jennifer Shehane <[email protected]>
Co-authored-by: cypresschris <[email protected]>
Co-authored-by: brady-cypressio <[email protected]>
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Jun 20, 2023

Released in 12.15.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v12.15.0, please open a new issue.

@cypress-bot cypress-bot bot locked as resolved and limited conversation to collaborators Jun 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants