feat: Cypress Cloud data on Specs page and Runs page use local Git data if available #26991

warrensplayer · 2023-06-09T17:31:53Z

Closes ACI - Update logic for pulling runs on Runs page and Specs list #26693

Additional details

Before this change, the Specs page and the Runs page did not use local Git data to determine which runs to show.

Specs page

For the Specs page, an algorithm was used within the Cloud query to find runs for the current Branch name and then fall back to the default branch in the Cloud before that. This did not always match the actual Git tree the developer was using. The new logic will use the same polling introduced with the Debug page to pull the "latest" runs that match the Git tree and pass those to new Cloud GQL fields to determine spec statuses for those runs. The code changes under packages/app/src/specs will reflect the update to the GQL fields used. The updates to packages/app/src/pages/Specs/Index.vue include the introduction of the useRelevantRuns composable.

Runs page

For the Runs page, it previously only just pulled the latest 10 runs from the Cloud project. It would cache these runs and then use a cursor pattern to pull new runs that were more recent than the latest run. Now, the runs page preserves that logic as a fallback and it has been encapsulated into the useProjectRuns composable. Another composable called useGitTreeRuns was added that uses the useRelevantRuns composable to pull runs relevant to the local Git tree and use those runs to display. This will always just show the latest 10 runs for the local Git tree. This may be expanded in a future initiative.

For either scenario, RunCard now uses a subscription to update itself if it is currently Running.

Additional changes on this branch

Updated the E2E test for the debug page to remove flake

Steps to test

Testing with Git

Find a project that is using Git
Verify that the Specs page shows the run dot information and flaky information related to the local Git tree
- Check the tooltips for the Runs and the flaky badges as well
- Create a new branch, break a test, and record a run
- Verify the Run status dots update while the tests are being recorded
Verify the Runs page shows runs related to the local Git tree
- Verify that a RUNNING run updates correctly.

Testing without Git

Use a project without Git to record runs
- Or what I did was copy the directory for a project that already had runs and remove the .git folder
Verify the Specs page does not show Cloud information (this is existing behavior)
Verify the Runs page shows the latest runs for the project

How has the user experience changed?

PR Tasks

Have tests been added/updated?
Has a PR for user-facing changes been opened in cypress-documentation?
[-] Have API changes been updated in the type definitions?

cypress · 2023-06-12T19:39:15Z

5 flaky tests on run #47774 ↗︎

0	5818	81	0	5

Details:

Replace DEBUG with NOTE
Project: cypress	Commit: `0b099a1188`
Status: Passed	Duration: 12:41 💡
Started: Jun 15, 2023 7:05 PM	Ended: Jun 15, 2023 7:18 PM

e2e/origin/commands/navigation.cy.ts • 1 flaky test • 5x-driver-chrome:beta

View Output Video

Test		Artifacts
cy.origin navigation > #consoleProps > .go()		`Output` `Video`

commands/net_stubbing.cy.ts • 1 flaky test • 5x-driver-chrome:beta

View Output Video

Test		Artifacts
network stubbing > intercepting request > can delay and throttle a StaticResponse		`Output` `Video`

cypress/cypress.cy.js • 3 flaky tests • 5x-driver-chrome:beta

View Output Video

Test		Artifacts
... > correctly returns currentRetry		`Output` `Video`
... > correctly returns currentRetry		`Output` `Video`
... > correctly returns currentRetry		`Output` `Video`

_{This comment has been generated by cypress-bot as a result of this
project's GitHub integration settings.}

lmiller1990

I am still going through this and testing it.

I'm posting my QA comments here: #26693

I wonder if I need to test with a built binary, and not the monorepo locally? Could that be causing the issues I'm seeing during my QA?

packages/app/src/specs/SpecsListRunWatcher.vue

lmiller1990 · 2023-06-14T00:10:22Z

packages/app/src/specs/SpecsListRunWatcher.vue

+import { gql, useSubscription } from '@urql/vue'
+import { computed } from 'vue'
+import { SpecsListRunWatcherDocument } from '../generated/graphql'
+import type { RelevantRunInfo } from '@packages/frontend-shared/cypress/support/generated/test-graphql-types.gen'


Not sure this import is correct - it is importing code from the test package into a production component.

Updated 029679a (#26991)

marktnoonan · 2023-06-14T22:00:02Z

Hmm, should this have any effect on the Debug page, or would that be a different issue. I can see a run on the Runs page, and the result of the run on the Specs List, but the Debug page doesn't know about that run yet.

Here's a loom, I would start at around 1m 30s to see this: https://www.loom.com/share/5d2a183eadf24fb8b41a7d8afd53fe8e

marktnoonan · 2023-06-14T22:12:04Z

When we talk about this using the local git tree, what should my expectations be here, while I am on the vue-conf-23-all-fixed branch of my project?

On specs and runs page, it shows runs from a branch that I am not currently on called lifecycle-hooks. I was thinking those runs would be excluded.

Update - based on #26693 (comment) my expectation wasn't correct, this seems to be working normally.

I will read everything once more in the morning, I think I am missing something about how these pages "use local Git data to determine which runs to show" 🤔

lmiller1990 · 2023-06-15T00:14:55Z

I think I found the issue I was having in testing @warrensplayer. It was not picking up runs associated with a GH Action triggered via creating a PR, like the ones in red rectangles in this image. I think this is because GH makes a merge commit in the background, which is not present in my local git repo, only the remote, so those were not picked up.

I got it working by adding those branches to my workflows.yml: https://github.com/cypress-io/aci-test-projects/commit/92e12b3e4406adb77e7b81c5cd89032dd763b577

This might be an issue, depending on how people have set up their GH Actions workflows. Your logic works as it is supposed to once I enabled running GH actions on my individual branches.

This change might cause issues for some users, hard to say, but I'll continue with testing/reviewing and disregard the case of remote only commits for now. I guess this is a fundamental difference between how Cypress Cloud and App works: App uses the local git, and Cloud doesn't care about that at all. I can see this might be a bit confusing for some users, if certain runs appear on Cloud but not locally (sure was confusing for me).

lmiller1990 · 2023-06-15T00:36:45Z

Update... Stokes pointed out we can work around ^^^ issue using a GH setting: https://github.com/actions/checkout#checkout-pull-request-head-commit-instead-of-merge-commit

Good to know for now, this is probably okay for now, we can figure the details as a fast follow.

lmiller1990

This is looking pretty good. I don't see any blocking issues right now. I'll continue testing, but other than the known edge case, which we discussed (GH Actions HEAD commit) the feature seems to be working as expected. The code is far cleaner now too, among the GraphQL patterns getting more streamlined and the improvements to the E2E tests. Great job!

lmiller1990 · 2023-06-15T00:56:31Z

packages/graphql/test/stubCloudTypes.ts

@@ -135,6 +136,24 @@ export function createCloudProject (config: Partial<ConfigFor<CloudProject>>) {
        nodes: connectionData.edges.map((e) => e.node),
      }
    },
+    runsByCommitShas (args: CloudProjectRunsByCommitShasArgs) {
+      console.log('***** runsByCommitShas', args)


Do we want to remove this log?

lmiller1990 · 2023-06-15T00:57:06Z

packages/graphql/test/stubCloudTypes.ts

+    runsByCommitShas (args: CloudProjectRunsByCommitShasArgs) {
+      console.log('***** runsByCommitShas', args)
+
+      return args.commitShas?.map((sha, i) => {


Nice, this test data is easy to read. Some of the stubs are a bit confusing, I like this style, though. 👍

lmiller1990 · 2023-06-15T00:58:54Z

packages/graphql/src/schemaTypes/objectTypes/gql-Subscription.ts

    t.field('relevantRuns', {
      type: RelevantRun,
      description: 'Subscription that polls the cloud for new relevant runs that match local git commit hashes',
      args: {
        location: nonNull(enumType({
          name: 'RelevantRunLocationEnum',
-          members: ['DEBUG', 'SIDEBAR'],
+          members: ['DEBUG', 'SIDEBAR', 'RUNS', 'SPECS'],


Should we make this into a separate enum, then re-use it in https://github.com/cypress-io/cypress/pull/26991/files#diff-1cfff3b283c5a3eb1e104bcd05900902c53efbc4ea7286747d8c504adc6cc978R51 ?

Actually we have it, RelevantRunLocationEnum, we could use on the front-end, too?

lmiller1990 · 2023-06-15T01:05:11Z

packages/app/src/runs/useProjectRuns.ts

+fragment RunsContainer_RunsConnection on CloudRunConnection {
+  nodes {
+    id
+    ...RunCard


Not a blocker but relevant to the continued development of GraphQL patterns - we've got two composables tied to the RunCard fragment, which is defined in UI component. My first instinct was it's not good to couple the data fetching layer to the component, but I'm starting to think this is actually the right pattern - the way we use GraphQL, the fragment is tied to the UI component, so it might actually make sense here.

It's definitely a bit counter intuitive at first, you really need to rethinking a lot of what you'd do with a traditional REST API when working with GraphQL.

I am taking a liking to the pattern when components are self contained and define/manage their own data, especially with the subscriptions and useSubscription composable.

lmiller1990 · 2023-06-15T01:10:03Z

packages/app/src/specs/SpecsListRunWatcher.vue

+  emits('runUpdate')
+}
+
+useSubscription({ query: SpecsListRunWatcherDocument, variables, pause: shouldPause }, handleUpdate)


Did not realize useSubscription has a second arg - looks quite useful for transforming data, but also neat usage here to handle an update with the date.

lmiller1990 · 2023-06-15T01:18:36Z

packages/data-context/src/DataContext.ts

@@ -220,11 +219,6 @@ export class DataContext {
    return new ProjectDataSource(this)
  }

-  @cached


Good bye remotePolling ⚰️

Seems our polling / subscriptions story is getting more well defined, any need to update any docs anywhere? We might want to do an update of https://github.com/cypress-io/cypress/blob/develop/guides/graphql-subscriptions.md at some point to capture the ins-and-outs of the whole polling situation, it's not really documented and not very standard - I think it'd be a bit challenging to onboard someone, even with GraphQL experience, to our current App <> Cloud setup.

Not going to block here, but might be a nice thing to do if we have some free time between now and Q3 getting under way.

warrensplayer · 2023-06-15T12:49:41Z

Hmm, should this have any effect on the Debug page, or would that be a different issue. I can see a run on the Runs page, and the result of the run on the Specs List, but the Debug page doesn't know about that run yet.

@marktnoonan No, it should not have any affect on the Debug page. I did note one scenario on that page when testing that I need to retest and write up regarding changing branches. In that case, the Specs and Runs page were seeing the new data for the branch that was switched to right away, but the Debug page was waiting for the next polling cycle (up to 30 sec) to see it.

In this case, I watched your Look and I wonder if it has something to do with that run that died that still appears to be running. I will try and recreate the scenario to test.

marktnoonan · 2023-06-15T18:02:58Z

packages/app/src/specs/SpecsListRunWatcher.vue

@@ -0,0 +1,50 @@
+/**
+ Non rendering component used to watch running runs and emit an event


I'm fine with a non-rendering component, but is this something that might as well be a composable that exposes a ref, instead of a component that emits an event?

I have definitely made at least one renderless component mostly to follow the convention we had of doing GQL requests in SFCs, but we seem to have not maintained that convention in other place. So just curious if there's some other reason to make this a component that I'm missing? (Idk, ease of hooking into lifecycle or something?).

I was not sure of another way of using the Urql useSubscription across an array of options without having this renderless component. This implementation allows it to get mounted for each run on the page after the original page has mounted and the GQL queries returned. If anyone has another pattern for this, please share!

This seems fine to me.

marktnoonan · 2023-06-15T18:23:29Z

packages/app/cypress/e2e/debug.cy.ts

+    cy.findByTestId('sidebar-link-debug-page').click()
+    cy.findByTestId('debug-container').should('be.visible')
+
+    cy.findByTestId('header-top').contains('chore: testing cypress')


In general I would like to add some more visibility assertions to this spec file where we use contains, particularly because we do not have any percy snapshots in app e2e tests.

Without visibility assertions, we are confirming what is in the DOM, but we won't catch problems caused by CSS that might still hide the elements in various ways.

For example, for this assertion, we are passing at this point, after DOM is updated but before CSS rendering has caught up:

makes me really want to make a containsVisible custom command and just never think about this again.

marktnoonan

Well done making testing better, I sure to love to see those deletions. Local testing all checks out, I poked around in the various test cases as well and all seems good to me.

cypress-bot · 2023-06-20T18:55:51Z

Released in 12.15.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v12.15.0, please open a new issue.

warrensplayer added 14 commits May 15, 2023 15:27

Move existing runs page logic to composable

fac3d86

Adding useGitTreeRuns composable and associated refactors

f392db4

Merge branch 'develop' into stokes/26693_spec_run_logic

f028693

Adding more detailed message to nexus slow guard

2972a25

Introduce RunsComposable type

d3152f3

Merge branch 'develop' into stokes/26693_spec_run_logic

59bc099

Add subscription to RunCard

6f90543

Update Debug E2E test to new patterns

c26768e

Merge branch 'develop' into stokes/26693_spec_run_logic

1e67eae

Updating runs tests

c745387

Fixing tests

991828a

SpecsList updates and tests [run ci]

8bc8dca

Fixing tests

dd8c5a3

Add running run support to specs list

5705c29

warrensplayer added 4 commits June 12, 2023 19:56

Fix typing

039d8d2

Adding more Runs page E2E tests

ac9e0b7

Merge branch 'develop' into stokes/26693_spec_run_logic

ddcfdc0

change log

16e63b0

warrensplayer changed the title ~~Move existing runs page logic to composable~~ feat: Cypress Cloud data on Specs page and Runs page use local Git data if available Jun 13, 2023

warrensplayer added 4 commits June 13, 2023 10:34

Update workflow to build all platforms

94eb5d8

Fixing tests

32807c8

One more test fix

8ab29db

Update CI workflow to persist artifacts

49b4475

warrensplayer marked this pull request as ready for review June 13, 2023 16:49

warrensplayer requested a review from a team June 13, 2023 16:49

marktnoonan assigned marktnoonan and unassigned marktnoonan Jun 13, 2023

marktnoonan self-requested a review June 13, 2023 17:42

Cleanup unused variables

d34c5b9

Remove comment

e9dc2ed

lmiller1990 reviewed Jun 14, 2023

View reviewed changes

warrensplayer added 4 commits June 14, 2023 13:43

Type fix and use correct relevant runs

029679a

Merge branch 'develop' into stokes/26693_spec_run_logic

67fd979

Reverting workflow changes

4babcb1

Removing unused code

493189c

lmiller1990 approved these changes Jun 15, 2023

View reviewed changes

warrensplayer added 2 commits June 15, 2023 08:20

Remove console.log

92c9da2

Merge branch 'develop' into stokes/26693_spec_run_logic

1c537d1

Fix type import

7a0c5e1

marktnoonan reviewed Jun 15, 2023

View reviewed changes

Replace DEBUG with NOTE

0b099a1

marktnoonan reviewed Jun 15, 2023

View reviewed changes

marktnoonan approved these changes Jun 15, 2023

View reviewed changes

warrensplayer merged commit 9cb7e1f into develop Jun 16, 2023

warrensplayer deleted the stokes/26693_spec_run_logic branch June 16, 2023 00:02

cypress-bot bot locked as resolved and limited conversation to collaborators Jun 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Cypress Cloud data on Specs page and Runs page use local Git data if available #26991

feat: Cypress Cloud data on Specs page and Runs page use local Git data if available #26991

warrensplayer commented Jun 9, 2023 •

edited

Loading

cypress bot commented Jun 12, 2023 •

edited

Loading

lmiller1990 left a comment •

edited

Loading

lmiller1990 Jun 14, 2023

warrensplayer Jun 14, 2023

marktnoonan commented Jun 14, 2023

marktnoonan commented Jun 14, 2023 •

edited

Loading

lmiller1990 commented Jun 15, 2023

lmiller1990 commented Jun 15, 2023

lmiller1990 left a comment

lmiller1990 Jun 15, 2023

lmiller1990 Jun 15, 2023

lmiller1990 Jun 15, 2023

lmiller1990 Jun 15, 2023

lmiller1990 Jun 15, 2023

lmiller1990 Jun 15, 2023

lmiller1990 Jun 15, 2023

lmiller1990 Jun 15, 2023

warrensplayer commented Jun 15, 2023

marktnoonan Jun 15, 2023

warrensplayer Jun 15, 2023

lmiller1990 Jun 15, 2023

marktnoonan Jun 15, 2023

marktnoonan Jun 15, 2023

marktnoonan left a comment

cypress-bot bot commented Jun 20, 2023

		@@ -0,0 +1,50 @@
		/**
		Non rendering component used to watch running runs and emit an event

feat: Cypress Cloud data on Specs page and Runs page use local Git data if available #26991

feat: Cypress Cloud data on Specs page and Runs page use local Git data if available #26991

Conversation

warrensplayer commented Jun 9, 2023 • edited Loading

Additional details

Specs page

Runs page

Additional changes on this branch

Steps to test

Testing with Git

Testing without Git

How has the user experience changed?

PR Tasks

cypress bot commented Jun 12, 2023 • edited Loading

5 flaky tests on run #47774 ↗︎

lmiller1990 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marktnoonan commented Jun 14, 2023

marktnoonan commented Jun 14, 2023 • edited Loading

lmiller1990 commented Jun 15, 2023

lmiller1990 commented Jun 15, 2023

lmiller1990 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

warrensplayer commented Jun 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marktnoonan left a comment

Choose a reason for hiding this comment

cypress-bot bot commented Jun 20, 2023

warrensplayer commented Jun 9, 2023 •

edited

Loading

cypress bot commented Jun 12, 2023 •

edited

Loading

lmiller1990 left a comment •

edited

Loading

marktnoonan commented Jun 14, 2023 •

edited

Loading