test(queryClient): Adjust performance threshholds

[SebKranz]

The pipeline was found to sometimes exceed the threshold of 500ms. Since I expect the the required time to increase linearly, increase the row count by *1.5 and quadruple the time-threshhold.

Local Tests on my Machine

(yellow is prior to #6918, red is after, all times in ms)

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
query	⬜️ Ignored (Inspect)	Visit Preview		Feb 19, 2024 7:45pm

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit bc6e65e. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 1 target

• nx affected --targets=test:format,test:sherif,test:knip,test:eslint,test:lib,test:types,test:build,build --parallel=3

---

Sent with 💌 from NxCloud.

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit bc6e65e:

Sandbox	Source
@tanstack/query-example-angular-basic	Configuration
@tanstack/query-example-react-basic-typescript	Configuration
@tanstack/query-example-solid-basic-typescript	Configuration
@tanstack/query-example-svelte-basic	Configuration
@tanstack/query-example-vue-basic	Configuration

[SebKranz]

Unfortunately, I'm a bit unsure whether the best approach would be to increase or decrease the threshold & query-count here.

Increasing will potentially slow down the pipeline whereas decreasing might lead to false negatives. 500ms felt like a good compromise but apparently wasn't lenient enough. (see #6918 (comment))

@milhamm do you have opinions on this?

[TkDodo]

how about a test that tests if execution doesn't grow exponentially instead ?

[SebKranz]

how about a test that tests if execution doesn't grow exponentially instead ?

I fear we would run into similar flakiness, because the cpu-load might not be stable over the course of the test (eg see the variance in my local tests). Also, the tests would become more complex since it would need some kind of regression algorithm.

I would suggest the following plan for now:

1. Try these new defaults and see if the problem occurs again
2. If it does, enable retries in vitest like so:

   test(
       'should set 15k data in less than 2000ms',
       () => {
            // ...
       }
       { retry: 3 },
    )

3. If that still fails, I'll consider an alternative testing approach.

[TkDodo]

we have retries in vitest enabled

[SebKranz]

There are some unrelated tests that depend on the event loop not being blocked (such as useIsMutating).
If the performance tests are scheduled at the same time as these tests, their timers execute delayed and fail the unrelated test.

Unless we run them sequentially, there's no feasible way of doing performance-testing IMO.

I'll check if there's a good way of marking specific tests for sequential execution in NX, but it might be best to just remove the performance tests for now.

[SebKranz]

I've created a PR to remove these tests: #6946