Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable another flaky MemoryCache test #54578

Merged
merged 1 commit into from
Jun 23, 2021
Merged

Disable another flaky MemoryCache test #54578

merged 1 commit into from
Jun 23, 2021

Conversation

danmoseley
Copy link
Member

relates to #33993

This particular test is failing ~8 times a month

Execute: Web | Desktop | Web (Lens) | Desktop (SAW)

https://engsrvprod.kusto.windows.net/engineeringdata

TestResults 
| join kind=inner WorkItems on WorkItemId 
| join kind=inner Jobs on JobId
| where Finished >= datetime(2021-01-01) and Type contains "MemoryCacheSetAndRemoveTests" and Result == 'Fail'
| order by Finished desc 
| project Finished, Branch, Type, Method, Arguments, QueueName, MachineName, Message, StackTrace

example results:
Message
Assert.Equal() Failure
Expected: 2
Actual: 4
Assert.Equal() Failure
Expected: 7
Actual: 8
Assert.Equal() Failure
Expected: 7
Actual: 9
Assert.Equal() Failure
Expected: 5
Actual: 7
Assert.Equal() Failure
Expected: 8
Actual: 9
Assert.Equal() Failure
Expected: 6
Actual: 10
Assert.Equal() Failure
Expected: 3
Actual: 7
Assert.Equal() Failure
Expected: 6
Actual: 9
Assert.Equal() Failure
Expected: 11
Actual: 14
Assert.Equal() Failure
Expected: 1
Actual: 4

I am not sure whether these issues are new in 6.0 or not: unfortunately there is not enough history and MemoryCacheSetAndRemoveTests tests do not fail enough to know.

@ghost
Copy link

ghost commented Jun 22, 2021

Tagging subscribers to this area: @eerhardt, @maryamariyan, @michaelgsharp
See info in area-owners.md if you want to be subscribed.

Issue Details

relates to #33993

This particular test is failing ~8 times a month

Execute: Web | Desktop | Web (Lens) | Desktop (SAW)

https://engsrvprod.kusto.windows.net/engineeringdata

TestResults 
| join kind=inner WorkItems on WorkItemId 
| join kind=inner Jobs on JobId
| where Finished >= datetime(2021-01-01) and Type contains "MemoryCacheSetAndRemoveTests" and Result == 'Fail'
| order by Finished desc 
| project Finished, Branch, Type, Method, Arguments, QueueName, MachineName, Message, StackTrace

example results:
Message
Assert.Equal() Failure
Expected: 2
Actual: 4
Assert.Equal() Failure
Expected: 7
Actual: 8
Assert.Equal() Failure
Expected: 7
Actual: 9
Assert.Equal() Failure
Expected: 5
Actual: 7
Assert.Equal() Failure
Expected: 8
Actual: 9
Assert.Equal() Failure
Expected: 6
Actual: 10
Assert.Equal() Failure
Expected: 3
Actual: 7
Assert.Equal() Failure
Expected: 6
Actual: 9
Assert.Equal() Failure
Expected: 11
Actual: 14
Assert.Equal() Failure
Expected: 1
Actual: 4

I am not sure whether these issues are new in 6.0 or not: unfortunately there is not enough history and MemoryCacheSetAndRemoveTests tests do not fail enough to know.

Author: danmoseley
Assignees: -
Labels:

area-Extensions-Caching
Milestone: -

@danmoseley danmoseley requested a review from maryamariyan June 22, 2021 20:27
@danmoseley
Copy link
Member Author

It looks like we've disabled tests in this area for flakiness back in 5.0 timeframe:
#39930
#34207
#34142

So there is at least some evidence that we have not regressed this since we shipped last, which is my main concern.

@maryamariyan maryamariyan requested a review from adamsitnik June 22, 2021 23:09
@stephentoub stephentoub merged commit 1f5033c into dotnet:main Jun 23, 2021
@danmoseley danmoseley deleted the disablememorycache branch June 23, 2021 01:18
thaystg added a commit to thaystg/runtime that referenced this pull request Jun 23, 2021
@thaystg
Merge remote-tracking branch 'origin/main' into thays_fix_exception_i…
ae2a17c 
…nitial_config

* origin/main: (362 commits)
  [wasm][debugger] Reuse debugger-agent on wasm debugger (dotnet#52300)
  Put Crossgen2 in sync with dotnet#54235 (dotnet#54438)
  Move System.Object serialization to ObjectConverter (dotnet#54436)
  Move setting fHasVirtualStaticMethods out of sanity check section (dotnet#54574)
  [wasm] Compile .bc->.o in parallel, before passing to the linker (dotnet#54053)
  Change PathInternal.IsCaseSensitive to a constant (dotnet#54340)
  Make mono_polling_required a public symbol (dotnet#54592)
  Respect EventSource::IsSupported setting in more codepaths (dotnet#51977)
  Root ComActivator for hosting (dotnet#54524)
  Add ILLink annotations to S.D.Common related to DbConnectionStringBuilder (dotnet#54280)
  Fix finalizer issue with regions (dotnet#54550)
  Add support for multi-arch install locations (dotnet#53763)
  Update library testing docs page to reduce confusion (dotnet#54324)
  [FileStream] handle UNC and device paths (dotnet#54483)
  Update NetAnalyzers version (dotnet#54511)
  Added runtime dependency to fix the intermittent test failures (dotnet#54587)
  Disable failing System.Reflection.Tests.ModuleTests.GetMethods (dotnet#54564)
  [wasm] Move AOT builds from `runtime-staging` to `runtime` (dotnet#54577)
  Keep obj node for ArrayIndex. (dotnet#54584)
  Disable another failing MemoryCache test (dotnet#54578)
  ...
@karelz karelz mentioned this pull request Jun 24, 2021
Closed
@ghost ghost locked as resolved and limited conversation to collaborators Jul 23, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@eerhardt eerhardt eerhardt approved these changes

@stephentoub stephentoub stephentoub approved these changes

@maryamariyan maryamariyan Awaiting requested review from maryamariyan

@adamsitnik adamsitnik Awaiting requested review from adamsitnik

Assignees
No one assigned
Labels
area-Extensions-Caching
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@danmoseley @stephentoub @eerhardt