Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unreliable test: shows warning 5 minutes before shutdown for 60 minute timeout #6213

Open
Hweinstock opened this issue Dec 11, 2024 · 2 comments · Fixed by #6265
Open

unreliable test: shows warning 5 minutes before shutdown for 60 minute timeout #6213

Hweinstock opened this issue Dec 11, 2024 · 2 comments · Fixed by #6265

Comments

@Hweinstock
Copy link
Contributor

Test Details

  • Name of test: shows warning 5 minutes before shutdown for 60 minute timeout
  • OS of failure: Linux
  • VSCode version of failure (minimum/insider/stable): (1.95.3)
  • Link to failing run:
  • Link to failing test:
    it('shows warning 5 minutes before shutdown for 60 minute timeout', async function () {
    setInitialOffset(54)
    await inactivityMsg.init(60, devEnvActivity as unknown as DevEnvActivity, relativeMinuteMillis)
    await assertMessagesShown([
    ['Your CodeCatalyst Dev Environment has been inactive for 55 minutes, and will stop soon.', 55],
    ])
    })

Log of Test Failure

AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
+ actual - expected

  {
    message: 'Your CodeCatalyst Dev Environment has been inactive for 55 minutes, and will stop soon.',
+   minute: 54
-   minute: 55
  }
    at assertMessagesShown (/codebuild/output/src1534573611/src/github.com/aws/aws-toolkit-vscode/packages/core/src/testInteg/codecatalyst/devEnv.test.ts:144:20)
    at async Context.<anonymous> (/codebuild/output/src1534573611/src/github.com/aws/aws-toolkit-vscode/packages/core/src/testInteg/codecatalyst/devEnv.test.ts:76:9)

      + expected - actual

       {
         "message": "Your CodeCatalyst Dev Environment has been inactive for 55 minutes, and will stop soon."
      -  "minute": 54
      +  "minute": 55
       }

Additional Information

Integ test

@justinmk3
Copy link
Contributor

This test is definitely a known issue. If we can just retry it, we should. Or possibly skip() for now.

justinmk3 pushed a commit that referenced this issue Dec 18, 2024
## Problem
fix #6213

## Solution
- The important part of this test is that the message comes after being inactive. If the message comes slightly early/late, that is okay. If it doesn't come at all, then we have problems and should fail. 
- Pass test if message comes within a minute of expected.
karanA-aws pushed a commit to karanA-aws/aws-toolkit-vscode that referenced this issue Jan 17, 2025
## Problem
fix aws#6213

## Solution
- The important part of this test is that the message comes after being inactive. If the message comes slightly early/late, that is okay. If it doesn't come at all, then we have problems and should fail. 
- Pass test if message comes within a minute of expected.
@Hweinstock
Copy link
Contributor Author

Another failure on 0d8c8fc. Looks like 1 minute buffer period is no longer effective at mitigating flakiness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants