test: simplify endpoint integration tests #6373

trivikr · 2024-08-09T19:11:21Z

Issue

Internal JS-5234

Description

Simplifies endpoint integration tests as follows:

removes unused variables/functions
use describe.each
pass namespace when running tests
run tests which were skipped in the past

Testing

Before

$ yarn jest -c tests/endpoints-2.0/jest.config.js tests/endpoints-2.0/endpoints-integration.spec.ts
...
Test Suites: 1 passed, 1 total
Tests:       124 skipped, 14548 passed, 14672 total

After

$ yarn jest -c tests/endpoints-2.0/jest.config.js tests/endpoints-2.0/endpoints-integration.spec.ts
...
Test Suites: 1 passed, 1 total
Tests:       14672 passed, 14672 total

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Test Suites: 1 failed, 1 total Tests: 22 failed, 124 skipped, 14526 passed, 14672 total Snapshots: 0 total Time: 61.488 s

trivikr · 2024-08-09T21:03:14Z

Following tests are failing after passing Command from namespace

$ yarn jest -c tests/endpoints-2.0/jest.config.js tests/endpoints-2.0/endpoints-integration.spec.ts
...
● client list › client-s3-control endpoint test cases › Vanilla outposts without ARN region + access point ARN@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.us-west-2.amazonaws.com/"
    Received string:    "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › Vanilla outposts with ARN region + access point ARN@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.us-east-1.amazonaws.com/"
    Received string:    "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › accept an access point ARN@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.us-west-2.amazonaws.com/"
    Received string:    "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › vanilla outposts china@cn-north-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.cn-north-1.amazonaws.com.cn/"
    Received string:    "https://123456789012.s3-control.cn-north-1.amazonaws.com.cn/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › gov region@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.us-west-2.amazonaws.com/"
    Received string:    "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › gov cloud with fips@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts-fips.us-west-2.amazonaws.com/"
    Received string:    "https://123456789012.s3-control-fips.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › govcloud with fips + arn region@us-gov-west-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts-fips.us-gov-east-1.amazonaws.com/"
    Received string:    "https://123456789012.s3-control-fips.us-gov-west-1.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › gov region@cn-north-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.cn-north-1.amazonaws.com.cn/"
    Received string:    "https://123456789012.s3-control.cn-north-1.amazonaws.com.cn/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › gov cloud with fips@cn-north-1

    EndpointError: Partition does not support FIPS

      85 |                 params
      86 |               )) as EndpointParams;
    > 87 |               const observed = resolveEndpoint(ruleSet, { endpointParams });
         |                                               ^
      88 |               assertEndpointResolvedCorrectly(endpoint, observed);
      89 |             }
      90 |           } else {

      at evaluateErrorRule (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:401:9)
      at evaluateRules (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:431:7)
      at evaluateTreeRule (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:416:10)
      at evaluateRules (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:433:35)
      at evaluateTreeRule (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:416:10)
      at evaluateRules (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:433:35)
      at evaluateTreeRule (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:416:10)
      at evaluateRules (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:433:35)
      at resolveEndpoint (../../node_modules/@smithy/util-endpoints/dist-cjs/index.js:462:20)
      at Object.<anonymous> (endpoints-integration.spec.ts:87:47)

  ● client list › client-s3-control endpoint test cases › govcloud with fips + arn region@us-gov-west-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts-fips.us-gov-east-1.amazonaws.com/"
    Received string:    "https://123456789012.s3-control-fips.us-gov-west-1.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › gov region@af-south-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.af-south-1.amazonaws.com/"
    Received string:    "https://123456789012.s3-control.af-south-1.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › gov cloud with fips@af-south-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts-fips.af-south-1.amazonaws.com/"
    Received string:    "https://123456789012.s3-control-fips.af-south-1.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › govcloud with fips + arn region@us-gov-west-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts-fips.us-gov-east-1.amazonaws.com/"
    Received string:    "https://123456789012.s3-control-fips.us-gov-west-1.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › outpost access points support dualstack@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.us-west-2.api.aws/"
    Received string:    "https://123456789012.s3-control.dualstack.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › outpost access points support dualstack@af-south-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.af-south-1.api.aws/"
    Received string:    "https://123456789012.s3-control.dualstack.af-south-1.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › outpost access points support fips + dualstack@af-south-1

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts-fips.af-south-1.api.aws/"
    Received string:    "https://123456789012.s3-control-fips.dualstack.af-south-1.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › Account ID set inline and in ARN but they both match@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.us-west-2.amazonaws.com/"
    Received string:    "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › Account ID set inline and in ARN and they do not match@us-west-2

    expect(received).toBeUndefined()

    Received: "https://999999999999.s3-control.us-west-2.amazonaws.com/"

      107 |               const observedError = await (async () => defaultEndpointResolver(endpointParams as any))().catch(pass);
      108 |               expect(observedError).not.toBeUndefined();
    > 109 |               expect(observedError?.url).toBeUndefined();
          |                                          ^
      110 |               // expect(normalizeQuotes(String(observedError))).toContain(normalizeQuotes(error));
      111 |             }
      112 |           } else {

      at Object.<anonymous> (endpoints-integration.spec.ts:109:42)

  ● client list › client-s3-control endpoint test cases › endpoint url with s3-outposts@us-west-2

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://beta.example.com/"
    Received string:    "https://123456789012.beta.example.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › Outpost Accesspoint ARN with arn region and client region mismatch with UseArnRegion=false

    expect(received).toBeUndefined()

    Received: "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      107 |               const observedError = await (async () => defaultEndpointResolver(endpointParams as any))().catch(pass);
      108 |               expect(observedError).not.toBeUndefined();
    > 109 |               expect(observedError?.url).toBeUndefined();
          |                                          ^
      110 |               // expect(normalizeQuotes(String(observedError))).toContain(normalizeQuotes(error));
      111 |             }
      112 |           } else {

      at Object.<anonymous> (endpoints-integration.spec.ts:109:42)

  ● client list › client-s3-control endpoint test cases › Accesspoint ARN with region mismatch and UseArnRegion unset

    expect(received).toContain(expected) // indexOf

    Expected substring: "https://s3-outposts.us-east-1.amazonaws.com/"
    Received string:    "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      129 |   const { authSchemes } = properties || {};
      130 |   if (url) {
    > 131 |     expect(observed.url.href).toContain(new URL(url).href);
          |                               ^
      132 |     expect(Math.abs(observed.url.href.length - url.length)).toBeLessThan(2);
      133 |   }
      134 |   if (headers) {

      at assertEndpointResolvedCorrectly (endpoints-integration.spec.ts:131:31)
      at Object.<anonymous> (endpoints-integration.spec.ts:88:15)

  ● client list › client-s3-control endpoint test cases › Accesspoint ARN with partition mismatch and UseArnRegion=true

    expect(received).toBeUndefined()

    Received: "https://123456789012.s3-control.us-west-2.amazonaws.com/"

      107 |               const observedError = await (async () => defaultEndpointResolver(endpointParams as any))().catch(pass);
      108 |               expect(observedError).not.toBeUndefined();
    > 109 |               expect(observedError?.url).toBeUndefined();
          |                                          ^
      110 |               // expect(normalizeQuotes(String(observedError))).toContain(normalizeQuotes(error));
      111 |             }
      112 |           } else {

      at Object.<anonymous> (endpoints-integration.spec.ts:109:42)

Test Suites: 1 failed, 1 total
Tests:       22 failed, 124 skipped, 14526 passed, 14672 total
Snapshots:   0 total
Time:        28.648 s

trivikr · 2024-08-12T16:27:06Z

The fix in #6376 was verified

$ yarn jest -c tests/endpoints-2.0/jest.config.js tests/endpoints-2.0/endpoints-integration.spec.ts
...
Test Suites: 1 passed, 1 total
Tests:       124 skipped, 14548 passed, 14672 total
Snapshots:   0 total
Time:        23.957 s, estimated 29 s

…nRegion

…global' tests

tests/endpoints-2.0/endpoints-integration.spec.ts

github-actions · 2024-08-27T00:03:35Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

trivikr added 5 commits August 9, 2024 18:08

test(endpoints): remove unused namespace

e13b54f

test(endpoints): move code inside describe

45a1129

test(endpoints): use describe.each

728b7b9

test: remove redundant function is(Endpoint|Error)Expectation

8d6af17

test(endpoints): read tests from serviceModel

c366769

trivikr force-pushed the refactor-endpoints-integration-test branch from 4bb89fa to c366769 Compare August 9, 2024 19:25

trivikr added 3 commits August 9, 2024 19:34

test(endpoint): move runTestCase inside runTestCases

4767f6c

test(endpoints): re-introduce namespace

a360f3e

Test Suites: 1 failed, 1 total Tests: 22 failed, 124 skipped, 14526 passed, 14672 total Snapshots: 0 total Time: 61.488 s

test(endpoints): read serviceId from service model

a50628b

test(endpoints): skip reading defaultEndpointResolver

6e5b22d

trivikr force-pushed the refactor-endpoints-integration-test branch from 038e03f to 6e5b22d Compare August 9, 2024 22:11

This was referenced Aug 12, 2024

Populate memberName as contextParams value smithy-lang/smithy-typescript#1365

Merged

fix(client-s3-control): populate memberName as contextParams value #6376

Merged

trivikr added 6 commits August 12, 2024 19:31

test(endpoints): use names imports from Node.js core

5d16bbe

test(endpoints): make namespace and model variables const

d24bff5

test(endpoints): remove redudnant check for skipping S3Control::UseAr…

caa0e01

…nRegion

test(endpoints): run params.UseGlobalEndpoint and params.Region 'aws-…

9632b56

…global' tests

test(endpoints): use variable command for storing Command import

bb7b535

test(endpoints): enable error validation except s3 empty arn type

f456c0e

trivikr marked this pull request as ready for review August 12, 2024 20:57

trivikr requested a review from a team as a code owner August 12, 2024 20:57

kuhe reviewed Aug 12, 2024

View reviewed changes

tests/endpoints-2.0/endpoints-integration.spec.ts Show resolved Hide resolved

kuhe approved these changes Aug 12, 2024

View reviewed changes

trivikr merged commit 88d384c into aws:main Aug 12, 2024
5 checks passed

trivikr deleted the refactor-endpoints-integration-test branch August 12, 2024 21:21

github-actions bot locked as resolved and limited conversation to collaborators Aug 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: simplify endpoint integration tests #6373

test: simplify endpoint integration tests #6373

trivikr commented Aug 9, 2024 •

edited

Loading

trivikr commented Aug 9, 2024 •

edited

Loading

trivikr commented Aug 12, 2024

github-actions bot commented Aug 27, 2024

test: simplify endpoint integration tests #6373

test: simplify endpoint integration tests #6373

Conversation

trivikr commented Aug 9, 2024 • edited Loading

Issue

Description

Testing

Before

After

trivikr commented Aug 9, 2024 • edited Loading

trivikr commented Aug 12, 2024

github-actions bot commented Aug 27, 2024

trivikr commented Aug 9, 2024 •

edited

Loading

trivikr commented Aug 9, 2024 •

edited

Loading