fix: use in memory provider for e2e suites

[znonthedev]

This PR

Removes setup using Flagd Provider and uses In-Memory Provider

Fixes #712

[znonthedev]

I had issues running the e2e locally

Running one project: client-e2e
 FAIL   client-e2e  packages/client/e2e/step-definitions/evaluation.spec.ts
  ● Test suite failed to run

    packages/client/e2e/step-definitions/evaluation.spec.ts:10:63 - error TS2307: Cannot find module '../..' or its corresponding type declarations.

    10 import { OpenFeature, ProviderEvents, InMemoryProvider } from '../..';

I saw a PR related to this #471

I changed the path locally to '../../src/' to get it working

[lukas-reining]

Hey @znonthedev,
thanks for submitting that change, this looks good!

The thing left would be to remove the flagd-testbed from the GitHub Action.

https://github.com/open-feature/js-sdk/blob/main/.github/workflows/pr-checks.yaml#L68-L72

[toddbaert]

Thanks @znonthedev ! This is a stop forward. The e2e tests are still failing though because you'll need to set up the InMemoryProvider to have the expected flags (you can do this in the constructor).

FAIL server-e2e packages/server/e2e/step-definitions/evaluation.spec.ts
  Flag evaluation
    ✕ Resolves boolean value (9 ms)
    ✕ Resolves string value (2 ms)
    ✕ Resolves integer value (1 ms)
    ✕ Resolves float value
    ✕ Resolves object value
    ✕ Resolves boolean details (1 ms)
    ✕ Resolves string details (1 ms)
    ✕ Resolves integer details
    ✕ Resolves float details
    ✕ Resolves object details (1 ms)
    ✕ Resolves based on context (1 ms)
    ✓ Flag not found (1 ms)
    ✕ Type error (2 ms)

The flagd test-bed has these flags configured. All the flags are defined in the gherkin suite here. The setup would like something like this (I've only included the first flag)

    OpenFeature.setProvider(
      new InMemoryProvider({
        'boolean-flag': {
          variants: {
            on: true,
            off: false,
          },
          disabled: false,
          defaultVariant: 'on',
        },
        // ...other flags...
      }),
    );

[znonthedev]

@lukas-reining @toddbaert . Okay will make the necessary changes

[znonthedev]

@lukas-reining @toddbaert I have updated the PR with the requested changes. Please review. Why don't we remove the workflow and docker file related to flagd-testbed? I assume we don't need it anymore

[toddbaert]

Why don't we remove the workflow and docker file related to flagd-testbed? I assume we don't need it anymore

Do you mean the service in the workflow (I assume so, since you removed it)? If so, yes! That's what @lukas-reining was suggesting as well, I think.

[toddbaert]

@znonthedev Great work! everything passes, I just have a couple small nits, then I'm ready to approve.

Would you like an invite to the org? It comes with no obligation, but it's the first step if you'd like to be more involved.

[znonthedev]

Do you mean the service in the workflow (I assume so, since you removed it)? If so, yes! That's what @lukas-reining was suggesting as well, I think.

Not only the service but also the files within the test-harness submodule. I might be missing some context here. Are we using the flagd-testbed container for testing other SDKs as well?

[znonthedev]

@znonthedev Great work! everything passes, I just have a couple small nits, then I'm ready to approve.

Would you like an invite to the org? It comes with no obligation, but it's the first step if you'd like to be more involved.

@toddbaert Thanks a lot for the positive feedback! I'm glad everything is passing smoothly. I appreciate the opportunity to contribute.
I'd be honoured to receive an invite to the org! While my availability may vary, I'm eager to contribute more and be part of the team.
Looking forward to the next steps, and thanks again for the opportunity!

[toddbaert]

Not only the service but also the files within the test-harness submodule. I might be missing some context here.

I'm not 100% sure whether we need that submodule now, actually. I will take a quick look and follow up. Good catch...

Are we using the flagd-testbed container for testing other SDKs as well?

The flagd-testbed containers (and the associated gherkin suites) are used in tons of tests across the org, but they should all be used for testing flagd or flagd-providers. We have been trying to disentangle the actual SDKs from these flagd dependencies (mostly by using the in-memory providers as you did here).

[znonthedev]

@toddbaert Okay, then I think no changes are needed for the shared test harness files. However, since we removed the flagd-testbed dependency in this repo, we could remove the submodule itself from this repo. I may be wrong.

[beeme1mr]

@znonthedev Todd won't be back until Monday. I'm going to leave this open until he has a chance to approve it. I think we should be good but I would like to confirm with him. Thanks for your effort with this.

[toddbaert]

So the evaluation.feature gherkin file is currently being supplied from the test-harness module, but again, in our efforts to remove all flagd-related dependencies from the SDKs, this gherkin is now available in the spec (with a few minor changes).

I've made a new issue to replace the test-harness submodule with a new spec submodule that will supply this gherkin.

[toddbaert]

Approved see comment here about submodule.

@beeme1mr @lukas-reining feel free to merge.