Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cucumber integration tests #70

Closed
wants to merge 1 commit into from
Closed

Add cucumber integration tests #70

wants to merge 1 commit into from

Conversation

toddbaert
Copy link
Member

@toddbaert toddbaert commented Jul 12, 2022

This PR adds a PoC for a cross-language integration test suite. The suite works by running a flagd image with an extra layer of config.

Specifically, the PR:

  • Adds sample gerkin tests. These are committed to this branch, but they should reside somewhere else, so all SDKs can use them. We certainly need more tests, but they are quite easy to write. These are just to prove the concept.
  • Adds a jest test suite that parses and runs the gerkin.
  • Adds CI which runs a custom flagd container with the flags in question.

My main question is where this gerkin should reside... It could be in the test harness repo, or it could even be embedded in the spec and parsed out by our spec parser.

@codecov-commenter
Copy link

codecov-commenter commented Jul 12, 2022

Codecov Report

Merging #70 (f36a4ff) into main (06652c5) will not change coverage.
The diff coverage is n/a.

❗ Current head f36a4ff differs from pull request most recent head 4a3046f. Consider uploading reports for the commit 4a3046f to get more accurate results

@@           Coverage Diff           @@
##             main      #70   +/-   ##
=======================================
  Coverage   96.17%   96.17%           
=======================================
  Files           9        9           
  Lines         445      445           
  Branches       57       57           
=======================================
  Hits          428      428           
  Misses         17       17           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8fc41b6...4a3046f. Read the comment docs.

.vscode/settings.json Outdated Show resolved Hide resolved
tsconfig.json Outdated Show resolved Hide resolved
Comment on lines +1 to +16
Feature: Flag evaluation

Scenario: Resolves boolean value
Given A boolean flag called boolean-flag with value true exists
When Flag is evaluated with default value false
Then The resolved value should match the flag value

Scenario: Resolves string value
Given A string flag called string-flag with value #CC0000 exists
When Flag is evaluated with default value #0000CC
Then The resolved value should match the flag value

Scenario: Resolves number value
Given A number flag called number-flag with value 1 exists
When Flag is evaluated with default value 2
Then The resolved value should match the flag value
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cannot reside here long term, it has to be in some common repo, and probably included as a submodule. It could be parsed out of the spec, or committed here

@toddbaert toddbaert requested review from beeme1mr and agentgonzo July 12, 2022 20:29
// get a client (flagd provider registered in setup)
const client = OpenFeature.getClient();

defineFeature(feature, (test) => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests are associated with the gerkin. If any test descriptions don't line up with the gerkin clauses, the test fails.

Then The resolved value should match the flag value

Scenario: Resolves string value
Given A string flag called string-flag with value #CC0000 exists
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These plain-text statements are parsed by our jest-gerkin tests. It works similarly in other languages (Java, Go, dotnet)

Comment on lines +13 to +18
services:
flagd:
image: ghcr.io/open-feature/flagd-testbed:latest
ports:
- 8080:8080

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runs the custom flagd image in CI

@toddbaert
Copy link
Member Author

This was implemented in another PR

@toddbaert toddbaert closed this Sep 20, 2022
lukas-reining pushed a commit that referenced this pull request Jun 29, 2023
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants