Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(segments): Segment AND'ing #1915

Merged
merged 57 commits into from
Aug 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
206b5cc
feat(segments): POC of API parity with segment anding
yquansah Jul 27, 2023
4946e53
feat(segment_anding): feature parity fixes and new logic for and-ing
yquansah Jul 27, 2023
4868a4b
feat(rollouts): feature parity with rollouts
yquansah Jul 28, 2023
777f817
feat(rollouts): Additional functionality working with multiple segmen…
yquansah Jul 28, 2023
56e7b68
feat(rules/rollouts): Add tests for server and storage
yquansah Jul 28, 2023
3cd24eb
chore(rollouts): evaluation storage tests
yquansah Jul 28, 2023
629c048
feat: Merge main and make relevant fixes
yquansah Jul 28, 2023
c8d13e2
feat(fs): Add FS implementation of segment AND-ing
yquansah Jul 28, 2023
b74affb
chore(tests): Fix all tests after FS implementation
yquansah Jul 28, 2023
e5198e1
chore(importer): Account for segmentKeys in importer
yquansah Jul 28, 2023
49001ff
chore(testing): Add integration tests for segment AND-ing feature
yquansah Jul 31, 2023
8e7bc82
feat(dbs): Add additional database DDL
yquansah Jul 31, 2023
efc0013
Merge branch 'main' into segment-anding
yquansah Jul 31, 2023
d00ad9c
chore: move alter statements to separate file
yquansah Aug 1, 2023
dd803a9
Merge remote-tracking branch 'origin/segment-anding' into segment-anding
yquansah Aug 1, 2023
80644af
feat(update): Add update methods for rollouts and rules
yquansah Aug 1, 2023
17af43d
chore(rollback): Rollback transaction in a defer
yquansah Aug 1, 2023
f847fc0
feat(update): Tests for UpdateRule and fixing linter errors
yquansah Aug 1, 2023
a7dec84
Merge remote-tracking branch 'origin/main' into segment-anding
yquansah Aug 1, 2023
2101228
chore(tests/rules/rollouts): fixing overall tests
yquansah Aug 1, 2023
ec2e991
chore(style): add better length checks
yquansah Aug 1, 2023
36b728b
feat(EvaluationResponse): Add SegmentKeys to legacy evaluation response
yquansah Aug 1, 2023
b28c7c9
chore(testing): Add tests for readonly on updated import/export format
yquansah Aug 1, 2023
901a927
chore: get import/export integration test working
yquansah Aug 2, 2023
72dab7e
chore(mysql): fix mysql test
yquansah Aug 3, 2023
d0b4ebf
Merge branch 'main' into segment-anding
yquansah Aug 3, 2023
d53550e
chore(storage): Do a length check of 1
yquansah Aug 3, 2023
f9801b3
chore: fix sqlite migrations
yquansah Aug 3, 2023
ceb7d5c
Merge remote-tracking branch 'origin/segment-anding' into segment-anding
yquansah Aug 3, 2023
9e20cb9
chore(rollouts): Fixes and removing of print statement
yquansah Aug 3, 2023
5878a6f
Merge branch 'main' into segment-anding
yquansah Aug 3, 2023
21088db
feat(segment-anding): Create RuleSegments and RolloutSegments (#1941)
yquansah Aug 4, 2023
5b864bb
Merge branch 'main' into segment-anding
yquansah Aug 4, 2023
bf420e0
feat(sql): Add unique constraints on new tables
yquansah Aug 5, 2023
e86bc13
Merge remote-tracking branch 'origin/segment-anding' into segment-anding
yquansah Aug 5, 2023
93508db
feat(rollouts/rules): Add SegmentKeysUnaryInterceptor for relevant re…
yquansah Aug 6, 2023
811faa6
chore(rules/rollouts): fix up integration tests
yquansah Aug 6, 2023
8c1d4bc
chore: last cleanup move back to non-interceptor
yquansah Aug 7, 2023
d0422a0
Merge branch 'main' into segment-anding
yquansah Aug 7, 2023
47a8dd8
Merge branch 'main' into segment-anding
yquansah Aug 7, 2023
3f81765
feat(rollouts/rules): UI for rollout/rule editing of segments (#1953)
yquansah Aug 8, 2023
a5088ce
chore: fix merge conflict
yquansah Aug 8, 2023
c3870c1
chore: fix reset of rules/rollouts (#1974)
yquansah Aug 9, 2023
45149d6
feat(rules/rollouts): hide or and and based on segment keys length, a…
yquansah Aug 9, 2023
190b3cd
Merge branch 'main' into segment-anding
yquansah Aug 10, 2023
524f277
feat(rules): Make segment be one of two types (#1978)
yquansah Aug 10, 2023
7db60c2
feat(rules): Revise query for optimization (#1979)
yquansah Aug 10, 2023
78b4995
Merge branch 'main' into segment-anding
yquansah Aug 10, 2023
b4b00cc
Merge branch 'main' into segment-anding
yquansah Aug 11, 2023
1139322
chore(evaluation): Use JSON tags for EvaluationSegment (#1984)
yquansah Aug 11, 2023
0856141
feat(segments): introduce cue validation for new segment definitions …
yquansah Aug 14, 2023
59dd229
chore(cockroach): Remove sql_safe_update setting
yquansah Aug 14, 2023
48cfbfd
Merge remote-tracking branch 'origin/segment-anding' into segment-anding
yquansah Aug 14, 2023
f04dd9c
chore: merge main branch in
yquansah Aug 14, 2023
6dee605
chore: use SegmentKey for backwards compatibility
yquansah Aug 14, 2023
6b535e9
chore: address comments related to squirrel statements
yquansah Aug 15, 2023
b5c10ce
Merge branch 'main' into segment-anding
yquansah Aug 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions build/internal/cmd/generate/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,10 @@ func main() {

for k := 0; k < *flagRuleCount; k++ {
rule := &ext.Rule{
Rank: uint(k + 1),
SegmentKey: doc.Segments[k%len(doc.Segments)].Key,
Rank: uint(k + 1),
Segment: &ext.SegmentEmbed{
IsSegment: ext.SegmentKey(doc.Segments[k%len(doc.Segments)].Key),
},
}

for l := 0; l < *flagRuleDistCount; l++ {
Expand Down
2 changes: 1 addition & 1 deletion build/testing/integration.go
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ func importExport(ctx context.Context, _ *dagger.Client, base, flipt *dagger.Con
if namespace == "" {
namespace = "default"
// replace namespace in expected yaml
expected = strings.ReplaceAll(expected, "version: \"1.1\"\n", fmt.Sprintf("version: \"1.1\"\nnamespace: %s\n", namespace))
expected = strings.ReplaceAll(expected, "version: \"1.2\"\n", fmt.Sprintf("version: \"1.2\"\nnamespace: %s\n", namespace))
}

// use target flipt binary to invoke import
Expand Down
207 changes: 199 additions & 8 deletions build/testing/integration/api/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -351,6 +351,64 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.Equal(t, "eq", updatedConstraint.Operator)
assert.Equal(t, "baz", updatedConstraint.Value)
assert.Equal(t, "newdesc", updatedConstraint.Description)

t.Log(`Create two additional segments.`)

createdSegment, err = client.Flipt().CreateSegment(ctx, &flipt.CreateSegmentRequest{
NamespaceKey: namespace,
Key: "segment",
Name: "Segment",
MatchType: flipt.MatchType_ALL_MATCH_TYPE,
})

require.NoError(t, err)

assert.Equal(t, "segment", createdSegment.Key)
assert.Equal(t, "Segment", createdSegment.Name)
assert.Equal(t, flipt.MatchType_ALL_MATCH_TYPE, createdSegment.MatchType)

createdConstraint, err := client.Flipt().CreateConstraint(ctx, &flipt.CreateConstraintRequest{
NamespaceKey: namespace,
SegmentKey: createdSegment.Key,
Type: flipt.ComparisonType_STRING_COMPARISON_TYPE,
Property: "first",
Operator: "eq",
Value: "segment",
})

require.NoError(t, err)

assert.Equal(t, "first", createdConstraint.Property)
assert.Equal(t, "eq", createdConstraint.Operator)
assert.Equal(t, "segment", createdConstraint.Value)

createdSegment, err = client.Flipt().CreateSegment(ctx, &flipt.CreateSegmentRequest{
NamespaceKey: namespace,
Key: "another-segment",
Name: "Another Segment",
MatchType: flipt.MatchType_ALL_MATCH_TYPE,
})

require.NoError(t, err)

assert.Equal(t, "another-segment", createdSegment.Key)
assert.Equal(t, "Another Segment", createdSegment.Name)
assert.Equal(t, flipt.MatchType_ALL_MATCH_TYPE, createdSegment.MatchType)

createdConstraint, err = client.Flipt().CreateConstraint(ctx, &flipt.CreateConstraintRequest{
NamespaceKey: namespace,
SegmentKey: createdSegment.Key,
Type: flipt.ComparisonType_STRING_COMPARISON_TYPE,
Property: "second",
Operator: "eq",
Value: "another-segment",
})

require.NoError(t, err)

assert.Equal(t, "second", createdConstraint.Property)
assert.Equal(t, "eq", createdConstraint.Operator)
assert.Equal(t, "another-segment", createdConstraint.Value)
})

t.Run("Rules and Distributions", func(t *testing.T) {
Expand Down Expand Up @@ -397,6 +455,35 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.Equal(t, "everyone", ruleTwo.SegmentKey)
assert.Equal(t, int32(2), ruleTwo.Rank)

t.Log(`Create rule "rank 3".`)

ruleThree, err := client.Flipt().CreateRule(ctx, &flipt.CreateRuleRequest{
NamespaceKey: namespace,
FlagKey: "test",
SegmentKey: "another-segment",
Rank: 3,
})

require.NoError(t, err)

assert.Equal(t, "test", ruleThree.FlagKey)
assert.Equal(t, "another-segment", ruleThree.SegmentKey)
assert.Equal(t, int32(3), ruleThree.Rank)

updatedRuleThree, err := client.Flipt().UpdateRule(ctx, &flipt.UpdateRuleRequest{
Id: ruleThree.Id,
NamespaceKey: namespace,
FlagKey: "test",
SegmentKeys: []string{"segment", "another-segment"},
SegmentOperator: flipt.SegmentOperator_AND_SEGMENT_OPERATOR,
})
require.NoError(t, err)

assert.Equal(t, "test", updatedRuleThree.FlagKey)
assert.Contains(t, updatedRuleThree.SegmentKeys, "another-segment")
assert.Contains(t, updatedRuleThree.SegmentKeys, "segment")
assert.Equal(t, int32(3), updatedRuleThree.Rank)

// ensure you can not link flags and segments from different namespaces.
if !namespaceIsDefault(namespace) {
t.Log(`Ensure that rules can only link entities in the same namespace.`)
Expand Down Expand Up @@ -428,17 +515,18 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
})
require.NoError(t, err)

assert.Len(t, allRules.Rules, 2)
assert.Len(t, allRules.Rules, 3)

assert.Equal(t, ruleOne.Id, allRules.Rules[0].Id)
assert.Equal(t, ruleTwo.Id, allRules.Rules[1].Id)
assert.Equal(t, ruleThree.Id, allRules.Rules[2].Id)

t.Log(`Re-order rules.`)

err = client.Flipt().OrderRules(ctx, &flipt.OrderRulesRequest{
NamespaceKey: namespace,
FlagKey: "test",
RuleIds: []string{ruleTwo.Id, ruleOne.Id},
RuleIds: []string{ruleTwo.Id, ruleOne.Id, ruleThree.Id},
})
require.NoError(t, err)

Expand All @@ -450,13 +538,15 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
})
require.NoError(t, err)

assert.Len(t, allRules.Rules, 2)
assert.Len(t, allRules.Rules, 3)

// ensure the order has switched
assert.Equal(t, ruleTwo.Id, allRules.Rules[0].Id)
assert.Equal(t, int32(1), allRules.Rules[0].Rank)
assert.Equal(t, ruleOne.Id, allRules.Rules[1].Id)
assert.Equal(t, int32(2), allRules.Rules[1].Rank)
assert.Equal(t, ruleThree.Id, allRules.Rules[2].Id)
assert.Equal(t, int32(3), allRules.Rules[2].Rank)

t.Log(`Create distribution "rollout 100".`)

Expand Down Expand Up @@ -533,6 +623,15 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au

assert.Equal(t, ruleTwo.Id, distribution.RuleId)
assert.Equal(t, float32(100), distribution.Rollout)

_, err = client.Flipt().CreateDistribution(ctx, &flipt.CreateDistributionRequest{
NamespaceKey: namespace,
FlagKey: "test",
RuleId: ruleThree.Id,
VariantId: flag.Variants[0].Id,
Rollout: 100,
})
require.NoError(t, err)
})

t.Run("Boolean Rollouts", func(t *testing.T) {
Expand Down Expand Up @@ -572,11 +671,51 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.Equal(t, "everyone", rolloutSegment.Rule.(*flipt.Rollout_Segment).Segment.SegmentKey)
assert.Equal(t, true, rolloutSegment.Rule.(*flipt.Rollout_Segment).Segment.Value)

anotherRolloutSegment, err := client.Flipt().CreateRollout(ctx, &flipt.CreateRolloutRequest{
NamespaceKey: namespace,
FlagKey: "boolean_disabled",
Description: "matches a segment",
Rank: 2,
Rule: &flipt.CreateRolloutRequest_Segment{
Segment: &flipt.RolloutSegment{
SegmentKey: "another-segment",
Value: false,
},
},
})
require.NoError(t, err)

assert.Equal(t, namespace, anotherRolloutSegment.NamespaceKey)
assert.Equal(t, "boolean_disabled", anotherRolloutSegment.FlagKey)
assert.Equal(t, int32(2), anotherRolloutSegment.Rank)
assert.Equal(t, "another-segment", anotherRolloutSegment.Rule.(*flipt.Rollout_Segment).Segment.SegmentKey)
assert.Equal(t, false, anotherRolloutSegment.Rule.(*flipt.Rollout_Segment).Segment.Value)

updatedAnotherRolloutSegment, err := client.Flipt().UpdateRollout(ctx, &flipt.UpdateRolloutRequest{
Id: anotherRolloutSegment.Id,
NamespaceKey: namespace,
FlagKey: "boolean_disabled",
Rule: &flipt.UpdateRolloutRequest_Segment{
Segment: &flipt.RolloutSegment{
SegmentKeys: []string{"another-segment", "segment"},
Value: false,
},
},
})
require.NoError(t, err)

assert.Equal(t, namespace, updatedAnotherRolloutSegment.NamespaceKey)
assert.Equal(t, "boolean_disabled", updatedAnotherRolloutSegment.FlagKey)
assert.Equal(t, int32(2), updatedAnotherRolloutSegment.Rank)
assert.Contains(t, updatedAnotherRolloutSegment.Rule.(*flipt.Rollout_Segment).Segment.SegmentKeys, "segment")
assert.Contains(t, updatedAnotherRolloutSegment.Rule.(*flipt.Rollout_Segment).Segment.SegmentKeys, "another-segment")
assert.Equal(t, false, updatedAnotherRolloutSegment.Rule.(*flipt.Rollout_Segment).Segment.Value)

rolloutThreshold, err := client.Flipt().CreateRollout(ctx, &flipt.CreateRolloutRequest{
NamespaceKey: namespace,
FlagKey: "boolean_disabled",
Description: "50% disabled",
Rank: 2,
Rank: 3,
Rule: &flipt.CreateRolloutRequest_Threshold{
Threshold: &flipt.RolloutThreshold{
Percentage: 50,
Expand All @@ -589,7 +728,7 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.Equal(t, namespace, rolloutThreshold.NamespaceKey)
assert.Equal(t, "boolean_disabled", rolloutThreshold.FlagKey)
assert.Equal(t, "50% disabled", rolloutThreshold.Description)
assert.Equal(t, int32(2), rolloutThreshold.Rank)
assert.Equal(t, int32(3), rolloutThreshold.Rank)
assert.Equal(t, float32(50.0), rolloutThreshold.Rule.(*flipt.Rollout_Threshold).Threshold.Percentage)
assert.Equal(t, true, rolloutThreshold.Rule.(*flipt.Rollout_Threshold).Threshold.Value)

Expand Down Expand Up @@ -635,6 +774,7 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au

assert.Empty(t, cmp.Diff([]*flipt.Rollout{
rolloutSegment,
updatedAnotherRolloutSegment,
rolloutThreshold,
}, rollouts.Rules, protocmp.Transform()))

Expand Down Expand Up @@ -669,7 +809,7 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.Equal(t, namespace, updatedRollout.NamespaceKey)
assert.Equal(t, "boolean_disabled", updatedRollout.FlagKey)
assert.Equal(t, "50% enabled", updatedRollout.Description)
assert.Equal(t, int32(2), updatedRollout.Rank)
assert.Equal(t, int32(3), updatedRollout.Rank)
assert.Equal(t, float32(50.0), updatedRollout.Rule.(*flipt.Rollout_Threshold).Threshold.Percentage)
assert.Equal(t, false, updatedRollout.Rule.(*flipt.Rollout_Threshold).Threshold.Value)

Expand Down Expand Up @@ -779,7 +919,7 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au

t.Run("Evaluation", func(t *testing.T) {
t.Run("Variant", func(t *testing.T) {
t.Run("successful match", func(t *testing.T) {
t.Run("successful match (rank 1)", func(t *testing.T) {
result, err := client.Evaluation().Variant(ctx, &evaluation.EvaluationRequest{
NamespaceKey: namespace,
FlagKey: "test",
Expand All @@ -797,6 +937,25 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.Equal(t, evaluation.EvaluationReason_MATCH_EVALUATION_REASON, result.Reason)
})

t.Run("successful match (rank 3)", func(t *testing.T) {
result, err := client.Evaluation().Variant(ctx, &evaluation.EvaluationRequest{
NamespaceKey: namespace,
FlagKey: "test",
EntityId: uuid.Must(uuid.NewV4()).String(),
Context: map[string]string{
"first": "segment",
"second": "another-segment",
},
})
require.NoError(t, err)

require.True(t, result.Match, "Evaluation should have matched.")
assert.Contains(t, result.SegmentKeys, "segment")
assert.Contains(t, result.SegmentKeys, "another-segment")
assert.Equal(t, "one", result.VariantKey)
assert.Equal(t, evaluation.EvaluationReason_MATCH_EVALUATION_REASON, result.Reason)
})

t.Run("no match", func(t *testing.T) {
result, err := client.Evaluation().Variant(ctx, &evaluation.EvaluationRequest{
NamespaceKey: namespace,
Expand Down Expand Up @@ -875,7 +1034,7 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.False(t, result.Enabled, "value should be threshold match value")
})

t.Run("segment match", func(t *testing.T) {
t.Run("segment match (rank 1)", func(t *testing.T) {
result, err := client.Evaluation().Boolean(ctx, &evaluation.EvaluationRequest{
NamespaceKey: namespace,
FlagKey: "boolean_disabled",
Expand All @@ -891,6 +1050,23 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
assert.Equal(t, evaluation.EvaluationReason_MATCH_EVALUATION_REASON, result.Reason)
assert.True(t, result.Enabled, "value should be segment match value")
})

t.Run("segment match (rank 2)", func(t *testing.T) {
result, err := client.Evaluation().Boolean(ctx, &evaluation.EvaluationRequest{
NamespaceKey: namespace,
FlagKey: "boolean_disabled",
EntityId: "fixed",
Context: map[string]string{
"first": "segment",
"second": "another-segment",
},
})

require.NoError(t, err)

assert.Equal(t, evaluation.EvaluationReason_MATCH_EVALUATION_REASON, result.Reason)
assert.False(t, result.Enabled, "value should be segment match value")
})
})

t.Run("Batch", func(t *testing.T) {
Expand Down Expand Up @@ -999,6 +1175,21 @@ func API(t *testing.T, ctx context.Context, client sdk.SDK, namespace string, au
NamespaceKey: namespace,
Key: "everyone",
})

require.NoError(t, err)

err = client.Flipt().DeleteSegment(ctx, &flipt.DeleteSegmentRequest{
NamespaceKey: namespace,
Key: "segment",
})

require.NoError(t, err)

err = client.Flipt().DeleteSegment(ctx, &flipt.DeleteSegmentRequest{
NamespaceKey: namespace,
Key: "another-segment",
})

require.NoError(t, err)

if !namespaceIsDefault(namespace) {
Expand Down
Loading