Feature Toggle Part II: Plumbing HTS

[autumnust]

Summary

This PR contains all changes needed to create a new MySQL table so that HTS can accept inquiries with respect to feature-toggle. As a result of this PR, a new table will be bootstrapped within MySQL when HTS is initialized, as well as one entry inserted into this table for demo purpose.

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Introducing a new endpoint /hts/togglestatuses in HTS, which essentially answers the question: Given a tri-tuple featureId, databaseId and tableId, whether the feature is active for the given table.

Note that:

• This is a read-only API for its client, i.e. tables-services.
• tables-service is only aware an explicit tuple consisting of databaseId, tableId, featureId. The rule engine within HTS will need to determine, given the set of rules stored in mysql, whether the inquired entity, here specifically for databaseId, tableId has been activated with the featureId
• When developers are activating new features for some tables, they will need to update data.sql and PR-review the change. They don't have to make the server side deployment, but manually inserting row into the MySQL backend is also acceptable.
• The rule engine that support wildcard/regex-based matching is not part of this PR. So although the repository's API is accepting databasePattern and tablePattern, tests are assuming that rules written in the repository are having the exact match.

Testing Done

• Manually Tested on local docker setup. Please include commands ran, and their output.
• Added new tests for the changes made.
• Updated existing tests to reflect the changes made.
• No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
• Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Verified locally that when HTS is bootstrapped, a new table will show up with a record inserted:

➜  openhouse git:(stitch_all_test) ✗ docker exec -it 'local.mysql' /bin/sh

sh-4.4# history
    1  history
sh-4.4# mysql -uoh_user -poh_password oh_db
mysql: [Warning] Using a password on the command line interface can be insecure.
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 18
Server version: 8.1.0 MySQL Community Server - GPL

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show tables ;
+-------------------+
| Tables_in_oh_db   |
+-------------------+
| job_row           |
| table_toggle_rule |
| user_table_row    |
+-------------------+
3 rows in set (0.00 sec)

mysql> select * from table_toggle_rule
    -> ;
+---------+------------------+---------------+----+------------------+----------------------------+
| feature | database_pattern | table_pattern | id | creation_time_ms | ETL_TS                     |
+---------+------------------+---------------+----+------------------+----------------------------+
| demo    | demodb           | demotable     |  1 |             NULL | 2024-04-15 23:38:40.006120 |
+---------+------------------+---------------+----+------------------+----------------------------+
1 row in set (0.00 sec)

Additional Information

• Breaking Changes
• Deprecations
• Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

[HotSushi]

Looking good overall, have feedback on:

• Not sharing ToggleEnum
• Impact of MODE=MYSQL on tables-test-fixtures and non-mysql dbs. There seems to be another way of doing this
• some repositories/ services are not enabled, maybe I missed the description on its plan

[autumnust]

Looking good overall, have feedback on:

• Not sharing ToggleEnum
• Impact of MODE=MYSQL on tables-test-fixtures and non-mysql dbs. There seems to be another way of doing this
• some repositories/ services are not enabled, maybe I missed the description on its plan

There are something really weird going on for H2 if using direct JPA to insert the record. I have spent 5+ hrs on that, the gist of it is it doesn't really honor the constraints declared in DDL and somehow H2 supports oracle's dialect as DEFAULT ON NULL syntax as the workaround for my case, but this syntax is not acceptable by MySQL. From my research H2 is partially supporting some of the features by default, that's why enabling MODE is needed here.

[HotSushi]

minor comments.
spec has changed after this, will you raise a PR for docs as well?

[autumnust]

minor comments. spec has changed after this, will you raise a PR for docs as well?

I can update this doc, but the changed spec is within HTS and this doc doesn't publish spec for HTS either. Is that still necessary?

[autumnust]

Looking good overall, have feedback on:

• Not sharing ToggleEnum
• Impact of MODE=MYSQL on tables-test-fixtures and non-mysql dbs. There seems to be another way of doing this
• some repositories/ services are not enabled, maybe I missed the description on its plan

There are something really weird going on for H2 if using direct JPA to insert the record. I have spent 5+ hrs on that, the gist of it is it doesn't really honor the constraints declared in DDL and somehow H2 supports oracle's dialect as DEFAULT ON NULL syntax as the workaround for my case, but this syntax is not acceptable by MySQL. From my research H2 is partially supporting some of the features by default, that's why enabling MODE is needed here.

References I found: JetBrains/Exposed#186

[HotSushi]

LGTM