Restructure Haskell/Warp for db modularity. #4595
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Haskell/Warp dir has a lot of code duplication for testing various backend libraries as separate backend specific libraries. - Refactored the project structure to move database specific modules into their own libraries so we can have a single/shared backend agnostic warp project. This makes it easier to add new backends with minimal effort.
This completes the code changes and docs following the earlier pure file moves (separate commit to preserve git history). - Added `warp-shared` cabal file with mutliple executables using various backend drivers and the same shared server code. - Updated stack.yaml to adopt the earlier directory layout changes. We now have backend specific libraries (`shared/tfb-*`) and a single re-usable server (`warp-shared`) lib. - Updated docker file to build and install the new warp shared server -- the cabal file is responsible for producing backend specific executables that match `TFB_TEST_NAME`. - Added ReadMes for each `shared/tfb-*` library. - Updated `shared/tfb-types` to adopt to conform to uniform types that can play nice with all of the `shared/tfb-*` backends. - Updated `shared/tfb-*` cabal files to adopt its new dir and concern of only being a backend driver. - Updated `shared/tfb-*` `Db.hs` files to conform to the exact same public API so `warp-shared` can plug and play into any database driver without any custom code. - Added ReadMe for `warp-shared` to document the executables/tests it produces. - Updated `warp-shared` Lib to delegate connection pool management to `shared/tfb-*` backends as some use their own pools and others require use of `resource-pool` lib. - Updated `warp-shared` Main to print test name so we can manually sanity check the right executable is running in the right TFB docker image.
|
Thanks @naushadh As far as the context that the docker container pulls in, I'll see if I can add another benchmark_config option to pull in context from a directory above the framework test. I don't want do this automatically for all frameworks because I don't want to pull in the entire context of the language directory as it will make each test container larger. As it is, all these test containers take up a ton of disk space. |
|
See comment for performance results: #4571 (comment) |
naushadh
added a commit
to naushadh/FrameworkBenchmarks
that referenced
this pull request
Apr 7, 2019
- Starting an identical refactoring effort as done for Warp in: TechEmpower#4595 - For now we're just moving files we intend to re-use and purging the ones we will no longer need due to upcoming db modularity.
NateBrady23
pushed a commit
that referenced
this pull request
Apr 8, 2019
* Restructure servant for db modularity. - Starting an identical refactoring effort as done for Warp in: #4595 - For now we're just moving files we intend to re-use and purging the ones we will no longer need due to upcoming db modularity. * Copy warp/shared into servant/shared. Due to a docker restriction that disallows referencing shared code directories from a parent dir, and lack of tooling support to alter the scope dir of docker build invoke, we are unfortunately resorting to copy pasting code between frameworks. Documented this in the servant-shared README. * Complete restructuring servant for code re-use. - Updated benchmark config to reference updated docker file path. - Updated benchmark config to add `postgres-wire` variation as we now support it. - Updated stack.yaml to adopt the earlier directory layout changes. We now have backend specific libraries (`shared/tfb-*`) and a single re-usable server (`servant-shared`) executable. - Updated cabal file with mutliple executables using various backend drivers and the same shared server code. - Updated docker file to build and install the new `servant-shared` server -- the cabal file is responsible for producing backend specific executables that match `TFB_TEST_NAME`. - Updated `servant-shared` Lib to delegate database queries to `shared/tfb-*` backends so we can keep the server code backend agnostic. - Added MIME module to store all MIME things. Separated it out to keep the main Lib module focusing on pure server and response logic.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Was about to make some changes to Warp but quickly realized how cumbersome it was to re-apply patches by hand to each db. So decided to do some upfront one off refactoring to deduplicate db logic.
The database libs are now generic enough it can be trivially re-used with any server/framework. But that will require making tooling changes to make Docker run from parent dir so we can have say
servantandwarpshare the reusable database libs. Will create a separate issue/PR for that.Baseline stats from last PR (#4571):
Current stats:
Looks mostly the same or better except for some odd regressions for
JSON Serialization-- even that seems to get better on the later runs (warp-mysql-haskellvswarp) in the same test using the same code. I'm not too worried myself.