Broadsea 3.0 merge into develop #52
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…cker arch variable.
Added OMOP vocab and phoebe loading services
alondhe
added a commit
that referenced
this pull request
Apr 19, 2023
* Broadsea 3.0 merge into develop (#52) * Initial commit of adding SSL to traefik and Solr * Added Ares profile, using forked version * Added profiles, support for git-based webapi and atlas builds * Fixed WebAPI from git. Began security env variable sections. Added docker arch variable. * Fixed yml error * Added AD section * Added more security provider env vars, some cleanup * Added volume for cacerts (LDAPS) * Removed original config-local * Added security provider enabled entries, default values for sec items * New README, removed extraneous db specific WebAPI stuff, organized .env into sections * Fixed md issue in readme * Added ldaps item to readme * Clarification on ssl in readme * Added OMOP vocab and phoebe loading services. With updated readme * Workaround for if cacerts file isn't present * Removed tmpfs assignment from postgres-load * Fixes for phoebe unzip, clarity on vocab and phoebe vars * Added vocab cpt conversion option based on if env var is fulfilled * Added Ares data folder env and updated README * Using new MAVEN_PARAMS in webapi from git. Renamed SOLR_ENDPOINT. * Added dynamic splash page, ares entry, changed routes for content and hades. Added env var for certs folder. * Updated hades and atlasdb to amd64 arch * Added restart attribute to solr vocab * Fixes #62, images were blocked due to content security policy meta tag * Fixes div show/hide by removing extra quotes * Updated Ares copy * Tweak to hades config for backslash issue. Updated Mac Silicon language in readme * Added .jar to gitignore
alondhe
added a commit
that referenced
this pull request
Apr 29, 2024
* Broadsea 3.0 merge into develop (#52) * Initial commit of adding SSL to traefik and Solr * Added Ares profile, using forked version * Added profiles, support for git-based webapi and atlas builds * Fixed WebAPI from git. Began security env variable sections. Added docker arch variable. * Fixed yml error * Added AD section * Added more security provider env vars, some cleanup * Added volume for cacerts (LDAPS) * Removed original config-local * Added security provider enabled entries, default values for sec items * New README, removed extraneous db specific WebAPI stuff, organized .env into sections * Fixed md issue in readme * Added ldaps item to readme * Clarification on ssl in readme * Added OMOP vocab and phoebe loading services. With updated readme * Workaround for if cacerts file isn't present * Removed tmpfs assignment from postgres-load * Fixes for phoebe unzip, clarity on vocab and phoebe vars * Added vocab cpt conversion option based on if env var is fulfilled * Added Ares data folder env and updated README * Using new MAVEN_PARAMS in webapi from git. Renamed SOLR_ENDPOINT. * Added dynamic splash page, ares entry, changed routes for content and hades. Added env var for certs folder. * Updated hades and atlasdb to amd64 arch * Added restart attribute to solr vocab * Fixes #62, images were blocked due to content security policy meta tag * Fixes div show/hide by removing extra quotes * Updated Ares copy * Tweak to hades config for backslash issue. Updated Mac Silicon language in readme * Added .jar to gitignore * Added openldap and posit connect services * Adds OpenLDAP, Open Source Shiny Server, and Posit Connect services. Updates Traefik to new stable release. * Added atlas cohort cache settings. Added mount option for jdbc jar files in webapi. * Removed unnecessary flyway migration params. Updated comment on Atlas disable browser check * Reverted flyway changes for now. Added Broadsea content title variable. Added WebAPI multi-language variable. Added WebAPI security token expiration variable. * Initial support of execution engine in webapi and atlas * Added option for flyway baseline on migrate. Removed faulty localStorage clear operation in atlas config-local. * Added initial achilles container * Added achilles ares export * Added logging level variables for webapi Shifted jdbc drivers to a new docker volume path Added security oid customparams setting for webapi * Changed ares git location from fork to OHDSI repo * Syncing with issue 96 fix. Removed OID customparams variable for now. * Fix omop load script to use UMLS key when available Fix Solr Vocab import step to have command on 1 line Added vocab pg files path env var Brought back security oid customparams env var Updated alpine to 3.18.2 for pg load containers to fix zip install issue Added phoebe gitignores for expanded files * Added more Atlas configLocal settings * Added dbt, achilles, dqd, aresIndexer. Moved other compose files into compose folder. Changed broadsea content folder name. Enabled traefik dashboard. * Shifted to local Dockerfiles for post processing services * Added perseus services. Removed deprecated postgresql folder for webtools. Added arachne golden file creation - still needs work. * Added docker secrets to all services, except the tokens for perseus-user Still needs testing. * Shifted secrets toward specific file env vars * Added github action for default profile * Added missing dbt vars to env * Added missing default value for phoebe vocab files in env * Changed patterns for bringing jdbc files for solr and webapi Added none.jar placeholder to jdbc folder * Relative path issue for webapi none.jar * Updated README file Changed env file to use ares shared volume by default for ares web * Introducing quarto for content page * content typos For now removing the subdirectory from default ares data path * added wget install back * apt-get issue * Using RUN over EOF syntax * Adjustments to tests * checkout action with branch Added Perseus testing * Fixed image issue with posit connect, errant path issue in webapi jdbc file path * trying classpath-based method to additional webapi jar * [skip ci] copy update in env * removed ref tag from ci files * Default tests split up, perseus test improved Fixes for webapi additional jdbc * Added debug to webapi container * Fix to atlasdb env * Using fork of atlasdb * added branch name to atlasdb fork * Using new atlasdb image with secret Removed option to set atlasdb password * Adding sleep hack for atlasdb in github action * Adding more sleep and psql test * Disabled mac * added sudo * Adding public webapi test * sleep on webapi build * Adding path to env * Adding relative env path * ip address echo and certs folder create * with env_file set * Attempting separate config files * Changes to traefik dynamic yml, separate tls out * Reordering steps, curl fails fast * Trying public ip for broadsea host * revert atlasdb test to localhost * debug webapi with logs * Trying non-secret atlasdb * more pw debug * added sleep back * readme as debug * trying to write file * testing only macos * testing atlasdb password * shifting back to linux * changed test to use secret file * testing without password set * adding webapi back, but trying to ensure env is used correctly * fix of cd command * Fixes to secrets in webapi and solr * default solr vocab to disabled. Added curl webapi test back. * added mac psql. trying simple default test. * Adding sleep back for WebAPI test. Fix to atlas curl url. * Trying https in test * fix for self signed action * trying GA with default ssl params * changing ssl test to linux only * fix for runner os * removed macos * trying new openssl steps * copying certs before use * trying sudo * hostkey fix * trying to add crt to ca * using manual ssl creation * added ca step back * adding insecure flag * Adding mac back * Trying more jobs * Added needs * combined into 1 workflow changed ldap to disabled by default * moved openldap and solr to separate test ymls * Fixes for perseus web - to override port Added atlasdb to solr vocab testing * Reverted to standard perseus web and backend updated port of backend * Adds new cdm-builder settings file * Initial achilles GA test Removed extra sleep from openldap GA test. * Added achilles log test to GA Renamed some GA files Added db auth to webapi GA test Updated secret for db auth pw * Added sleep to achilles GA, various renaming of GAs * Reorg of atlas files, updated quarto content. Preparing for add cdm profile. Removed arachne for now. * comment out new cdm password secrets * Added improved cacerts method. Added Snowflake PK support. Added Github PAT support. * Change from < to cat in omop vocab script * Updated < to cat in phoebe script * Added dummy github pat secret file * Adding apk update to load omop script * Updated readme to add BYO JDBC driver * Update Basic Authentication (#113) * changing default password to match the broadsea-atlasdb POSTGRES_PASSWORD * updating .env values for basic security * adding an authentication query env var * adding setup instructions for basic security --------- Co-authored-by: Ajit Londhe <[email protected]> * Cleanup of readme, some new notes * Adding GITHUB TOKEN to Achilles GA script * Trying to use org secret for Github pat * Trying to set github pat as env within achilles dockerfile * shift achilles github pat secret to build * Added secrets mount in achilles dockerfile * Trying docker compose (no hyphen) in achilles ci * Adopt Docker compose v2 and hyphen-less docker compose * Removal of extraneous ports, removal of perseus-web, disable traefik dashboard Add MAVEN_PROFILE to webapi-from-git pattern * Re-enabled traefik dashboard. Added copy to Readme about Perseus * Simplify perseus ci test to just http * README editing (#121) * cmd editing and highlighting for shell * added codeblock highlighting * formatting * transformed profiles nested lists to tables * Added contents section for navigation * expanded on Shutdown section * Fixed cacerts item in Readme. Restore certs folder step in perseus ci. * Adding missing config-gis.js file to Atlas container * Added jupyter notebook profile. Fixes for Achilles Dockerfile and traefik routing. Update traefik to 2.11.2. Set atlasdb to restart unless stopped. * Added pgadmin from @leeevans commits and updated the content page to show it Also borrowed some WebAPI DB security improvements from @leeevans * Updating to 3.5 release number * Changing OSs in CI matrices * Using only Linux OS for CI for now. Mac image runners not functioning properly. --------- Co-authored-by: Will Roddy <[email protected]> Co-authored-by: Hayden B Spence <[email protected]>
alondhe
added a commit
that referenced
this pull request
Apr 29, 2024
* Broadsea 3.0 merge into develop (#52) * Initial commit of adding SSL to traefik and Solr * Added Ares profile, using forked version * Added profiles, support for git-based webapi and atlas builds * Fixed WebAPI from git. Began security env variable sections. Added docker arch variable. * Fixed yml error * Added AD section * Added more security provider env vars, some cleanup * Added volume for cacerts (LDAPS) * Removed original config-local * Added security provider enabled entries, default values for sec items * New README, removed extraneous db specific WebAPI stuff, organized .env into sections * Fixed md issue in readme * Added ldaps item to readme * Clarification on ssl in readme * Added OMOP vocab and phoebe loading services. With updated readme * Workaround for if cacerts file isn't present * Removed tmpfs assignment from postgres-load * Fixes for phoebe unzip, clarity on vocab and phoebe vars * Added vocab cpt conversion option based on if env var is fulfilled * Added Ares data folder env and updated README * Using new MAVEN_PARAMS in webapi from git. Renamed SOLR_ENDPOINT. * Added dynamic splash page, ares entry, changed routes for content and hades. Added env var for certs folder. * Updated hades and atlasdb to amd64 arch * Added restart attribute to solr vocab * Fixes #62, images were blocked due to content security policy meta tag * Fixes div show/hide by removing extra quotes * Updated Ares copy * Tweak to hades config for backslash issue. Updated Mac Silicon language in readme * Added .jar to gitignore * Added openldap and posit connect services * Adds OpenLDAP, Open Source Shiny Server, and Posit Connect services. Updates Traefik to new stable release. * Added atlas cohort cache settings. Added mount option for jdbc jar files in webapi. * Removed unnecessary flyway migration params. Updated comment on Atlas disable browser check * Reverted flyway changes for now. Added Broadsea content title variable. Added WebAPI multi-language variable. Added WebAPI security token expiration variable. * Initial support of execution engine in webapi and atlas * Added option for flyway baseline on migrate. Removed faulty localStorage clear operation in atlas config-local. * Added initial achilles container * Added achilles ares export * Added logging level variables for webapi Shifted jdbc drivers to a new docker volume path Added security oid customparams setting for webapi * Changed ares git location from fork to OHDSI repo * Syncing with issue 96 fix. Removed OID customparams variable for now. * Fix omop load script to use UMLS key when available Fix Solr Vocab import step to have command on 1 line Added vocab pg files path env var Brought back security oid customparams env var Updated alpine to 3.18.2 for pg load containers to fix zip install issue Added phoebe gitignores for expanded files * Added more Atlas configLocal settings * Added dbt, achilles, dqd, aresIndexer. Moved other compose files into compose folder. Changed broadsea content folder name. Enabled traefik dashboard. * Shifted to local Dockerfiles for post processing services * Added perseus services. Removed deprecated postgresql folder for webtools. Added arachne golden file creation - still needs work. * Added docker secrets to all services, except the tokens for perseus-user Still needs testing. * Shifted secrets toward specific file env vars * Added github action for default profile * Added missing dbt vars to env * Added missing default value for phoebe vocab files in env * Changed patterns for bringing jdbc files for solr and webapi Added none.jar placeholder to jdbc folder * Relative path issue for webapi none.jar * Updated README file Changed env file to use ares shared volume by default for ares web * Introducing quarto for content page * content typos For now removing the subdirectory from default ares data path * added wget install back * apt-get issue * Using RUN over EOF syntax * Adjustments to tests * checkout action with branch Added Perseus testing * Fixed image issue with posit connect, errant path issue in webapi jdbc file path * trying classpath-based method to additional webapi jar * [skip ci] copy update in env * removed ref tag from ci files * Default tests split up, perseus test improved Fixes for webapi additional jdbc * Added debug to webapi container * Fix to atlasdb env * Using fork of atlasdb * added branch name to atlasdb fork * Using new atlasdb image with secret Removed option to set atlasdb password * Adding sleep hack for atlasdb in github action * Adding more sleep and psql test * Disabled mac * added sudo * Adding public webapi test * sleep on webapi build * Adding path to env * Adding relative env path * ip address echo and certs folder create * with env_file set * Attempting separate config files * Changes to traefik dynamic yml, separate tls out * Reordering steps, curl fails fast * Trying public ip for broadsea host * revert atlasdb test to localhost * debug webapi with logs * Trying non-secret atlasdb * more pw debug * added sleep back * readme as debug * trying to write file * testing only macos * testing atlasdb password * shifting back to linux * changed test to use secret file * testing without password set * adding webapi back, but trying to ensure env is used correctly * fix of cd command * Fixes to secrets in webapi and solr * default solr vocab to disabled. Added curl webapi test back. * added mac psql. trying simple default test. * Adding sleep back for WebAPI test. Fix to atlas curl url. * Trying https in test * fix for self signed action * trying GA with default ssl params * changing ssl test to linux only * fix for runner os * removed macos * trying new openssl steps * copying certs before use * trying sudo * hostkey fix * trying to add crt to ca * using manual ssl creation * added ca step back * adding insecure flag * Adding mac back * Trying more jobs * Added needs * combined into 1 workflow changed ldap to disabled by default * moved openldap and solr to separate test ymls * Fixes for perseus web - to override port Added atlasdb to solr vocab testing * Reverted to standard perseus web and backend updated port of backend * Adds new cdm-builder settings file * Initial achilles GA test Removed extra sleep from openldap GA test. * Added achilles log test to GA Renamed some GA files Added db auth to webapi GA test Updated secret for db auth pw * Added sleep to achilles GA, various renaming of GAs * Reorg of atlas files, updated quarto content. Preparing for add cdm profile. Removed arachne for now. * comment out new cdm password secrets * Added improved cacerts method. Added Snowflake PK support. Added Github PAT support. * Change from < to cat in omop vocab script * Updated < to cat in phoebe script * Added dummy github pat secret file * Adding apk update to load omop script * Updated readme to add BYO JDBC driver * Update Basic Authentication (#113) * changing default password to match the broadsea-atlasdb POSTGRES_PASSWORD * updating .env values for basic security * adding an authentication query env var * adding setup instructions for basic security --------- Co-authored-by: Ajit Londhe <[email protected]> * Cleanup of readme, some new notes * Adding GITHUB TOKEN to Achilles GA script * Trying to use org secret for Github pat * Trying to set github pat as env within achilles dockerfile * shift achilles github pat secret to build * Added secrets mount in achilles dockerfile * Trying docker compose (no hyphen) in achilles ci * Adopt Docker compose v2 and hyphen-less docker compose * Removal of extraneous ports, removal of perseus-web, disable traefik dashboard Add MAVEN_PROFILE to webapi-from-git pattern * Re-enabled traefik dashboard. Added copy to Readme about Perseus * Simplify perseus ci test to just http * README editing (#121) * cmd editing and highlighting for shell * added codeblock highlighting * formatting * transformed profiles nested lists to tables * Added contents section for navigation * expanded on Shutdown section * Fixed cacerts item in Readme. Restore certs folder step in perseus ci. * Adding missing config-gis.js file to Atlas container * Added jupyter notebook profile. Fixes for Achilles Dockerfile and traefik routing. Update traefik to 2.11.2. Set atlasdb to restart unless stopped. * Added pgadmin from @leeevans commits and updated the content page to show it Also borrowed some WebAPI DB security improvements from @leeevans * Updating to 3.5 release number * Changing OSs in CI matrices * Using only Linux OS for CI for now. Mac image runners not functioning properly. * Cleanup of items from the merge * More cleanup of deprecated files --------- Co-authored-by: Will Roddy <[email protected]> Co-authored-by: Hayden B Spence <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Broadsea 3.0 merge into develop