WX-966 Add Cascades, remove Directory from Biscayne #7105

aednichols · 2023-03-28T01:01:57Z

Before This PR:

WDL Version	Version declaration	Codename in Cromwell	Shipping Status
`draft-2`	(none)	`draft-2`	Supported
`1.0`	`1.0`	`draft-3`	Supported
Next (`1.1` or `2.0`)	`development`	`biscayne`	In-development

After This PR:

WDL Version	Version declaration	Codename in Cromwell	Shipping Status
`draft-2`	(none)	`draft-2`	Supported
`1.0`	`1.0`	`draft-3`	Supported
`1.1`	`development-1.1`	`biscayne`	In-development
Next (`1.2` or `2.0`)	`development`	`cascades`	Future

THWiseman · 2023-04-06T17:21:05Z

...est/scala/wdl/transforms/cascades/linking/expression/values/CascadesValueEvaluatorSpec.scala

+  it should "evaluate an 'as_pairs' expression correctly" in {
+    // Repeat this a few times ensure we aren't occasionally reordering by accident:
+    10 times {
+      val str = """ as_pairs( { 1: "one", 2: "two", 3: three } ) """


Just confirming it's intentional that three is not in quotes.

(This test is a duplicate of one from Biscayne) I believe what's going on here is support for bare words, without quotes. I'll verify with the spec that WDL should support that.

I wasn't able to find any evidence for this in either the 1.1 or development specs, so assuming typo. Test passes fine with quotes, so checking that in.

Is this assuming that three is a WDL variable, and checking that the as_pairs function is correctly evaluating sub-clauses correctly?

THWiseman · 2023-04-06T17:31:00Z

wdl/transforms/cascades/src/test/cases/no_input_no_output_workflow.wdl

+version development
+
+workflow no_input_no_output {
+  call no_inputs as noi1


If we're testing formatting stuff here, it might be useful to have cases with tabs and spaces, too.

These tests are really a bit too sensitive to normal variation in the AST...

jgainerdewar · 2023-04-19T13:54:44Z

I know we've talked about it at length, but I don't know if it's written down anywhere - could you (either in the PR description or the ticket) lay out the WDL versions supported by Cromwell as of this PR, and what user-facing changes this PR introduces?

jgainerdewar · 2023-04-19T13:50:46Z

...dardTestCases/wdl_biscayne/biscayne_http_relative_imports/biscayne_http_relative_imports.wdl


 # Is there a better way to test this?
-import "https://raw.githubusercontent.com/broadinstitute/cromwell/develop/womtool/src/test/resources/validate/biscayne/valid/relative_imports/sub_wfs/foo.wdl"
+import "https://raw.githubusercontent.com/broadinstitute/cromwell/aen_wx_966/womtool/src/test/resources/validate/biscayne/valid/relative_imports/sub_wfs/foo.wdl"


Assuming this change will be reverted before merge?

jgainerdewar · 2023-04-19T15:47:09Z

core/src/main/resources/reference.conf

+          strict-validation: true
+          enabled: true
+        }
+      }


I suspect the answer is "yes" but want to confirm - this is all the config needed to support this change, right? We don't need to adjust anything in Terra deployments?

For the sake of completeness, filed https://github.com/broadinstitute/firecloud-develop/pull/3349. Also fixes the current development not having caching - not a big deal but should be consistent.

jgainerdewar · 2023-04-19T15:49:59Z

...est/scala/wdl/transforms/cascades/linking/expression/values/CascadesValueEvaluatorSpec.scala

+
+class CascadesValueEvaluatorSpec extends AnyFlatSpec with CromwellTimeoutSpec with Matchers {
+
+  behavior of "biscayne value evaluator"


This value needs to be updated

jgainerdewar · 2023-04-19T15:50:28Z

...l/transforms/cascades/linking/expression/consumed/CascadesExpressionValueConsumersSpec.scala

+import wdl.transforms.cascades.ast2wdlom._
+
+class CascadesExpressionValueConsumersSpec extends AnyFlatSpec with CromwellTimeoutSpec with Matchers {
+  behavior of "WDL Biscayne's expressionValueConsumer"


Update test name

jgainerdewar · 2023-04-19T15:52:01Z

womtool/src/test/resources/validate/biscayne/valid/object_struct/object_struct.wdl

@@ -1,4 +1,4 @@
-version development
+version development-1.1


Do we need equivalent tests for the new development version?

Has the development version gone away? If so, some people's WDLs may break.

... on the other hand, some people's WDLs may break anyway, and it's literally called "development" so maybe we don't care?

Note: this is a different version declaration than the one listed in wdl/transforms/biscayne/src/main/resources/CHANGELOG.txt. Presumably this one is correct and the changelog needs updating?

Chris, I updated the description with info about the version declaration. tl;dr no WDLs will stop working.

aednichols · 2023-04-20T13:06:41Z

@jgainerdewar added versions summary to PR description, great idea.

cjllanwarne · 2023-04-20T18:50:04Z

...ories/language-factory-core/src/main/scala/cromwell/languages/util/LanguageFactoryUtil.scala

@@ -94,7 +94,7 @@ object LanguageFactoryUtil {
    }

    val firstCodeLine = fileWithoutInitialWhitespace.headOption.map(_.dropWhile(_.isWhitespace))
-    firstCodeLine.exists { line => startsWithOptions.exists(line.startsWith) }
+    firstCodeLine.exists { line => startsWithOptions.contains(line) }


Does this catch things we don't want, like ?

# Thank goodness this WDL is not version 1.1!

The commentIndicators value is defined as List("#") and appears to drop comments from consideration.

cjllanwarne · 2023-04-20T18:51:20Z

...est/scala/wdl/transforms/biscayne/linking/expression/values/BiscayneValueEvaluatorSpec.scala

@@ -46,7 +46,7 @@ class BiscayneValueEvaluatorSpec extends AnyFlatSpec with CromwellTimeoutSpec wi
  it should "evaluate an 'as_pairs' expression correctly" in {
    // Repeat this a few times ensure we aren't occasionally reordering by accident:
    10 times {
-      val str = """ as_pairs( { 1: "one", 2: "two", 3: three } ) """
+      val str = """ as_pairs( { 1: "one", 2: "two", 3: "three" } ) """


As mentioned on the cascades side, I think this was deliberately a variable lookup rather than a plain string

Undone, good catch. Consistency is not always the only goal, as it turns out 🙂

jgainerdewar · 2023-05-18T18:28:03Z

I think there are still some paths in here that reference this branch name, other than that LGTM!

aednichols · 2023-05-18T18:31:50Z

By the book, the customized branch names have to merge so that tests pass, and then can be updated in a follow-on PR.

jgainerdewar · 2023-05-18T19:05:08Z

Sounds good as long as this branch isn't deleted!

aednichols · 2023-05-19T14:01:30Z

Cromwell is not one of the repos with the newfangled auto-delete branch configured, fortunately.

THWiseman

Nice work, LGTM

THWiseman

Nice work, LGTM

@kevinRenaersBio

* BT-732 Checksum validation for blobs read by engine (broadinstitute#6838) * Draft support for optional FileHash * Draft getMd5 for BlobPath * Resolve non-parallel IO to fix tests * Checksum validation for BlobPath * Nicer error message * Test for missing Blob hash * Break attr acquisition into separate method * Cleanup, comments * In-progress tests of blob hash command * Remove test * Remove unused import * BT-711 Refresh SAS token for filesystem on expiry (broadinstitute#6831) * BT-711 Refresh SAS token for filesystem on expiry * Rough cut of token refresh using exceptions * Ignore tests, and minor cleanup * Remove stray line * Draft of manager class for handling expiring file systems * Style fixes * Refactor of blobfilesystemManager and tests covering its functionality * Refined tests to validate close filesystem as separate unit * Ignore connected tests * Clean up of some things * Refactor BlobFileSystemManager to separate file, and some other cleanup * Some additional scala-ifying * Small cleanup * Correcting imports * trigger tests * trigger tests * Batch 1 of scala steward updates (broadinstitute#6903) * Batch 1 of scala steward updates * Rollback snakeYAML * Attempt 3, with only the passing dependancies * Revert google API and Big Query udpates * Winding back other google deps * rollback remaining google updates * trigger tests * trigger tests * [BW-1398] Migrate PKs to BIGINT (broadinstitute#6907) * BT-745 Batch 2 of scala steward updates (broadinstitute#6906) * Update SBT to 2.0.0 * Fix sbt-git import * Update mouse to 1.0.11 * Update rhino 1.7.14 * SUP-692 Retry with more memory after RC 137 (broadinstitute#6912) * Reorder execution result checks so 137 can retry with more memory * Test for memory retry after 137 RC * Fix test expectations * Make memory retry checks consistent * Revert changes to existing test * Rename retryWithMoreMemory to outOfMemoryDetected * Scala steward updates batch 3 (broadinstitute#6913) * Scala steward updates batch 3 * WX-745 Batch 4 scala steward updates (broadinstitute#6916) * WX-746 Localize all DRS inputs in a single Action (broadinstitute#6914) Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-755 Build all images instead of just Cromwell (broadinstitute#6919) * WX-755 Add `isRelease` option for Docker builds (broadinstitute#6923) * WX-755 Cromwell/CromIAM automatically board train (broadinstitute#6924) * WX-755 Fix environment variable syntax (broadinstitute#6926) * WX-743 Enable TES task creation with BlobPaths (broadinstitute#6921) * Give blob SAS tokens write permission * Case class wrapper for subscription id * Resolve duplicate container name in absolute BlobPath * Ignored test demonstrating correct absolute path generation * Update filesystems/blob/src/test/scala/cromwell/filesystems/blob/BlobPathBuilderSpec.scala Co-authored-by: Brian Reilly <[email protected]> * PR feedback Co-authored-by: Brian Reilly <[email protected]> * [WX-765] Update snakeyaml to 1.33 (broadinstitute#6927) * update snakeyaml to 1.33 * Don't use deprecated no-arg Constructor constructor Co-authored-by: Janet Gainer-Dewar <[email protected]> * WM-1414 Refactoring WesRunLog to omit Cromwell's "workflowLog" object (broadinstitute#6925) * Upgrade Postgres to 42.4.1 (broadinstitute#6932) * WX-735 Fix incorrect and/or nondeterministic filesystem ordering (broadinstitute#6930) * WX-772 Update Scala to 2.13.9 (broadinstitute#6928) * Update Scala to 2.13.9 * Try updating sbt-scoverage * Does this version exist anywhere we can see? * This version actually exists * Update library version to remove conflict * Codegen version * Fix fun new 2.13.9 compiler errors * Resolve warnings * Newest Scala? * I guess not * Does this please Travis? * force ci * Back out changes to generated code Co-authored-by: Adam Nichols <[email protected]> * WX-781 Bump jackson-databind in /CromwellRefdiskManifestCreator (broadinstitute#6935) Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.13.2.2 to 2.13.4.1. - [Release notes](https://github.com/FasterXML/jackson/releases) - [Commits](https://github.com/FasterXML/jackson/commits) --- updated-dependencies: - dependency-name: com.fasterxml.jackson.core:jackson-databind dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * WX-808 Host allowlist for HTTP imports (broadinstitute#6938) * `hostAllowlist` that allows everything * Refactor * Stick allow list in HttpResolver * Better default config * Allow list tests * Make it build Co-authored-by: Janet Gainer-Dewar <[email protected]> * Update commons text to 1.10.0 (broadinstitute#6937) * WX-751 Token refresh signal for monitoring (broadinstitute#6939) * Log messages * `DEBUG` -> `INFO` * WX-744 Optionally rewrite blob paths to appear as local paths (broadinstitute#6941) * Modify blob paths for TES * Make blob transformation configurable * Update supportedBackends/tes/src/main/scala/cromwell/backend/impl/tes/TesTask.scala Co-authored-by: Adam Nichols <[email protected]> * Apply PR feedback in second place Co-authored-by: Adam Nichols <[email protected]> * Update changelog for wdl http allow list (broadinstitute#6944) * WM-1491 Fixing Cromwell-client (broadinstitute#6943) * More updated client for use in cbas * Removing excess code * Fix client build script (broadinstitute#6945) * WX-837: Remove CWL references from documentation (broadinstitute#6949) * wx-837 removed cwl references in markdown doc files * wx-837 removed cwlParsingOverview.md, updated mkdocs.yml * wx-837 updated cromwell.yaml, generated new RESTAPI file * WX-728 Add configurable WSM client to Cromwell (broadinstitute#6948) * Dependencies * Compiles but no tests * Formatting * Moar exclusions * Update to latest WSM * Add additional dependency * We need some UUID here to make the request * Formatting * Clarify what is fake * Formatting * Use our own version of Jersey and Jackson stuff * Port-in Khalid's changes (thank you!) Co-authored-by: Khalid Shakir <[email protected]> * Test longevity Don't break the test if someone decides to add a cert to `ws.org` * Cleanup * Cleanup * Cleanup * Adjust TES config file for CI Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Khalid Shakir <[email protected]> * CROM-6554: Removed PAPIv1 references from doc (broadinstitute#6950) * crom-6554 removed references to PAPI v1 from doc * crom-6554 pr feedback, reworded doc to use example conf as a starting point * WX-833 Real Azure DRS Credentials (broadinstitute#6952) * Remove B2C reference from name * Get token for current user rather than getting from KeyVault * Remove KeyVault config for engine * Remove KeyVault config for DRSLocalizer * Remove KeyVault dependency * Remove KeyVault support from localizer repo template * Cleaned up and working Azure token acquisition for engine * Collapse localizer's AccessTokenStrategy into DrsCredentials * Cleanup * WX-853 Remove most CWL (broadinstitute#6955) * WX-696 Enable getting SAS token from WSM (broadinstitute#6954) * WX-696 Enable getting SAS token from WSM * Wire container resource id from config * Move resource-container-id config path * First pass at config for WSM * Remove unused singleton config * Tests for new config * Fix config parsing * Modified b2c token to be provided each time * Remove singletonConfig arg from factory * Restore types to factory configs * Clean up comments and empty token default * Default to config b2c before searching environment * Fix token default on api client * Fix test * Refactor error handling for when there is no token * Remove token constructor arg for clientProvider * Move configs to global singleton config * Update filesystems/blob/src/main/scala/cromwell/filesystems/blob/BlobFileSystemManager.scala * default -> override * Add override token to test * Update filesystems/blob/src/main/scala/cromwell/filesystems/blob/BlobFileSystemManager.scala Co-authored-by: Adam Nichols <[email protected]> * Parentheses * Reduce token timeout * Move AzureCredentials to separate file * Make AzureCredentials an object * WSM token cleanup * Config refactor (broadinstitute#6960) Co-authored-by: Janet Gainer-Dewar <[email protected]> * Initial blob token documentation * Refine language in BlobSasTokenGenerator * Update comment and formatting Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Adam Nichols <[email protected]> * WX-853 Remove CWL language factory, Centaur runner (broadinstitute#6961) * WX-842 Add Pact Dependency for Cromwell (broadinstitute#6962) * WX-842 Add Pact Dependency for Cromwell * Remove incomplete test spec * Initial Pact Test * Fix pact so it compiles * Add breadcrumb comment and clean up * ID-125 Add support for drshub, rename all the things (broadinstitute#6959) * Add support for drshub, rename all the things * fallback to martha if resolver is not in config * WX-867 Translate crc32c hashes to b64 for getm (broadinstitute#6970) * Translate crc32c hashes to b64 for getm * Update tests * Remove obsolete b64 handling for md5, centralize hex validation * Restore old test, fix other test * WX-843 Workflow failure reason should accurately indicate issues opening blob filesystem (broadinstitute#6965) * WX-859 Accept workflow execution identity in config (broadinstitute#6967) * WX-892 Trim down `ValueStore` logging to prevent OOMs (broadinstitute#6981) * Add Nirvana 3.18.1 reference image test, minor cleanup [VS-705] (broadinstitute#6975) * WX-863 Turn off Azure NIO logging (broadinstitute#6982) * Turn off Azure NIO logging * Poke Travis * WM-1616: Allow repeating attempts at initialization (take 2) (broadinstitute#6985) * WX-878 Single shared BlobFileSystemManager (broadinstitute#6986) * Make BlobFileSystemManager shared across all BlobPathBuilders * Update TES conf file to reflect new singleton config * Shell escape reference image files [VS-796] [WX-910] (broadinstitute#6989) * WX-769 `disks` compatibility for TES backend (broadinstitute#6991) * Update FiveMinuteIntro.md (broadinstitute#6994) * WX-906 Sbt Unit Tests as Github Actions (broadinstitute#6992) * WX-926 Support falling back to OCI Manifest Format (broadinstitute#7003) * WX-926 Support falling back to OCI Manifest Forma * Only mount reference disks if requested [WX-925] (broadinstitute#7001) * [WM-1646] Add missing fields for `WorkflowDescription` for WomTool /describe endpoint to Swagger (broadinstitute#7004) * WX-876 Surface TES System Logs to Cromwell when TES backend returns task error status (broadinstitute#6980) * WX-876 Surface TES System Logs to Cromwell when TES backend returns task error status * Address feedback * Address feedback (broadinstitute#6997) * Address additional feedback (broadinstitute#7000) * Fix copy/paste error (broadinstitute#7005) * Address additional feedback * Fix copy/paste error * Trigger CI --------- Co-authored-by: Blair Murri <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> * Centaur reference image test should validate symlinks [VS-796] (broadinstitute#6996) * WX-903 Pre-GHA test suite disablement * WX-877 Update CHANGELOG for release 85 (broadinstitute#7011) * Update cromwell version from 85 to 86 * WX-905 (broadinstitute#7012) Co-authored-by: Thomas Wiseman <[email protected]> Co-authored-by: Tom Wiseman <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-719 modernize `cromwell-publish` image (broadinstitute#7013) * WX-930 Add `CODEOWNERS` file (broadinstitute#7015) * Update FiveMinuteIntro: Java 11 requirement (broadinstitute#6830) Co-authored-by: Adam Nichols <[email protected]> * WX-952 Fix Cromwell version update in Helm chart (broadinstitute#7014) * Reference Disk Manifest Builder App Test (broadinstitute#7017) * [WM-1696] Update Womtool (broadinstitute#7019) Co-authored-by: Tom Wiseman <[email protected]> * WX-958 write_map() should write its last entry with a newline (broadinstitute#7022) * Centaur slurm (broadinstitute#7083) * Remove Deploy Key (broadinstitute#7084) * WX-950 Set user agent in Java client (broadinstitute#7087) * WX-950 Upgrade Azure libs to probably fix extraneous log (broadinstitute#7088) * WX-984 Revert "WX-950 Upgrade Azure libs to probably fix extraneous log" (broadinstitute#7090) * Add cron run integrations each day of the week (broadinstitute#7089) * 85 release (#28) * Update cromwell version from 83 to 84 * BW-1255 Implement POST /runs endpoint (broadinstitute#6779) * Adding route * Fixing HTTP method error * All formFields made optional * A compliling state * Saving * Saving * All three endpoints functioning as expected; updated RESTAPI.md * Updated response for submission from 200 to 201 to pass tests * Test submission response * Moved updated submission response to askSubmit * test * updating RESTAPI.md * saving * Adding utility file for submitRequest * cleanup * Update awssdkv from 2.17.152 to 2.17.194 (broadinstitute#6814) * BW-1305 Swagger Update (broadinstitute#6818) * Properly documenting metadataArchiveStatus in WorkflowQueryResult model * Update docs * BT-710 Add configs for BlobPathBuilderFactory (broadinstitute#6817) BT-710 Add configs for BlobPathBuilderFactory * BW-1305 Make "name" optional in workflow query response (broadinstitute#6821) * BT-724 Fix BlobPathBuilder failing on retrieving existing filesystem (broadinstitute#6816) Modify blobPathBuilder to fallback to creating a filesystem if one is not found * Logging updates: (broadinstitute#6813) * [BT-698] first pass on BlobTokenGenerator with E2E test (broadinstitute#6824) * first pass on BlobTokenGenerator with E2E test * update BlobPathBuilder constructor args in test * account -> container level client * [BT-687] specify correct types (broadinstitute#6829) * specify correct types * fix test with new type * remove type declarations in function call * remove unnecessary sas-token config * BW-1206 - Combine all Wes Endpoints & add Tests (broadinstitute#6833) * Add tests, getting frid of WesRunRoutes.scala * wesWorkflowId fix, ec implicits errors gone * Refactoring path for GET /runs * Indentation fix * Commit to rollback * Revert "Indentation fix" This reverts commit 63fc484. * PR trigger * Optimize imports * Missed import * BW-1354 - Porting CBAS preliminary step (broadinstitute#6837) * Getting rid of shared utility file; Adding/Updating WES version of submit. * Edit spec file * Adding Wes-like error * BW-1378 Addl CromIAM user enablement checks (broadinstitute#6826) * Update cromwell version from 84 to 85 * BW-1393 Release doc updates (broadinstitute#6839) * BT-732 Checksum validation for blobs read by engine (broadinstitute#6838) * Draft support for optional FileHash * Draft getMd5 for BlobPath * Resolve non-parallel IO to fix tests * Checksum validation for BlobPath * Nicer error message * Test for missing Blob hash * Break attr acquisition into separate method * Cleanup, comments * In-progress tests of blob hash command * Remove test * Remove unused import * BT-711 Refresh SAS token for filesystem on expiry (broadinstitute#6831) * BT-711 Refresh SAS token for filesystem on expiry * Rough cut of token refresh using exceptions * Ignore tests, and minor cleanup * Remove stray line * Draft of manager class for handling expiring file systems * Style fixes * Refactor of blobfilesystemManager and tests covering its functionality * Refined tests to validate close filesystem as separate unit * Ignore connected tests * Clean up of some things * Refactor BlobFileSystemManager to separate file, and some other cleanup * Some additional scala-ifying * Small cleanup * Correcting imports * trigger tests * trigger tests * Batch 1 of scala steward updates (broadinstitute#6903) * Batch 1 of scala steward updates * Rollback snakeYAML * Attempt 3, with only the passing dependancies * Revert google API and Big Query udpates * Winding back other google deps * rollback remaining google updates * trigger tests * trigger tests * [BW-1398] Migrate PKs to BIGINT (broadinstitute#6907) * BT-745 Batch 2 of scala steward updates (broadinstitute#6906) * Update SBT to 2.0.0 * Fix sbt-git import * Update mouse to 1.0.11 * Update rhino 1.7.14 * SUP-692 Retry with more memory after RC 137 (broadinstitute#6912) * Reorder execution result checks so 137 can retry with more memory * Test for memory retry after 137 RC * Fix test expectations * Make memory retry checks consistent * Revert changes to existing test * Rename retryWithMoreMemory to outOfMemoryDetected * Scala steward updates batch 3 (broadinstitute#6913) * Scala steward updates batch 3 * WX-745 Batch 4 scala steward updates (broadinstitute#6916) * WX-746 Localize all DRS inputs in a single Action (broadinstitute#6914) Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-755 Build all images instead of just Cromwell (broadinstitute#6919) * WX-755 Add `isRelease` option for Docker builds (broadinstitute#6923) * WX-755 Cromwell/CromIAM automatically board train (broadinstitute#6924) * WX-755 Fix environment variable syntax (broadinstitute#6926) * WX-743 Enable TES task creation with BlobPaths (broadinstitute#6921) * Give blob SAS tokens write permission * Case class wrapper for subscription id * Resolve duplicate container name in absolute BlobPath * Ignored test demonstrating correct absolute path generation * Update filesystems/blob/src/test/scala/cromwell/filesystems/blob/BlobPathBuilderSpec.scala Co-authored-by: Brian Reilly <[email protected]> * PR feedback Co-authored-by: Brian Reilly <[email protected]> * [WX-765] Update snakeyaml to 1.33 (broadinstitute#6927) * update snakeyaml to 1.33 * Don't use deprecated no-arg Constructor constructor Co-authored-by: Janet Gainer-Dewar <[email protected]> * WM-1414 Refactoring WesRunLog to omit Cromwell's "workflowLog" object (broadinstitute#6925) * Upgrade Postgres to 42.4.1 (broadinstitute#6932) * WX-735 Fix incorrect and/or nondeterministic filesystem ordering (broadinstitute#6930) * WX-772 Update Scala to 2.13.9 (broadinstitute#6928) * Update Scala to 2.13.9 * Try updating sbt-scoverage * Does this version exist anywhere we can see? * This version actually exists * Update library version to remove conflict * Codegen version * Fix fun new 2.13.9 compiler errors * Resolve warnings * Newest Scala? * I guess not * Does this please Travis? * force ci * Back out changes to generated code Co-authored-by: Adam Nichols <[email protected]> * WX-781 Bump jackson-databind in /CromwellRefdiskManifestCreator (broadinstitute#6935) Bumps [jackson-databind](https://github.com/FasterXML/jackson) from 2.13.2.2 to 2.13.4.1. - [Release notes](https://github.com/FasterXML/jackson/releases) - [Commits](https://github.com/FasterXML/jackson/commits) --- updated-dependencies: - dependency-name: com.fasterxml.jackson.core:jackson-databind dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * WX-808 Host allowlist for HTTP imports (broadinstitute#6938) * `hostAllowlist` that allows everything * Refactor * Stick allow list in HttpResolver * Better default config * Allow list tests * Make it build Co-authored-by: Janet Gainer-Dewar <[email protected]> * Update commons text to 1.10.0 (broadinstitute#6937) * WX-751 Token refresh signal for monitoring (broadinstitute#6939) * Log messages * `DEBUG` -> `INFO` * WX-744 Optionally rewrite blob paths to appear as local paths (broadinstitute#6941) * Modify blob paths for TES * Make blob transformation configurable * Update supportedBackends/tes/src/main/scala/cromwell/backend/impl/tes/TesTask.scala Co-authored-by: Adam Nichols <[email protected]> * Apply PR feedback in second place Co-authored-by: Adam Nichols <[email protected]> * Update changelog for wdl http allow list (broadinstitute#6944) * WM-1491 Fixing Cromwell-client (broadinstitute#6943) * More updated client for use in cbas * Removing excess code * Fix client build script (broadinstitute#6945) * WX-837: Remove CWL references from documentation (broadinstitute#6949) * wx-837 removed cwl references in markdown doc files * wx-837 removed cwlParsingOverview.md, updated mkdocs.yml * wx-837 updated cromwell.yaml, generated new RESTAPI file * WX-728 Add configurable WSM client to Cromwell (broadinstitute#6948) * Dependencies * Compiles but no tests * Formatting * Moar exclusions * Update to latest WSM * Add additional dependency * We need some UUID here to make the request * Formatting * Clarify what is fake * Formatting * Use our own version of Jersey and Jackson stuff * Port-in Khalid's changes (thank you!) Co-authored-by: Khalid Shakir <[email protected]> * Test longevity Don't break the test if someone decides to add a cert to `ws.org` * Cleanup * Cleanup * Cleanup * Adjust TES config file for CI Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Khalid Shakir <[email protected]> * CROM-6554: Removed PAPIv1 references from doc (broadinstitute#6950) * crom-6554 removed references to PAPI v1 from doc * crom-6554 pr feedback, reworded doc to use example conf as a starting point * WX-833 Real Azure DRS Credentials (broadinstitute#6952) * Remove B2C reference from name * Get token for current user rather than getting from KeyVault * Remove KeyVault config for engine * Remove KeyVault config for DRSLocalizer * Remove KeyVault dependency * Remove KeyVault support from localizer repo template * Cleaned up and working Azure token acquisition for engine * Collapse localizer's AccessTokenStrategy into DrsCredentials * Cleanup * WX-853 Remove most CWL (broadinstitute#6955) * WX-696 Enable getting SAS token from WSM (broadinstitute#6954) * WX-696 Enable getting SAS token from WSM * Wire container resource id from config * Move resource-container-id config path * First pass at config for WSM * Remove unused singleton config * Tests for new config * Fix config parsing * Modified b2c token to be provided each time * Remove singletonConfig arg from factory * Restore types to factory configs * Clean up comments and empty token default * Default to config b2c before searching environment * Fix token default on api client * Fix test * Refactor error handling for when there is no token * Remove token constructor arg for clientProvider * Move configs to global singleton config * Update filesystems/blob/src/main/scala/cromwell/filesystems/blob/BlobFileSystemManager.scala * default -> override * Add override token to test * Update filesystems/blob/src/main/scala/cromwell/filesystems/blob/BlobFileSystemManager.scala Co-authored-by: Adam Nichols <[email protected]> * Parentheses * Reduce token timeout * Move AzureCredentials to separate file * Make AzureCredentials an object * WSM token cleanup * Config refactor (broadinstitute#6960) Co-authored-by: Janet Gainer-Dewar <[email protected]> * Initial blob token documentation * Refine language in BlobSasTokenGenerator * Update comment and formatting Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Adam Nichols <[email protected]> * WX-853 Remove CWL language factory, Centaur runner (broadinstitute#6961) * WX-842 Add Pact Dependency for Cromwell (broadinstitute#6962) * WX-842 Add Pact Dependency for Cromwell * Remove incomplete test spec * Initial Pact Test * Fix pact so it compiles * Add breadcrumb comment and clean up * ID-125 Add support for drshub, rename all the things (broadinstitute#6959) * Add support for drshub, rename all the things * fallback to martha if resolver is not in config * WX-867 Translate crc32c hashes to b64 for getm (broadinstitute#6970) * Translate crc32c hashes to b64 for getm * Update tests * Remove obsolete b64 handling for md5, centralize hex validation * Restore old test, fix other test * WX-843 Workflow failure reason should accurately indicate issues opening blob filesystem (broadinstitute#6965) * WX-859 Accept workflow execution identity in config (broadinstitute#6967) * WX-892 Trim down `ValueStore` logging to prevent OOMs (broadinstitute#6981) * Add Nirvana 3.18.1 reference image test, minor cleanup [VS-705] (broadinstitute#6975) * WX-863 Turn off Azure NIO logging (broadinstitute#6982) * Turn off Azure NIO logging * Poke Travis * WM-1616: Allow repeating attempts at initialization (take 2) (broadinstitute#6985) * WX-878 Single shared BlobFileSystemManager (broadinstitute#6986) * Make BlobFileSystemManager shared across all BlobPathBuilders * Update TES conf file to reflect new singleton config * Shell escape reference image files [VS-796] [WX-910] (broadinstitute#6989) * WX-769 `disks` compatibility for TES backend (broadinstitute#6991) * Update FiveMinuteIntro.md (broadinstitute#6994) * WX-906 Sbt Unit Tests as Github Actions (broadinstitute#6992) * WX-926 Support falling back to OCI Manifest Format (broadinstitute#7003) * WX-926 Support falling back to OCI Manifest Forma * Only mount reference disks if requested [WX-925] (broadinstitute#7001) * [WM-1646] Add missing fields for `WorkflowDescription` for WomTool /describe endpoint to Swagger (broadinstitute#7004) * WX-876 Surface TES System Logs to Cromwell when TES backend returns task error status (broadinstitute#6980) * WX-876 Surface TES System Logs to Cromwell when TES backend returns task error status * Address feedback * Address feedback (broadinstitute#6997) * Address additional feedback (broadinstitute#7000) * Fix copy/paste error (broadinstitute#7005) * Address additional feedback * Fix copy/paste error * Trigger CI --------- Co-authored-by: Blair Murri <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> * Centaur reference image test should validate symlinks [VS-796] (broadinstitute#6996) * WX-903 Pre-GHA test suite disablement * WX-877 Update CHANGELOG for release 85 (broadinstitute#7011) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Katrina P <[email protected]> Co-authored-by: Chris Llanwarne <[email protected]> Co-authored-by: Christian Freitas <[email protected]> Co-authored-by: Saloni Shah <[email protected]> Co-authored-by: kshakir <[email protected]> Co-authored-by: mspector <[email protected]> Co-authored-by: Adam Nichols <[email protected]> Co-authored-by: Brian Reilly <[email protected]> Co-authored-by: Adam Nichols <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Justin Variath Thomas <[email protected]> Co-authored-by: Christian Freitas <[email protected]> Co-authored-by: Trevyn Langsford <[email protected]> Co-authored-by: Miguel Covarrubias <[email protected]> Co-authored-by: ekiernan <[email protected]> Co-authored-by: Tom Wiseman <[email protected]> Co-authored-by: Blair Murri <[email protected]> * Remove Yaml (broadinstitute#7098) * Azure blob read/write integration test (broadinstitute#7024) Co-authored-by: Tom Wiseman <[email protected]> * Goodbye Travis (broadinstitute#7100) * Single workflow runner (broadinstitute#7106) * WX-975 Revert special TES path munging (broadinstitute#7103) * Centaur blob filesystem (broadinstitute#7104) * WX-1010 Update Github Actions set output syntax (broadinstitute#7111) * WX-1001 Upgrade JSON Small and Fast Parser (broadinstitute#7116) * WX-1015 Override glob behavior for TES to use absolute path (broadinstitute#7120) * WX-990 Make TES request backoff behavior configurable (broadinstitute#7122) * WX-1103 Revert CI to old TES polling behavior (broadinstitute#7126) * Develop aws (#29) * stuck on globbing * efs works, no callcaching * update readme * extended EFS support * fix for globbing in nested scatters * updated config for globbing, to prevent issues with empty folders * WX-1103 Accelerate TES CI (broadinstitute#7130) * Fix Horicromtal Deadlock Test (broadinstitute#7131) * WM-1963: Validate PRs begin with Jira tags (broadinstitute#7127) * WX-1106 Add logging for failed docker manifest pulls (broadinstitute#7135) Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Adam Nichols <[email protected]> * [WX-1108] Disable drs tests (broadinstitute#7145) * ID-377 Setup pact for consumer contract testing. (broadinstitute#7123) Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-1092 Support `size` engine function for public HTTP files (broadinstitute#7128) * WX-1105 Fix interpretation of full http blob paths (broadinstitute#7138) Co-authored-by: Adam Nichols <[email protected]> * Revert "ID-377 Setup pact for consumer contract testing." (broadinstitute#7146) * WX-966 Add Cascades, remove Directory from Biscayne (broadinstitute#7105) * WX-966 Post-merge branch cleanup + test fix (broadinstitute#7149) * WX-1122 Enable Azure ApplicationInsights (broadinstitute#7143) * [WX-1108] Add tests back (broadinstitute#7148) * [WX-1136] Docker Build Test (broadinstitute#7151) * [WX-1136] Self Hosted (broadinstitute#7152) * WX-1145 Fix minor regression introduced in WDL 1.1 foundation (broadinstitute#7153) * ID-377 Setup pact for consumer contract testing. Retry (broadinstitute#7147) Co-authored-by: Tom Wiseman <[email protected]> Co-authored-by: Ivan <[email protected]> * WX-1133 Shorten TES localized blob paths (broadinstitute#7150) * WX-1154 Fix Slack Messaging (broadinstitute#7159) * efs fixes : support paths with over 127 characters, fix delocalization of efs-based globs (#32) * WX-1160 Fix bad MariaDB version assumptions (broadinstitute#7160) * WX-1126 Upgrade to modern Python, 3.8 not available in package repo anymore (broadinstitute#7164) * WX-1122 Use legacy AppInsights to get better control over logging (broadinstitute#7157) * WX-1137 Replace 4-byte unicode chars in PAPI event descriptions (broadinstitute#7166) * WX-1101 Update library versions to support batch (broadinstitute#7155) Co-authored-by: Adam Nichols <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-1195 Update Azure identity libs (broadinstitute#7173) * WX-1114 Initial inclusion of Azure NIO 'fork' (broadinstitute#7168) * WX-1110[risk=low] Added endpoint to fetch failed tasks by root workflow id (broadinstitute#7165) * WX-1210 Added JIRA ID for Cromwhelm auto commit message (broadinstitute#7180) * WX-1179 DRS image build updates, remove self-hosted runners (broadinstitute#7179) * WX-1210-action-fix Use PR title to find JIRA ID for cromwhelm commit message (broadinstitute#7184) * WX-1110-query-fix Corrected Query to pull in attributes outside of executionStatus and backendStatus (broadinstitute#7185) * WX-1179 GCP Batch new base (broadinstitute#7177) Co-authored-by: dspeck <[email protected]> Co-authored-by: jarroyo28 <[email protected]> Co-authored-by: Jacob Jennings <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Tom Wiseman <[email protected]> Co-authored-by: Chris Llanwarne <[email protected]> Co-authored-by: Christian Freitas <[email protected]> Co-authored-by: Tristan Garwood <[email protected]> * WM-2100: Update terra-helmfile on publish (broadinstitute#7187) * WM-2100: Correct commit and PR description of terra-helmfile auto update (broadinstitute#7188) * WX-1230 Ignore `invalidate_bad_caches_use_good_local.test` (broadinstitute#7197) * WX-1078 ACR support (broadinstitute#7192) * WX-1179 Enable GCP Batch Integration Tests (broadinstitute#7199) Co-authored-by: Adam Nichols <[email protected]> Co-authored-by: Adam Nichols <[email protected]> * WX-1179 GCP Batch Docs Update (broadinstitute#7196) Co-authored-by: Jacob Jennings <[email protected]> Co-authored-by: Adam Nichols <[email protected]> Co-authored-by: Adam Nichols <[email protected]> * ID-734 Increase Timeout for DRSHub Communication (broadinstitute#7198) * [WX-1156] internal_path_prefix for TES 4.4 (broadinstitute#7190) * [WM-2184] Remove brackets from Jira ID (broadinstitute#7206) * WX-1153 Azure blob read md5 from metadata for large files (broadinstitute#7204) Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-1156 Fix internal_path_prefix (broadinstitute#7208) Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-1256 Temporarily turn off engine hashing for blob files (broadinstitute#7209) Co-authored-by: Adam Nichols <[email protected]> * WX-1173 Reopen filesystem for blob storage outside workspace (broadinstitute#7178) Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Tom Wiseman <[email protected]> Co-authored-by: Adam Nichols <[email protected]> * WX-1174 Adjust NIO Copy functionality (broadinstitute#7207) Co-authored-by: Adam Nichols <[email protected]> * [WX-1168] TES Log Paths (broadinstitute#7210) * Develop aws (#34) * efs fixes : support paths with over 127 characters, fix delocalization of efs-based globs * add deployment manual, fix issue with empty disks * update documentation * update documentation * update documentation * WX-1264 Don't expire an unexpirable filesystem (broadinstitute#7216) * [WX-495] DRS Parallel Downloads (broadinstitute#7214) * WX-1225 Print TES error messages to job logger (broadinstitute#7220) * WX-1217 Workflow completion callback (broadinstitute#7213) Co-authored-by: Chris Llanwarne <[email protected]> * [WM-2199] Changeset for modifying ownership of tables for WORKFLOWS app (broadinstitute#7218) Co-authored-by: Chris Llanwarne <[email protected]> * [WX-1234] Update CHANGELOG.md (broadinstitute#7227) Merging past CI since this is doc only. * WM-2252: Configurable metadata write role (broadinstitute#7225) Co-authored-by: dvoet <[email protected]> * Update cromwell version from 86 to 87 * WX-1282 Update failedJobs Query to use `lo_get` instead of INNER JOIN against pg_largeobject (broadinstitute#7228) * [WX-1234] Update Release Process with Docker Instructions (broadinstitute#7231) Co-authored-by: Adam Nichols <[email protected]> * WM-2294: Allow role-setter action to run on change (broadinstitute#7233) * WM-2296: Callback should supply fully qualified output names (broadinstitute#7234) * [WX-499] DRS Parallel Downloads Follow-up (broadinstitute#7229) * WX-1318 gcp batch: Add GPU driver install (broadinstitute#7235) Co-authored-by: Adam Nichols <[email protected]> * WX-1232 Include useful workflow ids in TES tags (broadinstitute#7221) * Options to publish status only (#36) * add options to publish status only * updated readme.md --------- Co-authored-by: quekx <[email protected]> * WX-1307 Azure E2E test (broadinstitute#7239) Co-authored-by: Janet Gainer-Dewar <[email protected]> * Fix aws unit tests (#39) * checkpoint * fix ecr and batch tests * fix AwsBatchJobSpec.scala --------- Co-authored-by: quekx <[email protected]> * return bucket directly instead of listing and checking it (#38) Co-authored-by: quekx <[email protected]> * WX-1340 GCP Batch: Mount with extra colon issue and multiple zones support (broadinstitute#7240) Co-authored-by: Adam Nichols <[email protected]> * WX-1339 Make `throwExceptionOnExecuteError` false for PAPI aborts (broadinstitute#7245) * WX-1338 Fix cron invocation of E2E test (broadinstitute#7244) * WX-1341 Better logging when a runner stops picking up new workflows (broadinstitute#7246) * Add evaluteOnExit for aws batch retry (#40) Co-authored-by: quekx <[email protected]> * [WX-1260] Acquire sas token from task runner (broadinstitute#7241) Co-authored-by: Adam Nichols <[email protected]> * [WX-1183] Ice Lake (broadinstitute#7252) * WX-1333 Improve logging visibility for load management (broadinstitute#7253) * [WX-1391] Fix Bash Bug (broadinstitute#7326) * [WX-1393] Add Content Length to Curl request (broadinstitute#7328) * [WX-1346] Scalafmt (broadinstitute#7257) * WX-1351 Speed up `Centaur Horicromtal PapiV2 Beta` (broadinstitute#7329) * WX-1351 CI CWL cleanup (broadinstitute#7327) * WX-1351 Remove slow/misbehaving localization test (broadinstitute#7330) * WX-1351 Split out restart tests (broadinstitute#7333) * [WX-1345] Automatic Token Acquisition for TES Config (broadinstitute#7256) * WX-1387 Remove unused case classes (broadinstitute#7334) * [WX-1394] Upgrade Logback Core to address CVE-2023-6378 (broadinstitute#7332) * WX-1351 Another round of CI cleanup (broadinstitute#7336) * [WX-1395] Scalafmt GitHub Action (broadinstitute#7337) * WX-1351 SBT compile speedup (broadinstitute#7339) * WX-1407 Fix contract tests (broadinstitute#7340) Co-authored-by: Ivan <[email protected]> * WX-1409 Java 17 (broadinstitute#7342) * Revert "WX-1409 Java 17 (broadinstitute#7342)" (broadinstitute#7343) * WX-1409 Restore Java 17 (broadinstitute#7342) (broadinstitute#7344) * Revert "WX-1409 Java 17" (broadinstitute#7346) * [WX-1395] Scala formatter Github Action (broadinstitute#7341) Co-authored-by: ScalaFmt Fixer <[email protected]> Co-authored-by: Broad Bot <[email protected]> * Improved tagging support (#37) * efs fixes : support paths with over 127 characters, fix delocalization of efs-based globs * add deployment manual, fix issue with empty disks * update documentation * update documentation * update documentation * support for tagging instances and volumes used in jobs, some support for spaces in file paths * corrected workflow id in tagging * redirect exit code 137 to retry-with-more-memory routine --------- Co-authored-by: Henrique Ribeiro <[email protected]> * add gpu count (#41) * add gpu count * fix typo --------- Co-authored-by: quekx <[email protected]> Co-authored-by: Henrique Ribeiro <[email protected]> * WX-1385 Reject blob URLs with external SAS tokens as unparsable (broadinstitute#7347) * WX-1411 Require preinstalled `jq` and `curl` for just-in-time SAS feature (broadinstitute#7350) * WX-1396 Cromwell client allow additional properties (broadinstitute#7352) * WM-2428: Include full error context when failing to abort TES jobs (broadinstitute#7354) * WX-1420 Fix GCP Batch label regex restriction (broadinstitute#7355) Co-authored-by: Beibei Chen <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-1385 Remove SAS tokens from TES input log printouts (broadinstitute#7358) * [WX-1184] PostgreSQL Docker Image for Local Cromwell (broadinstitute#7172) Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-1444 Use MySQL LTS in DBMS tests (broadinstitute#7360) * WX-1449 Add `latest` Docker tag (broadinstitute#7362) * WX-767 Upgrade Cloud SDK to 461.0.0 (broadinstitute#7361) * WX-1443 Adopt `gcloud storage` for localization only (broadinstitute#7359) * WX-964 suffix() (broadinstitute#7363) * WX-1417 New database role strategy (broadinstitute#7366) * [WM-2291] Callback API contract tests between Cromwell and CBAS (broadinstitute#7251) Co-authored-by: Chris Llanwarne <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> * WX-1445 Update docker image regex to handle python:3 correctly (broadinstitute#7367) * WX-1252 Runtime attributes cleanup – CWL runtime attributes (broadinstitute#7370) * WX-1485 Upgrade postgres lib (broadinstitute#7371) * WM-2454: Private GitHub support on describe api (broadinstitute#7365) * WX-1252 Runtime attributes cleanup – CWL runtime environment (broadinstitute#7369) * WX-1462 POSIX-flavored sub() (broadinstitute#7374) * [WX-963] Unzip Engine Function (broadinstitute#7368) * WX-1489 Hopefully reduce CI flakiness by modernizing deadlock test (broadinstitute#7376) * [WX-965] quote() and squote() engine functions. (broadinstitute#7375) Co-authored-by: Ryan Saperstein <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> * [WX-1317] Remove Akka 'server' header from all HTTP responses (broadinstitute#7379) * WM-2461] Add support for running private workflows on Azure (broadinstitute#7373) * WX-1252 Per-backend runtime attributes (broadinstitute#7380) * ID-347 Remove Martha References (broadinstitute#7384) * WX-1461 Remove `womtool upgrade` command (broadinstitute#7382) * WX-1488 Supply cloud platform when making DRS requests (broadinstitute#7381) * WX-757 Fix workflow stuck in aborting after WDL type error (broadinstitute#7385) * [WX-1506] Add jobLogger output for subworkflow running (with parent and subworkflow ID) (broadinstitute#7388) Co-authored-by: jlester-msft <[email protected]> * WX-757 Fix `stdout`, `stderr` in workflow body causing crashes (broadinstitute#7386) * WX-1519 Stop leaking DRS Localizer images on every CI run (broadinstitute#7390) * [WX-1460] WDL 1.1 Struct Literal Parsing (broadinstitute#7391) Co-authored-by: Janet Gainer-Dewar <[email protected]> * [WM-2500][WM-2502] Fetch Github token from ECM for importing and running private workflows (broadinstitute#7392) * WX-1542 Fix workflow cancellation (broadinstitute#7398) * WX-1550 Don't pointlessly continue running integration tests (broadinstitute#7400) * WX-1557 Fix default values from Configuring.md and Scaling.md (broadinstitute#7393) Co-authored-by: Adam Nichols <[email protected]> * WX-1557 Fix actor-factory in Batch101.md (broadinstitute#7377) Co-authored-by: Adam Nichols <[email protected]> * WX-1557 Add more tests to the GCP Batch backend (broadinstitute#7394) Co-authored-by: Adam Nichols <[email protected]> * [WX-1531] Struct Literal Type Checking (broadinstitute#7402) * [WX-1468] Implement `returnCodes` runtime attribute (broadinstitute#7389) Co-authored-by: Ryan Saperstein <[email protected]> Co-authored-by: Ryan Saperstein <[email protected]> * [WM-2555] Cromwell -> ECM contract test (broadinstitute#7405) * [WX-1568] Bump Akka max-response-reason-length limit (broadinstitute#7406) * WX-1530 Strip pesky URL bits when creating local paths for HTTP inputs (broadinstitute#7404) * WX-1538 Add more tests to the GCP Batch backend (broadinstitute#7410) * [WX-1605] Fix Codecov Report Uploads (broadinstitute#7418) * [WX-1410] Sanitize 4 byte UTF-8 characters before inserting into METADATA_ENTRY (broadinstitute#7414) * WX-1566 Special Docker build for debugging (broadinstitute#7417) Run `sbt -Dproject.isDebug=true server/docker` * [WX-1448] Add verbose logging and timeout for getm (broadinstitute#7416) Co-authored-by: Adam Nichols <[email protected]> * WX-1566 Fix Morgan's call cache file hash CPU thrash Cromwell crash (broadinstitute#7419) * WX-1566 Fix RTD so this ticket's docs get published (broadinstitute#7420) * fix womFileType coercion (#47) Co-authored-by: quekx <[email protected]> * PR #44: Multi-aserisk globbing by @kevinRenaersBio. Reapplied selection of changes on aws_develop branch * PR #45 reapplied to aws_develop: support for sharedMemory. skipped EFS fixes for now while waiting for feedback * upstream PR broadinstitute#7488 : fix compilation error due to swagger2markup * Fix issue with caching on EFS by replacing touch with existOrThrow routine * make sure that md5 sibling file of EFS data is newer than main file to be valid * revise sharedmemory attribute to take reguluar GB/MB specification * updated AWS README file --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Janet Gainer-Dewar <[email protected]> Co-authored-by: Christian Freitas <[email protected]> Co-authored-by: Saloni Shah <[email protected]> Co-authored-by: Brian Reilly <[email protected]> Co-authored-by: Adam Nichols <[email protected]> Co-authored-by: Katrina P <[email protected]> Co-authored-by: Adam Nichols <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Llanwarne <[email protected]> Co-authored-by: Justin Variath Thomas <[email protected]> Co-authored-by: Khalid Shakir <[email protected]> Co-authored-by: Christian Freitas <[email protected]> Co-authored-by: Trevyn Langsford <[email protected]> Co-authored-by: Miguel Covarrubias <[email protected]> Co-authored-by: ekiernan <[email protected]> Co-authored-by: Tom Wiseman <[email protected]> Co-authored-by: Blair Murri <[email protected]> Co-authored-by: Thomas Wiseman <[email protected]> Co-authored-by: Tom Wiseman <[email protected]> Co-authored-by: Stephen Fleming <[email protected]> Co-authored-by: Henrique Ribeiro <[email protected]> Co-authored-by: mspector <[email protected]> Co-authored-by: Tristan Garwood <[email protected]> Co-authored-by: Ivan <[email protected]> Co-authored-by: dspeck1 <[email protected]> Co-authored-by: jarroyo28 <[email protected]> Co-authored-by: Jacob Jennings <[email protected]> Co-authored-by: Dillon Scott <[email protected]> Co-authored-by: Chris Llanwarne <[email protected]> Co-authored-by: dvoet <[email protected]> Co-authored-by: xquek <[email protected]> Co-authored-by: quekx <[email protected]> Co-authored-by: ScalaFmt Fixer <[email protected]> Co-authored-by: Broad Bot <[email protected]> Co-authored-by: Beibei Chen <[email protected]> Co-authored-by: Maria Yazykova <[email protected]> Co-authored-by: rsaperst <[email protected]> Co-authored-by: Ryan Saperstein <[email protected]> Co-authored-by: jlester-msft <[email protected]> Co-authored-by: Alexis Hernandez <[email protected]> Co-authored-by: dkj <[email protected]> Co-authored-by: Ryan Saperstein <[email protected]> Co-authored-by: Alexis Hernandez <[email protected]>

aednichols added 13 commits March 27, 2023 21:01

Add Cascades, remove Directory from Biscayne

0076eea

Merge remote-tracking branch 'origin/develop' into aen_wx_966

0778940

Test fixes

88233b3

Fix tests

8c287c7

Fix tests

299406e

Fix tests

0a0b64b

Fix tests

c9bd54f

Fix tests

588dca2

Fix tests

5097cd4

OMG

d812b7d

Necessary but not sufficient

6164bf3

Fix tests

7248929

Fix tests

df18c8c

aednichols marked this pull request as ready for review April 4, 2023 04:53

aednichols requested a review from a team as a code owner April 4, 2023 04:53

THWiseman reviewed Apr 6, 2023

View reviewed changes

aednichols added 4 commits April 12, 2023 18:18

Merge remote-tracking branch 'origin/develop' into aen_wx_966

265e7e6

PR feedback

a4d2d63

PR feedback

672c056

Fix expectations

f482c41

These tests are really a bit too sensitive to normal variation in the AST...

jgainerdewar reviewed Apr 19, 2023

View reviewed changes

aednichols requested a review from cjllanwarne April 20, 2023 14:55

cjllanwarne reviewed Apr 20, 2023

View reviewed changes

aednichols added 2 commits May 15, 2023 15:33

Merge remote-tracking branch 'origin/develop' into aen_wx_966

176de35

Merge remote-tracking branch 'origin/develop' into aen_wx_966

d5fa2fc

aednichols changed the title ~~Add Cascades, remove Directory from Biscayne~~ WX-966 Add Cascades, remove Directory from Biscayne May 16, 2023

PR updates

a738ca3

aednichols requested review from THWiseman, cjllanwarne and jgainerdewar May 18, 2023 15:13

jgainerdewar approved these changes May 18, 2023

View reviewed changes

THWiseman approved these changes May 19, 2023

View reviewed changes

aednichols added 2 commits May 22, 2023 20:05

Merge branch 'develop' into aen_wx_966

a3351bf

Merge branch 'develop' into aen_wx_966

5995dca

aednichols self-assigned this May 24, 2023

aednichols added 2 commits May 24, 2023 09:23

Merge branch 'develop' into aen_wx_966

fecdc12

Merge branch 'develop' into aen_wx_966

45aefa7

aednichols merged commit 28cdba7 into develop May 25, 2023

aednichols deleted the aen_wx_966 branch May 25, 2023 03:49

aednichols restored the aen_wx_966 branch May 25, 2023 13:21

aednichols deleted the aen_wx_966 branch May 25, 2023 21:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WX-966 Add Cascades, remove Directory from Biscayne #7105

WX-966 Add Cascades, remove Directory from Biscayne #7105

aednichols commented Mar 28, 2023 •

edited

Loading

THWiseman Apr 6, 2023

aednichols Apr 12, 2023

aednichols Apr 12, 2023

cjllanwarne Apr 20, 2023

THWiseman Apr 6, 2023

jgainerdewar commented Apr 19, 2023

jgainerdewar Apr 19, 2023

jgainerdewar Apr 19, 2023

aednichols May 16, 2023

jgainerdewar Apr 19, 2023

jgainerdewar Apr 19, 2023

jgainerdewar Apr 19, 2023

cjllanwarne Apr 20, 2023

cjllanwarne Apr 20, 2023

aednichols Apr 20, 2023 •

edited

Loading

aednichols commented Apr 20, 2023

cjllanwarne Apr 20, 2023

aednichols May 16, 2023

cjllanwarne Apr 20, 2023

aednichols May 16, 2023

jgainerdewar commented May 18, 2023

aednichols commented May 18, 2023 •

edited

Loading

jgainerdewar commented May 18, 2023

aednichols commented May 19, 2023

THWiseman left a comment

THWiseman left a comment


		class CascadesValueEvaluatorSpec extends AnyFlatSpec with CromwellTimeoutSpec with Matchers {

		behavior of "biscayne value evaluator"

WX-966 Add Cascades, remove Directory from Biscayne #7105

WX-966 Add Cascades, remove Directory from Biscayne #7105

Conversation

aednichols commented Mar 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgainerdewar commented Apr 19, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aednichols Apr 20, 2023 • edited Loading

Choose a reason for hiding this comment

aednichols commented Apr 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgainerdewar commented May 18, 2023

aednichols commented May 18, 2023 • edited Loading

jgainerdewar commented May 18, 2023

aednichols commented May 19, 2023

THWiseman left a comment

Choose a reason for hiding this comment

THWiseman left a comment

Choose a reason for hiding this comment

aednichols commented Mar 28, 2023 •

edited

Loading

aednichols Apr 20, 2023 •

edited

Loading

aednichols commented May 18, 2023 •

edited

Loading