Inconsistent types for "memory" runtime attribute?

[natechols]

I'm trying to pass the memory attribute to various backends, including both AWS and SGE (using a modified config where I added String? memory in place of Float? memory_gb). Passing values like "4G" as the manual suggests works fine on AWS, but on SGE I get this:

cromwell.core.CromwellFatalException: java.lang.RuntimeException: Unsupported wdl type for memory: WomStringType

Am I missing something or is the type inconsistent between backends? That's going to make it a lot more work to set up.

[illusional]

Hey @natechols, not part of broad team, but what does your config looks like for the SGE cluster. The memory variable gets turned into memory_mb or memory_gb in your runtime attributes which you can use to prepare the SGE job:

https://cromwell.readthedocs.io/en/stable/RuntimeAttributes/

[natechols]

@illusional The key part was this:

        runtime-attributes = """
            Int cpu = 1
            String? memory
        """

plus passing memory to the submit command - which in my case is a wrapper script that can interpret strings like "4GB".

After another hour of trial-and-error I did finally get a working config using memory_gb in the config file instead of memory. But I have to say, a colleague and I read the documentation on memory repeatedly and we're still confused about what it's trying to say and how that relates to what actually happens. (In particular, it is not clear that you can't pass memory directly to non-SFS backends.) The fact that both memory and memory_gb can be task runtime parameters, yet one will presumably be overridden anyway, seems unwise.

[illusional]

Yeah I think there are a few pieces to the puzzle that I also didn't understand for a long time:

• The OpenWDL spec doesn't make opinions on how memory, disks, cpus should be specified. You can specify literally any key-value pairs you want.

  • NB, they're talking about this on the openwdl forums: Clarified Runtime and Added Hints openwdl/wdl#315

• Cromwell have opinions on what runtime fields it recognises (on the RuntimeAttributes link).

• Cromwell specifically transforms memory into memory_gb and memory_mb for shared-filesystems.

My recommendation is in your wdl file, keep passing memory: "4G", then use the:

        runtime-attributes = """
            Int cpu = 1
            Int memory_mb = 2048
        """

And using the `~{memory_mb}` placeholder in your `submit` / `submit-docker` block.