Per-job Instance Types and AMIs

[AndrewPlayer3]

No description provided.

[jhkennedy]

which specifies that there should be a "deployment-specified gpu ami and instance type fleet" and a "deployment-specified cpu ami and instance type fleet". This is probably more flexible so perhaps better than what was written for the acceptance criteria, but we should get buy-in from at least @forrestfwilliams on this approach.

Looking ahead, we'll be moving ARIA S1 GUNW (INSAR_ISCE) jobs into production HyP3 soon, which are currently running on the C or M instance family (we found them to be the most cost-optimal). So, I think this approach is better overall as we'll have broader needs than just GPU vs CPU.

This will have implications for our current cost control strategy.

Right now, we do cost control by limiting the max vcpus, which works well with one compute environment. Multiple compute environments will definitely reduce the effectiveness of this strategy. For the LAVAs deployment, I don't think we'll be using the EDC cost controls, so won't matter there. We can side-step the cost control worry in EDC by keeping it a single compute environment there for now, but we'll need to tackle it when we bring in GUNWs if we want to use the C/M instance families.

I'm not sure how forward-looking we want to be overall here, but we definitely don't want to put ourselves in a corner later.

[jtherrmann]

Looks like these are just extra blank lines, not a trailing newline.

[jtherrmann]

All of the code changes look good, but I still want to render the templates locally and review the changes.

[jtherrmann]

I also confirmed that none of our existing CF resources have been renamed.

[jtherrmann]

I've confirmed that the templates render locally as expected.

[jtherrmann]

The snyk failure is unrelated to this PR.