Generate unique SageMaker training job name based on pipeline and ste…

[christianversloot]

…p name

Describe changes

Attempts fixing #1502.

Pre-requisites

Please ensure you have done the following:

• I have read the CONTRIBUTING.md document.
• If my change requires a change to docs, I have updated the documentation accordingly.
• If I have added an integration, I have updated the integrations table and the corresponding website section.
• I have added tests to cover my changes.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Other (add details above)

[christianversloot]

On Slack, we agreed to the following approach:

Suggested approach:
Compose name composed_name as {pipeline_name}-{step-name}, cut it off to 63-5 = 58 chars if necessary.
Then add the suffix as {composed_name}-{random_suffix}, where random suffic is a length-4 random suffix.

...with the slight change that I am using 60 characters as a safety margin; let me know if this must be omitted.

Using the StepRunInfo object available to the step operator, and the PipelineConfiguration and StepConfiguration objects available in there, this PR first attempts to construct the {pipeline_name}-{step_name} composition as discussed above.

In doing so, I need to use Client() (I borrowed that idea from get_image(...) in the StepRunInfo object), but I am not yet sure if that's the right approach. Feel free to make other suggestions.

Then, as the suffix plus the - character will be 5 characters, the composed name is cut off to the first 55 characters. To that, the generated (pseudo)random suffix is appended as well as the - separator. This is sanitized (as was done previously) and subsequently used to fit the estimator.

Looking forward to your thoughts!

[strickvl]

Looks good, but I had one query about whether to take some of the string construction logic outside the main launch method..

[strickvl]

Reasonable to use the Client in there, as well, IMO.

A bit hard separate the PR from the various formatting changes. Assuming you branched off develop, unsure why it wants to make all those changes since running the format script on my end doesn't make any such changes on develop.

[christianversloot]

Agreed, I don't know either. Without it the build, lint tests etc would break.

[strickvl]

Ok I think this looks good now. I would like to fix the formatting issue, however, before merging this, and will let the CI / tests run through as well.

[christianversloot]

Just saw your previous comment. I have reverted the mass formatting commit in af51e83 and ran ruff and black manually on the src/zenml/integrations/aws/step_operators containing the only changes in this PR in 5daed8b. This should mean that the Files changed section is ok again while lint checks etc should pass.