As a user, I can specify a single monorepo Habushu module and Habushu will stage the source code required to build that module in my build directory, so I can containerize my python services

[Cho-William]

Description

This ticket is the first of a series to implement a more resilient and consistent pattern for installing Python monorepo dependencies to Docker containers. The scope of this ticket will cover augmenting Habushu's existing functionality to utilize the Maven session to stage Python module source code files to the docker build context (without needing to specify any relative-paths for monorepo Python dependencies), and leveraging these changes with the aiSSEMBLE baseline's relevant docker images.

DOD

• Update the relevant docker module poms in the aiSSEMBLE baseline:
  • Remove the Orphedomos retrieve-wheels execution
  • Add the new Habushu execution of containerize-dependencies
• Update the relevant docker module Dockerfiles in the aiSSEMBLE baseline:
  • Modify build to work with the Habushu-injected logic
• Ensure the modified docker modules in the baseline can build successfully and run

Test Steps

1. OTS Only: Build python Docker images in the baseline and ensure they build successfully

mvn clean install -am -pl :aissemble-model-training-api,:aissemble-model-training-api-sagemaker,:aissemble-versioning

2. Run aissemble-model-training-api-sagemaker and ensure it starts successfully (i.e. uvicorn starts up):

docker run --rm -it ghcr.io/boozallen/aissemble-model-training-api-sagemaker:1.8.0-SNAPSHOT

3. Create a new project:

mvn archetype:generate -DarchetypeGroupId=com.boozallen.aissemble \
                       -DarchetypeArtifactId=foundation-archetype \
                       -DarchetypeVersion=1.8.0-SNAPSHOT \
                       -DgroupId=org.test \
                       -DartifactId=test-150 \
                       -DprojectGitUrl=test.org/test-150.git \
                       -DprojectName="Test Issue 150" \
&& cd test-150

4. Add TrainingPipeline.json to test-150-pipeline-models/src/main/resources/pipelines
5. Run mvn clean install and resolve manual actions
6. Repeat until no more actions are printed
7. Deploy the project and wait for everything to start: tilt up
8. Verify that the versioning service started up and is listening on port 80
9. Submit the training job to the model-training-api:

curl -X POST -H "Content-Type: application/json" 'localhost:5001/training-jobs?pipeline_step=ExampleTrainingStep' -d '{}'

11. Retrieve the submitted jobs for the training step and verify the new job appears:

curl 'localhost:5001/training-jobs?pipeline_step=ExampleTrainingStep' | sed 's/\\n/\n/g'

12. Retrieve the logs for the submitted job and verify the job completed successfully:

curl localhost:5001/training-jobs/<JOB NAME FROM PREVIOUS STEPS> | sed 's/\\n/\n/g'

[ewilkins-csi]

OTS completed with @Cho-William

[Cho-William]

Final test passed!