Reintroduce a pipeline type

[StewartW]

Hey,
We've got a business requirement for having some pipelines that have drastically different shapes.
As such we've made some changes to adf-build/shared/cdk/cdk_stacks/main.py (and schema)

example_snippet.txt

In the example snippet, you can see that we have two skyscanner specific pipelines, these are currently situated in adf/shared/cdk/cdk_stacks/skyscanner_stacks.py.
(Ideally, the dream would be to install these as part of the requirements.txt)
I think a touch would be if, what we call, the adf_default pipeline was also situated outside of main.py and imported in.

I think it would be quite beneficial for other users as it means that any updates to the default pipeline won't require extensive merge conflicts, and gives it a certain flexibility that would probably be beneficial to most ?

Let me know what you think? This is something I'm happy to work together on if you think it needs refined.

Edit: I've got a branch with how I think this would look.

[bundyfx]

Thanks for opening the issue, I think there is something we can do further to make this more modularized for sure. Im going to have a think/discussion about this one a bit more and take your example into mind. Will update this shortly with some more thoughts but I think you are on the right train of thought here.

[sbkok]

Thank you for your patience. I am happy to inform you that this issue has been resolved in our latest release v3.2.0 just now.
I'm hereby closing this issue. Please open a new issue if you are experiencing any issues with the latest release.