Thoughts on managing CI tooling? Maybe using bitbucket pipes?

[aaronbauman]

ref: https://confluence.atlassian.com/bitbucket/how-to-write-a-pipe-for-bitbucket-pipelines-966051288.html

I'm finding that, as the volume of CI projects we're managing grows, the proliferation of CI scripts is quickly becoming unweildy. It's not practical for my team to support an entirely decoupled CI stack for every new project. If there were a way that my CI-related devops tooling could inherit from an upstream, that would lift a big management burden for us.

Could Bitbucket Pipes provide such a solution?
Any other ideas?

Not sure if this is a good place to start this discussion or not - feel free to move.

[ataylorme]

The Pantheon example-*-composer repositories include bash scripts intentionally, rather than putting them in a Dockerfile, so that folks new to the process can easily see them and learn.

@aaronbauman creating a custom Dockerfile with shared scripts is a good way to go for folks who know how CI works and want to use it in multiple projects.

The Dockerfile FROM syntax to extend an existing image is very useful (this repo uses it).

You could either extend our image or use it as a starting place and remove what you don't need (maybe some Terminus plugins your scripts won't use, clu if you are using the Terminus CLU plugin instead, etc.). The advantage to the latter would be getting the file size down.

Then, move the bash scripts from the individual project repositories to the Dockerfile and maintain them there for all your projects, having your CI config reference them.

I've done this on GitLab and it worked really well. GitLab also has a container registry so the Dockerfile could be cached well, which improved job times in addition to providing a central place to maintain the build/deploy/test scripts.

If you try out BitBucket pipes let us know how it goes.

[aaronbauman]

Thanks for the input. I will try it out and let you know.

[nelsonamaya82]

Hi @aaronbauman, I'm also trying to optimize my CI workflow with a a custom Dockerfile or any other solution, I use Bitbucket too. Did you try out BitBucket pipes to do this? Thanks!

[aaronbauman]

I haven't had time to test this out yet, no.

[nelsonamaya82]

Thanks. I'm testing it but I don't quite understand the difference between a custom Pipe and a custom Docker Image, are there any advantages of using Pipes over Images?

[aaronbauman]

Seems like Pipes are meant to be more modular.
But they appear to be Docker images with some additional decoration, so I guess you could use either to achieve the same results.

[nelsonamaya82]

Great, thanks @aaronbauman.