dynamic list of stacks

[elliottmurray]

Hey mate,

We have a requirement to run the near same stack many times but with a slightly different config. That is a dynamic list really. I want to pass in and have a stack run a set config (say dev) but then override with my runtime params and most importantly give it an aws cfn name rather than a static one in bora.yml. So something like

bora apply my-stack-dev --cfn-stack-id=my-stack-dev-1 --params 'example=1'

Looked at the code - think it is possible if I treat cfn-stack-id as a new type of param and in stack.rb would have to do some work so that:

@cfn_stack_name = stack_config['stack_name'] || @stack_name

can also be set by a 3rd option (which takes precedence)

But do you want to take it in this direction as it kinda adds a new concept and is a bit of a largish change. Have to deal with all the other tasks, look at resolvers etc

Otherwise I'll use output the and fall back to aws tooling.

[ampedandwired]

I've been thinking about doing this as well, but I haven't landed on a solution that I really like yet. The idea is to have a bora stack definition that acts as kind of a "template" that you can use to stamp out multiple CFN stacks with very similar configuration.

There's a stack_name parameter in bora.yml that you can use to override the name of the stack that gets created in CloudFormation. So maybe the solution is as simple as allowing this to be set as a command line option? I'm already doing something like this for "region".

Anyway, I'm planning a hacking session on Bora this week, I'll have a play with a few things and see how they look.

[ampedandwired]

... which is of course pretty much what you were suggesting :)

Another option might be to allow a "default" stack config that gets used whenever there is no matching stack found. However I'm not sure I like this approach, it's a bit more non-obvious and limiting (what if you want many "default" stack configs?).

[ampedandwired]

Looking at the code I think it should be pretty easy to do it in a similar way to how we're handing region overrides from the command line. This wouldn't be available for the rake tasks though, but TBH I'm thinking of deprecating the rake functionality, as it's pretty limiting and you can always invoke the bora process from Rake if you really need to.

What sort of an impact did you envisage on the resolvers? As far as I can see they wouldn't need to change?

[elliottmurray]

I wasn't sure if the resolvers just got applied to params? I hadn't followed it all the way through. I had (not MVP!) an idea around being able to pass a param and interpolate that into a cfn_stack_name in bora. You do some of that with creds and outputs I think? So in my case I'm identifying stacks by a number id so if I pass that as a param I can use that number in my cfndsl and in my cfn stack name.
cfn_stack_name: dev-stack-${id}

But short term I think yeah some way of having another named param like region to set the cloudformation stack name over the bora name would be good enough. Just need to make sure that change is propogated to status, delete and outputs I think? The bora stack name gets used a lot so wasn't sure how to approach changing that.

I've stopped using the rake tasks - the Thor command is much nicer :)

[ampedandwired]

Resolvers are just applied to params at present. Applying them to stack_name would be an interesting idea, although it does make it easy to shoot yourself in the foot by forgetting to supply the parameter.

[ampedandwired]

I'm tossing up between the various ways to implement this. Interested in your thoughts from a user perspective. To recap:

Option 1 is to introduce a --stack-name parameter that overrides the stack_name defined in the yaml. This would allow you to instantiate multiple instances of any bora stack by giving them unique names. How you manage those names is up to you. The downside is there's no real validation on whether you've supplied the stack_name param or not, so it's easy to forget and do the wrong thing.

Option 2 is to allow parameter interpolation in the stack_name field in the yaml, so you can say stack_name: dev-stack-${id}. Has the advantage that you can validate that the parameter is supplied and make it mandatory, thus reducing the chance of shooting yourself in the foot. Possibly a bit cleaner than introducing another command line param. Also serves as a form of documentation that this stack is intended to be instantiated multiple times.

Option 3 is to introduce a first-class top-level abstraction for this. I want to use the name "stack template", it's too confusing with the existing "template" terminology, but will go with it just for this discussion. So there would be syntax in bora.yml to declare a stack as a "template" and command line options to pass in the "stack instance" name or ID. Downside is more concepts and more complexity. Upside is a possibly a cleaner way to work with these concepts.

Overall I'm probably leaning towards option 2 as a good balance between the various considerations.

Thoughts?

[elliottmurray]

I think 1 and 3 are both valid but 2 would solve my particular problem the best :)

[ampedandwired]

Cool, I'll give this a crack. It will lay the groundwork for another feature I've been wanting to add too - being able to refer to other params in property substitutions

[ampedandwired]

After looking at this more closely, I decided that option 2 was actually not the best way to go about this. I think conflating params with stack name is confusing. It would require pretty much every command to accept a --params flag, for example bora outputs my-stack-dev --params id=1, which I think is not obvious at all.

Thinking on it, I think option 3 is really what we need - I've opened a new ticket to track that in #10. In the meantime, I've added a new flag --cfn-stack-name that allows you to specify the CloudFormation stack name on the command line. Hopefully this will at least unblock you while I think about more explicit support for this use case.

[ampedandwired]

Released as 1.2.0

[ampedandwired]

Sorry for the breaking change... I'm changing the short version of this param from -c to -n. Will release that as 1.3 soon...

[ampedandwired]

Released 1.3. Use that.