-
Notifications
You must be signed in to change notification settings - Fork 47
Shortcomings with current answers/params structure towards implementing nested Nulecule application #187
Comments
I have been struggling with something that I think is vague about the nulecule usage model. On one hand I want to define a nulecule as metadata to help consume a microservice container image. There I define my unresolved variables and give them names that will map them into my artifacts. On the other hand I want to create an application that uses a nulecule as input. Here I want several things which you described, and perhaps one that was missed.
Given those two not-really-identical cases (provider and consumer) I'm trying to come up with a syntax that would represent all these operations well. It almost feels like you need two sections:
This would lead to two "parameters" sections, one declaration for each consumed graph element and one assignment section for the nulecule as a whole. |
OK, I could edit that and avoid looking silly, but I'll re-comment instead. I guess the second "parameters" section I mention, the definition/assignment only needs to be in the global scope of the answers file. The only reason to have entries in the graph element sections of the answers file is if I need to map two mismatched parameter names from different consumed nulecules to a single provided value. |
It seems like something as simple as: {
"params": [
{
"name": "some_param",
"value": "some_value"
}
],
"graph": [
{
"name": "some-external-app",
"source": "docker://some/external-app"
}
]
} could be made to have Would that resolve the problem? UPDATE: It would not solve the problem for {
"params": [
{
"name": "some_param",
"value": "some_value"
}
],
"graph": [
{
"name": "some-external-app",
"source": "docker://some/external-app",
"params": [
{
"name": "some_param",
"value": "a_completely_different_value"
}
]
}
]
} |
@kanarip it could.. but having it be more explicit by defining the params as well as the args (like in [3]) would be better for knowing which params apply to which nulecules rather than just having them all inherit all params somehow. In this model the Nulecule is kind of like a function call. The nulecule defines it's inputs (params) and then makes other function calls to other nulecules and provides them their defined inputs as args. |
Right, I had not seen that example but it makes sense. I reckon This way, |
I think you've just identified another dimension which I've been thinking about. Ideally, the list of parameters (of all kinds) would be provided from the container image, perhaps in the form of an included Dockerfile, but ideally from some other component of the image. [1]. The Using labels, the container image producer can embed metadata about the run-time requirements of the container: environment variables, CLI arguments, external data volumes (location and permissions) [3]. This allows the definition of these variables to be inferred from the container image rather than declared directly in the Nulecule file [4]. Note that the This presents a problem for the image consumers when composing services. There is no way for the consumer to ensure that a parameter used by two images (database service and client) will have the same variable name in the two images. To resolve this, the Nulecule will need to provide a way to map variables with different names from different container images to a single value (DB username, password for example) In the top level of the Assume a DB service container named
And a client container named
The container images provide these environment variable names by defining a
and
The Nulecule then needs two things:
The Nulecule designer will likely want to provide a unified name for prompting or for lookup in an Since variables come from two different images and because they may have the same name, the mapping in part 1 must use the name of the container image to disambiguate in that case. The parameters section would reside in the top level of the Nulecule file. It would list all of the inputs a user may need to provide to the containers and a mapping from the Nulecule variable names to the container ones.
This would also allow the artifact writers to use the Nulecule parameter names in the artifacts rather than the container image variable names. NOTE: It might be desirable for the mapping to be provided in the [1] I think that LABEL directives are proper layers and so independent of whether a Dockerfile is actually included in the image. |
First of all, it too bad we already had this figured out after the meeting in Brno:( @markllama I am missing context in your example - are those global parameters? If yes, does Also relying on labels in this case is a bit dangerous as those can change independently from Nulecule metadata, although the same can happen for the code inside the image, so never mind:) How does this mapping work with nested components? I.e. |
I was personally excited about the notion of having unresolved variables in Nulecule, which get resolved during the various stages of deploying the Nulecule, for use in the subsequent deployment process. We could also introspect the Nulecule to find the dependencies across components and schedule deployment in a proper order, so that a component B depending on a value x resolved during deploying component A would be deployed after A has been deployed. This'd help Nulecule to perform some basic orchestration as well, useful for providers like Docker, Docker Swam (in the future), not that useful for kubernetes and openshift. However, it turned out that orchestration is not in the scope of Nulecule. Nulecule is just aimed towards packaging multi container applications and delivering them.
I am doing the needed mapping when assigning args to the Nulecule components, e.g., here.
I would like to keep the implementation of Nulecule separate from possible delivery mechanisms: docker or other container images, tarball, etc. |
@rtnpro thanks, It looks like the work you're doing to refactor the paremeters/args structures answers some of what I would like. You're doing the mapping "in reverse" so to speak, but with the same result. Regardless of the implementation of the containers, it would seem like we should insist that the container developer tell the user how they are meant to be used. This is the point of adding labels ala grashopper and OpenShift continer image metadata. It would seem that the only information in the nulecule would be that which is not provided by the container images. This is really two issues:
I guess I'll wait on your changes to see how #1 works out. #2 remains unresolved, but it's out of scope right now. |
Hey folks, I have implemented an intial POC for the proposed refactor of Nulecule params. You can try it in the following way: Pull changes in atomicapp
Pull refactored sentry nulecule application in nulecule-library and run it
Give it a shot and share your feedback. |
Problem statement
The way it is now, Nulecule apps are not that reusable. It's easier for us
to add dependencies internally, rather than, add external dependency. This
can be seen in most of our examples in [[1]]. There are a few shortcomings
in having dependency on external Nulecule applications.
Namespace conflict
When we consume external Nulecule applications in our Nulecule application,
we have no control over the component names chosen in the external
applications. For example, if our app contains two external Nulecule
applications: A, B, and both A and B contains a component named
db, then params for both these different components with same name
db would get merged into the same section
[db]
inanswers.conf
.As a consumer, of a Nulecule application, why should I bother with the
internal components of the consumed Nulecule application?
Redundant params in ANSWERS
Let's take the example of a our nulecule wordpress[[2]] application. The sample
answers
file generated during installing this app looks like:We see that params like
db_user
,db_pass
,db_name
, etc. getsduplicated in multiple sections. Currently, there's no model in place params
inheritance, which seems like a sensible thing to do.
Brainstorming
The way we currently load a Nulecule application is by doing a DFS
(Depth First Search) traversal of the Nulecule application tree, similar to
nested function calls. Each node, or component, in our Nulecule application
tree, is like a node in a tree data structure. Each node (except for root)
has a parent and may have some children.
However, we do not apply the same concept when dealing with params for our
Nulecule app. The current parameter model is a flat one, rather than
a hierarchical one, devoid of any notion of inheritance.
Proposed solution
So, why don't we translate nested nulecule applications to something like
nested function calls. The root/master nulecule application defines
params: mandatory and optional (with defaults), that it needs to run or
be consumed by another nulecule application. These params are defined at the
root level of the Nulecule SPEC data, rather than at the component level.
Now, we use these defined params, as required, at the component level, to
supply the components with necessary data they need to know.
An example will make it easier to understand the solution we are proposing.
We refactor the wordpress and mariadb nulecule application as an example
to demo our solution[[3]].
[1] https://github.com/projectatomic/nulecule-library
[2] https://github.com/projectatomic/nulecule-library/tree/master/wordpress-centos7-atomicapp
[3] projectatomic/nulecule-library@master...rtnpro:refactor_answers
The text was updated successfully, but these errors were encountered: