Proposal RFC: Java Agents and More Layers #455

coollog · 2018-06-27T14:42:39Z

Update: This proposal will probably be retired in favor of the incubating src/main/jib convention. (#489)

Request for comments. Approval upon merge.

Preview at https://github.com/GoogleContainerTools/jib/blob/more-layers-proposal/proposals/java_agents_and_more_layers.md

@linux-china
@saturnism

Fixes #378
Fixes #213 (closes in favor of this)
Regards #403

briandealwis · 2018-06-28T20:33:10Z

proposals/java_agents_and_more_layers.md

+addFile 'another/file', '/another/path/on/image'
+```
+
+All of the files would be added to a single new layer.


Do we need to specify this implementation detail? It might make sense to place larger file trees in separate layers?

Placing in separate layers may be a lot more complicated, since we would need some sort of decider that would deterministically place the files into a certain number of layers. I can remove this as a detail though since it is implementation-level.

briandealwis · 2018-06-28T20:39:47Z

proposals/java_agents_and_more_layers.md

+
+The configuration for adding additional files (including Java agents) would look something like:
+
+*(The exact configuration is to be decided before this PR is merged. Alternative naming for this parameter include: `additionalFiles`, `extraFiles`, and `copyFiles`.)*


Reading these examples, the singular addFile implies that each item is for a single file, and that I'd need to list out each file to copy to a directory. I thought we would allow specifying a directory to have the directory tree to be copied?

In which case we also need to specify the semantics for directories. Perhaps should adopt the rsync source-path semantics?

If the from location is a directory and has a trailing slash then the contents of that directory are copied. Otherwise the contents are copied into a new directory named for the last component of the from location. All intermediate directories are created as required. The destination specifies a directory in the image. The copy fails if the destination is a file.
For example, given source files config/ssh/ssh_config and config/ssh/sshd_config and wish to copy them into /etc/ssh so that the image contains /etc/ssh/ssh_config and /etc/ssh/sshd_config then we can do either of the following:

<from>config/ssh</from><to>/etc</to>

<from>config/ssh/</from><to>/etc/ssh</to>

Yea that sounds like a good way to do it! I'll update this with these semantics for directories. In terms of naming for addFile I was thinking of File as referring to both a directory and a regular file.

Actually, I was just thinking and maybe we should have it be the semantics of Dockerfile COPY, where essentially:

if from is a directory, only the contents are copied, and not the directory itself

if to ends with a trailing slash, it is considered a directory. Otherwise, it is considered a regular file.

if from refers to multiple files (like the contents of a directory), to must end with a trailing slash, and the from files will be added to that directory.

We won't support glob matching though, at least not until someone says they need it.

Using Docker's COPY semantics makes sense.

briandealwis · 2018-06-28T20:44:10Z

proposals/java_agents_and_more_layers.md

+    <from>another/file</from>
+    <to>/another/path/on/image</to>
+  </addFile>
+</addFiles>


I tried playing with some examples, and this felt a bit more concise?

<files> <copy from="path/to/file" to="/path/in/image"/> <copy from="path/to/file" to="/path/in/image"/> </files>

and with groovy

copyIn 'path/to/file', '/path/on/image'

Using copy avoids any confusion with Dockerfile's ADD and its unpacking? One downside to both is that Docker creates a separate layer for each COPY and ADD, which we don't do.

I don't think Maven configuration can support duplicate tags or tag parameters? @loosebazooka ?

Oh jeepers 🤦‍♂️ There does seem to be a way to support relatively arbitrary XML but it's not pretty and I don't think it's warranted by this situation.

yeah that might be a little un-maven-y

briandealwis · 2018-06-28T21:05:36Z

proposals/java_agents_and_more_layers.md

+
+### Alternative (Rejected) Proposal
+
+The alternative proposal was rejected because we deemed that layering should be an implementation detail that should not be exposed to the user.


Maybe call this out with **The alternative proposal was rejected …**?

briandealwis · 2018-06-29T18:11:29Z

proposals/java_agents_and_more_layers.md

@@ -58,7 +58,7 @@ addFile 'path/to/file', '/path/on/image'
 addFile 'another/file', '/another/path/on/image'
 ```

-All of the files would be added to a single new layer.
+*Directories will be handled using [rsync](https://linux.die.net/man/1/rsync) source-path semantics, where, for example, `addFile 'path/to/directory', '/path/on/image'` would put all the files in `path/to/directory` into the directory `/path/on/image/directory`, but `addFile 'path/to/directory', '/path/on/image'` (trailing slash) would put all the files into the directory `/path/on/image`.* 


Second example is missing the trailing slash!

Oops, will add that.

loosebazooka · 2018-07-02T14:55:44Z

I just tested copies, and it does become copy (maven knows how to do this)

<copies>
  <copy>asdf</copy>
</copies>

but I'm not sure if this actually reads well.

coollog · 2018-07-14T05:25:48Z

This proposal will probably be retired in favor of the incubating src/main/jib convention.

TadCordle · 2018-08-21T19:36:30Z

Are there any plans for this proposal, or should we just close this PR?

coollog · 2018-08-22T14:07:34Z

I think this will be good to have as a record in the proposals (and as a reference for the design behind the current solution). I'll touch this up with our current solution (src/main/jib) and move the previous solutions to sections about rejected proposals.

coollog · 2018-08-22T22:04:46Z

Changed the main proposal to regard src/main/jib, moving the previous one to rejected. Also moved this to archives since src/main/jib is already implemented.

briandealwis · 2018-08-23T02:29:59Z

proposals/archives/java_agents_and_more_layers.md

+
+The directory should also be able to be configured to override the `src/main/jib` default.
+
+### Alternative (Rejected) Proposal


Would be good to have a ## Alternative Proposals here as they otherwise fall under the ## Proposal.

coollog added 2 commits June 27, 2018 10:37

Proposal RFC: Java Agents and More Layers

cc0f440

Fixes link.

95e8e49

coollog added the proposal label Jun 27, 2018

coollog requested review from briandealwis, TadCordle, loosebazooka, patflynn and chanseokoh June 27, 2018 14:42

coollog added 2 commits June 27, 2018 14:34

Merge branch 'master' into more-layers-proposal

43afb4e

Merge branch 'master' into more-layers-proposal

b48ca91

briandealwis reviewed Jun 28, 2018

View reviewed changes

coollog added 2 commits June 29, 2018 12:31

Addresses briandealwis comments.

498d069

Merge branch 'master' into more-layers-proposal

df29447

briandealwis reviewed Jun 29, 2018

View reviewed changes

Adds trailing slash.

a557978

coollog added this to the Implement Proposal: Java Agents and More Layers milestone Jun 29, 2018

coollog requested a review from a team June 29, 2018 19:02

Changes to be Dockerfile COPY semantics.

40772db

coollog mentioned this pull request Jul 11, 2018

Builds extra layer with contents of src/main/jib. #562

Merged

googlebot added the cla: yes label Aug 22, 2018

coollog added 2 commits August 22, 2018 17:57

Merge branch 'master' into more-layers-proposal

b645468

Adds src/main/jib and moves to archives.

de36407

briandealwis approved these changes Aug 23, 2018

View reviewed changes

Puts alternative proposals under H2.

250d978

coollog merged commit 940684b into master Aug 23, 2018

coollog deleted the more-layers-proposal branch August 23, 2018 14:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Proposal RFC: Java Agents and More Layers #455

Proposal RFC: Java Agents and More Layers #455

coollog commented Jun 27, 2018 •

edited

Loading

briandealwis Jun 28, 2018

coollog Jun 28, 2018

briandealwis Jun 28, 2018

coollog Jun 28, 2018

coollog Jun 29, 2018

briandealwis Jun 29, 2018

briandealwis Jun 28, 2018

coollog Jun 29, 2018

briandealwis Jun 29, 2018

loosebazooka Jun 29, 2018

briandealwis Jun 28, 2018

briandealwis Jun 29, 2018

coollog Jun 29, 2018

loosebazooka commented Jul 2, 2018

coollog commented Jul 14, 2018

TadCordle commented Aug 21, 2018

coollog commented Aug 22, 2018

coollog commented Aug 22, 2018

briandealwis Aug 23, 2018


		The configuration for adding additional files (including Java agents) would look something like:

		(The exact configuration is to be decided before this PR is merged. Alternative naming for this parameter include: `additionalFiles`, `extraFiles`, and `copyFiles`.)


		### Alternative (Rejected) Proposal

		The alternative proposal was rejected because we deemed that layering should be an implementation detail that should not be exposed to the user.


		The directory should also be able to be configured to override the `src/main/jib` default.

		### Alternative (Rejected) Proposal

Proposal RFC: Java Agents and More Layers #455

Proposal RFC: Java Agents and More Layers #455

Conversation

coollog commented Jun 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

loosebazooka commented Jul 2, 2018

coollog commented Jul 14, 2018

TadCordle commented Aug 21, 2018

coollog commented Aug 22, 2018

coollog commented Aug 22, 2018

Choose a reason for hiding this comment

coollog commented Jun 27, 2018 •

edited

Loading