Enable running custom images in acs-engine #2513

0xmichalis · 2018-03-22T18:27:02Z

@khenidak this is the fix we talked yesterday about enabling us to run custom images in acs-engine. Any pointers on adding an e2e test that validates generation would be appreciated.

cc @jim-minter @pweil-

jackfrancis · 2018-03-23T20:57:54Z

@Kargakis I have rebased this locally after cloning your fork. Permission to force push? ( if you have local changes that haven't been pushed up, say no :) )

0xmichalis · 2018-03-23T21:19:20Z

@Kargakis I have rebased this locally after cloning your fork. Permission to force push? ( if you have local changes that haven't been pushed up, say no :) )

@jackfrancis I rebased my branch if that's all you wanted. My editor messes up the satori import.. Other than that, I have no local changes - feel free to do what you need to do.

jackfrancis · 2018-03-23T21:20:56Z

@Kargakis I'll force push w/ rebase from recent master, stand by

0xmichalis · 2018-03-27T09:03:50Z

I just verified that ARM does not allow data disks when an image resource ID is provided.

Cannot specify user image overrides for a disk already defined in the specified image reference.

I will update the PR shortly to exclude them.

@jim-minter @pschiffe after submitting this PR I realized it's not only going to be useful for playing around with acs-engine but will be likely how we do PR testing in OpenShift.

seanknox · 2018-03-27T16:04:54Z

Thanks for this @Kargakis, this will help with a use case I need, too!

seanknox · 2018-03-27T20:41:31Z

@Kargakis could you include an example of how to use this, either an API model (see examples/) and/or update documentation?

pweil-

tests and improved error lgtm. I think we're just needing an examples/ file based on current feedback.

0xmichalis · 2018-03-28T14:51:49Z

A question to folks, the example I am adding is going to work for generate but not for deploy. Is that OK? I would guess so.

0xmichalis · 2018-03-28T14:56:29Z

A question to folks, the example I am adding is going to work for generate but not for deploy. Is that OK? I would guess so.

Seems that some examples don't work ootb even for generate. Anyway, example added. PTAL

jackfrancis · 2018-03-28T21:22:12Z

@Kargakis I assume we intend for these image references to be available from the public internet? Are you able to publish a known-working prototype that we can test this cluster configuration workflow against?

jackfrancis · 2018-03-28T21:24:26Z

examples/custom-image.json

+      "ssh": {
+        "publicKeys": [
+          {
+            "keyData": "mykeyData"


nit: let's keep these boilerplate non-vals empty strings in the example api model (ditto dnsPrefix above and clientId/secret below)

jackfrancis · 2018-03-28T21:26:30Z

pkg/api/vlabs/types.go

@@ -311,6 +311,8 @@ type MasterProfile struct {
 	Extensions               []Extension       `json:"extensions"`
 	Distro                   Distro            `json:"distro,omitempty"`
 	KubernetesConfig         *KubernetesConfig `json:"kubernetesConfig,omitempty"`
+	ImageName                string            `json:"imageName,omitempty"`


Did we consider delivering a struct to represent these properties instead of this flat structure? Instinctually, the advantage of that is the ability to scale out in the future without expanding the overhead of the parent struct, and easier re-use among typed API versions.

jackfrancis · 2018-03-28T21:26:55Z

Added some remarks, this looks close, eager to test this soon! Thanks!

0xmichalis · 2018-03-29T07:49:10Z

@Kargakis I assume we intend for these image references to be available from the public internet? Are you able to publish a known-working prototype that we can test this cluster configuration workflow against?

If an image is public then can't we just use the pre-existing fields? I don't think I can publish an image atm (@kwoodson is going through this process right now, he can share how long does it take). This is important for unpublished images I think and will be likely used by our CI process since we are going to be building images and deploying them via acs-engine on every PR that lands in our repo.

kwoodson · 2018-03-29T13:42:18Z

If an image is public then can't we just use the pre-existing fields? I don't think I can publish an image atm (@kwoodson is going through this process right now, he can share how long does it take). This is important for unpublished images I think and will be likely used by our CI process since we are going to be building images and deploying them via acs-engine on every PR that lands in our repo.

@Kargakis @jackfrancis
The process to build an image is pretty straight forward and takes < 15 minutes. The publishing which allows separate subscriptions to try the image takes much longer. Inside of the cloud partner portal there is a form which requires information regarding the image. OS type, description, references to the vhds from a storage blob, and more. Once all of this information is set correctly, one can publish an image. The current MS process that is followed takes anywhere from 2-7 days. This involves testing the image, propagating the image to all regions, some certification process, etc.

We will be producing an origin image in the very near future. Hope this helps.

jackfrancis · 2018-03-30T16:50:55Z

Thanks @Kargakis! Running E2E now.

jackfrancis · 2018-03-30T20:00:48Z

This generally LGTM and back-compat tests are showing green. Is the inclusion of changes to agentPoolOnlyApi intentional at this stage?

0xmichalis · 2018-03-31T01:31:44Z

Initially I included the changes to it for consistency. Is it used only on the hosted master deployment mode? I feel we are going to need it eventually for testing but can leave it out for now if desired.

jackfrancis · 2018-04-02T15:56:14Z

@Kargakis Let's leave it out for now if it's not immediately operable, because its inclusion will (reasonably) engage an add'l review/validation process. Make sense to you?

0xmichalis · 2018-04-03T09:46:44Z

@jackfrancis fine by me, done+squashed.

jackfrancis · 2018-04-04T18:27:32Z

@Kargakis Added basic docs, feel free to validate. Will run E2E and then lgtm

jim-minter · 2018-04-04T18:33:16Z

@jackfrancis docs look fine to me, thanks for adding these. Assuming tests pass, please merge.

jackfrancis

lgtm, thanks @Kargakis!

ghost assigned jackfrancis Mar 23, 2018

ghost added the in progress label Mar 23, 2018

pweil- reviewed Mar 28, 2018

View reviewed changes

jackfrancis reviewed Mar 28, 2018

View reviewed changes

Enable running custom images in acs-engine

efbee9f

Added basic docs reference to ImageReference properties

0819b6b

jackfrancis approved these changes Apr 4, 2018

View reviewed changes

jackfrancis merged commit d5ec309 into Azure:master Apr 4, 2018

ghost removed the in progress label Apr 4, 2018

0xmichalis deleted the enable-using-private-images branch April 5, 2018 07:36

0xmichalis mentioned this pull request Apr 6, 2018

Add missing image variables #2613

Merged

seanknox mentioned this pull request Apr 16, 2018

Feature req: be able to use custom VHD images as source image #2698

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable running custom images in acs-engine #2513

Enable running custom images in acs-engine #2513

0xmichalis commented Mar 22, 2018

jackfrancis commented Mar 23, 2018

0xmichalis commented Mar 23, 2018

jackfrancis commented Mar 23, 2018

0xmichalis commented Mar 27, 2018

seanknox commented Mar 27, 2018

seanknox commented Mar 27, 2018

pweil- left a comment

0xmichalis commented Mar 28, 2018

0xmichalis commented Mar 28, 2018

jackfrancis commented Mar 28, 2018

jackfrancis Mar 28, 2018

0xmichalis Mar 29, 2018

jackfrancis Mar 28, 2018

0xmichalis Mar 29, 2018

jackfrancis commented Mar 28, 2018

0xmichalis commented Mar 29, 2018

kwoodson commented Mar 29, 2018

jackfrancis commented Mar 30, 2018

jackfrancis commented Mar 30, 2018

0xmichalis commented Mar 31, 2018

jackfrancis commented Apr 2, 2018

0xmichalis commented Apr 3, 2018

jackfrancis commented Apr 4, 2018

jim-minter commented Apr 4, 2018

jackfrancis left a comment

Enable running custom images in acs-engine #2513

Enable running custom images in acs-engine #2513

Conversation

0xmichalis commented Mar 22, 2018

jackfrancis commented Mar 23, 2018

0xmichalis commented Mar 23, 2018

jackfrancis commented Mar 23, 2018

0xmichalis commented Mar 27, 2018

seanknox commented Mar 27, 2018

seanknox commented Mar 27, 2018

pweil- left a comment

Choose a reason for hiding this comment

0xmichalis commented Mar 28, 2018

0xmichalis commented Mar 28, 2018

jackfrancis commented Mar 28, 2018

jackfrancis Mar 28, 2018

Choose a reason for hiding this comment

0xmichalis Mar 29, 2018

Choose a reason for hiding this comment

jackfrancis Mar 28, 2018

Choose a reason for hiding this comment

0xmichalis Mar 29, 2018

Choose a reason for hiding this comment

jackfrancis commented Mar 28, 2018

0xmichalis commented Mar 29, 2018

kwoodson commented Mar 29, 2018

jackfrancis commented Mar 30, 2018

jackfrancis commented Mar 30, 2018

0xmichalis commented Mar 31, 2018

jackfrancis commented Apr 2, 2018

0xmichalis commented Apr 3, 2018

jackfrancis commented Apr 4, 2018

jim-minter commented Apr 4, 2018

jackfrancis left a comment

Choose a reason for hiding this comment