Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updates to the custom install draft #13468

Merged
merged 1 commit into from
Jan 31, 2019
Merged

updates to the custom install draft #13468

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion _topic_map.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Topics:
File: architecture
---
Name: Installing clusters on AWS
Dir: installation-AWS
Dir: installing-aws
Distros: openshift-origin, openshift-enterprise
Topics:
- Name: Configuring an AWS account
Expand Down
0 installation-AWS/images → installing-aws/images
View file
Open in desktop
File renamed without changes.
4 changes: 2 additions & 2 deletions installation-AWS/installing-aws-account.adoc → installing-aws/installing-aws-account.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@ include::modules/installation-aws-iam-user.adoc[leveloffset=+1]
.Next steps

* Install a {product-title} cluster. You can
xref:../installing-AWS/installing-customizations-cloud.adoc#installing-customizations-cloud[install a customized cluster]
or xref:../installing-AWS/installing-quickly-cloud.adoc#installing-quickly-cloud[quickly install a cluster]
xref:../installing-aws/installing-customizations-cloud.adoc#installing-customizations-cloud[install a customized cluster]
or xref:../installing-aws/installing-quickly-cloud.adoc#installing-quickly-cloud[quickly install a cluster]
with default options.
16 changes: 3 additions & 13 deletions ...-AWS/installing-customizations-cloud.adoc → ...-aws/installing-customizations-cloud.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,32 +11,22 @@ cluster on Amazon Web Services (AWS).

.Prerequisites

* xref:../installing-AWS/installing-aws-account.adoc#installing-aws-account[Configure an AWS account]
* xref:../installing-aws/installing-aws-account.adoc#installing-aws-account[Configure an AWS account]
to host the cluster.

include::modules/installation-overview.adoc[leveloffset=+1]

include::modules/installation-clouds.adoc[leveloffset=+1]

include::modules/installation-about-custom.adoc[leveloffset=+1]

include::modules/installation-preparing-custom.adoc[leveloffset=+1]

include::modules/installation-configuration-parameters.adoc[leveloffset=+2]

include::modules/installation-render-options.adoc[leveloffset=+2]

include::modules/installation-provide-credentials.adoc[leveloffset=+1]

include::modules/installation-obtaining-installer.adoc[leveloffset=+1]

include::modules/installation-initializing.adoc[leveloffset=+1]

include::modules/installation-rendering.adoc[leveloffset=+1]

include::modules/installation-customizing.adoc[leveloffset=+1]
include::modules/installation-aws-config-yaml.adoc[leveloffset=+1]

include::modules/installation-preparing-assets.adoc[leveloffset=+1]
include::modules/installation-configuration-parameters.adoc[leveloffset=+2]

include::modules/installation-launching-installer.adoc[leveloffset=+1]

Expand Down
4 changes: 3 additions & 1 deletion ...llation-AWS/installing-quickly-cloud.adoc → installing-aws/installing-quickly-cloud.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Amazon Web Services (AWS) that uses the default configuration options.

.Prerequisites

* xref:../installing-AWS/installing-aws-account.adoc#installing-aws-account[Configure an AWS account]
* xref:../installing-aws/installing-aws-account.adoc#installing-aws-account[Configure an AWS account]
to host the cluster.

include::modules/installation-overview.adoc[leveloffset=+1]
Expand All @@ -23,3 +23,5 @@ include::modules/installation-provide-credentials.adoc[leveloffset=+1]
include::modules/installation-obtaining-installer.adoc[leveloffset=+1]

include::modules/installation-launching-installer.adoc[leveloffset=+1]

include::modules/installation-default-aws-config-yaml.adoc[leveloffset=+1]
0 installation-AWS/modules → installing-aws/modules
View file
Open in desktop
File renamed without changes.
0 ...llation-AWS/uninstalling-cluster-aws.adoc → installing-aws/uninstalling-cluster-aws.adoc
View file
Open in desktop
File renamed without changes.
0 ...ation-BYOH/installing-existing-hosts.adoc → ...lling-byoh/installing-existing-hosts.adoc
View file
Open in desktop
File renamed without changes.
0 ...ion-BYOH/uninstalling-existing-hosts.adoc → ...ing-byoh/uninstalling-existing-hosts.adoc
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion modules/glusterfs-hardware-requirements.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Module included in the following assemblies:
//
// * installing-BYOH/installing-existing-hosts.adoc
// * installing-byoh/installing-existing-hosts.adoc

[id='glusterfs-hardware-requirements-{context}']
= {gluster} hardware requirements
Expand Down
135 changes: 46 additions & 89 deletions modules/installation-about-custom.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,95 +1,52 @@
// Module included in the following assemblies:
//
// * installing-AWS/installing-customizations-cloud.adoc
// * orphaned

[id='installation-about-custom-{context}']
= About the custom installation

The custom installation is divided into four phases:

* Initialization, where you create and customize the installation configuration
file
* Rendering, where you provide YAML files that describe further customizations
* Preparing, where the system creates assets based on the rendered data
* Launching, where you create the cluster in your cloud.

During initialization, you generate and customize a cluster configuration file that
defines high level configuration and topology and specifies your cloud provider
credentials. This configuration file contains a limited set of parameters that
most clusters require.

During rendering, you do a thing with a bunch of yaml files that allows you to
provide more customization to your cluster. You can specify ...

During the preparing phase, the assets that you render...

Finally, you launch the installation. The installation program first creates the
infrastructure for the cluster, such as the load balancers, security groups, and
VPCs, and then creates an initial configuration on a bootstrap master node.
The bootstrap master then completes the configuration of the cluster by
self-configuring and loading other components. By default, all the cluster nodes
run Red Hat CoreOS as the operating system.

After the installation and configuration process is complete, instructions for
connecting to the cluster displays.

[discrete]
== Key concepts

Before you complete a customized installation, review the following key concepts.

[discrete]
=== Targets

The {product-title} installer operates on the notion of creating and destroying
targets. Similar to other tools that operate on a graph of dependencies, like
`make` and `systemd`, each target represents a subset of the dependencies in the
graph. The main target in the installation program creates a cluster, but the other targets
allow the user to interrupt this process and use or modify the intermediate
artifacts, such as the the Kubernetes manifests that are installed into the
cluster. Only the immediate dependencies of a target are written to disk by the
installer, but you can run the installation program multiple times to update more
dependencies.

The installation program can create the following targets:

`install-config`::
The installation configuration contains the main parameters
for the installation process. This configuration provides you with more options
than the interactive prompts and comes pre-populated with default values.
`manifest-templates`:: These are the unrendered Kubernetes manifest templates
that `manifests` target uses. This target is unstable.
`manifests`::
You can generate the Kubernetes manifests that will be installed on the cluster.
This target is unstable.
`ignition-configs`::
These files contain the Ignition configurations for the bootstrap, master, and
worker machines.
`cluster`::
This target provisions the cluster and its associated infrastructure.

The following targets can be destroyed by the installation program:

`cluster`::
This destroys the created cluster and its associated infrastructure.
`bootstrap`::
This destroys the bootstrap infrastructure.

[discrete]
=== Multiple invocations

Because you must run the installation program to generate the assets that you need to
customize your installation, you can run the installation program multiple times. The
installation state is stored in a hidden file in the asset directory which
contains all of the intermediate artifacts. Storing the artifacts in this way
lets you pause installation and modify the intermediate artifacts.

For example, if you wanted to create a different number of worker machines, you
must first run the installation program with the `install-config` target with the
`openshift-install create install-config` command. After you provide the required
parameters to the interactive installer, it writes the installation
configuration into the target directory. You can then modify the installation
configuration and run the installation program with the `cluster` target with the
`openshift-install create cluster` command. The installation program reads the install
configuration from disk, removes it from the target directory, and creates a
cluster from the provided configuration.
You can use the {product-title} installation program to customize four levels
kalexand-rh marked this conversation as resolved.
Show resolved Hide resolved
of the program:

* {product-title} itself
* The cluster platform
* Kubernetes
* The cluster operating system

Changes to {product-title} and its platform are managed and supported, but
changes to Kubernetes and the cluster operating system currently are not. If
you customize unsupported levels program levels, future installation and
upgrades might fail.

When you select values for the prompts that the installation program presents,
you customize {product-title}. You can further modify the cluster platform
by modifying the `install-config.yaml` file that the installation program
uses to deploy your cluster. In this file, you can make changes like setting the
number of machines that the control plane uses, the type of virtual machine
that the cluster deploys, or the CIDR range for the Kubernetes service network.

It is possible, but not supported, to modify the Kubernetes objects that are injected into the cluster.
A common modification is additional manifests in the initial installation.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of this will probably be supported, see openshift/installer#1144. And the Kubernetes objects that we really care about should be managed (possibly indirectly) by the cluster-version operator. What is unsupported is turning off the cluster-version operator or otherwise overriding it. Pushing Kubernetes objects that do not fall under the CVO umbrella should be fine (e.g. creating and populating user-defined projects).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wking, I'll follow that PR and make more updates when it's merged.

Is there a good way for users to identify which Kubernetes objects are safe to change? I'm happy to add a section about modifying them back in, but only if we can specifically tell them how to confirm that what they're doing will be supported. (Product docs can discuss only supported scenarios.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a good way for users to identify which Kubernetes objects are safe to change?

You can find out what the CVO manages by inspecting the update payload, but that doesn't cover indirect dependencies (e.g. objects pushed by CVO-managed operators). And it looks like Kubernetes considered and then rejected recording creator information. As long as they stick to namespaces/projects they create, they'll be fine. For namespaces we create and cluster-scoped objects, I'm not sure what an easy way would be to check whether a change would be supported short of making the edit and seeing if it gets stomped ;). Maybe @abhinavdahiya or @smarterclayton have some more elegant ideas.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it looks like Kubernetes considered and then rejected recording creator information.

Although creators can set ownerReference explicitly if they need to. Maybe we just need to push more operators to do that (if they aren't already).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jianlinliu, are you testing any modifications to Kubernetes manifests during installation?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we had some testing to modify Kubernetes manifests for customized install until some discussion in upstream repo - https://github.com/openshift/installer/pull/1088/files#r250505048. Since that, we lower the testing of any modifications to Kubernetes manifests, and any issue found during testing is not treated as a blocker issue any more. Because QE also thought modification via manifest and ignition is not officially supported.
I agree with you, if we support it, we have to tell user which Kubernetes objects are safe to change, or a better approach is to prompt the modification to install-config level, we treat everything in install-config is officially supported to allow modification.
@qinpingli pls follow up this discussion, make sure what we tested is on the same page like what we published on our official doc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jianlinliu! @qinpingli, please check during the E2E tests and follow up. Per @vikram-redhat, we're merging frequently instead of holding docs for 2/22.
@sferich888, are we planning to support modifying Kubernetes manifests during installation at 4.0 GA?

No validation is available to confirm the validity of any modifications that
you make to these manifests, so if you modify these objects, you might render
your cluster non-functional.
[IMPORTANT]
====
Modifying the Kubernetes objects is not supported.
====

Similarly it is possible, but not supported, to modify the
Ignition Configs for the bootstrap and other machines. No validation is
available to confirm the validity of any modifications that
you make to these Ignition Configs, so if you modify these objects, you might render
your cluster non-functional.

[IMPORTANT]
====
Modifying the Ignition Configs is not supported.
====

To complete a custom installation, you use the installation program to generate
the installation files and then customize them.
The installation status is stored in a hidden
file in the asset directory and contains all of the installation files.
56 changes: 56 additions & 0 deletions modules/installation-aws-config-yaml.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Module included in the following assemblies:
//
// * installing-aws/installing-customizations-cloud.adoc

[id='installation-aws-config-yaml-{context}']
= Customized `install-config.yaml` file for AWS

You can customize the `install-config.yaml` file to specify more details about
your {product-title} cluster's platform or modify the values of the required
parameters.

[source,yaml]
----
apiVersion: v1beta1
baseDomain: example.com <1>
machines:
- name: master <2>
platform:
aws:
zones:
- us-west-2a
- us-west-2b
replicas: 3 <2>
- name: worker <2>
platform:
aws:
iamRoleName: elastictranscoder-access
rootVolume:
iops: 4000
size: 500
type: io1
type: c5.9xlarge
zones:
- us-west-2c
replicas: 5 <2>
metadata:
name: test-cluster <1>
networking: <2>
clusterNetworks:
- cidr: 10.128.0.0/14
hostSubnetLength: 9
machineCIDR: 10.0.0.0/16
serviceCIDR: 172.30.0.0/16
type: OpenshiftSDN
platform:
aws:
region: us-west-2 <1>
userTags:
adminContact: jdoe
costCenter: 7536
pullSecret: '{"auths": ...}' <1>
sshKey: ssh-ed25519 AAAA... <1>
----
<1> Required. The installation program prompts you for this value.
<2> If you do not provide these parameters and values, the installation program
provides the default value.
4 changes: 2 additions & 2 deletions modules/installation-aws-iam-user.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
// Module included in the following assemblies:
//
// * installing-AWS/installing-aws-account.adoc
// * installing-aws/installing-aws-account.adoc

[id='installation-aws-iam-user-{context}']
[id='installing-aws-iam-user-{context}']
= Creating an IAM user

Each Amazon Web Services (AWS) account contains a root user account that is
Expand Down
4 changes: 2 additions & 2 deletions modules/installation-aws-limits.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
// Module included in the following assemblies:
//
// * installing-AWS/installing-aws-account.adoc
// * installing-aws/installing-aws-account.adoc

[id='installation-aws-limits-{context}']
[id='installing-aws-limits-{context}']
= AWS account limits

The {product-title} cluster uses a number of Amazon Web Services (AWS)
Expand Down
4 changes: 2 additions & 2 deletions modules/installation-aws-route53.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
// Module included in the following assemblies:
//
// * installing-AWS/installing-aws-account.adoc
// * installing-aws/installing-aws-account.adoc

[id='installation-aws-route53-{context}']
[id='installing-aws-route53-{context}']
= Configuring Route53

To install {product-title}, the Amazon Web Services (AWS) account you use must
Expand Down
2 changes: 1 addition & 1 deletion modules/installation-base-packages.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Module included in the following assemblies:
//
// * installing-BYOH/installing-existing-hosts.adoc
// * installing-byoh/installing-existing-hosts.adoc

[id='installing-base-packages-{context}']
= Installing base packages
Expand Down
2 changes: 1 addition & 1 deletion modules/installation-byo-cloud-requirements.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Module included in the following assemblies:
//
// * installing-BYOH/installing-existing-hosts.adoc
// * installing-byoh/installing-existing-hosts.adoc

[id='installation-BYO-cloud-requirements-{context}']
= Cloud requirements for BYO installations
Expand Down
2 changes: 1 addition & 1 deletion modules/installation-byo-system-requirements.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// Module included in the following assemblies:
//
// * installing-BYOH/installing-existing-hosts.adoc
// * installing-byoh/installing-existing-hosts.adoc

[id='installation-byo-system-requirements-{context}']
= System requirements for installing on BYO
Expand Down
24 changes: 17 additions & 7 deletions modules/installation-clouds.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Module included in the following assemblies:
//
// * installing-AWS/installing-quickly-cloud.adoc
// * installing-AWS/installing-customizations-cloud.adoc
// * installing-aws/installing-quickly-cloud.adoc
// * installing-aws/installing-customizations-cloud.adoc

[id='cloud-installations-{context}']
= {product-title} clusters on Installer Provisioned Infrastructure
Expand All @@ -13,14 +13,24 @@ Web Services (AWS).
====

You can install either a standard cluster or a customized cluster. With a
standard cluster, you provide only some details about your AWS account to the
installer, and it creates a cluster.
standard cluster, you provide minimum details that you need to install the
cluster. With a customized cluster, you can specify more details about the
platform, such as the number of machines that the control plane uses, the type
of virtual machine that the cluster deploys, or the CIDR range for the
Kubernetes service network.

With a customized cluster, you can specify more details of the installation,
such as ...
[IMPORTANT]
====
It is possible to modify Kubernetes and the Ignition Configs that control
the underlying Red Hat CoreOS operating system during installation. However,
no validation is available to confirm the suitability of any modifications that
you make to these objects. If you modify these objects, you might render
your cluster non-functional. Because of this risk, modifying Kubernetes and
Ignition Configs is not supported.
====

When you install {product-title} cluster with Installer Provisioned Infrastructure (IPI), you download the
installer from link:try.openshift.com. This site manages:
installer from link:try.openshift.com[the OpenShift start page]. This site manages:

* REST API for accounts
* Registry tokens, which are the pull secrets that you use to obtain the required
Expand Down
Loading