Skip to content

Latest commit

 

History

History
472 lines (388 loc) · 41.3 KB

README.md

File metadata and controls

472 lines (388 loc) · 41.3 KB

@datafire/google_composer

Client library for Cloud Composer API

Installation and Usage

npm install --save @datafire/google_composer
let google_composer = require('@datafire/google_composer').create({
  access_token: "",
  refresh_token: "",
  client_id: "",
  client_secret: "",
  redirect_uri: ""
});

.then(data => {
  console.log(data);
});

Description

Manages Apache Airflow environments on Google Cloud Platform.

Actions

oauthCallback

Exchange the code passed to your redirect URI for an access_token

google_composer.oauthCallback({
  "code": ""
}, context)

Input

  • input object
    • code required string

Output

  • output object
    • access_token string
    • refresh_token string
    • token_type string
    • scope string
    • expiration string

oauthRefresh

Exchange a refresh_token for an access_token

google_composer.oauthRefresh(null, context)

Input

This action has no parameters

Output

  • output object
    • access_token string
    • refresh_token string
    • token_type string
    • scope string
    • expiration string

composer.projects.locations.operations.delete

Deletes a long-running operation. This method indicates that the client is no longer interested in the operation result. It does not cancel the operation. If the server doesn't support this method, it returns google.rpc.Code.UNIMPLEMENTED.

google_composer.composer.projects.locations.operations.delete({
  "name": ""
}, context)

Input

  • input object
    • name required string: The name of the operation resource to be deleted.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

composer.projects.locations.operations.get

Gets the latest state of a long-running operation. Clients can use this method to poll the operation result at intervals as recommended by the API service.

google_composer.composer.projects.locations.operations.get({
  "name": ""
}, context)

Input

  • input object
    • name required string: The name of the operation resource.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

composer.projects.locations.environments.patch

Update an environment.

google_composer.composer.projects.locations.environments.patch({
  "name": ""
}, context)

Input

  • input object
    • name required string: The relative resource name of the environment to update, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}"
    • updateMask string: Required. A comma-separated list of paths, relative to Environment, of fields to update. For example, to set the version of scikit-learn to install in the environment to 0.19.0 and to remove an existing installation of argparse, the updateMask parameter would include the following two paths values: "config.softwareConfig.pypiPackages.scikit-learn" and "config.softwareConfig.pypiPackages.argparse". The included patch environment would specify the scikit-learn version as follows: { "config":{ "softwareConfig":{ "pypiPackages":{ "scikit-learn":"==0.19.0" } } } } Note that in the above example, any existing PyPI packages other than scikit-learn and argparse will be unaffected. Only one update type may be included in a single request's updateMask. For example, one cannot update both the PyPI packages and labels in the same request. However, it is possible to update multiple members of a map field simultaneously in the same request. For example, to set the labels "label1" and "label2" while clearing "label3" (assuming it already exists), one can provide the paths "labels.label1", "labels.label2", and "labels.label3" and populate the patch environment as follows: { "labels":{ "label1":"new-label1-value" "label2":"new-label2-value" } } Note that in the above example, any existing labels that are not included in the updateMask will be unaffected. It is also possible to replace an entire map field by providing the map field's path in the updateMask. The new value of the field will be that which is provided in the patch environment. For example, to delete all pre-existing user-specified PyPI packages and install botocore at version 1.7.14, the updateMask would contain the path "config.softwareConfig.pypiPackages", and the patch environment would be the following: { "config":{ "softwareConfig":{ "pypiPackages":{ "botocore":"==1.7.14" } } } } Note: Only the following fields can be updated: * config.softwareConfig.pypiPackages * Replace all custom custom PyPI packages. If a replacement package map is not included in environment, all custom PyPI packages are cleared. It is an error to provide both this mask and a mask specifying an individual package. * config.softwareConfig.pypiPackages.packagename * Update the custom PyPI package packagename, preserving other packages. To delete the package, include it in updateMask, and omit the mapping for it in environment.config.softwareConfig.pypiPackages. It is an error to provide both a mask of this form and the "config.softwareConfig.pypiPackages" mask. * labels * Replace all environment labels. If a replacement labels map is not included in environment, all labels are cleared. It is an error to provide both this mask and a mask specifying one or more individual labels. * labels.labelName * Set the label named labelName, while preserving other labels. To delete the label, include it in updateMask and omit its mapping in environment.labels. It is an error to provide both a mask of this form and the "labels" mask. * config.nodeCount * Horizontally scale the number of nodes in the environment. An integer greater than or equal to 3 must be provided in the config.nodeCount field. * config.webServerNetworkAccessControl * Replace the environment's current WebServerNetworkAccessControl. * config.softwareConfig.airflowConfigOverrides * Replace all Apache Airflow config overrides. If a replacement config overrides map is not included in environment, all config overrides are cleared. It is an error to provide both this mask and a mask specifying one or more individual config overrides. * config.softwareConfig.airflowConfigOverrides.section- name * Override the Apache Airflow config property name in the section named section, preserving other properties. To delete the property override, include it in updateMask and omit its mapping in environment.config.softwareConfig.airflowConfigOverrides. It is an error to provide both a mask of this form and the "config.softwareConfig.airflowConfigOverrides" mask. * config.softwareConfig.envVariables * Replace all environment variables. If a replacement environment variable map is not included in environment, all custom environment variables are cleared. It is an error to provide both this mask and a mask specifying one or more individual environment variables. * config.softwareConfig.imageVersion * Upgrade the version of the environment in-place. Refer to SoftwareConfig.image_version for information on how to format the new image version. Additionally, the new image version cannot effect a version downgrade and must match the current image version's Composer major version and Airflow major and minor versions. Consult the Cloud Composer Version List for valid values. * config.databaseConfig.machineType * Cloud SQL machine type used by Airflow database. It has to be one of: db-n1-standard-2, db-n1-standard-4, db-n1-standard-8 or db-n1-standard-16. * config.webServerConfig.machineType * Machine type on which Airflow web server is running. It has to be one of: composer-n1-webserver-2, composer-n1-webserver-4 or composer-n1-webserver-8. * config.maintenanceWindow * Maintenance window during which Cloud Composer components may be under maintenance.
    • body Environment
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

composer.projects.locations.operations.list

Lists operations that match the specified filter in the request. If the server doesn't support this method, it returns UNIMPLEMENTED. NOTE: the name binding allows API services to override the binding to use different resource name schemes, such as users/*/operations. To override the binding, API services can add a binding such as "/v1/{name=users/*}/operations" to their service configuration. For backwards compatibility, the default name includes the operations collection id, however overriding users must ensure the name binding is the parent resource, without the operations collection id.

google_composer.composer.projects.locations.operations.list({
  "name": ""
}, context)

Input

  • input object
    • name required string: The name of the operation's parent resource.
    • filter string: The standard list filter.
    • pageSize integer: The standard list page size.
    • pageToken string: The standard list page token.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

composer.projects.locations.environments.restartWebServer

Restart Airflow web server.

google_composer.composer.projects.locations.environments.restartWebServer({
  "name": ""
}, context)

Input

  • input object
    • name required string: The resource name of the environment to restart the web server for, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}"
    • body RestartWebServerRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

composer.projects.locations.environments.list

List environments.

google_composer.composer.projects.locations.environments.list({
  "parent": ""
}, context)

Input

  • input object
    • parent required string: List environments in the given project and location, in the form: "projects/{projectId}/locations/{locationId}"
    • pageSize integer: The maximum number of environments to return.
    • pageToken string: The next_page_token value returned from a previous List request, if any.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

composer.projects.locations.environments.create

Create a new environment.

google_composer.composer.projects.locations.environments.create({
  "parent": ""
}, context)

Input

  • input object
    • parent required string: The parent must be of the form "projects/{projectId}/locations/{locationId}".
    • body Environment
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

composer.projects.locations.imageVersions.list

List ImageVersions for provided location.

google_composer.composer.projects.locations.imageVersions.list({
  "parent": ""
}, context)

Input

  • input object
    • parent required string: List ImageVersions in the given project and location, in the form: "projects/{projectId}/locations/{locationId}"
    • includePastReleases boolean: Whether or not image versions from old releases should be included.
    • pageSize integer: The maximum number of image_versions to return.
    • pageToken string: The next_page_token value returned from a previous List request, if any.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

Definitions

AllowedIpRange

  • AllowedIpRange object: Allowed IP range with user-provided description.
    • description string: Optional. User-provided description. It must contain at most 300 characters.
    • value string: IP address or range, defined using CIDR notation, of requests that this rule applies to. Examples: 192.168.1.1 or 192.168.0.0/16 or 2001:db8::/32 or 2001:0db8:0000:0042:0000:8a2e:0370:7334. IP range prefixes should be properly truncated. For example, 1.2.3.4/24 should be truncated to 1.2.3.0/24. Similarly, for IPv6, 2001:db8::1/32 should be truncated to 2001:db8::/32.

DatabaseConfig

  • DatabaseConfig object: The configuration of Cloud SQL instance that is used by the Apache Airflow software.
    • machineType string: Optional. Cloud SQL machine type used by Airflow database. It has to be one of: db-n1-standard-2, db-n1-standard-4, db-n1-standard-8 or db-n1-standard-16. If not specified, db-n1-standard-2 will be used.

Date

  • Date object: Represents a whole or partial calendar date, such as a birthday. The time of day and time zone are either specified elsewhere or are insignificant. The date is relative to the Gregorian Calendar. This can represent one of the following: * A full date, with non-zero year, month, and day values * A month and day value, with a zero year, such as an anniversary * A year on its own, with zero month and day values * A year and month value, with a zero day, such as a credit card expiration date Related types are google.type.TimeOfDay and google.protobuf.Timestamp.
    • day integer: Day of a month. Must be from 1 to 31 and valid for the year and month, or 0 to specify a year by itself or a year and month where the day isn't significant.
    • month integer: Month of a year. Must be from 1 to 12, or 0 to specify a year without a month and day.
    • year integer: Year of the date. Must be from 1 to 9999, or 0 to specify a date without a year.

Empty

  • Empty object: A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } The JSON representation for Empty is empty JSON object {}.

EncryptionConfig

  • EncryptionConfig object: The encryption options for the Composer environment and its dependencies.
    • kmsKeyName string: Optional. Customer-managed Encryption Key available through Google's Key Management Service. Cannot be updated. If not specified, Google-managed key will be used.

Environment

  • Environment object: An environment for running orchestration tasks.
    • config EnvironmentConfig
    • createTime string: Output only. The time at which this environment was created.
    • labels object: Optional. User-defined labels for this environment. The labels map can contain no more than 64 entries. Entries of the labels map are UTF8 strings that comply with the following restrictions: * Keys must conform to regexp: \p{Ll}\p{Lo}{0,62} * Values must conform to regexp: [\p{Ll}\p{Lo}\p{N}_-]{0,63} * Both keys and values are additionally constrained to be <= 128 bytes in size.
    • name string: The resource name of the environment, in the form: "projects/{projectId}/locations/{locationId}/environments/{environmentId}" EnvironmentId must start with a lowercase letter followed by up to 63 lowercase letters, numbers, or hyphens, and cannot end with a hyphen.
    • state string (values: STATE_UNSPECIFIED, CREATING, RUNNING, UPDATING, DELETING, ERROR): The current state of the environment.
    • updateTime string: Output only. The time at which this environment was last modified.
    • uuid string: Output only. The UUID (Universally Unique IDentifier) associated with this environment. This value is generated when the environment is created.

EnvironmentConfig

  • EnvironmentConfig object: Configuration information for an environment.
    • airflowUri string: Output only. The URI of the Apache Airflow Web UI hosted within this environment (see Airflow web interface).
    • dagGcsPrefix string: Output only. The Cloud Storage prefix of the DAGs for this environment. Although Cloud Storage objects reside in a flat namespace, a hierarchical file tree can be simulated using "/"-delimited object name prefixes. DAG objects for this environment reside in a simulated directory with the given prefix.
    • databaseConfig DatabaseConfig
    • encryptionConfig EncryptionConfig
    • gkeCluster string: Output only. The Kubernetes Engine cluster used to run this environment.
    • maintenanceWindow MaintenanceWindow
    • nodeConfig NodeConfig
    • nodeCount integer: The number of nodes in the Kubernetes Engine cluster that will be used to run this environment.
    • privateEnvironmentConfig PrivateEnvironmentConfig
    • softwareConfig SoftwareConfig
    • webServerConfig WebServerConfig
    • webServerNetworkAccessControl WebServerNetworkAccessControl

IPAllocationPolicy

  • IPAllocationPolicy object: Configuration for controlling how IPs are allocated in the GKE cluster.
    • clusterIpv4CidrBlock string: Optional. The IP address range used to allocate IP addresses to pods in the cluster. This field is applicable only when use_ip_aliases is true. Set to blank to have GKE choose a range with the default size. Set to /netmask (e.g. /14) to have GKE choose a range with a specific netmask. Set to a CIDR notation (e.g. 10.96.0.0/14) from the RFC-1918 private networks (e.g. 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) to pick a specific range to use. Specify cluster_secondary_range_name or cluster_ipv4_cidr_block but not both.
    • clusterSecondaryRangeName string: Optional. The name of the cluster's secondary range used to allocate IP addresses to pods. Specify either cluster_secondary_range_name or cluster_ipv4_cidr_block but not both. This field is applicable only when use_ip_aliases is true.
    • servicesIpv4CidrBlock string: Optional. The IP address range of the services IP addresses in this cluster. This field is applicable only when use_ip_aliases is true. Set to blank to have GKE choose a range with the default size. Set to /netmask (e.g. /14) to have GKE choose a range with a specific netmask. Set to a CIDR notation (e.g. 10.96.0.0/14) from the RFC-1918 private networks (e.g. 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) to pick a specific range to use. Specify services_secondary_range_name or services_ipv4_cidr_block but not both.
    • servicesSecondaryRangeName string: Optional. The name of the services' secondary range used to allocate IP addresses to the cluster. Specify either services_secondary_range_name or services_ipv4_cidr_block but not both. This field is applicable only when use_ip_aliases is true.
    • useIpAliases boolean: Optional. Whether or not to enable Alias IPs in the GKE cluster. If true, a VPC-native cluster is created.

ImageVersion

  • ImageVersion object: Image Version information
    • creationDisabled boolean: Whether it is impossible to create an environment with the image version.
    • imageVersionId string: The string identifier of the ImageVersion, in the form: "composer-x.y.z-airflow-a.b(.c)"
    • isDefault boolean: Whether this is the default ImageVersion used by Composer during environment creation if no input ImageVersion is specified.
    • releaseDate Date
    • supportedPythonVersions array: supported python versions
      • items string
    • upgradeDisabled boolean: Whether it is impossible to upgrade an environment running with the image version.

ListEnvironmentsResponse

  • ListEnvironmentsResponse object: The environments in a project and location.
    • environments array: The list of environments returned by a ListEnvironmentsRequest.
    • nextPageToken string: The page token used to query for the next page if one exists.

ListImageVersionsResponse

  • ListImageVersionsResponse object: The ImageVersions in a project and location.
    • imageVersions array: The list of supported ImageVersions in a location.
    • nextPageToken string: The page token used to query for the next page if one exists.

ListOperationsResponse

  • ListOperationsResponse object: The response message for Operations.ListOperations.
    • nextPageToken string: The standard List next-page token.
    • operations array: A list of operations that matches the specified filter in the request.

MaintenanceWindow

  • MaintenanceWindow object: The configuration settings for Cloud Composer maintenance window. The following example: { "startTime":"2019-08-01T01:00:00Z" "endTime":"2019-08-01T07:00:00Z" "recurrence":"FREQ=WEEKLY;BYDAY=TU,WE" } would define a maintenance window between 01 and 07 hours UTC during each Tuesday and Wednesday.
    • endTime string: Required. Maintenance window end time. It is used only to calculate the duration of the maintenance window. The value for end_time must be in the future, relative to start_time.
    • recurrence string: Required. Maintenance window recurrence. Format is a subset of RFC-5545 RRULE. The only allowed values for FREQ field are FREQ=DAILY and FREQ=WEEKLY;BYDAY=... Example values: FREQ=WEEKLY;BYDAY=TU,WE, FREQ=DAILY.
    • startTime string: Required. Start time of the first recurrence of the maintenance window.

NodeConfig

  • NodeConfig object: The configuration information for the Kubernetes Engine nodes running the Apache Airflow software.
    • tags array: Optional. The list of instance tags applied to all node VMs. Tags are used to identify valid sources or targets for network firewalls. Each tag within the list must comply with RFC1035. Cannot be updated.
      • items string
    • diskSizeGb integer: Optional. The disk size in GB used for node VMs. Minimum size is 20GB. If unspecified, defaults to 100GB. Cannot be updated.
    • ipAllocationPolicy IPAllocationPolicy
    • location string: Optional. The Compute Engine zone in which to deploy the VMs used to run the Apache Airflow software, specified as a relative resource name. For example: "projects/{projectId}/zones/{zoneId}". This location must belong to the enclosing environment's project and location. If both this field and nodeConfig.machineType are specified, nodeConfig.machineType must belong to this location; if both are unspecified, the service will pick a zone in the Compute Engine region corresponding to the Cloud Composer location, and propagate that choice to both fields. If only one field (location or nodeConfig.machineType) is specified, the location information from the specified field will be propagated to the unspecified field.
    • machineType string: Optional. The Compute Engine machine type used for cluster instances, specified as a relative resource name. For example: "projects/{projectId}/zones/{zoneId}/machineTypes/{machineTypeId}". The machineType must belong to the enclosing environment's project and location. If both this field and nodeConfig.location are specified, this machineType must belong to the nodeConfig.location; if both are unspecified, the service will pick a zone in the Compute Engine region corresponding to the Cloud Composer location, and propagate that choice to both fields. If exactly one of this field and nodeConfig.location is specified, the location information from the specified field will be propagated to the unspecified field. The machineTypeId must not be a shared-core machine type. If this field is unspecified, the machineTypeId defaults to "n1-standard-1".
    • maxPodsPerNode integer: Optional. The maximum number of pods per node in the Cloud Composer GKE cluster. The value must be between 8 and 110 and it can be set only if the environment is VPC-native. The default value is 32. Values of this field will be propagated both to the default-pool node pool of the newly created GKE cluster, and to the default "Maximum Pods per Node" value which is used for newly created node pools if their value is not explicitly set during node pool creation. For more information, see [Optimizing IP address allocation] (https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr). Cannot be updated.
    • network string: Optional. The Compute Engine network to be used for machine communications, specified as a relative resource name. For example: "projects/{projectId}/global/networks/{networkId}". If unspecified, the default network in the environment's project is used. If a Custom Subnet Network is provided, nodeConfig.subnetwork must also be provided. For Shared VPC subnetwork requirements, see nodeConfig.subnetwork.
    • oauthScopes array: Optional. The set of Google API scopes to be made available on all node VMs. If oauth_scopes is empty, defaults to ["https://www.googleapis.com/auth/cloud-platform"]. Cannot be updated.
      • items string
    • serviceAccount string: Optional. The Google Cloud Platform Service Account to be used by the node VMs. If a service account is not specified, the "default" Compute Engine service account is used. Cannot be updated.
    • subnetwork string: Optional. The Compute Engine subnetwork to be used for machine communications, specified as a relative resource name. For example: "projects/{projectId}/regions/{regionId}/subnetworks/{subnetworkId}" If a subnetwork is provided, nodeConfig.network must also be provided, and the subnetwork must belong to the enclosing environment's project and location.

Operation

  • Operation object: This resource represents a long-running operation that is the result of a network API call.
    • done boolean: If the value is false, it means the operation is still in progress. If true, the operation is completed, and either error or response is available.
    • error Status
    • metadata object: Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata. Any method that returns a long-running operation should document the metadata type, if any.
    • name string: The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the name should be a resource name ending with operations/{unique_id}.
    • response object: The normal response of the operation in case of success. If the original method returns no data on success, such as Delete, the response is google.protobuf.Empty. If the original method is standard Get/Create/Update, the response should be the resource. For other methods, the response should have the type XxxResponse, where Xxx is the original method name. For example, if the original method name is TakeSnapshot(), the inferred response type is TakeSnapshotResponse.

OperationMetadata

  • OperationMetadata object: Metadata describing an operation.
    • createTime string: Output only. The time the operation was submitted to the server.
    • endTime string: Output only. The time when the operation terminated, regardless of its success. This field is unset if the operation is still ongoing.
    • operationType string (values: TYPE_UNSPECIFIED, CREATE, DELETE, UPDATE): Output only. The type of operation being performed.
    • resource string: Output only. The resource being operated on, as a relative resource name.
    • resourceUuid string: Output only. The UUID of the resource being operated on.
    • state string (values: STATE_UNSPECIFIED, PENDING, RUNNING, SUCCESSFUL, FAILED): Output only. The current operation state.

PrivateClusterConfig

  • PrivateClusterConfig object: Configuration options for the private GKE cluster in a Cloud Composer environment.
    • enablePrivateEndpoint boolean: Optional. If true, access to the public endpoint of the GKE cluster is denied.
    • masterIpv4CidrBlock string: Optional. The CIDR block from which IPv4 range for GKE master will be reserved. If left blank, the default value of '172.16.0.0/23' is used.
    • masterIpv4ReservedRange string: Output only. The IP range in CIDR notation to use for the hosted master network. This range is used for assigning internal IP addresses to the cluster master or set of masters and to the internal load balancer virtual IP. This range must not overlap with any other ranges in use within the cluster's network.

PrivateEnvironmentConfig

  • PrivateEnvironmentConfig object: The configuration information for configuring a Private IP Cloud Composer environment.
    • cloudSqlIpv4CidrBlock string: Optional. The CIDR block from which IP range in tenant project will be reserved for Cloud SQL. Needs to be disjoint from web_server_ipv4_cidr_block
    • enablePrivateEnvironment boolean: Optional. If true, a Private IP Cloud Composer environment is created. If this field is set to true, IPAllocationPolicy.use_ip_aliases must be set to true.
    • privateClusterConfig PrivateClusterConfig
    • webServerIpv4CidrBlock string: Optional. The CIDR block from which IP range for web server will be reserved. Needs to be disjoint from private_cluster_config.master_ipv4_cidr_block and cloud_sql_ipv4_cidr_block.
    • webServerIpv4ReservedRange string: Output only. The IP range reserved for the tenant project's App Engine VMs.

RestartWebServerRequest

  • RestartWebServerRequest object: Restart Airflow web server.

SoftwareConfig

  • SoftwareConfig object: Specifies the selection and configuration of software inside the environment.
    • airflowConfigOverrides object: Optional. Apache Airflow configuration properties to override. Property keys contain the section and property names, separated by a hyphen, for example "core-dags_are_paused_at_creation". Section names must not contain hyphens ("-"), opening square brackets ("["), or closing square brackets ("]"). The property name must not be empty and must not contain an equals sign ("=") or semicolon (";"). Section and property names must not contain a period ("."). Apache Airflow configuration property names must be written in snake_case. Property values can contain any character, and can be written in any lower/upper case format. Certain Apache Airflow configuration property values are blocked, and cannot be overridden.
    • envVariables object: Optional. Additional environment variables to provide to the Apache Airflow scheduler, worker, and webserver processes. Environment variable names must match the regular expression a-zA-Z_*. They cannot specify Apache Airflow software configuration overrides (they cannot match the regular expression AIRFLOW__[A-Z0-9_]+__[A-Z0-9_]+), and they cannot match any of the following reserved names: * AIRFLOW_HOME * C_FORCE_ROOT * CONTAINER_NAME * DAGS_FOLDER * GCP_PROJECT * GCS_BUCKET * GKE_CLUSTER_NAME * SQL_DATABASE * SQL_INSTANCE * SQL_PASSWORD * SQL_PROJECT * SQL_REGION * SQL_USER
    • imageVersion string: The version of the software running in the environment. This encapsulates both the version of Cloud Composer functionality and the version of Apache Airflow. It must match the regular expression composer-([0-9]+\.[0-9]+\.[0-9]+|latest)-airflow-[0-9]+\.[0-9]+(\.[0-9]+.*)?. When used as input, the server also checks if the provided version is supported and denies the request for an unsupported version. The Cloud Composer portion of the version is a semantic version or latest. When the patch version is omitted, the current Cloud Composer patch version is selected. When latest is provided instead of an explicit version number, the server replaces latest with the current Cloud Composer version and stores that version number in the same field. The portion of the image version that follows airflow- is an official Apache Airflow repository release name. See also Version List.
    • pypiPackages object: Optional. Custom Python Package Index (PyPI) packages to be installed in the environment. Keys refer to the lowercase package name such as "numpy" and values are the lowercase extras and version specifier such as "==1.12.0", "[devel,gcp_api]", or "[devel]>=1.8.2, <1.9.2". To specify a package without pinning it to a version specifier, use the empty string as the value.
    • pythonVersion string: Optional. The major version of Python used to run the Apache Airflow scheduler, worker, and webserver processes. Can be set to '2' or '3'. If not specified, the default is '2'. Cannot be updated.

Status

  • Status object: The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details. You can find out more about this error model and how to work with it in the API Design Guide.
    • code integer: The status code, which should be an enum value of google.rpc.Code.
    • details array: A list of messages that carry the error details. There is a common set of message types for APIs to use.
      • items object
    • message string: A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.

WebServerConfig

  • WebServerConfig object: The configuration settings for the Airflow web server App Engine instance.
    • machineType string: Optional. Machine type on which Airflow web server is running. It has to be one of: composer-n1-webserver-2, composer-n1-webserver-4 or composer-n1-webserver-8. If not specified, composer-n1-webserver-2 will be used. Value custom is returned only in response, if Airflow web server parameters were manually changed to a non-standard values.

WebServerNetworkAccessControl

  • WebServerNetworkAccessControl object: Network-level access control policy for the Airflow web server.
    • allowedIpRanges array: A collection of allowed IP ranges with descriptions.