diff --git a/CHANGELOG.md b/CHANGELOG.md index d32a654c750..ddfa74667ca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ FEATURES: __BACKWARDS INCOMPATIBILITIES:__ * csi: The `attachment_mode` and `access_mode` field are required for `volume` blocks in job specifications. Registering a volume requires at least one `capability` block with the `attachment_mode` and `access_mode` fields set. [[GH-10330](https://github.com/hashicorp/nomad/issues/10330)] + * licensing: Enterprise licenses are no longer stored in raft or synced between servers. Loading the Enterprise license from disk or environment is required. The `nomad license put` command has been removed. [[GH-10458](https://github.com/hashicorp/nomad/issues/10458)] IMPROVEMENTS: * api: Added an API endpoint for fuzzy search queries [[GH-10184](https://github.com/hashicorp/nomad/pull/10184)] diff --git a/website/content/docs/commands/license/get.mdx b/website/content/docs/commands/license/get.mdx index a31e24329ac..5a3f9f36842 100644 --- a/website/content/docs/commands/license/get.mdx +++ b/website/content/docs/commands/license/get.mdx @@ -7,7 +7,9 @@ description: | # Command: license get -The `license get` command is used to retrieve the current Nomad Enterprise license. +The `license get` command is used to retrieve the current Nomad Enterprise +license. The command is not forwarded to the Nomad leader, and will return +the license from the specific server being contacted. ~> License commands are new in Nomad 0.12.0 and are only available with Nomad Enterprise. @@ -25,15 +27,6 @@ capability. @include 'general_options_no_namespace.mdx' -## License Get Options - -- `-stale`: By default the `license get` command will be forwarded to the Nomad - leader. If `-stale` is set to `true`, the command will not be forwarded to - the leader and will return the license from the specific server being - contacted. This option may be useful during upgrade scenarios when a server - is given a new file license and is a follower so the new license has not yet - been propagated to raft. - ## Examples ```shell-session diff --git a/website/content/docs/commands/license/put.mdx b/website/content/docs/commands/license/put.mdx deleted file mode 100644 index 4fad372dbc1..00000000000 --- a/website/content/docs/commands/license/put.mdx +++ /dev/null @@ -1,61 +0,0 @@ ---- -layout: docs -page_title: 'Commands: license put' -description: | - The license put command is used to set the Nomad Enterprise license. ---- - -# Command: license put - -The `license put` command is used to set the Nomad Enterprise license. - -~> License commands are new in Nomad 0.12.0 and are only available with Nomad -Enterprise. - -## Usage - -```plaintext -nomad license put -``` - -When ACLs are enabled, this command requires a token with the 'operator:write' -capability. - -## General Options - -@include 'general_options_no_namespace.mdx' - -## Put Options - -- `-force`: Force the license to be applied. By default, Nomad will only accept - a new license if it is newer than the one currently applied (specified by the - license issue date). Use `-force` to override and apply an older, unexpired - license. - -```plaintext -Install a new license from a file: - - $ nomad license put - -Install a new license from stdin: - - $ nomad license put - -``` - -## Examples - -Set a license using a path - -```shell-session -$ nomad license put -force /path/to/my/license.hclic -``` - -Set a license using a path - -```shell-session -$ nomad license put - - -02MV4UU43BK5HGYYTOJZWFQMTMNNEWU33JLJKES6S2NJMXUWLKIV2E2RCCNRMXSMBQLJKFSM2MK5KTKTLKLF2FS6SJPFNFIYZVJV5GI2S2K5LGUSLJO5UVSM2WPJSEOOLULJMEUZTBK5IWST3JJF5E4VCBPJHFIWTMJZJTANCZK5LGUTCXJJVVU2SRORHUIVLYJVBTC2KNNJATCTKEMM2VSMSONBNEISLJJRBUU4DCNZHDAWKXPBZVSWCSOBRDENLGMFLVC2KPNFEXCSLJO5UWCWCOPJSFOVTGMRDWY5C2KNETMSLKJF3U22SBORGUIVLUJVCGQVKNKRITMTLKJU3E26TDOVHHURJRJVVEC52NPJATGV3JJFZUS3SOGBMVQSRQLAZVE4DCK5KWST3JJF4U2RCJO5GFIQJRJRKECNCWIRAXOT3KIF3U62SBO5LWSSLTJFWVMNDDI5WHSWKYKJYGEMRVMZSEO3DULJJUSNSJNJEXOTLKIF2E2RCVORGUI3CVJVCECNSNIRATMTKEIJQUS2LXNFSEOVTZMJLWY5KZLBJHAYRSGVTGIR3MORNFGSJWJFVES52NNJAXITKEKV2E2VCCKVGUIQJWJVCECNSNIRBGCSLJO5UWGSCKOZNEQVTKMRBUSNSJNU2XMYSXIZVUS2LXNFNG26DILIZU22KPNZZWSYSXHFVWIV3YNRRXSSJWK54UU3TCGNNGYY3NGVUGE3KONRGFQQ - - -``` diff --git a/website/content/docs/configuration/index.mdx b/website/content/docs/configuration/index.mdx index 3909af667a6..5f6b1135702 100644 --- a/website/content/docs/configuration/index.mdx +++ b/website/content/docs/configuration/index.mdx @@ -331,6 +331,12 @@ its configuration. The fields that currently support reloading are: In order to reload any other configuration values, you must restart the Nomad agent. + +Nomad Enterprise requires a license. If the `server.license_path` +configuration or `NOMAD_LICENSE_PATH` environment variable are set, the +license will be reloaded from the file on a configuration reload. + + If the Nomad agent receives a `SIGHUP` during initialization, it may crash (see [GH-3885]). Ensure that the Nomad agent is able to receive RPC traffic before attempting to reload its configuration. diff --git a/website/content/docs/enterprise/license.mdx b/website/content/docs/enterprise/license.mdx index 59ba93a3094..62d241292c1 100644 --- a/website/content/docs/enterprise/license.mdx +++ b/website/content/docs/enterprise/license.mdx @@ -8,9 +8,9 @@ description: >- ## Nomad Enterprise Licensing -Licensing capabilities were added to Nomad Enterprise v0.12.0. The license is -set once for a region and automatically propagates to all servers within the -region. Nomad Enterprise can be downloaded from the [releases site]. +Licensing capabilities were added to Nomad Enterprise v0.12.0. Each server in +the cluster must have its own license. Nomad Enterprise can be downloaded from +the [releases site]. Click [here](https://www.hashicorp.com/go/nomad-enterprise) to set up a demo or request a trial of Nomad Enterprise. @@ -20,24 +20,19 @@ source version of Nomad. Servers running the open source version of Nomad will panic if they are joined to a Nomad Enterprise cluster. See issue [gh-9958] for more details. -## Evaluating Nomad Enterprise - -Nomad Enterprise can be used without a valid license for 6 hours. When a Nomad -Enterprise server starts without a license configuration option (see [license -configuration]) it uses a temporary trial license. This license is valid -for 6 hours. +## Expiring Licenses -You can inspect the temporary license using `nomad license get`. +Nomad Enterprise license have an expiration time. You can read the license on +a server with the `nomad license get` command: ``` $ nomad license get Product = nomad License Status = valid -License ID = temporary-license -Customer ID = temporary license customer +License ID = my-license +Customer ID = my license customer Issued At = 2021-03-29 14:47:29.024191 -0400 EDT Expires At = 2021-03-29 20:47:29.024191 -0400 EDT -Terminates At = 2021-03-29 20:47:29.024191 -0400 EDT Datacenter = * Modules: governance-policy @@ -56,17 +51,9 @@ Licensed Features: Dynamic Application Sizing ``` -After the trial period, if you attempt to start Nomad with the same state or -`data_dir`, Nomad will wait a brief grace period time to allow an operator to -set a valid license before shutting down. - -## Expiring Licenses - -### Temporary Licenses - -As a Nomad Enterprise license approaches its expiration time, Nomad will -periodically log a warning message about the approaching expiration. Below -shows log excerpts of the warnings. +As a Nomad Enterprise license approaches its expiration time, Nomad servers +will periodically log a warning message about the approaching +expiration. Below shows log excerpts of the warnings. ``` 2021-03-29T15:02:28.100-0400 [WARN] nomad.licensing: license expiring: time_left=5m0s @@ -75,64 +62,24 @@ shows log excerpts of the warnings. 2021-03-29T15:05:28.109-0400 [WARN] nomad.licensing: license expiring: time_left=2m0s 2021-03-29T15:06:28.112-0400 [WARN] nomad.licensing: license expiring: time_left=1m0s 2021-03-29T15:07:28.114-0400 [WARN] nomad.licensing: license expiring: time_left=0s - 2021-03-29T15:07:30.160-0400 [WARN] nomad.licensing: temporary license too old for evaluation period. Nomad will - wait an additional grace period for valid Enterprise license to be applied - before shutting down: grace period=1m0s - 2021-03-29T15:07:58.104-0400 [ERROR] nomad.licensing: license expired, please update license: error="invalid license or license is - 2021-03-29T15:08:30.163-0400 [ERROR] nomad.licensing: cluster age is greater than temporary license lifespan. Please apply a valid license - 2021-03-29T15:08:30.163-0400 [ERROR] nomad.licensing: cluster will shutdown soon. Please apply a valid license - 2021-03-29T15:09:30.164-0400 [ERROR] nomad.licensing: temporary license grace period expired. shutting down - 2021-03-29T15:09:30.164-0400 [INFO] agent: requesting shutdown - 2021-03-29T15:09:30.164-0400 [INFO] client: shutting down - 2021-03-29T15:09:30.164-0400 [INFO] client.plugin: shutting down plugin manager: plugin-type=device - 2021-03-29T15:09:30.164-0400 [INFO] client.plugin: plugin manager finished: plugin-type=device - 2021-03-29T15:09:30.164-0400 [INFO] client.plugin: shutting down plugin manager: plugin-type=driver - 2021-03-29T15:09:30.164-0400 [INFO] client.plugin: plugin manager finished: plugin-type=driver - 2021-03-29T15:09:30.164-0400 [INFO] client.plugin: shutting down plugin manager: plugin-type=csi - 2021-03-29T15:09:30.164-0400 [INFO] client.plugin: plugin manager finished: plugin-type=csi - 2021-03-29T15:09:30.164-0400 [DEBUG] client.server_mgr: shutting down - 2021-03-29T15:09:30.164-0400 [INFO] nomad: shutting down server - 2021-03-29T15:09:30.164-0400 [WARN] nomad: serf: Shutdown without a Leave - 2021-03-29T15:09:30.165-0400 [DEBUG] nomad: shutting down leader loop - 2021-03-29T15:09:30.165-0400 [INFO] nomad: cluster leadership lost - 2021-03-29T15:09:30.170-0400 [INFO] agent: shutdown complete + 2021-03-29T15:07:58.104-0400 [ERROR] nomad.licensing: license expired, please update license: error="invalid license or license is expired" ``` -Since this was a temporary license, when -the temporary license expires, the agent shuts down. - -### Valid, Non-Temporary Licenses +When the license expires, enterprise functionality will become limited. Only +read operations on enterprise endpoints will be supported, and write +operations will return an error. -License expiry is handled differently for valid enterprise licenses. Nomad -licensing will continue to log about the expiring license above, but when the -license fully expires (the Termination Time is reached) the server _will not_ -shut down. Instead, of shutting down, enterprise functionality will become limited. Only -read operations on enterprise endpoints will be supported, and write operations -will return an error. +Note that if the server is restarted with an expired license, it will +immediately stop. -~> **Note:** When an enterprise server starts and the license is expired, Nomad -will wait for a short grace period to apply a valid license before shutting -down. - -## Setting the License +## Configuring the License See the server [license configuration] reference documentation on all the -options to set an enterprise license. - -When setting a Nomad Enterprise license there are two options to pick from. You -can set the license via the CLI or API after the server is running, or Nomad -can automatically load the file from disk or environment when it starts. - -To set the license via CLI, see the [license command] documentation. To set the -license programmatically see the [license endpoint] API documentation. - -To configure Nomad to load the license from disk or environment see the server -[license configuration]. - -## Operating Nomad Enterprise with a License +options to set an enterprise license. Nomad will load the license file from +disk or environment when it starts. In order to immediately alert operators of a bad configuration setting, if a -license configuration option is a completely invalid license, the nomad server +license configuration option is an invalid or expired license, the Nomad server will exit with an error. ``` @@ -142,45 +89,6 @@ NOMAD_LICENSE=misconfigured nomad agent -dev ==> Error starting agent: server setup failed: failed to initialize enterprise licensing: a file license was configured but the license is invalid: error decoding version: expected integer ``` -Some Nomad servers are controlled with a level of automation or could be part -of an autoscaling group. If an operator accidentally has an old, expired -license set as the disk or environment license, the server will emit a warning -log, but not exit if a valid license exists in raft. If a valid license -doesn't exist in raft then the server will enter a grace period before exiting. - -``` -2021-03-29T16:33:01.691-0400 [WARN] nomad.licensing: Configured enterprise -license file is expired! Falling back to temporary license. Please update, or -remove license configuration if setting the license via CLI/API -``` - -## Overriding a File or Environment License - -A Nomad Enterprise server that starts with an automatically loaded file or -environment variable license is able to be overridden using the CLI or API. -When setting a different license from the server's file license a warning will -be emitted. - -If an older (determined by license issue date), but valid license is applied, -an error is returned. - -``` -$ nomad license put nomadlicense.hclic -Error putting license: Unexpected response code: 500 (error setting license: requested license is older than current one, use force to override) -``` - -This can be overridden by setting the `-force` flag. - -``` -$ nomad license put -force nomadlicense.hclic - -WARNING: The server's configured file license is now outdated. Please update or -remove the server's license configuration to prevent initialization issues with -potentially expired licenses. - -Successfully applied license -``` - See the [License commands](/docs/commands/license) for more information on interacting with the Enterprise License. diff --git a/website/content/docs/upgrade/upgrade-specific.mdx b/website/content/docs/upgrade/upgrade-specific.mdx index 8b621fd48d2..098f30e01c2 100644 --- a/website/content/docs/upgrade/upgrade-specific.mdx +++ b/website/content/docs/upgrade/upgrade-specific.mdx @@ -41,6 +41,14 @@ Connect native tasks running in host networking mode will now have `CONSUL_HTTP_ set automatically. Before this was only the case for bridge networking. If an operator already explicitly set `CONSUL_HTTP_ADDR` then it will not get overriden. +#### Enterprise licenses + +Nomad Enterprise licenses are no longer stored in raft or synced between +servers. Nomad Enterprise servers will not start without a license. Before +upgrading, you must provide each server with its own license on disk or in its +environment (see the [Enterprise licensing] documentation for details). The +`nomad license put` command has been removed. + #### iptables Nomad now appends its iptables rules to the `NOMAD-ADMIN` chain instead of @@ -1047,3 +1055,4 @@ deleted and then Nomad 0.3.0 can be launched. [`volume create`]: /docs/commands/volume/create [`volume register`]: /docs/commands/volume/register [`volume`]: /docs/job-specification/volume +[Enterprise licensing]: /docs/enterprise/license