Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Change API key docs to Kibana UI #5953

Merged
merged 6 commits into from
Aug 23, 2021
Merged

docs: Change API key docs to Kibana UI #5953

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e068790
docs: swap to kibana UI
bmorelli25 Aug 16, 2021
56730f5
Merge branch 'master' of github.com:elastic/apm-server into doc-api-c…
bmorelli25 Aug 16, 2021
ba87d95
docs: bugfix
bmorelli25 Aug 16, 2021
3ed3acc
Merge branch 'master' into doc-api-creation
bmorelli25 Aug 19, 2021
291fd18
Merge branch 'master' of github.com:elastic/apm-server into doc-api-c…
bmorelli25 Aug 23, 2021
bf16de4
feedback: decouple beats, update privs, fix NOTE
bmorelli25 Aug 23, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 82 additions & 60 deletions docs/copied-from-beats/docs/security/api-keys.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,71 +19,114 @@ IMPORTANT: Review <<feature-roles>> before creating API keys for {beatname_uc}.
[float]
[[beats-api-key-publish]]
=== Create an API key for publishing
To create an API key to use for writing data to {es}, use the
{ref}/security-api-create-api-key.html[Create API key API], for example:

[source,console,subs="attributes,callouts"]
------------------------------------------------------------
POST /_security/api_key
In {kib}, navigate to **Stack Management** > **API keys** and click **Create API key**.

[role="screenshot"]
image::images/server-api-key-create.png[API key creation]

Enter a name for your API key and select **Restrict privileges**.
In the role descriptors box, assign the appropriate privileges to the new API key.
For example:

[source,json,subs="attributes,callouts"]
----
{
"name": "{beat_default_index_prefix}_host001", <1>
"role_descriptors": {
"{beat_default_index_prefix}_writer": { <2>
"cluster": ["monitor", "read_ilm"],
"index": [
{
"names": ["{beat_default_index_prefix}-*"],
"privileges": ["view_index_metadata", "create_doc"]
}
"{beat_default_index_prefix}_writer": {
"cluster": ["monitor", "read_ilm"], <1>
"index": [ <2>
{
"names": ["{beat_default_index_prefix}-*"],
"privileges": ["view_index_metadata", "create_doc"]
}
Copy link
Member Author

@bmorelli25 bmorelli25 Aug 16, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for reviewers. Are these permissions correct for a publishing user? I didn't change anything here, but I did notice that our Grant privileges and roles needed for writing events documentation shows different req'd privs.

These docs Privilege docs
Cluster: monitor + read_ilm Cluster: none
Index (apm-*): view_index_metadata + create_doc Index (apm-*): create_index + create_doc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! The linked docs have a NOTE at the beginning that the setup role is separated from the writer role to minimize the privileges required during writing. That's why the dedicated writer role has less privileges. But even with that, there are still discrepancies.

I'd only put a minimal example here, using the same privileges as documented for the writer role, and add a note that this assumes previous setup and link to the privileges docs for more details on that. Otherwise I am certain we will diverge again.

I did not explicitly test the privileges again, but remember that we did extensive testing when overhauling the privileges docs, so I think we can rely on them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd only put a minimal example here, using the same privileges as documented for the writer role, and add a note that this assumes previous setup and link to the privileges docs for more details on that. Otherwise I am certain we will diverge again.

Makes sense! I'll get that fixed.

but remember that we did extensive testing when overhauling the privileges docs, so I think we can rely on them.

💯

Thanks!

]
}
}
}
------------------------------------------------------------
<1> Name of the API key
<2> Granted privileges, see <<feature-roles>>
----
<1> Defines cluster-level privileges
<2> Defines index-level privileges

NOTE: See <<privileges-to-publish-events>> for the list of privileges required to publish events.

The return value will look something like this:
To set an expiration date for the API key, select **Expire after time**
and input the lifetime of the API key in days.

[source,console-result,subs="attributes,callouts"]
--------------------------------------------------
{
"id":"TiNAGG4BaaMdaH1tRfuU", <1>
"name":"{beat_default_index_prefix}_host001",
"api_key":"KnR6yE41RrSowb0kQ0HWoA" <2>
}
--------------------------------------------------
<1> Unique id for this API key
<2> Generated API key
Click **Create API key**. In the dropdown, switch to **Beats** and copy the API key.

You can now use this API key in your +{beatname_lc}.yml+ configuration file like this:
["source","yaml"]
You can now use this API key in your +{beatname_lc}.yml+ configuration file:

["source","yml",subs="attributes"]
--------------------
output.elasticsearch:
api_key: TiNAGG4BaaMdaH1tRfuU:KnR6yE41RrSowb0kQ0HWoA <1>
--------------------
<1> Format is `id:api_key` (as returned by {ref}/security-api-create-api-key.html[Create API key])
<1> Format is `id:api_key` (as shown in the Beats dropdown)

[float]
[[beats-api-key-monitor]]
=== Create an API key for monitoring
To create an API key to use for sending monitoring data to {es}, use the
{ref}/security-api-create-api-key.html[Create API key API], for example:

In {kib}, navigate to **Stack Management** > **API keys** and click **Create API key**.

[role="screenshot"]
image::images/server-api-key-create.png[API key creation]

Enter a name for your API key and select **Restrict privileges**.
In the role descriptors box, assign the appropriate privileges to the new API key.
For example:

[source,json,subs="attributes,callouts"]
----
{
"{beat_default_index_prefix}_monitoring": {
"cluster": ["monitor"], <1>
"index": [ <2>
{
"names": [".monitoring-beats-*"],
"privileges": ["create_index", "create"]
}
Copy link
Member Author

@bmorelli25 bmorelli25 Aug 16, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here. What we had is different from what we show in Grant privileges and roles needed for monitoring.

These docs Privilege docs
Cluster: monitor Cluster: none
Index (.monitoring-beats-*): create_index + create_doc Index (.monitoring-beats-*): create_index + create_doc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, let's rely on the privilege docs.

]
}
}
----
<1> Defines cluster-level privileges
<2> Defines index-level privileges

NOTE: See <<privileges-to-publish-monitoring>> for the list of privileges required to send monitoring data.

To set an expiration date for the API key, select **Expire after time**
and input the lifetime of the API key in days.

Click **Create API key**. In the dropdown, switch to **Beats** and copy the API key.

You can now use this API key in your +{beatname_lc}.yml+ configuration file like this:

["source","yml",subs="attributes"]
--------------------
monitoring.elasticsearch:
api_key: TiNAGG4BaaMdaH1tRfuU:KnR6yE41RrSowb0kQ0HWoA <1>
--------------------
<1> Format is `id:api_key` (as shown in the Beats dropdown)

[float]
[[beats-api-key-es]]
=== Create an API key with {es} APIs

You can also use {es}'s {ref}/security-api-create-api-key.html[Create API key API] to create a new API key.
For example:

[source,console,subs="attributes,callouts"]
------------------------------------------------------------
POST /_security/api_key
{
"name": "{beat_default_index_prefix}_host001", <1>
"role_descriptors": {
"{beat_default_index_prefix}_monitoring": { <2>
"cluster": ["monitor"],
"{beat_default_index_prefix}_writer": { <2>
"cluster": ["monitor", "read_ilm"],
"index": [
{
"names": [".monitoring-beats-*"],
"privileges": ["create_index", "create"]
"names": ["{beat_default_index_prefix}-*"],
"privileges": ["view_index_metadata", "create_doc"]
}
]
}
Expand All @@ -93,28 +136,7 @@ POST /_security/api_key
<1> Name of the API key
<2> Granted privileges, see <<feature-roles>>

NOTE: See <<privileges-to-publish-monitoring>> for the list of privileges required to send monitoring data.

The return value will look something like this:

[source,console-result,subs="attributes,callouts"]
--------------------------------------------------
{
"id":"TiNAGG4BaaMdaH1tRfuU", <1>
"name":"{beat_default_index_prefix}_host001",
"api_key":"KnR6yE41RrSowb0kQ0HWoA" <2>
}
--------------------------------------------------
<1> Unique id for this API key
<2> Generated API key

You can now use this API key in your +{beatname_lc}.yml+ configuration file like this:
["source","yml",subs="attributes"]
--------------------
monitoring.elasticsearch:
api_key: TiNAGG4BaaMdaH1tRfuU:KnR6yE41RrSowb0kQ0HWoA <1>
--------------------
<1> Format is `id:api_key` (as returned by {ref}/security-api-create-api-key.html[Create API key])
See the {ref}/security-api-create-api-key.html[Create API key] reference for more information.

[[learn-more-api-keys]]
[float]
Expand Down
Binary file added docs/images/api-key-copy.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/api-key-create.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/server-api-key-create.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
137 changes: 94 additions & 43 deletions docs/secure-communication-agents.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,30 +31,37 @@ include::./ssl-input.asciidoc[]
[[api-key]]
=== API keys

You can configure API keys to authorize requests to the APM Server.

NOTE: API keys are sent as plain-text,
so they only provide security when used in combination with <<ssl-setup,SSL/TLS>>.
They are not applicable for agents running on clients, like the RUM agent,
as there is no way to prevent them from being publicly exposed.

By enabling `apm-server.auth.api_key.enabled: true`, you ensure that only agents with a valid API Key
are able to successfully use APM Server's API (except for RUM endpoints).
Configure API keys to authorize requests to the APM Server.
To enable API key authorization, set `apm-server.auth.api_key.enabled` to `true`.

To secure the communication between APM Agents and the APM Server with API keys:
There are multiple, unique privileges you can assign to each API key.
API keys can have one or more of these privileges:

* *Agent configuration* (`config_agent:read`): Required for agents to read
{kibana-ref}/agent-configuration.html[Agent configuration remotely].
* *Ingest* (`event:write`): Required for ingesting Agent events.
* *Sourcemap* (`sourcemap:write`): Required for <<sourcemaps,uploading sourcemaps>>.

To secure the communication between APM Agents and the APM Server with API keys,
make sure <<ssl-setup,SSL/TLS>> is enabled, then complete these steps:

. Make sure <<ssl-setup,SSL/TLS>> is enabled
. <<configure-api-key,Enable and configure API keys>>
. <<create-api-key,Create an API key with APM Server>>
. <<set-api-key,Set the API key in your APM Agents>>
. <<create-api-key,Create an API key in {kib}>>
. <<set-api-key,Set the API key in your APM agents>>

NOTE: API Keys are not applicable for the RUM Agent,
as there is no way to prevent them from being publicly exposed.
NOTE: API Keys
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leftover?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leftover 😬


[[configure-api-key]]
[float]
=== Enable and configure API keys

API keys are disabled by default. You can change this, and additional settings,
in the `apm-server.auth.api_key` section of the +{beatname_lc}.yml+ configuration file.
API keys are disabled by default. Enable and configure this feature in the `apm-server.auth.api_key`
section of the +{beatname_lc}.yml+ configuration file.

At a minimum, you must enable API keys,
and should set a limit on the number of unique API keys that APM Server allows per minute.
Expand All @@ -73,35 +80,95 @@ All other configuration options are described in <<api-key-settings>>.

[[create-api-key]]
[float]
=== Create and validate an API key
=== Create an API key

[role="screenshot"]
image::images/api-key-create.png[API key creation]

In {kib}, navigate to **Stack Management** > **API keys** and click **Create API key**.

Enter a name for your API key and select **Restrict privileges**.
In the role descriptors box, copy and paste the following JSON.
This example creates an API key with privileges for ingesting APM events,
reading agent central configuration, uploading a sourcemap:

[source,json]
----
{
"apm": {
"applications": [
{
"application": "apm",
"privileges": ["sourcemap:write", "event:write", "config_agent:read"], <1>
"resources": ["*"]
}
]
}
}
----
<1> This example adds all three API privileges to the new API key.
Privileges are described <<api-key,above>>. Remove any privileges that you do not need.

To set an expiration date for the API key, select **Expire after time**
and input the lifetime of the API key in days.

Click **Create API key** and then copy the Base64 encoded API key.
You will need this for the next step, and you will not be able to view it again.

[role="screenshot"]
image::images/api-key-copy.png[API key copy base64]

[[set-api-key]]
[float]
=== Set the API key in your APM agents

You can now apply your newly created API keys in the configuration of each of your APM agents.
See the relevant agent documentation for additional information:

// Not relevant for RUM and iOS
* *Go agent*: {apm-go-ref}/configuration.html#config-api-key[`ELASTIC_APM_API_KEY`]
* *.NET agent*: {apm-dotnet-ref}/config-reporter.html#config-api-key[`ApiKey`]
* *Java agent*: {apm-java-ref}/config-reporter.html#config-api-key[`api_key`]
* *Node.js agent*: {apm-node-ref}/configuration.html#api-key[`apiKey`]
* *PHP agent*: {apm-php-ref-v}/configuration-reference.html#config-api-key[`api_key`]
* *Python agent*: {apm-py-ref}/configuration.html#config-api-key[`api_key`]
* *Ruby agent*: {apm-ruby-ref}/configuration.html#config-api-key[`api_key`]

[[configure-api-key-alternative]]
[float]
=== Alternate API key creation methods

API keys can also be created and validated outside of Kibana:

* <<create-api-key-workflow-apm-server>>
* <<create-api-key-workflow-es>>

[[create-api-key-workflow-apm-server]]
[float]
==== APM Server API key workflow

APM Server provides a command line interface for creating, retrieving, invalidating, and verifying API keys.
Keys created using this method can only be used for Agent/Server communication.
Keys created using this method can only be used for communication with APM Server.

[[create-api-key-subcommands]]
[float]
==== `apikey` subcommands
===== `apikey` subcommands

include::{libbeat-dir}/command-reference.asciidoc[tag=apikey-subcommands]

[[create-api-key-privileges]]
[float]
==== Privileges
===== Privileges

There are three unique privileges you can assign to each API keys.
If privileges are not specified at creation time, the created key will have all privileges.

* *Agent configuration*: Required for agents to read
{kibana-ref}/agent-configuration.html[Agent configuration remotely].
`--agent-config` gives the `config_agent:read` privilege to the created key.
* *Ingest*: Required for ingesting Agent events.
`--ingest` gives the `event:write` privilege to the created key.
* *Sourcemap*: Required for <<sourcemaps,uploading sourcemaps>>.
`--sourcemap` gives the `sourcemap:write` privilege to the created key.
* `--agent-config` grants the `config_agent:read` privilege
* `--ingest` grants the `event:write` privilege
* `--sourcemap` grants the `sourcemap:write` privilege

[[create-api-key-workflow]]
[float]
==== API key workflow example
===== Create an API key

Create an API key with the `create` subcommand.

Expand Down Expand Up @@ -164,9 +231,9 @@ A full list of `apikey` subcommands and flags is available in the <<apikey-comma

[[create-api-key-workflow-es]]
[float]
==== Alternate API key workflow example
==== {es} API key workflow

Instead of using the APM Server CLI, it is possible to create API keys using the Elasticsearch
It is also possible to create API keys using the Elasticsearch
{ref}/security-api-create-api-key.html[create API key API].

This example creates an API key named `java-002`:
Expand Down Expand Up @@ -276,22 +343,6 @@ Authorized for privilege "event:write"...: Yes
Authorized for privilege "sourcemap:write"...: Yes
----

[[set-api-key]]
[float]
=== Set the API key in your APM agents

You can now apply your newly created API keys in the configuration of each of your APM agents.
See the relevant agent documentation for additional information:

// Not relevant for RUM and iOS
* *Go agent*: {apm-go-ref}/configuration.html#config-api-key[`ELASTIC_APM_API_KEY`]
* *.NET agent*: {apm-dotnet-ref}/config-reporter.html#config-api-key[`ApiKey`]
* *Java agent*: {apm-java-ref}/config-reporter.html#config-api-key[`api_key`]
* *Node.js agent*: {apm-node-ref}/configuration.html#api-key[`apiKey`]
* *PHP agent*: {apm-php-ref-v}/configuration-reference.html#config-api-key[`api_key`]
* *Python agent*: {apm-py-ref}/configuration.html#config-api-key[`api_key`]
* *Ruby agent*: {apm-ruby-ref}/configuration.html#config-api-key[`api_key`]

[float]
[[api-key-settings]]
=== API key configuration options
Expand Down