[DOCS] Add data streams to security docs (#59084) (#59237)

This commit is contained in:
James Rodewig 2020-07-08 12:53:56 -04:00 committed by GitHub
parent 00c859bfca
commit 838f717e5f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 161 additions and 44 deletions

View File

@ -8,6 +8,7 @@ To set up a data stream, follow these steps:
. <<create-a-data-stream-template>>. . <<create-a-data-stream-template>>.
. <<create-a-data-stream>>. . <<create-a-data-stream>>.
. <<get-info-about-a-data-stream>> to verify it exists. . <<get-info-about-a-data-stream>> to verify it exists.
. <<secure-a-data-stream>>.
After you set up a data stream, you can <<use-a-data-stream, use the data After you set up a data stream, you can <<use-a-data-stream, use the data
stream>> for indexing, searches, and other supported operations. stream>> for indexing, searches, and other supported operations.
@ -345,6 +346,13 @@ information for the `logs` stream's write index, `.ds-logs-000002`.
// TESTRESPONSE[skip:unable to assert responses with top level array] // TESTRESPONSE[skip:unable to assert responses with top level array]
==== ====
[discrete]
[[secure-a-data-stream]]
=== Secure a data stream
You can use {es} {security-features} to control access to a data stream and its
data. See <<data-stream-privileges>>.
[discrete] [discrete]
[[delete-a-data-stream]] [[delete-a-data-stream]]
=== Delete a data stream === Delete a data stream

View File

@ -438,7 +438,7 @@ PUT /_security/role_mapping/oidc-example
<1> The `example_role` role is *not* a builtin Elasticsearch role. <1> The `example_role` role is *not* a builtin Elasticsearch role.
This example assumes that you have created a custom role of your own, with This example assumes that you have created a custom role of your own, with
appropriate access to your <<roles-indices-priv,indices>> and appropriate access to your <<roles-indices-priv,data streams, indices,>> and
{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features]. {kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features].
The user properties that are mapped via the realm configuration are used to process The user properties that are mapped via the realm configuration are used to process

View File

@ -648,7 +648,7 @@ PUT /_security/role_mapping/saml-example
<1> The `example_role` role is *not* a builtin Elasticsearch role. <1> The `example_role` role is *not* a builtin Elasticsearch role.
This example assumes that you have created a custom role of your own, with This example assumes that you have created a custom role of your own, with
appropriate access to your <<roles-indices-priv,indices>> and appropriate access to your <<roles-indices-priv,data streams, indices,>> and
{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features]. {kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features].
The attributes that are mapped via the realm configuration are used to process The attributes that are mapped via the realm configuration are used to process

View File

@ -1,9 +1,118 @@
[role="xpack"] [role="xpack"]
[[securing-aliases]] [[securing-aliases]]
=== Granting privileges for indices and aliases === Granting privileges for data streams and index aliases
Elasticsearch allows to execute operations against {ref}/indices-aliases.html[index aliases], {es} {security-features} allow you to secure operations executed against
which are effectively virtual indices. An alias points to one or more indices, <<data-streams,data streams>> and <<indices-aliases,index aliases>>.
[[data-stream-privileges]]
==== Data stream privileges
A data stream consists of one or more backing indices, which store the stream's
data. Most requests sent to a data stream are routed to one or more of these
backing indices.
Similar to an index, you can use <<privileges-list-indices,indices privileges>>
to control access to a data stream. Any role or user granted privileges to a
data stream are automatically granted the same privileges to its backing
indices.
.*Example*
[%collapsible]
====
`logs` is a data stream that consists of two backing indices: `.ds-logs-000001`
and `.ds-logs-000002`.
A user is granted the `read` privilege to the `logs` data stream.
[source,js]
--------------------------------------------------
{
"names" : [ "logs" ],
"privileges" : [ "read" ]
}
--------------------------------------------------
// NOTCONSOLE
Because the user is automatically granted the same privileges to the stream's
backing indices, the user can retrieve a document directly from `.ds-logs-000002`:
////
[source,console]
----
PUT /_index_template/logs_data_stream
{
"index_patterns": [ "logs*" ],
"data_stream": {
"timestamp_field": "@timestamp"
},
"template": {
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
}
}
}
}
}
PUT /_data_stream/logs
POST /logs/_rollover/
PUT /logs/_create/2?refresh=wait_for
{
"@timestamp": "2020-12-07T11:06:07.000Z"
}
----
////
[source,console]
----
GET /.ds-logs-000002/_doc/2
----
// TEST[continued]
Later the `logs` data stream <<manually-roll-over-a-data-stream,rolls over>>.
This creates a new backing index: `.ds-logs-000003`. Because the user still has
the `read` privilege for the `logs` data stream, the user can retrieve documents
directly from `.ds-logs-000003`:
////
[source,console]
----
POST /logs/_rollover/
PUT /logs/_create/2?refresh=wait_for
{
"@timestamp": "2020-12-07T11:06:07.000Z"
}
----
// TEST[continued]
////
[source,console]
----
GET /.ds-logs-000003/_doc/2
----
// TEST[continued]
////
[source,console]
----
DELETE /_data_stream/*
DELETE /_index_template/*
----
// TEST[continued]
////
====
[[index-alias-privileges]]
==== Index alias privileges
An index alias points to one or more indices,
holds metadata and potentially a filter. The {es} {security-features} treat holds metadata and potentially a filter. The {es} {security-features} treat
aliases and indices aliases and indices
the same. Privileges for indices actions are granted on specific indices or the same. Privileges for indices actions are granted on specific indices or

View File

@ -152,7 +152,7 @@ Grants the necessary privileges to create snapshots of **all** the indices and
to view their metadata. This role enables users to view the configuration of to view their metadata. This role enables users to view the configuration of
existing snapshot repositories and snapshot details. It does not grant authority existing snapshot repositories and snapshot details. It does not grant authority
to remove or add repositories or to restore snapshots. It also does not enable to remove or add repositories or to restore snapshots. It also does not enable
to change index settings or to read or update index data. to change index settings or to read or update data stream or index data.
[[built-in-roles-superuser]] `superuser`:: [[built-in-roles-superuser]] `superuser`::
Grants full access to the cluster, including all indices and data. A user with Grants full access to the cluster, including all indices and data. A user with

View File

@ -7,12 +7,12 @@ In particular, it restricts which documents can be accessed from document-based
read APIs. read APIs.
To enable document level security, you use a query to specify the documents that To enable document level security, you use a query to specify the documents that
each role can access. The document query is associated with a particular index each role can access. The document query is associated with a particular data
or index pattern and operates in conjunction with the privileges specified for stream, index, or wildcard (`*`) pattern and operates in conjunction with the
the indices. privileges specified for the data streams and indices.
The following role definition grants read access only to documents that The following role definition grants read access only to documents that
belong to the `click` category within all the `events-*` indices: belong to the `click` category within all the `events-*` data streams and indices:
[source,console] [source,console]
-------------------------------------------------- --------------------------------------------------

View File

@ -2,16 +2,16 @@
[[field-and-document-access-control]] [[field-and-document-access-control]]
=== Setting up field and document level security === Setting up field and document level security
You can control access to data within an index by adding field and document level You can control access to data within a data stream or index by adding field and document level
security permissions to a role. security permissions to a role.
<<field-level-security,Field level security permissions>> restrict access to <<field-level-security,Field level security permissions>> restrict access to
particular fields within a document. particular fields within a document.
<<document-level-security,Document level security permissions>> restrict access <<document-level-security,Document level security permissions>> restrict access
to particular documents within an index. to particular documents.
NOTE: Document and field level security is currently meant to operate with NOTE: Document and field level security is currently meant to operate with
read-only privileged accounts. Users with document and field level read-only privileged accounts. Users with document and field level
security enabled for an index should not perform write operations. security enabled for a data stream or index should not perform write operations.
A role can define both field and document level permissions on a per-index basis. A role can define both field and document level permissions on a per-index basis.
A role that doesnt specify field level permissions grants access to ALL fields. A role that doesnt specify field level permissions grants access to ALL fields.
@ -22,7 +22,7 @@ to ALL documents in the index.
===================================================================== =====================================================================
When assigning users multiple roles, be careful that you don't inadvertently When assigning users multiple roles, be careful that you don't inadvertently
grant wider access than intended. Each user has a single set of field level and grant wider access than intended. Each user has a single set of field level and
document level permissions per index. See <<multiple-roles-dls-fls>>. document level permissions per data stream or index. See <<multiple-roles-dls-fls>>.
===================================================================== =====================================================================
NOTE: Document- and field-level security disables the NOTE: Document- and field-level security disables the
@ -32,11 +32,11 @@ NOTE: Document- and field-level security disables the
==== Multiple roles with document and field level security ==== Multiple roles with document and field level security
A user can have many roles and each role can define different permissions on the A user can have many roles and each role can define different permissions on the
same index. It is important to understand the behavior of document and field same data stream or index. It is important to understand the behavior of document and field
level security in this scenario. level security in this scenario.
Document level security takes into account each role held by the user and Document level security takes into account each role held by the user and
combines each document level security query for a given index with an "OR". This combines each document level security query for a given data stream or index with an "OR". This
means that only one of the role queries must match for a document to be returned. means that only one of the role queries must match for a document to be returned.
For example, if a role grants access to an index without document level security For example, if a role grants access to an index without document level security
and another grants access with document level security, document level security and another grants access with document level security, document level security
@ -44,7 +44,7 @@ is not applied; the user with both roles has access to all of the documents in
the index. the index.
Field level security takes into account each role the user has and combines Field level security takes into account each role the user has and combines
all of the fields listed into a single set for each index. For example, if a all of the fields listed into a single set for each data stream or index. For example, if a
role grants access to an index without field level security and another grants role grants access to an index without field level security and another grants
access with field level security, field level security is not be applied for access with field level security, field level security is not be applied for
that index; the user with both roles has access to all of the fields in the that index; the user with both roles has access to all of the fields in the

View File

@ -8,11 +8,11 @@ read APIs.
To enable field level security, specify the fields that each role can access To enable field level security, specify the fields that each role can access
as part of the indices permissions in a role definition. Field level security is as part of the indices permissions in a role definition. Field level security is
thus bound to a well-defined set of indices (and potentially a set of thus bound to a well-defined set of data streams or indices (and potentially a set of
<<document-level-security, documents>>). <<document-level-security, documents>>).
The following role definition grants read access only to the `category`, The following role definition grants read access only to the `category`,
`@timestamp`, and `message` fields in all the `events-*` indices. `@timestamp`, and `message` fields in all the `events-*` data streams and indices.
[source,console] [source,console]
-------------------------------------------------- --------------------------------------------------
@ -162,7 +162,7 @@ An empty array for `grant` (for example, `"grant" : []`) means that access has
not been granted to any fields. not been granted to any fields.
When a user has several roles that specify field level permissions, the When a user has several roles that specify field level permissions, the
resulting field level permissions per index are the union of the individual role resulting field level permissions per data stream or index are the union of the individual role
permissions. For example, if these two roles are merged: permissions. For example, if these two roles are merged:
[source,console] [source,console]

View File

@ -55,14 +55,14 @@ The following describes the structure of an indices permissions entry:
------- -------
// NOTCONSOLE // NOTCONSOLE
<1> A list of indices (or index name patterns) to which the permissions in this <1> A list of data streams, indices, and index aliases to which the permissions
entry apply. in this entry apply. Wildcard (`*`) expressions are supported.
<2> The index level privileges the owners of the role have on the associated <2> The index level privileges the owners of the role have on the associated
indices (those indices that are specified in the `names` field) data streams and indices specified in the `names` argument.
<3> Specification for document fields the owners of the role have read access to. <3> Specification for document fields the owners of the role have read access to.
See <<field-and-document-access-control>> for details. See <<field-and-document-access-control>> for details.
<4> A search query that defines the documents the owners of the role have read <4> A search query that defines the documents the owners of the role have read
access to. A document within the associated indices must match this query access to. A document within the associated data streams and indices must match this query
in order for it to be accessible by the owners of the role. in order for it to be accessible by the owners of the role.
<5> Restricted indices are a special category of indices that are used <5> Restricted indices are a special category of indices that are used
internally to store configuration data. Only internal system internally to store configuration data. Only internal system
@ -75,8 +75,8 @@ The following describes the structure of an indices permissions entry:
[TIP] [TIP]
============================================================================== ==============================================================================
When specifying index names, you can use indices and aliases with their full The `names` parameter accepts wildcard and regular expressions that may refer to
names or regular expressions that refer to multiple indices. multiple data streams, indices, and index aliases.
* Wildcard (default) - simple wildcard matching where `*` is a placeholder * Wildcard (default) - simple wildcard matching where `*` is a placeholder
for zero or more characters, `?` is a placeholder for a single character for zero or more characters, `?` is a placeholder for a single character

View File

@ -34,7 +34,7 @@ see <<security-privileges>>.
_Permissions_:: _Permissions_::
A set of one or more privileges against a secured resource. Permissions can A set of one or more privileges against a secured resource. Permissions can
easily be described in words, here are few examples: easily be described in words, here are few examples:
* `read` privilege on the `products` index * `read` privilege on the `products` data stream or index
* `manage` privilege on the cluster * `manage` privilege on the cluster
* `run_as` privilege on `john` user * `run_as` privilege on `john` user
* `read` privilege on documents that match query X * `read` privilege on documents that match query X

View File

@ -12,7 +12,7 @@ role query for document level security. This is a situation where the
{ref}/ingest-node-set-security-user-processor.html[Set Security User Processor] ingest processor can help. {ref}/ingest-node-set-security-user-processor.html[Set Security User Processor] ingest processor can help.
NOTE: Document level security doesn't apply to write APIs. You must use unique NOTE: Document level security doesn't apply to write APIs. You must use unique
ids for each user that uses the same index, otherwise they might overwrite other ids for each user that uses the same data stream or index, otherwise they might overwrite other
users' documents. The ingest processor just adds properties for the current users' documents. The ingest processor just adds properties for the current
authenticated user to the documents that are being indexed. authenticated user to the documents that are being indexed.

View File

@ -2,21 +2,20 @@
==== {ccs-cap} and {kib} ==== {ccs-cap} and {kib}
When {kib} is used to search across multiple clusters, a two-step authorization When {kib} is used to search across multiple clusters, a two-step authorization
process determines whether or not the user can access indices on a remote process determines whether or not the user can access data streams and indices on a remote
cluster: cluster:
* First, the local cluster determines if the user is authorized to access remote * First, the local cluster determines if the user is authorized to access remote
clusters. (The local cluster is the cluster {kib} is connected to.) clusters. (The local cluster is the cluster {kib} is connected to.)
* If they are, the remote cluster then determines if the user has access * If they are, the remote cluster then determines if the user has access
to the specified indices. to the specified data streams and indices.
To grant {kib} users access to remote clusters, assign them a local role To grant {kib} users access to remote clusters, assign them a local role
with read privileges to indices on the remote clusters. You specify remote with read privileges to indices on the remote clusters. You specify data streams and indices in a remote cluster as `<remote_cluster_name>:<target>`.
cluster indices as `<remote_cluster_name>:<index_name>`.
To enable users to actually read the remote indices, you must create a matching To enable users to actually read the remote data streams and indices, you must create a matching
role on the remote clusters that grants the `read_cross_cluster` privilege role on the remote clusters that grants the `read_cross_cluster` privilege
and access to the appropriate indices. and access to the appropriate data streams and indices.
For example, if {kib} is connected to the cluster where you're actively For example, if {kib} is connected to the cluster where you're actively
indexing {ls} data (your _local cluster_) and you're periodically indexing {ls} data (your _local cluster_) and you're periodically

View File

@ -75,7 +75,8 @@ PUT _cluster/settings
==== Example Configuration of Cross Cluster Search ==== Example Configuration of Cross Cluster Search
In the following example, we will configure the user `alice` to have permissions In the following example, we will configure the user `alice` to have permissions
to search any index starting with `logs-` in cluster `two` from cluster `one`. to search any data stream or index starting with `logs-` in cluster `two` from
cluster `one`.
First, enable cluster `one` to perform cross cluster search on remote cluster First, enable cluster `one` to perform cross cluster search on remote cluster
`two` by running the following request as the superuser on cluster `one`: `two` by running the following request as the superuser on cluster `one`:

View File

@ -154,7 +154,7 @@ themselves, and run the `authenticate` API. If you want them to do more than
that, you need to give them one or more _roles_. that, you need to give them one or more _roles_.
Each role defines a specific set of actions (such as read, create, or delete) Each role defines a specific set of actions (such as read, create, or delete)
that can be performed on specific secured resources (such as indices, aliases, that can be performed on specific secured resources (such as data streams, indices, aliases,
documents, fields, or clusters). To help you get up and running, there are documents, fields, or clusters). To help you get up and running, there are
built-in roles. built-in roles.

View File

@ -18,18 +18,18 @@ source or not) and therefore we cannot guarantee their compliance with
officially supported on clusters with {security-features} enabled. officially supported on clusters with {security-features} enabled.
[float] [float]
=== Changes in index wildcard behavior === Changes in wildcard behavior
Elasticsearch clusters with the {security-features} enabled apply the `/_all` Elasticsearch clusters with the {security-features} enabled apply the `/_all`
wildcard, and all other wildcards, to the indices that the current user has wildcard, and all other wildcards, to the data streams, indices, and index aliases that the current user has
privileges for, not the set of all indices on the cluster. privileges for, not all data streams, indices, and index aliases on the cluster.
[float] [float]
=== Multi document APIs === Multi document APIs
Multi get and multi term vectors API throw IndexNotFoundException when trying to access non existing indices that the user is Multi get and multi term vectors API throw IndexNotFoundException when trying to access non existing indices that the user is
not authorized for. By doing that they leak information regarding the fact that the index doesn't exist, while the user is not not authorized for. By doing that they leak information regarding the fact that the data stream or index doesn't exist, while the user is not
authorized to know anything about those indices. authorized to know anything about those data streams or indices.
[float] [float]
=== Filtered index aliases === Filtered index aliases
@ -44,14 +44,14 @@ documents through the
[float] [float]
=== Field and document level security limitations === Field and document level security limitations
When a user's role enables document or field level security for an index: When a user's role enables document or field level security for a data stream or index:
* The user cannot perform write operations: * The user cannot perform write operations:
** The update API isn't supported. ** The update API isn't supported.
** Update requests included in bulk requests aren't supported. ** Update requests included in bulk requests aren't supported.
* The request cache is disabled for search requests. * The request cache is disabled for search requests.
When a user's role enables document level security for an index: When a user's role enables document level security for a data stream or index:
* Document level security isn't applied for APIs that aren't document based. * Document level security isn't applied for APIs that aren't document based.
An example is the field stats API. An example is the field stats API.