Shield Docs: Updated Marvel config and license management for 2.0.
Original commit: elastic/x-pack-elasticsearch@3d225c2562
This commit is contained in:
debadair
parent
aeffd696ff
commit
0123d33e87
|
|
@ -12,6 +12,6 @@ include::configuring-clients-integrations/hadoop.asciidoc[]
|
|||
|
||||
include::configuring-clients-integrations/logstash.asciidoc[]
|
||||
|
||||
include::configuring-clients-integrations/marvel.asciidoc[]
|
||||
|
||||
include::configuring-clients-integrations/kibana.asciidoc[]
|
||||
|
||||
include::configuring-clients-integrations/marvel.asciidoc[]
|
||||
|
|
@ -96,16 +96,6 @@ kibana3_monitoring:
|
|||
- "cn=Jack Nicholson,dc=example,dc=com"
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
** If you are using a PKI realm, you assign roles on a per user basis in the role mapping file.
|
||||
For example, the following snippet assigns the `kibana3_monitoring` role to the users named `Jack Nicholson` and `Robert De Niro.
|
||||
+
|
||||
[source,yaml]
|
||||
--------------------------------------------------------------------------------
|
||||
kibana3_monitoring:
|
||||
- "cn=Jack Nicholson,ou=example,o=com"
|
||||
- "cn=Robert De Niro,ou=example,o=com"
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
. If you have <<ssl-tls, enabled SSL encryption>> in Shield and are using your own Certificate Authority (CA) to sign certificates for your nodes, configure your browser or operating system to trust your CA. When you access Kibana, your browser verifies that the certificate received from the Elasticsearch node is trusted before sending a request to the node. Establishing this trust requires that either your browser or operating system trust the CA that signed the node's certificate. Consult your local IT professional for information about the recommended procedure for adding trusted CAs in your organization.
|
||||
|
||||
. Access Kibana 3 from your browser and verify that you can sign in as a user. For example, you could log in as the `jacknich` user created in step 4.
|
||||
|
|
@ -217,16 +207,6 @@ kibana4_monitoring:
|
|||
- "cn=Jack Nicholson,dc=example,dc=com"
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
** If you are using a PKI realm, you assign roles on a per user basis in the role mapping file.
|
||||
For example, the following snippet assigns the `kibana4_monitoring` role to the users named `Jack Nicholson` and `Robert De Niro`.
|
||||
+
|
||||
[source,yaml]
|
||||
--------------------------------------------------------------------------------
|
||||
kibana4_monitoring:
|
||||
- "cn=Jack Nicholson,ou=example,o=com"
|
||||
- "cn=Robert De Niro,ou=example,o=com"
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
. If you have enabled SSL encryption in Shield, configure Kibana 4 to connect to Elasticsearch via HTTPS. To do this:
|
||||
|
||||
.. Specify the HTTPS protocol in the `elasticsearch.url` setting in the Kibana configuration file, `kibana.yml`:
|
||||
|
|
|
|||
|
|
@ -1,124 +1,115 @@
|
|||
[[marvel]]
|
||||
=== Using Marvel with Shield
|
||||
|
||||
Marvel consists of a user interface over a data exporter known as the _agent_. The agent runs on each node and accesses
|
||||
that node's monitoring API. The agent can store this collected data locally, on the cluster, or send the data to an
|
||||
external monitoring cluster. Users can view and analyze the collected data with the Marvel UI.
|
||||
https://www.elastic.co/guide/en/marvel/current/introduction.html[Marvel] consists of two
|
||||
components: a Marvel agent that you install on on each node in your cluster, and a Marvel application you install in https://www.elastic.co/guide/en/kibana/current/introduction.html[Kibana]. The Marvel agent collects and indexes metrics from Elasticsearch and you visualize the data through the Marvel dashboards in Kibana. The agent can index data on the same cluster, or send it to an external monitoring cluster.
|
||||
|
||||
To work with the Shield plugin, Marvel's configuration needs to be adapted for the _production_ cluster, which is the
|
||||
cluster being monitored, as well as the _monitoring_ cluster, where the monitoring data is stored. For clusters that
|
||||
store their own monitoring data, apply both sets of settings to the single, production cluster.
|
||||
|
||||
You will configure at least two users to work with Marvel. These users have to exist on the monitoring cluster.
|
||||
|
||||
* The Agent needs to be assigned a user with the correct <<roles,privileges>> to write data to the Marvel indices
|
||||
named `.marvel-*`, check the Marvel index template, and upload the Marvel index template. You need only one agent user.
|
||||
* Marvel UI users must authenticate and have privileges to read data from the Marvel indices. These users also
|
||||
need to able to call the Nodes Info API in order to get the monitoring cluster's Elasticsearch version.
|
||||
This version check allows Marvel to be compatible with many versions of Elasticsearch. You can have as many of
|
||||
these end users configured as you would like.
|
||||
|
||||
The default `roles.yml` file includes definitions for these two roles. The steps below show you how to create these
|
||||
users on the monitoring cluster.
|
||||
To use Marvel with Shield enabled, you need to <<kibana, set up Kibana to work with Shield>> and create at least one user for the Marvel app. If you are using an external monitoring cluster, you also need to configure a user for the Marvel agent and configure the agent to use the appropriate
|
||||
credentials when communicating with the monitoring cluster.
|
||||
|
||||
[float]
|
||||
[[monitoring-cluster]]
|
||||
==== Monitoring Cluster Settings
|
||||
[[marvel-app-users]]
|
||||
==== Setting Up Marvel App Users
|
||||
|
||||
The monitoring cluster is used to both store and view the Marvel data. When configuring Shield, you need to perform the
|
||||
following actions:
|
||||
When Shield is enabled, Kibana users are prompted to log in when they access the UI. To use
|
||||
the Marvel app, a user must have access to the Kibana indices and permission to read from the
|
||||
Marvel indices.
|
||||
|
||||
* Make sure there is a user created with the `marvel_agent` role. Marvel uses this to export the data.
|
||||
* Make sure there is a user created with the `marvel_user` role. You use this to view the Marvel UI and get license information.
|
||||
* When using Marvel on a production server, you must enter your Marvel License. This license is stored in the
|
||||
monitoring cluster. This step needs to be performed once, by a user with permissions to write to the `.marvel-kibana`
|
||||
index. The .marvel-kibana index is used to store Marvel UI settings (for example, set custom warning levels) and
|
||||
therefore write permission for `.marvel-kibana` is required for UI customizations. Both storing license and storing
|
||||
settings can be done by any user added to the marvel_user role.
|
||||
You set up Marvel app users on the cluster where the monitoring data is being stored. To grant
|
||||
all of the necessary permissions, assign user the `kibana_user` and `marvel_user` roles defined
|
||||
in `roles.yml`:
|
||||
|
||||
This is in the default `roles.yml`
|
||||
* If you're using the default `esusers` realm, you can assign roles when you <<esusers-add, add a
|
||||
user>>, or modify the role assignments with the <<esusers-roles, `roles`>> command. For example,
|
||||
the following command creates a user named `jacknich` and assigns the `marvel_user` role:
|
||||
+
|
||||
[source,console]
|
||||
--------------------------------------------------------------------------------
|
||||
esusers useradd jacknich -r kibana_user,marvel_user -p password
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
* If you are using an LDAP or Active Directory realm, you can either assign roles on a per user
|
||||
basis, or assign roles to groups of users. By default, role mappings are configured in
|
||||
<<mapping-roles, `config/shield/role_mapping.yml`>>. For example, the following snippet assigns
|
||||
the user named Jack Nicholson to the `kibana_user` and `marvel_user` roles:
|
||||
+
|
||||
[source,yaml]
|
||||
--------------------------------------------------------------------------------
|
||||
kibana_user:
|
||||
- "cn=Jack Nicholson,dc=example,dc=com"
|
||||
marvel_user:
|
||||
- "cn=Jack Nicholson,dc=example,dc=com"
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
[float]
|
||||
[[configuring-marvel-agent-shield]]
|
||||
==== Configuring Marvel Agent to Communicate with a Shield-Enabled Monitoring Cluster
|
||||
|
||||
To configure the Marvel agent to communicate with a secured monitoring cluster:
|
||||
|
||||
. Configure a user on the monitoring cluster who has the `marvel_agent` role, which is defined in
|
||||
`roles.yml`. For example:
|
||||
+
|
||||
[source,console]
|
||||
--------------------------------------------------------------------------------
|
||||
esusers useradd agent-user -r marvel_agent -p password
|
||||
--------------------------------------------------------------------------------
|
||||
+
|
||||
.Marvel Agent Role
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
marvel_agent:
|
||||
cluster: indices:admin/template/get, indices:admin/template/put
|
||||
indices:
|
||||
'.marvel-*': indices:data/write/bulk, create_index
|
||||
|
||||
marvel_user:
|
||||
cluster: cluster:monitor/nodes/info, cluster:admin/plugin/license/get
|
||||
indices:
|
||||
'.marvel-*': all
|
||||
--------------------------------------------------
|
||||
|
||||
Once the roles are configured, create a user for the agent:
|
||||
. On each node in the cluster being monitored, configure a Marvel HTTP exporter
|
||||
in `elasticsearch.yml` and resart Elasticsearch. In the exporter configuration,
|
||||
you need to:
|
||||
+
|
||||
--
|
||||
.. Set the `type` to `http`.
|
||||
.. Specify the location of the monitoring cluster in the `host` setting.
|
||||
.. Provide the agent user credentials with the `username` and `password` settings.
|
||||
|
||||
[source,shell]
|
||||
--------------------------------------------------
|
||||
bin/shield/esusers useradd marvel_export -p strongpassword -r marvel_agent
|
||||
--------------------------------------------------
|
||||
|
||||
Then create one or more users for the Marvel UI:
|
||||
|
||||
[source,shell]
|
||||
--------------------------------------------------
|
||||
bin/shield/esusers useradd USER -p strongerpassword -r marvel_user
|
||||
--------------------------------------------------
|
||||
|
||||
[float]
|
||||
==== Production Cluster Settings
|
||||
|
||||
The Marvel agent is installed on every node in the production cluster. The agent collects monitoring data from the
|
||||
production cluster and stores the data on the monitoring cluster. The agent's configuration specifies a list of
|
||||
hostname and port combinations for access to the monitoring cluster.
|
||||
|
||||
When the monitoring cluster uses the Shield plugin and is configured to accept only HTTPS requests, you must configure the agent
|
||||
on the production cluster to use HTTPS instead of the default HTTP protocol.
|
||||
|
||||
Authentication and protocol configuration are both controlled by the `marvel.agent.exporter.es.hosts` setting in the
|
||||
node's `elasticsearch.yml` file. The setting accepts a list of monitoring cluster servers to serve as a fallback
|
||||
in case a server is unavailable. Each of these servers must be properly configured, as in the following example:
|
||||
|
||||
Example `marvel.agent.exporter.es.hosts` setting
|
||||
[source,yaml]
|
||||
-------------------------------------------------------------------------------------------------------------------
|
||||
marvel.agent.exporter.es.hosts: [ "https<1>://USER:PASSWORD<2>@node01:9200", "https://USER:PASSWORD@node02:9200"]
|
||||
-------------------------------------------------------------------------------------------------------------------
|
||||
|
||||
<1> Indicates to use HTTPS.
|
||||
<2> Username and password. The user needs to be configured on the Monitoring Cluster as described in the next section.
|
||||
|
||||
When the monitoring cluster uses HTTPS, the Marvel agent will attempt to validate the certificate of the Elasticsearch
|
||||
node in the monitoring cluster. If you are using your own CA you should specify a trust store that has the signing
|
||||
certificate of the CA. Here is an example config for the `marvel.agent.exporter.es.truststore.*` settings:
|
||||
For example:
|
||||
|
||||
[source,yaml]
|
||||
-------------------------------------------------------------------------------------------------------------
|
||||
marvel.agent.exporter.es.hosts: [ "https://USER:PASSWORD@node01:9200", "https://USER:PASSWORD@node02:9200"]
|
||||
marvel.agent.exporter.es.ssl.truststore.path: FULL_FILE_PATH
|
||||
marvel.agent.exporter.es.ssl.truststore.password: PASSWORD
|
||||
-------------------------------------------------------------------------------------------------------------
|
||||
--------------------------------------------------
|
||||
marvel.agent.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: [http://es-mon1:9200, http://es-mon2:9200]
|
||||
auth:
|
||||
username: agent-user
|
||||
password: password
|
||||
--------------------------------------------------
|
||||
|
||||
See the http://www.elastic.co/guide/en/marvel/current/configuration.html[Marvel documentation] for more details about
|
||||
other SSL related settings.
|
||||
If SSL/TLS is enabled on the monitoring cluster:
|
||||
|
||||
NOTE: The 1.3.0 release of Marvel adds HTTPS support.
|
||||
.. Specify the HTTPS protocol when setting the monitoring server host.
|
||||
.. Specify a truststore that contains the CA certificate to use to verify the identities of the
|
||||
nodes in the monitoring cluster. You need to set the `truststore.path` and
|
||||
`truststore.password`.
|
||||
|
||||
[float]
|
||||
==== Marvel user interface & Sense
|
||||
For example:
|
||||
|
||||
The Marvel UI supports SSL without the need for any additional configuration. You can change URL access scheme for Marvel to
|
||||
HTTPS.
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
marvel.agent.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: [ https://es-mon1:9200, https://es-mon2:9200]
|
||||
|
||||
Users attempting to access the Marvel UI with the URL `https://HOST:9200/_plugin/marvel` must provide valid
|
||||
credentials. See <<monitoring-cluster,Monitoring Cluster settings>> for information on the required user configuration.
|
||||
auth:
|
||||
username: agent-user
|
||||
password: password
|
||||
|
||||
Sense also supports HTTPS access. Users that access Sense over URLs of the form
|
||||
`https://host:9200/_plugin/marvel/sense/index.html` must provide valid credentials if they have not already
|
||||
authenticated to a dashboard.
|
||||
|
||||
Users connecting to the production cluster with Sense must provide valid credentials. Clusters must be configured to
|
||||
enable cross-origin requests to enable users to connect with Sense. See the <<cors, CORS>> documentation for details.
|
||||
|
||||
NOTE: Providing user credentials to Sense in order to access another cluster is only supported in releases 1.3.0 and
|
||||
later of Marvel.
|
||||
ssl:
|
||||
truststore.path: /path/to/file
|
||||
truststore.password: password
|
||||
id2:
|
||||
type: local
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
|
@ -1,109 +1,114 @@
|
|||
[[license-management]]
|
||||
== Managing Shield Licenses
|
||||
== Managing Your License
|
||||
|
||||
When installing Shield for the first time, having the license plugin installed is the minimum required for Shield to work.
|
||||
You can just start up the node and everything will just work as expected. The first time you start up the node, a 30 days
|
||||
trial license will automatically be created which will enable Shield to be fully operational. Within these 30 days, you
|
||||
will be able to replace the trial license with another one that will be provided to you up on purchase.
|
||||
When you initially install Shield, a 30 day trial license is installed that allows access to all features. At the end of the trial period, you can https://www.elastic.co/subscriptions/shield[purchase a subscription] to keep using the full functionality of Shield along with Marvel and Watcher.
|
||||
|
||||
IMPORTANT: When your license expires, Shield operates in a degraded mode where access to the Elasticsearch cluster health, cluster stats, and index stats APIs is blocked. Shield keeps on protecting your cluster, but you won't be able to monitor its operation until you update your license. For more information, see <<license-expiration, License Expiration>>.
|
||||
|
||||
[float]
|
||||
[[installing-license]]
|
||||
=== Updating Your Shield License
|
||||
Updating the
|
||||
license can be done at runtime (no need to shutdown the nodes) using a dedicated API.
|
||||
=== Updating Your License
|
||||
|
||||
IMPORTANT: With a valid license, Shield will be fully operational. Upon license expiry, Shield will operate in a
|
||||
degraded mode, where cluster health, cluster stats, and index stats APIs will be blocked. All other operations will
|
||||
continue operating normally. Find out more about <<license-expiration, Shield license expiration>>.
|
||||
You can update your license at runtime without shutting down your nodes. License updates take
|
||||
effect immediately. The license is provided as a _JSON_ file that you install with the `license`
|
||||
API. You need cluster admin privileges to install the license.
|
||||
|
||||
To update your license:
|
||||
|
||||
The license itself is a _JSON_ file containing all information about the license (e.g. feature name, expiry date, etc...).
|
||||
To install or update the license use the following REST API:
|
||||
|
||||
. Send a request to the `license` API and specify the file that contains your new license:
|
||||
+
|
||||
[source,shell]
|
||||
-----------------------------------------------------------------------
|
||||
curl -XPUT -u admin 'http://<host>:<port>/_license' -d @license.json
|
||||
-----------------------------------------------------------------------
|
||||
|
||||
+
|
||||
Where:
|
||||
|
||||
+
|
||||
* `<host>` is the hostname of the Elasticsearch node (`localhost` if executing locally)
|
||||
* `<port>` is the http port (defaults to `9200`)
|
||||
* `license.json` is the license json file
|
||||
* `license.json` is the license JSON file
|
||||
|
||||
NOTE: The put license API is protected under the cluster admin privilege, therefore it has to be executed
|
||||
by a user with the appropriate permissions.
|
||||
. If the license you are installing does not support all of the features available with your
|
||||
previous license, you will be notified in the response. To complete the license installation,
|
||||
you must resubmit the license update request and set the `acknowledge` parameter to `true` to
|
||||
indicate that you are aware of the changes.
|
||||
+
|
||||
[source,shell]
|
||||
-----------------------------------------------------------------------
|
||||
curl -XPUT -u admin 'http://<host>:<port>/_license?acknowledge=true' -d @license.json
|
||||
-----------------------------------------------------------------------
|
||||
|
||||
[float]
|
||||
[[listing-licenses]]
|
||||
=== Listing Installed Licenses
|
||||
=== Viewing the Installed License
|
||||
|
||||
You can list all currently installed licenses by executing the following REST API:
|
||||
You can also use the `license` API to retrieve the currently installed license:
|
||||
|
||||
[source,shell]
|
||||
-----------------------------------------------------
|
||||
curl -XGET -u admin:password 'http://<host>:<port>/_license'
|
||||
-----------------------------------------------------
|
||||
|
||||
The response of this command will be a JSON listing all available licenses. In the case of Shield, the following
|
||||
entry will be shown:
|
||||
|
||||
[source,json]
|
||||
--------------------------------------------
|
||||
{
|
||||
licenses: [
|
||||
...
|
||||
{
|
||||
status: "active",
|
||||
uid: "sample_uid",
|
||||
type: "sample_type",
|
||||
subscription_type: "sample_subscription_type",
|
||||
"issue_date" : "2015-01-26T00:00:00.000Z",
|
||||
"issue_date_in_millis" : 1422230400000,
|
||||
feature: "shield",
|
||||
"expiry_date" : "2015-04-26T23:59:59.999Z",
|
||||
"expiry_date_in_millis" : 1430092799999,
|
||||
max_nodes: 1,
|
||||
issued_to: "sample customer",
|
||||
issuer: "elasticsearch"
|
||||
"license" : {
|
||||
"status" : "active",
|
||||
"uid" : "0a98411f-73f4-4c67-954c-724874ed5488",
|
||||
"type" : "trial",
|
||||
"issue_date" : "2015-10-13T18:18:20.709Z",
|
||||
"issue_date_in_millis" : 1444760300709,
|
||||
"expiry_date" : "2015-11-12T18:18:20.709Z",
|
||||
"expiry_date_in_millis" : 1447352300709,
|
||||
"max_nodes" : 1000,
|
||||
"issued_to" : "elasticsearch",
|
||||
"issuer" : "elasticsearch"
|
||||
}
|
||||
...
|
||||
]
|
||||
}
|
||||
--------------------------------------------
|
||||
|
||||
NOTE: The get license API is protected under the cluster admin privilege, therefore it has to be executed
|
||||
by a user with the appropriate permissions.
|
||||
NOTE: You need cluster admin privileges to retrieve the license.
|
||||
|
||||
[float]
|
||||
[[license-expiration]]
|
||||
=== License Expiration
|
||||
|
||||
License expiration should never be a surprise. Beginning 30 days from license expiration, Shield will begin logging daily messages
|
||||
containing the license expiration date and a brief description of unlicensed behavior. Beginning 7 days from license expiration,
|
||||
Shield will begin logging error messages every 10 minutes with the same information. After expiration, Shield will continue to
|
||||
log error messages informing you that the license has expired. These messages will also be generated at node startup, to ensure
|
||||
that there are no surprises. Here is an example message:
|
||||
License expiration should never be a surprise. If you're using Marvel, a license expiration
|
||||
warning is displayed prominently if your license expires within 30 days. Warnings are
|
||||
also displayed on startup and written to the Elasticsearch log starting 30 days from the expiration date. These error messages tell you when the license expires and what features will be disabled if
|
||||
you fail to update it:
|
||||
|
||||
[source,sh]
|
||||
---------------------------------------------------------------------------------------------------------------------------------
|
||||
[ERROR][shield.license] Shield license will expire on 1/1/1970. Cluster health, cluster stats and indices stats operations are
|
||||
blocked on Shield license expiration. All data operations (read and write) continue to work. If you have a new license, please
|
||||
update it. Otherwise, please reach out to your support contact.
|
||||
---------------------------------------------------------------------------------------------------------------------------------
|
||||
--------------------------------------------
|
||||
# License will expire on [Thursday, November 12, 2015]. If you have a new license, please update it.
|
||||
# Otherwise, please reach out to your support contact.
|
||||
#
|
||||
# Commercial plugins operate with reduced functionality on license expiration:
|
||||
# - marvel
|
||||
# - The agent will stop collecting cluster and indices metrics
|
||||
# - shield
|
||||
# - Cluster health, cluster stats and indices stats operations are blocked
|
||||
# - All data operations (read and write) continue to work
|
||||
--------------------------------------------
|
||||
|
||||
When the license for Shield is expired, Shield will block requests to the cluster health, cluster stats, and index stats APIs.
|
||||
Calls to these APIs will fail with a LicenseExpiredException, and will return HTTP status code 401. By disabling only these APIs,
|
||||
any automated cluster monitoring should detect the license failure, while users of the cluster should not be immediately impacted.
|
||||
It is not recommended to run for any length of time with a disabled Shield license; cluster health and stats APIs are critical
|
||||
for monitoring and management of an Elasticsearch cluster.
|
||||
Once the license expires, calls to the cluster health, cluster stats, and index stats APIs
|
||||
fail with a `ElasticsearchSecurityException` and return a 401 HTTP status code:
|
||||
|
||||
Example error response the clients will receive when license is expired and cluster health, cluster stats or index stats APIs are called:
|
||||
[source,sh]
|
||||
--------------------------------------------
|
||||
{
|
||||
"error": {
|
||||
"root_cause": [{
|
||||
"type": "security_exception",
|
||||
"reason": "current license is non-compliant for [shield]",
|
||||
"license.expired.feature": "shield"
|
||||
}],
|
||||
"type": "security_exception",
|
||||
"reason": "current license is non-compliant for [shield]",
|
||||
"license.expired.feature": "shield"
|
||||
},
|
||||
"status": 401
|
||||
}
|
||||
--------------------------------------------
|
||||
|
||||
[source,json]
|
||||
----------------------------------------------------------------------------------------------------------------------------------------------
|
||||
{"error":"LicenseExpiredException[license expired for feature [shield]]","status":401}
|
||||
----------------------------------------------------------------------------------------------------------------------------------------------
|
||||
This enables automatic monitoring systems to easily detect the license failure without immediately impacting other users.
|
||||
|
||||
IMPORTANT: You should update your license as soon as possible. You're essentially flying blind when running with an expired license. Access to the cluster health and stats APIs is critical
|
||||
for monitoring and managing an Elasticsearch cluster.
|
||||
|
||||
If you receive a new license file and <<installing-license, install it>>, it will take effect immediately and the health and
|
||||
stats APIs will be available.
|
||||
|
|
|
|||
Loading…
Reference in New Issue