[DOCS] Create X-Pack installation and introduction for Elasticsearch Ref (elastic/x-pack-elasticsearch#1698)

* [DOCS] Create X-Pack installation and introduction for Elasticsearch Reference

* [DOCS] Address feedback in X-Pack install and intro info

* [DOCS] Add X-Pack setup pages

* [DOCS] Add ML settings to Elasticsearch

* [DOCS] Add table for X-Pack settings

* [DOCS]Add logstash settings

Original commit: elastic/x-pack-elasticsearch@65786cff78

docs/en/index.asciidoc

@@ -23,6 +23,12 @@
 
 include::{es-repo-dir}/Versions.asciidoc[]
 include::{es-repo-dir}/reference/index-shared1.asciidoc[]
+
+ifdef::include-xpack[]
+:edit_url!:
+include::setup-xes.asciidoc[]
+endif::include-xpack[]
+
 include::{es-repo-dir}/reference/index-shared2.asciidoc[]
 
 ifdef::include-xpack[]

docs/en/installing-xes.asciidoc

@@ -0,0 +1,182 @@
+[role="xpack"]
+[[installing-xpack-es]]
+== Installing X-Pack
+
+After you install {es}, you can optionally obtain and install {xpack}.
+For more information about how to obtain {xpack},
+see https://www.elastic.co/products/x-pack.
+
+You must run the version of {xpack} that matches the version of {es} you are running.
+
+IMPORTANT: If you are installing {xpack} for the first time on an existing
+cluster, you must perform a full cluster restart. Installing {xpack} enables
+security and security must be enabled on ALL nodes in a cluster for the cluster
+to operate correctly. When upgrading you can usually perform
+a {ref}/rolling-upgrades.html[rolling upgrade].
+
+To install {xpack} in {es}:
+
+. Run `bin/elasticsearch-plugin install` from `ES_HOME` on each node in your cluster:
++
+--
+[source,shell]
+----------------------------------------------------------
+bin/elasticsearch-plugin install x-pack
+----------------------------------------------------------
+//NOTCONSOLE
+
+NOTE: If you are using a <<xpack-package-installation, DEB/RPM distribution>>
+      of {es}, run the installation with superuser permissions. To
+      perform an offline installation, <<xpack-installing-offline, download the X-Pack binaries>>.
+
+--
+
+. Confirm that you want to grant {xpack} additional permissions.
++
+--
+TIP:  Specify the `--batch` option when running the install command to
+      automatically grant these permissions and bypass these install prompts.
+
+--
++
+  .. {xpack} needs these permissions to set the threat context loader during
+  install so {watcher} can send email notifications.
++
+--
+[source,shell]
+----------------------------------------------------------
+@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
+@     WARNING: plugin requires additional permissions     @
+@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
+* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
+* java.lang.RuntimePermission getClassLoader
+* java.lang.RuntimePermission setContextClassLoader
+* java.lang.RuntimePermission setFactory
+* java.security.SecurityPermission createPolicy.JavaPolicy
+* java.security.SecurityPermission getPolicy
+* java.security.SecurityPermission putProviderProperty.BC
+* java.security.SecurityPermission setPolicy
+* java.util.PropertyPermission * read,write
+* java.util.PropertyPermission sun.nio.ch.bugLevel write
+* javax.net.ssl.SSLPermission setHostnameVerifier
+See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
+for descriptions of what these permissions allow and the associated risks.
+
+Continue with installation? [y/N]y
+----------------------------------------------------------
+//NOTCONSOLE
+--
+  .. {xpack} requires permissions to enable {es} to launch the {ml} analytical
+  engine. The native controller ensures that the launched process is a valid
+  {ml} component. Once launched, communications between the {ml} processes and
+  {es} are limited to the operating system user that {es} runs as.
++
+--
+[source,shell]
+----------------------------------------------------------
+@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
+@        WARNING: plugin forks a native controller        @
+@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
+This plugin launches a native controller that is not subject to
+the Java security manager nor to system call filters.
+
+Continue with installation? [y/N]y
+----------------------------------------------------------
+//NOTCONSOLE
+--
+
+. If you have disabled automatic index creation in {es}, configure
+{ref}/docs-index_.html#index-creation[`action.auto_create_index`] in
+`elasticsearch.yml` to allow {xpack} to create the following indices:
++
+--
+[source,yaml]
+-----------------------------------------------------------
+action.auto_create_index: .security,.monitoring*,.watches,.triggered_watches,.watcher-history*
+-----------------------------------------------------------
+//NOTCONSOLE
+--
+
+. Start {es}.
++
+--
+[source,shell]
+----------------------------------------------------------
+bin/elasticsearch
+----------------------------------------------------------
+//NOTCONSOLE
+--
+
+For information about installing {xpack} in {kib} and Logstash, see
+{xpack-ref}/installing-xpack.html[Installing {xpack}].
+
+[IMPORTANT]
+=============================================================================
+SSL/TLS encryption is disabled by default, which means user credentials are
+passed in the clear. **Do not deploy to production without enabling encryption!**
+For more information, see {xpack-ref}/encrypting-communications.html[Encrypting
+Communications].
+
+You must also **change the passwords for the built-in `elastic` user and the
+`kibana` user that enables {kib} to communicate with {es} before
+deploying to production**. For more information,
+see {xpack-ref}/setting-up-authentication.html[Setting Up User Authentication].
+=============================================================================
+
+[float]
+[[xpack-package-installation]]
+=== Installing {xpack} on a DEB/RPM Package Installation
+
+If you use the DEB/RPM packages to install {es}, by default {es} is installed
+in `/usr/share/elasticsearch` and the configuration files are stored
+in `/etc/elasticsearch`. (For the complete list of default paths, see
+{ref}/deb.html#deb-layout[Debian Directory Layout] and
+{ref}/rpm.html#rpm-layout[RPM Directory Layout] in the {es} Reference.)
+
+To install {xpack} on a DEB/RPM package installation, you need to run
+`bin/plugin install` from the `/usr/share/elasticsearch` directory with superuser
+permissions:
+
+[source,shell]
+----------------------------------------------------------
+cd /usr/share/elasticsearch
+sudo bin/elasticsearch-plugin install x-pack
+----------------------------------------------------------
+//NOTCONSOLE
+
+NOTE: If the configuration files are not in `/etc/elasticsearch` you
+      need to specify the location of the configuration files by
+      setting the system property `es.path.conf` to the config path via
+      `ES_JAVA_OPTS="-Des.path.conf=<path>"` or by setting the
+      environment variable `CONF_DIR` via `CONF_DIR=<path>`.
+
+[float]
+[[xpack-installing-offline]]
+=== Installing {xpack} on Offline Machines
+The plugin install scripts require direct Internet access to download and
+install {xpack}. If your server doesn’t have Internet access, you
+can manually download and install {xpack}.
+
+To install {xpack} on a machine that doesn't have Internet access:
+
+. Manually download the {xpack} zip file:
+https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-{version}.zip[
++https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-{version}.zip+]
+(https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-{version}.zip.sha1[sha1])
+
+. Transfer the zip file to a temporary directory on the offline machine. (Do NOT
+put the file in the {es} plugins directory.)
+
+. Run `bin/elasticsearch-plugin install` from the {es} install directory
+and specify the location of the {xpack} zip file. For example:
++
+--
+["source","sh",subs="attributes"]
+----------------------------------------------------------
+bin/elasticsearch-plugin install file:///path/to/file/x-pack-{version}.zip
+----------------------------------------------------------
+//NOTCONSOLE
+
+NOTE:  You must specify an absolute path to the zip file after the `file://` protocol.
+
+--

docs/en/ml/configuring.asciidoc

@@ -4,7 +4,7 @@
 If you want to use {xpackml} features, there must be at least one {ml} node in
 your cluster and all master-eligible nodes must have {ml} enabled. By default,
 when you install {xpack}, all nodes are {ml} nodes. For more information about
-these settings, see <<ml-settings>>.
+these settings, see <<xpack-settings>>.
 
 To use the {xpackml} features to analyze your data, you must create a job and
 send your data to that job.

docs/en/ml/getting-started.asciidoc

@@ -59,7 +59,7 @@ When you install {xpack} into {es} and {kib}, the {ml} features are
 enabled by default. If you have multiple nodes in your cluster, you can
 optionally dedicate nodes to specific purposes. If you want to control which
 nodes are _machine learning nodes_ or limit which nodes run resource-intensive
-activity related to jobs, see <<ml-settings>>.
+activity related to jobs, see <<xpack-settings>>.
 
 
 [float]

docs/en/ml/overview.asciidoc

@@ -52,7 +52,7 @@ A {ml} node is a node that has `xpack.ml.enabled` and `node.ml` set to `true`,
 which is the default behavior. If you set `node.ml` to `false`, the node can
 service API requests but it cannot run jobs. If you want to use {xpackml}
 features, there must be at least one {ml} node in your cluster. For more
-information about this setting, see <<ml-settings>>.
+information about this setting, see <<xpack-settings>>.
 
 [float]
 [[ml-function-overview]]

docs/en/node.asciidoc

@@ -96,8 +96,7 @@ If you want to use {xpackml} features in your cluster, you must enable {ml}
 (set `xpack.ml.enabled` to `true`) on all master-eligible nodes. Do not use
 these settings if you do not have {xpack} installed.
 
-For more information about these settings,
+For more information about these settings, see <<ml-settings>>.
-see {xpack-ref}/ml-settings.html[Machine Learning Settings].
 
 To create a dedicated {ml} node, set:
 

docs/en/settings/configuring-xes.asciidoc

@@ -0,0 +1,23 @@
+[role="xpack"]
+[[settings-xpack]]
+== Configuring X-Pack
+
+You can configure {es} settings for {xpack} features in the `elasticsearch.yml`
+file.
+
+If you are using {kib}, there are also settings in the `kibana.yml` file. See
+{kibana}/settings.html[Configuring {kib}].
+
+//TODO: Add link to "Configuring XPack" in Kibana Reference.
+
+The following settings pertain to specific {xpack} features:
+
+* <<ml-settings,Machine Learning Settings>>
+* {xpack-ref}/monitoring-settings.html[Monitoring Settings]
+* {xpack-ref}/security-settings.html[Security Settings]
+* {xpack-ref}/notification-settings.html[Watcher Settings]
+
+For more information, see <<settings>> and
+{xpack-ref}/xpack-settings.html[{xpack} Settings].
+
+include::ml-settings.asciidoc[]

docs/en/settings/index.asciidoc

@@ -6,11 +6,23 @@
 You configure settings for X-Pack features in the `elasticsearch.yml` and `kibana.yml`
 configuration files.
 
-* <<security-settings, Security Settings>>
+[options="header,footer"]
-* <<monitoring-settings, Monitoring Settings>>
+|=======================
-* <<notification-settings, Watcher Settings>>
+|{xpack} Feature  |{es} Settings                  |{kib} Settings              |Logstash Settings
-* <<reporting-settings, Reporting Settings>>
+|Graph            |No                             |<<graph-settings,Yes>>      |No
-* <<ml-settings, Machine Learning Settings>>
+//{kib-ref}/settings-xpack-kb.html[Yes]
+|Machine learning |{ref}/settings-xpack.html[Yes] |Yes                         |No
+//{kib-ref}/settings-xpack-kb.html[Yes]
+|Monitoring       |<<monitoring-settings,Yes>>    |<<monitoring-settings,Yes>> |<<monitoring-settings,Yes>>
+//{ref}/settings-xpack.html[Yes] {kib-ref}/settings-xpack-kb.html[Yes]
+|Reporting        |No                             |<<reporting-settings,Yes>>  |No
+//{kib-ref}/settings-xpack-kb.html[Yes]
+|Security         |<<security-settings,Yes>>      |<<security-settings,Yes>>   |No
+//{ref}/settings-xpack.html[Yes] {kib-ref}/settings-xpack-kb.html[Yes]
+|Watcher          |<<notification-settings,Yes>>  |No                          |No
+//{ref}/settings-xpack.html[Yes] No
+|=======================
+
 --
 
 include::security-settings.asciidoc[]
@@ -18,4 +30,4 @@ include::monitoring-settings.asciidoc[]
 include::graph-settings.asciidoc[]
 include::notification-settings.asciidoc[]
 include::reporting-settings.asciidoc[]
-include::ml-settings.asciidoc[]
+//include::ml-settings.asciidoc[]

docs/en/settings/ml-settings.asciidoc

@@ -1,18 +1,18 @@
+[role="xpack"]
 [[ml-settings]]
-== Machine Learning Settings
+=== Machine Learning Settings
 You do not need to configure any settings to use {ml}. It is enabled by default.
 
 [float]
 [[general-ml-settings]]
-=== General Machine Learning Settings
+==== General Machine Learning Settings
 
 `xpack.ml.enabled`::
 Set to `true` (default) to enable {ml}. +
 +
 If set to `false` in `elasticsearch.yml`, the {ml} APIs are disabled.
 You also cannot open jobs, start {dfeeds}, or receive transport (internal)
-communication requests related to {ml} APIs.
+communication requests related to {ml} APIs. +
-If set to `false` in `kibana.yml`, the {ml} icon is not visible in {kib}. +
 +
 IMPORTANT: If you want to use {ml} features in your cluster, you must have
 `xpack.ml.enabled` set to `true` on all master-eligible nodes. This is the

docs/en/setup-xes.asciidoc

@@ -0,0 +1,14 @@
+[role="xpack"]
+[[setup-xpack]]
+= Setup X-Pack
+
+[partintro]
+--
+{xpack} is an Elastic Stack extension that bundles security, alerting,
+monitoring, reporting, machine learning, and graph capabilities into one
+easy-to-install package. To access this functionality, you must
+<<installing-xpack-es,install {xpack} in {es}>>.
+--
+
+include::installing-xes.asciidoc[]
+include::settings/configuring-xes.asciidoc[]