From 5809bbdb5de5cd9a900b52b608ac9bbfdb670e5d Mon Sep 17 00:00:00 2001 From: debadair Date: Wed, 18 Nov 2015 17:46:21 -0800 Subject: [PATCH] Docs: Deleting unused Marvel doc files. Marvel docs are currently in elasticsearch-marvel/docs. Original commit: elastic/x-pack-elasticsearch@be7a7e2243ef865ef72134a910eb4b5f192c8ce3 --- marvel/docs/configuring-marvel.asciidoc | 172 ---------------- marvel/docs/getting-started.asciidoc | 47 ----- marvel/docs/index.asciidoc | 17 -- marvel/docs/installing-marvel.asciidoc | 141 ------------- marvel/docs/introduction.asciidoc | 23 --- marvel/docs/release-notes.asciidoc | 252 ------------------------ 6 files changed, 652 deletions(-) delete mode 100644 marvel/docs/configuring-marvel.asciidoc delete mode 100644 marvel/docs/getting-started.asciidoc delete mode 100644 marvel/docs/index.asciidoc delete mode 100644 marvel/docs/installing-marvel.asciidoc delete mode 100644 marvel/docs/introduction.asciidoc delete mode 100644 marvel/docs/release-notes.asciidoc diff --git a/marvel/docs/configuring-marvel.asciidoc b/marvel/docs/configuring-marvel.asciidoc deleted file mode 100644 index dc4882c0332..00000000000 --- a/marvel/docs/configuring-marvel.asciidoc +++ /dev/null @@ -1,172 +0,0 @@ -[[configuration]] -== Configuring Marvel - -[float] -[[stats-export]] -=== Controlling Marvel Data Collection - -You can set `marvel.agent` options in a node's `elasticsearch.yml` file to control how Marvel data -is collected from the node. - -`marvel.agent.enabled`:: - -Controls whether or not data is collected from the node. Enabled by default. Set -`marvel.agent.enabled` to `false` to disable data collection. Use to disable data collection -on the monitoring nodes when you use a separate monitoring cluster. - -`marvel.agent.exporter.es.hosts`:: - -Specifies where the collected Marvel data should be stored. Defaults to the address of the -local node (typically `localhost:9200`). To send data to a monitoring cluster, -set to a comma-seprarated list of nodes in `hostname:port` format . Marvel attempts to send data -to the hosts in the order they are listed--if the first node is unreachable, the second node is -tried, and so on. added[1.3.0, before the default was `localhost:9200`] -+ -You can update this setting through the Cluster Update Settings API. -+ -added[1.0.2] - HTTP Basic authentication credentials can be specified as part of the host name, - i.e., ["user:pwd@host:9200"] -+ -added[1.3.0] - next to the host and port, you can specify a protocol to use, - i.e., ["https://host:9200"] (defaults to "http") -+ -added[1.3.0] - the `hostname:port` part can be extended with a base path - i.e., ["host:9200/monitor1"] - -`marvel.agent.indices`:: - -Controls which indices Marvel collects data for. Defaults to all indices. Specify the index names -as a comma-separated list, for example `test1,test2,test3`. Names can include wildcards, for -example `test*`. You can explicitly include or exclude indices by prepending -`+` to include the index, or `-` to exclude the index. For example, to include all indices that -start with `test` except `test3`, you could specify `+test*,-test3`. -+ -You can update this setting through the Cluster Update Settings API. - -`marvel.agent.interval`:: - -Controls how often data samples are collected from the node. Defaults to `10s`. Set to -`-1` to temporarily disable data collection. -+ -You can update this setting through the Cluster Update Settings API. - -`marvel.agent.exporter.es.index.timeformat`:: - -Controls the time component in the index name to which data is exported. -Defaults to `"YYYY.MM.dd"`, which produces index names such as -`.marvel-2015.08.23`. Supports date formats as explained -http://joda-time.sourceforge.net/api-release/org/joda/time/format/DateTimeFormat.html[here]. - -`marvel.agent.exporter.es.timeout`:: - -Sets the connection timeout for sending data. Defaults to `6s`. -+ -You can update this setting through the Cluster Update Settings API. - -`marvel.agent.stats.timeout`:: - -added[1.3.1] - -Sets the timeout when collecting statistics from the master node. Defaults to `10m`. - -`marvel.agent.exporter.es.ssl.truststore.path`:: - -added[1.3.0] - -The location of the truststore to use for HTTPS connections. Specified as an absolute path. - -`marvel.agent.exporter.es.ssl.truststore.password`:: - -The password to use to access the truststore. - -`marvel.agent.exporter.es.ssl.truststore.algorithm`:: - -The truststore format. Defaults to SunX509. - -`marvel.agent.exporter.es.ssl.hostname_verification`:: -+ -added[1.3.1] - -Controls whether or not the hostname is verified when using HTTPS. Set to `false` to disable -hostname verification when sending data to a remote host. -+ -You can update this setting through the Cluster Update Settings API. - -[float] -[[marvel-indices]] -=== Configuring Marvel Indices - -Marvel stores it's data using time-based indices. By default, Marvel generates -an index per day, with one shard and one replica. We expect this to be a good -default for most use cases. For very large clusters, you might need to make changes -by overriding the settings in the default Marvel index template. - -[[config-marvel-indices]] - -Marvel uses an {ref}/indices-templates.html[index template] to preconfigure newly created indices. -You can retrieve it with: - -[source,sh] ----------------------------------- -GET /_template/marvel ----------------------------------- - -You can override the default settings by adding your own template. Make sure your template uses -the `.marvel-*` matching pattern and has an order of 1 or higher. For example, the following -template increases the number of shards to 5: - -[source,json] ----------------------------------- -PUT /_template/custom_marvel -{ - "template": ".marvel*", - "order": 1, - "settings": { - "number_of_shards": 5 - } -} ----------------------------------- - -IMPORTANT: We recommend only changing the `settings` section. Other sections are -important for the correct operation of the dashboards. - -For reference, the following snippet shows the `settings` section of the default template. - -[source,json] ----------------------------------- -{ - "template": ".marvel*", - "settings": { - "number_of_shards": 1, - "number_of_replicas": 1, - "analysis": { - "analyzer": { - "default": { - "type": "standard", - "stopwords": "_none_" - } - } - }, - "mapper.dynamic": true, - "marvel.index_format": 1 - } - ..... -} ----------------------------------- - -[float] -[[relevant-settings]] -=== Enabling Automatic Index Creation - -Marvel relies on Elasticsearch's ability to automatically create new indices -when indexing documents. If you have disabled automatic index creation, you -need to configure the `action.auto_create_index` setting to allow the -creation of Marvel indices: - -[source,yaml] ----------------------- -action.auto_create_index: .marvel-* ----------------------- - -For more information see {ref}/docs-index_.html#index-creation[Index Creation] in the Elasticsearch -Reference. diff --git a/marvel/docs/getting-started.asciidoc b/marvel/docs/getting-started.asciidoc deleted file mode 100644 index a4ee053c4a0..00000000000 --- a/marvel/docs/getting-started.asciidoc +++ /dev/null @@ -1,47 +0,0 @@ -[[getting-started]] -== Getting Started - -This getting started guide walks you through installing Marvel -and using the Marvel Kibana app to monitor your Elastisearch cluster. - -To install Marvel: - -. Install the Marvel plugin on each node in your cluster: - -.. Run `bin/plugin install` from `ES_HOME` to install the License plugin: -+ -[source,shell] ----------------------------------------------------------- -bin/plugin install license ----------------------------------------------------------- - -.. Run `bin/plugin install` to install the Marvel plugin. -+ -[source,shell] ----------------------------------------------------------- -bin/plugin install marvel-agent ----------------------------------------------------------- -+ -NOTE: If you are using a <> of Elasticsearch, run the installation with superuser permissions. To perform an offline installation, <>. - -.. Start Elasticsearch. -+ -[source,shell] ----------------------------------------------------------- -bin/elasticsearch ----------------------------------------------------------- - -. Install the Marvel app into Kibana. - -. Run Kibana and open the Marvel app to verify the installation. You should -see an overview of your cluster's status: -+ -// image:images/overview_thumb.png["Overview Dashboard",link="images/overview.png"] - - -Now you're ready to use Marvel to monitor and analyze your cluster! For example, you can: - -* -* - - diff --git a/marvel/docs/index.asciidoc b/marvel/docs/index.asciidoc deleted file mode 100644 index c07f713d15a..00000000000 --- a/marvel/docs/index.asciidoc +++ /dev/null @@ -1,17 +0,0 @@ -= Marvel Documentation - -:ref: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current -:esversion: 2.0 -:licenseversion: 2.0 -:shieldversion: 2.0 -:kibanaversion: 4.2+ - -include::introduction.asciidoc[] - -include::getting-started.asciidoc[] - -include::installing-marvel.asciidoc[] - -include::configuring-marvel.asciidoc[] - -include::release-notes.asciidoc[] \ No newline at end of file diff --git a/marvel/docs/installing-marvel.asciidoc b/marvel/docs/installing-marvel.asciidoc deleted file mode 100644 index b9d597f9ca2..00000000000 --- a/marvel/docs/installing-marvel.asciidoc +++ /dev/null @@ -1,141 +0,0 @@ -[[installing-marvel]] -== Installing Marvel - -To use Marvel, you need to install two components: - -* An Elasticsearch plugin that collects data from each node in your cluster. -This plugin must be installed on every node. -* A Kibana app that provides the Marvel monitoring and management UI. - -By default, the Marvel plugin stores data in the same Elasticsearch cluster -where it is installed. If you are monitoring a production cluster we recommend -that you store the Marvel data in a separate monitoring cluster. Sending the Marvel -data to a monitoring cluster helps ensure that you can continue to monitor your -production cluster if it's in an unhealthy state. - -For basic installation instructions, see <>. For -information about storing Marvel data in a separate monitoring cluster, see <>. - -[float] -[[marvel-prequisites]] -=== Marvel Installation Prequisites - -* Elasticsearch {esversion} or later. -* Kibana {kibanaversion} or later. -* A modern web browser - http://www.elastic.co/subscriptions/matrix#matrix_browsers[Supported -Browsers]. - -[float] -[[package-installation]] -=== Installing Marvel on a DEB/RPM Package Installation - -If you use the DEB/RPM packages to install Elasticsearch, by default Elasticsearch is installed in -`/usr/share/elasticsearch` and the configuration files are stored in `/etc/elasticsearch`. (For the -complete list of default paths, see {ref}/setup-dir-layout.html#_deb_and_rpm[Directory Layout] in -the Elasticsearch Reference.) - -To install the Marvel and License plugins on a DEB/RPM package installation, you need to run -`bin/plugin install` from the `/usr/share/elasticsearch` directory with superuser permissions, and -specify the location of the configuration files by setting `-Des.path.conf`. For example: - -[source,shell] ----------------------------------------------------------- -cd /usr/share/elasticsearch -sudo bin/plugin install license -sudo bin/plugin install marvel-agent ----------------------------------------------------------- - -[float] -[[offline-installation]] -=== Installing Marvel on Offline Machines - -Elasticsearch’s `bin/plugin` script requires direct Internet access to download and install the -License and Marvel plugins. If your server doesn’t have Internet access, you can manually -download and install the plugins. - -To install Marvel on a machine that doesn't have Internet access: - -. Manually download the License and Marvel binaries: -+ -** https://download.elastic.co/elasticsearch/license/license-latest.zip[ -https://download.elastic.co/elasticsearch/license/license-latest.zip] -** https://download.elastic.co/elasticsearch/marvel/marvel-agent-latest.zip[ -https://download.elastic.co/elasticsearch/marvel/marvel-agent-latest.zip] - -. Transfer the zip files to the offline machine. - -. Run `bin/plugin` with the `-u` option to install the plugins using the zip files. For example: -+ -[source,shell] ----------------------------------------------------------- -bin/plugin install file:///path/to/file/license-latest.zip <1> -bin/plugin install file:///path/to/file/marvel-agent-latest.zip ----------------------------------------------------------- -<1> Note that you must specify an absolute path to the zip file after the `file://` protocol. - -[float] -[[monitoring-cluster]] -=== Setting up a Separate Monitoring Cluster - -To store Marvel data in a separate monitoring cluster: - -. Set up the Elasticsearch cluster you want to use for monitoring. For example, a two host cluster -with the nodes `es-mon-1` and `es-mon-2`. - -. Disable Marvel data collection for the nodes in your monitoring cluster by configuring the -`marvel.agent.enabled` setting in their `elasticsearch.yml` configuration files. -+ -[source,yaml] ------------------------- -marvel.agent.enabled: false ------------------------- - -. Install the Marvel and License plugins on the nodes in your monitoring cluster: -+ -[source,sh] ----------------- -bin/plugin install license -bin/plugin install marvel-agent ----------------- - -. Start Elasticsearch on the nodes in your monitoring cluster: -+ -[source,shell] ----------------------------------------------------------- -bin/elasticsearch ----------------------------------------------------------- - -. Configure the nodes in your production cluster to send Marvel data to your monitoring cluster by -configuring the `marvel.agent.exporter.es.hosts` setting in their `elasticsearch.yml` -configuration files: -+ -[source,yaml] ------------------------- -marvel.agent.exporter.es.hosts: ["es-mon-1:9200","es-mon-2:9200"] ------------------------- - -. Install the Marvel and License plugins on the nodes in your production cluster: -+ -[source,sh] ----------------- -bin/plugin install license -bin/plugin install marvel-agent ----------------- - - -. Restart Elasticsearch on the nodes in your production cluster -+ -[source,shell] ----------------------------------------------------------- -bin/elasticsearch ---------------------------------------------------------- -+ -TIP: You may want to temporarily {ref}/modules-cluster.html[disable shard -allocation] before you restart your nodes to avoid unnecessary shard -reallocation during the install process. - -. Install the Marvel app into Kibana. - -. Run Kibana and open the Marvel app to verify the installation. You should -see an overview of your cluster's status: \ No newline at end of file diff --git a/marvel/docs/introduction.asciidoc b/marvel/docs/introduction.asciidoc deleted file mode 100644 index a7d1fb039af..00000000000 --- a/marvel/docs/introduction.asciidoc +++ /dev/null @@ -1,23 +0,0 @@ -[[introduction]] -== Introduction - -_Marvel_ is a plugin for Elasticsearch that enables you to easily monitor and manage -your Elasticsearch cluster. Marvel aggregates cluster-wide statistics and events and provides a -Kibana app that makes it easy to view and analyze the data. - -When you open the Marvel app in Kibana, you see the key metrics that indicate the health of your -cluster. - -From there, you can dive into the details for particular nodes and indices. - -[float] -=== Where to Go Next - -* <> steps through how to install and start using Marvel to -monitor Elasticsearch. - -[float] -=== Have Comments, Questions, or Feedback? - -Head over to our https://discuss.elastic.co/c/marvel[Marvel Discussion Forum] to share your -experience, questions, and suggestions. \ No newline at end of file diff --git a/marvel/docs/release-notes.asciidoc b/marvel/docs/release-notes.asciidoc deleted file mode 100644 index 491afba07b3..00000000000 --- a/marvel/docs/release-notes.asciidoc +++ /dev/null @@ -1,252 +0,0 @@ -[[release-notes]] -== Release Notes - -[float] -[[version-compatibility]] -=== Version Compatibility - -Marvel {marvelversion} is compatible with: - -* Elasticsearch {esversion} -* License {licenseversion} -* Shield {shieldversion} -* Kibana {kibanaversion} - -[float] -[[upgrading]] -=== Upgrading Marvel -When upgrading Marvel, you must upgrade *every node in the cluster*. If you're using a monitoring -cluster, upgrade the nodes in the monitoring cluster before upgrading your production cluster. You -do not need to fully shut down your production or monitoring clusters to perform the upgrade, you -can perform a rolling upgrade. - -To perform a rolling upgrade of Marvel: - -. Disable shard reallocation. While this is optional, it enables a faster startup after cluster -shutdown. If you don't disable shard reallocation, the nodes immediately start trying to -replicate shards to each other on startup and spend a lot of time on wasted I/O. With shard -reallocation disabled, the nodes join the cluster with their indices intact and do not attempt to -rebalance. After startup is complete, you can turn reallocation back on. -+ -[source,sh] --------------------------------------------------- -curl -XPUT localhost:9200/_cluster/settings -d '{ - "transient" : { - "cluster.routing.allocation.enable" : "none" - } -}' --------------------------------------------------- - -. Upgrade each node, one at a time: - -.. Stop Elasticsearch. -.. Remove the old version of the Marvel plugin: -+ -[source,sh] --------------------------------------------------- -bin/plugin remove marvel-agent --------------------------------------------------- - -.. Install the latest version of the Marvel plugin: -+ -[source,sh] --------------------------------------------------- -bin/plugin install marvel-agent --------------------------------------------------- - -.. Start Elasticsearch and confirm that the node rejoins the cluster and that there are no errors -in the logs. - -. When you've upgraded all of the nodes in the cluster, reenable shard allocation: -+ -[source,sh] --------------------------------------------------- -curl -XPUT localhost:9200/_cluster/settings -d '{ - "transient" : { - "cluster.routing.allocation.enable" : "all" - } -}' --------------------------------------------------- - -[float] -[[change_list]] -=== Change List - -[float] -==== 2.0.0-beta2 - -Marvel versions are now aligned with Elasticsearch version. - -- Agent: - * Added: add license expiration support to stop collecting data 7 days - after license expiration - * Added: add integration tests - * Added: add support to index cluster license information - * Added: add support to collect node statistics - * Added: add support to collect index recoveries - * Added: add support to collect index level statistics - * Added: add support to collect cluster statistics - * Added: add support to collect cluster state - * Improved: update project to work with Elasticsearch 2.x - -- Monitoring UI: - * TODO - -- Sense: - * Sense has been removed from Marvel starting version 2.x and is now shipped as - a Kibana application - -[float] -==== 1.3.1 - -- Agent: - * Added: add timeouts to better deal with unresponsive ES nodes / hiccups. - * Added: Allow SSL hostname verification to be disabled. - * Fixed: Node failed to start if HTTP is disabled. - * Fixed: Potential NPE if HTTP server didn't start fast enough. - * Fixed: `marvel.agent.indices` wasn't dynamically updatable when using a single value or a - comma separated list. - * Fixed: unused shield SSL settings caused errors during start up. - -- Monitoring UI: - * Fixed: Upgraded Kibana3 to latest version, fixing a wrap around issue in Safari. - -- Sense: - * Added: Cluster health's level url parameter. - * Added: _recovery API. - * Fixed: trailing space after URL broke request parsing. - * Added: _search_shards API. - -[float] -==== 1.3.0 - -- Agent: - * Added: support for shipping over https. - * Removed: support for optional shard level stats due to an incompatible change in ES 1.4. - * Fixed: an issue causing a tribe node (ES 1.4.0) not to initialize when Marvel is installed. - * Improved: resiliency and error checking around the marvel index template (both checking for it and adding it). - * Improved: logging upon error, supressing repreated logs. - * Added: Automcally detect the local node's port when using not default Marvel settings (previously was always 9200) - * Improved: Change _bulk export command to set the index name in the url param. This is usefull when `rest.action.multi.allow_explicit_index` is set to false. - -- Monitoring UI - * Added: charts for new circuit breakers introduce with ES 1.4.0 - * Added: a chart to plot circuit break limit. - * Added: a charts for query cache. - * Added: charts for index throttling. - * Added: charts to expose memory usage of the index writer and version map. - * Fixed: Network Transport Bytes Received chart actually shows bytes sent - * Fixed: Node Stats dashboard missed some thread pools. - -- Sense: - * Added: a settings to allow disabling mappings and/or indices autocomplete. This is usefull for extremly large deployments where parsing by the browser is unrealistic. - * Added: Custer Reroute API. - * Added: Get Field Mappings API, - * Fixed: Url auto complete failed with completing fully qualified urls (i.e. with protocol and host). - * Added: Query Cache parmaters to the Search API. - * Added: Analyze API. - * Added: Validate Query API. - * Fixed: include_in_parent and include_in_root is missing for nested type mapping. - * Added: Put Percolator API. - * Fixed: Range filter template to use gt, gte, lt and lte. - * Added: cluster.routing.allocation.* settings - * Added: weight param to the Function Score query. - * Added: Flush API. - * Added: show_term_doc_count_error parameter to the Terms Aggregation. - * Added: Update API - * Added: _geo_distance as a sort option. - * Added: Updated the Significant Terms aggregation to 1.4.0 features. - * Added: metadata fields to the Mapping API. - * Added: Get Index API. - * Added: Scripted Metric Aggregation. - * Added: simple_query_string query. - * Added: Updated the More Like This query to 1.4.0 features. - * Added: min_childeren, max_children options to the has_child query and dilter. - * Added: Updated execution hint options in terms and significant terms aggs. - * Added: transform section of Mappings API. - * Added: indexed scripts and templates. - * Added: Geo Bounds aggregation. - * Added: Top Hits aggregation. - * Added: collect_mode option the Terms aggregation. - * Added: Percentiles Rank aggregation. - * Added: Disk Threshold Allocator settings. - * Fixed: Exists filter auto complete. - * Fixed: Snapshot and Restore API failed to autocomplete repository settings. - -[float] -==== 1.2.1 - - Fix a cluster state data shipping for cluster states larger than 16K (in `SMILE` format and without meta data). - -[float] -==== 1.2.0 - - New Shard Allocation Dashboard. - - Simplified navigation and dashboard customization. - - Sense: - * Update the KB to the ES 1.2.0 API, adding the following: - * `_cat/plugins` - * `_cat/segments` - * `_search/template` - * `_count` - * `_snapshot` - * Alias support for index creation. - * Significant terms aggregation. - * Percentiles aggregation. - * Cardinality aggregation. - * Time_zone keyword to date_histogram facet/aggregation. - * Removed deprecated `custom_score` & `custom_boost_factor` from the 1.0 API. - * Fixed a bug causing the query panel to loose focus after running a command. - - - Charts and Dashboards changes: - * Added an information icon next to the status information of Cluster Summary panel. Hovering on it will show a - short explanation of current status. - * The indices stats table in the Overview dashboard now shows an information icon next to red and yellow indices. - Hovering on it will show a short shard level summary. - * Marvel's index template will now be automatically updated upon upgrade. - * Added field data & filter cache eviction charts to Node Stats dashboard and Index Stats dashboard. - * Added field data circuit breaker charts to Node Stats dashboard. - * Added a registration & purchasing form. - * Hidden indexes are now shown by default. - * Default cluster pulse default time span to 7 days. - * Fixed: Split brain detection algorithm didn't fire in some configurations. - - - - `marvel.agent.exporter.es.host` configuration option now defaults to port 9200. - -[float] -==== 1.1.1 - - Fixed: agent did not interpret timeout settings correctly, causing potential connectivity errors when shipping data. - -[float] -==== 1.1.0 - - Improved Sense's autocomplete suggestions: - * Added Snapshot & Restore - * Added Aggregations - * Added support for url query string parameters - * Updated for breaking changes in Elasticsearch 1.0 - - Updated welcome splash screen. - - Sense now uses the last used server when opened (previously used the hostname used to access it). - - The agent's keep-alive thread is now stopped upon errors to reduce log noise. It will be restarted - upon successful connection. - - Improved error reporting for failures of items in the agent's bulk requests. - - Index Statistics Dashboard: Indexing Rate Primaries chart was based on the wrong field. - - Introduced `marvel.agent.shard_stats.enabled` to control exporting of shard level statistics. Defaults to `false`. - - Changed agent's default sampling rate to 10s (was 5s) - - Added a visual indication for the master node at the Nodes section of the Overview Dashboard - - Node and Indices tables visually indicate stale data - - Added error reporting to nodes and indices tables - - Made the following agent settings changeable via the Cluster Update Settings API: - * marvel.agent.interval (also supports setting to -1 to disable exporting) - * marvel.agent.exporter.es.hosts - * marvel.agent.exporter.es.timeout - * marvel.agent.shard_stats.enabled - -[float] -==== 1.0.2 - - Kibana uses `window.location.protocol` (http or https) to make ES calls. - - Added support for basic authentication when sending data from agent. See <>. - - Reduced DEBUG logging verbosity. - -[float] -==== 1.0.1 - - fixed an issue with usage statistics report. - - improve logging message when running on old Elasticsearch versions.