honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/docs/reference/cat/indices.asciidoc

126 lines
4.2 KiB
Plaintext
Raw Normal View History

First pass at cat docs.
2013-11-14 20:14:39 -05:00
[[cat-indices]]
[DOCS] Remove heading offsets for REST APIs (#44568) Several files in the REST APIs nav section are included using :leveloffset: tags. This increments headings (h2 -> h3, h3 -> h4, etc.) in those files and removes the :leveloffset: tags. Other supporting changes: * Alphabetizes top-level REST API nav items. * Change 'indices APIs' heading to 'index APIs.' * Changes 'Snapshot lifecycle management' heading to sentence case.
2019-07-19 14:35:36 -04:00
=== cat indices
First pass at cat docs.
2013-11-14 20:14:39 -05:00
The `indices` command provides a cross-section of each index. This
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
information *spans nodes*. For example:
First pass at cat docs.
2013-11-14 20:14:39 -05:00
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
[source,js]
First pass at cat docs.
2013-11-14 20:14:39 -05:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
GET /_cat/indices/twi*?v&s=index
First pass at cat docs.
2013-11-14 20:14:39 -05:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
// CONSOLE
// TEST[setup:huge_twitter]
// TEST[s/^/PUT twitter2\n{"settings": {"number_of_replicas": 0}}\n/]
Might respond with:
Convert more docs to // CONSOLE Converts docs for `_cat/segments`, `_cat/plugins` and `_cat/repositories` from `curl` to `// CONSOLE` so they are tested as part of the build and are cleaner to use in Console. They should work fine with `curl` with the `COPY AS CURL` link. Also swaps the `source` type of the response from `js` to `txt` because that is more correct. The syntax highlighter doesn't care. It looks at the text to figure out the language. So it looks a little funny for `_cat` responses regardless. Relates to #18160
2016-10-25 10:56:30 -04:00
[source,txt]
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
--------------------------------------------------
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open twitter u8FNjxh8Rfy_awN11oDKYQ 1 1 1200 0 88.1kb 88.1kb
Default to one shard (#30539) This commit changes the default out-of-the-box configuration for the number of shards from five to one. We think this will help address a common problem of oversharding. For users with time-based indices that need a different default, this can be managed with index templates. For users with non-time-based indices that find they need to re-shard with the split API in place they no longer need to resort only to reindexing. Since this has the impact of changing the default number of shards used in REST tests, we want to ensure that we still have coverage for issues that could arise from multiple shards. As such, we randomize (rarely) the default number of shards in REST tests to two. This is managed via a global index template. However, some tests check the templates that are in the cluster state during the test. Since this template is randomly there, we need a way for tests to skip adding the template used to set the number of shards to two. For this we add the default_shards feature skip. To avoid having to write our docs in a complicated way because sometimes they might be behind one shard, and sometimes they might be behind two shards we apply the default_shards feature skip to all docs tests. That is, these tests will always run with the default number of shards (one).
2018-05-14 12:22:35 -04:00
green open twitter2 nYFWZEO7TUiOjLQXBaYJpA 1 0 0 0 260b 260b
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
--------------------------------------------------
// TESTRESPONSE[s/\d+(\.\d+)?[tgmk]?b/\\d+(\\.\\d+)?[tgmk]?b/]
[DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006)
2019-06-10 09:33:32 -04:00
// TESTRESPONSE[s/u8FNjxh8Rfy_awN11oDKYQ|nYFWZEO7TUiOjLQXBaYJpA/.+/ non_json]
First pass at cat docs.
2013-11-14 20:14:39 -05:00
We can tell quickly how many shards make up an index, the number of
doc: highlight that doc counts come from lucene (#23522) The docs don't clearly explain that the deleted doc count also comes from lucene. IMHO, it is worth highlighting this information separately, as a Note. Apart from that, there should be an official recommended alternative as well.
2017-04-17 21:52:29 -04:00
docs, deleted docs, primary store size, and total store size (all shards including replicas).
Add some clarification regarding docs.count * Add some clarification regarding docs.count * Some improvements as suggested by @jasontedor
2016-04-07 05:15:32 -04:00
All these exposed metrics come directly from Lucene APIs.
First pass at cat docs.
2013-11-14 20:14:39 -05:00
doc: highlight that doc counts come from lucene (#23522) The docs don't clearly explain that the deleted doc count also comes from lucene. IMHO, it is worth highlighting this information separately, as a Note. Apart from that, there should be an official recommended alternative as well.
2017-04-17 21:52:29 -04:00
*Notes:*
1. As the number of documents and deleted documents shown in this are at the lucene level,
it includes all the hidden documents (e.g. from nested documents) as well.
[Docs] Unify spelling of Elasticsearch (#27567) Removes occurences of "elasticsearch" or "ElasticSearch" in favour of "Elasticsearch" where appropriate.
2017-11-29 03:44:25 -05:00
2. To get actual count of documents at the Elasticsearch level, the recommended way
doc: highlight that doc counts come from lucene (#23522) The docs don't clearly explain that the deleted doc count also comes from lucene. IMHO, it is worth highlighting this information separately, as a Note. Apart from that, there should be an official recommended alternative as well.
2017-04-17 21:52:29 -04:00
is to use either the <<cat-count>> or the <<search-count>>
[DOCS] Update cat/indices to reflect ?pri flag
2014-01-13 15:18:14 -05:00
[float]
[[pri-flag]]
[DOCS] Remove heading offsets for REST APIs (#44568) Several files in the REST APIs nav section are included using :leveloffset: tags. This increments headings (h2 -> h3, h3 -> h4, etc.) in those files and removes the :leveloffset: tags. Other supporting changes: * Alphabetizes top-level REST API nav items. * Change 'indices APIs' heading to 'index APIs.' * Changes 'Snapshot lifecycle management' heading to sentence case.
2019-07-19 14:35:36 -04:00
==== Primaries
[DOCS] Update cat/indices to reflect ?pri flag
2014-01-13 15:18:14 -05:00
The index stats by default will show them for all of an index's
shards, including replicas. A `pri` flag can be supplied to enable
the view of relevant stats in the context of only the primaries.
First pass at cat docs.
2013-11-14 20:14:39 -05:00
[float]
[[examples]]
[DOCS] Remove heading offsets for REST APIs (#44568) Several files in the REST APIs nav section are included using :leveloffset: tags. This increments headings (h2 -> h3, h3 -> h4, etc.) in those files and removes the :leveloffset: tags. Other supporting changes: * Alphabetizes top-level REST API nav items. * Change 'indices APIs' heading to 'index APIs.' * Changes 'Snapshot lifecycle management' heading to sentence case.
2019-07-19 14:35:36 -04:00
==== Examples
First pass at cat docs.
2013-11-14 20:14:39 -05:00
Which indices are yellow?
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
[source,js]
--------------------------------------------------
GET /_cat/indices?v&health=yellow
--------------------------------------------------
// CONSOLE
// TEST[continued]
Which looks like:
Convert more docs to // CONSOLE Converts docs for `_cat/segments`, `_cat/plugins` and `_cat/repositories` from `curl` to `// CONSOLE` so they are tested as part of the build and are cleaner to use in Console. They should work fine with `curl` with the `COPY AS CURL` link. Also swaps the `source` type of the response from `js` to `txt` because that is more correct. The syntax highlighter doesn't care. It looks at the text to figure out the language. So it looks a little funny for `_cat` responses regardless. Relates to #18160
2016-10-25 10:56:30 -04:00
[source,txt]
First pass at cat docs.
2013-11-14 20:14:39 -05:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open twitter u8FNjxh8Rfy_awN11oDKYQ 1 1 1200 0 88.1kb 88.1kb
First pass at cat docs.
2013-11-14 20:14:39 -05:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
// TESTRESPONSE[s/\d+(\.\d+)?[tgmk]?b/\\d+(\\.\\d+)?[tgmk]?b/]
[DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006)
2019-06-10 09:33:32 -04:00
// TESTRESPONSE[s/u8FNjxh8Rfy_awN11oDKYQ/.+/ non_json]
First pass at cat docs.
2013-11-14 20:14:39 -05:00
Fix recurring doc test failures with the cat API. (#21561) This failure is due to the fact that we sort on store size, which is cached. So it might happen that the store size that is taken into account is not the right one, which makes the indices sorted in the wrong order. This changes the doc example to sort on the number of docs instead. Closes #21062
2016-11-15 10:00:44 -05:00
Which index has the largest number of documents?
First pass at cat docs.
2013-11-14 20:14:39 -05:00
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
[source,js]
--------------------------------------------------
Fix recurring doc test failures with the cat API. (#21561) This failure is due to the fact that we sort on store size, which is cached. So it might happen that the store size that is taken into account is not the right one, which makes the indices sorted in the wrong order. This changes the doc example to sort on the number of docs instead. Closes #21062
2016-11-15 10:00:44 -05:00
GET /_cat/indices?v&s=docs.count:desc
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
--------------------------------------------------
// CONSOLE
// TEST[continued]
Which looks like:
Convert more docs to // CONSOLE Converts docs for `_cat/segments`, `_cat/plugins` and `_cat/repositories` from `curl` to `// CONSOLE` so they are tested as part of the build and are cleaner to use in Console. They should work fine with `curl` with the `COPY AS CURL` link. Also swaps the `source` type of the response from `js` to `txt` because that is more correct. The syntax highlighter doesn't care. It looks at the text to figure out the language. So it looks a little funny for `_cat` responses regardless. Relates to #18160
2016-10-25 10:56:30 -04:00
[source,txt]
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
--------------------------------------------------
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open twitter u8FNjxh8Rfy_awN11oDKYQ 1 1 1200 0 88.1kb 88.1kb
Default to one shard (#30539) This commit changes the default out-of-the-box configuration for the number of shards from five to one. We think this will help address a common problem of oversharding. For users with time-based indices that need a different default, this can be managed with index templates. For users with non-time-based indices that find they need to re-shard with the split API in place they no longer need to resort only to reindexing. Since this has the impact of changing the default number of shards used in REST tests, we want to ensure that we still have coverage for issues that could arise from multiple shards. As such, we randomize (rarely) the default number of shards in REST tests to two. This is managed via a global index template. However, some tests check the templates that are in the cluster state during the test. Since this template is randomly there, we need a way for tests to skip adding the template used to set the number of shards to two. For this we add the default_shards feature skip. To avoid having to write our docs in a complicated way because sometimes they might be behind one shard, and sometimes they might be behind two shards we apply the default_shards feature skip to all docs tests. That is, these tests will always run with the default number of shards (one).
2018-05-14 12:22:35 -04:00
green open twitter2 nYFWZEO7TUiOjLQXBaYJpA 1 0 0 0 260b 260b
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
--------------------------------------------------
// TESTRESPONSE[s/\d+(\.\d+)?[tgmk]?b/\\d+(\\.\\d+)?[tgmk]?b/]
[DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006)
2019-06-10 09:33:32 -04:00
// TESTRESPONSE[s/u8FNjxh8Rfy_awN11oDKYQ|nYFWZEO7TUiOjLQXBaYJpA/.+/ non_json]
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
How many merge operations have the shards for the `twitter` completed?
[source,js]
First pass at cat docs.
2013-11-14 20:14:39 -05:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
GET /_cat/indices/twitter?pri&v&h=health,index,pri,rep,docs.count,mt
[DOCS] Update cat/indices to reflect ?pri flag
2014-01-13 15:18:14 -05:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
// CONSOLE
// TEST[continued]
[DOCS] Update cat/indices to reflect ?pri flag
2014-01-13 15:18:14 -05:00
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
Might look like:
[DOCS] Update cat/indices to reflect ?pri flag
2014-01-13 15:18:14 -05:00
Enforce that responses in docs are valid json (#26249) All of the snippets in our docs marked with `// TESTRESPONSE` are checked against the response from Elasticsearch but, due to the way they are implemented they are actually parsed as YAML instead of JSON. Luckilly, all valid JSON is valid YAML! Unfurtunately that means that invalid JSON has snuck into the exmples! This adds a step during the build to parse them as JSON and fail the build if they don't parse. But no! It isn't quite that simple. The displayed text of some of these responses looks like: ``` { ... "aggregations": { "range": { "buckets": [ { "to": 1.4436576E12, "to_as_string": "10-2015", "doc_count": 7, "key": "*-10-2015" }, { "from": 1.4436576E12, "from_as_string": "10-2015", "doc_count": 0, "key": "10-2015-*" } ] } } } ``` Note the `...` which isn't valid json but we like it anyway and want it in the output. We use substitution rules to convert the `...` into the response we expect. That yields a response that looks like: ``` { "took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits, "aggregations": { "range": { "buckets": [ { "to": 1.4436576E12, "to_as_string": "10-2015", "doc_count": 7, "key": "*-10-2015" }, { "from": 1.4436576E12, "from_as_string": "10-2015", "doc_count": 0, "key": "10-2015-*" } ] } } } ``` That is what the tests consume but it isn't valid JSON! Oh no! We don't want to go update all the substitution rules because that'd be huge and, ultimately, wouldn't buy much. So we quote the `$body.took` bits before parsing the JSON. Note the responses that we use for the `_cat` APIs are all converted into regexes and there is no expectation that they are valid JSON. Closes #26233
2017-08-17 09:02:10 -04:00
[source,txt]
[DOCS] Update cat/indices to reflect ?pri flag
2014-01-13 15:18:14 -05:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
health index pri rep docs.count mt pri.mt
yellow twitter 1 1 1200 16 16
First pass at cat docs.
2013-11-14 20:14:39 -05:00
--------------------------------------------------
[DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006)
2019-06-10 09:33:32 -04:00
// TESTRESPONSE[s/16/\\d+/ non_json]
Admin: add total index memory in `_cat/indices` This patch adds to `_cat/indices` information about memory usage per index by adding memory used by FieldData, IdCache, Percolate, Segments (memory, index writer, version map). ``` % curl 'localhost:9200/_cat/indices?v&h=i,tm' i tm wiki 8.1gb test 30.5kb user 1.9mb ``` Closes #7008
2014-09-22 16:03:50 -04:00
How much memory is used per index?
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
[source,js]
--------------------------------------------------
GET /_cat/indices?v&h=i,tm&s=tm:desc
--------------------------------------------------
// CONSOLE
// TEST[continued]
Might look like:
Enforce that responses in docs are valid json (#26249) All of the snippets in our docs marked with `// TESTRESPONSE` are checked against the response from Elasticsearch but, due to the way they are implemented they are actually parsed as YAML instead of JSON. Luckilly, all valid JSON is valid YAML! Unfurtunately that means that invalid JSON has snuck into the exmples! This adds a step during the build to parse them as JSON and fail the build if they don't parse. But no! It isn't quite that simple. The displayed text of some of these responses looks like: ``` { ... "aggregations": { "range": { "buckets": [ { "to": 1.4436576E12, "to_as_string": "10-2015", "doc_count": 7, "key": "*-10-2015" }, { "from": 1.4436576E12, "from_as_string": "10-2015", "doc_count": 0, "key": "10-2015-*" } ] } } } ``` Note the `...` which isn't valid json but we like it anyway and want it in the output. We use substitution rules to convert the `...` into the response we expect. That yields a response that looks like: ``` { "took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits, "aggregations": { "range": { "buckets": [ { "to": 1.4436576E12, "to_as_string": "10-2015", "doc_count": 7, "key": "*-10-2015" }, { "from": 1.4436576E12, "from_as_string": "10-2015", "doc_count": 0, "key": "10-2015-*" } ] } } } ``` That is what the tests consume but it isn't valid JSON! Oh no! We don't want to go update all the substitution rules because that'd be huge and, ultimately, wouldn't buy much. So we quote the `$body.took` bits before parsing the JSON. Note the responses that we use for the `_cat` APIs are all converted into regexes and there is no expectation that they are valid JSON. Closes #26233
2017-08-17 09:02:10 -04:00
[source,txt]
Admin: add total index memory in `_cat/indices` This patch adds to `_cat/indices` information about memory usage per index by adding memory used by FieldData, IdCache, Percolate, Segments (memory, index writer, version map). ``` % curl 'localhost:9200/_cat/indices?v&h=i,tm' i tm wiki 8.1gb test 30.5kb user 1.9mb ``` Closes #7008
2014-09-22 16:03:50 -04:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
i tm
twitter 8.1gb
twitter2 30.5kb
Admin: add total index memory in `_cat/indices` This patch adds to `_cat/indices` information about memory usage per index by adding memory used by FieldData, IdCache, Percolate, Segments (memory, index writer, version map). ``` % curl 'localhost:9200/_cat/indices?v&h=i,tm' i tm wiki 8.1gb test 30.5kb user 1.9mb ``` Closes #7008
2014-09-22 16:03:50 -04:00
--------------------------------------------------
CONSOLEify _cat/indices docs Relates to #18160 Uses the new sorting (#20658) in the `_cat` API to support all use cases natively. We can still resort to piping things through `sort` if we need to, but we don't have to for basic stuff like sorting!
2016-10-11 08:46:59 -04:00
// TESTRESPONSE[s/\d+(\.\d+)?[tgmk]?b/\\d+(\\.\\d+)?[tgmk]?b/]
[DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006)
2019-06-10 09:33:32 -04:00
// TESTRESPONSE[non_json]