honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/docs/reference
History
David Turner 532ade7816 More logging for slow cluster state application (#45007) 
Today the lag detector may remove nodes from the cluster if they fail to apply
a cluster state within a reasonable timeframe, but it is rather unclear from
the default logging that this has occurred and there is very little extra
information beyond the fact that the removed node was lagging. Moreover the
only forewarning that the lag detector might be invoked is a message indicating
that cluster state publication took unreasonably long, which does not contain
enough information to investigate the problem further.

This commit adds a good deal more detail to make the issues of slow nodes more
prominent:

- after 10 seconds (by default) we log an INFO message indicating that a
  publication is still waiting for responses from some nodes, including the
  identities of the problematic nodes.

- when the publication times out after 30 seconds (by default) we log a WARN
  message identifying the nodes that are still pending.

- the lag detector logs a more detailed warning when a fatally-lagging node is
  detected.

- if applying a cluster state takes too long then the cluster applier service
  logs a breakdown of all the tasks it ran as part of that process.
 		2019-08-01 13:20:46 +01:00
..
administering [DOC] Backup & Restore Security Configuration (#42970) 2019-07-10 14:53:56 +03:00
aggregations [DOCS] Correct sum_other_doc_count value in terms agg example (#45028) 2019-07-31 14:10:36 -04:00
analysis [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
cat [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
ccr [DOCS] Reformats API parameter details (#44194) 2019-07-12 08:28:49 -07:00
cluster [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
commands [DOCS] Adds command reference for elasticsearch-croneval (#43946) 2019-07-25 11:41:05 -07:00
data-frames/apis [DOCS] Clarifies dataframe transform validations 2019-07-25 15:05:42 -07:00
docs Return seq_no and primary_term in noop update (#44603) 2019-07-25 19:16:56 -04:00
graph Remove X-Pack centric graph endpoints (#36010) 2018-11-29 07:09:37 -05:00
how-to Expand docs on force-merge and global ordinals (#44684) 2019-07-23 07:33:33 +01:00
ilm Add option to filter ILM explain response (#44777) 2019-07-26 11:57:38 -04:00
images Add RareTerms aggregation (#35718) 2019-07-01 10:30:02 -04:00
index-modules [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
indices Add Clone Index API (#44267) 2019-07-25 22:02:28 +02:00
ingest Fix GeoIP custom database directory in docs (#43383) 2019-06-19 13:26:07 -04:00
licensing [DOCS] Remind users to include @ symbol when applying license file (#40688) 2019-04-04 11:50:20 -04:00
mapping Expand docs on force-merge and global ordinals (#44684) 2019-07-23 07:33:33 +01:00
migration [DOCS] Adds notable 7.3 breaking changes (#45057) 2019-07-31 11:47:10 -07:00
ml [DOCS] Updates terms in machine learning get APIs (#44986) 2019-07-30 11:29:25 -07:00
modules More logging for slow cluster state application (#45007) 2019-08-01 13:20:46 +01:00
monitoring [DOCS] Describe setup for monitoring logs (#42655) 2019-06-20 08:17:27 -07:00
query-dsl [DOCS] Reformat rank feature query. Add relevance score section. (#44975) 2019-07-31 14:31:50 -04:00
release-notes [DOCS] Edits 7.3 release highlights (#45066) 2019-07-31 15:49:27 -07:00
rest-api [DOCS] Amends REST API TOC. (#44948) 2019-07-29 14:08:57 +02:00
rollup [DOCS] Reformats API parameter details (#44194) 2019-07-12 08:28:49 -07:00
scripting [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
search [Docs] Fix typo in rank-eval.asciidoc (#44978) 2019-07-31 12:37:49 +02:00
security [DOCS] Remove :edit_url: overrides. (#44445) 2019-07-16 15:04:44 -04:00
settings [ML][Data Frame] adding dynamic cluster setting for failure retries (#44577) (#44639) 2019-07-19 16:17:39 -05:00
setup [DOCS] Remove xpack terminology from installation pages (#44973) 2019-07-30 15:29:59 -07:00
sql [DOCS] Edits 7.3 release highlights (#45066) 2019-07-31 15:49:27 -07:00
testing Uppercasing some docs section title (#37781) 2019-01-24 22:54:55 +01:00
upgrade [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
vectors Add l1norm and l2norm distances for vectors (#44116) 2019-07-11 14:30:02 -04:00
administering.asciidoc [DOC] Backup & Restore Security Configuration (#42970) 2019-07-10 14:53:56 +03:00
aggregations.asciidoc [Docs] Update aggregations.asciidoc (#29265) 2018-03-28 15:01:45 +02:00
analysis.asciidoc Remove more include_type_name and types from docs (#37601) 2019-01-18 14:11:18 +01:00
api-conventions.asciidoc [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
cat.asciidoc [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
cluster.asciidoc [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
docs.asciidoc [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
frozen-indices.asciidoc [DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006) 2019-06-10 09:53:05 -04:00
getting-started.asciidoc [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
glossary.asciidoc Update glossary.asciidoc (#41364) 2019-04-19 10:25:07 +02:00
gs-index.asciidoc [DOCS] Adding index file for GS "mini book". 2017-07-18 13:44:08 -07:00
how-to.asciidoc Correct grammar in list in how-to docs 2017-01-17 20:57:22 -05:00
index-modules.asciidoc [DOCS] Rewrite `regexp` query (#42711) 2019-07-24 08:38:41 -04:00
index.asciidoc [DOCS] Move Elasticsearch APIs to REST APIs section. (#44238) (#44372) 2019-07-17 09:18:31 -04:00
index.x.asciidoc [DOCS] Removes redundant index.asciidoc files (#30707) 2018-05-18 11:05:40 -07:00
indices.asciidoc Add Clone Index API (#44267) 2019-07-25 22:02:28 +02:00
ingest.asciidoc show a full ingest example in the index page, to let user fast understand ingest node. (#43476) 2019-07-01 08:04:57 +02:00
intro.asciidoc [DOCS] Edited title/subtitle. (#43552) 2019-06-24 15:31:19 -07:00
mapping.asciidoc [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
modules.asciidoc [DOCS] Remove :edit_url: overrides. (#44445) 2019-07-16 15:04:44 -04:00
query-dsl.asciidoc [DOCS] Rewrite `regexp` query (#42711) 2019-07-24 08:38:41 -04:00
redirects.asciidoc [DOCS] Move Elasticsearch APIs to REST APIs section. (#44238) (#44372) 2019-07-17 09:18:31 -04:00
release-notes.asciidoc [DOCS] Adds ml-cpp PRs to release notes (#44354) 2019-07-15 09:22:36 -07:00
scripting.asciidoc [DOCS] Move 'Scripting' section to top-level navigation. (#42939) 2019-06-06 10:46:02 -04:00
search.asciidoc [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
setup.asciidoc [ML][Data Frame] adding dynamic cluster setting for failure retries (#44577) (#44639) 2019-07-19 16:17:39 -05:00
testing.asciidoc [Docs] Unify spelling of Elasticsearch (#27567) 2017-11-29 09:44:25 +01:00
upgrade.asciidoc fix assumption that 6.7 is last 6.x release (#42255) 2019-05-20 14:35:28 -05:00