honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/docs/reference
History
Nhat Nguyen 15aa3764a4
Reduce recovery time with compress or secure transport (#36981) 
Today file-chunks are sent sequentially one by one in peer-recovery. This is a
correct choice since the implementation is straightforward and recovery is
network bound in most of the time. However, if the connection is encrypted, we
might not be able to saturate the network pipe because encrypting/decrypting
are cpu bound rather than network-bound.

With this commit, a source node can send multiple (default to 2) file-chunks
without waiting for the acknowledgments from the target.

Below are the benchmark results for PMC and NYC_taxis.

- PMC (20.2 GB)

| Transport | Baseline | chunks=1 | chunks=2 | chunks=3 | chunks=4 |
| ----------| ---------| -------- | -------- | -------- | -------- |
| Plain     | 184s     | 137s     | 106s     | 105s     | 106s     |
| TLS       | 346s     | 294s     | 176s     | 153s     | 117s     |
| Compress  | 1556s    | 1407s    | 1193s    | 1183s    | 1211s    |

- NYC_Taxis (38.6GB)

| Transport | Baseline | chunks=1 | chunks=2 | chunks=3 | chunks=4 |
| ----------| ---------| ---------| ---------| ---------| -------- |
| Plain     | 321s     | 249s     | 191s     |  *       | *        |
| TLS       | 618s     | 539s     | 323s     | 290s     | 213s     |
| Compress  | 2622s    | 2421s    | 2018s    | 2029s    | n/a      |

Relates #33844
 		2019-01-14 15:14:46 -05:00
..
aggregations [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
analysis [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
cat [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
ccr [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
cluster [DOCS] Adds overview and API ref for cluster voting configurations (#36954) 2019-01-07 09:11:14 -08:00
commands [DOCS] Update X-Pack terminology in security docs (#36564) 2018-12-19 14:53:37 -08:00
docs followup to #37284 with additional feedback 2019-01-11 08:07:35 -08:00
graph Remove X-Pack centric graph endpoints (#36010) 2018-11-29 07:09:37 -05:00
how-to [DOCS] Fixes broken links 2019-01-04 17:41:28 +10:00
ilm add link to freeze from cold phase (#37332) 2019-01-11 08:58:41 -06:00
images Move ODBC documentation to `elasticsearch` repository (#36955) 2018-12-27 11:55:20 +02:00
index-modules [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
indices Introduce retention lease persistence (#37375) 2019-01-12 14:43:19 -08:00
ingest [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
licensing [DOCS] Synchs titles of X-Pack APIs 2018-12-20 10:27:24 -08:00
mapping [DOCS] copy_to only works one level deep, not recursively (#37249) 2019-01-13 16:24:34 +10:00
migration [Analysis] Deprecate Standard Html Strip Analyzer in master (#26719) 2019-01-09 12:42:00 +09:00
ml [DOCS] Remove unused screenshots 2019-01-10 11:10:25 -08:00
modules Reduce recovery time with compress or secure transport (#36981) 2019-01-14 15:14:46 -05:00
monitoring remove `+` references for xpack.monitoring.collection.indices (#37282) 2019-01-10 17:06:44 -06:00
query-dsl Deprecate reference to _type in lookup queries (#37016) 2019-01-08 18:46:41 -08:00
release-notes [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
rest-api [DOCS] Update X-Pack terminology in security docs (#36564) 2018-12-19 14:53:37 -08:00
rollup [Docs] Fix wrong math in overview.asciidoc (#37209) 2019-01-08 10:25:02 +01:00
search [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
security [Zen2] Update documentation for Zen2 (#34714) 2018-12-20 13:02:44 +00:00
settings Add whitelist to watcher HttpClient (#36817) 2019-01-11 09:22:47 +01:00
setup [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
sql SQL: Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) (#37254) 2019-01-10 09:51:51 +02:00
testing [Docs] Remove repeating words (#33087) 2018-08-28 13:16:43 +02:00
upgrade [DOCS] Rolling upgrade with old internal indices (#37184) 2019-01-09 18:14:22 +02:00
aggregations.asciidoc [Docs] Update aggregations.asciidoc (#29265) 2018-03-28 15:01:45 +02:00
analysis.asciidoc [Docs] Add clarification to analysis example (#31826) 2018-07-06 14:36:58 +02:00
api-conventions.asciidoc [DOCS] Synchronizes captialization in top-level titles (#33605) 2018-09-27 08:36:18 -07:00
cat.asciidoc Rename the bulk thread pool to write thread pool (#29593) 2018-04-19 08:18:58 -04:00
cluster.asciidoc [DOCS] Adds overview and API ref for cluster voting configurations (#36954) 2019-01-07 09:11:14 -08:00
docs.asciidoc Document Seq No powered optimistic concurrency control (#37284) 2019-01-11 07:59:15 -08:00
frozen-indices.asciidoc Expose `search.throttled` on `_cat/indices` (#37073) 2019-01-04 13:49:40 +01:00
getting-started.asciidoc Add doc's sequence number + primary term to GetResult and use it for updates (#36680) 2018-12-17 15:22:13 +01:00
glossary.asciidoc Default to one shard (#30539) 2018-05-14 12:22:35 -04:00
gs-index.asciidoc [DOCS] Adding index file for GS "mini book". 2017-07-18 13:44:08 -07:00
how-to.asciidoc Correct grammar in list in how-to docs 2017-01-17 20:57:22 -05:00
index-modules.asciidoc ingest: document index.default_pipeline (#34500) 2018-10-23 13:49:25 -05:00
index.asciidoc Add a `_freeze` / `_unfreeze` API (#35592) 2018-11-20 08:03:24 +01:00
index.x.asciidoc [DOCS] Removes redundant index.asciidoc files (#30707) 2018-05-18 11:05:40 -07:00
indices.asciidoc add split index reference in indices.asciidoc 2017-11-06 12:55:41 +01:00
ingest.asciidoc ingest: document index.default_pipeline (#34500) 2018-10-23 13:49:25 -05:00
mapping.asciidoc Make sure to use the type _doc in the REST documentation. (#34662) 2018-10-22 11:54:04 -07:00
modules.asciidoc [Zen2] Update documentation for Zen2 (#34714) 2018-12-20 13:02:44 +00:00
query-dsl.asciidoc Update query-dsl.asciidoc (#27669) 2017-12-11 18:06:08 +01:00
redirects.asciidoc [Zen2] Update documentation for Zen2 (#34714) 2018-12-20 13:02:44 +00:00
release-notes.asciidoc [DOCS] Adds placeholder for alpha2 release notes 2018-12-11 14:26:41 -08:00
search.asciidoc Make sure to use the type _doc in the REST documentation. (#34662) 2018-10-22 11:54:04 -07:00
setup.asciidoc [DOCS] Starting Elasticsearch (#31701) 2018-07-03 13:40:37 -07:00
testing.asciidoc [Docs] Unify spelling of Elasticsearch (#27567) 2017-11-29 09:44:25 +01:00
upgrade.asciidoc Revert "[DOCS] Added 6.3 info & updated the upgrade table. (#30940)" 2018-06-11 22:04:36 -04:00