honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/docs/reference
History
Benjamin Trent 24d41eb695
[ML] partitions model definitions into chunks (#55260) (#55484) 
This paves the data layer way so that exceptionally large models are partitioned across multiple documents.

This change means that nodes before 7.8.0 will not be able to use trained inference models created on nodes on or after 7.8.0.

I chose the definition document limit to be 100. This *SHOULD* be plenty for any large model. One of the largest models that I have created so far had the following stats:
~314MB of inflated JSON, ~66MB when compressed, ~177MB of heap.
With the chunking sizes of `16 * 1024 * 1024` its compressed string could be partitioned to 5 documents.
Supporting models 20 times this size (compressed) seems adequate for now.
 		2020-04-20 16:08:54 -04:00
..
aggregations [7.x] Add support for filters to T-Test aggregation (#54980) (#55066) 2020-04-13 12:28:58 -04:00
analysis [DOCS] Reformat `flatten_graph` token filter (#54268) 2020-04-16 08:35:08 -04:00
autoscaling Add get autoscaling policy API (#54762) 2020-04-04 18:04:25 -04:00
cat [DOCS] Remove unused cat tasks request parameters (#54539) (#54741) 2020-04-03 15:33:28 -04:00
ccr [DOCS] Collapse nested objects in CCR APIs (#54697) 2020-04-03 12:04:33 -07:00
cluster Adjust docs for voting config exclusions API (#55006) 2020-04-20 19:47:33 +01:00
commands Clarify the remove keystore command can handle many (#54244) 2020-03-26 08:49:43 -04:00
docs Correct description for mget API request URI index (#52305) 2020-04-02 09:08:34 +10:00
eql [DOCS] EQL: Document `indexOf` function (#55071) 2020-04-15 11:29:50 -04:00
graph [DOCS] Collapse nested objects in Explore API docs (#55067) 2020-04-13 09:27:03 -04:00
high-availability [DOCS] Move snapshot-restore out of modules. (#49618) (#50829) 2020-01-09 16:55:46 -08:00
how-to [DOCS] Add example to "avoid scripts" advice (#54719) 2020-04-07 15:25:10 -04:00
ilm ILM DOCS: mention forcemerge is best effort (#54794) (#55401) 2020-04-17 15:42:23 +01:00
images [DOCS] Add token graph concept docs (#53339) 2020-03-19 07:43:18 -04:00
index-modules Clarify available processors (#54907) 2020-04-10 08:48:27 -04:00
indices Deprecate local parameter for get field mapping request (#55014) (#55099) 2020-04-12 13:48:47 +10:00
ingest [7.x] [ML] adding prediction_field_type to inference config (#55128) (#55230) 2020-04-15 09:45:22 -04:00
licensing Remove `_xpack` from license API example (#54698) (#54763) 2020-04-07 09:51:37 +03:00
mapping [DOCS] Add `ip_range` datatype to core datatypes range list (#55446) 2020-04-20 08:55:09 -04:00
migration Deprecate disabling basic-license features (#54816) (#55405) 2020-04-17 15:04:17 -04:00
ml [ML] partitions model definitions into chunks (#55260) (#55484) 2020-04-20 16:08:54 -04:00
modules Adjust docs for voting config exclusions API (#55006) 2020-04-20 19:47:33 +01:00
monitoring Deprecate disabling basic-license features (#54816) (#55405) 2020-04-17 15:04:17 -04:00
query-dsl [DOCS] Add query reference docs template (#52292) 2020-04-10 08:47:54 -04:00
release-notes Document the index corruption bug that gets fixed via Lucene 8.5.1. (#55232) 2020-04-17 13:37:37 +02:00
rest-api [7.x] Add analytics plugin usage stats to _xpack/usage (#54911) (#55162) 2020-04-14 17:03:14 -04:00
rollup [DOCS] Adds collapsible sections to rollup APIs (#54690) 2020-04-02 17:51:16 -07:00
scripting Scripting: Add char position of script errors (#51069) (#51266) 2020-01-21 13:45:59 -07:00
search [DOCS] Document `max_concurrent_searches` default (#55116) 2020-04-15 10:04:23 -04:00
searchable-snapshots/apis Provide repository-level stats for searchable snapshots (#55051) 2020-04-14 14:34:08 +02:00
settings Deprecate disabling basic-license features (#54816) (#55405) 2020-04-17 15:04:17 -04:00
setup [7.x] Fix ReloadSecureSettings API to consume password (#54771) (#55059) 2020-04-13 09:50:55 +03:00
slm Deprecate disabling basic-license features (#54816) (#55405) 2020-04-17 15:04:17 -04:00
snapshot-restore Backporting updates to ILM org, overview, & GS (#51898) 2020-02-04 16:45:18 -08:00
sql [DOCS] Remove Wikipedia link from `SUM_OF_SQUARES` SQL function docs (#52398) 2020-04-20 09:59:59 -04:00
testing Uppercasing some docs section title (#37781) 2019-01-24 22:54:55 +01:00
transform [DOCS] Removes transform performance note (#55177) 2020-04-15 10:42:52 -07:00
upgrade [DOCS] Add maintenance releases to upgrade table (#55012) 2020-04-14 11:28:55 -04:00
vectors Remove the 'experimental' marking from vector fields. (#49120) 2019-11-18 12:42:46 -08:00
aggregations.asciidoc [Docs] Update aggregations.asciidoc (#29265) 2018-03-28 15:01:45 +02:00
analysis.asciidoc [DOCS] Reformat `stop` token filter (#53059) 2020-03-03 13:22:52 -05:00
api-conventions.asciidoc [7.x] Add prefer_v2_templates flag and index setting (#55411) (#55476) 2020-04-20 12:05:42 -06:00
async-search.asciidoc [DOCS] add docs for async search (#53675) 2020-03-20 14:46:38 +01:00
cat.asciidoc [Transform] renamed _cat/transform to _cat/transforms (#53743) (#53771) 2020-03-18 19:54:03 -04:00
cluster.asciidoc Password-protected Keystore Feature Branch PR (#51123) (#51510) 2020-01-28 05:32:32 -05:00
data-rollup-transform.asciidoc [DOCS] Adds transforms to Elasticsearch book (#46846) (#47055) 2019-09-25 08:11:37 -07:00
docs.asciidoc [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:36:06 -04:00
frozen-indices.asciidoc Add heuristics to compute pre_filter_shard_size when unspecified (#53873) (#54007) 2020-03-24 02:05:15 +01:00
getting-started.asciidoc [DOCS] Switch to standard ESS trial links (#52552) 2020-02-21 12:07:10 -05:00
glossary.asciidoc Backporting updates to ILM org, overview, & GS (#51898) 2020-02-04 16:45:18 -08:00
gs-index.asciidoc [DOCS] Adding index file for GS "mini book". 2017-07-18 13:44:08 -07:00
high-availability.asciidoc [DOCS] Remove leveloffset for CCR docs (#46818) 2019-09-18 09:44:43 -04:00
how-to.asciidoc Correct grammar in list in how-to docs 2017-01-17 20:57:22 -05:00
index-modules.asciidoc Deprecate creation of dot-prefixed index names except for hidden and system indices (#49959) 2020-01-28 10:01:16 -07:00
index.asciidoc [DOCS] add docs for async search (#53675) 2020-03-20 14:46:38 +01:00
index.x.asciidoc [DOCS] Removes redundant index.asciidoc files (#30707) 2018-05-18 11:05:40 -07:00
indices.asciidoc [DOCS] Reorder index APIs alphabetically (#46981) (#47402) 2019-10-01 17:07:28 -04:00
ingest.asciidoc Replace required pipeline with final pipeline (#49470) 2019-11-22 14:37:36 -05:00
intro.asciidoc [7.x][DOCS] Updates ML links (#50387) (#50409) 2019-12-20 10:01:19 -08:00
mapping.asciidoc Remove the top-level 'mapping type' section. (#54035) 2020-03-23 15:34:23 -07:00
modules.asciidoc Backporting updates to ILM org, overview, & GS (#51898) 2020-02-04 16:45:18 -08:00
query-dsl.asciidoc Add a cluster setting to disallow expensive queries (#51385) (#52279) 2020-02-12 22:56:14 +01:00
redirects.asciidoc [DOCS] Add redirect for changed anchor ID (#54533) 2020-03-31 16:42:16 -04:00
release-notes.asciidoc value_count Aggregation optimization (backport of #54854) (#55076) 2020-04-10 13:16:39 -04:00
scripting.asciidoc [DOCS] Move 'Scripting' section to top-level navigation. (#42939) 2019-06-06 10:46:02 -04:00
search.asciidoc [DOCS] add docs for async search (#53675) 2020-03-20 14:46:38 +01:00
setup.asciidoc [DOCS] Adds transform-settings page and its subpage to redirects (#52944) 2020-03-02 16:36:40 +01:00
testing.asciidoc [Docs] Unify spelling of Elasticsearch (#27567) 2017-11-29 09:44:25 +01:00
upgrade.asciidoc [DOCS] Add maintenance releases to upgrade table (#55012) 2020-04-14 11:28:55 -04:00