Commit Graph

5097 Commits

Author SHA1 Message Date
Lisa Cawley 44e83f30e2
[DOCS] Edits rollup API description () 2019-01-15 08:46:36 -08:00
niloct e71542ffd3 Update delete-by-query.asciidoc ()
Tried my best to clarify sentence on `_delete_by_query` docs.
2019-01-15 08:05:24 -07:00
Boaz Leskes f7eb517eb8 Update the Flush API documentation ()
The semantics of the API changed considerably since the documentation was written.

The main change is to remove references to memory reduction (this is related to refresh).
Instead, flush refers to recovery times. I also removed the references to trimming the translog
as the translog may be required for other purposes (operation history for ops based recovery
and complement ongoing file based recoveries).

Closes 
2019-01-15 11:45:10 +01:00
Julie Tibshirani 36a3b84fc9
Update the default for include_type_name to false. ()
* Default include_type_name to false for get and put mappings.

* Default include_type_name to false for get field mappings.

* Add a constant for the default include_type_name value.

* Default include_type_name to false for get and put index templates.

* Default include_type_name to false for create index.

* Update create index calls in REST documentation to use include_type_name=true.

* Some minor clean-ups around the get index API.

* In REST tests, use include_type_name=true by default for index creation.

* Make sure to use 'expression == false'.

* Clarify the different IndexTemplateMetaData toXContent methods.

* Fix FullClusterRestartIT#testSnapshotRestore.

* Fix the ml_anomalies_default_mappings test.

* Fix GetFieldMappingsResponseTests and GetIndexTemplateResponseTests.

We make sure to specify include_type_name=true during xContent parsing,
so we continue to test the legacy typed responses. XContent generation
for the typeless responses is currently only covered by REST tests,
but we will be adding unit test coverage for these as we implement
each typeless API in the Java HLRC.

This commit also refactors GetMappingsResponse to follow the same appraoch
as the other mappings-related responses, where we read include_type_name
out of the xContent params, instead of creating a second toXContent method.
This gives better consistency in the response parsing code.

* Fix more REST tests.

* Improve some wording in the create index documentation.

* Add a note about types removal in the create index docs.

* Fix SmokeTestMonitoringWithSecurityIT#testHTTPExporterWithSSL.

* Make sure to mention include_type_name in the REST docs for affected APIs.

* Make sure to use 'expression == false' in FullClusterRestartIT.

* Mention include_type_name in the REST templates docs.
2019-01-14 13:08:01 -08:00
Jay Modi f3edbe2911
Security: remove SSL settings fallback ()
This commit removes the fallback for SSL settings. While this may be
seen as a non user friendly change, the intention behind this change
is to simplify the reasoning needed to understand what is actually
being used for a given SSL configuration. Each configuration now needs
to be explicitly specified as there is no global configuration or
fallback to some other configuration.

Closes 
2019-01-14 14:06:22 -07:00
Nhat Nguyen 15aa3764a4
Reduce recovery time with compress or secure transport ()
Today file-chunks are sent sequentially one by one in peer-recovery. This is a
correct choice since the implementation is straightforward and recovery is
network bound in most of the time. However, if the connection is encrypted, we
might not be able to saturate the network pipe because encrypting/decrypting
are cpu bound rather than network-bound.

With this commit, a source node can send multiple (default to 2) file-chunks
without waiting for the acknowledgments from the target.

Below are the benchmark results for PMC and NYC_taxis.

- PMC (20.2 GB)

| Transport | Baseline | chunks=1 | chunks=2 | chunks=3 | chunks=4 |
| ----------| ---------| -------- | -------- | -------- | -------- |
| Plain     | 184s     | 137s     | 106s     | 105s     | 106s     |
| TLS       | 346s     | 294s     | 176s     | 153s     | 117s     |
| Compress  | 1556s    | 1407s    | 1193s    | 1183s    | 1211s    |

- NYC_Taxis (38.6GB)

| Transport | Baseline | chunks=1 | chunks=2 | chunks=3 | chunks=4 |
| ----------| ---------| ---------| ---------| ---------| -------- |
| Plain     | 321s     | 249s     | 191s     |  *       | *        |
| TLS       | 618s     | 539s     | 323s     | 290s     | 213s     |
| Compress  | 2622s    | 2421s    | 2018s    | 2029s    | n/a      |

Relates 
2019-01-14 15:14:46 -05:00
Peter Dyson 96cfa000a5
[DOCS] copy_to only works one level deep, not recursively () 2019-01-13 16:24:34 +10:00
Jason Tedor 03be4dbaca
Introduce retention lease persistence ()
This commit introduces the persistence of retention leases by persisting
them in index commits and recovering them when recovering a shard from
store.
2019-01-12 14:43:19 -08:00
Boaz Leskes 093d4a371c followup to with additional feedback 2019-01-11 08:07:35 -08:00
Boaz Leskes cae71cddfe
Document Seq No powered optimistic concurrency control ()
Add documentation to describe the new sequence number powered optimistic concurrency control

Relates 
Relates 
2019-01-11 07:59:15 -08:00
Jake Landis fad6329808
add link to freeze from cold phase () 2019-01-11 08:58:41 -06:00
Alexander Reelsen bbd093059f
Add whitelist to watcher HttpClient ()
This adds a configurable whitelist to the HTTP client in watcher. By
default every URL is allowed to retain BWC. A dynamically configurable
setting named "xpack.http.whitelist" was added that allows to
configure an array of URLs, which can also contain simple regexes.

Closes 
2019-01-11 09:22:47 +01:00
Jake Landis 9fd4d6ba4b
remove `+` references for xpack.monitoring.collection.indices ()
`+` for index name inclusions is no longer supported for 6.x+. This
commit removes references of the `+` from the documenation. System
indices additional example is also included.

fixes 
2019-01-10 17:06:44 -06:00
Lee Hinman 04dcb13ac4
Remove "beta" modifier from ILM documentation () 2019-01-10 13:38:14 -07:00
Lee Hinman a2d63ecdc0
Link ILM management and policy information in ILM API documentation ()
Previously these were only linked in a circuitous way rather than being
available from the top level API documentation and "Put Lifecycle" API docs.
This makes them slightly easier to find for a user.
2019-01-10 12:34:44 -07:00
lcawl 2d5a8ec59d [DOCS] Remove unused screenshots 2019-01-10 11:10:25 -08:00
Andrei Stefan 4a92de214a
SQL: Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) ()
* provide overriden `hashCode` and toString methods to account for `DISTINCT`
* change the analyzer for scenarios where `COUNT <field_name>` and `COUNT DISTINCT` have different paths
* defined a new `filter` aggregation encapsulating an `exists` query to filter out null or missing values
2019-01-10 09:51:51 +02:00
Albert Zaharovits 4efbda1bc0
[DOCS] Rolling upgrade with old internal indices ()
Upgrading the Elastic Stack perfectly documents the process to
upgrade ES from 5 to 6 when internal indices are present. However,
the rolling upgrade docs do not mention anything about internal indices.

This adds a warning in the rolling upgrade procedure, highlighting that
internal indices should be upgraded before the rolling upgrade procedure
can be started.
2019-01-09 18:14:22 +02:00
Christoph Büscher c149bb8cc2
Support 'include_type_name' in RestGetIndicesAction ()
This change adds support for the 'include_type_name' parameter for the
indices.get API. This parameter, which defaults to `false` starting in 7.0,
changes the response to not include the indices type names any longer.

If the parameter is set in the request, we additionally emit a deprecation
warning since using the parameter should be only temporarily necessary while
adapting to the new response format and we will remove it with the next major
version.
2019-01-09 14:17:17 +01:00
David Turner d9e2ebca67
Add more detail to recovery bandwidth limit docs () 2019-01-09 08:18:25 +00:00
Jun Ohtani 38b698d455
[Analysis] Deprecate Standard Html Strip Analyzer in master ()
* [Analysis] Deprecate Standard Html Strip Analyzer

Deprecate only Standard Html Strip Analyzer
If user create index with the analyzer since 7.0, es throws an exception.
If an index was created before 7.0, es issue deprecation log
We will remove it in 8.0

Related 
2019-01-09 12:42:00 +09:00
Mayya Sharipova ec32e66088 Deprecate reference to _type in lookup queries ()
Relates to 
2019-01-08 18:46:41 -08:00
Andrew Banchich 1d371a427f [Docs] Fix wrong math in overview.asciidoc () 2019-01-08 10:25:02 +01:00
lcawl 382e4d39ef [DOCS] Cleans up xpackml attributes 2019-01-07 14:33:10 -08:00
Lisa Cawley f307847f29
[DOCS] Adds overview and API ref for cluster voting configurations () 2019-01-07 09:11:14 -08:00
Josh Soref edb48321ba [DOCS] Various spelling corrections () 2019-01-07 14:44:12 +01:00
Andrei Stefan 39a072389c
SQL: add sub-selects to the Limitations page () 2019-01-07 10:08:51 +02:00
Jim Ferenczi e38cf1d0dc
Add the ability to set the number of hits to track accurately ()
In Lucene 8 searches can skip non-competitive hits if the total hit count is not requested.
It is also possible to track the number of hits up to a certain threshold. This is a trade off to speed up searches while still being able to know a lower bound of the total hit count. This change adds the ability to set this threshold directly in the track_total_hits search option. A boolean value (true, false) indicates whether the total hit count should be tracked in the response. When set as an integer this option allows to compute a lower bound of the total hits while preserving the ability to skip non-competitive hits when enough matches have been collected.

Relates 
2019-01-04 20:36:49 +01:00
Adam Thomson ac4aecc92d [Docs] Update ingest-node.asciidoc () 2019-01-04 19:33:06 +01:00
Simon Willnauer 41d7e3a2fe
Expose `search.throttled` on `_cat/indices` ()
Today it's very difficult to see which indices are frozen or rather
throttled via the commonly used monitoring APIs. This change adds
a cell to the `_cat/indices` API to render if an index is `search.throttled`

Relates to 
2019-01-04 13:49:40 +01:00
Luca Cavanna bfe6f091da [DOCS] fix typo in 7.0.0-alpha2 release notes 2019-01-04 10:38:34 +01:00
Dimitris Athanasiou 586453fef1
[ML] Remove types from datafeed ()
Closes 
2019-01-04 09:43:44 +02:00
lcawley 00997b4f60 [DOCS] Fixes broken links 2019-01-04 17:41:28 +10:00
Peter Dyson 7839cec301
subsequent fix to edit in recent cherry-pick 2019-01-04 17:34:24 +10:00
Peter Dyson 7cc9754d94
fix to edit in recent cherry-pick 2019-01-04 17:26:42 +10:00
Peter Dyson 0ff2707c9f Add Profile API to search speed tuning howto ()
* Add Profile API to search speed tuning howto

Seemed useful to mention the Profile API in the context of tuning for search speed.
2019-01-04 16:49:12 +10:00
Tal Levy eaeccd8401
[ILM] Add Freeze Action ()
This commit adds a new ILM Action for
freezing indices in the cold phase.

Closes .
2019-01-03 15:00:40 -08:00
Igor Motov d6acd8e15f
Docs: add clarification about geohash use in geohashgrid agg ()
Adds an example on translating geohashes returned by geohashgrid 
agg as bucket keys into geo bounding box filters in elasticsearch as well
as 3rd party applications.

Closes 
2019-01-03 15:40:48 -05:00
Lisa Cawley 366b6b5e11
[DOCS] Adds cluster setting to enable/disable config migration () 2019-01-03 09:26:57 -08:00
Marios Trivyzas 33137907cf
SQL: Enhance message for PERCENTILE[_RANK] with field as 2nd arg ()
Enhance error message for the case that the 2nd argument of PERCENTILE
and PERCENTILE_RANK is not a foldable, as it doesn't make sense to have
a dynamic value coming from a field.

Fixes: 
2019-01-03 13:55:09 +02:00
Christoph Büscher 046f86f274
Deprecate use of type in reindex request body ()
Types can be used both in the source and dest section of the body which will
be translated to search and index requests respectively. Adding a deprecation warning
for those cases and removing examples using more than one type in reindex since
support for this is going to be removed.
2019-01-03 10:29:14 +01:00
David Turner a40f0545e6
Make ILM examples larger ()
There are a handful of examples in the ILM documentation that could result in
rolling over indices more quickly than we might normally recommend,
contributing to over-sharding in cases where the examples are copied without
modification. This change makes some numbers bigger to try and avoid this.
2019-01-03 08:28:49 +00:00
Daniel Mitterdorfer 75f3443c62
Rename setting to enable mmap
With this commit we rename `node.store.allow_mmapfs` to
`node.store.allow_mmap`. Previously this setting has controlled whether
`mmapfs` could be used as a store type. With the introduction of
`hybridfs` which also relies on memory-mapping,
`node.store.allow_mmapfs` also applies to `hybridfs` and thus we rename
it in order to convey that it is actually used to allow memory-mapping
but not a specific store type.

Relates 
Relates 
2019-01-03 07:10:34 +01:00
Luca Cavanna 42ea644903
Remove single shard optimization when suggesting shard_size ()
When executing terms aggregations we set the shard_size, meaning the
number of buckets to collect on each shard, to a value that's higher than
the number of requested buckets, to guarantee some basic level of
precision. We have an optimization in place so that we leave shard_size
set to size whenever we are searching against a single shard, in which
case maximum precision is guaranteed by definition.

Such optimization requires us access to the total number of shards that
the search is executing against. In the context of cross-cluster search,
once we will introduce multiple reduction steps (one per cluster) each
cluster will only know the number of local shards, which is problematic
as we should only optimize if we are searching against a single shard in a
single cluster. It could be that we are searching against one shard per cluster
in which case the current code would optimize number of terms causing
a loss of precision.

While discussing how to address the CCS scenario, we decided that we do
not want to introduce further complexity caused by this single shard
optimization, as it benefits only a minority of cases, especially when
the benefits are not so great.

This commit removes the single shard optimization, meaning that we will
always have heuristic enabled on how many number of buckets to collect
on the shards, even when searching against a single shard.

This will cause more buckets to be collected when searching against a single
shard compared to before. If that becomes a problem for some users, they
can work around that by setting the shard_size equal to the size.

Relates to 
2019-01-02 17:45:49 +01:00
Abdullah DURSUN 8a02bacf76 Fix typo in multi-search.asciidoc () 2019-01-02 10:32:42 +01:00
Daniel Mitterdorfer f0052b1a7a
Add hybridfs store type
With this commit we introduce a new store type `hybridfs` that is a
hybrid between `mmapfs` and `niofs`. This store type chooses different
strategies to read Lucene files based on the read access pattern (random
or linear) in order to optimize performance.

This store type has been available in earlier versions of Elasticsearch
as `default_fs`. We have chosen a different name now in order to convey
the intent of the store type instead of tying it to the fact whether it
is the default choice.

Relates 
2019-01-02 10:10:32 +01:00
Peter Lyons 794303af73 [Docs] Fix URL path to aliases ()
This API URL contains /aliases/aliases and that seems to be a duplicate.
2018-12-31 17:58:49 +01:00
Andrei Stefan 9c283aed0a
SQL: Move the SQL ODBC client applications pages from the ODBC section into the general ES SQL Client Applications page () 2018-12-31 14:07:39 +02:00
Andrei Stefan 0b1cb5300e
Move ODBC documentation to `elasticsearch` repository () 2018-12-27 11:55:20 +02:00
Damien Alexandre 47f0a47f3d [Docs] Fix badly escaped keywords in migrate_7_0/search.asciidoc () 2018-12-27 09:18:28 +01:00