Commit Graph

5294 Commits

Author SHA1 Message Date
Van0SS 4478f10a2a Rest High Level client: Add List Tasks (#29546)
This change adds a `listTasks` method to the high level java
ClusterClient which allows listing running tasks through the 
task management API.

Related to #27205
2018-05-16 13:31:37 +02:00
lukens 9434f25ee3 [Docs] Update code snippet in has-child-query.asciidoc (#30510)
Changed `InetSocketTransportAddress` to `TransportAddress`, as that 
seems to be  the thing now.
2018-05-16 09:25:48 +02:00
Vladimir Dolzhenko fe3e0257ae
Allow date math for naming newly-created snapshots (#7939) (#30479)
Allow date math for naming newly-created snapshots (#7939)
2018-05-16 07:23:25 +02:00
Michael Basnight b94bc70aee
Add Create Repository High Level REST API (#30501)
This commit adds Create Repository, the associated docs and tests
for the high level REST API client. A few small changes to the
PutRepository Request and Response went into the commit as well.
2018-05-15 21:21:11 -05:00
Jason Tedor abc06d5b79
Expose master version in REST test context (#30623)
This commit exposes the master version to the REST test context. This
will be needed in a follow-up where the master version will be used to
determine whether or not a certain warning header is expected.
2018-05-15 17:26:43 -04:00
Julie Tibshirani 4f9dd37169
Add support for search templates to the high-level REST client. (#30473) 2018-05-15 13:07:58 -07:00
lcawl 5894e3574f [DOCS] Restores 7.0.0 release notes and highlights 2018-05-15 08:48:41 -07:00
Julie Tibshirani ab0be394e9
Remove assert statements from field caps documentation. (#30601)
Reorganize the test in `SearchDocumentationIT` so the assertions aren't shown in the generated documentation.
2018-05-15 08:37:50 -07:00
Albert Zaharovits 801973fa9f
Repository GCS plugin new client library (#30168)
This does away with the deprecated `com.google.api-client:google-api-client:1.23`
and replaces it with `com.google.cloud:google-cloud-storage:1.28.0`.
It also changes security permissions for the repository-gcs plugin.
2018-05-15 18:22:58 +03:00
javanna e1d675c690 [DOCS] Remove references to changelog and to highlights
highlights reference the changelog and it currently breaks the docs.

This aligns changes in master with the ones made in other branches.
2018-05-15 12:42:15 +02:00
javanna 098b3b7fb4 [DOCS] Remove references to removed changelog 2018-05-15 11:47:56 +02:00
srini-raman 0592b685b9 [Docs] Improve section detailing translog usage (#30573) 2018-05-15 10:43:57 +02:00
Jason Tedor 0f85c6429c
Remove the changelog (#30593)
We are starting over on the changelog with a different approach. This
commit removes the existing incarnation of the changelog to remove
confusion that we need to continue adding entries to it.
2018-05-14 21:57:08 -04:00
Lisa Cawley 21d67d1bd7
[DOCS] Adds release highlight pages (#30590) 2018-05-14 15:49:00 -07:00
Nik Everett 9881bfaea5
Docs: Document how to rebuild analyzers (#30498)
Adds documentation for how to rebuild all the built in analyzers and
tests for that documentation using the mechanism added in #29535.

Closes #29499
2018-05-14 18:40:54 -04:00
Igor Motov b30f2913cf
Docs: document precision limitations of geo_bounding_box (#30540)
The geo_bounding_box query might produce false positives alongside
the right and upper edges and false negatives alongside left and
bottom edges. This commit documents the behavior and defines the
maximum error.

Closes #29196
2018-05-14 15:54:42 -04:00
Jason Tedor 4a4e3d70d5
Default to one shard (#30539)
This commit changes the default out-of-the-box configuration for the
number of shards from five to one. We think this will help address a
common problem of oversharding. For users with time-based indices that
need a different default, this can be managed with index templates. For
users with non-time-based indices that find they need to re-shard with
the split API in place they no longer need to resort only to
reindexing.

Since this has the impact of changing the default number of shards used
in REST tests, we want to ensure that we still have coverage for issues
that could arise from multiple shards. As such, we randomize (rarely)
the default number of shards in REST tests to two. This is managed via a
global index template. However, some tests check the templates that are
in the cluster state during the test. Since this template is randomly
there, we need a way for tests to skip adding the template used to set
the number of shards to two. For this we add the default_shards feature
skip. To avoid having to write our docs in a complicated way because
sometimes they might be behind one shard, and sometimes they might be
behind two shards we apply the default_shards feature skip to all docs
tests. That is, these tests will always run with the default number of
shards (one).
2018-05-14 12:22:35 -04:00
Nik Everett 41148e4bb1
Docs: Update HighLevelRestClient migration docs (#30544)
The High Level REST Client's documentation suggested that users should
use the Low Level REST Client for index management activities. This
change removes that suggestion because the high level REST client
supports those APIs now.

This also changes the examples in the migration docs to that still use
the Low Level REST Client to use the non-deprecated varieats of
`performRequest`.
2018-05-14 11:11:27 -04:00
Yannick Welsch c96f2d7bf7
Document woes between auto-expand-replicas and allocation filtering (#30531)
Relates to #2869
2018-05-14 12:14:37 +02:00
Jason Tedor 901436148b Adjust copy settings versions
This commit adjusts the versions on the copy settings behavior now
that the default behavior is configured in 7.0.0.
2018-05-13 22:23:13 -04:00
Jason Tedor 593fdd40ed
Deprecate not copy settings and explicitly disallow (#30404)
We want copying settings to be the default behavior. This commit
deprecates not copying settings, and disallows explicitly not copying
settings. This gives users a transition path to the future default
behavior.
2018-05-13 10:30:05 -04:00
Costin Leau 52580b5ca8
SQL: Fix parsing of dates with milliseconds (#30419)
Dates internally contain milliseconds (which appear when converting them
to Strings) however parsing does not accept them (and is being strict).
The parser has been changed so that Date is mandatory but the time
(including its fractions such as millis) are optional.

Fix #30002
2018-05-10 20:14:54 +03:00
Nik Everett b4502dbf74
LLClient: Add setJsonEntity (#30447)
Adds `Request#setJsonEntity(String)` which short circuits the process of
sending a json string which is super common.
2018-05-09 18:33:03 -04:00
Mueed Chaudhry bf141a3fd1 [docs] add warning for read-write indices in force merge documentation (#28869) 2018-05-09 18:53:55 +02:00
Nik Everett f9dc86836d
Docs: Test examples that recreate lang analyzers (#29535)
We have a pile of documentation describing how to rebuild the built in
language analyzers and, previously, our documentation testing framework
made sure that the examples successfully built *an* analyzer but they
didn't assert that the analyzer built by the documentation matches the
built in anlayzer. Unsuprisingly, some of the examples aren't quite
right.

This adds a mechanism that tests that the analyzers built by the docs.
The mechanism is fairly simple and brutal but it seems to be working:
build a hundred random unicode sequences and send them through the
`_analyze` API with the rebuilt analyzer and then again through the
built in analyzer. Then make sure both APIs return the same results.
Each of these calls to `_anlayze` takes about 20ms on my laptop which
seems fine.
2018-05-09 09:23:10 -04:00
Michael Basnight 3b9c8204a6
Add GET Repository High Level REST API (#30362)
This commit adds the Snapshot Client with a first API call within it,
the get repositories call in snapshot/restore module. This also creates
a snapshot namespace for the docs, as well as get repositories docs.

Relates #27205
2018-05-09 07:25:23 -05:00
Yu 106bed90c7 Add `coordinating_only` node selector (#30313)
Today we can execute cluster API actions on only master, data or ingest nodes
using the `master:true`, `data:true` and `ingest:true` filters, but it is not
so easy to select coordinating-only nodes (i.e. those nodes that are neither
master nor data nor ingest nodes). This change fixes this by adding support for
a `coordinating_only` filter such that `coordinating_only:true` adds all
coordinating-only nodes to the set of selected nodes, and 
`coordinating_only:false` deletes them.

Resolves #28831.
2018-05-09 12:14:07 +01:00
Ke Li 0c6789bc72 Use date format in `date_range` mapping before fallback to default (#29310)
If the date format is not forced in query, use the format in mapping before 
fallback to the default format.

Closes #29282
2018-05-09 09:41:44 +02:00
Alexander Reelsen f00890ee38
Watcher: Increase HttpClient parallel sent requests (#30130)
The HTTPClient used in watcher is based on the apache http client. The
current client is using a lot of defaults - which are not always
optimal. Two of those defaults are the maximum number of total
connections and the maximum number of connections to a single route.

If one of those limits is reached, the HTTPClient waits for a connection
to be finished thus acting in a blocking fashion. In order to prevent
this when many requests are being executed, we increase the limit of
total connections as well as the connections per route (a route is
basically an endpoint, which also contains proxy information, not
containing an URL, just hosts).

On top of that an additional option has been set to evict
long running connections, which can potentially be reused after some
time. As this requires an additional background thread, this required
some changes to ensure that the httpclient is closed properly. Also the
timeout for this can be configured.
2018-05-09 09:37:47 +02:00
Nik Everett b062ce5634
Client: Deprecate many argument performRequest (#30315)
Deprecate the many arguments versions of `performRequest` and
`performRequestAsync` in favor of the `Request` object flavored variants
introduced in #29623. We'll be dropping the many arguments variants in
7.0 because they make it difficult to add new features in a backwards
compatible way and they create a *ton* of intellisense noise.
2018-05-08 14:38:55 -04:00
Nik Everett d20e8e2bb4
Docs: Use task_id in examples of tasks (#30436)
We had been using `task_id:1` or `taskId:1` because it is parses as a
valid task identifier but the `:1` part is confusing. This replaces
those examples with `task_id` which matches the response from the list
tasks API.

Closes #28314
2018-05-08 14:23:32 -04:00
Karim Frenn 3acca0b35c [Docs] Fix typo in cardinality-aggregation.asciidoc (#30434) 2018-05-08 16:12:36 +02:00
aditya-agrawal 27ddb4ffea Avoid NPE in `more_like_this` when field has zero tokens (#30365)
Fixes and edge case when using `more_like_this` where TermVectorsWriter
could throw an NPE when a field produced zero tokens after analysis. This
changes the implementation to use an empty list of tokens in this case.

Closes #30148
2018-05-08 15:13:07 +02:00
Yannick Welsch 82b251adcf
Auto-expand replicas when adding or removing nodes (#30423)
Auto-expands replicas in the same cluster state update (instead of a follow-up reroute) where nodes are added or removed.

Closes #1873, fixing an issue where nodes drop their copy of auto-expanded data when coming up, only to sync it again later.
2018-05-07 22:26:31 +02:00
Igor Motov 44c6dcf5be Docs: fix changelog merge 2018-05-07 14:25:03 -04:00
Igor Motov 6fb189ce47
Add stricter geohash parsing (#30376)
Adds verification that geohashes are not empty and contain only
valid characters. It fixes the issue when en empty geohash is
treated as [-180, -90] and geohashes with non-geohash character
are getting resolved into invalid coordinates.

Closes #23579
2018-05-07 13:56:39 -04:00
javanna c9f5a7893b [DOCS] convert forcemerge snippet
Relates to #30113
2018-05-07 16:09:03 +02:00
Matija Bruncic e5653e635d Update forcemerge.asciidoc (#30113) 2018-05-07 14:56:12 +02:00
Dave Moore 391bcbcbe1 Added zentity to the list of API extension plugins (#29143) 2018-05-07 14:46:47 +02:00
Ke Li d373e1b49c Fix the search request default operation behavior doc (#29302) (#29405) 2018-05-07 14:43:45 +02:00
Tanguy Leroux 1987d6261f
Do not fail snapshot when deleting a missing snapshotted file (#30332)
When deleting or creating a snapshot for a given shard, elasticsearch 
usually starts by listing all the existing snapshotted files in the repository. 
Then it computes a diff and deletes the snapshotted files that are not 
needed anymore. During this deletion, an exception is thrown if the file 
to be deleted does not exist anymore.

This behavior is challenging with cloud based repository implementations 
like S3 where a file that has been deleted can still appear in the bucket for 
few seconds/minutes (because the deletion can take some time to be fully 
replicated on S3). If the deleted file appears in the listing of files, then the 
following deletion will fail with a NoSuchFileException and the snapshot 
will be partially created/deleted.

This pull request makes the deletion of these files a bit less strict, ie not 
failing if the file we want to delete does not exist anymore. It introduces a 
new BlobContainer.deleteIgnoringIfNotExists() method that can be used 
at some specific places where not failing when deleting a file is 
considered harmless.

Closes #28322
2018-05-07 09:35:55 +02:00
Nhat Nguyen 3e58463256 DOCS: Correct mapping tags in put-template api
The mapping tags were not named consistently and not linked correctly.

Relates #30400
2018-05-06 15:56:49 -04:00
Nhat Nguyen eed8a3b585
Add put index template api to high level rest client (#30400)
Relates #27205
2018-05-06 09:47:36 -04:00
Jim Ferenczi d3ee35ef18 [Docs] Add snippets for POS stop tags default value
relates #30397
2018-05-05 07:53:50 +02:00
Jason Tedor 10fcf30ce1 Move respect accept header on no handler to 6.3.1
This commit moves the changelog entry for the change to respect the
accept header on no handler from the 7.0.0 section of the docs to 6.3.1.
2018-05-04 20:50:30 -04:00
Jason Tedor beee5fe004
Respect accept header on no handler (#30383)
Today when processing a request for a URL path for which we can not find
a handler we send back a plain-text response. Yet, we have the accept
header in our hand and can respect the accepted media type of the
request. This commit addresses this.
2018-05-04 18:13:50 -04:00
Jim Ferenczi ec187ed3be [Docs] Fix bad link
relates #30397
2018-05-04 22:07:12 +02:00
Jim Ferenczi d7c2a99347 [Docs] Fix end of section in the korean plugin docs
relates #30397
2018-05-04 21:41:50 +02:00
Jim Ferenczi 891d3bd9c3
Expose the Lucene Korean analyzer module in a plugin (#30397)
This change adds a new plugin called `analysis-nori` that exposes
Korean text analysis in es using the new Lucene Korean analyzer module named (`nori`).
The plugin adds:
* a Korean analyzer: `nori`
* a Korean tokenizer: `nori_tokenizer`
* a part of speech stop filter: `nori_part_of_speech`
* a filter that can replace Hanja characters with their Hangul transcription: `nori_readingform`
2018-05-04 20:46:13 +02:00
Zachary Tong 1c0d339904
[Rollup] Validate timezone in range queries (#30338)
When validating the search request, we make sure any date_histogram
aggregations have timezones that match the jobs.  But we didn't
do any such validation on range queries.

While it wouldn't produce incorrect results, it would be confusing
to the user as no documents would match the aggregation (because we
add a filter clause on the timezone for the agg).

Now the user gets an exception up front, and some helpful text about
why the range query didnt match, and which timezones are acceptable
2018-05-04 10:45:16 -07:00
tomcallahan 0a93956194
Add Get Settings API support to java high-level rest client (#29229)
This PR adds support for the Get Settings API to the java high-level rest client.
Furthermore, logic related to the retrieval of default settings has been moved from the rest layer into the transport layer and now default settings may be retrieved consistency via both the rest API and the transport API.
2018-05-04 11:14:28 -04:00
Jim Ferenczi dbd857341f
Upgrade to 7.4.0-snapshot-1ed95c097b (#30357)
Upgrade to lucene-7.4.0-snapshot-1ed95c097b

This version contains:
* An Analyzer for Korean
* An IntervalQuery and IntervalsSource that retrieve minimum intervals of positional queries.
* A new API to retrieve matches (offsets and positions) of a query for a single document.
* Support for soft deletes in the index writer.
* A fixed shingle filter that handles index time synonyms.
* Support for emoji sequence in ICUTokenizer (with an upgrade to icu 61.1)
2018-05-04 11:44:22 +02:00
lcawley 137ce702a4 [DOCS] Added coming qualifiers in changelog 2018-05-03 19:38:19 -07:00
debadair 19624466e8
[DOCS] Commented out empty sections in the changelog to fix the doc build. (#30372) 2018-05-03 16:31:32 -07:00
Jay Modi aa0d7c73f8
Security: reduce garbage during index resolution (#30180)
The IndexAndAliasesResolver resolves the indices and aliases for each
request and also handles local and remote indices. The current
implementation uses the ResolvedIndices class to hold the resolved
indices and aliases. While evaluating the indices and aliases against
the user's permissions, the final value for ResolvedIndices is
constructed. Prior to this change, this was done by creating a
ResolvedIndices for the first set of indices and for each additional
addition, a new ResolvedIndices object is created and merged with
the existing one. With a small number of indices and aliases this does
not pose a large problem; however as the number of indices/aliases
grows more list allocations and array copies are needed resulting in a
large amount of garbage and severely impacted performance.

This change introduces a builder for ResolvedIndices that appends to
mutable lists until the final value has been constructed, which will
ultimately reduce the amount of garbage generated by this code.
2018-05-03 12:48:23 -06:00
Sue Gallagher 09a6ba4fea
Change quad tree max levels to 29. Closes #21191 (#29663)
* [DOCS] Changed quad tree max levels to 29. Clears 21191

* Changed QuadPrefixTree max levels to 29 and added defaults. Closes #21191
2018-05-03 09:48:21 -07:00
Dimitris Athanasiou 3b260dcfc1
[ML] Account for gaps in data counts after job is reopened (#30294)
This commit fixes an issue with the data diagnostics were
empty buckets are not reported even though they should. Once
a job is reopened, the diagnostics do not get initialized from
the current data counts (especially the latest record timestamp).
The result is that if the data that is sent have a time gap compared
to the previous ones, that gap is not accounted for in the empty bucket
count.

This commit fixes that by initializing the diagnostics with the current
data counts.

Closes #30080
2018-05-03 15:08:24 +01:00
wmellouli c8d8407012 [Docs] Add term query with normalizer example 2018-05-03 10:23:14 +02:00
Alexander Reelsen 2c38d12e23
Watcher: Make start/stop cycle more predictable and synchronous (#30118)
The current implementation starts/stops watcher using an executor. This
can result in our of order operations.

This commit reduces those executor calls to an absolute minimum in order
to be able to do state changes within the cluster state listener method,
which runs in sequence.

When a state change occurs that forces the watcher service to pause
(like no watcher index, no master node, no local shards), the service is
now in a paused state.

Pausing is a super lightweight operation, which marks the
ExecutionService as paused and waits for the currently executing watches
to finish in the background via an executor. The same applies for
stopping, the potentially long running operation is outsourced in to an
executor, as waiting for executed watches is decoupled from the current
state.

The only other long running operation is starting, where watches need to
be loaded. This is also done via an executor, but has an additional
protection by checking the cluster state version it was started with. If
another cluster state version was trying to load the watches, then this
loading will not take effect.

This PR also cleans up some unused states, like the a simple boolean in
the HistoryStore/TriggeredWatchStore marking it as started or stopped,
as this can now be caught in the execution service.

Another advantage of this approach is the fact, that now only triggered
watches are not getting executed, while watches that are run via the
Execute Watch API will still be executed regardless if watcher is
stopped or not.

Lastly the TickerScheduleTriggerEngine thread now only starts on data nodes.
2018-05-03 09:47:12 +02:00
Zachary Tong 3c2d2a7d4a
Fix NPE when CumulativeSum agg encounters null/empty bucket (#29641)
Fix NPE when CumulativeSum agg encounters null/empty bucket

If the cusum agg encounters a null value, it's because the value is
missing (like the first value from a derivative agg), the path is
not valid, or the bucket in the path was empty.

Previously cusum would just explode on the null, but this changes it
so we only increment the sum if the value is non-null and finite.
This is safe because even if the cusum encounters all null or empty
buckets, the cumulative sum is still zero (like how the sum agg returns
zero even if all the docs were missing values)

I went ahead and tweaked AggregatorTestCase to allow testing pipelines,
so that I could delete the IT test and reimplement it as AggTests.

Closes #27544
2018-05-02 12:22:55 -07:00
Ryan Ernst fb0aa562a5
Network: Remove http.enabled setting (#29601)
This commit removes the http.enabled setting. While all real nodes (started with bin/elasticsearch) will always have an http binding, there are many tests that rely on the quickness of not actually needing to bind to 2 ports. For this case, the MockHttpTransport.TestPlugin provides a dummy http transport implementation which is used by default in ESIntegTestCase.

closes #12792
2018-05-02 11:42:05 -07:00
Lisa Cawley 0d7ac9a74c
[DOCS] Enables edit links for X-Pack pages (#30278) 2018-05-02 10:13:42 -07:00
Ryan Ernst fba2f00a73
Packaging: Unmark systemd service file as a config file (#29004)
Systemd overrides should happen through /etc/systemd/system, not
directly editing the service file. This commit removes marking the
service file as configuration for rpm and deb packages.
2018-05-02 09:48:49 -07:00
Ryan Ernst 62f2918abc
Added changelog entry for deb prerelease version change (#30184)
This commit adds a changelog entry for the change in #29000.
2018-05-02 09:00:35 -07:00
Conor Landry 5deda6929a [Docs] Clarify `fuzzy_like_this` redirect (#30183) 2018-05-02 11:45:37 +02:00
Adrien Grand 5991ede9ef Fix docs of the `_ignored` meta field.
Relates #29658
2018-05-02 11:43:50 +02:00
Adrien Grand 7358946bda
Add a new `_ignored` meta field. (#29658)
This adds a new `_ignored` meta field which indexes and stores fields that have
been ignored at index time because of the `ignore_malformed` option. It makes
malformed documents easier to identify by using `exists` or `term(s)` queries
on the `_ignored` field.

Closes #29494
2018-05-02 10:47:02 +02:00
Lisa Cawley fd20370145
[DOCS] Adds new installation package details (#29590) 2018-05-01 17:04:16 -07:00
Lisa Cawley 092dd6cb89
[DOCS] Removes X-Pack Elasticsearch release notes (#30272) 2018-05-01 16:05:23 -07:00
Julie Tibshirani 5c9f08402e
Correct an example in the top-level suggester documentation. (#30224) 2018-05-01 15:16:28 -07:00
Julie Tibshirani 6506edfd9c
Fix a reference to match_phrase_prefix in the match query docs. (#30282) 2018-05-01 13:46:33 -07:00
Lisa Cawley c5dc60718f
[DOCS] Fix 6.4-specific link in changelog (#30314) 2018-05-01 13:20:19 -07:00
Nik Everett 0be443c5bb
REST Client: Add Request object flavored methods (#29623)
Adds two new methods to `RestClient` that take a `Request` object. These
methods will allows us to add more per-request customizable options
without creating more and more and more overloads of the `performRequest`
and `performRequestAsync` methods. These new methods look like:

```
Response performRequest(Request request)
```

and

```
void performRequestAsync(Request request, ResponseListener responseListener)
```

This change doesn't add any actual features but enables adding things like
per request timeouts and per request node selectors. This change *does*
rework the `HighLevelRestClient` and its tests to use these new `Request`
objects and it does update the docs.
2018-05-01 14:31:23 -04:00
Lisa Cawley 5b5c98c96b
[DOCS] Adds changelog to Elasticsearch Reference (#30271) 2018-05-01 10:34:26 -07:00
Boaz Leskes f4901b8a19 Remove usage of transient settings to enable allocations in rolling upgrade docs (#29671)
Since we disable allocation using persistent settings, we should be consistent and remove
the setting from the persistent storage. Otherwise an accidental restart will lead for shards
not being allocated.

Relates to #28757
2018-05-01 15:56:14 +02:00
Boaz Leskes 6229c0ce88 Remove usage of transient settings to enable allocations in full restart upgrade docs (#29670)
Since we disable allocation using persistent settings, we should be consistent and remove
the setting from the persistent storage. Otherwise an accidental restart will leed for shards
not being allocated.

Relates to https://github.com/elastic/elasticsearch/pull/28757
2018-05-01 15:21:46 +02:00
Jason Tedor 5de6f4ff7b Adjust copy settings on resize BWC version
This commit adjusts the BWC version for copy settings on resize
operations after the behavior was backported to 6.x.
2018-05-01 08:49:16 -04:00
Jason Tedor 50535423ff
Allow copying source settings on resize operation (#30255)
Today when an index is created from shrinking or splitting an existing
index, the target index inherits almost none of the source index
settings. This is surprising and a hassle for operators managing such
indices. Given this is the default behavior, we can not simply change
it. Instead, we start by introducing the ability to copy settings. This
flag can be set on the REST API or on the transport layer and it has the
behavior that it copies all settings from the source except non-copyable
settings (a property of a setting introduced in this
change). Additionally, settings on the request will always override.

This change is the first step in our adventure:
 - this flag is added here in 7.0.0 and immediately deprecated
 - this flag will be backported to 6.4.0 and remain deprecated
 - then, we will remove the ability to set this flag to false in 7.0.0
 - finally, in 8.0.0 we will remove this flag and the only behavior will
   be for settings to be copied
2018-05-01 08:48:19 -04:00
Paul Sanwald e11070bcfa
Fix macros in changelog (#30269)
remove comments for macros which caused macros not to work correctly
2018-04-30 14:09:32 -07:00
David Turner d553a8be2f
Improve docs for disk watermarks (#30249)
* Clarify that the low watermark does not affect brand-new shards.
* Replace ES -> Elasticsearch.
* Format to 80 columns.

Resolves #25163
2018-04-30 17:31:11 +01:00
Chris Earle 725a5af2c6
_cluster/state should always return cluster_uuid (#30143)
Currently, the only way to get the REST response for the `/_cluster/state`
call to return the `cluster_uuid` is to request the `metadata` metrics,
which is one of the most expensive response structures. However, external
monitoring agents will likely want the `cluster_uuid` to correlate the
response with other API responses whether or not they want cluster
metadata.
2018-04-30 10:16:11 -04:00
David Turner 5e4d0b4510
Minor tweaks to reroute documentation (#30246)
Add yet another warning about data loss to the introductory paragraph about the
unsafe commands. Also move this paragraph next to the details of the unsafe
commands, below the section on the `retry_failed` flag.

Be more specific about how to use the URI parameters and in-body flags.

Clarify statements about when rebalancing takes place (i.e. it respects
settings)

Resolves #16113.
2018-04-30 13:09:03 +01:00
Jason Tedor 811f5b4efc
Do not ignore request analysis/similarity on resize (#30216)
Today when a resize operation is performed, we copy the analysis,
similarity, and sort settings from the source index. It is possible for
the resize request to include additional index settings including
analysis, similarity, and sort settings. We reject sort settings when
validating the request. However, we silently ignore analysis and
similarity settings on the request that are already set on the source
index. Since it is possible to change the analysis and similarity
settings on an existing index, this should be considered a bug and the
sort of leniency that we abhor. This commit addresses this bug by
allowing the request analysis/similarity settings to override the
existing analysis/similarity settings on the target.
2018-04-30 07:31:36 -04:00
Jason Tedor f381e2a00c
Add migration note on thread pool API changes (#29192)
A previous change modified the output of the thread pool info contained
in the nodes info API. This commit adds a note to the migration docs for
this change.
2018-04-28 00:11:17 -04:00
Julie Tibshirani f5978d6d33
In the field capabilities API, remove support for providing fields in the request body. (#30185) 2018-04-27 16:14:11 -07:00
Jason Tedor 4494565d8e
Bump changelog version to 6.4 (#30217)
This commit bumps the changelog version to 6.4 as now that 6.3 is
feature frozen there would be no additional entries in the changelog for
6.3.0.
2018-04-27 16:22:27 -04:00
Nik Everett f4ed902698
CCS: Drop http address from remote cluster info (#29568)
They are expensive to fetch and no longer needed by Kibana so they
*shouldn't* be needed by anyone else either.

Closes #29207
2018-04-27 14:19:00 -04:00
Sue Gallagher dd666599f7
[DOCS] Added 'on a single shard' to description of max_thread_count. Closes 28518 (#29686) 2018-04-27 09:29:27 -07:00
Tanguy Leroux 63148dd9ba
Fail snapshot operations early on repository corruption (#30140)
A NullPointerException is thrown when trying to create or delete
a snapshot in a repository that has been written to by an older 
Elasticsearch after writing to it with a newer Elasticsearch version.

This is because the way snapshots are formatted in the repository 
snapshots index file changed in #24477.

This commit changes the parsing of the repository index file so that 
it now detects a corrupted index file and fails early the snapshot 
operation.

closes #29052
2018-04-27 16:29:59 +02:00
Nik Everett a7e69b07a1 Docs: Document `failures` on reindex and friends
We already had *some* documentation of the batch nature of `reindex` and
friends but it wasn't super obvious how it interacted with the
`failures` element in the response. This adds some more documentation
the `failures` element.
2018-04-27 10:07:38 -04:00
Alexander Reelsen e1a16a6018
REST: Remove GET support for clear cache indices (#29525)
Clearing the cache indices can be done via GET and POST. As GET should
only support read only operations, this removes the support for using
GET for clearing the indices caches.
2018-04-27 08:41:36 +02:00
Christoph Büscher 21dbf9fab0
Document time unit limitations for date histograms (#30177)
Adding some allowed abbreviated values for intervals in date histograms
as well as documenting the limitations of intervals larger than days.

Closes #23294
2018-04-26 19:44:21 +02:00
Julie Tibshirani d40116d260
Add support for field capabilities to the high-level REST client. (#29664) 2018-04-26 09:50:37 -07:00
David Turner 05ef60135b
Reinstate missing documentation (#28781)
The documentation for settings index.routing.allocation.enable,
index.routing.rebalance.enable and index.gc_deletes was lost in
f123a53d72. This change reinstates it.
2018-04-26 11:42:23 +01:00
Saren Currie 0b4d2f5225 Clarify documentation of scroll_id (#29424)
* Clarify documentation of scroll_id

The Scroll API may return the same scroll ID for multiple requests due to server side state. This is not clear from the current documentation.

* Further clarify scroll ID return behaviour
2018-04-26 09:45:48 +01:00
Julie Tibshirani 32dfb65144 In the field capabilities API, deprecate support for providing fields in the request body. (#30157)
(cherry picked from commit d8d884b29d4aa7d01070484fee5de8d3db60cb25)
2018-04-25 23:01:53 -07:00
Jason Tedor a103533f1d
Correct transport compression algorithm in docs (#29645)
We use DEFLATE when compressing byte streams on the transport layer yet
the docs say we use LZF. This commit correct this.
2018-04-25 15:49:54 -04:00
Jack Conradson 5a9a1cda53
Painless: Docs Clean Up (#29592)
As part of the lang spec: separated identifiers into its own section, and cleaned up variables section.
2018-04-25 09:38:41 -07:00
Adrien Grand 0a5a9a2086 Remove reference to `not_analyzed`.
Relates #30122.
2018-04-25 15:00:53 +02:00
ZarHenry96 d9ea0dd6c3 [Docs] Add community analysis plugin (#29612) 2018-04-25 14:11:35 +02:00
Jason Tedor 2c3e71f116
Remove the suggest metric from stats APIs (#29635)
This metric previously existed for backwards compatibility reasons
although the suggest stats were folded into search stats. This metric
was deprecated in 6.3.0 and this commit removes them for 7.0.0.
2018-04-24 19:03:48 -04:00
Jason Tedor d99d0fa669 Add distribution type to startup scripts
This commit adds the distribution type to the startup scripts so that we
can discern from log output and the main response the type of the
distribution (deb/rpm/tar/zip).
2018-04-20 15:34:01 -07:00
Jason Tedor e64e6d8996 Add distribution flavor to startup scripts
This commit adds the distribution flavor (default versus oss) to the
build process which is passed through the startup scripts to
Elasticsearch. This change will be used to customize the message on
attempting to install/remove x-pack based on the distribution flavor.
2018-04-20 15:33:58 -07:00
Ryan Ernst fab5e21e7d Build: Split distributions into oss and default
This commit makes x-pack a module and adds it to the default
distrubtion. It also creates distributions for zip, tar, deb and rpm
which contain only oss code.
2018-04-20 15:33:57 -07:00
Jason Tedor 0045111ce2 Deprecate the suggest metrics (#29627)
The suggest stats were folded into the search stats as part of the
indices stats API in 5.0.0. However, the suggest metric remained as a
synonym for the search metric for BWC reasons. This commit deprecates
usage of the suggest metric on the indices stats API.

Similarly, due to the changes to fold the suggest stats into the search
stats, requesting the suggest index metric on the indices metric on the
nodes stats API has produced an empty object as the response since
5.0.0. This commit deprecates this index metric on the indices metric on
the nodes stats API.
2018-04-20 09:47:38 -04:00
Ryan Ernst 7975280383
Remove remaining tribe node references (#29574)
While tribe node was removed in
https://github.com/elastic/elasticsearch/pull/28443, there remained a
couple lingering references to it in docs and code. This commit removes
those remaining references.
2018-04-19 18:02:01 -07:00
Jason Tedor 5d767e449a
Remove bulk fallback for write thread pool (#29609)
The name of the bulk thread pool was renamed to "write" with "bulk" as a
fallback name. This change was made in 6.x for BWC reasons yet in 7.0.0
we are removing this fallback. This commit removes this fallback for the
write thread pool.
2018-04-19 16:59:58 -04:00
Jason Tedor a28c0d2271 Remove extra spaces from changelog
This commit removes to extra spaces at the end of lines in the
changelog.
2018-04-19 15:10:19 -04:00
Julie Tibshirani b9e1a00213
Add support to match_phrase query for zero_terms_query. (#29598) 2018-04-19 11:25:27 -07:00
Adrien Grand 6e62b481b4
Update plan for the removal of mapping types. (#29586)
8.x will no longer allow types in APIs and 7.x will issue deprecation warnings
when `include_type_name` is set to `false`.
2018-04-19 15:09:14 +02:00
Christoph Büscher 124fecd221 [Docs] Add rankEval method for Jva HL client
This change adds documentation about using the ranking evaluation API
from the high level Java REST client.

Closes #28694
2018-04-19 14:39:42 +02:00
Jason Tedor c12c2a6cc9 Rename the bulk thread pool to write thread pool (#29593)
This commit renames the bulk thread pool to the write thread pool. This
is to better reflect the fact that the underlying thread pool is used to
execute any document write request (single-document index/delete/update
requests, and bulk requests).

With this change, we add support for fallback settings
thread_pool.bulk.* which will be supported until 7.0.0.

We also add a system property so that the display name of the thread
pool remains as "bulk" if needed to avoid breaking users.
2018-04-19 08:18:58 -04:00
Martijn van Groningen 8afa7c174f
Added painless execute api. (#29164)
Added an api that allows to execute an arbitrary script and a result to be returned.

```
POST /_scripts/painless/_execute
{
  "script": {
    "source": "params.var1 / params.var2",
    "params": {
      "var1": 1,
      "var2": 1
    }
  }
}
```

Relates to #27875
2018-04-19 09:33:34 +02:00
debadair 0c9baebe15
[DOCS] Added include for internal highlighters section. (#29597) 2018-04-18 16:56:09 -07:00
Mayya Sharipova bf6cfff080
[DOCS] Update highlighting docs (#28802)
- add more explanation to some highlighting parameters
- add a document describing how highlighters work internally
2018-04-18 17:41:19 -04:00
Paul Sanwald 3e7fccddaf
Add a CHANGELOG file for release notes. (#29450)
* Add a CHANGELOG file for 7.x release notes.

* update file to include 6.x

* remove confusing comment and small edit to section title

* moving CHANGELOG file under docs directory, as it pertains to release notes.
2018-04-18 07:42:05 -07:00
Jason Tedor 2b47d67d95
Remove the index thread pool (#29556)
Now that single-document indexing requests are executed on the bulk
thread pool the index thread pool is no longer needed. This commit
removes this thread pool from Elasticsearch.
2018-04-18 09:18:08 -04:00
Jack Conradson c833167d84
Painless Spec Documentation Clean Up (#29441)
Created a flatter structure for the different sections.  Cleaned up comments, keywords, and literals.  Used callouts for examples where it made sense.
2018-04-17 12:16:08 -07:00
Luca Cavanna 9c8ebb608f
Remove `flatSettings` support from request classes (#29560)
As part of adding support for new API to the high-level REST client,
we added support for the `flat_settings` parameter to some of our
request classes. We added documentation that such flag is only ever
read by the high-level REST client, but the truth is that it doesn't
do anything given that settings are always parsed back into a `Settings`
object, no matter whether they are returned in a flat format or not.

It was a mistake to add support for this flag in the context of the
high-level REST client, hence this commit removes it.
2018-04-17 18:18:21 +02:00
Igor Motov 983d6c15a2
Add null_value support to geo_point type (#29451)
Adds support for null_value attribute to the geo_point types.

Closes #12998
2018-04-17 10:19:54 -04:00
Adrien Grand 3367948be6
Add documentation about the include_type_name option. (#29555)
This option will be useful in 7.x to prepare for upgrade to 8.0 which won't
know about types anymore.
2018-04-17 15:04:46 +02:00
Jason Tedor faa7fe86c5
Introduce analyze thread pool (#29541)
We want to remove the index thread pool as it is no longer needed since
single-document indexing requests are executed as bulk requests
now. Analyze requests are also executed on the index thread pool though
and they need a thread pool to execute on. The bulk thread does not seem
like the right thread pool, let us keep that thread pool conceptually
for bulk requests and free for bulk requests. None of the existing
thread pools make sense for analyze requests either. The generic thread
pool would be a terrible choice since it has an unbounded queue and that
is a bad idea for user-facing APIs. This commit introduces a small by
default (size=1, queue_size=16) thread pool for analyze requests.
2018-04-17 06:46:15 -04:00
olcbean b3e3b80f1b REST high-level client: add support for Indices Update Settings API [take 2] (#29327)
Relates to #27205
2018-04-16 21:39:11 +02:00
Igor Motov e334baf6fc
Fix overflow error in parsing of long geohashes (#29418)
Fixes a possible overflow error that geohashes longer than 12 characters
can cause during parsing.

Fixes #24616
2018-04-16 12:37:38 -04:00
Ke Li 0bfb59dcf2 Using ObjectParser in UpdateRequest (#29293)
CRUD: Parsing changes for UpdateRequest (#29293)

Use `ObjectParser` to parse `UpdateRequest` so we reject unknown fields
and drop support for the `_fields` parameter because it was deprecated
in 5.x.
2018-04-16 08:39:35 -04:00
Bolarinwa Saheed Olayemi 7931cf87f0 [Docs] Add definitions to glossary (#29127)
Definitions for "filter" and "query" are added to the glossary of terms.

Closes #29127
2018-04-16 10:49:54 +02:00
Mayya Sharipova 5dcfdb09cb
Control max size and count of warning headers (#28427)
Control max size and count of warning headers

Add a static persistent cluster level setting
"http.max_warning_header_count" to control the maximum number of
warning headers in client HTTP responses.
Defaults to unbounded.

Add a static persistent cluster level setting
"http.max_warning_header_size" to control the maximum total size of
warning headers in client HTTP responses.
Defaults to unbounded.

With every warning header that exceeds these limits,
a message will be logged in the main ES log,
and any more warning headers for this response will be
ignored.
2018-04-13 05:55:33 -04:00
Adrien Grand ebd6b5b7ba
Deprecate filtering on `_type`. (#29468)
As indices are only allowed to have one type now, and types are going away in
the future, we should deprecate filtering by `_type`.

Relates #15613
2018-04-13 09:07:51 +02:00
Jason Tedor d44bbb7602
Fix auto-generated ID example format (#29461)
This commit fixes the format of an example auto-generated ID in the docs
where the format misleadingly looked like a UUID rather than a base64
ID.
2018-04-12 22:33:38 -04:00
Sachin Frayne 183ba9a7e5 Fix typo in max number of threads check docs (#29469)
Historically, the bootstrap checks used 2048 as the minimum limit for
the maximum number of threads. This limit was guided by the fact that
the number of processors was artificially capped at 32. This limit was
removed in 6.0.0 and the minimum limit was raised to 4096 to accommodate
this. However, the docs were not updated and this commit addresses that
miss.
2018-04-12 21:16:37 -04:00
Nik Everett 0a21533097 Docs: Fix callouts after _parent removed
Fix the callouts in the docs for the high level rest client after
_parent support was removed.
2018-04-11 12:56:13 -04:00
Adrien Grand 6a6c0ea5e6
Add an `include_type_name` option. (#29453)
This adds an `include_type_name` option to the `indices.create`,
`indices.get_mapping` and `indices.put_mapping` APIs, which defaults to `true`.
When set to `false`, then mappings will be returned directly in the body of
the `indices.get_mapping` API, without keying them by the type name, the
`indices.create` will expect mappings directly under the `mappings` key, and
the `indices.put_mapping` will use `_doc` as a type name and fail if a `type`
is provided explicitly.

Relates #15613
2018-04-11 15:54:16 +02:00
Adrien Grand 3a147b442a Fix docs build. 2018-04-11 13:48:53 +02:00
Jim Ferenczi 1b6d5e531b
Fail _search request with trailing tokens (#29428)
This change validates that the `_search` request does not have trailing
tokens after the main object and fails the request with a parsing exception otherwise.

Closes #28995
2018-04-11 13:10:22 +02:00
Adrien Grand 4918924fae
Remove legacy mapping code. (#29224)
Some features have been deprecated since `6.0` like the `_parent` field or the
ability to have multiple types per index. This allows to remove quite some
code, which in-turn will hopefully make it easier to proceed with the removal
of types.
2018-04-11 09:41:37 +02:00
Adrien Grand aeac682869
Make purely negative queries return scores of 0. (#26015)
It would make them consistent with queries that are only made of filters.

Closes #23449
2018-04-10 14:31:06 +02:00
Christoph Büscher 9f0c5ccf34 [Docs] Correct typos in rank-eval and multi-search 2018-04-10 12:48:16 +02:00
D Pinto 8d6a368402 [Docs] Correct typo in pipeline.asciidoc (#29431) 2018-04-10 10:42:07 +02:00
Lisa Cawley 28e9ef3c83
[DOCS] Updated installation pages with X-Pack indices (#29077) 2018-04-09 08:19:38 -07:00
yangyang.liu 73019ae887 [Docs] Update painless-lang-spec.asciidoc (#29425)
Remove redundant word.
2018-04-09 10:42:16 +02:00
Van0SS 4310ef0be6 [Docs] Update Copyright notices to 2018 (#29404) 2018-04-06 16:21:20 +02:00
Christoph Büscher 231fd4eb18
Remove `delimited_payload_filter` (#27705)
From 7.0 on, using `delimited_payload_filter` should throw an error. 
It was deprecated in 6.2 in favour of `delimited_payload` (#26625).

Relates to #27704
2018-04-05 18:41:04 +02:00
Jason Tedor 8fdca6a89a
Align cat thread pool info to thread pool config (#29195)
Today we report thread pool info using a common object. This means that
we use a shared set of terminology that is not consistent with the
terminology used to the configure thread pools. This holds in particular
for the minimum and maximum number of threads in the thread pool where
we use the following terminology:
 thread pool info | fixed | scaling
  min                core    size
  max                max     size

A previous change addressed this for the nodes info API. This commit
changes the display of thread pool info in the cat thread pool API too
to be dependent on the type of the thread pool so that we can align the
terminology in the output of thread pool info with the terminology used
to configure a thread pool.
2018-04-03 17:27:26 -04:00
Adrien Grand 569d0c0e89
Improve similarity integration. (#29187)
This improves the way similarities are plugged in in order to:
 - reject the classic similarity on 7.x indices and emit a deprecation
   warning otherwise
 - reject unkwown parameters on 7.x indices and emit a deprecation
   warning otherwise

Even though this breaks the plugin API, I'd like to backport to 7.x so
that users can get deprecation warnings when they are doing something
that will become unsupported in the future.

Closes #23208
Closes #29035
2018-04-03 16:45:25 +02:00
Christoph Büscher dc1c16964a [Docs] Correct experimental note formatting 2018-04-03 16:16:21 +02:00
Ragnar Nevries 5adfe2d8e3 [Docs] Update getting-started.asciidoc (#29294)
Description after first use of search API with body does not refer to verb POST.
2018-04-03 14:57:42 +02:00
Yannick Welsch 3be98edc69
[DOCS] Refer back to index API for full-document updates in _update API section (#28677)
This clarifies how full-document updates are performed in ES.
2018-04-03 13:53:15 +02:00
rzmf 080cefec73 Fix missing comma in ingest-node.asciidoc (#29343) 2018-04-03 11:33:44 +01:00
Adrien Grand 3bdfc8f3fb
Upgrade to lucene-7.3.0-snapshot-98a6b3d. (#29298)
Most notable changes include:
 - this release doesn't have the 7.2.1 version constant so I had to create one
 - spatial4j and jts were upgraded
2018-04-03 09:27:14 +02:00
Jason Tedor 1df43a09b7
Remove HTTP max content length leniency (#29337)
I am not sure why we have this leniency for HTTP max content length, it
has been there since the beginning
(5ac51ee93f) with no explanation of its
source. That said, our philosophy today is different than the philosophy
of the past where Elasticsearch would be quite lenient in its handling
of settings and today we aim for predictability for both users and
us. This commit removes leniency in the parsing of
http.max_content_length.
2018-04-02 20:20:01 -04:00
David Turner 40d19532bc
Clarify expectations of false positives/negatives (#27964)
Today this part of the documentation just says that Geo queries are not 100% 
accurate, but in fact we can be more precise about which kinds of queries see
which kinds of error. This commit clarifies this point.
2018-04-02 10:03:42 +01:00
David Turner 3ca9310aee
Update docs on vertex ordering (#27963)
At time of writing, GeoJSON did not enforce a specific ordering of vertices in
a polygon, but it now does. We occasionally get reports of Elasticsearch
rejecting apparently-valid GeoJSON because of badly oriented polygons, and it's
helpful to be able to point at this bit of the documentation when responding.
2018-04-02 09:59:12 +01:00
Mayya Sharipova e70cd35bda
Revert "REST high-level client: add support for Indices Update Settings API (#28892)" (#29323)
This reverts commit b67b5b1bbd.
2018-03-30 16:26:46 -07:00
Fabien Baligand 199d131385 Improve query string docs (#28882)
fix query string syntax doc when OR operator is missed
2018-03-30 16:36:40 +02:00
Fabien Baligand 437ad06e40 fix query string example for boolean query (#28881) 2018-03-30 15:10:14 +02:00
olcbean b67b5b1bbd REST high-level client: add support for Indices Update Settings API (#28892)
Relates to #27205
2018-03-30 10:53:29 +02:00
Sue Gallagher 5518640d46
[DOCS] Added info on WGS-84. Closes issue #3590 (#29305) 2018-03-29 15:50:05 -07:00
Boaz Leskes eb8b31746a Move trimming unsafe commits from engine ctor to store (#29260)
As follow up to #28245 , this PR removes the logic for selecting the 
right start commit from the Engine constructor in favor of explicitly
trimming them in the Store, before the engine is opened. This makes the
constructor in engine follow standard Lucene semantics and use the last
commit.

Relates #28245
Relates #29156
2018-03-29 13:35:57 -04:00
Bolarinwa Saheed Olayemi a3e5773522 Docs: Link to Ansible playbook for Elasticsearch (#29238)
Links to the official Ansible playbook for Elasticsearch.
2018-03-28 18:18:42 -04:00
Christoph Büscher c3fdf8fbfb [Docs] Fix small typo in ranking evaluation docs 2018-03-28 17:45:44 +02:00
Diwas Joshi 63203b228b [Docs] Update aggregations.asciidoc (#29265)
Add note about accuracy of some aggregation results.
2018-03-28 15:01:45 +02:00
Andrew Banchich d2baf4b191 [Docs] Spelling and grammar changes to reindex.asciidoc (#29232) 2018-03-27 12:17:46 +02:00
Boaz Leskes f5d4550e93
Fold EngineDiskUtils into Store, for better lock semantics (#29156)
#28245 has introduced the utility class`EngineDiskUtils` with a set of methods to prepare/change
translog and lucene commit points. That util class bundled everything that's needed to create and
empty shard, bootstrap a shard from a lucene index that was just restored etc. 

In order to safely do these manipulations, the util methods acquired the IndexWriter's lock. That
would sometime fail due to concurrent shard store fetching or other short activities that require the
files not to be changed while they read from them. 

Since there is no way to wait on the index writer lock, the `Store` class has other locks to make
sure that once we try to acquire the IW lock, it will succeed. To side step this waiting problem, this
PR folds `EngineDiskUtils` into `Store`. Sadly this comes with a price - the store class doesn't and
shouldn't know about the translog. As such the logic is slightly less tight and callers have to do the
translog manipulations on their own.
2018-03-26 14:08:03 +02:00
Jim Ferenczi 5288235ca3
Optimize the composite aggregation for match_all and range queries (#28745)
This change refactors the composite aggregation to add an execution mode that visits documents in the order of the values
present in the leading source of the composite definition. This mode does not need to visit all documents since it can early terminate
the collection when the leading source value is greater than the lowest value in the queue.
Instead of collecting the documents in the order of their doc_id, this mode uses the inverted lists (or the bkd tree for numerics) to collect documents
in the order of the values present in the leading source.
For instance the following aggregation:

```
"composite" : {
  "sources" : [
    { "value1": { "terms" : { "field": "timestamp", "order": "asc" } } }
  ],
  "size": 10
}
```
... can use the field `timestamp` to collect the documents with the 10 lowest values for the field instead of visiting all documents.
For composite aggregation with more than one source the execution can early terminate as soon as one of the 10 lowest values produces enough
composite buckets. For instance if visiting the first two lowest timestamp created 10 composite buckets we can early terminate the collection since it
is guaranteed that the third lowest timestamp cannot create a composite key that compares lower than the one already visited.

This mode can execute iff:
 * The leading source in the composite definition uses an indexed field of type `date` (works also with `date_histogram` source), `integer`, `long` or `keyword`.
 * The query is a match_all query or a range query over the field that is used as the leading source in the composite definition.
 * The sort order of the leading source is the natural order (ascending since postings and numerics are sorted in ascending order only).

If these conditions are not met this aggregation visits each document like any other agg.
2018-03-26 09:51:37 +02:00
Christoph Büscher afe95a7738
[Docs] Add rank_eval size parameter k (#29218)
The rank_eval documentation was missing an explanation of the parameter
`k` that controls the number of top hits that are used in the ranking evaluation.

Closes #29205
2018-03-23 18:04:32 +01:00
Nicholas Knize d400a08788 [DOCS] Remove ignore_z_value parameter link
Removes invalid ignore_z_value parameter link in geo-point.asciidoc.
2018-03-23 11:07:24 -05:00
Jean-Charles Legras 687fe860ac Docs: Update docs/index_.asciidoc (#29172)
Use `_doc` in the routing example instead of `tweet` to agree with the
text and line up with the other examples.
2018-03-23 11:35:10 -04:00
Petr Novák 16bffc7394 Docs: Link C++ client lib elasticlient (#28949)
elasticlient is simple library for simplified work with Elasticsearch in C++
2018-03-23 11:30:01 -04:00
Yannick Welsch 3b8a8867c4
[DOCS] Unregister repository instead of deleting it (#29206)
Relates to #15426
2018-03-23 15:53:36 +01:00
Nik Everett 8c59e43ac7
Docs: HighLevelRestClient#multiSearch (#29144)
Adds docs for `HighLevelRestClient#multiSearch`. Unlike the `multiGet`
docs these are much more sparse because multi-search doesn't support
setting many options on the `MultiSearchRequest` and instead just wraps
a list of `SearchRequest`s.

Closes #28389
2018-03-23 10:11:50 -04:00
Nicholas Knize fede633563 Add Z value support to geo_shape
This enhancement adds Z value support (source only) to geo_shape fields. If vertices are provided with a third dimension, the third dimension is ignored for indexing but returned as part of source. Like beofre, any values greater than the 3rd dimension are ignored.

closes #23747
2018-03-23 08:50:55 -05:00
Jason Tedor 111f0788a2 Add error file docs to important settings
This commit adds the error file documentation to the important settings
docs so that the page is actually visible.
2018-03-22 23:06:53 -04:00
Jason Tedor a9677023da
Add note to low-level client docs for DNS caching (#29213)
This commit adds a note to the low-level REST client docs regarding the
possibility of being impacted by the JVM DNS cache policy under a
default security manager policy.
2018-03-22 21:23:52 -04:00
Jim Ferenczi c93c7f3121
Remove deprecated options for query_string (#29203)
This commit removes some parameters deprecated in 6.x (or 5.x):
`use_dismax`, `split_on_whitespace`, `all_fields` and `lowercase_expanded_terms`.

Closes #25551
2018-03-22 18:37:08 +01:00
Yu 24c8d8f5ef REST high-level client: add force merge API (#28896)
Relates to #27205
2018-03-22 17:17:16 +01:00
Tanguy Leroux 6c3278b8e8 [Docs] Fix missing closing block in cluster/misc.asciidoc 2018-03-22 12:02:53 +01:00
Tanguy Leroux edf27a599e
Add new setting to disable persistent tasks allocations (#29137)
This commit adds a new setting `cluster.persistent_tasks.allocation.enable`
that can be used to enable or disable the allocation of persistent tasks.
The setting accepts the values `all` (default) or `none`. When set to
none, the persistent tasks that are created (or that must be reassigned)
won't be assigned to a node but will reside in the cluster state with
a no "executor node" and a reason describing why it is not assigned:

```
"assignment" : {
  "executor_node" : null,
  "explanation" : "persistent task [foo/bar] cannot be assigned [no
  persistent task assignments are allowed due to cluster settings]"
}
```
2018-03-22 09:18:07 +01:00
Adrien Grand 8f9d2ee4e2
Reject updates to the `_default_` mapping. (#29165)
This will reject mapping updates to the `_default_` mapping with 7.x indices
and still emit a deprecation warning with 6.x indices.

Relates #15613
Supersedes #28248
2018-03-21 10:44:11 +01:00
Adrien Grand 1d6ed824c7
Improve similarity docs. (#29089)
This adds links to the relevant Lucene javadocs and warnings regarding
similarities that might return 0 as a score.

Close #29015
2018-03-21 10:41:10 +01:00
joadha 08c530907a [Docs] Update api.asciidoc (#29166)
The parent page has the same title, and the URL path indicates this is about 
API changes, so include "API" in the title.
2018-03-21 10:14:26 +01:00
Ryan Ernst 744777a6a0
Docs: Add note about missing mapping for doc values field (#29036)
This commit adds a documentation note about the behavior when trying to
access docvalues for a field which does not exist in mappings.

closes #22056
2018-03-20 22:15:34 -07:00
Boaz Leskes 602145a562 Add a note about using the `retry_failed` flag before accepting data loss (#29160) 2018-03-20 18:11:45 +01:00
tnsatish 70f67b17dd Fix typo in percolate-query.asciidoc (#29155) 2018-03-20 16:47:53 +00:00
Luca Cavanna ff09c82319
REST high-level client: add clear cache API (#28866)
* REST high-level client: add clear cache API

Relates to #27205

Also Closes #26947 (rest-spec were outdated)
2018-03-20 10:39:36 +01:00
Ryan Ernst 52a517e216
Docs: Add example of resetting index setting (#29048)
This commit adds an example using `null` to reset an index settings.

closes #22870
2018-03-19 20:04:58 -07:00
Sue Gallagher 3530a676e0
[Docs]Corrected spelling errors. (#28976) 2018-03-19 10:22:40 -07:00
David Turner 7608480a62
Update allocation awareness docs (#29116)
Update allocation awareness docs

Today, the docs imply that if multiple attributes are specified the the
whole combination of values is considered as a single entity when
performing allocation. In fact, each attribute is considered separately. This
change fixes this discrepancy.

It also replaces the use of the term "awareness zone" with "zone or domain", and
reformats some paragraphs to the right width.

Fixes #29105
2018-03-19 07:04:47 +00:00
Jason Tedor 0abf51af3d
Configure error file for archive packages (#29129)
This is a follow up to a previous change which set the error file path
for the package distributions. The observation here is that we always
set the working directory of Elasticsearch to the root of the
installation (i.e., Elasticsearch home). Therefore, we can specify the
error file path relative to this directory and default it to the logs
directory, similar to the package distributions.
2018-03-18 15:33:17 -04:00
Jason Tedor 29fedb2669
Configure heap dump path for archive packages (#29130)
This is a follow up to a previous change which set the heap dump path
for the package distributions. The observation here is that we always
set the working directory of Elasticsearch to to the root of
installation (i.e., Elasticsearch home). Therefore, we can specify the
heap dump path relative to this directory and default it to the data
directory, similar to the package distributions.
2018-03-18 15:32:37 -04:00
Nik Everett 2f21dc7129
Docs: HighLevelRestClient#multiGet (#29095)
Add documentation for HighLevelRestClient#multiGet.

Relates to #28389.
2018-03-16 18:52:34 -04:00
Deb Adair 9b41917266 [DOCS] Removed prerelease footnote from upgrade table. 2018-03-16 10:33:10 -07:00
Nik Everett 762226bee9
Docs: Support triple quotes (#28915)
Adds support for triple quoted strings to the documentation test
generator. Kibana's CONSOLE tool has supported them for a year but we
were unable to use them in Elasticsearch's docs because the process that
converts example snippets into tests couldn't handle this. This change
adds code to convert them into standard JSON so we can pass them to
Elasticsearch.
2018-03-16 12:46:39 -04:00
Adrien Grand 708c06896b
Explain why Elasticsearch doesn't support incremental resharding. (#29082)
I have seen this question a couple times already, most recently at
https://twitter.com/dimosr7/status/973872744965332993

I tried to keep the explanation as simple as I could, which is not always easy
as this is a matter of trade-offs.
2018-03-16 16:50:58 +01:00
Adrien Grand 0755ff425f
Clarify requirements of strict date formats. (#29090)
Closes #29014
2018-03-16 14:39:36 +01:00
Adrien Grand 695ec05160
Clarify that dates are always rendered as strings. (#29093)
Even in the case that the date was originally supplied as a long in the
JSON document.

Closes #26504
2018-03-16 14:34:33 +01:00
Jiri Tyr c713d62f88 [Docs] Fix link to Grok patterns (#29088) 2018-03-16 14:13:17 +01:00
Martijn van Groningen 069a876542
Added minimal docs for reindex api in java-api docs
Additionally:
* Included the existing update by query java api docs in java-api docs.
  (for some reason it was never included, it needed some tweaking and
  then it was good to go)
* moved delete-by-query / update-by-query code samples to java file so
  that we can verify that these samples at least compile.

Closes #24203
2018-03-16 07:42:48 +01:00
Lisa Cawley f82376c4f8
[DOCS] Add X-Pack upgrade details (#29038) 2018-03-15 11:40:20 -07:00
Nik Everett cf60e93a21
Docs: HighLevelRestClient#exists (#29073)
Docs: HighLevelRestClient#exists

Add documentation for `HighLevelRestClient#exists`.

Relates to #28389
2018-03-15 12:10:30 -04:00
Christoph Büscher ae912cbde4
[Docs] Fix Java Api index administration usage (#28260)
The Java API documentation for index administration currenty is wrong because
the PutMappingRequestBuilder#setSource(Object... source) an
CreateIndexRequestBuilder#addMapping(String type, Object... source) methods
delegate to methods that check that the input arguments are valid key/value
pairs. This changes the docs so the java api code examples are included from
documentation integration tests so we detect compile and runtime issues earlier.

Closes #28131
2018-03-14 22:02:06 +01:00