Commit Graph

23767 Commits

Author SHA1 Message Date
Martijn van Groningen 2fdf79d8d4 Deprecate template query.
Closes #19390
2016-07-27 09:50:44 +02:00
Colin Goodheart-Smithe 3f344d3154 [DOCS] fix documentation for selecting algorithm for percentiles agg 2016-07-27 08:48:51 +01:00
Martijn van Groningen 24d7fa6d54 ingest: Change the `foreach` processor to use the `_ingest._value` ingest metadata attribute to store the current array element being processed.
Closes #19592
2016-07-27 09:35:09 +02:00
Ali Beyad 21ff90fed3 Fixes debug logging on index creation waiting for shards to be started (#19612) 2016-07-26 19:17:02 -04:00
Boaz Leskes 6f76740a58 await fix testConcurrentSendRespondAndDisconnect 2016-07-26 23:42:10 +02:00
Lee Hinman 0876247bca [TEST] Assert that shard has been released before running truncate tool
It's possible that the shard has been closed but the resources
associated with it have not yet been released. This waits until the
index lock can be obtained before running the tool.
2016-07-26 14:14:04 -06:00
Igor Motov 7275291f35 Tests: add more logging to testCorruptFileThenSnapshotAndRestore
This test fails because of an unknown exceptions in FsService.stats() method, which causes no stats to be returned. With this change the exception that is causing this issue is going to be logged.

Related to #19591 and #17964
2016-07-26 15:08:19 -04:00
kingrhoton 643ccb8cc1 [docs] Switch contraction to possesive 2016-07-26 14:01:30 -04:00
Nik Everett 9270e8b22b Rename client yaml test infrastructure
This makes it obvious that these tests are for running the client yaml
suites. Now that there are other ways of running tests using the REST
client against a running cluster we can't go on calling the shared
client yaml tests "REST tests". They are rest tests, but they aren't
**the** rest tests.
2016-07-26 13:53:44 -04:00
Chris Earle 0553ba9151 [Ingest] Add REST _ingest/pipeline to get all pipelines
This adds an extra REST handler for "_ingest/pipeline" so that users do not need to supply "_ingest/pipeline/*" to get all of them.

- Also adds a teardown section to related REST-tests for ingest.
2016-07-26 13:48:15 -04:00
David Pilato 0d3edee928 Merge branch 'master' into pr/15724-gce-network-host-master 2016-07-26 18:51:01 +02:00
Nik Everett 3c0288ee98 Consolify term and phrase suggester docs
This includes a working example of reverse filters to support
correcting prefix errors.
2016-07-26 12:28:31 -04:00
David Pilato fde15ae470 Move custom name resolvers to NetworkService CTOR
Instead of using NetworkModule we can directly inject them in NetworkService CTOR.

See https://github.com/elastic/elasticsearch/pull/15765#issuecomment-235307974
2016-07-26 18:26:30 +02:00
Christoph Büscher e1415d6519 Merge pull request #19595 from cbuescher/fix-19422
Allow empty json object in request body in `_count` API.
2016-07-26 18:17:52 +02:00
Boaz Leskes fabfd425f0 remove socket timeout from MockTcpTransport
added in b208a7dbae
2016-07-26 18:04:05 +02:00
Boaz Leskes dbdb6341a5 increase logging information in testConcurrentSendRespondAndDisconnect 2016-07-26 18:02:22 +02:00
Boaz Leskes 8151224883 add `Socket closed` variant to NetworkExceptionHelper.isCloseConnectionException 2016-07-26 18:01:57 +02:00
Yannick Welsch 95ae58b531 Set up project substitution for client:transport
This ensures that projects depending on client:transport (such as client:benchmark) properly resolve this dependency.
2016-07-26 17:58:39 +02:00
Lee Hinman e538c1c6d6 Merge remote-tracking branch 'dakrone/translog-cli' 2016-07-26 09:39:11 -06:00
Nik Everett a182e356d3 Fix unit test build failure
We didn't catch the failure because we tested against the fork instead
of master. I think.
2016-07-26 11:35:17 -04:00
Luca Cavanna 8a51cfb5b3 Simplify Sniffer initialization and automatically create the default HostsSniffer (#19599)
Simplify Sniffer initialization and automatically create the default HostsSniffer

Take Sniffer.Builder out to its own top level class. Remove HostsSniffer.Builder and let SnifferBuilder create the default HostsSniffer. This simplifies the Sniffer initialization as the HostsSniffer is not mandatory anymore. It can still be specified though in case the configuration needs to be changed or a different impl has to be used. Also make HostsSniffer an interface.
2016-07-26 17:28:37 +02:00
Alexander Lin 8f2882a442 Add _operation field to index, update, delete responses
Performing the bulk request shown in #19267 now results in the following:
```
{"_index":"test","_type":"test","_id":"1","_version":1,"_operation":"create","forced_refresh":false,"_shards":{"total":2,"successful":1,"failed":0},"status":201}
{"_index":"test","_type":"test","_id":"1","_version":1,"_operation":"noop","forced_refresh":false,"_shards":{"total":2,"successful":1,"failed":0},"status":200}
```
2016-07-26 11:16:19 -04:00
Lee Hinman ac53c90ff4 Add 'elasticsearch-translog' CLI tool with 'translog' command
This adds the `bin/elasticsearch-translate` bin file that will be used
for CLI tasks pertaining to Elasticsearch. Currently it implements only
a single sub-command, `truncate-translog`, that creates a truncated
translog for a given folder.

Here's what running the tool looks like:

```
λ bin/elasticsearch-translog truncate -d data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/
Checking existing translog files
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!   WARNING: Elasticsearch MUST be stopped before running this tool   !
!                                                                     !
!   WARNING:    Documents inside of translog files will be lost       !
!                                                                     !
!   WARNING:          The following files will be DELETED!            !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-10.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-18.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-21.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-12.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-25.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-29.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-2.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-5.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-41.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-6.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-37.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-24.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-11.ckp

Continue and DELETE files? [y/N] y
Reading translog UUID information from Lucene commit from shard at [data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index]
Translog Generation: 3
Translog UUID      : AxqC4rocTC6e0fwsljAh-Q
Removing existing translog files
Creating new empty checkpoint at [data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog.ckp]
Creating new empty translog at [data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-3.tlog]
Done.
```

It also includes a `-b` batch operation that can be used to skip the
confirmation diaglog.

Resolves #19123
2016-07-26 08:34:07 -06:00
Daniel Mitterdorfer b208a7dbae Add socket timeout in MockTcpTransport
With this commit we set an explicit socket timeout in
MockTcpTransport to avoid hanging tests in case of disconnections.
2016-07-26 16:04:51 +02:00
Christoph Büscher 4bac61425c Adding unit tests for QueryParseContext 2016-07-26 15:27:25 +02:00
Colin Goodheart-Smithe 7ed64af639 [DOCS] fix callout in buckets path docs 2016-07-26 11:33:54 +01:00
Isabel Drost-Fromm 1080df51fc Merge branch 'master' into docs/add_console_to_search 2016-07-26 11:29:35 +02:00
Colin Goodheart-Smithe cef65eba54 #19571 Add _bucket_count option to buckets_path
Add _bucket_count option to buckets_path
2016-07-26 10:29:27 +01:00
Daniel Mitterdorfer c33f85bc37 Add client benchmark
With this commit we add a benchmark for the new REST client and the
existing transport client.

Closes #19281
2016-07-26 11:01:22 +02:00
Colin Goodheart-Smithe 2c12c3e628 Add _bucket_count option to buckets_path
This change adds a new special path to the buckets_path syntax
`_bucket_count`. This new option will return the number of buckets for a
multi-bucket aggregation, which can then be used in pipeline
aggregations.

Closes #19553
2016-07-26 09:28:21 +01:00
Christoph Büscher b861ec1cc0 Allow empty json object in request body in `_count` API
When the request body is missing, all documents in the target index are counted.
As mentioned in #19422, the same should happen when the request body is an empty
json object. This is also the behaviour for the `_search` endpoint and the two
APIs should behave in the same way.
2016-07-26 09:54:05 +02:00
Martijn van Groningen c7c0faa54d aggs: Changed how `nested` and `reverse_nested` aggs know about their nested depth level.
Before the aggregation tree was traversed to figure out what the parent level is, this commit
changes that by using `NestedScope` to figure out the nested depth level. The big upsides
are that this cleans up `NestedAggregator` (it used a hack to lazily figure out the nested parent filter)
 and this is also what `nested` query uses and therefor the `nested` query can be included inside `nested`
 aggregation and work correctly.

Closes #11749
Closes #12410
2016-07-26 09:04:51 +02:00
Daniel Mitterdorfer 94bc489275 Temporarily increase Netty4RestIT timeout
Recently, we experience timeouts on our Windows build slaves for
Netty4RestIT. Until we have figured out what's going on, we
increase this test suite's timeout temporarily to ensure this
timeout does not mask other problems.
2016-07-26 08:27:47 +02:00
Daniel Mitterdorfer d7d3b1a874 Remove duplicate dependency declaration for http client (#19580)
We disable transitive dependencies in our build plugin
for all dependencies except for the group `org.elasticsearch`.
However, in the reindex plugin we depend on the REST client
and declare its dependencies again which is not necessary
(and led to problems with conflicting versions in #19281).

With this PR we remove the duplicate declaration.
2016-07-26 06:29:04 +02:00
Ryan Ernst 0be363d611 Merge pull request #19589 from rjernst/license_header_generic
Allow license header check to be customized
2016-07-25 17:40:41 -07:00
Ryan Ernst 7bf6676d21 Use string concat instead of gstring 2016-07-25 17:30:15 -07:00
Ryan Ernst c2c9b51732 Add javadocs for adding additional license types to the license headers
check
2016-07-25 17:22:08 -07:00
Ryan Ernst 0e6bdd025d Construct hashmap directly instead of using crazy groovy syntax 2016-07-25 17:17:12 -07:00
Ryan Ernst 0ecaa6ec3c Build: Allow license header check to be customized
This change allows setting which license families are approved, as well
as adding matchers for additional license types.
2016-07-25 17:05:40 -07:00
Nik Everett a95d4f4ee7 Add Location header and improve REST testing
This adds a header that looks like `Location: /test/test/1` to the
response for the index/create/update API. The requirement for the header
comes from https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

https://tools.ietf.org/html/rfc7231#section-7.1.2 claims that relative
URIs are OK. So we use an absolute path which should resolve to the
appropriate location.

Closes #19079

This makes large changes to our rest test infrastructure, allowing us
to write junit tests that test a running cluster via the rest client.
It does this by splitting ESRestTestCase into two classes:
* ESRestTestCase is the superclass of all tests that use the rest client
to interact with a running cluster.
* ESClientYamlSuiteTestCase is the superclass of all tests that use the
rest client to run the yaml tests. These tests are shared across all
official clients, thus the `ClientYamlSuite` part of the name.
2016-07-25 17:02:40 -04:00
Boaz Leskes b90dff7292 increase log level to debug in testConcurrentSendRespondAndDisconnect 2016-07-25 22:01:09 +02:00
Lee Hinman 1623cff6c0 Merge remote-tracking branch 'dakrone/bucket-circuit-breaker' 2016-07-25 13:37:26 -06:00
Ali Beyad 2f831c3abb BytesArray tests fix: offsets don't matter on a zero bytes array
Closes #19582
2016-07-25 15:22:08 -04:00
Lee Hinman 124a9fabe3 Circuit break on aggregation bucket numbers with request breaker
This adds new circuit breaking with the "request" breaker, which adds
circuit breaks based on the number of buckets created during
aggregations. It consists of incrementing during AggregatorBase creation

This also bumps the REQUEST breaker to 60% of the JVM heap now.

The output when circuit breaking an aggregation looks like:

```json
{
  "shard" : 0,
  "index" : "i",
  "node" : "a5AvjUn_TKeTNYl0FyBW2g",
  "reason" : {
    "type" : "exception",
    "reason" : "java.util.concurrent.ExecutionException: QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: CircuitBreakingException[[request] Data too large, data for [<agg [otherthings]>] would be larger than limit of [104857600/100mb]];",
    "caused_by" : {
      "type" : "execution_exception",
      "reason" : "QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: CircuitBreakingException[[request] Data too large, data for [<agg [myagg]>] would be larger than limit of [104857600/100mb]];",
      "caused_by" : {
        "type" : "circuit_breaking_exception",
        "reason" : "[request] Data too large, data for [<agg [otherthings]>] would be larger than limit of [104857600/100mb]",
        "bytes_wanted" : 104860781,
        "bytes_limit" : 104857600
      }
    }
  }
}
```

Relates to #14046
2016-07-25 11:33:37 -06:00
Martijn van Groningen a9ab095b8c Moved all mustache classes into one package.
No need for multiple packages inside a small module.
2016-07-25 18:50:12 +02:00
Martijn van Groningen a784055db1 Cleaned up the tests in lang-mustache.
Messy tests with mustache were either moved to core, moved to a rest test or remained untouched if they actually tested mustache.
Also removed tests that were redundant.
2016-07-25 17:57:39 +02:00
Jim Ferenczi 5fc503342a Merge pull request #19579 from jimferenczi/docvalue_fields_fetch
Rename FieldDataFieldsContext and FieldDataFieldsFetchSubPhase in DocValueFieldsContext and DocValueFieldsFetchSubPhase
2016-07-25 17:20:27 +02:00
Tanguy Leroux f745c96949 Clean up more messy tests
After #13834 many tests that used Groovy scripts (for good or bad reason) in their tests have been moved in the lang-groovy module and the issue #13837 has been created to track these messy tests in order to clean them up.

This commit moves more tests back in core, removes the dependency on Groovy, changes the scripts in order to use the mocked script engine, and change the tests  to integration tests.
2016-07-25 17:02:49 +02:00
Jim Ferenczi 33461a8432 Rename FieldDataFieldsContext and FieldDataFieldsFetchSubPhase in DocValueFieldsContext and DocValueFieldsFetchSubPhase
This change renames the package org.elasticsearch.search.fetch.fielddata in org.elasticsearch.search.fetch.docvalues and renames the
FieldData* classes in DocValue*. This is a follow up of the renaming that happened in #18943
2016-07-25 16:20:59 +02:00
David Pilato b62ec1d300 Remove TODO about Timeout in Azure
In #15950 #15080 #16084 we added the support of TimeOut for Requests with a default client`setTimeoutIntervalInMs`.
So we can remove this useless todo which was added for only one method.

Closes #18617.
2016-07-25 16:19:15 +02:00