Commit Graph

23634 Commits

Author SHA1 Message Date
Nik Everett fb45f6a8a8 Add authentication to reindex-from-remote
The tests for authentication extend ESIntegTestCase and use a mock
authentication plugin. This way the clients don't have to worry about
running it. Sadly, that means we don't really have good coverage on the
REST portion of the authentication.

This also adds ElasticsearchStatusException, and exception on which you
can set an explicit status. The nice thing about it is that you can
set the RestStatus that it returns to whatever arbitrary status you like
based on the status that comes back from the remote system.
reindex-from-remote then uses it to wrap all remote failures, preserving
the status from the remote Elasticsearch or whatever proxy is between us
and the remove Elasticsearch.
2016-07-27 14:17:41 -04:00
kingrhoton 1307aa7e77 clarify awkward text (#19608) 2016-07-27 20:03:20 +02:00
Jared McQueen d97b3fd817 [docs] missing a comma in the terms aggregation example 2016-07-27 12:59:38 -04:00
Tyler Langlois a449b8aca1 Document proxy flags explicitly and nuke Windows section
Closes #19503
2016-07-27 18:21:20 +02:00
Tyler Langlois 0a9089beb1 Correct Java setting to pass proxy options to plugin install
It appears that `proxyHost` and `proxyPort` are not valid Java
properties; they should be prefixed with either `http` or `https`. There
is no code that the flags pass through in `plugin` that munges the flags
in any way, so they should be passed through as indicated in the Java
docs.

See:
https://docs.oracle.com/javase/7/docs/technotes/guides/net/proxies.html
https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html
2016-07-27 18:17:19 +02:00
Clinton Gormley 8315a64a33 provide code example for processors setting
A simple example but was missing

Closes #19567
2016-07-27 17:54:52 +02:00
Areek Zillur 4e3602a790 Add zero-padding to auto-generated rollover index name increment
closes #19484
2016-07-27 10:50:47 -04:00
Martijn van Groningen 7e3d5b21bb test: fix generic type 2016-07-27 14:56:17 +02:00
Isabel Drost-Fromm 2a148a5b25 Update to current format. 2016-07-27 14:30:14 +02:00
David Pilato 9cb1e79e84 Fix comments and method name 2016-07-27 13:35:58 +02:00
David Pilato fb9bad23de Rename GceMetadataServiceImpl to GceMetadataService
See https://github.com/elastic/elasticsearch/pull/15765/files#r65527203
2016-07-27 13:28:53 +02:00
David Pilato 3d9f2bf531 Revert last change and make generateCustomNameResolvers private in Node class 2016-07-27 12:19:08 +02:00
David Pilato e949101cc7 Move generateCustomNameResolvers to DiscoveryPlugin interface 2016-07-27 11:36:06 +02:00
David Pilato e9339a1960 Merge branch 'master' into pr/15724-gce-network-host-master 2016-07-27 11:24:53 +02:00
David Pilato b62bb47663 Move registerCustomNameResolvers to Node class and rename it 2016-07-27 11:23:25 +02:00
David Pilato 1c03d0bb2a Fix after merge 2016-07-27 11:23:18 +02:00
Martijn van Groningen d1b7a335cb removed useless comment and make sure all public constructors delegate to the package protected constructor. 2016-07-27 10:01:36 +02:00
Martijn van Groningen 2fdf79d8d4 Deprecate template query.
Closes #19390
2016-07-27 09:50:44 +02:00
Colin Goodheart-Smithe 3f344d3154 [DOCS] fix documentation for selecting algorithm for percentiles agg 2016-07-27 08:48:51 +01:00
Martijn van Groningen 24d7fa6d54 ingest: Change the `foreach` processor to use the `_ingest._value` ingest metadata attribute to store the current array element being processed.
Closes #19592
2016-07-27 09:35:09 +02:00
Ali Beyad 21ff90fed3 Fixes debug logging on index creation waiting for shards to be started (#19612) 2016-07-26 19:17:02 -04:00
Boaz Leskes 6f76740a58 await fix testConcurrentSendRespondAndDisconnect 2016-07-26 23:42:10 +02:00
Lee Hinman 0876247bca [TEST] Assert that shard has been released before running truncate tool
It's possible that the shard has been closed but the resources
associated with it have not yet been released. This waits until the
index lock can be obtained before running the tool.
2016-07-26 14:14:04 -06:00
Igor Motov 7275291f35 Tests: add more logging to testCorruptFileThenSnapshotAndRestore
This test fails because of an unknown exceptions in FsService.stats() method, which causes no stats to be returned. With this change the exception that is causing this issue is going to be logged.

Related to #19591 and #17964
2016-07-26 15:08:19 -04:00
kingrhoton 643ccb8cc1 [docs] Switch contraction to possesive 2016-07-26 14:01:30 -04:00
Nik Everett 9270e8b22b Rename client yaml test infrastructure
This makes it obvious that these tests are for running the client yaml
suites. Now that there are other ways of running tests using the REST
client against a running cluster we can't go on calling the shared
client yaml tests "REST tests". They are rest tests, but they aren't
**the** rest tests.
2016-07-26 13:53:44 -04:00
Chris Earle 0553ba9151 [Ingest] Add REST _ingest/pipeline to get all pipelines
This adds an extra REST handler for "_ingest/pipeline" so that users do not need to supply "_ingest/pipeline/*" to get all of them.

- Also adds a teardown section to related REST-tests for ingest.
2016-07-26 13:48:15 -04:00
David Pilato 0d3edee928 Merge branch 'master' into pr/15724-gce-network-host-master 2016-07-26 18:51:01 +02:00
Nik Everett 3c0288ee98 Consolify term and phrase suggester docs
This includes a working example of reverse filters to support
correcting prefix errors.
2016-07-26 12:28:31 -04:00
David Pilato fde15ae470 Move custom name resolvers to NetworkService CTOR
Instead of using NetworkModule we can directly inject them in NetworkService CTOR.

See https://github.com/elastic/elasticsearch/pull/15765#issuecomment-235307974
2016-07-26 18:26:30 +02:00
Christoph Büscher e1415d6519 Merge pull request #19595 from cbuescher/fix-19422
Allow empty json object in request body in `_count` API.
2016-07-26 18:17:52 +02:00
Boaz Leskes fabfd425f0 remove socket timeout from MockTcpTransport
added in b208a7dbae
2016-07-26 18:04:05 +02:00
Boaz Leskes dbdb6341a5 increase logging information in testConcurrentSendRespondAndDisconnect 2016-07-26 18:02:22 +02:00
Boaz Leskes 8151224883 add `Socket closed` variant to NetworkExceptionHelper.isCloseConnectionException 2016-07-26 18:01:57 +02:00
Yannick Welsch 95ae58b531 Set up project substitution for client:transport
This ensures that projects depending on client:transport (such as client:benchmark) properly resolve this dependency.
2016-07-26 17:58:39 +02:00
Lee Hinman e538c1c6d6 Merge remote-tracking branch 'dakrone/translog-cli' 2016-07-26 09:39:11 -06:00
Nik Everett a182e356d3 Fix unit test build failure
We didn't catch the failure because we tested against the fork instead
of master. I think.
2016-07-26 11:35:17 -04:00
Luca Cavanna 8a51cfb5b3 Simplify Sniffer initialization and automatically create the default HostsSniffer (#19599)
Simplify Sniffer initialization and automatically create the default HostsSniffer

Take Sniffer.Builder out to its own top level class. Remove HostsSniffer.Builder and let SnifferBuilder create the default HostsSniffer. This simplifies the Sniffer initialization as the HostsSniffer is not mandatory anymore. It can still be specified though in case the configuration needs to be changed or a different impl has to be used. Also make HostsSniffer an interface.
2016-07-26 17:28:37 +02:00
Alexander Lin 8f2882a442 Add _operation field to index, update, delete responses
Performing the bulk request shown in #19267 now results in the following:
```
{"_index":"test","_type":"test","_id":"1","_version":1,"_operation":"create","forced_refresh":false,"_shards":{"total":2,"successful":1,"failed":0},"status":201}
{"_index":"test","_type":"test","_id":"1","_version":1,"_operation":"noop","forced_refresh":false,"_shards":{"total":2,"successful":1,"failed":0},"status":200}
```
2016-07-26 11:16:19 -04:00
Lee Hinman ac53c90ff4 Add 'elasticsearch-translog' CLI tool with 'translog' command
This adds the `bin/elasticsearch-translate` bin file that will be used
for CLI tasks pertaining to Elasticsearch. Currently it implements only
a single sub-command, `truncate-translog`, that creates a truncated
translog for a given folder.

Here's what running the tool looks like:

```
λ bin/elasticsearch-translog truncate -d data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/
Checking existing translog files
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!   WARNING: Elasticsearch MUST be stopped before running this tool   !
!                                                                     !
!   WARNING:    Documents inside of translog files will be lost       !
!                                                                     !
!   WARNING:          The following files will be DELETED!            !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-10.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-18.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-21.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-12.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-25.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-29.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-2.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-5.tlog
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-41.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-6.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-37.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-24.ckp
--> data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-11.ckp

Continue and DELETE files? [y/N] y
Reading translog UUID information from Lucene commit from shard at [data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/index]
Translog Generation: 3
Translog UUID      : AxqC4rocTC6e0fwsljAh-Q
Removing existing translog files
Creating new empty checkpoint at [data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog.ckp]
Creating new empty translog at [data/nodes/0/indices/P45vf_YQRhqjfwLMUvSqDw/0/translog/translog-3.tlog]
Done.
```

It also includes a `-b` batch operation that can be used to skip the
confirmation diaglog.

Resolves #19123
2016-07-26 08:34:07 -06:00
Daniel Mitterdorfer b208a7dbae Add socket timeout in MockTcpTransport
With this commit we set an explicit socket timeout in
MockTcpTransport to avoid hanging tests in case of disconnections.
2016-07-26 16:04:51 +02:00
Christoph Büscher 4bac61425c Adding unit tests for QueryParseContext 2016-07-26 15:27:25 +02:00
Colin Goodheart-Smithe 7ed64af639 [DOCS] fix callout in buckets path docs 2016-07-26 11:33:54 +01:00
Isabel Drost-Fromm 1080df51fc Merge branch 'master' into docs/add_console_to_search 2016-07-26 11:29:35 +02:00
Colin Goodheart-Smithe cef65eba54 #19571 Add _bucket_count option to buckets_path
Add _bucket_count option to buckets_path
2016-07-26 10:29:27 +01:00
Daniel Mitterdorfer c33f85bc37 Add client benchmark
With this commit we add a benchmark for the new REST client and the
existing transport client.

Closes #19281
2016-07-26 11:01:22 +02:00
Colin Goodheart-Smithe 2c12c3e628 Add _bucket_count option to buckets_path
This change adds a new special path to the buckets_path syntax
`_bucket_count`. This new option will return the number of buckets for a
multi-bucket aggregation, which can then be used in pipeline
aggregations.

Closes #19553
2016-07-26 09:28:21 +01:00
Christoph Büscher b861ec1cc0 Allow empty json object in request body in `_count` API
When the request body is missing, all documents in the target index are counted.
As mentioned in #19422, the same should happen when the request body is an empty
json object. This is also the behaviour for the `_search` endpoint and the two
APIs should behave in the same way.
2016-07-26 09:54:05 +02:00
Martijn van Groningen c7c0faa54d aggs: Changed how `nested` and `reverse_nested` aggs know about their nested depth level.
Before the aggregation tree was traversed to figure out what the parent level is, this commit
changes that by using `NestedScope` to figure out the nested depth level. The big upsides
are that this cleans up `NestedAggregator` (it used a hack to lazily figure out the nested parent filter)
 and this is also what `nested` query uses and therefor the `nested` query can be included inside `nested`
 aggregation and work correctly.

Closes #11749
Closes #12410
2016-07-26 09:04:51 +02:00
Daniel Mitterdorfer 94bc489275 Temporarily increase Netty4RestIT timeout
Recently, we experience timeouts on our Windows build slaves for
Netty4RestIT. Until we have figured out what's going on, we
increase this test suite's timeout temporarily to ensure this
timeout does not mask other problems.
2016-07-26 08:27:47 +02:00