Commit Graph

23642 Commits

Author SHA1 Message Date
javanna 835d8cecdc [TEST] add unit tests for internal TrackingFailureListener
Any provided listener will always be wrapped into FailureTrackingListener to handle retries
2016-07-22 17:31:11 +02:00
javanna d13a3d3761 Reindex from remote: add fallback in case content-type header is not set
We better read the header, but who knows what can happen, maybe headers are filtered out for some reasons and we don't want to run into an NPE, then we fallback to auto-detection.
2016-07-22 16:46:17 +02:00
javanna a6a685b0f6 make Response class final 2016-07-22 16:36:14 +02:00
Simon Willnauer 6adfeb4dde [TEST] Add Netty3RestIT to actually run REST tests on netty3 2016-07-22 16:29:14 +02:00
javanna db8beeba3b Merge branch 'master' into feature/async_rest_client 2016-07-22 15:51:03 +02:00
javanna a579866b42 rename mayRetry to isRetryStatus 2016-07-22 15:49:39 +02:00
javanna 37e075a506 Make SyncResponseListener safer
Throw explicit IllegalStateException in unexpected situations, like where both response and exception are set, or when both are unset. Add unit test for SyncResponseListener.
2016-07-22 15:48:15 +02:00
Boaz Leskes bd574d92ae Verify lower level transport exceptions don't bubble up on disconnects (#19518)
#19096 introduced a generic TCPTransport base class so we can have multiple TCP based transport implementation. These implementations can vary in how they respond internally to situations where we concurrently send, receive and handle disconnects and can have different exceptions. However, disconnects are important events for the rest of the code base and should be distinguished from other errors (for example, it signals TransportMasterAction that it needs to retry and wait for the a (new) master to come back).  Therefore, we should make sure that all the implementations do the proper translation from their internal exceptions into ConnectTransportException which is used externally. 

Similarly we should make sure that the transport implementation properly recognize errors that were caused by a disconnect as such and deal with them correctly. This was, for example, the source of a build failure at https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-intake/1080 , where a concurrency issue cause SocketException to bubble out of MockTcpTransport.

This PR adds a tests which concurrently simulates connects, disconnects, sending and receiving and makes sure the above holds. It also fixes anything (not much!) that was found it.
2016-07-22 14:35:47 +02:00
Tal Levy 19e7b1c737 fix: no other processors should be executed after on_failure is called in a compound processor (#19545) 2016-07-21 14:27:04 -07:00
Ali Beyad 9765b4a6ff Fixes the ActiveShardsObserverIT tests that have a very short index (#19540)
creation timeout so they process the index creation cluster state update
before the test finishes and attempts to cleanup. Otherwise, the index
creation cluster state update could be processed after the test finishes
and cleans up, thereby leaking an index in the cluster state that could
cause issues for other tests that wouldn't expect the index to exist.

Closes #19530
2016-07-21 11:47:21 -04:00
Nik Everett 4d89aa97e9 Vagrant tests should use plugin zips
Fixes failing packaging tests:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+packaging-tests/1049/console
2016-07-21 10:02:51 -04:00
Isabel Drost-Fromm c8d3098d3e Move github review comments to TODOs 2016-07-21 15:44:47 +02:00
Ali Beyad 83a137b25c Fixes REST test that is designed to timeout on index creation by
making the test wait until all urgent requests are completed before
finishing, so that tear down can properly delete the created index
and cleanup. Without this wait, it was possible that the test would
finish and cleanup the deleted indices would happen before the
index creation even processed, causing the test to leave a created
index behind.
2016-07-21 09:14:41 -04:00
David Pilato 3bcf2653bb ping_timeout is documented in discovery-ec2 but does not exist in code
Also mentioned in discovery-gce

Actually ping timeout can be set using `discovery.zen.ping_timeout`.

Closes #16600.
2016-07-21 15:03:56 +02:00
Isabel Drost-Fromm 2d57fd10b1 Merge branch 'master' into feature/rank-eval 2016-07-21 14:54:49 +02:00
Yannick Welsch d4771b993f Use executor's describeTasks method to log task information in cluster service (#19531)
This fixes the log output in some places of ClusterService where the executor's describeTasks wasn't used to log task information.
2016-07-21 14:32:37 +02:00
Sakthipriyan Vairamani 96b0b1091f minor documentation improvements (#19500)
* minor documentation improvements

* remove unnecessary commas
2016-07-21 14:26:56 +02:00
Karel Minarik 8c721b10af Test: Fixed incorrect YAML indentation in the `indices.put_template/10_basic.yaml` test
The Ruby YAML parser ignores the `do` actions when they are not indented,
making the test suite fail.

Related: #19506

Closes #19529
2016-07-21 14:17:17 +02:00
David Pilato e22bd08e3c Clarify cloud.service.name entry
Closes #18577.
2016-07-21 14:10:43 +02:00
javanna 175c327e17 validate bufferLimit is positive in HeapBufferedAsyncResponseConsumer 2016-07-21 13:59:28 +02:00
Isabel Drost-Fromm 2b3abadad1 Merge pull request #19283 from cbuescher/rank-add-restparsing
Query evaluation: Adding rest layer parsing and response rendering
2016-07-21 13:11:31 +02:00
Folusho Oladipo 1e7495a7fa corrected the use of two synonymous words (#19498)
Two synonyms were jointly used in the sentence(i.e "problems" and "issues"), so I deleted one of them.
2016-07-21 12:21:12 +02:00
David Pilato 5e57febe53 Add DiscoveryPlugin interface
So we have a Pull interface easier to use which reduce the need of Guice.

See 2a9d7f68a1 (commitcomment-18335161)
2016-07-21 11:35:29 +02:00
David Pilato 2a9d7f68a1 Move custom name resolver registration to the NetworkModule
As explained in https://github.com/elastic/elasticsearch/pull/15765#discussion_r65804713
2016-07-21 10:27:38 +02:00
David Pilato 11ec3a4af6 Fix path_style_access after merge with master 2016-07-21 09:45:12 +02:00
Simon Willnauer 302c7a521a Fix analyzer alias processing (#19506)
In the lack of tests the analyzer.alias feature was pretty much not working
at all on current master. Issues like #19163 showed some serious problems for users
using this feature upgrading to an alpha version.
This change fixes the processing order and allows aliases to be set for
existing analyzers like `default`. This change also ensures that if `default`
is aliased the correct analyzer is used for `default_search` etc.

Closes #19163
2016-07-21 09:32:47 +02:00
David Pilato 98fd5833cc Merge branch 'master' into pr/15724-gce-network-host-master 2016-07-21 09:29:59 +02:00
Jun Ohtani cebad703fe Analyze: Specify anonymous char_filters/tokenizer/token_filters in the analyze API
Add parser for anonymous char_filters/tokenizer/token_filters
Using Settings in AnalyzeRequest for anonymous definition
Add breaking changes document

Closed #8878
2016-07-21 11:06:36 +09:00
Tal Levy f7cd86ef6d rethrow script compilation exceptions into ingest configuration exceptions (#19318)
* rethrow script compilation exceptions into ingest configuration exceptions
* update readProcessor to rethrow any exception as an ElasticsearchException
2016-07-20 10:37:56 -07:00
Nik Everett 3a82c613e4 Migrate query registration from push to pull
Remove `ParseField` constants used for names where there are no deprecated
names and just use the `String` version of the registration method instead.

This is step 2 in cleaning up the plugin interface for extending
search time actions. Aggregations are next.

This is breaking for plugins because those that register a new query should
now implement `SearchPlugin` rather than `onModule(SearchModule)`.
2016-07-20 12:33:51 -04:00
Yannick Welsch 2cf94d2d8a Fix race in testCreateIndexWaitsForAllActiveShards
When index creation is not acknowledged (due to a very low request timeout) it is possible that the index is still created.
If a subsequent index-exists request completes before the cluster state of the index creation has been fully applied, it
might miss the newly created index.
2016-07-20 18:28:28 +02:00
javanna 59ccc88c73 rename mustRetry method to mayRetry 2016-07-20 16:24:57 +02:00
javanna fccfe7dcb8 RestClient javadocs adjustments 2016-07-20 16:24:29 +02:00
Nik Everett fc4b439635 Remove AggregationStreams and friends
* Remove outdated aggregation registration method
* Remove AggregationStreams
* Adds StreamInput#readNamedWriteableList and
StreamOutput#writeNamedWriteableList convenience methods. We strive to
make the reading and writing from the streams terse so they are easier
to scan visually.
* Remove PipelineAggregatorStreams
* Remove stream info from InternalAggreation.Type
* Remove InternalAggregation#type
* Remove Streamable from PipelineAggregator
* Remove Streamable from MultiBucketsAggregation.Bucket
2016-07-20 09:46:04 -04:00
javanna 569d7b3ecc notify the listener if the request gets cancelled 2016-07-20 15:24:14 +02:00
Daniel Mitterdorfer a4f09d2b81 Restore parameter name auto_generate_phrase_queries (#19514)
During query refactoring the query string query parameter
'auto_generate_phrase_queries' was accidentally renamed
to 'auto_generated_phrase_queries'.

With this commit we restore the old name.

Closes #19512
2016-07-20 13:13:57 +02:00
David Pilato 55887457fa Don't register repository settings in S3 plugin
Follow up for https://github.com/elastic/elasticsearch/pull/17784#discussion_r64575845

Today we are registering repository settings when `S3RepositoryPlugin` starts:

```java
        settingsModule.registerSetting(S3Repository.Repository.KEY_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.SECRET_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.BUCKET_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.ENDPOINT_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.PROTOCOL_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.REGION_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.SERVER_SIDE_ENCRYPTION_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.BUFFER_SIZE_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.MAX_RETRIES_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.CHUNK_SIZE_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.COMPRESS_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.STORAGE_CLASS_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.CANNED_ACL_SETTING);
        settingsModule.registerSetting(S3Repository.Repository.BASE_PATH_SETTING);
```

We don't need to register those settings as they are repository level settings and not node level settings.

Closes #18945.
2016-07-20 11:26:03 +02:00
Adrien Grand 1ed6c5d110 Docs: Add more points to the chart that gives accuracy for the cardinality aggregation.
This also adds instructions how to regenerate the chart.
2016-07-20 10:37:12 +02:00
Martijn van Groningen 9b1a477120 Fix ClusterInfo serialization 2016-07-20 09:16:27 +02:00
Ryan Ernst 48fe4326b0 Merge pull request #19510 from rjernst/mapper_keyword_positions
Mappings: Fix not_analyzed string fields to error when position_increment_gap is set
2016-07-19 23:27:02 -07:00
Ryan Ernst 0f2d7a84a8 Add tests for disabling positions and copy the check to text fields 2016-07-19 19:07:56 -07:00
Ryan Ernst c85cb37cc4 Mappings: Fix not_analyzed string fields to error when position_increment_gap is set
Currently if a string field is not_analyzed, but a
position_increment_gap is set, it will lookup the default analyzer and
set it, along with the position_increment_gap, before the code which
handles setting the keyword analyzer for not_analyzed fields has a
chance to run. This change adds a parsing check and test for that case.
2016-07-19 17:54:13 -07:00
Jason Tedor 128f0276d9 Fix Javadocs for ThreadPool#schedule
This commit fixes an issue with an @throws tag on ThreadPool#schedule
not containing a description.
2016-07-19 18:35:30 -04:00
Jason Tedor 770186f6cf Catch the right rejected execution exception
ThreadPool#schedule can throw a rejected execution exception. Yet, the
rejected execution exception that it throws comes from the EsAbortPolicy
which throws an EsRejectedExecutionException. This exception does not
inherit from RejectedExecutionException so instead we must catch the
former instead of the latter.
2016-07-19 16:45:12 -04:00
Jason Tedor 720b53b018 Handle rejected execution exception on reschedule
A self-rescheduling runnable can hit a rejected execution exception but
this exception goes uncaught. Instead, this exception should be caught
and passed to the onRejected handler. Not catching handling this
rejected execution exception can lead to test failures. Namely, a race
condition can arise between the shutting down of the thread pool and
cancelling of the rescheduling of the task. If another reschedule fires
right as the thread pool is being terminated, the rescheduled task will
be rejected leading to an uncaught exception which will cause a test
failure. This commit addresses these issues.

Relates #19505
2016-07-19 15:35:51 -04:00
Nik Everett 9e2221cae5 Migrate remaining aggregations to NamedWriteable
After this we'll be able to remove AggregationStreams and
PipelineAggregatorStreams.
2016-07-19 14:43:29 -04:00
jaymode 11389638f9 Require executor name when calling scheduleWithFixedDelay
The ThreadPool#scheduleWithFixedDelay method does not make it clear that all scheduled runnable instances
will be run on the scheduler thread. This becomes problematic if the actions being performed include
blocking operations since there is a single thread and tasks may not get executed due to a blocking task.

This change includes a few different aspects around trying to prevent this situation. The first is that
the scheduleWithFixedDelay method now requires the name of the executor that should be used to execute
the runnable. All existing calls were updated to use Names.SAME to preserve the existing behavior.

The second aspect is the removal of using ScheduledThreadPoolExecutor#scheduleWithFixedDelay in favor of
a custom runnable, ReschedulingRunnable. This runnable encapsulates the logic to deal with rescheduling a
runnable with a fixed delay and mimics the behavior of executing using a ScheduledThreadPoolExecutor and
provides a ScheduledFuture implementation that also mimics that of the typed returned by a
ScheduledThreadPoolExecutor.

Finally, an assertion was added to BaseFuture to detect blocking calls that are being made on the scheduler
thread.
2016-07-19 12:47:47 -04:00
Adrien Grand 0854b03f13 Elasticsearch should reject dynamic templates with unknown `match_mapping_type`. #17285
When looking at the logstash template, I noticed that it has definitions for
dynamic temilates with `match_mapping_type` equal to `byte` for instance.
However elasticsearch never tries to find templates that match the byte type
(only long or double as far as numbers are concerned). This commit changes
template parsing in order to ignore bad values of `match_mapping_type` (given
how the logstash template is popular, this would break many upgrades
otherwise). Then I hope to fail the parsing on bad values in 6.0.
2016-07-19 15:38:00 +02:00
David Pilato 32a5e16c7d Merge branch 'master' into pr/s3-path-style-access 2016-07-19 15:32:41 +02:00
javanna a9b5c5adbe restore throws IOException clause on all performRequest sync methods
We throw IOException, which is the exception that is going to be thrown in 99% of the cases. A more generic exception can happen, and if it is a runtime one we just let it bubble up as is, otherwise we wrap it into runtime one so that we don't require to catch Exception everywhere, which seems odd.

Also adjusted javadocs for all performRequest methods
2016-07-19 15:18:05 +02:00