Commit Graph

23523 Commits

Author SHA1 Message Date
Simon Willnauer 29fd0f1bd8 [TEST] Remove wrong transportName from MockTcpTransport#ctor 2016-07-13 12:50:52 +02:00
Simon Willnauer ae98d59899 Don't assert that files exists if recovery has been cancled
Today we assert that the tmp files are present but if the recovery
was canceled this might not be the case while still a valid state.
This chance only throws the AssertionError if the recovery is still active.
2016-07-13 10:52:17 +02:00
Simon Willnauer 067ca1f996 [TEST] Use a semaphore to block unitl all in-flight requests are released 2016-07-13 10:31:05 +02:00
Simon Willnauer 814c7224f9 Merge pull request #19392 from elastic/modularize_netty
This moves all netty related code into modules/transport-netty the module is build as a zip file as well as a JAR to serve as a dependency for transport client. For the time being this is required otherwise we have no network based impl. for transport client users. This might be subject to change given that we move forward http client.
2016-07-13 09:52:03 +02:00
Martijn van Groningen 2bdc55c9ff fvh: Also extract terms from the nested query' inner query.
Closes #19265
2016-07-13 08:15:46 +02:00
Nik Everett d14e06ce51 Migrate top_hits, histogram, and ip_range aggregations to NamedWriteable
This is just another step towards removing AggregationStreams
in favor of NamedWriteable.
2016-07-12 23:02:32 -04:00
Nik Everett f2978f41b9 Migrate nested, reverse_nested, and children aggregations to NamedWriteable
Just another step in removing AggregationStreams in favor of NamedWriteable.
2016-07-12 22:38:51 -04:00
Nik Everett 06bd896ce0 Migrate geohash_grid and geo_bounds to NamedWriteable
Just another small step in removing Aggregation's custom streams
implementation in favor of NamedWriteable.
2016-07-12 22:22:51 -04:00
Nik Everett f479219ca7 Clean up significant terms aggregation results
* Clean up the generics around significant terms aggregation results
* Reduce code duplicated between `SignificantLongTerms` and
`SignificantStringTerms` by creating `InternalMappedSignificantTerms`
and moving common things there where possible.
* Migrate to `NamedWriteable`
* Line length fixes while I was there
2016-07-12 22:08:09 -04:00
Ryan Ernst 920bd0cf68 Merge pull request #19401 from rjernst/more_plugin_services
Plugins: Add resource watcher to services available for plugin components
2016-07-12 15:28:17 -07:00
Lee Hinman 95cf2407ee Merge remote-tracking branch 'dakrone/include-cluster-info-in-explain-api' 2016-07-12 16:26:46 -06:00
Jason Tedor ce5a382c69 Remove support for properties
This commit removes support for properties syntax and config files:
 - removed support for elasticsearch.properties
 - removed support for logging.properties
 - removed support for properties content detection in REST APIs
 - removed support for properties content detection in Java API

Relates #19398
2016-07-12 17:55:18 -04:00
Lee Hinman 58db63b610 Expose the ClusterInfo object in the allocation explain output
This adds an optional parameter to the cluster allocation explain API
that will return the cluster info object, `include_disk_info`, the
output looks like:

GET /_cluster/allocation/explain?include_disk_info -d'
{"index": "i", "shard": 0, "primary": false}'

{
  ... other info ...

  "cluster_info" : {
    "nodes" : {
      "7Uws-vL7R6WVm3ZwQA1n5A" : {
        "node_name" : "Kraven the Hunter",
        "least_available" : {
          "path" : "/path/to/data1",
          "total_bytes" : 165999570944,
          "used_bytes" : 118180614144,
          "free_bytes" : 47818956800,
          "free_disk_percent" : 28.80667493781158,
          "used_disk_percent" : 71.19332506218842
        },
        "most_available" : {
          "path" : "/path/to/data2",
          "total_bytes" : 165999570944,
          "used_bytes" : 118180614144,
          "free_bytes" : 47818956800,
          "free_disk_percent" : 28.80667493781158,
          "used_disk_percent" : 71.19332506218842
        }
      }
    },
    "shard_sizes" : {
      "[i][2][p]_bytes" : 0,
      "[i][4][p]_bytes" : 130,
      "[i][1][p]_bytes" : 0,
      "[i][3][p]_bytes" : 0,
      "[i][0][p]_bytes" : 130
    },
    "shard_paths" : {
      "[i][3], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=LegZLDniTVaw0Y1urv7s3g]" : "/path/to/data1/nodes/0",
      "[i][1], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=lAU_4vf_SKmoRdtg0ACnjQ]" : "/path/to/data1/nodes/0",
      "[i][2], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=Aurpeuj7SeGeyPDDpCtRgg]" : "/path/to/data1/nodes/0",
      "[i][0], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=Vgg8GlQTQ82C2j6HYBq8DQ]" : "/path/to/data1/nodes/0",
      "[i][4], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=t8hQlVSxQe-58fSeaXcAqg]" : "/path/to/data1/nodes/0"
    }
  }
}

Resolves #14405
2016-07-12 15:52:20 -06:00
Ryan Ernst 9b5ac4f68e Plugins: Add resource watcher to services available for plugin components 2016-07-12 14:51:50 -07:00
Simon Willnauer ce28aa9160 [TEST] HttpCompressionIT is a real IT 2016-07-12 23:49:17 +02:00
Simon Willnauer eba69ffade [TEST] First decrement in-flight requests before releasing the latch 2016-07-12 22:58:03 +02:00
Michael Sander c493774093 Fix typo in cluster module docs
This commit fixes a simple typo in the cluster module docs.

Closes #19393
2016-07-12 16:32:23 -04:00
Ryan Ernst e7b3bea8d3 Tweak and document gradle setup for adding transport modules to integ-test-zip 2016-07-12 12:13:52 -07:00
Simon Willnauer ec55f9fff7 [TEST] Make AbstractSimpleTransportTestCase#testTimeoutSendExceptionWithDelayedResponse more robust and wait for in-flight request 2016-07-12 20:41:37 +02:00
Simon Willnauer c463083537 minor cleanups and an additional BogusPlugin for HttpSmokeTestCase 2016-07-12 17:55:05 +02:00
Simon Willnauer 2d80a53b09 fix test plugin visibility 2016-07-12 17:42:29 +02:00
Simon Willnauer 9cb247287f consolidate security code in on place an allow test based on the jar dependency to opt out of netty internal property setting assertion 2016-07-12 17:41:21 +02:00
Simon Willnauer 4fb79707bd Fix remaining tests that either need access to the netty module or require explict configuration
Some tests still start http implicitly or miss configuring the transport clients correctly.
This commit fixes all remaining tests and adds a depdenceny to `transport-netty` from
`qa/smoke-test-http` and `modules/reindex` since they need an http server running on the nodes.

This also moves all required permissions for netty into it's module and out of core.
2016-07-12 16:29:57 +02:00
Martijn van Groningen 075cb970c0 inner_hits: Ensure that that InnerHitBuilder uses rewritten queries
If a nested, has_child or has_parent query's inner query gets rewritten then the InnerHitBuilder should use that rewritten form too, otherwise this can cause exceptions in a later phase.

Also fixes a bug that HasChildQueryBuilder's rewrite method overwrites max_children with min_children value.

Closes #19353
2016-07-12 16:26:57 +02:00
Britta Weber fed6b72460 Add test for #19389 new type always creates _parent#null field 2016-07-12 15:45:31 +02:00
Luca Cavanna f6aec3fdb5 Merge pull request #19373 from javanna/enhancement/rest_client_builder_callback
Rest Client: add callback to customize http client settings
2016-07-12 13:30:27 +02:00
javanna 512b8be791 RestClient: simplify ssl configuration and make http config callback functional friendly 2016-07-12 13:25:55 +02:00
Boaz Leskes 081d04afac Make NotMasterException a first class citizen (#19385)
That exception is currently serialized as its current base class IllegalStateException which confuses code supposed to deal with the stepping down of a master. This is an important exception and we should be able to serialize it correctly. This commit fixes it by moving the exception to inherit from ElasticsearchException and properly register it.

As a bonus I adapted CapturingTransport to properly simulate serialized exceptions.
2016-07-12 12:44:40 +02:00
javanna fa0b354e66 Rest Client: add callback to customize http client settings
The callback replaces the ability to fully replace the http client instance. By doing that, one used to lose any default that the RestClient had set for the underlying http client. Given that you'd usually override one or two things only, like a couple of timeout values, the ssl factory or the default credentials providers, it is not uder friendly if by doing that users end up replacing the whole http client instance and lose any default set by us.
2016-07-12 12:31:28 +02:00
Simon Willnauer 199a5a1f04 Fix TcpTransport#sendRequest to raise NotConnectedExcepiton if we get disconnected while sending
This also fixes a race in AbstractSimpleTransportTestCase where we never wait long enough
for all response to finish causing expected failures.
2016-07-12 10:56:20 +02:00
Ryan Ernst d2ac7d450b Merge pull request #19380 from rjernst/plugin_components_inputs
Add some basic services to createComponents for plugins
2016-07-11 23:58:38 -07:00
Ryan Ernst 93aebbef0f Merge branch 'master' into modularize_netty 2016-07-11 23:49:00 -07:00
Ryan Ernst 81429aed8c Remove createComponents from transport client 2016-07-11 23:33:49 -07:00
Ryan Ernst 86d0d67036 Plugins: Add some basic services to createComponents
This adds the first few basic services needed for any plugin to create
its own components that interact with the rest of the system.
2016-07-11 23:22:20 -07:00
Ryan Ernst 7195d1e0ff Fix plugins service to not double bind plugin components 2016-07-11 17:05:56 -07:00
Nik Everett 8263873783 Switch search extension from push to pull
Switches most search behavior extensions from push (`onModule(SearchModule)`)
to pull (`implements SearchPlugin`). This effort in general gives plugin
authors a much cleaner view of how to extend Elasticsearch and starts to
set up portions of Elasticsearch as "the plugin API". This commit in
particular does that for search-time behavior like customized suggesters,
highlighters, score functions, and significance heuristics.

It also switches most such customization to being done at search module
construction time which is much, much easier to reason about from a testing
perspective. It also helps significantly in the process of de-guice-ing
Elasticsearch's startup.

There are at least two major search time extensions that aren't covered in
this commit that will simply have to wait for the next commit on the topic
because this one has already grown large: custom aggregations and custom
queries. These will likely live in the same SearchPlugin interface as well.
2016-07-11 18:49:05 -04:00
David Pilato 922230bd75 Fix after merge with master 2016-07-11 23:50:12 +02:00
Ryan Ernst 535b60cb2b Merge pull request #19371 from rjernst/plugin_components
Add components getter as bridge between guice and new plugin init world
2016-07-11 14:23:45 -07:00
Ryan Ernst 6b4d0001a2 Remove unnecessary warning suppression 2016-07-11 14:20:47 -07:00
David Pilato bdebaba8f5 Merge branch 'master' into pr/s3-path-style-access
# Conflicts:
#	docs/plugins/repository-s3.asciidoc
#	plugins/repository-s3/src/main/java/org/elasticsearch/cloud/aws/AwsS3Service.java
#	plugins/repository-s3/src/main/java/org/elasticsearch/cloud/aws/InternalAwsS3Service.java
#	plugins/repository-s3/src/main/java/org/elasticsearch/repositories/s3/S3Repository.java
#	plugins/repository-s3/src/test/java/org/elasticsearch/cloud/aws/TestAwsS3Service.java
2016-07-11 23:17:38 +02:00
Ryan Ernst 05ea943def Rename local plugin componenents list for clarity 2016-07-11 14:16:23 -07:00
Ryan Ernst 99ac65931a Plugins: Add components creator as bridge between guice and new plugin init world
This change adds a createComponents() method to Plugin implementations
which they can use to return already constructed componenents/services.
Eventually this should be just services ("components" don't really do
anything), but for now it allows any object so that preconstructed
instances by plugins can still be bound to guice. Over time we should
add basic services as arguments to this method, but for now I have left
it empty so as to not presume what is a necessary service.
2016-07-11 14:14:06 -07:00
Simon Willnauer 048e4416e7 Move netty transport and http into a module
This moves all netty code and it's dependency into a module.
2016-07-11 22:21:29 +02:00
Nik Everett c680bd57da Fix scroll test
It was relying on unreasonably large windows crashing. Those large windows
abort the request immediately now.
2016-07-11 16:04:17 -04:00
Ali Beyad 7759c23272 Fix line length formatting for ClusterStateHealthTests 2016-07-11 15:32:13 -04:00
Ali Beyad 0faf638710 Blocked allocations on primary causes RED health
If the allocation decision for a primary shard was NO, this should
cause the cluster health for the shard to go RED, even if the shard
belongs to a newly created index or is part of cluster recovery.

Relates #9126
2016-07-11 15:32:13 -04:00
Ali Beyad 417bd0cd63 Index creation does not cause the cluster health to go RED
Previously, index creation would momentarily cause the cluster health to
go RED, because the primaries were still being assigned and activated.
This commit ensures that when an index is created or an index is being
recovered during cluster recovery and it does not have any active
allocation ids, then the cluster health status will not go RED, but
instead be YELLOW.

Relates #9126
2016-07-11 15:30:47 -04:00
Nik Everett 3ea1360625 Limit batch size when scrolling
Limits the batch size from scrolling using the same setting as interactive
search: `index.max_result_window`.

Closes #19249
2016-07-11 15:29:45 -04:00
Simon Willnauer 47bd2f9ca5 More cleanups aroung tests that require HTTP to be enalbed. (#19363)
this commit moves the most of the http related integ tests out into it's own 
`qa/smoke-test-http` project where most of the test can run against the external cluster.
2016-07-11 20:44:57 +02:00
Nik Everett 89614586e9 Migrate range, date_range, and geo_distance aggregations to NamedWriteable 2016-07-11 13:00:36 -04:00