Commit Graph

5695 Commits

Author SHA1 Message Date
Mathias Fussenegger 8c0b954466 Complete load-settings error message 2016-07-13 23:39:16 +02:00
Tal Levy ed768b101f show ignored errors in verbose simulate result (#19404)
Closes #19319.
2016-07-13 13:32:10 -07:00
Tal Levy 8fd01554bc update foreach processor to only support one applied processor. (#19402)
Closes #19345.
2016-07-13 13:13:00 -07:00
gfyoung 3f2e1066d3 Removed duplicate deleteBlob methods (#18813)
Removed the following methods from the
BlobContainer interface to clean up the interface:

1) deleteBlobs
2) deleteBlobsByPrefix

Closes #18529
2016-07-13 14:36:23 -04:00
Chris Earle ce65ab6eb7 Add RestController method for deprecating in one step
This adds an extra method, registerWithDeprecatedHandler, to register both a normal handler and a deprecated handler at the same time. This helps with renaming methods as opposed to _just_ deprecated methods.
2016-07-13 13:03:23 -04:00
Nik Everett 2422b969c1 Migrate matrix_stats to NamedWriteable
This is the last consumer of the old style register method so I removed
the method.
2016-07-13 10:48:20 -04:00
Nik Everett d95fbba8cb Switch remaining builtin aggs to new registration method 2016-07-13 10:48:20 -04:00
Martijn van Groningen 2c3165d080 Removed deprecated 1.x script and template syntax
Closes #13729
2016-07-13 15:07:36 +02:00
Nik Everett 88d3527178 Migrate derivative pipeline aggregation to NamedWriteable
This is another step in the effort to remove AggregationStreams and
instead use NamedWriteableRegistry like the rest of the code base.
2016-07-13 07:12:22 -04:00
Simon Willnauer 29fd0f1bd8 [TEST] Remove wrong transportName from MockTcpTransport#ctor 2016-07-13 12:50:52 +02:00
Simon Willnauer ae98d59899 Don't assert that files exists if recovery has been cancled
Today we assert that the tmp files are present but if the recovery
was canceled this might not be the case while still a valid state.
This chance only throws the AssertionError if the recovery is still active.
2016-07-13 10:52:17 +02:00
Simon Willnauer 814c7224f9 Merge pull request #19392 from elastic/modularize_netty
This moves all netty related code into modules/transport-netty the module is build as a zip file as well as a JAR to serve as a dependency for transport client. For the time being this is required otherwise we have no network based impl. for transport client users. This might be subject to change given that we move forward http client.
2016-07-13 09:52:03 +02:00
Martijn van Groningen 2bdc55c9ff fvh: Also extract terms from the nested query' inner query.
Closes #19265
2016-07-13 08:15:46 +02:00
Nik Everett d14e06ce51 Migrate top_hits, histogram, and ip_range aggregations to NamedWriteable
This is just another step towards removing AggregationStreams
in favor of NamedWriteable.
2016-07-12 23:02:32 -04:00
Nik Everett f2978f41b9 Migrate nested, reverse_nested, and children aggregations to NamedWriteable
Just another step in removing AggregationStreams in favor of NamedWriteable.
2016-07-12 22:38:51 -04:00
Nik Everett 06bd896ce0 Migrate geohash_grid and geo_bounds to NamedWriteable
Just another small step in removing Aggregation's custom streams
implementation in favor of NamedWriteable.
2016-07-12 22:22:51 -04:00
Nik Everett f479219ca7 Clean up significant terms aggregation results
* Clean up the generics around significant terms aggregation results
* Reduce code duplicated between `SignificantLongTerms` and
`SignificantStringTerms` by creating `InternalMappedSignificantTerms`
and moving common things there where possible.
* Migrate to `NamedWriteable`
* Line length fixes while I was there
2016-07-12 22:08:09 -04:00
Ryan Ernst 920bd0cf68 Merge pull request #19401 from rjernst/more_plugin_services
Plugins: Add resource watcher to services available for plugin components
2016-07-12 15:28:17 -07:00
Lee Hinman 95cf2407ee Merge remote-tracking branch 'dakrone/include-cluster-info-in-explain-api' 2016-07-12 16:26:46 -06:00
Jason Tedor ce5a382c69 Remove support for properties
This commit removes support for properties syntax and config files:
 - removed support for elasticsearch.properties
 - removed support for logging.properties
 - removed support for properties content detection in REST APIs
 - removed support for properties content detection in Java API

Relates #19398
2016-07-12 17:55:18 -04:00
Lee Hinman 58db63b610 Expose the ClusterInfo object in the allocation explain output
This adds an optional parameter to the cluster allocation explain API
that will return the cluster info object, `include_disk_info`, the
output looks like:

GET /_cluster/allocation/explain?include_disk_info -d'
{"index": "i", "shard": 0, "primary": false}'

{
  ... other info ...

  "cluster_info" : {
    "nodes" : {
      "7Uws-vL7R6WVm3ZwQA1n5A" : {
        "node_name" : "Kraven the Hunter",
        "least_available" : {
          "path" : "/path/to/data1",
          "total_bytes" : 165999570944,
          "used_bytes" : 118180614144,
          "free_bytes" : 47818956800,
          "free_disk_percent" : 28.80667493781158,
          "used_disk_percent" : 71.19332506218842
        },
        "most_available" : {
          "path" : "/path/to/data2",
          "total_bytes" : 165999570944,
          "used_bytes" : 118180614144,
          "free_bytes" : 47818956800,
          "free_disk_percent" : 28.80667493781158,
          "used_disk_percent" : 71.19332506218842
        }
      }
    },
    "shard_sizes" : {
      "[i][2][p]_bytes" : 0,
      "[i][4][p]_bytes" : 130,
      "[i][1][p]_bytes" : 0,
      "[i][3][p]_bytes" : 0,
      "[i][0][p]_bytes" : 130
    },
    "shard_paths" : {
      "[i][3], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=LegZLDniTVaw0Y1urv7s3g]" : "/path/to/data1/nodes/0",
      "[i][1], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=lAU_4vf_SKmoRdtg0ACnjQ]" : "/path/to/data1/nodes/0",
      "[i][2], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=Aurpeuj7SeGeyPDDpCtRgg]" : "/path/to/data1/nodes/0",
      "[i][0], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=Vgg8GlQTQ82C2j6HYBq8DQ]" : "/path/to/data1/nodes/0",
      "[i][4], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=t8hQlVSxQe-58fSeaXcAqg]" : "/path/to/data1/nodes/0"
    }
  }
}

Resolves #14405
2016-07-12 15:52:20 -06:00
Ryan Ernst 9b5ac4f68e Plugins: Add resource watcher to services available for plugin components 2016-07-12 14:51:50 -07:00
Simon Willnauer 4fb79707bd Fix remaining tests that either need access to the netty module or require explict configuration
Some tests still start http implicitly or miss configuring the transport clients correctly.
This commit fixes all remaining tests and adds a depdenceny to `transport-netty` from
`qa/smoke-test-http` and `modules/reindex` since they need an http server running on the nodes.

This also moves all required permissions for netty into it's module and out of core.
2016-07-12 16:29:57 +02:00
Martijn van Groningen 075cb970c0 inner_hits: Ensure that that InnerHitBuilder uses rewritten queries
If a nested, has_child or has_parent query's inner query gets rewritten then the InnerHitBuilder should use that rewritten form too, otherwise this can cause exceptions in a later phase.

Also fixes a bug that HasChildQueryBuilder's rewrite method overwrites max_children with min_children value.

Closes #19353
2016-07-12 16:26:57 +02:00
Britta Weber fed6b72460 Add test for #19389 new type always creates _parent#null field 2016-07-12 15:45:31 +02:00
Boaz Leskes 081d04afac Make NotMasterException a first class citizen (#19385)
That exception is currently serialized as its current base class IllegalStateException which confuses code supposed to deal with the stepping down of a master. This is an important exception and we should be able to serialize it correctly. This commit fixes it by moving the exception to inherit from ElasticsearchException and properly register it.

As a bonus I adapted CapturingTransport to properly simulate serialized exceptions.
2016-07-12 12:44:40 +02:00
Simon Willnauer 199a5a1f04 Fix TcpTransport#sendRequest to raise NotConnectedExcepiton if we get disconnected while sending
This also fixes a race in AbstractSimpleTransportTestCase where we never wait long enough
for all response to finish causing expected failures.
2016-07-12 10:56:20 +02:00
Ryan Ernst 93aebbef0f Merge branch 'master' into modularize_netty 2016-07-11 23:49:00 -07:00
Ryan Ernst 81429aed8c Remove createComponents from transport client 2016-07-11 23:33:49 -07:00
Ryan Ernst 86d0d67036 Plugins: Add some basic services to createComponents
This adds the first few basic services needed for any plugin to create
its own components that interact with the rest of the system.
2016-07-11 23:22:20 -07:00
Ryan Ernst 7195d1e0ff Fix plugins service to not double bind plugin components 2016-07-11 17:05:56 -07:00
Nik Everett 8263873783 Switch search extension from push to pull
Switches most search behavior extensions from push (`onModule(SearchModule)`)
to pull (`implements SearchPlugin`). This effort in general gives plugin
authors a much cleaner view of how to extend Elasticsearch and starts to
set up portions of Elasticsearch as "the plugin API". This commit in
particular does that for search-time behavior like customized suggesters,
highlighters, score functions, and significance heuristics.

It also switches most such customization to being done at search module
construction time which is much, much easier to reason about from a testing
perspective. It also helps significantly in the process of de-guice-ing
Elasticsearch's startup.

There are at least two major search time extensions that aren't covered in
this commit that will simply have to wait for the next commit on the topic
because this one has already grown large: custom aggregations and custom
queries. These will likely live in the same SearchPlugin interface as well.
2016-07-11 18:49:05 -04:00
Ryan Ernst 535b60cb2b Merge pull request #19371 from rjernst/plugin_components
Add components getter as bridge between guice and new plugin init world
2016-07-11 14:23:45 -07:00
Ryan Ernst 6b4d0001a2 Remove unnecessary warning suppression 2016-07-11 14:20:47 -07:00
Ryan Ernst 05ea943def Rename local plugin componenents list for clarity 2016-07-11 14:16:23 -07:00
Ryan Ernst 99ac65931a Plugins: Add components creator as bridge between guice and new plugin init world
This change adds a createComponents() method to Plugin implementations
which they can use to return already constructed componenents/services.
Eventually this should be just services ("components" don't really do
anything), but for now it allows any object so that preconstructed
instances by plugins can still be bound to guice. Over time we should
add basic services as arguments to this method, but for now I have left
it empty so as to not presume what is a necessary service.
2016-07-11 14:14:06 -07:00
Simon Willnauer 048e4416e7 Move netty transport and http into a module
This moves all netty code and it's dependency into a module.
2016-07-11 22:21:29 +02:00
Nik Everett c680bd57da Fix scroll test
It was relying on unreasonably large windows crashing. Those large windows
abort the request immediately now.
2016-07-11 16:04:17 -04:00
Nik Everett 3ea1360625 Limit batch size when scrolling
Limits the batch size from scrolling using the same setting as interactive
search: `index.max_result_window`.

Closes #19249
2016-07-11 15:29:45 -04:00
Simon Willnauer 47bd2f9ca5 More cleanups aroung tests that require HTTP to be enalbed. (#19363)
this commit moves the most of the http related integ tests out into it's own 
`qa/smoke-test-http` project where most of the test can run against the external cluster.
2016-07-11 20:44:57 +02:00
Nik Everett 89614586e9 Migrate range, date_range, and geo_distance aggregations to NamedWriteable 2016-07-11 13:00:36 -04:00
Christoph Büscher 0d428b6ba8 Add test for GeoHashUtils#bbox() 2016-07-11 10:46:31 -05:00
Nicholas Knize f77f79c24a GeoBoundingBoxQueryBuilder should fail when topLeft and bottomRight are the same coordinate 2016-07-11 10:25:33 -05:00
Jason Tedor df7ad9970b Batch process node left and node failure
Today when a node is removed the cluster (it leaves or it fails), we
submit a cluster state update task. These cluster state update tasks are
processed serially on the master. When nodes are removed en masse (e.g.,
a rack is taken down or otherwise becomes unavailable), the master will
be slow to process these failures because of the resulting reroutes and
publishing of each subsequent cluster state. We improve this in this
commit by processing the node removals using the cluster state update
task batch processing framework.

Relates #19289
2016-07-11 08:30:09 -04:00
Simon Willnauer 3f3c93ec65 Add blocking socket based MockTcpTransport (#19332)
Today we have a bunch of tests that use netty transport for several reasons
these tests use it because they need to run some tcp based transport. Yet, this
couples our tests tightly to the netty implementation which should be tested on it's own.
This change adds a plain socket based blocking TcpTransport implementation that is used by
default in tests if local transport is suppressed or if network is selected.
It also adds another tcp network implementation as a showcase how the interface works.
2016-07-11 12:17:52 +02:00
Simon Willnauer 1d03a1409c Catch assertion errors on commit and turn it into a real exception (#19357)
Lucene IndexWriter asserts on files existing on the filesystem but
some tests throw IOException explicitly on those operatiosn such that
some tests trip asserts. We had this before on InternalEngine#ctor
and added some logic there to catch only a specific assertions based
on some excepition stack analysis. This change applies the same logic
to the IndexWriter#commit part of the engine since it can hit the same
issue.
This also fixes a self-suppression issue in Store.java.

Closes #19356
2016-07-11 11:20:56 +02:00
javanna 942e342662 Rest Client: use short performRequest methods when possible 2016-07-11 10:36:26 +02:00
Simon Willnauer 36dbe7250f Cleanup usage of http.enabled (#19351)
Several tests required http.enabled where it was unnecessary.
We also had RestMainActionIT which tests what two of our REST tests
test already so I removed it.
The explicit use of http.enabled: false is also obsolet since our
test do that by default.
2016-07-11 10:21:03 +02:00
Ryan Ernst 25ed93dd28 Fix test edge case for random bytes reference iter.
Getting an offset to the last byte means we can only stream one byte and
then we are done, we can't get another offset after it.
2016-07-10 09:11:32 -07:00
Nicholas Knize ab8b577aea Use GeoDistanceIT.testDuelOptimizations for bwc testing only
This test is only relevant for testing the GeoDistanceBuilder with old indices (pre GeoPointV2).

closes #19263
2016-07-09 19:08:06 -05:00