Commit Graph

23545 Commits

Author SHA1 Message Date
Simon Willnauer 5616251f22 Remove `node.mode` and `node.local` settings (#19428)
Today `node.mode` and `node.local` serve almost the same purpose, they
are a shortcut for `discovery.type` and `transport.type`. If `node.local: true`
or `node.mode: local` is set elasticsearch will start in _local_ mode which means
only nodes within the same JVM are discovered and a non-network based transport
is used. The _local_ mode it only really used in tests or if nodes are embedded.
For both, embedding and tests explicit configuration via `discovery.type` and `transport.type`
should be preferred.

This change removes all the usage of these settings and by-default doesn't
configure a default transport implemenation since netty is now a module. Yet, to make
the user expericence flawless, plugins or modules can set a `http.type.default` and
`transport.type.default`. Plugins set this via `PluginService#additionalSettings()`
which enforces _set-once_ which prevents node startup if set multiple times. This means
that our distributions will just startup with netty transport since it's packaged as a
module unless `transport.type` or `http.transport.type` is explicitly set.

This change also found a bunch of bugs since several NamedWriteables were not registered if a
transport client is used. Now that we don't rely on the `node.mode` leniency which is inherited
instead of using explicit settings, `TransportClient` uses `AssertingLocalTransport` which detects these problems since it serializes all messages.

Closes #16234
2016-07-14 13:21:10 +02:00
Simon Willnauer 4156a4bebb Add support for `wait_for_events` to the `_cluster/health` REST endpoint (#19432)
The Java API supports this while mostly used for tests it can also be useful in
production environments. For instance if something is automated like a settings change
and we execute some health right after it the settings update might have some consequences
like a reroute which hasn't been fully applied since the preconditions are not fulfilled yet.
For instance if not all shards started the settings update is applied but the reroute won't move
currently initializing shards like in the shrink API test. Sure this could be done by waiting for
green before but if the cluster moves shards due to some side-effects waiting for all events is
still useful. I also took the chance to add unittests to Priority.java

Closes #19419
2016-07-14 12:33:29 +02:00
Boaz Leskes ef33183a19 update migration docs to include removal of `netty.epollBugWorkaround` 2016-07-14 12:20:35 +02:00
Ryan Ernst 9b63fc37c8 Make module client jars go to org.elasticsearch.plugin groupId 2016-07-14 02:58:27 -07:00
Ryan Ernst c36850f114 Add client flag for percolator module build 2016-07-14 02:41:58 -07:00
Martijn van Groningen 1bc12f5214 docs: fix broken link
Closes #19430
2016-07-14 11:12:47 +02:00
Ryan Ernst 20b038d976 Build: Add flag for plugins/modules which can be used in the transport client
This changes adds a flag which can be set in the esplugin closure in
build.gradle for plugins and modules which contain pieces that must be
published to maven, for use in the transport client. The jar/pom and
source/javadoc jars are moved to a new name that has the suffix
"-client".

I enabled this for the two modules that I know definitely need this;
there may be more. One open question is which groupId to use for the
generated pom.

closes #19411
2016-07-14 02:07:52 -07:00
Simon Willnauer 2138a55eab Merge pull request #19424 from mfussenegger/error-msg
Complete load-settings error message
2016-07-13 23:59:02 +02:00
Mathias Fussenegger 8c0b954466 Complete load-settings error message 2016-07-13 23:39:16 +02:00
Tal Levy ed768b101f show ignored errors in verbose simulate result (#19404)
Closes #19319.
2016-07-13 13:32:10 -07:00
Tal Levy 8fd01554bc update foreach processor to only support one applied processor. (#19402)
Closes #19345.
2016-07-13 13:13:00 -07:00
gfyoung 3f2e1066d3 Removed duplicate deleteBlob methods (#18813)
Removed the following methods from the
BlobContainer interface to clean up the interface:

1) deleteBlobs
2) deleteBlobsByPrefix

Closes #18529
2016-07-13 14:36:23 -04:00
Chris Earle ce65ab6eb7 Add RestController method for deprecating in one step
This adds an extra method, registerWithDeprecatedHandler, to register both a normal handler and a deprecated handler at the same time. This helps with renaming methods as opposed to _just_ deprecated methods.
2016-07-13 13:03:23 -04:00
Nik Everett f084469c27 Fix syntax for template tests 2016-07-13 11:34:06 -04:00
Nik Everett e9d292b450 Fix names of tar tests 2016-07-13 11:33:55 -04:00
Nik Everett 2422b969c1 Migrate matrix_stats to NamedWriteable
This is the last consumer of the old style register method so I removed
the method.
2016-07-13 10:48:20 -04:00
Nik Everett d95fbba8cb Switch remaining builtin aggs to new registration method 2016-07-13 10:48:20 -04:00
Clinton Gormley 1e2d0c1000 More bad asciidoc 2016-07-13 16:30:49 +02:00
Clinton Gormley 599727e38f Fixed bad ASCIIDOC 2016-07-13 16:09:41 +02:00
Clinton Gormley ab7a976e49 Make Prefer Parameters admon block linkable 2016-07-13 16:02:34 +02:00
Martijn van Groningen 2c3165d080 Removed deprecated 1.x script and template syntax
Closes #13729
2016-07-13 15:07:36 +02:00
Nik Everett 88d3527178 Migrate derivative pipeline aggregation to NamedWriteable
This is another step in the effort to remove AggregationStreams and
instead use NamedWriteableRegistry like the rest of the code base.
2016-07-13 07:12:22 -04:00
Simon Willnauer 29fd0f1bd8 [TEST] Remove wrong transportName from MockTcpTransport#ctor 2016-07-13 12:50:52 +02:00
Simon Willnauer ae98d59899 Don't assert that files exists if recovery has been cancled
Today we assert that the tmp files are present but if the recovery
was canceled this might not be the case while still a valid state.
This chance only throws the AssertionError if the recovery is still active.
2016-07-13 10:52:17 +02:00
Simon Willnauer 067ca1f996 [TEST] Use a semaphore to block unitl all in-flight requests are released 2016-07-13 10:31:05 +02:00
Simon Willnauer 814c7224f9 Merge pull request #19392 from elastic/modularize_netty
This moves all netty related code into modules/transport-netty the module is build as a zip file as well as a JAR to serve as a dependency for transport client. For the time being this is required otherwise we have no network based impl. for transport client users. This might be subject to change given that we move forward http client.
2016-07-13 09:52:03 +02:00
Martijn van Groningen 2bdc55c9ff fvh: Also extract terms from the nested query' inner query.
Closes #19265
2016-07-13 08:15:46 +02:00
Nik Everett d14e06ce51 Migrate top_hits, histogram, and ip_range aggregations to NamedWriteable
This is just another step towards removing AggregationStreams
in favor of NamedWriteable.
2016-07-12 23:02:32 -04:00
Nik Everett f2978f41b9 Migrate nested, reverse_nested, and children aggregations to NamedWriteable
Just another step in removing AggregationStreams in favor of NamedWriteable.
2016-07-12 22:38:51 -04:00
Nik Everett 06bd896ce0 Migrate geohash_grid and geo_bounds to NamedWriteable
Just another small step in removing Aggregation's custom streams
implementation in favor of NamedWriteable.
2016-07-12 22:22:51 -04:00
Nik Everett f479219ca7 Clean up significant terms aggregation results
* Clean up the generics around significant terms aggregation results
* Reduce code duplicated between `SignificantLongTerms` and
`SignificantStringTerms` by creating `InternalMappedSignificantTerms`
and moving common things there where possible.
* Migrate to `NamedWriteable`
* Line length fixes while I was there
2016-07-12 22:08:09 -04:00
Ryan Ernst 920bd0cf68 Merge pull request #19401 from rjernst/more_plugin_services
Plugins: Add resource watcher to services available for plugin components
2016-07-12 15:28:17 -07:00
Lee Hinman 95cf2407ee Merge remote-tracking branch 'dakrone/include-cluster-info-in-explain-api' 2016-07-12 16:26:46 -06:00
Jason Tedor ce5a382c69 Remove support for properties
This commit removes support for properties syntax and config files:
 - removed support for elasticsearch.properties
 - removed support for logging.properties
 - removed support for properties content detection in REST APIs
 - removed support for properties content detection in Java API

Relates #19398
2016-07-12 17:55:18 -04:00
Lee Hinman 58db63b610 Expose the ClusterInfo object in the allocation explain output
This adds an optional parameter to the cluster allocation explain API
that will return the cluster info object, `include_disk_info`, the
output looks like:

GET /_cluster/allocation/explain?include_disk_info -d'
{"index": "i", "shard": 0, "primary": false}'

{
  ... other info ...

  "cluster_info" : {
    "nodes" : {
      "7Uws-vL7R6WVm3ZwQA1n5A" : {
        "node_name" : "Kraven the Hunter",
        "least_available" : {
          "path" : "/path/to/data1",
          "total_bytes" : 165999570944,
          "used_bytes" : 118180614144,
          "free_bytes" : 47818956800,
          "free_disk_percent" : 28.80667493781158,
          "used_disk_percent" : 71.19332506218842
        },
        "most_available" : {
          "path" : "/path/to/data2",
          "total_bytes" : 165999570944,
          "used_bytes" : 118180614144,
          "free_bytes" : 47818956800,
          "free_disk_percent" : 28.80667493781158,
          "used_disk_percent" : 71.19332506218842
        }
      }
    },
    "shard_sizes" : {
      "[i][2][p]_bytes" : 0,
      "[i][4][p]_bytes" : 130,
      "[i][1][p]_bytes" : 0,
      "[i][3][p]_bytes" : 0,
      "[i][0][p]_bytes" : 130
    },
    "shard_paths" : {
      "[i][3], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=LegZLDniTVaw0Y1urv7s3g]" : "/path/to/data1/nodes/0",
      "[i][1], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=lAU_4vf_SKmoRdtg0ACnjQ]" : "/path/to/data1/nodes/0",
      "[i][2], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=Aurpeuj7SeGeyPDDpCtRgg]" : "/path/to/data1/nodes/0",
      "[i][0], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=Vgg8GlQTQ82C2j6HYBq8DQ]" : "/path/to/data1/nodes/0",
      "[i][4], node[7Uws-vL7R6WVm3ZwQA1n5A], [P], s[STARTED], a[id=t8hQlVSxQe-58fSeaXcAqg]" : "/path/to/data1/nodes/0"
    }
  }
}

Resolves #14405
2016-07-12 15:52:20 -06:00
Ryan Ernst 9b5ac4f68e Plugins: Add resource watcher to services available for plugin components 2016-07-12 14:51:50 -07:00
Simon Willnauer ce28aa9160 [TEST] HttpCompressionIT is a real IT 2016-07-12 23:49:17 +02:00
Simon Willnauer eba69ffade [TEST] First decrement in-flight requests before releasing the latch 2016-07-12 22:58:03 +02:00
Michael Sander c493774093 Fix typo in cluster module docs
This commit fixes a simple typo in the cluster module docs.

Closes #19393
2016-07-12 16:32:23 -04:00
Ryan Ernst e7b3bea8d3 Tweak and document gradle setup for adding transport modules to integ-test-zip 2016-07-12 12:13:52 -07:00
Simon Willnauer ec55f9fff7 [TEST] Make AbstractSimpleTransportTestCase#testTimeoutSendExceptionWithDelayedResponse more robust and wait for in-flight request 2016-07-12 20:41:37 +02:00
Simon Willnauer c463083537 minor cleanups and an additional BogusPlugin for HttpSmokeTestCase 2016-07-12 17:55:05 +02:00
Simon Willnauer 2d80a53b09 fix test plugin visibility 2016-07-12 17:42:29 +02:00
Simon Willnauer 9cb247287f consolidate security code in on place an allow test based on the jar dependency to opt out of netty internal property setting assertion 2016-07-12 17:41:21 +02:00
Simon Willnauer 4fb79707bd Fix remaining tests that either need access to the netty module or require explict configuration
Some tests still start http implicitly or miss configuring the transport clients correctly.
This commit fixes all remaining tests and adds a depdenceny to `transport-netty` from
`qa/smoke-test-http` and `modules/reindex` since they need an http server running on the nodes.

This also moves all required permissions for netty into it's module and out of core.
2016-07-12 16:29:57 +02:00
Martijn van Groningen 075cb970c0 inner_hits: Ensure that that InnerHitBuilder uses rewritten queries
If a nested, has_child or has_parent query's inner query gets rewritten then the InnerHitBuilder should use that rewritten form too, otherwise this can cause exceptions in a later phase.

Also fixes a bug that HasChildQueryBuilder's rewrite method overwrites max_children with min_children value.

Closes #19353
2016-07-12 16:26:57 +02:00
Britta Weber fed6b72460 Add test for #19389 new type always creates _parent#null field 2016-07-12 15:45:31 +02:00
Luca Cavanna f6aec3fdb5 Merge pull request #19373 from javanna/enhancement/rest_client_builder_callback
Rest Client: add callback to customize http client settings
2016-07-12 13:30:27 +02:00
javanna 512b8be791 RestClient: simplify ssl configuration and make http config callback functional friendly 2016-07-12 13:25:55 +02:00
Boaz Leskes 081d04afac Make NotMasterException a first class citizen (#19385)
That exception is currently serialized as its current base class IllegalStateException which confuses code supposed to deal with the stepping down of a master. This is an important exception and we should be able to serialize it correctly. This commit fixes it by moving the exception to inherit from ElasticsearchException and properly register it.

As a bonus I adapted CapturingTransport to properly simulate serialized exceptions.
2016-07-12 12:44:40 +02:00