2508 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe
be64bfb135 Revert "Aggregations: Pass extended bounds into HistogramAggregator when creating an unmapped aggregator"
This reverts commit e1bf9798bff698d23537b53b5a31287da70f9ccb.
2015-11-13 12:48:51 +00:00
Colin Goodheart-Smithe
e1bf9798bf Aggregations: Pass extended bounds into HistogramAggregator when creating an unmapped aggregator
This fixes an issue where if the field for the aggregation was unmapped the extended bounds would get dropped and the resulting buckets would not cover the extended bounds requested.

Closes #14735
2015-11-13 12:33:06 +00:00
Jason Tedor
a9ab35a487 Add sync_id to cat shards API
This commit adds the ability to get the sync_id from the cat shards API.

Closes #14705
2015-11-13 05:13:08 -05:00
Robert Muir
776bb288b5 fix gradle check under jigsaw
closes #14726

Squashed commit of the following:

commit 5b591e98570e3fa481b2816a44063b98bff36ddf
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 13 00:54:08 2015 -0500

    add assumption for self-signing in PluginManagerTests

commit ed11e5371b6f71591dc41c6f60d033502cfcf029
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 13 00:20:59 2015 -0500

    show error output from integ test startup

commit d8b187a10e95d89a0e775333dcbe1aaa903fb376
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Nov 12 22:14:11 2015 -0500

    fix gradle check under jigsaw
2015-11-13 00:57:12 -05:00
Nicholas Knize
dc77815744 Enable GeoPointV2 with backward compatibility testing
This commit removes all noreleases and cuts over to Lucene 5.4 GeoPointField type. Included are randomized testing updates to unit and integration test suites for ensuring full backward compatability with existing geo_point indexes.
2015-11-12 23:01:51 -06:00
Robert Muir
720ebe347d Handle system policy correctly.
Just suck in the system policy, so its compatible with any version of java.
It means it also respects configuration (e.g. for monitoring agents)

Closes #14704
2015-11-12 18:08:13 -05:00
xuzha
f33184329f Add extra validation into cluster/stats
The cluster health and cluster stats disagree on the status.
Add a extra validation step in `cluster/stats`.

closes #7390
2015-11-12 10:46:18 -08:00
Yannick Welsch
4729386347 Merge pull request #14494 from ywelsch/fix/delayed-allocation-reroute
Delayed allocation can miss a reroute
2015-11-12 17:58:08 +01:00
Yannick Welsch
c32de1f72b [TEST] Use TransportService/TransportAddress instead of DiscoveryNode for disruption rules
The disruption rules are changed to work on all transport addresses that are bound by a node (not only publish address).
This is important as UnicastZenPing creates fake DiscoveryNode instances which match one of the bound addresses and not necessarily the publish address.

Closes #14625
Closes #14653
2015-11-12 17:20:07 +01:00
Jason Tedor
fac472f90c Remove unused imports in o.e.c.a.s.ShardStateActionTests 2015-11-12 09:08:27 -05:00
Yannick Welsch
0220e45e2f Fix missing reroute in case of multiple delayed shards
After a delayed reroute of a shard, RoutingService misses to schedule a new delayed reroute of other delayed shards.

Closes #14494
Closes #14010
Closes #14445
2015-11-12 10:57:40 +01:00
Jason Tedor
c2aec53b46 Move field to local variable 2015-11-11 12:50:21 -05:00
Jason Tedor
3a92012938 Use String#equals instead of reference equality 2015-11-11 12:50:21 -05:00
Jason Tedor
e4e68c0463 Sanity check exception message in tests 2015-11-11 12:50:21 -05:00
Jason Tedor
06c07ce275 Remove unnecessary generic type parameters 2015-11-11 12:50:21 -05:00
Robert Muir
914cee213e Upgrade to aws 1.10.33
Security issues have been fixed. This removes our hacks.
2015-11-10 22:51:33 -05:00
Ryan Ernst
4b5f87cb7d Build: Remove transitive dependencies
Transitive dependencies can be confusing and hard to deal with when
conflicts arise between them. This change removes transitive
dependencies from elasticsearch, and forces any dependency conflicts to
be resolved manually, instead of automatically by gradle.

closes #14627
2015-11-10 15:01:41 -08:00
Tanguy Leroux
4063354dbe Fix Delete-by-Query with Shield
closes #14527
2015-11-10 20:39:38 +01:00
Lee Hinman
bf3e32e8be Add breaker name to logging package
This allows different circuit breakers to have different logging levels.
It's useful when diagnosing problems (say for instance with the
fielddata breaker) and not seeing the enormous amount of logging from
the request breaker.

The log messages use the breaker name for logging, so example logging
will look like:

```
[2015-11-10 09:51:52,993][TRACE][indices.breaker.fielddata] [fielddata] Adding [27b][body] to used bytes [new used: [27b], limit: 623326003 [594.4mb], estimate: 27 [27b]]
[2015-11-10 09:51:53,000][TRACE][indices.breaker.fielddata] [fielddata] Adjusted breaker by [453] bytes, now [480]
[2015-11-10 09:51:53,016][TRACE][indices.breaker.request  ] [request] Adjusted breaker by [16440] bytes, now [16440]
[2015-11-10 09:51:53,018][TRACE][indices.breaker.request  ] [request] Adjusted breaker by [-16440] bytes, now [0]
```
2015-11-10 10:02:34 -07:00
Martijn van Groningen
973c094a5d Merge pull request #14654 from s1monw/fix_searcher_wrapper_interface
Fix IndexSearcherWrapper interface to not depend on the EngineConfig
2015-11-10 23:09:32 +07:00
Ryan Ernst
7bb4e10353 Merge pull request #14645 from rjernst/version_cleanup
Consolidate dependencies specified in multiple places
2015-11-10 07:49:10 -08:00
David Pilato
a61fcfd8ca Remove AbstractLegacyBlobContainer
`AbstractLegacyBlobContainer` was kept for historical reasons (see #13434).
We can migrate Azure and S3 repositories to use the new methods added in #13434 so we can remove `AbstractLegacyBlobContainer` class.
2015-11-10 16:04:44 +01:00
Simon Willnauer
049430621d Fix IndexSearcherWrapper interface to not depend on the EngineConfig
We only passed the engine config since we had no chance to get the cache
etc. from the IndexSearcher. Now that we have these getters we can just pass the
searcher instead.
2015-11-10 14:53:32 +01:00
Simon Willnauer
17913c56fb Merge pull request #14591 from s1monw/issues/14387
Only allow rebalance operations to run if all shard store data is available
2015-11-10 12:06:31 +01:00
Simon Willnauer
84033f3f2e Use syntactic sugar for starting data only nodes 2015-11-10 12:00:29 +01:00
Ryan Ernst
7a6155e12f Build: Consolidate dependencies specified in multiple places
Some dependencies must be specified in a couple places in the build.
e.g. randomized runner is specified both in buildSrc (for the gradle
wrapper plugin), as well as in the test-framework.

This change creates buildSrc/versions.properties which acts similar to
the set of shared version properties we used to have in the maven parent
pom.
2015-11-09 23:59:56 -08:00
Nicholas Knize
3037970c33 Update GeoPoint FieldData for GeoPointV2
This commit adds an abstraction layer to GeoPoint FieldData for supporting new Lucene 5.4 GeoPointField and backwards compatibility.
2015-11-09 15:39:24 -06:00
Adrien Grand
d0853e9253 Merge pull request #14619 from jpountz/upgrade/lucene-5.4.0-snapshot-1712973
Upgrade to lucene-5.4.0-snapshot-1712973.
2015-11-09 22:38:44 +01:00
Ryan Ernst
7034feeceb Merge pull request #14451 from andrejserafim/run
Add gradle run command to replace run.bat and run.sh
2015-11-09 11:15:53 -08:00
Lee Hinman
25f8db9aa9 Merge remote-tracking branch 'dakrone/node-limit-decider' 2015-11-09 11:03:19 -07:00
Lee Hinman
145374b762 Add cluster-wide setting for total shard limit
This adds the `cluster.routing.allocation.total_shards_per_node`
setting, which limits the total number of shards across all indices on
each node. It defaults to -1 and can be dynamically configured.

Resolves #14456
2015-11-09 11:03:07 -07:00
Nicholas Knize
36e294a03e Fix GeoDistanceRangeQueryTest tolerance
GeoDistanceRangeQueryTest was using an inconsistent tolerance when validating from=0 with include_lower/upper set to false. This commit changes the assertion such that validation is consistent with builder logic.
2015-11-09 11:03:22 -06:00
Yannick Welsch
33ca2ec597 Merge pull request #14222 from ywelsch/fix/refactor-transport-master-node-action
Refactor retry logic for TransportMasterNodeAction
2015-11-09 17:59:16 +01:00
Yannick Welsch
0575744a39 Refactor retry logic for TransportMasterNodeAction
- Same as for TransportInstanceSingleOperationAction and TransportReplicationAction, onClusterServiceClose consistently throws a NodeClosedException now.
- Added retry logic if master could not publish cluster state or stepped down before publishing (ZenDiscovery). The test IndexingMasterFailoverIT shows the issue.
- Simplified retry logic by moving bits from different places into shared retry method.
  - Removed boolean flag retrying that aborted retrying after a single master node change (now we retry until timeout).
  - Two existing predicates that deal with master node changes unified in a single predicate masterNodeChangedPredicate

Closes #14222
2015-11-09 17:56:18 +01:00
Adrien Grand
d6d7af0a6c Upgrade to lucene-5.4.0-snapshot-1712973. 2015-11-09 15:53:27 +01:00
javanna
43b729f34c [TEST] test query and search source parsing from different XContent types
We used to test only json parsing as we relied on QueryBuilder#toString which uses the json format. This commit makes sure that we now output the randomly generated queries using a random format, and that we are always able to parse them correctly.

This revealed a couple of issues with binary objects that haven't been migrated yet to be structured Writeable objects. We used to keep them in the format they were sent while parsing, which led to problems when printing them out as we expected them to always be in json format. Also we can't compare different BytesReference objects that hold the same content but in different formats (unless we want to parse them as part of equal and hashcode, doesn't seem like a good idea) and verify that we have parsed the right objects if they can be different formats. The fix is to always keep binary objects in json format. Best fix would be not to have binary objects, which we'll get to once we are done with the search refactoring.

Closes #14415
2015-11-09 15:09:27 +01:00
javanna
10ddd691a3 Internal: move to lucene BoostQuery
Latest version of lucene deprecated Query#setBoost and Query#getBoost which made queries effectively immutable. Those methods need to be replaced with `BoostQuery` that wraps any query that needs boosting.

This commit replaces usages of setBoost with BoostQuery and adds it to forbidden-apis for prod code.

Usages of `getBoost` are only partially removed, as some will have to stay for backwards compatibility.

Closes #14264
2015-11-09 15:01:35 +01:00
Simon Willnauer
7b5e323ec0 add IT for #14387 2015-11-09 15:00:56 +01:00
Robert Muir
102e25413d Merge pull request #14605 from rmuir/migrate-mapper-attachments
Migrate mapper attachments plugin to main repository
2015-11-09 08:55:29 -05:00
Simon Willnauer
219fe8f30f apply @bleskes review changes 2015-11-09 12:58:11 +01:00
Daniel Mitterdorfer
1c5b0b3c5b Merge branch 'pr/14611' 2015-11-09 12:08:56 +01:00
Adrien Grand
f400f58132 Merge pull request #14472 from jpountz/fix/simplify_xcontent_detection
Simplify XContent detection.
2015-11-09 12:02:39 +01:00
Adrien Grand
c350a3dfae Simplify XContent detection.
Currently we duplicate the logic for BytesReferences and InputStreams.
2015-11-09 12:00:03 +01:00
Martijn van Groningen
72e9bd8506 Merge pull request #14597 from wbowling/tophitsbuilder-addfield
Put method addField on TopHitsBuilder
2015-11-09 17:40:13 +07:00
Camilo Díaz Repka
d795ce3b6e XContentFactory.xContentType: allow for possible UTF-8 BOM for JSON XContentType
Fixes #14442
2015-11-09 11:19:54 +01:00
Robert Muir
0de503b34b fixup issues with 32-bit jvm
If you run tests under a 32-bit jvm, you will get a test failure in IndexStoreTests,
the logic there is wrong in the case of 32-bit (its NIOFSDirectory on linux).

Also if mlockall fails, you'll see huge bogus values (because of use of `long` instead of `NativeLong`)

finally add seccomp support for 32 bit too, and clean up all its `long` usage as well.
2015-11-08 21:08:08 -05:00
andrejserafim
2bd353d72d replacing run.bat and run.sh with gradle run
run.sh and run.bat were calling out to the old maven build system.
This is no longer in place, so we've created new gradle tasks to
start an elasticsearch node from the current codebase.

fixed #14423
2015-11-08 17:07:19 +00:00
Simon Willnauer
47769cc4b4 add more logging to test for better readability 2015-11-08 12:43:24 +01:00
Simon Willnauer
fa6a2932f5 Instead of relying on the in-flight number use a boolean flag on the
allocation to ensure that we skip rebalance if we have not processed
all pending shard / store fetches.
2015-11-08 12:40:05 +01:00
Robert Muir
e88896e40e apply a few post-merge cleanups 2015-11-08 02:31:26 -05:00