2490 Commits

Author SHA1 Message Date
Lee Hinman
bf3e32e8be Add breaker name to logging package
This allows different circuit breakers to have different logging levels.
It's useful when diagnosing problems (say for instance with the
fielddata breaker) and not seeing the enormous amount of logging from
the request breaker.

The log messages use the breaker name for logging, so example logging
will look like:

```
[2015-11-10 09:51:52,993][TRACE][indices.breaker.fielddata] [fielddata] Adding [27b][body] to used bytes [new used: [27b], limit: 623326003 [594.4mb], estimate: 27 [27b]]
[2015-11-10 09:51:53,000][TRACE][indices.breaker.fielddata] [fielddata] Adjusted breaker by [453] bytes, now [480]
[2015-11-10 09:51:53,016][TRACE][indices.breaker.request  ] [request] Adjusted breaker by [16440] bytes, now [16440]
[2015-11-10 09:51:53,018][TRACE][indices.breaker.request  ] [request] Adjusted breaker by [-16440] bytes, now [0]
```
2015-11-10 10:02:34 -07:00
Martijn van Groningen
973c094a5d Merge pull request #14654 from s1monw/fix_searcher_wrapper_interface
Fix IndexSearcherWrapper interface to not depend on the EngineConfig
2015-11-10 23:09:32 +07:00
Ryan Ernst
7bb4e10353 Merge pull request #14645 from rjernst/version_cleanup
Consolidate dependencies specified in multiple places
2015-11-10 07:49:10 -08:00
David Pilato
a61fcfd8ca Remove AbstractLegacyBlobContainer
`AbstractLegacyBlobContainer` was kept for historical reasons (see #13434).
We can migrate Azure and S3 repositories to use the new methods added in #13434 so we can remove `AbstractLegacyBlobContainer` class.
2015-11-10 16:04:44 +01:00
Simon Willnauer
049430621d Fix IndexSearcherWrapper interface to not depend on the EngineConfig
We only passed the engine config since we had no chance to get the cache
etc. from the IndexSearcher. Now that we have these getters we can just pass the
searcher instead.
2015-11-10 14:53:32 +01:00
Simon Willnauer
17913c56fb Merge pull request #14591 from s1monw/issues/14387
Only allow rebalance operations to run if all shard store data is available
2015-11-10 12:06:31 +01:00
Simon Willnauer
84033f3f2e Use syntactic sugar for starting data only nodes 2015-11-10 12:00:29 +01:00
Ryan Ernst
7a6155e12f Build: Consolidate dependencies specified in multiple places
Some dependencies must be specified in a couple places in the build.
e.g. randomized runner is specified both in buildSrc (for the gradle
wrapper plugin), as well as in the test-framework.

This change creates buildSrc/versions.properties which acts similar to
the set of shared version properties we used to have in the maven parent
pom.
2015-11-09 23:59:56 -08:00
Nicholas Knize
3037970c33 Update GeoPoint FieldData for GeoPointV2
This commit adds an abstraction layer to GeoPoint FieldData for supporting new Lucene 5.4 GeoPointField and backwards compatibility.
2015-11-09 15:39:24 -06:00
Adrien Grand
d0853e9253 Merge pull request #14619 from jpountz/upgrade/lucene-5.4.0-snapshot-1712973
Upgrade to lucene-5.4.0-snapshot-1712973.
2015-11-09 22:38:44 +01:00
Ryan Ernst
7034feeceb Merge pull request #14451 from andrejserafim/run
Add gradle run command to replace run.bat and run.sh
2015-11-09 11:15:53 -08:00
Lee Hinman
25f8db9aa9 Merge remote-tracking branch 'dakrone/node-limit-decider' 2015-11-09 11:03:19 -07:00
Lee Hinman
145374b762 Add cluster-wide setting for total shard limit
This adds the `cluster.routing.allocation.total_shards_per_node`
setting, which limits the total number of shards across all indices on
each node. It defaults to -1 and can be dynamically configured.

Resolves #14456
2015-11-09 11:03:07 -07:00
Nicholas Knize
36e294a03e Fix GeoDistanceRangeQueryTest tolerance
GeoDistanceRangeQueryTest was using an inconsistent tolerance when validating from=0 with include_lower/upper set to false. This commit changes the assertion such that validation is consistent with builder logic.
2015-11-09 11:03:22 -06:00
Yannick Welsch
33ca2ec597 Merge pull request #14222 from ywelsch/fix/refactor-transport-master-node-action
Refactor retry logic for TransportMasterNodeAction
2015-11-09 17:59:16 +01:00
Yannick Welsch
0575744a39 Refactor retry logic for TransportMasterNodeAction
- Same as for TransportInstanceSingleOperationAction and TransportReplicationAction, onClusterServiceClose consistently throws a NodeClosedException now.
- Added retry logic if master could not publish cluster state or stepped down before publishing (ZenDiscovery). The test IndexingMasterFailoverIT shows the issue.
- Simplified retry logic by moving bits from different places into shared retry method.
  - Removed boolean flag retrying that aborted retrying after a single master node change (now we retry until timeout).
  - Two existing predicates that deal with master node changes unified in a single predicate masterNodeChangedPredicate

Closes #14222
2015-11-09 17:56:18 +01:00
Adrien Grand
d6d7af0a6c Upgrade to lucene-5.4.0-snapshot-1712973. 2015-11-09 15:53:27 +01:00
javanna
43b729f34c [TEST] test query and search source parsing from different XContent types
We used to test only json parsing as we relied on QueryBuilder#toString which uses the json format. This commit makes sure that we now output the randomly generated queries using a random format, and that we are always able to parse them correctly.

This revealed a couple of issues with binary objects that haven't been migrated yet to be structured Writeable objects. We used to keep them in the format they were sent while parsing, which led to problems when printing them out as we expected them to always be in json format. Also we can't compare different BytesReference objects that hold the same content but in different formats (unless we want to parse them as part of equal and hashcode, doesn't seem like a good idea) and verify that we have parsed the right objects if they can be different formats. The fix is to always keep binary objects in json format. Best fix would be not to have binary objects, which we'll get to once we are done with the search refactoring.

Closes #14415
2015-11-09 15:09:27 +01:00
javanna
10ddd691a3 Internal: move to lucene BoostQuery
Latest version of lucene deprecated Query#setBoost and Query#getBoost which made queries effectively immutable. Those methods need to be replaced with `BoostQuery` that wraps any query that needs boosting.

This commit replaces usages of setBoost with BoostQuery and adds it to forbidden-apis for prod code.

Usages of `getBoost` are only partially removed, as some will have to stay for backwards compatibility.

Closes #14264
2015-11-09 15:01:35 +01:00
Simon Willnauer
7b5e323ec0 add IT for #14387 2015-11-09 15:00:56 +01:00
Robert Muir
102e25413d Merge pull request #14605 from rmuir/migrate-mapper-attachments
Migrate mapper attachments plugin to main repository
2015-11-09 08:55:29 -05:00
Simon Willnauer
219fe8f30f apply @bleskes review changes 2015-11-09 12:58:11 +01:00
Daniel Mitterdorfer
1c5b0b3c5b Merge branch 'pr/14611' 2015-11-09 12:08:56 +01:00
Adrien Grand
f400f58132 Merge pull request #14472 from jpountz/fix/simplify_xcontent_detection
Simplify XContent detection.
2015-11-09 12:02:39 +01:00
Adrien Grand
c350a3dfae Simplify XContent detection.
Currently we duplicate the logic for BytesReferences and InputStreams.
2015-11-09 12:00:03 +01:00
Martijn van Groningen
72e9bd8506 Merge pull request #14597 from wbowling/tophitsbuilder-addfield
Put method addField on TopHitsBuilder
2015-11-09 17:40:13 +07:00
Camilo Díaz Repka
d795ce3b6e XContentFactory.xContentType: allow for possible UTF-8 BOM for JSON XContentType
Fixes #14442
2015-11-09 11:19:54 +01:00
Robert Muir
0de503b34b fixup issues with 32-bit jvm
If you run tests under a 32-bit jvm, you will get a test failure in IndexStoreTests,
the logic there is wrong in the case of 32-bit (its NIOFSDirectory on linux).

Also if mlockall fails, you'll see huge bogus values (because of use of `long` instead of `NativeLong`)

finally add seccomp support for 32 bit too, and clean up all its `long` usage as well.
2015-11-08 21:08:08 -05:00
andrejserafim
2bd353d72d replacing run.bat and run.sh with gradle run
run.sh and run.bat were calling out to the old maven build system.
This is no longer in place, so we've created new gradle tasks to
start an elasticsearch node from the current codebase.

fixed #14423
2015-11-08 17:07:19 +00:00
Simon Willnauer
47769cc4b4 add more logging to test for better readability 2015-11-08 12:43:24 +01:00
Simon Willnauer
fa6a2932f5 Instead of relying on the in-flight number use a boolean flag on the
allocation to ensure that we skip rebalance if we have not processed
all pending shard / store fetches.
2015-11-08 12:40:05 +01:00
Robert Muir
e88896e40e apply a few post-merge cleanups 2015-11-08 02:31:26 -05:00
Areek Zillur
6444194dae fix conflicts from merge 2015-11-07 18:22:28 -05:00
Areek Zillur
d5198b326c fix comments and throw ElasticsearchException when we fail to report fst index size 2015-11-07 17:50:56 -05:00
Areek Zillur
d37e011526 add query context builders 2015-11-07 17:50:56 -05:00
Areek Zillur
40022e2168 remove nocommit 2015-11-07 17:50:55 -05:00
Areek Zillur
e87b4e00eb add back awaitfix tests 2015-11-07 17:50:55 -05:00
Simon Willnauer
6f9a486071 more parsers 2015-11-07 17:50:55 -05:00
Simon Willnauer
f5560a3087 add parsers 2015-11-07 17:47:46 -05:00
Simon Willnauer
8911469e63 some basic cleanups 2015-11-07 17:46:28 -05:00
Areek Zillur
75d55e11ad cut over to using ObjectParser for context mappings 2015-11-07 17:46:27 -05:00
Areek Zillur
96c9849f5a added FuzzyOptionsBuilder javadocs and cleanups 2015-11-07 17:46:27 -05:00
Areek Zillur
dd1c687ace Completion Suggester V2
The completion suggester provides auto-complete/search-as-you-type functionality.
This is a navigational feature to guide users to relevant results as they are typing, improving search precision.
It is not meant for spell correction or did-you-mean functionality like the term or phrase suggesters.

The completions are indexed as a weighted FST (finite state transducer) to provide fast Top N prefix-based
searches suitable for serving relevant results as a user types.

closes #10746
2015-11-07 17:46:27 -05:00
Robert Muir
e06cae84f3 only allow code to bind to the user's configured port numbers/ranges
Random code shouldn't be listening on sockets elsewhere.

Today its the wild west, but we only need to grant access to what the user configured.

This means e.g. multicast plugin has to declare its intentions in its security.policy

Closes #14549
2015-11-07 12:22:12 -05:00
William Bowling
202a735d8f Put method addField on TopHitsBuilder 2015-11-07 17:15:02 +11:00
Robert Muir
f03196193f Fix Build to correctly treat URLs and not leak a file handle.
Closes #14595

Squashed commit of the following:

commit d0b2b262e9dcdbc2aee163b9a84db082c8b5b96b
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 6 22:36:54 2015 -0500

    Switch to JarInputStream, to contain suppressforbidden. Also add a test that fails if path is not accessible (regardless of whether its a jar)

commit f99c1d240db23ceb2a06987b3bd69eae0229550b
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 6 22:16:16 2015 -0500

    remove leniency in i/o here

commit b160d4303ee81a8c9298729596ecbc893f5f8894
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Nov 6 21:58:21 2015 -0500

    Fix Build to correctly treat URLs and to not leak a file handle
2015-11-06 22:40:58 -05:00
Ryan Ernst
59a1cda9bb Merge pull request #14592 from rjernst/eclipse_cycles
Build: Fix eclipse generation to add a core-tests projects
2015-11-06 11:55:47 -08:00
Ryan Ernst
548430072f Build: Fix eclipse generation to add a core-tests projects
Eclipse does not have the ability to differentiate test dependencies
from main dependencies. This causes what looks like a circular
dependency through test-framework. This change sets up an additional
core-tests project for eclipse only, which removes this problem.
2015-11-06 11:41:18 -08:00
Simon Willnauer
01ca95a648 Only allow rebalance operations to run if all shard store data is available
This commit prevents running rebalance operations if the store allocator is
still fetching async shard / store data to prevent pre-mature rebalance decisions
which need to be reverted once shard store data is available. This is typically happening
on rolling restarts which can make those restarts extremely painful.

Closes #14387
2015-11-06 20:18:01 +01:00
Nicholas Knize
afaf96732a Add GeoPointV2 Field Mapping
This commit adds the abstraction layer to GeoPointFieldMapper needed to cut over to Lucene 5.4's new GeoPointField type while maintaining backward compatibility with 'legacy' geo_point indexes.
2015-11-06 13:14:04 -06:00