2678 Commits

Author SHA1 Message Date
Ryan Ernst
7cbb955d19 Move forbidden suppression for InetSocketAddress to its own method 2015-12-11 18:08:39 -08:00
Ryan Ernst
5c8a0da1fd Build: Change rest integ tests to not have hardcoded ports
This change removes hardcoded ports from cluster formation. It passes
port 0 for http and transport, and then uses a special property to have
the node log the ports used for http and transport (just for tests).
This does not yet work for multi node tests. This brings us one step
closer to working with --parallel.
2015-12-11 17:36:08 -08:00
Jason Tedor
a42823ad45 Improve ThreadLocal handling in o.e.c.Randomness
This commit improves the handling of ThreadLocal Random instance
allocation in o.e.c.Randomness.
 - the seed per instance is no longer fixed
 - a non-dangerous race to create the ThreadLocal instance has been
   removed
 - encapsulated all state into an static nested class for safe and lazy
   instantiation
2015-12-11 19:06:59 -05:00
Nicholas Knize
5f3d807f61 Update geo_shape/query docs, fix TermStrategy defaults
This commit adds the following:
* SpatialStrategy documentation to the geo-shape reference docs.
* Updates relation documentation to geo-shape-query reference docs.
* Updates GeoShapeFiledMapper to set points_only to true if TERM strategy is used (to be consistent with documentation)
2015-12-11 17:14:22 -06:00
Adrien Grand
003e19b5fb Merge pull request #15398 from jpountz/remove/_source_format
Remove the `format` option of the `_source` field.
2015-12-11 18:27:27 +01:00
Jason Tedor
a087d557f9 Fix NullPointerException in o.e.c.Randomness 2015-12-11 12:10:43 -05:00
Adrien Grand
9fd843748e Remove the format option of the _source field.
This option allows to force the xcontent type to use to store the `_source`
document. The default is to use the same format as the input format.

This commit makes this option ignored for 2.x indices and rejected for 3.0
indices.
2015-12-11 18:06:20 +01:00
Jason Tedor
cd40cd78ba Merge pull request #15299 from jasontedor/collections-shuffle-reproducibility
Remove and forbid use of Collections#shuffle(List) and Random#<init>()
2015-12-11 11:24:56 -05:00
Jim Ferenczi
d78e24689b Merge pull request #15364 from jimferenczi/missing_query_removal
Remove the MissingQueryBuilder which was deprecated in 2.2.0.
2015-12-11 17:20:47 +01:00
Jason Tedor
3383c24be0 Remove and forbid use of Collections#shuffle(List) and Random#<init>()
This commit removes and now forbids all uses of
Collections#shuffle(List) and Random#<init>() across the codebase. The
rationale for removing and forbidding these methods is to increase test
reproducibility. As these methods use non-reproducible seeds, production
code and tests that rely on these methods contribute to
non-reproducbility of tests.

Instead of Collections#shuffle(List) the method
Collections#shuffle(List, Random) can be used. All that is required then
is a reproducible source of randomness. Consequently, the utility class
Randomness has been added to assist in creating reproducible sources of
randomness.

Instead of Random#<init>(), Random#<init>(long) with a reproducible seed
or the aforementioned Randomess class can be used.

Closes #15287
2015-12-11 11:16:38 -05:00
Adrien Grand
ed53edd4d0 Merge pull request #15344 from jpountz/fix/remove_source_backcompat
Remove back compat for the `_source` compression options.
2015-12-11 17:01:15 +01:00
Daniel Mitterdorfer
f5d307e426 BulkItemResponse returns status code instead of status name
In commit fafeb3a, we've refactored REST response handling logic
and returned HTTP status names instead of HTTP status codes for
bulk item responses. With this commit we restore the original
behavior.

Checked with @bleskes.
2015-12-11 16:22:36 +01:00
Simon Willnauer
90ff1ad5fb Merge pull request #15235 from s1monw/remove_recovery_settings
Restore chunksize of 512kb on recovery and remove configurability
2015-12-11 15:27:20 +01:00
Simon Willnauer
db15682b3c apply review comments from @bleskes 2015-12-11 15:16:14 +01:00
Adrien Grand
afcaa593ae Merge pull request #15358 from jpountz/fix/safe_write_raw
Make XContentGenerator.writeRaw* safer.
2015-12-11 12:18:04 +01:00
Isabel Drost-Fromm
8dad4d03f4 Merge pull request #11598 from srogljan/master
UNICODE_CHARACTER_CLASS fix
2015-12-11 11:53:35 +01:00
Adrien Grand
d408313974 Make XContentGenerator.writeRaw* safer.
This method currently allows to write arbitrary bytes in an xcontent stream.
I changed it so that it can only write data to the same stream as the xcontent
(the bos parameter is removed) and that it yells at you if you try to write
raw bytes that can't be recognized as xcontent. Also the logic to copy the
structure instead of appending the bytes directly if the source and target
are of a different xcontent type have been moved to the low-level
XContentGenerator.
2015-12-11 11:13:57 +01:00
Yannick Welsch
db5594a2af Support wildcards for getting repositories or snapshots
Closes #15151
2015-12-11 10:20:27 +01:00
Simon Willnauer
12f905a675 Make chunkSize configurabel for tests and use correct close handling for closing streams to not hide original exception 2015-12-11 09:44:08 +01:00
Ryan Ernst
81d6363967 Remove benchmark package
Tons of ancient "benchmarks" exist in elasticsearch. These are main
methods that do some kind of construction of ES classes and time various
things. The problem with these is they are not maintained, and not run.
Refactorings that touch anything that is common in these classes is very
painful. Going through these, almost all would simply not work in 2.x
without modifications (because they do not set path.home).

This change removes the entire benchmark package. If someone needs to
run a benchmark like this, they can look at history for examples if
necessary (although these examples are often not realistic and should
just start real elasticsearch processes in a shell script). Longer term,
we should make this easier to do by having the build support adding real
benchmarks which can be run in jenkins (so we know they actually run,
instead of doing refactorings with pure guesswork as to whether the
benchmark would run correctly).
2015-12-10 19:29:54 -08:00
Ryan Ernst
93de1ed606 Merge pull request #15354 from rjernst/just_node
Remove NodeBuilder
2015-12-10 19:19:55 -08:00
Robert Muir
d190283fea Merge pull request #15378 from rmuir/tame_reflection
Remove RuntimePermission("accessDeclaredMembers")
2015-12-10 15:25:44 -05:00
Robert Muir
2741888498 Remove RuntimePermission("accessDeclaredMembers")
Upgrades lucene to 5.5.0-1719088, randomizedtesting to 2.3.2, and securemock to 1.2
2015-12-10 14:26:55 -05:00
Boaz Leskes
09053f5c87 remove safety check from IndexShard#prepareDeleteOnReplica
we are not ready for this yet:

```
        if (shardRouting.primary() && shardRouting.isRelocationTarget() == false) {
            throw new IllegalIndexShardStateException(shardId, state, "shard is not a replica");
        }
```
2015-12-10 16:04:47 +01:00
Boaz Leskes
fafeb3abdd Introduce a common base response class to all single doc write ops
IndexResponse, DeleteResponse and UpdateResponse share some logic. This can be unified to a single DocWriteResponse base class. On top, some replication actions are now not about write operations anymore. This commit renames ActionWriteResponse to ReplicationResponse

Last some toXContent is moved from the Rest layer to the actual response classes, for more code re-sharing.

Closes #15334
2015-12-10 15:14:02 +01:00
Jun Ohtani
fab44398d9 Analysis: Add detail response support
add explain option
fix char_filter bug

Closes #11076 #15257
2015-12-10 23:10:51 +09:00
Robert Muir
1ef24d2a85 Merge pull request #15328 from rmuir/shave_mustache
Factor mustache -> modules/lang-mustache
2015-12-10 08:46:57 -05:00
Daniel Mitterdorfer
fc9afa2bb8 Reduce maximum number of concurrent requests in BulkProcessorIT
The test configuration with seed A23029712A7EFB34 overwhelmed the pool which is invoked
in TransportService#sendLocalRequest().

With this commit we reduce the maximum number of concurrent requests from 10 to 7 and
add the failure message to the test output on the failing assertion for easier analysis.
2015-12-10 14:24:35 +01:00
Jim Ferenczi
176a4fc8da Review ++ 2015-12-10 14:19:19 +01:00
Robert Muir
e454fadc22 Merge branch 'master' into shave_mustache 2015-12-10 07:58:24 -05:00
Jim Ferenczi
437488ae64 Remove the MissingQueryBuilder which was deprecated in 2.2.0.
As a replacement use ExistsQueryBuilder inside a mustNot() clause.
So instead of using `new ExistsQueryBuilder(name)` now use:
`new BoolQueryBuilder().mustNot(new ExistsQueryBuilder(name))`.

Closes #14112
2015-12-10 12:55:05 +01:00
Robert Muir
7595c4a3c8 Improve network docs
This makes some minor improvements (does not fix all problems!)

It reorders unicast disco in elasticsearch.yml to be right after the network host,
for better locality.

It removes the warning (unreleased) about publish addresses, lets try to really discourage setting
that unless you need to (behind a proxy server). Most people should be fine with `network.host`

Finally it reorganizes the network docs page a bit:

We add a table of 4 "basic" settings at the very beginning:

* network.host
* discovery.zen.ping.unicast.hosts
* http.port
* transport.tcp.port

The first two being the most important, which addresses to bind and talk to, and the other two
being the port numbers.

The rest of the stuff I tried to simplify and reorder under "advanced" headers.

This is just a quick stab, I still think we need more effort into this thing, but we gotta start somewhere.
2015-12-10 12:01:18 +01:00
Christoph Büscher
79cdc40afe Merge pull request #15324 from cbuescher/highlight-builder-searchContextHighlight
Enable HighlightBuilder to create SearchContextHighlight
2015-12-10 11:15:43 +01:00
Adrien Grand
29859a946d Merge pull request #15337 from jpountz/remove/estimatedNumberType
Remove XContentParser.estimatedNumberType().
2015-12-10 10:09:09 +01:00
Yannick Welsch
17da7fb0ee Merge pull request #15305 from ywelsch/feature/multi-alias-rest
Add support to _aliases endpoint to specify multiple indices and aliases in one action
2015-12-10 09:44:36 +01:00
Ryan Ernst
6f166dd0ff Remove NodeBuilder
The NodeBuilder is currently used to construct a Node. However, this is
really just yet-another-builder that wraps around a Settings.Builder
witha couple convenience methods. But there are very few uses of these
convenience methods.  This change removes NodeBuilder, in favor of just
using the Node constructor.
2015-12-10 00:01:44 -08:00
Areek Zillur
025e9818e7 Refactors TransportReplicationAction to decouple request routing and shard operation logic 2015-12-10 01:32:07 -05:00
Jack Conradson
da5b07ae13 Added a new scripting language (PlanA).
Closes #15136
2015-12-09 16:32:37 -08:00
David Pilato
f675801b26 add more tests to network service
Follow up for #15340

We test that bind with wilcard IP + fixed IP it raises an exception
We test binding multiple IPs

(cherry picked from commit 2cc5bb7)
2015-12-09 23:17:59 +01:00
Adrien Grand
ab44366d93 Remove back compat for the _source compression options.
These options have been deprecated before 2.0 so they don't need to be supported
by 3.0.
2015-12-09 19:22:34 +01:00
Yannick Welsch
bef0bedba9 Add support to _aliases endpoint to specify multiple indices and aliases in one action
Closes #15305
2015-12-09 19:08:27 +01:00
Christoph Büscher
e4721fd02a Addressing review comments 2015-12-09 18:04:58 +01:00
Adrien Grand
3e47f90460 Remove XContentParser.estimatedNumberType().
The goal of this method is to know whether the xcontent impl knows how to
differenciate floats from doubles or longs from ints or if it's just guessing.
However, all implementations return true (which is correct for yaml and json,
but cbor and smile should be able to differenciate). I first tried to implement
this method correctly but it raised many issues because eg. most impls write a
long as an integer when it is small enough. So I suggest that we remove this
method and just treat cbor and smile like yaml and json, which is already what
is happening today anyway.
2015-12-09 14:56:50 +01:00
Colin Goodheart-Smithe
b4e0c876d5 Merge pull request #15321 from martinstuga/Refactor-StatsAggegator#15264
Correct typo in class name of StatsAggregator (Closes #14730)
2015-12-09 13:39:27 +00:00
Christoph Büscher
cb84b1ff1a Make HighlightBuilder produce SearchContextHighlight
The HighlightBuilder should be able to procude a SeachContextHighlight
object which contains the merged global and field options, also
contains objects that can only be created on the index shard (like
the actual lucene Query object used during highlighting).
This is done by the build() method of the HighlighBuilder. Also
adding tests that make sure the produced SearchContextHighlighter is
similar to the one we would get when parsing the xContent directly
with the current HighlightParseElement.
2015-12-09 11:44:45 +01:00
Boaz Leskes
ccd0543172 CancellableThreads should also treat ThreadInterruptedException as InterruptedException
RecoverySource uses the RateLimiter under a cancelable thread. The SimpleRateLimiter used in throws   ThreadInterruptedException on interruption. We should treat it as InterruptedException
2015-12-09 09:29:23 +01:00
Adrien Grand
1909c3d8cc Only text fields should accept analyzer and term vector settings.
Currently almost all our fields accept the `analyzer` and `term_vector` settings
although they only make sense on text fields. This commit forbids those settings
on all fields but `string` and `_all` for indices created on or after version
2.2.0.
2015-12-09 08:42:56 +01:00
Adrien Grand
473e880a7c Merge pull request #15319 from jpountz/enhancement/dynamic_mappings_float
Dynamically map floating-point numbers as floats instead of doubles.
2015-12-09 08:38:27 +01:00
Robert Muir
0bf9e36eaa Merge pull request #15311 from rmuir/symlinks_are_bad
punch thru symlinks when loading plugins/modules
2015-12-09 01:35:00 -05:00
Ryan Ernst
b7fb0824be Fix render search template tests 2015-12-08 21:33:48 -08:00