18172 Commits

Author SHA1 Message Date
Boaz Leskes
3978508a29 Merge pull request from bleskes/inject_remove
IndexService: remove unneed inject annotation
2015-12-15 15:33:50 +01:00
Boaz Leskes
07044e02b9 IndexService: remove unneed inject annotation from 2015-12-15 15:25:17 +01:00
umeku
0ce88b5887 Fix inaccurate docs for nested datatype
Closes 
2015-12-15 15:15:00 +01:00
Adrien Grand
c7bcb32ae6 Merge pull request from jpountz/remove/back_compat_path
Remove back compat for the `path` option.
2015-12-15 15:04:04 +01:00
Adrien Grand
d94bba2d9c Remove back compat for the path option.
The `path` option allowed to index/store a field `a.b.c` under just `c` when
set to `just_name`. This "feature" has been removed in 2.0 in favor of `copy_to`
so we can remove the back compat in 3.x.
2015-12-15 14:55:23 +01:00
Costin Leau
4426ed0a09 [DOCS] Link docs on repository-hdfs plugin
relates 
2015-12-15 14:52:36 +02:00
Clinton Gormley
82eb498b29 Docs: Updated plugin author help for Gradle
Relates to 
2015-12-15 12:27:47 +01:00
Jim Ferenczi
715528a567 Merge pull request from jimferenczi/terms_agg_getKeyAsString
getKeyAsString and key_as_string should be the same for terms aggregation.
2015-12-15 12:11:23 +01:00
Jim Ferenczi
76713611e9 Format the term with the formatter in LongTerms.Bucket.getKeyAsString in order to be consistent
with the json response field `key_as_string` of the terms aggregation.
2015-12-15 12:10:06 +01:00
Clinton Gormley
061446b25a Merge pull request from cjohansen/patch-1
Fix typo
2015-12-15 10:57:38 +01:00
Adrien Grand
c40099d4ef Merge pull request from jpountz/fix/field_uniqueness
Validate that fields are defined only once.
2015-12-15 10:38:47 +01:00
Adrien Grand
5d5c6591aa Validate that fields are defined only once.
There are two ways that a field can be defined twice:
 - by reusing the name of a meta mapper in the root object (`_id`, `_routing`,
   etc.)
 - by defining a sub-field both explicitly in the mapping and through the code
   in a field mapper (like ExternalMapper does)

This commit adds new checks in order to make sure this never happens.

Close 
2015-12-15 10:37:18 +01:00
Adrien Grand
7473905620 Merge pull request from jpountz/fix/immutable_mappings
Make mappings immutable.
2015-12-15 10:23:34 +01:00
Adrien Grand
50eeafa75c Make mappings immutable.
Today mappings are mutable because of two APIs:
 - Mapper.merge, which expects changes to be performed in-place
 - IncludeInAll, which allows to change whether values should be put in the
   `_all` field in place.

This commit changes both APIs to return a modified copy instead of modifying in
place so that mappings can be immutable. For now, only the type-level object is
immutable, but in the future we can imagine making them immutable at the
index-level so that mapping updates could be completely atomic at the index
level.

Close 
2015-12-15 10:20:28 +01:00
Martijn van Groningen
ec908ddfd6 Use transport service to handle RetryOnReplicaException and execute replica action on the current node.
Transport service will delegate to threadpool internally.
2015-12-15 09:32:58 +01:00
Robert Muir
5f7b863067 Merge pull request from rmuir/who_tests_the_tester
smoke test plugins does not test any plugins
2015-12-15 01:11:27 -05:00
Ryan Ernst
60d35c81af Plugins: Expose http.type setting, and collapse al(most all) modules relating to transport/http
This change adds back the http.type setting. It also cleans up all the
transport related guice code to be consolidated within the
NetworkModule (as transport and http related stuff is what and how ES
exposes over the network). The setter methods previously used by some
plugins to override eg the TransportService or HttpServerTransport are
removed, and those plugins should now register a custom implementation
of the class with a name and set that using the appropriate config
setting. Note that I think ActionModule should also be moved into here,
to sit along side the rest actions, but I left that for a followup.

closes 
2015-12-14 22:01:04 -08:00
Robert Muir
2d42e99c7a smoke test plugins does not test any plugins
Currently the build has a bug and it loads 0 plugins.
2015-12-15 00:05:14 -05:00
Jason Tedor
ffdda793a0 Merge pull request from jasontedor/intellij-type-inference
Fix IntelliJ query builder type inference issues
2015-12-14 19:35:31 -05:00
Jason Tedor
2e721a0328 Fix IntelliJ query builder type inference issues
This commit addresses two type inference issues that the IntelliJ source
editor struggles with when registering query builder prototypes in
o/e/i/q/IndicesQueriesRegistry.java and
o/e/i/q/f/ScoreFunctionParserMapper.java.
2015-12-14 18:54:35 -05:00
Jason Tedor
d64baf0f30 Explicitly log cluster state update failures
This commit adds explicit logging at the DEBUG level for cluster state
update failures. Currently this responsibility is left to the cluster
state task listener, but we should expliclty log these with a generic
message to address cases where the listener might not.

Relates , relates , relates 
2015-12-14 17:20:22 -05:00
Jason Tedor
1f0c23a21d Merge pull request from jasontedor/trace-log-shard-not-available-exceptions
Only trace log shard not available exceptions
2015-12-14 16:20:47 -05:00
Jason Tedor
6b39ff6081 Only trace log shard not available exceptions
This commit changes the behavior of the logging in
TransportBroadcastByNodeAction#onNodeFailure to only trace log
exceptions that are considered shard-not-available exceptions. This
makes the logging consistent with how these exceptions are handled in
the response.

Relates 
2015-12-14 16:20:13 -05:00
Costin Leau
7bca97bba6 HDFS Snapshot/Restore plugin
Migrated from ES-Hadoop. Contains several improvements regarding:

* Security
Takes advantage of the pluggable security in ES 2.2 and uses that in order
to grant the necessary permissions to the Hadoop libs. It relies on a
dedicated DomainCombiner to grant permissions only when needed only to the
libraries installed in the plugin folder
Add security checks for SpecialPermission/scripting and provides out of
the box permissions for the latest Hadoop 1.x (1.2.1) and 2.x (2.7.1)

* Testing
Uses a customized Local FS to perform actual integration testing of the
Hadoop stack (and thus to make sure the proper permissions and ACC blocks
are in place) however without requiring extra permissions for testing.
If needed, a MiniDFS cluster is provided (though it requires extra
permissions to bind ports)
Provides a RestIT test

* Build system
Picks the build system used in ES (still Gradle)
2015-12-14 21:50:09 +02:00
Christoph Büscher
c7a001dcb4 Merge pull request from cbuescher/highlight-refactoring-searchSource
Use HighlightBuilder in SearchSourceBuilder
2015-12-14 20:35:07 +01:00
Simon Willnauer
bc8745dcc2 Merge pull request from s1monw/fail_translog_on_tragic_event
Fail and close translog hard if writing to disk fails
2015-12-14 18:30:04 +01:00
Simon Willnauer
943915e05a simplify code and use members directly 2015-12-14 18:06:49 +01:00
Christoph Büscher
59bd21f24f Adding test for Order enum ordinals 2015-12-14 17:29:25 +01:00
Christoph Büscher
6f16317a4b Switch internal representation of order to enum 2015-12-14 16:56:39 +01:00
Simon Willnauer
b0144c557c apply feedback from @mikemccand 2015-12-14 16:09:20 +01:00
Christoph Büscher
659bab7105 Use HighlightBuilder in SearchSourceBuilder
After HighlightBuilder implements Writable now, we can remove
the temporary solution for transporting the highlight section in
SearchSourceBuilder from the coordinating node to the shard as
BytesReference and use HighlightBuilder instead.
2015-12-14 15:20:49 +01:00
Christoph Büscher
7742c1e181 Merge pull request from cbuescher/highlight-builder-shared-parsing
Joint parsing of common global Hightlighter and subfield parameters
2015-12-14 15:19:03 +01:00
Christoph Büscher
2ab937d64d Joint parsing of common global Hightlighter and subfield parameters
The top-level highlighter has many options that can be overwritten per
field. Currently there is very similar code for this in two places.
This PR pulls out the parsing of the common parameters into
AbstractHighlighterBuilder for better reuse and to keep parsing of
common parameters more consistent.
2015-12-14 15:09:00 +01:00
Simon Willnauer
6cefdc82f6 Expose tragic event to translog, close translog once we hit a tragic even and fail engine if we hit one too 2015-12-14 15:01:55 +01:00
Simon Willnauer
c6003b6f13 apply feedback from @bleskes 2015-12-14 14:36:20 +01:00
Clinton Gormley
f20f41e02e Merge pull request from alexg-dev/patch-1
More detailed explanation of some similarity types
2015-12-14 14:28:43 +01:00
Clinton Gormley
6144457f01 Merge pull request from fforbeck/master
Added description for parameter <local>, FIX 
2015-12-14 14:24:49 +01:00
Clinton Gormley
83ee1fc903 Merge pull request from TheDude05/fix-match_pattern-docs
Fix docs with `match_pattern` in dynamic templates
2015-12-14 14:18:59 +01:00
Simon Willnauer
2d03a6b808 Fail and close translog hard if writing to disk fails
Today we are super lenient (how could I missed that for f**k sake) with failing
/ closing the translog writer when we hit an exception. It's actually worse, we allow
to further write to it and don't care what has been already written to disk and what hasn't.
We keep the buffer in memory and try to write it again on the next operation.

When we hit a disk-full expcetion due to for instance a big merge we are likely adding document to the
translog but fail to write them to disk. Once the merge failed and freed up it's diskspace (note this is
a small window when concurrently indexing and failing the shard due to out of space exceptions) we will
allow in-flight operations to add to the translog and then once we fail the shard fsync it. These operations
are written to disk and fsynced which is fine but the previous buffer flush might have written some bytes
to disk which are not corrupting the translog. That wouldn't be an issue if we prevented the fsync.

Closes 
2015-12-14 13:16:17 +01:00
Christoph Büscher
f520f8bfb9 Tests: Add test for parsing "_name" field in RangeQueryParser 2015-12-14 10:07:35 +01:00
Jason Tedor
2602439a51 Delegate to j.u.c.ThreadLocalRandom for Random instances 2015-12-12 17:51:46 -05:00
Ryan Ernst
089a9e6977 Merge pull request from rjernst/port_zero
Change rest integ tests to not have hardcoded ports
2015-12-11 18:20:55 -08:00
Ryan Ernst
a0c69fe7f9 Remove forbidden suppressions for InetSocketAddress 2015-12-11 18:20:09 -08:00
Ryan Ernst
7cbb955d19 Move forbidden suppression for InetSocketAddress to its own method 2015-12-11 18:08:39 -08:00
Ryan Ernst
5c8a0da1fd Build: Change rest integ tests to not have hardcoded ports
This change removes hardcoded ports from cluster formation. It passes
port 0 for http and transport, and then uses a special property to have
the node log the ports used for http and transport (just for tests).
This does not yet work for multi node tests. This brings us one step
closer to working with --parallel.
2015-12-11 17:36:08 -08:00
Felipe Forbeck
708abcc59a Added desc for parameter <local> 2015-12-11 22:26:33 -02:00
Jason Tedor
a42823ad45 Improve ThreadLocal handling in o.e.c.Randomness
This commit improves the handling of ThreadLocal Random instance
allocation in o.e.c.Randomness.
 - the seed per instance is no longer fixed
 - a non-dangerous race to create the ThreadLocal instance has been
   removed
 - encapsulated all state into an static nested class for safe and lazy
   instantiation
2015-12-11 19:06:59 -05:00
Nicholas Knize
5f3d807f61 Update geo_shape/query docs, fix TermStrategy defaults
This commit adds the following:
* SpatialStrategy documentation to the geo-shape reference docs.
* Updates relation documentation to geo-shape-query reference docs.
* Updates GeoShapeFiledMapper to set points_only to true if TERM strategy is used (to be consistent with documentation)
2015-12-11 17:14:22 -06:00
Andrew Williams
e7127c9f6f Fix docs with match_pattern in dynamic templates 2015-12-11 14:03:54 -06:00
Jim Ferenczi
9ab168dbf6 Removes all the reference of the query in the docs 2015-12-11 20:07:57 +01:00