Commit Graph

6285 Commits

Author SHA1 Message Date
Simon Willnauer c499427166 Use _refresh instead of reading from Translog in the RT GET case (#20102)
Today we do a lot of accounting inside the engine to maintain locations
of documents inside the transaction log. This is only needed to ensure
we can return the documents source from the engine if it hasn't been refreshed.
Aside of the added complexity to be able to read from the currently writing translog,
maintainance of pointers into the translog this also caused inconsistencies like different values
of the `_ttl` field if it was read from the tlog or not. TermVectors are totally different if
the document is fetched from the tranlog since copy fields are ignored etc.

This chance will simply call `refresh` if the documents latest version is not in the index. This
streamlines the semantics of the `_get` API and allows for more optimizations inside the engine
and on the transaction log. Note: `_refresh` is only called iff the requested document is not refreshed
yet but has recently been updated or added.

#Relates to #19787
2016-08-24 15:30:08 +02:00
Simon Willnauer 1b1a1acad8 Don't index the `_version` field (#20132)
The `_version` field doesn't allow to be searched anyway since it's
set `IndexOptions#NONE` for it instead.
2016-08-24 10:04:27 +02:00
Adrien Grand 5d6c9b0745 Fix RAM usage estimation of LiveVersionMap. #20123
I was writing tests for RAM usage estimation of LiveVersionMap and found a
couple issues:
 - The BytesRef objects used as uids were oversized since they were created
   via `new BytesRef(CharSequence)` which creates a `byte[]` whose size is 3x
   the length of the provided char sequence. Given that our uids are most of
   times ASCII sequences, this is a waste of memory.
 - `VersionValue` was using `translogLocation.size` instead of
   `translogLocation.ramBytesUsed()` for RAM estimation, which is completely
   unrelated to the memory footprint of the `Translog.Location` object.

In particular, the latter issue could cause RAM usage estimation to be
significantly overestimated, especially on large documents.

I also added tests for ram accounting.
2016-08-24 09:54:06 +02:00
Lee Hinman 3298a4ed38 Revert "Merge remote-tracking branch 'dakrone/exclude-numerics-from-all'"
This reverts commit 514585290c, reversing
changes made to 8563c8d897.
2016-08-23 09:24:33 -06:00
Nicholas Knize 8234fad9ca Deprecate geohash parameters for geo_point parser
This commit deprecates all geohash parameters in the geo_point field parser.
2016-08-23 09:19:21 -05:00
Nicholas Knize 28ed0e7abf Deprecate optimize_bbox on geodistance queries
Deprecates the optimize_bbox parameter on geodistance queries. This has no longer been needed since version 2.2 because lucene geo distance queries (postings and LatLonPoint) already optimize by bounding box.
2016-08-23 09:14:54 -05:00
Michael McCandless 668dac722a Don't suppress AlreadyClosedException (#19975)
Catching and suppressing AlreadyClosedException from Lucene is dangerous because it can mean there is a bug in ES since ES should normally guard against invoking Lucene classes after they were closed.

I reviewed the cases where we catch AlreadyClosedException from Lucene and removed the ones that I believe are not needed, or improved comments explaining why ACE is OK in that case.

I think (@s1monw can you confirm?) that holding the engine's readLock means IW will not be closed, except if disaster strikes (failEngine) at which point I think it's fine to see the original ACE in the logs?

Closes #19861
2016-08-23 12:37:38 +02:00
Masaru Hasegawa f3cddef61e Merge pull request #20046 from masaruh/same_shard_host_setting
Move cluster.routing.allocation.same_shard.host setting to new settings infrastructure
2016-08-23 11:34:59 +09:00
Jack Conradson 131e370a16 Make Painless the default scripting language.
Closes #20017
2016-08-22 17:38:02 -07:00
Lee Hinman 514585290c Merge remote-tracking branch 'dakrone/exclude-numerics-from-all' 2016-08-22 12:36:25 -06:00
Thiago Souza 8563c8d897 Merge pull request #20042 from tsouza/fix/issue-19364
Use internal from/to when creating InternalDateRange.Bucket
2016-08-22 14:38:13 -03:00
Simon Willnauer 29336b231b Add ref-counting to SearchContext to prevent accessing already closed readers (#20095)
When a SearchContext is closed it's reader / searcher reference is closed too.
If this happens while a search is accessing it's reader reference it can lead
to an unexpected `AlreadyClosedException` or worst case, an already closed MMapDirectory
is access causing a `SIGSEV` like in #20008 (even though the window for this is very small).

SearchContext can be closed concurrently if:
 * an index is deleted / removed from the node
 * a search context is idle for too long and is cleaned by the reaper
 * an explicit freeContext message is received

This change adds reference counting to the SearchContext base class and it's used
inside SearchService each time the context is accessed.

Closes #20008
2016-08-22 15:41:05 +02:00
Masaru Hasegawa c7e36536f6 Move cluster.routing.allocation.same_shard.host setting to new settings infrastructure
Fixes #20045
2016-08-22 11:07:42 +09:00
Ryan Ernst e7393529b1 Merge branch 'master' into remove_index_template_filter 2016-08-19 21:14:12 -07:00
Ryan Ernst 1a7a9d3c62 Merge pull request #20071 from rjernst/pull_shards_allocator
Plugins: Switch custom ShardsAllocators to pull based model
2016-08-19 20:55:31 -07:00
Ryan Ernst 3a9055b55d Merge pull request #20073 from rjernst/deguice_indices_service
Deguice IndicesService
2016-08-19 20:47:07 -07:00
Lee Hinman d7e516c0b4 Default `include_in_all` for numeric-like types to false
This includes:

- All regular numeric types such as int, long, scaled-float, double, etc
- IP addresses
- Dates
- Geopoints and Geoshapes

Relates to #19784
2016-08-19 15:50:38 -06:00
Jason Tedor 6cda12871c Merge pull request #20083 from jasontedor/improve-startup-exception
Improve startup exception
2016-08-19 16:44:41 -04:00
Ali Beyad 1c9b64e09a Adds ignoreUnavailable option to the snapshot status API (#20066)
Adds ignoreUnavailable to the snapshot status API to be consistent
with the get snapshots API which has a similar parameter. If
ignoreUnavailable is set to true, then the snapshot status request
will ignore any snapshots that were not found in the repository,
instead of throwing a SnapshotMissingException.

Closes #18522
2016-08-19 16:19:56 -04:00
Jason Tedor c3849d9e7d Add print stack trace override to StartupException
StartupException overrides Throwable#printStackTrace(PrintStream) but
not Throwable#printStackTrace(PrintWriter). The former override is used
when the JVM terminates with an exception, but the latter override can
be used in some logging frameworks when rendering an exception (e.g.,
log4j). This commit adds an override for the latter, with the behavior
for the two overrides being the same.
2016-08-19 15:10:54 -04:00
Jason Tedor 3a6f7eb07a Rename StartupError to StartupException
This commit renames StartupError to StartupException. This rename is due
to the fact that this class inherits from Exception not Error in the
Throwable class hierarchy.
2016-08-19 14:53:08 -04:00
Ali Beyad cf32f8de34 Fixes tests so allocation ids in IndexMetaData is in sync with
what is in the RoutingTable
2016-08-19 14:42:02 -04:00
Jason Tedor 069fc22696 Remove minimum master nodes bootstrap check
This commit removes the minimum master nodes bootstrap check. The
motivation for this check was to raise awareness of the minimum master
nodes setting but this check gives a false sense of security because
it's too easy to set the setting to one when first standing up a cluster
and never update it when adding master-eligible nodes, or have it out of
sync on various nodes and still pass this check. Since this check does
not have the security that other bootstrap checks provide, it should be
removed in favor of a stronger guarantee in the future. We do log a
warning if an election occurs with minimum master nodes less than a
quorum of master-eligible nodes that participated in an election and
this is the best that we can do right now.

Relates #20082
2016-08-19 14:21:17 -04:00
Thiago Souza 9ea3f4ace3 Use supported random methods instead of DateTime.now() 2016-08-19 14:09:15 -03:00
Thiago Souza 2ba508a761 Use a better name for unit test method 2016-08-19 13:53:15 -03:00
Yannick Welsch 57c3dcb7d7 Merge pull request #20075 from ywelsch/fix/update-cs-with-routingresult
Some time ago, AllocationService.reroute was changed to not only return updates to the routing table but also to the metadata (which contain primary terms and in-sync allocation ids). A lot of test code still only updates the routing table though, which is fixed by this PR.
2016-08-19 18:18:30 +02:00
Yannick Welsch 771668f380 Use routingResult method to update cluster state after reroute
This ensures that the routing table as well as the metadata (with the primary terms and in-sync allocation ids) is updated.
2016-08-19 17:15:02 +02:00
Adrien Grand b586465a4c Make generics explicit to please ECJ. 2016-08-19 15:55:24 +02:00
Yannick Welsch a74f77b632 Check that all active shards have their allocation id in the in-sync set 2016-08-19 10:41:11 +02:00
Ryan Ernst 59636a0844 Internal: Deguice IndicesService
Almost all the dependencies of indices service are already created outside of
guice. This change deguices MetaStateService, and then IndicesService.
2016-08-19 00:27:37 -07:00
Adrien Grand a4ea7e7223 Switch indices.exists_type from `{index}/{type}` to `{index}/_mapping/{type}`. #20055
This will help remove types as we will need `{index}/{id}` to tell whether a
document exists.

Relates #15613
2016-08-19 09:18:24 +02:00
Ryan Ernst 207d3a60e7 Fix staging url for official plugins
This was incorrectly setup in #19996, without the version in the staging
build id.
2016-08-18 23:06:14 -07:00
Ryan Ernst 00c123b59f Plugins: Remove IndexTemplateFilter
How index templates match is currently controlled by the
IndexTemplateFilter interface. It is pluggable, to add additional
filter implementations to the default glob matcher.

This change removes the IndexTemplateFilter interface completely. This
is a very esoteric extension point, and not worth maintaining. Instead,
any improvements should be made to all of our glob matching.
2016-08-18 22:41:25 -07:00
Ryan Ernst ab404d90ed Plugins: Switch custom ShardsAllocators to pull based model
This change moves custom ShardsAllocators from registration on
ClusterModule, to implementing getShardsAllocators() in ClusterPlugin.
It also removes the legacy alias "even_shard" for the balanced allocator
which was removed in 2.0.
2016-08-18 22:18:33 -07:00
Thiago Souza 8281a3ce79 Merge pull request #20041 from tsouza/fix/issue-19142
Make exception message more descriptive
2016-08-18 17:31:16 -03:00
Ryan Ernst 165565a817 Merge pull request #20040 from rjernst/pull_allocation_deciders
Make custom allocation deciders use pull based extensions
2016-08-18 12:07:09 -07:00
Ryan Ernst 45144edd73 Fix cat allocation test line length violations 2016-08-18 10:51:59 -07:00
Adrien Grand 8f8ae8f577 Mapping updates on objects should propagate `include_an_all`. #20051
Today you can't update `include_an_all` on an existing object. The bug affects
2.x too.
2016-08-18 12:45:28 +02:00
Martijn van Groningen 825edd8dba tests for Script parsing and serialization 2016-08-18 12:19:43 +02:00
Adrien Grand d805266d94 Revert "Save one utf8 conversion in KeywordFieldMapper. #19867"
This reverts commit c44679d952.

Conflicts:
	core/src/main/java/org/elasticsearch/index/mapper/BaseGeoPointFieldMapper.java
	core/src/main/java/org/elasticsearch/index/mapper/GeoPointFieldMapperLegacy.java
	core/src/test/java/org/elasticsearch/index/mapper/GeoPointFieldMapperTests.java
2016-08-18 08:17:28 +02:00
Adrien Grand a7a7123d74 Simplify inclusion in `_all`. #20028
Currently, when you set `include_in_all` on an object, it will propagate the
information to its sub mappers immediately. This is annoying because this is
done using a different mechanism than regular mapping updates.

This PR changes object fields to propagate the information at document parsing
time rather than when `include_an_all` is updated. While moving this cost to
document parsing time rather than mapping update time is probably a bad
trade-off, I am confident that this cost is very low and think this new way
makes things simpler.
2016-08-18 08:13:55 +02:00
Thiago Souza d9bc2693a3 Use internal from/to when creating InternalDateRange.Bucket
InternalDateRange.Factory.createBucket should use
prototype's internal from/to

Fixes https://github.com/elastic/elasticsearch/issues/19364
2016-08-18 00:26:37 -03:00
Ryan Ernst 1ff348ed7f Plugins: Make custom allocation deciders use pull based extensions
This change converts AllocationDecider registration from push based on
ClusterModule to implementing with a new ClusterPlugin interface.
AllocationDecider instances are allowed to use only Settings and
ClusterSettings.
2016-08-17 15:55:31 -07:00
Thiago Souza 8e8614483b Make exception message more descriptive
Exception message should be more descriptive about what to do when
inner_hit names colides.

Fixes https://github.com/elastic/elasticsearch/issues/19142
2016-08-17 19:54:42 -03:00
Lee Hinman f6b166f19e Merge remote-tracking branch 'dakrone/forbid-simpleregex-in-index-name' 2016-08-17 16:01:09 -06:00
Lee Hinman 6030acb43b Disallow creating indices starting with '-' or '+'
Previously this was possible, which was problematic when issuing a
request like `DELETE /-myindex`, which was interpretted as "delete
everything except for myindex".

Resolves #19800
2016-08-17 15:13:03 -06:00
Areek Zillur fe5cdd30d5 Set created flag in index operation
Now document created flag is set in the index operation instead of
being returned from engine operation. This change makes the engine
index and delete operations have the same signature.
2016-08-17 17:09:34 -04:00
Ryan Ernst 2ea50bc162 Merge pull request #20018 from rjernst/split_disk_threshold
Internal: Split disk threshold monitoring from decider
2016-08-17 07:57:50 -07:00
Ryan Ernst efd8d837e8 Make disk threshold settings final 2016-08-17 07:58:27 -07:00
Yannick Welsch 27a760f9c1 Add routing changes API to RoutingAllocation (#19992)
Adds a class that records changes made to RoutingAllocation, so that at the end of the allocation round other values can be more easily derived based on these changes. Most notably, it:

- replaces the explicit boolean flag that is passed around everywhere to denote changes to the routing table. The boolean flag is automatically updated now when changes actually occur, preventing issues where it got out of sync with actual changes to the routing table.
- records actual changes made to RoutingNodes so that primary term and in-sync allocation ids, which are part of index metadata, can be efficiently updated just by looking at the shards that were actually changed.
2016-08-17 10:46:59 +02:00
Adrien Grand d894db1590 Only use `PUT` for index creation, not POST. #20001
Currently both `PUT` and `POST` can be used to create indices. This commit
removes support for `POST index_name` so that we can use it to index documents
with auto-generated ids once types are removed.

Relates #15613
2016-08-17 10:15:42 +02:00
Adrien Grand ffee9e8833 Automatically upgrade analyzed string fields that have `index_options` or `position_increment_gap` set. #20002
Closes #19974
2016-08-17 10:14:25 +02:00
Ryan Ernst b2c0f2d08f Internal: Split disk threshold monitoring from decider
In addition to be an allocation decider, DiskThresholdDecider also
monitors the used disk in order to trigger a reroute when the thresholds
are crossed. This change splits out the settings for disk thresholds
into DiskThresholdSettings, and moves the monitoring to a new
DiskThresholdMonitor.  DiskThresholdDecider is then in line with other
allocation deciders, needing only Settings and ClusterSettings for
construction, which will allow deguicing allocation deciders.
2016-08-17 00:22:16 -07:00
Lee Hinman 1825d8060c Merge remote-tracking branch 'dakrone/lockobtainfailed-replacement' 2016-08-16 14:41:27 -06:00
Lee Hinman 1de3388fa3 Switching LockObtainFailedException over to ShardLockObtainFailedException
`LobObtainFailedException` should be reserved for on-disk locks that
Lucene attempts (like `write.lock`). This switches our in-memory
semaphore locks for shards to use a different exception. Additionally,
ShardLockObtainFailedException no longer subclasses IOException, since
no IO is being done is this case.

Resolves #19978
2016-08-16 14:37:36 -06:00
Areek Zillur 75d4a9f6e4 Allow plugins to upgrade global custom metadata on startup
Currently plugins can not inspect or upgrade custom
meta data on startup. This commit allow plugins
to check and/or upgrade global custom meta data on startup.
Plugins can stop a node if any custom meta data is not supported.
2016-08-16 16:24:43 -04:00
Ryan Ernst 743d9fd008 Merge branch 'master' into search_parser 2016-08-16 11:28:59 -07:00
Ryan Ernst f716a86f40 Add comment about making parser members private instead of public 2016-08-16 11:25:34 -07:00
Nik Everett fdd50612ae Fix reindex under the transport client
The big change here is cleaning up the `TaskListResponse` so it doesn't
have a breaky `toString` implementation. That was causing the reindex
tests to break.

Also removed `NetworkModule#registerTaskStatus` which is part of the
Plugin API. Use `Plugin#getNamedWriteables` instead.
2016-08-16 12:15:15 -04:00
Ali Beyad 88aff40eef Primary shard allocator observes limits in forcing allocation (#19811)
Primary shard allocation observes limits in forcing allocation

Previously, during primary shards allocation of shards
with prior allocation IDs, if all nodes returned a
NO decision for allocation (e.g. the settings blocked
allocation on that node), we would chose one of those
nodes and force the primary shard to be allocated to it.

However, this meant that primary shard allocation
would not adhere to the decision of the MaxRetryAllocationDecider,
which would lead to attempting to allocate a shard
which has failed N number of times already (presumably
due to some configuration issue).

This commit solves this issue by introducing the
notion of force allocating a primary shard to a node
and each decider implementation must implement whether
this is allowed or not. In the case of MaxRetryAllocationDecider,
it just forwards the request to canAllocate.

Closes #19446
2016-08-16 11:25:45 -04:00
Nik Everett 46bf8baf2e Switch aggregation registration for push to pull
Adds `getAggregations` to `SearchPlugin` which can be used to register
aggregations.

Fixup MockNode which wasn't createing MockBigArrays.
2016-08-16 09:08:36 -04:00
Ryan Ernst 7fde410586 Internal: Consolidate search parser registries
Parsing a search request is currently split up among a number of
classes, using multiple public static methods, which take multiple
regstries of elements that may appear in the search request like query
parsers and aggregations. This change begins consolidating all this code
by collapsing the registries normally used for parsing search requests
into a single SearchRequestParsers class. It is also made available to
plugin services to enable templating of search requests.  Eventually all
of the actual parsing logic should move to the class, and the registries
should be hidden, but for now they are at least co-located to reduce the
number of objects that must be passed around.
2016-08-16 01:59:24 -07:00
Ryan Ernst 0996ae03a4 Merge pull request #19996 from rjernst/plugin_location
Plugins: Update official plugin location with unified release
2016-08-15 20:36:01 -07:00
Nik Everett 1452ab4b9f Squash the rest of o.e.rest.action
Squashes all the subpackages of `org.elasticsearch.rest.action` down to
the following:
* `o.e.rest.action.admin` - Administrative actions
* `o.e.rest.action.cat` - Actions that make tables for `grep`ing
* `o.e.rest.action.document` - Actions that act on documents
* `o.e.rest.action.ingest` - Actions that act on ingest pipelines
* `o.e.rest.action.search` - Actions that search

I'm tempted to merge `search` into `document` but the `document`
package feels fairly complete as is and `Suggest` isn't actually always
about documents either....

I'm also tempted to merge `ingest` into `admin.cluster` because the
latter contains the actions for dealing with stored scripts.

I've moved the `o.e.rest.action.support` into `o.e.rest.action`.

I've also added `package-info.java`s to all packges in `o.e.rest`. I
figure if the package is too small to deserve a `package-info.java` file
then it is too small to deserve to be a package....

Also fixes checkstyle in all moved classes.
2016-08-15 21:06:32 -04:00
chengpohi 2adc2a1971 Enable BoostingQuery with FVH highlighter (#19984)
* Enable BoostingQuery with FVH highlighter
* apply boost with negativeBoost
* flatten boosting query with its own boost and update boost query to a single layer
2016-08-15 21:00:16 -04:00
Nik Everett 4f262ce11e Clear some more static state in tests
This was causing CI build failures that didn't reproduce consistently
locally. Hopefully this will fix the error on CI.
2016-08-15 18:51:17 -04:00
Nik Everett eb9b84e6c3 Fix broken test
Randomized testing requires that we clean all the static state
in test classess.
2016-08-15 17:27:01 -04:00
Luca Cavanna 8804035205 Restore assignment of time value when deserializing a scroll instance (#19977)
* Assign scroll keepAlive when deserializing

The scroll time value was never assign when deserializing from the transport layer, meaning that it would always be null when received from another node, although the originating search request might have it set to some value.

* add tests for SearchRequest serialization and fail fast with illegal arguments

To ease testing, also introduced equals, hashcode and toString methods in SearchRequest and Scroll.

The serialization test brought up a few wrong assumptions about non null instance members, for which some null checks were needed to avoid NPEs when serializing.

* make Scroll implement Writeable rather than Streamable

* [TEST] add serialization test for ShardSearchTransportRequest

This also covers ShardSearchLocalRequest implicitly as most of the serialization code is in it.
2016-08-15 17:26:48 -04:00
Ryan Ernst fe5e99a408 Plugins: Update official plugin locaion with unified release
This change updates the url pattern for official plugins to be inline
with what the unified release will produce.
2016-08-15 13:24:11 -07:00
Ali Beyad 5ba06b6487 Removes support for adding aliases to analyzers. Indices created pre 5.x (#19994)
that have analyzer aliases in their analysis settings will still work, but
any attempts to create an alias for analyzers in newly created indices
will result in an IllegalArgumentException.

As a result, the setting `index.analysis.analyzer.{analyzerName}.alias` is
no longer supported.

Closes #18244
2016-08-15 16:17:58 -04:00
Igor Motov 10a766704e Rename Task Persistence into Storing Task Results
The term persisted task was used to indicate that a task should store its results upon its completion. We would like to use this term to indicate that a task can survive restart of nodes instead. This commit removes usages of the term "persist" when it means store results.
2016-08-15 10:02:43 -04:00
Jason Tedor d94e388904 Fix number of nodes in discovery disruption tests
This commit fixes the number of max local storage nodes setting used in
the discovery disruption tests. In some cases (randomly but rarely), the
acked indexing test can run with five nodes instead of three, breaching
the max local storage nodes configuration.
2016-08-14 21:03:05 -04:00
Nik Everett 153b2ae180 Checkstyle 2016-08-12 18:21:15 -04:00
Nik Everett cf6e1a4362 Move all FetchSubPhases to `o.e.search.fetch.subphase`
As the most complicated `FetchSubPhase` highlighting gets its own package
(`o.e.seach.fetch.subphase.highlight`. No other `FetchSubPhase`s get their
own package. Instead they all reside together in `o.e.search.fetch.subphase`.

Add package descriptions to `o.e.search.fetch` and subpackages.
2016-08-12 18:21:15 -04:00
Areek Zillur 40d7ebc515 Fix bug in single shard optimization when sorting documents in search request
This commit adds a function to shard-level query result to determine whether
there are any hits that needs fetching. Currently, a shard-level query result
can have hits when there are search hits and/or completion suggestion hits.
The newly added function encapsulates the checks to determine if a shard-level
query result has any fetchable hits, which is used in optimizing for sorting
documents and releasing search request contexts.
2016-08-12 17:32:22 -04:00
javanna efc32746eb fix typo getMovingAverageMdelParserRegistry->getMovingAverageModelParserRegistry in SearchModule 2016-08-12 20:33:06 +02:00
javanna 20e4fed65c fix javadocs for SearchExtensionSpec 2016-08-12 20:30:08 +02:00
Xiang Chen 77f28dbdde fix CompletionSuggestion test failed caused by shard is 1 2016-08-13 00:20:46 +08:00
Yannick Welsch 35e4f24467 Remove dead code that promotes replica relocation target to primary (#19973)
If a primary fails, an active replica is promoted to primary. Once we do the promotion, however, we are sure that the active replica is not relocating anymore. The reason is that when the primary fails, we first remove/cancel all initializing replicas (also if they are relocation targets). This is the only safe thing to do anyhow, because promoting relocating replica to primary would also mean that the replica recovery of the replica relocation target is suddenly promoted to primary relocation, which the recovery code treats in a different way.
2016-08-12 16:42:10 +02:00
Jun Ohtani 8d4bc0b2a8 Merge pull request #19929 from johtani/fix/stop_using_cached_components_in_analyze_api
Stop using cached component in _analyze API
2016-08-12 23:00:54 +09:00
Jim Ferenczi bf312f4203 Add the shard ID and the node name in the output of the search slow log.
This change outputs '[nodeName] [indexName][shardId]' instead of [indexName/indexUUID]

closes #19735
2016-08-12 15:32:40 +02:00
Jason Tedor 1f0673c9bd Default max local storage nodes to one
This commit defaults the max local storage nodes to one. The motivation
for this change is that a default value greather than one is dangerous
as users sometimes end up unknowingly starting a second node and start
thinking that they have encountered data loss.

Relates #19964
2016-08-12 09:26:20 -04:00
Jun Ohtani 2cde3b07cd Stop using cached component in _analyze API
Add javadoc some methods

Closes #19827
2016-08-12 21:54:45 +09:00
Jim Ferenczi b73751a4b5 Fix explain output for dfs query
ContextIndexSearcher#explain ignores the dfs data to create the normalized weight.
This change fixes this discrepancy by using the dfs data to create the normalized weight when needed.
2016-08-12 12:14:38 +02:00
Nik Everett 9f8f2ea54b Remove ESIntegTestCase#pluginList
It was a useful method in 1.7 when javac's type inference wasn't as
good, but now we can just replace it with `Arrays.asList`.
2016-08-11 15:44:02 -04:00
javanna 2f360ecc16 fix typo and make parseIndexConstraints method static in FieldStatsRequest 2016-08-11 20:29:27 +02:00
Ali Beyad 50b31ce620 Remove //norelease from IndexWithShadowReplicasIT test that checks
asserts the indices directory is deleted on index deletion, as we
are no longer considering it a blocker for releasing.

Relates #17695
2016-08-11 13:07:39 -04:00
Yannick Welsch 522b137097 Make NetworkPartition disruption scheme configurable (#19534)
This commit separates the description of the links in the network that are to be disrupted from the failure that is to be applied to the links (disconnect/unresponsive/delay). Previously we had subclasses for the various kind of network disruption schemes combining on one hand failure mode (disconnect/unresponsive/delay) as well as the network links to cut (two partitions / bridge partitioning) into a single class.
2016-08-11 14:55:06 +02:00
Yannick Welsch 4b33d8bb94 Mute test CompletionSuggestionTests.testToReduce
Relates to #19896
2016-08-11 14:46:12 +02:00
Jim Ferenczi 6130677a96 Merge pull request #19945 from jimferenczi/ttl_version_lookup
Remove useless PK lookup in IndicesTTLService
2016-08-11 14:19:03 +02:00
Jim Ferenczi 729f443199 Remove useless PK lookup in IndicesTTLService
This is a follow up of https://github.com/elastic/elasticsearch/pull/19944#issuecomment-239119859
Since the docid is known we can directly access the version doc value.
2016-08-11 12:30:22 +02:00
Jim Ferenczi 1f75d05a2a VersionFetchSubPhase should not use Versions#loadDocIdAndVersion
Since we already know the docId, the PK lookup is useless and we can directly get the value from the numeric doc values.
2016-08-11 11:39:01 +02:00
Yannick Welsch a1538de1a1 [TEST] Leave default ping timeouts on tests that don't simulate network failures
Reducing the ping timeouts on a test that does not simulate network failures can cause node disconnects within the test on a slow CI machine.
The test testSearchWithRelocationAndSlowClusterStateProcessing does not expect such disconnects, leading to shard relocation in the test to abort prematurely.
2016-08-11 11:05:38 +02:00
Jason Tedor c3253130d4 Mark halting the virtual machine as privileged
Today in the uncaught exception handler, we attempt to halt the virtual
machine on fatal errors. Yet, halting the virtual machine requires
privileges which might not be granted to the caller when the exception
is thrown for example from a scripting engine. This means that if an
OutOfMemoryError or another fatal error is hit inside a script, the
virtual machine will not exit because the halt call will be denied for
securiry privileges. In this commit, we mark this halt call as trusted
so that the virtual machine can be halted if a fatal error is
encountered in a script.

Relates #19923
2016-08-10 21:22:53 -04:00
Ryan Ernst 82fc86553c remove dots in field names tests for mapping api 2016-08-10 17:11:02 -07:00
Ryan Ernst 58c15f01b5 Merge branch 'master' into dots_in_mapper_names 2016-08-10 15:41:23 -07:00
Luca Cavanna 8a0d71924c Merge pull request #19926 from javanna/enhancement/threadcontext_cleanup
Reduce ThreadContext's inner classes visibility
2016-08-10 20:38:33 +02:00
Jun Ohtani f63fcefbd0 Stop using cached component in _analyze API
Stop calling tokenizer/tokenFilters/chaFilter method of IndexService
Add some getAnalysisProvider methods
Change SynonymTokenFilterFactory constructor

Closes #19827
2016-08-11 02:41:34 +09:00
Christoph Büscher 563bf0154c Merge pull request #19920 from cbuescher/remove-SuggestUtil
Remove SuggestUtil helper class
2016-08-10 19:22:22 +02:00
javanna ea6b7b46c9 reduce ThreadContext's inner classes visibility 2016-08-10 18:06:35 +02:00
Christoph Büscher d11521318d Renaming method according to review comments 2016-08-10 18:03:39 +02:00
Adrien Grand 0d6ac57acf Collapse o.e.index.mapper packages. #19921
I also reduced the visibility of a couple classes and renamed/consolidated some
test classes for consistency, eg. removing the `Simple` prefix or using the
`<Type>FieldMapperTests` convention for testing field mappers.
2016-08-10 17:51:11 +02:00
Christoph Büscher 9c91ced029 Removing use of ParseFields where we have alternative in other classes already 2016-08-10 16:20:34 +02:00
Christoph Büscher e6d57af0c5 Moving join() helper function to WordScorer 2016-08-10 16:20:33 +02:00
Christoph Büscher cdc77648a1 Move analysis helper methods to DirectCandidateGenerator 2016-08-10 16:20:29 +02:00
Christoph Büscher d6e16b6e74 Move getDirectSpellChecker to DirectSpellcheckerSettings 2016-08-10 16:06:05 +02:00
javanna a13dbc12e2 SuggestUtils#analyze: assign success variable a value 2016-08-10 12:57:24 +02:00
javanna a0e32e9dfe move SuggestUtils methods to their respective callers
These methods are called only once, they are then moved to the classes that call them, and become private.
2016-08-10 12:54:38 +02:00
javanna ae78394c03 Remove redundant generics type declaration 2016-08-10 12:28:06 +02:00
javanna 297b2d6739 remove unused methods from SuggestUtils
Parsing code was moved to the builder objects, these methods were left behind unused
2016-08-10 12:28:06 +02:00
javanna 2c44278ce8 [TEST] use ParseField instead of plain strings in query tests 2016-08-10 12:21:25 +02:00
javanna 0a98b5e56e [TEST] make AbstractQueryTestCase#testUnknownObjectException more accurate
testUnknownObjectException used to generate malformed json objects in some cases, due to the existence of arrays as it was not closing the injected object correctly. That is why the test was catching JsonParseException among the exception that are expected to be thrown. That is fixed by tracking where the new object is placed and placing its end object marker to the right level rather than always at the end.

Also introduced a mechanism to explicitly declare objects that won't cause any exception when they get additional objects injected, so that there is no need to override the method anymore as that caused copy pasting of the whole test method. This also makes sure that changes are reflected in tests, as those inner objects are not skipped but we actually check that what is declared is true (no exceptions get thrown when an additional object is added within them.
2016-08-10 11:48:51 +02:00
javanna f221b0ce52 [TEST] inner_hits is now parsed on the coord node, no need to skip such objects in testUnknownObjectException 2016-08-10 11:48:51 +02:00
javanna 57b90cb6ce rename local loop variable ingore->ignore 2016-08-10 10:17:54 +02:00
Adrien Grand 42725e9339 Fix expectations of GeoPointFieldMapperTests.
Closes #19895
2016-08-10 09:30:39 +02:00
Ryan Ernst 38d4382565 Mappings: Support dots in field names in mapping parsing
This change adds support for treating dots in field names found in
mappings as path separators, like was previously done for dynamic
mappings and document parsing.

closes #19443
2016-08-09 14:35:35 -07:00
Ryan Ernst 6efbe54255 Remove alpha5 bwc indexes
We don't have bwc indexes for alpha releases.
2016-08-09 13:25:16 -07:00
Ali Beyad 601602b364 Check restores in progress before deleting a snapshot (#19853)
Currently, when attempting to delete a snapshot, we check
if a snapshot is in progress before proceeding with the
delete. However, we do not check if a restore is taking
place before deleting. This can lead to concurrency issues
where a restore is in progress but the snapshotted files
for the restore are being deleted underneath.

This commit first checks if a restore is in progress and
if so, it prevents the deletion of a snapshot with an
exception.

Note that this is not a complete solution because it is
still possible that a restore of the same snapshot is
started after the deletion commenced but before the
deletion finished. But there is a much smaller window
for this to occur and this commit is a quick way to
check for the common case.
2016-08-09 15:07:09 -05:00
Areek Zillur 16d93e5a53 Merge pull request #19877 from areek/fix/remove_completion_payload
Remove payload option from completion suggester
2016-08-09 15:27:29 -04:00
David Pilato 90dbce9682 Merge branch 'fix/19772-toString' 2016-08-09 20:37:27 +02:00
Lee Hinman 5849c488b5 Merge remote-tracking branch 'dakrone/compliation-breaker' 2016-08-09 11:57:26 -06:00
David Pilato 8bc15039cd Fix after review 2016-08-09 19:44:42 +02:00
Clinton Gormley eac14f6e3d Bumped version to 5.0.0-alpha6 and added bwc indices for alpha5 2016-08-09 18:31:27 +02:00
Lee Hinman 2be52eff09 Circuit break the number of inline scripts compiled per minute
When compiling many dynamically changing scripts, parameterized
scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>)
should be preferred. This enforces a limit to the number of scripts that
can be compiled within a minute. A new dynamic setting is added -
`script.max_compilations_per_minute`, which defaults to 15.

If more dynamic scripts are sent, a user will get the following
exception:

```json
{
  "error" : {
    "root_cause" : [
      {
        "type" : "circuit_breaking_exception",
        "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
        "bytes_wanted" : 0,
        "bytes_limit" : 0
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "i",
        "node" : "a5V1eXcZRYiIk8lecjZ4Jw",
        "reason" : {
          "type" : "general_script_exception",
          "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
          "caused_by" : {
            "type" : "circuit_breaking_exception",
            "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
            "bytes_wanted" : 0,
            "bytes_limit" : 0
          }
        }
      }
    ],
    "caused_by" : {
      "type" : "general_script_exception",
      "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
      "caused_by" : {
        "type" : "circuit_breaking_exception",
        "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
        "bytes_wanted" : 0,
        "bytes_limit" : 0
      }
    }
  },
  "status" : 500
}
```

This also fixes a bug in `ScriptService` where requests being executed
concurrently on a single node could cause a script to be compiled
multiple times (many in the case of a powerful node with many shards)
due to no synchronization between checking the cache and compiling the
script. There is now synchronization so that a script being compiled
will only be compiled once regardless of the number of concurrent
searches on a node.

Relates to #19396
2016-08-09 10:26:27 -06:00
Yannick Welsch 6abcd42a05 Simplify RoutingNodes interface (#19870)
Slims the public interface of RoutingNodes down to 4 methods to update routing entries:
- initializeShard() -> initializes an unassigned shard
- startShard() -> starts an initializing shard / completes relocation of a shard
- relocateShard() -> starts relocation of a started shard
- failShard() -> fails/cancels an assigned shard

In the spirit of PR #19743, where deassociateDeadNodes was moved to its own public method to be only called when nodes have actually left the cluster and not on every reroute step, this commit also removes electPrimariesAndUnassignedDanglingReplicas from AllocationService and folds it into the shard failure logic. This means that an active replica is promoted to primary in the same method where the primary was failed. Previously we would scan in each reroute iteration for active replicas to be promoted to primary.
2016-08-09 17:07:13 +02:00
David Pilato 9b10bb7693 Fix toString method
See https://github.com/elastic/elasticsearch/pull/19773#issuecomment-238564524

Was introduced with #18939
2016-08-09 16:32:05 +02:00
David Pilato d28cc73046 Fix after merge 2016-08-09 12:34:52 +02:00
David Pilato 2a05030e22 Fix after merge 2016-08-09 12:14:50 +02:00
David Pilato 4d272cc9b2 Merge branch 'master' into fix/19772-toString
# Conflicts:
#	core/src/test/java/org/elasticsearch/action/admin/cluster/node/tasks/TransportTasksActionTests.java
2016-08-09 11:53:29 +02:00
Luca Cavanna af5fbcddfc Merge pull request #19871 from javanna/fix/short_query_multiple_fields
Throw exception when multiple field names are provided as part of query short syntax
2016-08-09 11:15:36 +02:00
Adrien Grand c44679d952 Save one utf8 conversion in KeywordFieldMapper. #19867
If a `keyword` field is both indexed and doc-valued, then we will convert the
input string to utf8 bytes twice: once for indexing/storing, and once for doc
values. This commit changes `keyword` fields to compute the utf8 representation
up-front and then feed both the inverted index and doc values with it.

Rather than adding version-based bw compat logic, I broke the `keyword` field
(they are now indexed/stored as a binary field rather than string), which is
fine since we are still on alpha releases for 5.0.
2016-08-09 10:06:30 +02:00
javanna f9a40344b2 Modify term query error when multiple fields are provided to comply with all other queries 2016-08-09 10:01:56 +02:00
javanna 0f54cb69ab Throw parsing error if span term query contains multiple fields in its short version 2016-08-09 09:53:03 +02:00
javanna d4db987825 Add common method that throws exception whenever multiple fields are provided in a query that support one field only
This makes sure that error messages are unified, and makes us save a few lines of code too.
2016-08-09 09:52:28 +02:00
javanna bbf40ca0cf [TEST] test that term query short syntax throws error when multiple fields are provided 2016-08-09 09:50:12 +02:00
Jason Tedor 1aba907ea2 Remove dead OOM handling in engine
Previously, the engine would catch an out of memory error and would try
to handle the error (it would try to fail the engine, and then it would
swallow the out of memory error). Catching the out of memory errors was
removed in 3343ceeae4 so this code path is
not effectively dead. This commit removes this dead code from the
engine.

Relates #19881
2016-08-08 21:59:49 -04:00
Areek Zillur d107141bf6 Remove payload option from completion suggester
The payload option was introduced with the new completion
suggester implementation in v5, as a stop gap solution
to return additional metadata with suggestions.

Now we can return associated documents with suggestions
(#19536) through fetch phase using stored field (_source).
The additional fetch phase ensures that we only fetch
the _source for the global top-N suggestions instead of
fetching _source of top results for each shard.
2016-08-08 16:04:06 -04:00
javanna f547886a9b [TEST] remove AwaitsFix that was fixed with #16615 2016-08-08 20:39:55 +02:00
javanna 9beb82b036 [TEST] remove unused argument from GeoPolygonQueryBuilderTests#randomPolygon 2016-08-08 20:39:55 +02:00
javanna 27a6983646 Throw parsing error if wildcard query contains multiple fields in its short version 2016-08-08 19:42:48 +02:00
javanna 796bc74163 Throw parsing error if regexp query contains multiple fields in its short version 2016-08-08 19:42:37 +02:00
javanna 8f485b3614 Throw parsing error if prefix query contains multiple fields in its short version 2016-08-08 19:42:26 +02:00
javanna 040f9c6be6 Throw parsing error if match query contains multiple fields in its short version 2016-08-08 19:42:14 +02:00
javanna d5316b2783 Throw parsing error if match phrase query contains multiple fields in its short version 2016-08-08 19:42:01 +02:00
javanna cb41f304f2 Throw parsing error if match phrase prefix query contains multiple fields in its short version 2016-08-08 19:41:45 +02:00
javanna 5d238e86f6 Throw parsing error if fuzzy query contains multiple fields in its short version 2016-08-08 19:40:54 +02:00
javanna 1db3c67e31 Throw parsing error if common terms query contains multiple fields in its short version 2016-08-08 19:40:23 +02:00
Colin Goodheart-Smithe bf0e42aaeb #19855 Throw exception when maxBounds greater than minBounds
Throw exception when maxBounds greater than minBounds
2016-08-08 13:17:25 +01:00
Colin Goodheart-Smithe 4735e0a9d3 Throw exception when maxBounds greater than minBounds
The recent changes to the Histogram Aggregator introduced a bug where
an exception would not be thrown if the maxBound of the extended bounds
is less that the minBound. This change fixes that bug.

Closes #19833
2016-08-08 12:09:43 +01:00
Yannick Welsch 180eff14dd Fix issue when relocation source and target routings are failed in same batch update
PR #19715 made AllocationService less lenient, requiring ShardRouting instances that are passed to its applyStartedShards and
applyFailedShards methods to exist in the routing table. As primary shard failures also fail initializing replica shards,
concurrent replica shard failures that are treated in the same cluster state update might not reference existing replica entries
in the routing table anymore. To solve this, PR #19715 ordered the failures by first handling replica before
primary failures. There are other failures that influence more than one routing entry, however. When we have a failed shard entry
for both a relocation source and target, then, depending on the order, either one or the other might point to an out-dated shard
entry. As finding a good order is more difficult than applying the failures, this commit re-adds parts of the ShardRouting
re-resolve logic so that the applyFailedShards method can properly treat shard failure batches.
2016-08-08 11:46:48 +02:00
Nicholas Knize ab0a0cd4d4 fix rogue license header 2016-08-05 23:21:16 -05:00
Nicholas Knize 2d590af593 Deprecate GeoDistance enumerators and remove geo distance script helpers
GeoDistance is implemented using a crazy enum that causes issues with the scripting modules. This commit moves all distance calculations to arcDistance and planeDistance static methods in GeoUtils. It also removes unnecessary distance helper methods from ScriptDocValues.GeoPoints.
2016-08-05 18:42:06 -05:00
Areek Zillur 469eb2546d Merge pull request #19536 from areek/enhancement/completion_suggester_documents
Add support for returning documents with completion suggester
2016-08-05 18:55:08 -04:00
Areek Zillur fee013c07c Add support for returning documents with completion suggester
This commit enables completion suggester to return documents
associated with suggestions. Now the document source is returned
with every suggestion, which respects source filtering options.

In case of suggest queries spanning more than one shard, the
suggest is executed in two phases, where the last phase fetches
the relevant documents from shards, implying executing suggest
requests against a single shard is more performant due to the
document fetch overhead when the suggest spans multiple shards.
2016-08-05 17:51:45 -04:00
Christoph Büscher fbbb633d81 Merge pull request #19825 from cbuescher/register-namedWritables-transportClient
Add NamedWriteables from plugins to TransportClient
2016-08-05 22:51:04 +02:00
Christoph Büscher 6ccb70e1ab Avoid using injector and more test to TransportClientTests 2016-08-05 21:39:44 +02:00
Christoph Büscher 37c433aace Merge pull request #19837
Ensure PutMappingRequest.buildFromSimplifiedDef input are pairs
2016-08-05 20:31:49 +02:00
Christoph Büscher e57f76aa2d Ensure PutMappingRequest.buildFromSimplifiedDef fails when input isn't pairs
The method requires pairs of fieldnames and property arguments and will fail if
the varargs input is an uneven number. We should check this and fail with an
appropriate IllegalArgumentException instead.
2016-08-05 19:25:20 +02:00
Britta Weber 981478e4a9 mute test 2016-08-05 19:10:13 +02:00
Britta Weber 899cddefb6 make ctors protected (#19831)
This is useful if we need an acknowledged instance in a test
2016-08-05 17:13:26 +02:00
Nik Everett 8bebf2599e Add note explaining analysis caching for plugins
```
Elasticsearch doesn't have any automatic mechanism to share these
components between indexes. If any component is heavy enough to
warrant such sharing then it is the Pugin's responsibility to do
it in their {@link AnalysisProvider} implementation. We recommend
against doing this unless absolutely necessary because it can be
difficult to get the caching right given things like behavior
changes across versions.
```

Closes #19814
2016-08-05 11:11:53 -04:00
Christoph Büscher e162935656 Add test to check that plugin NamedWriteables are registerd with TransportClient 2016-08-05 17:08:59 +02:00
Luca Cavanna 4c1a3b9a53 Merge pull request #19791 from javanna/fix/multiple_fields_queries
Query parsers to throw exception when multiple field names are provided
2016-08-05 15:53:35 +02:00
Ali Beyad f59ca9083b Snapshot repository cleans up empty index folders (#19751)
This commit cleans up indices in a snapshot repository when all
snapshots containing the index are all deleted. Previously, empty
indices folders would lay around after all snapshots containing
them were deleted.
2016-08-05 09:39:02 -04:00
Adrien Grand 284b9794c0 Do not parse the created version from the settings every time a field is parsed. #19824
I found it while looking at some jfr telemetry reports from Rally.
2016-08-05 15:35:53 +02:00
Christoph Büscher c32a4324b0 Add NamedWriteables from plugins to TransportClient
Plugins provide NamedWriteables that are added to the
NamedWriteableRegistry. Those are added on Nodes already, the same mechanism is
added to the setup for TransportClient.
2016-08-05 14:11:01 +02:00
javanna 7f0bd56094 [TEST] use expectThrows wherever possible in query builder unit tests 2016-08-05 13:55:18 +02:00
Tanguy Leroux 841d5a210e Update to Jackson 2.8.1
This commit updates Jackson to the 2.8.1 version, which is more strict when it comes to build objects. It also adds the snakeyaml dependency that was previously shaded in jackson libs.

It also closes #18076
2016-08-05 12:26:06 +02:00
javanna 6a5c44a271 fix line length in FuzzyQueryBuilder 2016-08-05 10:58:19 +02:00
javanna 0ac7dd6137 Make query parsing stricter by requiring each parser to stop at END_OBJECT token
Instead of being lenient in QueryParseContext#parseInnerQueryBuilder we check that the token where the parser stopped reading was END_OBJECT, and throw error otherwise. This is a best effort to verify that the parsers read a whole object rather than stepping out in the middle of it due to malformed queries.
2016-08-05 10:58:19 +02:00
javanna 43fee1d7fa Throw parsing error if fuzzy query contains multiple fields
Fuzzy Query, like many other queries, used to parse even when the query referred to multiple fields and the first one would win. We rather throw an exception now instead.
Also added test for short prefix query variant and modified the parsing code to consume the whole query object.
2016-08-05 10:58:19 +02:00
javanna 6d228bb09c [TEST] test that term query throws error when made against multiple fields 2016-08-05 10:58:19 +02:00
javanna 389bd06846 [TEST] check validation error messages in AbstractTermQueryTestCase 2016-08-05 10:58:19 +02:00
javanna 1bcf0722c4 Throw parsing error if span_term query contains multiple fields
Span term Query, like many other queries, used to parse even when the query referred to multiple fields and the first one would win. We rather throw an exception now instead.
Also modified the parsing code to consume the whole query object.
2016-08-05 10:58:19 +02:00
javanna c3dfe0846c Throw parsing error if common terms query contains multiple fields
Common Terms Query, like many other queries, used to parse even when the query referred to multiple fields and the first one would win. We rather throw an exception now instead.
Also added test for short prefix query variant and modified the parsing code to consume the whole query object.
2016-08-05 10:58:19 +02:00
javanna 1e45fd5850 Throw parsing error if match query contains multiple fields
Match Query, like many other queries, used to parse even when the query referred to multiple fields and the first one would win. We rather throw an exception now instead.
Also added test for short prefix query variant and modified the parsing code to consume the whole query object.
2016-08-05 10:58:19 +02:00
javanna f7b3dce4bc Throw parsing error if match_phrase_prefix query contains multiple fields
Match phrase prefix Query, like many other queries, used to parse even when the query referred to multiple fields and the first one would win. We rather throw an exception now instead.
Also added test for short prefix query variant and modified the parsing code to consume the whole query object.
2016-08-05 10:58:19 +02:00
javanna ad8f5e7e4b Throw parsing error if geo_distance query contains multiple fields
Geo distance Query, like many other queries, used to parse even when the query referred to multiple fields and the last one would win. We rather throw an exception now instead.
2016-08-05 10:58:19 +02:00
javanna 195320f2d6 [TEST] check validation error messages in IdsQueryBuilderTests 2016-08-05 10:58:19 +02:00
javanna f56333048a Throw parsing error if match_phrase query contains multiple fields
Match phrase Query, like many other queries, used to parse even when the query referred to multiple fields and the first one would win. We rather throw an exception now instead.
Also added test for short prefix query variant and modified the parsing code to consume the whole query object.
2016-08-05 10:58:19 +02:00
javanna 51ea913248 Throw parsing error if wildcard query contains multiple fields
Wildcard Query, like many other queries, used to parse even when the query referred to multiple fields and the first one would win. We rather throw an exception now instead.
Also added test for short prefix query variant and modified the parsing code to consume the whole query object.
2016-08-05 10:58:19 +02:00
javanna 003a7b6eb3 Throw parsing error if regexp query contains multiple fields
Regexp Query, like many other queries, used to parse even when the query referred to multiple fields and the last one would win. We rather throw an exception now instead.
Also added test for short prefix query variant.
2016-08-05 10:58:19 +02:00
javanna 69c2deedc7 Throw parsing error if prefix query contains multiple fields
Prefix Query, like many other queries, used to parse when the query refers to multiple fields and the last one would win. We rather throw an exception now instead.
Also added tests for short prefix quer variant.
2016-08-05 10:58:19 +02:00
javanna 11e4b0168b Throw parsing error if range query contains multiple fields
Range Query, like many other queries, used to parse when the query refers to multiple fields and the last one would win. We rather throw an exception now instead.

Closes #19547
2016-08-05 10:58:19 +02:00
Colin Goodheart-Smithe a01475a20b #19781 Refactored Rounding simplify Date Histogram code
Refactored Rounding simplify Date Histogram code
2016-08-05 09:28:38 +01:00
Boaz Leskes 609a199bd4 Upon being elected as master, prefer joins' node info to existing cluster state (#19743)
When we introduces [persistent node ids](https://github.com/elastic/elasticsearch/pull/19140) we were concerned that people may copy data folders from one to another resulting in two nodes competing for the same id in the cluster. To solve this we elected to not allow an incoming join if a different with same id already exists in the cluster, or if some other node already has the same transport address as the incoming join. The rationeel there was that it is better to prefer existing nodes and that we can rely on node fault detection to remove any node from the cluster that isn't correct any more, making room for the node that wants to join (and will keep trying).

Sadly there were two problems with this:
1) One minor and easy to fix - we didn't allow for the case where the existing node can have the same network address as the incoming one, but have a different ephemeral id (after node restart). This confused the logic in `AllocationService`, in this rare cases. The cluster is good enough to detect this and recover later on, but it's not clean.
2) The assumption that Node Fault Detection will clean up is *wrong* when the node just won an election (it wasn't master before) and needs to process the incoming joins in order to commit the cluster state and assume it's mastership. In those cases, the Node Fault Detection isn't active. 

This PR fixes these two and prefers incoming nodes to existing node when finishing an election. 
On top of the, on request by @ywelsch , `AllocationService` synchronization between the nodes of the cluster and it's routing table is now explicit rather than something we do all the time. The same goes for promotion of replicas to primaries.
2016-08-05 08:58:03 +02:00
Jason Tedor 3f6a3c01da Merge pull request #19803 from elastic/fix/transportClientTests
Fix PreBuiltTransportClientTests to run and pass
2016-08-04 16:55:08 -04:00
Simon Willnauer e08f11dabc Remove BWC serialization logic for pre 2.2 nodes (#19810)
This change removes all pre 2.2 logic from InternalSearchResponse serialization. It's unneeded in 5.0 since we require full cluster restart
2016-08-04 22:47:39 +02:00
Daniel Mitterdorfer 4598c36027 Fix various concurrency issues in transport (#19675)
Due to various issues (most notably a missing happens-before edge
between socket accept and channel close in MockTcpTransport),
MockTcpTransportTests sometimes did not terminate.

With this commit we fix various concurrency issues that led to
this hanging test.

Failing example build: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=oraclelinux/835/console
2016-08-04 21:00:59 +02:00
Boaz Leskes 7010082112 Add checksumming and versions to the Translog's Checkpoint files (#19797)
This prepares the infrastructure to be able to extend the checkpoint file to store more information.
2016-08-04 20:42:12 +02:00
javanna cd9388ce66 [TEST] parse query alternate versions in strict mode
AbstractQueryTestCase parses the main version of the query in strict mode, meaning that it will fail if any deprecated syntax is used. It should do the same for alternate versions (e.g. short versions). This is the way it is because the two alternate versions for ids query are both deprecated. Moved testing for those to a specific test method that isolates the deprecations and actually tests that the two are deprecated.
2016-08-04 19:49:43 +02:00
Colin Goodheart-Smithe b6ef99195d Remove offset rounding
This is in favour of doing the offset calculations in the date histogram
2016-08-04 16:24:19 +01:00
Colin Goodheart-Smithe c14155e4a8 Remove TimeZoneRounding abstraction
Because the Rounding class now only deals with date based rounding of
values we can remove the TimeZoneRounding abstraction to simplify the
code.
2016-08-04 16:24:19 +01:00
Colin Goodheart-Smithe 5ab5cc69b8 Remove unused rounding code
Factor rounding and Interval rounding (the non-date based rounding)
was no longer used so it has been removed. Offset rounding has been
retained for no since both date based rounding classes rely on it
2016-08-04 16:24:19 +01:00
Ali Beyad 34bb150863 [TEST] Fixes primary term in TransportReplicationActionTests#testReplicaProxy 2016-08-04 10:18:48 -04:00
Colin Goodheart-Smithe b0730bb214 Fix PreBuiltTransportClientTests to run and pass
This change does three things:

1. Makes PreBuiltTransportClientTests run since it was silently
failing on a missing dependency
2. Makes PreBuiltTransportClientTests pass
3. Removes the http.type and transport.type from being set in the
transport clients additional settings since these are set to `netty4` by
default anyway.
2016-08-04 14:15:28 +01:00
Ali Beyad 8bbc312fdd Fixes issue with dangling index being deleted instead of re-imported (#19666)
Fixes an issue where a node that receives a cluster state
update with a brand new cluster UUID but without an
initial persistence block could cause indices to be wiped out,
preventing them from being reimported as dangling indices.
This commit only removes the in-memory data structures and
thus, are subsequently reimported as dangling indices.
2016-08-04 08:47:46 -04:00
Yannick Welsch ede78ad231 Use primary terms as authority to fail shards (#19715)
A primary shard currently instructs the master to fail a replica shard that it fails to replicate writes to before acknowledging the writes to the client. To ensure that the primary instructing the master to fail the replica is still the current primary in the cluster state on the master, it submits not only the identity of the replica shard to fail to the master but also its own shard identity. This can be problematic however when the primary is relocating. After primary relocation handoff but before the primary relocation target is activated, the primary relocation target is replicating writes through the authority of the primary relocation source. This means that the primary relocation target should probably send the identity of the primary relocation source as authority. However, this is not good enough either, as primary shard activation and shard failure instructions can arrive out-of-order. This means that the relocation target would have to send both relocation source and target identity as authority. Fortunately, there is another concept in the cluster state that represents this joint authority, namely primary terms. The primary term is only increased on initial assignment or when a replica is promoted. It stays the same however when a primary relocates.

This commit changes ShardStateAction to rely on primary terms for shard authority. It also changes the wire format to only transmit ShardId and allocation id of the shard to fail (instead of the full ShardRouting), so that the same action can be used in a subsequent PR to remove allocation ids from the active allocation set for which there exist no ShardRouting in the cluster anymore. Last but not least, this commit also makes AllocationService less lenient, requiring ShardRouting instances that are passed to its applyStartedShards and applyFailedShards methods to exist in the routing table. ShardStateAction, which is calling these methods, now has the responsibility to resolve the ShardRouting objects that are to be started / failed, and remove duplicates.
2016-08-04 12:00:37 +02:00
Boaz Leskes d327dd46b1 Recovery: don't log an error when listing an empty folder 2016-08-04 10:23:36 +02:00
Jason Tedor 533412e36f Improve cat thread pool API
Today, when listing thread pools via the cat thread pool API, thread
pools are listed in a column-delimited format. This is unfriendly to
command-line tools, and inconsistent with other cat APIs. Instead,
thread pools should be listed in a row-delimited format.

Additionally, the cat thread pool API is limited to a fixed list of
thread pools that excludes certain built-in thread pools as well as all
custom thread pools. These thread pools should be available via the cat
thread pool API.

This commit improves the cat thread pool API by listing all thread pools
(built-in or custom), and by listing them in a row-delimited
format. Finally, for each node, the output thread pools are sorted by
thread pool name.

Relates #19721
2016-08-03 23:02:13 -04:00
David Pilato 54603903f3 Remove ListTasksResponse#setDiscoveryNodes 2016-08-04 02:02:51 +02:00
Ali Beyad be87d50f32 Fixes CreateIndexIT test that assumes an index create propogated
before calling delete.
2016-08-03 16:24:24 -04:00
Ryan Ernst c3a5e4fa48 Merge pull request #19765 from rjernst/metadata_mapper_dup
Mappings: Fix detection of metadata fields in documents
2016-08-03 11:58:24 -07:00
Ryan Ernst ef425f4b7c Merge pull request #19770 from rjernst/script_service_component
Add ScriptService to dependencies available for plugin components
2016-08-03 11:57:58 -07:00
javanna 4805250ecf Throw ParsingException if a query is wrapped in an array
Our parsing code accepted up until now queries in the following form (note that the query starts with `[`:

```
{
    "bool" : [
        {
          "must" : []
        }
    ]
}
```

This would lead to a null pointer exception as most parsers assume that the field name ("must" in this example) is the first thing that can be found in a query if its json is valid, hence always non null while parsing. Truth is that the additional array layer doesn't make the json invalid, hence the following code fragment would cause NPE within ParseField, because null gets passed to `parseContext.isDeprecatedSetting`:

```
if (token == XContentParser.Token.FIELD_NAME) {
    currentFieldName = parser.currentName();
} else if (parseContext.isDeprecatedSetting(currentFieldName)) {
    // skip
} else if (token == XContentParser.Token.START_OBJECT) {
```

We could add null checks in each of our parsers in lots of places, but we rely on `currentFieldName` being non null in all of our parsers, and we should consider it a bug when these unexpected situations are not caught explicitly. It would be best to find a way to prevent such queries altogether without changing all of our parsers.

The reason why such a query goes through is that we've been allowing a query to start with either `[` or `{`. The only reason I found is that we accept `match_all : []`. This seems like an undocumented corner case that we could drop support for. Then we can be stricter and accept only `{` as start token of a query. That way the only next token that the parser can encounter if the json is valid (otherwise the json parser would barf earlier) is actually a field_name, hence the assumption that all our parser makes hold.

The downside of this is simply dropping support for `match_all : []`

Relates to #12887
2016-08-03 17:05:14 +02:00
javanna 51bbe2c5c4 [TEST] fix log statement in ESIndexLevelReplicationTestCase 2016-08-03 16:56:19 +02:00
Clinton Gormley 39081af9d6 Added version 2.3.5 with bwc indices 2016-08-03 15:50:47 +02:00
David Pilato a1633d6444 ListTasksResponse#toString() should not group by nodes
We just overwrite `toString()` method so it calls toXContent
with `group_by` = "whatever" so we don't try to group by nodes
which does not make sense in a toString() method.

We keep the old behavior for `toXContent()` method which
means that there is no impact in the REST layer but
only in logs and tests (where we call `toString()`).

Closes #19772.
2016-08-03 14:56:09 +02:00
Robert Muir ef5debc6ce Merge pull request #19754 from rmuir/docker_seccomp
ignore some docker craziness in seccomp environment checks
2016-08-03 05:50:25 -04:00
Britta Weber abcb4c8a97 [Test] move methods from bwc test to test package for use in plugins (#19738)
* [Test] move methods from bwc test to test package for use in other plugins
2016-08-03 11:41:46 +02:00
Adrien Grand 0e64117512 package-info.java should be in src/main only. 2016-08-03 11:11:25 +02:00
Ryan Ernst 18f242b069 Merge pull request #19764 from rjernst/writeable_registry
Make NamedWriteableRegistry immutable and add extension point for named writeables
2016-08-03 01:36:38 -07:00
Ryan Ernst fe823c857b Plugins: Add ScriptService to dependencies available for plugin components 2016-08-03 00:43:04 -07:00
Adrien Grand a0818d3b87 Split regular histograms from date histograms. #19551
Currently both aggregations really share the same implementation. This commit
splits the implementations so that regular histograms can support decimal
intervals/offsets and compute correct buckets for negative decimal values.

However the response API is still the same. So for intance both regular
histograms and date histograms will produce an
`org.elasticsearch.search.aggregations.bucket.histogram.Histogram`
aggregation.

The optimization to compute an identifier of the rounded value and the
rounded value itself has been removed since it was only used by regular
histograms, which now do the rounding themselves instead of relying on the
Rounding abstraction.

Closes #8082
Closes #4847
2016-08-03 08:39:48 +02:00
Boaz Leskes f6aeb35ce8 Tighten up concurrent store metadata listing and engine writes (#19684)
In several places in our code we need to get a consistent list of files + metadata of the current index. We currently have a couple of ways to do in the `Store` class, which also does the right things and tries to verify the integrity of the smaller files. Sadly, those methods can run into trouble if anyone writes into the folder while they are busy. Most notably, the index shard's engine decides to commit half way and remove a `segment_N` file before the store got to checksum (but did already list it). This race condition typically doesn't happen as almost all of the places where we list files also happen to be places where the relevant shard doesn't yet have an engine. There  is however an exception (of course :)) which is the API to list shard stores, used by the master when it is looking for shard copies to assign to.

I already took one shot at fixing this in #19416 , but it turns out not to be enough - see for example https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=sles/822.

The first inclination to fix this was to add more locking to the different Store methods and acquire the `IndexWriter` lock, thus preventing any engine for accessing if if the a shard is offline and use the current index commit snapshotting logic already existing in `IndexShard` for when the engine is started. That turned out to be a bad idea as we create more subtleties where, for example, a store listing can prevent a shard from starting up (the writer lock doesn't wait if it can't get access, but fails immediately, which is good). Another example is running on a shared directory where some other engine may actually hold the lock.

Instead I decided to take another approach:
1) Remove all the various methods on store and keep one, which accepts an index commit (which can be null) and also clearly communicates that the *caller* is responsible for concurrent access. This also tightens up the API which is a plus.
2) Add a `snapshotStore` method to IndexShard that takes care of all the concurrency aspects with the engine, which is now possible because it's all in the same place. It's still a bit ugly but at least it's all in one place and we can evaluate how to improve on this later on. I also renamed the  `snapshotIndex` method to `acquireIndexCommit` to avoid confusion and I think it communicates better what it does.
2016-08-03 08:34:09 +02:00
Ryan Ernst 7bfe1bd628 Check inner field with metadata field name is ok 2016-08-02 17:03:21 -07:00
Ryan Ernst 4e48154130 Mappings: Fix detection of metadata fields in documents
In 2.0, the ability to specify metadata fields like _routing and _ttl
inside a document was removed. However, the ability to break through
this restriction has lingered, and the check that enforced it is
completely broken.

This change fixes the check, and adds a parsing test.
2016-08-02 16:54:44 -07:00
Ryan Ernst df8dc64e9b Plugins: Make NamedWriteableRegistry immutable and add extenion point for named writeables
Currently any code that wants to added NamedWriteables to the
NamedWriteableRegistry can do so via guice injection of the registry,
and registering at construction time. However, this makes the registry
complex: it has both get and register methods synchronized, and there is
likely contention on the read side from multiple threads.  The
registration has mostly already been contained to guice modules at node
construction time.

This change makes the registry immutable, taking all of the
NamedWriteable readers at construction time. It also allows plugins to
added arbitrary named writables that it may use in its own transport
actions.
2016-08-02 15:56:25 -07:00
Lee Hinman a9b2e172fa [TEST] Increase time waiting for all shards to move off/on to a node 2016-08-02 16:18:39 -06:00
Ali Beyad c28eee77df Fixes the active shard count check in the case of (#19760)
ActiveShardCount.ALL by checking for active shards,
not just started shards, as a shard could be active
but in the relocating state (i.e. not in the started
state).
2016-08-02 18:00:39 -04:00
Igor Motov 22e63b4783 Fixes cat tasks operation in detailed mode
Currently the cat tasks operation fails in the detailed mode.

Closes #19755
2016-08-02 15:21:31 -04:00
Robert Muir f77e8a512c ignore some docker craziness in scccomp environment checks 2016-08-02 12:19:38 -04:00
Ali Beyad c4ae23f5d8 Enables implementations of the BlobContainer interface to (#19749)
conform with the requirements of the writeBlob method by
throwing a FileAlreadyExistsException if attempting to write
to a blob that already exists. This change means implementations
of BlobContainer should never overwrite blobs - to overwrite a
blob, it must first be deleted and then can be written again.

Closes #15579
2016-08-02 09:48:21 -04:00
Nik Everett 42fe2f0aca Add docs for a few packages
This'll make javadocs slightly more useful....
2016-08-02 09:30:30 -04:00
Ali Beyad 456ea56527 Cleans up the BlobContainer interface by removing the (#19727)
writeBlob method takes a BytesReference in favor of just
the writeBlob method that takes an InputStream.

Closes #18528
2016-08-02 09:21:43 -04:00
Ali Beyad 3d2a105825 Merge pull request #19454 from abeyad/remove-write-consistency-level
Removes write consistency level across replication action APIs in favor of wait_for_active_shards
2016-08-02 09:01:11 -04:00
Daniel Mitterdorfer 419e9e090e Document and enforce cancellation policy of CancellableThreads (#19712)
With this commit we add documentation and additional checks to
enforce the cancellation policy of CancellableThreads (which is
disallow `Thread#interrupt()` on any of the threads managed by
it).
2016-08-02 08:46:38 +02:00
Ali Beyad 4923da93c8 Refactors wait_for_active_shards index settings tests 2016-08-01 19:14:37 -04:00
Lee Hinman f9fd64fc78 Revert to older exception message
If the uuidBytes and ref are converted to utf8, it's possible they can
trip an assertion related to valid UTF-8/UTF-16 ranges, so display them
as hex, not as strings.
2016-08-01 11:51:39 -06:00
Ali Beyad 6a7d005081 Makes the index.write.wait_for_active_shards setting index-level and
dynamically updatable for both index creation and write operations.
2016-08-01 13:37:05 -04:00
Ali Beyad 4a51ea8c8e Before, transport replication actions implemented a checkWriteConsistency()
method to determine if a write consistency check should be performed
before proceeding with the action.  This commit removes this method from
the transport replication actions in favor of setting the ActiveShardCount
on the request, with setting the value to ActiveShardCount.NONE if the
transport action's checkWriteConsistency() method returned false.
2016-08-01 13:35:30 -04:00
Ali Beyad d93f7d6085 Refactors ActiveShardCount 2016-08-01 13:35:29 -04:00
Ali Beyad 25d8eca62d Removes the notion of write consistency level across all APIs in
favor of waiting for active shard copy count (wait_for_active_shards).
2016-08-01 13:35:29 -04:00
Ali Beyad 9f88a8194a Merge pull request #19706 from elastic/enhancement/snapshot-blob-handling
More resilient blob handling in snapshot repositories
2016-08-01 12:03:53 -04:00
Tanguy Leroux 386902903e [TEST] Kill remaining lang-groovy messy tests
After #13834 many tests that used Groovy scripts (for good or bad reason) in their tests have been moved in the lang-groovy module and the issue #13837 has been created to track these messy tests in order to clean them up.

The work started with #19280, #19302 and #19336 and this PR moves the remaining messy tests back in core, removes the dependency on Groovy, changes the scripts in order to use the mocked script engine, and change the tests to integration tests.

It also moves IndexLookupIT test back (even if it has good chance to be removed soon) and fixes its tests.

It also changes AbstractQueryTestCase to use custom script plugins in tests.

closes #13837
2016-08-01 16:59:47 +02:00
Alexander Lin 9ac6389e43 Rename operation to result and reworking responses
* Rename operation to result and reworking responses
* Rename DocWriteResponse.Operation enum to DocWriteResponse.Result

These are just easier to interpret names.

Closes #19664
2016-08-01 10:42:58 -04:00
Nik Everett 12fd4ed8f8 Add description to org.elasticsearch.tasks package (#19700)
Yet more readable docs!
2016-08-01 07:43:32 -04:00
Nik Everett aefc36bfaa Add descriptions for o.e.search.suggest packages (#19699)
Let's have readable javadoc!
2016-08-01 07:43:13 -04:00
Boaz Leskes 7c6527ed09 make election stop not be a failure (#19705)
During our master elections, nodes "vote" for a master being issuing a join request to it. Since this is done in an async fashion, joins may arrive before the master itself has realized it had won the election. Therefore we start accumulating node joins on every node at election start (we don't know the result yet). When the election finish nodes that did not become the master (i.e., joined another node which won the election) need to potentially process and fail any incoming join request they may have received during the election. This is currently achieved by always issuing a cluster state update task that is doomed to fail, even if no pending joins are actually there. That aspect results in confusing (debug) log messages, making it seems like something is wrong. For example (note that `NotMasterException`)

```
[2016-07-30 22:25:53,040][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-process-pending-joins [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400} elected]]: execute
[2016-07-30 22:25:53,041][DEBUG][transport                ] [node_t1] connected to node [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400}]
[2016-07-30 22:25:53,045][DEBUG][cluster.service          ] [node_t1] cluster state update task [zen-disco-process-pending-joins [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400} elected]] failed
NotMasterException[Node [{node_t1}{eAQts270TiGFpoCDE-0PQQ}{or5bsv2ET220su78DLJk5g}{127.0.0.1}{127.0.0.1:9401}] not master for join request]
[2016-07-30 22:25:53,048][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-process-pending-joins [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400} elected]]: took [7ms] no change in cluster_state
```

This commit cleans up  the logic a bit to only use failure where there are actual joins that are failed. The result is cleaner logs as well:

```
[2016-07-30 22:23:12,880][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-election-stop [{node_t0}{jMR5HCpOQnOM4pGeFkUjng}{B5WIZQAdQk2cWbjGZ21mvQ}{127.0.0.1}{127.0.0.1:9400} elected]]: execute
[2016-07-30 22:23:12,881][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-election-stop [{node_t0}{jMR5HCpOQnOM4pGeFkUjng}{B5WIZQAdQk2cWbjGZ21mvQ}{127.0.0.1}{127.0.0.1:9400} elected]]: took [0s] no change in cluster_state
[2016-07-30 22:23:12,881][DEBUG][transport                ] [node_t1] connected to node [{node_t0}{jMR5HCpOQnOM4pGeFkUjng}{B5WIZQAdQk2cWbjGZ21mvQ}{127.0.0.1}{127.0.0.1:9400}]
```
2016-08-01 13:08:50 +02:00
Tanguy Leroux 737db98bd7 /_cat/shards should support wilcards for indices
closes #19634
2016-08-01 11:09:48 +02:00
Christoph Büscher 87a4995bed Merge pull request #19665 from cbuescher/missing-field-MultiMatchQuery
`multi_match` query should produce MatchNoDocs query on unknown field
2016-08-01 10:59:52 +02:00
Tanguy Leroux 7d4f557aa3 Allow routing table to be filtered by index pattern
Before this commit when an index pattern is used to filter the cluster state, only indices metadata are populated and routing table is just empty. This commit aligns the behavior of the filtering of cluster state's routing table with the filtering of cluster state's metadata so that coherent data are returned for both routing table & metadata when index pattern is requested.
2016-08-01 09:22:12 +02:00
chengpohi 8aa1eb6aa4 Fix EquivalenceIT#testRandomRanges failed with -Dtest.seed A4648847991E5C27
Set double value to double type mapping in EquivalenceIT.

Closes #19697
2016-07-31 12:49:28 -04:00
Ali Beyad 0f335ac873 Removes legacy format in RepositoryData 2016-07-30 18:46:58 -04:00
Nik Everett 303c9faca5 Squash o.e.rest.action.admin.cluster
In an effort to reduce the number of tiny packages we have in the
code base this moves all the files that were in subdirectories of
`org.elasticsearch.rest.action.admin.cluster` into
`org.elasticsearch.rest.action.admin.cluster`.

Also fixes line length in these packages.
2016-07-29 20:31:24 -04:00
Michael McCandless 71166c020a Merge pull request #19554 from mikemccand/negative_usable_space
Guard against negative result from FileStore.getUsableSpace when picking data path for a new shard
2016-07-29 20:26:30 -04:00
Mike McCandless 59181c8a66 use mockito instead 2016-07-29 17:13:01 -04:00
Nik Everett bdebd02d8c Only write forced_refresh if we forced a refresh
Otherwise it just adds noise to the response.

Closes #19629
2016-07-29 15:00:30 -04:00
Christoph Büscher 0d7c289f4c Adressing review comments 2016-07-29 20:28:17 +02:00
Alexander Lin 119026b4fb Remove isCreated and isFound from the Java API
This is cleanup work from #19566, where @nik9000 suggested trying to nuke the isCreated and isFound methods. I've combined nuking the two methods with removing UpdateHelper.Operation in favor of DocWriteResponse.Operation here.

Closes #19631.
2016-07-29 14:21:43 -04:00