Commit Graph

27381 Commits

Author SHA1 Message Date
Nik Everett 94e3796908 Docs tests: cat/health can have max_task_wait_time
Make the doc test assertions ok with a non `-` value for
`max_task_wait_time`. These are rare, but possible:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-unix-compatibility/os=oraclelinux/900/consoleFull
2017-04-28 09:58:53 -04:00
Nik Everett e3b7b88756 Fix compilation in Ecipse (#24391)
Eclipse doesn't allow extra semicolons after an import statement:
```
import foo.Bar;;   // <-- syntax error!
```

Here is the Eclipse bug:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=425140
which the Eclipse folks closed as "the spec doesn't allow these
semicolons so why should we?" Which is fair. Here is the bug
against javac for allowing them:
https://bugs.openjdk.java.net/browse/JDK-8027682
which hasn't been touched since 2013 without explanation. There
is, however, a rather educations mailing list thread:
http://mail.openjdk.java.net/pipermail/compiler-dev/2013-August/006956.html

which contains gems like, "In general, it is better/simpler to
change javac to conform to the spec. (Except when it is not.)"

I suspect the reason this hasn't been fixed is:
```
FWIW, if we change javac such that the set of programs accepted by javac
is changed, we have an process (currently Oracle internal) to get
approval for such a change.   So, we would not simply change javac on a
whim to meet the spec; we would at least have other eyes looking at the
behavioral change to determine if it is "acceptable".
```
from http://mail.openjdk.java.net/pipermail/compiler-dev/2013-August/006973.html
2017-04-28 09:52:14 -04:00
Guillaume Le Floch 382a617d34 Handle multiple aliases in _cat/aliases api (#23698)
The alias parameter was documented as a list in our rest-spec, yet only the first value out of a list was getting read and processed. This commit adds support for multiple aliases to _cat/aliases

Closes #23661
2017-04-28 15:21:44 +02:00
Nik Everett 2412574e49 Docs: Upserts no longer support version
Closes #16671
2017-04-28 08:43:46 -04:00
Christoph Büscher 0d5cca8feb Fixing checktyle error for modifier order 2017-04-28 14:19:00 +02:00
Kunal Kapoor a5bd2012b6 Added validation for upserd request (#24282)
The version on an update request is a syntactic sugar
for get of a specific version, doc merge and a version
index. This changes it to reject requests with both
upsert and a version.
If the upsert index request is versioned, we also
reject the op.
2017-04-28 08:02:09 -04:00
Christoph Büscher a6c38b8f8a Add parsing for InternalGeoBounds (#24365) 2017-04-28 12:10:16 +02:00
Yannick Welsch a72db191f2 Weaken assertion in ZenDiscovery.publish
The previous commit (35f78d098a) introduced an assertion in ZenDiscovery that was overly restrictive - it could trip when a cluster state that was
successfully published would not be applied locally because a master with a better cluster state came along in the meantime.
2017-04-28 11:33:53 +02:00
Martijn van Groningen b77254871b
docs: document alternative for nested inner hits source
Closes #24110
2017-04-28 11:09:24 +02:00
Yannick Welsch 35f78d098a Separate publishing from applying cluster states (#24236)
Separates cluster state publishing from applying cluster states:

- ClusterService is split into two classes MasterService and ClusterApplierService. MasterService has the responsibility to calculate cluster state updates for actions that want to change the cluster state (create index, update shard routing table, etc.). ClusterApplierService has the responsibility to apply cluster states that have been successfully published and invokes the cluster state appliers and listeners.
- ClusterApplierService keeps track of the last applied state, but MasterService is stateless and uses the last cluster state that is provided by the discovery module to calculate the next prospective state. The ClusterService class is still kept around, which now just delegates actions to ClusterApplierService and MasterService.
- The discovery implementation is now responsible for managing the last cluster state that is used by the consensus layer and the master service. It also exposes the initial cluster state which is used by the ClusterApplierService. The discovery implementation is also responsible for adding the right cluster-level blocks to the initial state.
- NoneDiscovery has been renamed to TribeDiscovery as it is exclusively used by TribeService. It adds the tribe blocks to the initial state.
- ZenDiscovery is synchronized on state changes to the last cluster state that is used by the consensus layer and the master service, and does not submit cluster state update tasks anymore to make changes to the disco state (except when becoming master).

Control flow for cluster state updates is now as follows:

- State updates are sent to MasterService
- MasterService gets the latest committed cluster state from the discovery implementation and calculates the next cluster state to publish
- MasterService submits the new prospective cluster state to the discovery implementation for publishing
- Discovery implementation publishes cluster states to all nodes and, once the state is committed, asks the ClusterApplierService to apply the newly committed state.
- ClusterApplierService applies state to local node.
2017-04-28 09:34:31 +02:00
Toby McLaughlin e4bb360ae0 Fix typo in node environment exception message
This commit fixes a typo in an exception message when trying to create a
node environment.

Relates #24381
2017-04-28 00:48:52 -04:00
Christoph Büscher 9cb3666f0a Merge branch 'master' into feature/client_aggs_parsing 2017-04-28 00:41:16 +02:00
Zachary Tong 350573290f Agg builder accessibility fixes (#24323)
- Getters for DateHisto `interval` and `offset` should return a
long, not double
- Add getter for the filter in a FilterAgg
- Add getters for subaggs / pipelines in base AggregationBuilder
2017-04-27 16:50:59 -04:00
Ryan Ernst cdcc75dd2a Plugins: Add support for platform specific plugins (#24265)
This commit adds support for plugins having a platform specific variant.
It also adds unit tests for all official and maven urls.
2017-04-27 11:27:29 -07:00
Yannick Welsch b4cff19842 Run gradle vagrantUpdateVersions to add version 5.3.2 2017-04-27 17:29:33 +02:00
Nik Everett 06aeb2307f Docs: remove experimental tag from update-by-query
We meant to remove it in #23621 but we removed it from the
documentation for the java client but not the rest documetnaiton.
2017-04-27 10:36:28 -04:00
Ali Beyad 2facc42a55 Change snapshot status error to use generic SnapshotException (#24355)
Changes the snapshot status read exception from the (misleading)
IndexShardRestoreFailedException to the generic SnapshotException

Closes #24225
2017-04-27 09:36:26 -04:00
Yannick Welsch 2fa1c9fff1 Provide target allocation id as part of start recovery request (#24333)
This makes it possible for the recovery source to verify that it is talking to the shard it thinks it is talking to.

Closes #24167
2017-04-27 14:45:44 +02:00
Tim Vernum 65f90b25e0 Pass Context to ConstructingObjectParser's function (#24230)
Allow the `Context` to be used in the builder function used within ConstructingObjectParser.
This facilitates scenarios where a constructor argument comes from a URL parameter, or from document id.
2017-04-27 20:26:10 +10:00
Christoph Büscher 93415cdfb4 Add parsing for InternalPercentilesBucket (#24330) 2017-04-27 11:04:57 +02:00
Clinton Gormley 8a8410b5ce Added bwc indices for v2.4.5 2017-04-27 10:30:53 +02:00
Ryan Ernst 4a5c3c5a4a Test: Write node ports file before starting tribe service (#24351)
The tribe service can take a while to initialize, depending on how many cluster it needs to connect to. This change moves writing the ports file used by tests to before the tribe service is started.
2017-04-27 09:59:54 +02:00
Adrien Grand 1be2800120 Only allow one type on 7.0 indices (#24317)
This adds the `index.mapping.single_type` setting, which enforces that indices
have at most one type when it is true. The default value is true for 6.0+ indices
and false for old indices.

Relates #15613
2017-04-27 08:43:20 +02:00
Jason Tedor 74acc594a9 Fix inconsistencies in long GC disruption
This commit fixes some inconsistencies in long GC disruption where we
mixed stopping and suspending when the action we are performing on
threads is suspending which is distinct from stopping a thread.
2017-04-26 21:23:19 -04:00
Koen De Groote 7f9d84cb1a The parseObject method in DocumentParse can be void. There is no point in the code that actually expects the return, plus the variable created for it was never actually used. (#24350) 2017-04-26 16:54:23 -06:00
Ali Beyad d387dcfd6c [TEST] fixes NPE in RoutingTableTests 2017-04-26 18:39:26 -04:00
Christoph Büscher b46c39ea2e Fixed another sneaky LineLength violation 2017-04-27 00:02:08 +02:00
Christoph Büscher 1f221154a6 Merge branch 'master' into feature/client_aggs_parsing 2017-04-26 23:09:45 +02:00
Christoph Büscher e2b95f9758 Fix order of static field declarations for parser 2017-04-26 23:07:49 +02:00
Ali Beyad 0e74f5ddb1 [TEST] fixes shard count of source shard index in a restore shrink index
test
2017-04-26 16:34:53 -04:00
Christoph Büscher 774368d2ac Add parsing for InternalStatsBucket and InternalExtendedStatsBucket (#24312) 2017-04-26 22:32:30 +02:00
Nik Everett bc45d10e82 Remove most usages of 1-arg Script ctor (#24325)
The one argument ctor for `Script` creates a script with the
default language but most usages of are for testing and either
don't care about the language or are for use with
`MockScriptEngine`. This replaces most usages of the one argument
ctor on `Script` with calls to `ESTestCase#mockScript` to make
it clear that the tests don't need the default scripting language.

I've also factored out some copy and pasted script generation
code into a single place. I would have had to change that code
to use `mockScript` anyway, so it was easier to perform the
refactor.

Relates to #16314
2017-04-26 16:04:38 -04:00
Luca Cavanna 149629fec6 Cross Cluster Search: propagate original indices per cluster (#24328)
In case of a Cross Cluster Search, the coordinating node should split the original indices per cluster, and send over to each cluster only its own set of original indices, rather than the set taken from the original search request which contains all the indices.

In fact, each remote cluster should not be aware of the indices belonging to other remote clusters.
2017-04-26 21:45:49 +02:00
Jason Tedor 2ed1f7a339 Avoid leaks in Long GC disruption tests
We can leak disrupted threads here since we never wait for them to
complete after freeing them from their loops. This commit addresses this
by joining on disrupted threads, and addresses fallout from trying to
join here.

Relates #24338
2017-04-26 15:26:36 -04:00
Yannick Welsch b7bf651738 [TEST] Fix cluster forming in testDynamicUpdateMinimumMasterNodes
This test can run into a split-brain situation as minimum_master_nodes is not properly set. To prevent this, make sure that at least one of the two
master nodes that are initially started has minimum_master_nodes correctly set.
2017-04-26 21:13:27 +02:00
Martijn van Groningen ebe98f9d62
test: don't randomly wrap index reader 2017-04-26 21:07:56 +02:00
Ali Beyad 0e52e3420e Fixes restore of a shrunken index when initial recovery node is gone (#24322)
When an index is shrunk using the shrink APIs, the shrink operation adds
some internal index settings to the shrink index, for example
`index.shrink.source.name|uuid` to denote the source index, as well as
`index.routing.allocation.initial_recovery._id` to denote the node on
which all shards for the source index resided when the shrunken index
was created.  However, this presents a problem when taking a snapshot of
the shrunken index and restoring it to a cluster where the initial
recovery node is not present, or restoring to the same cluster where the
initial recovery node is offline or decomissioned.  The restore
operation fails to allocate the shard in the shrunken index to a node
when the initial recovery node is not present, and a restore type of
recovery will *not* go through the PrimaryShardAllocator, meaning that
it will not have the chance to force allocate the primary to a node in
the cluster.  Rather, restore initiated shard allocation goes through
the BalancedShardAllocator which does not attempt to force allocate a
primary.

This commit fixes the aforementioned problem by not requiring allocation
to occur on the initial recovery node when the recovery type is a
restore of a snapshot.  This commit also ensures that the internal
shrink index settings are recognized and not archived (which can trip an
assertion in the restore scenario).

Closes #24257
2017-04-26 14:48:10 -04:00
Koen De Groote 3187ed73fc Removal of dead code in ScriptedMetricAggregationBuilder (#24346)
This code removes a few lines of dead code from ScriptedMetricAggregationBuilder. 
Just completely dead code, it adds things to a Set that is then not used in any way.
2017-04-26 14:44:03 -04:00
Koen De Groote 4c0eb35c22 Removal of dead code from SnapshotsService (#24347)
This code removes a few lines of dead code from SnapshotsService. 
Looks like a forgotten remnant of a past implementation.
2017-04-26 14:32:35 -04:00
Nik Everett 416feeb7f9 Rewrite description of `bool`'s `should` (#24342)
Docs: rewrite description of `bool`'s `should`

Rewrites the description of the `bool` query's `should`
clauses so it is (hopefully) more clear what the defaults
for `minimum_should_match` are.

There is still an `[IMPORTANT]` section about `minimum_should_match`
in a filter context. I think it is worth keeping because it is, well,
important.

Closes #23831
2017-04-26 14:09:26 -04:00
Nik Everett 7c3efb829b Move char filters into analysis-common (#24261)
Another step down the road to dropping the
lucene-analyzers-common dependency from core.

Note that this removes some tests that no longer compile from
core. I played around with adding them to the analysis-common
module where they would compile but we already test these in
the tests generated from the example usage in the documentation.

I'm not super happy with the way that `requriesAnalysisSettings`
works with regards to plugins. I think it'd be fairly bug-prone
for plugin authors to use. But I'm making it visible as is for
now and I'll rethink later.

A part of #23658
2017-04-26 13:25:34 -04:00
Christoph Büscher db1b243343 InternalPercentilesBucket should not rely on ordered percents array (#24336)
Currently InternalPercentilesBucket#percentile() relies on the percent array passed in
to be in sorted order. This changes the aggregation to store an internal lookup table that
is constructed from the percent/percentiles arrays passed in that can be used to look up 
the percentile values.

Closes #24331
2017-04-26 19:15:48 +02:00
Nik Everett 0c12d0ce37 Make bats tests refuse to start on non-VMs (#24315)
The bats tests are descructive and must be run as root. This is a
horrible combination on any sane system but perfectly fine to do
in a VM. This change modifies the tests so they revuse to start
unless they are in an environment with an `/etc/is_vagrant_vm`
file. The Vagrantfile creates it on startup.

Closes #24137
2017-04-26 10:40:35 -04:00
Yannick Welsch 91b61ce569 [TEST] Do a reroute with retry_failed after a bridge partition on testAckedIndexing
In case of a bridge partition, shard allocation can fail "index.allocation.max_retries" times if the master is the super-connected node and recovery
source and target are on opposite sides of the bridge. This commit adds a reroute with retry_failed after healing the network partition so that the
ensureGreen check succeeds.
2017-04-26 16:08:16 +02:00
Jay Modi 7f8fe8b81d StreamInput throws exceptions instead of using assertions (#24294)
StreamInput has methods such as readVInt that perform sanity checks on the data using assertions,
which will catch bad data in tests but provide no safety when running as a node without assertions
enabled. The use of assertions also make testing with invalid data difficult since we would need
to handle assertion errors in the code using the stream input and errors like this should not be
something we try to catch. This commit introduces a flag that will throw an IOException instead of
using an assertion.
2017-04-26 07:23:07 -04:00
Martijn van Groningen c17de49a6d
[percolator] Fix memory leak when percolator uses bitset or field data cache.
The percolator doesn't close the IndexReader of the memory index any more.
Prior to 2.x the percolator had its own SearchContext (PercolatorContext) that did this,
but that was removed when the percolator was refactored as part of the 5.0 release.

I think an alternative way to fix this is to let percolator not use the bitset and fielddata caches,
that way we prevent the memory leak.

Closes #24108
2017-04-26 11:08:15 +02:00
Christoph Büscher fede0bab2c Correcting 140 character line length 2017-04-26 11:04:33 +02:00
Ryan Ernst 51b33f1fd5 S3 Repository: Deprecate remaining `repositories.s3.*` settings (#24144)
Most of these settings should always be pulled from the repository
settings. A couple were leftover that should be moved to client
settings. The path style access setting should be removed altogether.
This commit adds deprecations for all of these existing settings, as
well as adding new client specific settings for max retries and
throttling.

relates #24143
2017-04-25 23:43:20 -07:00
Koen De Groote 3c845727f8 Replace alternating regex with character classes
This commit replaces two alternating regular expressions (that is,
regular expressions that consist of the form a|b where a and b are
characters) with the equivalent regular expression rewritten as a
character class (that is, [ab]) The reason this is an improvement is
because a|b involves backtracking while [ab] does not.

Relates #24316
2017-04-25 22:15:00 -04:00
Nik Everett fc97e25b56 Add task to look for tests in src/main (#24298)
Creates a new task `namingConventionsMain`, that runs on the
`buildSrc` and `test:framework` projects and fails the build if
any of the classes in the main artifacts are named like tests or
are non-abstract subclasses of ESTestCase.

It also fixes the three tests that would cause it to fail.
2017-04-25 21:11:47 -04:00