Commit Graph

14213 Commits

Author SHA1 Message Date
Simon Willnauer cdcfcf4b00 Allow shards to be allocated if leftover shard from different index exists.
If an index name is reused but a leftover shard still exists on any node
we fail repeatedly to allocate the shard since we now check the index UUID
before reusing data. This commit allows to recover even if there is such a
leftover shard by deleting the leftover shard.

Closes #10677
2015-07-15 09:39:15 +02:00
Boaz Leskes 1e35bf3171 Discovery: wait on incoming joins before electing local node as master
During master election each node pings in order to discover other nodes and validate the liveness of existing nodes. Based on this information the node either discovers an existing master or, if enough nodes are found (based on `discovery.zen.minimum_master_nodes>>) a new master will be elected.

Currently, the node that is elected as master will currently update it the cluster state to indicate the result of the election. Other nodes will submit a join request to the newly elected master node. Instead of immediately processing the election result, the elected master
node should wait for the incoming joins from other nodes, thus validating the elections result is properly applied. As soon as enough nodes have sent their joins request (based on the `minimum_master_nodes` settings) the cluster state is modified.

Note that if `minimum_master_nodes` is not set, this change has no effect.

Closes #12161
2015-07-15 07:43:49 +02:00
Robert Muir 4804bf747b skip this target too, to not confuse POM builds 2015-07-15 00:30:26 -04:00
Robert Muir 4b33d2e5df Merge pull request #12252 from rmuir/refactor_integ
Refactor integration tests
2015-07-15 00:18:37 -04:00
Robert Muir 68143dedf2 Refactor integration tests
1. tests don't have a bogus test dependency on zips anymore,
   instead we handle this in pre-integration-test. This reduces
   lots of confusion for e.g. mvn clean test.
2. refactor integ logic so that core/ and plugin/ share it.
   previously they were duplicates but the above change simplifies life.
   it also makes it easier for doing more interesting stuff
2015-07-14 23:37:04 -04:00
Robert Muir 5174232c11 Merge pull request #12248 from rmuir/systemassertions
enable system assertions
2015-07-14 21:35:42 -04:00
Robert Muir ed2f8ace51 enable system assertions 2015-07-14 21:25:32 -04:00
Robert Muir aa41074ed6 Merge pull request #12247 from rmuir/remove_shadow_eclipse_build
remove shadow eclipse build
2015-07-14 20:44:31 -04:00
Robert Muir 882b46d65f remove shadow m2e build 2015-07-14 20:31:12 -04:00
Igor Motov f2911117ef Add index name to the upgrade exception 2015-07-14 19:05:39 -04:00
Igor Motov 24a93840d5 Add url repository whitelist
Require urls for URL repository to be listed in repositories.url.allowed_urls setting. This change ensures that only authorized URLs can be accessed by elasticsearch
2015-07-14 18:38:26 -04:00
Martijn van Groningen 0af70614ac test: removed AwaitsFix annotation as the associated issue is fixed and the failures don't reproduce any more 2015-07-14 22:50:47 +02:00
Simon Willnauer 3fe48c89cf Merge pull request #12240 from s1monw/m_structured_response
Render structured exceptions in mget / mpercolate
2015-07-14 22:03:20 +02:00
Robert Muir caab1df111 more fine-grained jython hack that also plays with java 9 2015-07-14 15:42:21 -04:00
Robert Muir 38054ff4ee Merge pull request #12243 from rmuir/enforcer
Use enforcer:display-info to print version info
2015-07-14 15:01:14 -04:00
Simon Willnauer b159a36559 Merge pull request #12227 from s1monw/release_profile
Add a release profile to the parent pom.xml
2015-07-14 21:00:40 +02:00
Robert Muir 15f233f672 Use enforcer:display-info to print version info
Currently this is done with ant tasks, but this is simpler and
designed for this purpose.
2015-07-14 14:48:20 -04:00
Martijn van Groningen f5e351aaf1 parent/child: Removed AwaitsFix annotation
Closes #9461
2015-07-14 20:05:35 +02:00
Robert Muir 0808c98eeb enable enforcer plugin 2015-07-14 13:42:12 -04:00
Clinton Gormley 2b512f1f29 Docs: Use "js" instead of "json" and "sh" instead of "shell" for source highlighting 2015-07-14 18:14:09 +02:00
Robert Muir 01601e9a3d Add permissions to all classpath elements in BootstrapForTesting
The previous strategy (target/xxx + .m2/repository) is obviously broken for multi-module
builds.

Includes hack for crazy jython, which "finds its own jar" then looks for a Lib/ beside it
2015-07-14 12:01:03 -04:00
Simon Willnauer def08bd594 Render structured exceptions in mget / mpercolate
Instead of rendering only the exception message this commit
adds structured exception rendering to mget and mpercolate
2015-07-14 17:52:28 +02:00
Clinton Gormley a4f7ede953 Asciidoc fix to API Conventions 2015-07-14 17:33:47 +02:00
Simon Willnauer 09bd19b947 Don't allow fuzziness specified as a and require edits [0,2]
Lucene deprecated this in 4.0 and we only try best effort to support it.
Folks should only use edit distance rather than some length based
similarity. Yet the formular is simple enough such that users can
still do it in the client if they really need to.

Closes #10638
2015-07-14 17:10:29 +02:00
Simon Willnauer 738a0bc8a2 [TEST] Don't rely on iteration oder in tests
Closes #12238
2015-07-14 17:08:26 +02:00
Colin Goodheart-Smithe bccfcfa522 Aggregation: Removed Old Script Java API from metrics aggregations
The old script syntax has been removed from the Java API but the metrics aggregations were missed. This change removes the old script API from the ValuesSourceMetricsAggregationBuilder and removes the relevant test methods for the metrics aggregations.
2015-07-14 15:41:11 +01:00
Simon Willnauer 7db293c616 Generify Index and Shard exceptions
Today we have a intermediate hierarchy for shard and index exceptions
which makes it hard to introduce generic exceptions like ResourceNotFoundException
intoduced in this commit. This commit breaks up the hierarchy by adding index and shard
as a special internal header that gets rendered for every exception that fills that header.
This commit removes dedicated exceptions like `IndexMissingException` or
`IndexShardMissingException` in favour of `ResourceNotFoundException`
2015-07-14 16:31:49 +02:00
Shay Banon c6b110c6ef Simplify assignToNode to only do initializing
The method really only should do the move from unassigned to initializing, all the other moves have explicit methods like relocate
2015-07-14 16:05:48 +02:00
Clinton Gormley 5324855224 Merge pull request #12223 from awislowski/patch-1
Update index.asciidoc
2015-07-14 14:36:03 +02:00
Martijn van Groningen 7d59fd6ea0 Merge pull request #12215 from martijnvg/percolator/filter_by_now
Support filtering percolator queries by date using `now`
2015-07-14 14:35:31 +02:00
Colin Goodheart-Smithe b76c394bf3 [DOCS] More updates to script parameters section of breaking changes doc 2015-07-14 13:34:11 +01:00
Colin Goodheart-Smithe 2e6af3ba4c [DOCS] updates to script parameters section of braking changes doc 2015-07-14 13:33:37 +01:00
Martijn van Groningen 5cdbe60f6b percolator: Support filtering percolator queries by date using `now`
Closes #12185
2015-07-14 12:35:32 +02:00
Simon Willnauer 295f1971d2 Add a release profile to the parent pom.xml
This commit adds a release profile that runs additional checks like:
  * check for `norelease` in the source
  * check for `AwaitsFix` annotations
  * deploys only if all reactor builds where successful
  * signs all artifacts with the GPG key provided (required for release)
  * builds the RPM together with the zip, tar.gz and .deb during package phase

Note this is WIP of replacing major functionality of the release pythong script but
won't work by itself.
2015-07-14 12:04:16 +02:00
Shay Banon e598f16b58 Default delayed allocation timeout to 1m from 0
Change the default delayed allocation timeout from 0 (no delayed allocation) to 1m. The value came from a test of having a node with 50 shards being indexed into (so beefy translog requiring flush on shutdown), then shutting it down and starting it back up and waiting for it to join the cluster. This took, on a slow machine, about 30s.
The value is conservatively low and does not try to address a virtual machine / OS restart for now, in order to not have the affect of node going away and users being concerned that shards are not being allocated to the rest of the cluster as a result of that. The setting can always be changed in order to increase the delayed allocation if needed.
closes #12166
2015-07-14 11:31:16 +02:00
Boaz Leskes 7fec35b678 Merge pull request #12224 from awislowski/patch-2
Doc: fix github source link in resiliency page

Closes #12224
2015-07-14 11:15:44 +02:00
Simon Willnauer fbafd14f23 [TEST] Don't use replicas to stabelize assertion.
This test asserts that the first recovery result is of type SNAPSHOT.
This assertion might not be true depending on the rendering order if
a replica is recovered quick enough. This commit disables replicas and
their recovery since it's not the purpose of this test.
2015-07-14 10:32:20 +02:00
Martijn van Groningen 44275f3e3e Merge pull request #12214 from martijnvg/percolator/_parent_field/npe
Fix NPE when percolating a document that has a _parent field configured in its mapping
2015-07-14 10:31:10 +02:00
Andrzej Wisłowski 7b4824f318 fix github source link 2015-07-14 10:14:15 +02:00
Martijn van Groningen 7bc26d1edd percolator: Don't throw NPE when percolating a document that has a _parent field configured in its mapping
Closes #12192
2015-07-14 10:01:16 +02:00
Andrzej Wisłowski aaea4a2f52 Update index.asciidoc 2015-07-14 09:35:15 +02:00
Robert Muir 6c466ed0d8 Merge pull request #12221 from rmuir/version_hell
fail plugins on version mismatch
2015-07-13 21:46:09 -04:00
Robert Muir 5d48f93557 fail plugins on version mismatch 2015-07-13 21:01:39 -04:00
Ryan Ernst fd6bc9fc57 tests: explicitly disable security manager for maven integration tests 2015-07-13 17:09:06 -07:00
Ryan Ernst 2628ecfacf Merge pull request #12219 from rjernst/fix/integ-test-exceptions
Tests: Explicitly use newer junit runner for maven integ tests
2015-07-13 15:37:34 -07:00
Ryan Ernst adfd55bf89 Tests: Explicitly use newer junit runner for maven integ tests
The older runner (junit 4.0?) hides exceptions in static initialization
(and runs the tests anyways!). With this change, the actual cause
of the recent windows integ test failures with security manager
are shown.
2015-07-13 15:33:02 -07:00
uboness 686413771c Removed duplication in parent pom
Found 2 definition for  `maven-install-plugin`
2015-07-13 22:40:18 +02:00
Boaz Leskes d12fb5f725 Add MetaData.clusterUUID to ClusterState.toXContent
The MetaData.clusterUUID is guaranteed to be unique across clusters and is handy (which may or may not have the same human readable cluster name).

Closes #11832
2015-07-13 20:31:11 +02:00
uboness b40186652c updated the elasticsearch versioning format
Moving to from `X.Y.Z.beta1`/`X.Y.Z.RC1` to `X.Y.Z-beta1`/`X.Y.Z-rc1`
2015-07-13 20:26:37 +02:00
Boaz Leskes 17906ca7d6 Rename MetaData.uuid -> MetaData.clusterUUID and IndexMetaData.uuid-> IndexMetaData.indexUUID
As explained in #11831, we currently have uuid fields on the cluster state, meta data and index metadata. The latter two are persistent across changes are being effectively used as a persistent uuid for this cluster and a persistent uuid for an index. The first (ClusterState.uuid) is ephemeral and changes with every change to the cluster state. This is confusing,

We settled on having the following, new names:

-> ClusterState.uuid -> stateUUID (transient)
-> MetaData.uuid -> clusterUUID (persistent)
-> IndexMetaData.uuid -> indexUUID (persistent).

Closes #11914
Closes #11831
2015-07-13 20:15:54 +02:00