Commit Graph

18333 Commits

Author SHA1 Message Date
Jason Tedor e7952e2023 Fix test bug in ClusterServiceIT#testClusterStateBatchedUpdates
This commit fixes a test bug in
ClusterService#testClusterStateBatchedUpdates. In particular, in the
case that an executor did not receive a task assignment from the random
assignments, it would not have an entry in the map of executors to
counts of assigned tasks. The fix is to just check if each executor has
an entry in the counts map.
2015-12-04 16:25:36 -05:00
Jason Tedor 3ab7451ca5 Merge pull request #15252 from jasontedor/indexing-memory-controller-check-idle-simplification
Simplify IndexingMemoryController#checkIdle
2015-12-04 16:11:04 -05:00
Jason Tedor b1a67b1c69 Simplify IndexingMemoryController#checkIdle
This commit further simplifies IndexingMemoryController#checkIdle after
the changes in #15251.
2015-12-04 16:10:43 -05:00
Robert Muir 46377778a9 Merge branch 'master' into getClassLoader 2015-12-04 15:58:36 -05:00
Robert Muir b0c64910b0 ban RuntimePermission("getClassLoader")
this gives more isolation between modules and plugins.
2015-12-04 15:58:02 -05:00
Jason Tedor 5a391f116b Merge pull request #15251 from jasontedor/stateless-indexing-memory-controller
IndexingMemoryController should not track shard index states
2015-12-04 15:18:40 -05:00
Jason Tedor 56328160c9 Absorb core ShardsIndicesStatusChecker logic into body of run 2015-12-04 15:16:22 -05:00
Jason Tedor 5341404f01 IndexingMemoryController should not track shard index states
This commit modifies IndexingMemoryController to be stateless. Rather
than statefully tracking the indexing status of shards,
IndexingMemoryController can grab all available shards, check their idle
state, and then resize the buffers based on the number of and which
shards are not idle.

The driver for this change is a performance regression that can arise in
some scenarios after #13918. One scenario under which this performance
regression can arise is if an index is deleted and then created
again. Because IndexingMemoryController was previously statefully
tracking the state of shards via a map of ShardIds, the new shards with
the same ShardIds as previously existing shards would not be detected
and therefore their version maps would never be resized from the
defaults. This led to an explosion in the number of merges causing a
degradation in performance.

Closes #15225
2015-12-04 15:01:47 -05:00
Ryan Ernst 70107c5c3c Merge pull request #15233 from rjernst/jigsaw
Add modules to distributions, and move lang-expression and lang-groovy to them
2015-12-04 11:44:27 -08:00
Ryan Ernst 2dbd93d3b0 Tweaked module error message against non integ-test-zip and made checks
for limiting configuration to certain distributions prettier
2015-12-04 11:40:08 -08:00
Ryan Ernst 01d48e2062 Merge branch 'master' into jigsaw 2015-12-04 11:29:49 -08:00
Ryan Ernst a6752c4da9 Merge pull request #15210 from rjernst/buildsrc_test_cleanup
Cleanup integ test classes in buildSrc to be less groovyish
2015-12-04 11:24:21 -08:00
Lee Hinman 46ca2d679d Merge remote-tracking branch 'dakrone/remove-get-forcemerge' 2015-12-04 10:21:57 -07:00
David Pilato 619fb998e8 Update Azure Service Management API to 0.9.0
Azure team released new versions of their Java SDK.

According to https://github.com/Azure/azure-sdk-for-java/wiki/Azure-SDK-for-Java-Features, it comes with 2 versions.
We should at least update to `0.9.0` of V1 but also consider moving to the new APIs (V2).

This commit first updates to latest API V1.

```xml
<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-svc-mgmt-compute</artifactId>
    <version>0.9.0</version>
</dependency>
```

Closes #15209
2015-12-04 17:32:11 +01:00
Adrien Grand be033f1362 Make MappedFieldType.checkTypeName part of MappedFieldType.checkCompatibility. 2015-12-04 17:15:49 +01:00
Yannick Welsch 1905f495bf Merge pull request #15212 from ywelsch/fix/rejectedexecution-on-shutdown
Ignore RejectedExecutionException in NodesFaultDetection
2015-12-04 16:36:08 +01:00
Yannick Welsch 8dcc692373 Ignore RejectedExecutionException in NodesFaultDetection
Closes #15212
2015-12-04 16:35:20 +01:00
Adrien Grand f2832c491b Merge pull request #15175 from jpountz/fix/check_mapping_compat_up_front
Check mapping compatibility up-front.
2015-12-04 14:46:29 +01:00
Adrien Grand e5ed5b908f Check mapping compatibility up-front.
Today we only check mapping compatibility when adding mappers to the
lookup structure. However, at this stage, the mapping has already been merged
partially, so we can leave mappings in a bad state. This commit removes the
compatibility check from Mapper.merge entirely and performs it _before_ we
call Mapper.merge.

One minor regression is that the exception messages don't group together errors
that come from MappedFieldType.checkCompatibility and Mapper.merge. Since we
run the former before the latter, Mapper.merge won't even have a chance to let
the user know about conflicts if conflicts were discovered by
MappedFieldType.checkCompatibility.

Close #15049
2015-12-04 14:02:04 +01:00
Adrien Grand 3f18487afa Merge pull request #15177 from jpountz/remove/MergeMappingException
Remove MergeMappingException.
2015-12-04 13:55:54 +01:00
Christoph Büscher 5d0689581b Geo: Remove `translated` state from LineStringBuilder
The `translated` flag makes LineStringBuilder stateful and gets set
to true under certain conditions when building a Shape or Geometry
from the ShapeBuilder. This makes building operations not be idempotent,
so calling build() more than once on a LineStringBuilder might change the
builder itself. This PR fixes this by replacing the instance variable by
a local `translated` flag that is only updated internally during the
building process and created again on any subsequent calls to build()
or buildGeometry().
2015-12-04 12:57:52 +01:00
Adrien Grand 3f86adddbf Remove MergeMappingException.
Failures to merge a mapping can either come as a MergeMappingException if they
come from Mapper.merge or as an IllegalArgumentException if they come from
FieldTypeLookup.checkCompatibility. I think we should settle on one: this pull
request replaces all usage of MergeMappingException with
IllegalArgumentException.
2015-12-04 12:56:26 +01:00
javanna fae494aa84 Java api: allow to specify ttl as a time value (either as a string or using TimeValue)
The ttl could be specified as a time value only via the REST layer. That is now possible via java api too, either as a string or as a proper TimeValue. The internal format in IndexRequest becomes now TimeValue, which will then still converted to a long before storing the document.

Closes #15047
2015-12-04 11:51:22 +01:00
Yannick Welsch 0c2c7e7ef5 Merge pull request #14964 from ywelsch/feature/persist-allocid-indexmetadata
Persist currently started allocation IDs to index metadata
2015-12-04 11:33:50 +01:00
Yannick Welsch fef043a5bf Persist currently started allocation IDs to index metadata
Closes #14964
2015-12-04 11:31:48 +01:00
Yannick Welsch 342665300b Add capabilities for serializing map-based cluster state diffs
- Supports ImmutableOpenIntMap besides java.util.Map and ImmutableOpenMap
- Map keys can be any value (not only String)
- Map values do not have to implement Diffable interface. In that case custom value serializer needs to be provided.
2015-12-04 11:24:22 +01:00
Clinton Gormley 69fff60624 Merge pull request #15231 from bt5e/master
fixed minor typo
2015-12-04 10:01:02 +01:00
Clinton Gormley 543b9440b7 Merge pull request #15227 from popol1991/patch-1
Update scripting.asciidoc
2015-12-04 09:58:03 +01:00
Simon Willnauer 536e7e1268 Remove ancient deprecated and alternative recovery settings
Several settings have been deprecated or are replaced with new settings after refactorings
in version 1.x. This commit removes the support for these settings.

The settings are:

 * `index.shard.recovery.translog_size`
 * `index.shard.recovery.translog_ops`
 * `index.shard.recovery.file_chunk_size`
 * `index.shard.recovery.concurrent_streams`
 * `index.shard.recovery.concurrent_small_file_streams`
 * `indices.recovery.max_size_per_sec`
2015-12-04 09:48:00 +01:00
Robert Muir 784ebb1e1b fix bats tests 2015-12-04 03:25:43 -05:00
David Pilato 012dfb2988 Merge pull request #15205 from imacube/imacube-plugins-s3-update
Update repository-s3.asciidoc
2015-12-04 08:59:01 +01:00
Robert Muir 688935df1c fix broken plugin warning 2015-12-04 02:38:56 -05:00
Ryan Ernst 4b1fc8ff22 Merge pull request #15226 from rjernst/fix_update_shas
Fix updateShas to not barf on disabled license checks and even compile correctly
2015-12-03 23:34:33 -08:00
Robert Muir c78c8bae08 improve logger output on startup 2015-12-04 02:23:34 -05:00
Ryan Ernst f5b6b40a92 Fix distros to put modules dir int he right place and qa rest tests to
use the real zip
2015-12-03 23:03:14 -08:00
Ryan Ernst 7245d34f51 add modules check to plugins smoke test 2015-12-03 22:48:51 -08:00
Robert Muir 699b140d69 update lists, rename stuff to be clearer, remove public method that is unused 2015-12-04 01:43:07 -05:00
Robert Muir 7160c5ec15 list modules separately in pluginservice 2015-12-04 01:13:17 -05:00
Ryan Ernst afedd24877 force integ tests for modules to run before distribution integ tests 2015-12-03 22:07:22 -08:00
Ben Tse 3cede749f9 fixed minor typo 2015-12-03 23:53:48 -05:00
Ryan Ernst a8e9403204 added gradle checks for modules configuration, and ability to add
modules to integ test cluster
2015-12-03 20:53:06 -08:00
Robert Muir 763747df71 don't let pluginmanager install modules 2015-12-03 23:36:02 -05:00
Robert Muir 208b46d94b add leniency for tests 2015-12-03 23:25:57 -05:00
Ryan Ernst f2cebd9b3b Merge branch 'master' into jigsaw 2015-12-03 20:02:44 -08:00
Robert Muir 794264023a add module loading logic (untested) 2015-12-03 23:01:03 -05:00
Robert Muir 001a28bb98 Merge pull request #15230 from rjernst/buildsrc_eclipse_settings
Add .settings for buildSrc on gradle eclipse
2015-12-03 22:59:33 -05:00
Ryan Ernst 0be141e021 Build: Add .settings for buildSrc on gradle eclipse
We have eclipse settings added to all projects when running gradle
eclipse, but buildSrc is its own special project that is not
encapsulated by allprojects blocks. This adds eclipse settings to
buildSrc.
2015-12-03 19:56:11 -08:00
Robert Muir 88c538879b add test for modules/ 2015-12-03 22:37:27 -05:00
Robert Muir da8fe687fc add modules/ infra (but no loading via pluginservice yet) 2015-12-03 22:34:24 -05:00
Ryan Ernst b50d94efab Merge pull request #15228 from rjernst/cli_error
Cli tools: Use toString instead of getMessage for exceptions
2015-12-03 19:01:33 -08:00