Commit Graph

2580 Commits

Author SHA1 Message Date
Robert Muir 506fa617da Merge pull request #15253 from rmuir/getClassLoader
ban RuntimePermission("getClassLoader")
2015-12-04 18:34:08 -05:00
Ryan Ernst 50716c3941 Plugins: Add nicer error message when an existing plugin's descriptor is missing
Currently, when a user tries to install an old plugin (pre 2.x) on a 2.x
node, the error message is cryptic (just printing the file path that was
missing, when looking for the descriptor). This improves the message to
be more explicit that the descriptor is missing, and suggests the
problem might be the plugin was built before 2.0.

closes #15197
2015-12-04 14:11:37 -08:00
Jason Tedor bbef8acd3c Cleanup ClusterServiceIT#testClusterStateBatchedUpdates
This commit addresses some issues that arose during the review of #14899
but were lost during squash while integrating into master.
 - the number of test threads is dropped to at most eight
 - a local variable is renamed for clarity
 - task priorities are randomized
2015-12-04 16:36:29 -05:00
Jason Tedor e7952e2023 Fix test bug in ClusterServiceIT#testClusterStateBatchedUpdates
This commit fixes a test bug in
ClusterService#testClusterStateBatchedUpdates. In particular, in the
case that an executor did not receive a task assignment from the random
assignments, it would not have an entry in the map of executors to
counts of assigned tasks. The fix is to just check if each executor has
an entry in the counts map.
2015-12-04 16:25:36 -05:00
Jason Tedor b1a67b1c69 Simplify IndexingMemoryController#checkIdle
This commit further simplifies IndexingMemoryController#checkIdle after
the changes in #15251.
2015-12-04 16:10:43 -05:00
Robert Muir 46377778a9 Merge branch 'master' into getClassLoader 2015-12-04 15:58:36 -05:00
Robert Muir b0c64910b0 ban RuntimePermission("getClassLoader")
this gives more isolation between modules and plugins.
2015-12-04 15:58:02 -05:00
Jason Tedor 5a391f116b Merge pull request #15251 from jasontedor/stateless-indexing-memory-controller
IndexingMemoryController should not track shard index states
2015-12-04 15:18:40 -05:00
Jason Tedor 56328160c9 Absorb core ShardsIndicesStatusChecker logic into body of run 2015-12-04 15:16:22 -05:00
Jason Tedor 5341404f01 IndexingMemoryController should not track shard index states
This commit modifies IndexingMemoryController to be stateless. Rather
than statefully tracking the indexing status of shards,
IndexingMemoryController can grab all available shards, check their idle
state, and then resize the buffers based on the number of and which
shards are not idle.

The driver for this change is a performance regression that can arise in
some scenarios after #13918. One scenario under which this performance
regression can arise is if an index is deleted and then created
again. Because IndexingMemoryController was previously statefully
tracking the state of shards via a map of ShardIds, the new shards with
the same ShardIds as previously existing shards would not be detected
and therefore their version maps would never be resized from the
defaults. This led to an explosion in the number of merges causing a
degradation in performance.

Closes #15225
2015-12-04 15:01:47 -05:00
Ryan Ernst 01d48e2062 Merge branch 'master' into jigsaw 2015-12-04 11:29:49 -08:00
Lee Hinman 46ca2d679d Merge remote-tracking branch 'dakrone/remove-get-forcemerge' 2015-12-04 10:21:57 -07:00
Yannick Welsch 8dcc692373 Ignore RejectedExecutionException in NodesFaultDetection
Closes #15212
2015-12-04 16:35:20 +01:00
Adrien Grand e5ed5b908f Check mapping compatibility up-front.
Today we only check mapping compatibility when adding mappers to the
lookup structure. However, at this stage, the mapping has already been merged
partially, so we can leave mappings in a bad state. This commit removes the
compatibility check from Mapper.merge entirely and performs it _before_ we
call Mapper.merge.

One minor regression is that the exception messages don't group together errors
that come from MappedFieldType.checkCompatibility and Mapper.merge. Since we
run the former before the latter, Mapper.merge won't even have a chance to let
the user know about conflicts if conflicts were discovered by
MappedFieldType.checkCompatibility.

Close #15049
2015-12-04 14:02:04 +01:00
Adrien Grand 3f18487afa Merge pull request #15177 from jpountz/remove/MergeMappingException
Remove MergeMappingException.
2015-12-04 13:55:54 +01:00
Christoph Büscher 5d0689581b Geo: Remove `translated` state from LineStringBuilder
The `translated` flag makes LineStringBuilder stateful and gets set
to true under certain conditions when building a Shape or Geometry
from the ShapeBuilder. This makes building operations not be idempotent,
so calling build() more than once on a LineStringBuilder might change the
builder itself. This PR fixes this by replacing the instance variable by
a local `translated` flag that is only updated internally during the
building process and created again on any subsequent calls to build()
or buildGeometry().
2015-12-04 12:57:52 +01:00
Adrien Grand 3f86adddbf Remove MergeMappingException.
Failures to merge a mapping can either come as a MergeMappingException if they
come from Mapper.merge or as an IllegalArgumentException if they come from
FieldTypeLookup.checkCompatibility. I think we should settle on one: this pull
request replaces all usage of MergeMappingException with
IllegalArgumentException.
2015-12-04 12:56:26 +01:00
javanna fae494aa84 Java api: allow to specify ttl as a time value (either as a string or using TimeValue)
The ttl could be specified as a time value only via the REST layer. That is now possible via java api too, either as a string or as a proper TimeValue. The internal format in IndexRequest becomes now TimeValue, which will then still converted to a long before storing the document.

Closes #15047
2015-12-04 11:51:22 +01:00
Yannick Welsch fef043a5bf Persist currently started allocation IDs to index metadata
Closes #14964
2015-12-04 11:31:48 +01:00
Yannick Welsch 342665300b Add capabilities for serializing map-based cluster state diffs
- Supports ImmutableOpenIntMap besides java.util.Map and ImmutableOpenMap
- Map keys can be any value (not only String)
- Map values do not have to implement Diffable interface. In that case custom value serializer needs to be provided.
2015-12-04 11:24:22 +01:00
Robert Muir 688935df1c fix broken plugin warning 2015-12-04 02:38:56 -05:00
Robert Muir c78c8bae08 improve logger output on startup 2015-12-04 02:23:34 -05:00
Robert Muir 699b140d69 update lists, rename stuff to be clearer, remove public method that is unused 2015-12-04 01:43:07 -05:00
Robert Muir 7160c5ec15 list modules separately in pluginservice 2015-12-04 01:13:17 -05:00
Robert Muir 763747df71 don't let pluginmanager install modules 2015-12-03 23:36:02 -05:00
Robert Muir 208b46d94b add leniency for tests 2015-12-03 23:25:57 -05:00
Ryan Ernst f2cebd9b3b Merge branch 'master' into jigsaw 2015-12-03 20:02:44 -08:00
Robert Muir 794264023a add module loading logic (untested) 2015-12-03 23:01:03 -05:00
Robert Muir da8fe687fc add modules/ infra (but no loading via pluginservice yet) 2015-12-03 22:34:24 -05:00
Ryan Ernst b50d94efab Merge pull request #15228 from rjernst/cli_error
Cli tools: Use toString instead of getMessage for exceptions
2015-12-03 19:01:33 -08:00
Ryan Ernst f78c732ca9 Cli tools: Use toString instead of getMessage for exceptions
When not in debug mode, we currently only print the message of an
exception. However, this is not usually useful without knowing what the
exception type was. This change makes cli tools use toString() on the
exception so we get the type + message.
2015-12-03 17:27:45 -08:00
Jason Tedor 6613b3a0d6 Merge pull request #15221 from jasontedor/avoid-trace-logging-allocations-in-transport-broadcast-by-node-action
Avoid trace logging allocations in TransportBroadcastByNodeAction
2015-12-03 17:17:01 -05:00
Lee Hinman f709b7283f Remove `GET` option for /_forcemerge
POST should be used to indicate this is not just a retrieval operation.

Resolves #15165
2015-12-03 13:56:15 -07:00
Jason Tedor de0e1f5e2f Avoid trace logging allocations in TransportBroadcastByNodeAction
This commit wraps the trace logging statements in
TransportBroadcastByNodeAction in trace enabled checks to avoid
unnecessarily allocating objects.

The most egregious offenders were the two trace logging statements in
BroadcastByNodeTransportRequestHandler#onShardOperation. Aside from the
usual object allocations that occur when invoking ESLogger#trace (the
allocated object array for the varargs Object... parameter), these two
logging statements were invoking ShardRouting#shortSummary generating a
bunch of char arrays and Strings (from the StringBuilder, and so a bunch
of array copies as well). In a scenario where there are a lot of shards
and this method is being invoked frequently (e.g., constantly hitting
the _stats endpoint), these two unprotected trace logging statements
were generating a lot of unnecessary allocations.
2015-12-03 14:38:16 -05:00
Jason Tedor b58d82f66c Remove obsolete flag in ShardStateAction$ShardRoutingEntry 2015-12-03 14:08:35 -05:00
Jason Tedor 928d53a884 Apply shard starts in a single batch 2015-12-03 14:04:10 -05:00
Jason Tedor 99eac0e7d7 Use general cluster state batching mechanism for shard started
This commit modifies the handling of shard started cluster state updates
to use the general cluster state batching mechanism. An advantage of
this approach is we now get correct per-listener notification on
failures.
2015-12-03 14:04:10 -05:00
Jason Tedor 182c22f23f Merge pull request #15016 from jasontedor/shard-failure-batch
Use general cluster state batching mechanism for shard failures
2015-12-03 14:02:48 -05:00
Jason Tedor b70d97f36b Remove unnecessary method in AllocationService 2015-12-03 13:42:05 -05:00
Jason Tedor 73a2d3085a Rename variable for clarity in ShardFailedClusterStateHandler#execute 2015-12-03 13:41:34 -05:00
Adrien Grand d2e07f21e0 Merge pull request #15217 from jpountz/fix/simplify_MetaDataMappingService
Simplify MetaDataMappingService.
2015-12-03 16:26:08 +01:00
Adrien Grand 3ff91baedf Simplify MetaDataMappingService.
Now that we create all types on indices that receive a mapping update, creation
of temporary indices could get simpler.
2015-12-03 16:06:51 +01:00
Jason Tedor d1fd1fb908 Remove unnecessary early-out in MetaDataMappingService#executeRefresh
This commit removes a simple early-out check in
MetaDataMappingService#executeRefresh. The early-out is unnecessary
because the cluster state task execution framework will not invoke
ClusterStateTaskExecutor#execute if the list of tasks is empty.
2015-12-03 09:05:19 -05:00
Jason Tedor 310e98f51d Update stale Javadoc in MetaDataMappingService#executeRefresh
This commit updates a stale Javadoc on
MetaDataMappingService#executeRefresh. Previously this method handled
refresh and update tasks. Update tasks have been removed and the method
was renamed, but the Javadoc was not updated to reflect this.
2015-12-03 09:05:19 -05:00
Martijn van Groningen d0fbd9d1c4 index template: Disallow index template pattern to be the same as an alias name
This can cause index creation to fail down the line.
2015-12-03 13:35:16 +01:00
Boaz Leskes c5301e853a reduced the number of shards in RecoveryWhileUnderLoadIT.testRecoveryWhileRelocating 2015-12-03 10:00:48 +01:00
Boaz Leskes 9930e6883d Move IndicesService.canDeleteShardContent to use IndexSettings
Just a minor cleanup/simplification

Closes #15059
Closes #15150
2015-12-03 09:55:07 +01:00
Adrien Grand 6ff82260a7 Merge pull request #15029 from jpountz/remove/onIndexService
Remove Plugin.onIndexService.
2015-12-03 09:46:30 +01:00
Adrien Grand 612ef4488e Add missing license header. 2015-12-03 09:41:06 +01:00
Adrien Grand 79ff2ddd20 Merge pull request #15202 from jpountz/fix/clear_empty_query_cache
Fix NPE when a segment with an empty cache gets closed.
2015-12-03 09:38:08 +01:00