Commit Graph

18413 Commits

Author SHA1 Message Date
Daniel Mitterdorfer b2b7b30d10 BulkProcessor retries after request handling has been rejected due to a full thread pool
With this commit we introduce limited retries with a backoff logic to BulkProcessor
when a bulk request has been rejeced with an EsRejectedExecutionException.

Fixes #14620.
2015-12-17 14:05:00 +01:00
Adrien Grand 86d6e28b7f MetaDataMappingService should call MapperService.merge with the original mapping update.
Currently MetaDataMappingService parses the mapping updates, reserializes it and
finally calls MapperService.merge with the serialized mapping. Given that mapping
serialization only writes differences from the default, this is a bit unfair to
parsers since they can't know whether some option has been explicitly set or not.

Furthermore this can cause bugs with metadata fields given that these fields use
existing field types as defaults.

This commit changes MetaDataMappingService to call MapperService.merge with the
original mapping update.
2015-12-17 13:09:15 +01:00
Michael McCandless c1d8d3b28b AllTermQuery's scorer should skip segments that never saw the requested term 2015-12-17 05:26:54 -05:00
Adrien Grand 6ccc759691 Merge pull request #15480 from jpountz/fix/mapping_explicit_defaults
Make mapping serialization more robust.
2015-12-17 11:23:27 +01:00
Robert Muir a7cc91e868 Merge pull request #15501 from rmuir/sheisty_classes
thirdPartyAudit round 2
2015-12-17 03:44:27 -05:00
Robert Muir 6692e42d9a thirdPartyAudit round 2
This fixes the `lenient` parameter to be `missingClasses`. I will remove this boolean and we can handle them via the normal whitelist.
It also adds a check for sheisty classes (jar hell with the jdk).
This is inspired by the lucene "sheisty" classes check, but it has false positives. This check is more evil, it validates every class file against the extension classloader as a resource, to see if it exists there. If so: jar hell.

This jar hell is a problem for several reasons:

1. causes insanely-hard-to-debug problems (like bugs in forbidden-apis)
2. hides problems (like internal api access)
3. the code you think is executing, is not really executing
4. security permissions are not what you think they are
5. brings in unnecessary dependencies
6. its jar hell

The more difficult problems are stuff like jython, where these classes are simply 'uberjared' directly in, so you cant just fix them by removing a bogus dependency. And there is a legit reason for them to do that, they want to support java 1.4.
2015-12-17 02:35:00 -05:00
Jack Conradson 4523eaec88 Added plumbing for compile time script parameters.
Closes #15464
2015-12-16 18:29:21 -08:00
Ryan Ernst a1312a5350 Merge pull request #15495 from rjernst/disable_mock_plugins
Allow integ tests to exclude mock plugins
2015-12-16 18:22:04 -08:00
Ryan Ernst 17a806631d Test: Allow integ tests to exclude mock plugins
We currently randomly add a set of mock plugins to integ tests.
Sometimes it is necessary to omit this mock plugins, but other times you
may just want to suppress a particular mock plugin. For example, if you
have your own transport, you want to omit the asserting local transport
mock, since they would both try to set the transport.type.
2015-12-16 17:59:43 -08:00
Jason Bryan 9a1133ca50 Fix typo in scroll.asciidoc
Fix scroll request with sort.

Closes #15493
2015-12-16 20:31:46 -05:00
Robert Muir 4f9d4103f2 Merge pull request #15491 from rmuir/forbidden_third_party
Add gradle thirdPartyAudit to precommit tasks
2015-12-16 18:56:50 -05:00
Robert Muir 42138007db add some more comments about internal api usage 2015-12-16 18:56:02 -05:00
Jason Tedor 7e14245c69 Add callback for publication of new cluster state
This commit adds a callback for a cluster state task executor that will
be invoked if the execution of a batch of cluster state update tasks
led to a new cluster state and that new cluster state was successfully
published.

Closes #15482
2015-12-16 18:43:31 -05:00
Michael McCandless 207f0a4eda use async translog for this test 2015-12-16 17:53:55 -05:00
Jason Tedor 12e241ff1a Ensure compressed oops flag is written as a string 2015-12-16 17:29:08 -05:00
Jason Tedor cfc46da18e Merge pull request #15489 from jasontedor/compressed-oops
Info on compressed ordinary object pointers
2015-12-16 17:00:14 -05:00
Jason Tedor 9aa41d092e Info on compressed ordinary object pointers
This commit adds to JvmInfo the status of whether or not compressed
ordinary object pointers are enabled. Additionally, logging of the max
heap size and the status of the compressed ordinary object pointers flag
are provided on startup.

Relates #13187, relates elastic/elasticsearch-definitive-guide#455
2015-12-16 16:59:41 -05:00
Robert Muir 49f37a526c fix bad indent 2015-12-16 16:50:52 -05:00
Robert Muir ee79d46583 Add gradle thirdPartyAudit to precommit tasks 2015-12-16 16:38:16 -05:00
Jason Tedor b4fbe1e1b8 Merge pull request #15486 from jasontedor/fix-delimited-payload-token-filter-docs
Fix minor issues in delimited payload token filter docs
2015-12-16 15:26:15 -05:00
Ryan Ernst c46482b7f6 Merge pull request #15467 from rjernst/organize_gitignore
Organize and annotate gitignores
2015-12-16 11:57:31 -08:00
Ryan Ernst 67e0b5190a Remove old docs gitignores and comment on docs build dir 2015-12-16 11:56:55 -08:00
Ryan Ernst a2b8f4b90a Merge pull request #15434 from rjernst/http_type
Expose http.type setting, and collapse al(most all) modules relating to transport/http
2015-12-16 11:54:30 -08:00
Simon Willnauer 71b204ea49 Merge branch 'master' into settings_prototype 2015-12-16 20:29:21 +01:00
Simon Willnauer 161cabbef4 organize tests 2015-12-16 20:14:00 +01:00
Simon Willnauer 45b7afe14f more tests 2015-12-16 20:06:43 +01:00
Jason Tedor d9a24961c5 Fix minor issues in delimited payload token filter docs
This commit addresses a few minor issues in the delimited payload token
filter docs:
  - the provided example reversed the payloads associated with the
    tokens "the" and "fox"
  - two additional typos in the same sentence
    - "per default" -> "by default"
    - "default int to" -> "default into"
  - adds two serial commas
2015-12-16 13:00:20 -05:00
Jason Tedor 709740efd2 Merge pull request #14993 from jasontedor/tribe-node-cluster-state-batch
Tribe nodes should apply cluster state updates in batches
2015-12-16 12:10:10 -05:00
Jason Tedor 53b3cd83a5 Tribe nodes should apply cluster state updates in batches
This commit applies the general mechanism for applying cluster state
updates in batches to tribe nodes.

Relates #14899, relates #14725
2015-12-16 12:08:27 -05:00
Yannick Welsch 6b49d728ec Merge pull request #15476 from ywelsch/fix/script-service-tests
[TEST] Fix ScriptServiceTests.testFineGrainedSettings that can loop indefinitely
2015-12-16 18:05:55 +01:00
Yannick Welsch 88a0104687 [TEST] ScriptServiceTests only test single ScriptEngineService 2015-12-16 17:36:14 +01:00
Britta Weber db357f078a Merge pull request #15396 from brwe/java-api-for-synced-flush
Add java API for synced flush

closes #12812
2015-12-16 17:18:18 +01:00
javanna 99f9bd7cfe Throw exception when trying to write map with null keys
Closes #14346
2015-12-16 17:01:04 +01:00
Jason Tedor 89c960b00a Merge pull request #15428 from jasontedor/cluster-state-batch-logging
Explicitly log cluster state update failures
2015-12-16 10:55:14 -05:00
Jason Tedor 02354fc491 More useful assertions in o.e.c.s.InternalClusterService 2015-12-16 10:53:59 -05:00
Jason Tedor 3e8768f9ee Merge pull request #15468 from jasontedor/master-side-of-wait-on-shard-failures
Master should wait on cluster state publication when failing a shard
2015-12-16 10:39:38 -05:00
Jason Tedor 89a09b9bed Master should wait on cluster state publication when failing a shard
When a client sends a request to fail a shard to the master, the current
behavior is that the master will submit the cluster state update task
and then immediately send a successful response back to the client;
additionally, if there are any failures while processing the cluster
state update task to fail the shard, then the client will never be
notified of these failures.

This commit modifies the master behavior when handling requests to fail
a shard. In particular, the master will now wait until successful
publication of the cluster state update before notifying the request
client that the shard is marked as failed; additionally, the client is
now notified of any failures during the execution of the cluster state
update task.

Relates #14252
2015-12-16 10:37:51 -05:00
Adrien Grand 8ac8c1f547 Make mapping serialization more robust.
When creating a metadata mapper for a new type, we reuse an existing
configuration from an existing type (if any) in order to avoid introducing
conflicts. However this field type that is provided is considered as both an
initial configuration and the default configuration. So at serialization time,
we might only serialize the difference between the current configuration and
this default configuration, which might be different to what is actually
considered the default configuration.

This does not cause bugs today because metadata mappers usually override the
toXContent method and compare the current field type with Defaults.FIELD_TYPE
instead of defaultFieldType() but I would still like to do this change to
avoid future bugs.
2015-12-16 16:08:45 +01:00
Simon Willnauer c1ff619781 fix cluster settings to accept complex list settigns 2015-12-16 14:13:22 +01:00
Jason Tedor 44467df353 Merge pull request #15456 from jasontedor/reorganize-shard-state-action
Reorganize o/e/c/a/s/ShardStateAction.java
2015-12-16 07:37:10 -05:00
Clinton Gormley 4597a22ace Merge pull request #15473 from jmluy/patch-1
Update sample in sort for consistency
2015-12-16 12:53:09 +01:00
Simon Willnauer e83c728265 Merge pull request #15475 from s1monw/beef_up_translog_tests
Beef up TranslogTests with concurrent fatal exceptions test
2015-12-16 12:43:56 +01:00
Simon Willnauer e5dc124811 apply review from @bleskes 2015-12-16 12:27:07 +01:00
Jason Tedor e084e50edb Merge pull request #15461 from jasontedor/cleanup-routing-nodes
Cleanup o/e/c/r/RoutingNodes.java
2015-12-16 06:21:01 -05:00
Simon Willnauer e586c966be fix several .getKey usages 2015-12-16 12:08:48 +01:00
Simon Willnauer 32881c08b3 fix more excpption useage 2015-12-16 12:00:34 +01:00
Simon Willnauer 9cf2f42d15 fix list/array settings 2015-12-16 11:52:30 +01:00
Yannick Welsch 20dc855644 [TEST] Fix ScriptServiceTests.testFineGrainedSettings that can loop indefinitely 2015-12-16 11:27:45 +01:00
Britta Weber 7a469538bc serialize os name, arch and version too
These three properties are build in the jason response but were not
transported when a node sends the response.

closes #15422
2015-12-16 11:24:11 +01:00
Simon Willnauer d0dbfce49a allow -1 for concurrent rebalance, it's a special value :( 2015-12-16 10:38:19 +01:00