323 Commits

Author SHA1 Message Date
Simon Willnauer
eb498282dd Add infrastructure to wrap/filter all clients exposed by the test clusters.
This commit allows an integ test to wrap all clients that are exposed by the
InternalTestCluster which is useful for request interception or to add general headers
to request or to intercept client to server call even if the client calls are done from within
the test framework.
2016-01-19 20:12:47 +01:00
Nik Everett
0d6d77328d Merge branch 'master' into feature/reindex 2016-01-19 13:27:24 -05:00
Simon Willnauer
fa0e1488aa Merge branch 'master' into new_index_settings 2016-01-19 10:50:11 +01:00
Jason Tedor
296ab3ec16 Fix transport exceptions for shard state actions
This commit fixes an issue in the handling of TransportExceptions in
ShardStateAction. There were two cases not being handled correctly.
 - when the local node is shutting down, handlers will be notified with
   a TransportException with a message starting "transport stopped"
 - when the remote node disconnects, handlers will be notified with a
   NodeDisconnectedException

In both of these cases, the cause of the exception will be null and this
was incorrectly being handled. The first case can passed to the listener
like any other critical non-channel failure, and the second case can be
handled by modifying the logic for detecting master channel exceptions.

There was a third case of NodeNotConnectedException that was not being
treated as a master channel exception but should be.

This commit adds an integration test that simulates the handling of a
shard failure request during a network partition. By isolating the
master from the cluster while a shard failed request is in flight, this
test simulates that we wait until a new master is elected and then retry
sending that shard failed request to the newly elected master.

This commit adds methods to CapturingTransport to separate local and
remote transport exceptions. The motivation for this change is that
local transport exceptions are delivered to listeners (usually, but not
always) wrapped in SendRequestTransportException while remote transport
exceptions are delivered to listeners wrapped in
RemoteTransportException. By making this distinction clear in the
CapturingTransport, this makes it less likely that tests will make
incorrect assumptions about the exceptions coming out of the transport
layer to listeners.

Closes #16057
2016-01-19 10:43:50 +01:00
Simon Willnauer
fbfa9f4925 Merge branch 'master' into new_index_settings 2016-01-19 10:13:48 +01:00
Simon Willnauer
b73c737608 apply feedback from @nik9000 2016-01-19 09:54:00 +01:00
Ryan Ernst
ef4f0a8699 Test: Make rest test framework accept http directly for the test cluster
The rest test framework, because it used to be tightly integrated with
ESIntegTestCase, currently expects the addresses for the test cluster to
be passed using the transport protocol port. However, it only uses this
to then find the http address.

This change makes ESRestTestCase extend from ESTestCase instead of
ESIntegTestCase, and changes the sysprop used to tests.rest.cluster,
which now takes the http address.

closes #15459
2016-01-18 16:44:14 -08:00
Nik Everett
093600cd63 Cleanup ContextAndHeaderTransportTests
We have two similar tests with the same name, ContextAndHeaderTransportTests.
They shared lots of common code so I extracted much of it into
ActionRecordingPlugin, a plugin which records all action requests for later
inspection.

I also removed all the warnings from both tests. That made lang-mustache
compile cleanly without any custom -Xlint so I removed those. To remove
the warnings I had to add type parameters to ActionFilter which seemed
like a good idea anyway.
2016-01-18 16:14:58 -05:00
Simon Willnauer
c2e8ba8970 Register index.version_created on unittest that set it 2016-01-18 15:04:35 +01:00
Simon Willnauer
dce29d29c0 Normalize template index settings before validating 2016-01-18 14:39:21 +01:00
Martijn van Groningen
62089539db s/processorId/processorTag 2016-01-18 13:11:46 +01:00
Simon Willnauer
9562fb76bc expose default settings via rest API 2016-01-18 12:48:47 +01:00
Simon Willnauer
7925e2ef84 convert IndexModule settings 2016-01-18 09:23:35 +01:00
Simon Willnauer
79f4697f3e Register MockFSDirectoryService settings 2016-01-18 09:23:34 +01:00
Simon Willnauer
a8eedd0457 fix all kinds of crazy test failures 2016-01-18 09:23:34 +01:00
Simon Willnauer
8a4fca2947 make all unittests pass 2016-01-18 09:23:33 +01:00
Simon Willnauer
04045a072f First steps of integrating converted settings into the cluster/index settings infrastructure
The old infa has been removed in this commit such that nothing uses `DynamicSettings` anymore
and all index-scoped settings require to be registered before the node has fully started up.
2016-01-18 09:23:33 +01:00
Simon Willnauer
211f80fcad first cut at integrating new settings infra 2016-01-18 09:23:33 +01:00
Simon Willnauer
abfef7aa73 convert index allocation filtering 2016-01-18 09:23:31 +01:00
Britta Weber
a9ff6bc405 convert compound_format 2016-01-18 09:23:31 +01:00
Britta Weber
c4a019d93d convert translog.flush_threshold_size 2016-01-18 09:23:31 +01:00
Simon Willnauer
2e1bc1ea44 cut over index.routing.rebalance.enable and index.routing.allocation.enable 2016-01-18 09:23:30 +01:00
Simon Willnauer
0bef73269a cut over index.unassigned.node_left.delayed_timeout 2016-01-18 09:23:30 +01:00
Simon Willnauer
b7b104a585 convert index.translog.durability 2016-01-18 09:23:29 +01:00
Simon Willnauer
da0424c75f Convert index level setting to the new setting infrastrucutre
this is an initial commit of cutting over simple string key based settings
to a more contained scoped settings infrastructure.
2016-01-18 09:23:29 +01:00
Ryan Ernst
3b78267c71 Plugins: Remove site plugins
Site plugins used to be used for things like kibana and marvel, but
there is no longer a need since kibana (and marvel as a kibana plugin)
uses node.js. This change removes site plugins, as well as the flag for
jvm plugins. Now all plugins are jvm plugins.
2016-01-16 22:45:37 -08:00
Tal Levy
9f48df9736 Add on_failure support for verbose _simulate execution and introduce optional processor_tag to Processors 2016-01-15 14:56:20 -08:00
Nik Everett
645b053061 Merge branch 'master' into feature/reindex 2016-01-15 14:04:37 -05:00
javanna
9c06736dbd Merge branch 'master' into feature/ingest 2016-01-15 10:11:56 +01:00
Colin Goodheart-Smithe
e6f9cbce8f Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamInput.java
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java
2016-01-14 15:00:26 +00:00
Simon Willnauer
aaf7e55c5f Remove index.flush_on_close entirely
This undocumented setting was mainly used for testing and a safety net for
a new flushOnClose feature back in 1.x. We can now safely remove this setting.
The test usage now uses a mock plugin setting to achive the same.
2016-01-14 10:34:58 +01:00
Simon Willnauer
4526f117a2 Randomize which MockPlugins are used for IT 2016-01-13 23:00:59 +01:00
Simon Willnauer
437d7c179a Move all dynamic settings and their config classes to the index level
Today we maintain a lot of settings on the shard level which are all index level settings.
In order to cut over to the new settings API where we register update listener we have to move
all of them on to the index level otherwise we need a way to un-register listeners which is error-prone
and requires additional handling when shards are closed. It's simpler and also more accurate to handle all of
them on the index level where we can trash the entire registry for update listener once the index goes out of scope.
2016-01-13 14:25:37 +01:00
Simon Willnauer
574d1b35b3 Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation
ContextAndHeaders has a massive impact on the core infrastructure since it has to
be manually passed on to all relevant places across threads/network calls etc. For the same reason
it's also very error prone and easily forgotten on potentially relevant APIs.

The new ThreadContext is associated with a ThreadPool (node or transport client) and ensures that
headers and context registered on a current thread are inherited to new threads spawned, send across
the network to be deserialized on the receiver end as well as restored on the response handling thread
once the response is received.
2016-01-13 11:53:32 +01:00
javanna
ea8065aa3d Merge branch 'master' into feature/ingest 2016-01-12 18:28:42 +01:00
Nik Everett
78dc1c32de Merge pull request #15899 from nik9000/xlint_serial
Ban Serializable
2016-01-12 10:04:50 -05:00
Jason Tedor
ed723fbe5a Merge pull request #15929 from jasontedor/capturing-transport-clear
Add convenience method for capturing and clearing requests

Closes #15897
2016-01-12 09:44:23 -05:00
Jason Tedor
c19981c67b Add convenience method for capturing and clearing requests
This commit adds convenience methods to o.e.t.t.CapturingTransport
that enables capturing requests and clearing the captured requests
with a single method. This is to simplify a common pattern in tests of
capturing requests, and then clearing the captured requests.
2016-01-12 09:20:54 -05:00
Colin Goodheart-Smithe
f429cb08d7 Merge branch 'master' into feature/aggs-refactoring 2016-01-12 12:59:52 +00:00
Martijn van Groningen
7bdd2583aa Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-12 01:01:30 +01:00
Nik Everett
01ce49e94e Ban Serializable
1. Uses forbidden patterns to prevent things from referencing
java.io.Serializable or from mentioning serialVersionUID.
2. Uses -Xlint:-serial so we don't have to hear from javac that we aren't
declaring serialVersionUID on any classes that we make that happen to extend
Serializable.
3. Remove Serializable and serialVersionUID declarations.

I didn't use forbidden apis because it doesn't look like it has a way to ban
explicitly implementing Serializable. If you try to ban Serializable with
forbidden apis you end up banning all Exceptions and all Strings.

Closes #15847
2016-01-11 16:57:31 -05:00
Nik Everett
b29c416b7c Merge pull request #15848 from nik9000/xlint3
Remove -Xlint:-deprecation from all but core
2016-01-11 16:55:13 -05:00
Colin Goodheart-Smithe
9d7d89d04d Merge branch 'master' into feature/aggs-refactoring 2016-01-11 12:50:31 +00:00
Jason Tedor
abaf816d00 Remove and forbid use of IndexWriter#isLocked
This commit removes and now forbids use of
org.apache.lucene.index.IndexWriter#isLocked as this method was
deprecated in LUCENE-6508. The deprecation is due to the fact that
checking if a lock is held before acquiring that lock is subject to a
time-of-check-to-time-of-use race condition. There were three uses of
IndexWriter#isLocked in the code base:
 - a logging statement in o.e.i.e.InternalEngine where we are already in
   an exceptional condition that the lock was held; in this case,
   logging whether or not the directory is locked is superfluous
 - in o.e.c.l.u.VersionsTests where we were verifying that a write lock
   is released upon closing an IndexWriter; in this case, the check is
   not needed as successfully closing an IndexWriter releases its
   write lock
 - in o.e.t.s.MockFSDirectoryService where we were verifying that a
   directory is not write-locked before (implicitly) trying to obtain
   such a write lock in org.apache.lucene.index.CheckIndex#<init> (this
   is the exact type of a situation that is subject to a race
   condition); in this case we can proceed by just (implicitly) trying
   to obtain the write lock and failing if we encounter a
   LockObtainFailedException
2016-01-10 08:28:44 -05:00
Nik Everett
00edc8bc87 Remove -Xlint:-deprecated from test/framwork
There were a few uses of deprecation I couldn't clear up quickly so I filed
issues for them and suppressed the warnings.
2016-01-07 21:22:20 -05:00
javanna
1ea690e814 Merge branch 'feature/ingest' into enhancement/move_to_core 2016-01-07 15:13:31 +01:00
Colin Goodheart-Smithe
3b7d1b47f7 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/percolator/PercolatorService.java
#	core/src/main/java/org/elasticsearch/percolator/QueryCollector.java
2016-01-07 12:42:02 +00:00
Nik Everett
20e7fa97db Remove Xlint:-override,-fallthrough,-static
Adds `@SuppressWarnings("fallthrough")` in two places where the fallthrough
is used to implement well known hashing algorithms.
2016-01-06 22:27:14 -05:00
javanna
9079a7e891 wip: move all the ingest infra to core 2016-01-06 19:10:44 +01:00
javanna
456351de2e move common test classes to test module 2016-01-06 17:31:39 +01:00