Commit Graph

19287 Commits

Author SHA1 Message Date
Simon Willnauer 054465287e Don't guard IndexShard#refresh calls by a check to isRefreshNeeded
While conceptually correct this call can be confusing since if a
realtime GET is exectued after refresh is called with realtime=true it might
fetch the wrong document out of the translog since the isRefreshNeeded guard
might return false once the new searcher is published but it will not
wait until the verision map is flushed and that can cause a slight race condition
if a subsequent call GETs a document that it expects to come from the index.

Ie. the `_size` mapper tests do this and expecte the GET call to return the document
from the index but it comes from the t-log due to this race.

This change only uses the guard in an async refresh that we schedule to prevent unnecessary
refresh calls but will allow API Refresh calls to have the somewhat less confusing semantics.

This was introduces lately only on unrelease major version branches.
2016-01-20 11:42:04 +01:00
Martijn van Groningen e383e96f58 replaced custom PipelineMissingException with ResourceNotFoundException 2016-01-20 11:40:17 +01:00
Robert Muir ebe3fd2825 Merge pull request #16114 from rmuir/lucene_1725675
Update lucene to r1725675
2016-01-20 05:04:01 -05:00
Yannick Welsch 1eb8d5bd6f Prefer nodes that previously held primary shard for primary shard allocation 2016-01-20 11:02:19 +01:00
Simon Willnauer f5b68cd0c0 Merge pull request #16101 from s1monw/client_wrapper_for_test_cluster
Add infrastructure to wrap/filter all clients exposed by the test clusters
2016-01-20 09:34:14 +01:00
Robert Muir 6e7e3a2274 Update lucene to r1725675
Adds DFI (divergence from independence) provider.
Fixes test bugs passing invalid values for BM25 parameters.
2016-01-20 03:32:51 -05:00
Robert Muir 00bc239eec Merge pull request #16104 from rmuir/nosys
set -Djna.nosys when running tests.
2016-01-19 22:07:24 -05:00
Nik Everett 9e3e024358 Cleanup the raw generics in ActionFilter
Mostly these were pretty easy to clean up by insisting that the request
and response stays consistent across the filter. There are a few places
where we have to make assumptions in tests but those are valid assumptions
for the test.
2016-01-19 21:53:47 -05:00
Robert Muir 79cc3c2dd0 set -Djna.nosys when running tests.
This is consistent with what happens in elasticsearch.sh/.bat, and it means
tests will work even if there is a crazy "system" JNA installed on the machine.
2016-01-19 18:49:47 -05:00
Nik Everett 3178d24bea Fix tests I just broke 2016-01-19 18:27:52 -05:00
Nik Everett bd68af1dee Merge pull request #16103 from nik9000/standalone_xlint
Use our standard xlint with standalone-test
2016-01-19 18:03:30 -05:00
Nik Everett 2cb7e8ce76 Use our standard xlint with standalone-test
We were not changing the xlint settings there at all. Also cleans up some
generic array warnings that this found by switching them to an ArrayList.
2016-01-19 17:54:47 -05:00
Martijn van Groningen e2e207687d Fixes due to changes in master branch. 2016-01-19 22:03:26 +01:00
Martijn van Groningen 602a0f183e Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-19 22:01:38 +01:00
Christoph Büscher df5f1b1ee9 Fix compilation issue with final assignment in lambda on windows jdk
Relates to #16097
2016-01-19 20:25:46 +01:00
Simon Willnauer eb498282dd Add infrastructure to wrap/filter all clients exposed by the test clusters.
This commit allows an integ test to wrap all clients that are exposed by the
InternalTestCluster which is useful for request interception or to add general headers
to request or to intercept client to server call even if the client calls are done from within
the test framework.
2016-01-19 20:12:47 +01:00
Nik Everett 5c88164364 Merge pull request #16099 from nik9000/default_for_executableScript_unwrap
Create default for ExecutableScript#unwrap
2016-01-19 14:04:56 -05:00
Nik Everett e1e73d9914 Create default for ExecutableScript#unwrap
Tons of scripts just return the variable they are passed and that is the
most intuitive behavior so that may as well be the default implementation.
2016-01-19 13:17:11 -05:00
Jack Conradson 2249a640bb Plan A Update...
New Features:

Exceptions (throw, try/catch)
Infinite Loop Checking
Iterators added to the API

Fixes:

Improved loop path analysis
Fixed set/add issue with shortcuts in lists
Fixed minor promotion issue with incorrect type
Fixed score issue related to score extending Number

Documentation:

Added JavaDocs for some files
2016-01-19 09:49:03 -08:00
Simon Willnauer da699ceae2 Merge pull request #16095 from s1monw/use_updated_settings_for_transport_client
TransportClient should use updated setting for initialization of modules and service
2016-01-19 18:03:47 +01:00
David Pilato 725d7f968b Merge branch 'fix/12567-azure-timeout' 2016-01-19 18:02:33 +01:00
David Pilato 6db5b5033c Change exception message for wrong timeout in azure repository settings
Backport in master this change: https://github.com/elastic/elasticsearch/pull/15950#discussion-diff-50128378

Related to #15080
Related to #15950
2016-01-19 18:01:54 +01:00
Simon Willnauer efece285d3 TransportClient should use updated setting for initialization of modules and service
Today the TransportClient uses the given settings rather than the updated setting from the plugin
service to pass on to it's modules etc. It should use the updates settings instead.
2016-01-19 17:35:38 +01:00
Jason Tedor 98f9222fff Log master node when sending shard state action 2016-01-19 11:31:13 -05:00
David Pilato d6e6ef4ec4 Merge branch 'pr/tests-azure-repo' 2016-01-19 17:07:59 +01:00
Nik Everett 6b6d01c4fe Merge pull request #16072 from nik9000/xlint_plugins
Remove remaining xlints from plugins
2016-01-19 10:54:15 -05:00
Nik Everett 4efa8c4ff5 Remove remaining xlints from plugins 2016-01-19 10:53:48 -05:00
Jim Ferenczi b9f1c7242c Add explanation about minimum should match and coordination factor 2016-01-19 16:04:09 +01:00
Simon Willnauer 2d4d1a943b Merge pull request #16091 from s1monw/check_node_level_settings
Validate known global settings on startup
2016-01-19 16:01:53 +01:00
Simon Willnauer bc5d6cdff5 Validate known global settings on startup
Today we already validate all index level setting on startup. For global
settings we are not fully there yet since not all settings are registered.
Yet we can already validate the ones that are know if their values are parseable/correct.
This is better than nothing and an improvement to what we had before. Over time there will
be more an dmore setting converted and we can finally flip the switch.
2016-01-19 15:25:19 +01:00
Jim Ferenczi 12df1e7a6f Merge pull request #16078 from jimferenczi/mss_synonym
Do not apply minimum-should-match on a boolean query if the coords are disabled
2016-01-19 15:07:32 +01:00
Jim Ferenczi 9b1daff9b5 Do not apply minimum-should-match on a boolean query if the coords are disabled.
Fixes #15858
2016-01-19 14:50:01 +01:00
Jason Tedor d2ad2e8eee Fix typo in scroll timeout example
Closes #16075
2016-01-19 08:42:29 -05:00
David Pilato 8d35bca4db Add more tests for Azure Repository client selection
One test we forgot in #14843 and #13779 is the default client selection.

Most of the time, users won't define explicitly which client they want to use because they are providing only one connection to Azure storage:

```yml
cloud:
    azure:
        storage:
            my_account:
                account: your_azure_storage_account
                key: your_azure_storage_key
```

Then using the default client like this:

```sh
# This one will use the default account (my_account1)
curl -XPUT localhost:9200/_snapshot/my_backup1?pretty -d '{
  "type": "azure"
}'
```

This commit adds tests to check that the right client is still selected when no client is explicitly set when creating the snapshot.
2016-01-19 14:23:08 +01:00
David Pilato 11e7a746ae Replace server side timeout by client side timeout
This commit replaces server side timeout (which is BTW not correctly implemented in azure client 2.0.0 but fixed later #16084) with a client side timeout.

As a consequence, for each request sent to azure, azure client will raise an exception after a given amount of time (timeout).

Closes #12567
2016-01-19 13:56:08 +01:00
Isabel Drost-Fromm e7aae1b79e Merge pull request #16085 from MaineC/bug-fix/random-fuzziness-date
Fix for MatchQueryBuilderTests.testToQuery random fuzziness generation caused test failure.

Relates to #13284
2016-01-19 13:26:44 +01:00
Jason Tedor 089a7a321c Merge pull request #16041 from jasontedor/shard-started-channel-failures
Add handling of channel failures when starting a shard
2016-01-19 07:05:45 -05:00
Jason Tedor b7c1847c41 Add logging statement on shard state request send
This commit restores a debug-level logging statement when sending shard
state requests to the master node.
2016-01-19 06:59:42 -05:00
Isabel Drost-Fromm 3adeac149d Fix random fuzziness generation.
For all query testing we offer the option to initialise them with random
fuzziness objects. So far there is a chance to generate an edit distance based
fuzziness for non-string fields. This is fixed by this change.

Fixes the following test failure:

http://build-us-00.elastic.co/job/es_core_master_small/6606/testReport/junit/org.elasticsearch.index.query/MatchQueryBuilderTests/testToQuery/
2016-01-19 12:45:02 +01:00
Simon Willnauer 5a64f08f04 Merge pull request #16054 from s1monw/new_index_settings
Cut over all index scope settings to the new setting infrastrucuture

This change moves all index.* settings over to the new infrastructure. This means in short that:

 - every setting that has an index scope must be registered up-front
 - index settings are validated on index creation, template creation, index settings update
 - node level settings starting with index.* are validated on node startup
 - settings that are private to ES like index.version.created can only be set by tests when they install a specific test plugin.
 - all index settings can be reset by passing null as their value on update
 - all index settings defaults can be listed via the settings APIs

Closes #12854
Closes #6732
Closes #16032
Closes #12790
2016-01-19 12:26:46 +01:00
Yannick Welsch 0a69f1e283 Merge pull request #15708 from ywelsch/feature/reroute-stale-primary
Extend reroute with an option to force assign stale primary shard copies
2016-01-19 12:26:02 +01:00
Simon Willnauer bd37ba22e3 Merge branch 'master' into new_index_settings 2016-01-19 12:11:57 +01:00
Yannick Welsch d5b691b68e Extend reroute with an option to force assign stale primary shard copies
Closes #15708
2016-01-19 12:07:01 +01:00
Rachit Gupta 5b2ded5c96 Fix typo in doc values docs
Closes #16067
2016-01-19 05:58:39 -05:00
Jason Tedor ac237b269c Add handling of channel failures when starting a shard
This commit adds handling of channel failures when starting a shard to
o.e.c.a.s.ShardStateAction. This means that shard started requests
that timeout or occur when there is no master or the master leaves
after the request is sent will now be retried from here. The listener
for a shard state request will now only be notified upon successful
completion of the shard state request, or when a catastrophic
non-channel failure occurs.

This commit also refactors the handling of shard failure requests so
that the two shard state actions of shard failure and shard started
now share the same channel-retry and notification logic.
2016-01-19 05:31:26 -05:00
Yannick Welsch 6ca79095df Allow changing nested fields limit for open indices 2016-01-19 10:55:11 +01:00
Simon Willnauer fa0e1488aa Merge branch 'master' into new_index_settings 2016-01-19 10:50:11 +01:00
Simon Willnauer f2311fd8b3 register index.mapping.nested_fields.limit 2016-01-19 10:49:49 +01:00
Jason Tedor 296ab3ec16 Fix transport exceptions for shard state actions
This commit fixes an issue in the handling of TransportExceptions in
ShardStateAction. There were two cases not being handled correctly.
 - when the local node is shutting down, handlers will be notified with
   a TransportException with a message starting "transport stopped"
 - when the remote node disconnects, handlers will be notified with a
   NodeDisconnectedException

In both of these cases, the cause of the exception will be null and this
was incorrectly being handled. The first case can passed to the listener
like any other critical non-channel failure, and the second case can be
handled by modifying the logic for detecting master channel exceptions.

There was a third case of NodeNotConnectedException that was not being
treated as a master channel exception but should be.

This commit adds an integration test that simulates the handling of a
shard failure request during a network partition. By isolating the
master from the cluster while a shard failed request is in flight, this
test simulates that we wait until a new master is elected and then retry
sending that shard failed request to the newly elected master.

This commit adds methods to CapturingTransport to separate local and
remote transport exceptions. The motivation for this change is that
local transport exceptions are delivered to listeners (usually, but not
always) wrapped in SendRequestTransportException while remote transport
exceptions are delivered to listeners wrapped in
RemoteTransportException. By making this distinction clear in the
CapturingTransport, this makes it less likely that tests will make
incorrect assumptions about the exceptions coming out of the transport
layer to listeners.

Closes #16057
2016-01-19 10:43:50 +01:00
Simon Willnauer 0066fc9d5f Merge branch 'master' into new_index_settings 2016-01-19 10:42:25 +01:00