3756 Commits

Author SHA1 Message Date
Simon Willnauer
e9e300c42e Add Javadocs for ThreadContext 2016-01-15 09:36:49 +01:00
Martijn van Groningen
a05ea535ad percolator: Make sure that start time is serialized on the mpercolate shard requests
Closes #15908
2016-01-15 09:25:01 +01:00
Jason Tedor
7f78d52c72 Use capture and clear convenience method 2016-01-14 16:51:36 -05:00
Lee Hinman
03087cfc47 Skip capturing least/most FS info for an FS with no total
If an operating system reports -1 for the total bytes of a filesystem
path, we should ignore it when capturing the least and most available
statistics.

Relates to #15919

Squashed commit of the following:

commit 5d2258ffeff8a0d156295dcc754ab9b6cbb4b02e
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Jan 14 14:14:27 2016 -0700

    Change test to test positive total with negative 'free' value

commit 927e61d4b39692fc147220a955b63b291ad80db5
Author: Lee Hinman <lee@writequit.org>
Date:   Thu Jan 14 13:09:28 2016 -0700

    Skip capturing least/most FS info for an FS with no total

    If an operating system reports -1 for the total bytes of a filesystem
    path, we should ignore it when capturing the least and most available
    statistics.

    Relates to #15919
2016-01-14 14:44:46 -07:00
Simon Willnauer
2008a5bdc9 apply review comments 2016-01-14 22:19:36 +01:00
Simon Willnauer
903d0ff96c Use trace for annoying schedule logs 2016-01-14 22:09:45 +01:00
Jason Tedor
8f67dcc348 Add Javadocs for exceptions that are handled by ShardStateAction 2016-01-14 15:07:17 -05:00
Jason Tedor
5a5d7881ed Remove dead field in o.e.c.a.s.ShardStateActionTests 2016-01-14 15:07:17 -05:00
Jason Tedor
d55c5f6c4d Handle FailedToCommitClusterStateException in ShardStateAction
This commit adds Discovery.FailedToCommitClusterStateException to the
list of channel failures that ShardStateAction handles and retries.
2016-01-14 15:07:17 -05:00
Jason Tedor
f17f9a5f36 Remove timeout mechanism from ShardStateAction
This commit removes the timeout retry mechanism from ShardStateAction
allowing it to instead be handled by the general master channel retry
mechanism. The idea is that if there is a network issue, the master will
miss a ping timeout causing the channel to be closed which will expose
itself via a NodeDisconnectedException. At this point, we can just wait
for a new master and retry, as with any other master channel exception.
2016-01-14 15:07:16 -05:00
Jason Tedor
f49435c78b Centrally handle channel failures when failing a shard
This commit moves the handling of channel failures when failing a shard
to o.e.c.a.s.ShardStateAction. This means that shard failure requests
that timeout or occur when there is no master or the master leaves after
the request is sent will now be retried from here. The listener for a
shard failed request will now only be notified upon successful
completion of the shard failed request, or when a catastrophic
non-channel failure occurs.
2016-01-14 15:07:16 -05:00
Jason Tedor
58c2a3b687 Add simulation of master leaving in TransportReplicationActionTests
This commit adds a simulation of the master leaving after a shard
failure request has been sent. In this case, after a new cluster state
is published (simulating a new master having been elected), the request
to fail the shard should be retried.
2016-01-14 15:05:46 -05:00
Jason Tedor
703ff2b5ab Wait for new master when failing shard
This commit handles the situation when we are failing a shard and either
no master is known, or the known master left while failing the shard. We
handle this situation by waiting for a new master to be reelected, and
then sending the shard failed request to the new master.
2016-01-14 15:05:46 -05:00
javanna
57b88076f7 [TEST] remove thread pool tests that check whether the ingest tp was started
The ingest thread pool has been removed, no reason to test it ;)
2016-01-14 17:32:09 +01:00
Luca Cavanna
7ed4bbdf64 Merge pull request #15991 from javanna/enhancement/plugin_todos
move grok patterns loading out of GrokProcessorFactory
2016-01-14 17:25:30 +01:00
Martijn van Groningen
63ee2224f7 ingest: remove ingest threadpool and use index threadpool instead.
To main concern with the dedicated ingest TP is that there are already many TPs and in the case with beefy nodes we would many more threads. In the case ingest isn't used the all these threads are just idle.
2016-01-14 16:33:14 +01:00
Jason Tedor
a954e4e8e5 Safely notify cluster state task executor of publish 2016-01-14 10:07:08 -05:00
javanna
1e68ad0887 add ingest-geoip to official plugins 2016-01-14 15:50:59 +01:00
Christoph Büscher
3d98756e64 Merge branch 'refactor/rescore-builder-equals-hash' 2016-01-14 15:40:34 +01:00
Christoph Büscher
76192024a8 Make RescoreBuilder and nested QueryRescorer Writable
Adding serialization capabilities to RescoreBuilder and make
all QueryRescorer implement NamedWritable, also requiring
all implementations of RescoreBuilder.Rescorer to implement
equals() and hashCode.

In addition, the current rescore mode enumeration is pulled out to a
separate class to make sharing of constants easier between
the query builders XContent rendering coder and the parser.
2016-01-14 15:26:18 +01:00
Luca Cavanna
1cb1607c29 Merge pull request #15954 from javanna/enhancement/get_alias_expand_closed
make get alias expand to open and closed indices by default
2016-01-14 14:24:40 +01:00
Simon Willnauer
4bef475985 [TEST] Add test to show how to parse an enum from a string in ObjectParser.java 2016-01-14 14:20:33 +01:00
Martijn van Groningen
4d88da5ad5 ingest: Use BiConsumer instead of Cunsumer to pass down the failed index request with throwable 2016-01-14 14:09:42 +01:00
javanna
07a82d0c09 make get alias expand to open and closed indices by default
This change affects get alias, get aliases as well as cat aliases. They all return closed indices too by default. get alias and get aliases also allow to return open indices only through the `expand_wildcards` option (set it to `open`).

Closes #14982
2016-01-14 10:40:31 +01:00
Simon Willnauer
aaf7e55c5f Remove index.flush_on_close entirely
This undocumented setting was mainly used for testing and a safety net for
a new flushOnClose feature back in 1.x. We can now safely remove this setting.
The test usage now uses a mock plugin setting to achive the same.
2016-01-14 10:34:58 +01:00
Jason Tedor
d5e20fb763 Merge pull request #15934 from jasontedor/freebsd-load-average
Add load averages to OS stats on FreeBSD

Closes #15917
2016-01-13 20:11:22 -05:00
Simon Willnauer
1ddfa42c2e Merge pull request #15966 from s1monw/randomize_mock_plugins
Randomize which MockPlugins are used for IT
2016-01-13 23:58:49 +01:00
Simon Willnauer
22834bce55 fix possible race in IndexServiceTests#testBaseAsyncTask and add mock plugin SearchWithRandomIOExceptionsIT 2016-01-13 23:39:25 +01:00
Martijn van Groningen
a7730b05b2 Fix some todos 2016-01-13 23:35:41 +01:00
Simon Willnauer
e849674f5c add required plugin to PrimaryAllocationIT 2016-01-13 23:11:35 +01:00
Martijn van Groningen
f3883343cb Move the pipeline configuration from the dedicated index to the cluster state.
Closes #15842
2016-01-13 22:59:36 +01:00
Simon Willnauer
8c10616df8 Merge pull request #15964 from s1monw/flush_on_close_final
Remove updatability of `index.flush_on_close`
2016-01-13 22:55:26 +01:00
Jason Tedor
4b47c38922 Fix typo in comment in o/e/c/c/Cache.java 2016-01-13 15:01:59 -05:00
Simon Willnauer
24b0f1f025 Remove updateability of index.flush_on_close
`index.flush_on_close` is a test setting and doesn't need to be updateable.

Relates to #15955
2016-01-13 20:39:07 +01:00
Simon Willnauer
b333133183 Merge pull request #15955 from s1monw/move_settings_to_index_level
Move all dynamic settings and their config classes to the index level
2016-01-13 20:35:33 +01:00
Adrien Grand
aa52ab365a Merge pull request #15958 from jpountz/fix/remove_dead_fielddata_code
Remove dead fielddata code.
2016-01-13 18:33:35 +01:00
Britta Weber
87f0edbf51 Merge pull request #15952 from brwe/fix-cicular-reference-exception
Avoid circular reference in exception
2016-01-13 16:28:04 +01:00
Britta Weber
0c08d796c5 assert before we set the suppressed 2016-01-13 16:27:26 +01:00
Britta Weber
e0edb07cc6 disable testCreateAndDeleteIndexConcurrently, it is flaky and we have an issue for it 2016-01-13 16:21:58 +01:00
Adrien Grand
08d05b201e Remove dead fielddata code.
This became dead when we removed in-memory fielddata for numeric fields.
2016-01-13 16:09:50 +01:00
Martijn van Groningen
5e185befa8 Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-13 15:51:14 +01:00
Martijn van Groningen
a7a45cd941 test: ensure that no percolate request are executed when stopping a node 2016-01-13 15:45:28 +01:00
Nik Everett
b7cb04cbf4 Merge pull request #15809 from nik9000/unraw
Remove some raw from ActionRequest subclasses
2016-01-13 09:43:57 -05:00
Nik Everett
df44754605 Renames Self to Request in ActionRequests 2016-01-13 09:40:32 -05:00
Simon Willnauer
437d7c179a Move all dynamic settings and their config classes to the index level
Today we maintain a lot of settings on the shard level which are all index level settings.
In order to cut over to the new settings API where we register update listener we have to move
all of them on to the index level otherwise we need a way to un-register listeners which is error-prone
and requires additional handling when shards are closed. It's simpler and also more accurate to handle all of
them on the index level where we can trash the entire registry for update listener once the index goes out of scope.
2016-01-13 14:25:37 +01:00
Britta Weber
2c2264d8d0 Avoid circular reference in exception
Don't set the suppressed Exception in Translog.closeOnTragicEvent(Exception ex) if it is an
AlreadyClosedException. ACE is thrown by the TranslogWriter and as cause might
contain the Exception that we add the suppressed ACE to. We then end up with a
circular reference where Exception A has a suppressed Exception B that has as cause A.
This would cause a stackoverflow when we try to serialize it.
For a more detailed description see #15941

closes #15941
2016-01-13 13:56:04 +01:00
Simon Willnauer
47c6ba41d5 remove final keyword 2016-01-13 13:23:12 +01:00
Simon Willnauer
aa464778b1 fix compile errors 2016-01-13 12:16:26 +01:00
Simon Willnauer
ce42ae4cf3 Merge pull request #15922 from s1monw/feature/ingest
Review feedback and several cleanups
2016-01-13 12:04:04 +01:00
Simon Willnauer
574d1b35b3 Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation
ContextAndHeaders has a massive impact on the core infrastructure since it has to
be manually passed on to all relevant places across threads/network calls etc. For the same reason
it's also very error prone and easily forgotten on potentially relevant APIs.

The new ThreadContext is associated with a ThreadPool (node or transport client) and ensures that
headers and context registered on a current thread are inherited to new threads spawned, send across
the network to be deserialized on the receiver end as well as restored on the response handling thread
once the response is received.
2016-01-13 11:53:32 +01:00