Commit Graph

19362 Commits

Author SHA1 Message Date
Igor Motov 8fbb3686cd Improve stability of the testBatchingShardUpdateTask test
On slow machines when this test randomly picks a large number of shards it can occasionally take more than 32.5 seconds to snapshot all shards. That is causing the test to miss the second to last assert in awaitsBusy at 32.5 seconds and then timeout in BlockingClusterStateListener at 60 seconds. Due to the timeout, the pending task queue is cleaned before the last awaitsBusy assert at 65 seconds and as a result the last assert runs on a completely empty queue and fails with a very confusing assert error.

This commit makes the timeout in BlockingClusterStateListener to occur after the last assert in assertBusyPendingTasks and therefore allows assertBusyPendingTasks to perform the last assert before cleaning the pending tasks queue takes place.

 This commit also reduces the maximum number of shards used in the test to 10 in order to speed up this test.
2016-01-07 19:33:50 -05:00
Adrien Grand 3ef9ec25f8 BitSetFilterCache duplicates its content.
We have a bug that makes all per-index bitset caches store bitsets for all
indices. In the case that you have many indices, which is fairly common with
time-based data, this could translate to a lot of wasted memory.

Closes #15820
2016-01-07 18:50:14 +01:00
javanna adac314328 revert move of IngestPlugin class
This was moved accidentally as part of a previous refactoring.
2016-01-07 18:32:46 +01:00
Jason Tedor dcd8a8207f Merge pull request #15831 from jasontedor/global-search-timeout-docs
Add documentation for global search timeout

Relates #12211
2016-01-07 11:09:41 -05:00
Nik Everett 42cb2a5f02 Merge pull request #15811 from nik9000/def_cleanup
Cleanups for Def
2016-01-07 11:03:01 -05:00
Britta Weber bc5a6be0e3 Merge pull request #15832 from brwe/sync-translog-to-disk-after-recovery
sync translog to disk after recovery from primary
2016-01-07 16:54:56 +01:00
Jason Tedor 6243d375eb Add documentation for global search timeout
This commit adds documentation for the global search timeout setting
“search.default_search_timeout”.
2016-01-07 10:42:18 -05:00
Britta Weber f93b4cb215 sync translog to disk after recovery from primary
Otherwise if that node is shutdown and restarted it might will have lost all operations
that were in the translog.
2016-01-07 16:27:40 +01:00
javanna 79c1377661 [TEST] set node.ingest to true otherwise we may try to modify values for a threadpool that hasn't been started 2016-01-07 16:16:14 +01:00
Luca Cavanna e149704ba2 Merge pull request #15769 from javanna/enhancement/move_to_core
Move ingest api to core
2016-01-07 16:04:16 +01:00
javanna 03fe38681e renamed qa package o.e.plugin.ingest to o.e.ingest
This way InternalTemplateService constructor can be set back to package private visibility
2016-01-07 15:51:52 +01:00
javanna 694eeffe92 catch the right exception... 2016-01-07 15:36:34 +01:00
Martijn van Groningen 0d70d92aa0 Also ignore rejected execution exception when installing index template 2016-01-07 15:25:35 +01:00
Martijn van Groningen c44b83dc29 Also ignore rejected execution exception when retying restart 2016-01-07 15:21:21 +01:00
Adrien Grand 8bd54dbf5a Merge pull request #15828 from jpountz/enhancement/stricter_metadata_parsing
Make MetaData parsing less lenient.
2016-01-07 15:20:51 +01:00
Adrien Grand 6ce7a972bc Make MetaData parsing less lenient.
Today this simply ignores everything that is not recognized.
2016-01-07 15:20:16 +01:00
javanna 1ea690e814 Merge branch 'feature/ingest' into enhancement/move_to_core 2016-01-07 15:13:31 +01:00
javanna 7d7e0db91d Merge branch 'master' into feature/ingest 2016-01-07 15:09:13 +01:00
javanna 234371811d awaited fix a couple of index template tests till we have removed the ingest index template 2016-01-07 15:06:05 +01:00
Martijn van Groningen fa9aab91eb Ignore rejected execution exception 2016-01-07 14:52:15 +01:00
Martijn van Groningen ba97b7f7bd Don't activate ingest on tribe nodes 2016-01-07 14:36:23 +01:00
Nik Everett 52f28888d5 Merge pull request #15813 from nik9000/xlint1
Remove Xlint:-override,-fallthrough,-static
2016-01-07 08:34:40 -05:00
Nik Everett 439832d0b3 Merge pull request #15815 from nik9000/xlint2
Remove a few more Xlint skips
2016-01-07 08:34:28 -05:00
javanna eca1594969 start ingest thread pool only when node.ingest is set to true 2016-01-07 14:33:56 +01:00
Boaz Leskes d5e6eb58a8 Log uncaught exceptions from scheduled once tasks
`ScheduledThreadPoolExecutor` allows you to schedule tasks to run once or periodically at the future. If such a task throws an exception, that exception is caught and reported in the future that `ScheduledThreadPoolExecutor#schedule` returns. However, we typically do not capture the future / do not test it for errors. This results in exception being swallowed and not reported. To mitigate this we now wrap any command in a LoggingRunnable  (already used for periodic tasks).  Also, RunnableCommand is changed not to swallow exception but percolate them further for reporting by the future.

Closes #15824
2016-01-07 14:04:35 +01:00
Martijn van Groningen 0b06648205 template installation is async, so use assertBusy 2016-01-07 14:00:50 +01:00
javanna 0bfe6de75c move all transport actions under same package org.elasticsearch.action.ingest 2016-01-07 13:52:42 +01:00
Martijn van Groningen e6cc79a976 test: added ingest script context 2016-01-07 13:48:36 +01:00
javanna 95bc0ed7a2 move constants to IngestActionFilter 2016-01-07 13:40:28 +01:00
javanna 2803ae09dc addProcessor -> registerProcessor 2016-01-07 13:25:25 +01:00
javanna 18aabd67c8 adapt qa tests for when ingest.node is set to false
CRUD and simulate apis work now fine, every node has the pipelines in memory, but node.ingest disables ingestion, meaning that any index or bulk request with a pipeline id is going to fail
2016-01-07 13:21:06 +01:00
javanna 52c2a273f9 amended error message 2016-01-07 13:21:06 +01:00
Martijn van Groningen 7e56f65939 since ingest is part of core the ingest template always gets installed, so the assumptions this test had had to be updated 2016-01-07 12:58:13 +01:00
Martijn van Groningen 4dca3cb38f fix thread pools, use management the do the operation and response handeling 2016-01-07 12:14:13 +01:00
javanna c185c1339a remove grok and join usage from IngestClientIT 2016-01-07 11:30:02 +01:00
javanna e036b5896d move registerFilter up 2016-01-07 11:29:43 +01:00
Martijn van Groningen 9ec2e140b8 Merge branch 'master' into feature/ingest 2016-01-07 10:44:21 +01:00
Simon Willnauer e7f9d685f1 [TEST] Test that translog can recover after random IOException
This commit adds a new test that can throw an IOException at any point in time
and ensures that all previously synced documents can be successfully recovered after hitting
an excepiton.

Relates to #15788
2016-01-07 10:17:31 +01:00
Adrien Grand 132df10342 Merge pull request #15614 from jpountz/remove/warmers
Remove query warmers and the warmer API.
2016-01-07 09:59:14 +01:00
Adrien Grand 67d233cecd Remove warmers and the warmer API.
Warmers are now barely useful and will be removed in 3.0. Note that this only
removes the warmer API and query-based warmers. We still have warmers internally
for eg. global ordinals.

Close #15607
2016-01-07 09:57:07 +01:00
Martijn van Groningen 604d59a95e muted test 2016-01-07 09:54:59 +01:00
Nik Everett 244120a065 Remove more Xlint skips 2016-01-06 23:53:05 -05:00
Nik Everett 0786c506dc Remove a few more Xlint skips 2016-01-06 23:28:13 -05:00
Nik Everett 20e7fa97db Remove Xlint:-override,-fallthrough,-static
Adds `@SuppressWarnings("fallthrough")` in two places where the fallthrough
is used to implement well known hashing algorithms.
2016-01-06 22:27:14 -05:00
Nik Everett 1325c1442b Merge pull request #15812 from nik9000/varargs
Standardize some methods on varargs
2016-01-06 21:42:13 -05:00
Nik Everett 74c132afc6 Standardize some methods on varargs
Right now we define the same sort of methods as taking String arrays and
string varargs. We should standardize on one and varargs is easier to
call so lets use varargs!
2016-01-06 21:01:58 -05:00
Nik Everett 32605ecb4f Cleanups for Def
Manually I:
1. Added some missing raw types warnings suppressions.
2. Removed some unused unchecked cast warning suppressions.
3. Added <?> to Class.

I let my IDE:
1. Remove unneeded casts.
2. Reorder imports (just ignore these, everyone does).
2016-01-06 20:28:49 -05:00
Jason Tedor a583edb2df Merge pull request #15801 from jasontedor/cyclic-barriers-for-boaz
Use CyclicBarriers for sychronizing driver and test threads
2016-01-06 20:09:28 -05:00
Jason Tedor c147fe5691 Do not lose CacheTest failure stack traces 2016-01-06 20:00:11 -05:00
Nik Everett 91464c7687 Remove some raw from ActionRequest subclasses
Renames "T" used in many subclasses of ActionRequest to "Self" and tightens
the type bounds.
2016-01-06 19:01:58 -05:00