Commit Graph

18837 Commits

Author SHA1 Message Date
Martijn van Groningen cd2155311f renamed ingest plugin to ingest-geoip plugin, since it only contains the geoip processor 2016-01-08 22:50:54 +01:00
Martijn van Groningen 1637fe9e0b Moved the grok processor to its own module, so that it will available out-of-the-box, while its dependencies are isolated 2016-01-08 21:59:23 +01:00
Robert Muir 46a8a48d23 Merge pull request #15851 from rmuir/geoip-nodns
[ingest] Don't do DNS lookups from GeoIpProcessor
2016-01-08 13:17:20 -05:00
javanna 3e08c24991 clarify comment in TemplateService, explain why we have this class 2016-01-08 14:31:57 +01:00
javanna de2eac4c49 removed leftover jodatime dependency 2016-01-08 14:31:35 +01:00
Martijn van Groningen b9dd99636d dynamic option is a string, not a boolean 2016-01-08 11:36:52 +01:00
Luca Cavanna c8aeb11fd6 Merge pull request #15837 from javanna/enhancement/move_processors_to_core
Move ingest rest spec and tests to core with basic processors
2016-01-08 11:28:55 +01:00
javanna 8675784e3f adapt error message when node.ingest is set to false and we fail executing a pipeline 2016-01-08 10:39:39 +01:00
javanna ae69d46f92 move processors that have no deps to core, also move to core rest spec and tests and set node.inget to true by default 2016-01-08 10:39:39 +01:00
Martijn van Groningen e35e9bd736 Removed index template and instead create `.ingest` with settings and mapping when we find out that the index is missing during the put pipeline call. 2016-01-08 10:35:52 +01:00
Robert Muir 9c3ebb83a7 Don't do DNS lookups from GeoIpProcessor
There is no need to involve DNS in this!
2016-01-07 23:12:44 -05:00
javanna adac314328 revert move of IngestPlugin class
This was moved accidentally as part of a previous refactoring.
2016-01-07 18:32:46 +01:00
javanna 79c1377661 [TEST] set node.ingest to true otherwise we may try to modify values for a threadpool that hasn't been started 2016-01-07 16:16:14 +01:00
Luca Cavanna e149704ba2 Merge pull request #15769 from javanna/enhancement/move_to_core
Move ingest api to core
2016-01-07 16:04:16 +01:00
javanna 03fe38681e renamed qa package o.e.plugin.ingest to o.e.ingest
This way InternalTemplateService constructor can be set back to package private visibility
2016-01-07 15:51:52 +01:00
javanna 694eeffe92 catch the right exception... 2016-01-07 15:36:34 +01:00
Martijn van Groningen 0d70d92aa0 Also ignore rejected execution exception when installing index template 2016-01-07 15:25:35 +01:00
Martijn van Groningen c44b83dc29 Also ignore rejected execution exception when retying restart 2016-01-07 15:21:21 +01:00
javanna 1ea690e814 Merge branch 'feature/ingest' into enhancement/move_to_core 2016-01-07 15:13:31 +01:00
javanna 7d7e0db91d Merge branch 'master' into feature/ingest 2016-01-07 15:09:13 +01:00
javanna 234371811d awaited fix a couple of index template tests till we have removed the ingest index template 2016-01-07 15:06:05 +01:00
Martijn van Groningen fa9aab91eb Ignore rejected execution exception 2016-01-07 14:52:15 +01:00
Martijn van Groningen ba97b7f7bd Don't activate ingest on tribe nodes 2016-01-07 14:36:23 +01:00
Nik Everett 52f28888d5 Merge pull request #15813 from nik9000/xlint1
Remove Xlint:-override,-fallthrough,-static
2016-01-07 08:34:40 -05:00
Nik Everett 439832d0b3 Merge pull request #15815 from nik9000/xlint2
Remove a few more Xlint skips
2016-01-07 08:34:28 -05:00
javanna eca1594969 start ingest thread pool only when node.ingest is set to true 2016-01-07 14:33:56 +01:00
Boaz Leskes d5e6eb58a8 Log uncaught exceptions from scheduled once tasks
`ScheduledThreadPoolExecutor` allows you to schedule tasks to run once or periodically at the future. If such a task throws an exception, that exception is caught and reported in the future that `ScheduledThreadPoolExecutor#schedule` returns. However, we typically do not capture the future / do not test it for errors. This results in exception being swallowed and not reported. To mitigate this we now wrap any command in a LoggingRunnable  (already used for periodic tasks).  Also, RunnableCommand is changed not to swallow exception but percolate them further for reporting by the future.

Closes #15824
2016-01-07 14:04:35 +01:00
Martijn van Groningen 0b06648205 template installation is async, so use assertBusy 2016-01-07 14:00:50 +01:00
javanna 0bfe6de75c move all transport actions under same package org.elasticsearch.action.ingest 2016-01-07 13:52:42 +01:00
Martijn van Groningen e6cc79a976 test: added ingest script context 2016-01-07 13:48:36 +01:00
javanna 95bc0ed7a2 move constants to IngestActionFilter 2016-01-07 13:40:28 +01:00
javanna 2803ae09dc addProcessor -> registerProcessor 2016-01-07 13:25:25 +01:00
javanna 18aabd67c8 adapt qa tests for when ingest.node is set to false
CRUD and simulate apis work now fine, every node has the pipelines in memory, but node.ingest disables ingestion, meaning that any index or bulk request with a pipeline id is going to fail
2016-01-07 13:21:06 +01:00
javanna 52c2a273f9 amended error message 2016-01-07 13:21:06 +01:00
Martijn van Groningen 7e56f65939 since ingest is part of core the ingest template always gets installed, so the assumptions this test had had to be updated 2016-01-07 12:58:13 +01:00
Martijn van Groningen 4dca3cb38f fix thread pools, use management the do the operation and response handeling 2016-01-07 12:14:13 +01:00
javanna c185c1339a remove grok and join usage from IngestClientIT 2016-01-07 11:30:02 +01:00
javanna e036b5896d move registerFilter up 2016-01-07 11:29:43 +01:00
Martijn van Groningen 9ec2e140b8 Merge branch 'master' into feature/ingest 2016-01-07 10:44:21 +01:00
Simon Willnauer e7f9d685f1 [TEST] Test that translog can recover after random IOException
This commit adds a new test that can throw an IOException at any point in time
and ensures that all previously synced documents can be successfully recovered after hitting
an excepiton.

Relates to #15788
2016-01-07 10:17:31 +01:00
Adrien Grand 132df10342 Merge pull request #15614 from jpountz/remove/warmers
Remove query warmers and the warmer API.
2016-01-07 09:59:14 +01:00
Adrien Grand 67d233cecd Remove warmers and the warmer API.
Warmers are now barely useful and will be removed in 3.0. Note that this only
removes the warmer API and query-based warmers. We still have warmers internally
for eg. global ordinals.

Close #15607
2016-01-07 09:57:07 +01:00
Martijn van Groningen 604d59a95e muted test 2016-01-07 09:54:59 +01:00
Nik Everett 244120a065 Remove more Xlint skips 2016-01-06 23:53:05 -05:00
Nik Everett 0786c506dc Remove a few more Xlint skips 2016-01-06 23:28:13 -05:00
Nik Everett 20e7fa97db Remove Xlint:-override,-fallthrough,-static
Adds `@SuppressWarnings("fallthrough")` in two places where the fallthrough
is used to implement well known hashing algorithms.
2016-01-06 22:27:14 -05:00
Nik Everett 1325c1442b Merge pull request #15812 from nik9000/varargs
Standardize some methods on varargs
2016-01-06 21:42:13 -05:00
Nik Everett 74c132afc6 Standardize some methods on varargs
Right now we define the same sort of methods as taking String arrays and
string varargs. We should standardize on one and varargs is easier to
call so lets use varargs!
2016-01-06 21:01:58 -05:00
Jason Tedor a583edb2df Merge pull request #15801 from jasontedor/cyclic-barriers-for-boaz
Use CyclicBarriers for sychronizing driver and test threads
2016-01-06 20:09:28 -05:00
Jason Tedor c147fe5691 Do not lose CacheTest failure stack traces 2016-01-06 20:00:11 -05:00