Himanshu Gupta
4d4aa8bfc6
refactor IngestSegmentFirehoseFactory so that IngestSegmentFirehose becomes reusable
...
Conflicts:
indexing-service/src/main/java/io/druid/indexing/firehose/IngestSegmentFirehoseFactory.java
2015-08-14 14:44:22 -05:00
Gian Merlino
bc0c7dd65d
Avoid the Hadoop objectMapper in the local IndexTask. Fixes #1545 .
2015-08-11 10:40:53 -07:00
Charles Allen
1ddaa3fb33
Merge pull request #1592 from metamx/clean-test-files
...
clean temporary files
2015-08-03 11:47:20 -07:00
Nishant
2679efee7a
clean temporary files
2015-08-03 23:32:58 +05:30
Fangjin Yang
6f65e6d3ef
Merge pull request #1547 from pjain1/improve_overlord_test
...
add test to OverlordResourceTest
2015-07-28 07:35:48 -10:00
Parag Jain
2e1b617346
add more tests
2015-07-24 15:12:08 -05:00
Fangjin Yang
97242356b4
Merge pull request #1480 from guobingkun/kill_task_test
...
Unit tests for KillTask and MetadataTaskStorage
2015-07-20 16:31:45 -07:00
Xavier Léauté
4cfb00bc8a
inrement version
2015-07-15 13:09:05 -07:00
Fangjin Yang
3f7ba58227
Merge pull request #1504 from metamx/fix-1447
...
fix for #1447
2015-07-14 08:50:08 -07:00
Himanshu
e2ddfb7a1a
Merge pull request #1511 from pjain1/remove_test
...
remove flaky overlord test
2015-07-13 18:38:34 -05:00
Parag Jain
59dec89f6a
remove flaky overlord test
2015-07-13 15:32:12 -05:00
Himanshu
725086cc89
Merge pull request #1506 from gianm/realtime-plumber-nulls
...
Consider null inputRows and parse errors as unparseable during realtime ingestion.
2015-07-13 10:12:12 -05:00
Gian Merlino
9068bcd062
Consider null inputRows and parse errors as unparseable during realtime ingestion.
...
Also, harmonize exception handling between the RealtimeIndexTask and the RealtimeManager.
Conditions other than null inputRows and parse errors bubble up in both.
2015-07-11 20:40:03 -07:00
Himanshu
cac722968e
Merge pull request #1503 from metamx/fix-leaking-zk-nodes
...
Fix leaking Status Path nodes in ZK
2015-07-10 17:40:18 -05:00
Fangjin Yang
9f19e96658
Merge pull request #1477 from pjain1/overlord_test
...
overlord and task master test
2015-07-10 14:27:14 -07:00
Parag Jain
55c4fe64f3
overlord and task master test
2015-07-10 16:17:45 -05:00
Nishant
5fe27fe4ad
fix for #1447
...
fixes #1447
2015-07-09 19:05:48 +05:30
Nishant
8d7a566bae
Fix leaking Status Path nodes in ZK
...
- remove ZK status path nodes for workers after they are removed
2015-07-09 17:20:09 +05:30
Charles Allen
c0b60c0d2f
I'm not your mom, indexing-service/test... cleanup after yourself
2015-07-01 15:00:09 -07:00
Bingkun Guo
282a0f9760
Unit tests for KillTask and MetadataTaskStorage
2015-06-29 17:55:41 -05:00
Himanshu
b5b9ca1446
Merge pull request #1470 from pjain1/rtindex_test
...
Realtime Index Task test
2015-06-29 16:51:35 -05:00
Parag Jain
284b80b09e
Realtime Index Task test
2015-06-29 09:52:41 -05:00
Himanshu
4a83a22f8c
Merge pull request #1445 from metamx/JSWorkerSelectStrategy
...
JavaScript Worker Select Strategy
2015-06-22 17:19:13 -05:00
nishant
fb4052d577
JavaScript Worker Select Strategy
...
this PR adds a JavaScriptWorkerSelectStrategy which allows defining
arbitrary logic for selecting workers to run task using a JavaScript
function.
This gives users full control to implement complex worker selection
strategies based on task attributes.
more tests and a complex javascript config
fix for java8 modify for nashorn compatibility
2015-06-20 02:01:34 +05:30
Xavier Léauté
0a5bb909a2
[maven-release-plugin] prepare for next development iteration
2015-06-18 17:35:19 -07:00
Xavier Léauté
59c6b2b279
[maven-release-plugin] prepare release druid-0.8.0-rc1
2015-06-18 17:35:14 -07:00
Charles Allen
acc0a3fbf7
Add jitter to the retries for RemoteTaskActionClient
2015-06-12 17:43:25 -07:00
nishant
e9afec4a2b
fix task status issues on zk outages
...
docs
review comments
fix test
review comments
Review comments
fix compilation
fix typo
2015-06-11 00:49:52 +05:30
Xavier Léauté
78d468700b
Merge pull request #1388 from metamx/fix-1360
...
fix race described in 1360
2015-06-10 11:59:36 -07:00
Xavier Léauté
f6b336ac3e
Merge pull request #1432 from metamx/config-fix
...
fix passing of config from IndexTuningConfig to RealtimeTuningConfig
2015-06-10 11:42:58 -07:00
nishant
963682d696
Add check for valid rowFlushBoundary configuration and fix tests
2015-06-10 21:38:34 +05:30
nishant
191b302f6a
fix passing of config from IndexTuningConfig to RealtimeTuningConfig
...
- pass rowFlushboundary correctly instead of using default.
- fixes indexTask failing with
io.druid.segment.incremental.IndexSizeExceededException when
rowFlushboundary is set higher than
RealtimeTuningConfig.defaultMaxRowsInMemory
rename test method
2015-06-10 21:07:25 +05:30
nishant
af9ea08041
fix race described in 1360
...
review comments
review comments
review comments
no need to remove
fix test
review comments
2015-06-10 12:19:12 +05:30
Charles Allen
056cab93ed
Add Hadoop Converter Job and task
...
* Fixes https://github.com/druid-io/druid/issues/1363
* Add extra utils in JobHelper based on PR feedback
2015-06-09 14:47:38 -07:00
Charles Allen
ef9b67cce3
Merge pull request #1422 from metamx/fix-ec2-public-ip
...
fix public IP not working in EC2 autoscaling
2015-06-03 16:30:51 -07:00
Xavier Léauté
4ebdfea76f
fix public IP not working in EC2 autoscaling
2015-06-03 16:05:59 -07:00
Charles Allen
8289914f76
Make AbstractTask.makeId use AbstractTask.joinId
...
* Also remove TaskUtil
2015-06-03 13:24:20 -07:00
Fangjin Yang
ac9057c00e
Merge pull request #1401 from metamx/ec2-public-ip
...
flag to enable public IP in EC2 autoscaling
2015-05-28 20:21:32 -07:00
Xavier Léauté
d834a974ba
flag to enable public IP in EC2-VPC autoscaling
2015-05-28 18:14:12 -07:00
fjy
bb1145ef56
Make the index task use indexmerger and not indexmaker
2015-05-28 13:34:57 -07:00
Xavier Léauté
5ad5d7d18b
Merge pull request #1379 from flowbehappy/fix-hadoop-ha
...
bug fix: hdfs task log and indexing task not work properly with Hadoop HA
2015-05-22 09:14:50 -04:00
flow
07659f30ab
bug fix: hdfs task log and indexing task not work properly with Hadoop HA
2015-05-21 20:49:42 +08:00
Charles Allen
29ba05c04f
Abstractify HadoopTask
...
* Add `invokeForeignLoader` to commonize the way tasks are attempted to be launched in a foreign class loader
* Add `buildClassLoader` to accomplish the common tasks for hadoop jobs when building a ClassLoader
2015-05-14 17:04:43 -07:00
fjy
7a6acf5c1b
update pom to 0.8
2015-05-11 19:41:58 -06:00
Gian Merlino
e69d82a2b4
Realtime: Delay firehose connection until job is started.
...
Some firehoses (like the Kafka firehose) acquire input resources when they
connect, so it helps to delay this until after plumber.startJob() runs.
2015-05-04 10:54:07 -07:00
Xavier Léauté
721505c017
Merge pull request #1208 from druid-io/rework-metrics
...
Schemaless metrics + additional metrics for things we care about
2015-04-27 15:04:54 -07:00
fjy
963e5765bf
Schemaless metrics + additional metrics for things we care about
2015-04-27 13:39:40 -07:00
Charles Allen
633fdb029e
Add option to ConvertSegmentTask to skip validation
...
* Validation is enabled by default
2015-04-27 08:37:55 -07:00
Charles Allen
29341f9837
Fix random unit test failure from NoopTask ID collision
2015-04-24 13:07:48 -07:00
Xavier Léauté
f73f14ab91
Merge pull request #1297 from metamx/versionConverterTaskUpdates
...
Update VersionConverterTask for IndexSpec and allowing Forced updates
2015-04-20 16:44:35 -07:00
