725086cc89
Merge pull request #1506 from gianm/realtime-plumber-nulls
...
Consider null inputRows and parse errors as unparseable during realtime ingestion.
2015-07-13 10:12:12 -05:00
9068bcd062
Consider null inputRows and parse errors as unparseable during realtime ingestion.
...
Also, harmonize exception handling between the RealtimeIndexTask and the RealtimeManager.
Conditions other than null inputRows and parse errors bubble up in both.
2015-07-11 20:40:03 -07:00
cac722968e
Merge pull request #1503 from metamx/fix-leaking-zk-nodes
...
Fix leaking Status Path nodes in ZK
2015-07-10 17:40:18 -05:00
9f19e96658
Merge pull request #1477 from pjain1/overlord_test
...
overlord and task master test
2015-07-10 14:27:14 -07:00
55c4fe64f3
overlord and task master test
2015-07-10 16:17:45 -05:00
8d7a566bae
Fix leaking Status Path nodes in ZK
...
- remove ZK status path nodes for workers after they are removed
2015-07-09 17:20:09 +05:30
c0b60c0d2f
I'm not your mom, indexing-service/test... cleanup after yourself
2015-07-01 15:00:09 -07:00
b5b9ca1446
Merge pull request #1470 from pjain1/rtindex_test
...
Realtime Index Task test
2015-06-29 16:51:35 -05:00
284b80b09e
Realtime Index Task test
2015-06-29 09:52:41 -05:00
4a83a22f8c
Merge pull request #1445 from metamx/JSWorkerSelectStrategy
...
JavaScript Worker Select Strategy
2015-06-22 17:19:13 -05:00
fb4052d577
JavaScript Worker Select Strategy
...
this PR adds a JavaScriptWorkerSelectStrategy which allows defining
arbitrary logic for selecting workers to run task using a JavaScript
function.
This gives users full control to implement complex worker selection
strategies based on task attributes.
more tests and a complex javascript config
fix for java8 modify for nashorn compatibility
2015-06-20 02:01:34 +05:30
0a5bb909a2
[maven-release-plugin] prepare for next development iteration
2015-06-18 17:35:19 -07:00
59c6b2b279
[maven-release-plugin] prepare release druid-0.8.0-rc1
2015-06-18 17:35:14 -07:00
acc0a3fbf7
Add jitter to the retries for RemoteTaskActionClient
2015-06-12 17:43:25 -07:00
e9afec4a2b
fix task status issues on zk outages
...
docs
review comments
fix test
review comments
Review comments
fix compilation
fix typo
2015-06-11 00:49:52 +05:30
78d468700b
Merge pull request #1388 from metamx/fix-1360
...
fix race described in 1360
2015-06-10 11:59:36 -07:00
f6b336ac3e
Merge pull request #1432 from metamx/config-fix
...
fix passing of config from IndexTuningConfig to RealtimeTuningConfig
2015-06-10 11:42:58 -07:00
963682d696
Add check for valid rowFlushBoundary configuration and fix tests
2015-06-10 21:38:34 +05:30
191b302f6a
fix passing of config from IndexTuningConfig to RealtimeTuningConfig
...
- pass rowFlushboundary correctly instead of using default.
- fixes indexTask failing with
io.druid.segment.incremental.IndexSizeExceededException when
rowFlushboundary is set higher than
RealtimeTuningConfig.defaultMaxRowsInMemory
rename test method
2015-06-10 21:07:25 +05:30
af9ea08041
fix race described in 1360
...
review comments
review comments
review comments
no need to remove
fix test
review comments
2015-06-10 12:19:12 +05:30
056cab93ed
Add Hadoop Converter Job and task
...
* Fixes https://github.com/druid-io/druid/issues/1363
* Add extra utils in JobHelper based on PR feedback
2015-06-09 14:47:38 -07:00
ef9b67cce3
Merge pull request #1422 from metamx/fix-ec2-public-ip
...
fix public IP not working in EC2 autoscaling
2015-06-03 16:30:51 -07:00
4ebdfea76f
fix public IP not working in EC2 autoscaling
2015-06-03 16:05:59 -07:00
8289914f76
Make AbstractTask.makeId use AbstractTask.joinId
...
* Also remove TaskUtil
2015-06-03 13:24:20 -07:00
ac9057c00e
Merge pull request #1401 from metamx/ec2-public-ip
...
flag to enable public IP in EC2 autoscaling
2015-05-28 20:21:32 -07:00
d834a974ba
flag to enable public IP in EC2-VPC autoscaling
2015-05-28 18:14:12 -07:00
bb1145ef56
Make the index task use indexmerger and not indexmaker
2015-05-28 13:34:57 -07:00
5ad5d7d18b
Merge pull request #1379 from flowbehappy/fix-hadoop-ha
...
bug fix: hdfs task log and indexing task not work properly with Hadoop HA
2015-05-22 09:14:50 -04:00
07659f30ab
bug fix: hdfs task log and indexing task not work properly with Hadoop HA
2015-05-21 20:49:42 +08:00
29ba05c04f
Abstractify HadoopTask
...
* Add `invokeForeignLoader` to commonize the way tasks are attempted to be launched in a foreign class loader
* Add `buildClassLoader` to accomplish the common tasks for hadoop jobs when building a ClassLoader
2015-05-14 17:04:43 -07:00
7a6acf5c1b
update pom to 0.8
2015-05-11 19:41:58 -06:00
e69d82a2b4
Realtime: Delay firehose connection until job is started.
...
Some firehoses (like the Kafka firehose) acquire input resources when they
connect, so it helps to delay this until after plumber.startJob() runs.
2015-05-04 10:54:07 -07:00
721505c017
Merge pull request #1208 from druid-io/rework-metrics
...
Schemaless metrics + additional metrics for things we care about
2015-04-27 15:04:54 -07:00
963e5765bf
Schemaless metrics + additional metrics for things we care about
2015-04-27 13:39:40 -07:00
633fdb029e
Add option to ConvertSegmentTask to skip validation
...
* Validation is enabled by default
2015-04-27 08:37:55 -07:00
29341f9837
Fix random unit test failure from NoopTask ID collision
2015-04-24 13:07:48 -07:00
f73f14ab91
Merge pull request #1297 from metamx/versionConverterTaskUpdates
...
Update VersionConverterTask for IndexSpec and allowing Forced updates
2015-04-20 16:44:35 -07:00
7479ac9012
Update VersionConverterTask for IndexSepc and allowing Forced updates
2015-04-20 16:17:06 -07:00
d260515a43
update druid-api version
2015-04-17 14:58:35 -07:00
ea5572d001
Merge pull request #1271 from metamx/strictErrorChecking
...
Add stricter checking for potential coding errors
2015-04-15 15:21:41 -07:00
abdeaa0746
Add stricter checking for potential coding errors
...
Can use via `mvn clean compile test-compile -P strict'
2015-04-15 14:52:25 -07:00
3a3046ccf3
add support for dimension compression
...
- compression for single-value dimensions using CompressedVSizeIntsIndexedSupplier
- makes dimension compression configurable via IndexSpec
- IndexSpec also enables configuring bitmap and metric compression
2015-04-14 10:44:18 -07:00
195a3b8bb8
ignore rows with invalid interval
2015-04-06 16:08:40 -07:00
208e307915
Merge pull request #1251 from metamx/uriSegmentLoaders
...
Revert "Revert "Overhaul of SegmentPullers to add consistency and retries""
2015-03-30 17:43:51 -07:00
aea7f9d192
[maven-release-plugin] prepare for next development iteration
2015-03-30 16:35:24 -07:00
060d7aef03
[maven-release-plugin] prepare release druid-0.7.1
2015-03-30 16:35:20 -07:00
1c6cbea89c
Revert "Revert "Overhaul of SegmentPullers to add consistency and retries""
...
This reverts commit f904bc7858
2015-03-30 13:40:04 -07:00
f904bc7858
Revert "Overhaul of SegmentPullers to add consistency and retries"
2015-03-30 13:15:50 -07:00
6d407e8677
Add URI handling to SegmentPullers
...
* Requires https://github.com/druid-io/druid-api/pull/37
* Requires https://github.com/metamx/java-util/pull/22
* Moves the puller logic to use a more standard workflow going through java-util helpers instead of re-writing the handlers for each impl
* General workflow goes like this: 1) LoadSpec makes sure the correct Puller is called with the correct parameters. 2) The Puller sets up general information like how to make an InputStream, how to find a file name (for .gz files for example), and when to retry. 3) CompressionUtils does most of the heavy lifting when it can
2015-03-30 12:33:23 -07:00
942c17a2aa
Remove timeline chunk count assumptions.
...
* Replace with generic iterables
2015-03-24 22:40:49 +01:00
Himanshu
Gian Merlino
Fangjin Yang
Parag Jain
Nishant
Charles Allen
Xavier Léauté
Xavier Léauté
flow
msprunck