5ccfdcc48b
Fix testDeadlock timeout delay ( #3979 )
...
* No more singleton. Reduce iterations
* Granularities
* Fix the delay in the test
* Add license header
* Remove unused imports
* Lot more unused imports from all the rearranging
* CR feedback
* Move javadoc to constructor
2017-02-28 12:51:41 -06:00
c3bf40108d
One granularity ( #3850 )
...
* Refactor Segment Granularity
* Beginning of one granularity
* Copy the fix for custom periods in segment-grunalrity over here.
* Remove the custom serialization for now.
* Compilation cleanup
* Reformat code
* Fixing unit tests
* Unify to use a single iterable
* Backward compatibility for rolling upgrade
* Minor check style. Cosmetic changes.
* Rename length and millis to duration
* CR feedback
* Minor changes.
2017-02-25 01:02:29 -06:00
1f263fe50b
alert when resetting offsets ( #3931 )
...
* alert when resetting offsets
* add more data to alerts
2017-02-13 13:49:24 -08:00
8e31a465ad
report hand off count finite appenderator driver ( #3925 )
2017-02-13 10:41:24 -08:00
12317fd001
Bump version to 0.10.0-SNAPSHOT. ( #3913 )
2017-02-06 17:54:35 -08:00
1aabb45a09
auto reset option for Kafka Indexing service ( #3842 )
...
* auto reset option for Kafka Indexing service in case message at the offset being fetched is not present anymore at kafka brokers
* review comments
* review comments
* reverted last change
* review comments
* review comments
* fix typo
2017-02-02 14:57:45 -06:00
b3dae0efc3
catch all errors ( #3844 )
2017-01-24 18:01:30 -07:00
6440ddcbca
Fix #3795 (Java 7 compatibility). ( #3796 )
...
* Fix #3795 (Java 7 compatibility).
Also introduce Animal Sniffer checks during build, which would
have caught the original problems.
* Add Animal Sniffer on caffeine-cache for JDK8.
2016-12-21 10:19:13 -08:00
0b9dff0bc1
fix worker thread pool exhaustion bug ( #3760 )
...
* fix worker thread pool exhaustion bug
* code review changes
* code review changes
2016-12-09 15:23:11 -08:00
7f087cdd3b
allow Kafka consumer group.id to be overriden by config ( #3765 )
2016-12-08 15:53:13 -08:00
7ee6bb7410
option to reset offest automatically in case of OffsetOutOfRangeException ( #3678 )
...
* option to reset offset automatically in case of OffsetOutOfRangeException
if the next offset is less than the earliest available offset for that partition
* review comments
* refactoring
* refactor
* review comments
2016-11-21 16:29:46 -06:00
bcd20441be
Make buildV9Directly the default. ( #3688 )
2016-11-14 09:29:32 -08:00
4b3bd8bd63
Migrating java-util from Metamarkets. ( #3585 )
...
* Migrating java-util from Metamarkets.
* checkstyle and updated license on java-util files.
* Removed unused imports from whole project.
* cherry pick metamx/java-util@826021f .
* Copyright changes on java-util pom, address review comments.
2016-10-21 14:57:07 -07:00
c2ae734848
KafkaIndexTask: Allow run thread to stop gracefully instead of interrupting ( #3534 )
...
* allow run thread to gracefully complete instead of interrupting when stopGracefully() is called
* add comments
2016-10-17 10:52:19 -04:00
592903571a
add context to kafka supervisor for the kafka indexing task ( #3464 )
2016-10-04 20:08:43 -05:00
e419407eba
handle supervisor spec metadata failures ( #3456 )
...
close kafka consumer in case supervisor start fails
2016-10-04 10:15:28 -07:00
40f2fe7893
Bump versions to 0.9.3-SNAPSHOT ( #3524 )
2016-09-29 13:53:32 -07:00
15c9918c65
log exceptions while trying to pause task ( #3504 )
2016-09-23 16:53:23 -07:00
9226d4af3c
configurable shutdownTimeout for Kakfa supervisor ( #3497 )
...
* configurable shutdownTimeout
* cr change
2016-09-23 13:26:45 -06:00
ca9114b41b
add supervisor reset API ( #3484 )
...
* add supervisor reset API
* CR doc changes and kill running tasks / clear offsets from supervisor
2016-09-22 17:51:06 -07:00
05ea88df5c
fix kafka-indexing-service pom to not reference specific version but parent version for druid core dependencies ( #3472 )
2016-09-20 15:18:21 -07:00
96fcca18ea
update KafkaSupervisor to make HTTP requests to tasks in parallel where possible ( #3452 )
2016-09-20 22:51:15 +05:30
bcff08826b
KafkaIndexTask: Treat null values as unparseable. ( #3453 )
2016-09-13 10:56:38 -07:00
df766b2bbd
Add dimension handling interface for ingestion and segment creation ( #3217 )
...
* Add dimension handling interface for ingestion and segment creation
* update javadocs for DimensionHandler/DimensionIndexer
* Move IndexIO row validation into DimensionHandler
* Fix null column skipping in mergerV9
* Add deprecation note for 'numeric_dims' filename pattern in IndexIO v8->v9 conversion
* Fix java7 test failure
2016-09-12 12:54:02 -07:00
146a17de48
KafkaIndexTask: allow pause to break out of retry loop ( #3401 )
2016-09-06 22:29:37 -06:00
5b1ae21bd1
retry calls to getStartTime ( #3429 )
2016-09-06 14:02:22 -07:00
ed924bf214
allow registrants to opt out of announcing themselves when registering as a chat handler ( #3360 )
2016-08-16 10:51:28 +05:30
8030f1cb67
Be more respectful of maxRowsInMemory. ( #3284 )
...
- Appenderator: Respect maxRowsInMemory across all sinks.
- KafkaIndexTask: Respect maxRowsInMemory across all partitions.
2016-07-26 15:02:35 -06:00
1d40df4bb7
fix kafka consumer concurrent access during shutdown ( #3193 )
2016-06-28 13:23:17 -07:00
ebf890fe79
Update master version to 0.9.2-SNAPSHOT. ( #3133 )
2016-06-13 13:10:38 -07:00
4faa298977
update kafka client for kafka indexing service to 0.9.0.1 ( #3109 )
2016-06-08 06:51:03 -07:00
a2290a8f05
support seamless config changes ( #3051 )
2016-06-03 13:50:19 -07:00
f6c39cc844
Kafka task minimum message time ( #3035 )
...
* add KafkaIndexTask support for minimumMessageTime
* add Kafka supervisor support for lateMessageRejectionPeriod
2016-05-31 11:37:00 -07:00
3ef24c03b3
Validate X-Druid-Task-Id header in request/response and support retrying on outdated TaskLocation information, add KafkaIndexTaskClient unit tests ( #3006 )
...
* validate X-Druid-Task-Id header in request and add header to response
* modify KafkaIndexTaskClient to take a TaskLocationProvider as the TaskLocation may not remain constant
2016-05-25 22:05:18 -07:00
15ccf451f9
Move QueryGranularity static fields to QueryGranularities ( #2980 )
...
* Move QueryGranularity static fields to QueryGranularityUtil
* Fixes #2979
* Add test showing #2979
* change name to QueryGranularities
2016-05-17 16:23:48 -07:00
b489f63698
Supervisor for KafkaIndexTask ( #2656 )
...
* supervisor for kafka indexing tasks
* cr changes
2016-05-04 23:13:13 -07:00
f8ddfb9a4b
Split SegmentInsertAction and SegmentTransactionalInsertAction for backwards compat. ( #2922 )
...
Fixes #2912 .
2016-05-04 13:54:34 -07:00
7641f2628f
add control and status endpoints to KafkaIndexTask ( #2730 )
2016-04-21 15:34:59 -07:00
08c784fbf6
KafkaIndexTask: Use a separate sequence per Kafka partition in order to make ( #2844 )
...
segment creation deterministic.
This means that each segment will contain data from just one Kafka
partition. So, users will probably not want to have a super high number
of Kafka partitions...
Fixes #2703 .
2016-04-18 22:29:52 -07:00
7e7a886f65
Move druid-api into the druid repo.
...
This is from druid-api-0.3.17, as of commit 51884f1d05d5512cacaf62cedfbb28c6ab2535cf
in the druid-api repo.
2016-03-24 11:04:34 -07:00
a59c9ee1b1
Support use of DimensionSchema class in DimensionsSpec
2016-03-21 13:12:04 -07:00
738dcd8cd9
Update version to 0.9.1-SNAPSHOT.
...
Fixes #2462
2016-03-17 10:34:20 -07:00
f22fb2c2cf
KafkaIndexTask.
...
Reads a specific offset range from specific partitions, and can use dataSource metadata
transactions to guarantee exactly-once ingestion.
Each task has a finite lifecycle, so it is expected that some process will be supervising
existing tasks and creating new ones when needed.
2016-03-10 18:41:43 -08:00
praveev
Parag Jain
Gian Merlino
David Lim
Akash Dwivedi
Himanshu
Jonathan Wei
Charles Allen
jon-wei