0402636598
configurable handoffConditionTimeout in realtime tasks for segment handoff wait
2016-03-05 10:14:54 -06:00
4fa08a1329
Merge pull request #2592 from himanshug/inc_index_storage_adapter_fix
...
fix regression - reintroducing the safety check removed in commit-1d602be
2016-03-04 12:31:28 -08:00
2ad134638d
Merge pull request #2589 from b-slim/fix_real_time
...
Make realtime kafka firehose skip corrupt message
2016-03-04 12:14:23 -08:00
623e89aa54
skip corrupt message
2016-03-04 08:30:40 -06:00
c544ebf25e
reintroducing the safety check removed in commit-1d602be so that dim value ids are less than cardinality
2016-03-03 23:34:23 -06:00
9fbcd86930
Merge pull request #2590 from jisookim0513/master
...
fix default number of connections on broker config documentation
2016-03-03 15:08:52 -08:00
177b575d41
fix default number of connections on broker config documentation
2016-03-03 13:50:48 -08:00
703dc7a48f
Merge pull request #2585 from gianm/overlord-workers-api
...
OverlordResource: Use getZkWorkers on RemoteTaskRunner.
2016-03-02 18:15:05 -08:00
d06c1c5c85
Merge pull request #2583 from guobingkun/fix_multiple_specs_2
...
update querySegmentSpec when passing query to getQueryRunner
2016-03-02 18:05:34 -08:00
e9c23bf376
OverlordResource: Use getZkWorkers on RemoteTaskRunner.
...
Restores old behavior of this api, from before #2249 when getWorkers returned ZkWorkers.
2016-03-02 17:31:34 -08:00
9e74772d6b
Merge pull request #2574 from gianm/allostuff
...
Make first few allocatePendingSegment retries quiet.
2016-03-02 16:16:53 -07:00
cfe2dbf1eb
Merge pull request #2580 from gianm/rtc-basePersist
...
RealtimeTuningConfig: Use different default basePersistDirectory per instance.
2016-03-02 16:56:49 -06:00
4a58462fc7
update querySegmentSpec when passing query to getQueryRunner
...
After finding the FireChief for a specific partition, Druid will need to find the specific queryRunner for each segment being queried by passing the query to FireChief. Currently Druid is passing the original query that contains all the segments need to be queried, it's possible that fireChief.getQueryRunner(query) returns more than 1 queryRunner because query.getIntervals() is not specific to a single segment.
In this patch, for each segment being queried, Druid will update the query with its corresponding SpecificSegmentSpec.
2016-03-02 16:44:56 -06:00
e65e6a49a5
RealtimeTuningConfig: Use different default basePersistDirectory per instance.
2016-03-02 13:57:53 -08:00
004028b887
Make first few allocatePendingSegment retries quiet.
...
Some light retrying can happen during normal operation (SELECT -> INSERT races) and the
ensuing log messages would be scary for users.
2016-03-02 13:40:29 -08:00
612e327426
Merge pull request #2581 from gianm/fix-deadlock
...
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
2016-03-02 11:37:49 -08:00
7557eb2800
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
...
See stack traces here, from current master: https://gist.github.com/gianm/bd9a66c826995f97fc8f
1. The thread "qtp925672150-62" holds the lock on InternalInjectorCreator.class,
used by Scopes.SINGLETON, and wants the lock on "handlers" in Lifecycle.addMaybeStartHandler
called by DiscoveryModule.getServiceAnnouncer.
2. The main thread holds the lock on "handlers" in Lifecycle.addMaybeStartHandler, which it
took because it's trying to add the ExecutorLifecycle to the lifecycle. main is trying
to get the InternalInjectorCreator.class lock because it's running ExecutorLifecycle.start,
which does some Jackson deserialization, and Jackson needs that lock in order to inject
stuff into the Task it's deserializing.
This patch eagerly instantiates ChatHandlerResource (which I believe is what's trying to
create the ServiceAnnouncer in the qtp925672150-62 jetty thread) and the ExecutorLifecycle.
2016-03-02 10:53:42 -08:00
441234eafb
Merge pull request #2573 from gianm/sqlystuff
...
SQLMetadataConnector: Fix overzealous retries on "insert".
2016-03-02 10:21:00 -08:00
80d954578d
Merge pull request #2572 from gianm/fix-rit-taskresource
...
Fix default TaskResource for RealtimeIndexTasks.
2016-03-02 10:20:27 -08:00
d002b09e36
Merge pull request #2575 from gianm/tasklocation-toString
...
TaskLocation: Add toString method.
2016-03-01 21:13:20 -06:00
acd95d3e28
TaskLocation: Add toString method.
...
Necessary because these objects are used in log messages.
2016-03-01 17:52:06 -08:00
102fc92120
SQLMetadataConnector: Fix overzealous retries that were preventing EntryExistsException from making it out.
2016-03-01 17:20:33 -08:00
a355bfb7a9
Fix default TaskResource for RealtimeIndexTasks.
...
It was supposed to be the same as the task id, but it wasn't because
"makeTaskId" has a random component.
2016-03-01 16:54:22 -08:00
c82b680cd4
Merge pull request #2565 from sascha-coenen/feature-pulldeps-local-mvn-repo
...
pointing pull-deps tool to local maven repository
2016-03-01 13:26:18 -08:00
9340cae985
Merge pull request #2457 from bjozet/docs/fixes
...
Default value for maxRowsInMemory
2016-03-01 07:43:26 -08:00
2462c82c0e
New defaults for maxRowsInMemory rowFlushBoundary
...
To bring consistency to docs and source this commit changes the default
values for maxRowsInMemory and rowFlushBoundary to 75000 after
discussion in PR https://github.com/druid-io/druid/pull/2457 .
The previous default was 500000 and it's lower now on the grounds that
it's better for a default to be somewhat less efficient, and work,
than to reach for the stars and possibly result in
"OutOfMemoryError: java heap space" errors.
2016-03-01 13:50:28 +01:00
31b502773a
Merge pull request #2480 from navis/pagingfail-over-segments
...
Select query cannot span to next segment with paging
2016-03-01 11:42:41 +05:30
55c9320671
Merge pull request #2542 from guobingkun/fix_multiple_specs
...
[Fix Bug] Realtime Node may double count query results when multiple spec files are specified.
2016-02-29 15:53:14 -06:00
4edcb1b861
Refactor FireChief + UTs for RealtimeManagerTest
...
Add tests that verify whether RealtimeManager is querying the correct FireChief for a specific partition
make FireChief static and package private, add latches in the UT
2016-02-29 14:41:10 -06:00
e5c25725c0
Merge pull request #2562 from himanshug/fix_2556
...
with nested GpBy query outer query results need to be further merged
2016-02-29 12:17:33 -08:00
5b3d99c412
pointing pull-deps tool to local maven repository
2016-02-29 21:13:00 +01:00
68631d89e9
Allow realtime nodes to have multiple shards of the same datasource
2016-02-29 12:30:25 -06:00
14a306c553
Merge pull request #2553 from guobingkun/versioning_mysql
...
Give mysql tarball a version
2016-02-29 10:59:23 -06:00
0722ced413
with GpBy query outer query results need to be further merged
2016-02-29 10:16:25 -06:00
5aa1164b1a
Merge pull request #2482 from navis/lazy-init-boundfilter
...
Lazily initialize predicate for bound filter
2016-02-29 07:18:35 -06:00
b1ff920831
Lazily initialize predicate for bound filter
2016-02-29 15:35:52 +09:00
5f1e60324a
Added more complex test case with versioned segments
2016-02-29 14:48:24 +09:00
aa871b9353
versioning mysql tarball
2016-02-28 16:21:26 -06:00
2686bfa394
Select query cannot span to next segment with paging
2016-02-29 00:01:46 +09:00
8f97b1e40c
Merge pull request #2554 from guobingkun/improve_include_extensions
...
improve doc on including druid and hadoop extensions
2016-02-26 20:05:57 -08:00
dd060eb826
Merge pull request #2558 from gianm/hdfs-log-overwrite
...
HdfsTaskLogs: Allow overwriting existing logs.
2016-02-26 15:50:51 -08:00
7e70881101
HdfsTaskLogs: Allow overwriting existing logs.
...
Necessary because ForkingTaskRunner pushes logs when gracefully stopping,
but it may need to re-push those logs when the task finishes for real after
restoring.
2016-02-26 15:10:19 -08:00
ca1bf648d1
Merge pull request #2533 from metamx/javaOptsArray
...
Allow specifying peon javaOpts as an array
2016-02-26 14:05:49 -08:00
c6803c4364
Allow specifying peon javaOpts as an array
2016-02-26 13:24:35 -08:00
18f9e05f0f
improve doc on including druid and hadoop extensions
2016-02-26 13:53:08 -06:00
eb13d7afe3
Merge pull request #2521 from himanshug/fix_2497
...
RTR has multiple threads for assignment of pending tasks now
2016-02-26 08:14:15 -08:00
6b3c96c63a
better exception for invalid interval
2016-02-26 10:02:38 -06:00
9f8faabddb
Merge pull request #2469 from pdeva/patch-10
...
correct service names
2016-02-26 21:15:58 +05:30
bc156effe7
RTR has multiple threads for assignment of pending tasks now.
2016-02-26 09:27:03 -06:00
1fe277ee29
Merge pull request #2367 from se7entyse7en/feature-rackspace-cloud-files-static-firehose
...
Adds support to use Rackspace's cloudfiles as static firehose
2016-02-25 17:31:06 -08:00
Himanshu Gupta
Charles Allen
Slim Bouguerra
Xavier Léauté
jisookim
Fangjin Yang
Gian Merlino
David Lim
Bingkun Guo
Björn Zettergren
Nishant
Sascha Coenen
Eric Tschetter
Slim
navis.ryu
Parag Jain