Apache/druid
1
0
Fork 0
mirror of https://github.com/apache/druid.git synced 2025-02-08 19:14:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,867 Commits 116 Branches 507 Tags
Commit Graph

6867 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
dclim
df29667a89 add combineText property and cleanup batch ingestion doc 2016-03-08 13:10:34 -07:00
Fangjin Yang
db11d8bbe5 Merge pull request #2609 from metamx/showBadLog 
Add LogTest to show bad log behavior
 2016-03-08 10:53:27 -08:00
Slim
3afa9e916b Merge pull request #2578 from b-slim/fix_metric 
override metric name
 2016-03-08 12:04:50 -06:00
Charles Allen
908eb7eb4d Add LogTest to show bad log behavior 2016-03-08 09:46:26 -08:00
Slim Bouguerra
c72438ead0 override metric name 2016-03-08 10:58:12 -06:00
Charles Allen
4fa08a1329 Merge pull request #2592 from himanshug/inc_index_storage_adapter_fix 
fix regression - reintroducing the safety check removed in commit-1d602be
 2016-03-04 12:31:28 -08:00
Charles Allen
2ad134638d Merge pull request #2589 from b-slim/fix_real_time 
Make realtime kafka firehose skip corrupt message
 2016-03-04 12:14:23 -08:00
Slim Bouguerra
623e89aa54 skip corrupt message 2016-03-04 08:30:40 -06:00
Himanshu Gupta
c544ebf25e reintroducing the safety check removed in commit-1d602be so that dim value ids are less than cardinality 2016-03-03 23:34:23 -06:00
Xavier Léauté
9fbcd86930 Merge pull request #2590 from jisookim0513/master 
fix default number of connections on broker config documentation
 2016-03-03 15:08:52 -08:00
jisookim
177b575d41 fix default number of connections on broker config documentation 2016-03-03 13:50:48 -08:00
Fangjin Yang
703dc7a48f Merge pull request #2585 from gianm/overlord-workers-api 
OverlordResource: Use getZkWorkers on RemoteTaskRunner.
 2016-03-02 18:15:05 -08:00
Fangjin Yang
d06c1c5c85 Merge pull request #2583 from guobingkun/fix_multiple_specs_2 
update querySegmentSpec when passing query to getQueryRunner
 2016-03-02 18:05:34 -08:00
Gian Merlino
e9c23bf376 OverlordResource: Use getZkWorkers on RemoteTaskRunner. 
Restores old behavior of this api, from before #2249 when getWorkers returned ZkWorkers.
 2016-03-02 17:31:34 -08:00
David Lim
9e74772d6b Merge pull request #2574 from gianm/allostuff 
Make first few allocatePendingSegment retries quiet.
 2016-03-02 16:16:53 -07:00
Bingkun Guo
cfe2dbf1eb Merge pull request #2580 from gianm/rtc-basePersist 
RealtimeTuningConfig: Use different default basePersistDirectory per instance.
 2016-03-02 16:56:49 -06:00
Bingkun Guo
4a58462fc7 update querySegmentSpec when passing query to getQueryRunner 
After finding the FireChief for a specific partition, Druid will need to find the specific queryRunner for each segment being queried by passing the query to FireChief. Currently Druid is passing the original query that contains all the segments need to be queried, it's possible that fireChief.getQueryRunner(query) returns more than 1 queryRunner because query.getIntervals() is not specific to a single segment.

In this patch, for each segment being queried, Druid will update the query with its corresponding SpecificSegmentSpec.
 2016-03-02 16:44:56 -06:00
Gian Merlino
e65e6a49a5 RealtimeTuningConfig: Use different default basePersistDirectory per instance. 2016-03-02 13:57:53 -08:00
Gian Merlino
004028b887 Make first few allocatePendingSegment retries quiet. 
Some light retrying can happen during normal operation (SELECT -> INSERT races) and the
ensuing log messages would be scary for users.
 2016-03-02 13:40:29 -08:00
Fangjin Yang
612e327426 Merge pull request #2581 from gianm/fix-deadlock 
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
 2016-03-02 11:37:49 -08:00
Gian Merlino
7557eb2800 CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource. 
See stack traces here, from current master: https://gist.github.com/gianm/bd9a66c826995f97fc8f

1. The thread "qtp925672150-62" holds the lock on InternalInjectorCreator.class,
   used by Scopes.SINGLETON, and wants the lock on "handlers" in Lifecycle.addMaybeStartHandler
   called by DiscoveryModule.getServiceAnnouncer.
2. The main thread holds the lock on "handlers" in Lifecycle.addMaybeStartHandler, which it
   took because it's trying to add the ExecutorLifecycle to the lifecycle. main is trying
   to get the InternalInjectorCreator.class lock because it's running ExecutorLifecycle.start,
   which does some Jackson deserialization, and Jackson needs that lock in order to inject
   stuff into the Task it's deserializing.

This patch eagerly instantiates ChatHandlerResource (which I believe is what's trying to
create the ServiceAnnouncer in the qtp925672150-62 jetty thread) and the ExecutorLifecycle.
 2016-03-02 10:53:42 -08:00
Fangjin Yang
441234eafb Merge pull request #2573 from gianm/sqlystuff 
SQLMetadataConnector: Fix overzealous retries on "insert".
 2016-03-02 10:21:00 -08:00
Fangjin Yang
80d954578d Merge pull request #2572 from gianm/fix-rit-taskresource 
Fix default TaskResource for RealtimeIndexTasks.
 2016-03-02 10:20:27 -08:00
Bingkun Guo
d002b09e36 Merge pull request #2575 from gianm/tasklocation-toString 
TaskLocation: Add toString method.
 2016-03-01 21:13:20 -06:00
Gian Merlino
acd95d3e28 TaskLocation: Add toString method. 
Necessary because these objects are used in log messages.
 2016-03-01 17:52:06 -08:00
Gian Merlino
102fc92120 SQLMetadataConnector: Fix overzealous retries that were preventing EntryExistsException from making it out. 2016-03-01 17:20:33 -08:00
Gian Merlino
a355bfb7a9 Fix default TaskResource for RealtimeIndexTasks. 
It was supposed to be the same as the task id, but it wasn't because
"makeTaskId" has a random component.
 2016-03-01 16:54:22 -08:00
Charles Allen
c82b680cd4 Merge pull request #2565 from sascha-coenen/feature-pulldeps-local-mvn-repo 
pointing pull-deps tool to local maven repository
 2016-03-01 13:26:18 -08:00
Fangjin Yang
9340cae985 Merge pull request #2457 from bjozet/docs/fixes 
Default value for maxRowsInMemory
 2016-03-01 07:43:26 -08:00
Björn Zettergren
2462c82c0e New defaults for maxRowsInMemory rowFlushBoundary 
To bring consistency to docs and source this commit changes the default
values for maxRowsInMemory and rowFlushBoundary to 75000 after
discussion in PR https://github.com/druid-io/druid/pull/2457.

The previous default was 500000 and it's lower now on the grounds that
it's better for a default to be somewhat less efficient, and work,
than to reach for the stars and possibly result in
"OutOfMemoryError: java heap space" errors.
 2016-03-01 13:50:28 +01:00
Nishant
31b502773a Merge pull request #2480 from navis/pagingfail-over-segments 
Select query cannot span to next segment with paging
 2016-03-01 11:42:41 +05:30
Himanshu
55c9320671 Merge pull request #2542 from guobingkun/fix_multiple_specs 
[Fix Bug] Realtime Node may double count query results when multiple spec files are specified.
 2016-02-29 15:53:14 -06:00
Bingkun Guo
4edcb1b861 Refactor FireChief + UTs for RealtimeManagerTest 
Add tests that verify whether RealtimeManager is querying the correct FireChief for a specific partition
make FireChief static and package private, add latches in the UT
 2016-02-29 14:41:10 -06:00
Fangjin Yang
e5c25725c0 Merge pull request #2562 from himanshug/fix_2556 
with nested GpBy query outer query results need to be further merged
 2016-02-29 12:17:33 -08:00
Sascha Coenen
5b3d99c412 pointing pull-deps tool to local maven repository 2016-02-29 21:13:00 +01:00
Eric Tschetter
68631d89e9 Allow realtime nodes to have multiple shards of the same datasource 2016-02-29 12:30:25 -06:00
Bingkun Guo
14a306c553 Merge pull request #2553 from guobingkun/versioning_mysql 
Give mysql tarball a version
 2016-02-29 10:59:23 -06:00
Himanshu Gupta
0722ced413 with GpBy query outer query results need to be further merged 2016-02-29 10:16:25 -06:00
Slim
5aa1164b1a Merge pull request #2482 from navis/lazy-init-boundfilter 
Lazily initialize predicate for bound filter
 2016-02-29 07:18:35 -06:00
navis.ryu
b1ff920831 Lazily initialize predicate for bound filter 2016-02-29 15:35:52 +09:00
navis.ryu
5f1e60324a Added more complex test case with versioned segments 2016-02-29 14:48:24 +09:00
Bingkun Guo
aa871b9353 versioning mysql tarball 2016-02-28 16:21:26 -06:00
navis.ryu
2686bfa394 Select query cannot span to next segment with paging 2016-02-29 00:01:46 +09:00
Fangjin Yang
8f97b1e40c Merge pull request #2554 from guobingkun/improve_include_extensions 
improve doc on including druid and hadoop extensions
 2016-02-26 20:05:57 -08:00
Fangjin Yang
dd060eb826 Merge pull request #2558 from gianm/hdfs-log-overwrite 
HdfsTaskLogs: Allow overwriting existing logs.
 2016-02-26 15:50:51 -08:00
Gian Merlino
7e70881101 HdfsTaskLogs: Allow overwriting existing logs. 
Necessary because ForkingTaskRunner pushes logs when gracefully stopping,
but it may need to re-push those logs when the task finishes for real after
restoring.
 2016-02-26 15:10:19 -08:00
Charles Allen
ca1bf648d1 Merge pull request #2533 from metamx/javaOptsArray 
Allow specifying peon javaOpts as an array
 2016-02-26 14:05:49 -08:00
Charles Allen
c6803c4364 Allow specifying peon javaOpts as an array 2016-02-26 13:24:35 -08:00
Bingkun Guo
18f9e05f0f improve doc on including druid and hadoop extensions 2016-02-26 13:53:08 -06:00
Gian Merlino
eb13d7afe3 Merge pull request #2521 from himanshug/fix_2497 
RTR has multiple threads for assignment of pending tasks now
 2016-02-26 08:14:15 -08:00