eab8a0b54d
in DatasourceInputFormat code for determining segment block locations avoid the split calulation by helper TextInputFormat
2016-03-10 14:28:53 -06:00
7b1bfbf704
Add documentation to modules about what should be excluded.
2016-03-10 10:18:33 -08:00
d299540efc
Make HadoopTask load hadoop dependency classes LAST for local isolated classrunner
2016-03-10 10:18:23 -08:00
ba1185963b
Fix a bunch of dependencies
...
* Eliminate exclusion groups from pull-deps
* Only consider dependency nodes in pull-deps if they are not in the following scopes
* provided
* test
* system
* Fix a bunch of `<scope>provided</scope>` missing tags
* Better exclusions for a couple of problematic libs
2016-03-10 10:18:08 -08:00
cf3965c82e
Merge pull request #2625 from gianm/clarify-parser-docs
...
Clarify parser docs.
2016-03-10 09:44:23 -08:00
a2b1652787
Clarify parser docs.
...
- Clarify what parseSpecs are used for.
- Avro, Protobuf should use timeAndDims parseSpecs.
- Hadoop jobs should use hadoopyString string parsers.
2016-03-10 08:45:04 -08:00
68cffe1d91
Merge pull request #2615 from gianm/timeseries-skipEmptyBuckets-cache
...
Fix caching of skipEmptyBuckets for TimeseriesQuery.
2016-03-09 18:45:59 -08:00
5bb9fcf785
Merge pull request #2620 from guobingkun/fix_log
...
log exception correctly in DatasourceInputFormat and IndexGeneratorJob
2016-03-09 17:33:48 -06:00
0f3a7c94c6
Merge pull request #2616 from gianm/getContextBoolean
...
Make specifying query context booleans more consistent.
2016-03-09 14:23:10 -08:00
c20d7682a9
log exceptions correctly in DatasourceInputFormat and IndexGeneratorJob
2016-03-09 13:41:31 -06:00
4c3a3f8da6
Merge pull request #2537 from druid-io/refactor-ext
...
refactor extensions into core and contrib
2016-03-09 08:18:42 -08:00
94da1f8f59
Merge pull request #2608 from gaodayue/hdfs_federation_support
...
support Federationed HDFS
2016-03-08 20:41:13 -08:00
708bc674fa
Make specifying query context booleans more consistent.
...
Before, some needed to be strings and some needed to be real booleans. Now
they can all be either one.
2016-03-08 19:38:26 -08:00
40dad6dff4
Fix caching of skipEmptyBuckets for TimeseriesQuery.
2016-03-08 19:22:12 -08:00
e3e932a4d4
refactor extensions into core and contrib
2016-03-08 17:12:09 -08:00
163e536415
Merge pull request #2601 from navis/fix-combine-sequence
...
Relay final value to yielder in CombineSequence (Fix for #2586 )
2016-03-08 15:59:08 -08:00
ca5de3f583
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-08 15:03:59 -06:00
8e36e6fa43
Merge pull request #2610 from dclim/add-combineText-doc
...
add combineText property and cleanup batch ingestion doc
2016-03-08 12:54:16 -08:00
9e31e2ce0e
Merge pull request #2605 from himanshug/robust_hdfs_kill
...
let hdfs segment kill be a success when segment file does not exist
2016-03-08 12:43:59 -08:00
9c2420a1bc
Merge pull request #2599 from himanshug/datasource_isolation
...
make coordinator db polling for list of segments more robust
2016-03-08 12:43:49 -08:00
e7018f524f
Merge pull request #2598 from himanshug/handoff_timeout
...
optional ability to configure handoff wait timeout on realtime tasks
2016-03-08 12:43:36 -08:00
de869f6efc
Merge pull request #2584 from himanshug/gp_by_max_limit
...
allow groupBy maxRows limit be overridable by context
2016-03-08 12:43:25 -08:00
df29667a89
add combineText property and cleanup batch ingestion doc
2016-03-08 13:10:34 -07:00
db11d8bbe5
Merge pull request #2609 from metamx/showBadLog
...
Add LogTest to show bad log behavior
2016-03-08 10:53:27 -08:00
3afa9e916b
Merge pull request #2578 from b-slim/fix_metric
...
override metric name
2016-03-08 12:04:50 -06:00
908eb7eb4d
Add LogTest to show bad log behavior
2016-03-08 09:46:26 -08:00
1b3fd8a8aa
added more tests and fixed concat+combine
2016-03-09 02:08:52 +09:00
c72438ead0
override metric name
2016-03-08 10:58:12 -06:00
a6dc3703ca
use ISODataTimeFormat for both hdfs and viewfs schema to support Federationed HDFS
2016-03-08 13:55:05 +08:00
4ff1620131
Relay final value to yielder in CombineSequence (Fix for #2586 )
2016-03-08 10:31:15 +09:00
ea408f531e
let the hdfs segment kill be success when segment file does not exist
2016-03-07 16:51:41 -06:00
099acb4966
allow groupBy max[Intermediate]Rows limit be overridable by context
2016-03-07 15:22:41 -06:00
1288784bde
in coordinator db polling for available segments, ignore corrupted entries in segments table so that coordinator continues to load new segments even if there are few corrupted segment entries
2016-03-07 15:13:10 -06:00
0402636598
configurable handoffConditionTimeout in realtime tasks for segment handoff wait
2016-03-05 10:14:54 -06:00
4fa08a1329
Merge pull request #2592 from himanshug/inc_index_storage_adapter_fix
...
fix regression - reintroducing the safety check removed in commit-1d602be
2016-03-04 12:31:28 -08:00
2ad134638d
Merge pull request #2589 from b-slim/fix_real_time
...
Make realtime kafka firehose skip corrupt message
2016-03-04 12:14:23 -08:00
623e89aa54
skip corrupt message
2016-03-04 08:30:40 -06:00
c544ebf25e
reintroducing the safety check removed in commit-1d602be so that dim value ids are less than cardinality
2016-03-03 23:34:23 -06:00
9fbcd86930
Merge pull request #2590 from jisookim0513/master
...
fix default number of connections on broker config documentation
2016-03-03 15:08:52 -08:00
177b575d41
fix default number of connections on broker config documentation
2016-03-03 13:50:48 -08:00
703dc7a48f
Merge pull request #2585 from gianm/overlord-workers-api
...
OverlordResource: Use getZkWorkers on RemoteTaskRunner.
2016-03-02 18:15:05 -08:00
d06c1c5c85
Merge pull request #2583 from guobingkun/fix_multiple_specs_2
...
update querySegmentSpec when passing query to getQueryRunner
2016-03-02 18:05:34 -08:00
e9c23bf376
OverlordResource: Use getZkWorkers on RemoteTaskRunner.
...
Restores old behavior of this api, from before #2249 when getWorkers returned ZkWorkers.
2016-03-02 17:31:34 -08:00
9e74772d6b
Merge pull request #2574 from gianm/allostuff
...
Make first few allocatePendingSegment retries quiet.
2016-03-02 16:16:53 -07:00
cfe2dbf1eb
Merge pull request #2580 from gianm/rtc-basePersist
...
RealtimeTuningConfig: Use different default basePersistDirectory per instance.
2016-03-02 16:56:49 -06:00
4a58462fc7
update querySegmentSpec when passing query to getQueryRunner
...
After finding the FireChief for a specific partition, Druid will need to find the specific queryRunner for each segment being queried by passing the query to FireChief. Currently Druid is passing the original query that contains all the segments need to be queried, it's possible that fireChief.getQueryRunner(query) returns more than 1 queryRunner because query.getIntervals() is not specific to a single segment.
In this patch, for each segment being queried, Druid will update the query with its corresponding SpecificSegmentSpec.
2016-03-02 16:44:56 -06:00
e65e6a49a5
RealtimeTuningConfig: Use different default basePersistDirectory per instance.
2016-03-02 13:57:53 -08:00
004028b887
Make first few allocatePendingSegment retries quiet.
...
Some light retrying can happen during normal operation (SELECT -> INSERT races) and the
ensuing log messages would be scary for users.
2016-03-02 13:40:29 -08:00
612e327426
Merge pull request #2581 from gianm/fix-deadlock
...
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
2016-03-02 11:37:49 -08:00
7557eb2800
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
...
See stack traces here, from current master: https://gist.github.com/gianm/bd9a66c826995f97fc8f
1. The thread "qtp925672150-62" holds the lock on InternalInjectorCreator.class,
used by Scopes.SINGLETON, and wants the lock on "handlers" in Lifecycle.addMaybeStartHandler
called by DiscoveryModule.getServiceAnnouncer.
2. The main thread holds the lock on "handlers" in Lifecycle.addMaybeStartHandler, which it
took because it's trying to add the ExecutorLifecycle to the lifecycle. main is trying
to get the InternalInjectorCreator.class lock because it's running ExecutorLifecycle.start,
which does some Jackson deserialization, and Jackson needs that lock in order to inject
stuff into the Task it's deserializing.
This patch eagerly instantiates ChatHandlerResource (which I believe is what's trying to
create the ServiceAnnouncer in the qtp925672150-62 jetty thread) and the ExecutorLifecycle.
2016-03-02 10:53:42 -08:00
Himanshu Gupta
Charles Allen
Nishant
Fangjin Yang
Gian Merlino
Bingkun Guo
Xavier Léauté
dclim
Slim
navis.ryu
Slim Bouguerra
gaodayue
jisookim
David Lim