96c981cd0a
fix broken link for Tasks
2016-03-11 11:36:34 -06:00
05397a9b4f
ThreadPoolTaskRunner: Make graceful shutdown logs less scary.
...
- It's okay to suppress InterruptedException during graceful shutdown, as
tasks may use it to accelerate their own shutdown.
- It's okay to ignore return statuses during graceful shutdown (which may
be FAILED!) because it actually doesn't matter what they are.
2016-03-11 07:49:29 -08:00
f4ab1c2e52
Merge pull request #2632 from gianm/examples-druid-provided
...
examples: Switch druid-server, druid-common to "provided".
2016-03-10 19:57:32 -08:00
d63473e0d5
examples: Switch druid-server, druid-common to "provided".
2016-03-10 18:43:29 -08:00
f22fb2c2cf
KafkaIndexTask.
...
Reads a specific offset range from specific partitions, and can use dataSource metadata
transactions to guarantee exactly-once ingestion.
Each task has a finite lifecycle, so it is expected that some process will be supervising
existing tasks and creating new ones when needed.
2016-03-10 18:41:43 -08:00
187569e702
DataSource metadata.
...
Geared towards supporting transactional inserts of new segments. This involves an
interface "DataSourceMetadata" that allows combining of partially specified metadata
(useful for partitioned ingestion).
DataSource metadata is stored in a new "dataSource" table.
2016-03-10 17:41:50 -08:00
3d2214377d
Appenderatoring.
...
Appenderators are a way of getting more control over the ingestion process
than a Plumber allows. The idea is that existing Plumbers could be implemented
using Appenderators, but you could also implement things that Plumbers can't do.
FiniteAppenderatorDrivers help simplify indexing a finite stream of data.
Also:
- Sink: Ability to consider itself "finished" vs "still writable".
- Sink: Ability to return the number of rows contained within the sink.
2016-03-10 17:41:50 -08:00
08284fea62
Publish test-jar for indexing-service.
2016-03-10 16:50:37 -08:00
92c828f904
Make SegmentHandoffNotifier Closeable.
2016-03-10 16:50:37 -08:00
ad5ffdf483
Nix Committers.supplierOf; Suppliers.ofInstance is good enough.
2016-03-10 16:50:37 -08:00
8a11161b20
Plumbers: Move plumber.add out of try/catch for ParseException.
...
The incremental indexes handle that now so it's not necessary.
Also, add debug logging and more detailed exceptions to the incremental
indexes for the case where there are parse exceptions during aggregation.
2016-03-10 16:39:26 -08:00
dc0214bddb
while GroupBy merging use unsorted facts in IncrementalIndex wherever possible
2016-03-10 16:11:48 -06:00
02dfd5cd80
update IncrementalIndex to support unsorted facts map that can be used in groupBy merging to improve performance
2016-03-10 16:11:48 -06:00
1e49092ce7
Merge pull request #2627 from himanshug/fix_datasource_inputformat_locations
...
fix regression - bug in DatasourceInputFormat best effort split location finder code
2016-03-10 13:46:04 -08:00
90d7409e1a
Merge pull request #2611 from himanshug/gp_by_max_limit
...
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-10 13:44:13 -08:00
863aa66808
Merge pull request #2597 from metamx/forwardPortMMX23
...
Forward port - Fix dependency problems
2016-03-10 14:56:52 -06:00
eab8a0b54d
in DatasourceInputFormat code for determining segment block locations avoid the split calulation by helper TextInputFormat
2016-03-10 14:28:53 -06:00
7b1bfbf704
Add documentation to modules about what should be excluded.
2016-03-10 10:18:33 -08:00
d299540efc
Make HadoopTask load hadoop dependency classes LAST for local isolated classrunner
2016-03-10 10:18:23 -08:00
ba1185963b
Fix a bunch of dependencies
...
* Eliminate exclusion groups from pull-deps
* Only consider dependency nodes in pull-deps if they are not in the following scopes
* provided
* test
* system
* Fix a bunch of `<scope>provided</scope>` missing tags
* Better exclusions for a couple of problematic libs
2016-03-10 10:18:08 -08:00
cf3965c82e
Merge pull request #2625 from gianm/clarify-parser-docs
...
Clarify parser docs.
2016-03-10 09:44:23 -08:00
a2b1652787
Clarify parser docs.
...
- Clarify what parseSpecs are used for.
- Avro, Protobuf should use timeAndDims parseSpecs.
- Hadoop jobs should use hadoopyString string parsers.
2016-03-10 08:45:04 -08:00
68cffe1d91
Merge pull request #2615 from gianm/timeseries-skipEmptyBuckets-cache
...
Fix caching of skipEmptyBuckets for TimeseriesQuery.
2016-03-09 18:45:59 -08:00
5bb9fcf785
Merge pull request #2620 from guobingkun/fix_log
...
log exception correctly in DatasourceInputFormat and IndexGeneratorJob
2016-03-09 17:33:48 -06:00
0f3a7c94c6
Merge pull request #2616 from gianm/getContextBoolean
...
Make specifying query context booleans more consistent.
2016-03-09 14:23:10 -08:00
c20d7682a9
log exceptions correctly in DatasourceInputFormat and IndexGeneratorJob
2016-03-09 13:41:31 -06:00
4c3a3f8da6
Merge pull request #2537 from druid-io/refactor-ext
...
refactor extensions into core and contrib
2016-03-09 08:18:42 -08:00
94da1f8f59
Merge pull request #2608 from gaodayue/hdfs_federation_support
...
support Federationed HDFS
2016-03-08 20:41:13 -08:00
708bc674fa
Make specifying query context booleans more consistent.
...
Before, some needed to be strings and some needed to be real booleans. Now
they can all be either one.
2016-03-08 19:38:26 -08:00
40dad6dff4
Fix caching of skipEmptyBuckets for TimeseriesQuery.
2016-03-08 19:22:12 -08:00
e3e932a4d4
refactor extensions into core and contrib
2016-03-08 17:12:09 -08:00
163e536415
Merge pull request #2601 from navis/fix-combine-sequence
...
Relay final value to yielder in CombineSequence (Fix for #2586 )
2016-03-08 15:59:08 -08:00
ca5de3f583
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-08 15:03:59 -06:00
8e36e6fa43
Merge pull request #2610 from dclim/add-combineText-doc
...
add combineText property and cleanup batch ingestion doc
2016-03-08 12:54:16 -08:00
9e31e2ce0e
Merge pull request #2605 from himanshug/robust_hdfs_kill
...
let hdfs segment kill be a success when segment file does not exist
2016-03-08 12:43:59 -08:00
9c2420a1bc
Merge pull request #2599 from himanshug/datasource_isolation
...
make coordinator db polling for list of segments more robust
2016-03-08 12:43:49 -08:00
e7018f524f
Merge pull request #2598 from himanshug/handoff_timeout
...
optional ability to configure handoff wait timeout on realtime tasks
2016-03-08 12:43:36 -08:00
de869f6efc
Merge pull request #2584 from himanshug/gp_by_max_limit
...
allow groupBy maxRows limit be overridable by context
2016-03-08 12:43:25 -08:00
df29667a89
add combineText property and cleanup batch ingestion doc
2016-03-08 13:10:34 -07:00
db11d8bbe5
Merge pull request #2609 from metamx/showBadLog
...
Add LogTest to show bad log behavior
2016-03-08 10:53:27 -08:00
3afa9e916b
Merge pull request #2578 from b-slim/fix_metric
...
override metric name
2016-03-08 12:04:50 -06:00
908eb7eb4d
Add LogTest to show bad log behavior
2016-03-08 09:46:26 -08:00
1b3fd8a8aa
added more tests and fixed concat+combine
2016-03-09 02:08:52 +09:00
c72438ead0
override metric name
2016-03-08 10:58:12 -06:00
a6dc3703ca
use ISODataTimeFormat for both hdfs and viewfs schema to support Federationed HDFS
2016-03-08 13:55:05 +08:00
4ff1620131
Relay final value to yielder in CombineSequence (Fix for #2586 )
2016-03-08 10:31:15 +09:00
ea408f531e
let the hdfs segment kill be success when segment file does not exist
2016-03-07 16:51:41 -06:00
099acb4966
allow groupBy max[Intermediate]Rows limit be overridable by context
2016-03-07 15:22:41 -06:00
1288784bde
in coordinator db polling for available segments, ignore corrupted entries in segments table so that coordinator continues to load new segments even if there are few corrupted segment entries
2016-03-07 15:13:10 -06:00
0402636598
configurable handoffConditionTimeout in realtime tasks for segment handoff wait
2016-03-05 10:14:54 -06:00
Bingkun Guo
Gian Merlino
Fangjin Yang
Himanshu Gupta
Xavier Léauté
Charles Allen
Nishant
dclim
Slim
navis.ryu
Slim Bouguerra
gaodayue