Gian Merlino
cd5c5419bb
Make docs deploying better. ( #3040 )
...
- Make redirects for old links based on _redirects.json
- Replace #{DRUIDVERSION} tokens in docs with current version
- Allow origins named something other than "origin"
- Can use either s3cmd or awscli, depending on availability
2016-05-31 15:34:58 -07:00
David Lim
f6c39cc844
Kafka task minimum message time ( #3035 )
...
* add KafkaIndexTask support for minimumMessageTime
* add Kafka supervisor support for lateMessageRejectionPeriod
2016-05-31 11:37:00 -07:00
scusjs
ebb6831770
rm , of jobProperties. jackson can not parse it ( #3012 )
2016-05-26 09:46:33 -07:00
Charles Allen
245077b47f
Fix formatting in lookups-cached-global.md ( #3009 )
2016-05-24 17:28:39 -07:00
Charles Allen
c738c0e1cd
Silly Typo in docs
2016-05-24 13:31:58 -07:00
Charles Allen
8024b915e2
[QTL] Implement LookupExtractorFactory of namespaced lookup ( #2926 )
...
* support LookupReferencesManager registration of namespaced lookup and eliminate static configurations for lookup from namespecd lookup extensions
- druid-namespace-lookup and druid-kafka-extraction-namespace are modified
- However, druid-namespace-lookup still has configuration about ON/OFF
HEAP cache manager selection, which is not namespace wide
configuration but node wide configuration as multiple namespace shares
the same cache manager
* update KafkaExtractionNamespaceTest to reflect argument signature changes
* Add more synchronization functionality to NamespaceLookupExtractorFactory
* Remove old way of using extraction namespaces
* resolve compile error by supporting LookupIntrospectHandler
* Remove kafka lookups
* Remove unused stuff
* Fix start and stop behavior to be consistent with new javadocs
* Remove unused strings
* Add timeout option
* Address comments on configurations and improve docs
* Add more options and update hash key and replaces
* Move monitoring to the overriding classes
* Add better start/stop logging
* Remove old docs about namespace names
* Fix bad comma
* Add `@JsonIgnore` to lookup factory
* Address code review comments
* Remove ExtractionNamespace from module json registration
* Fix problems with naming and initialization. Add tests
* Optimize imports / reformat
* Fix future not being properly cancelled on failed initial scheduling
* Fix delete returns
* Add more docs about whole introspection
* Add `/version` introspection point for lookups
* Add more tests and address comments
* Add StaticMap extraction namespace for testing. Also add a bunch of tests
* Move cache system property to `druid.lookup.namespace.cache.type`
* Make VERSION lower case
* Change poll period to 0ms for StaticMap
* Move cache key to bytebuffer
* Change hashCode and equals on static map extraction fn
* Add more comments on StaticMap
* Address comments
* Make scheduleAndWait use a latch
* Sanity renames and fix imports
* Remove extra info in docs
* Fix review comments
* Strengthen failure on start from warn to error
* Address comments
* Rename namespace-lookup to lookups-cached-global
* Fix injective mis-naming
* Also add serde test
2016-05-24 10:56:40 -07:00
Nishant
0ac1b27d53
Allow manually setting of shutoffTime for EventReceiverFirehose ( #2803 )
...
* Allow dynamically setting of shutoffTime for EventReceiverFirehose
Allow dynamically setting shutoffTime for EventReceiverFirehose
review comments and tests
* shut down exec on close
2016-05-24 07:24:00 -07:00
Nishant
dea4391a49
fix broken links ( #3003 )
2016-05-23 06:38:21 -07:00
Fangjin Yang
00de26c76a
fix extensions docs ( #2995 )
...
* fix extensions docs
* fix mistakes
2016-05-19 14:01:06 -07:00
Charles Allen
eaaad01de7
[QTL] Datasource as lookupTier ( #2955 )
...
* Datasource as lookup tier
* Adds an option to let indexing service tasks pull their lookup tier from the datasource they are working for.
* Fix bad docs for lookups lookupTier
* Add Datasource name holder
* Move task and datasource to be pulled from Task file
* Make LookupModule pull from bound dataSource
* Fix test
* Fix code style on imports
* Fix formatting
* Make naming better
* Address code comments about naming
2016-05-17 15:44:42 -07:00
Shekhar Gulati
c41bfe50d0
Using quotes around the cp ( #2934 )
2016-05-16 15:16:48 -07:00
Parag Jain
e3ea842cd3
add available query granularity strings ( #2960 )
2016-05-12 18:49:31 -07:00
Joe Pettersson
2288c78395
chore_fix-quickstart-docs ( #2946 )
...
Fixes a small grammatical error in the `./docs/content/tutorials/quickstart.md` whereby a sentence didn't make sense
2016-05-10 09:52:24 -07:00
Slim
45b2e65d75
[QTL] adding listDelimiter to lookup parser spec ( #2941 )
...
* adding listDelimiter to lookup parser spec
* cleaning code
2016-05-10 15:41:16 +05:30
Gian Merlino
b8af84d1fc
Update tutorials to tranquility v0.8.0. ( #2937 )
2016-05-09 11:50:37 -07:00
Gian Merlino
fffa9c8265
Fix flattenSpec docs, "nested" should be "path". ( #2924 )
2016-05-05 08:59:41 -07:00
David Lim
b489f63698
Supervisor for KafkaIndexTask ( #2656 )
...
* supervisor for kafka indexing tasks
* cr changes
2016-05-04 23:13:13 -07:00
Charles Allen
44e52acfc0
Link up metrics configuration to what they mean ( #2921 )
2016-05-04 10:30:02 -07:00
Himanshu
8e2742b7e8
adding QueryGranularity to segment metadata and optionally expose same from segmentMetadata query ( #2873 )
2016-05-03 11:31:10 -07:00
Navis Ryu
45a3a26ef7
Add more math functions ( #2822 )
...
* Add more math functions
* added function list
2016-05-03 10:55:13 -07:00
Gian Merlino
e680665f1c
Fix Avro parseSpec example, "type" should be "format". ( #2918 )
2016-05-03 09:22:43 -07:00
Himanshu
6c5bf91f9a
publish metrics numJettyConns to see how number of active jetty connections change over time ( #2839 )
...
this can be compared with numer of active queries to see if requests are waiting in jetty queue
2016-05-02 14:08:25 -07:00
Charles Allen
6b957aa072
[QTL] Make URI Exctraction Namespace take more sane arguments ( #2738 )
...
* Make URI Exctraction Namespace take more sane arguments
* Fixes https://github.com/druid-io/druid/issues/2669
* Update docs
* Rename error message
* Undo overzealous deletion of docs
* Explain caching mechanism a bit more in docs
2016-05-02 12:54:34 -07:00
Charles Allen
54b717bdc3
[QTL] Move kafka-extraction-namespace to the Lookup framework. ( #2800 )
...
* Move kafka-extraction-namespace to the Lookup framework.
* Address comments
* Fix missing kafka introspection
* Fix tests to be less racy
* Make testing a bit more leniant
* Make tests even more forgiving
* Add comments to kafka lookup cache method
* Move startStopLock to just use started
* Make start() and stop() idempotent
* Forgot to update test after last change, test now accounts for idempotency
* Add extra idempotency on stop check
* Add more descriptive docs of behavior
2016-05-02 09:45:13 -07:00
michaelschiff
2203a812bc
statsd-emitter ( #2410 )
2016-04-28 18:41:02 -07:00
David Lim
890bdb543d
doc fixes ( #2897 )
2016-04-28 15:34:58 -07:00
Slim
58510d826b
fix emit wait time ( #2869 )
2016-04-26 17:07:03 -07:00
Slim
55785267e4
postAgg filedName must match name of AGG ( #2874 )
2016-04-22 11:11:54 -07:00
binlijin
9151099e08
add document for druid.segmentCache.numBootstrapThreads ( #2872 )
2016-04-22 12:06:08 +08:00
Himanshu
3cfd9c64c9
make singleThreaded groupBy query config overridable at query time ( #2828 )
...
* make isSingleThreaded groupBy query processing overridable at query time
* refactor code in GroupByMergedQueryRunner to make processing of single threaded and parallel merging of runners consistent
2016-04-21 17:12:58 -07:00
Slim
984a518c9f
Merge pull request #2734 from b-slim/LookupIntrospection2
...
[QTL][Lookup] adding introspection endpoint
2016-04-21 12:15:57 -05:00
Gian Merlino
c74391e54c
JavaScript: Ability to disable. ( #2853 )
...
Fixes #2852 .
2016-04-21 09:43:15 -05:00
Nishant
dbf63f738f
Add ability to filter segments for specific dataSources on broker without creating tiers ( #2848 )
...
* Add back FilteredServerView removed in a32906c7fd
to reduce memory usage using watched tiers.
* Add functionality to specify "druid.broker.segment.watchedDataSources"
2016-04-19 10:10:06 -07:00
Gaurav Kumar
f5822faca3
Fixed wrong parseSpec in Avro Hadoop Parser ( #2846 )
...
`parseSpec` should contain `format` instead of `type`. It was wrongly defaulting to `tsv`
2016-04-16 11:34:54 -07:00
du00cs
639d0630b8
jackson conflict workaround in hadooop ingestio & parquet extension coordinate update ( #2817 )
2016-04-13 14:20:33 -07:00
Fangjin Yang
0c4a42bb6f
change toc entry ( #2834 )
2016-04-13 13:45:07 -07:00
Gian Merlino
e320d13385
Fix various broken links in the docs. ( #2833 )
2016-04-13 13:30:01 -07:00
Gian Merlino
725ee1401d
Update tranquility version in the docs. ( #2832 )
2016-04-13 11:33:59 -07:00
Gian Merlino
aa25cc1f68
Fix up Kafka tutorial ( #2831 )
...
1) Remove extraneous section
2) Remove -SNAPSHOT version
2016-04-13 11:33:45 -07:00
Fangjin Yang
abd951df1a
Document how to use roaring bitmaps ( #2824 )
...
* Document how to use roaring bitmaps
This fixes #2408 .
While not all indexSpec properties are explained, it does explain how roaring bitmaps can be turned on.
* fix
* fix
* fix
* fix
2016-04-12 19:28:02 -07:00
Charles Allen
ed5377465a
add AirBnB Caravel to list of libraries ( #2719 )
2016-04-12 12:53:50 -07:00
Sébastien Launay
37d2ab623e
Merge pull request #2815 from slaunay/documentation/hadoop-classpath-issue-fix-with-configuration
...
Doc for mapreduce.job.user.classpath.first=true
2016-04-12 10:51:51 -07:00
Nishant
deb6ecf919
handle review comments for PR 2784
...
https://github.com/druid-io/druid/pull/2784#discussion_r59062021
2016-04-12 21:52:00 +05:30
Fangjin Yang
bd6bd34cd8
Merge pull request #2090 from himanshug/math_exp
...
math expression support
2016-04-11 21:36:17 -07:00
Fangjin Yang
234125878a
Merge pull request #2808 from metamx/moveLookupSaveStateConfigDocs
...
Move lookup config doc to proper location
2016-04-08 13:50:42 -06:00
Himanshu Gupta
308211cc18
math expression language with parser/lexer generated using ANTLR
2016-04-08 11:40:29 -05:00
Himanshu Gupta
36ccfbd20e
math expression language with hand written parser/lexer
2016-04-08 11:40:29 -05:00
Charles Allen
2b99f717e4
Move lookup config doc to proper location
2016-04-08 08:15:38 -07:00
Nishant
edd74f2b67
Allow Lite DataSegment Announcements
...
separate config for each skipping dimensions, metrics and loadSpec
Add test
fix test comment
Add docs
2016-04-07 18:24:12 +05:30
Charles Allen
f915a59138
Merge pull request #2691 from metamx/lookupExtrFn
...
Add ExtractionFn to LookupExtractor bridge
2016-04-06 09:13:08 -07:00
jon-wei
0e481d6f93
Allow filters to use extraction functions
2016-04-05 13:24:56 -07:00
Fangjin Yang
eea7a47870
Merge pull request #2576 from navis/paging-from-next
...
Add option for select query to get next page without modifying returned paging identifiers
2016-04-01 13:50:36 -07:00
Fangjin Yang
4eb5a2c4f1
Merge pull request #2715 from navis/stringformat-null-handling
...
stringFormat extractionFn should be able to return null on null values (Fix for #2706 )
2016-04-01 13:45:28 -07:00
navis.ryu
077522a46f
stringFormat extractionFn should be able to return null on null values (Fix for #2706 )
2016-04-01 13:40:56 +09:00
navis.ryu
29bb00535b
Add option for select query to get next page without modifying returned paging identifiers
2016-04-01 09:03:03 +09:00
fjy
14dbc431ef
clean up for extensions docs
2016-03-30 17:14:58 -07:00
Fangjin Yang
a8b28879f1
Merge pull request #2369 from du00cs/master
...
[Feature] Extension: Offline Ingestion with limited Parquet Support
2016-03-29 23:19:35 -07:00
Fangjin Yang
23a8830bc2
Merge pull request #2757 from druid-io/fix-conf
...
Update libraries.md
2016-03-29 21:32:01 -07:00
DuNinglin [杜宁林]
0f67ff7dfb
reoganize code folder according to recent upstream folder changes, seperate it from avro code and take it into extensions-conrib. docs rewite too
2016-03-30 11:21:41 +08:00
Gian Merlino
1853f36e9f
More consistent empty-set filtering behavior on multi-value columns.
...
The behavior is now that filters on "null" will match rows with no
values. The behavior in the past was inconsistent; sometimes these
filters would match and sometimes they wouldn't.
Adds tests for this behavior to SelectorFilterTest and
BoundFilterTest, for query-level filters and filtered aggregates.
Fixes #2750 .
2016-03-29 15:32:13 -07:00
r4ruchir
4bff008d65
Update libraries.md
...
Adding embedded-druid information in helper libraries
2016-03-29 15:16:36 -07:00
Fangjin Yang
1e02eeab13
Merge pull request #2683 from metamx/default_retry
...
Better defaults for Retry policy for task actions
2016-03-29 08:02:59 -07:00
fjy
c418a55638
cleanup distinct count agg
2016-03-28 17:29:41 -07:00
Fangjin Yang
62c1dc7a09
Merge pull request #2602 from binlijin/distinctcount
...
implement special distinctcount
2016-03-28 17:20:17 -07:00
Fangjin Yang
9cb197adec
Merge pull request #2722 from himanshug/fix_hadoop_jar_upload
...
config to explicitly specify classpath for hadoop container during hadoop ingestion
2016-03-28 14:49:03 -07:00
Charles Allen
4764e86409
Add docs for RegisteredDimensionExtractionFn
2016-03-28 13:27:49 -07:00
Gian Merlino
dbdfcd2443
Fix extension reference in Kafka namespaced lookup docs.
...
The reference to io.druid.extensions:kafka-extraction-namespace is wrong (should
be druid-kafka-extraction-namespace) and unnecessary (the extension id is written
at the top of the doc file).
2016-03-28 09:23:24 -07:00
Fangjin Yang
a0216dcf7d
Merge pull request #2735 from metamx/fixlookupDocs
...
Move lookup docs that are in druid-proper back into lookups.md
2016-03-26 15:38:48 -07:00
Charles Allen
ab324e4ac0
Move lookup docs that are in druid-proper back into lookups.md
2016-03-25 10:46:50 -07:00
Gian Merlino
6d18382fb2
Fix broken link in datasketches-aggregators.md.
2016-03-25 09:32:40 -07:00
Himanshu Gupta
e78a469fb7
UTs for ExtensionsConfig
2016-03-25 10:51:28 -05:00
Himanshu Gupta
004b00bb96
config to explicitly specify classpath for hadoop container during hadoop ingestion
2016-03-25 10:51:28 -05:00
Bingkun Guo
0fa04305a6
refine description for mergeBytesLimit
2016-03-24 13:17:24 -05:00
binlijin
2729efca71
implement special distinctcount
2016-03-24 11:11:11 +08:00
Robin
448e0127b9
dynamic config endpoint is at coordinator
2016-03-23 17:22:19 -05:00
Fangjin Yang
a5d5529749
Merge pull request #2711 from gianm/filtered-aggregator-impls
...
All Filters should work with FilteredAggregators.
2016-03-23 13:37:21 -07:00
Gian Merlino
dd86198902
All Filters should work with FilteredAggregators.
...
This removes Filter.makeMatcher(ColumnSelectorFactory) and adds a
ValueMatcherFactory implementation to FilteredAggregatorFactory so it can
take advantage of existing makeMatcher(ValueMatcherFactory) implementations.
This patch also removes the Bound-based method from ValueMatcherFactory. Its
only user was the SpatialFilter, which could use the Predicate-based method.
Fixes #2604 .
2016-03-23 12:24:01 -07:00
Gian Merlino
2dfd3877c0
Fix a bunch of broken links in the docs.
2016-03-23 10:21:28 -07:00
Fangjin Yang
d1f8f2b2fd
Merge pull request #2698 from druid-io/fix-ext-docs
...
refactor extensions into their own docs
2016-03-22 22:04:12 -07:00
fjy
943cbe6e76
refactor extensions into their own docs
2016-03-22 18:54:10 -07:00
Fangjin Yang
041350c31b
Merge pull request #2701 from gianm/mvd-docs
...
Improved docs for multi-value dimensions.
2016-03-22 18:09:37 -07:00
Gian Merlino
451c0bc6d8
Merge pull request #2702 from pjain1/improve_docs
...
how to query in the querying section, correct default for select strategy, formatting
2016-03-22 16:40:35 -07:00
Parag Jain
39ecb9929d
how to query, correct default for select strategy, formatting
2016-03-22 17:06:15 -05:00
Gian Merlino
ff25325f3b
Improved docs for multi-value dimensions.
...
- Add central doc for multi-value dimensions, with some content from other docs.
- Link to multi-value dimension doc from topN and groupBy docs.
- Fixes a broken link from dimensionspecs.md, which was presciently already
linking to this nonexistent doc.
- Resolve inconsistent naming in docs & code (sometimes "multi-valued", sometimes
"multi-value") in favor of "multi-value".
2016-03-22 14:40:55 -07:00
Himanshu
3220b109ad
Merge pull request #2570 from binlijin/single_dimension_partitioning
...
Single dimension hash-based partitioning
2016-03-22 11:51:06 -05:00
binlijin
bce600f5d5
Single dimension hash-based partitioning
2016-03-22 13:15:33 +08:00
Nishant
11b8d1ed70
Merge pull request #2686 from gianm/fix-analysistypes-docs
...
Fix analysisTypes docs for SegmentMetadataQuery.
2016-03-18 16:15:38 -07:00
Gian Merlino
76ae30604e
Fix analysisTypes docs for SegmentMetadataQuery.
2016-03-18 13:17:33 -07:00
Nishant
ed8f39fcfe
Better defaults for Retry policy for task actions
...
This PR changes the retry of task actions to be a bit more aggressive
by reducing the maxWait. Current defaults were 1 min to 10 mins, which
lead to a very delayed recovery in case there are any transient network
issues between the overlord and the peons.
doc changes.
2016-03-18 11:59:55 -07:00
Charles Allen
5da9a280b6
Query Time Lookup - Dynamic Configuration
2016-03-18 09:45:05 -07:00
Slim
cf342d8d3c
Merge pull request #2517 from b-slim/adding_lookup_snapshot_utility
...
[QTL][Lookup] lookup module with the snapshot utility
2016-03-17 11:39:47 -05:00
Slim Bouguerra
0c86b29ef0
lookup module with the snapshot utility
2016-03-17 09:20:41 -05:00
Fangjin Yang
8cea85816d
Merge pull request #2668 from navis/fix-document-selectquery
...
Document for search query was not updated properly (Fix for #2662 )
2016-03-15 20:34:27 -07:00
navis.ryu
71ee9e2aac
Document for search query is not updated properly (Fix for #2662 )
2016-03-16 09:22:26 +09:00
dclim
553b677971
caching doc fix
2016-03-15 17:09:33 -06:00
Gian Merlino
a938f0853b
Additional ports docs.
2016-03-14 19:11:18 -07:00
Jonathan Wei
5ec5ac92c6
Merge pull request #2382 from himanshug/broker_segment_tier_selection
...
at broker, if configured, only add segments from specific tiers to the timeline
2016-03-14 16:53:06 -07:00
Fangjin Yang
a41a70d370
Merge pull request #2651 from gianm/ports-docs
...
Docs on default ports.
2016-03-14 14:15:52 -07:00
Fangjin Yang
dbdbacaa18
Merge pull request #2260 from navis/cardinality-for-searchquery
...
Support cardinality for search query
2016-03-14 13:24:40 -07:00
Gian Merlino
e51277b96c
Docs on default ports.
2016-03-14 11:25:21 -07:00
rasahner
2861e854f0
Merge pull request #2540 from pjain1/remove_kill
...
Remove extra parameter from deleteDataSourceSpecificInterval endpoint and correct exception message for invalid interval
2016-03-14 11:16:23 -05:00
navis.ryu
be341bf4e3
Support cardinality for search query (Fix for #2260 )
2016-03-12 09:51:01 +09:00
Bingkun Guo
96c981cd0a
fix broken link for Tasks
2016-03-11 11:36:34 -06:00
Xavier Léauté
90d7409e1a
Merge pull request #2611 from himanshug/gp_by_max_limit
...
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-10 13:44:13 -08:00
Charles Allen
7b1bfbf704
Add documentation to modules about what should be excluded.
2016-03-10 10:18:33 -08:00
Gian Merlino
a2b1652787
Clarify parser docs.
...
- Clarify what parseSpecs are used for.
- Avro, Protobuf should use timeAndDims parseSpecs.
- Hadoop jobs should use hadoopyString string parsers.
2016-03-10 08:45:04 -08:00
fjy
e3e932a4d4
refactor extensions into core and contrib
2016-03-08 17:12:09 -08:00
Himanshu Gupta
ca5de3f583
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-08 15:03:59 -06:00
Fangjin Yang
8e36e6fa43
Merge pull request #2610 from dclim/add-combineText-doc
...
add combineText property and cleanup batch ingestion doc
2016-03-08 12:54:16 -08:00
Fangjin Yang
e7018f524f
Merge pull request #2598 from himanshug/handoff_timeout
...
optional ability to configure handoff wait timeout on realtime tasks
2016-03-08 12:43:36 -08:00
dclim
df29667a89
add combineText property and cleanup batch ingestion doc
2016-03-08 13:10:34 -07:00
Himanshu Gupta
099acb4966
allow groupBy max[Intermediate]Rows limit be overridable by context
2016-03-07 15:22:41 -06:00
Himanshu Gupta
0402636598
configurable handoffConditionTimeout in realtime tasks for segment handoff wait
2016-03-05 10:14:54 -06:00
Charles Allen
2ad134638d
Merge pull request #2589 from b-slim/fix_real_time
...
Make realtime kafka firehose skip corrupt message
2016-03-04 12:14:23 -08:00
Slim Bouguerra
623e89aa54
skip corrupt message
2016-03-04 08:30:40 -06:00
jisookim
177b575d41
fix default number of connections on broker config documentation
2016-03-03 13:50:48 -08:00
Björn Zettergren
2462c82c0e
New defaults for maxRowsInMemory rowFlushBoundary
...
To bring consistency to docs and source this commit changes the default
values for maxRowsInMemory and rowFlushBoundary to 75000 after
discussion in PR https://github.com/druid-io/druid/pull/2457 .
The previous default was 500000 and it's lower now on the grounds that
it's better for a default to be somewhat less efficient, and work,
than to reach for the stars and possibly result in
"OutOfMemoryError: java heap space" errors.
2016-03-01 13:50:28 +01:00
Fangjin Yang
8f97b1e40c
Merge pull request #2554 from guobingkun/improve_include_extensions
...
improve doc on including druid and hadoop extensions
2016-02-26 20:05:57 -08:00
Charles Allen
c6803c4364
Allow specifying peon javaOpts as an array
2016-02-26 13:24:35 -08:00
Bingkun Guo
18f9e05f0f
improve doc on including druid and hadoop extensions
2016-02-26 13:53:08 -06:00
Gian Merlino
eb13d7afe3
Merge pull request #2521 from himanshug/fix_2497
...
RTR has multiple threads for assignment of pending tasks now
2016-02-26 08:14:15 -08:00
Nishant
9f8faabddb
Merge pull request #2469 from pdeva/patch-10
...
correct service names
2016-02-26 21:15:58 +05:30
Himanshu Gupta
bc156effe7
RTR has multiple threads for assignment of pending tasks now.
2016-02-26 09:27:03 -06:00
Charles Allen
1fe277ee29
Merge pull request #2367 from se7entyse7en/feature-rackspace-cloud-files-static-firehose
...
Adds support to use Rackspace's cloudfiles as static firehose
2016-02-25 17:31:06 -08:00
Parag Jain
b82b487f20
remove extra kill parameter
2016-02-24 17:16:18 -06:00
jon-wei
fd3782522c
Rename 'replaceMissingValues...' parameters in RegexExtractionFn
2016-02-24 13:12:56 -08:00
Nishant
6c9e1a28ad
Merge pull request #2519 from gianm/unparseable-handling
...
Better handling of ParseExceptions.
2016-02-24 04:46:29 +05:30
Fangjin Yang
93540c0631
Merge pull request #2503 from gianm/jetty-qos
...
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
2016-02-23 10:35:53 -08:00
Fangjin Yang
4f300cfe49
Merge pull request #2526 from druid-io/b-slim-patch-1
...
fix docs about sketches
2016-02-23 10:23:53 -08:00
Gian Merlino
3534483433
Better handling of ParseExceptions.
...
Two changes:
- Allow IncrementalIndex to suppress ParseExceptions on "aggregate".
- Add "reportParseExceptions" option to realtime tuning configs. By default this is "false".
Behavior of the counters should now be:
- processed: Number of rows indexed, including rows where some fields could be parsed and some could not.
- thrownAway: Number of rows thrown away due to rejection policy.
- unparseable: Number of rows thrown away due to being completely unparseable (no fields salvageable at all).
If "reportParseExceptions" is true then "unparseable" will always be zero (because a parse error would
cause an exception to be thrown). In addition, "processed" will only include fully parseable rows
(because even partial parse failures will cause exceptions to be thrown).
Fixes #2510 .
2016-02-23 10:11:43 -08:00
Slim
86c4900347
fix thetaSketch post aggregator doc
2016-02-23 10:43:54 -06:00
Himanshu Gupta
c7cb5bff14
fix thetaSketchSetOp doc
2016-02-23 09:17:49 -06:00
Fangjin Yang
0c984f9e32
Merge pull request #2109 from himanshug/segments_in_delta_ingestion
...
idempotent batch delta ingestion
2016-02-22 14:00:45 -08:00
Himanshu Gupta
f7679dd5a9
updating thetaSketchSetOp post agg documentation to reflect the possibility of nesting
2016-02-22 09:38:58 -06:00
Himanshu Gupta
21b0b8a07d
new coordinator endpoint to get list of used segment given a dataSource and list of intervals
2016-02-21 23:17:58 -06:00
Himanshu Gupta
09ffcae4ae
give user the option to specify the segments for dataSource inputSpec
2016-02-21 23:15:31 -06:00
Gian Merlino
23c993c9e7
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
...
- Add druid.indexer.server.maxChatRequests, which sets up a QoSFilter on the main Jetty server.
- Deprecate druid.indexer.runner.separateIngestionEndpoint
- Deprecate druid.indexer.server.chathandler.*
2016-02-19 13:36:09 -08:00
Bingkun Guo
9e4c908922
generate tarball by mvn package
2016-02-18 16:42:41 -06:00
Fangjin Yang
083f019a48
Merge pull request #2465 from druid-io/more-doc-fix
...
more doc fixes
2016-02-17 11:00:38 -08:00
Fangjin Yang
661c3aac4f
Merge pull request #2483 from druid-io/b-slim-patch-1
...
clarification about monitoring period
2016-02-17 09:46:25 -08:00
fjy
7da6594bfe
more doc fixes
2016-02-17 09:43:47 -08:00
Gian Merlino
3a996216bd
Multivalued dimensions can be compressed since 0.8.0.
2016-02-17 08:33:21 -08:00
Slim
e9f1c94822
Update metrics.md
2016-02-17 09:27:15 -06:00
Slim
ebbb1aa74e
Update metrics.md
2016-02-17 09:05:16 -06:00
pdeva
dd81b5ebe4
correct service names
...
use a `/` instead of `:` cause thats how the service names are declared in the respective config files of coordinator and overlord
2016-02-13 15:26:19 -08:00
Gian Merlino
e0c049c0b0
Make startup properties logging optional.
...
Off by default, but enabled in the example config files. See also #2452 .
2016-02-12 14:12:16 -08:00
Jonathan Wei
d63eec65a1
Merge pull request #2208 from navis/metadataquery-minmax
...
Support min/max values for metadata query
2016-02-11 17:28:07 -08:00
navis.ryu
dd2375477a
Support min/max values for metadata query ( #2208 )
2016-02-12 09:35:58 +09:00
navis.ryu
4d63196535
Support dimension spec for select query
2016-02-12 08:54:28 +09:00
Slim
368988d187
Merge pull request #2291 from druid-io/lookupManager
...
Promoting LookupExtractor state and LookupExtractorFactory to be a first class druid state object.
2016-02-11 16:07:27 -06:00
Fangjin Yang
6b45a9e4a7
Merge pull request #2449 from b-slim/fix_docs
...
fix docs about search query limit
2016-02-11 13:01:41 -08:00
Slim Bouguerra
438a4a9970
fix docs about search query limit
2016-02-11 13:20:59 -06:00
Himanshu
5779b32742
Merge pull request #2439 from metamx/fix2435
...
Make QuotableWhiteSpaceSplitter able to take JSON
2016-02-11 13:14:43 -06:00
Slim Bouguerra
4e119b7a24
Adding lookup ref manager and lookup dimension spec impl
2016-02-11 12:11:51 -06:00
Fangjin Yang
1430bc2c88
Merge pull request #2276 from harshjain2/feature-2021
...
Fix for issue 2021.
2016-02-10 17:04:45 -08:00
Fangjin Yang
f204dfbebe
Merge pull request #2413 from pdeva/patch-9
...
added note about including extension lib
2016-02-10 17:01:27 -08:00
Charles Allen
3a6452c6d4
Make QuotableWhiteSpaceSplitter able to take json
...
* Fixes #2435
2016-02-10 16:42:14 -08:00
Himanshu Gupta
d1cb17d3f7
at broker - only add segments from specific tiers to the timeline
2016-02-09 22:33:22 -06:00
Fangjin Yang
69a6bdcf03
Merge pull request #2427 from gianm/rolling-doc
...
Freshen up rolling update docs
2016-02-09 15:32:27 -08:00
Gian Merlino
95d5526e7c
Freshen up rolling update docs
...
1. Clarify what "Indexing Service / Realtime" means
2. Add info about restore-based middle manager rolling restarts
3. Add info about what happens in middle manager updates
4. More consistent capitalization and spelling of node types
2016-02-09 13:57:04 -08:00
Himanshu
f6eebf5884
Merge pull request #2422 from rasahner/docMinorFixes
...
some minor doc changes
2016-02-09 10:03:22 -06:00
Robin
1d57e3267d
some minor doc changes
2016-02-09 08:20:53 -06:00
pdeva
b75862da7e
make 0.9 compatible
2016-02-08 17:25:34 -08:00
David Lim
57bf61029b
Merge pull request #2416 from druid-io/fix-docs
...
fix docs
2016-02-08 15:09:20 -07:00
fjy
6fc5bcb1ef
fix docs
2016-02-08 13:40:53 -08:00
pdeva
525a911a3c
added note about including extension lib
2016-02-08 12:59:41 -08:00
Himanshu Gupta
b40c342cd1
make Global stupid pool cache size configurable
2016-02-05 14:18:06 -06:00
fjy
9e2295aa61
whitespace fixes
2016-02-04 16:25:51 -08:00
fjy
b52e1e9161
fix spacing again
2016-02-04 16:13:12 -08:00
fjy
962e7bac14
fix rendering
2016-02-04 15:58:20 -08:00
fjy
8405a7710e
tutorial fix
2016-02-04 15:37:47 -08:00
Fangjin Yang
e3273230cc
Merge pull request #2399 from metamx/md-to
...
move table of contents to Markdown and fix broken link
2016-02-04 15:28:29 -08:00
Xavier Léauté
eb593aeb90
css link is broken
...
removing in favor of adjusting on the site
2016-02-04 15:23:20 -08:00
Xavier Léauté
bbc5e5d470
move table of contents to Markdown
...
fixes #2397
2016-02-04 15:23:11 -08:00
fjy
003f54e268
add doc rendering
2016-02-04 14:21:59 -08:00
fjy
1aa363cea7
new quickstart
2016-02-04 09:37:38 -08:00
bin liu
39a4666387
fix doc format
2016-02-03 16:29:31 +08:00
Lou Marvin Caraig
9de57eb1c8
Added documentation
2016-02-02 14:32:12 +01:00
Sameer Al-Sakran
ee2a0e4afa
Update libraries.md
2016-02-01 11:47:50 -08:00
Fangjin Yang
459c2a49ca
Merge pull request #2364 from metamx/fix2356
...
Add more docs around timezone handling
2016-02-01 10:58:15 -08:00
Charles Allen
c9393e5289
Add more docs around timezone handling
...
* Fixes #2356
2016-02-01 08:51:07 -08:00
Jaebin Yoon
66a74a2b88
Fixed the broken link
2016-02-01 01:07:24 -08:00
Fangjin Yang
bbfb8aa7dd
Merge pull request #2358 from druid-io/addCommunityExtensions
...
Add Community Extensions
2016-01-31 17:45:11 -08:00
Erik Dubbelboer
246473c58a
Remove duplicate doc section
2016-01-30 13:50:32 +00:00
Charles Allen
5ec5c7221b
Add Community Extensions
...
Add a "Community Extensions" section to the known libraries
2016-01-29 13:09:15 -08:00
Fangjin Yang
728055f0eb
Merge pull request #2353 from bjozet/doc-fixes
...
DOCs: Missing 'type' for leaveIntermediate
2016-01-29 09:24:16 -08:00
Björn Zettergren
d373573c25
DOCs: Missing 'type' for leaveIntermediate
...
Added missing 'Boolean' as type for leaveIntermediate row in table TuningConfig
2016-01-29 14:42:19 +01:00
navis.ryu
55a888ea2f
time-descending result of select queries
2016-01-29 10:06:05 +09:00
Himanshu Gupta
b3437825f0
add ignoreWhenNoSegments flag to optionally ignore the dataSource inputSpec when no segments were found
2016-01-26 17:23:55 -06:00
Bingkun Guo
b07db4089c
fix doc: Setting druid.coordinator.merge.on will trigger an Append Task instead of Merge Task.
2016-01-26 10:20:32 -06:00
Rafael Abbondanza
145c65c72d
Updates number of parts in a topN query
...
This threw me off a bit, so I'm sure it may throw others off, too.
Updating from 10 to 11 parts.
2016-01-25 10:29:25 -05:00
Harsh Jain
41730b96d4
Fix for issue 2021
2016-01-25 02:48:22 +05:30
binlijin
cd1c71ceb4
rename persistBackgroundCount to numBackgroundPersistThreads
2016-01-22 14:29:41 +08:00
Gian Merlino
d416279c14
SegmentMetadataQuery support for returning aggregators.
2016-01-21 17:27:25 -08:00
Gian Merlino
87c8046c6c
Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments.
...
SegmentMetadataQuery stuff:
- Simplify implementation of SegmentAnalyzer.
- Fix type names for realtime complex columns; this used to try to merge a nice type
name (like "hyperUnique") from mmapped segments with the word "COMPLEX" from incremental
index segments, leading to a merge failure. Now it always uses the nice name.
- Add hasMultipleValues to ColumnAnalysis.
- Add tests for both mmapped and incremental index segments.
- Update docs to include errorMessage.
2016-01-21 15:50:33 -08:00
Slim Bouguerra
e0d90f875c
Graphite emitter
2016-01-21 13:43:37 -06:00
Fangjin Yang
ab635c5cd3
Merge pull request #2315 from rasahner/docUpdateBuild
...
docs: edits related to loadList
2016-01-21 09:52:27 -08:00
Robin
c9368702fa
do some editing of the instructions for using mysql for metadata
2016-01-21 10:37:30 -06:00
Slim
201539260c
Merge pull request #2076 from b-slim/issue_2010_upper_lower_extractionFN
...
adding lower and upper extraction fn
2016-01-21 09:58:07 -06:00
Slim Bouguerra
78feb3a13e
adding lower and upper extraction fn
2016-01-21 08:59:05 -06:00
Nishant
dcb7830330
Merge pull request #984 from drcrallen/thread-priority-rebase
...
Use thread priorities. (aka set `nice` values for background-like tasks)
2016-01-21 15:02:34 +05:30
Charles Allen
2a69a58570
Merge pull request #2149 from binlijin/master
...
Do persist IncrementalIndex in another thread in IndexGeneratorReducer
2016-01-20 17:06:42 -08:00
Charles Allen
2e1d6aaf3d
Use thread priorities. (aka set `nice` values for background-like tasks)
...
* Defaults the thread priority to java.util.Thread.NORM_PRIORITY in io.druid.indexing.common.task.AbstractTask
* Each exec service has its own Task Factory which is assigned a priority for spawned task. Therefore each priority class has a unique exec service
* Added priority to tasks as taskPriority in the task context. <0 means low, 0 means take default, >0 means high. It is up to any particular implementation to determine how to handle these numbers
* Add options to ForkingTaskRunner
* Add "-XX:+UseThreadPriorities" default option
* Add "-XX:ThreadPriorityPolicy=42" default option
* AbstractTask - Removed unneded @JsonIgnore on priority
* Added priority to RealtimePlumber executors. All sub-executors (non query runners) get Thread.MIN_PRIORITY
* Add persistThreadPriority and mergeThreadPriority to realtime tuning config
2016-01-20 14:00:31 -08:00
Logan Linn
c3bdaefe1f
Update batch-ingestion.md
...
Fix documented type of the `dataGranularity` config
2016-01-19 17:20:47 -08:00
binlijin
8e43e2c446
Do persist IncrementalIndex in another thread in IndexGeneratorReducer
2016-01-20 09:20:09 +08:00
Fangjin Yang
0c31f007fc
Merge pull request #1728 from himanshug/aggregators_in_segment_metadata
...
Store AggregatorFactory[] in segment metadata
2016-01-19 12:55:49 -08:00
Himanshu Gupta
a99aef29a1
adding aggregators to segment metadata
2016-01-19 14:23:39 -06:00
zhxiaog
3459a202ce
fixed #1873 , add ability to express CONCAT as an extractionFn
2016-01-18 15:03:17 -08:00
Fangjin Yang
592638f6b3
Merge pull request #2209 from sirpkt/cascade_extraction_functions
...
support cascade execution of extraction filters in extraction dimension spec
2016-01-18 07:59:36 -08:00
Gian Merlino
42ee473052
ZooKeeper 3.4.7 was unreleased, revert to 3.4.6.
2016-01-17 21:31:29 -08:00
Keuntae Park
238dd3be3c
support cascade execution of extraction filters in extraction dimension spec
2016-01-18 11:10:19 +09:00
Gian Merlino
2d3f6e7705
Some more multitenancy docs
2016-01-17 17:47:49 -08:00
Harsh Jain
71f1cd5e34
Fix for issue 2021
2016-01-17 16:10:04 +05:30
Fangjin Yang
f6a1a4ae20
Merge pull request #2138 from KurtYoung/feature-build-v9
...
build v9 directly
2016-01-16 13:35:46 -06:00
Kurt Young
82ff98c2bf
add config for build v9 directly and update docs
2016-01-16 11:26:34 +08:00
Fangjin Yang
2e54553a8f
Merge pull request #1990 from himanshug/schedule_kill_task
...
support periodic hard delete of segments
2016-01-15 15:22:33 -06:00
David Lim
34cd8f8c72
Merge pull request #2258 from fjy/acl-zk
...
acl for zookeeper is added
2016-01-15 10:27:08 -07:00
Nikita Geer
1908d63162
acl for zookeeper is added
2016-01-13 14:56:05 -08:00
navis.ryu
18479bb757
time-descending result of timeseries queries
2016-01-13 12:23:01 +09:00
Fangjin Yang
d7ad93debc
Merge pull request #2221 from binlijin/topN_minTopNThreshold
...
Allow change minTopNThreshold per topN query
2016-01-12 16:22:20 -08:00
Himanshu Gupta
eb2d251ac8
support periodic hard delete of segments
2016-01-12 16:55:05 -06:00
binlijin
2751f785f8
add doc
2016-01-12 11:25:11 +08:00
Fangjin Yang
fafc7c07ba
Merge pull request #2238 from rasahner/docServerviewEndpoint
...
doc: add information about new serverview coordinator endpoint
2016-01-11 10:44:18 -08:00
Robin
c1b813011c
doc: add information about new serverview coordinator endpoint
2016-01-10 19:47:23 -06:00
Himanshu
d255f4baac
Merge pull request #2234 from pjain1/emit_realtime_metrics
...
emit handoff count metrics
2016-01-08 14:24:16 -06:00
Parag Jain
9dba0f67e7
emit handoff count metrics
2016-01-08 12:36:13 -06:00
Himanshu Gupta
0d5f82aee7
document size attribute in thetaSketchSetOp post aggregator
2016-01-07 23:59:03 -06:00
Fangjin Yang
3048b1f0a5
Merge pull request #2174 from metamx/ingest-size-metrics
...
Add metrics for ingest/bytes/received for EventReceiverFirehose
2016-01-06 22:05:55 -08:00
Fangjin Yang
4cc81d3eff
Merge pull request #2096 from b-slim/add_use_case_unapply
...
Add use case unapply
2016-01-06 21:58:12 -08:00
Fangjin Yang
dd262f0451
Merge pull request #2215 from pjain1/fix_doc_metrics
...
correct metric name - segment/added/count -> segment/assigned/count
2016-01-06 16:21:54 -08:00
Parag Jain
768d07b702
correct metric name - segment/added/count -> segment/assigned/count
2016-01-06 15:55:11 -06:00
Slim Bouguerra
032d3bf6e6
Optimization of extraction filter by reversing the lookup
2016-01-06 11:16:11 -06:00
Himanshu
82bdfbbbf1
Merge pull request #2155 from metamx/taskConfigTmpdir
...
Make TaskConfig pull from java.io.tmpdir
2016-01-05 13:58:39 -06:00
Nishant
14989f272d
Add metrics for ingest/bytes/received for EventReceiverFirehose
...
review comments
review comments
2016-01-05 20:06:09 +05:30
Zhao Weinan
5e57ddb8cc
Adding avro support to realtime & hadoop batch indexing.
2016-01-05 10:21:27 +08:00
Robin
0961c0b703
trivial documentation fix
2016-01-04 12:39:10 -06:00
Himanshu
ebccea6bb0
Merge pull request #2188 from druid-io/doc-fix
...
small fixes to documentation
2016-01-04 12:25:26 -06:00
Charles Allen
e18301d99c
Make TaskConfig pull from java.io.tmpdir
...
* Also makes paths built off of java.nio.file.Paths instead of String.format
2016-01-04 10:17:08 -08:00
Robin
e280ab5f07
update zookeeper version to 3.4.7
2016-01-04 11:47:02 -06:00
fjy
d3d2ee03ce
minor fixes to docs
2016-01-03 11:37:06 -08:00
Gian Merlino
5a63c3dd63
Merge pull request #2186 from druid-io/dev-docs2
...
Add intro developer docs
2016-01-03 11:36:41 -05:00
fjy
88f6b9b5ad
Multiple improvements for docs
2016-01-02 21:54:54 -08:00
fjy
06a8e14820
Add intro developer docs
2016-01-02 14:44:45 -08:00
Himanshu Gupta
48de9dfafa
doc update to make it easy to find how to do re-indexing or delta ingestion
2015-12-30 23:58:09 -06:00
Himanshu Gupta
e1ea93b6fc
documenting querying behavior on multi-valued dimensions
2015-12-30 18:14:25 -06:00
Himanshu Gupta
b47d807738
Add support for filtering at DimensionSpec level so that multivalued dimensions can be filtered correctly
...
also adding UTs for multi-valued dimensions
2015-12-30 17:59:47 -06:00
Bingkun Guo
3c107c5757
Merge pull request #2150 from himanshug/emit_query_bytes
...
emit query/bytes metric
2015-12-30 13:44:19 -06:00
Fangjin Yang
e14ad74088
Merge pull request #1936 from b-slim/between_range_with_predicat
...
adding Upper/Lower Bound Filter
2015-12-29 10:11:22 -08:00
Fangjin Yang
b1261035a7
Merge pull request #1861 from guobingkun/insert_segment_tool
...
insert-segment tool
2015-12-29 10:06:07 -08:00
Fangjin Yang
5042b20eb4
Merge pull request #2164 from pdeva/patch-7
...
correct docs
2015-12-27 14:43:51 -08:00
pdeva
77863285e9
fix typo
2015-12-27 14:28:23 -08:00
pdeva
b308a13483
correct docs
2015-12-27 14:27:20 -08:00
pdeva
98fc08b0de
List who can access metadata storage
...
This is useful to know which ip addresses you need to allow to access your RDS instance for example
2015-12-27 14:25:17 -08:00
Himanshu Gupta
1a8546a682
emit query/bytes metric
2015-12-23 00:29:44 -06:00
Fangjin Yang
daaa24d7ad
Merge pull request #2145 from gianm/restoring-option
...
druid.indexer.task.restoreTasksOnRestart configuration.
2015-12-22 15:22:32 -08:00
Gian Merlino
bad270b6c4
druid.indexer.task.restoreTasksOnRestart configuration.
2015-12-22 10:59:15 -08:00
Himanshu Gupta
b96f560255
emit query/node/bytes metric
2015-12-21 23:23:20 -06:00
Charles Allen
c2a2d19d62
Merge pull request #2126 from guobingkun/remove_single_announcer
...
Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer
2015-12-21 08:55:07 -08:00
Bingkun Guo
951a4e9b35
Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer
2015-12-21 00:05:53 -06:00
Bingkun Guo
89b477970f
DataSegmentFinder tool
...
`insert-segment-to-db` is a tool that can insert segments into Druid metadata storage. It is intended to be used
to update the segment table in metadata storage after people manually migrate segments from one place to another.
It can also be used to insert missing segment into Druid, or even recover metadata storage by telling it where the
segments are stored.
Note: This tool expects users to have Druid cluster running in a "safe" mode, where there are no active tasks to interfere
the segments being inserted. Users can optionally bring down the cluster to make 100% sure nothing is interfering.
2015-12-21 00:02:04 -06:00
Fangjin Yang
7019d3c421
Merge pull request #2107 from jon-wei/fix_smq
...
More efficient SegmentMetadataQuery
2015-12-18 16:40:47 -08:00
fjy
398a3ec620
add docs for more specs
2015-12-17 18:06:30 -08:00
jon-wei
356b07c6c3
More efficient SegmentMetadataQuery
2015-12-17 12:46:23 -08:00
Jonathan Wei
f8cf84f466
Merge pull request #1995 from himanshug/num_rows_seg_metadata_query
...
add numRows to segment metadata query response
2015-12-17 12:23:46 -08:00
Fangjin Yang
cb085f99d3
Merge pull request #2097 from metamx/update-coordinator-docs
...
update the coordinator API docs to better explain how to format ISO86…
2015-12-15 17:35:48 -08:00
desaianuj
2f32cb1fb6
update the coordinator API docs to better explain how to format ISO8601 intervals in the request url
2015-12-15 16:17:54 -08:00
Gian Merlino
e6c2db89da
Merge pull request #1921 from jon-wei/flat_json
...
Add docs and benchmark for JSON flattening parser
2015-12-14 20:53:54 -08:00
Fangjin Yang
e7f06cf61c
Merge pull request #2075 from jon-wei/regex_extract
...
Configurable value replacement on match failure for RegexExtractionFn
2015-12-14 19:10:50 -08:00
jon-wei
c88f75df7c
Configurable value replacement on match failure for RegexExtractionFn
2015-12-14 17:57:41 -08:00
Steve M
2b5a010332
Change sample worker config spec with host:port instead of ip:port.
...
Also extend description of the 'affinity' property of the worker strategy
fillCapacityWithAffinity and fix a couple typos of middle manager (to
be more consistent throughout the page).
Add additional verbiage about appropriate middle manager host value.
2015-12-14 14:59:23 -08:00
Slim Bouguerra
77afdf25e3
adding Bound Filter
2015-12-10 08:47:21 -06:00
jon-wei
c53bf85d83
Add docs and benchmark for JSON flattening parser
2015-12-09 16:13:30 -08:00
Fangjin Yang
b0ab363022
Merge pull request #2052 from gianm/service-names
...
Change service names in docs, examples to match defaults in the code.
2015-12-08 15:40:35 -08:00
Xavier Léauté
dcd15735cd
Merge pull request #2015 from metamx/handoff-notifier-coordinator
...
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoff information
2015-12-08 14:06:39 -08:00
Nishant
9491e8de3b
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoffs
...
- fixes #1970
- extracted out segment handoff callbacks in SegmentHandoffNotifier
which is responsible for tracking segment handoffs and doing callbacks
when handoff is complete.
- Coordinator now maintains a view of segments in the cluster, this
will affect the jam heap requirements for the overlord for large
clusters.
realtime index task and nodes now use HTTP end points exposed by the
coordinator to get serverView
review comment
fix realtime node guide injection
review comments
make test not rely on scheduled exec
fix compilation
fix import
review comment
introduce immutableSegmentLoadInfo
fix son reading
remove unnecessary logging
2015-12-09 01:54:09 +05:30
Nishant
6c23d8edb4
Merge pull request #2043 from mangeshpardeshiyahoo/master
...
Add dimension selector support for groupby/having filters
2015-12-08 12:08:53 +05:30
Mangesh Pardeshi
d7ce120929
Add dimension selector support for groupby/having quries
2015-12-08 01:51:11 +00:00
michael.schiff
4ed0128b3b
removed documentation about (removed) DeleteTask
2015-12-07 15:56:46 -08:00
Fangjin Yang
00416e26ff
Merge pull request #2012 from himanshug/add_comment
...
adding more comments to why drops do not happen
2015-12-07 11:50:06 -08:00
Fangjin Yang
fe05545f58
Merge pull request #2055 from jkukul/documentation-fixes
...
Minor documentation fixes.
2015-12-07 10:50:38 -08:00
Jakub Kukul
8248f7f40c
Minor docs fixes in 'Evaluate Druid' section.
2015-12-07 19:42:52 +01:00
AdrieanKhisbe
5d10088e2d
Update Tutorial with new Zookeper Version and instructions
2015-12-07 06:50:42 +01:00
Gian Merlino
8e594a2e72
Change service names in docs, examples to match defaults in the code.
2015-12-06 10:04:21 -08:00
Himanshu Gupta
efe3c9f4a5
update the examples for batch reindexing/delta ingestion to use "intervals" instead of deprecated "interval"
2015-12-06 00:22:20 -06:00
Himanshu Gupta
61aaa09012
support multiple intervals in dataSource input spec
2015-12-03 21:28:04 -06:00
Himanshu
0eab8417cb
Merge pull request #2008 from codingwhatever/regex-search-query
...
Regex search query
2015-12-03 09:57:34 -06:00
Sam Groth
596b7ebd9a
Adding RegexSearchQuerySpec
2015-12-03 09:16:02 -06:00
Clint Wylie
68ef5f437a
fixes #1874 adding a substring extraction function, tests, and documentation
2015-12-01 23:50:32 -08:00
Himanshu Gupta
7c91219130
adding more comments to why drops do not happen when there are no segments in db
2015-11-25 16:26:51 -06:00
jon-wei
95dca4440f
Update data formats doc with info about JSON multi-value dimensions
2015-11-24 14:38:06 -08:00
Fangjin Yang
8e83d800d6
Merge pull request #1881 from gianm/restartable-tasks
...
Restorable indexing tasks
2015-11-23 21:14:37 -08:00
Gian Merlino
501dcb43fa
Some changes that make it possible to restart tasks on the same hardware.
...
This is done by killing and respawning the jvms rather than reconnecting to existing
jvms, for a couple reasons. One is that it lets you restore tasks after server reboots
too, and another is that it lets you upgrade all the software on a box at once by just
restarting everything.
The main changes are,
1) Add "canRestore" and "stopGracefully" methods to Tasks that say if a task can
stop gracefully, and actually do a graceful stop. RealtimeIndexTask is the only
one that currently implements this.
2) Add "stop" method to TaskRunners that attempts to do an orderly shutdown.
ThreadPoolTaskRunner- call stopGracefully on restorable tasks, wait for exit
ForkingTaskRunner- close output stream to restorable tasks, wait for exit
RemoteTaskRunner- do nothing special, we actually don't want to shutdown
3) Add "restore" method to TaskRunners that attempts to bootstrap tasks from last run.
Only ForkingTaskRunner does anything here. It maintains a "restore.json" file with
a list of restorable tasks.
4) Have the CliPeon's ExecutorLifecycle lock the task base directory to avoid a restored
task and a zombie old task from stomping on each other.
2015-11-23 11:22:08 -08:00
Gian Merlino
c0580bf063
Merge pull request #1928 from druid-io/new-compares
...
New comparisons for Druid
2015-11-20 16:40:59 -08:00
Himanshu Gupta
7a89b2e1a6
add numRows to segment metadata query response
2015-11-20 01:25:02 -06:00
Himanshu Gupta
fde9df2720
update to sketches-core-0.2.2 .
...
adds support for "cardinality" aggregator.
do not create sketch per event at ingestion time to make realtime ingestion faster
2015-11-19 01:05:59 -06:00
Fangjin Yang
87c43fbf02
Merge pull request #1981 from rasahner/fixFormattingSchemaDesign
...
fix formatting in schema-design
2015-11-17 17:51:16 -08:00
Charles Allen
8fcf2403e3
Merge pull request #1943 from metamx/realtime-caching
...
Enable caching on intermediate realtime persists
2015-11-17 15:06:43 -08:00
sahner
a4ed2ce2d1
fix formatting in schema-design
2015-11-17 16:50:53 -06:00
Himanshu
9118b010c0
Merge pull request #1976 from noddi/feature/count-parameter-rules-history
...
Add count parameter to rules history endpoint.
2015-11-17 14:41:50 -06:00
Charles Allen
dbe201aeed
Merge pull request #1929 from pjain1/jetty_threads
...
separate ingestion and query thread pool
2015-11-17 12:14:25 -08:00