Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,722 Commits 111 Branches 503 Tags 728 MiB
Commit Graph

1345 Commits

Author SHA1 Message Date
Gian Merlino c12712e8b8 Move "libraries.md" out of docs, onto the main site. (#3159) 2016-06-16 18:14:35 -07:00
Fangjin Yang 6c2fd75e46 update vs spark doc (#3116) 
* update vs spark doc

* update based on comments
 2016-06-15 10:30:19 -07:00
Gian Merlino 7da4a283a9 Add missing layout: toc to TOC. (#3144) 2016-06-14 10:48:05 -07:00
Gian Merlino dc2bf9efa5 Update absolute TOC links. (#3138) 
See druid-io/druid-io.github.io#286.
 2016-06-13 17:57:52 -07:00
Gian Merlino 3b3e772748 Add --no-default-remote-repositories flag to pull-deps. (#3120) 2016-06-13 17:01:18 +05:30
michaelschiff 7294ea87c3 link to statsd metrics emitter docs from development/extensions.html doc page (#3125) 2016-06-10 16:27:16 -07:00
Gian Merlino 5321ba3e8f Switch to absolute TOC (#3110) 
Depends on druid-io/druid-io.github.io#282
 2016-06-07 21:39:34 -07:00
Jonathan Wei c5dbf364e3 Fix JSON flatten docs, add link to path expression tester (#3105) 2016-06-07 14:39:57 -07:00
Kirill Kozlov 4ab675e863 Fix command name in example (#3088) 2016-06-07 10:44:27 -07:00
Kirill Kozlov 9f93be448e Fix logical operator in example (#3093) 2016-06-07 10:44:18 -07:00
Gian Merlino 99ee3f4dc3 Fixups, clarifications to lookup docs. (#3060) 2016-06-07 10:43:35 -07:00
Charles Allen fa41a6466a Cleanup the base lookup cluster wide config docs (#3061) 
* Cleanup the base lookup cluster wide config docs

* Add better examples in lookups-cached-global.md

* Use actual valid stock lookups

* Fixed maps with :

* Add mix of lookups

* Better examples in extension

* Remove unneeded namespace requirement

* Add extra line space

* Add link to lookup tiers

* Renamed header
 2016-06-07 10:42:41 -07:00
Charles Allen 8cac710546 Async lookups-cached-global by default (#3074) 
* Async lookups-cached-global by default
* Also better lookup docs

* Fix test timeouts

* Fix timing of deserialized test

* Fix problem with 0 wait failing immediately
 2016-06-03 15:58:10 -05:00
David Lim a2290a8f05 support seamless config changes (#3051) 2016-06-03 13:50:19 -07:00
Gian Merlino 2db5f49f35 Fix JavaScriptConfig. (#3062) 2016-06-02 23:59:00 -07:00
Gian Merlino 603fbbcc20 Fix docs for "contains" search spec. (#3066) 2016-06-02 19:03:40 -07:00
Vadim Ogievetsky 13c267bfee Added new line for site formatting (#3059) 2016-06-02 11:36:45 -07:00
Parag Jain 44237e25d9 fix duration format and number format (#3057) 2016-06-02 10:09:21 -07:00
Erik Dubbelboer b4737336e5 Added info about Google Cloud Storage (#3056) 2016-06-02 10:06:07 -07:00
Vadim Ogievetsky 767190d5db Clear up confusing wording (#3052) 
There is no such thing as a "Java aggregator" in Druid from a user's point of view, there are just native aggregator that happen to be implemented in Java.
 2016-06-01 15:41:50 -07:00
Gian Merlino 5e44ed7132 Fix redirect-making and version-replacing in deploy-docs. (#3041) 2016-05-31 16:36:10 -07:00
Gian Merlino cd5c5419bb Make docs deploying better. (#3040) 
- Make redirects for old links based on _redirects.json
- Replace #{DRUIDVERSION} tokens in docs with current version
- Allow origins named something other than "origin"
- Can use either s3cmd or awscli, depending on availability
 2016-05-31 15:34:58 -07:00
David Lim f6c39cc844 Kafka task minimum message time (#3035) 
* add KafkaIndexTask support for minimumMessageTime

* add Kafka supervisor support for lateMessageRejectionPeriod
 2016-05-31 11:37:00 -07:00
scusjs ebb6831770 rm , of jobProperties. jackson can not parse it (#3012) 2016-05-26 09:46:33 -07:00
Charles Allen 245077b47f Fix formatting in lookups-cached-global.md (#3009) 2016-05-24 17:28:39 -07:00
Charles Allen c738c0e1cd Silly Typo in docs 2016-05-24 13:31:58 -07:00
Charles Allen 8024b915e2 [QTL] Implement LookupExtractorFactory of namespaced lookup (#2926) 
* support LookupReferencesManager registration of namespaced lookup and eliminate static configurations for lookup from namespecd lookup extensions

- druid-namespace-lookup and druid-kafka-extraction-namespace are modified
- However, druid-namespace-lookup still has configuration about ON/OFF
  HEAP cache manager selection, which is not namespace wide
  configuration but node wide configuration as multiple namespace shares
  the same cache manager

* update KafkaExtractionNamespaceTest to reflect argument signature changes

* Add more synchronization functionality to NamespaceLookupExtractorFactory

* Remove old way of using extraction namespaces

* resolve compile error by supporting LookupIntrospectHandler

* Remove kafka lookups

* Remove unused stuff

* Fix start and stop behavior to be consistent with new javadocs

* Remove unused strings

* Add timeout option

* Address comments on configurations and improve docs

* Add more options and update hash key and replaces

* Move monitoring to the overriding classes

* Add better start/stop logging

* Remove old docs about namespace names

* Fix bad comma

* Add `@JsonIgnore` to lookup factory

* Address code review comments

* Remove ExtractionNamespace from module json registration

* Fix problems with naming and initialization. Add tests

* Optimize imports / reformat

* Fix future not being properly cancelled on failed initial scheduling

* Fix delete returns

* Add more docs about whole introspection

* Add `/version` introspection point for lookups

* Add more tests and address comments

* Add StaticMap extraction namespace for testing. Also add a bunch of tests

* Move cache system property to `druid.lookup.namespace.cache.type`

* Make VERSION lower case

* Change poll period to 0ms  for StaticMap

* Move cache key to bytebuffer

* Change hashCode and equals on static map extraction fn

* Add more comments on StaticMap

* Address comments

* Make scheduleAndWait use a latch

* Sanity renames and fix imports

* Remove extra info in docs

* Fix review comments

* Strengthen failure on start from warn to error

* Address comments

* Rename namespace-lookup to lookups-cached-global

* Fix injective mis-naming
* Also add serde test
 2016-05-24 10:56:40 -07:00
Nishant 0ac1b27d53 Allow manually setting of shutoffTime for EventReceiverFirehose (#2803) 
* Allow dynamically setting of shutoffTime for EventReceiverFirehose

Allow dynamically setting shutoffTime for EventReceiverFirehose

review comments and tests

* shut down exec on close
 2016-05-24 07:24:00 -07:00
Nishant dea4391a49 fix broken links (#3003) 2016-05-23 06:38:21 -07:00
Fangjin Yang 00de26c76a fix extensions docs (#2995) 
* fix extensions docs

* fix mistakes
 2016-05-19 14:01:06 -07:00
Charles Allen eaaad01de7 [QTL] Datasource as lookupTier (#2955) 
* Datasource as lookup tier
* Adds an option to let indexing service tasks pull their lookup tier from the datasource they are working for.

* Fix bad docs for lookups lookupTier

* Add Datasource name holder

* Move task and datasource to be pulled from Task file

* Make LookupModule pull from bound dataSource

* Fix test

* Fix code style on imports

* Fix formatting

* Make naming better

* Address code comments about naming
 2016-05-17 15:44:42 -07:00
Shekhar Gulati c41bfe50d0 Using quotes around the cp (#2934) 2016-05-16 15:16:48 -07:00
Parag Jain e3ea842cd3 add available query granularity strings (#2960) 2016-05-12 18:49:31 -07:00
Joe Pettersson 2288c78395 chore_fix-quickstart-docs (#2946) 
Fixes a small grammatical error in the `./docs/content/tutorials/quickstart.md` whereby a sentence didn't make sense
 2016-05-10 09:52:24 -07:00
Slim 45b2e65d75 [QTL] adding listDelimiter to lookup parser spec (#2941) 
* adding listDelimiter to lookup parser spec

* cleaning code
 2016-05-10 15:41:16 +05:30
Gian Merlino b8af84d1fc Update tutorials to tranquility v0.8.0. (#2937) 2016-05-09 11:50:37 -07:00
Gian Merlino fffa9c8265 Fix flattenSpec docs, "nested" should be "path". (#2924) 2016-05-05 08:59:41 -07:00
David Lim b489f63698 Supervisor for KafkaIndexTask (#2656) 
* supervisor for kafka indexing tasks

* cr changes
 2016-05-04 23:13:13 -07:00
Charles Allen 44e52acfc0 Link up metrics configuration to what they mean (#2921) 2016-05-04 10:30:02 -07:00
Himanshu 8e2742b7e8 adding QueryGranularity to segment metadata and optionally expose same from segmentMetadata query (#2873) 2016-05-03 11:31:10 -07:00
Navis Ryu 45a3a26ef7 Add more math functions (#2822) 
* Add more math functions

* added function list
 2016-05-03 10:55:13 -07:00
Gian Merlino e680665f1c Fix Avro parseSpec example, "type" should be "format". (#2918) 2016-05-03 09:22:43 -07:00
Himanshu 6c5bf91f9a publish metrics numJettyConns to see how number of active jetty connections change over time (#2839) 
this can be compared with numer of active queries to see if requests are waiting in jetty queue
 2016-05-02 14:08:25 -07:00
Charles Allen 6b957aa072 [QTL] Make URI Exctraction Namespace take more sane arguments (#2738) 
* Make URI Exctraction Namespace take more sane arguments
* Fixes https://github.com/druid-io/druid/issues/2669

* Update docs

* Rename error message

* Undo overzealous deletion of docs

* Explain caching mechanism a bit more in docs
 2016-05-02 12:54:34 -07:00
Charles Allen 54b717bdc3 [QTL] Move kafka-extraction-namespace to the Lookup framework. (#2800) 
* Move kafka-extraction-namespace to the Lookup framework.

* Address comments

* Fix missing kafka introspection

* Fix tests to be less racy

* Make testing a bit more leniant

* Make tests even more forgiving

* Add comments to kafka lookup cache method

* Move startStopLock to just use started

* Make start() and stop() idempotent

* Forgot to update test after last change, test now accounts for idempotency

* Add extra idempotency on stop check

* Add more descriptive docs of behavior
 2016-05-02 09:45:13 -07:00
michaelschiff 2203a812bc statsd-emitter (#2410) 2016-04-28 18:41:02 -07:00
David Lim 890bdb543d doc fixes (#2897) 2016-04-28 15:34:58 -07:00
Slim 58510d826b fix emit wait time (#2869) 2016-04-26 17:07:03 -07:00
Slim 55785267e4 postAgg filedName must match name of AGG (#2874) 2016-04-22 11:11:54 -07:00
binlijin 9151099e08 add document for druid.segmentCache.numBootstrapThreads (#2872) 2016-04-22 12:06:08 +08:00
Himanshu 3cfd9c64c9 make singleThreaded groupBy query config overridable at query time (#2828) 
* make isSingleThreaded groupBy query processing overridable at query time

* refactor code in GroupByMergedQueryRunner to make processing of single threaded and parallel merging of runners consistent
 2016-04-21 17:12:58 -07:00
Slim 984a518c9f Merge pull request #2734 from b-slim/LookupIntrospection2 
[QTL][Lookup] adding introspection endpoint
 2016-04-21 12:15:57 -05:00
Gian Merlino c74391e54c JavaScript: Ability to disable. (#2853) 
Fixes #2852.
 2016-04-21 09:43:15 -05:00
Nishant dbf63f738f Add ability to filter segments for specific dataSources on broker without creating tiers (#2848) 
* Add back FilteredServerView removed in a32906c7fd to reduce memory usage using watched tiers.

* Add functionality to specify "druid.broker.segment.watchedDataSources"
 2016-04-19 10:10:06 -07:00
Gaurav Kumar f5822faca3 Fixed wrong parseSpec in Avro Hadoop Parser (#2846) 
`parseSpec` should contain `format` instead of `type`. It was wrongly defaulting to `tsv`
 2016-04-16 11:34:54 -07:00
du00cs 639d0630b8 jackson conflict workaround in hadooop ingestio & parquet extension coordinate update (#2817) 2016-04-13 14:20:33 -07:00
Fangjin Yang 0c4a42bb6f change toc entry (#2834) 2016-04-13 13:45:07 -07:00
Gian Merlino e320d13385 Fix various broken links in the docs. (#2833) 2016-04-13 13:30:01 -07:00
Gian Merlino 725ee1401d Update tranquility version in the docs. (#2832) 2016-04-13 11:33:59 -07:00
Gian Merlino aa25cc1f68 Fix up Kafka tutorial (#2831) 
1) Remove extraneous section
2) Remove -SNAPSHOT version
 2016-04-13 11:33:45 -07:00
Fangjin Yang abd951df1a Document how to use roaring bitmaps (#2824) 
* Document how to use roaring bitmaps

This fixes #2408.
While not all indexSpec properties are explained, it does explain how roaring bitmaps can be turned on.

* fix

* fix

* fix

* fix
 2016-04-12 19:28:02 -07:00
Charles Allen ed5377465a add AirBnB Caravel to list of libraries (#2719) 2016-04-12 12:53:50 -07:00
Sébastien Launay 37d2ab623e Merge pull request #2815 from slaunay/documentation/hadoop-classpath-issue-fix-with-configuration 
Doc for mapreduce.job.user.classpath.first=true
 2016-04-12 10:51:51 -07:00
Nishant deb6ecf919 handle review comments for PR 2784 
https://github.com/druid-io/druid/pull/2784#discussion_r59062021
 2016-04-12 21:52:00 +05:30
Fangjin Yang bd6bd34cd8 Merge pull request #2090 from himanshug/math_exp 
math expression support
 2016-04-11 21:36:17 -07:00
Fangjin Yang 234125878a Merge pull request #2808 from metamx/moveLookupSaveStateConfigDocs 
Move lookup config doc to proper location
 2016-04-08 13:50:42 -06:00
Himanshu Gupta 308211cc18 math expression language with parser/lexer generated using ANTLR 2016-04-08 11:40:29 -05:00
Himanshu Gupta 36ccfbd20e math expression language with hand written parser/lexer 2016-04-08 11:40:29 -05:00
Charles Allen 2b99f717e4 Move lookup config doc to proper location 2016-04-08 08:15:38 -07:00
Nishant edd74f2b67 Allow Lite DataSegment Announcements 
separate config for each skipping dimensions, metrics and loadSpec

Add test

fix test comment

Add docs
 2016-04-07 18:24:12 +05:30
Charles Allen f915a59138 Merge pull request #2691 from metamx/lookupExtrFn 
Add ExtractionFn to LookupExtractor bridge
 2016-04-06 09:13:08 -07:00
jon-wei 0e481d6f93 Allow filters to use extraction functions 2016-04-05 13:24:56 -07:00
Fangjin Yang eea7a47870 Merge pull request #2576 from navis/paging-from-next 
Add option for select query to get next page without modifying returned paging identifiers
 2016-04-01 13:50:36 -07:00
Fangjin Yang 4eb5a2c4f1 Merge pull request #2715 from navis/stringformat-null-handling 
stringFormat extractionFn should be able to return null on null values (Fix for #2706)
 2016-04-01 13:45:28 -07:00
navis.ryu 077522a46f stringFormat extractionFn should be able to return null on null values (Fix for #2706) 2016-04-01 13:40:56 +09:00
navis.ryu 29bb00535b Add option for select query to get next page without modifying returned paging identifiers 2016-04-01 09:03:03 +09:00
fjy 14dbc431ef clean up for extensions docs 2016-03-30 17:14:58 -07:00
Fangjin Yang a8b28879f1 Merge pull request #2369 from du00cs/master 
[Feature] Extension: Offline Ingestion with limited Parquet Support
 2016-03-29 23:19:35 -07:00
Fangjin Yang 23a8830bc2 Merge pull request #2757 from druid-io/fix-conf 
Update libraries.md
 2016-03-29 21:32:01 -07:00
DuNinglin [杜宁林] 0f67ff7dfb reoganize code folder according to recent upstream folder changes, seperate it from avro code and take it into extensions-conrib. docs rewite too 2016-03-30 11:21:41 +08:00
Gian Merlino 1853f36e9f More consistent empty-set filtering behavior on multi-value columns. 
The behavior is now that filters on "null" will match rows with no
values. The behavior in the past was inconsistent; sometimes these
filters would match and sometimes they wouldn't.

Adds tests for this behavior to SelectorFilterTest and
BoundFilterTest, for query-level filters and filtered aggregates.

Fixes #2750.
 2016-03-29 15:32:13 -07:00
r4ruchir 4bff008d65 Update libraries.md 
Adding embedded-druid information in helper libraries
 2016-03-29 15:16:36 -07:00
Fangjin Yang 1e02eeab13 Merge pull request #2683 from metamx/default_retry 
Better defaults for Retry policy for task actions
 2016-03-29 08:02:59 -07:00
fjy c418a55638 cleanup distinct count agg 2016-03-28 17:29:41 -07:00
Fangjin Yang 62c1dc7a09 Merge pull request #2602 from binlijin/distinctcount 
implement special distinctcount
 2016-03-28 17:20:17 -07:00
Fangjin Yang 9cb197adec Merge pull request #2722 from himanshug/fix_hadoop_jar_upload 
config to explicitly specify classpath for hadoop container during hadoop ingestion
 2016-03-28 14:49:03 -07:00
Charles Allen 4764e86409 Add docs for RegisteredDimensionExtractionFn 2016-03-28 13:27:49 -07:00
Gian Merlino dbdfcd2443 Fix extension reference in Kafka namespaced lookup docs. 
The reference to io.druid.extensions:kafka-extraction-namespace is wrong (should
be druid-kafka-extraction-namespace) and unnecessary (the extension id is written
at the top of the doc file).
 2016-03-28 09:23:24 -07:00
Fangjin Yang a0216dcf7d Merge pull request #2735 from metamx/fixlookupDocs 
Move lookup docs that are in druid-proper back into lookups.md
 2016-03-26 15:38:48 -07:00
Charles Allen ab324e4ac0 Move lookup docs that are in druid-proper back into lookups.md 2016-03-25 10:46:50 -07:00
Gian Merlino 6d18382fb2 Fix broken link in datasketches-aggregators.md. 2016-03-25 09:32:40 -07:00
Himanshu Gupta e78a469fb7 UTs for ExtensionsConfig 2016-03-25 10:51:28 -05:00
Himanshu Gupta 004b00bb96 config to explicitly specify classpath for hadoop container during hadoop ingestion 2016-03-25 10:51:28 -05:00
Bingkun Guo 0fa04305a6 refine description for mergeBytesLimit 2016-03-24 13:17:24 -05:00
binlijin 2729efca71 implement special distinctcount 2016-03-24 11:11:11 +08:00
Robin 448e0127b9 dynamic config endpoint is at coordinator 2016-03-23 17:22:19 -05:00
Fangjin Yang a5d5529749 Merge pull request #2711 from gianm/filtered-aggregator-impls 
All Filters should work with FilteredAggregators.
 2016-03-23 13:37:21 -07:00
Gian Merlino dd86198902 All Filters should work with FilteredAggregators. 
This removes Filter.makeMatcher(ColumnSelectorFactory) and adds a
ValueMatcherFactory implementation to FilteredAggregatorFactory so it can
take advantage of existing makeMatcher(ValueMatcherFactory) implementations.

This patch also removes the Bound-based method from ValueMatcherFactory. Its
only user was the SpatialFilter, which could use the Predicate-based method.

Fixes #2604.
 2016-03-23 12:24:01 -07:00
Gian Merlino 2dfd3877c0 Fix a bunch of broken links in the docs. 2016-03-23 10:21:28 -07:00
Fangjin Yang d1f8f2b2fd Merge pull request #2698 from druid-io/fix-ext-docs 
refactor extensions into their own docs
 2016-03-22 22:04:12 -07:00
fjy 943cbe6e76 refactor extensions into their own docs 2016-03-22 18:54:10 -07:00
Fangjin Yang 041350c31b Merge pull request #2701 from gianm/mvd-docs 
Improved docs for multi-value dimensions.
 2016-03-22 18:09:37 -07:00
Gian Merlino 451c0bc6d8 Merge pull request #2702 from pjain1/improve_docs 
how to query in the querying section, correct default for select strategy, formatting
 2016-03-22 16:40:35 -07:00
Parag Jain 39ecb9929d how to query, correct default for select strategy, formatting 2016-03-22 17:06:15 -05:00
Gian Merlino ff25325f3b Improved docs for multi-value dimensions. 
- Add central doc for multi-value dimensions, with some content from other docs.
- Link to multi-value dimension doc from topN and groupBy docs.
- Fixes a broken link from dimensionspecs.md, which was presciently already
  linking to this nonexistent doc.
- Resolve inconsistent naming in docs & code (sometimes "multi-valued", sometimes
  "multi-value") in favor of "multi-value".
 2016-03-22 14:40:55 -07:00
Himanshu 3220b109ad Merge pull request #2570 from binlijin/single_dimension_partitioning 
Single dimension hash-based partitioning
 2016-03-22 11:51:06 -05:00
binlijin bce600f5d5 Single dimension hash-based partitioning 2016-03-22 13:15:33 +08:00
Nishant 11b8d1ed70 Merge pull request #2686 from gianm/fix-analysistypes-docs 
Fix analysisTypes docs for SegmentMetadataQuery.
 2016-03-18 16:15:38 -07:00
Gian Merlino 76ae30604e Fix analysisTypes docs for SegmentMetadataQuery. 2016-03-18 13:17:33 -07:00
Nishant ed8f39fcfe Better defaults for Retry policy for task actions 
This PR changes the retry of task actions to be a bit more aggressive
by reducing the maxWait. Current defaults were 1 min to 10 mins, which
lead to a very delayed recovery in case there are any transient network
issues between the overlord and the peons.

doc changes.
 2016-03-18 11:59:55 -07:00
Charles Allen 5da9a280b6 Query Time Lookup - Dynamic Configuration 2016-03-18 09:45:05 -07:00
Slim cf342d8d3c Merge pull request #2517 from b-slim/adding_lookup_snapshot_utility 
[QTL][Lookup] lookup module with the snapshot utility
 2016-03-17 11:39:47 -05:00
Slim Bouguerra 0c86b29ef0 lookup module with the snapshot utility 2016-03-17 09:20:41 -05:00
Fangjin Yang 8cea85816d Merge pull request #2668 from navis/fix-document-selectquery 
Document for search query was not updated properly (Fix for #2662)
 2016-03-15 20:34:27 -07:00
navis.ryu 71ee9e2aac Document for search query is not updated properly (Fix for #2662) 2016-03-16 09:22:26 +09:00
dclim 553b677971 caching doc fix 2016-03-15 17:09:33 -06:00
Gian Merlino a938f0853b Additional ports docs. 2016-03-14 19:11:18 -07:00
Jonathan Wei 5ec5ac92c6 Merge pull request #2382 from himanshug/broker_segment_tier_selection 
at broker, if configured, only add segments from specific tiers to the timeline
 2016-03-14 16:53:06 -07:00
Fangjin Yang a41a70d370 Merge pull request #2651 from gianm/ports-docs 
Docs on default ports.
 2016-03-14 14:15:52 -07:00
Fangjin Yang dbdbacaa18 Merge pull request #2260 from navis/cardinality-for-searchquery 
Support cardinality for search query
 2016-03-14 13:24:40 -07:00
Gian Merlino e51277b96c Docs on default ports. 2016-03-14 11:25:21 -07:00
rasahner 2861e854f0 Merge pull request #2540 from pjain1/remove_kill 
Remove extra parameter from deleteDataSourceSpecificInterval endpoint and correct exception message for invalid interval
 2016-03-14 11:16:23 -05:00
navis.ryu be341bf4e3 Support cardinality for search query (Fix for #2260) 2016-03-12 09:51:01 +09:00
Bingkun Guo 96c981cd0a fix broken link for Tasks 2016-03-11 11:36:34 -06:00
Xavier Léauté 90d7409e1a Merge pull request #2611 from himanshug/gp_by_max_limit 
only allow lowering maxResults and maxIntermediateRows from groupBy query context
 2016-03-10 13:44:13 -08:00
Charles Allen 7b1bfbf704 Add documentation to modules about what should be excluded. 2016-03-10 10:18:33 -08:00
Gian Merlino a2b1652787 Clarify parser docs. 
- Clarify what parseSpecs are used for.
- Avro, Protobuf should use timeAndDims parseSpecs.
- Hadoop jobs should use hadoopyString string parsers.
 2016-03-10 08:45:04 -08:00
fjy e3e932a4d4 refactor extensions into core and contrib 2016-03-08 17:12:09 -08:00
Himanshu Gupta ca5de3f583 only allow lowering maxResults and maxIntermediateRows from groupBy query context 2016-03-08 15:03:59 -06:00
Fangjin Yang 8e36e6fa43 Merge pull request #2610 from dclim/add-combineText-doc 
add combineText property and cleanup batch ingestion doc
 2016-03-08 12:54:16 -08:00
Fangjin Yang e7018f524f Merge pull request #2598 from himanshug/handoff_timeout 
optional ability to configure handoff wait timeout on realtime tasks
 2016-03-08 12:43:36 -08:00
dclim df29667a89 add combineText property and cleanup batch ingestion doc 2016-03-08 13:10:34 -07:00
Himanshu Gupta 099acb4966 allow groupBy max[Intermediate]Rows limit be overridable by context 2016-03-07 15:22:41 -06:00
Himanshu Gupta 0402636598 configurable handoffConditionTimeout in realtime tasks for segment handoff wait 2016-03-05 10:14:54 -06:00
Charles Allen 2ad134638d Merge pull request #2589 from b-slim/fix_real_time 
Make realtime kafka firehose skip corrupt message
 2016-03-04 12:14:23 -08:00
Slim Bouguerra 623e89aa54 skip corrupt message 2016-03-04 08:30:40 -06:00
jisookim 177b575d41 fix default number of connections on broker config documentation 2016-03-03 13:50:48 -08:00
Björn Zettergren 2462c82c0e New defaults for maxRowsInMemory rowFlushBoundary 
To bring consistency to docs and source this commit changes the default
values for maxRowsInMemory and rowFlushBoundary to 75000 after
discussion in PR https://github.com/druid-io/druid/pull/2457.

The previous default was 500000 and it's lower now on the grounds that
it's better for a default to be somewhat less efficient, and work,
than to reach for the stars and possibly result in
"OutOfMemoryError: java heap space" errors.
 2016-03-01 13:50:28 +01:00
Fangjin Yang 8f97b1e40c Merge pull request #2554 from guobingkun/improve_include_extensions 
improve doc on including druid and hadoop extensions
 2016-02-26 20:05:57 -08:00
Charles Allen c6803c4364 Allow specifying peon javaOpts as an array 2016-02-26 13:24:35 -08:00
Bingkun Guo 18f9e05f0f improve doc on including druid and hadoop extensions 2016-02-26 13:53:08 -06:00
Gian Merlino eb13d7afe3 Merge pull request #2521 from himanshug/fix_2497 
RTR has multiple threads for assignment of pending tasks now
 2016-02-26 08:14:15 -08:00
Nishant 9f8faabddb Merge pull request #2469 from pdeva/patch-10 
correct service names
 2016-02-26 21:15:58 +05:30
Himanshu Gupta bc156effe7 RTR has multiple threads for assignment of pending tasks now. 2016-02-26 09:27:03 -06:00
Charles Allen 1fe277ee29 Merge pull request #2367 from se7entyse7en/feature-rackspace-cloud-files-static-firehose 
Adds support to use Rackspace's cloudfiles as static firehose
 2016-02-25 17:31:06 -08:00
Parag Jain b82b487f20 remove extra kill parameter 2016-02-24 17:16:18 -06:00
jon-wei fd3782522c Rename 'replaceMissingValues...' parameters in RegexExtractionFn 2016-02-24 13:12:56 -08:00
Nishant 6c9e1a28ad Merge pull request #2519 from gianm/unparseable-handling 
Better handling of ParseExceptions.
 2016-02-24 04:46:29 +05:30
Fangjin Yang 93540c0631 Merge pull request #2503 from gianm/jetty-qos 
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
 2016-02-23 10:35:53 -08:00
Fangjin Yang 4f300cfe49 Merge pull request #2526 from druid-io/b-slim-patch-1 
fix docs about sketches
 2016-02-23 10:23:53 -08:00
Gian Merlino 3534483433 Better handling of ParseExceptions. 
Two changes:
- Allow IncrementalIndex to suppress ParseExceptions on "aggregate".
- Add "reportParseExceptions" option to realtime tuning configs. By default this is "false".

Behavior of the counters should now be:

- processed: Number of rows indexed, including rows where some fields could be parsed and some could not.
- thrownAway: Number of rows thrown away due to rejection policy.
- unparseable: Number of rows thrown away due to being completely unparseable (no fields salvageable at all).

If "reportParseExceptions" is true then "unparseable" will always be zero (because a parse error would
cause an exception to be thrown). In addition, "processed" will only include fully parseable rows
(because even partial parse failures will cause exceptions to be thrown).

Fixes #2510.
 2016-02-23 10:11:43 -08:00
Slim 86c4900347 fix thetaSketch post aggregator doc 2016-02-23 10:43:54 -06:00
Himanshu Gupta c7cb5bff14 fix thetaSketchSetOp doc 2016-02-23 09:17:49 -06:00
Fangjin Yang 0c984f9e32 Merge pull request #2109 from himanshug/segments_in_delta_ingestion 
idempotent batch delta ingestion
 2016-02-22 14:00:45 -08:00
Himanshu Gupta f7679dd5a9 updating thetaSketchSetOp post agg documentation to reflect the possibility of nesting 2016-02-22 09:38:58 -06:00
Himanshu Gupta 21b0b8a07d new coordinator endpoint to get list of used segment given a dataSource and list of intervals 2016-02-21 23:17:58 -06:00
Himanshu Gupta 09ffcae4ae give user the option to specify the segments for dataSource inputSpec 2016-02-21 23:15:31 -06:00
Gian Merlino 23c993c9e7 Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports. 
- Add druid.indexer.server.maxChatRequests, which sets up a QoSFilter on the main Jetty server.
- Deprecate druid.indexer.runner.separateIngestionEndpoint
- Deprecate druid.indexer.server.chathandler.*
 2016-02-19 13:36:09 -08:00
Bingkun Guo 9e4c908922 generate tarball by mvn package 2016-02-18 16:42:41 -06:00
Fangjin Yang 083f019a48 Merge pull request #2465 from druid-io/more-doc-fix 
more doc fixes
 2016-02-17 11:00:38 -08:00
Fangjin Yang 661c3aac4f Merge pull request #2483 from druid-io/b-slim-patch-1 
clarification about monitoring period
 2016-02-17 09:46:25 -08:00
fjy 7da6594bfe more doc fixes 2016-02-17 09:43:47 -08:00
Gian Merlino 3a996216bd Multivalued dimensions can be compressed since 0.8.0. 2016-02-17 08:33:21 -08:00
Slim e9f1c94822 Update metrics.md 2016-02-17 09:27:15 -06:00
Slim ebbb1aa74e Update metrics.md 2016-02-17 09:05:16 -06:00
pdeva dd81b5ebe4 correct service names 
use a `/` instead of `:` cause thats how the service names are declared in the respective config files of coordinator and overlord
 2016-02-13 15:26:19 -08:00
Gian Merlino e0c049c0b0 Make startup properties logging optional. 
Off by default, but enabled in the example config files. See also #2452.
 2016-02-12 14:12:16 -08:00
Jonathan Wei d63eec65a1 Merge pull request #2208 from navis/metadataquery-minmax 
Support min/max values for metadata query
 2016-02-11 17:28:07 -08:00
navis.ryu dd2375477a Support min/max values for metadata query (#2208) 2016-02-12 09:35:58 +09:00
navis.ryu 4d63196535 Support dimension spec for select query 2016-02-12 08:54:28 +09:00
Slim 368988d187 Merge pull request #2291 from druid-io/lookupManager 
Promoting LookupExtractor state and LookupExtractorFactory to be a first class druid state object.
 2016-02-11 16:07:27 -06:00
Fangjin Yang 6b45a9e4a7 Merge pull request #2449 from b-slim/fix_docs 
fix docs about search query limit
 2016-02-11 13:01:41 -08:00
Slim Bouguerra 438a4a9970 fix docs about search query limit 2016-02-11 13:20:59 -06:00
Himanshu 5779b32742 Merge pull request #2439 from metamx/fix2435 
Make QuotableWhiteSpaceSplitter able to take JSON
 2016-02-11 13:14:43 -06:00
Slim Bouguerra 4e119b7a24 Adding lookup ref manager and lookup dimension spec impl 2016-02-11 12:11:51 -06:00
Fangjin Yang 1430bc2c88 Merge pull request #2276 from harshjain2/feature-2021 
Fix for issue 2021.
 2016-02-10 17:04:45 -08:00
Fangjin Yang f204dfbebe Merge pull request #2413 from pdeva/patch-9 
added note about including extension lib
 2016-02-10 17:01:27 -08:00
Charles Allen 3a6452c6d4 Make QuotableWhiteSpaceSplitter able to take json 
* Fixes #2435
 2016-02-10 16:42:14 -08:00
Himanshu Gupta d1cb17d3f7 at broker - only add segments from specific tiers to the timeline 2016-02-09 22:33:22 -06:00
Fangjin Yang 69a6bdcf03 Merge pull request #2427 from gianm/rolling-doc 
Freshen up rolling update docs
 2016-02-09 15:32:27 -08:00
Gian Merlino 95d5526e7c Freshen up rolling update docs 
1. Clarify what "Indexing Service / Realtime" means
2. Add info about restore-based middle manager rolling restarts
3. Add info about what happens in middle manager updates
4. More consistent capitalization and spelling of node types
 2016-02-09 13:57:04 -08:00
Himanshu f6eebf5884 Merge pull request #2422 from rasahner/docMinorFixes 
some minor doc changes
 2016-02-09 10:03:22 -06:00
Robin 1d57e3267d some minor doc changes 2016-02-09 08:20:53 -06:00
pdeva b75862da7e make 0.9 compatible 2016-02-08 17:25:34 -08:00
David Lim 57bf61029b Merge pull request #2416 from druid-io/fix-docs 
fix docs
 2016-02-08 15:09:20 -07:00
fjy 6fc5bcb1ef fix docs 2016-02-08 13:40:53 -08:00
pdeva 525a911a3c added note about including extension lib 2016-02-08 12:59:41 -08:00
Himanshu Gupta b40c342cd1 make Global stupid pool cache size configurable 2016-02-05 14:18:06 -06:00
fjy 9e2295aa61 whitespace fixes 2016-02-04 16:25:51 -08:00
fjy b52e1e9161 fix spacing again 2016-02-04 16:13:12 -08:00
fjy 962e7bac14 fix rendering 2016-02-04 15:58:20 -08:00
fjy 8405a7710e tutorial fix 2016-02-04 15:37:47 -08:00
Fangjin Yang e3273230cc Merge pull request #2399 from metamx/md-to 
move table of contents to Markdown and fix broken link
 2016-02-04 15:28:29 -08:00
Xavier Léauté eb593aeb90 css link is broken 
removing in favor of adjusting on the site
 2016-02-04 15:23:20 -08:00
Xavier Léauté bbc5e5d470 move table of contents to Markdown 
fixes #2397
 2016-02-04 15:23:11 -08:00
fjy 003f54e268 add doc rendering 2016-02-04 14:21:59 -08:00
fjy 1aa363cea7 new quickstart 2016-02-04 09:37:38 -08:00
bin liu 39a4666387 fix doc format 2016-02-03 16:29:31 +08:00
Lou Marvin Caraig 9de57eb1c8 Added documentation 2016-02-02 14:32:12 +01:00
Sameer Al-Sakran ee2a0e4afa Update libraries.md 2016-02-01 11:47:50 -08:00