Commit Graph

1793 Commits

Author SHA1 Message Date
Gian Merlino c74391e54c JavaScript: Ability to disable. (#2853)
Fixes #2852.
2016-04-21 09:43:15 -05:00
Nishant dbf63f738f Add ability to filter segments for specific dataSources on broker without creating tiers (#2848)
* Add back FilteredServerView removed in a32906c7fd to reduce memory usage using watched tiers.

* Add functionality to specify "druid.broker.segment.watchedDataSources"
2016-04-19 10:10:06 -07:00
Gaurav Kumar f5822faca3 Fixed wrong parseSpec in Avro Hadoop Parser (#2846)
`parseSpec` should contain `format` instead of `type`. It was wrongly defaulting to `tsv`
2016-04-16 11:34:54 -07:00
du00cs 639d0630b8 jackson conflict workaround in hadooop ingestio & parquet extension coordinate update (#2817) 2016-04-13 14:20:33 -07:00
Fangjin Yang 0c4a42bb6f change toc entry (#2834) 2016-04-13 13:45:07 -07:00
Gian Merlino e320d13385 Fix various broken links in the docs. (#2833) 2016-04-13 13:30:01 -07:00
Gian Merlino 725ee1401d Update tranquility version in the docs. (#2832) 2016-04-13 11:33:59 -07:00
Gian Merlino aa25cc1f68 Fix up Kafka tutorial (#2831)
1) Remove extraneous section
2) Remove -SNAPSHOT version
2016-04-13 11:33:45 -07:00
Fangjin Yang abd951df1a Document how to use roaring bitmaps (#2824)
* Document how to use roaring bitmaps

This fixes #2408.
While not all indexSpec properties are explained, it does explain how roaring bitmaps can be turned on.

* fix

* fix

* fix

* fix
2016-04-12 19:28:02 -07:00
Charles Allen ed5377465a add AirBnB Caravel to list of libraries (#2719) 2016-04-12 12:53:50 -07:00
Sébastien Launay 37d2ab623e Merge pull request #2815 from slaunay/documentation/hadoop-classpath-issue-fix-with-configuration
Doc for mapreduce.job.user.classpath.first=true
2016-04-12 10:51:51 -07:00
Nishant deb6ecf919 handle review comments for PR 2784
https://github.com/druid-io/druid/pull/2784#discussion_r59062021
2016-04-12 21:52:00 +05:30
Fangjin Yang bd6bd34cd8 Merge pull request #2090 from himanshug/math_exp
math expression support
2016-04-11 21:36:17 -07:00
Fangjin Yang 234125878a Merge pull request #2808 from metamx/moveLookupSaveStateConfigDocs
Move lookup config doc to proper location
2016-04-08 13:50:42 -06:00
Himanshu Gupta 308211cc18 math expression language with parser/lexer generated using ANTLR 2016-04-08 11:40:29 -05:00
Himanshu Gupta 36ccfbd20e math expression language with hand written parser/lexer 2016-04-08 11:40:29 -05:00
Charles Allen 2b99f717e4 Move lookup config doc to proper location 2016-04-08 08:15:38 -07:00
Nishant edd74f2b67 Allow Lite DataSegment Announcements
separate config for each skipping dimensions, metrics and loadSpec

Add test

fix test comment

Add docs
2016-04-07 18:24:12 +05:30
Charles Allen f915a59138 Merge pull request #2691 from metamx/lookupExtrFn
Add ExtractionFn to LookupExtractor bridge
2016-04-06 09:13:08 -07:00
jon-wei 0e481d6f93 Allow filters to use extraction functions 2016-04-05 13:24:56 -07:00
Fangjin Yang eea7a47870 Merge pull request #2576 from navis/paging-from-next
Add option for select query to get next page without modifying returned paging identifiers
2016-04-01 13:50:36 -07:00
Fangjin Yang 4eb5a2c4f1 Merge pull request #2715 from navis/stringformat-null-handling
stringFormat extractionFn should be able to return null on null values (Fix for #2706)
2016-04-01 13:45:28 -07:00
navis.ryu 077522a46f stringFormat extractionFn should be able to return null on null values (Fix for #2706) 2016-04-01 13:40:56 +09:00
navis.ryu 29bb00535b Add option for select query to get next page without modifying returned paging identifiers 2016-04-01 09:03:03 +09:00
fjy 14dbc431ef clean up for extensions docs 2016-03-30 17:14:58 -07:00
Fangjin Yang a8b28879f1 Merge pull request #2369 from du00cs/master
[Feature] Extension: Offline Ingestion with limited Parquet Support
2016-03-29 23:19:35 -07:00
Fangjin Yang 23a8830bc2 Merge pull request #2757 from druid-io/fix-conf
Update libraries.md
2016-03-29 21:32:01 -07:00
DuNinglin [杜宁林] 0f67ff7dfb reoganize code folder according to recent upstream folder changes, seperate it from avro code and take it into extensions-conrib. docs rewite too 2016-03-30 11:21:41 +08:00
Gian Merlino 1853f36e9f More consistent empty-set filtering behavior on multi-value columns.
The behavior is now that filters on "null" will match rows with no
values. The behavior in the past was inconsistent; sometimes these
filters would match and sometimes they wouldn't.

Adds tests for this behavior to SelectorFilterTest and
BoundFilterTest, for query-level filters and filtered aggregates.

Fixes #2750.
2016-03-29 15:32:13 -07:00
r4ruchir 4bff008d65 Update libraries.md
Adding embedded-druid information in helper libraries
2016-03-29 15:16:36 -07:00
Fangjin Yang 1e02eeab13 Merge pull request #2683 from metamx/default_retry
Better defaults for Retry policy for task actions
2016-03-29 08:02:59 -07:00
fjy c418a55638 cleanup distinct count agg 2016-03-28 17:29:41 -07:00
Fangjin Yang 62c1dc7a09 Merge pull request #2602 from binlijin/distinctcount
implement special distinctcount
2016-03-28 17:20:17 -07:00
Fangjin Yang 9cb197adec Merge pull request #2722 from himanshug/fix_hadoop_jar_upload
config to explicitly specify classpath for hadoop container during hadoop ingestion
2016-03-28 14:49:03 -07:00
Charles Allen 4764e86409 Add docs for RegisteredDimensionExtractionFn 2016-03-28 13:27:49 -07:00
Gian Merlino dbdfcd2443 Fix extension reference in Kafka namespaced lookup docs.
The reference to io.druid.extensions:kafka-extraction-namespace is wrong (should
be druid-kafka-extraction-namespace) and unnecessary (the extension id is written
at the top of the doc file).
2016-03-28 09:23:24 -07:00
Fangjin Yang a0216dcf7d Merge pull request #2735 from metamx/fixlookupDocs
Move lookup docs that are in druid-proper back into lookups.md
2016-03-26 15:38:48 -07:00
Charles Allen ab324e4ac0 Move lookup docs that are in druid-proper back into lookups.md 2016-03-25 10:46:50 -07:00
Gian Merlino 6d18382fb2 Fix broken link in datasketches-aggregators.md. 2016-03-25 09:32:40 -07:00
Himanshu Gupta e78a469fb7 UTs for ExtensionsConfig 2016-03-25 10:51:28 -05:00
Himanshu Gupta 004b00bb96 config to explicitly specify classpath for hadoop container during hadoop ingestion 2016-03-25 10:51:28 -05:00
Bingkun Guo 0fa04305a6 refine description for mergeBytesLimit 2016-03-24 13:17:24 -05:00
binlijin 2729efca71 implement special distinctcount 2016-03-24 11:11:11 +08:00
Robin 448e0127b9 dynamic config endpoint is at coordinator 2016-03-23 17:22:19 -05:00
Fangjin Yang a5d5529749 Merge pull request #2711 from gianm/filtered-aggregator-impls
All Filters should work with FilteredAggregators.
2016-03-23 13:37:21 -07:00
Gian Merlino dd86198902 All Filters should work with FilteredAggregators.
This removes Filter.makeMatcher(ColumnSelectorFactory) and adds a
ValueMatcherFactory implementation to FilteredAggregatorFactory so it can
take advantage of existing makeMatcher(ValueMatcherFactory) implementations.

This patch also removes the Bound-based method from ValueMatcherFactory. Its
only user was the SpatialFilter, which could use the Predicate-based method.

Fixes #2604.
2016-03-23 12:24:01 -07:00
Gian Merlino 2dfd3877c0 Fix a bunch of broken links in the docs. 2016-03-23 10:21:28 -07:00
Fangjin Yang d1f8f2b2fd Merge pull request #2698 from druid-io/fix-ext-docs
refactor extensions into their own docs
2016-03-22 22:04:12 -07:00
fjy 943cbe6e76 refactor extensions into their own docs 2016-03-22 18:54:10 -07:00
Fangjin Yang 041350c31b Merge pull request #2701 from gianm/mvd-docs
Improved docs for multi-value dimensions.
2016-03-22 18:09:37 -07:00
Gian Merlino 451c0bc6d8 Merge pull request #2702 from pjain1/improve_docs
how to query in the querying section, correct default for select strategy, formatting
2016-03-22 16:40:35 -07:00
Parag Jain 39ecb9929d how to query, correct default for select strategy, formatting 2016-03-22 17:06:15 -05:00
Gian Merlino ff25325f3b Improved docs for multi-value dimensions.
- Add central doc for multi-value dimensions, with some content from other docs.
- Link to multi-value dimension doc from topN and groupBy docs.
- Fixes a broken link from dimensionspecs.md, which was presciently already
  linking to this nonexistent doc.
- Resolve inconsistent naming in docs & code (sometimes "multi-valued", sometimes
  "multi-value") in favor of "multi-value".
2016-03-22 14:40:55 -07:00
Himanshu 3220b109ad Merge pull request #2570 from binlijin/single_dimension_partitioning
Single dimension hash-based partitioning
2016-03-22 11:51:06 -05:00
binlijin bce600f5d5 Single dimension hash-based partitioning 2016-03-22 13:15:33 +08:00
Nishant 11b8d1ed70 Merge pull request #2686 from gianm/fix-analysistypes-docs
Fix analysisTypes docs for SegmentMetadataQuery.
2016-03-18 16:15:38 -07:00
Gian Merlino 76ae30604e Fix analysisTypes docs for SegmentMetadataQuery. 2016-03-18 13:17:33 -07:00
Nishant ed8f39fcfe Better defaults for Retry policy for task actions
This PR changes the retry of task actions to be a bit more aggressive
by reducing the maxWait. Current defaults were 1 min to 10 mins, which
lead to a very delayed recovery in case there are any transient network
issues between the overlord and the peons.

doc changes.
2016-03-18 11:59:55 -07:00
Charles Allen 5da9a280b6 Query Time Lookup - Dynamic Configuration 2016-03-18 09:45:05 -07:00
Slim cf342d8d3c Merge pull request #2517 from b-slim/adding_lookup_snapshot_utility
[QTL][Lookup] lookup module with the snapshot utility
2016-03-17 11:39:47 -05:00
Slim Bouguerra 0c86b29ef0 lookup module with the snapshot utility 2016-03-17 09:20:41 -05:00
Fangjin Yang 8cea85816d Merge pull request #2668 from navis/fix-document-selectquery
Document for search query was not updated properly (Fix for #2662)
2016-03-15 20:34:27 -07:00
navis.ryu 71ee9e2aac Document for search query is not updated properly (Fix for #2662) 2016-03-16 09:22:26 +09:00
dclim 553b677971 caching doc fix 2016-03-15 17:09:33 -06:00
Gian Merlino a938f0853b Additional ports docs. 2016-03-14 19:11:18 -07:00
Jonathan Wei 5ec5ac92c6 Merge pull request #2382 from himanshug/broker_segment_tier_selection
at broker, if configured, only add segments from specific tiers to the timeline
2016-03-14 16:53:06 -07:00
Fangjin Yang a41a70d370 Merge pull request #2651 from gianm/ports-docs
Docs on default ports.
2016-03-14 14:15:52 -07:00
Fangjin Yang dbdbacaa18 Merge pull request #2260 from navis/cardinality-for-searchquery
Support cardinality for search query
2016-03-14 13:24:40 -07:00
Gian Merlino e51277b96c Docs on default ports. 2016-03-14 11:25:21 -07:00
rasahner 2861e854f0 Merge pull request #2540 from pjain1/remove_kill
Remove extra parameter from deleteDataSourceSpecificInterval endpoint and correct exception message for invalid interval
2016-03-14 11:16:23 -05:00
navis.ryu be341bf4e3 Support cardinality for search query (Fix for #2260) 2016-03-12 09:51:01 +09:00
Bingkun Guo 96c981cd0a fix broken link for Tasks 2016-03-11 11:36:34 -06:00
Xavier Léauté 90d7409e1a Merge pull request #2611 from himanshug/gp_by_max_limit
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-10 13:44:13 -08:00
Charles Allen 7b1bfbf704 Add documentation to modules about what should be excluded. 2016-03-10 10:18:33 -08:00
Gian Merlino a2b1652787 Clarify parser docs.
- Clarify what parseSpecs are used for.
- Avro, Protobuf should use timeAndDims parseSpecs.
- Hadoop jobs should use hadoopyString string parsers.
2016-03-10 08:45:04 -08:00
fjy e3e932a4d4 refactor extensions into core and contrib 2016-03-08 17:12:09 -08:00
Himanshu Gupta ca5de3f583 only allow lowering maxResults and maxIntermediateRows from groupBy query context 2016-03-08 15:03:59 -06:00
Fangjin Yang 8e36e6fa43 Merge pull request #2610 from dclim/add-combineText-doc
add combineText property and cleanup batch ingestion doc
2016-03-08 12:54:16 -08:00
Fangjin Yang e7018f524f Merge pull request #2598 from himanshug/handoff_timeout
optional ability to configure handoff wait timeout on realtime tasks
2016-03-08 12:43:36 -08:00
dclim df29667a89 add combineText property and cleanup batch ingestion doc 2016-03-08 13:10:34 -07:00
Himanshu Gupta 099acb4966 allow groupBy max[Intermediate]Rows limit be overridable by context 2016-03-07 15:22:41 -06:00
Himanshu Gupta 0402636598 configurable handoffConditionTimeout in realtime tasks for segment handoff wait 2016-03-05 10:14:54 -06:00
Charles Allen 2ad134638d Merge pull request #2589 from b-slim/fix_real_time
Make realtime kafka firehose skip corrupt message
2016-03-04 12:14:23 -08:00
Slim Bouguerra 623e89aa54 skip corrupt message 2016-03-04 08:30:40 -06:00
jisookim 177b575d41 fix default number of connections on broker config documentation 2016-03-03 13:50:48 -08:00
Björn Zettergren 2462c82c0e New defaults for maxRowsInMemory rowFlushBoundary
To bring consistency to docs and source this commit changes the default
values for maxRowsInMemory and rowFlushBoundary to 75000 after
discussion in PR https://github.com/druid-io/druid/pull/2457.

The previous default was 500000 and it's lower now on the grounds that
it's better for a default to be somewhat less efficient, and work,
than to reach for the stars and possibly result in
"OutOfMemoryError: java heap space" errors.
2016-03-01 13:50:28 +01:00
Fangjin Yang 8f97b1e40c Merge pull request #2554 from guobingkun/improve_include_extensions
improve doc on including druid and hadoop extensions
2016-02-26 20:05:57 -08:00
Charles Allen c6803c4364 Allow specifying peon javaOpts as an array 2016-02-26 13:24:35 -08:00
Bingkun Guo 18f9e05f0f improve doc on including druid and hadoop extensions 2016-02-26 13:53:08 -06:00
Gian Merlino eb13d7afe3 Merge pull request #2521 from himanshug/fix_2497
RTR has multiple threads for assignment of pending tasks now
2016-02-26 08:14:15 -08:00
Nishant 9f8faabddb Merge pull request #2469 from pdeva/patch-10
correct service names
2016-02-26 21:15:58 +05:30
Himanshu Gupta bc156effe7 RTR has multiple threads for assignment of pending tasks now. 2016-02-26 09:27:03 -06:00
Charles Allen 1fe277ee29 Merge pull request #2367 from se7entyse7en/feature-rackspace-cloud-files-static-firehose
Adds support to use Rackspace's cloudfiles as static firehose
2016-02-25 17:31:06 -08:00
Parag Jain b82b487f20 remove extra kill parameter 2016-02-24 17:16:18 -06:00
jon-wei fd3782522c Rename 'replaceMissingValues...' parameters in RegexExtractionFn 2016-02-24 13:12:56 -08:00
Nishant 6c9e1a28ad Merge pull request #2519 from gianm/unparseable-handling
Better handling of ParseExceptions.
2016-02-24 04:46:29 +05:30
Fangjin Yang 93540c0631 Merge pull request #2503 from gianm/jetty-qos
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
2016-02-23 10:35:53 -08:00
Fangjin Yang 4f300cfe49 Merge pull request #2526 from druid-io/b-slim-patch-1
fix docs about sketches
2016-02-23 10:23:53 -08:00
Gian Merlino 3534483433 Better handling of ParseExceptions.
Two changes:
- Allow IncrementalIndex to suppress ParseExceptions on "aggregate".
- Add "reportParseExceptions" option to realtime tuning configs. By default this is "false".

Behavior of the counters should now be:

- processed: Number of rows indexed, including rows where some fields could be parsed and some could not.
- thrownAway: Number of rows thrown away due to rejection policy.
- unparseable: Number of rows thrown away due to being completely unparseable (no fields salvageable at all).

If "reportParseExceptions" is true then "unparseable" will always be zero (because a parse error would
cause an exception to be thrown). In addition, "processed" will only include fully parseable rows
(because even partial parse failures will cause exceptions to be thrown).

Fixes #2510.
2016-02-23 10:11:43 -08:00
Slim 86c4900347 fix thetaSketch post aggregator doc 2016-02-23 10:43:54 -06:00
Himanshu Gupta c7cb5bff14 fix thetaSketchSetOp doc 2016-02-23 09:17:49 -06:00
Fangjin Yang 0c984f9e32 Merge pull request #2109 from himanshug/segments_in_delta_ingestion
idempotent batch delta ingestion
2016-02-22 14:00:45 -08:00
Himanshu Gupta f7679dd5a9 updating thetaSketchSetOp post agg documentation to reflect the possibility of nesting 2016-02-22 09:38:58 -06:00
Himanshu Gupta 21b0b8a07d new coordinator endpoint to get list of used segment given a dataSource and list of intervals 2016-02-21 23:17:58 -06:00
Himanshu Gupta 09ffcae4ae give user the option to specify the segments for dataSource inputSpec 2016-02-21 23:15:31 -06:00
Gian Merlino 23c993c9e7 Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
- Add druid.indexer.server.maxChatRequests, which sets up a QoSFilter on the main Jetty server.
- Deprecate druid.indexer.runner.separateIngestionEndpoint
- Deprecate druid.indexer.server.chathandler.*
2016-02-19 13:36:09 -08:00
Bingkun Guo 9e4c908922 generate tarball by mvn package 2016-02-18 16:42:41 -06:00
Fangjin Yang 083f019a48 Merge pull request #2465 from druid-io/more-doc-fix
more doc fixes
2016-02-17 11:00:38 -08:00
Fangjin Yang 661c3aac4f Merge pull request #2483 from druid-io/b-slim-patch-1
clarification about monitoring period
2016-02-17 09:46:25 -08:00
fjy 7da6594bfe more doc fixes 2016-02-17 09:43:47 -08:00
Gian Merlino 3a996216bd Multivalued dimensions can be compressed since 0.8.0. 2016-02-17 08:33:21 -08:00
Slim e9f1c94822 Update metrics.md 2016-02-17 09:27:15 -06:00
Slim ebbb1aa74e Update metrics.md 2016-02-17 09:05:16 -06:00
pdeva dd81b5ebe4 correct service names
use a `/` instead of `:` cause thats how the service names are declared in the respective config files of coordinator and overlord
2016-02-13 15:26:19 -08:00
Gian Merlino e0c049c0b0 Make startup properties logging optional.
Off by default, but enabled in the example config files. See also #2452.
2016-02-12 14:12:16 -08:00
Jonathan Wei d63eec65a1 Merge pull request #2208 from navis/metadataquery-minmax
Support min/max values for metadata query
2016-02-11 17:28:07 -08:00
navis.ryu dd2375477a Support min/max values for metadata query (#2208) 2016-02-12 09:35:58 +09:00
navis.ryu 4d63196535 Support dimension spec for select query 2016-02-12 08:54:28 +09:00
Slim 368988d187 Merge pull request #2291 from druid-io/lookupManager
Promoting LookupExtractor state and LookupExtractorFactory to be a first class druid state object.
2016-02-11 16:07:27 -06:00
Fangjin Yang 6b45a9e4a7 Merge pull request #2449 from b-slim/fix_docs
fix docs about search query limit
2016-02-11 13:01:41 -08:00
Slim Bouguerra 438a4a9970 fix docs about search query limit 2016-02-11 13:20:59 -06:00
Himanshu 5779b32742 Merge pull request #2439 from metamx/fix2435
Make QuotableWhiteSpaceSplitter able to take JSON
2016-02-11 13:14:43 -06:00
Slim Bouguerra 4e119b7a24 Adding lookup ref manager and lookup dimension spec impl 2016-02-11 12:11:51 -06:00
Fangjin Yang 1430bc2c88 Merge pull request #2276 from harshjain2/feature-2021
Fix for issue 2021.
2016-02-10 17:04:45 -08:00
Fangjin Yang f204dfbebe Merge pull request #2413 from pdeva/patch-9
added note about including extension lib
2016-02-10 17:01:27 -08:00
Charles Allen 3a6452c6d4 Make QuotableWhiteSpaceSplitter able to take json
* Fixes #2435
2016-02-10 16:42:14 -08:00
Himanshu Gupta d1cb17d3f7 at broker - only add segments from specific tiers to the timeline 2016-02-09 22:33:22 -06:00
Fangjin Yang 69a6bdcf03 Merge pull request #2427 from gianm/rolling-doc
Freshen up rolling update docs
2016-02-09 15:32:27 -08:00
Gian Merlino 95d5526e7c Freshen up rolling update docs
1. Clarify what "Indexing Service / Realtime" means
2. Add info about restore-based middle manager rolling restarts
3. Add info about what happens in middle manager updates
4. More consistent capitalization and spelling of node types
2016-02-09 13:57:04 -08:00
Himanshu f6eebf5884 Merge pull request #2422 from rasahner/docMinorFixes
some minor doc changes
2016-02-09 10:03:22 -06:00
Robin 1d57e3267d some minor doc changes 2016-02-09 08:20:53 -06:00
pdeva b75862da7e make 0.9 compatible 2016-02-08 17:25:34 -08:00
David Lim 57bf61029b Merge pull request #2416 from druid-io/fix-docs
fix docs
2016-02-08 15:09:20 -07:00
fjy 6fc5bcb1ef fix docs 2016-02-08 13:40:53 -08:00
pdeva 525a911a3c added note about including extension lib 2016-02-08 12:59:41 -08:00
Himanshu Gupta b40c342cd1 make Global stupid pool cache size configurable 2016-02-05 14:18:06 -06:00
fjy 9e2295aa61 whitespace fixes 2016-02-04 16:25:51 -08:00
fjy b52e1e9161 fix spacing again 2016-02-04 16:13:12 -08:00
fjy 962e7bac14 fix rendering 2016-02-04 15:58:20 -08:00
fjy 8405a7710e tutorial fix 2016-02-04 15:37:47 -08:00
Fangjin Yang e3273230cc Merge pull request #2399 from metamx/md-to
move table of contents to Markdown and fix broken link
2016-02-04 15:28:29 -08:00
Xavier Léauté eb593aeb90 css link is broken
removing in favor of adjusting on the site
2016-02-04 15:23:20 -08:00
Xavier Léauté bbc5e5d470 move table of contents to Markdown
fixes #2397
2016-02-04 15:23:11 -08:00
fjy 003f54e268 add doc rendering 2016-02-04 14:21:59 -08:00
fjy 1aa363cea7 new quickstart 2016-02-04 09:37:38 -08:00
bin liu 39a4666387 fix doc format 2016-02-03 16:29:31 +08:00
Lou Marvin Caraig 9de57eb1c8 Added documentation 2016-02-02 14:32:12 +01:00
Sameer Al-Sakran ee2a0e4afa Update libraries.md 2016-02-01 11:47:50 -08:00
Fangjin Yang 459c2a49ca Merge pull request #2364 from metamx/fix2356
Add more docs around timezone handling
2016-02-01 10:58:15 -08:00
Charles Allen c9393e5289 Add more docs around timezone handling
* Fixes #2356
2016-02-01 08:51:07 -08:00
Jaebin Yoon 66a74a2b88 Fixed the broken link 2016-02-01 01:07:24 -08:00
Fangjin Yang bbfb8aa7dd Merge pull request #2358 from druid-io/addCommunityExtensions
Add Community Extensions
2016-01-31 17:45:11 -08:00
Erik Dubbelboer 246473c58a Remove duplicate doc section 2016-01-30 13:50:32 +00:00
Charles Allen 5ec5c7221b Add Community Extensions
Add a "Community Extensions" section to the known libraries
2016-01-29 13:09:15 -08:00
Fangjin Yang 728055f0eb Merge pull request #2353 from bjozet/doc-fixes
DOCs: Missing 'type' for leaveIntermediate
2016-01-29 09:24:16 -08:00
Björn Zettergren d373573c25 DOCs: Missing 'type' for leaveIntermediate
Added missing 'Boolean' as type for leaveIntermediate row in table TuningConfig
2016-01-29 14:42:19 +01:00
navis.ryu 55a888ea2f time-descending result of select queries 2016-01-29 10:06:05 +09:00
Himanshu Gupta b3437825f0 add ignoreWhenNoSegments flag to optionally ignore the dataSource inputSpec when no segments were found 2016-01-26 17:23:55 -06:00
Bingkun Guo b07db4089c fix doc: Setting druid.coordinator.merge.on will trigger an Append Task instead of Merge Task. 2016-01-26 10:20:32 -06:00
Rafael Abbondanza 145c65c72d Updates number of parts in a topN query
This threw me off a bit, so I'm sure it may throw others off, too.
Updating from 10 to 11 parts.
2016-01-25 10:29:25 -05:00
Harsh Jain 41730b96d4 Fix for issue 2021 2016-01-25 02:48:22 +05:30
binlijin cd1c71ceb4 rename persistBackgroundCount to numBackgroundPersistThreads 2016-01-22 14:29:41 +08:00
Gian Merlino d416279c14 SegmentMetadataQuery support for returning aggregators. 2016-01-21 17:27:25 -08:00
Gian Merlino 87c8046c6c Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments.
SegmentMetadataQuery stuff:

- Simplify implementation of SegmentAnalyzer.
- Fix type names for realtime complex columns; this used to try to merge a nice type
  name (like "hyperUnique") from mmapped segments with the word "COMPLEX" from incremental
  index segments, leading to a merge failure. Now it always uses the nice name.
- Add hasMultipleValues to ColumnAnalysis.
- Add tests for both mmapped and incremental index segments.
- Update docs to include errorMessage.
2016-01-21 15:50:33 -08:00
Slim Bouguerra e0d90f875c Graphite emitter 2016-01-21 13:43:37 -06:00
Fangjin Yang ab635c5cd3 Merge pull request #2315 from rasahner/docUpdateBuild
docs: edits related to loadList
2016-01-21 09:52:27 -08:00
Robin c9368702fa do some editing of the instructions for using mysql for metadata 2016-01-21 10:37:30 -06:00
Slim 201539260c Merge pull request #2076 from b-slim/issue_2010_upper_lower_extractionFN
adding lower and upper extraction fn
2016-01-21 09:58:07 -06:00
Slim Bouguerra 78feb3a13e adding lower and upper extraction fn 2016-01-21 08:59:05 -06:00
Nishant dcb7830330 Merge pull request #984 from drcrallen/thread-priority-rebase
Use thread priorities. (aka set `nice` values for background-like tasks)
2016-01-21 15:02:34 +05:30
Charles Allen 2a69a58570 Merge pull request #2149 from binlijin/master
Do persist IncrementalIndex in another thread in IndexGeneratorReducer
2016-01-20 17:06:42 -08:00
Charles Allen 2e1d6aaf3d Use thread priorities. (aka set `nice` values for background-like tasks)
* Defaults the thread priority to java.util.Thread.NORM_PRIORITY in io.druid.indexing.common.task.AbstractTask
 * Each exec service has its own Task Factory which is assigned a priority for spawned task. Therefore each priority class has a unique exec service
 * Added priority to tasks as taskPriority in the task context. <0 means low, 0 means take default, >0 means high. It is up to any particular implementation to determine how to handle these numbers
 * Add options to ForkingTaskRunner
    * Add "-XX:+UseThreadPriorities" default option
    * Add "-XX:ThreadPriorityPolicy=42" default option
 * AbstractTask - Removed unneded @JsonIgnore on priority
 * Added priority to RealtimePlumber executors. All sub-executors (non query runners) get Thread.MIN_PRIORITY
 * Add persistThreadPriority and mergeThreadPriority to realtime tuning config
2016-01-20 14:00:31 -08:00
Logan Linn c3bdaefe1f Update batch-ingestion.md
Fix documented type of the `dataGranularity` config
2016-01-19 17:20:47 -08:00
binlijin 8e43e2c446 Do persist IncrementalIndex in another thread in IndexGeneratorReducer 2016-01-20 09:20:09 +08:00
Fangjin Yang 0c31f007fc Merge pull request #1728 from himanshug/aggregators_in_segment_metadata
Store AggregatorFactory[] in segment metadata
2016-01-19 12:55:49 -08:00
Himanshu Gupta a99aef29a1 adding aggregators to segment metadata 2016-01-19 14:23:39 -06:00
zhxiaog 3459a202ce fixed #1873, add ability to express CONCAT as an extractionFn 2016-01-18 15:03:17 -08:00
Fangjin Yang 592638f6b3 Merge pull request #2209 from sirpkt/cascade_extraction_functions
support cascade execution of extraction filters in extraction dimension spec
2016-01-18 07:59:36 -08:00
Gian Merlino 42ee473052 ZooKeeper 3.4.7 was unreleased, revert to 3.4.6. 2016-01-17 21:31:29 -08:00
Keuntae Park 238dd3be3c support cascade execution of extraction filters in extraction dimension spec 2016-01-18 11:10:19 +09:00
Gian Merlino 2d3f6e7705 Some more multitenancy docs 2016-01-17 17:47:49 -08:00
Harsh Jain 71f1cd5e34 Fix for issue 2021 2016-01-17 16:10:04 +05:30
Fangjin Yang f6a1a4ae20 Merge pull request #2138 from KurtYoung/feature-build-v9
build v9 directly
2016-01-16 13:35:46 -06:00
Kurt Young 82ff98c2bf add config for build v9 directly and update docs 2016-01-16 11:26:34 +08:00
Fangjin Yang 2e54553a8f Merge pull request #1990 from himanshug/schedule_kill_task
support periodic hard delete of segments
2016-01-15 15:22:33 -06:00
David Lim 34cd8f8c72 Merge pull request #2258 from fjy/acl-zk
acl for zookeeper is added
2016-01-15 10:27:08 -07:00
Nikita Geer 1908d63162 acl for zookeeper is added 2016-01-13 14:56:05 -08:00
navis.ryu 18479bb757 time-descending result of timeseries queries 2016-01-13 12:23:01 +09:00
Fangjin Yang d7ad93debc Merge pull request #2221 from binlijin/topN_minTopNThreshold
Allow change minTopNThreshold per topN query
2016-01-12 16:22:20 -08:00
Himanshu Gupta eb2d251ac8 support periodic hard delete of segments 2016-01-12 16:55:05 -06:00
binlijin 2751f785f8 add doc 2016-01-12 11:25:11 +08:00
Fangjin Yang fafc7c07ba Merge pull request #2238 from rasahner/docServerviewEndpoint
doc: add information about new serverview coordinator endpoint
2016-01-11 10:44:18 -08:00
Robin c1b813011c doc: add information about new serverview coordinator endpoint 2016-01-10 19:47:23 -06:00
Himanshu d255f4baac Merge pull request #2234 from pjain1/emit_realtime_metrics
emit handoff count metrics
2016-01-08 14:24:16 -06:00
Parag Jain 9dba0f67e7 emit handoff count metrics 2016-01-08 12:36:13 -06:00
Himanshu Gupta 0d5f82aee7 document size attribute in thetaSketchSetOp post aggregator 2016-01-07 23:59:03 -06:00
Fangjin Yang 3048b1f0a5 Merge pull request #2174 from metamx/ingest-size-metrics
Add metrics for ingest/bytes/received for EventReceiverFirehose
2016-01-06 22:05:55 -08:00
Fangjin Yang 4cc81d3eff Merge pull request #2096 from b-slim/add_use_case_unapply
Add use case unapply
2016-01-06 21:58:12 -08:00
Fangjin Yang dd262f0451 Merge pull request #2215 from pjain1/fix_doc_metrics
correct metric name - segment/added/count -> segment/assigned/count
2016-01-06 16:21:54 -08:00
Parag Jain 768d07b702 correct metric name - segment/added/count -> segment/assigned/count 2016-01-06 15:55:11 -06:00
Slim Bouguerra 032d3bf6e6 Optimization of extraction filter by reversing the lookup 2016-01-06 11:16:11 -06:00
Himanshu 82bdfbbbf1 Merge pull request #2155 from metamx/taskConfigTmpdir
Make TaskConfig pull from java.io.tmpdir
2016-01-05 13:58:39 -06:00
Nishant 14989f272d Add metrics for ingest/bytes/received for EventReceiverFirehose
review comments

review comments
2016-01-05 20:06:09 +05:30
Zhao Weinan 5e57ddb8cc Adding avro support to realtime & hadoop batch indexing. 2016-01-05 10:21:27 +08:00
Robin 0961c0b703 trivial documentation fix 2016-01-04 12:39:10 -06:00
Himanshu ebccea6bb0 Merge pull request #2188 from druid-io/doc-fix
small fixes to documentation
2016-01-04 12:25:26 -06:00
Charles Allen e18301d99c Make TaskConfig pull from java.io.tmpdir
* Also makes paths built off of java.nio.file.Paths instead of String.format
2016-01-04 10:17:08 -08:00
Robin e280ab5f07 update zookeeper version to 3.4.7 2016-01-04 11:47:02 -06:00
fjy d3d2ee03ce minor fixes to docs 2016-01-03 11:37:06 -08:00
Gian Merlino 5a63c3dd63 Merge pull request #2186 from druid-io/dev-docs2
Add intro developer docs
2016-01-03 11:36:41 -05:00
fjy 88f6b9b5ad Multiple improvements for docs 2016-01-02 21:54:54 -08:00
fjy 06a8e14820 Add intro developer docs 2016-01-02 14:44:45 -08:00
Himanshu Gupta 48de9dfafa doc update to make it easy to find how to do re-indexing or delta ingestion 2015-12-30 23:58:09 -06:00
Himanshu Gupta e1ea93b6fc documenting querying behavior on multi-valued dimensions 2015-12-30 18:14:25 -06:00
Himanshu Gupta b47d807738 Add support for filtering at DimensionSpec level so that multivalued dimensions can be filtered correctly
also adding UTs for multi-valued dimensions
2015-12-30 17:59:47 -06:00
Bingkun Guo 3c107c5757 Merge pull request #2150 from himanshug/emit_query_bytes
emit query/bytes metric
2015-12-30 13:44:19 -06:00
Fangjin Yang e14ad74088 Merge pull request #1936 from b-slim/between_range_with_predicat
adding Upper/Lower Bound Filter
2015-12-29 10:11:22 -08:00
Fangjin Yang b1261035a7 Merge pull request #1861 from guobingkun/insert_segment_tool
insert-segment tool
2015-12-29 10:06:07 -08:00
Fangjin Yang 5042b20eb4 Merge pull request #2164 from pdeva/patch-7
correct docs
2015-12-27 14:43:51 -08:00
pdeva 77863285e9 fix typo 2015-12-27 14:28:23 -08:00
pdeva b308a13483 correct docs 2015-12-27 14:27:20 -08:00
pdeva 98fc08b0de List who can access metadata storage
This is useful to know which ip addresses you need to allow to access your RDS instance for example
2015-12-27 14:25:17 -08:00
Himanshu Gupta 1a8546a682 emit query/bytes metric 2015-12-23 00:29:44 -06:00
Fangjin Yang daaa24d7ad Merge pull request #2145 from gianm/restoring-option
druid.indexer.task.restoreTasksOnRestart configuration.
2015-12-22 15:22:32 -08:00
Gian Merlino bad270b6c4 druid.indexer.task.restoreTasksOnRestart configuration. 2015-12-22 10:59:15 -08:00
Himanshu Gupta b96f560255 emit query/node/bytes metric 2015-12-21 23:23:20 -06:00
Charles Allen c2a2d19d62 Merge pull request #2126 from guobingkun/remove_single_announcer
Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer
2015-12-21 08:55:07 -08:00
Bingkun Guo 951a4e9b35 Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer 2015-12-21 00:05:53 -06:00
Bingkun Guo 89b477970f DataSegmentFinder tool
`insert-segment-to-db` is a tool that can insert segments into Druid metadata storage. It is intended to be used
to update the segment table in metadata storage after people manually migrate segments from one place to another.
It can also be used to insert missing segment into Druid, or even recover metadata storage by telling it where the
segments are stored.

Note: This tool expects users to have Druid cluster running in a "safe" mode, where there are no active tasks to interfere
the segments being inserted. Users can optionally bring down the cluster to make 100% sure nothing is interfering.
2015-12-21 00:02:04 -06:00
Fangjin Yang 7019d3c421 Merge pull request #2107 from jon-wei/fix_smq
More efficient SegmentMetadataQuery
2015-12-18 16:40:47 -08:00
fjy 398a3ec620 add docs for more specs 2015-12-17 18:06:30 -08:00
jon-wei 356b07c6c3 More efficient SegmentMetadataQuery 2015-12-17 12:46:23 -08:00
Jonathan Wei f8cf84f466 Merge pull request #1995 from himanshug/num_rows_seg_metadata_query
add numRows to segment metadata query response
2015-12-17 12:23:46 -08:00
Fangjin Yang cb085f99d3 Merge pull request #2097 from metamx/update-coordinator-docs
update the coordinator API docs to better explain how to format ISO86…
2015-12-15 17:35:48 -08:00
desaianuj 2f32cb1fb6 update the coordinator API docs to better explain how to format ISO8601 intervals in the request url 2015-12-15 16:17:54 -08:00
Gian Merlino e6c2db89da Merge pull request #1921 from jon-wei/flat_json
Add docs and benchmark for JSON flattening parser
2015-12-14 20:53:54 -08:00
Fangjin Yang e7f06cf61c Merge pull request #2075 from jon-wei/regex_extract
Configurable value replacement on match failure for RegexExtractionFn
2015-12-14 19:10:50 -08:00
jon-wei c88f75df7c Configurable value replacement on match failure for RegexExtractionFn 2015-12-14 17:57:41 -08:00
Steve M 2b5a010332 Change sample worker config spec with host:port instead of ip:port.
Also extend description of the 'affinity' property of the worker strategy
fillCapacityWithAffinity and fix a couple typos of middle manager (to
be more consistent throughout the page).

Add additional verbiage about appropriate middle manager host value.
2015-12-14 14:59:23 -08:00
Slim Bouguerra 77afdf25e3 adding Bound Filter 2015-12-10 08:47:21 -06:00
jon-wei c53bf85d83 Add docs and benchmark for JSON flattening parser 2015-12-09 16:13:30 -08:00
Fangjin Yang b0ab363022 Merge pull request #2052 from gianm/service-names
Change service names in docs, examples to match defaults in the code.
2015-12-08 15:40:35 -08:00
Xavier Léauté dcd15735cd Merge pull request #2015 from metamx/handoff-notifier-coordinator
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoff information
2015-12-08 14:06:39 -08:00
Nishant 9491e8de3b Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoffs
- fixes #1970
- extracted out segment handoff callbacks in SegmentHandoffNotifier
which is responsible for tracking segment handoffs and doing callbacks
when handoff is complete.
- Coordinator now maintains a view of segments in the cluster, this
will affect the jam heap requirements for the overlord for large
clusters.
realtime index task and nodes now use HTTP end points exposed by the
coordinator to get serverView

review comment

fix realtime node guide injection

review comments

make test not rely on scheduled exec

fix compilation

fix import

review comment

introduce immutableSegmentLoadInfo

fix son reading

remove unnecessary logging
2015-12-09 01:54:09 +05:30
Nishant 6c23d8edb4 Merge pull request #2043 from mangeshpardeshiyahoo/master
Add dimension selector support for groupby/having filters
2015-12-08 12:08:53 +05:30
Mangesh Pardeshi d7ce120929 Add dimension selector support for groupby/having quries 2015-12-08 01:51:11 +00:00
michael.schiff 4ed0128b3b removed documentation about (removed) DeleteTask 2015-12-07 15:56:46 -08:00
Fangjin Yang 00416e26ff Merge pull request #2012 from himanshug/add_comment
adding more comments to why drops do not happen
2015-12-07 11:50:06 -08:00
Fangjin Yang fe05545f58 Merge pull request #2055 from jkukul/documentation-fixes
Minor documentation fixes.
2015-12-07 10:50:38 -08:00
Jakub Kukul 8248f7f40c Minor docs fixes in 'Evaluate Druid' section. 2015-12-07 19:42:52 +01:00
AdrieanKhisbe 5d10088e2d Update Tutorial with new Zookeper Version and instructions 2015-12-07 06:50:42 +01:00
Gian Merlino 8e594a2e72 Change service names in docs, examples to match defaults in the code. 2015-12-06 10:04:21 -08:00
Himanshu Gupta efe3c9f4a5 update the examples for batch reindexing/delta ingestion to use "intervals" instead of deprecated "interval" 2015-12-06 00:22:20 -06:00
Himanshu Gupta 61aaa09012 support multiple intervals in dataSource input spec 2015-12-03 21:28:04 -06:00
Himanshu 0eab8417cb Merge pull request #2008 from codingwhatever/regex-search-query
Regex search query
2015-12-03 09:57:34 -06:00
Sam Groth 596b7ebd9a Adding RegexSearchQuerySpec 2015-12-03 09:16:02 -06:00
Clint Wylie 68ef5f437a fixes #1874 adding a substring extraction function, tests, and documentation 2015-12-01 23:50:32 -08:00
Himanshu Gupta 7c91219130 adding more comments to why drops do not happen when there are no segments in db 2015-11-25 16:26:51 -06:00
jon-wei 95dca4440f Update data formats doc with info about JSON multi-value dimensions 2015-11-24 14:38:06 -08:00
Fangjin Yang 8e83d800d6 Merge pull request #1881 from gianm/restartable-tasks
Restorable indexing tasks
2015-11-23 21:14:37 -08:00
Gian Merlino 501dcb43fa Some changes that make it possible to restart tasks on the same hardware.
This is done by killing and respawning the jvms rather than reconnecting to existing
jvms, for a couple reasons. One is that it lets you restore tasks after server reboots
too, and another is that it lets you upgrade all the software on a box at once by just
restarting everything.

The main changes are,

1) Add "canRestore" and "stopGracefully" methods to Tasks that say if a task can
   stop gracefully, and actually do a graceful stop. RealtimeIndexTask is the only
   one that currently implements this.

2) Add "stop" method to TaskRunners that attempts to do an orderly shutdown.
   ThreadPoolTaskRunner- call stopGracefully on restorable tasks, wait for exit
   ForkingTaskRunner- close output stream to restorable tasks, wait for exit
   RemoteTaskRunner- do nothing special, we actually don't want to shutdown

3) Add "restore" method to TaskRunners that attempts to bootstrap tasks from last run.
   Only ForkingTaskRunner does anything here. It maintains a "restore.json" file with
   a list of restorable tasks.

4) Have the CliPeon's ExecutorLifecycle lock the task base directory to avoid a restored
   task and a zombie old task from stomping on each other.
2015-11-23 11:22:08 -08:00
Gian Merlino c0580bf063 Merge pull request #1928 from druid-io/new-compares
New comparisons for Druid
2015-11-20 16:40:59 -08:00
Himanshu Gupta 7a89b2e1a6 add numRows to segment metadata query response 2015-11-20 01:25:02 -06:00
Himanshu Gupta fde9df2720 update to sketches-core-0.2.2 .
adds support for "cardinality" aggregator.
do not create sketch per event at ingestion time to make realtime ingestion faster
2015-11-19 01:05:59 -06:00
Fangjin Yang 87c43fbf02 Merge pull request #1981 from rasahner/fixFormattingSchemaDesign
fix formatting in schema-design
2015-11-17 17:51:16 -08:00
Charles Allen 8fcf2403e3 Merge pull request #1943 from metamx/realtime-caching
Enable caching on intermediate realtime persists
2015-11-17 15:06:43 -08:00
sahner a4ed2ce2d1 fix formatting in schema-design 2015-11-17 16:50:53 -06:00
Himanshu 9118b010c0 Merge pull request #1976 from noddi/feature/count-parameter-rules-history
Add count parameter to rules history endpoint.
2015-11-17 14:41:50 -06:00
Charles Allen dbe201aeed Merge pull request #1929 from pjain1/jetty_threads
separate ingestion and query thread pool
2015-11-17 12:14:25 -08:00
Parag Jain 6c498b7d4a separate ingestion and query thread pool 2015-11-17 13:42:41 -06:00
Bartosz Ługowski b0a444eafc Add count parameter to rules history endpoint. 2015-11-17 20:14:12 +01:00
Xavier Léauté d7eb2f717e enable query caching on intermediate realtime persists 2015-11-17 10:58:00 -08:00
Fangjin Yang 4f46d457f1 Merge pull request #1947 from noddi/feature/count-parameter-history-endpoints
Add count parameter to history endpoints
2015-11-12 10:23:44 -08:00
Himanshu Gupta 7788f7c2a1 update doc with new thetaSketch api 2015-11-12 00:04:34 -06:00
Himanshu Gupta 6c6a38cedb adding datasketches aggregator to documentation 2015-11-12 00:04:33 -06:00
Bartosz Ługowski 6e5d2c6745 Add count parameter to history endpoints. 2015-11-11 23:03:57 +01:00
Gian Merlino e75c2a407d Merge pull request #1944 from druid-io/fix-doc
fix website rendering for this doc
2015-11-10 16:04:40 -08:00
fjy e923de3eea fix website rendering for this doc 2015-11-10 15:36:30 -08:00
Xavier Léauté cf779946ef Merge pull request #1791 from guobingkun/event_receiver_firehose_monitor
EventReceiverFirehoseMonitor
2015-11-10 11:09:42 -08:00
fjy 46bf1ba5ef remove unneeded 2015-11-09 17:03:00 -08:00
fjy 8a8bb0369e address more comments 2015-11-09 16:56:43 -08:00
fjy b99576d854 rework compares again 2015-11-09 16:40:07 -08:00
fjy 0b319093df New comparisons for Druid 2015-11-09 14:31:23 -08:00
Bingkun Guo b24eccfb9e add doc for bundling custom extensions with other Druid extensions 2015-11-09 13:11:22 -06:00
Slim Bouguerra c511273efd adding in filter 2015-11-06 16:23:24 -06:00
Xavier Léauté e9533db987 Merge pull request #1850 from metamx/friendlyBardCache
Allow setting upper limit on the number of cache segments a broker will try to fetch.
2015-11-06 10:25:49 -08:00
Himanshu 3c3ea128cc Merge pull request #1895 from druid-io/cleanup-codebase
Cleanup the Druid codebase
2015-11-04 17:16:33 -06:00
fjy 8f231fd3e3 cleanup druid codebase 2015-11-04 13:59:53 -08:00
Fangjin Yang 1cbc514eb5 Merge pull request #1719 from se7entyse7en/feature-rackspace-cloud-files-deep-storage
Add Rackspace Cloud Files Deep Storage Extension
2015-11-04 11:13:25 -08:00
Fangjin Yang 861338c5e9 Merge pull request #1904 from rohitkochar/doc
modifying the documentation as per Issue#1903
2015-11-04 09:48:45 -08:00
Himanshu Gupta 67c218bed6 since topN is descending by default, inversion makes is ascending 2015-11-04 11:36:49 -06:00
Lou Marvin Caraig c924f9fe56 Added cloudfiles-extensions in order to support Rackspace's cloudfiles as deep storage 2015-11-04 17:44:48 +01:00
rohit.kochar 69a3b38f81 made changes as per review comments 2015-11-04 11:10:28 +05:30
Bingkun Guo 3ee28c35ce fix curator compress doc 2015-11-03 16:48:59 -06:00
Gian Merlino 8defe29270 Merge pull request #1901 from guobingkun/fix_typo_and_rename
Fix metadata typo and rename default extension directory
2015-11-03 14:02:11 -08:00
Xavier Léauté 3b3d88bd03 Merge pull request #1857 from noddi/feature/intervals-route
Queryable intervals
2015-11-03 13:19:04 -08:00
Bingkun Guo 962f65cc76 fix metadata typo and rename default extension directory 2015-11-03 14:50:42 -06:00
Fangjin Yang cec09a9967 Merge pull request #1804 from himanshug/objectify_index_creators
static to non-static conversion for methods in Index[Merger/Maker/IO]
2015-11-03 11:25:32 -08:00
rohit.kochar 2a617c777e modifying the documentation as per Issue#1903 2015-11-03 21:48:19 +05:30
Fangjin Yang f681c84014 Merge pull request #1892 from navis/DRUID-1878
Make 'search' filter have a case sensitive option (for #1878)
2015-11-03 07:11:00 -08:00
Himanshu Gupta c74a4490e1 add metamarket histogram post to approx-histo doc 2015-11-03 01:19:22 -06:00
Himanshu Gupta 8b67417ac8 make methods in Index[Merger,Maker,IO] non-static so that they can have
appropriate ObjectMapper injected instead of creating one statically
2015-11-02 23:24:26 -06:00
Gian Merlino 0d85774a27 Merge pull request #1899 from itsmee/docs-historical-improvements
Docs improved: more details about caching and memory for segments on historicals
2015-11-02 12:48:56 -08:00
Nishant efc49da073 fix doc - correct default value for maxRowsInMemory 2015-11-01 22:09:24 -08:00
navis.ryu 69c86716d6 addressed comments 2015-11-02 14:23:13 +09:00
Oleg Zaezdny 95a5ae0373 Docs improved by adding more details about local cache and memory for segments on historicals. 2015-11-01 21:56:28 +02:00
Bingkun Guo c3b6fcce9d Add EventReceiverFirehoseMonitor
add an EventReceiverFirehoseMonitor so that we can monitor how many
events have been queued in the EventReceiverFirehose and get a sense
about whether the firehose is under too much pressure.
2015-10-30 11:40:02 -05:00
Bingkun Guo 657a5ac346 fix pull-deps remoteRepository option 2015-10-30 11:32:56 -05:00
Bartosz Ługowski 6de51e022d Coordinator - add intervals route. 2015-10-30 11:51:42 +01:00
Charles Allen dfce14ed17 Allow setting upper limit on the number of cache segments a broker will try to fetch. 2015-10-29 11:50:00 -07:00
Angel M de Miguel a2510c9b0b Update ruby-druid URL 2015-10-28 10:31:30 +01:00
Angel M de Miguel 04c5d0f8e2 Update Ruby libraries in docs 2015-10-28 09:08:26 +01:00
Xavier Léauté 59872bd0cd Merge pull request #1809 from metamx/fifoPriorityExecutorService
Make PrioritizedExecutorService optionally FIFO
2015-10-27 15:19:32 -07:00
Gian Merlino 7df7370935 Merge pull request #1862 from metamx/indexingServiceMMGone
Add timeout to shutdown request to middle manager for indexing service
2015-10-27 14:38:01 -07:00
Charles Allen ecdafa87c5 Make PrioritizedExecutorService optionally FIFO 2015-10-27 14:16:22 -07:00
Charles Allen 44a2b204df Add timeout to shutdown request to middle manager for indexing service 2015-10-27 13:56:03 -07:00
Bingkun Guo 44aad069da add description for /druid/coordinator/v1/loadqueue?full 2015-10-26 14:19:13 -05:00
Fangjin Yang 7e1cde25b6 Merge pull request #1820 from himanshug/doc_files
Removing the md files created for redirection
2015-10-26 09:37:11 -07:00
Fangjin Yang 5f23703216 Merge pull request #1638 from guobingkun/remove_maven_client_code
Remove Maven client at runtime + Provide a way to load Druid extensions through local file system
2015-10-26 09:30:05 -07:00
Slim 16f84161da Fix documentation about lookup 2015-10-23 16:40:54 -07:00
Xavier Léauté 72c408cf2d Merge pull request #1770 from metamx/merge-time
Add segment merge time as a metric
2015-10-22 22:03:41 -07:00
Nishant 7cecc55045 Add segment merge time as a metric
Add merge and persist cpu time

Fix typo

review comment

move cpu time measuring to VMUtils

review comments.
2015-10-22 12:28:03 +05:30
Gian Merlino d068ec3b9d "druid.manager.segment" should be "druid.manager.segments" 2015-10-21 15:50:44 -07:00
Bingkun Guo 4914925d65 New extension loading mechanism
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
2015-10-21 14:22:36 -05:00
Himanshu 8dd30e6e18 Merge pull request #1832 from guobingkun/improve_granularity_doc
add examples for duration and period granularities
2015-10-21 13:13:24 -05:00
Bingkun Guo 2de5e9cefb add examples for duration and period granularities 2015-10-21 11:23:00 -05:00
Fangjin Yang 170cce69d4 Merge pull request #1834 from guobingkun/fix_doc
fix ingestion faq link
2015-10-18 17:31:22 -07:00
Fangjin Yang ccd17f818a Merge pull request #1837 from druid-io/update-paper
add more details to radstack paper
2015-10-18 17:31:08 -07:00
fjy 89785b2f52 add more details to paper 2015-10-18 17:30:24 -07:00
Bingkun Guo 620e334d0f fix ingestion faq link 2015-10-16 10:14:14 -05:00
sahner 83d31844db fix doubled word 2015-10-15 15:59:07 -05:00
Himanshu Gupta 24f33938d8 Removing the md files created for redirection 2015-10-12 15:21:55 -05:00
Gian Merlino 933cbdf780 Adjust realtime constraints in the docs. 2015-10-09 10:52:52 -07:00
Charles Allen 020a706ac4 Merge pull request #1800 from metamx/hybrid-cache
Hybrid L1/L2 cache
2015-10-06 15:18:07 -07:00
Xavier Léauté b11c0859e8 hybrid l1/l2 cache to combine local and remote cache 2015-10-06 14:32:08 -07:00
Xavier Léauté e5b3226b62 Merge pull request #1793 from himanshug/fix_chunking_link
fixing the link to chunkPeriod doc
2015-10-02 10:10:11 -04:00
Himanshu Gupta 9006e255ef fixing the link to chunkPeriod doc 2015-10-01 13:03:46 -05:00
Xavier Léauté faf4c865d5 update R / Python clients 2015-10-01 13:42:09 -04:00
Nishant 573aa96bd6 fix #1727 - Union bySegment queries fix
Fixes #1727.
revert to doing merging for results for union queries on broker.

revert unrelated changes

Add test for union query runner

Add test

remove unused imports

fix imports

fix renamed file

fix test

update docs.
2015-09-29 23:32:36 +05:30
Himanshu Gupta 6e550d5346 update doc about aggregation field in merge task and a null check 2015-09-24 22:25:07 -05:00
Xavier Léauté 2cb0fb4669 Merge pull request #1753 from jon-wei/segmentmetadataquery_flags
Allow SegmentMetadataQuery to skip cardinality and size calculations
2015-09-22 14:52:28 -07:00
jon-wei e6a6284ebd Allow SegmentMetadataQuery to skip cardinality and size calculations 2015-09-22 13:51:55 -07:00
Xavier Léauté df4c8a3aa5 balance memcached cache across multiple connections 2015-09-18 09:52:16 -07:00
fjy c9ccf9a565 add docs for search filter 2015-09-16 16:08:59 -07:00
Gian Merlino 9705c5139b Merge pull request #1732 from jon-wei/segmentmeta
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
2015-09-16 12:36:25 -07:00
Gian Merlino b29cbf97a6 Docs: Suggest hadoopyString parser for Hadoop. 2015-09-16 10:19:42 -07:00
Xavier Léauté b464da438c Merge pull request #1688 from metamx/moreMemcachedMetrics
More memcached metrics
2015-09-15 15:33:51 -07:00
fjy beab6fd487 add pivot as a UI 2015-09-15 14:58:32 -07:00
jon-wei 193fb4fdfc Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups 2015-09-14 19:41:42 -07:00
Himanshu 5ff92664f8 Merge pull request #1696 from metamx/cpuTimeReporting
Add CPU time to metrics for segment scanning.
2015-09-14 10:53:55 -05:00
Bingkun Guo e222e6b86b fix typo in segments.md 2015-09-11 16:32:50 -05:00
Himanshu Gupta 075b6d4385 update ingestion faq to mention dataSource inputSpec as an option of reindexing via hadoop 2015-09-10 14:41:13 -05:00
Himanshu Gupta 39d23ffc45 in configuration/index.md s/instantialize/initialize 2015-09-09 21:18:26 -05:00
Himanshu Gupta ebdb612933 composing emitter module to use multiple emitters together 2015-09-09 16:45:50 -05:00
Charles Allen 5813856819 More memcached metrics 2015-09-08 13:34:58 -07:00
Charles Allen fcf5cae81d Add CPU time to metrics for segment scanning. 2015-09-08 13:34:19 -07:00
Fangjin Yang 4546652b3b Merge pull request #1698 from gianm/docs-skipEmptyBuckets
Timeseries skipEmptyBucket docs.
2015-09-02 15:30:43 -07:00
Gian Merlino cbef4a15b0 Timeseries skipEmptyBucket docs. 2015-09-02 14:59:21 -07:00
Charles Allen ce3506b8e1 Merge pull request #1542 from metamx/audit-rule-history
add endpoint to fetch rule history for all datasources
2015-09-01 13:57:28 -07:00
cheddar 4f61b42f40 Merge pull request #1578 from b-slim/fix_extraction_filter_2
Fix UT and documentation to the extraction filter
2015-09-01 10:46:20 -07:00
Xavier Léauté d89b0fa76a Merge pull request #1662 from qix/pathFormat-doc
Add documentation for pathFormat in batch ingestion
2015-08-31 11:14:54 -07:00
Josh Yudaken 29c29b42d3 Add default value and link to joda docs 2015-08-31 11:09:54 -07:00
nishant 0835e12f2a add endpoint to fetch rule history for all datasources.
review comments

Review comment

fix compilation
2015-08-31 14:05:52 +05:30
Himanshu Gupta 2e0dd1d792 adding UTs and addressing review comments to
firehoseV2 addition to Realtime[Manager|Plumber],
essential segment metadata persist support,
kafka-simple-consumer-firehose extension patch
2015-08-27 20:50:46 -05:00
lvjq 2237a8cf0f kafka 8 simple consumer firehose 2015-08-27 20:50:46 -05:00
Bingkun ae1f104c10 Fix batch ingestion doc 2015-08-26 15:16:21 -05:00
fjy bdfe67fe17 fix formatting 2015-08-26 09:21:32 -07:00
Gian Merlino 10946610f4 Merge pull request #1656 from druid-io/all-the-docs
more docs for common questions
2015-08-25 17:49:47 -07:00
fjy 4055f9ca48 more docs for common questions 2015-08-25 17:49:04 -07:00
sahner 3def847e28 add documentation about TimedShutoff firehose 2015-08-24 20:41:42 -05:00
Josh Yudaken 5e42aee49e Add documentation for pathFormat in batch ingestion 2015-08-24 14:39:57 -07:00
Xavier Léauté 5a48376788 add postgresql setup docs 2015-08-21 13:35:58 -07:00
Himanshu Gupta 0daeb830b0 update approx-histogram document to explain how to ignore rows with no value at ingestion time 2015-08-19 15:20:37 -05:00
Xavier Léauté 0cbda0c01d update version numbers in docs 2015-08-17 16:41:21 -07:00
Charles Allen b9792b57bc Merge pull request #1374 from himanshug/batch_delta_ingestion3
Feature for hadoop batch re-ingesion and delta ingestion
2015-08-17 15:52:49 -07:00
Fangjin Yang f9e3f922e6 Merge pull request #1633 from pjain1/edit_metrics
swap description and dimension column for some JVM metrics
2015-08-17 13:07:57 -07:00
Parag Jain 41fa9bf994 swap description and dimension for some JVM metrics 2015-08-17 15:03:06 -05:00
Himanshu Gupta cfd81bfac7 updating the docs on how to do hadoop batch re-ingesion and delta ingestion 2015-08-16 14:07:35 -05:00
Fangjin Yang 958dd1a451 Merge pull request #1629 from metamx/fix-1584
Update post-aggregations.md
2015-08-14 11:07:47 -07:00
MarConSchneid 3f15d81f76 Update post-aggregations.md
fix for  the issue: https://github.com/druid-io/druid/issues/1584
2015-08-14 11:03:58 -07:00
Xavier Léauté 24dc23967f Merge pull request #1607 from druid-io/even-more-docs
Fixes and more docs across many areas
2015-08-13 11:53:53 -07:00
fjy 43978eba99 Fixes and more docs across many areas 2015-08-13 10:35:53 -07:00
Xavier Léauté f583cad2e2 disclaimer + more docs for approximate histograms 2015-08-10 13:13:49 -07:00
Slim Bouguerra 64d638a386 optimize makeMatcher 2015-08-06 17:04:36 -05:00
Charles Allen e6226968a6 Merge pull request #1589 from druid-io/fix-firehose-doc
Add a lot more docs for firehoses
2015-08-06 12:45:24 -07:00
Charles Allen 0326a7ac69 Merge pull request #1597 from metamx/fix-doc
some docs were missing before, hopfully this is more clear
2015-08-06 11:17:23 -07:00
Xavier Léauté 0357dc2cd8 some docs were missing before, hopfully this is more clear 2015-08-04 11:11:00 -07:00
fjy 012fff6616 fix firehose docs 2015-08-04 09:52:23 -07:00
Fangjin Yang ba76806f3a Merge pull request #1591 from b-slim/patch-1
typo in the documentation after changing the code
2015-08-04 09:23:25 -07:00
Himanshu 3078d905fa Merge pull request #1586 from onlychoice/fix_javaOpts
Fix configuration for javaOpts, remove quotation marks
2015-08-03 21:27:45 -05:00
pdeva 27997f9682 added required config option 2015-08-03 19:08:05 -07:00
Slim 1fdfc67ffa typo in the documentation after changing the code 2015-08-03 10:45:03 -05:00
Slim Bouguerra 83de5a4716 addressing reviewers comments 2015-08-03 09:03:28 -05:00
Slim Bouguerra 65654ddbf9 adding documentation about extraction filter 2015-08-03 09:02:17 -05:00
Zhihui Jiao d3e838ff34 Fix configuration for javaOpts, remove quotation marks 2015-08-01 11:26:37 +08:00
kete.yangkt 2b2bba8d59 fix realtime index task json description in doc 2015-07-31 13:59:58 +08:00
Himanshu Gupta 7ee509bcd0 fix mysql references in tutorial docs 2015-07-30 22:05:05 -05:00
Fangjin Yang f4567c995c Merge pull request #1572 from himanshug/cache_doc_update
update doc with useCache and populateCache flags
2015-07-28 11:50:13 -10:00
Fangjin Yang 22567946cf Merge pull request #1259 from metamx/queryTimeLookup
Query Time Lookup
2015-07-28 11:43:05 -10:00
Himanshu Gupta c17bc3fd74 update doc with useCache and populateCache flags 2015-07-28 15:46:22 -05:00
fjy 9946eebc02 Add setup to metadata stoer page 2015-07-28 08:36:48 -10:00
Charles Allen ba59f8afc4 Merge pull request #1550 from himanshug/optionally_log_all_requests
print *all* HTTP requests to log if configured
2015-07-28 11:19:36 -07:00
Charles Allen 86ede702b1 Add namespaced lookups as extensions
* Adds kafka, URI, and JDBC namespace defintions
* Add ability to explicitly rename using a "namespace" which is a particular data collection that is loaded on all realtime, historic nodes, and brokers. If any of these nodes has the namespace extension, ALL nodes have the namespace extension.
* Add namespace caching and populating (can be on heap or off heap)
* Add NamespaceExtractionCacheManager for handling caches
* Added ExtractionNamespace for handling metadata on the extraction namespaces
* Added ExtractionNamespaceUpdate for handling metadata related to updates
* Add extension which caches renames from a kafka stream (requires kafka8)
* Added README.md for the namespace kafka extension
* Added docs
* Added namespace/size, namespace/count, namespace/deltaTasksStarted metrics

Add static config for namespaces via `druid.query.extraction.namespace`
* This is a rebase of https://github.com/b-slim/druid/tree/static_config_only
2015-07-28 11:14:14 -07:00
Himanshu Gupta 90b475975e Log all the requests to debug level in logger io.druid.jetty.RequestLog 2015-07-28 12:56:04 -05:00
pdeva ef0439229d Specify dynamic dimension schema
Document how druid can dynamically infer dimension columns
2015-07-27 20:20:53 -07:00
sahner 4801de62a2 make "announce" the chathandler default in realtime node,
remove doc references to chathandler type "announce" since it is the default now,
2015-07-27 12:14:28 -05:00
pdeva 76bf8ccd8c correct key name 2015-07-25 21:58:37 -07:00
Charles Allen b9359b7531 Add convert.md to document conversion task 2015-07-24 17:37:49 -07:00
fjy 92293ef094 Added section on best practices for schema designa and a few other edits 2015-07-24 14:06:20 -07:00
Himanshu Gupta 119ec13d23 updating hadoop tuningConfig doc with useCombiner flag 2015-07-22 13:55:00 -05:00
Maxime Beauchemin db4928e73b Documentation entry for Panormix (a druid UI) 2015-07-21 18:23:46 -07:00
Fangjin Yang 020827349e Merge pull request #1539 from mvfast/cdh_workaround
Cdh workaround
2015-07-20 09:16:42 -07:00
Qi Wang 7211791585 add workaround for cdh 2015-07-19 14:11:47 -07:00
Himanshu Gupta dd95ef77c0 recommend druid-hdfs-storage and hadoop dependencies to be in the classpath instead of added as an extension 2015-07-18 16:18:12 -05:00
Charles Allen e051e93d19 Merge pull request #1518 from RealROI/more-azure-features
Azure Blob Store support for Firehose and Indexing Service Logs
2015-07-17 16:10:22 -07:00
Zak Kristjanson 0bda7af52c Add more support for Azure Blob Store
Azure Blob Store support for Task Logs and a firehose for data ingestion
2015-07-17 15:38:21 -07:00
Fangjin Yang e21195f987 Merge pull request #1469 from guobingkun/table_config
Inconsistent property names for "druid.metadata.storage.tables.xxx"
2015-07-17 07:43:19 -07:00
Himanshu 19af3bc9bc Merge pull request #1535 from metamx/alphanum-docs-tests
Update alphanumeric sort docs + more tests / examples
2015-07-16 22:09:41 -05:00
Xavier Léauté 2c464ad936 correct reference in docs + more tests / examples 2015-07-16 19:50:05 -07:00
Fangjin Yang 6e4108c29b Merge pull request #1530 from sherry-q/ingestion-doc-fix
Fix Doc data-formats.md
2015-07-15 18:01:17 -07:00
fjy 0074138bb9 fix versions in tutorials 2015-07-15 16:18:16 -07:00
Shiyu Qiu bec8e8e23a fix doc data-formats.md 2015-07-15 17:13:33 -05:00
Tim 3b692fb6f7 fix #1525 - typo: "HadoopBatchIndexer" 2015-07-14 20:48:24 -07:00
Gian Merlino 183f51fe58 Clearer language around external dependencies in tutorials. 2015-07-14 10:50:45 -07:00
Fangjin Yang 798c3320d0 Merge pull request #1475 from himanshug/derby_in_examples
for example/tutorial, use derby instead of mysql
2015-07-14 10:18:21 -07:00
Fangjin Yang 14a07077a2 Merge pull request #1519 from solimant/master
add section: "What to Do When You Have a Firewall"
2015-07-13 23:20:59 -07:00
solimant 91c15a90a9 add section: "What to Do When You Have a Firewall" 2015-07-13 22:12:16 -07:00
samjhecht fc438e5a72 add alert on errors polling for rules
reset retrystarttime after alerting
2015-07-13 09:03:51 -07:00
fjy 08d00cc80f rework the realtime examples a bit; add more faq 2015-07-07 14:07:14 -07:00
Himanshu Gupta db00533bc8 make concepts link in toc more robust 2015-07-07 14:53:22 -05:00
Xavier Léauté 3e960dcc01 Merge pull request #1496 from druid-io/more-docs
Add more docs based on proposed wishlist
2015-07-07 10:03:02 -07:00
Fangjin Yang d846f3ced6 Merge pull request #1498 from rasahner/doc_localFirehose
say explicitly that local firehose searches dirs recursively for files
2015-07-05 15:34:36 -07:00
sahner acd20e8c00 say explicitly that local firehose searches directories recursively for files 2015-07-05 14:46:44 -05:00
fjy 42ac41d55e add more docs based on proposed wishlist 2015-07-02 17:46:08 -07:00
Kris Molendyke 5b5e3caba9 Make postAggregations example a list type. 2015-07-02 14:37:23 -04:00
Charles Allen f2919b98b1 Merge pull request #1471 from metamx/direct-client-metrics
add query/node/time metrics to DirectDruidClient
2015-06-30 22:06:44 -07:00
Himanshu Gupta 473bc101d7 for example/tutorial, use derby instead of mysql 2015-06-30 20:47:32 -05:00
Himanshu Gupta 12f5d1743e fix toc links per the shortened names from commit d2346b6834 2015-06-30 14:16:40 -05:00
fjy 8c551ed889 very minor fix to the TOC 2015-06-29 10:49:45 -07:00
Himanshu Gupta 4114b4902e fixing the links to doc images 2015-06-29 12:17:42 -05:00
Bingkun Guo 4a0ae7d8d5 Fix inconsistent druid property names for "druid.metadata.storage.tables.xxx" between document and code 2015-06-29 10:12:30 -05:00
Xavier Léauté 2da12de598 add back query/node/(time|ttfb) docs 2015-06-26 17:58:47 -07:00
Charles Allen fbcac10e00 Remove metrics emitting from caching clustered client 2015-06-26 10:49:13 -07:00
Fangjin Yang 9859555346 Merge pull request #1446 from conradlee/docs-segment-details
Improve segment file documentation
2015-06-24 12:14:59 -07:00
Conrad Lee f214b412ca Improve segment file documentation 2015-06-24 11:45:23 -07:00
Fangjin Yang 9104d08a6a Merge pull request #1461 from rasahner/doc_queryFixes
minor fixes to TimeseriesQuery and Aggregations
2015-06-24 08:12:09 -07:00
sahner a938e65315 minor fixes to TimeseriesQuery and Aggregations 2015-06-24 08:24:07 -05:00
Fangjin Yang 726ed432a1 Merge pull request #1451 from rasahner/doc_minorFixes
minor documentation fixes in Tasks.md, index.md, indexing-service.md
2015-06-23 10:15:47 -07:00
Fangjin Yang 2544f3655e Merge pull request #1457 from ravishrathod/rabbitmq-doc
updating doc for rabbitmq firehose
2015-06-23 08:24:49 -07:00
Xavier Léauté 36b4453789 Merge pull request #1455 from druid-io/fix-protobuf
Fix protobuf impl and docs
2015-06-22 23:15:40 -07:00
Xavier Léauté d9dfff0689 fix broken redirects 2015-06-22 23:04:31 -07:00
ravishrathod 9213fd3801 updating doc for rabbitmq firehose 2015-06-22 02:40:11 -04:00
fjy 9c74993559 fix protobuf impl and docs 2015-06-20 21:59:38 -07:00
sahner 4ba34fe43d minor documentation fixes in Tasks.md, index.md, indexing-service.md 2015-06-19 17:09:53 -05:00
nishant fb4052d577 JavaScript Worker Select Strategy
this PR adds a JavaScriptWorkerSelectStrategy which allows defining
arbitrary logic for selecting workers to run task using a JavaScript
function.

This gives users full control to implement complex worker selection
strategies based on task attributes.

more tests and a complex javascript config

fix for java8 modify for nashorn compatibility
2015-06-20 02:01:34 +05:30
Xavier Léauté 6763e3780a Merge pull request #1417 from metamx/rtr-fixes
fix behaviour of middlemanagers around ZK disconnects , fixes #709
2015-06-10 14:42:49 -07:00
Bingkun Guo bdf4d541e3 Fix issue #1016 that if user specifies extension coordinates, the local extensions will be loaded twice,
which could cause Guice duplicate binding errors.
Add unit test to replicate duplicate extension issue. Update documents. Add an package accessible getter for loadersMap for testing only.
Notice that extensions explicitly specified in druid.extensions.coordinates have a higher priority than ones included in the classpath.
Extension modules that don't have a canonical class name will be ignored.
2015-06-10 15:48:03 -05:00
nishant e9afec4a2b fix task status issues on zk outages
docs

review comments

fix test

review comments

Review comments

fix compilation

fix typo
2015-06-11 00:49:52 +05:30
fjy afdd9304cf rename servermonitor to historicalmetricsmonitor in docs 2015-06-05 17:32:04 -07:00
fjy 6744eb2889 Add more links for talks 2015-06-05 14:35:56 -07:00
Xavier Léauté 35e2fde18e Merge pull request #1386 from himanshug/aggregation_testing1
General class for testing any Aggregation Implementation
2015-06-03 23:43:36 -07:00
Charles Allen f48db09e35 Add optimizations for ExtractionFn by enabling MANY_TO_ONE vs ONE_TO_ONE codepaths
* Also adds LookupExtractionFn and MapLookupExtractor which takes in an explicit mapping of renames
* Add injective to javascript extraction fn
2015-06-02 12:22:56 -07:00
Himanshu Gupta 215c1ab01e UTs for hyperUnique aggregation 2015-06-01 12:52:40 -05:00
fjy 74d8840414 Change tranquility links 2015-05-31 10:59:38 -07:00
Himanshu Gupta be4ecc4b91 in batch ingestion metadataUpdateSpec->type is derby, mysql etc and not metadata 2015-05-29 22:16:18 -05:00
Xavier Léauté 96103e1227 remove whirr from documentation 2015-05-29 20:55:42 -05:00
Xavier Léauté d2346b6834 shorten links and file names
* remove redundant parts in file names
* delete unsupported "Druid-Personal-Demo-Cluster"
2015-05-29 20:55:42 -05:00
Himanshu Gupta 8edc2aaca3 renaming all *.md filenames to only have lowercase and dashes
so that they are editable on case-insensitive os as well
2015-05-29 20:55:42 -05:00
fjy 185c99deec Add schema check for overlap between dimension and metric names 2015-05-28 22:18:08 -07:00
janakg 5812b3dd5c druid production architecture diagram 2015-05-27 17:06:24 -07:00
gbuisson 9e90dc0055 added clj-druid to query libraries (clojure) 2015-05-22 17:49:31 +02:00
Xavier Léauté e6d22565ad Merge pull request #1368 from himanshug/fix_seg_metadata_complex_column
in segment metadata query return actual complex data type name
2015-05-15 19:52:16 -07:00
Himanshu Gupta 2fd3e9e8e5 return size = 0 in ColumnAnalysis if its unknown
that is if complex agg did not implement inputSizeFn() so
that segment metadata query shows atleast some information.
also instead of COMPLEX, return type of data stored.
2015-05-15 20:11:56 -05:00
Fangjin Yang 1d33f46ece Merge pull request #1373 from metamx/segmentMetadataQueryDocClarification
Modify SegmentMetadataQuery.md to refer to flat data instead of `TSV`
2015-05-15 17:27:26 -06:00
Charles Allen 03df582fe4 Modify SegmentMetadataQuery.md to refer to flat data instead of `TSV` 2015-05-15 13:49:51 -07:00
Xavier Léauté 87924c7c66 Merge pull request #1369 from druid-io/add-toc
Add missing toc entry for data ingestion
2015-05-15 10:18:00 -07:00
fjy 614e79a0da Add missing toc entry 2015-05-15 11:16:51 -06:00
Himanshu Gupta 4b907a93c5 updating the aggregators doc 2015-05-13 09:25:57 -05:00
Himanshu Gupta f2adb22350 documenting druid alerts schema 2015-05-09 21:03:10 -05:00
fjy 70f089594f Add back some links in TOC 2015-05-07 16:38:09 -07:00
fjy b19435d172 fix typos with batch ingestion in docs 2015-05-07 14:46:17 -07:00
Gian Merlino 1519a1c930 Tasks are not actually sigkilled. 2015-05-07 08:44:28 -07:00
Xavier Léauté 2ec87838d2 Merge pull request #1333 from himanshug/convert_props_ut
UTs for ConvertProperties
2015-05-03 15:11:53 -07:00
Himanshu Gupta eca69fd084 remove validation query config from docs as its not valid anymore 2015-05-03 08:41:35 -05:00
Himanshu Gupta fd3b23072b fixing defaults for druid.[host,port,service] in docs etc 2015-05-02 15:44:03 -05:00
Himanshu Gupta ce7ad41aaa removing port number from druid.host in production-cluster-configuration as its gives the intuition that port name is mandatory to be specified which is not true anymore 2015-05-02 15:34:36 -05:00
Himanshu Gupta 3982301484 add audit table name configuration doc 2015-04-30 06:45:57 -05:00
Xavier Léauté 8b16d70e19 Merge pull request #1320 from davrodpin/fix-deep-storage-doc
Remove duplicated Azure doc. section.
2015-04-29 16:09:37 -07:00
David Rodrigues 1dda3268d1 Remove duplicated Azure doc. section. 2015-04-29 11:42:57 -07:00
Himanshu Gupta 42ed88e1cb adding druid.broker.retryPolicy.numTries config to documentation 2015-04-28 13:22:32 -05:00
Himanshu Gupta 0beaec5578 adding druid.server.maxSize config doc 2015-04-28 09:34:05 -05:00
Xavier Léauté 721505c017 Merge pull request #1208 from druid-io/rework-metrics
Schemaless metrics + additional metrics for things we care about
2015-04-27 15:04:54 -07:00
fjy 963e5765bf Schemaless metrics + additional metrics for things we care about 2015-04-27 13:39:40 -07:00
Alexander Makarenko 070da15a66 Update Realtime-ingestion.md
Add note about sharding and firehose configurations
2015-04-26 16:55:49 +03:00
Vadim Ogievetsky 58879ca0bf added facetjs to the library list in the docs 2015-04-22 18:38:01 -07:00
Fangjin Yang f15a41270a Merge pull request #1302 from metamx/fix-groupby-doc
Updates groupBy doc:
2015-04-21 20:22:23 -07:00
Igal Levy 1216512e49 Updates groupBy doc:
* changes sample dimension names to be more meaningful
* fixes timestamps in example output
* adds formatting to names appearing in paragraphs for easier reading
2015-04-21 17:30:37 -07:00
fjy be69946751 fix typo 2015-04-21 11:14:26 -07:00
fjy 4575c5378c Rework the druid docs 2015-04-21 10:11:11 -07:00
sahner 6ef2941975 minor fixes:
1. leader endpoint returns a string, not a json object
2. "enabled and enabled" -> "enabled and disabled"
2015-04-20 17:02:44 -05:00
Fangjin Yang 8b68b8f3c2 Merge pull request #1282 from davrodpin/validate-deep-storage-impl
Add doc. on how to validate a deep storage implementation.
2015-04-16 20:40:54 -07:00
David Rodrigues 5b76a52c99 Reformat Deep Storage options to use table. 2015-04-16 16:30:42 -07:00
David Pinheiro f912e45464 Add doc. on how to validate a deep storage implementation. 2015-04-16 16:10:43 -07:00
David Pinheiro baeef08c4c Add Microsoft Azure as a Deep Storage option. 2015-04-16 15:39:36 -07:00
Xavier Léauté 49d1f4e568 add documentation for dimension compression 2015-04-14 10:44:19 -07:00
Xavier Léauté 66a4daa363 [docs] fix broken links 2015-04-09 17:36:20 -07:00
Slim d854ceebbd typo or backward compatibility problem
2147483648 will break the node since druid 0.7 it is parsing it as an `int`
2015-04-08 19:12:01 -05:00
Xavier Léauté 9962a33cdf Merge pull request #1270 from metamx/fix-broken-link-druid-cluster-tutorial
fixes #112 broken link
2015-04-08 13:36:40 -07:00
Igal Levy 905fa84d45 fixes #112 broken link 2015-04-08 13:23:10 -07:00
Slim a9b5529ff8 fix typo (missing ```) 2015-04-07 14:54:04 -05:00
Fangjin Yang 834fd71e34 Merge pull request #1254 from bobrik/on-the-right
Menu is on the right now
2015-03-31 15:58:57 -07:00
Prajwal Tuladhar def4cddfea fix doc link 2015-03-31 18:19:24 -04:00
Ian Babrou a6887da923 menu is on the right now 2015-03-30 21:59:15 -07:00
Slim 425c5f750c Fix typo in the query body
last `,` will break the query.
2015-03-30 09:34:38 -05:00
Xavier Léauté 2c58e355fd Merge pull request #1239 from metamx/log4j2forRealz
Exclude log4j from curator dependencies in favor of log4j-1.2-api
2015-03-26 13:21:13 -07:00
Charles Allen 9cd6c08e96 Exclude log4j from curator dependencies in favor of log4j-1.2-api 2015-03-26 13:05:12 -07:00
Xavier Léauté 47cc11829d Merge pull request #1240 from himanshug/async_query_forwarding_servlet_router_fix
adding router numMaxThread configuration for ProxyServlet HttpClient executor
2015-03-26 10:15:44 -07:00
nishantmonu51 99bf430844 fix examples and doc 2015-03-26 19:01:10 +05:30
Himanshu Gupta b131a4fd3b adding router numMaxThread configuration for ProxyServlet HttpClient executor 2015-03-23 21:58:01 -05:00
fjy 6a47c1530c update versions to prepare for rc release 2015-03-19 11:39:38 -07:00
fjy db06bda5e0 fix ingestion docs 2015-03-18 17:23:22 -07:00
fjy 3e664a4c32 fix plumber docs 2015-03-18 15:56:27 -07:00
fjy 8de3eab832 fix doc formatting 2015-03-18 15:53:00 -07:00
fjy bfe10bd156 This fixes arbitrary gran spec breaking 2015-03-17 12:19:43 -07:00
fjy 36b4c6a371 fix set up with first tutorial 2015-03-13 13:54:45 -07:00
Xavier Léauté 7fb4b1d2bb Merge pull request #1144 from metamx/javaLoggingManager
Add log4j2 hooks to other loggings
2015-03-12 14:38:51 -07:00
Charles Allen edfcea18d8 Add log4j2 hooks to standard java logging 2015-03-12 14:29:41 -07:00
nishantmonu51 3ec4a30ab5 initial commit
review comments

more refactoring and cleaning of redundant code

add UT + docs + more refactoring

fixes + review comments

more cleanup

end points to fetch history

review comments

remove unnecessary changes

review comments rename header name

review comments + add test for MetadataRulesManager

review comments docs
2015-03-12 22:50:29 +05:30
Xavier Léauté 11b3230602 update to kafka 0.8.2.1, because it's better™ 2015-03-12 09:59:24 -07:00
Xavier Léauté d3f5bddc5c Add ability to apply extraction functions to the time dimension
- Moves DimExtractionFn under a more generic ExtractionFn interface to
  support extracting dimension values other than strings
- pushes down extractionFn to the storage adapter from query engine
- 'dimExtractionFn' parameter has been deprecated in favor of 'extractionFn'
- adds a TimeFormatExtractionFn, allowing to project the '__time' dimension
- JavascriptDimExtractionFn renamed to JavascriptExtractionFn, adding
  support for any dimension value types that map directly to Javascript
- update documentation for time column extraction and related changes
2015-03-11 16:45:42 -07:00
Xavier Léauté 9f242ed1ba Merge pull request #1182 from druid-io/better-docs
Rework the druid docs and fix many mistakes
2015-03-10 14:23:38 -07:00
fjy 064d65a009 rework the druid docs and fix many mistakes 2015-03-10 14:19:19 -07:00
Fangjin Yang 0b467624ec Merge pull request #694 from druid-io/arithmetic-op-strategies
normal division & configurable ordering for ArithmeticPostAggregator
2015-03-10 13:48:27 -07:00
Fangjin Yang 2d79c6e5b9 Merge pull request #1187 from metamx/update-design-diagram
updates Realtime node's dataflow connection, s/historical/deep storage; ...
2015-03-10 13:46:21 -07:00
Fangjin Yang fa2c431589 Merge pull request #1186 from metamx/dataFormatDocsUpdate
Update Data_formats.md to match arguments used in 0.7.x
2015-03-10 13:45:37 -07:00
Igal Levy 705a7b3f23 updates Realtime node's dataflow connection, s/historical/deep storage; a bit more information on how Realtime nodes drop segments based on said segments' uptake by Historicals 2015-03-10 13:44:36 -07:00
Charles Allen 2ac610771e Update Data_formats.md to match arguments used in 0.7.x 2015-03-10 13:41:11 -07:00
Fangjin Yang c7ec8ba15b Merge pull request #1184 from metamx/docs-event-receiver-firehose
fix docs and add docs for event receiver firehose
2015-03-10 10:23:19 -07:00
nishantmonu51 425ad98261 fix docs and add docs for event receiver firehose 2015-03-10 21:29:57 +05:30
Fangjin Yang cee7e12cb7 Merge pull request #1183 from metamx/update-design-diagram
updates diagram: s/MySql/metadata storage
2015-03-09 16:54:03 -07:00
Igal Levy f98e168ed5 updates diagram: s/MySql/metadata storage' 2015-03-09 16:41:21 -07:00
Fangjin Yang 2abdce1dc0 Merge pull request #1180 from metamx/logging-groupBy-NPE
add null check early to catch root cause for groupBy NPE while running bySegment query
2015-03-09 09:16:33 -07:00
nishantmonu51 704f463dbf fix property name in doc 2015-03-09 21:19:29 +05:30
Himanshu Gupta cb5ed4942c simple typo fix on Concepts-and_Terminology.md to correct navigation to spec file info 2015-03-07 11:29:24 -06:00
Xavier Léauté 0d47c0c36d normal division and configurable ordering for ArithmeticPostAggregator
Fixes #510
2015-03-04 12:44:24 -08:00
Fangjin Yang 9578304d6f Merge pull request #1130 from housejester/jde-building
Various improvements to the tutorial and building experience.
2015-03-02 14:48:32 -08:00
Fangjin Yang e8605c63a9 Merge pull request #1150 from himanshug/broker-parallel-chunk-process
interval chunk query runner now processes individual chunk in a threadpool
2015-03-02 13:50:23 -08:00
Himanshu Gupta 29039fd541 interval chunk query runner now processes individual chunk in a thread pool and prints metrics query/time per chunk 2015-03-02 15:45:09 -06:00
Fangjin Yang a76a24742b Merge pull request #1149 from himanshug/broker-get-datasource-update
updating ClientInfoResource.getDatasource() with a full flag
2015-03-02 13:28:27 -08:00
Himanshu Gupta a39f4f8bf4 updating ClientInfoResource.getDatasource() with a full flag to return all the served intervals for given data source along with the dimension and metric information 2015-03-02 14:18:52 -06:00
Xavier Léauté 437d61ed8f Merge pull request #1164 from mrijke/docfix-cache-configs
Docfix: Update Cache config pages
2015-03-02 11:47:50 -08:00
Maarten Rijke 2531bb9664 Docfix: Update Cache config pages
* Update Broker-Config.md Cache options table
 * Update Historical-Config.md Cache options table (broker->historical)
2015-03-02 19:55:00 +01:00
Xavier Léauté 1d98717d24 Merge pull request #1162 from mrijke/fix-metric-monitor-names
Update metrics monitors names in Configuration.md
2015-03-02 10:33:19 -08:00
Maarten Rijke 092acaea9d Update metrics monitors names in Configuration.md
Updated the metric package names to reflect the new names in 0.7
2015-03-02 12:16:47 +01:00
Maarten Rijke d52f3bb817 Update default value for druid.emitter in Configuration.md 2015-03-02 12:06:30 +01:00
James Estes 9cb5d9bd00 Various improvements to the tutorial and building experience.
- Improve geo ip download stability
- Retry default geo ip download if corrupt
- Clarify docs on building & where to get artifact
2015-02-25 18:49:16 -07:00
Prajwal Tuladhar d1fb42c554 fix broken docs link 2015-02-24 23:41:13 -05:00
fjy 46bd614d63 fix broken links 2015-02-24 20:14:17 -08:00
Xavier Léauté 667c3d4f8c fix typo 2015-02-19 11:41:37 -08:00
James Estes 562de6c621 Update docs and examples for log4j2 usage.
- Put configs early in classpath in examples so log4j2.xml will get picked up properly
- Add an example log4j2.xml file.
- Update Logging doc.
2015-02-19 11:40:56 -07:00
Xavier Léauté 475ba9d068 remove build.sh in favor of mvn package 2015-02-19 10:32:41 -08:00
Xavier Léauté 8e33e58422 rename Druid binary assembly 2015-02-18 17:42:55 -08:00
fjy 6d05f4fe5e Add more compare docs 2015-02-18 15:57:36 -08:00
Fangjin Yang 146a6a3b2b Merge pull request #1135 from metamx/topNDocumentation
Add more TopN documentation regarding Aliasing
2015-02-18 15:39:27 -08:00
Xavier Léauté 19e11ae601 Use default ports in examples 2015-02-18 11:46:27 -08:00
Charles Allen 2e27b888ed Add more TopN documentation regarding Aliasing
* Addresses concerns in https://github.com/druid-io/druid/issues/1134
2015-02-18 11:27:53 -08:00
Xavier Léauté cff218ee20 update example + tutorial for kafka 0.8 2015-02-17 23:58:51 -08:00
Xavier Léauté 018653700b update docs jekyll config 2015-02-17 23:58:50 -08:00
fjy 3fab045b2d Go through and fix mistakes in tutorials and docs 2015-02-17 15:21:16 -08:00
cheddar 2a45501393 Merge pull request #1110 from metamx/move-to-sonatype
Move Druid release artifacts to Sonatype
2015-02-17 11:54:16 -08:00
Xavier Léauté 78df7f6165 Move Druid release artifacts to Sonatype
- Switch to using Druid parent POM
- Add required fields for Sonatype
- Common plugin versions and settings have been moved to the parent pom
- Cleanup artifacts and POMs for consistent formatting
- Remove org.hyperic.sigar dependency and update docs to reflect necessary jars to add at runtime when sigar is needed
2015-02-13 14:26:31 -08:00
James Estes 8d1aa2beb3 Add common config dir to classpath in examples
Also adds query examples in the Cluster tutorial
2015-02-13 12:59:08 -07:00
fjy 13de51494e update versions in tutorials and docs 2015-02-12 16:11:31 -08:00
Fangjin Yang e915675b97 Merge pull request #1112 from druid-io/fix-examples
Update realtime node configs for examples; fixes #1111
2015-02-12 15:44:15 -08:00
fjy 34ed474d40 Add docs for http endpoints 2015-02-11 16:12:51 -08:00
fjy 63338e6e35 Update realtime node configs for examples; fixes #1111 2015-02-11 14:34:02 -08:00
Xavier Léauté 72333731c3 Merge pull request #1101 from druid-io/update
Update poms and add docs for backwards compat JSON parser
2015-02-09 16:14:57 -08:00
fjy e5debb2dec update poms and add docs for backwards compat 2015-02-09 16:13:00 -08:00
Fangjin Yang cb4f894ef7 Merge pull request #1087 from metamx/fixIndexerBaseRelativity
Fix overriding base of IndexerZkConfig to be absolute instead of relative
2015-02-04 14:10:21 -08:00
Fangjin Yang 60b2d920bc Merge pull request #1088 from metamx/upload-docs-s3
Upload documentation to static.druid.io
2015-02-04 13:23:27 -08:00
Charles Allen 79a3e8f59f Fix overriding base of IndexerZkConfig to be absolute instead of relative
* Updated docs to clarify ZK config behavior
* Added unit tests for this case
2015-02-04 13:04:06 -08:00
Xavier Léauté ae78b1450e upload documentation to static.druid.io 2015-02-04 12:56:52 -08:00
fjy 67ecb07ccd update version in docs 2015-02-03 14:26:42 -08:00
fjy d05032b98a towards a community led druid 2015-01-31 20:57:36 -08:00
fjy b9337e883a I'm all about them doc fixes 2015-01-21 22:04:48 -08:00
fjy f437e16543 moar doc fixes 2015-01-21 21:59:16 -08:00
fjy e33e45e69e one more doc fix 2015-01-21 21:52:23 -08:00
fjy 74a4fddcfe fix task docs 2015-01-21 21:48:48 -08:00
fjy 7540f2f562 Add more docs for production clusters 2015-01-21 14:21:27 -08:00
Xavier Léauté 5a2030c49b Merge pull request #1059 from druid-io/remove-test
Remove test rejection policy - it has no place anywhere
2015-01-20 13:39:46 -08:00
Xavier Léauté a74ced97d9 Merge pull request #1058 from druid-io/equal-dist
Add a new equal distribution strategy for assigning tasks
2015-01-20 13:27:32 -08:00
fjy 1ff5829d6d remove test rejection policy 2015-01-20 13:22:30 -08:00
fjy 2d516fa591 Add a new equal distribution strategy for assigning tasks 2015-01-20 13:12:22 -08:00
Charles Allen 3d27747f7e Upgrade to log4j2
Default behavior is as before.
Added documentation for how to enable synchronous logging for select chatty classes:
* io.druid.client.ServerInventoryView
* io.druid.client.BatchServerInventoryView
* io.druid.curator.inventory.CuratorInventoryManager
* com.metamx.http.client.pool.ChannelResourceFactory
2015-01-20 12:35:18 -08:00
Fangjin Yang 91a79dbf95 Merge pull request #1031 from metamx/ingestmetadata-query
DataSourceMetadata query
2015-01-19 21:55:35 -08:00
Fangjin Yang 068518216b Merge pull request #1048 from metamx/update-tutorial
update tutorial to create db utf8 by default
2015-01-19 11:02:17 -08:00
Xavier Léauté 2443dda46b update tutorial to create db utf8 by default 2015-01-16 20:52:40 -08:00
nishantmonu51 503631f6ff review comments. 2015-01-16 20:21:40 +05:30
Igal Levy e86273a9ae New section to provide detail on how and why to set up sharding
* Different types of sharding strategies
* How and what to configure
* Redundancy and scale-out
2015-01-15 21:19:53 -08:00
nishantmonu51 c7452b75f6 Merge branch 'master' into ingestmetadata-query 2015-01-15 18:00:31 +05:30
Maarten Rijke 58b334c8e6 Fix some of the Task json examples to have the correct description 2015-01-15 09:41:23 +01:00
fjy a02ef1dbf1 update batch ingest docs 2015-01-08 14:38:10 -08:00
Gian Merlino dde21a16e1 Improve maxPendingPersists docs. 2015-01-07 09:55:33 -08:00
Xavier Léauté 47fa1b3d10 update docs to point to druid-io/druid 2014-12-23 11:26:52 -08:00
fjy ac407fb6ba clean up defaults 2014-12-15 15:05:02 -08:00
Fangjin Yang 335fbebb94 Merge pull request #960 from metamx/new-config
Rewrite config docs for 0.7.x
2014-12-15 13:35:25 -07:00
Charles Allen 225a039a01 Merge pull request #965 from metamx/doc-fix
Fix documented default value for druid.coordinator.merge.on.
2014-12-13 11:13:14 -08:00
Gian Merlino 020f8653f0 Fix documented default value for druid.coordinator.merge.on. 2014-12-13 08:03:09 -08:00
fjy 818f85e745 rewrite config docs 2014-12-11 15:52:26 -08:00
nishantmonu51 a0d3579a92 add docs + fix tests 2014-12-11 17:58:01 +05:30
Fangjin Yang d6d3ec6846 Merge pull request #948 from metamx/ingestion-docs
Redocumenting ingestion
2014-12-09 15:30:03 -07:00
fjy f0e5bb1ba9 remove excess tutorials 2014-12-09 14:29:19 -08:00
fjy 5727193873 more fixes to docs 2014-12-09 14:28:15 -08:00
fjy 4e3d8faed8 fix bad example 2014-12-09 14:21:47 -08:00
fjy 9596c11f42 address cr 2014-12-09 14:19:18 -08:00
fjy 38a369c0ff fix links and some errors 2014-12-08 16:20:35 -08:00
fjy 28b72a69ad redocumenting ingestion 2014-12-08 16:15:46 -08:00
Charles Allen 0fbbcd5885 Move some config description from Configuration.md to the Broker and Historical config pages 2014-12-08 15:53:58 -08:00
xvrl 1392e2731f Merge pull request #936 from metamx/cachingRunnerImprovements
General Caching Query Runners cleanup (40% query time reduction for HLL)
2014-12-08 14:07:52 -08:00
Charles Allen 7b65f0635d General Caching Query Runners cleanup
* Add type strictness to CachingClusteredClient.
* Add background caching to CachingClusteredClient. Gives between 0% and 5% query speed increase.
* Add @BackgroundCaching annotation for injected ExecutorService items
* Add `numBackgroundThreads' configuration options to CacheConfig (default 0 aka same thread legacy behavior)
* Add unit tests for CacheConfig
* Add an abstract caching query runner class, currently it doesn't do anything exceppt simply make the two caching queries distinct.
* Add caching to CachingQueryRunner. Gives up to a WHOPPING 40% reduction in query time on HLL queries
* Updated docs with more info on cache settings.
2014-12-08 13:29:32 -08:00
Fangjin Yang 86a837a653 Merge pull request #939 from 7eggs/doc-fixes
Fix link to Druid Cluster Tutorial in Booting a Druid Cluster page
2014-12-08 12:10:15 -07:00
Alexander Makarenko 6dfa0d20b8 Fix link to Druid Cluster Tutorial in Booting a Druid Cluster page 2014-12-07 21:59:06 +03:00
Fangjin Yang 1a4e798373 Merge pull request #938 from 7eggs/doc-fixes
Broker config small fix
2014-12-07 09:48:53 -07:00
Alexander Makarenko df5b525097 Broker config small fix 2014-12-07 18:59:22 +03:00
Alexander Makarenko bbd337235e Add Node.js client to Community Libraries 2014-12-07 03:51:35 +03:00
fjy 8ee4d12562 Refactor structure for examples and extensions 2014-11-21 14:45:24 -08:00
nishantmonu51 e3260aa177 Filtered Aggregator fixes + enhancements
- fix NPE on IncrementIndex
- refactor code to support AND, OR filter
- tests for AND & OR filter
- handling for missing column / null values
2014-11-20 15:17:18 -08:00
nishantmonu51 2f08ab85fc Revert "update doc"
This reverts commit d66d0de54f.
2014-11-20 10:17:07 +05:30
nishantmonu51 d66d0de54f update doc 2014-11-20 09:29:26 +05:30
Fangjin Yang db8a168b91 Merge pull request #881 from mrijke/update-ruleconf-docs
Update Rule-Configuration docs: match example with description
2014-11-18 16:41:49 -07:00
Fangjin Yang 87ea6fd377 Merge pull request #878 from metamx/default-extension-version
use Druid version as the default version for extensions
2014-11-18 13:56:54 -07:00
Xavier Léauté bf04ad0eb3 add documentation 2014-11-18 10:50:18 -08:00
Maarten Rijke 06e4535cb8 Update Rule-Configuration: match example with description
The example JSON snippets of the 'loadForever' and the 'dropForever' don't match their description, the 'type' description mentions 'loadByInterval' and 'dropByPeriod' respectively.
2014-11-18 17:00:42 +01:00
nishantmonu51 b2e8073c61 document combining firehose 2014-11-18 11:01:08 +05:30
fjy afd419ea29 fix bugs with docs 2014-11-14 13:25:21 -08:00
Fangjin Yang d85c172285 Merge pull request #808 from metamx/igalDruid
Clarify value types for granularities; link to more ISO info; more info on OrderByColumnSpec
2014-11-14 14:20:37 -07:00
Eric Bouchut e89f2f973d Fix typo in DimensionSpecs 2014-11-13 15:35:09 +01:00
Xavier Léauté ba442b9882 fix default FilteredServerView + batch annoucement 2014-11-12 23:17:16 -08:00
fjy 26b358e127 Merge branch 'master' of github.com:metamx/druid into druid-0.7.x 2014-11-11 17:30:30 -08:00
Igal Levy de7fb5a690 fixed typo, added note on matching schema requirement for unioned data sources 2014-11-11 11:56:22 -08:00
Igal Levy 34a6f7f7ca Merge branch 'master' into fix-datasource-doc 2014-11-11 11:55:47 -08:00
fjy 6188315293 Merge branch 'master' into druid-0.7.x
Conflicts:
	processing/src/test/java/io/druid/query/search/SearchQueryRunnerTest.java
2014-11-10 14:52:10 -08:00
Igal Levy 03515a4de6 fixes typo 2014-11-07 10:40:46 -08:00
xvrl 18b9c5cd2f db -> metadata 2014-11-06 21:46:35 -08:00
Igal Levy 20562492de clarified that dimension and metric can be String too, not just object 2014-11-06 15:16:36 -08:00
Igal Levy ec0accf0b1 clarified that dimensions and metrics are given in String arrays 2014-11-06 15:13:30 -08:00
Igal Levy 8f0b675e23 Added info to better explain how to use pagingSpec 2014-11-06 14:50:01 -08:00
Igal Levy 5a98fcf5b9 typo: s/metric_value/metric_name/ and added quotes 2014-11-06 11:18:30 -08:00
Igal Levy 73226f20cc typo: s/metric_value/metric_name/ 2014-11-06 11:01:31 -08:00
Igal Levy 81227b63a7 Merge branch 'master' into fix-query-docs 2014-11-04 11:09:13 -08:00
xvrl 2c62b9e6e3 Merge pull request #821 from metamx/minor-fix
Some minor fixes in docs and change jetty defaults
2014-11-04 10:18:57 -08:00
fjy 42cf20c9f1 some minor fixes in docs and change jetty defaults 2014-11-04 10:08:50 -08:00
Xavier Léauté f56d026d7d fix realtime module + cleaner metadata abstraction
- better metadata abstractions
- more db->metadata renaming
2014-11-03 16:40:23 -08:00
Igal Levy c58fc503d9 Clarified sort as optional, object 2014-11-03 11:59:04 -08:00
Igal Levy b416c3ff5c added quotes to clarify string value; added example of numeric filter 2014-11-03 11:57:52 -08:00
Xavier Léauté 48e6bef250 update documentation with new property names 2014-11-03 09:25:39 -08:00
Igal Levy e07b02f54b fixed typo and added info that union can take any number of dataSources 2014-10-31 11:07:55 -07:00
Igal Levy 0906b21079 Merge branch 'master' into fix-datasource-doc 2014-10-30 16:26:16 -07:00
Igal Levy 479c196aa4 Added section on union data sources 2014-10-29 12:32:24 -07:00
Igal Levy 23315e298d expanded and clarified OrderByColumnSpec section 2014-10-28 10:38:21 -07:00
Xavier Léauté 575919ec6c provide a little more involved example 2014-10-27 19:18:09 -07:00
Igal Levy f62221bfae added link to DimensionSpec for alternative way to provide dimensions 2014-10-23 14:30:02 -07:00
Igal Levy f76d68e2c9 added missing anchor to fix broken internal link 2014-10-23 12:49:44 -07:00
Igal Levy 374ba8ab5e minor edits based on Vad's feedback 2014-10-23 11:38:01 -07:00
fjy d76d57d95d update docs 2014-10-22 16:16:28 -07:00
nishantmonu51 0785baf54c review comments 2014-10-22 21:28:31 +05:30
nishantmonu51 6cb6ec39cf add support for not filter and add test 2014-10-22 10:25:24 +05:30
Xavier Léauté 34f2c74331 add docs 2014-10-21 17:24:44 -07:00
fjy ee392b6064 more updates to docs 2014-10-21 16:26:17 -07:00
fjy 2d96bc5f1f more FAQ docs 2014-10-21 16:08:56 -07:00
Fangjin Yang 5df92aff2c Merge pull request #797 from metamx/igalDruid
Igal druid
2014-10-21 16:47:47 -06:00
Xavier Léauté 9d5e65913d fix serde 2014-10-21 15:05:08 -07:00
Xavier Léauté b32ce31eff add documentation 2014-10-20 14:12:30 -07:00
Igal Levy 1dd9aeba4f removed link with no target 2014-10-19 21:22:10 -07:00
Igal Levy b7c4983a91 added syntax highlighting 2014-10-19 18:38:20 -07:00
Igal Levy 63a4f2dc05 fixed broken internal link 2014-10-19 18:29:34 -07:00
Fangjin Yang 82fc6e5358 Merge pull request #793 from metamx/partitions-docs
New PartitionsSpec docs.
2014-10-19 15:33:16 -06:00
Gian Merlino 20d29666af New PartitionsSpec docs. 2014-10-16 13:17:13 -07:00
fjy 35716e8dee add a best practices doc 2014-10-13 14:33:45 -07:00
fjy d4f5a71ded prepare for next release 2014-10-13 12:48:12 -07:00
Xavier Léauté 8304365948 move thanks page to main site 2014-10-10 10:19:33 -07:00
fjy 0532208c25 prepare for next release 2014-10-09 14:21:38 -07:00
Xavier Léauté fa27908ae4 and another one, fixes #782 2014-10-09 10:28:33 -07:00
Xavier Léauté e2e912cdc3 fix broken link, resolved #782 2014-10-09 10:26:47 -07:00
fjy 21ac3f2aab slight clarification about hadoop setup 2014-10-08 10:56:00 -07:00
fjy 8a34a55c0b add example hadoop config 2014-10-08 10:51:00 -07:00
fjy 1ce67dcfc4 prepare for next release 2014-10-07 14:27:47 -07:00
fjy 19dc32d314 add headers for druid vs pages 2014-10-07 13:50:32 -07:00
fjy f1d37d569c doc fixes 2014-10-07 13:01:41 -07:00
fjy ee0e856029 correct mistakes 2014-10-06 14:23:23 -07:00
fjy 546fca2d67 docs for router node 2014-10-06 14:18:25 -07:00
fjy d9453f29f1 fix docs 2014-09-30 15:02:06 -07:00
fjy 5b166134fb add docs 2014-09-30 14:59:02 -07:00
fjy e367f76824 Merge pull request #757 from pdeva/master
FAQ on changing data granularity
2014-09-29 15:40:37 -06:00
pdeva bc66b42acc FAQ on changing data granularity
added cause this question is asked a ton of times on google group and i had a similar question and had to resort to the forums cause there was no doc.
2014-09-25 01:48:45 -07:00
Xavier Léauté 13ac7de882 update doc versions 2014-09-23 16:28:18 -07:00
fjy c042ce5b4b prep next rel 2014-09-23 16:10:12 -07:00
fjy d19723e186 Merge pull request #749 from metamx/background-segment-announcing
Parallelize segment loading + background announcing
2014-09-23 17:07:51 -06:00
fjy bdea4f45ea fixes #733 2014-09-23 15:41:56 -07:00
Xavier Léauté d7b39fa7ea add documentation 2014-09-23 09:30:40 -07:00