Commit Graph

81 Commits

Author SHA1 Message Date
Jonathan Wei 1bddfc089c Additional docs/log for direct memory usage (#4631)
* Additional docs/log for direct memory usage

* Tweak docs

* Doc rewording
2017-08-10 23:33:20 -07:00
Himanshu ae6780f62a rolling upgrade order change to bring coordinator and overlord together (#4281)
* rolling upgrade order change to bring coordinator and overlord together

* mentioned merged Coordinator-Overlord in upgrade order doc

* revert autoscaling doc change

* auto scaling doc fix
2017-07-25 12:54:12 -05:00
Parag Jain 6e2f78f552 TLS support (#4270) 2017-07-06 17:40:12 -07:00
Parag Jain 4502c207af fix injection bug and documentation (#4243) 2017-05-03 15:07:43 -05:00
Jihoon Son 7411b18df9 Add BroadcastDistributionRule (#4077)
* Add BroadcastDistributionRule

* Add missing null check

* Rename variable 'colocateDataSource' to 'colocatedDatasource'

* Address comments

* Document for broadcast rules

* Drop segments which are not co-located anymore

* Remove duplicated segment loading and dropping

* Add caveat

* address comments
2017-05-01 09:55:17 -07:00
Akash Dwivedi 94da5e80f9 Namespace optimization for hdfs data segments. (#3877)
* NN optimization for hdfs data segments.

* HdfsDataSegmentKiller, HdfsDataSegment finder changes to use new storage
format.Docs update.

* Common utility function in DataSegmentPusherUtil.

* new static method `makeSegmentOutputPathUptoVersionForHdfs` in JobHelper

* reuse getHdfsStorageDirUptoVersion in
DataSegmentPusherUtil.getHdfsStorageDir()

* Addressed comments.

* Review comments.

* HdfsDataSegmentKiller requested changes.

* extra newline

* Add maprfs.
2017-03-01 09:51:20 -08:00
Nishant 35160e5595 Add metrics for Query Count statistics (#3470)
* Add metrics for Query Count statistics

This PR adds a new metrics monitor “QueryCountStatsMonitor” which emits
three new metrics -
1) query/success/count - number of successful queries
2) query/failed/count - number of failed queries
3) query/interrupted/count - number of interrupted/timedout queries

fix bindings

* make fields final

* fix imports

* AsyncQueryForwardingServlet implement QueryStatsProvider

* remove unused import
2016-12-19 09:47:58 -08:00
kaijianding 4be3eb0ce7 report message gap, source gap and sink count in RealtimePlumber (#3744)
* report message gap, source gap and sink count in RealtimePlumber

* report message gap, sink count in Appenderator

* add ingest/events/sourceGap in metrics.md

* remove source gap
2016-12-13 11:23:02 -06:00
Jonathan Wei 7c63bee7f5 Add mapreduce.job.classloader.system.classes property to 'Other Hadoop Versions' docs (#3706) 2016-11-18 16:16:50 -08:00
Himanshu b76b3f8d85 reset-cluster command to clean up druid state stored on metadata and deep storage (#3670) 2016-11-09 11:07:01 -06:00
jaehong choi 6f21778364 Support finding segments in AWS S3. (#3399)
* support finding segments from a AWS S3 storage.

* add more Uts

* address comments and add a document for the feature.

* update docs indentation

* update docs indentation

* address comments.
1. add a Ut for json ser/deser for the config object.
2. more informant error message in a Ut.

* address comments.
1. use @Min to validate the configuration object
2. change updateDescriptor to a string as it does not take an argument otherwise

* fix a Ut failure - delete a Ut for testing default max length.
2016-10-10 17:27:09 -07:00
Ashish 6b40bf8b32 doc: added note to README, about necessary hdfs config after insert-segment-to-db (#3402) 2016-08-28 16:39:33 -07:00
Chanh Le d624037698 Pull-deps: correct the library directory in the document (#3361)
* Pull-deps: correct the library directory in the document

* Pull-deps: correct the library directory in the document in the last example command
2016-08-16 17:18:15 -07:00
Fangjin Yang 6beb8ac342 fix some docs and add new content (#3369) 2016-08-16 15:00:18 -07:00
Himanshu ed5b92d612 document how to check MM enabled/disabled (#3331) 2016-08-06 05:56:51 +08:00
Gian Merlino e5397ed316 Link up Hadoop class loading docs better. (#3302) 2016-07-29 10:19:54 -07:00
Charles Allen 546e4f79b0 Add size of pending deletes to historical metrics (#3295)
* Add size of pending deletes to historical metrics
2016-07-27 11:30:47 -07:00
Charles Allen b1e3fe77f5 More logging around how the coordinator balancer is happening (#3279)
* More logging around how the coordinator balancer is happening

* Address comments

* Address code review comments and add actual logging
2016-07-27 13:24:32 +05:30
Gian Merlino dd4ec751d0 Update docs for working with Hadoop dependencies. (#3252)
- Attempt to make things clearer in general
- Point out that HDFS deep storage and MR jobs don't use the same loading mechanism
- Recommend using mapreduce.job.classloader = true when possible
2016-07-18 07:47:58 -05:00
Gian Merlino 6a03a0cfec Fix ingest/persist/backPressure docs. (#3243) 2016-07-13 21:56:28 -07:00
Gian Merlino b8a4f4ea7b DumpSegment: Add --dump bitmaps option. (#3221)
Also make --dump metadata respect --column.
2016-07-06 12:42:50 -07:00
Parag Jain 99844dfeb5 remove need for tmp extensions dir (#3211)
correct lib path relative to base distribution dir
2016-07-01 12:55:57 -07:00
michaelschiff 66d8ad36d7 adds new coordinator metrics 'segment/unavailable/count' and (#3176)
'segment/underReplicated/count' (#3173)
2016-06-23 14:53:15 -07:00
Gian Merlino da660bb592 DumpSegment tool. (#3182)
Fixes #2723.
2016-06-23 14:37:50 -07:00
Gian Merlino 3b3e772748 Add --no-default-remote-repositories flag to pull-deps. (#3120) 2016-06-13 17:01:18 +05:30
Kirill Kozlov 4ab675e863 Fix command name in example (#3088) 2016-06-07 10:44:27 -07:00
Gian Merlino cd5c5419bb Make docs deploying better. (#3040)
- Make redirects for old links based on _redirects.json
- Replace #{DRUIDVERSION} tokens in docs with current version
- Allow origins named something other than "origin"
- Can use either s3cmd or awscli, depending on availability
2016-05-31 15:34:58 -07:00
Fangjin Yang 00de26c76a fix extensions docs (#2995)
* fix extensions docs

* fix mistakes
2016-05-19 14:01:06 -07:00
Himanshu 6c5bf91f9a publish metrics numJettyConns to see how number of active jetty connections change over time (#2839)
this can be compared with numer of active queries to see if requests are waiting in jetty queue
2016-05-02 14:08:25 -07:00
du00cs 639d0630b8 jackson conflict workaround in hadooop ingestio & parquet extension coordinate update (#2817) 2016-04-13 14:20:33 -07:00
Sébastien Launay 37d2ab623e Merge pull request #2815 from slaunay/documentation/hadoop-classpath-issue-fix-with-configuration
Doc for mapreduce.job.user.classpath.first=true
2016-04-12 10:51:51 -07:00
fjy e3e932a4d4 refactor extensions into core and contrib 2016-03-08 17:12:09 -08:00
Bingkun Guo 18f9e05f0f improve doc on including druid and hadoop extensions 2016-02-26 13:53:08 -06:00
Fangjin Yang 083f019a48 Merge pull request #2465 from druid-io/more-doc-fix
more doc fixes
2016-02-17 11:00:38 -08:00
fjy 7da6594bfe more doc fixes 2016-02-17 09:43:47 -08:00
Slim e9f1c94822 Update metrics.md 2016-02-17 09:27:15 -06:00
Slim ebbb1aa74e Update metrics.md 2016-02-17 09:05:16 -06:00
Gian Merlino 95d5526e7c Freshen up rolling update docs
1. Clarify what "Indexing Service / Realtime" means
2. Add info about restore-based middle manager rolling restarts
3. Add info about what happens in middle manager updates
4. More consistent capitalization and spelling of node types
2016-02-09 13:57:04 -08:00
fjy 003f54e268 add doc rendering 2016-02-04 14:21:59 -08:00
fjy 1aa363cea7 new quickstart 2016-02-04 09:37:38 -08:00
Fangjin Yang 459c2a49ca Merge pull request #2364 from metamx/fix2356
Add more docs around timezone handling
2016-02-01 10:58:15 -08:00
Charles Allen c9393e5289 Add more docs around timezone handling
* Fixes #2356
2016-02-01 08:51:07 -08:00
Jaebin Yoon 66a74a2b88 Fixed the broken link 2016-02-01 01:07:24 -08:00
Robin c9368702fa do some editing of the instructions for using mysql for metadata 2016-01-21 10:37:30 -06:00
Gian Merlino 2d3f6e7705 Some more multitenancy docs 2016-01-17 17:47:49 -08:00
Himanshu d255f4baac Merge pull request #2234 from pjain1/emit_realtime_metrics
emit handoff count metrics
2016-01-08 14:24:16 -06:00
Parag Jain 9dba0f67e7 emit handoff count metrics 2016-01-08 12:36:13 -06:00
Fangjin Yang 3048b1f0a5 Merge pull request #2174 from metamx/ingest-size-metrics
Add metrics for ingest/bytes/received for EventReceiverFirehose
2016-01-06 22:05:55 -08:00
Fangjin Yang dd262f0451 Merge pull request #2215 from pjain1/fix_doc_metrics
correct metric name - segment/added/count -> segment/assigned/count
2016-01-06 16:21:54 -08:00
Parag Jain 768d07b702 correct metric name - segment/added/count -> segment/assigned/count 2016-01-06 15:55:11 -06:00