Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,259 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

87 Commits

Author SHA1 Message Date
Himanshu 2ecebb3173 Fix coordinator/overlord redirects when TLS is enabled (#5037) 
* Fix coordinator/overlord redirects when TLS is enabled

* address review comment

* fix UTs

* workaround to not ignore URL instance to fix the teamcity build

* update tls doc
 2017-11-09 13:10:28 -08:00
Jihoon Son 52d7f74226 Add streaming aggregation as the last step of ConcurrentGrouper if data are spilled (#4704) 
* Add steaming grouper

* Fix doc

* Use a single dictionary while combining

* Revert GroupByBenchmark

* Removed unused code

* More cleanup

* Remove unused config

* Fix some typos and bugs

* Refactor Groupers.mergeIterators()

* Add comments for combining tree

* Refactor buildCombineTree

* Refactor iterator

* Add ParallelCombiner

* Add ParallelCombinerTest

* Handle InterruptedException

* use AbstractPrioritizedCallable

* Address comments

* [maven-release-plugin] prepare release druid-0.11.0-sg

* [maven-release-plugin] prepare for next development iteration

* Address comments

* Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 5c6b31e488.

* Revert "[maven-release-plugin] prepare release druid-0.11.0-sg"

This reverts commit 0f5c3a8b82.

* Fix build failure

* Change list to array

* rename sortableIds

* Address comments

* change to foreach loop

* Fix comment

* Revert keyEquals()

* Remove loop

* Address comments

* Fix build fail

* Address comments

* Remove unused imports

* Fix method name

* Split intermediate and leaf combine degrees

* Add comments to StreamingMergeSortedGrouper

* Add more comments and fix overflow

* Address comments

* ConcurrentGrouperTest cleanup

* add thread number configuration for parallel combining

* improve doc

* address comments

* fix build
 2017-10-17 23:24:08 -07:00
Parag Jain 7cc18226cd add more tls configs to enable/disable specific cipher suites and protocols (#4902) 
* add more tls configs to enable/disable specific cipher suites and protocols

* fix doc, allow empty list
 2017-10-09 13:53:12 -07:00
Himanshu f69c9280c4 remove ServerConfig from DruidNode as all information needs to be present in DruidNode serialized form (#4858) 
* remove ServerConfig from DruidNode as all information needs to be present in DruidNode serialized form

* sanitize output of /druid/coordinator/v1/cluster endpoint
 2017-09-28 10:40:59 -05:00
Himanshu a36adc63e4 [documentation] add more jvm and os guidelines (#4793) 
* add more jvm and os guidelines

* address review comments

* add not so general guidelines too

* duplicate statement removal
 2017-09-20 13:12:57 -07:00
Parag Jain b5e839b3db injectable sslcontextfactory for jetty server and key manager factory algorithm (#4769) 
* injectable sslcontextfactory for jetty server

key manager factory algorithm

* explicitly set trustAll certificates to false in sslcontextfactory
 2017-09-12 11:45:03 -07:00
Jonathan Wei 1bddfc089c Additional docs/log for direct memory usage (#4631) 
* Additional docs/log for direct memory usage

* Tweak docs

* Doc rewording
 2017-08-10 23:33:20 -07:00
Himanshu ae6780f62a rolling upgrade order change to bring coordinator and overlord together (#4281) 
* rolling upgrade order change to bring coordinator and overlord together

* mentioned merged Coordinator-Overlord in upgrade order doc

* revert autoscaling doc change

* auto scaling doc fix
 2017-07-25 12:54:12 -05:00
Parag Jain 6e2f78f552 TLS support (#4270) 2017-07-06 17:40:12 -07:00
Parag Jain 4502c207af fix injection bug and documentation (#4243) 2017-05-03 15:07:43 -05:00
Jihoon Son 7411b18df9 Add BroadcastDistributionRule (#4077) 
* Add BroadcastDistributionRule

* Add missing null check

* Rename variable 'colocateDataSource' to 'colocatedDatasource'

* Address comments

* Document for broadcast rules

* Drop segments which are not co-located anymore

* Remove duplicated segment loading and dropping

* Add caveat

* address comments
 2017-05-01 09:55:17 -07:00
Akash Dwivedi 94da5e80f9 Namespace optimization for hdfs data segments. (#3877) 
* NN optimization for hdfs data segments.

* HdfsDataSegmentKiller, HdfsDataSegment finder changes to use new storage
format.Docs update.

* Common utility function in DataSegmentPusherUtil.

* new static method `makeSegmentOutputPathUptoVersionForHdfs` in JobHelper

* reuse getHdfsStorageDirUptoVersion in
DataSegmentPusherUtil.getHdfsStorageDir()

* Addressed comments.

* Review comments.

* HdfsDataSegmentKiller requested changes.

* extra newline

* Add maprfs.
 2017-03-01 09:51:20 -08:00
Nishant 35160e5595 Add metrics for Query Count statistics (#3470) 
* Add metrics for Query Count statistics

This PR adds a new metrics monitor “QueryCountStatsMonitor” which emits
three new metrics -
1) query/success/count - number of successful queries
2) query/failed/count - number of failed queries
3) query/interrupted/count - number of interrupted/timedout queries

fix bindings

* make fields final

* fix imports

* AsyncQueryForwardingServlet implement QueryStatsProvider

* remove unused import
 2016-12-19 09:47:58 -08:00
kaijianding 4be3eb0ce7 report message gap, source gap and sink count in RealtimePlumber (#3744) 
* report message gap, source gap and sink count in RealtimePlumber

* report message gap, sink count in Appenderator

* add ingest/events/sourceGap in metrics.md

* remove source gap
 2016-12-13 11:23:02 -06:00
Jonathan Wei 7c63bee7f5 Add mapreduce.job.classloader.system.classes property to 'Other Hadoop Versions' docs (#3706) 2016-11-18 16:16:50 -08:00
Himanshu b76b3f8d85 reset-cluster command to clean up druid state stored on metadata and deep storage (#3670) 2016-11-09 11:07:01 -06:00
jaehong choi 6f21778364 Support finding segments in AWS S3. (#3399) 
* support finding segments from a AWS S3 storage.

* add more Uts

* address comments and add a document for the feature.

* update docs indentation

* update docs indentation

* address comments.
1. add a Ut for json ser/deser for the config object.
2. more informant error message in a Ut.

* address comments.
1. use @Min to validate the configuration object
2. change updateDescriptor to a string as it does not take an argument otherwise

* fix a Ut failure - delete a Ut for testing default max length.
 2016-10-10 17:27:09 -07:00
Ashish 6b40bf8b32 doc: added note to README, about necessary hdfs config after insert-segment-to-db (#3402) 2016-08-28 16:39:33 -07:00
Chanh Le d624037698 Pull-deps: correct the library directory in the document (#3361) 
* Pull-deps: correct the library directory in the document

* Pull-deps: correct the library directory in the document in the last example command
 2016-08-16 17:18:15 -07:00
Fangjin Yang 6beb8ac342 fix some docs and add new content (#3369) 2016-08-16 15:00:18 -07:00
Himanshu ed5b92d612 document how to check MM enabled/disabled (#3331) 2016-08-06 05:56:51 +08:00
Gian Merlino e5397ed316 Link up Hadoop class loading docs better. (#3302) 2016-07-29 10:19:54 -07:00
Charles Allen 546e4f79b0 Add size of pending deletes to historical metrics (#3295) 
* Add size of pending deletes to historical metrics
 2016-07-27 11:30:47 -07:00
Charles Allen b1e3fe77f5 More logging around how the coordinator balancer is happening (#3279) 
* More logging around how the coordinator balancer is happening

* Address comments

* Address code review comments and add actual logging
 2016-07-27 13:24:32 +05:30
Gian Merlino dd4ec751d0 Update docs for working with Hadoop dependencies. (#3252) 
- Attempt to make things clearer in general
- Point out that HDFS deep storage and MR jobs don't use the same loading mechanism
- Recommend using mapreduce.job.classloader = true when possible
 2016-07-18 07:47:58 -05:00
Gian Merlino 6a03a0cfec Fix ingest/persist/backPressure docs. (#3243) 2016-07-13 21:56:28 -07:00
Gian Merlino b8a4f4ea7b DumpSegment: Add --dump bitmaps option. (#3221) 
Also make --dump metadata respect --column.
 2016-07-06 12:42:50 -07:00
Parag Jain 99844dfeb5 remove need for tmp extensions dir (#3211) 
correct lib path relative to base distribution dir
 2016-07-01 12:55:57 -07:00
michaelschiff 66d8ad36d7 adds new coordinator metrics 'segment/unavailable/count' and (#3176) 
'segment/underReplicated/count' (#3173)
 2016-06-23 14:53:15 -07:00
Gian Merlino da660bb592 DumpSegment tool. (#3182) 
Fixes #2723.
 2016-06-23 14:37:50 -07:00
Gian Merlino 3b3e772748 Add --no-default-remote-repositories flag to pull-deps. (#3120) 2016-06-13 17:01:18 +05:30
Kirill Kozlov 4ab675e863 Fix command name in example (#3088) 2016-06-07 10:44:27 -07:00
Gian Merlino cd5c5419bb Make docs deploying better. (#3040) 
- Make redirects for old links based on _redirects.json
- Replace #{DRUIDVERSION} tokens in docs with current version
- Allow origins named something other than "origin"
- Can use either s3cmd or awscli, depending on availability
 2016-05-31 15:34:58 -07:00
Fangjin Yang 00de26c76a fix extensions docs (#2995) 
* fix extensions docs

* fix mistakes
 2016-05-19 14:01:06 -07:00
Himanshu 6c5bf91f9a publish metrics numJettyConns to see how number of active jetty connections change over time (#2839) 
this can be compared with numer of active queries to see if requests are waiting in jetty queue
 2016-05-02 14:08:25 -07:00
du00cs 639d0630b8 jackson conflict workaround in hadooop ingestio & parquet extension coordinate update (#2817) 2016-04-13 14:20:33 -07:00
Sébastien Launay 37d2ab623e Merge pull request #2815 from slaunay/documentation/hadoop-classpath-issue-fix-with-configuration 
Doc for mapreduce.job.user.classpath.first=true
 2016-04-12 10:51:51 -07:00
fjy e3e932a4d4 refactor extensions into core and contrib 2016-03-08 17:12:09 -08:00
Bingkun Guo 18f9e05f0f improve doc on including druid and hadoop extensions 2016-02-26 13:53:08 -06:00
Fangjin Yang 083f019a48 Merge pull request #2465 from druid-io/more-doc-fix 
more doc fixes
 2016-02-17 11:00:38 -08:00
fjy 7da6594bfe more doc fixes 2016-02-17 09:43:47 -08:00
Slim e9f1c94822 Update metrics.md 2016-02-17 09:27:15 -06:00
Slim ebbb1aa74e Update metrics.md 2016-02-17 09:05:16 -06:00
Gian Merlino 95d5526e7c Freshen up rolling update docs 
1. Clarify what "Indexing Service / Realtime" means
2. Add info about restore-based middle manager rolling restarts
3. Add info about what happens in middle manager updates
4. More consistent capitalization and spelling of node types
 2016-02-09 13:57:04 -08:00
fjy 003f54e268 add doc rendering 2016-02-04 14:21:59 -08:00
fjy 1aa363cea7 new quickstart 2016-02-04 09:37:38 -08:00
Fangjin Yang 459c2a49ca Merge pull request #2364 from metamx/fix2356 
Add more docs around timezone handling
 2016-02-01 10:58:15 -08:00
Charles Allen c9393e5289 Add more docs around timezone handling 
* Fixes #2356
 2016-02-01 08:51:07 -08:00
Jaebin Yoon 66a74a2b88 Fixed the broken link 2016-02-01 01:07:24 -08:00
Robin c9368702fa do some editing of the instructions for using mysql for metadata 2016-01-21 10:37:30 -06:00