da4836f38c
Added titles and harmonized docs to improve usability and SEO ( #6731 )
...
* added titles and harmonized docs
* manually fixed some titles
2018-12-12 20:42:12 -08:00
607339003b
Add TaskCountStatsMonitor to monitor task count stats ( #6657 )
...
* Add TaskCountStatsMonitor to monitor task count stats
* address comments
* add file header
* tweak test
2018-12-04 13:37:17 -08:00
e2bedab665
fix links to use relative references ( #6696 )
2018-11-30 16:32:10 -08:00
b332021c49
remove extensions from default configs that have configuration/library dependencies and update docs ( #6694 )
2018-11-30 12:52:46 -08:00
d6539abd0a
Fix overlord api and console ( #6686 )
...
* Fix overlord APIs and console
* remove getRunningTasksByDataSource
* add missing path to isApplicable
2018-11-29 23:45:28 -08:00
c5405bb592
emit maxLag/avgLag in KafkaSupervisor ( #6587 )
...
* emit maxLag/totalLag/avgLag in KafkaSupervisor
* modify ingest/kafka/totalLag to ingest/kafka/lag for backwards compatibility
2018-11-28 02:11:14 -08:00
cdae2fe7b5
Deprecate IntervalChunkingQueryRunner ( #6591 )
...
* Deprecate IntervalChunkingQueryRunner
* add doc
* deprecate metric
* fix doc
2018-11-14 06:33:27 +08:00
7b262b7123
Remove unnecessary path param from auto compaction api ( #6594 )
...
* Remove unnecessary path param from auto compaction api
* fix ci
2018-11-13 09:46:13 -08:00
afb239b17a
add missing license headers, in particular to MD files; clean up RAT … ( #6563 )
...
* add missing license headers, in particular to MD files; clean up RAT exclusions
* revert inadvertent doc changes
* docs
* cr changes
* fix modified druid-production.svg
2018-11-13 09:38:37 -08:00
1224d8b746
overhaul 'druid-parquet-extensions' module, promoting from 'contrib' to 'core' ( #6360 )
...
* move parquet-extensions from contrib to core, adds new hadoop parquet parser that does not convert to avro first and supports flattenSpec and int96 columns, add support for flattenSpec for parquet-avro conversion parser, much test with a bunch of files lifted from spark-sql
* fix avro flattener to support nullable primitives for auto discovery and now only supports primitive arrays instead of all arrays
* remove leftover print
* convert micro timestamp to millis
* checkstyle
* add ignore for .parquet and .parq to rat exclude
* fix legit test failure from avro flattern behavior change
* fix rebase
* add exclusions to pom to cut down on redundant jars
* refactor tests, add support for unwrapping lists for parquet-avro, review comments
* more comment
* fix oops
* tweak parquet-avro list handling
* more docs
* fix style
* grr styles
2018-11-05 21:33:42 -08:00
a92c2a197b
Move supervisor APIs to api-reference ( #6555 )
...
* Move supervisor APIs to api-reference
* fix kafka-specific docs
* add ingestion stats report
2018-11-01 13:10:05 -07:00
7b34662462
Period load/drop/broadcast rules should include the future by default ( #6414 )
...
* Period load/drop/broadcast rules should include the future by default
* address comments
* adjust coordinator console and tweak docs
* address comments
* fix travis-ci
2018-11-01 09:43:34 -07:00
b2d9b6f23d
Allow custom TLS cert checks ( #6432 )
...
* Allow custom TLS cert checks
* PR comment
* Checkstyle, PR comment
2018-10-24 16:31:52 -07:00
601183b4c7
Add period drop before rule ( #6415 )
...
* Add period drop before rule
* add license header
* support period drop before rule in coordinator console
* address comments
2018-10-24 12:44:30 -07:00
f5f4171a45
QueryCountStatsMonitor: emit query/count ( #6473 )
...
Let `QueryCountStatsMonitor` emit `query/count`, then I can monitor QPS of my services, or I have to count it by myself.
2018-10-19 10:15:02 -03:00
c5872bef41
Improve GC metrics documentation ( #6423 )
2018-10-05 14:57:01 -07:00
a56ffe6ab2
Added backpressure metric to docs and defaultMetricDimensions ( #6405 )
...
* Added backpressure metric to docs and defaultMetricDimensions.json
* Reworded description for backpressure metric in docs
2018-09-29 17:57:29 -07:00
6fb503c073
Deprecate task audit logging ( #6368 )
...
* Deprecate task audit logging
* fix test
* fix it test
2018-09-26 16:28:02 -07:00
034d006d24
add to docs on including Drop rule with Load rule ( #6378 )
2018-09-25 20:13:52 -07:00
99428e20d2
Deprecate dimensions / metrics APIs on brokers ( #6361 )
...
* Deprecate dimensions / metrics APIs on brokers
* add segmentMetadataQuery link
* add more doc
2018-09-24 17:56:38 -07:00
ee7b565469
Docs for ingestion stat reports and new parse exception handling ( #6373 )
2018-09-24 17:45:05 -07:00
8972244c68
Mutual TLS support ( #6076 )
...
* Mutual TLS support
* Kafka test fixes
* TeamCity fix
* Split integration tests
* Use localhost DOCKER_IP
* Increase server thread count
* Increase SSL handshake timeouts
* Add broken pipe retries, use injected client config params
* PR comments, Rat license check exclusion
2018-09-19 09:56:15 -07:00
4caa61d8fa
Fix tutorial sample data filename, fix logger classname in metrics docs ( #6299 )
2018-09-04 21:47:12 -07:00
180e3ccfad
Docs consistency cleanup ( #6259 )
2018-09-04 12:54:41 -07:00
9b04846e6b
correct metric name in doc file ( #6271 )
2018-08-30 10:57:35 -07:00
431d3d8497
Rename io.druid to org.apache.druid. ( #6266 )
...
* Rename io.druid to org.apache.druid.
* Fix META-INF files and remove some benchmark results.
* MonitorsConfig update for metrics package migration.
* Reorder some dimensions in inner queries for some reason.
* Fix protobuf tests.
2018-08-30 09:56:26 -07:00
b0cf8d0252
'shutdownAllTasks' API for a dataSource ( #6185 )
...
* 'shutdownAllTasks' API for a dataSource
Change-Id: I30d14390457d39e0427d23a48f4f224223dc5777
* fix api path and return
Change-Id: Ib463f31ee2c4cb168cf2697f149be845b57c42e5
* optimize implementation
Change-Id: I50a8dcd44dd9d36c9ecbfa78e103eb9bff32eab9
2018-08-17 12:57:09 -04:00
2b0f03acb9
Unified API doc page ( #6128 )
...
* Unified API doc page
* PR comments
* Fix metadata endpoint
2018-08-09 14:27:42 -06:00
3525d4059e
Cache: Add maxEntrySize config, make groupBy cacheable by default. ( #5108 )
...
* Cache: Add maxEntrySize config.
The idea is this makes it more feasible to cache query types that
can potentially generate large result sets, like groupBy and select,
without fear of writing too much to the cache per query.
Includes a refactor of cache population code in CachingQueryRunner and
CachingClusteredClient, such that they now use the same CachePopulator
interface with two implementations: one for foreground and one for
background.
The main reason for splitting the foreground / background impls is
that the foreground impl can have a more effective implementation of
maxEntrySize. It can stop retaining subvalues for the cache early.
* Add CachePopulatorStats.
* Fix whitespace.
* Fix docs.
* Fix various tests.
* Add tests.
* Fix tests.
* Better tests
* Remove conflict markers.
* Fix licenses.
2018-08-07 10:23:15 -07:00
96feb479cd
add order change needed for KIS in 0.12.0 ( #5760 )
2018-06-08 15:25:26 -07:00
cfa94b747b
Update to jetty 9.4; Enable request decompression ( #5624 )
...
* Update to jetty 9.4; Enable request decompression; Add http compression config options
* Fix BadMessageException from jetty server at HttpGenerator.generateHeaders(...)
2018-06-08 14:53:08 -07:00
67d0b0ee42
Add taskType dimension to task metrics ( #5664 )
2018-05-07 09:42:26 -07:00
c12c16385e
support throw duplcate row during realtime ingestion in RealtimePlumber ( #5693 )
2018-05-04 10:12:25 -07:00
e2431ae161
Update defaultHadoopCoordinates in documentation. ( #5720 )
...
* Update defaultHadoopCoordinates in documentation.
To match changes applied in #5382 .
* Remove a parameter with defaults from example configuration file.
If it has reasonable defaults, then why would it be in an example config file?
Also, it is yet another place that has been forgotten to be updated and will be forgotten in the future.
Also, if someone is running different hadoop version, then there's much more work to be done than just changing this property, so why give users false hopes?
* Fix typo in documentation.
2018-04-30 20:49:14 -07:00
8ec2d2fe18
Use unique segment paths for Kafka indexing ( #5692 )
...
* support unique segment file paths
* forbiddenapis
* code review changes
* code review changes
* code review changes
* checkstyle fix
2018-04-29 21:59:48 -07:00
762f8829e4
Add task action metrics, add taskId metric dimension. ( #5714 )
...
* Add task action metrics, add taskId metric dimension.
Adds two new metrics: task/action/log/time and task/action/run/time. Also
adds taskId as a dimension, to give us the ability to drill down into metrics
for an individual task. Also standardizes metrics-attachment using two helper
methods in IndexTaskUtils.
* Fix typo
2018-04-29 21:24:06 -07:00
b084075279
Add http/https proxy options to PullDependencies.java ( #5450 )
2018-03-07 15:05:43 -08:00
80419752b5
Add metamx emitter, http clients, and metrics packages to druid java-util ( #5289 )
...
* Add metamx java-util emitter, http clients, and metrics packages to druid java-util
* Remove metamx java-util from pom.xml files
* Checkstyle fixes
* Import fix
* TeamCity inspection fixes
* Use slf4j, move some version defs to master pom.xml
* Use parent jvm-attach-api and maven-surefire-plugin versions
* Add ] to log msg, suppress inspection
2018-01-24 22:10:36 +01:00
241efafbb2
Automatic compaction by coordinators ( #5102 )
...
* Automatic compaction by coordinator
* add links
* skip compaction for very recent segments if they are small
* fix finding search interval
* fix finding search interval
* fix TimelineHolder iteration
* add test for newestSegmentFirstPolicy
* add CompactionSegmentIterator
* add numTargetCompactionSegments
* add missing config
* fix skipping huge shards
* fix handling large number of segments per shard
* fix test failure
* change recursive call to loop
* fix logging
* fix build
* fix test failure
* address comments
* change dataSources type
* check running pendingTasks at each run
* fix test
* address comments
* fix build
* fix test
* address comments
* address comments
* add doc for segment size optimization
* address comment
2018-01-13 13:52:37 +09:00
2ecebb3173
Fix coordinator/overlord redirects when TLS is enabled ( #5037 )
...
* Fix coordinator/overlord redirects when TLS is enabled
* address review comment
* fix UTs
* workaround to not ignore URL instance to fix the teamcity build
* update tls doc
2017-11-09 13:10:28 -08:00
52d7f74226
Add streaming aggregation as the last step of ConcurrentGrouper if data are spilled ( #4704 )
...
* Add steaming grouper
* Fix doc
* Use a single dictionary while combining
* Revert GroupByBenchmark
* Removed unused code
* More cleanup
* Remove unused config
* Fix some typos and bugs
* Refactor Groupers.mergeIterators()
* Add comments for combining tree
* Refactor buildCombineTree
* Refactor iterator
* Add ParallelCombiner
* Add ParallelCombinerTest
* Handle InterruptedException
* use AbstractPrioritizedCallable
* Address comments
* [maven-release-plugin] prepare release druid-0.11.0-sg
* [maven-release-plugin] prepare for next development iteration
* Address comments
* Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 5c6b31e4880f5c3a8b82
2017-10-17 23:24:08 -07:00
7cc18226cd
add more tls configs to enable/disable specific cipher suites and protocols ( #4902 )
...
* add more tls configs to enable/disable specific cipher suites and protocols
* fix doc, allow empty list
2017-10-09 13:53:12 -07:00
f69c9280c4
remove ServerConfig from DruidNode as all information needs to be present in DruidNode serialized form ( #4858 )
...
* remove ServerConfig from DruidNode as all information needs to be present in DruidNode serialized form
* sanitize output of /druid/coordinator/v1/cluster endpoint
2017-09-28 10:40:59 -05:00
a36adc63e4
[documentation] add more jvm and os guidelines ( #4793 )
...
* add more jvm and os guidelines
* address review comments
* add not so general guidelines too
* duplicate statement removal
2017-09-20 13:12:57 -07:00
b5e839b3db
injectable sslcontextfactory for jetty server and key manager factory algorithm ( #4769 )
...
* injectable sslcontextfactory for jetty server
key manager factory algorithm
* explicitly set trustAll certificates to false in sslcontextfactory
2017-09-12 11:45:03 -07:00
1bddfc089c
Additional docs/log for direct memory usage ( #4631 )
...
* Additional docs/log for direct memory usage
* Tweak docs
* Doc rewording
2017-08-10 23:33:20 -07:00
ae6780f62a
rolling upgrade order change to bring coordinator and overlord together ( #4281 )
...
* rolling upgrade order change to bring coordinator and overlord together
* mentioned merged Coordinator-Overlord in upgrade order doc
* revert autoscaling doc change
* auto scaling doc fix
2017-07-25 12:54:12 -05:00
6e2f78f552
TLS support ( #4270 )
2017-07-06 17:40:12 -07:00
4502c207af
fix injection bug and documentation ( #4243 )
2017-05-03 15:07:43 -05:00
7411b18df9
Add BroadcastDistributionRule ( #4077 )
...
* Add BroadcastDistributionRule
* Add missing null check
* Rename variable 'colocateDataSource' to 'colocatedDatasource'
* Address comments
* Document for broadcast rules
* Drop segments which are not co-located anymore
* Remove duplicated segment loading and dropping
* Add caveat
* address comments
2017-05-01 09:55:17 -07:00
Vadim Ogievetsky
Mingming Qiu
David Lim
Jihoon Son
Clint Wylie
Jonathan Wei
Roman Leventov
Shiv Toolsidass
Caroline1000
Gian Merlino
Hongze Zhang
Kirill Kozlov
kaijianding
Jakub Kukul
David Lim
Himanshu
Parag Jain
Parag Jain