Commit Graph

9184 Commits

Author SHA1 Message Date
Justin Borromeo 7baeade832 Changes based on Gian's comments 2019-02-19 17:52:06 -08:00
Dylan Wylie 554b0142c3 Autoclose old PRs using stale bot. (#7031)
* Autoclose old PRs using stale bot.

* add apache license

* Excempt bug label
2019-02-19 14:26:54 -08:00
Fangyuan Deng 7d1e8f353e bugfix: when building materialized-view, if taskCount>1, may cause concurrentModificationException (#6690)
* bugfix: when building materialized-view, if taskCount >1, may cause ConcurrentModificationException

* remove entry after iteration instead of using ConcurrentMap, and add unit test

* small change

* modify unit test for coverage

* remove unused method
2019-02-19 13:10:55 -08:00
Jonathan Wei 258485a2fb
Exclude github issue templates from license check (#7070)
* Exclude github issue templates from license check

* Adjust capitalization
2019-02-19 12:38:52 -08:00
Surekha 2b04e6d0bc add note on consistency of results for sys.segments queries (#7034)
* add doc

* change docs

* PR comments

* few more changes
2019-02-19 10:52:37 -08:00
Clint Wylie cadb6c5280 Missing Overlord and MiddleManager api docs (#7042)
* document middle manager api

* re-arrange

* correction

* document more missing overlord api calls, minor re-arrange of some code i was referencing

* fix it

* this will fix it

* fixup

* link to other docs
2019-02-19 10:52:05 -08:00
Surekha 80a2ef7be4 Support kafka transactional topics (#5404) (#6496)
* Support kafka transactional topics

* update kafka to version 2.0.0
* Remove the skipOffsetGaps option since it's not used anymore
* Adjust kafka consumer to use transactional semantics
* Update tests

* Remove unused import from test

* Fix compilation

* Invoke transaction api to fix a unit test

* temporary modification of travis.yml for debugging

* another attempt to get travis tasklogs

* update kafka to 2.0.1 at all places

* Remove druid-kafka-eight dependency from integration-tests, remove the kafka firehose test and deprecate kafka-eight classes

* Add deprecated in docs for kafka-eight and kafka-simple extensions

* Remove skipOffsetGaps and code changes for transaction support

* Fix indentation

* remove skipOffsetGaps from kinesis

* Add transaction api to KafkaRecordSupplierTest

* Fix indent

* Fix test

* update kafka version to 2.1.0
2019-02-18 11:50:08 -08:00
Jonathan Wei 61272d6daa Update handlebars dep to patch vulnerability (#7083) 2019-02-18 18:06:47 +08:00
Justin Borromeo 35150fe1a6 Small changes 2019-02-15 15:57:53 -08:00
Justin Borromeo c7eeeabf45 2528 Replace Incremental Index Global Flags with Getters (#7043)
* Eliminated reportParseExceptions and deserializeComplexMetrics

* Removed more global flags

* Cleanup

* Addressed Surekha's recommendations
2019-02-15 13:36:46 -08:00
scrawfor 0fa9000849 Add Postgresql SqlFirehose (#6813)
* Add Postgresql SqlFirehose

* Fix Code Style.

* Fix style.

* Fix Import Order.

* Add Line Break before package.
2019-02-14 22:52:03 -08:00
awelsh93 ee91e27fe7 Update api-reference.md doc (#7065)
- moving description of coordinator isLeader endpoint
2019-02-14 14:38:09 +00:00
Jonathan Wei 1f29940811
Fix momentsketch build issues (#7074)
* Fix momentsketch build issues

* Remove unused section in pom

* Fix test

* Remove unused method

* Checkstyle
2019-02-13 21:32:43 -08:00
Edward Gan 90c1a54b86 Moments Sketch custom aggregator (#6581)
* Moments Sketch Integration with Druid

* updates, add documentation, fix warnings

* nits

* disallowed base64

* update to druid 0.14
2019-02-13 14:03:47 -08:00
Jonathan Wei 673396ae74 Add proposal template (#7062)
* Add proposal template

Adds a proposal template based on the discussion in https://lists.apache.org/thread.html/bb9c5e1f8ce9b3148a5c26f95059f9b6629fae3bf8c617121d671395@%3Cdev.druid.apache.org%3E

* Add license
2019-02-13 13:43:31 -08:00
Jihoon Son 970308463d
Add doc for Hadoop-based ingestion vs Native batch ingestion (#7044)
* Add doc for Hadoop-based ingestion vs Native batch ingestion

* add links

* add links
2019-02-13 11:23:08 -08:00
Jihoon Son 1701fbcad3
Improve error message for revoked locks (#7035)
* Improve error message for revoked locks

* fix test

* fix test

* fix test

* fix toString
2019-02-13 11:22:48 -08:00
Jihoon Son b1c4a5de0d
Fix and improve doc for partitioning of local index (#7064) 2019-02-13 11:20:52 -08:00
Mingming Qiu d0abf5c20a fix kafka index task doesn't resume when recieve duplicate request (#6990)
* fix kafka index task doesn't resume when recieve duplicate request

* add unit test
2019-02-12 13:24:28 -08:00
Jonathan Wei 8ba11591b6 Add router conf to assembly.xml (#7051) 2019-02-12 10:33:18 +08:00
Surekha 02ef14f262 Fix num_rows in sys.segments (#6888)
* Fix the bug with num_rows in sys.segments

* Fix segmentMetadataInfo update in DruidSchema
* Add numRows to SegmentMetadataHolder builder's constructor, so it's not overwritten
* Rename SegSegmentSignature to setSegmentMetadataHolder and fix it so nested map is appended instead of recreated
* Replace Map<String, Set<String>> segmentServerMap with Set<String> for num_replica

* Remove unnecessary code and update test

* Add unit test for num_rows

* PR comments

* change access modifier to default package level

* minor changes to comments

* PR comments
2019-02-11 16:21:19 -08:00
Justin Borromeo 4e69276d57 Removed unused import to satisfy PMD check 2019-02-11 12:09:54 -08:00
Justin Borromeo ecb0f483a9 improved doc 2019-02-11 10:37:11 -08:00
Justin Borromeo f0eddee665 Added more javadoc 2019-02-11 10:18:45 -08:00
Justin Borromeo 5f92dd7325 Unused import 2019-02-11 10:05:58 -08:00
Justin Borromeo 93e1636287 Added javadoc on ScanResultValueTimestampComparator 2019-02-11 10:03:14 -08:00
Ankit Kothari 16a4a50e91 [Issue #6967] NoClassDefFoundError when using druid-hdfs-storage (#7015)
* Fix:
  1. hadoop-common dependency for druid-hdfs and druid-kerberos extensions
 Refactoring:
  2. Hadoop config call in the inner static class to avoid class path conflicts for stopGracefully kill

* Fix:
  1. hadoop-common test dependency

* Fix:
  1. Avoid issue of kill command once the job is actually completed
2019-02-08 18:26:37 -08:00
Justin Borromeo 134041c479 Renamed sort function 2019-02-08 13:13:54 -08:00
Jihoon Son d42de574d6 Add an api to get all lookup specs (#7025)
* Add an api to get all lookup specs

* add doc
2019-02-08 11:05:59 -08:00
Jihoon Son c9f21bc782 Fix filterSegments for TimeBoundary and DataSourceMetadata queries (#7023)
* Fix filterSegments for TimeBoundary and DataSourceMetadata queries

* add javadoc

* fix build
2019-02-08 10:03:02 -08:00
Don Bowman b3dcbe70ad Add docker container for druid (#6896)
* Add docker container for druid

This container is an 'omnibus' (since there is such a high
overlap with the various services). It includes all contrib
extension as well as the msql connector.

It is intended to be run as `docker run NAME SERVICE`
(e.g. docker run druid:latest broker)

* Add Apache license header

* Resolve issues from Pull Request review

* Add comments at top of script per PR comments

* Revert BUILDKIT. Not available everywhere.

* Don't set hostname, allow default (IP)

Some environments (e.g. Kubernetes Deployments) don't resolve
hostname to IP.

* Switch to amd64 glibc-based busybox from 32-bit uclibc

* Override service-specific configuration

* Replace MAINTAINER w/ LABEL

* Add mysql connector to application classpath

This works around issue #3770
https://github.com/apache/incubator-druid/issues/3770

* Add docker-compose and sample environment

Signed-off-by: Don Bowman <don@agilicus.com>
2019-02-08 12:12:28 +00:00
Jonathan Wei fafbc4a80e
Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
Justin Borromeo 2e3577cd3d Fixed benchmark queries 2019-02-07 13:01:25 -08:00
Furkan KAMACI 3097562adf Improper getter value is fixed. (#6930)
* Improper getter value is fixed.

* Test class is added.
2019-02-07 11:51:07 -08:00
Justin Borromeo d3b335af42 added all query types to scan benchmark 2019-02-07 11:08:07 -08:00
Justin Borromeo ab00eade9f Kicking travis with change to benchmark param 2019-02-07 09:42:48 -08:00
Justin Borromeo b432beaf84 Fixed failing calcite tests 2019-02-06 17:45:59 -08:00
Justin Borromeo b2c8c77ad4 Fixing tests WIP 2019-02-06 17:39:48 -08:00
Justin Borromeo 85e72a614e Set to spaces over tabs 2019-02-06 15:42:02 -08:00
Justin Borromeo 7e872a8ebc Created an error message for when someone tries to time order a result
set > threshold limit
2019-02-06 15:36:24 -08:00
Justin Borromeo e8a4b49044 Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 15:05:11 -08:00
Justin Borromeo 305876a434 nit 2019-02-06 15:02:02 -08:00
Jihoon Son 8e3a58f723
Improve druid.storage.sse.kms.keyId and druid.s3.protocol (#7012)
* Improve druid.storage.sse.kms.keyId and druid.s3.protocol

* fix article
2019-02-06 15:00:51 -08:00
Justin Borromeo 6723243ed2 Create Scan Benchmark (#6986)
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Remove todos

* Change number of benchmark iterations

* Changed benchmark params

* More param changes

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* Decrease segment size for less memory usage

* Committing a param change to kick teamcity
2019-02-06 14:45:01 -08:00
Justin Borromeo 8212a21caf Improved conciseness 2019-02-06 14:40:35 -08:00
Furkan KAMACI 58f9507ccf Improper equals override is fixed to prevent NullPointerException (#6938)
* Improper equals override is fixed to prevent NullPointerException

* Fixed curly brace indentation.

* Test method is added for equals method of TaskLockPosse class.
2019-02-06 14:08:50 -08:00
Justin Borromeo 10b5e0ca93 . 2019-02-06 13:42:12 -08:00
Justin Borromeo dfe4aa9681 Fixed codestyle and forbidden API errors 2019-02-06 13:41:18 -08:00
Justin Borromeo 148939e88b Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 13:26:17 -08:00
Justin Borromeo 5edbe2ae12 Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark 2019-02-06 13:18:55 -08:00