Commit Graph

9174 Commits

Author SHA1 Message Date
scrawfor 0fa9000849 Add Postgresql SqlFirehose (#6813)
* Add Postgresql SqlFirehose

* Fix Code Style.

* Fix style.

* Fix Import Order.

* Add Line Break before package.
2019-02-14 22:52:03 -08:00
awelsh93 ee91e27fe7 Update api-reference.md doc (#7065)
- moving description of coordinator isLeader endpoint
2019-02-14 14:38:09 +00:00
Jonathan Wei 1f29940811
Fix momentsketch build issues (#7074)
* Fix momentsketch build issues

* Remove unused section in pom

* Fix test

* Remove unused method

* Checkstyle
2019-02-13 21:32:43 -08:00
Edward Gan 90c1a54b86 Moments Sketch custom aggregator (#6581)
* Moments Sketch Integration with Druid

* updates, add documentation, fix warnings

* nits

* disallowed base64

* update to druid 0.14
2019-02-13 14:03:47 -08:00
Jonathan Wei 673396ae74 Add proposal template (#7062)
* Add proposal template

Adds a proposal template based on the discussion in https://lists.apache.org/thread.html/bb9c5e1f8ce9b3148a5c26f95059f9b6629fae3bf8c617121d671395@%3Cdev.druid.apache.org%3E

* Add license
2019-02-13 13:43:31 -08:00
Jihoon Son 970308463d
Add doc for Hadoop-based ingestion vs Native batch ingestion (#7044)
* Add doc for Hadoop-based ingestion vs Native batch ingestion

* add links

* add links
2019-02-13 11:23:08 -08:00
Jihoon Son 1701fbcad3
Improve error message for revoked locks (#7035)
* Improve error message for revoked locks

* fix test

* fix test

* fix test

* fix toString
2019-02-13 11:22:48 -08:00
Jihoon Son b1c4a5de0d
Fix and improve doc for partitioning of local index (#7064) 2019-02-13 11:20:52 -08:00
Mingming Qiu d0abf5c20a fix kafka index task doesn't resume when recieve duplicate request (#6990)
* fix kafka index task doesn't resume when recieve duplicate request

* add unit test
2019-02-12 13:24:28 -08:00
Jonathan Wei 8ba11591b6 Add router conf to assembly.xml (#7051) 2019-02-12 10:33:18 +08:00
Surekha 02ef14f262 Fix num_rows in sys.segments (#6888)
* Fix the bug with num_rows in sys.segments

* Fix segmentMetadataInfo update in DruidSchema
* Add numRows to SegmentMetadataHolder builder's constructor, so it's not overwritten
* Rename SegSegmentSignature to setSegmentMetadataHolder and fix it so nested map is appended instead of recreated
* Replace Map<String, Set<String>> segmentServerMap with Set<String> for num_replica

* Remove unnecessary code and update test

* Add unit test for num_rows

* PR comments

* change access modifier to default package level

* minor changes to comments

* PR comments
2019-02-11 16:21:19 -08:00
Justin Borromeo 4e69276d57 Removed unused import to satisfy PMD check 2019-02-11 12:09:54 -08:00
Justin Borromeo ecb0f483a9 improved doc 2019-02-11 10:37:11 -08:00
Justin Borromeo f0eddee665 Added more javadoc 2019-02-11 10:18:45 -08:00
Justin Borromeo 5f92dd7325 Unused import 2019-02-11 10:05:58 -08:00
Justin Borromeo 93e1636287 Added javadoc on ScanResultValueTimestampComparator 2019-02-11 10:03:14 -08:00
Ankit Kothari 16a4a50e91 [Issue #6967] NoClassDefFoundError when using druid-hdfs-storage (#7015)
* Fix:
  1. hadoop-common dependency for druid-hdfs and druid-kerberos extensions
 Refactoring:
  2. Hadoop config call in the inner static class to avoid class path conflicts for stopGracefully kill

* Fix:
  1. hadoop-common test dependency

* Fix:
  1. Avoid issue of kill command once the job is actually completed
2019-02-08 18:26:37 -08:00
Justin Borromeo 134041c479 Renamed sort function 2019-02-08 13:13:54 -08:00
Jihoon Son d42de574d6 Add an api to get all lookup specs (#7025)
* Add an api to get all lookup specs

* add doc
2019-02-08 11:05:59 -08:00
Jihoon Son c9f21bc782 Fix filterSegments for TimeBoundary and DataSourceMetadata queries (#7023)
* Fix filterSegments for TimeBoundary and DataSourceMetadata queries

* add javadoc

* fix build
2019-02-08 10:03:02 -08:00
Don Bowman b3dcbe70ad Add docker container for druid (#6896)
* Add docker container for druid

This container is an 'omnibus' (since there is such a high
overlap with the various services). It includes all contrib
extension as well as the msql connector.

It is intended to be run as `docker run NAME SERVICE`
(e.g. docker run druid:latest broker)

* Add Apache license header

* Resolve issues from Pull Request review

* Add comments at top of script per PR comments

* Revert BUILDKIT. Not available everywhere.

* Don't set hostname, allow default (IP)

Some environments (e.g. Kubernetes Deployments) don't resolve
hostname to IP.

* Switch to amd64 glibc-based busybox from 32-bit uclibc

* Override service-specific configuration

* Replace MAINTAINER w/ LABEL

* Add mysql connector to application classpath

This works around issue #3770
https://github.com/apache/incubator-druid/issues/3770

* Add docker-compose and sample environment

Signed-off-by: Don Bowman <don@agilicus.com>
2019-02-08 12:12:28 +00:00
Jonathan Wei fafbc4a80e
Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
Justin Borromeo 2e3577cd3d Fixed benchmark queries 2019-02-07 13:01:25 -08:00
Furkan KAMACI 3097562adf Improper getter value is fixed. (#6930)
* Improper getter value is fixed.

* Test class is added.
2019-02-07 11:51:07 -08:00
Justin Borromeo d3b335af42 added all query types to scan benchmark 2019-02-07 11:08:07 -08:00
Justin Borromeo ab00eade9f Kicking travis with change to benchmark param 2019-02-07 09:42:48 -08:00
Justin Borromeo b432beaf84 Fixed failing calcite tests 2019-02-06 17:45:59 -08:00
Justin Borromeo b2c8c77ad4 Fixing tests WIP 2019-02-06 17:39:48 -08:00
Justin Borromeo 85e72a614e Set to spaces over tabs 2019-02-06 15:42:02 -08:00
Justin Borromeo 7e872a8ebc Created an error message for when someone tries to time order a result
set > threshold limit
2019-02-06 15:36:24 -08:00
Justin Borromeo e8a4b49044 Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 15:05:11 -08:00
Justin Borromeo 305876a434 nit 2019-02-06 15:02:02 -08:00
Jihoon Son 8e3a58f723
Improve druid.storage.sse.kms.keyId and druid.s3.protocol (#7012)
* Improve druid.storage.sse.kms.keyId and druid.s3.protocol

* fix article
2019-02-06 15:00:51 -08:00
Justin Borromeo 6723243ed2 Create Scan Benchmark (#6986)
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Remove todos

* Change number of benchmark iterations

* Changed benchmark params

* More param changes

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* Decrease segment size for less memory usage

* Committing a param change to kick teamcity
2019-02-06 14:45:01 -08:00
Justin Borromeo 8212a21caf Improved conciseness 2019-02-06 14:40:35 -08:00
Furkan KAMACI 58f9507ccf Improper equals override is fixed to prevent NullPointerException (#6938)
* Improper equals override is fixed to prevent NullPointerException

* Fixed curly brace indentation.

* Test method is added for equals method of TaskLockPosse class.
2019-02-06 14:08:50 -08:00
Justin Borromeo 10b5e0ca93 . 2019-02-06 13:42:12 -08:00
Justin Borromeo dfe4aa9681 Fixed codestyle and forbidden API errors 2019-02-06 13:41:18 -08:00
Justin Borromeo 148939e88b Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 13:26:17 -08:00
Justin Borromeo 5edbe2ae12 Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark 2019-02-06 13:18:55 -08:00
Justin Borromeo 60b7684db7 Committing a param change to kick teamcity 2019-02-06 13:02:13 -08:00
Justin Borromeo 4f51024b31 Wrote more tests for scan result value sort 2019-02-06 12:08:12 -08:00
Justin Borromeo 8b7d5f5081 Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order
2019-02-06 11:55:09 -08:00
Justin Borromeo b6d4df3864 Decrease segment size for less memory usage 2019-02-05 16:45:20 -08:00
anantmf 315ccb76b8 Fix for getSingleObjectSummary, replacing keyCount with objectSummaries().size (#7000)
* Instead of using keyCount, changing it to check the size of objectSummaries.

For issue:
https://github.com/apache/incubator-druid/issues/6980
https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580

* Changing another usage of keyCount with size of objectSummaries.

* Adding some comments to explain why using keyCount is not working as expected.
2019-02-05 15:45:44 -08:00
Surekha ef451d3603 Add null checks in DruidSchema (#6830)
* Add null checks in DruidSchema

* Add unit tests

* Add VisibleForTesting annotation

* PR comments

* unused import
2019-02-05 13:42:20 -08:00
Justin Borromeo d1a1793f36 nit 2019-02-05 12:40:26 -08:00
Jihoon Son 75c70c2ccc Add doc for S3 permissions settings (#7011)
* Add doc for S3 permissions settings

* add a comment about additional settings
2019-02-05 11:52:09 -08:00
Justin Borromeo 7deb06f6df Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-05 10:53:38 -08:00
Justin Borromeo 86c5eee13b Broke some long lines into two lines 2019-02-05 10:31:27 -08:00