9570 Commits

Author SHA1 Message Date
Gian Merlino
b6941551ae Upgrade various build and doc links to https. (#7722)
* Upgrade various build and doc links to https.

Where it wasn't possible to upgrade build-time dependencies to https,
I kept http in place but used hardcoded checksums or GPG keys to ensure
that artifacts fetched over http are verified properly.

* Switch to https://apache.org.
2019-05-21 11:30:14 -07:00
Xue Yu
dd7dace70a Add TIMESTAMPDIFF sql support (#7695)
* add timestampdiff sql support

* feedback address
2019-05-21 08:05:38 -07:00
Vadim Ogievetsky
156322932f Update Druid Console docs for 0.15.0 (#7697)
* Update Druid Console docs for 0.15.0

* SQL -> query

* added links and fix typos
2019-05-21 04:00:42 -07:00
Vadim Ogievetsky
169d2493bc Web console: fix missing value input in timestampSpec step (#7698) 2019-05-21 03:59:48 -07:00
Gian Merlino
cb6ec2cab8 SqlOperatorConversion Javadoc fix. (#7713)
Appears to be a copypasta error; the toDruidFilter method was referred
to aggregations, but it's not handling aggregations.
2019-05-20 21:21:21 -07:00
Jihoon Son
d69aa6f7f6
Fix case insensitive of ParserUtils.findDuplicates (#7692)
* Fix case insensitive ParserUtils.findDuplicates

* unused import
2019-05-20 17:49:15 -07:00
Himanshu
fb0c846941
Virtual column updates for exploiting base column internal structure (#7618)
* VirtualColumn updates for exploiting base column internal structure

* unit tests for virtual column interface updates

* groupBy needs to use VirtualizedColumnSelectorFactory if outer query in
nested groupBy has virtual columns.

* fix strict compile checks

* fix teamcity build errors

* add comment explaining useVirtualizedColumnSelectorFactory flag in RowBasedGrouperHelper.createGrouperAccumulatorPair(..)
2019-05-20 17:04:35 -07:00
Clint Wylie
c4a4223c9c fix issue where result level cache was recomputing post aggs that were already cached, causing issues with finalizing aggregators (#7708) 2019-05-20 16:51:50 -07:00
Clint Wylie
ddda8b74cb update lz4-java to 1.6.0 (lz4 1.9.1) (#7700) 2019-05-20 13:01:48 -07:00
Merlin Lee
5f08b0b474 Add checkstyle for "Prohibit @author tags in Javadoc" (#7682)
* Add checkstyle for "Prohibit @author tags in Javadoc"

* Add "Do not use author tags/information in the code" back to CONTRIBUTING.md
2019-05-20 00:09:51 -07:00
andrewluotechnologies
1add566411 Fix typo (ComplexMetricSerde class name was spelled incorrectly) (#7694) 2019-05-19 09:49:54 -07:00
Vadim Ogievetsky
be16e4a4ae Web console, adding Apache Kafka and AWS Kinesis to the data loader (#7643)
* adding kafka and kinesis to the data loader

* feature detect

* copy fixes

* wording fixes

* added missing spec type

* increase timeout

* Call it Google Cloud Storage
2019-05-17 14:01:27 -07:00
Jihoon Son
94721de141 Add auto tagging milestone script (#7677)
* Add auto tagging milestone script

* fix usage

* missing newline

* missing newline
2019-05-16 23:11:16 -07:00
Clint Wylie
939b417379 Update tutorial-kafka.md (#7678) 2019-05-16 23:10:45 -07:00
David Lim
d38457933f Data loader (sampler component) - Kafka/Kinesis samplers (#7566)
* implement Kafka/Kinesis sampler

* add KafkaSamplerSpecTest and KinesisSamplerSpecTest

* code review changes
2019-05-16 20:26:23 -07:00
Jenny Zhu
ec0b7787cf Web console: fixed issue when grouping tasks by different attributes (#7657)
* deleted created_time in "status"

* added toString on the status object
2019-05-16 16:46:04 -07:00
Jonathan Wei
d99f77a01b
Add option to use YARN RM as fallback for JobHistory failure (#7673)
* Add option to use YARN RM as fallback for job status

* PR comments
2019-05-16 13:59:10 -07:00
Fangjin Yang
dc85a5309e
some more doc improvements (#7675) 2019-05-16 13:17:21 -07:00
Jonathan Wei
d667655871 Add basic tuning guide, getting started page, updated clustering docs (#7629)
* Add basic tuning guide, getting started page, updated clustering docs

* Add note about caching, fix tutorial paths

* Adjust hadoop wording

* Add license

* Tweak

* Shrink overlord heaps, fix tutorial urls

* Tweak xlarge peon, update peon sizing

* Update Data peon buffer size

* Fix cluster start scripts

* Add upper level _common to classpath

* Fix cluster data/query confs

* Address PR comments

* Elaborate on connection pools

* PR comments

* Increase druid.broker.http.maxQueuedBytes

* Add guidelines for broker backpressure

* PR comments
2019-05-16 11:13:48 -07:00
Himanshu
8687f424f9
make ComplexColumn an interface and ExtensionPoint (#7633)
* make ComplexColumn an interface and ExtensionPoint

* incorporate review comments

* make ColumnValueSelector @ExtensionPoint

* more java docs

* add close() method to ComplexColumn interface
2019-05-15 20:59:55 -07:00
Benedict Jin
3df364c472 Fix broken links in api-reference.md (#7670) 2019-05-15 18:53:34 -07:00
Clint Wylie
c2abbc24a7 minor web console doc fixes (#7668) 2019-05-15 18:52:51 -07:00
Jonathan Wei
7d63c295cc Fix compilation error in CoordinatorBasicAuthorizerResourceTest (#7667)
* Fix compilation error in CoordinatorBasicAuthorizerResourceTest

* Don't use simplifyPermissions
2019-05-15 17:47:38 -07:00
Surekha
d3545f5086 Show all server types in sys.servers table (#7654)
* update sys.servers table to show all servers

* update docs

* Fix integration test

* modify test query for batch integration test

* fix case in test queries

* make the server_type lowercase

* Apply suggestions from code review

Co-Authored-By: Himanshu <g.himanshu@gmail.com>

* Fix compilation from git suggestion

* fix unit test
2019-05-15 16:54:02 -07:00
Denis Boulas
0b739a5787 Added more mappings for new metrics. (#7665)
Signed-off-by: Denis Boulas <dene14@gmail.com>
2019-05-15 15:04:46 -07:00
Jonathan Wei
6901123a53 Fix compareAndSwap() in SQLMetadataConnector (#7661)
* Fix compareAndSwap() in SQLMetadataConnector

* Catch serialization_failure and retry for Postgres
2019-05-15 14:53:04 -07:00
Gian Merlino
0352f450d7 Fix broken links in docs, add broken link checker. (#7658)
Also adds back insert-segment-to-db.md with some docs about why and
when it was removed (in #6911).
2019-05-15 14:49:50 -07:00
Surekha
917106985f Update tutorial to delete data (#7577)
* Update tutorial to delete data

* update tutorial, remove old ways to drop data

* PR comments
2019-05-15 14:40:06 -07:00
Jonathan Wei
e874da7cea
Add simpler permissions option to BasicAuthorizer GET APIs (#7635)
* Add simpler permissions option to BasicAuthorizer GET APIs

* Adjust log message

Co-Authored-By: Himanshu <g.himanshu@gmail.com>

* Adjust log message

Co-Authored-By: Himanshu <g.himanshu@gmail.com>
2019-05-15 12:59:32 -07:00
Clint Wylie
b87c8f0314 fix lookup editor to use lookup tiers instead of historical tiers (#7647)
* fix lookup editor to use lookup tiers instead of historical tiers

* use default tier if empty response, fix if configured lookups is null

* fixes

* fix typo
2019-05-14 13:30:51 -07:00
Fokko Driesprong
2aa9613bed Bump Checkstyle to 8.20 (#7651)
* Bump Checkstyle to 8.20

Moderate severity vulnerability that affects:
com.puppycrawl.tools:checkstyle

Checkstyle prior to 8.18 loads external DTDs by default,
which can potentially lead to denial of service attacks
or the leaking of confidential information.

Affected versions: < 8.18

* Oops, missed one

* Oops, missed a few
2019-05-14 11:53:37 -07:00
Fokko Driesprong
e8a6575fb3 Remove Joda from indexing-hadoop (#7650) 2019-05-13 12:31:13 -07:00
Fokko Driesprong
4c709ddbc1 Bump Apache Parquet to 1.10.1 (#7645)
https://github.com/apache/parquet-mr/blob/master/CHANGES.md#version-1101
2019-05-12 14:38:33 -07:00
Xue Yu
35a1fbefea upgrade avatica to 1.12.0 (#7644) 2019-05-12 14:38:06 -07:00
Clint Wylie
c7b482625e travis.yml remove duplicate language entry (#7626)
* travis.yml remove duplicate language entry

* openjdk8 instead of oraclejdk8?
2019-05-10 17:09:54 -07:00
Xue Yu
f7b8b57c3b simpilfy DruidConvertletTable.java, remove STANDARD_CONVERTLET declare (#7632) 2019-05-10 14:08:32 -07:00
Alexander Saydakov
ca1a6649f6 Datasketches quantiles more post-aggs (#7550)
* rank and CDF post-aggs

* added post-aggs to the module

* added new post-aggs

* moved post-agg IDs

* moved post-agg IDs
2019-05-10 11:46:54 -07:00
Xavier Léauté
1d49364d08 Set direct memory if unable to detect JVM config (#7606)
* Set direct memory if unable to detect JVM config

Java 9 and above prevents us from detecting the maximum available direct
memory.

This change adds a fallback method to use at most 25% of maximum heap
size, which should be a reasonable default.

Unless -XX:MaxDirectMemorySize is set, recent JVMs will default maximum
direct memory to match the maximum heap size, so this should work out of
the box in most cases. For completeness we print instructions in the log
to explain how to adjust settings if necessary.

* skip test rather than succeeding

* reword log message

Co-Authored-By: Himanshu <g.himanshu@gmail.com>
2019-05-09 22:30:42 -07:00
Clint Wylie
402d76a10f make-redirects.py requires python3, explicitly specify it (#7625) 2019-05-09 21:32:58 -07:00
Clint Wylie
76e31937cb fix license.binary for new datasketches version (#7627) 2019-05-09 21:32:44 -07:00
Clint Wylie
6a6c6d573d
Add plain text README.txt, use relative link from README.md to build.md (#7611)
* use relative link to build instructions from top level readme

* add textfile to readme

* formatting

* make README.BINARY plaintext, move LABELS.md to LABELS, README.txt to README

* exclude README.BINARY still

* remove jdk links/recommmendations

* add script to use DRUIDVERSION in textfile README instead of latest, add links to recommended jdk to build.md

* license

* better readme template, links to latest if does not detect an apache release version

* fix
2019-05-09 21:29:26 -07:00
Jihoon Son
18e0d6acb4 Fix resultLevelCache for timeseries with grandTotal (#7624)
* Fix resultLevelCache for timeseries with grandTotal

* Address comment

* fix test
2019-05-09 18:11:04 -07:00
Alexander Saydakov
59f9ff38c7 fix issue #7607 (#7619)
* fix issue #7607

* exclude com.google.code.findbugs:annotations
2019-05-09 17:33:29 -07:00
Samarth Jain
b542bb9f34 TDigest backed sketch aggregators (#7331)
* First set of changes for tDigest histogram

* Add license

* Address code review comments

* Add a doc page for new T-Digest sketch aggregators. Minor code cleanup and comments.

* Remove synchronization from BufferAggregators. Address code review comments

* Fix typo
2019-05-09 17:22:55 -07:00
Jihoon Son
b32c9abdb9
Run npm ci instead of npm install (#7613) 2019-05-09 15:42:16 -07:00
Jonathan Wei
1b577c9b1d
Fix exception when using complex aggs with result level caching (#7614)
* Fix exception when using complex aggs with result level caching

* Add test comments

* checkstyle

* Add helper function for getting aggs from cache

* Move method to CacheStrategy

* Revert QueryToolChest changes

* Update test comments
2019-05-09 13:49:11 -07:00
Magnus Henoch
2ac112151f Fix formatting in scan query documentation (#7622)
Escape underscores in `__time`, so they're not interpreted as bold
formatting.
2019-05-09 11:32:37 -07:00
Jinseon Lee
0ef435a16c add postgresql meta db table schema configuration property (#7137) (#7183)
* add postgresql meta db table schema configuration property (#7137)

If the postgresql db schema changes, you must set the configuration
values.
You do not need to set it if there is no change from the default schema
'public'.
druid.metadata.postgres.dbTableSchema=public

* create postgresql metadb table schema configuration property (#7137)
If the postgresql db schema changes, you must set the configuration
values.
You do not need to set it if there is no change from the default schema
'public'.
druid.metadata.postgres.dbTableSchema=public
check PostgreSQLTablesConfig.java

* modify postgresql readme file. - metadb table schema (#7137)
If the postgresql db schema changes, you must set the configuration
values.
You do not need to set it if there is no change from the default schema
'public'.
druid.metadata.postgres.dbTableSchema=public
check PostgreSQLTablesConfig.java
2019-05-08 12:56:30 -07:00
Vadim Ogievetsky
9b197b436b added scrolling to data loader panes (#7605) 2019-05-07 08:45:20 -07:00
Jonathan Wei
dadf6a2f11
Add tool for migrating from local deep storage/Derby metadata (#7598)
* Add tool for migrating from local deep storage/Derby metadata

* Split deep storage and metadata migration docs

* Support import into Derby

* Fix create tables cmd

* Fix create tables cmd

* Fix commands

* PR comment

* Add -p
2019-05-06 23:39:40 -07:00