151edeec3c
expression virtual column selector fix for expressions which produce array types ( #7958 )
...
* fix bug in multi-value string expression column selector
* more test
* imports!!
* fixes
2019-06-26 16:57:13 -07:00
5464c8938f
Add array_slice and array_unshift function expr ( #7950 )
...
* add array_slice and array_unshift function expr
* feedback address
2019-06-26 16:56:09 -07:00
d677c83ce4
Web console: Power up the data loader init step ( #7947 )
...
* Power up the data loader init step
* update snapshot
* normalize spec
* allow deselect
* added HDFS tile
* update border style
* text updates
* goodies
* new reset icon
2019-06-26 15:50:48 -07:00
bc1413e4e3
Bump commons-cli from 1.2 to 1.3.1 ( #7966 )
2019-06-26 08:05:13 -07:00
a171b4a399
Web-console: Add action column to segments view ( #7954 )
...
* add actions column to segments view
* add sements action column
2019-06-25 20:14:06 -07:00
9d925f5086
Adjust NOTICE files ( #7945 )
...
* Adjust NOTICE files
* remove commons-logging
* Remove lucene from derby
* more adjustment
2019-06-25 09:08:54 -07:00
b9c6a26c0e
Use ComplexMetrics.registerSerde() across the codebase ( #7925 )
...
* refactor complexmetric registerserde
* fix error
* feedback address
2019-06-25 11:39:04 +03:00
16aafd5788
[ImgBot] Optimize images ( #7873 )
...
*Total -- 10,997.25kb -> 7,160.16kb (34.89%)
/publications/radstack/figures/precompute.png -- 54.20kb -> 16.97kb (68.69%)
/web-console/favicon.png -- 4.41kb -> 1.61kb (63.58%)
/docs/img/indexing_service.png -- 47.37kb -> 21.96kb (53.64%)
/docs/img/segmentPropagation.png -- 62.94kb -> 29.85kb (52.57%)
/docs/content/tutorials/img/tutorial-quickstart-01.png -- 55.62kb -> 29.13kb (47.62%)
/docs/content/tutorials/img/tutorial-deletion-02.png -- 791.43kb -> 429.30kb (45.76%)
/docs/content/tutorials/img/tutorial-deletion-03.png -- 786.79kb -> 427.05kb (45.72%)
/docs/content/tutorials/img/tutorial-retention-00.png -- 135.06kb -> 75.88kb (43.82%)
/docs/content/tutorials/img/tutorial-batch-data-loader-10.png -- 77.23kb -> 43.47kb (43.71%)
/docs/content/tutorials/img/tutorial-batch-data-loader-01.png -- 97.03kb -> 55.16kb (43.15%)
/docs/content/tutorials/img/tutorial-batch-data-loader-07.png -- 79.49kb -> 45.44kb (42.84%)
/docs/content/tutorials/img/tutorial-retention-02.png -- 401.30kb -> 234.68kb (41.52%)
/docs/content/tutorials/img/tutorial-compaction-06.png -- 343.27kb -> 201.87kb (41.19%)
/docs/content/tutorials/img/tutorial-batch-data-loader-09.png -- 105.14kb -> 61.86kb (41.16%)
/docs/content/tutorials/img/tutorial-retention-06.png -- 227.57kb -> 134.35kb (40.97%)
/docs/content/tutorials/img/tutorial-compaction-04.png -- 304.83kb -> 180.04kb (40.94%)
/docs/content/tutorials/img/tutorial-compaction-02.png -- 273.18kb -> 162.67kb (40.45%)
/docs/content/tutorials/img/tutorial-query-05.png -- 85.03kb -> 50.64kb (40.44%)
/publications/radstack/figures/druid_vs_bigquery.png -- 155.44kb -> 92.85kb (40.27%)
/docs/content/tutorials/img/tutorial-kafka-02.png -- 122.51kb -> 73.93kb (39.65%)
/docs/content/tutorials/img/tutorial-deletion-01.png -- 70.37kb -> 42.56kb (39.52%)
/docs/content/tutorials/img/tutorial-batch-data-loader-06.png -- 103.50kb -> 62.79kb (39.33%)
/docs/content/tutorials/img/tutorial-batch-submit-task-01.png -- 111.25kb -> 67.73kb (39.12%)
/docs/content/tutorials/img/tutorial-query-03.png -- 103.60kb -> 63.51kb (38.69%)
/docs/content/tutorials/img/tutorial-query-04.png -- 105.79kb -> 64.87kb (38.69%)
/docs/content/tutorials/img/tutorial-batch-data-loader-11.png -- 130.20kb -> 81.34kb (37.53%)
/docs/content/tutorials/img/tutorial-query-07.png -- 122.52kb -> 76.79kb (37.32%)
/docs/content/tutorials/img/tutorial-kafka-01.png -- 133.12kb -> 83.47kb (37.3%)
/docs/content/tutorials/img/tutorial-query-06.png -- 127.55kb -> 80.28kb (37.06%)
/docs/content/tutorials/img/tutorial-batch-submit-task-02.png -- 133.07kb -> 84.06kb (36.83%)
/docs/content/tutorials/img/tutorial-retention-05.png -- 60.19kb -> 38.08kb (36.74%)
/docs/content/tutorials/img/tutorial-batch-data-loader-03.png -- 211.92kb -> 134.22kb (36.66%)
/docs/content/tutorials/img/tutorial-batch-data-loader-05.png -- 250.36kb -> 158.68kb (36.62%)
/publications/radstack/figures/radstack.png -- 16.80kb -> 10.67kb (36.48%)
/docs/content/tutorials/img/tutorial-batch-data-loader-08.png -- 158.59kb -> 101.49kb (36%)
/docs/content/tutorials/img/tutorial-batch-data-loader-04.png -- 255.10kb -> 163.33kb (35.97%)
/docs/content/tutorials/img/tutorial-query-02.png -- 126.92kb -> 81.42kb (35.85%)
/docs/content/tutorials/img/tutorial-compaction-01.png -- 53.86kb -> 34.87kb (35.25%)
/docs/img/druid-architecture.png -- 202.23kb -> 130.97kb (35.24%)
/docs/content/tutorials/img/tutorial-retention-01.png -- 52.69kb -> 34.35kb (34.81%)
/docs/img/druid-timeline.png -- 35.87kb -> 23.59kb (34.22%)
/docs/content/tutorials/img/tutorial-query-01.png -- 149.53kb -> 98.56kb (34.08%)
/docs/content/tutorials/img/tutorial-retention-04.png -- 65.91kb -> 43.57kb (33.89%)
/docs/content/tutorials/img/tutorial-compaction-08.png -- 42.24kb -> 28.08kb (33.53%)
/docs/content/tutorials/img/tutorial-compaction-07.png -- 39.17kb -> 26.06kb (33.47%)
/docs/content/tutorials/img/tutorial-compaction-03.png -- 39.17kb -> 26.13kb (33.3%)
/docs/content/tutorials/img/tutorial-compaction-05.png -- 38.85kb -> 25.96kb (33.17%)
/publications/demo/figures/throughput_vs_cardinality.png -- 73.49kb -> 49.31kb (32.9%)
/publications/radstack/figures/throughput_vs_cardinality.png -- 73.49kb -> 49.31kb (32.9%)
/publications/whitepaper/figures/throughput_vs_cardinality.png -- 73.49kb -> 49.31kb (32.9%)
/docs/content/tutorials/img/tutorial-retention-03.png -- 43.11kb -> 29.33kb (31.97%)
/publications/radstack/figures/throughput_vs_num_dims.png -- 72.86kb -> 49.72kb (31.76%)
/publications/whitepaper/figures/throughput_vs_num_dims.png -- 72.86kb -> 49.72kb (31.76%)
/publications/demo/figures/throughput_vs_num_dims.png -- 72.86kb -> 49.72kb (31.76%)
/publications/radstack/figures/joined.png -- 164.14kb -> 113.47kb (30.87%)
/docs/content/tutorials/img/tutorial-batch-data-loader-02.png -- 508.93kb -> 351.85kb (30.87%)
/publications/radstack/figures/imps_clicks.png -- 190.95kb -> 132.70kb (30.51%)
/publications/radstack/figures/shuffled.png -- 180.46kb -> 128.21kb (28.95%)
/publications/radstack/figures/pipeline.png -- 392.54kb -> 281.93kb (28.18%)
/docs/img/druid-manage-1.png -- 108.94kb -> 78.53kb (27.92%)
/publications/radstack/figures/throughput_vs_num_metrics.png -- 85.25kb -> 61.80kb (27.51%)
/publications/demo/figures/throughput_vs_num_metrics.png -- 85.25kb -> 61.80kb (27.51%)
/publications/whitepaper/figures/throughput_vs_num_metrics.png -- 85.25kb -> 61.80kb (27.51%)
/docs/img/druid-production.png -- 50.00kb -> 39.18kb (21.63%)
/docs/img/druid-dataflow-3.png -- 88.25kb -> 69.75kb (20.96%)
/publications/demo/figures/realtime_flow.png -- 51.12kb -> 40.61kb (20.56%)
/publications/demo/figures/realtime_timeline.png -- 36.15kb -> 29.24kb (19.12%)
/publications/demo/figures/tpch_scaling.png -- 43.21kb -> 34.97kb (19.08%)
/publications/demo/figures/caching.png -- 35.26kb -> 29.09kb (17.49%)
/dev/intellij-sdk-config.jpg -- 1,019.35kb -> 864.37kb (15.2%)
/docs/img/druid-column-types.png -- 101.53kb -> 91.17kb (10.2%)
/docs/img/druid-dataflow-2x.png -- 138.30kb -> 127.11kb (8.09%)
2019-06-24 21:27:48 -07:00
cd01a48eab
Web-console: add timed button ( #7912 )
...
* add refresh button
* update snapshots
* fix spacing
* add supervisors button
* modify query manager
* add icon
* update snapshots
* fix space
* stop refreshing when option selected
* small fixes
* add default value
* fix spelling mistake
* fix query-input
* rename class
2019-06-24 21:18:19 -07:00
6cc8802b8e
#7875 : Setting ACL on S3 task logs on similar lines as that of data segments pushed to S3 ( #7907 )
...
* #7875 : Setting ACL on S3 task logs on similar lines as that of data segment pushed to S3
* #7875 1. Extracting a method (which uploads a file to S3 setting appropriate access control list to the file being uploaded) and moving it to utils class. 2. Adding S3TaskLogsTest.java file to test acl (permissions) on the task log files pushed to S3.
* fixing checkstyle errors
* #7875 Incorporating review comments
2019-06-24 17:25:59 -06:00
0d5fbfa0eb
Add missing license pointer for Porter Stemmer ( #7941 )
...
* Add missing license pointer for Porter Stemmer
* add missing file
2019-06-24 12:21:40 -07:00
82b248cc17
Spotbugs: Enable MS_SHOULD_BE_FINAL ( #7946 )
2019-06-23 15:42:18 -07:00
35601bb7a0
Add finalizeAsBase64Binary option to FixedBucketsHistogramAggregatorFactory ( #7784 )
...
* Add finalizeAsBase64Binary option to FixedBucketsHistogramAggregatorFactory
* Add finalizeAsBase64Binary option to ApproximateHistogramFactory
* Update approx histogram doc
2019-06-21 18:00:19 -07:00
6c8f9482c7
Web console: prettier code ( #7939 )
...
* prettier code
* update tests
* scss also
* more ,
* update pachage json prettier config
2019-06-21 16:52:33 -07:00
183d5be7ec
fix spotbugs failure in Expr.java ( #7944 )
2019-06-21 14:49:22 -07:00
2bc8e7c0e8
more acurate keyword auto complete ( #7934 )
2019-06-20 21:14:54 -07:00
d046d5aa8d
Stalebot config adjustments. ( #7936 )
...
* Stalebot config adjustments.
- Different wordings for PRs and issues.
- Extend stale timeout for issues.
- Exempt "Proposal" label.
- Exempt issues in projects and issues with assignees.
* Fix copy/paste error.
2019-06-20 14:16:57 -07:00
ceade03213
Enable stalebot for issues ( #7927 )
...
* Enable stalebot for issues
* fix comments
* fix comments
* fix comments
2019-06-20 11:09:40 -07:00
48f20fe754
Add Spotbugs ( #7894 )
...
* Add Spotbugs
Exclude all the issues for now, so we can add them one by one.
(cherry picked from commit ceda4754dc8c703d1e0de85b48cd5f5409cfd5b7)
* Add additional rules to the list
* More rules
* More rules
* Add comments to the xml
* Move the spotbugs-exclude.xml to codestyle/
2019-06-20 21:06:52 +03:00
494b8ebe56
multi-value string column support for expressions ( #7588 )
...
* array support for expression language for multi-value string columns
* fix tests?
* fixes
* more tests
* fixes
* cleanup
* more better, more test
* ignore inspection
* license
* license fix
* inspection
* remove dumb import
* more better
* some comments
* add expr rewrite for arrayfn args for more magic, tests
* test stuff
* more tests
* fix test
* fix test
* castfunc can deal with arrays
* needs more empty array
* more tests, make cast to long array more forgiving
* refactor
* simplify ExprMacro Expr implementations with base classes in core
* oops
* more test
* use Shuttle for Parser.flatten, javadoc, cleanup
* fixes and more tests
* unused import
* fixes
* javadocs, cleanup, refactors
* fix imports
* more javadoc
* more javadoc
* more
* more javadocs, nonnullbydefault, minor refactor
* markdown fix
* adjustments
* more doc
* move initial filter out
* docs
* map empty arg lambda, apply function argument validation
* check function args at parse time instead of eval time
* more immutable
* more more immutable
* clarify grammar
* fix docs
* empty array is string test, we need a way to make arrays better maybe in the future, or define empty arrays as other types..
2019-06-19 13:57:37 -07:00
28eaa620a9
Web console: Data loader respects parse spec columns for data preview ( #7922 )
...
* small fixes in the data loader
* respect columns
* fix test
2019-06-19 12:48:31 -07:00
41f23b5120
Bump commons-compress from 1.16 to 1.18 ( #7924 )
2019-06-19 10:43:01 -07:00
0a6fbbbb80
Bump Apache Avro to 1.9.0 ( #7772 )
...
* Bump Apache Avro to 1.9.0
Apache Avro 1.9.0 brings a lot of new features:
* Deprecate Joda-Time in favor of Java8 JSR310 and setting it as default
* Remove support for Hadoop 1.x
* Move from Jackson 1.x to 2.9
* Add ZStandard Codec
* Lots of updates on the dependencies to fix CVE's
* Remove Jackson classes from public API
* Apache Avro is built by default with Java 8
* Apache Avro is compiled and tested with Java 11 to guarantee compatibility
* Apache Avro MapReduce is compiled and tested with Hadoop 3
* Apache Avro is now leaner, multiple dependencies were removed: guava, paranamer, commons-codec, and commons-logging
* Introduce JMH Performance Testing Framework
* Add Snappy support for C++ DataFile
* and many, many more!
* Add exclusions for Jackson
2019-06-19 03:31:18 -07:00
417fcef385
WorkerTaskManager to create disk files atomically and ignore task file corruption ( #7917 )
...
* WorkerTaskManager to create disk files atomically and ignore task file
corruptions
* fixing weird checkstyle lambda indentation issues
2019-06-18 09:18:43 -07:00
20d1db9dff
bump fastutil to 8.2.3 ( #7920 )
2019-06-18 09:17:34 -07:00
71997c16a2
switch links from druid.io to druid.apache.org ( #7914 )
...
* switch links from druid.io to druid.apache.org
* fix it
2019-06-18 09:06:27 -07:00
e80297efef
Set Test timeout higher for robust performance ( #7890 )
2019-06-17 22:01:54 -07:00
c44620a2e3
clean auto generated doc file also ( #7911 )
2019-06-17 18:01:29 -07:00
24dd4573da
Added the web console to the quickstart tutorials and docs ( #7863 )
...
* added console to the quickstart tutorials
* feedback fixes
* feedback fixes
* more typo fixes
* moved reseting cluster section after load data
* update images
* stage -> step
* feedback fixes
* more feedback fixes
2019-06-17 18:00:54 -07:00
df9cdcf13b
Web console: added Query View metadata pane, more query details ( #7905 )
...
* refactor
* adding metadata hookup
* add scroll
* standerdize icons
* better loading state
* update snapshots
* fix lint
* do not mix up queryId and sqlQueryId
2019-06-17 13:14:42 -07:00
01881e3a98
Use only com.google.errorprone.annotations.concurrent.GuardedBy, not javax.annotations.concurrent.GuardedBy ( #7889 )
2019-06-17 15:58:51 +02:00
4bceab0d7b
Bump jmh from 1.19 to 1.21 ( #7876 )
2019-06-17 19:27:09 +08:00
fb7f8ec362
Bump RoaringBitmap from 0.8.0 to 0.8.6 ( #7906 )
2019-06-17 17:02:52 +08:00
daed1bc141
Fix HttpRemoteTaskRunnerConfig not loading ( #7902 )
2019-06-16 00:31:42 -07:00
b3328b2785
endpoint to delete lookup tier and remove tier on last lookup deletion ( #7852 )
2019-06-15 17:55:50 -07:00
f603498e11
Web console: Add download path to SQL Query ( #7898 )
...
* adding download path to Query
* add more checker
* updated snap tests
* change after Vad's comments
2019-06-15 13:51:22 -07:00
8e5003b01c
Scan Doc Change ( #7903 )
2019-06-15 01:21:34 -07:00
3cd9a7507d
Fix script for dependencies report for extensions ( #7899 )
2019-06-14 18:53:50 -07:00
d00a9676b7
Set aws.region for unit tests automatically ( #7868 )
...
* Set aws.region for unit tests automatically
* Update README.template
2019-06-14 15:34:21 -07:00
a648e1548d
Add support of --exclude-extension argument for dependency report script ( #7786 )
2019-06-14 15:18:59 -07:00
5e6d833fd9
Web-Console: tail log based on task status ( #7888 )
...
* clean up
* fix declarations
* spacing
2019-06-14 14:37:21 -07:00
f581118f05
Remove Apache Pig from the tests ( #7810 )
...
* Remove Apache Pig from the tests
* Remove the Pig specific part
* Fix the Checkstyle issues
* Cleanup a bit
* Add an additional test
* Revert the abstract class
2019-06-14 14:18:58 -07:00
3bee6adcf7
Use map.putIfAbsent() or map.computeIfAbsent() as appropriate instead of containsKey() + put() ( #7764 )
...
* https://github.com/apache/incubator-druid/issues/7316 Use Map.putIfAbsent() instead of containsKey() + put()
* fixing indentation
* Using map.computeIfAbsent() instead of map.putIfAbsent() where appropriate
* fixing checkstyle
* Changing the recommendation text
* Reverting auto changes made by IDE
* Implementing recommendation: A ConcurrentHashMap on which computeIfAbsent() is called should be assigned into variables of ConcurrentHashMap type, not ConcurrentMap
* Removing unused import
2019-06-14 17:59:36 +02:00
05346a9e0c
allow semicolon with comment ( #7892 )
2019-06-14 07:53:19 -07:00
456a3654ce
add PolygonBound and missing extentions list doc ( #7885 )
2019-06-13 12:03:58 -07:00
0763585208
Web-console: update supervisors table ( #7799 )
...
* add new columns
* fix syling
* fix spaces
* update snapshots
* fix Spelling
* fix capitalization
* reorder action dialog
* set color using state
* fix snapshots
* fix array
* update snapshots
* remove extra columns
* update snapshots
* update snapshots
* fixes
* update snapshots
* use cell
* fix spacing
* update snapshot
2019-06-11 13:06:35 -07:00
8117222da3
use right port for kafka tutorial, reinfoce that tutorials assume you are using micro-quickstart single-server configuration ( #7862 )
2019-06-11 08:50:52 -07:00
ce591d1457
Support var_pop, var_samp, stddev_pop and stddev_samp etc in sql ( #7801 )
...
* support var_pop, stddev_pop etc in sql
* fix sql compatible
* rebase on master
* update doc
2019-06-10 09:40:09 -07:00
c612ddc0f4
Support more JODA time formats ( #7857 )
2019-06-09 20:08:29 -07:00
c3d230b354
Web console: make the data loader more responsive when sampling data with many columns ( #7856 )
...
* Don't rerender data loader tables
* removed unused imports
* removed extra console.log
* fix typo
2019-06-09 10:44:13 -07:00
Clint Wylie
Xue Yu
Vadim Ogievetsky
Benedict Jin
mcbrewster
Jihoon Son
Sashidhar Thallam
Fokko Driesprong
Jonathan Wei
Gian Merlino
Fangjin Yang
Fokko Driesprong
Clint Wylie
Himanshu
Xue Yu
SandishKumarHN
litao91
Jenny Zhu
Justin Borromeo