Commit Graph

132 Commits

Author SHA1 Message Date
Xavier Léauté d105519558
Replace use of PowerMock with Mockito (#12282)
Mockito now supports all our needs and plays much better with recent Java versions.
Migrating to Mockito also simplifies running the kind of tests that required PowerMock in the past. 

* replace all uses of powermock with mockito-inline
* upgrade mockito to 4.3.1 and fix use of deprecated methods
* import mockito bom to align all our mockito dependencies
* add powermock to forbidden-apis to avoid accidentally reintroducing it in the future
2022-02-27 22:47:09 -08:00
Xavier Léauté 1434197ee1
update airline dependency to 2.x (#12270)
* upgrade Airline to Airline 2
  https://github.com/airlift/airline is no longer maintained, updating to
  https://github.com/rvesse/airline (Airline 2) to use an actively
  maintained version, while minimizing breaking changes.

  Note, this is a backwards incompatible change, and extensions relying on
  the CliCommandCreator extension point will also need to be updated.

* fix dependency checks where jakarta.inject is now resolved first instead
  of javax.inject, due to Airline 2 using jakarta
2022-02-27 15:19:28 -08:00
Xavier Léauté 009dd9e09a
upgrade core Apache Kafka dependencies to 3.1.0 (#12203)
Announcement: https://blogs.apache.org/kafka/entry/what-s-new-in-apache7
Release notes: https://dist.apache.org/repos/dist/release/kafka/3.1.0/RELEASE_NOTES.html

* upgrade core Apache Kafka dependencies to 3.1.0
* fix use of private Kafka APIs
* remove deprecated test rules
* remove mock calls that weren't verified in the first place
* remove the need for powermock in KafkaLookupExtractorFactoryTest
* align curator-test version with curator itself
* update easymock to 4.3.0
2022-02-23 18:42:51 -08:00
William Hyun 34bc361953
Update ORC to 1.7.2 (#12084) 2022-02-15 10:04:12 -08:00
Vadim Ogievetsky bc408bacc8
Web console: Adding a shard detail column to the segments view (#12212)
* shard spec details

* improve pattern match

* refactor spec cleanup

* better format detection

* update JSONbig

* add multiline option to autoform
2022-02-02 18:46:17 -08:00
aggarwalakshay eb4fafe08f
Upgrading follow-redirects to 1.14.7 (#12153)
* Upgrading follow-redirects to 1.14.7

* removed the existing follow-redirects i.e. 1.14.4 from package-lock.json
2022-01-13 14:01:36 -08:00
Vadim Ogievetsky 2a41b7bffa
Web console: correctly cancel JSON shaped SQL queries (#12134)
* misc fixes

* type typo
2022-01-10 14:24:05 -08:00
Frank Chen fe71fc414f
Update log4j2 to 2.17.1 (#12106)
Signed-off-by: frank chen <frank.chen021@outlook.com>
2021-12-30 19:18:16 -06:00
Vadim Ogievetsky 37112d24e2
Web console: new Ace, diff view, and cleanup. Decorating the console for the holidays 🎁 (#12085)
* Add diff view and upgrade AceEditor

* fix test

* function doc parsing fixes

* escape args

* allowKeys

* everyone gets a diff

* update snapshot
2021-12-22 16:31:17 -08:00
Frank Chen f345759360
Update to 2.17.0 (#12081) 2021-12-19 20:27:08 -08:00
Gian Merlino d917e0433e
Update to log4j 2.16.0. (#12061)
* Update to log4j 2.16.0.

* Update licenses.yaml
2021-12-13 19:06:00 -08:00
Suneet Saldanha ffa4783ce8
Adjust log4j version in licenses.yaml. (#12053)
Co-authored-by: Gian Merlino <gian@imply.io>
2021-12-10 08:12:54 -08:00
Vadim Ogievetsky 1d3c8c187b
Web console: query view improvements and other fixes (#12031)
* don't copy commas

* use numeric type information

* add VALUES keyword

* propogate rollup config into spec

* fix

* cleanup

* understand range partitioning

* update snapshots

* better comp apis

* fix segment pages

* update snapshots
2021-12-07 10:16:16 -08:00
William Hyun 3abca73ee8
Upgrade ORC to 1.7.1 (#11919) 2021-11-15 09:13:03 -08:00
Karan Kumar 90640bb316
Support for hadoop 3 via maven profiles (#11794)
Add support for hadoop 3 profiles . Most of the details are captured in #11791 .
We use a combination of maven profiles and resource filtering to achieve this. Hadoop2 is supported by default and a new maven profile with the name hadoop3 is created. This will allow the user to choose the profile which is best suited for the use case.
2021-10-30 22:46:24 +05:30
Jihoon Son 07a232d7b4
Bump netty4 to 4.1.68; suppress CVE-2021-37136 and CVE-2021-37137 for netty3 (#11844)
* bump netty4 to 4.1.68

* suppress CVE-2021-37136 and CVE-2021-37137 for netty3

* license
2021-10-25 21:09:15 -07:00
Alexander Saydakov 8cf1cbc4a9
latest datasketches-java and datasketches-memory (#11773)
* latest datasketches-java and datasketches-memory

* updated versions of datasketches-java and datasketches-memory

Co-authored-by: AlexanderSaydakov <AlexanderSaydakov@users.noreply.github.com>
2021-10-19 23:42:30 -07:00
Xavier Léauté bc3b038712
Update Apache Kafka client libraries to 3.0.0 (#11735)
Release notes:
https://downloads.apache.org/kafka/3.0.0/RELEASE_NOTES.html
https://blogs.apache.org/kafka/entry/what-s-new-in-apache6
2021-10-05 10:23:19 -07:00
William Hyun 9bff6bd70e
Upgrade ORC to 1.7.0 (#11726)
* Upgrade ORC to 1.7.0

* address comments

* address comments

* Add import
2021-09-27 13:20:09 -07:00
dependabot[bot] 776ddf76f4
Bump parquet.version from 1.11.1 to 1.12.0 (#11346)
* Bump parquet.version from 1.11.1 to 1.12.0

Bumps `parquet.version` from 1.11.1 to 1.12.0.

Updates `parquet-column` from 1.11.1 to 1.12.0
- [Release notes](https://github.com/apache/parquet-mr/releases)
- [Changelog](https://github.com/apache/parquet-mr/blob/master/CHANGES.md)
- [Commits](https://github.com/apache/parquet-mr/compare/apache-parquet-1.11.1...apache-parquet-1.12.0)

Updates `parquet-avro` from 1.11.1 to 1.12.0
- [Release notes](https://github.com/apache/parquet-mr/releases)
- [Changelog](https://github.com/apache/parquet-mr/blob/master/CHANGES.md)
- [Commits](https://github.com/apache/parquet-mr/compare/apache-parquet-1.11.1...apache-parquet-1.12.0)

Updates `parquet-hadoop` from 1.11.1 to 1.12.0
- [Release notes](https://github.com/apache/parquet-mr/releases)
- [Changelog](https://github.com/apache/parquet-mr/blob/master/CHANGES.md)
- [Commits](https://github.com/apache/parquet-mr/compare/apache-parquet-1.11.1...apache-parquet-1.12.0)

---
updated-dependencies:
- dependency-name: org.apache.parquet:parquet-column
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.apache.parquet:parquet-avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.apache.parquet:parquet-hadoop
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update license

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Suneet Saldanha <suneet@apache.org>
2021-08-13 19:17:57 -07:00
dependabot[bot] 2e850b5655
Bump aws.sdk.version from 1.11.884 to 1.12.37 (#11367)
* Bump aws.sdk.version from 1.11.884 to 1.12.8

Bumps `aws.sdk.version` from 1.11.884 to 1.12.8.

Updates `aws-java-sdk-core` from 1.11.884 to 1.12.8
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.11.884...1.12.8)

Updates `aws-java-sdk-ec2` from 1.11.884 to 1.12.8
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.11.884...1.12.8)

Updates `aws-java-sdk-s3` from 1.11.884 to 1.12.8
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.11.884...1.12.8)

Updates `aws-java-sdk-sts` from 1.11.884 to 1.12.8
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.11.884...1.12.8)

Updates `aws-java-sdk-kinesis` from 1.11.884 to 1.12.8
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.11.884...1.12.8)

Updates `aws-java-sdk-rds` from 1.11.884 to 1.12.8
- [Release notes](https://github.com/aws/aws-sdk-java/releases)
- [Changelog](https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-java/compare/1.11.884...1.12.8)

---
updated-dependencies:
- dependency-name: com.amazonaws:aws-java-sdk-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: com.amazonaws:aws-java-sdk-ec2
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: com.amazonaws:aws-java-sdk-s3
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: com.amazonaws:aws-java-sdk-sts
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: com.amazonaws:aws-java-sdk-kinesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: com.amazonaws:aws-java-sdk-rds
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix license and bump to latest aws

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Suneet Saldanha <suneet@apache.org>
2021-08-01 00:12:57 -07:00
Dongjoon Hyun dbed4424b5
Upgrade ORC to 1.6.9 (#11518) 2021-07-31 23:33:03 -07:00
Xavier Léauté 4bca7f014e
update error-prone to 2.8.0 with fix for crashing check (#11494)
* error-prone 2.8.0 fixes https://github.com/google/error-prone/issues/2396
* fix for a few ignored return values
* fix unknown args in sub-modules
2021-07-29 09:13:46 -07:00
Suneet Saldanha 3f456fe305
Address CVE-2021-35515 CVE-2021-36090 (#11496)
* Address CVE-2021-35515 CVE-2021-36090

Bump commons-compress to deal with new CVEs

* fix licenses
2021-07-26 14:54:32 -07:00
Dongjoon Hyun 5037493e45
Bump commons-io to 2.11.0 (#11460)
* Bump commons-io to 2.11.0

* Address comments

* Remove try catch

* Fix checkstyle
2021-07-19 15:47:14 -07:00
Vadim Ogievetsky 377b5e708c
Web console: Data loading walkthrough fixes (#11416)
* fix quotes

* fix sql doc parsing

* prevent array-input from losing position while the user is typing

* make group filter click-to-filterable

* fix casing bug in exact table search

* do not sort columns in smaples

* can bypass transform step

* fixed string json parsing

* improve PartitionMessage

* better error messages

* feedback fixes

* tool to order dimensions in schema view
2021-07-10 07:56:50 -07:00
zachjsh 8037a54525
revert commons-io to 2.6 (#11392)
* * revert commons-io to 2.6

* * fix failing tests
2021-06-29 23:04:38 -07:00
Vadim Ogievetsky f56a5b9ba2
Web console: Better hotkeys and library upgrades (#11365)
* improve hotkeys

* fix test name

* refactor explain dialog

* explain tests

* small fixes

* update to popover2

* use resize sensor 2

* misc cleanup
2021-06-17 18:24:29 -07:00
Xavier Léauté a1c20d7457
update jackson dependencies to use bom (#11353)
Switching to the bom dependency declaration simplifies managing jackson
dependencies. It also removes the need to override individual library
versions for CVE fixes, since the bom takes care of that internally.

This change aligns our jackson dependency versions on 2.10.5(.x):
- updates jackson libraries from 2.10.2 to 2.10.5
- jackson-databind remains at 2.10.5.1 as defined in the bom

Release notes: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10
2021-06-16 18:37:30 -07:00
Xavier Léauté 712f2a5d00
upgrade error-prone to 2.7.1 and support checks with Java 11+ (#11363)
* upgrade error-prone to 2.7.1 and support checks with Java 11+

- upgrade error-prone to 2.7.1
- support running error-prone with Java 11 and above using -Xplugin
  instead of custom compiler
- add compiler arguments to ignore warnings/errors in Java 15/16
- introduce strictCompile property to enable strict profiles since we
  now need multiple strict profiles for Java 8
- properly exclude all generated source files from error-prone
- fix druid-processing overriding annotation processors from parent pom
- fix druid-core disabling most non-default checks
- align plugin and annotation errorprone versions
- fix / suppress additional issues found by error-prone:
  * fix bug in SeekableStreamSupervisor initializing ArrayList size with
    the taskGroupdId
  * fix missing @Override annotations
- remove outdated compiler plugin in benchmarks
- remove deleted ParameterPackage error-prone rule
- re-enable checks on benchmark module as well

* fix IntelliJ inspections

* disable LongFloatConversion due to bug in error-prone with JDK 8

* add comment about InsecureCrypto
2021-06-16 12:55:34 -07:00
John Gozde ac9b1f81b4
Web console: Remove support for IE11 and other older browsers (#11357)
* Use common browserlist and update to drop IE11

* Change TypeScript target to ES2016

* Update browserslist for "supports es6" support

* Show a warning if accessed from an unsupported browser

* Inline browser-update styles; detect SyntaxErrors too

* Better wording

* Upgrade to the latest Blueprint

* Refactor RunButton to be FC, use useHotkeys

* Remove dead license

* Update snapshots

* Address feedback

* Wording

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>
2021-06-10 19:05:40 -07:00
dependabot[bot] 167044f715
Bump fastutil from 8.2.3 to 8.5.4 (#11347)
* Bump fastutil from 8.2.3 to 8.5.4

Bumps [fastutil](https://github.com/vigna/fastutil) from 8.2.3 to 8.5.4.
- [Release notes](https://github.com/vigna/fastutil/releases)
- [Changelog](https://github.com/vigna/fastutil/blob/master/CHANGES)
- [Commits](https://github.com/vigna/fastutil/compare/8.2.3...8.5.4)

---
updated-dependencies:
- dependency-name: it.unimi.dsi:fastutil
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* update licenses.yaml
* update maven dependency list for -core and -extra libraries to pass maven dependency checks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xavier Léauté <xvrl@apache.org>
2021-06-10 07:43:18 -07:00
dependabot[bot] be10a236d5
Bump commons-io from 2.6 to 2.9.0 (#11338)
* Bump commons-io from 2.6 to 2.9.0

Bumps commons-io from 2.6 to 2.9.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* update licenses.yaml to reflect version bumps
* fix tests relying on specific log messages

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xavier Léauté <xvrl@apache.org>
2021-06-08 10:02:59 -07:00
Vadim Ogievetsky 0c5d1c9725
Web console: add more query fixing auto suggestions (#11203)
* add more query fixing auto suggestions

* update query gen

* update toolkit

* update licenses

* fix funky quotes

* funky => fancy

* revert engine change

* separate web-console and website npm and node deps
2021-06-04 09:29:00 -07:00
Xavier Léauté b517c3339b
remove ZooKeeper 3.4 support + pass tests with Java 15 (#11073)
With this change, Druid will only support ZooKeeper 3.5.x and later.

In order to support Java 15 we need to switch to ZK 3.5.x client libraries and drop support for ZK 3.4.x
(see #10780 for the detailed reasons) 

* remove ZooKeeper 3.4.x compatibility
* exclude additional ZK 3.5.x netty dependencies to ensure we use our version
* keep ZooKeeper version used for integration tests in sync with client library version
* remove the need to specify ZK version at runtime for docker
* add support to run integration tests with JDK 15
* build and run unit tests with Java 15 in travis
2021-05-25 12:49:49 -07:00
Atul Mohan ac95f99d59
Upgrade jetty version (#11207) 2021-05-06 18:21:36 +08:00
Xavier Léauté 0296f20551
upgrade Apache Kafka to 2.8.0 (#11139)
* upgrade to Apache Kafka 2.8.0 (release notes:
  https://downloads.apache.org/kafka/2.8.0/RELEASE_NOTES.html)
* pass Kafka version as a Docker argument in integration tests
  to keep in sync with maven version
* fix use of internal Kafka APIs in integration tests
2021-04-24 08:27:07 -07:00
Vadim Ogievetsky 4caa221d72
Web console: Better inline docs (#11128)
* better highlight

* better highlighting

* add spec
2021-04-19 14:36:53 -07:00
John Gozde fdc3c2f362
Web console: update dev dependencies (#11119)
* Update some dev dependencies, prettify, tslint-fix

* Sort tsconfig keys for easy comparison

* Set noImplicitThis

* Slightly more accurate types

* Bump Jest and related

* Bump react to latest on v16

* Bump node-sass, sass-loader for node14 support

* Remove node-sass-chokidar (unused)

* More unused dependencies

* Fix blueprint imports

* Webpack 5

* Update webpack config for 'process' usage

* Update playwright-chromium

* Emit esnext modules for tree shaking

* Enable source maps in development

* Dedupe

* Bump babel and things

* npm audit fix

* Add .editorconfig file to match prettier settings

* Update licenses (tslib is 0BSD as of 1.11.2)

https://github.com/microsoft/tslib/pull/96

* Require node >= 10

* Use Node 10 to run e2e tests

* Use 'ws' transport mode for dev server (will be default in next version)

* Remove an 'any'

* No sourcemaps in prod

* Exclude .editorconfig from license checks

* Try nvm for setting node version
2021-04-16 20:15:19 -07:00
Sandeep 26d1074ade
[Security] Bump netty4.version from 4.1.48.Final to 4.1.63.Final (#11117) 2021-04-16 10:32:22 +08:00
Jonathan Wei d28d4e8ed3
Upgrade jetty to 9.4.39.v20210325 (#11076) 2021-04-06 20:20:40 -07:00
Himanshu 43638cc6f9
license.yaml fixes for code introduced related to AWS RDS token based password provider in PR #9518 (#10885)
* license.yaml fixes for code introduced related to AWS RDS token based password provider in PR #9518

* add notice for commons-dbcp in license file

* add version and update NOTICE file
2021-03-10 12:59:25 -08:00
Yi Yuan 36e86a2880
Add protobuf schema registry (#10839)
* dd_protobuf_schema_registry

* change licese

* delete some annotation

* nodify tests

* delete extra exception

* add licenses

* add descriptor and protoMessageType in ProtobufInputRowParser for adopt to old version

* seperate kafka-protobuf-provider

* modify protobuf.md

* refine protobuf.md

* add config and header

* bug fixed

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-03-09 15:15:51 -08:00
Atul Mohan 6040c30fcd
Upgrade jetty to latest version (#10937)
* Upgrade jetty

* Fix license
2021-03-04 08:28:50 -06:00
Abhishek Agarwal 7d9a61cf7f
Suppress CVE-2017-15288 and upgrade bcprov-ext-jdk15o (#10933) 2021-03-02 16:18:27 -08:00
spinatelli 99198c02af
Add config and header support for confluent schema registry. (#10314)
* Add config and header support for confluent schema registry. (porting code from https://github.com/apache/druid/pull/9096)

* Add Eclipse Public License 2.0 to license check

* Update licenses.yaml, revert changes to check-licenses.py and dependencies for integration-tests

* Add spelling exception and remove unused dependency

* Use non-deprecated getSchemaById() and remove duplicated license entry

* Update docs/ingestion/data-formats.md

Co-authored-by: Clint Wylie <cjwylie@gmail.com>

* Added check for schema being null, as per Confluent code

* Missing imports and whitespace

* Updated unit tests with AvroSchema

Co-authored-by: Sergio Spinatelli <sergio.spinatelli.extern@7-tv.de>
Co-authored-by: Sergio Spinatelli <sergio.spinatelli.extern@joyn.de>
Co-authored-by: Clint Wylie <cjwylie@gmail.com>
2021-02-27 14:25:35 -08:00
Alexander Saydakov f930cf14d6
Use the latest Apache DataSketches release 2.0.0 (#10917)
* use the latest Apache DataSketches release 2.0.0

* updated datasketches version

Co-authored-by: AlexanderSaydakov <AlexanderSaydakov@users.noreply.github.com>
2021-02-26 07:52:00 -06:00
Benedict Jin 32e801ceab
Bump Apache Parquet from 1.11.0 to 1.11.1 (#10889) 2021-02-17 12:18:17 +08:00
Jihoon Son a2b5e01142
Bump DataSketches memory to 1.3.0 (#10789) 2021-02-04 18:39:52 -08:00
Jonathan Wei a1a49811d9
Address CVE-2020-8570, suppress CVE-2020-8554 (#10826)
* Address CVE-2020-8570, suppress CVE-2020-8554

* Update licenses.yaml
2021-02-03 15:17:06 -08:00