Commit Graph

9766 Commits

Author SHA1 Message Date
Himanshu 1fe4ecf17a
StringDictionaryEncodedColumn dimSelector to return CARDINALITY_UNKNOWN with extractionFn (#8433)
* update DimensionDictionarySelector.getValueCardinality() javadoc

* unknown cardinality in StringDictionaryEncodedColumn dim selector

* revert StringDictionaryEncodedColumn change as that fails GroupBy-v1 execution for many working queries

* fix/add more comments
2019-09-06 14:19:25 -07:00
Rye 645799f977 disallow whitespace characters except space in data source names (#8465)
* disallow whitespace characters in data source names

* wrapped preconditions in a function, and simplify unit tests code

* Fixed regex to allow space, simplified repeat logic

* Fixed import style against mvn checkstyle

* Add msg in case test fails, use emptyMap(), improved naming

* Changes on assertion functions

* change wording of "whitespace" to "whitespace except space" to avoid misleading
2019-09-06 08:55:21 -07:00
Vadim Ogievetsky b8dc6a94b3 Web console: fix datasource name auto detection (#8479)
* fix datasource name guessing

* fix comment
2019-09-06 00:57:26 -07:00
Clint Wylie fd58fbc8d3
fix statds dogstatsdServiceAsTag docs example to match behavior (#8477) 2019-09-05 19:05:25 -07:00
Alexandre Yang 7f28561d17 Fix service tag by avoiding using a reserved tag name (#8472) 2019-09-05 15:43:09 -07:00
Vadim Ogievetsky bffaca672a swap ordering clauses in the menu (#8473) 2019-09-05 14:59:28 -07:00
Vadim Ogievetsky 85119b8721 fix iPad width (#8471) 2019-09-05 13:56:23 -07:00
SeKing 6a6893b406 Fix operator mistake of expression OR (#8452)
* Add realization for updating version of derived segments in MaterializedView

* add unit test, and change code style for the sake of ease of understanding

* fix document's mistake of expression
2019-09-04 21:27:18 -07:00
Lucas Capistrant bfb02f09f8 Add druid.segmentCache.numBootstrapThreads back to the docs (#8462) 2019-09-04 20:27:17 -07:00
Jonathan Wei f36fd73f60 Speed up StringDimensionIndexer.estimateEncodedKeyComponentSize (#8466)
* Speed up StringDimensionIndexer.estimateEncodedKeyComponentSize

* Remove print

* Move benchmark, add header
2019-09-04 20:26:04 -07:00
Benedict Jin de18840412
Fix inconsistent equals and hashCode (#8381)
* Fix inconsistent equals and hashCode

* Patch comments

* Remove equals and hashCode from InsensitiveContainsSearchQuerySpec
2019-09-04 13:48:08 +08:00
Himanshu ee4ebb496a
make single/multi value string column handling official in aggregation (#8428) 2019-09-03 13:47:09 -07:00
Vadim Ogievetsky 637a9e8614 Web console: Fix segment re-ingest (#8454)
* fixing ingest spec

* rm console.log

* ingest segment spec

* make sure step query always being run

* better example interface

* add keywords

* placeholders

* do not overwrite datasource name in data loader spec

* fix comment typo
2019-09-03 12:13:57 -07:00
Fokko Driesprong abd86467f8 Bump ORC library to 1.5.6 (#8405)
Changelog at:
https://orc.apache.org/docs/releases.html#current-release---156
2019-09-02 02:24:31 -07:00
SeKing 31732f0e21 Add realization for updating version of derived segments in Materiali… (#8281)
* Add realization for updating version of derived segments in MaterializedView

* add unit test, and change code style for the sake of ease of understanding
2019-09-02 01:24:53 -07:00
Grace Koo 58fbb69113 Fix hump nomenclature (#8447) 2019-09-01 00:07:29 +08:00
legendtkl 0be4a41c06 Website Doc: fix bash command (#8442)
* fix "gunzip -k" to "gunzip -c"
2019-08-30 22:22:09 -07:00
Clint Wylie 984958122b
packaging script adjustments (#8436)
* set encoding for license and notice scripts, split generate-license.py into generate-binary-license.py and check-licenses.py, check-licenses when -Papache-release is used

* missing docs

* doc fix

* more doc fix

* remove comments

* good catch travis +1

* fix lgtm alerts
2019-08-29 23:27:43 -07:00
Vadim Ogievetsky f55e1be80b Web console: fix task log tailing (#8434)
* fix log tailing

* update snapshots
2019-08-29 14:15:06 -07:00
mcbrewster beed07022a Web-Console: Increase snapshot coverage (#8408)
* increase test-coverage

* fix ordered inputs

* fixes

* prettier fix

* fixes
2019-08-29 08:41:34 -07:00
Xavier Léauté 4b69ce0f09 enable unit tests with JDK11 (#8400)
* enable unit tests with JDK11

This enables unit tests with openjdk11, splitting up the build into
stages to have it fail faster

The integration test docker image still uses openjdk8, so there is
little reason to run those tests with JDK11 yet

* remove stages
2019-08-28 10:29:13 -07:00
Chi Cao Minh 31e6280b75 Use Codecov (#8388)
* Use Codecov

Upload coverage reports to Codecov. For now, having Codecov comment on
PRs or enforcing a minimum coverage threshold are both disabled until
the Codecov coverage reports look reliable:
https://codecov.io/gh/apache/incubator-druid

* Split bash and curl into separate lines
2019-08-28 08:49:30 -07:00
Clint Wylie 6327cb3d4c add missing step in release process (#8422) 2019-08-28 08:34:17 -07:00
Clint Wylie 3baf31e9a8 add documentation for group by array based result format (#8416) 2019-08-28 08:30:31 -07:00
SandishKumarHN f4e2902226 Add Checkstyle check for String literal equality (#8386) 2019-08-28 17:53:42 +03:00
Clint Wylie c73a489335
bump master version to 0.17.0-incubating-SNAPSHOT (#8421) 2019-08-28 01:58:36 -07:00
Jonathan Wei cd750463fa Fix default CliIndexer TLS port (#8415) 2019-08-27 19:23:20 -07:00
Clint Wylie 7afe473fd3 How to asf release (#8370)
* add ASF release manager guide

* fix broken link

* fix bold

* fix order

* clean up

* oops

* pom

* more

* fix

* fixes

* fix

* fix
2019-08-27 18:36:13 -07:00
Himanshu 5c3db41c2b
string column handling for long/float min/max/sum aggregators (#8319)
* string column handling for long min/max/sum aggregators

* add apache license to new files

* use 'L' as suffix for long literal instead of 'l'

* return null in ParallelCombiner.SettableColumnSelectorFactory.getColumnCapabilities(String) as is required by contract of ColumnSelectorFactory interface

* fix more tests
2019-08-27 16:10:59 -07:00
Himanshu d5d170f866
skip unnecessary aggregate(..) calls with LimitedBufferHashGrouper (#8412)
* skip unnecessary aggregate(..) calls with LimitedBufferHashGrouper

* remove unused bucketWasUsed arg from canSkipAggregate(..)
2019-08-27 15:01:07 -07:00
Himanshu 4d87a19547
Logging emitter to publish query and other metric events as valid json objects (#8359)
* LoggingEmitter: print event as json

* use DefaultRequestLogEventBuilderFactory in emitting request logger by default

* print context in query metric as json

* removed unused jsonMapper from DefaultQueryMetrics

* add comment

* remove change to DefaultRequestLogEventBuilderFactory.java
2019-08-27 15:00:23 -07:00
Clint Wylie 44dd5b5f0d add jaxb-runtime to fix exception with newer versions of java (#8409)
* add jaxb-runtime to fix exception with jdk9+

* fix licenses

* oops
2019-08-27 14:25:05 -06:00
Chi Cao Minh d8b81f4fd9 Speed up javascript Travis CI jobs (#8361)
* Speed up javascript Travis CI jobs

Skip mvn install for javascript CI jobs since it is not needed.

* Specify base filepath for source files

* Remove coveralls
2019-08-27 12:03:48 -07:00
Jihoon Son e5ef5ddafa Fix the shuffle with TLS enabled for parallel indexing; add an integration test; improve unit tests (#8350)
* Fix shuffle with tls enabled; add an integration test; improve unit tests

* remove debug log

* fix tests

* unused import

* add javadoc

* rename to getContent
2019-08-26 19:27:41 -07:00
Benedict Jin 95f727f66c
Add docker shield (#8403) 2019-08-27 10:04:51 +08:00
Chi Cao Minh a4b842ac2e Speed up ExpressionSelectors.makeExprEvalSelector (#8373)
* Speed up ExpressionSelectors.makeExprEvalSelector

Addresses performance bottlenecks observed when running a search query
with an expression filter and granularity set to none by caching and
changing streams to for-loops. After changes, the query was observed to
run 2-3x faster.

Also fixes various IntelliJ inspection warnings.

* Fix static analysis errors
2019-08-26 14:34:16 -07:00
Dylan Wylie b2821a8371 do not exclude client core jar (#8339)
make indexing service depend on hadoop client
2019-08-26 13:48:24 -07:00
Vadim Ogievetsky 894b4efb36 Web console: Fix segment timeline styling (#8402)
* Fix segment timeline styling

* update snapshot
2019-08-25 21:31:20 -07:00
Vadim Ogievetsky 9254dc8b80 Web console: Save query context also (#8395)
* tidy up menus

* fix query output API

* save context also

* pull out auto run into a switch

* better copy

* add group_id

* support FLOAT also

* use built in time logic

* fix trunc direction

* add skipCache

* add manifest url

* remove depricated props
2019-08-25 16:56:27 -07:00
Xavier Léauté 496dfa3b15
fix JVMMonitor initialization with JDK11 (#8397)
JVMMonitor requires access to jdk.internal.perf.Perf to enable GC
counters, which requires additional JVM arguments with JDK11.

This change adds a fallback in case GC counters cannot be initialized,
and logs a warning message explaining how GC counters can be enabled.
2019-08-25 08:26:58 -04:00
Jonathan Wei c626452b47 Add nano-quickstart single server example configuration (#8390)
* Add nano-quickstart single server example configuration

* Use two workers

* Shrink processing buffers
2019-08-24 22:07:20 -07:00
Furkan KAMACI 02fe3db911 Zookeeper version is updated. (#8363)
* Zookeeper version is updated.

* Zookeeper version is updated at licenses.yaml

* licenses.yaml is updated and dependencies are fixed to make the project successfully build.

* Zookeeper versions are fixed at licenses.yaml
2019-08-24 22:00:43 -07:00
Xavier Léauté d5e3c53e74 workaround for Guava 16 bug using Java 9 and above 2019-08-25 00:58:59 -04:00
Xavier Léauté 8e0c307e54
Do not assume system classloader is URLClassLoader in Java 9+ (#8392)
* Fallback to parsing classpath for hadoop task in Java 9+
In Java 9 and above we cannot assume that the system classloader is an
instance of URLClassLoader. This change adds a fallback method to parse
the system classpath in that case, and adds a unit test to validate it matches
what JDK8 would do.

Note: This has not been tested in an actual hadoop setup, so this is mostly
to help us pass unit tests.

* Remove granularity test of dubious value
One of our granularity tests relies on system classloader being a URLClassLoaders to
catch a bug related to class initialization and static initializers using a subclass (see
#2979)
This test was added to catch a potential regression, but it assumes we would add back
the same type of static initializers to this specific class, so it seems to be of dubious value
as a unit test and mostly serves to illustrate the bug.

relates to #5589
2019-08-24 20:47:54 -04:00
Xavier Léauté 20f7db5d22
Fix ConcurrentModificationException in JDK11 (#8391)
When building column/dimension selectors, calling computeIfAbsent can
cause the applied function to modify the same cache through virtual
column references. The JDK11 map implementation detects this change and
will throw an exception.

This fix – while not as elegant – breaks the single call into two
steps to avoid this problem.
2019-08-24 18:24:50 -04:00
Xavier Léauté 5c7803fe6b fix powermock classloader issues with Java 9 and above 2019-08-24 18:20:52 -04:00
Vadim Ogievetsky c948422f32 Web console: Better build experience (#8389)
* rename sql docs

* rm old file

* add old files
2019-08-24 11:35:30 -07:00
Vadim Ogievetsky efbf1a5822 Restore old console (#8387) 2019-08-23 17:48:11 -07:00
Jihoon Son 95fa609615 Fix wrong partitionsSpec type names in the document (#8297)
* Fix wrong type names for partitionsSpec

* add unit tests; add json properties for backward compatibility

* beautify conf names

* remove maxRowsPerSegment from hashed partitionsSpec

* fix doc build
2019-08-23 13:44:58 -07:00
Clint Wylie 7749571a7f order and add more ports to hadoop docker container in hadoop indexing tutorial (#8329)
LGTM
2019-08-23 15:43:06 -05:00