b97b86aa96
Add 'Area - Metadata' tag to committer-instructions.md and stale.yml ( #8170 )
2019-07-31 15:02:35 -07:00
716ce7fdc7
Spelling Error ( #8206 )
2019-07-31 10:43:11 -07:00
faf51107d5
Add SuppressWarnings SS_SHOULD_BE_STATIC ( #8138 )
...
* Spotbugs: SS_SHOULD_BE_STATIC (#8073 )
* Add SuppressWarnings SS_SHOULD_BE_STATIC
Fixes #8073
* Fix the voilation
* Make them non-final
* Remove @Nonnull
2019-07-31 19:44:42 +03:00
63461311f8
HllSketch Merge/Build BufferAggregators: Speed up init with prebuilt sketch. ( #8194 )
...
* HllSketchMergeBufferAggregator: Speed up init by copying prebuilt sketch.
* Remove useless writableRegion call.
* POM variables.
* Fix missing reposition.
* Apply similar optimization to HllSketchBuildBufferAggregator.
* Rename emptySketch -> emptyUnion in merge flavor.
* Adjustments based on review.
* Comment update.
* Additional updates.
* Comment push.
2019-07-31 08:18:42 -07:00
15dd3d7256
Web console: better action linking between views ( #8203 )
...
* better actions
* better action linking
2019-07-31 07:13:25 -07:00
645fca53d8
Web console: segment timeline ( #8202 )
...
* Add segment timeline chart
* fix file
* Fix bugs: no data handling & scaling problems
* resolve conflict
* changed package-lock
* do not show by default
* trust the interop
* stricter type fixes
* fix sasslint
2019-07-30 22:35:30 -07:00
ddec5ea82e
Web console: Improve data loader time column detection ( #8199 )
...
* better timestamp suggestion
* misc fixes
* update dependancies and licenses
* add indexer, update snapshots
* add explicit render types
* pass through auto refresh in lookup view
2019-07-30 17:54:39 -07:00
385f492a55
Use PartitionsSpec for all task types ( #8141 )
...
* Use partitionsSpec for all task types
* fix doc
* fix typos and revert to use isPushRequired
* address comments
* move partitionsSpec to core
* remove hadoopPartitionsSpec
2019-07-30 17:24:39 -07:00
aba65bb675
removed hard-coded Kafka key and value deserializer ( #8112 )
...
* removed hard-coded Kafka key and value deserializer, leaving default deserializer as org.apache.kafka.common.serialization.ByteArrayDeserializer. Also added checks to ensure that any provided deserializer class extends org.apache.kafka.serialization.Deserializer and outputs a byte array.
* Addressed all comments from original pull request and also added a
unit test.
* Added additional test that uses "poll" to ensure that custom deserializer
works properly.
2019-07-30 16:25:32 -07:00
653b558134
sql firehose and firehose doc adjustments ( #8067 )
...
* firehose doc adjustments
* fix typo
* additional information on parser types in ingestion docs
* clarify ingest segment firehose docs, add sql firehose examples to sql extension pages
* fixit
* make sql firehose more forgiving my always constructing a MapInputRowParser from the parseSpec of whatever actual InputRowParser impl is provided, remove doc references to map based parsers
* transforms
* fix tests
2019-07-30 15:28:10 -07:00
4e60afc86e
Web-console: add Supervisor statistics table and show json query manager ( #8176 )
...
* add statistics table
* change error message
* add query managers
* update snapshots
* fixes
* add stricter types
* fix error type
* prettier fic
* fix undefined
* add typing
* fixe query manager
2019-07-30 12:53:46 -07:00
0f8c902e94
Spotbugs fix DM_NUMBER_CTOR ( #8072 )
2019-07-30 20:08:27 +03:00
e016995d1f
Enable Spotbugs: WMI_WRONG_MAP_ITERATOR ( #8005 )
...
* WMI_WRONG_MAP_ITERATOR
* Fixed missing loop
2019-07-30 19:51:53 +03:00
fb653ceef9
Add benchmark for VersionedIntervalTimeline ( #8161 )
...
* Add benchmark for VersionedIntervalTimeline
* rename
2019-07-30 08:10:00 -07:00
f7abc300a8
Web-console: add loader to show-json, add status-dialog ( #8172 )
...
* add loader to show json, add status-dialog
* fix ordered import
* fix changes
* order inputs
* update snapshots
* fix bad merge conflicts
2019-07-30 08:08:36 -07:00
a0270512a2
Web-console: add history menu to supervisors modal ( #8164 )
...
* Add history menu to supervisors dialog
* add scrolling
* change how tabs render
* remove copy button
* user query manager
2019-07-30 08:07:58 -07:00
80a0fbf254
fix circular reference in moving average query ( #8192 )
2019-07-29 22:31:44 -07:00
771c04841e
remove husky hooks ( #8193 )
2019-07-29 19:32:16 -07:00
640b7afc1c
Add CliIndexer process type and initial task runner implementation ( #8107 )
...
* Add CliIndexer process type and initial task runner implementation
* Fix HttpRemoteTaskRunnerTest
* Remove batch sanity check on PeonAppenderatorsManager
* Fix paralle index tests
* PR comments
* Adjust Jersey resource logging
* Additional cleanup
* Fix SystemSchemaTest
* Add comment to LocalDataSegmentPusherTest absolute path test
* More PR comments
* Use Server annotated with RemoteChatHandler
* More PR comments
* Checkstyle
* PR comments
* Add task shutdown to stopGracefully
* Small cleanup
* Compile fix
* Address PR comments
* Adjust TaskReportFileWriter and fix nits
* Remove unnecessary closer
* More PR comments
* Minor adjustments
* PR comments
* ThreadingTaskRunner: cancel task run future not shutdownFuture and remove thread from workitem
2019-07-29 17:06:33 -07:00
cc4450db12
Web console: add reindex (ingestSegment firehose) to the data loader ( #8181 )
...
* tidy up nulls
* standardize more on undefined
* updated licenses
* do not do heavy handed rendering
* reindex from druid
* tidy up
* add inline firehose
* add husky
* sass lint
* better suggestion
* fix script typo
* adjust time formats
* add missing time formats
* use term 'reindex'
* fix lodash.compact
2019-07-29 14:41:27 -07:00
ab71a2e1e4
Revert "Fix dependency analyze warnings ( #8128 )" ( #8189 )
...
This reverts commit 5dd0d8e873
2019-07-29 11:42:16 -07:00
0b3ede193e
Web console: adding Slack channel integration ( #8182 )
...
* adding slack channel integration
* update snapshots
2019-07-28 16:17:54 -07:00
8bd0f8c2ac
Web console: code quality improvements (null tidy up) ( #8162 )
...
* tidy up nulls
* standardize more on undefined
* updated licenses
* update snapshot
* do not do heavy handed rendering
* add placeholder to SQL view
* remove pointelss fragment
2019-07-27 01:46:37 -07:00
adf7bafb9f
Fix race between canHandle() and addSegment() in StorageLocation ( #8114 )
...
* Fix race between canHandle() and addSegment() in StorageLocation
* add comment
* add comments
* fix test
* address comments
* remove <p> tag from javadoc
* address comments
* comparingLong
2019-07-27 11:11:06 +03:00
5dd0d8e873
Fix dependency analyze warnings ( #8128 )
...
* Fix dependency analyze warnings
Update the maven dependency plugin to the latest version and fix all
warnings for unused declared and used undeclared dependencies in the
compile scope. Added new travis job to add the check to CI. Also fixed
some source code files to use the correct packages for their imports.
* Fix licenses and dependencies
* Fix licenses and dependencies again
* Fix integration test dependency
* Address review comments
* Fix unit test dependencies
* Fix integration test dependency
* Fix integration test dependency again
* Fix integration test dependency third time
* Fix integration test dependency fourth time
* Fix compile error
* Fix assert package
2019-07-26 10:49:03 -07:00
7b9bd5359d
[statsd-emitter] Fix ingest persists metrics typo ( #8160 )
2019-07-26 07:20:41 -07:00
31a29d8883
add noop type name to prevent jackson exception when setting type to noop ( #8133 )
2019-07-25 16:07:08 -07:00
ef9230f916
Web-console: Add continue to spec view ( #8144 )
...
* Add continue to spec view
* update spec when type selected
* remove redundancy
* fix mistakes
2019-07-25 14:48:26 -07:00
8ba1f06632
Web-console: add lookups tile to home view ( #8124 )
...
* add lookups-tile
* add uninitailized state
* remove error:
2019-07-25 13:08:58 -07:00
61f4abece4
Add more warning to the doc for resetOffsetAutomatically ( #8153 )
...
* Add more warnings to the doc for resetOffsetAutomatically
* fix kinesis doc
* fix typos
* revise the description
* capital
* capitalize
2019-07-24 17:37:32 -07:00
db14946207
Add support minor compaction with segment locking ( #7547 )
...
* Segment locking
* Allow both timeChunk and segment lock in the same gruop
* fix it test
* Fix adding same chunk to atomicUpdateGroup
* resolving todos
* Fix segments to lock
* fix segments to lock
* fix kill task
* resolving todos
* resolving todos
* fix teamcity
* remove unused class
* fix single map
* resolving todos
* fix build
* fix SQLMetadataSegmentManager
* fix findInputSegments
* adding more tests
* fixing task lock checks
* add SegmentTransactionalOverwriteAction
* changing publisher
* fixing something
* fix for perfect rollup
* fix test
* adjust package-lock.json
* fix test
* fix style
* adding javadocs
* remove unused classes
* add more javadocs
* unused import
* fix test
* fix test
* Support forceTimeChunk context and force timeChunk lock for parallel index task if intervals are missing
* fix travis
* fix travis
* unused import
* spotbug
* revert getMaxVersion
* address comments
* fix tc
* add missing error handling
* fix backward compatibility
* unused import
* Fix perf of versionedIntervalTimeline
* fix timeline
* fix tc
* remove remaining todos
* add comment for parallel index
* fix javadoc and typos
* typo
* address comments
2019-07-24 17:35:46 -07:00
714a2726a7
Spotbugs: RC_REF_COMPARISON_BAD_PRACTICE_BOOLEAN ( #8076 )
...
* Spotbugs: RC_REF_COMPARISON_BAD_PRACTICE_BOOLEAN
* Use Objects instead
2019-07-24 16:17:02 -07:00
c87b47e0fa
More documentation formatting fixes ( #8149 )
...
Add empty lines before bulleted lists and code blocks, to ensure that
they show up properly on the web site. See also #8079 .
2019-07-24 15:26:03 -07:00
0695e487e7
fix issue with CuratorLoadQueuePeon shutting down executors it does not own ( #8140 )
...
* fix issue with CuratorLoadQueuePeon shutting down executors it does not own
* use lifecycled executors
* maybe this
2019-07-24 10:59:43 -07:00
799d20249f
Response context refactoring ( #8110 )
...
* Response context refactoring
* Serialization/Deserialization of ResponseContext
* Added java doc comments
* Renamed vars related to ResponseContext
* Renamed empty() methods to createEmpty()
* Fixed ResponseContext usage
* Renamed multiple ResponseContext static fields
* Added PublicApi annotations
* Renamed QueryResponseContext class to ResourceIOReaderWriter
* Moved the protected method below public static constants
* Added createEmpty method to ResponseContext with DefaultResponseContext creation
* Fixed inspection error
* Added comments to the ResponseContext length limit and ResponseContext
http header name
* Added a comment of possible future refactoring
* Removed .gitignore file of indexing-service
* Removed a never-used method
* VisibleForTesting method reducing boilerplate
Co-Authored-By: Clint Wylie <cjwylie@gmail.com>
* Reduced boilerplate
* Renamed the method serialize to serializeWith
* Removed unused import
* Fixed incorrectly refactored test method
* Added comments for ResponseContext keys
* Fixed incorrectly refactored test method
* Fixed IntervalChunkingQueryRunnerTest mocks
2019-07-24 18:29:03 +03:00
0388581493
Revert "Spotbugs: SS_SHOULD_BE_STATIC ( #8073 )" ( #8145 )
...
This reverts commit 04a180a5fb
2019-07-23 22:57:19 -07:00
b8b22b7aaa
fix references to bin/supervise in tutorial docs ( #8087 )
2019-07-23 15:05:01 -07:00
cb82d72547
fix forking task runner task shutdown to be more graceful ( #8085 )
...
* fix forking task runner shutdown to be more graceful
* javadoc
2019-07-23 11:47:23 -07:00
83514958db
remove unnecessary lock in ForegroundCachePopulator leading to a lot of contention ( #8116 )
...
* remove unecessary lock in ForegroundCachePopulator leading to a lot of contention
* mutableboolean, javadocs,document some cache configs that were missing
* more doc stuff
* adjustments
* remove background documentation
2019-07-23 10:57:59 -07:00
04a180a5fb
Spotbugs: SS_SHOULD_BE_STATIC ( #8073 )
2019-07-23 18:18:49 +08:00
b2867c9d06
web console added to docker-compose ( #8117 )
2019-07-22 12:40:05 -06:00
e1a745717e
Spotbugs: NP_STORE_INTO_NONNULL_FIELD ( #8021 )
2019-07-21 21:23:47 +08:00
f24e2f16af
fix npe with sql metadata manager polling and empty database ( #8106 )
...
* fix npe with sql metadata manager polling and empty database
* treat null segments separately
* use preconditions check
* add test
2019-07-20 19:09:02 -07:00
2b2fcc0371
Web console: cleanup build, check licenses in test ( #8113 )
...
* cleanup build, check licenses
* cleanup filterMap type
* don't run license check twice
2019-07-19 22:59:07 -07:00
ea4bad7836
Druid SQL EXTRACT time function - adding support for additional Time Units ( #8068 )
...
* 1. Added TimestampExtractExprMacro.Unit for MILLISECOND 2. expr eval for MILLISECOND 3. Added a test case to test extracting millisecond from expression. #7935
* 1. Adding DATASOURCE4 in tests. 2. Adding test TimeExtractWithMilliseconds
* Fixing testInformationSchemaTables test
* Fixing failing tests in DruidAvaticaHandlerTest
* Adding cannotVectorize() call before the test
* Extract time function - Adding support for MICROSECOND, ISODOW, ISOYEAR and CENTURY time units, documentation changes.
* Adding MILLISECOND in test case
* Adding support DECADE and MILLENNIUM, updating test case and documentation
* Fixing expression eval for DECADE and MILLENIUM
2019-07-19 20:38:32 -07:00
ceb3a891bb
Fix druid sql group by queries returning complex aggregation type ( #8099 )
...
* Fix druid sql group by queries returning complex aggregation type
* Remove unnecessary check
2019-07-19 13:52:14 -07:00
01f3da6fda
Web Console: add Group by interval to segments view ( #7986 )
...
* add group by
* update snapshot
* fix spacing
* change design pattern
* rename interface
* add static function
* save
* add whereClause"
* add default page size
* add where to inner query
* snapshots
2019-07-19 11:25:25 -07:00
54a7b54d2d
avoid 'must return non-void type' warning ( #8105 )
2019-07-18 15:02:27 -07:00
c7eb7cd018
Add intermediary data server for shuffle ( #8088 )
...
* Add intermediary data server for shuffle
* javadoc
* adjust timeout
* resolved todo
* fix test
* style
* address comments
* rename to shuffleDataLocations
* Address comments
* bit adjustment StorageLocation
* fix test
* address comment & fix test
* handle interrupted exception
2019-07-18 14:46:47 -07:00
03e55d30eb
add CachingClusteredClient benchmark, refactor some stuff ( #8089 )
...
* add CachingClusteredClient benchmark, refactor some stuff
* revert WeightedServerSelectorStrategy to ConnectionCountServerSelectorStrategy and remove getWeight since felt artificial, default mergeResults in toolchest implementation for topn, search, select
* adjust javadoc
* adjustments
* oops
* use it
* use BinaryOperator, remove CombiningFunction, use Comparator instead of Ordering, other review adjustments
* rename createComparator to createResultComparator, fix typo, firstNonNull nullable parameters
2019-07-18 13:16:28 -07:00
Roman Leventov
Nathan
Fokko Driesprong
Gian Merlino
Vadim Ogievetsky
Jihoon Son
Aaron Bossert
Clint Wylie
mcbrewster
Eyal Yurman
Jonathan Wei
Chi Cao Minh
Alexandre YANG
Parag Jain
Magnus Henoch
Eugene Sevastianov
ilhanadiyaman
Sashidhar Thallam
Samarth Jain
Himanshu