3e5dae9b96
Rename SegmentMetadataHolder to AvailableSegmentMetadata ( #7372 )
2019-04-14 10:19:48 -07:00
9b5c216684
Fix datasource availability sorting ( #7451 )
...
* Fix availability sorting
* Style change
2019-04-14 10:09:57 -07:00
7a38d28cf3
run pending tasks when assigned a task that is already pending ( #6991 )
...
* run pending tasks when assigned a task that is already pending
* add unit test
* fix pending tasks aren't going to run
* address comments
2019-04-14 23:37:15 +08:00
233d485ad0
Display null in SQL table cell ( #7463 )
...
* Display null if cell in table is empty
* Add null
* re-indent
2019-04-13 12:05:14 -07:00
c57e62bcd6
7153: Isolated a supplier's close by removing its invocation ( #7459 )
...
from a non-notice-processing thread
2019-04-12 19:13:08 -07:00
85f10ed0d0
Support querying realtime segments using time-ordered scan queries and fix broken scan queries without time column ( #7454 )
...
* Update scan query runner factory to accept SpecificSegmentSpec
* nit
* Sorry travis
* Improve logging and fix doc
* Bug fix
* Friendlier error msgs and tests to cover bug
* Address Gian's comments
* Fix doc
* Added tests for empty and null column list
* Style
* Fix checking wrong order (looking at query param when it should be
looking at the null-handled order)
* Add test case for null order
* Fix ScanQueryRunnerTest
* Forbidden APIs fixed
2019-04-12 19:08:34 -07:00
7d9cb6944b
Adjust BufferAggregator.get() impls to return copies ( #7464 )
...
* Adjust BufferAggregator.get() impls to return copies
* Update BufferAggregator docs, more agg fixes
* Update BufferAggregator get() doc
2019-04-12 19:04:07 -07:00
1d9450da81
Some docs optimization ( #6890 )
...
* some markdown docs optimization
* markdown escape
2019-04-12 17:30:57 -07:00
60dd75d3d9
Wrap query with limit within the web console ( #7449 )
...
* wrap with limit
* make actual menu checkbox component
2019-04-12 08:51:37 -07:00
3aae4aaf8b
Ensuring that all the controls are aligned correctly ( #7457 )
...
* making sure all the controls are inline
* update deps
2019-04-12 08:51:03 -07:00
07eb2effe9
customHeader ( #7456 )
2019-04-11 21:58:09 -07:00
3854cfd15e
SQLMetadataSegmentManager: Comments, formatting adjustments ( #7452 )
...
Follow up to #7447 .
2019-04-11 21:57:50 -07:00
2470b3279f
SQL: Fix docs for STRING_FORMAT. ( #7455 )
2019-04-11 21:57:28 -07:00
7382dac44e
Fixed grouping and sorting by status in task table ( #7440 )
...
* Fixed group by status
* Shorten code
2019-04-11 16:10:37 -07:00
c5c2bf77f0
adding host to the error display ( #7453 )
2019-04-11 14:39:57 -07:00
08d0d6d20d
chore: pin upstream java container to jdk 8 ( #7313 )
2019-04-11 13:18:29 -07:00
a517f8ce49
Coordinator: Allow dropping all segments. ( #7447 )
...
Removes the coordinator sanity check that prevents it from dropping all
segments. It's useful to get rid of this, since the behavior is
unintuitive for dev/testing clusters where users might regularly want
to drop all their data to get back to a clean slate.
But the sanity check was there for a reason: to prevent a race condition
where the coordinator might drop all segments if it ran before the
first metadata store poll finished. This patch addresses that concern
differently, by allowing methods in MetadataSegmentManager to return
null if a poll has not happened yet, and canceling coordinator runs
in that case.
This patch also makes the "dataSources" reference in
SQLMetadataSegmentManager volatile. I'm not sure why it wasn't volatile
before, but it seems necessary to me: it's not final, and it's dereferenced
from multiple threads without synchronization.
2019-04-11 08:45:38 -07:00
4ea37e2614
Add time taken by last query in SQL view ( #7421 )
...
* Add time taken by query
* Fix time to 2 dp; set state in on state change; hide time taken at first
* Refactored code: defined query result interface; more concise shorthand expression
* Use single quote
2019-04-10 23:35:47 -07:00
bbb620125d
Table pagination style change ( #7435 )
...
* Style pagination
* Removed use of important
* Removed space
* Align items
* Switched back to react table 6.8.6
* Component name change
2019-04-10 23:34:03 -07:00
408e3e1b2a
Remove select execution code from SQL planner ( #7416 )
...
* Removed select execution code from SQL planner
* Update doc
2019-04-10 22:32:57 -07:00
78e6f6fb38
Updated Javascript Affinity config docs ( #7441 )
...
Updated with hostname:port rather than IP Address.
2019-04-10 21:44:50 -07:00
15c3124e7f
add auth check exclusion for console-config.js added in #7414 ( #7442 )
2019-04-10 21:44:26 -07:00
2771ed50b0
Support Kafka supervisor adopting running tasks between versions ( #7212 )
...
* Recompute hash in isTaskCurrent() and added tests
* Fixed checkstyle stuff
* Fixed failing tests
* Make TestableKafkaSupervisorWithCustomIsTaskCurrent static
* Add doc
* baseSequenceName change
* Added comment
* WIP
* Fixed imports
* Undid lambda change for diff sake
* Cleanup
* Added comment
* Reinsert Kafka tests
* Readded kinesis test
* Readd bad partition assignment in kinesis supervisor test
* Nit
* Misnamed var
2019-04-10 18:16:38 -07:00
2f64414ade
Add "REVERSE" / "REPEAT" / "RIGHT" / "LEFT" functions ( #7334 )
...
* Add "REVERSE" / "REPEAT" / "RIGHT" / "LEFT" functions
* Fix ImportOrder
* Use RuntimeException instead of OutOfMemoryError according to "Effective Java"
* Simplify
* Patch suggestions
2019-04-10 11:46:29 +08:00
89bb43f382
'core' ORC extension ( #7138 )
...
* orc extension reworked to use apache orc map-reduce lib, moved to core extensions, support for flattenSpec, tests, docs
* change binary handling to be compatible with avro and parquet, Rows.objectToStrings now converts byte[] to base64, change date handling
* better docs and tests
* fix it
* formatting
* doc fix
* fix it
* exclude redundant dependencies
* use latest orc-mapreduce, add hadoop jobProperties recommendations to docs
* doc fix
* review stuff and fix binaryAsString
* cache for root level fields
* more better
2019-04-09 09:03:26 -07:00
09abc4a856
Fix comments in KinesisIndexTaskIOConfig ( #7430 )
2019-04-09 08:40:08 -07:00
e87d6e32b3
Support kinesis compatibility ( #7351 )
2019-04-08 19:19:34 -07:00
7cd5477658
DoublesSketchComplexMetricSerde: Handle empty strings. ( #7429 )
2019-04-08 18:01:31 -07:00
98ee8637e6
Console SQL function doc generation ( #7406 )
...
* Change sql function doc generation script from Bash to Node.js
* Minor code change for better reading
* Disable tslint in file
* clean up code
* Use es6 multiline string
* Solved a bug which would generate invalid TS variable if no function sections are found; in this case an empty array should be created
* Edit build script
2019-04-08 17:38:26 -07:00
c69c37e99d
Update console packages and enforce tslint quotes ( #7419 )
...
* update packages and enforce tslint quotes
* remove direct dep on tslint-react
2019-04-07 20:19:08 -07:00
799c66d9ac
Allow max rows and max segments for time-ordered scans to be overridden using the scan query JSON spec ( #7413 )
...
* Initial changes
* Fixed NPEs
* Fixed failing spec test
* Fixed failing Calcite test
* Move configs to context
* Validated and added docs
* fixed weird indentation
* Update default context vals in doc
* Fixed allowable values
2019-04-07 20:12:52 -07:00
7778f29781
add travis check to make sure -Pdist works ( #7418 )
2019-04-07 20:12:23 -07:00
e28a15f9f5
fix expressions docs operator table ( #7420 )
...
* fix expressions docs operator table
* Update math-expr.md
2019-04-07 20:12:00 -07:00
e23fd41fa7
Update SQL doc for planning change ( #7415 )
2019-04-05 15:14:07 -07:00
76b4a5c62e
refactor lookups to be more chill to router ( #7222 )
...
* refactor lookups to be more chill to router
* remove accidental change
* fix and combine LookupIntrospectionResourceTest
* fix inspection
* rename RouterLookupModule to LookupSerdeModule and RouterLookupExtractorFactoryContainerProvider to NoopLookupExtractorFactoryContainerProvider
* make comment generic
* use ConfigResourceFilter instead of StateResourceFilter
* fix indentation
* unused import
* another unused import
* refactor some stuff into processing module, split up LookupModule.java classes into their own files
2019-04-05 14:49:41 -07:00
a66291a9cd
SQL explain in web console ( #7402 )
...
* Add explain for SQL query
* Terminate explain query manager
* Add signature for semi-joined queries
* Extract components out
* Hide more button in rune mode
* Add types for query explanation parser
* Add type for query-plan-dialog
2019-04-05 12:44:51 -07:00
0fa122ecf2
Adding ability to pass configs in and fixing misc bugs ( #7414 )
...
* Adding ability to pass configs in and fixing misc bugs
* update lock file
* remove dead version param
2019-04-05 12:40:43 -07:00
e2aa6e47f8
change Dockerfile maintainers email to dev mailing list ( #7417 )
2019-04-04 19:34:48 -07:00
78745fea84
Fix two issues with Coordinator -> Overlord communication. ( #7412 )
...
* Fix two issues with Coordinator -> Overlord communication.
1) ClientCompactQuery needs to recognize the potential for 'intervals'
to be set instead of 'segments'. The lack of this led to a
NullPointerException on DruidCoordinatorSegmentCompactor.java:102.
2) In two locations (DruidCoordinatorSegmentCompactor,
DruidCoordinatorCleanupPendingSegments) tasks were being retrieved
using waiting/pending/running tasks in the wrong order: by checking
'running' first and then 'pending', tasks could be missed if they
moved from 'pending' to 'running' in between the two calls. Replaced
these methods with calls to 'getActiveTasks', a new method that does
the calls in the right order.
* Remove unused import.
2019-04-04 10:25:18 -07:00
d29a32062f
upgrade to RoaringBitmap 0.8.0 and serialise directly to ByteBuffer ( #7408 )
2019-04-04 13:22:50 -04:00
0f6cb1e7e0
Update theta/hll sketch doc comparison ( #7407 )
2019-04-03 15:21:33 -07:00
8c104a115c
SQL: Add STRING_FORMAT function. ( #7327 )
2019-04-03 17:09:54 -04:00
28b4e8586d
use latest sketches-core-0.13.1 ( #7320 )
...
* use latest sketches-core-0.13.0
* latest release
2019-04-03 17:06:02 -04:00
6b741f0800
Overwrite index task maxTotalRows with computed maxRowsPerSegments ( #7338 )
...
fixed identation
set maxTotalRows to higher value in case mutiple segments in appendrator
with comments
2019-04-03 11:47:59 -07:00
c883c52cb1
Fix tests ( #7401 )
2019-04-02 16:49:21 -07:00
4584b5e139
SQL support for time-ordered scan ( #7373 )
...
* Squashed commit of the following:
commit 287a367f4107503ea5c0231a72e7d91df50de321480e932fdc7fea6ac8480e932fdf487f31fcf6fb858efbb7376e8bf9068a6bb1127c35692680fc219af478c8da4fc66403b822fc73df57033f36df8f01d8dd1657033f36df8f01d8dd186d9730fc8f01d8dd16ec470288c7ec470288c786d9730fc98b3b6b51eda87d02127c62dcedacde1b46b58aec49472162b743d490cc3a42f5246b8d4823dab8952528a561427bfa77d3c1a032c46ee7e49d47397e49d47391a032c46ee057b568265445e95bb1f4cce917ab8473f4038068fb966def836dc53b311c35c96d35572d1978d57183ec3fe1f3398d398283ec3fe1f147c970b5f45ff59f5ca6806166f977de83b11a1b5bd0e1a32c18cce9a649fa649b3b18cce9a646451e2b436569b24bd851763c43df7417b9f2fe763c43df7e06a52189173b923dac9c023538d831e1fc2955d3f57ff253fa1813a5472cf83e99655db13ff624a9fba6b022f0c9142e721ccd489a020554b0142ccd489a02087baeade83235150fe1a64e69276d57ecb0f483a9f0eddee6655f92dd732593e1636287134041c4792e3577cd3dd3b335af42ab00eade9fb432beaf84b2c8c77ad485e72a614e7e872a8ebce8a4b49044305876a438e3a58f72305876a4348212a21caf10b5e0ca93dfe4aa9681148939e88b4f51024b35edbe2ae15edbe2ae1260b7684db315ccb76b60b7684db74f51024b318b7d5f5081b6d4df3864d1a1793f367deb06f6dfb7d3a490086c5eee1386c5eee13bb7d3a4900a796083f2b8bc5eaa90737a83321d796083f2bb20c36644db9e6e71616b01b25ed112432acaf08512e51a2721e66339cd76ad731a362b989bd2d50e7b584713926930f8d27b5847139479e83193837a6080f63626930f8d20dd4ec1ac9cdba6e492a010e57d5f97d4cc287310e57d5f9e
2019-04-02 15:46:01 -07:00
a99f0ff450
prefix no-op aggs with "Noop" ( #6960 )
2019-04-02 15:05:07 -07:00
4e23c11345
Make IngestSegmentFirehoseFactory splittable for parallel ingestion ( #7048 )
...
* Make IngestSegmentFirehoseFactory splittable for parallel ingestion
* Code review feedback
- Get rid of WindowedSegment
- Don't document 'segments' parameter or support splitting firehoses that use it
- Require 'intervals' in WindowedSegmentId (since it won't be written by hand)
* Add missing @JsonProperty
* Integration test passes
* Add unit test
* Remove two FIXME comments from CompactionTask
I'd like to leave this PR in a potentially mergeable state, but I still would
appreciate reviewer eyes on the questions I'm removing here.
* Updates from code review
2019-04-02 14:59:17 -07:00
78fd5aff21
support radians and degrees in sql ( #7336 )
...
* support radians and degrees in sql
* update test case
2019-04-02 12:47:49 -07:00
134f71d1b4
Add documentation for Druid native query in SQL view of web console ( #7381 )
...
* Add docmentation for Druid native query in SQL view of web console
* Edit sentence
2019-04-02 12:20:51 -07:00
Surekha
Qi Shu
Mingming Qiu
Eugene Sevastyanov
Justin Borromeo
Jonathan Wei
zhaojiandong
Vadim Ogievetsky
Gian Merlino
Don Bowman
Benjamin Hopp
Clint Wylie
Benedict Jin
Jihoon Son
Richard Startin
Alexander Saydakov
lxqfy
Atul Mohan
David Glasser
Xue Yu