Commit Graph

176 Commits

Author SHA1 Message Date
Jihoon Son 4046c86d62
Stateful auto compaction (#8573)
* Stateful auto compaction

* javaodc

* add removed test back

* fix test

* adding indexSpec to compactionState

* fix build

* add lastCompactionState

* address comments

* extract CompactionState

* fix doc

* fix build and test

* Add a task context to store compaction state; add javadoc

* fix it test
2019-10-15 22:57:42 -07:00
Vadim Ogievetsky 6c609293b2 show full value in SQL view (#8660) 2019-10-10 14:16:11 -07:00
Vadim Ogievetsky 4c215b417e Web console: misc bug fixes and tidy up (#8654)
* fix updates

* fix status dialog

* fix scan query deserialization

* extract error message

* update snapshot
2019-10-10 10:52:46 -07:00
Jihoon Son 96d8523ecb Use hash of Segment IDs instead of a list of explicit segments in auto compaction (#8571)
* IOConfig for compaction task

* add javadoc, doc, unit test

* fix webconsole test

* add spelling

* address comments

* fix build and test

* address comments
2019-10-09 11:12:00 -07:00
Evan Ren 17d9d7daed Increase column size for taskID and createdTime, and decrease Type and Duration (#8594) 2019-09-27 14:19:04 -07:00
Evan Ren 0467cce7a0 Web console: Add frontend buttons to remove group by (#8537)
* Add frontend buttons to remove group by

* Change icon for remove group by

* Update web console to use latest toolkit

* Add test cases to verify that remove group by buttons are rendered

* Correct mistake of using incorrect components

* Update tests for two cases

* Put remove button after group by
2019-09-24 16:32:02 -07:00
Vadim Ogievetsky 7c14fa08f8 Web console: Expand filter UI (#8579)
* add controls to the filter UI

* fix double base
2019-09-24 12:27:46 -07:00
Vadim Ogievetsky 2104cee79b Web console: prevent extra trim in auto complete (#8543)
* prevent extra trim in auto complete

* add unit test
2019-09-22 15:18:51 -07:00
Vadim Ogievetsky 868bb42301 clean up default values and add infos (#8567) 2019-09-21 21:00:21 -07:00
Vadim Ogievetsky 36a6365d9f Web console: polish the data loader (#8554)
* rearrange ioConfig prop ordering in Tune step

* make useEarliestOffset mandatory

* required intent

* make segmentGranularity requred for batch ingestion

* update tests
2019-09-18 20:53:18 -07:00
Evan Ren 8650ee9fd0 Web console: Druid status displayed in a table (#8484)
* Retrieved data from endpoint and displayed on table

* Added view raw button, removed json, and fixed formatting for table

* Remove error var

* Fixed snapshot for updated status dialog

* Made changes based on PR review

* Added version tag

* Updated snapshot to match changes

* Made more changes based on review

* Fix filter and formatting

* Fix filter and add unit test

* fix styling of dialog

* Fix footer height

* Fixed testing of filtering
2019-09-17 15:26:51 -06:00
Vadim Ogievetsky 6c7f36d364 fix spec dialog highlighting (#8533) 2019-09-13 15:24:08 -07:00
Vadim Ogievetsky ad72f36360 fix forceGuaranteedRollup tooltip (#8529) 2019-09-12 17:24:29 -07:00
Vadim Ogievetsky a6eca5e935 Web console: Force intervals config (#8514)
* make sure intervals are required

* all truncated values everywhere

* continue to spec when going from tasks table

* remove unused thigns

* fix alert
2019-09-12 00:35:04 -07:00
Vadim Ogievetsky 1498250204 Fix snitch dialog final step (#8506) 2019-09-11 12:44:16 -07:00
Vadim Ogievetsky 2b04c22bfd split segment view query into two (#8485) 2019-09-06 21:32:39 -07:00
Vadim Ogievetsky b8dc6a94b3 Web console: fix datasource name auto detection (#8479)
* fix datasource name guessing

* fix comment
2019-09-06 00:57:26 -07:00
Vadim Ogievetsky bffaca672a swap ordering clauses in the menu (#8473) 2019-09-05 14:59:28 -07:00
Vadim Ogievetsky 85119b8721 fix iPad width (#8471) 2019-09-05 13:56:23 -07:00
Vadim Ogievetsky 637a9e8614 Web console: Fix segment re-ingest (#8454)
* fixing ingest spec

* rm console.log

* ingest segment spec

* make sure step query always being run

* better example interface

* add keywords

* placeholders

* do not overwrite datasource name in data loader spec

* fix comment typo
2019-09-03 12:13:57 -07:00
Vadim Ogievetsky f55e1be80b Web console: fix task log tailing (#8434)
* fix log tailing

* update snapshots
2019-08-29 14:15:06 -07:00
mcbrewster beed07022a Web-Console: Increase snapshot coverage (#8408)
* increase test-coverage

* fix ordered inputs

* fixes

* prettier fix

* fixes
2019-08-29 08:41:34 -07:00
Chi Cao Minh 31e6280b75 Use Codecov (#8388)
* Use Codecov

Upload coverage reports to Codecov. For now, having Codecov comment on
PRs or enforcing a minimum coverage threshold are both disabled until
the Codecov coverage reports look reliable:
https://codecov.io/gh/apache/incubator-druid

* Split bash and curl into separate lines
2019-08-28 08:49:30 -07:00
Clint Wylie c73a489335
bump master version to 0.17.0-incubating-SNAPSHOT (#8421) 2019-08-28 01:58:36 -07:00
Clint Wylie 7afe473fd3 How to asf release (#8370)
* add ASF release manager guide

* fix broken link

* fix bold

* fix order

* clean up

* oops

* pom

* more

* fix

* fixes

* fix

* fix
2019-08-27 18:36:13 -07:00
Vadim Ogievetsky 894b4efb36 Web console: Fix segment timeline styling (#8402)
* Fix segment timeline styling

* update snapshot
2019-08-25 21:31:20 -07:00
Vadim Ogievetsky 9254dc8b80 Web console: Save query context also (#8395)
* tidy up menus

* fix query output API

* save context also

* pull out auto run into a switch

* better copy

* add group_id

* support FLOAT also

* use built in time logic

* fix trunc direction

* add skipCache

* add manifest url

* remove depricated props
2019-08-25 16:56:27 -07:00
Vadim Ogievetsky c948422f32 Web console: Better build experience (#8389)
* rename sql docs

* rm old file

* add old files
2019-08-24 11:35:30 -07:00
Vadim Ogievetsky efbf1a5822 Restore old console (#8387) 2019-08-23 17:48:11 -07:00
Vadim Ogievetsky 20ea90a5a6 Web console: more explicit limit on run button (#8378)
* update sql doc parsing

* keyword fixes

* fix header default

* tidy

* fix tests
2019-08-23 13:40:48 -07:00
mcbrewster bfbae76031 Web-console: small fixes (#8376)
* fix column tree

* move remove filter

* fix lgtm

* remove from state

* stop mutating state diirectly

* fix column-tree

* remove component did mount

* remove expanded node
2019-08-22 21:54:11 -07:00
mcbrewster 5274c5ab73 Web-Console: Replace from clause (#8371)
* add replace from

* add failture state

* fix uninential downgrade

* use uppercase for SQL keywords
2019-08-22 09:09:37 -07:00
Vadim Ogievetsky aa51668ea7 Web console: LGTM alert squash + final fixes (#8351)
* mv old console

* update dep

* reduce card size

* fix tuning configs for forceGuaranteedRollup

* fix tests

* revert console mv

* fix some old console issues

* revert build change
2019-08-21 13:52:48 -07:00
Benedict Jin e1f94a5e26 Reduce the size of images with lossless compression (#8358) 2019-08-21 13:29:30 -07:00
Gian Merlino d007477742
Docusaurus build framework + ingestion doc refresh. (#8311)
* Docusaurus build framework + ingestion doc refresh.

* stick to npm instead of yarn

* fix typos

* restore some _bin

* Adjustments.

* detect and fix redirect anchors

* update anchor lint

* Web-console: remove specific column filters (#8343)

* add clear filter

* update tool kit

* remove usless check

* auto run

* add %

* Fix resource leak (#8337)

* Fix resource leak

* Patch comments

* Enable Spotbugs NP_NONNULL_RETURN_VIOLATION (#8234)

* Fixes from PR review.

* Fix more anchors.

* Preamble nix.

* Fix more anchors, headers

* clean up placeholder page

* add to website lint to travis config

* better broken link checking

* travis fix

* Fixed more broken links

* better redirects

* unfancy catch

* fix LGTM error

* link fixes

* fix md issues

* Addl fixes
2019-08-20 21:48:59 -07:00
mcbrewster e4aa7fb268 Web-console: fix alerts from lgtm (#8346)
* fix alerts from lgtm

* remove unordered imports

* fix introduced alert

* move getExpanded to renderSchemaSelector

* use getDerivedStateFromProps

* use prevState

* add semi colons, remove unused imports

* fixes
2019-08-20 20:40:47 -07:00
mcbrewster e64070b8bf Web-console: remove specific column filters (#8343)
* add clear filter

* update tool kit

* remove usless check

* auto run

* add %
2019-08-19 19:54:18 -07:00
Vadim Ogievetsky 56e440383f Web console: data loader native batch shuffle UI (#8229)
* data loader shuffle UI

* better support for inline firehose

* feedback changes

* make numShards mandatory sometimes

* edit context dialog fixes

* make auto form less agro

* download full log

* improve auto column detection

* add sample for example
2019-08-19 13:27:34 -07:00
mcbrewster cae1361c32 Web-console: gate auto complete on current table and schema (#8322)
* gate auto complete on current table and schema

* reset defaults

* add static functions

* move completions to state
2019-08-17 09:34:36 -07:00
Jihoon Son 31af4eb9ad
Rename maxNumSubTasks to maxNumConcurrentSubTasks for native parallel index task (#8324) 2019-08-16 15:57:13 -07:00
Vadim Ogievetsky 0d69438395 Web console: make console's Explain more honest (#8327)
* fix structure

* update snapshots
2019-08-16 15:51:49 -07:00
mcbrewster 8924d285dc Web-Console: add more side column functions to query view (#8283)
* small fixes

* run jest -u

* add trim

* add to groupby

* save

* add functionality to column tree menu

* remove history dialog

* rename file

* fixes

* add new values to time-menu

* add query dialog

* add more functions to collumn-tree menu

* add menu functions

* remove test

* update tests, fix extra 0s, clear in between time filters

* fixes

* add deffered

* save

* add alias's

* change menu style

* fix substring

* small fixes

* add auto run to run button

* move divider to a reasonable position

* fix auto expland

* run jest -u

* fix auto run

* prevent aggregate cooumn changes

* fix group by bug

* add quertAst to state

* ui fixes

* prevent filter on non existant ast

* add auto run to shows

* use handle run

* handle *

* add alias and tslint fix
2019-08-15 01:22:25 -07:00
mcbrewster d1f286b7f4 fix no from clause error (#8275) 2019-08-08 21:42:52 -07:00
mcbrewster 03a37936ec Web-Console: Column tree button fixes (#8253)
* small fixes

* run jest -u

* add trim

* remove trim, update package

* fix package-lock
2019-08-07 16:48:17 -07:00
Chi Cao Minh 05b44e3467 Speedup Travis CI jobs (#8240)
Reorganize Travis CI jobs into smaller faster (and more) jobs. Add
various maven options to skip unnecessary work and refactored Travis CI
job definitions to follow DRY.

Detailed changes:

.travis.yml
- Refactor build logic to get rid of copy-and-paste logic
- Skip static checks and enable parallelism for maven install
- Split static analysis into different jobs to ease triage
- Use "name" attribute instead of NAME environment variable
- Split "indexing" and "web console" out of "other modules test"
- Split 2 integration test jobs into multiple smaller jobs

build.sh
- Enable parallelism
- Disable more static checks

travis_script_integration.sh
travis_script_integration_part2.sh
integration-tests/README.md
- Use TestNG groups instead of shell scripts and move definition of jobs
  into Travis CI yaml

integration-tests/pom.xml
- Show elapsed time of individual tests to aid in future rebalancing of
  Travis CI integration test jobs run time

TestNGGroup.java
- Use TestNG groups to make it easy to have multiple Travis CI
  integration test jobs. TestNG groups also make it easier to have an
  "other" integration test group and make it less likely a test will
  accidentally not be included in a CI job.

IT*Test.java
AbstractITBatchIndexTest.java
AbstractKafkaIndexerTest.java
- Add TestNG group
- Fix various IntelliJ inspection warnings
- Reduce scope of helper methods since the TestNG group annotation on
  the class makes TestNG consider all public methods as test methods

pom.xml
- Allow enforce plugin to be run from command-line
- Bump resources plugin version so that "[debug] execute contextualize"
  output is correctly suppressed by "mvn -q"
- Bump exec plugin version so that skip property is renamed from "skip"
  to "exec.skip"

web-console/pom.xml
- Add property to allow disabling javascript-related work. This property
  is overridden in Travis CI to speed up the jobs.
2019-08-07 09:52:42 -07:00
mcbrewster e2d1d00fb8 Web-Console: Add show and copy actions to column tree (#8251)
* save

* add copy and show popup to column tree menu

* fixes

* nest css, uset functions

* fix copy state

* update package

* update package-lock
2019-08-06 23:58:00 -07:00
Vadim Ogievetsky b9c68a5b7b Web console: refactor home view, add tests (#8247)
* refactor home view

* updated mode button placement
2019-08-06 12:41:07 -07:00
mcbrewster 62da1efc0f Web-Console: add dimensions to datasources icon (#8232)
* add dimensions to datasources icon

* fix test

* update snapshots

* save

* use table

* update snapshots

* rename to columns

* run jest -u
2019-08-06 00:04:37 -07:00
Vadim Ogievetsky 0235b338fc Web console: celebrate array based groupBy by supporting resultAsArray in the console (#8231)
* teach table about resultAsArray

* use query result decoder

* fix snapshot
2019-08-05 18:54:39 -07:00
mcbrewster 5d0805dd48 Web-Console: change go to sql button to more button (#8227)
* change go to sql button

* rename to See in SQL view

* update snapshots
2019-08-05 14:12:16 -07:00