Commit Graph

6927 Commits

Author SHA1 Message Date
Gian Merlino 345299aefe Merge pull request #2054 from AdrieanKhisbe/update-tutorial
Update Tutorial with new Zookeper Version and instructions
2015-12-07 07:24:15 -08:00
AdrieanKhisbe 5d10088e2d Update Tutorial with new Zookeper Version and instructions 2015-12-07 06:50:42 +01:00
Himanshu Gupta 6cfaf59d7e add a unique string to intermediate path for the hadoop indexing task 2015-12-06 22:20:38 -06:00
Gian Merlino 8e594a2e72 Change service names in docs, examples to match defaults in the code. 2015-12-06 10:04:21 -08:00
Fangjin Yang b44fedb1cb Merge pull request #2051 from himanshug/fix_doc
update the examples for batch reindexing/delta ingestion to use "intervals"
2015-12-05 22:36:13 -08:00
Himanshu Gupta efe3c9f4a5 update the examples for batch reindexing/delta ingestion to use "intervals" instead of deprecated "interval" 2015-12-06 00:22:20 -06:00
Himanshu 5f2466afd1 Merge pull request #2045 from metamx/updateEmitter036
Update mmx emitter to 0.3.6
2015-12-05 23:20:17 -06:00
Himanshu Gupta 431469e9c1 remove min/max aggregator factory which are replaced by double[min/max] aggregator factories 2015-12-05 22:36:49 -06:00
Fangjin Yang b661f00dc8 Merge pull request #2050 from himanshug/update_license_hdr
unifying license header in all java files
2015-12-05 20:29:07 -08:00
Himanshu Gupta 62ba9ade37 unifying license header in all java files 2015-12-05 22:16:23 -06:00
Fangjin Yang b9760e1219 Merge pull request #2031 from liuqiyun/1914-v2
Druid Issue #1914: Added an injection test for ZkCoordinator.
2015-12-04 21:20:17 -08:00
刘麒贇 3781aa380f Added an injection test for ZkCoordinator. 2015-12-05 11:56:22 +08:00
Charles Allen ea5fdc30f8 Update mmx emitter to 0.3.6
* 0.3.5 updated better logging messages
* 0.3.6 updates validator dependency to help prevent stale validator jars from being pulled in
2015-12-04 12:50:22 -08:00
Gian Merlino d21a640695 Merge pull request #2034 from b-slim/fix_cache_key
Fix getCacheKey for DimFilters
2015-12-04 09:13:06 -08:00
Gian Merlino 20544d409b Merge pull request #1988 from himanshug/multi-interval-batch-delta
support multiple intervals in dataSource inputSpec
2015-12-04 09:07:52 -08:00
Slim Bouguerra fb4ff3cf54 fix getCacheKey 2015-12-04 08:07:08 -06:00
Gian Merlino c36edf6fdf Merge pull request #2037 from metamx/updateIncrementalIndexTestCopyright
Update IncrementalIndexTest copyright notice
2015-12-03 19:47:56 -08:00
Gian Merlino 020a5e7081 Merge pull request #2024 from metamx/fairBigTaskQueueLock
Make the TaskQueue big lock fair
2015-12-03 19:32:53 -08:00
Himanshu Gupta 61aaa09012 support multiple intervals in dataSource input spec 2015-12-03 21:28:04 -06:00
Himanshu Gupta 86f0a36e83 support multiple intervals in SegmentListUsedAction 2015-12-03 21:28:04 -06:00
Himanshu Gupta 221fb95d07 add support for getting used segments for multiple interval in IndexerMetadataStorageCoordinator 2015-12-03 21:28:04 -06:00
Xavier Léauté 2ae34aea3a Merge pull request #2036 from gianm/no-doclint
Disable javadoc linting.
2015-12-03 19:12:00 -08:00
Gian Merlino fde4753e25 Disable javadoc linting. 2015-12-03 19:11:29 -08:00
navis.ryu 4a70d8e775 ParseSpecValidator 2015-12-04 11:08:02 +09:00
Charles Allen 9d02f47201 Update IncrementalIndexTest copyright notice 2015-12-03 18:03:08 -08:00
Charles Allen be8c6fafb0 Merge pull request #2017 from tubemogul/issue/63
fixes issue #63
2015-12-03 18:01:11 -08:00
Gian Merlino 045df54404 Merge pull request #1961 from metamx/druidMetricsVersion
Add the druid artifact version to metrics when emitted
2015-12-03 17:34:57 -08:00
Michael Schiff b6cc2428e1 fixes issue #63 2015-12-03 17:30:47 -08:00
Xavier Léauté 1ca3166a26 Merge pull request #2035 from gianm/scala-formatting
Update Scala formatting in intellij_formatting.jar.
2015-12-03 13:37:51 -08:00
Gian Merlino d7d4382508 Update Scala formatting in intellij_formatting.jar, and rename style to "Druid Java and Scala style". 2015-12-03 13:35:44 -08:00
Himanshu 0eab8417cb Merge pull request #2008 from codingwhatever/regex-search-query
Regex search query
2015-12-03 09:57:34 -06:00
Sam Groth 596b7ebd9a Adding RegexSearchQuerySpec 2015-12-03 09:16:02 -06:00
Himanshu d02be6194d Merge pull request #1967 from metamx/realtime-metrics-improvements
Add datasource and taskId to metrics emitted by peons
2015-12-02 23:48:13 -06:00
Himanshu 00c6027777 Merge pull request #1986 from metamx/substring
fixes #1874 adding a substring extraction function, tests, and documentation
2015-12-02 23:45:47 -06:00
Fangjin Yang e71436fb34 Merge pull request #2018 from gianm/fix-fragment-search-query-spec
Fix serde for FragmentSearchQuerySpec and add some tests.
2015-12-03 07:38:11 +08:00
Clint Wylie 68ef5f437a fixes #1874 adding a substring extraction function, tests, and documentation 2015-12-01 23:50:32 -08:00
navis.ryu 87357a0534 fixed #2001 GenericIndexed.fromIterable compares all values even when it's not sorted 2015-12-02 15:11:14 +09:00
Charles Allen dbaaa6af92 Make the TaskQueue big lock fair 2015-12-01 19:13:07 -08:00
Nishant 1eb8211346 Add datasource and taskId to metrics emitted by peons
This PR adds the datasource and taskId to the jvm and sys metrics
emitted by the peons.

fix spelling

review comment

review comment
2015-12-01 23:20:59 +05:30
Gian Merlino cd2cff24ff Fix serde for FragmentSearchQuerySpec and add some tests. 2015-11-30 17:34:35 -08:00
Himanshu Gupta 7c91219130 adding more comments to why drops do not happen when there are no segments in db 2015-11-25 16:26:51 -06:00
Fangjin Yang bad96537a9 Merge pull request #2009 from jon-wei/multival_json_doc
Update data formats doc with info about JSON multi-value dimensions
2015-11-24 22:47:53 -08:00
Himanshu 78e07ea591 Merge pull request #2004 from navis/DRUID-2003
fixed #2003 ColumnSelectorBitmapIndexSelector throws NPE for dimension not supporting bitmap
2015-11-25 00:12:30 -06:00
jon-wei 95dca4440f Update data formats doc with info about JSON multi-value dimensions 2015-11-24 14:38:06 -08:00
Fangjin Yang 8e83d800d6 Merge pull request #1881 from gianm/restartable-tasks
Restorable indexing tasks
2015-11-23 21:14:37 -08:00
navis.ryu c73418c181 fixed #2003 ColumnSelectorBitmapIndexSelector throws NPE for dimension not supporting bitmap 2015-11-24 10:45:36 +09:00
Gian Merlino 13af260102 Merge pull request #1957 from metamx/skip-incremental-segment
Ability to skip Incremental Index during query using query context
2015-11-23 17:32:57 -08:00
Xavier Léauté f3185938a8 Merge pull request #1997 from gianm/fix-nullable-actions
Switch TaskActions from Optionals to nullable.
2015-11-23 16:02:40 -08:00
Gian Merlino 501dcb43fa Some changes that make it possible to restart tasks on the same hardware.
This is done by killing and respawning the jvms rather than reconnecting to existing
jvms, for a couple reasons. One is that it lets you restore tasks after server reboots
too, and another is that it lets you upgrade all the software on a box at once by just
restarting everything.

The main changes are,

1) Add "canRestore" and "stopGracefully" methods to Tasks that say if a task can
   stop gracefully, and actually do a graceful stop. RealtimeIndexTask is the only
   one that currently implements this.

2) Add "stop" method to TaskRunners that attempts to do an orderly shutdown.
   ThreadPoolTaskRunner- call stopGracefully on restorable tasks, wait for exit
   ForkingTaskRunner- close output stream to restorable tasks, wait for exit
   RemoteTaskRunner- do nothing special, we actually don't want to shutdown

3) Add "restore" method to TaskRunners that attempts to bootstrap tasks from last run.
   Only ForkingTaskRunner does anything here. It maintains a "restore.json" file with
   a list of restorable tasks.

4) Have the CliPeon's ExecutorLifecycle lock the task base directory to avoid a restored
   task and a zombie old task from stomping on each other.
2015-11-23 11:22:08 -08:00
Gian Merlino 36569094ce EventReceiverFirehose: Drain buffer when closed, until empty. 2015-11-20 19:12:59 -08:00