Commit Graph

6755 Commits

Author SHA1 Message Date
Gian Merlino fde4753e25 Disable javadoc linting. 2015-12-03 19:11:29 -08:00
navis.ryu 4a70d8e775 ParseSpecValidator 2015-12-04 11:08:02 +09:00
Charles Allen 9d02f47201 Update IncrementalIndexTest copyright notice 2015-12-03 18:03:08 -08:00
Charles Allen be8c6fafb0 Merge pull request #2017 from tubemogul/issue/63
fixes issue #63
2015-12-03 18:01:11 -08:00
Gian Merlino 045df54404 Merge pull request #1961 from metamx/druidMetricsVersion
Add the druid artifact version to metrics when emitted
2015-12-03 17:34:57 -08:00
Michael Schiff b6cc2428e1 fixes issue #63 2015-12-03 17:30:47 -08:00
Xavier Léauté 1ca3166a26 Merge pull request #2035 from gianm/scala-formatting
Update Scala formatting in intellij_formatting.jar.
2015-12-03 13:37:51 -08:00
Gian Merlino d7d4382508 Update Scala formatting in intellij_formatting.jar, and rename style to "Druid Java and Scala style". 2015-12-03 13:35:44 -08:00
Himanshu 0eab8417cb Merge pull request #2008 from codingwhatever/regex-search-query
Regex search query
2015-12-03 09:57:34 -06:00
Sam Groth 596b7ebd9a Adding RegexSearchQuerySpec 2015-12-03 09:16:02 -06:00
Himanshu d02be6194d Merge pull request #1967 from metamx/realtime-metrics-improvements
Add datasource and taskId to metrics emitted by peons
2015-12-02 23:48:13 -06:00
Himanshu 00c6027777 Merge pull request #1986 from metamx/substring
fixes #1874 adding a substring extraction function, tests, and documentation
2015-12-02 23:45:47 -06:00
Fangjin Yang e71436fb34 Merge pull request #2018 from gianm/fix-fragment-search-query-spec
Fix serde for FragmentSearchQuerySpec and add some tests.
2015-12-03 07:38:11 +08:00
Clint Wylie 68ef5f437a fixes #1874 adding a substring extraction function, tests, and documentation 2015-12-01 23:50:32 -08:00
navis.ryu 87357a0534 fixed #2001 GenericIndexed.fromIterable compares all values even when it's not sorted 2015-12-02 15:11:14 +09:00
Charles Allen dbaaa6af92 Make the TaskQueue big lock fair 2015-12-01 19:13:07 -08:00
Nishant 1eb8211346 Add datasource and taskId to metrics emitted by peons
This PR adds the datasource and taskId to the jvm and sys metrics
emitted by the peons.

fix spelling

review comment

review comment
2015-12-01 23:20:59 +05:30
Gian Merlino cd2cff24ff Fix serde for FragmentSearchQuerySpec and add some tests. 2015-11-30 17:34:35 -08:00
Himanshu Gupta 7c91219130 adding more comments to why drops do not happen when there are no segments in db 2015-11-25 16:26:51 -06:00
Fangjin Yang bad96537a9 Merge pull request #2009 from jon-wei/multival_json_doc
Update data formats doc with info about JSON multi-value dimensions
2015-11-24 22:47:53 -08:00
Himanshu 78e07ea591 Merge pull request #2004 from navis/DRUID-2003
fixed #2003 ColumnSelectorBitmapIndexSelector throws NPE for dimension not supporting bitmap
2015-11-25 00:12:30 -06:00
jon-wei 95dca4440f Update data formats doc with info about JSON multi-value dimensions 2015-11-24 14:38:06 -08:00
Fangjin Yang 8e83d800d6 Merge pull request #1881 from gianm/restartable-tasks
Restorable indexing tasks
2015-11-23 21:14:37 -08:00
navis.ryu c73418c181 fixed #2003 ColumnSelectorBitmapIndexSelector throws NPE for dimension not supporting bitmap 2015-11-24 10:45:36 +09:00
Gian Merlino 13af260102 Merge pull request #1957 from metamx/skip-incremental-segment
Ability to skip Incremental Index during query using query context
2015-11-23 17:32:57 -08:00
Xavier Léauté f3185938a8 Merge pull request #1997 from gianm/fix-nullable-actions
Switch TaskActions from Optionals to nullable.
2015-11-23 16:02:40 -08:00
Gian Merlino 501dcb43fa Some changes that make it possible to restart tasks on the same hardware.
This is done by killing and respawning the jvms rather than reconnecting to existing
jvms, for a couple reasons. One is that it lets you restore tasks after server reboots
too, and another is that it lets you upgrade all the software on a box at once by just
restarting everything.

The main changes are,

1) Add "canRestore" and "stopGracefully" methods to Tasks that say if a task can
   stop gracefully, and actually do a graceful stop. RealtimeIndexTask is the only
   one that currently implements this.

2) Add "stop" method to TaskRunners that attempts to do an orderly shutdown.
   ThreadPoolTaskRunner- call stopGracefully on restorable tasks, wait for exit
   ForkingTaskRunner- close output stream to restorable tasks, wait for exit
   RemoteTaskRunner- do nothing special, we actually don't want to shutdown

3) Add "restore" method to TaskRunners that attempts to bootstrap tasks from last run.
   Only ForkingTaskRunner does anything here. It maintains a "restore.json" file with
   a list of restorable tasks.

4) Have the CliPeon's ExecutorLifecycle lock the task base directory to avoid a restored
   task and a zombie old task from stomping on each other.
2015-11-23 11:22:08 -08:00
Gian Merlino 36569094ce EventReceiverFirehose: Drain buffer when closed, until empty. 2015-11-20 19:12:59 -08:00
Gian Merlino c0580bf063 Merge pull request #1928 from druid-io/new-compares
New comparisons for Druid
2015-11-20 16:40:59 -08:00
Fangjin Yang bec7dacd86 Merge pull request #1994 from himanshug/how_to_contrib_update
updating how to contribute guide
2015-11-20 10:33:34 -08:00
Gian Merlino 666d785787 Switch TaskActions from Optionals to nullable.
Deserialization of Optionals does not work quite right- they come back as actual
nulls, rather than absent Optionals. So these probably only ever worked for the local
task action client.
2015-11-20 09:14:07 -08:00
Himanshu Gupta 7a89b2e1a6 add numRows to segment metadata query response 2015-11-20 01:25:02 -06:00
Himanshu Gupta 5260783db4 updating how to contribute guide 2015-11-19 23:30:28 -06:00
Fangjin Yang 6305dfe1b9 Merge pull request #1991 from himanshug/sketch_agg_upgrade
datasketches module updates
2015-11-19 13:16:14 -08:00
Himanshu Gupta f99bad7988 reformat datasketches module to satisfy druid style guidelines 2015-11-19 01:07:03 -06:00
Himanshu Gupta fde9df2720 update to sketches-core-0.2.2 .
adds support for "cardinality" aggregator.
do not create sketch per event at ingestion time to make realtime ingestion faster
2015-11-19 01:05:59 -06:00
Himanshu 24ba20c52f Merge pull request #1979 from metamx/update-docker-stuff
switch integration tests docker image to Java 8 + cleanup
2015-11-18 23:24:05 -06:00
Fangjin Yang 21c84b5ff7 Merge pull request #1896 from gianm/allocate-segment
SegmentAllocateAction (fixes #1515)
2015-11-18 21:05:46 -08:00
Himanshu d93640bfcb Merge pull request #1974 from jon-wei/dim_order_merge
Allow IndexMerger to use non-lexicographic dim order when merging indexes
2015-11-18 19:51:34 -06:00
Fangjin Yang e52c156066 Merge pull request #1880 from gianm/rtr-adjust
RTR: Ensure that there is only one cleanup task scheduled for a worker at once.
2015-11-18 15:12:55 -08:00
Nishant 60f649dab1 Ability to skip Incremental Index during query using query context
This PR adds adds the ability to skip incremental index when querying
results from realtime nodes. default behaviour is to include
incrementalIndex in queries.

review comment
2015-11-19 00:29:41 +05:30
Xavier Léauté e3e6159336 Merge pull request #1985 from metamx/FixLookupCacheKey
Change LookupExtractionFn cache key to be unique
2015-11-18 10:13:55 -08:00
Charles Allen 7abe999418 Change LookupExtractionFn cache key to be unique 2015-11-17 18:02:40 -08:00
Fangjin Yang 87c43fbf02 Merge pull request #1981 from rasahner/fixFormattingSchemaDesign
fix formatting in schema-design
2015-11-17 17:51:16 -08:00
Charles Allen 7545ab7188 Merge pull request #1983 from metamx/fix-mysql-57
fix #1701 - MySQL 5.7 defaults break database character set check
2015-11-17 17:36:41 -08:00
Xavier Léauté ba41f37ce1 fix #1701 - MySQL 5.7 defaults break database character set check 2015-11-17 15:51:58 -08:00
Charles Allen 8fcf2403e3 Merge pull request #1943 from metamx/realtime-caching
Enable caching on intermediate realtime persists
2015-11-17 15:06:43 -08:00
sahner a4ed2ce2d1 fix formatting in schema-design 2015-11-17 16:50:53 -06:00
Xavier Léauté 350fdbeec1 switch to Java 8 + cleanup 2015-11-17 13:35:06 -08:00
jon-wei 4afc62be29 Allow IndexMerger to use non-lexicographic dim order when merging indexes 2015-11-17 13:02:31 -08:00