Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,451 Commits 111 Branches 503 Tags 726 MiB
Commit Graph

947 Commits

Author SHA1 Message Date
michael.schiff 4ed0128b3b removed documentation about (removed) DeleteTask 2015-12-07 15:56:46 -08:00
Fangjin Yang 00416e26ff Merge pull request #2012 from himanshug/add_comment 
adding more comments to why drops do not happen
 2015-12-07 11:50:06 -08:00
Fangjin Yang fe05545f58 Merge pull request #2055 from jkukul/documentation-fixes 
Minor documentation fixes.
 2015-12-07 10:50:38 -08:00
Jakub Kukul 8248f7f40c Minor docs fixes in 'Evaluate Druid' section. 2015-12-07 19:42:52 +01:00
AdrieanKhisbe 5d10088e2d Update Tutorial with new Zookeper Version and instructions 2015-12-07 06:50:42 +01:00
Gian Merlino 8e594a2e72 Change service names in docs, examples to match defaults in the code. 2015-12-06 10:04:21 -08:00
Himanshu Gupta efe3c9f4a5 update the examples for batch reindexing/delta ingestion to use "intervals" instead of deprecated "interval" 2015-12-06 00:22:20 -06:00
Himanshu Gupta 61aaa09012 support multiple intervals in dataSource input spec 2015-12-03 21:28:04 -06:00
Himanshu 0eab8417cb Merge pull request #2008 from codingwhatever/regex-search-query 
Regex search query
 2015-12-03 09:57:34 -06:00
Sam Groth 596b7ebd9a Adding RegexSearchQuerySpec 2015-12-03 09:16:02 -06:00
Clint Wylie 68ef5f437a fixes #1874 adding a substring extraction function, tests, and documentation 2015-12-01 23:50:32 -08:00
Himanshu Gupta 7c91219130 adding more comments to why drops do not happen when there are no segments in db 2015-11-25 16:26:51 -06:00
jon-wei 95dca4440f Update data formats doc with info about JSON multi-value dimensions 2015-11-24 14:38:06 -08:00
Fangjin Yang 8e83d800d6 Merge pull request #1881 from gianm/restartable-tasks 
Restorable indexing tasks
 2015-11-23 21:14:37 -08:00
Gian Merlino 501dcb43fa Some changes that make it possible to restart tasks on the same hardware. 
This is done by killing and respawning the jvms rather than reconnecting to existing
jvms, for a couple reasons. One is that it lets you restore tasks after server reboots
too, and another is that it lets you upgrade all the software on a box at once by just
restarting everything.

The main changes are,

1) Add "canRestore" and "stopGracefully" methods to Tasks that say if a task can
   stop gracefully, and actually do a graceful stop. RealtimeIndexTask is the only
   one that currently implements this.

2) Add "stop" method to TaskRunners that attempts to do an orderly shutdown.
   ThreadPoolTaskRunner- call stopGracefully on restorable tasks, wait for exit
   ForkingTaskRunner- close output stream to restorable tasks, wait for exit
   RemoteTaskRunner- do nothing special, we actually don't want to shutdown

3) Add "restore" method to TaskRunners that attempts to bootstrap tasks from last run.
   Only ForkingTaskRunner does anything here. It maintains a "restore.json" file with
   a list of restorable tasks.

4) Have the CliPeon's ExecutorLifecycle lock the task base directory to avoid a restored
   task and a zombie old task from stomping on each other.
 2015-11-23 11:22:08 -08:00
Gian Merlino c0580bf063 Merge pull request #1928 from druid-io/new-compares 
New comparisons for Druid
 2015-11-20 16:40:59 -08:00
Himanshu Gupta 7a89b2e1a6 add numRows to segment metadata query response 2015-11-20 01:25:02 -06:00
Himanshu Gupta fde9df2720 update to sketches-core-0.2.2 . 
adds support for "cardinality" aggregator.
do not create sketch per event at ingestion time to make realtime ingestion faster
 2015-11-19 01:05:59 -06:00
Fangjin Yang 87c43fbf02 Merge pull request #1981 from rasahner/fixFormattingSchemaDesign 
fix formatting in schema-design
 2015-11-17 17:51:16 -08:00
Charles Allen 8fcf2403e3 Merge pull request #1943 from metamx/realtime-caching 
Enable caching on intermediate realtime persists
 2015-11-17 15:06:43 -08:00
sahner a4ed2ce2d1 fix formatting in schema-design 2015-11-17 16:50:53 -06:00
Himanshu 9118b010c0 Merge pull request #1976 from noddi/feature/count-parameter-rules-history 
Add count parameter to rules history endpoint.
 2015-11-17 14:41:50 -06:00
Charles Allen dbe201aeed Merge pull request #1929 from pjain1/jetty_threads 
separate ingestion and query thread pool
 2015-11-17 12:14:25 -08:00
Parag Jain 6c498b7d4a separate ingestion and query thread pool 2015-11-17 13:42:41 -06:00
Bartosz Ługowski b0a444eafc Add count parameter to rules history endpoint. 2015-11-17 20:14:12 +01:00
Xavier Léauté d7eb2f717e enable query caching on intermediate realtime persists 2015-11-17 10:58:00 -08:00
Fangjin Yang 4f46d457f1 Merge pull request #1947 from noddi/feature/count-parameter-history-endpoints 
Add count parameter to history endpoints
 2015-11-12 10:23:44 -08:00
Himanshu Gupta 7788f7c2a1 update doc with new thetaSketch api 2015-11-12 00:04:34 -06:00
Himanshu Gupta 6c6a38cedb adding datasketches aggregator to documentation 2015-11-12 00:04:33 -06:00
Bartosz Ługowski 6e5d2c6745 Add count parameter to history endpoints. 2015-11-11 23:03:57 +01:00
Gian Merlino e75c2a407d Merge pull request #1944 from druid-io/fix-doc 
fix website rendering for this doc
 2015-11-10 16:04:40 -08:00
fjy e923de3eea fix website rendering for this doc 2015-11-10 15:36:30 -08:00
Xavier Léauté cf779946ef Merge pull request #1791 from guobingkun/event_receiver_firehose_monitor 
EventReceiverFirehoseMonitor
 2015-11-10 11:09:42 -08:00
fjy 46bf1ba5ef remove unneeded 2015-11-09 17:03:00 -08:00
fjy 8a8bb0369e address more comments 2015-11-09 16:56:43 -08:00
fjy b99576d854 rework compares again 2015-11-09 16:40:07 -08:00
fjy 0b319093df New comparisons for Druid 2015-11-09 14:31:23 -08:00
Bingkun Guo b24eccfb9e add doc for bundling custom extensions with other Druid extensions 2015-11-09 13:11:22 -06:00
Slim Bouguerra c511273efd adding in filter 2015-11-06 16:23:24 -06:00
Xavier Léauté e9533db987 Merge pull request #1850 from metamx/friendlyBardCache 
Allow setting upper limit on the number of cache segments a broker will try to fetch.
 2015-11-06 10:25:49 -08:00
Himanshu 3c3ea128cc Merge pull request #1895 from druid-io/cleanup-codebase 
Cleanup the Druid codebase
 2015-11-04 17:16:33 -06:00
fjy 8f231fd3e3 cleanup druid codebase 2015-11-04 13:59:53 -08:00
Fangjin Yang 1cbc514eb5 Merge pull request #1719 from se7entyse7en/feature-rackspace-cloud-files-deep-storage 
Add Rackspace Cloud Files Deep Storage Extension
 2015-11-04 11:13:25 -08:00
Fangjin Yang 861338c5e9 Merge pull request #1904 from rohitkochar/doc 
modifying the documentation as per Issue#1903
 2015-11-04 09:48:45 -08:00
Himanshu Gupta 67c218bed6 since topN is descending by default, inversion makes is ascending 2015-11-04 11:36:49 -06:00
Lou Marvin Caraig c924f9fe56 Added cloudfiles-extensions in order to support Rackspace's cloudfiles as deep storage 2015-11-04 17:44:48 +01:00
rohit.kochar 69a3b38f81 made changes as per review comments 2015-11-04 11:10:28 +05:30
Bingkun Guo 3ee28c35ce fix curator compress doc 2015-11-03 16:48:59 -06:00
Gian Merlino 8defe29270 Merge pull request #1901 from guobingkun/fix_typo_and_rename 
Fix metadata typo and rename default extension directory
 2015-11-03 14:02:11 -08:00
Xavier Léauté 3b3d88bd03 Merge pull request #1857 from noddi/feature/intervals-route 
Queryable intervals
 2015-11-03 13:19:04 -08:00
Bingkun Guo 962f65cc76 fix metadata typo and rename default extension directory 2015-11-03 14:50:42 -06:00
Fangjin Yang cec09a9967 Merge pull request #1804 from himanshug/objectify_index_creators 
static to non-static conversion for methods in Index[Merger/Maker/IO]
 2015-11-03 11:25:32 -08:00
rohit.kochar 2a617c777e modifying the documentation as per Issue#1903 2015-11-03 21:48:19 +05:30
Fangjin Yang f681c84014 Merge pull request #1892 from navis/DRUID-1878 
Make 'search' filter have a case sensitive option (for #1878)
 2015-11-03 07:11:00 -08:00
Himanshu Gupta c74a4490e1 add metamarket histogram post to approx-histo doc 2015-11-03 01:19:22 -06:00
Himanshu Gupta 8b67417ac8 make methods in Index[Merger,Maker,IO] non-static so that they can have 
appropriate ObjectMapper injected instead of creating one statically
 2015-11-02 23:24:26 -06:00
Gian Merlino 0d85774a27 Merge pull request #1899 from itsmee/docs-historical-improvements 
Docs improved: more details about caching and memory for segments on historicals
 2015-11-02 12:48:56 -08:00
Nishant efc49da073 fix doc - correct default value for maxRowsInMemory 2015-11-01 22:09:24 -08:00
navis.ryu 69c86716d6 addressed comments 2015-11-02 14:23:13 +09:00
Oleg Zaezdny 95a5ae0373 Docs improved by adding more details about local cache and memory for segments on historicals. 2015-11-01 21:56:28 +02:00
Bingkun Guo c3b6fcce9d Add EventReceiverFirehoseMonitor 
add an EventReceiverFirehoseMonitor so that we can monitor how many
events have been queued in the EventReceiverFirehose and get a sense
about whether the firehose is under too much pressure.
 2015-10-30 11:40:02 -05:00
Bingkun Guo 657a5ac346 fix pull-deps remoteRepository option 2015-10-30 11:32:56 -05:00
Bartosz Ługowski 6de51e022d Coordinator - add intervals route. 2015-10-30 11:51:42 +01:00
Charles Allen dfce14ed17 Allow setting upper limit on the number of cache segments a broker will try to fetch. 2015-10-29 11:50:00 -07:00
Angel M de Miguel a2510c9b0b Update ruby-druid URL 2015-10-28 10:31:30 +01:00
Angel M de Miguel 04c5d0f8e2 Update Ruby libraries in docs 2015-10-28 09:08:26 +01:00
Xavier Léauté 59872bd0cd Merge pull request #1809 from metamx/fifoPriorityExecutorService 
Make PrioritizedExecutorService optionally FIFO
 2015-10-27 15:19:32 -07:00
Gian Merlino 7df7370935 Merge pull request #1862 from metamx/indexingServiceMMGone 
Add timeout to shutdown request to middle manager for indexing service
 2015-10-27 14:38:01 -07:00
Charles Allen ecdafa87c5 Make PrioritizedExecutorService optionally FIFO 2015-10-27 14:16:22 -07:00
Charles Allen 44a2b204df Add timeout to shutdown request to middle manager for indexing service 2015-10-27 13:56:03 -07:00
Bingkun Guo 44aad069da add description for /druid/coordinator/v1/loadqueue?full 2015-10-26 14:19:13 -05:00
Fangjin Yang 7e1cde25b6 Merge pull request #1820 from himanshug/doc_files 
Removing the md files created for redirection
 2015-10-26 09:37:11 -07:00
Fangjin Yang 5f23703216 Merge pull request #1638 from guobingkun/remove_maven_client_code 
Remove Maven client at runtime + Provide a way to load Druid extensions through local file system
 2015-10-26 09:30:05 -07:00
Slim 16f84161da Fix documentation about lookup 2015-10-23 16:40:54 -07:00
Xavier Léauté 72c408cf2d Merge pull request #1770 from metamx/merge-time 
Add segment merge time as a metric
 2015-10-22 22:03:41 -07:00
Nishant 7cecc55045 Add segment merge time as a metric 
Add merge and persist cpu time

Fix typo

review comment

move cpu time measuring to VMUtils

review comments.
 2015-10-22 12:28:03 +05:30
Gian Merlino d068ec3b9d "druid.manager.segment" should be "druid.manager.segments" 2015-10-21 15:50:44 -07:00
Bingkun Guo 4914925d65 New extension loading mechanism 
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
 2015-10-21 14:22:36 -05:00
Himanshu 8dd30e6e18 Merge pull request #1832 from guobingkun/improve_granularity_doc 
add examples for duration and period granularities
 2015-10-21 13:13:24 -05:00
Bingkun Guo 2de5e9cefb add examples for duration and period granularities 2015-10-21 11:23:00 -05:00
Fangjin Yang 170cce69d4 Merge pull request #1834 from guobingkun/fix_doc 
fix ingestion faq link
 2015-10-18 17:31:22 -07:00
Fangjin Yang ccd17f818a Merge pull request #1837 from druid-io/update-paper 
add more details to radstack paper
 2015-10-18 17:31:08 -07:00
fjy 89785b2f52 add more details to paper 2015-10-18 17:30:24 -07:00
Bingkun Guo 620e334d0f fix ingestion faq link 2015-10-16 10:14:14 -05:00
sahner 83d31844db fix doubled word 2015-10-15 15:59:07 -05:00
Himanshu Gupta 24f33938d8 Removing the md files created for redirection 2015-10-12 15:21:55 -05:00
Gian Merlino 933cbdf780 Adjust realtime constraints in the docs. 2015-10-09 10:52:52 -07:00
Charles Allen 020a706ac4 Merge pull request #1800 from metamx/hybrid-cache 
Hybrid L1/L2 cache
 2015-10-06 15:18:07 -07:00
Xavier Léauté b11c0859e8 hybrid l1/l2 cache to combine local and remote cache 2015-10-06 14:32:08 -07:00
Xavier Léauté e5b3226b62 Merge pull request #1793 from himanshug/fix_chunking_link 
fixing the link to chunkPeriod doc
 2015-10-02 10:10:11 -04:00
Himanshu Gupta 9006e255ef fixing the link to chunkPeriod doc 2015-10-01 13:03:46 -05:00
Xavier Léauté faf4c865d5 update R / Python clients 2015-10-01 13:42:09 -04:00
Nishant 573aa96bd6 fix #1727 - Union bySegment queries fix 
Fixes #1727.
revert to doing merging for results for union queries on broker.

revert unrelated changes

Add test for union query runner

Add test

remove unused imports

fix imports

fix renamed file

fix test

update docs.
 2015-09-29 23:32:36 +05:30
Himanshu Gupta 6e550d5346 update doc about aggregation field in merge task and a null check 2015-09-24 22:25:07 -05:00
Xavier Léauté 2cb0fb4669 Merge pull request #1753 from jon-wei/segmentmetadataquery_flags 
Allow SegmentMetadataQuery to skip cardinality and size calculations
 2015-09-22 14:52:28 -07:00
jon-wei e6a6284ebd Allow SegmentMetadataQuery to skip cardinality and size calculations 2015-09-22 13:51:55 -07:00
Xavier Léauté df4c8a3aa5 balance memcached cache across multiple connections 2015-09-18 09:52:16 -07:00
fjy c9ccf9a565 add docs for search filter 2015-09-16 16:08:59 -07:00
Gian Merlino 9705c5139b Merge pull request #1732 from jon-wei/segmentmeta 
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
 2015-09-16 12:36:25 -07:00
Gian Merlino b29cbf97a6 Docs: Suggest hadoopyString parser for Hadoop. 2015-09-16 10:19:42 -07:00