Commit Graph

889 Commits

Author SHA1 Message Date
Fangjin Yang 5042b20eb4 Merge pull request #2164 from pdeva/patch-7
correct docs
2015-12-27 14:43:51 -08:00
pdeva 77863285e9 fix typo 2015-12-27 14:28:23 -08:00
pdeva b308a13483 correct docs 2015-12-27 14:27:20 -08:00
pdeva 98fc08b0de List who can access metadata storage
This is useful to know which ip addresses you need to allow to access your RDS instance for example
2015-12-27 14:25:17 -08:00
Fangjin Yang daaa24d7ad Merge pull request #2145 from gianm/restoring-option
druid.indexer.task.restoreTasksOnRestart configuration.
2015-12-22 15:22:32 -08:00
Gian Merlino bad270b6c4 druid.indexer.task.restoreTasksOnRestart configuration. 2015-12-22 10:59:15 -08:00
Himanshu Gupta b96f560255 emit query/node/bytes metric 2015-12-21 23:23:20 -06:00
Charles Allen c2a2d19d62 Merge pull request #2126 from guobingkun/remove_single_announcer
Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer
2015-12-21 08:55:07 -08:00
Bingkun Guo 951a4e9b35 Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer 2015-12-21 00:05:53 -06:00
Fangjin Yang 7019d3c421 Merge pull request #2107 from jon-wei/fix_smq
More efficient SegmentMetadataQuery
2015-12-18 16:40:47 -08:00
fjy 398a3ec620 add docs for more specs 2015-12-17 18:06:30 -08:00
jon-wei 356b07c6c3 More efficient SegmentMetadataQuery 2015-12-17 12:46:23 -08:00
Jonathan Wei f8cf84f466 Merge pull request #1995 from himanshug/num_rows_seg_metadata_query
add numRows to segment metadata query response
2015-12-17 12:23:46 -08:00
Fangjin Yang cb085f99d3 Merge pull request #2097 from metamx/update-coordinator-docs
update the coordinator API docs to better explain how to format ISO86…
2015-12-15 17:35:48 -08:00
desaianuj 2f32cb1fb6 update the coordinator API docs to better explain how to format ISO8601 intervals in the request url 2015-12-15 16:17:54 -08:00
Gian Merlino e6c2db89da Merge pull request #1921 from jon-wei/flat_json
Add docs and benchmark for JSON flattening parser
2015-12-14 20:53:54 -08:00
Fangjin Yang e7f06cf61c Merge pull request #2075 from jon-wei/regex_extract
Configurable value replacement on match failure for RegexExtractionFn
2015-12-14 19:10:50 -08:00
jon-wei c88f75df7c Configurable value replacement on match failure for RegexExtractionFn 2015-12-14 17:57:41 -08:00
Steve M 2b5a010332 Change sample worker config spec with host:port instead of ip:port.
Also extend description of the 'affinity' property of the worker strategy
fillCapacityWithAffinity and fix a couple typos of middle manager (to
be more consistent throughout the page).

Add additional verbiage about appropriate middle manager host value.
2015-12-14 14:59:23 -08:00
jon-wei c53bf85d83 Add docs and benchmark for JSON flattening parser 2015-12-09 16:13:30 -08:00
Fangjin Yang b0ab363022 Merge pull request #2052 from gianm/service-names
Change service names in docs, examples to match defaults in the code.
2015-12-08 15:40:35 -08:00
Xavier Léauté dcd15735cd Merge pull request #2015 from metamx/handoff-notifier-coordinator
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoff information
2015-12-08 14:06:39 -08:00
Nishant 9491e8de3b Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoffs
- fixes #1970
- extracted out segment handoff callbacks in SegmentHandoffNotifier
which is responsible for tracking segment handoffs and doing callbacks
when handoff is complete.
- Coordinator now maintains a view of segments in the cluster, this
will affect the jam heap requirements for the overlord for large
clusters.
realtime index task and nodes now use HTTP end points exposed by the
coordinator to get serverView

review comment

fix realtime node guide injection

review comments

make test not rely on scheduled exec

fix compilation

fix import

review comment

introduce immutableSegmentLoadInfo

fix son reading

remove unnecessary logging
2015-12-09 01:54:09 +05:30
Nishant 6c23d8edb4 Merge pull request #2043 from mangeshpardeshiyahoo/master
Add dimension selector support for groupby/having filters
2015-12-08 12:08:53 +05:30
Mangesh Pardeshi d7ce120929 Add dimension selector support for groupby/having quries 2015-12-08 01:51:11 +00:00
michael.schiff 4ed0128b3b removed documentation about (removed) DeleteTask 2015-12-07 15:56:46 -08:00
Fangjin Yang 00416e26ff Merge pull request #2012 from himanshug/add_comment
adding more comments to why drops do not happen
2015-12-07 11:50:06 -08:00
Fangjin Yang fe05545f58 Merge pull request #2055 from jkukul/documentation-fixes
Minor documentation fixes.
2015-12-07 10:50:38 -08:00
Jakub Kukul 8248f7f40c Minor docs fixes in 'Evaluate Druid' section. 2015-12-07 19:42:52 +01:00
AdrieanKhisbe 5d10088e2d Update Tutorial with new Zookeper Version and instructions 2015-12-07 06:50:42 +01:00
Gian Merlino 8e594a2e72 Change service names in docs, examples to match defaults in the code. 2015-12-06 10:04:21 -08:00
Himanshu Gupta efe3c9f4a5 update the examples for batch reindexing/delta ingestion to use "intervals" instead of deprecated "interval" 2015-12-06 00:22:20 -06:00
Himanshu Gupta 61aaa09012 support multiple intervals in dataSource input spec 2015-12-03 21:28:04 -06:00
Himanshu 0eab8417cb Merge pull request #2008 from codingwhatever/regex-search-query
Regex search query
2015-12-03 09:57:34 -06:00
Sam Groth 596b7ebd9a Adding RegexSearchQuerySpec 2015-12-03 09:16:02 -06:00
Clint Wylie 68ef5f437a fixes #1874 adding a substring extraction function, tests, and documentation 2015-12-01 23:50:32 -08:00
Himanshu Gupta 7c91219130 adding more comments to why drops do not happen when there are no segments in db 2015-11-25 16:26:51 -06:00
jon-wei 95dca4440f Update data formats doc with info about JSON multi-value dimensions 2015-11-24 14:38:06 -08:00
Fangjin Yang 8e83d800d6 Merge pull request #1881 from gianm/restartable-tasks
Restorable indexing tasks
2015-11-23 21:14:37 -08:00
Gian Merlino 501dcb43fa Some changes that make it possible to restart tasks on the same hardware.
This is done by killing and respawning the jvms rather than reconnecting to existing
jvms, for a couple reasons. One is that it lets you restore tasks after server reboots
too, and another is that it lets you upgrade all the software on a box at once by just
restarting everything.

The main changes are,

1) Add "canRestore" and "stopGracefully" methods to Tasks that say if a task can
   stop gracefully, and actually do a graceful stop. RealtimeIndexTask is the only
   one that currently implements this.

2) Add "stop" method to TaskRunners that attempts to do an orderly shutdown.
   ThreadPoolTaskRunner- call stopGracefully on restorable tasks, wait for exit
   ForkingTaskRunner- close output stream to restorable tasks, wait for exit
   RemoteTaskRunner- do nothing special, we actually don't want to shutdown

3) Add "restore" method to TaskRunners that attempts to bootstrap tasks from last run.
   Only ForkingTaskRunner does anything here. It maintains a "restore.json" file with
   a list of restorable tasks.

4) Have the CliPeon's ExecutorLifecycle lock the task base directory to avoid a restored
   task and a zombie old task from stomping on each other.
2015-11-23 11:22:08 -08:00
Gian Merlino c0580bf063 Merge pull request #1928 from druid-io/new-compares
New comparisons for Druid
2015-11-20 16:40:59 -08:00
Himanshu Gupta 7a89b2e1a6 add numRows to segment metadata query response 2015-11-20 01:25:02 -06:00
Himanshu Gupta fde9df2720 update to sketches-core-0.2.2 .
adds support for "cardinality" aggregator.
do not create sketch per event at ingestion time to make realtime ingestion faster
2015-11-19 01:05:59 -06:00
Fangjin Yang 87c43fbf02 Merge pull request #1981 from rasahner/fixFormattingSchemaDesign
fix formatting in schema-design
2015-11-17 17:51:16 -08:00
Charles Allen 8fcf2403e3 Merge pull request #1943 from metamx/realtime-caching
Enable caching on intermediate realtime persists
2015-11-17 15:06:43 -08:00
sahner a4ed2ce2d1 fix formatting in schema-design 2015-11-17 16:50:53 -06:00
Himanshu 9118b010c0 Merge pull request #1976 from noddi/feature/count-parameter-rules-history
Add count parameter to rules history endpoint.
2015-11-17 14:41:50 -06:00
Charles Allen dbe201aeed Merge pull request #1929 from pjain1/jetty_threads
separate ingestion and query thread pool
2015-11-17 12:14:25 -08:00
Parag Jain 6c498b7d4a separate ingestion and query thread pool 2015-11-17 13:42:41 -06:00
Bartosz Ługowski b0a444eafc Add count parameter to rules history endpoint. 2015-11-17 20:14:12 +01:00