Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,222 Commits 111 Branches 503 Tags 726 MiB
Commit Graph

6222 Commits

Author SHA1 Message Date
Gian Merlino e4e5f0375b SegmentAllocateAction (fixes #1515) 
This is a feature meant to allow realtime tasks to work without being told upfront
what shardSpec they should use (so we can potentially publish a variable number
of segments per interval).

The idea is that there is a "pendingSegments" table in the metadata store that
tracks allocated segments. Each one has a segment id (the same segment id we know
and love) and is also part of a sequence.

The sequences are an idea from @cheddar that offers a way of doing replication.
If there are N tasks reading exactly the same data with exactly the same logic
(think Kafka tasks reading a fixed range of offsets) then you can place them
in the same sequence, and they will generate the same sequence of segments.
 2015-11-11 16:54:35 -08:00
Charles Allen dc4ae59aaf Merge pull request #1865 from noddi/bugfix/issue-560 
Fix #560 - datasource segments summary shouldn't include replication.
 2015-11-11 15:29:08 -08:00
Bartosz Ługowski 6e5d2c6745 Add count parameter to history endpoints. 2015-11-11 23:03:57 +01:00
Fangjin Yang 9f78598672 Merge pull request #1950 from zhxiaogg/feature/ut_remote_task_action_client 
add unit test for RemoteTaskActionClientTest
 2015-11-11 12:29:54 -08:00
Gian Merlino 8e743b70c6 SegmentIdentifier, like a pre-DataSegment. 2015-11-11 12:25:43 -08:00
Xavier Léauté e26f7fca36 cleanup cache interface 2015-11-11 12:25:22 -08:00
Xavier Léauté fa6142e217 cleanup and remove unused imports 2015-11-11 12:25:21 -08:00
zhxiaog c197a4cf32 fix #1918, add unit tests for RemoteTaskActionClient 2015-11-12 03:15:17 +08:00
Fangjin Yang 9e12a501fe Merge pull request #1946 from gianm/local-storage-better-permission-error 
Better error message when LocalDataSegmentPusher cannot create its directory.
 2015-11-11 09:02:25 -08:00
Gian Merlino 07157f9e9d Merge pull request #1945 from dclim/hadoop-spatial-dim 
fix spatial dimension transformer to work with hadoop
 2015-11-10 20:01:20 -08:00
Gian Merlino f2c271e500 Better error message when LocalDataSegmentPusher cannot create its directory. 2015-11-10 19:49:01 -08:00
dclim fd0935ecb9 fix spatial dimension transformer to work with hadoop 2015-11-10 19:16:51 -07:00
Gian Merlino e75c2a407d Merge pull request #1944 from druid-io/fix-doc 
fix website rendering for this doc
 2015-11-10 16:04:40 -08:00
fjy e923de3eea fix website rendering for this doc 2015-11-10 15:36:30 -08:00
Xavier Léauté cf779946ef Merge pull request #1791 from guobingkun/event_receiver_firehose_monitor 
EventReceiverFirehoseMonitor
 2015-11-10 11:09:42 -08:00
Xavier Léauté a57cbfd2c3 Merge pull request #1387 from metamx/enableShutdownLogging 
Add special handler to allow logger messages during shutdown
 2015-11-09 17:20:09 -08:00
fjy 46bf1ba5ef remove unneeded 2015-11-09 17:03:00 -08:00
fjy 8a8bb0369e address more comments 2015-11-09 16:56:43 -08:00
fjy b99576d854 rework compares again 2015-11-09 16:40:07 -08:00
fjy 0b319093df New comparisons for Druid 2015-11-09 14:31:23 -08:00
Gian Merlino afd67a39a1 Merge pull request #1940 from metamx/repackageGuiceJackson 
Move Jackson Guice adapters into io.druid
 2015-11-09 14:08:38 -08:00
Fangjin Yang 78b42c1087 Merge pull request #1941 from guobingkun/custom_extension 
add doc for bundling custom extensions with other Druid extensions
 2015-11-09 11:32:33 -08:00
Bingkun Guo b24eccfb9e add doc for bundling custom extensions with other Druid extensions 2015-11-09 13:11:22 -06:00
Charles Allen 1df4baf489 Move Jackson Guice adapters into io.druid 
* Removes access to protected methods in com.fasterxml
* Eliminates druid-common's use of foreign package com.fasterxml
 2015-11-09 10:50:45 -08:00
Xavier Léauté 7816d67148 Merge pull request #1935 from gianm/unbuffered-ftr-logging 
ForkingTaskRunner: Log without buffering.
 2015-11-07 17:17:24 -08:00
Fangjin Yang fe6efb6367 Merge pull request #1932 from gianm/data-schema-exclude-metric-names 
DataSchema: Exclude metric names from dimension list.
 2015-11-07 15:22:33 -08:00
Gian Merlino fc55314d1c ForkingTaskRunner: Log without buffering. 
In #933 the ForkingTaskRunner's logging was changed to buffered from
unbuffered. This means that the last few KB of the logs are generally
not visible while a task is running, which makes debugging running
tasks difficult.
 2015-11-07 15:16:53 -08:00
Bartosz Ługowski 8b5de492cc Fix #560 - datasource segments summary shouldn't include replication. 2015-11-07 21:43:47 +01:00
Gian Merlino 6b908a5061 DataSchema: Exclude metric names from dimension list. 
Otherwise we could end up with a metric and a dimension with the same column name.
 2015-11-07 12:25:36 -08:00
Charles Allen 61139b9dfa Merge pull request #1912 from b-slim/in_filter 
Adding in filter
 2015-11-06 14:47:47 -08:00
Slim Bouguerra c511273efd adding in filter 2015-11-06 16:23:24 -06:00
Xavier Léauté e9533db987 Merge pull request #1850 from metamx/friendlyBardCache 
Allow setting upper limit on the number of cache segments a broker will try to fetch.
 2015-11-06 10:25:49 -08:00
Gian Merlino dfbd0e2b60 Merge pull request #1925 from gianm/fix-index-generator 
Fix reference to INDEX_MAKER in IndexGeneratorJob.
 2015-11-06 09:56:30 -08:00
Gian Merlino 0e75642e01 Merge pull request #1923 from himanshug/fix_index_generator_job 
remove LoggingProgressIndicator from IndexGeneratorJob
 2015-11-06 09:26:06 -08:00
Charles Allen cf8b9a8613 Merge pull request #1920 from metamx/upgrade-curator 
Update curator to 2.9.1
 2015-11-06 09:25:27 -08:00
Gian Merlino 75122dc396 Fix reference to INDEX_MAKER in IndexGeneratorJob. 2015-11-06 09:19:58 -08:00
Himanshu Gupta 6bed633121 do not use LoggingProcessIndicator in IndexGeneratorJob because that uses Stopwatch methods from guava not available in older guava versions, this makes the behavior same as LegacyIndexGeneratorJob 2015-11-06 00:40:51 -06:00
Xavier Léauté c896818241 Update curator to 2.9.1 
Lots of bugfixes since 2.8.0
- https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314425&version=12333324
- https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314425&version=12332392
 2015-11-05 15:53:01 -08:00
Fangjin Yang 379ca87e6a Merge pull request #1919 from metamx/old-todo 
fix a very old todo
 2015-11-05 13:33:06 -08:00
Xavier Léauté 223d1ebe9f fix a very old todo 2015-11-05 13:00:30 -08:00
Xavier Léauté 7cbff731e3 Merge pull request #1917 from gianm/math-is-hard 
RemoteTaskActionClient: Fix statusCode check.
 2015-11-05 11:25:41 -08:00
Gian Merlino cb409ee928 RemoteTaskActionClient: Fix statusCode check. 2015-11-05 10:03:49 -08:00
Xavier Léauté 2d4d3a40de Merge pull request #1913 from druid-io/fix-zkc 
fix zkcoordinator inject
 2015-11-04 21:34:30 -08:00
fjy 4efc90dda3 fix zkcoordinator inject 2015-11-04 17:30:17 -08:00
Himanshu 3c3ea128cc Merge pull request #1895 from druid-io/cleanup-codebase 
Cleanup the Druid codebase
 2015-11-04 17:16:33 -06:00
fjy 8f231fd3e3 cleanup druid codebase 2015-11-04 13:59:53 -08:00
Fangjin Yang 1cbc514eb5 Merge pull request #1719 from se7entyse7en/feature-rackspace-cloud-files-deep-storage 
Add Rackspace Cloud Files Deep Storage Extension
 2015-11-04 11:13:25 -08:00
Fangjin Yang 861338c5e9 Merge pull request #1904 from rohitkochar/doc 
modifying the documentation as per Issue#1903
 2015-11-04 09:48:45 -08:00
Gian Merlino af12c44880 Merge pull request #1911 from himanshug/fix_topn_doc 
since topN is descending by default, inversion makes it ascending
 2015-11-04 09:38:49 -08:00
Himanshu Gupta 67c218bed6 since topN is descending by default, inversion makes is ascending 2015-11-04 11:36:49 -06:00