Commit Graph

3674 Commits

Author SHA1 Message Date
fjy ef325f15ff update pom and fix broken UT 2014-07-17 13:47:59 -07:00
fjy c6078ca841 address code review 2014-07-17 13:34:05 -07:00
fjy ba978d8b79 some minor cleanups to ingest firehose 2014-07-17 13:05:59 -07:00
fjy bc650a1c80 Merge pull request #627 from metamx/druid-firehose
Functionality to ingest a Druid segment and change the schema
2014-07-17 13:41:16 -06:00
fjy 5197ea527a disable middlemanagers based on worker version 2014-07-17 12:35:45 -07:00
nishantmonu51 e59c9ebdbc minor fixes
fix IndexOutOfBoundsException
fix ingestFirehose
2014-07-17 17:24:57 +05:30
fjy 831d4e9515 update README 2014-07-16 15:38:30 -07:00
fjy bf71d079f3 fix #636 2014-07-16 15:20:20 -07:00
fjy b70a6b1061 address code review comments 2014-07-16 13:52:06 -07:00
fjy de8cb55260 Merge branch 'master' into rejigger-cache
Conflicts:
	pom.xml
2014-07-16 13:37:20 -07:00
fjy 0f3c474dee Merge branch 'master' of github.com:metamx/druid 2014-07-15 22:50:46 -07:00
fjy 534acc9eab fix mm 2014-07-15 22:50:38 -07:00
nishantmonu51 b74845e942 fix test
committed it by mistake.
2014-07-16 09:39:51 +05:30
fjy 935b642b2c fix overlord binding problems 2014-07-15 16:13:11 -07:00
fjy 27c4750780 fix infinite loop in test 2014-07-15 16:00:56 -07:00
fjy bdfeccd092 doc and timeout fix 2014-07-15 15:57:58 -07:00
fjy 6961abaa61 Merge pull request #634 from metamx/shardspec-optimization
Add ShardSpec Lookup
2014-07-15 15:10:24 -06:00
fjy 2f90bfcf9a rejiggering the cache for groupBy and select 2014-07-15 12:12:33 -07:00
Xavier Léauté 12bc3ac27b fix broken S3 moving test 2014-07-15 10:58:53 -07:00
fjy f2e0dc9327 Merge pull request #637 from metamx/fix-s3-storageclass-check
fix storage class npe
2014-07-14 20:23:56 -06:00
Xavier Léauté f56d60b451 fix storage class npe 2014-07-14 16:24:01 -07:00
nishantmonu51 972c5dac31 improve memory usage and rename firehose 2014-07-14 21:17:53 +05:30
nishantmonu51 a31376ee83 point to correct version 2014-07-10 12:23:20 +05:30
nishantmonu51 a12688bc8b fix partitionNum calculation & remove extra sorting 2014-07-10 12:21:27 +05:30
nishantmonu51 fa43049240 review comments & pom changes 2014-07-10 11:48:46 +05:30
Jae Hyeon Bae 193c7259f6 refresh thread should be daemon 2014-07-09 15:52:17 -07:00
Gian Merlino 7a7386be74 Merge pull request #633 from metamx/remove-config
Remove redundant forking task runner config
2014-07-08 11:30:40 -07:00
nishantmonu51 7168adcca7 Use Ingest task instead of Noop Task 2014-07-08 21:20:17 +05:30
nishantmonu51 a1a5e4254f CombiningFirehoseFactory
can be used to ingest data from multiple sources.
2014-07-08 21:14:48 +05:30
nishantmonu51 36fc85736c Add ShardSpec Lookup
Optimize choosing shardSpec for Hash Partitions
2014-07-08 18:01:31 +05:30
Jae Hyeon Bae 3a849b0341 update refresh manually every hour 2014-07-07 23:11:08 -07:00
fjy ce478be899 Remove redundant forking task runner config 2014-07-07 18:08:37 -06:00
fjy b8347cf4af Merge pull request #618 from realfun/master
Add HdfsTaskLogs
2014-07-04 14:17:51 -06:00
nishantmonu51 97b58eb193 review comments
1) Rename firehose to IngestSegment
2) fix segment overlapping, intervals
3) fix overshadow
2014-07-04 12:31:33 +05:30
Jae Hyeon Bae 4ab9af36b0 License header and utf-8 2014-07-03 16:52:47 -07:00
fjy 12afaae2df Merge branch 'master' of github.com:metamx/druid 2014-07-02 23:41:47 -06:00
fjy c1cd9ff2ce fix broken rename of rejection policies 2014-07-02 23:41:37 -06:00
fjy 19494eeb8c Merge pull request #630 from metamx/port-finder
improve port finding strategy for task runner
2014-07-02 23:41:19 -06:00
nishantmonu51 518ab473f3 improve port finding strategy for task runner
1) Recycle free ports
2) Choose only ports that are free & not used by any other application
2014-07-03 09:58:12 +05:30
Jae Hyeon Bae 5e5fa1f357 A few fixes
- EasyMock instead of Mockito
- FileSessionCredentialsProvider fields should be volatile
- getRestS3Service should create AWSCredentials not a AWSSessionCredentials with fixed credentials
2014-07-02 15:53:39 -07:00
Jae Hyeon Bae 31f7bd8bcb AWSCredentialsProvider for s3-extentions
- fileSessionCredentials property is added
- AWSSessionCredentialsAdapter is adapter class between aws sdk AWSCredentialsProvider to jets3t Credentials
- FileSessionCredentialsProvider will refresh its session credentials from the file
- S3StorageDruidModule should return different AWSCredentialsProvider on the config settings
2014-07-02 14:44:02 -07:00
fjy c646648319 fix examples start port 2014-07-02 10:07:04 -07:00
nishantmonu51 1de390801f Druid Fireshose
Add druidFirehose, can be used to reIndex filtered data from a
datasource
2014-07-02 20:34:18 +05:30
fjy d37527ca79 make histos work with groupby maybe 2014-07-01 16:46:58 -07:00
fjy dba7e972d8 refactor docs for histograms 2014-07-01 16:39:24 -07:00
fjy a16bdf9759 Merge pull request #625 from metamx/approx-histo
Approximate Histograms and Quantiles
2014-07-01 17:16:07 -06:00
Xavier Léauté d0b8494735 fix approximate histogram serde to not change buffer limit 2014-07-01 16:13:00 -07:00
Xavier Léauté 70ae6e1fcd Approximate Histogram Module 2014-07-01 15:10:23 -07:00
fjy 484b91db5a Merge pull request #617 from metamx/igalDruid
Igal druid
2014-06-30 19:58:38 -06:00
fjy 1c1f65ab5f Merge pull request #604 from metamx/more-tests
All manners of "new" tests
2014-06-30 19:41:59 -06:00