a2584d214a
Delagate creation of segmentPath/LoadSpec to DataSegmentPushers and add S3a support ( #4116 )
...
* Adding s3a schema and s3a implem to hdfs storage module.
* use 2.7.3
* use segment pusher to make loadspec
* move getStorageDir and makeLoad spec under DataSegmentPusher
* fix uts
* fix comment part1
* move to hadoop 2.8
* inject deep storage properties
* set version to 2.7.3
* fix build issue about static class
* fix comments
* fix default hadoop default coordinate
* fix create filesytem
* downgrade aws sdk
* bump the version
2017-06-04 00:55:09 -06:00
733dfc9b30
Add PrefetchableTextFilesFirehoseFactory for cloud storage types ( #4193 )
...
* Add PrefetcheableTextFilesFirehoseFactory
* fix comment
* exception handling
* Fix wrong json property
* Remove ReplayableFirehoseFactory and fix misspelling
* Defer object initialization
* Add a temporaryDirectory parameter to FirehoseFactory.connect()
* fix when cache and fetch are disabled
* Address comments
* Add more test
* Increase timeout for test
* Add wrapObjectStream
* Move methods to Firehose from PrefetchableFirehoseFactory
* Cleanup comment
* add directory listing to s3 firehose
* Rename a variable
* Addressing comments
* Update document
* Support disabling prefetch
* Fix race condition
* Add fetchLock
* Remove ReplayableFirehoseFactoryTest
* Fix compilation error
* Fix test failure
* Address comments
* Add default implementation for new method
2017-05-18 15:37:18 +09:00
4b3bd8bd63
Migrating java-util from Metamarkets. ( #3585 )
...
* Migrating java-util from Metamarkets.
* checkstyle and updated license on java-util files.
* Removed unused imports from whole project.
* cherry pick metamx/java-util@826021f .
* Copyright changes on java-util pom, address review comments.
2016-10-21 14:57:07 -07:00
485e381387
remove datasource from hadoop output path ( #3196 )
...
fixes #2083 , follow-up to #1702
2016-06-29 08:53:45 -07:00
45f553fc28
Replace the deprecated usage of NoneShardSpec ( #3166 )
2016-06-25 10:27:25 -07:00
e3e932a4d4
refactor extensions into core and contrib
2016-03-08 17:12:09 -08:00
Slim
Jihoon Son
Akash Dwivedi
Xavier Léauté
Hyukjin Kwon
fjy