Roman Leventov
9625993c9a
Fix bugs in Google extensions and RocketMQ extension ( #4340 )
2017-05-30 14:25:35 -07:00
Jihoon Son
733dfc9b30
Add PrefetchableTextFilesFirehoseFactory for cloud storage types ( #4193 )
...
* Add PrefetcheableTextFilesFirehoseFactory
* fix comment
* exception handling
* Fix wrong json property
* Remove ReplayableFirehoseFactory and fix misspelling
* Defer object initialization
* Add a temporaryDirectory parameter to FirehoseFactory.connect()
* fix when cache and fetch are disabled
* Address comments
* Add more test
* Increase timeout for test
* Add wrapObjectStream
* Move methods to Firehose from PrefetchableFirehoseFactory
* Cleanup comment
* add directory listing to s3 firehose
* Rename a variable
* Addressing comments
* Update document
* Support disabling prefetch
* Fix race condition
* Add fetchLock
* Remove ReplayableFirehoseFactoryTest
* Fix compilation error
* Fix test failure
* Address comments
* Add default implementation for new method
2017-05-18 15:37:18 +09:00
Gian Merlino
2ca7b00346
Update versions to 0.10.1-SNAPSHOT. ( #4191 )
2017-04-20 18:12:28 -07:00
Roman Leventov
81a5f9851f
TmpFileIOPeons to create files under the merging output directory, instead of java.io.tmpdir ( #3990 )
...
* In IndexMerger and IndexMergerV9, create temporary files under the output directory/tmpPeonFiles, instead of java.io.tmpdir
* Use FileUtils.forceMkdir() across the codebase and remove some unused code
* Fix test
* Fix PullDependencies.run()
* Unused import
2017-03-02 14:05:12 -08:00
Gian Merlino
cc20133e70
Checkstyle rule to outlaw tabs. ( #3988 )
...
Tabs are the worst.
2017-02-28 23:52:53 -08:00
Akash Dwivedi
8854ce018e
File.deleteOnExit() ( #3923 )
...
* Less use of File.deleteOnExit()
* removed deleteOnExit from most of the tests/benchmarks/iopeon
* Made IOpeon closable
* Formatting.
* Revert DeterminePartitionsJobTest, remove cleanup method from IOPeon
2017-02-13 15:12:14 -08:00
Gian Merlino
12317fd001
Bump version to 0.10.0-SNAPSHOT. ( #3913 )
2017-02-06 17:54:35 -08:00
Himanshu
4ca3b7f1e4
overlord helpers framework and tasklog auto cleanup ( #3677 )
...
* overlord helpers framework and tasklog auto cleanup
* review comment changes
* further review comments addressed
2016-12-21 15:18:55 -08:00
Erik Dubbelboer
7d36f540e8
WIP: Add Google Storage support ( #2458 )
...
Also excludes the correct artifacts from #2741
2016-11-16 14:06:45 +05:30