Steve Loughran
bdde7edba5
HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong
2016-07-23 17:59:50 +01:00
mattf-horton
ea10e1384f
HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley.
2016-07-21 15:58:05 -07:00
Steve Loughran
d3df810dce
HADOOP-13387 users always get told off for using S3 —even when not using it. Contributed by stevel.
2016-07-20 14:01:30 +01:00
Steve Loughran
5490ed5f87
HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan)
2016-07-20 13:42:33 +01:00
Jitendra Pandey
328866905c
HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu.
2016-07-15 14:29:59 -07:00
Jian He
e94e6be772
YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan
2016-07-11 22:37:52 -07:00
Andrew Wang
38b1eafdbc
HDFS-10300. TestDistCpSystem should share MiniDFSCluster. Contributed by John Zhuge.
...
(cherry picked from commit f292624bd8
)
2016-07-11 18:06:35 -07:00
Steve Loughran
89121745c0
HADOOP-13139 Branch-2: S3a to use thread pool that blocks clients. Contributed by Pieter Reuse.
2016-07-11 17:21:00 +01:00
Akira Ajisaka
f4b63a391b
Revert "HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux."
...
This reverts commit 1d175f0f3f
.
2016-07-09 21:00:13 -07:00
Jitendra Pandey
63b64a20c2
HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.
2016-07-07 15:00:34 -07:00
Jitendra Pandey
06b89f0d14
HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu.
2016-07-01 15:35:50 -07:00
Steve Loughran
70a45e3122
HADOOP-13239 Deprecate s3:// in branch-2
2016-06-29 13:16:26 +01:00
Yongjun Zhang
8a121dc3b8
HDFS-10396. Using -diff option with DistCp may get "Comparison method violates its general contract" exception. Contributed by Yongjun Zhang.
...
(cherry picked from commit 8113855b3a
)
2016-06-28 23:29:48 -07:00
Jing Zhao
856bc4e28b
HDFS-10556. DistCpOptions should be validated automatically. Contributed by Mingliang Liu.
2016-06-22 15:15:47 -07:00
Steve Loughran
fb9167fdef
HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel.
2016-06-22 15:42:30 +01:00
Chris Nauroth
e6ebe9ab4e
HADOOP-13287. TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains +. Contributed by Chris Nauroth.
...
(cherry picked from commit b2c596cdda
)
2016-06-21 11:32:11 -07:00
Jitendra Pandey
8f93bbcb11
HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu.
2016-06-20 16:28:55 -07:00
Chris Nauroth
e4b63ed7b4
HADOOP-13241. document s3a better. Contributed by Steve Loughran.
...
(cherry picked from commit 127d2c7281
)
2016-06-16 11:18:40 -07:00
Ravi Prakash
b216dbfb55
HADOOP-3733. "s3x:" URLs break when Secret Key contains a slash, even if encoded. Contributed by Steve Loughran.
...
(cherry picked from commit 4aefe119a0
)
2016-06-16 11:14:38 -07:00
Wangda Tan
863bfa4d6a
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:12:00 -07:00
Chris Nauroth
1925942294
Revert "HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane."
...
This reverts commit a8f03ef7ea
.
2016-06-09 15:17:18 -07:00
Chris Nauroth
a8f03ef7ea
HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane.
...
(cherry picked from commit 9581fb715c
)
Conflicts:
hadoop-tools/hadoop-tools-dist/pom.xml
hadoop-tools/pom.xml
2016-06-09 14:49:05 -07:00
Steve Loughran
6350e4b1fb
HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory.
2016-06-09 20:59:54 +01:00
Steve Loughran
df29f77846
HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth
2016-06-09 16:36:27 +01:00
Steve Loughran
0cbbc2f0db
HADOOP-12807 S3AFileSystem should read AWS credentials from environment variables. Contributed by Tobin Baker.
2016-06-06 23:41:54 +02:00
Chris Nauroth
043a0c2e6c
HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran.
2016-06-03 08:56:07 -07:00
Chris Nauroth
e9942c5b34
HADOOP-13131. Add tests to verify that S3A supports SSE-S3 encryption. Contributed by Steve Loughran.
...
(cherry picked from commit 16b1cc7af9
)
2016-06-01 14:49:31 -07:00
Steve Loughran
b29bf5d335
HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel)
2016-06-01 14:17:50 +01:00
Varun Vasudev
4f36c3d214
YARN-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 20:55:39 +05:30
Yongjun Zhang
82a88c7e67
HADOOP-13199. Add doc for distcp -filters. (John Zhuge via Yongjun Zhang)
...
(cherry picked from commit cfb860dee7
)
2016-05-26 23:32:46 -07:00
Steve Loughran
15f997cca6
HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve Loughran)
2016-05-21 14:19:56 +01:00
Steve Loughran
eb160561e0
HADOOP-12723 S3A: Add ability to plug in any AWSCredentialsProvider. Contributed by Steven Wong.
2016-05-20 13:52:30 +01:00
Steve Loughran
e29d7fb585
HADOOP-13145 In DistCp, prevent unnecessary getFileStatus call when not preserving metadata. Contributed by Chris Nauroth.
2016-05-20 12:21:47 +01:00
Chris Nauroth
60e5397a9b
HADOOP-13183. S3A proxy tests fail after httpclient/httpcore upgrade. Contributed by Steve Loughran.
...
(cherry picked from commit 93258459fa
)
2016-05-19 22:00:32 -07:00
Arun Suresh
1b4600abcf
YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh)
...
(cherry picked from commit 341888a0aa
)
2016-05-18 22:35:55 -07:00
Jing Zhao
dd7f5a3b88
HDFS-10397. Distcp should ignore -delete option if -diff option is provided instead of exiting. Contributed by Mingliang Liu.
...
(cherry picked from commit 03788d3015
)
2016-05-17 15:53:39 -07:00
Steve Loughran
dd0a8201bc
HADOOP-13158 S3AFileSystem#toString might throw NullPointerException due to null cannedACL. (Chris Nauroth via stevel)
2016-05-17 13:19:08 +01:00
Steve Loughran
dfc062a092
HADOOP-13163 Reuse pre-computed filestatus in Distcp-CopyMapper (Rajesh Balamohan via stevel)
2016-05-17 13:01:50 +01:00
Chris Nauroth
bb41547fb4
HADOOP-13148. TestDistCpViewFs to include IOExceptions in test error reports. Contributed by Steve Loughran.
...
(cherry picked from commit b9685e85d5
)
2016-05-16 11:53:25 -07:00
Steve Loughran
f14208d362
HADOOP-13113 Enable parallel test execution for hadoop-aws. Chris Nauroth via stevel
2016-05-13 10:46:37 +01:00
Andrew Wang
f6f20616c6
HADOOP-12868. Fix hadoop-openstack undeclared and unused dependencies. Contributed by Masatake Iwasaki.
...
(cherry picked from commit fa440a39b7
)
2016-05-12 23:58:24 -07:00
Steve Loughran
1370dfc775
HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel
...
patch includes
HADOOP-12844 Recover when S3A fails on IOException in read()
HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge
HADOOP-13047 S3a Forward seek in stream length to be configurable
2016-05-12 19:23:31 +01:00
Steve Loughran
389dd91dd4
HADOOP-13116 Jets3tNativeS3FileSystemContractTest does not run.
2016-05-12 16:36:23 +01:00
Steve Loughran
257b2f5beb
HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel.
2016-05-12 13:57:05 +01:00
Chris Nauroth
ba48834431
HADOOP-12801. Suppress obsolete S3FileSystem tests. Contributed by Chris Nauroth.
...
(cherry picked from commit d3dbfa7feee7922ddcd0b2e97898adf4e0c37de3)
2016-05-10 15:14:36 -07:00
Steve Loughran
f47ea0cc3f
HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel)
2016-05-10 21:40:13 +01:00
Jian He
007d6d1d57
YARN-4778. Support specifying resources for task containers in SLS. Contributed by Wangda Tan
...
(cherry picked from commit 996a210ab0
)
2016-05-09 14:50:00 -07:00
Jason Lowe
ee86cef2fe
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
...
(cherry picked from commit d0da13229c
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
2016-05-05 14:33:01 +00:00
Jing Zhao
d43d8a1bcd
HADOOP-12469. distcp should not ignore the ignoreFailures option. Contributed by Mingliang Liu.
...
(cherry picked from commit af942585a1
)
2016-05-04 10:24:06 -07:00
Masatake Iwasaki
ff233dfca0
HADOOP-12378. Fix findbugs warnings in hadoop-tools module. Contributed by Akira AJISAKA.
...
(cherry picked from commit ef0870ad03
)
2016-04-29 20:27:45 +09:00
Yongjun Zhang
7955b6f66e
HDFS-10313. Distcp need to enforce the order of snapshot names passed to -diff. (Lin Yiqun via Yongjun Zhang)
...
(cherry picked from commit 959a28dd12
)
2016-04-26 16:27:59 -07:00
Akira Ajisaka
58642d2244
HDFS-10298. Document the usage of distcp -diff option. Contributed by Takashi Ohnishi.
...
(cherry picked from commit 02c51c27d9
)
2016-04-25 22:34:18 +09:00
Steve Loughran
edcb2a8410
HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel)
2016-04-22 11:24:50 +01:00
Yongjun Zhang
f98f4151ac
HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang)
...
(cherry picked from commit a749ba0cea
)
2016-04-21 12:23:42 -07:00
Jing Zhao
864baf23aa
HDFS-10216. Distcp -diff throws exception when handling relative path. Contributed by Takashi Ohnishi.
...
(cherry picked from commit 404f57f328
)
2016-04-14 10:35:43 -07:00
Steve Loughran
e124c3a2ae
HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel)
2016-04-14 12:45:10 +01:00
Jason Lowe
f1a370ce8b
Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla"
...
This reverts commit 814ceeb489
.
2016-04-11 15:56:29 +00:00
Steve Loughran
9acfde3a03
HADOOP-12444 Support lazy seek in S3AInputStream. Rajesh Balamohan via stevel
2016-04-09 11:24:55 +01:00
Chris Nauroth
26a23eff82
HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran.
...
(cherry picked from commit 843ee8d59d
)
2016-04-08 13:37:46 -07:00
Steve Loughran
1533403a9a
HADOOP-12753 S3A JUnit tests failing if using HTTP proxy. (Zoran Rajic via stevel)
2016-04-08 17:09:47 +01:00
Jason Lowe
814ceeb489
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
...
(cherry picked from commit 1cbcd4a491
)
2016-04-05 13:41:18 +00:00
Steve Loughran
caaadf9f3e
HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse. - omitted new S3A subclass
2016-04-04 17:01:37 +01:00
Steve Loughran
20e808d83e
HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse.
2016-04-03 16:39:51 +01:00
Harsh J
7c5b55d4e5
HADOOP-11687. Ignore x-* and response headers when copying an Amazon S3 object. Contributed by Aaron Peterson and harsh.
...
(cherry picked from commit 256c82fe29
)
2016-04-01 14:35:58 +05:30
Akira Ajisaka
7b2cf29f4f
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
...
(cherry picked from commit 0050fa5f1c
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
2016-03-29 18:20:01 +09:00
Akira Ajisaka
2b1f066d2d
Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote."
...
This reverts commit 0dc1453626
.
2016-03-28 16:05:55 +09:00
Akira Ajisaka
0dc1453626
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
...
(cherry picked from commit 2c268cc936
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
2016-03-25 15:23:23 +09:00
Chris Nauroth
57a0e6543c
HDFS-10199. Unit tests TestCopyFiles, TestDistCh, TestLogalyzer under org.apache.hadoop.tools are failing. Contributed by Tibor Kiss.
2016-03-23 12:17:27 -07:00
Chris Nauroth
2e60387401
HADOOP-12717. NPE when trying to rename a directory in Windows Azure Storage FileSystem. Contributed by Robert Yokota and Gaurav Kanade.
...
(cherry picked from commit c50aad0f85
)
2016-03-04 22:57:56 -08:00
Masatake Iwasaki
722182fdfb
HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
...
(cherry picked from commit cbd31328a6
)
Conflicts:
hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md
2016-03-04 14:12:44 +09:00
Masatake Iwasaki
8095c612a3
HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims)
...
(cherry picked from commit 33a412e8a4
)
2016-03-03 18:57:41 +09:00
Chris Nauroth
bce33b5179
HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty.
...
(cherry picked from commit f98dff329b
)
2016-03-01 13:18:57 -08:00
Chris Nauroth
e402371b6a
HADOOP-12851. S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders. Contributed by Larry McCay.
...
(cherry picked from commit d251e55415
)
2016-02-29 22:00:27 -08:00
Masatake Iwasaki
32c7791a6c
HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims)
...
(cherry picked from commit 809ebc0b14
)
2016-02-29 18:23:33 +09:00
Chris Nauroth
2e761dabd8
HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay.
...
(cherry picked from commit 7634d404b7
)
2016-02-28 11:23:02 -08:00
cnauroth
9ea19a861f
HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty.
...
(cherry picked from commit 3369a4f691
)
2016-02-23 21:37:56 -08:00
cnauroth
a5511debdb
HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay.
...
(cherry picked from commit 27b77751c1
)
2016-02-22 17:30:32 -08:00
cnauroth
3f14eed580
HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki.
...
(cherry picked from commit d4f5fc23b2
)
2016-02-22 13:39:45 -08:00
Yongjun Zhang
b144e0f15d
HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang)
...
(cherry picked from commit ba1c9d484a
)
2016-02-19 10:29:27 -08:00
cnauroth
b4c4f30fa4
HADOOP-11031. Design Document for Credential Provider API. Contributed by Larry McCay.
...
(cherry picked from commit 8ab7658025
)
2016-02-18 14:07:09 -08:00
cnauroth
6731bb468d
HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay.
...
(cherry picked from commit 76fab26c5c
)
Conflicts:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
2016-02-17 12:20:41 -08:00
cnauroth
ac5a793590
HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty.
...
(cherry picked from commit 91a96eaa53
)
2016-02-12 15:50:19 -08:00
Steve Loughran
60474a769e
HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel)
2016-02-06 18:20:24 +00:00
Yongjun Zhang
2fb711bde9
HDFS-9638. Improve DistCp Help and documentation. (Wei-Chiu Chuang via Yongjun Zhang)
...
(cherry picked from commit eddd823cd6
)
2016-01-29 12:13:03 -08:00
cnauroth
bd01fdb64d
MAPREDUCE-6563. Streaming documentation contains a stray % character. Contributed by Chris Nauroth.
...
(cherry picked from commit 41da9a0773
)
Conflicts:
hadoop-mapreduce-project/CHANGES.txt
2016-01-28 14:46:42 -08:00
Xuan
69f3fa71a2
YARN-4612. Fix rumen and scheduler load simulator handle killed tasks
...
properly. Contributed by Ming Ma.
(cherry picked from commit 4efdf3a979
)
2016-01-26 18:18:03 -08:00
Akira Ajisaka
4edd7bd820
HADOOP-12731. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
...
(cherry picked from commit 736eb17a79
)
2016-01-25 13:48:28 +09:00
Akira Ajisaka
6368d102b9
HADOOP-12730. Hadoop streaming -mapper and -reducer options are wrongly documented as required. Contributed by Kengo Seki.
...
(cherry picked from commit 0bae506c22
)
2016-01-22 21:09:09 +09:00
Akira Ajisaka
7f215ffdd5
MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki.
2016-01-22 18:37:17 +09:00
Xuan
66b030f608
YARN-4611. Fix scheduler load simulator to support multi-layer network
...
location. Contributed by Ming Ma.
(cherry picked from commit 1708a4cd23
)
2016-01-20 15:43:02 -08:00
Ravi Prakash
2cbb8bbd72
HADOOP-12696. Add tests for S3FileSystem Contract. Contributed by Matt Paduano
...
(cherry picked from commit 1acc509b45
)
2016-01-19 14:00:54 -08:00
Wangda Tan
23deeb4230
HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda)
...
(cherry picked from commit 89d1fd5dac
)
2016-01-19 21:28:51 +08:00
Wangda Tan
704b37d0e1
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
...
(cherry picked from commit a44ce3f14f
)
2016-01-19 09:31:13 +08:00
Wangda Tan
1625281ccb
Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)"
...
This reverts commit 3fe5728563
.
Conflicts:
hadoop-yarn-project/CHANGES.txt
(cherry picked from commit 150f5ae034
)
2016-01-19 09:31:07 +08:00
cnauroth
62d6166211
HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth.
...
(cherry picked from commit 8bc93db2e7
)
2016-01-18 09:09:05 -08:00
Wangda Tan
6dab78c71a
YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)
...
(cherry picked from commit 805a9ed85e
)
2016-01-18 11:15:59 +08:00
Yongjun Zhang
ee012c1f6f
HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang)
...
(cherry picked from commit a9c69ebeb7
)
2016-01-15 10:10:28 -08:00
Colin Patrick Mccabe
cea1c71087
HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe)
...
(cherry picked from commit b1ed28fa77
)
Conflicts:
hadoop-common-project/hadoop-common/pom.xml
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
2016-01-14 11:13:13 -08:00
Colin Patrick Mccabe
2baeaa20a3
HDFS-9517. Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe)
...
(cherry picked from commit 8315582c4f
)
2016-01-13 16:33:02 -08:00
Lei Xu
63b8be149a
HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei)
2016-01-12 12:21:29 -08:00
Xiaoyu Yao
521f60c094
HDFS-8584. NPE in distcp when ssl configuration file does not exist in class path. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit c2e2e13455
)
2016-01-11 17:12:43 -08:00
Zhe Zhang
17dfa04be6
HDFS-9630. DistCp minor refactoring and clean up. Contributed by Kai Zheng.
...
Change-Id: I363c4ffcac32116ddcdc0a22fac3db92f14a0db0
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2016-01-11 09:53:20 -08:00
cnauroth
813841e10b
HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth.
...
(cherry picked from commit 0e76f1fcea
)
2016-01-09 22:18:22 -08:00
cnauroth
debd13387d
HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty.
...
(cherry picked from commit f0fa6d869b
)
2016-01-08 20:19:11 -08:00
cnauroth
4753676a34
HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade.
...
(cherry picked from commit 978bbdfeb2
)
2016-01-06 11:16:18 -08:00
Ravi Prakash
6330683778
HADOOP-12689. S3 filesystem operations stopped working correctly
...
(cherry picked from commit 2d16f40dab
)
2016-01-05 23:43:17 -08:00
Wangda Tan
6eefae1b33
YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda)
2015-12-30 15:34:39 -08:00
Wangda Tan
9a7d5f69ec
YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda)
...
(cherry picked from commit 8310b2e9ff
)
2015-12-30 15:31:11 -08:00
Xiaoyu Yao
a65e159c59
HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth.
...
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
2015-12-30 11:12:02 -08:00
Wangda Tan
9f4a6ec663
YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda)
...
(cherry picked from commit 79c41b1d83
)
2015-12-16 13:28:56 -08:00
Vinod Kumar Vavilapalli
b20e77eba3
Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0
2015-11-25 17:47:22 -08:00
Robert Kanter
7b4bf23b2a
MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor (rkanter)
...
(cherry picked from commit 6d84cc16b3
)
2015-11-25 17:13:00 -08:00
Akira Ajisaka
4257991257
HADOOP-12598. Add XML namespace declarations for some hadoop/tools modules. Contributed by Xin Wang.
...
(cherry picked from commit b53f20135599cc7825b6213794e2f999bc43e2ad)
2015-11-26 01:21:32 +09:00
Karthik Kambatla
49ed955c91
YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha)
...
(cherry picked from commit 52948bb20b
)
2015-11-24 13:52:29 +05:30
Haohui Mai
1d175f0f3f
HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux.
2015-11-22 18:12:53 -08:00
Haohui Mai
ac0694eb6e
HADOOP-12098. Remove redundant test dependencies in Hadoop Archives. Contributed by Varun Saxena.
2015-11-22 17:25:29 -08:00
Haohui Mai
f504d95e87
HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su.
2015-11-22 15:57:23 -08:00
Karthik Kambatla
d3f9508096
YARN-4367. SLS webapp doesn't load. (kasha)
2015-11-18 13:45:00 -08:00
Akira Ajisaka
c5c663c05d
HADOOP-12545. Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp. Contributed by Arshad Mohammad.
...
(cherry picked from commit f94d892704
)
2015-11-13 21:53:23 +09:00
Lei Xu
47941858af
Revert "HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei)"
...
This reverts commit 01ae30796d
.
2015-11-06 09:52:21 -08:00
Lei Xu
01ae30796d
HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei)
...
(cherry picked from commit bff7c90a56
)
2015-11-05 18:36:55 -08:00
cnauroth
81f7e8af22
HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade.
...
(cherry picked from commit 0fb1867fd6
)
2015-11-04 10:29:01 -08:00
cnauroth
c1d7b26e94
HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade.
...
(cherry picked from commit 9e7dcab185
)
2015-11-02 10:21:46 -08:00
cnauroth
c260d7ec78
HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth.
...
(cherry picked from commit 28790692624177d89fb1e4f59e2f83a659fc3089)
Conflicts:
hadoop-common-project/hadoop-common/CHANGES.txt
2015-11-02 09:39:27 -08:00
Kihwal Lee
dae9f0ef53
Addendum to MAPREDUCE-6451
...
(cherry picked from commit b24fe06483
)
2015-10-30 16:06:50 -05:00
Kihwal Lee
52819fe4ef
MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla.
...
(cherry picked from commit 2868ca0328
)
2015-10-30 14:57:57 -05:00
cnauroth
c7f87dc2d0
HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth.
...
(cherry picked from commit 2d10cb8e00
)
2015-10-28 09:44:21 -07:00
cnauroth
0377795e06
HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu.
...
(cherry picked from commit 1f7ecb0c84
)
2015-10-27 22:56:33 -07:00
cnauroth
a23f79b92c
HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth.
...
(cherry picked from commit 73822de7c3
)
2015-10-27 22:49:04 -07:00
Varun Vasudev
c5bf1cb7af
YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt.
...
(cherry picked from commit c28e16b40c
)
2015-10-27 20:56:45 +05:30
cnauroth
b023539c19
HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade.
...
(cherry picked from commit cb282d5b89
)
2015-10-22 14:30:09 -07:00
cnauroth
10398daadf
HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade.
...
(cherry picked from commit 47641fcbc9
)
2015-10-22 12:21:44 -07:00
Robert Kanter
492142097b
MAPREDUCE-6495. Docs for archive-logs tool (rkanter)
...
(cherry picked from commit 0c4af0f998
)
2015-10-20 17:34:56 -07:00
Haohui Mai
ee04add696
HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu.
2015-10-13 15:24:29 -07:00
Haohui Mai
f6b0fcdc75
Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu."
...
This reverts commit a412edf5c4
.
2015-10-10 03:25:16 -07:00
Haohui Mai
a412edf5c4
HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu.
2015-10-09 22:38:13 -07:00
Robert Kanter
e5bf4594cc
MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME (rkanter)
...
(cherry picked from commit 1bca1bb517
)
2015-10-06 16:50:18 -07:00
cnauroth
535762fadb
HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth.
...
(cherry picked from commit 5f6edb30c2
)
2015-10-05 20:12:33 -07:00
Akira Ajisaka
c6f30e5913
HADOOP-10296. Incorrect null check in SwiftRestClient#buildException(). Contributed by Rahul Palamuttam and Kanaka Kumar Avvaru.
...
(cherry picked from commit ecbfd68974
)
2015-10-01 18:35:19 +02:00
Robert Kanter
46ef5aa8eb
MAPREDUCE-6494. Permission issue when running archive-logs tool as different users (rkanter)
...
(cherry picked from commit 5db371f52f
)
2015-09-30 17:34:03 -07:00
Kihwal Lee
95b8f1f3a6
HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu.
...
(cherry picked from commit 3abbdc929b
)
2015-09-28 17:33:20 -05:00
Harsh J
03e370e306
MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian.
...
(cherry picked from commit 66dad854c0
)
2015-09-28 13:14:20 +05:30
cnauroth
85c0cb0075
HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu.
...
(cherry picked from commit 7fe521b1dd
)
2015-09-25 22:34:10 -07:00
Robert Kanter
d8a5d2b2fc
MAPREDUCE-6480. archive-logs tool may miss applications (rkanter)
...
(cherry picked from commit 8fec882cfaac047af1892615fe32d8c3ca02e7f9)
2015-09-25 15:00:07 -07:00
Jian He
d458805690
YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan
...
(cherry picked from commit 89cab1ba5f
)
Conflicts:
hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
2015-09-24 10:30:04 -07:00
Harsh J
38420754f5
MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused. Contributed by Neelesh Srinivas Salian.
...
(cherry picked from commit de1322aa76007832698af682a3b76042116043b9)
2015-09-22 21:38:31 +05:30
Tsuyoshi Ozawa
52cc2ca488
HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
...
(cherry picked from commit dfd807afab
)
2015-09-22 12:54:44 +09:00
Steve Loughran
988749e6ea
HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel).
2015-09-13 14:25:06 +01:00
Karthik Kambatla
999aa22509
MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files. (Robert Kanter via kasha)
2015-09-09 17:49:23 -07:00
Tsuyoshi Ozawa
7f1abf0b27
HADOOP-10318. Incorrect reference to nodeFile in RumenToSLSConverter error message. Contributed by Wei Yan.
...
(cherry picked from commit 355eaaa33d
)
2015-09-03 22:49:12 +09:00
Lei Xu
de42fce390
HADOOP-12346. Increase some default timeouts / retries for S3a connector. (Sean Mackrory via Lei (Eddy) Xu)
2015-08-29 09:58:57 -07:00
Rohith Sharma K S
6dc732f2f7
YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks)
...
(cherry picked from commit feaf034994
)
2015-08-24 11:30:46 +05:30
Yongjun Zhang
b085c5ef6f
HDFS-8828. Utilize Snapshot diff report to build diff copy list in distcp. (Yufei Gu via Yongjun Zhang)
...
(cherry picked from commit 0bc15cb6e6
)
2015-08-20 08:10:34 -07:00
cnauroth
b83d475252
HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth.
...
(cherry picked from commit 3da0bedaec
)
2015-08-05 14:56:54 -07:00
Lei Xu
f7b0f292e7
HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu)
...
(cherry picked from commit d5403747b5
)
2015-08-04 18:56:07 -07:00
cnauroth
bb080b32d4
HADOOP-12239. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.
...
(cherry picked from commit efa97243ec
)
2015-07-22 11:17:00 -07:00
Vinayakumar B
03d68b5575
HADOOP-12017. Hadoop archives command should use configurable replication factor when closing (Contributed by Bibin A Chundatt)
...
(cherry picked from commit 94c6a4aa85
)
2015-07-22 10:27:03 +05:30
Wangda Tan
3dd113fa7c
YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda)
...
(cherry picked from commit c39ca541f4
)
2015-07-21 09:57:53 -07:00
Vinayakumar B
8ef3821356
HADOOP-12214. Parse 'HadoopArchive' commandline using cli Options. (Contributed by Vinayakumar B)
...
(cherry picked from commit 87f29c6b8a
)
2015-07-21 13:13:31 +05:30
Steve Loughran
f3296a1984
HADOOP-12235 hadoop-openstack junit & mockito dependencies should be "provided". (Ted Yu via stevel)
2015-07-20 11:22:22 +01:00
Chris Douglas
c6b0bbabc7
HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl
...
(cherry picked from commit a431ed9075
)
2015-07-13 15:36:25 -07:00
Ming Ma
b169889f01
YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma)
...
(cherry picked from commit 08244264c0
)
2015-07-10 08:34:01 -07:00
Chris Douglas
66419d3672
HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl
...
(cherry picked from commit 1a0752d85a
)
2015-07-09 17:48:56 -07:00
Robert Kanter
3f79ee7494
HADOOP-12159. Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces (rchiang via rkanter)
...
(cherry picked from commit aaafa0b2ee
)
2015-06-30 16:50:42 -07:00
cnauroth
399df10c58
HADOOP-12089. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.
...
(cherry picked from commit 460e98f7b3
)
2015-06-29 13:49:12 -07:00
Colin Patrick Mccabe
df58fdaf21
HADOOP-12104. Migrate Hadoop Pipes native build to new CMake framework (Alan Burlison via Colin P. McCabe)
...
(cherry picked from commit 5a27c3fd76
)
2015-06-29 12:31:43 -07:00
cnauroth
f5b0cce7fa
HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic.
...
(cherry picked from commit c45784bc90
)
2015-06-08 22:42:24 -07:00
Andrew Wang
83a97907cc
HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey.
...
(cherry picked from commit 2dbc40e608
)
2015-06-05 13:11:12 -07:00
Robert Kanter
ee3002933a
MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter)
...
(cherry picked from commit cc70df98e7
)
2015-06-03 18:42:16 -07:00
cnauroth
39f451e721
HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic.
...
(cherry picked from commit 94e7d49a6d
)
Conflicts:
hadoop-project/pom.xml
2015-05-28 12:32:55 -07:00
Colin Patrick Mccabe
3dec58dd78
HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe)
...
(cherry picked from commit 7dba7005b7
)
2015-05-26 12:40:41 -07:00
Vinod Kumar Vavilapalli
a67cb4826b
YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev.
...
(cherry picked from commit 500a1d9c76
)
2015-05-26 11:39:40 -07:00
Colin Patrick Mccabe
97ebe1b2cf
HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe)
...
(cherry picked from commit f34638399a
)
2015-05-22 12:57:29 -07:00
Jing Zhao
5caea4cd46
HADOOP-1540. Support file exclusion list in distcp. Contributed by Rich Haase.
...
(cherry picked from commit 0790275f05
)
2015-05-18 13:26:09 -07:00
Akira Ajisaka
b26d99c394
HADOOP-11939. Deprecate DistCpV1 and Logalyzer. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit f5c48238d5
)
2015-05-18 16:14:19 +09:00
cnauroth
91855c2340
HADOOP-11960. Enable Azure-Storage Client Side logging. Contributed by Dushyanth.
...
(cherry picked from commit cb8e69a80c
)
2015-05-14 22:22:46 -07:00
Akira Ajisaka
b9a92c4452
HADOOP-9723. Improve error message when hadoop archive output path already exists. Contributed by Jean-Baptiste Onofré and Yongjun Zhang.
...
(cherry picked from commit 92c38e41e1
)
2015-05-13 17:30:49 +09:00
Vinayakumar B
14d88a85e4
HADOOP-11922. Misspelling of threshold in log4j.properties for tests in hadoop-tools (Contributed by Gabor Liptak)
...
(cherry picked from commit f33efbc827
)
2015-05-08 10:23:27 +05:30
Jonathan Eagles
9ed6b77da1
HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles)
...
(cherry picked from commit 2d7363b273
)
Conflicts:
pom.xml
2015-05-01 19:04:41 -07:00
Akira Ajisaka
47ba08a119
HADOOP-11821. Fix findbugs warnings in hadoop-sls. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit f384a063a6
)
2015-04-30 19:49:32 +09:00
Ravi Prakash
122262a1fb
HADOOP-11876. Refactor code to make it more readable, minor maybePrintStats bug (Zoran Dimitrijevic via raviprak)
2015-04-24 13:39:48 -07:00
Steve Loughran
a6a5d1d6b5
HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel)
2015-04-23 21:39:43 +01:00
Ravi Prakash
aa34aa5caa
HADOOP-11827. Speed-up distcp buildListing() using threadpool (Zoran Dimitrijevic via raviprak)
2015-04-21 16:50:40 -07:00
Jing Zhao
d4dd97eabd
HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao.
...
(cherry picked from commit 4c097e473b
)
2015-04-15 12:39:09 -07:00
Akira Ajisaka
f603b9ccca
MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina.
...
(cherry picked from commit 83979e61a4
)
Conflicts:
hadoop-mapreduce-project/CHANGES.txt
2015-04-10 20:23:46 +09:00
Colin Patrick Mccabe
386b90a700
HADOOP-11785. Reduce the number of listStatus operation in distcp buildListing (Zoran Dimitrijevic via Colin P. McCabe)
...
(cherry picked from commit 932730df7d
)
2015-04-03 14:08:53 -07:00
Haohui Mai
9e114ee607
HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao.
2015-04-01 16:51:09 -07:00
Harsh J
274db918c3
MAPREDUCE-6291. Correct mapred queue usage command. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit f5432cb47dfacaa57faed32a29d00bbeff8a5385)
2015-03-28 11:58:44 +05:30
Tsuyoshi Ozawa
24b60a7d1a
HADOOP-11760. Fix typo of javadoc in DistCp. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit e074952bd6
)
2015-03-27 23:16:37 +09:00
Yongjun Zhang
be0d228ed1
HADOOP-11724. DistCp throws NPE when the target directory is root. (Lei Eddy Xu via Yongjun Zhang)
...
(cherry-picked from commit 65ee02d9ed50e026c2d37ec80babd4176075ff84)
2015-03-25 15:50:28 -07:00
Harsh J
d85c14afbf
MAPREDUCE-579. Streaming slowmatch documentation.
...
(cherry picked from commit a2e42d2deee715f6255d6fd2c95f34e80888dc5f)
2015-03-25 14:39:46 +05:30
Harsh J
c58786794b
MAPREDUCE-4653. TestRandomAlgorithm has an unused import statement. Contributed by Amir Sanjar.
...
(cherry picked from commit 75e4670408a058efa95eaa768fedbe614008658f)
2015-03-17 14:13:23 +05:30
Tsuyoshi Ozawa
77297017d8
HADOOP-11720. [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools. Contributed by Akira AJISAKA.
...
(cherry picked from commit ef9946cd52
)
2015-03-17 16:09:38 +09:00
Tsuyoshi Ozawa
4489a02634
HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto.
...
(cherry picked from commit 79426f3334
)
2015-03-15 14:30:12 +09:00
Tsuyoshi Ozawa
1802df8373
Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki."
...
This reverts commit 54a73a6b5f
.
2015-03-15 14:27:56 +09:00
Tsuyoshi Ozawa
54a73a6b5f
HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki.
...
(cherry picked from commit 7da136ecca
)
2015-03-15 14:18:01 +09:00
cnauroth
dcc4850ff3
HADOOP-11693. Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving. Contributed by Duo Xu.
...
(cherry picked from commit 7a346bcb4f
)
2015-03-11 14:37:22 -07:00
Haohui Mai
e75e6c66ea
HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki.
2015-03-11 14:23:44 -07:00
Tsuyoshi Ozawa
b46f9e72db
HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa)
2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli
d9416317a9
Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2.
2015-03-08 20:32:51 -07:00
Tsuyoshi Ozawa
ccde4aed18
MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA.
2015-03-09 12:00:04 +09:00
Steve Loughran
b8a2087951
HADOOP-11670. Regression: s3a auth setup broken. (Adam Budde via stevel)
2015-03-08 11:22:01 -07:00
cnauroth
02cadbd24b
HADOOP-11642. Upgrade azure sdk version from 0.6.0 to 2.0.0. Contributed by Shashank Khandelwal and Ivan Mitic.
...
(cherry picked from commit 608ebd52ba
)
Conflicts:
hadoop-project/pom.xml
2015-03-06 15:29:22 -08:00
Jing Zhao
b2ccf54c14
HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao.
...
(cherry picked from commit ed70fa142c
)
2015-03-04 10:32:39 -08:00
Vinod Kumar Vavilapalli
6604613bc5
MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public API in DistCp. Contributed by Jing Zhao.
...
(cherry picked from commit 5af693fde2
)
2015-03-03 16:29:36 -08:00
Steve Loughran
824c32de1a
HADOOP-11183. Memory-based S3AOutputstream. (Thomas Demoor via stevel)
2015-03-03 16:18:39 -08:00
Akira Ajisaka
f7a724ca9e
HADOOP-10976. Backport "moving the source code of hadoop-tools docs to the directory under hadoop-tools" to branch-2. Contributed by Masatake Iwasaki.
...
(cherry picked from commit 9112f093cd
)
Conflicts:
hadoop-common-project/hadoop-common/CHANGES.txt
2015-02-28 17:06:58 -08:00
cnauroth
1bc4c6808d
HADOOP-11629. WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true. Contributed by Shanyu Zhao.
...
(cherry picked from commit 5731c0e0d0
)
2015-02-25 09:09:15 -08:00
Akira Ajisaka
e97613b429
HADOOP-11480. Typo in hadoop-aws/index.md uses wrong scheme for test.fs.s3.name. Contributed by Ted Yu.
...
(cherry picked from commit 1a625b8158
)
2015-02-24 17:13:35 -08:00
Steve Loughran
a36cad0522
HADOOP-11584 s3a file block size set to 0 in getFileStatus. (Brahma Reddy Battula via stevel)
2015-02-21 12:02:41 +00:00
Steve Loughran
4276f8b315
HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel)
2015-02-20 20:51:00 +00:00
Steve Loughran
411c8a7cda
HADOOP-11521. Make connection timeout configurable in s3a. (Thomas Demoor via stevel)
2015-02-17 20:00:00 +00:00
Steve Loughran
f52fcdc2e0
HADOOP-11522. Update S3A Documentation. (Thomas Demoor via stevel)
2015-02-17 18:14:31 +00:00
Steve Loughran
701b96ca8e
HADOOP-11570. S3AInputStream.close() downloads the remaining bytes of the object from S3. (Dan Hecht via stevel).
2015-02-17 16:36:32 +00:00
Steve Loughran
af7368cf03
HADOOP-11520. Clean incomplete multi-part uploads in S3A tests. (Thomas Demoor via stevel)
2015-02-06 20:59:56 +00:00
Steve Loughran
4cd9657b71
HADOOP-11463 Replace method-local TransferManager object with S3AFileSystem#transfers. (Ted Yu via stevel)
2015-02-05 12:19:49 +00:00
Akira Ajisaka
0a030871dd
MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. Contributed by Masatake Iwasaki.
...
(cherry picked from commit 34fe11c987
)
2015-02-04 09:27:01 -08:00
Tsuyoshi Ozawa
f92d4fa20f
HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa)
...
(cherry picked from commit ba58da29f4
)
2015-02-05 00:51:40 +09:00
Akira Ajisaka
8d98e5c960
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka)
...
(cherry picked from commit bd69fb2d44
)
Conflicts:
hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
2015-02-03 14:35:15 -08:00
Karthik Kambatla
c981b040ba
YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha)
...
(cherry picked from commit c559df2219
)
2015-02-03 13:20:57 -08:00
Haohui Mai
6c5ae1571c
HADOOP-11529. Fix findbugs warnings in hadoop-archives. Contributed by Masatake Iwasaki.
2015-02-03 10:53:26 -08:00
cnauroth
dedfb982f1
HADOOP-11442. hadoop-azure: Create test jar. Contributed by Shashank Khandelwal.
...
(cherry picked from commit 1c09ca2ba4
)
2015-02-02 11:51:01 -08:00
Harsh J
648510e031
HADOOP-11488. Difference in default connection timeout for S3A FS. Contributed by Daisuke Kobayashi.
...
(cherry picked from commit ffc75d6ebe
)
2015-02-01 00:17:24 +05:30
Allen Wittenauer
a754f94ed7
MAPREDUCE-6151. Update document of GridMix (Masatake Iwasaki via aw)
2015-01-30 11:09:03 -08:00
cnauroth
4607ca2854
HADOOP-11523. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.
...
(cherry picked from commit f2c91098c4
)
2015-01-29 17:02:48 -08:00
Allen Wittenauer
3165e778f3
MAPREDUCE-6150. Update document of Rumen (Masatake Iwasaki via aw)
2015-01-29 14:18:25 -08:00
cnauroth
fa2378d866
HADOOP-11441. Hadoop-azure: Change few methods scope to public. Contributed by Shashank Khandelwal.
...
(cherry picked from commit ad55083f75
)
2015-01-29 13:07:59 -08:00
Allen Wittenauer
874e38d81f
HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw)
2015-01-27 14:32:21 -08:00
Tsuyoshi Ozawa
2551cc6e6f
HADOOP-11450. Cleanup DistCpV1 not to use deprecated methods and fix javadocs. Contributed by Varun Saxena.
...
(cherry picked from commit 7b82c4ab4e
)
2015-01-26 13:02:00 +09:00
Allen Wittenauer
683cc18bba
HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw)
2015-01-21 08:46:19 -08:00
Steve Loughran
15865587ec
HADOOP-11171 Enable using a proxy server to connect to S3a. (Thomas Demoor via stevel)
2015-01-17 18:40:30 +00:00
Steve Loughran
ed2f8a1ba0
HADOOP-10542 Potential null pointer dereference in Jets3tFileSystemStore retrieveBlock(). (Ted Yu via stevel)
2015-01-17 18:26:03 +00:00
Steve Loughran
651c7debe4
HADOOP-11261 Set custom endpoint for S3A. (Thomas Demoor via stevel)
2015-01-16 10:15:22 +00:00
Arpit Agarwal
7a75f4eed9
HADOOP-11465. Fix findbugs warnings in hadoop-gridmix. (Contributed by Varun Saxena)
2015-01-12 21:23:13 -08:00
Steve Loughran
2800e73c27
HADOOP-11446 S3AOutputStream should use shared thread pool to avoid OutOfMemoryError
2015-01-05 12:59:48 +00:00
Karthik Kambatla
798ab51289
YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha)
...
(cherry picked from commit e7257acd8a
)
2014-12-30 19:55:50 -08:00
Tsuyoshi Ozawa
7739f819a6
HADOOP-11283. SequenceFile.Writer can leak file descriptors in DistCpV1#setup(). (Contributed by Varun Saxena via ozawa)
...
(cherry picked from commit a164ce2ac9
)
2014-12-25 23:32:34 +09:00
Jian He
411836b74c
YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan
...
(cherry picked from commit fdf042dfff
)
2014-12-22 16:52:15 -08:00
Haohui Mai
e6662ec6a4
HADOOP-11429. Findbugs warnings in hadoop extras. Contributed by Varun Saxena.
2014-12-21 14:57:41 -08:00
arp
0a5b28605f
HADOOP-11395. Add site documentation for Azure Storage FileSystem integration. (Contributed by Chris Nauroth)
2014-12-19 18:54:35 -08:00
Haohui Mai
99c176ea6a
HADOOP-11431. Clean up redundant maven-site-plugin configuration. Contributed by Herve Boutemy.
2014-12-18 16:24:17 -08:00
Haohui Mai
c28449d4b5
HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack. Contributed by Li Lu.
...
(cherry picked from commit 2e98ad34ce
)
Conflicts:
hadoop-common-project/hadoop-common/CHANGES.txt
2014-12-17 14:57:14 -08:00
Steve Loughran
e71484be18
HADOOP-11412 POMs mention "The Apache Software License" rather than "Apache License". (Herve Boutemy via stevel)
...
(cherry picked from commit 07bb0b0bbb
)
Conflicts:
hadoop-common-project/hadoop-common/CHANGES.txt
2014-12-17 14:57:13 -08:00
cnauroth
5c32c4b2e4
HADOOP-10809. SyncableDataOutputStream must implement sync method on branch-2.
2014-12-17 14:57:13 -08:00
Aaron T. Myers
66808e6aac
HADOOP-10400. Incorporate new S3A FileSystem implementation. Contributed by Jordan Mendelson and Dave Wang.
...
(cherry picked from commit 24d920b80e
)
Conflicts:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/BasicAWSCredentialsProvider.java
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AOutputStream.java
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/TestS3AContractRename.java
hadoop-tools/hadoop-aws/src/test/resources/contract/s3a.xml
2014-12-17 14:57:13 -08:00
cnauroth
535d98bbb3
HADOOP-11248. Add hadoop configuration to disable Azure Filesystem metrics collection. Contributed by Shanyu Zhao.
...
(cherry picked from commit 734eeb4f35
)
Conflicts:
hadoop-common-project/hadoop-common/CHANGES.txt
2014-12-17 14:57:13 -08:00
cnauroth
1646cc9f29
HADOOP-11188. hadoop-azure: automatically expand page blobs when they become full. Contributed by Eric Hanson.
...
(cherry picked from commit f4b7e99f4e
)
2014-12-17 14:57:13 -08:00
cnauroth
5a737026cc
HADOOP-10809. hadoop-azure: page blob support. Contributed by Dexter Bradshaw, Mostafa Elhemali, Eric Hanson, and Mike Liddell.
...
(cherry picked from commit 2217e2f8ff
)
2014-12-17 14:57:13 -08:00
Allen Wittenauer
9a2e4f4f98
HADOOP-10946. Fix a bunch of typos in log messages (Ray Chiang via aw)
...
(cherry picked from commit 9f03a7c018
)
Conflicts:
hadoop-common-project/hadoop-common/CHANGES.txt
2014-12-17 14:57:13 -08:00
Chris Nauroth
cb6eb5931f
HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604236 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 61bf9f7799
)
2014-12-17 14:57:12 -08:00