Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,937 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

386 Commits

Author SHA1 Message Date
cnauroth a5511debdb HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay. 
(cherry picked from commit 27b77751c1)
 2016-02-22 17:30:32 -08:00
cnauroth 3f14eed580 HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki. 
(cherry picked from commit d4f5fc23b2)
 2016-02-22 13:39:45 -08:00
Yongjun Zhang b144e0f15d HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit ba1c9d484a)
 2016-02-19 10:29:27 -08:00
cnauroth b4c4f30fa4 HADOOP-11031. Design Document for Credential Provider API. Contributed by Larry McCay. 
(cherry picked from commit 8ab7658025)
 2016-02-18 14:07:09 -08:00
cnauroth 6731bb468d HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay. 
(cherry picked from commit 76fab26c5c)

Conflicts:
	hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
 2016-02-17 12:20:41 -08:00
cnauroth ac5a793590 HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty. 
(cherry picked from commit 91a96eaa53)
 2016-02-12 15:50:19 -08:00
Steve Loughran 60474a769e HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel) 2016-02-06 18:20:24 +00:00
Yongjun Zhang 2fb711bde9 HDFS-9638. Improve DistCp Help and documentation. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit eddd823cd6)
 2016-01-29 12:13:03 -08:00
cnauroth bd01fdb64d MAPREDUCE-6563. Streaming documentation contains a stray % character. Contributed by Chris Nauroth. 
(cherry picked from commit 41da9a0773)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-01-28 14:46:42 -08:00
Xuan 69f3fa71a2 YARN-4612. Fix rumen and scheduler load simulator handle killed tasks 
properly. Contributed by Ming Ma.

(cherry picked from commit 4efdf3a979)
 2016-01-26 18:18:03 -08:00
Akira Ajisaka 4edd7bd820 HADOOP-12731. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 736eb17a79)
 2016-01-25 13:48:28 +09:00
Akira Ajisaka 6368d102b9 HADOOP-12730. Hadoop streaming -mapper and -reducer options are wrongly documented as required. Contributed by Kengo Seki. 
(cherry picked from commit 0bae506c22)
 2016-01-22 21:09:09 +09:00
Akira Ajisaka 7f215ffdd5 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 2016-01-22 18:37:17 +09:00
Xuan 66b030f608 YARN-4611. Fix scheduler load simulator to support multi-layer network 
location. Contributed by Ming Ma.

(cherry picked from commit 1708a4cd23)
 2016-01-20 15:43:02 -08:00
Ravi Prakash 2cbb8bbd72 HADOOP-12696. Add tests for S3FileSystem Contract. Contributed by Matt Paduano 
(cherry picked from commit 1acc509b45)
 2016-01-19 14:00:54 -08:00
Wangda Tan 23deeb4230 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)
 2016-01-19 21:28:51 +08:00
Wangda Tan 704b37d0e1 YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit a44ce3f14f)
 2016-01-19 09:31:13 +08:00
Wangda Tan 1625281ccb Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe5728563.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 150f5ae034)
 2016-01-19 09:31:07 +08:00
cnauroth 62d6166211 HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth. 
(cherry picked from commit 8bc93db2e7)
 2016-01-18 09:09:05 -08:00
Wangda Tan 6dab78c71a YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85e)
 2016-01-18 11:15:59 +08:00
Yongjun Zhang ee012c1f6f HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit a9c69ebeb7)
 2016-01-15 10:10:28 -08:00
Colin Patrick Mccabe cea1c71087 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 
(cherry picked from commit b1ed28fa77)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
 2016-01-14 11:13:13 -08:00
Colin Patrick Mccabe 2baeaa20a3 HDFS-9517. Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit 8315582c4f)
 2016-01-13 16:33:02 -08:00
Lei Xu 63b8be149a HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei) 2016-01-12 12:21:29 -08:00
Xiaoyu Yao 521f60c094 HDFS-8584. NPE in distcp when ssl configuration file does not exist in class path. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit c2e2e13455)
 2016-01-11 17:12:43 -08:00
Zhe Zhang 17dfa04be6 HDFS-9630. DistCp minor refactoring and clean up. Contributed by Kai Zheng. 
Change-Id: I363c4ffcac32116ddcdc0a22fac3db92f14a0db0

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-01-11 09:53:20 -08:00
cnauroth 813841e10b HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth. 
(cherry picked from commit 0e76f1fcea)
 2016-01-09 22:18:22 -08:00
cnauroth debd13387d HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty. 
(cherry picked from commit f0fa6d869b)
 2016-01-08 20:19:11 -08:00
cnauroth 4753676a34 HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade. 
(cherry picked from commit 978bbdfeb2)
 2016-01-06 11:16:18 -08:00
Ravi Prakash 6330683778 HADOOP-12689. S3 filesystem operations stopped working correctly 
(cherry picked from commit 2d16f40dab)
 2016-01-05 23:43:17 -08:00
Wangda Tan 6eefae1b33 YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda) 2015-12-30 15:34:39 -08:00
Wangda Tan 9a7d5f69ec YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda) 
(cherry picked from commit 8310b2e9ff)
 2015-12-30 15:31:11 -08:00
Xiaoyu Yao a65e159c59 HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth. 
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
 2015-12-30 11:12:02 -08:00
Wangda Tan 9f4a6ec663 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83)
 2015-12-16 13:28:56 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Robert Kanter 7b4bf23b2a MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor (rkanter) 
(cherry picked from commit 6d84cc16b3)
 2015-11-25 17:13:00 -08:00
Akira Ajisaka 4257991257 HADOOP-12598. Add XML namespace declarations for some hadoop/tools modules. Contributed by Xin Wang. 
(cherry picked from commit b53f20135599cc7825b6213794e2f999bc43e2ad)
 2015-11-26 01:21:32 +09:00
Karthik Kambatla 49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20b)
 2015-11-24 13:52:29 +05:30
Haohui Mai 1d175f0f3f HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux. 2015-11-22 18:12:53 -08:00
Haohui Mai ac0694eb6e HADOOP-12098. Remove redundant test dependencies in Hadoop Archives. Contributed by Varun Saxena. 2015-11-22 17:25:29 -08:00
Haohui Mai f504d95e87 HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su. 2015-11-22 15:57:23 -08:00
Karthik Kambatla d3f9508096 YARN-4367. SLS webapp doesn't load. (kasha) 2015-11-18 13:45:00 -08:00
Akira Ajisaka c5c663c05d HADOOP-12545. Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp. Contributed by Arshad Mohammad. 
(cherry picked from commit f94d892704)
 2015-11-13 21:53:23 +09:00
Lei Xu 47941858af Revert "HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei)" 
This reverts commit 01ae30796d.
 2015-11-06 09:52:21 -08:00
Lei Xu 01ae30796d HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei) 
(cherry picked from commit bff7c90a56)
 2015-11-05 18:36:55 -08:00
cnauroth 81f7e8af22 HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade. 
(cherry picked from commit 0fb1867fd6)
 2015-11-04 10:29:01 -08:00
cnauroth c1d7b26e94 HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade. 
(cherry picked from commit 9e7dcab185)
 2015-11-02 10:21:46 -08:00
cnauroth c260d7ec78 HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth. 
(cherry picked from commit 28790692624177d89fb1e4f59e2f83a659fc3089)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-11-02 09:39:27 -08:00
Kihwal Lee dae9f0ef53 Addendum to MAPREDUCE-6451 
(cherry picked from commit b24fe06483)
 2015-10-30 16:06:50 -05:00
Kihwal Lee 52819fe4ef MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla. 
(cherry picked from commit 2868ca0328)
 2015-10-30 14:57:57 -05:00