Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,883 Commits 326 Branches 371 Tags
Commit Graph

569 Commits

Author SHA1 Message Date
Wangda Tan
1625281ccb Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe57285635e8058c34aa40a103845b49ca7d6ff.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 150f5ae0343e872ee8bef39c57008c1389f0ba9e)
 2016-01-19 09:31:07 +08:00
cnauroth
62d6166211 HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth. 
(cherry picked from commit 8bc93db2e7c64830b6a662f28c8917a9eef4e7c9)
 2016-01-18 09:09:05 -08:00
Wangda Tan
6dab78c71a YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85eb34c8125cfb7d26d07cdfac12b3579)
 2016-01-18 11:15:59 +08:00
Yongjun Zhang
ee012c1f6f HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit a9c69ebeb707801071db3cc22bfcd14f87be443a)
 2016-01-15 10:10:28 -08:00
Colin Patrick Mccabe
cea1c71087 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 
(cherry picked from commit b1ed28fa77cb2fab80c54f9dfeb5d8b7139eca34)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
 2016-01-14 11:13:13 -08:00
Colin Patrick Mccabe
2baeaa20a3 HDFS-9517. Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit 8315582c4ff2951144b096c23a64e753f397572d)
 2016-01-13 16:33:02 -08:00
Lei Xu
63b8be149a HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei) 2016-01-12 12:21:29 -08:00
Xiaoyu Yao
521f60c094 HDFS-8584. NPE in distcp when ssl configuration file does not exist in class path. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit c2e2e134555010ec28da296bcfef4ba2613a5c6c)
 2016-01-11 17:12:43 -08:00
Zhe Zhang
17dfa04be6 HDFS-9630. DistCp minor refactoring and clean up. Contributed by Kai Zheng. 
Change-Id: I363c4ffcac32116ddcdc0a22fac3db92f14a0db0

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-01-11 09:53:20 -08:00
cnauroth
813841e10b HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth. 
(cherry picked from commit 0e76f1fceaaaeb66bdb4818f43b9a55fc092bf79)
 2016-01-09 22:18:22 -08:00
cnauroth
debd13387d HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty. 
(cherry picked from commit f0fa6d869b9abb5a900ea1c9eb4eb19ec9831dc4)
 2016-01-08 20:19:11 -08:00
cnauroth
4753676a34 HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade. 
(cherry picked from commit 978bbdfeb2d12efd6e750da6a14849e072fb814b)
 2016-01-06 11:16:18 -08:00
Ravi Prakash
6330683778 HADOOP-12689. S3 filesystem operations stopped working correctly 
(cherry picked from commit 2d16f40dab291a29b3fc005221b12fd587615d4e)
 2016-01-05 23:43:17 -08:00
Wangda Tan
6eefae1b33 YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda) 2015-12-30 15:34:39 -08:00
Wangda Tan
9a7d5f69ec YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda) 
(cherry picked from commit 8310b2e9ff3d6804bad703c4c15458b0dfeeb4af)
 2015-12-30 15:31:11 -08:00
Xiaoyu Yao
a65e159c59 HADOOP-12608. Fix exception message in WASB when connecting with anonymous credential. Contributed by Dushyanth. 
(cherry picked from commit 007a8decc61750720033e06b6d08861ac3788e41)
 2015-12-30 11:12:02 -08:00
Wangda Tan
9f4a6ec663 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83e981ae74cb8b58ffcf7907b7612ad4)
 2015-12-16 13:28:56 -08:00
Vinod Kumar Vavilapalli
b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Robert Kanter
7b4bf23b2a MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor (rkanter) 
(cherry picked from commit 6d84cc16b3e0685fef01d0e3526b0f7556ceff51)
 2015-11-25 17:13:00 -08:00
Akira Ajisaka
4257991257 HADOOP-12598. Add XML namespace declarations for some hadoop/tools modules. Contributed by Xin Wang. 
(cherry picked from commit b53f20135599cc7825b6213794e2f999bc43e2ad)
 2015-11-26 01:21:32 +09:00
Karthik Kambatla
49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20bd1446164df1d3920c46c96dad750ae)
 2015-11-24 13:52:29 +05:30
Haohui Mai
1d175f0f3f HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux. 2015-11-22 18:12:53 -08:00
Haohui Mai
ac0694eb6e HADOOP-12098. Remove redundant test dependencies in Hadoop Archives. Contributed by Varun Saxena. 2015-11-22 17:25:29 -08:00
Haohui Mai
f504d95e87 HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su. 2015-11-22 15:57:23 -08:00
Karthik Kambatla
d3f9508096 YARN-4367. SLS webapp doesn't load. (kasha) 2015-11-18 13:45:00 -08:00
Akira Ajisaka
c5c663c05d HADOOP-12545. Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp. Contributed by Arshad Mohammad. 
(cherry picked from commit f94d89270464ea8e0d19e26e425835cd6a5dc5de)
 2015-11-13 21:53:23 +09:00
Lei Xu
47941858af Revert "HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei)" 
This reverts commit 01ae30796d0a120e4317e819533fb52bdae76885.
 2015-11-06 09:52:21 -08:00
Lei Xu
01ae30796d HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei) 
(cherry picked from commit bff7c90a5686de106ca7a866982412c5dfa01632)
 2015-11-05 18:36:55 -08:00
cnauroth
81f7e8af22 HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade. 
(cherry picked from commit 0fb1867fd62b5df664ad66386d6067db8fbf2317)
 2015-11-04 10:29:01 -08:00
cnauroth
c1d7b26e94 HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade. 
(cherry picked from commit 9e7dcab185abf2fdabb28f2799b9952b5664a4b0)
 2015-11-02 10:21:46 -08:00
cnauroth
c260d7ec78 HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth. 
(cherry picked from commit 28790692624177d89fb1e4f59e2f83a659fc3089)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-11-02 09:39:27 -08:00
Kihwal Lee
dae9f0ef53 Addendum to MAPREDUCE-6451 
(cherry picked from commit b24fe0648348d325d14931f80cee8a170fb3358a)
 2015-10-30 16:06:50 -05:00
Kihwal Lee
52819fe4ef MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla. 
(cherry picked from commit 2868ca0328d908056745223fb38d9a90fd2811ba)
 2015-10-30 14:57:57 -05:00
cnauroth
c7f87dc2d0 HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth. 
(cherry picked from commit 2d10cb8e00fd84be26b51f9fba819e3e5b6cdcbf)
 2015-10-28 09:44:21 -07:00
cnauroth
0377795e06 HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu. 
(cherry picked from commit 1f7ecb0c84042783f9fcf3f77d7d889dc58c9ead)
 2015-10-27 22:56:33 -07:00
cnauroth
a23f79b92c HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth. 
(cherry picked from commit 73822de7c38e189f7654444ff48d15cbe0df7404)
 2015-10-27 22:49:04 -07:00
Varun Vasudev
c5bf1cb7af YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt. 
(cherry picked from commit c28e16b40caf1e22f72cf2214ebc2fe2eaca4d03)
 2015-10-27 20:56:45 +05:30
cnauroth
b023539c19 HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade. 
(cherry picked from commit cb282d5b89fdece4719cc4ad37a6e27f13371534)
 2015-10-22 14:30:09 -07:00
cnauroth
10398daadf HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade. 
(cherry picked from commit 47641fcbc9c41f4a338d8899501e4a310d2e81ad)
 2015-10-22 12:21:44 -07:00
Robert Kanter
492142097b MAPREDUCE-6495. Docs for archive-logs tool (rkanter) 
(cherry picked from commit 0c4af0f99811a7138954391df3761aef9ff09155)
 2015-10-20 17:34:56 -07:00
Haohui Mai
ee04add696 HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:29 -07:00
Haohui Mai
f6b0fcdc75 Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu." 
This reverts commit a412edf5c4838e007217652e1cb6a24e73b7e735.
 2015-10-10 03:25:16 -07:00
Haohui Mai
a412edf5c4 HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu. 2015-10-09 22:38:13 -07:00
Robert Kanter
e5bf4594cc MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME (rkanter) 
(cherry picked from commit 1bca1bb5175b5976a5a0e45b6e1c4486bdf87a0e)
 2015-10-06 16:50:18 -07:00
cnauroth
535762fadb HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth. 
(cherry picked from commit 5f6edb30c2bb648d5564c951edc25645e17e6636)
 2015-10-05 20:12:33 -07:00
Akira Ajisaka
c6f30e5913 HADOOP-10296. Incorrect null check in SwiftRestClient#buildException(). Contributed by Rahul Palamuttam and Kanaka Kumar Avvaru. 
(cherry picked from commit ecbfd68974691cc9103ae477d10c15c7f21b6dea)
 2015-10-01 18:35:19 +02:00
Robert Kanter
46ef5aa8eb MAPREDUCE-6494. Permission issue when running archive-logs tool as different users (rkanter) 
(cherry picked from commit 5db371f52f5c6e894a7e6a5d523084f4b316a7ab)
 2015-09-30 17:34:03 -07:00
Kihwal Lee
95b8f1f3a6 HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 
(cherry picked from commit 3abbdc929bde05f8819f5410cef1eaeb8940203f)
 2015-09-28 17:33:20 -05:00
Harsh J
03e370e306 MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian. 
(cherry picked from commit 66dad854c0aea8c137017fcf198b165cc1bd8bdd)
 2015-09-28 13:14:20 +05:30
cnauroth
85c0cb0075 HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 7fe521b1dd49f81ae325f78cf531cfff15be6641)
 2015-09-25 22:34:10 -07:00