Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,272 Commits 326 Branches 371 Tags
Commit Graph

9272 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
f0520a2c94 YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit 1ba31fe9e906dbd093afd4b254216601967a4a7b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2016-05-25 21:49:21 +00:00
Tsz-Wo Nicholas Sze
d3bdea7f7f HDFS-9365. Balaner does not work with the HDFS-6376 HA setup. 2016-05-24 13:52:05 -07:00
Naganarasimha
94a9a5d014 YARN-4751. In 2.7, Labeled queue usage not shown properly in capacity scheduler UI. Contributed by Eric Payne 2016-05-25 04:48:14 +08:00
Jason Lowe
27aeab63c7 YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
 2016-05-23 15:57:54 +00:00
Chris Nauroth
0abb366364 HADOOP-13154. S3AFileSystem printAmazonServiceException/printAmazonClientException appear copy & paste of AWS examples. Contributed by Steve Loughran. 2016-05-19 21:40:17 -07:00
Eric Payne
0ee36fd511 YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger. 2016-05-16 21:12:26 +00:00
Naganarasimha
8f17395029 YARN-3362. Add node label usage in RM CapacityScheduler web UI. Contributed by Eric Payne. 2016-05-17 02:39:50 +08:00
Jason Lowe
7ded648aee MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f373a99cb66ad7d6bacef8dcf336f77)
 2016-05-13 14:37:55 +00:00
Jian He
1918da8fc9 MAPREDUCE-6513. Job got hanged forever when one NM unstable for some 
time. Contributed by Varun Saxena & Wangda Tan
 2016-05-12 15:14:08 -07:00
Kihwal Lee
6e62b6fd83 HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. 
(cherry picked from commit 0c4c7b3b62fd9e58dc8adef42b81b8b96e1f4ab1)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
 2016-05-12 11:26:27 -05:00
Colin Patrick Mccabe
a61e04171f HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe) 
(cherry picked from commit f0252ad2525c90b2b89ae2ff1de12482da2c8ddd)
(cherry picked from commit 43f32d3f4a86d8f2a302530831003496c78fd561)
(cherry picked from commit 154332bb5463998d5e8f2251c8010d624f5e85a9)
 2016-05-10 11:22:13 -07:00
Kihwal Lee
0467efcc09 HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah. 
(cherry picked from commit b9e5a32fa14b727b44118ec7f43fb95de05a7c2c)
Added a CHANGES.txt entry.
 2016-05-09 15:56:51 -05:00
Akira Ajisaka
b8e01da1b3 HADOOP-13084. Fix ASF License warnings in branch-2.7. Contributed by Brahma Reddy Battula. 2016-05-08 15:41:59 -07:00
Jason Lowe
09fc02be67 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae73197990a950ce71ece80c0f23221c384)
 2016-05-06 23:03:23 +00:00
Jason Lowe
5f698df901 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81fc925e33ccc0b02c98cc2d929df120)
 2016-05-06 22:29:52 +00:00
Robert Kanter
b0bcb4e728 Remove parent's env vars from child processes 2016-05-06 09:33:36 -07:00
Masatake Iwasaki
dd8aba3fad HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 2016-05-06 15:57:06 +09:00
Vinod Kumar Vavilapalli
9f51a3f430 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
Updated the 2.7 branch patch to fix a minor issue in TestRMContainerAllocator.java

(cherry picked from commit 8d48266720dcf0e71cfd87fef18b60a53aa1bef9)
 2016-05-05 19:39:25 -07:00
Tsz-Wo Nicholas Sze
2340e29084 HADOOP-13103 Group resolution from LDAP may fail on javax.naming.ServiceUnavailableException 2016-05-05 16:07:08 -07:00
Jason Lowe
4311e5f353 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b48391680c1b81a86aabc3ad4c725bfade6d2e)
 2016-05-03 17:38:19 +00:00
Harsh J
d490cfc42c HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9f877e58f7f153c43cb4147876826d3)
 2016-05-02 17:52:14 -07:00
Kihwal Lee
6347f8f4d4 HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 
(cherry picked from commit 7da540d03eccc2f97950bf47e8b35ce8c889d1e0)
Updated CHANGES.txt
 2016-04-29 16:07:20 -05:00
Tsz-Wo Nicholas Sze
c927f938fe HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:28:23 -07:00
Jason Lowe
e12c9e3348 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501bc753858693dacdafba61d604702f71)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-04-29 15:07:07 +00:00
Jason Lowe
dbb7458735 Revert "YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe" 
This reverts commit 77ef473981c1af0166ed6307b5a2789ca7caa7ba.
 2016-04-29 14:38:30 +00:00
Jian He
77ef473981 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501bc753858693dacdafba61d604702f71)
 2016-04-28 22:00:42 -07:00
Jian He
d7b2da6518 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 
(cherry picked from commit a9707dceaf181ea0f6d3c4291d1f3ccc2681f273)
 2016-04-28 21:45:17 -07:00
Kihwal Lee
a3ece8b5b4 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 2016-04-28 16:47:04 -05:00
Arpit Agarwal
92548e09c6 HADOOP-13039. Add documentation for configuration property ipc.maximum.data.length (Contributed by Mingliang Liu) 2016-04-26 20:47:53 -07:00
Eric Payne
2ec33c4e3c YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda / Eric Badger 2016-04-26 16:14:07 +00:00
Akira Ajisaka
076215024b HDFS-10245. Fix the findbugs warnings in branch-2.7. Contributed by Brahma Reddy Battula. 2016-04-25 22:03:48 +09:00
Kihwal Lee
0bb23e22ce HADOOP-13052. ChecksumFileSystem mishandles crc file permissions. Contributed by Daryn Sharp. 
(cherry picked from commit 9dbdc8e12d009e76635b2d20ce940851725cb069)
 2016-04-22 15:22:42 -05:00
Kihwal Lee
9e3b91b141 HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a80156d2cacfea8184b41805070223d3a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-04-22 11:47:18 -05:00
Tsz-Wo Nicholas Sze
e7f1b8fcb3 HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:36:28 -07:00
Andrew Wang
b8febb774a HADOOP-13042. Restore lost leveldbjni LICENSE and NOTICE changes. 
(cherry picked from commit fea50c5440d83225958c5e346299334559fc37a4)
(cherry picked from commit c28e4b0651cfd5ed142980dc10f4c2a7388a39a0)
(cherry picked from commit bae229ae27605d039d692fdd1edc5b4d946ddbe8)

 Conflicts:
	LICENSE.txt
 2016-04-21 13:51:52 -07:00
Jian He
f1e4f4d83a MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe5800975ea0c4295c9911ae3f572ed37d)
 2016-04-20 19:03:53 -07:00
Arpit Agarwal
12c6c2c9a6 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:28:01 -07:00
Karthik Kambatla
cc6ae6fa7e MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab8d17e839a1ef0d7076efc78d28724a)

(cherry picked from commit b826168173c3386738acc12a5d62577f12aa06e9)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-18 21:58:53 -07:00
Eric Payne
2ec5bb0fe8 MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037d2b22dafc64f33d4f1231bef4ceba8)
 2016-04-18 16:12:53 +00:00
Walter Su
91ca90fdd1 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9d353677184ff5602966b11ffb408b9)
 2016-04-18 20:30:48 +08:00
Jason Lowe
0ad4634585 YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5c3ab0c79cacffc22b1f59408622ae7)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
 2016-04-15 20:52:11 +00:00
Akira Ajisaka
80d279b481 HADOOP-12989. Some tests in org.apache.hadoop.fs.shell.find occasionally time out. Contributed by Takashi Ohnishi. 
(cherry picked from commit 6e6b6dd5aaf93cfb373833cd175ee722d2cb708f)
(cherry picked from commit 5ae74507c0d64eef4ea65973d798981bc56de61b)
 2016-04-15 14:26:52 +09:00
Steve Loughran
d3a806666d HADOOP-12613. TestFind.processArguments occasionally fails. (Wei-Chiu Chuang via stevel) 
(cherry picked from commit c329cab92c49e7dbe4c48557908124c9792ae699)
 2016-04-15 14:22:53 +09:00
Robert Kanter
c9ba6fa9ba MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77d9438f0954ae4e1497d63997bb7347)
(cherry picked from commit f9f57265cf0ad2685d0d96ec59b8b78ac8bed175)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:58:45 -07:00
Jason Lowe
a996889313 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 9b5c5bd42f0cb240d0fe7754967765a99dd5be46)
 2016-04-14 19:44:31 +00:00
Junping Du
b4d24d7b0e Add entry of MAPREDUCE-6670 to CHANGES.txt after backport. 2016-04-13 09:02:33 -07:00
Junping Du
683f313c1f MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a42cd54bd88ce2de63122998e967efa)
(cherry picked from commit 5574f827991cfc3d14f71184296866d387ed174f)
 2016-04-13 08:59:22 -07:00
Walter Su
2ee244ac28 HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf946827b4d58c7c577ed0c574a7cff029)
 2016-04-13 17:51:01 +08:00
Rohith Sharma K S
3afdac6e2c YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 2016-04-13 11:13:42 +05:30
Vinayakumar B
b52ac3cb4d Addendum: HDFS-10271. 
Added CHANGES.txt entry
 2016-04-12 11:57:00 +05:30