Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,306 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

9306 Commits

Author SHA1 Message Date
Masatake Iwasaki dd8aba3fad HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 2016-05-06 15:57:06 +09:00
Vinod Kumar Vavilapalli 9f51a3f430 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
Updated the 2.7 branch patch to fix a minor issue in TestRMContainerAllocator.java

(cherry picked from commit 8d48266720)
 2016-05-05 19:39:25 -07:00
Tsz-Wo Nicholas Sze 2340e29084 HADOOP-13103 Group resolution from LDAP may fail on javax.naming.ServiceUnavailableException 2016-05-05 16:07:08 -07:00
Jason Lowe 4311e5f353 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b4839168)
 2016-05-03 17:38:19 +00:00
Harsh J d490cfc42c HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9)
 2016-05-02 17:52:14 -07:00
Kihwal Lee 6347f8f4d4 HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 
(cherry picked from commit 7da540d03e)
Updated CHANGES.txt
 2016-04-29 16:07:20 -05:00
Tsz-Wo Nicholas Sze c927f938fe HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:28:23 -07:00
Jason Lowe e12c9e3348 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-04-29 15:07:07 +00:00
Jason Lowe dbb7458735 Revert "YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe" 
This reverts commit 77ef473981.
 2016-04-29 14:38:30 +00:00
Jian He 77ef473981 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501b)
 2016-04-28 22:00:42 -07:00
Jian He d7b2da6518 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 
(cherry picked from commit a9707dceaf)
 2016-04-28 21:45:17 -07:00
Kihwal Lee a3ece8b5b4 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 2016-04-28 16:47:04 -05:00
Arpit Agarwal 92548e09c6 HADOOP-13039. Add documentation for configuration property ipc.maximum.data.length (Contributed by Mingliang Liu) 2016-04-26 20:47:53 -07:00
Eric Payne 2ec33c4e3c YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda / Eric Badger 2016-04-26 16:14:07 +00:00
Akira Ajisaka 076215024b HDFS-10245. Fix the findbugs warnings in branch-2.7. Contributed by Brahma Reddy Battula. 2016-04-25 22:03:48 +09:00
Kihwal Lee 0bb23e22ce HADOOP-13052. ChecksumFileSystem mishandles crc file permissions. Contributed by Daryn Sharp. 
(cherry picked from commit 9dbdc8e12d)
 2016-04-22 15:22:42 -05:00
Kihwal Lee 9e3b91b141 HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a801)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-04-22 11:47:18 -05:00
Tsz-Wo Nicholas Sze e7f1b8fcb3 HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:36:28 -07:00
Andrew Wang b8febb774a HADOOP-13042. Restore lost leveldbjni LICENSE and NOTICE changes. 
(cherry picked from commit fea50c5440)
(cherry picked from commit c28e4b0651)
(cherry picked from commit bae229ae27)

 Conflicts:
	LICENSE.txt
 2016-04-21 13:51:52 -07:00
Jian He f1e4f4d83a MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe58)
 2016-04-20 19:03:53 -07:00
Arpit Agarwal 12c6c2c9a6 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:28:01 -07:00
Karthik Kambatla cc6ae6fa7e MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab)

(cherry picked from commit b826168173)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-18 21:58:53 -07:00
Eric Payne 2ec5bb0fe8 MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037)
 2016-04-18 16:12:53 +00:00
Walter Su 91ca90fdd1 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9)
 2016-04-18 20:30:48 +08:00
Jason Lowe 0ad4634585 YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
 2016-04-15 20:52:11 +00:00
Akira Ajisaka 80d279b481 HADOOP-12989. Some tests in org.apache.hadoop.fs.shell.find occasionally time out. Contributed by Takashi Ohnishi. 
(cherry picked from commit 6e6b6dd5aa)
(cherry picked from commit 5ae74507c0)
 2016-04-15 14:26:52 +09:00
Steve Loughran d3a806666d HADOOP-12613. TestFind.processArguments occasionally fails. (Wei-Chiu Chuang via stevel) 
(cherry picked from commit c329cab92c)
 2016-04-15 14:22:53 +09:00
Robert Kanter c9ba6fa9ba MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77)
(cherry picked from commit f9f57265cf)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:58:45 -07:00
Jason Lowe a996889313 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 9b5c5bd42f)
 2016-04-14 19:44:31 +00:00
Junping Du b4d24d7b0e Add entry of MAPREDUCE-6670 to CHANGES.txt after backport. 2016-04-13 09:02:33 -07:00
Junping Du 683f313c1f MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a)
(cherry picked from commit 5574f82799)
 2016-04-13 08:59:22 -07:00
Walter Su 2ee244ac28 HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf94)
 2016-04-13 17:51:01 +08:00
Rohith Sharma K S 3afdac6e2c YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 2016-04-13 11:13:42 +05:30
Vinayakumar B b52ac3cb4d Addendum: HDFS-10271. 
Added CHANGES.txt entry
 2016-04-12 11:57:00 +05:30
Vinayakumar B 37e4e45390 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 2016-04-12 11:32:28 +05:30
Vinod Kumar Vavilapalli 66072070d0 HADOOP-12406. Fixed AbstractMapWritable.readFields to use the thread's ClassLoader to load class instead of System ClassLoader. Contributed by Nadeem Douba. 
(cherry picked from commit 069c6c62de)
 2016-04-11 12:04:12 -07:00
Akira Ajisaka c748ced474 YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger. 
(cherry picked from commit 1b78b2ba17)
(cherry picked from commit 88556294e2)
 2016-04-11 01:36:04 +09:00
Wangda Tan 801c95c1cc MAPREDUCE-6302. Preempt reducers after a configurable timeout irrespective of headroom. (kasha via wangda) 2016-04-08 14:35:28 -07:00
Tsz-Wo Nicholas Sze 262d769b4d HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:43:09 +08:00
Kihwal Lee 8e784afb15 Revert "HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo." 
This reverts commit 9bc9e13a97.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-04-06 17:35:54 -05:00
Vinayakumar B 62da8f6fa6 HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 48ca23def1)
(cherry picked from commit d21bc811d8)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
 2016-04-06 10:49:34 +08:00
Kihwal Lee dd701c9800 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 
(cherry picked from commit 9ba1e5af06)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java

(cherry picked from commit 6e37c5fe80)
 2016-04-05 17:00:42 -05:00
Masatake Iwasaki 76e0bb7a19 YARN-4917. Fix typos in documentation of Capacity Scheduler. (Takashi Ohnishi via iwasakims) 2016-04-06 04:03:01 +09:00
Masatake Iwasaki 6576fc6ccc YARN-4915. Fix typo in YARN Secure Containers documentation (Takashi Ohnishi via iwasakims) 2016-04-06 03:50:01 +09:00
Kihwal Lee 9608601330 HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit ef3da82357)
 2016-04-05 09:20:38 -05:00
Jason Lowe 49428ab6bb YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 2016-04-05 13:51:46 +00:00
Vinayakumar B fc0ffc48d4 HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 2016-04-05 09:59:37 +08:00
Kihwal Lee 69d4fa0deb HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 
(cherry picked from commit a7d1fb0cd2)
TestClientProtocolForPipelineRecovery modified to use DFSConfigKeys instead of HdfsClientConfigKeys.
 2016-04-04 16:54:04 -05:00
Li Lu f242b78fd9 YARN-4706. UI Hosting Configuration in TimelineServer doc is broken. (Akira AJISAKA via gtcarrera9) 
(cherry picked from commit f61de41736)
 2016-04-04 14:46:28 -07:00
Allen Wittenauer 353f379168 YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw) 
(cherry picked from commit b1394d6307)
(cherry picked from commit 92a3dbe44f)
 2016-04-01 16:58:33 -07:00