Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,252 Commits 326 Branches 371 Tags
Commit Graph

9252 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Harsh J
d490cfc42c HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9f877e58f7f153c43cb4147876826d3)
 2016-05-02 17:52:14 -07:00
Kihwal Lee
6347f8f4d4 HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 
(cherry picked from commit 7da540d03eccc2f97950bf47e8b35ce8c889d1e0)
Updated CHANGES.txt
 2016-04-29 16:07:20 -05:00
Tsz-Wo Nicholas Sze
c927f938fe HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:28:23 -07:00
Jason Lowe
e12c9e3348 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501bc753858693dacdafba61d604702f71)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-04-29 15:07:07 +00:00
Jason Lowe
dbb7458735 Revert "YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe" 
This reverts commit 77ef473981c1af0166ed6307b5a2789ca7caa7ba.
 2016-04-29 14:38:30 +00:00
Jian He
77ef473981 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501bc753858693dacdafba61d604702f71)
 2016-04-28 22:00:42 -07:00
Jian He
d7b2da6518 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 
(cherry picked from commit a9707dceaf181ea0f6d3c4291d1f3ccc2681f273)
 2016-04-28 21:45:17 -07:00
Kihwal Lee
a3ece8b5b4 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 2016-04-28 16:47:04 -05:00
Arpit Agarwal
92548e09c6 HADOOP-13039. Add documentation for configuration property ipc.maximum.data.length (Contributed by Mingliang Liu) 2016-04-26 20:47:53 -07:00
Eric Payne
2ec33c4e3c YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda / Eric Badger 2016-04-26 16:14:07 +00:00
Akira Ajisaka
076215024b HDFS-10245. Fix the findbugs warnings in branch-2.7. Contributed by Brahma Reddy Battula. 2016-04-25 22:03:48 +09:00
Kihwal Lee
0bb23e22ce HADOOP-13052. ChecksumFileSystem mishandles crc file permissions. Contributed by Daryn Sharp. 
(cherry picked from commit 9dbdc8e12d009e76635b2d20ce940851725cb069)
 2016-04-22 15:22:42 -05:00
Kihwal Lee
9e3b91b141 HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a80156d2cacfea8184b41805070223d3a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-04-22 11:47:18 -05:00
Tsz-Wo Nicholas Sze
e7f1b8fcb3 HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:36:28 -07:00
Andrew Wang
b8febb774a HADOOP-13042. Restore lost leveldbjni LICENSE and NOTICE changes. 
(cherry picked from commit fea50c5440d83225958c5e346299334559fc37a4)
(cherry picked from commit c28e4b0651cfd5ed142980dc10f4c2a7388a39a0)
(cherry picked from commit bae229ae27605d039d692fdd1edc5b4d946ddbe8)

 Conflicts:
	LICENSE.txt
 2016-04-21 13:51:52 -07:00
Jian He
f1e4f4d83a MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe5800975ea0c4295c9911ae3f572ed37d)
 2016-04-20 19:03:53 -07:00
Arpit Agarwal
12c6c2c9a6 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:28:01 -07:00
Karthik Kambatla
cc6ae6fa7e MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab8d17e839a1ef0d7076efc78d28724a)

(cherry picked from commit b826168173c3386738acc12a5d62577f12aa06e9)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-18 21:58:53 -07:00
Eric Payne
2ec5bb0fe8 MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037d2b22dafc64f33d4f1231bef4ceba8)
 2016-04-18 16:12:53 +00:00
Walter Su
91ca90fdd1 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9d353677184ff5602966b11ffb408b9)
 2016-04-18 20:30:48 +08:00
Jason Lowe
0ad4634585 YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5c3ab0c79cacffc22b1f59408622ae7)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
 2016-04-15 20:52:11 +00:00
Akira Ajisaka
80d279b481 HADOOP-12989. Some tests in org.apache.hadoop.fs.shell.find occasionally time out. Contributed by Takashi Ohnishi. 
(cherry picked from commit 6e6b6dd5aaf93cfb373833cd175ee722d2cb708f)
(cherry picked from commit 5ae74507c0d64eef4ea65973d798981bc56de61b)
 2016-04-15 14:26:52 +09:00
Steve Loughran
d3a806666d HADOOP-12613. TestFind.processArguments occasionally fails. (Wei-Chiu Chuang via stevel) 
(cherry picked from commit c329cab92c49e7dbe4c48557908124c9792ae699)
 2016-04-15 14:22:53 +09:00
Robert Kanter
c9ba6fa9ba MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77d9438f0954ae4e1497d63997bb7347)
(cherry picked from commit f9f57265cf0ad2685d0d96ec59b8b78ac8bed175)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:58:45 -07:00
Jason Lowe
a996889313 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 9b5c5bd42f0cb240d0fe7754967765a99dd5be46)
 2016-04-14 19:44:31 +00:00
Junping Du
b4d24d7b0e Add entry of MAPREDUCE-6670 to CHANGES.txt after backport. 2016-04-13 09:02:33 -07:00
Junping Du
683f313c1f MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a42cd54bd88ce2de63122998e967efa)
(cherry picked from commit 5574f827991cfc3d14f71184296866d387ed174f)
 2016-04-13 08:59:22 -07:00
Walter Su
2ee244ac28 HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf946827b4d58c7c577ed0c574a7cff029)
 2016-04-13 17:51:01 +08:00
Rohith Sharma K S
3afdac6e2c YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 2016-04-13 11:13:42 +05:30
Vinayakumar B
b52ac3cb4d Addendum: HDFS-10271. 
Added CHANGES.txt entry
 2016-04-12 11:57:00 +05:30
Vinayakumar B
37e4e45390 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 2016-04-12 11:32:28 +05:30
Vinod Kumar Vavilapalli
66072070d0 HADOOP-12406. Fixed AbstractMapWritable.readFields to use the thread's ClassLoader to load class instead of System ClassLoader. Contributed by Nadeem Douba. 
(cherry picked from commit 069c6c62def4a0f94382e9f149581d8e22f6d31c)
 2016-04-11 12:04:12 -07:00
Akira Ajisaka
c748ced474 YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger. 
(cherry picked from commit 1b78b2ba17cffdd19ae056555dd7f89b7a3eb06a)
(cherry picked from commit 88556294e20f118ecd8295f492b9617da413c83d)
 2016-04-11 01:36:04 +09:00
Wangda Tan
801c95c1cc MAPREDUCE-6302. Preempt reducers after a configurable timeout irrespective of headroom. (kasha via wangda) 2016-04-08 14:35:28 -07:00
Tsz-Wo Nicholas Sze
262d769b4d HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:43:09 +08:00
Kihwal Lee
8e784afb15 Revert "HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo." 
This reverts commit 9bc9e13a979c775354b2394d926f466a0d5be514.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-04-06 17:35:54 -05:00
Vinayakumar B
62da8f6fa6 HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 48ca23def1d1c28448a65238814070e79c8f4c4e)
(cherry picked from commit d21bc811d82f685b0a1338bc513d9a925d305a17)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
 2016-04-06 10:49:34 +08:00
Kihwal Lee
dd701c9800 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 
(cherry picked from commit 9ba1e5af06070ba01dcf46e1a4c66713a1d43352)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java

(cherry picked from commit 6e37c5fe80ee9ce46afc26838391cc27ca6b1e6f)
 2016-04-05 17:00:42 -05:00
Masatake Iwasaki
76e0bb7a19 YARN-4917. Fix typos in documentation of Capacity Scheduler. (Takashi Ohnishi via iwasakims) 2016-04-06 04:03:01 +09:00
Masatake Iwasaki
6576fc6ccc YARN-4915. Fix typo in YARN Secure Containers documentation (Takashi Ohnishi via iwasakims) 2016-04-06 03:50:01 +09:00
Kihwal Lee
9608601330 HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit ef3da823573cbf16fd1d84479330dd457f95e0ff)
 2016-04-05 09:20:38 -05:00
Jason Lowe
49428ab6bb YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 2016-04-05 13:51:46 +00:00
Vinayakumar B
fc0ffc48d4 HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 2016-04-05 09:59:37 +08:00
Kihwal Lee
69d4fa0deb HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 
(cherry picked from commit a7d1fb0cd2fdbf830602eb4dbbd9bbe62f4d5584)
TestClientProtocolForPipelineRecovery modified to use DFSConfigKeys instead of HdfsClientConfigKeys.
 2016-04-04 16:54:04 -05:00
Li Lu
f242b78fd9 YARN-4706. UI Hosting Configuration in TimelineServer doc is broken. (Akira AJISAKA via gtcarrera9) 
(cherry picked from commit f61de4173684aa1767cef20b3cb4d54df20273cd)
 2016-04-04 14:46:28 -07:00
Allen Wittenauer
353f379168 YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw) 
(cherry picked from commit b1394d6307425a41d388c71e39f0880babb2c7a9)
(cherry picked from commit 92a3dbe44ffb113dead560fc179c2bc15fd6fce8)
 2016-04-01 16:58:33 -07:00
Sangjin Lee
856a131c65 YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 
(cherry picked from commit 6d67420dbc5c6097216fa40fcec8ed626b2bae14)
 2016-03-31 10:52:39 -07:00
Akira Ajisaka
480b01ba95 HADOOP-12902. JavaDocs for SignerSecretProvider are out-of-date in AuthenticationFilter. Contributed by Gabor Liptak. 
(cherry picked from commit acca149ec96f2932bebc492452a63a159de9ce47)
(cherry picked from commit b4fbc22688124020e90be04827d0f8b35bbe50bb)
 2016-03-31 16:07:22 +09:00
Arpit Agarwal
7f14dc3294 HDFS-9478. Reason for failing ipc.FairCallQueue contruction should be thrown. (Contributed by Ajith S) 2016-03-29 13:07:31 -07:00
Kihwal Lee
efb210cc30 Fix CHANGES.txt 2016-03-28 11:31:07 -05:00