Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,030 Commits 326 Branches 371 Tags
Commit Graph

11030 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Walter Su
09ffc65d94 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9d353677184ff5602966b11ffb408b9)
 2016-04-18 20:32:36 +08:00
Steve Loughran
e9699aa443 HDFS-10291 TestShortCircuitLocalRead failing (stevel) 2016-04-18 10:26:35 +01:00
Walter Su
7f47d984fa HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. 
(cherry picked from commit 67523ffcf491f4f2db5335899c00a174d0caaa9b)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java

(cherry picked from commit a6d0e9e56a7eef226c8a891cf741ab22d8205e1c)
 2016-04-18 09:39:46 +08:00
Xuan
84ab382939 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012ffeacad2bcaf712d86a7e5d1420b26)
(cherry picked from commit 94e4d349b018496ab25afc164e2f25c780010cfd)
 2016-04-16 19:40:46 -07:00
Eric Payne
1e33c05a7e MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:37:00 +00:00
Arun Suresh
aa457912ca YARN-4468. Document the general ReservationSystem functionality, and the REST API. (subru and carlo via asuresh) 
(cherry picked from commit 8460a323ce454c99e690728bc3a045513c1142e4)
 2016-04-15 17:24:44 -07:00
Jason Lowe
d389afb3da YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5c3ab0c79cacffc22b1f59408622ae7)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
 2016-04-15 20:45:20 +00:00
Naganarasimha
3e223d5dc0 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 
(cherry picked from commit fdbafbc9e59314d9f9f75e615de9d2dfdced017b)
 2016-04-16 00:00:03 +05:30
Steve Loughran
1dddf1ac8a HADOOP-13026 Should not wrap IOExceptions into a AuthenticationException in KerberosAuthenticator. Xuan Gong via stevel 2016-04-15 17:43:38 +01:00
Akira Ajisaka
71b7f1f85f HADOOP-12989. Some tests in org.apache.hadoop.fs.shell.find occasionally time out. Contributed by Takashi Ohnishi. 
(cherry picked from commit 6e6b6dd5aaf93cfb373833cd175ee722d2cb708f)
(cherry picked from commit 5ae74507c0d64eef4ea65973d798981bc56de61b)
 2016-04-15 14:25:16 +09:00
Steve Loughran
302d499cf1 HADOOP-12613. TestFind.processArguments occasionally fails. (Wei-Chiu Chuang via stevel) 
(cherry picked from commit c329cab92c49e7dbe4c48557908124c9792ae699)
 2016-04-15 14:20:11 +09:00
Wangda Tan
c202ad6721 MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 17:19:26 -07:00
Robert Kanter
8da0a490c6 MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77d9438f0954ae4e1497d63997bb7347)
(cherry picked from commit f9f57265cf0ad2685d0d96ec59b8b78ac8bed175)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:54:42 -07:00
Kihwal Lee
8b1e7842e3 HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit b9c9d03591a49be31f3fbc738d01a31700bfdbc4)
 2016-04-14 15:30:59 -05:00
Jason Lowe
9b5c5bd42f YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 3150ae8108a1fc40a67926be6254824c1e37cb38)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/recovery/NMLeveldbStateStoreService.java
 2016-04-14 19:40:10 +00:00
Kihwal Lee
7dd8798933 HDFS-10292. Add block id when client got Unable to close file exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 2c155afe2736a5571bbb3bdfb2fe6f9709227229)
 2016-04-14 14:26:59 -05:00
Kihwal Lee
019ca8f7c1 HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit c970f1d00525e4273075cff7406dcbd71305abd5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2016-04-14 12:50:36 -05:00
Steve Loughran
4d90014560 HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel) 2016-04-14 12:45:26 +01:00
Steve Loughran
6b1f86f7f6 HADOOP-12969 Mark IPC.Client and IPC.Server as @Public, @Evolving (Xiaobing Zhou via stevel) 2016-04-14 10:35:04 +01:00
Akira Ajisaka
284718decc HADOOP-12822. Change "Metrics intern cache overflow" log level from WARN to INFO. Contributed by Andras Bokor. 
(cherry picked from commit 27b131e79c5fa99de3ed4fb529d854dd5da55bde)
(cherry picked from commit 03bf09865f2cf7fe503e90bdeed1ccff1097eb89)
 2016-04-14 10:58:05 +09:00
Xiaoyu Yao
df78ea84df HADOOP-12993. Change ShutdownHookManger complete shutdown log from INFO to DEBUG. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8ced42daff5cd0cb11d26042ae8c8255ef629a40)
 2016-04-13 16:43:45 -07:00
Andrew Wang
1dee3e1200 HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun. 
(cherry picked from commit 314aa21a89134fac68ac3cb95efdeb56bd3d7b05)
(cherry picked from commit 4c18f54c1d98e7ad61966af3384904e33a38daae)
 2016-04-13 16:40:08 -07:00
Xuan
3127b86492 YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu 
(cherry picked from commit e0cb426758b3d716ff143f723fc16ef2f1e4971b)
(cherry picked from commit 5bc64dafc3b20b31a6175dd556a730d197e0e737)
 2016-04-13 10:40:04 -07:00
Kihwal Lee
ccaf746eea HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 
(cherry picked from commit d2f3bbc29046435904ad9418073795439c71b441)
 2016-04-13 11:24:45 -05:00
Junping Du
c85a63aba1 YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák. 
(cherry picked from commit 08ddb3ac6da634c293c638d76b53af163ede3057)
(cherry picked from commit a3f8491410c5c8b50640904c95cc78d65410ce04)
 2016-04-13 09:05:09 -07:00
Junping Du
80c151b889 MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a42cd54bd88ce2de63122998e967efa)
(cherry picked from commit 5574f827991cfc3d14f71184296866d387ed174f)
 2016-04-13 08:58:24 -07:00
Walter Su
f3dbabb13a HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf946827b4d58c7c577ed0c574a7cff029)
 2016-04-13 17:50:47 +08:00
Colin Patrick Mccabe
59b0661c7f HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 
(cherry picked from commit 35f07705552ef5636f8b8d2599a6af5ec1426203)
(cherry picked from commit 2b0b332e2faedcee34db7b0c640aa1aba54b2f14)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2016-04-12 16:30:58 -07:00
Vinayakumar B
b804b20843 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 
(cherry picked from commit a9a607f8fc0d996af3fb37f7efa7591d6655900d)
(cherry picked from commit 04621537c94c8e406337e82800d2119ea25d8146)
 2016-04-12 11:47:01 +05:30
Akira Ajisaka
e2b8bdbe0a HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran. 
(cherry picked from commit a409508b3f4c46b419c41b9cdff83429d9d025ce)
(cherry picked from commit 8f968171bd5d672c8078275a243e3d590feef6e4)
 2016-04-12 14:55:42 +09:00
Rohith Sharma K S
9d4dedd65f YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 
(cherry picked from commit ff722bbbdfaf9715643507628e94ece26d904b9d)
 2016-04-12 10:05:35 +05:30
Vinod Kumar Vavilapalli
3589b9e10e YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi. 
(cherry picked from commit 44bbc50d919388e4ad08be2e9ba80ac7502d2579)
 2016-04-11 12:13:37 -07:00
Jason Lowe
3cf7856878 Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 9d3c51eb5a701ae4ba296e2fd3fba154fa0097c0.
 2016-04-11 15:57:56 +00:00
Akira Ajisaka
13be7a849d YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 1ff27f9d12e8124c1b9a722708264c5b07fd0fde)
(cherry picked from commit 8cf6630fc6cedbd86eff9da6f35ce1da4ed7ed2f)
 2016-04-11 14:55:54 +09:00
Akira Ajisaka
35fd4d4124 YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger. 
(cherry picked from commit 1b78b2ba17cffdd19ae056555dd7f89b7a3eb06a)
(cherry picked from commit 88556294e20f118ecd8295f492b9617da413c83d)
 2016-04-11 01:33:02 +09:00
Eric Payne
3de0ff030c MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037d2b22dafc64f33d4f1231bef4ceba8)
 2016-04-09 18:29:44 +00:00
Steve Loughran
f6351519b1 HADOOP-12444 Support lazy seek in S3AInputStream. Rajesh Balamohan via stevel 2016-04-09 11:24:39 +01:00
Wangda Tan
67ee302723 YARN-4826. Document configuration of ReservationSystem for CapacityScheduler. (Subru Krishnan via wangda) 
(cherry picked from commit fcb3fcd4c674d58e5f40b17eff7048e8ae45d151)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md

(cherry picked from commit dac2021f870af674ab1a64e937c4f2a388dcc692)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
 2016-04-08 16:33:55 -07:00
Chris Nauroth
1b5b85e560 HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran. 
(cherry picked from commit 843ee8d59d8bacbca0d87ccf0790772e39d16138)
(cherry picked from commit 26a23eff828a3f4fcbd4362761c329657603277d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
 2016-04-08 13:44:27 -07:00
Jian He
4cf0b1932f YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong 2016-04-08 11:21:34 -07:00
Andrew Wang
bc148530d4 HADOOP-12951. Improve documentation on KMS ACLs and delegation tokens. Contributed by Xiao Chen. 
(cherry picked from commit 594c70f779b277bd0b9d0a5dc98c3e9cc49b7e91)
(cherry picked from commit 2f983b161ffcbed23cfd7c1fac237244aec7e78d)
 2016-04-07 23:51:19 -07:00
Karthik Kambatla
ddb1407980 YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 
(cherry picked from commit e82f961a3925aadf9e53a009820a48ba9e4f78b6)
(cherry picked from commit 2b97a50eec8e9f7167a44b8ca0391fce0aae571c)
 2016-04-07 17:35:06 -07:00
Tsz-Wo Nicholas Sze
413e5b4926 HADOOP-12909. Change ipc.Client to support asynchronous calls. Contributed by Xiaobing Zhou 2016-04-07 14:51:50 +08:00
Suresh Srinivas
cbd10bfa8c HADOOP-8813. Add InterfaceAudience and InterfaceStability annotations to RPC Server and Client classes. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389649 13f79535-47bb-0310-9956-ffa450edef68
 2016-04-07 14:47:04 +08:00
Tsz-Wo Nicholas Sze
f872d519db HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:42:27 +08:00
Colin Patrick Mccabe
354801fe61 HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose 
(cherry picked from commit 4bd7cbc29d142fc56324156333b9a8a7d7b68042)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java

(cherry picked from commit b5e6ad457f39dd5f2176de700612e2757b3c1812)
 2016-04-06 21:11:48 -07:00
Eric Payne
497c65ad0c HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee 
(cherry picked from commit aede8c10ecad4f2a8802a834e4bd0b8286cebade)
 2016-04-06 20:33:40 +00:00
Colin Patrick Mccabe
8514ab8cb5 HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe) 
(cherry picked from commit 188f65287d5b2f26a8862c88198f83ac59035016)
(cherry picked from commit 387d3f25c192dc89484c3372bbc401a57175ff8e)
 2016-04-06 11:38:19 -07:00
Akira Ajisaka
e426d1bb3f HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits. Contributed by Allen Wittenauer. 
(cherry picked from commit 1371f993a9fa59d7e91321f022eab9974319d641)
 2016-04-06 17:44:54 +09:00
Wangda Tan
b7c83d4883 YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda) 
(cherry picked from commit 21eb4284487d6f8e4beedb8a0c3168e952f224fc)
 2016-04-05 16:30:56 -07:00