Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,249 Commits 326 Branches 371 Tags
Commit Graph

11249 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Akira Ajisaka
9891f51829 HADOOP-13033. Add missing Javadoc entries to Interns.java. Contributed by Andras Bokor. 
(cherry picked from commit c610031cabceebe9fe63106471476be862d6013c)
(cherry picked from commit 5949436010649d4088d8acfabf8e4c2c33ffdbd7)
 2016-04-23 06:21:03 +09:00
Kihwal Lee
989cd895e3 HADOOP-13052. ChecksumFileSystem mishandles crc file permissions. Contributed by Daryn Sharp. 
(cherry picked from commit 9dbdc8e12d009e76635b2d20ce940851725cb069)
 2016-04-22 15:06:25 -05:00
Kihwal Lee
3d3ed9b68e HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a80156d2cacfea8184b41805070223d3a9)
 2016-04-22 11:10:36 -05:00
Steve Loughran
5df89f9a8a HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel) 2016-04-22 11:24:24 +01:00
Larry McCay
adf937fbc0 HADOOP-13011 - Clearly Document the Password Details for Keystore-based 
Credential Providers
 2016-04-21 20:04:29 -07:00
Tsz-Wo Nicholas Sze
60c68c65c1 HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand 2016-04-21 15:45:36 -07:00
Tsz-Wo Nicholas Sze
704c57e3eb HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:25:49 -07:00
Eric Payne
b726ae0876 YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda 
(cherry picked from commit 3dce486d88895dcdf443f4d0064d1fb6e9116045)
 2016-04-21 22:01:07 +00:00
Andrew Wang
22403be0c3 HADOOP-13043. Add LICENSE.txt entries for bundled javascript dependencies. 
(cherry picked from commit 08b7efa95202b6d6ada143cab9369fac4ebb4c49)
(cherry picked from commit e16040b2af5046e3919a1f20cb801ba2ebc823c9)
 2016-04-21 14:05:11 -07:00
Andrew Wang
bae229ae27 HADOOP-13042. Restore lost leveldbjni LICENSE and NOTICE changes. 
(cherry picked from commit fea50c5440d83225958c5e346299334559fc37a4)
(cherry picked from commit c28e4b0651cfd5ed142980dc10f4c2a7388a39a0)
 2016-04-21 13:50:09 -07:00
Yongjun Zhang
a1beb5fe1e HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit a749ba0ceaa843aa83146b6bea19e031c8dc3296)
 2016-04-21 12:29:49 -07:00
Jian He
0b008dc21c MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe5800975ea0c4295c9911ae3f572ed37d)
 2016-04-20 19:03:19 -07:00
Chris Nauroth
7fb1cfc591 HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth. 
(cherry picked from commit 63ac2db59af2b50e74dc892cae1dbc4d2e061423)
(cherry picked from commit cc56bed6867f844810f5fda0cba3cbc9a9367269)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
 2016-04-20 14:00:06 -07:00
Colin Patrick Mccabe
e2968f5da2 HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe) 
(cherry picked from commit ad36fa6f42870c517526618a30204b443bfc6b5a)
(cherry picked from commit 76d963acb422f7b7e8e83768ab4be538ea5ad94e)
 2016-04-20 11:52:49 -07:00
Arpit Agarwal
a13628fe4a HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:27:46 -07:00
Walter Su
1860b8ceb5 HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun. 
(cherry picked from commit 90d1af141666ba12f18d44464cb6169c445d0902)
 2016-04-19 18:56:23 +08:00
Colin Patrick Mccabe
e1d9e5ab82 HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe) 
(cherry picked from commit cb3ca460efb97be8c031bdb14bb7705cc25f2117)
(cherry picked from commit a69b6b1e8b1e734ba9a106061cc6d22583d1d2d1)
 2016-04-18 13:49:05 -07:00
Vinod Kumar Vavilapalli
92e45b8321 Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 
(cherry picked from commit 477003730e6a7c7eff11892f5cedf74073ca867b)
 2016-04-18 11:51:21 -07:00
Kihwal Lee
7ac2e5ec7b HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun. 
(cherry picked from commit d8b729e16fb253e6c84f414d419b5663d9219a43)
 2016-04-18 08:01:17 -05:00
Walter Su
09ffc65d94 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9d353677184ff5602966b11ffb408b9)
 2016-04-18 20:32:36 +08:00
Steve Loughran
e9699aa443 HDFS-10291 TestShortCircuitLocalRead failing (stevel) 2016-04-18 10:26:35 +01:00
Walter Su
7f47d984fa HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. 
(cherry picked from commit 67523ffcf491f4f2db5335899c00a174d0caaa9b)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java

(cherry picked from commit a6d0e9e56a7eef226c8a891cf741ab22d8205e1c)
 2016-04-18 09:39:46 +08:00
Xuan
84ab382939 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012ffeacad2bcaf712d86a7e5d1420b26)
(cherry picked from commit 94e4d349b018496ab25afc164e2f25c780010cfd)
 2016-04-16 19:40:46 -07:00
Eric Payne
1e33c05a7e MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:37:00 +00:00
Arun Suresh
aa457912ca YARN-4468. Document the general ReservationSystem functionality, and the REST API. (subru and carlo via asuresh) 
(cherry picked from commit 8460a323ce454c99e690728bc3a045513c1142e4)
 2016-04-15 17:24:44 -07:00
Jason Lowe
d389afb3da YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5c3ab0c79cacffc22b1f59408622ae7)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
 2016-04-15 20:45:20 +00:00
Naganarasimha
3e223d5dc0 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 
(cherry picked from commit fdbafbc9e59314d9f9f75e615de9d2dfdced017b)
 2016-04-16 00:00:03 +05:30
Steve Loughran
1dddf1ac8a HADOOP-13026 Should not wrap IOExceptions into a AuthenticationException in KerberosAuthenticator. Xuan Gong via stevel 2016-04-15 17:43:38 +01:00
Akira Ajisaka
71b7f1f85f HADOOP-12989. Some tests in org.apache.hadoop.fs.shell.find occasionally time out. Contributed by Takashi Ohnishi. 
(cherry picked from commit 6e6b6dd5aaf93cfb373833cd175ee722d2cb708f)
(cherry picked from commit 5ae74507c0d64eef4ea65973d798981bc56de61b)
 2016-04-15 14:25:16 +09:00
Steve Loughran
302d499cf1 HADOOP-12613. TestFind.processArguments occasionally fails. (Wei-Chiu Chuang via stevel) 
(cherry picked from commit c329cab92c49e7dbe4c48557908124c9792ae699)
 2016-04-15 14:20:11 +09:00
Wangda Tan
c202ad6721 MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 17:19:26 -07:00
Robert Kanter
8da0a490c6 MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77d9438f0954ae4e1497d63997bb7347)
(cherry picked from commit f9f57265cf0ad2685d0d96ec59b8b78ac8bed175)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:54:42 -07:00
Kihwal Lee
8b1e7842e3 HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit b9c9d03591a49be31f3fbc738d01a31700bfdbc4)
 2016-04-14 15:30:59 -05:00
Jason Lowe
9b5c5bd42f YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 3150ae8108a1fc40a67926be6254824c1e37cb38)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/recovery/NMLeveldbStateStoreService.java
 2016-04-14 19:40:10 +00:00
Kihwal Lee
7dd8798933 HDFS-10292. Add block id when client got Unable to close file exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 2c155afe2736a5571bbb3bdfb2fe6f9709227229)
 2016-04-14 14:26:59 -05:00
Kihwal Lee
019ca8f7c1 HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit c970f1d00525e4273075cff7406dcbd71305abd5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2016-04-14 12:50:36 -05:00
Steve Loughran
4d90014560 HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel) 2016-04-14 12:45:26 +01:00
Steve Loughran
6b1f86f7f6 HADOOP-12969 Mark IPC.Client and IPC.Server as @Public, @Evolving (Xiaobing Zhou via stevel) 2016-04-14 10:35:04 +01:00
Akira Ajisaka
284718decc HADOOP-12822. Change "Metrics intern cache overflow" log level from WARN to INFO. Contributed by Andras Bokor. 
(cherry picked from commit 27b131e79c5fa99de3ed4fb529d854dd5da55bde)
(cherry picked from commit 03bf09865f2cf7fe503e90bdeed1ccff1097eb89)
 2016-04-14 10:58:05 +09:00
Xiaoyu Yao
df78ea84df HADOOP-12993. Change ShutdownHookManger complete shutdown log from INFO to DEBUG. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8ced42daff5cd0cb11d26042ae8c8255ef629a40)
 2016-04-13 16:43:45 -07:00
Andrew Wang
1dee3e1200 HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun. 
(cherry picked from commit 314aa21a89134fac68ac3cb95efdeb56bd3d7b05)
(cherry picked from commit 4c18f54c1d98e7ad61966af3384904e33a38daae)
 2016-04-13 16:40:08 -07:00
Xuan
3127b86492 YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu 
(cherry picked from commit e0cb426758b3d716ff143f723fc16ef2f1e4971b)
(cherry picked from commit 5bc64dafc3b20b31a6175dd556a730d197e0e737)
 2016-04-13 10:40:04 -07:00
Kihwal Lee
ccaf746eea HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 
(cherry picked from commit d2f3bbc29046435904ad9418073795439c71b441)
 2016-04-13 11:24:45 -05:00
Junping Du
c85a63aba1 YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák. 
(cherry picked from commit 08ddb3ac6da634c293c638d76b53af163ede3057)
(cherry picked from commit a3f8491410c5c8b50640904c95cc78d65410ce04)
 2016-04-13 09:05:09 -07:00
Junping Du
80c151b889 MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a42cd54bd88ce2de63122998e967efa)
(cherry picked from commit 5574f827991cfc3d14f71184296866d387ed174f)
 2016-04-13 08:58:24 -07:00
Walter Su
f3dbabb13a HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf946827b4d58c7c577ed0c574a7cff029)
 2016-04-13 17:50:47 +08:00
Colin Patrick Mccabe
59b0661c7f HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 
(cherry picked from commit 35f07705552ef5636f8b8d2599a6af5ec1426203)
(cherry picked from commit 2b0b332e2faedcee34db7b0c640aa1aba54b2f14)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2016-04-12 16:30:58 -07:00
Vinayakumar B
b804b20843 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 
(cherry picked from commit a9a607f8fc0d996af3fb37f7efa7591d6655900d)
(cherry picked from commit 04621537c94c8e406337e82800d2119ea25d8146)
 2016-04-12 11:47:01 +05:30
Akira Ajisaka
e2b8bdbe0a HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran. 
(cherry picked from commit a409508b3f4c46b419c41b9cdff83429d9d025ce)
(cherry picked from commit 8f968171bd5d672c8078275a243e3d590feef6e4)
 2016-04-12 14:55:42 +09:00
Rohith Sharma K S
9d4dedd65f YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 
(cherry picked from commit ff722bbbdfaf9715643507628e94ece26d904b9d)
 2016-04-12 10:05:35 +05:30