Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,330 Commits 326 Branches 371 Tags
Commit Graph

11330 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiaoyu Yao
5dfcfb2d2d HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao. 
(cherry picked from commit 45a753ccf79d334513c7bc8f2b81c89a4697075d)
 2016-05-02 19:50:10 -07:00
Xiaoyu Yao
643d2fd3c1 HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao. 
(cherry picked from commit 5bd7b592e5fbe4d448fd127c15d29f3121b8a833)
(cherry picked from commit a2a65922dd9783b76d9abacc07ba8cb32e81caea)
 2016-05-02 19:38:03 -07:00
Harsh J
cf6efccebb HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9f877e58f7f153c43cb4147876826d3)
 2016-05-02 17:53:10 -07:00
Andrew Wang
1207cf1cf6 MAPREDUCE-6537. Include hadoop-pipes examples in the release tarball. Contributed by Kai Sasaki. 
(cherry picked from commit 9e8411d0a630cabbfe7493763c172ff6754e51be)
(cherry picked from commit 5f46076f2bfc646a074a01f7968e3277f7e4abac)
 2016-05-02 15:35:21 -07:00
Tsz-Wo Nicholas Sze
a1ba6eee38 HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou 2016-05-02 11:34:59 -07:00
Tsz-Wo Nicholas Sze
bdc45bef64 HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-02 11:34:22 -07:00
Lei Xu
5477c31c5c HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 0a152103f19a3e8e1b7f33aeb9dd115ba231d7b7)
(cherry picked from commit 4289cb8b36bcb96510b9e63e3e966e306c6e3893)
 2016-05-02 10:21:59 -07:00
Colin Patrick Mccabe
ef9e7a10e4 HADOOP-13072. WindowsGetSpaceUsed constructor should be public (Vinayakumar B via cmccabe) 
(cherry picked from commit 2beedead72ee9efb69218aaf587de585158d6a1c)
(cherry picked from commit b16c376c3b62ec60aac4041099c4235bdaca815e)
 2016-05-02 09:44:07 -07:00
Robert Kanter
3c3d003402 Remove parent's env vars from child processes 
(cherry picked from commit ac8fb579c6058fec60caf30682f902413d68edf3)
 2016-04-29 15:30:06 -07:00
Kihwal Lee
e9cd88d484 HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 
(cherry picked from commit 7da540d03eccc2f97950bf47e8b35ce8c889d1e0)
 2016-04-29 15:58:45 -05:00
Chris Nauroth
3222dea8ed MAPREDUCE-6672. TestTeraSort fails on Windows. Contributed by Tibor Kiss. 
(cherry picked from commit 26ac0f8f0b28758a6404cd7b4b81d79b61d809d9)
(cherry picked from commit 752a8132832fd59027df5cb1bbedb9b3b7c54681)
 2016-04-29 11:33:54 -07:00
Tsz-Wo Nicholas Sze
31ed34c982 HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:27:14 -07:00
Kihwal Lee
a4db3729be HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah. 
(cherry picked from commit af9b000535cc987f66327e2b2bfe08923ba24c13)
 2016-04-29 10:05:58 -05:00
Masatake Iwasaki
50d06acb09 HADOOP-12378. Fix findbugs warnings in hadoop-tools module. Contributed by Akira AJISAKA. 
(cherry picked from commit ef0870ad038a1b72ea14ce550e34139b81eb901b)
 2016-04-29 20:40:47 +09:00
Jian He
5ba79d77fb YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501bc753858693dacdafba61d604702f71)
 2016-04-28 21:54:53 -07:00
Jian He
e62162db46 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 
(cherry picked from commit dd80042c42aadaa347db93028724f69c9aca69c6)
 2016-04-28 21:36:23 -07:00
Andrew Wang
0d4fbf01be HADOOP-13030. Handle special characters in passwords in KMS startup script. Contributed by Xiao Chen. 
(cherry picked from commit 7271e91b791ba277fd2a9b8f870eb9bc4f395566)
 2016-04-27 17:14:00 -07:00
Andrew Wang
bdb0e98483 HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge. 2016-04-27 17:03:36 -07:00
Li Lu
76b4ec32e6 YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit f16722d2ef31338a57a13e2c8d18c1c62d58bbaf)
 2016-04-27 10:58:50 -07:00
Arpit Agarwal
7cd0177310 HADOOP-5470. RunJar.unJar() should write the last modified time found in the jar entry to the uncompressed file. (Contributed by Andras Bakor) 2016-04-27 10:07:20 -07:00
Kihwal Lee
dd7c9f5b8f HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla. 
(cherry picked from commit e181092b86a8822e0f252b6ffa77baad4f711e77)
 2016-04-27 08:30:29 -05:00
Arpit Agarwal
b39e9efeeb HADOOP-13039. Add documentation for configuration property ipc.maximum.data.length (Contributed by Mingliang Liu) 2016-04-26 20:48:13 -07:00
Masatake Iwasaki
b4ed3285a1 HADOOP-12738. Create unit test to automatically compare Common related classes and core-default.xml. Contributed by Ray Chiang. 
(cherry picked from commit 68b4564e78380a2fac1a9000fb862104d4bc86e5)
 2016-04-27 11:05:56 +09:00
Yongjun Zhang
7ff4fb89ca HDFS-10313. Distcp need to enforce the order of snapshot names passed to -diff. (Lin Yiqun via Yongjun Zhang) 
(cherry picked from commit 959a28dd1216dfac78d05b438828e8503108d963)
(cherry picked from commit 7955b6f66ee670a57ccdeb7b6a9e6af25c0d67c7)
 2016-04-26 16:30:54 -07:00
Jing Zhao
b98b057f45 HDFS-10216. Distcp -diff throws exception when handling relative path. Contributed by Takashi Ohnishi. 
(cherry picked from commit 404f57f328b00a42ec8b952ad08cd7a80207c7f2)
(cherry picked from commit 864baf23aa3df201f81b721852221c6475388295)
 2016-04-26 16:30:46 -07:00
Kihwal Lee
621be60592 HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun. 
(cherry picked from commit c6a2430b47692a72c500373d6fc1bce78652675b)
 2016-04-26 11:47:15 -05:00
Kihwal Lee
973c73aa68 HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor. 
(cherry picked from commit 10f0f7851a3255caab775777e8fb6c2781d97062)
 2016-04-25 11:42:17 -05:00
Masatake Iwasaki
e01242edb4 HADOOP-11418. Property "io.compression.codec.lzo.class" does not work with other value besides default. (Yuanbo Liu via iwasakims) 
(cherry picked from commit bec5b4cd8c1fb27a5b707d601b508224a9c107ed)
 2016-04-25 22:50:07 +09:00
Akira Ajisaka
6cb501e431 HDFS-10298. Document the usage of distcp -diff option. Contributed by Takashi Ohnishi. 
(cherry picked from commit 02c51c27d92e74be7089083555e85da0e73f7fda)
(cherry picked from commit 58642d2244c3062672cc8a8b8a7cdbcd8929cd14)
 2016-04-25 22:34:37 +09:00
Masatake Iwasaki
c4d8090b72 HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims) 
(cherry picked from commit 6fcde2e38da04cae3aad6b13cf442af211f71506)
 2016-04-23 23:55:27 +09:00
Wangda Tan
358b54d063 YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda) 2016-04-22 15:59:49 -07:00
Akira Ajisaka
9891f51829 HADOOP-13033. Add missing Javadoc entries to Interns.java. Contributed by Andras Bokor. 
(cherry picked from commit c610031cabceebe9fe63106471476be862d6013c)
(cherry picked from commit 5949436010649d4088d8acfabf8e4c2c33ffdbd7)
 2016-04-23 06:21:03 +09:00
Kihwal Lee
989cd895e3 HADOOP-13052. ChecksumFileSystem mishandles crc file permissions. Contributed by Daryn Sharp. 
(cherry picked from commit 9dbdc8e12d009e76635b2d20ce940851725cb069)
 2016-04-22 15:06:25 -05:00
Kihwal Lee
3d3ed9b68e HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a80156d2cacfea8184b41805070223d3a9)
 2016-04-22 11:10:36 -05:00
Steve Loughran
5df89f9a8a HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel) 2016-04-22 11:24:24 +01:00
Larry McCay
adf937fbc0 HADOOP-13011 - Clearly Document the Password Details for Keystore-based 
Credential Providers
 2016-04-21 20:04:29 -07:00
Tsz-Wo Nicholas Sze
60c68c65c1 HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand 2016-04-21 15:45:36 -07:00
Tsz-Wo Nicholas Sze
704c57e3eb HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:25:49 -07:00
Eric Payne
b726ae0876 YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda 
(cherry picked from commit 3dce486d88895dcdf443f4d0064d1fb6e9116045)
 2016-04-21 22:01:07 +00:00
Andrew Wang
22403be0c3 HADOOP-13043. Add LICENSE.txt entries for bundled javascript dependencies. 
(cherry picked from commit 08b7efa95202b6d6ada143cab9369fac4ebb4c49)
(cherry picked from commit e16040b2af5046e3919a1f20cb801ba2ebc823c9)
 2016-04-21 14:05:11 -07:00
Andrew Wang
bae229ae27 HADOOP-13042. Restore lost leveldbjni LICENSE and NOTICE changes. 
(cherry picked from commit fea50c5440d83225958c5e346299334559fc37a4)
(cherry picked from commit c28e4b0651cfd5ed142980dc10f4c2a7388a39a0)
 2016-04-21 13:50:09 -07:00
Yongjun Zhang
a1beb5fe1e HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit a749ba0ceaa843aa83146b6bea19e031c8dc3296)
 2016-04-21 12:29:49 -07:00
Jian He
0b008dc21c MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe5800975ea0c4295c9911ae3f572ed37d)
 2016-04-20 19:03:19 -07:00
Chris Nauroth
7fb1cfc591 HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth. 
(cherry picked from commit 63ac2db59af2b50e74dc892cae1dbc4d2e061423)
(cherry picked from commit cc56bed6867f844810f5fda0cba3cbc9a9367269)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
 2016-04-20 14:00:06 -07:00
Colin Patrick Mccabe
e2968f5da2 HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe) 
(cherry picked from commit ad36fa6f42870c517526618a30204b443bfc6b5a)
(cherry picked from commit 76d963acb422f7b7e8e83768ab4be538ea5ad94e)
 2016-04-20 11:52:49 -07:00
Arpit Agarwal
a13628fe4a HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:27:46 -07:00
Walter Su
1860b8ceb5 HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun. 
(cherry picked from commit 90d1af141666ba12f18d44464cb6169c445d0902)
 2016-04-19 18:56:23 +08:00
Colin Patrick Mccabe
e1d9e5ab82 HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe) 
(cherry picked from commit cb3ca460efb97be8c031bdb14bb7705cc25f2117)
(cherry picked from commit a69b6b1e8b1e734ba9a106061cc6d22583d1d2d1)
 2016-04-18 13:49:05 -07:00
Vinod Kumar Vavilapalli
92e45b8321 Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 
(cherry picked from commit 477003730e6a7c7eff11892f5cedf74073ca867b)
 2016-04-18 11:51:21 -07:00
Kihwal Lee
7ac2e5ec7b HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun. 
(cherry picked from commit d8b729e16fb253e6c84f414d419b5663d9219a43)
 2016-04-18 08:01:17 -05:00