Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,770 Commits 326 Branches 371 Tags
Commit Graph

12770 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arpit Agarwal
fd3b1ca26c HDFS-11148. Update DataNode to use StorageLocationChecker at startup. 
Change-Id: I61dfae840e767f9a33c0bb45a058f2b866cbffa4
 2017-01-05 20:49:34 -08:00
Arpit Agarwal
76f1ab524c HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup. 
Change-Id: Iddedbeefd056af165be3557ccad199fe2878b591
 2017-01-05 20:49:30 -08:00
Daniel Templeton
d1aa844dc6 MAPREDUCE-6715. Fix Several Unsafe Practices (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 0b8a7c18ddbe73b356b3c9baf4460659ccaee095)
 2017-01-05 17:57:14 -08:00
Chris Nauroth
9c61858ee7 HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 4c8f9e130230457fc897ed7a2a09e14d078be90a)
 2017-01-05 15:05:01 -08:00
Haohui Mai
20486730e8 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2017-01-04 21:03:05 -08:00
Andrew Wang
d4ca1c5226 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 2017-01-04 15:10:15 -08:00
Jason Lowe
db947fb870 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:44 +00:00
Rohith Sharma K S
3207762192 YARN-5988. RM unable to start in secure setup. Contributed by Ajith S. 
(cherry picked from commit e49e0a6e37f4a32535d7d4a07015fbf9eb33c74a)
 2017-01-04 12:10:05 +05:30
Akira Ajisaka
3e2d26a97f HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe9389ccdc153d7f4bf6574bf8e509867c1)
 2017-01-04 14:15:00 +09:00
Junping Du
dc2e44e8f8 YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 8fadd69047143c9c389cc09ca24100b5f90f79d2)
 2017-01-03 15:19:36 -08:00
Wangda Tan
23f6a33156 YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312c48e26b0cb2c1e1e803ef71d1839cf)
(cherry picked from commit 554e6656bc9cf898d14d1fe2da5be5bff8fc4284)
 2017-01-03 14:52:31 -08:00
Wangda Tan
cb64e8eb19 Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" because forgot to add JIRA Number 
This reverts commit 554e6656bc9cf898d14d1fe2da5be5bff8fc4284.
 2017-01-03 14:51:59 -08:00
Wangda Tan
554e6656bc Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312c48e26b0cb2c1e1e803ef71d1839cf)
 2017-01-03 14:50:19 -08:00
Mingliang Liu
1b8cb0b9c9 Revert "HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran" 
This reverts commit a447efa2abd550abd268a6402070f529db3d69ff.
 2017-01-03 14:45:19 -08:00
Mingliang Liu
a447efa2ab HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran 
(cherry picked from commit 451efb08fe0680d002c6856c104ebb366acee8a0)
 2017-01-03 13:09:29 -08:00
Robert Kanter
2f32a6aea2 YARN-5529. Create new DiskValidator class with metrics (yufeigu via rkanter) 
(cherry picked from commit 591fb159444037bf4cb651aa1228914f5d71e1bf)
 2017-01-03 12:14:13 -08:00
Andrew Wang
2b5b62f9cc Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 80e664084894bbff9b1cc3102e71b161873fb6ef.
 2017-01-03 11:12:53 -08:00
Andrew Wang
80e6640848 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
(cherry picked from commit 7fcc73fc0d248aae1edbd4e1514c5818f6198928)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
 2017-01-03 09:58:45 -08:00
Brahma Reddy Battula
029b6fbe71 Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao." 
This reverts commit 1bc9b316ba60b1ecc8d6fca8d78642933d2c186b.
 2017-01-03 19:26:44 +05:30
Brahma Reddy Battula
d6e5e4288c HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 
(cherry picked from commit 6938b677111a5fa6bf654717b310220f98b7cf20)
 2016-12-31 16:17:15 +05:30
Haohui Mai
1bc9b316ba HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2016-12-30 22:59:24 -08:00
Daniel Templeton
8cb7aa2b52 YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 
(cherry picked from commit 165d01a73e5f328108a9e876c7a751a3c2cec4a8)
 2016-12-30 08:56:03 -08:00
Mingliang Liu
ec646b1a9d HADOOP-13942. Build failure due to errors of javadoc build in hadoop-azure. Contributed by Kai Sasaki 
(cherry picked from commit c6a5b689db573046819f0e9193cb042e6c7298f4)
 2016-12-29 16:24:38 -08:00
Lei Xu
db81cc3add HDFS-11267. Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage. (Manoj Govindassamy via lei) 
(cherry picked from commit a4f66655ec22ca8c960f971f2b0cdafbd3430ad7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
 2016-12-29 17:06:21 +08:00
Lei Xu
f349c7692d HDFS-11251. ConcurrentModificationException during DataNode#refreshVolumes. (Manoj Govindassamy via lei) 
(cherry picked from commit e9f1396834174646a8d7aa8fc6c4a4f724ca5b28)
 2016-12-29 15:35:17 +08:00
Daniel Templeton
af266c8c85 HDFS-11278. Add missing @Test annotation for TestSafeMode.testSafeModeUtils() (Contributed by Lukas Majercak via Daniel Templeton) 
(cherry picked from commit ce3613c9656e57efdc73edd1787bd9622d1b716e)
 2016-12-28 20:11:09 -08:00
Robert Kanter
f9b48206dd YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter) 
(cherry picked from commit f216276d2164c6564632c571fd3adbb03bc8b3e4)
 2016-12-28 15:29:06 -08:00
Daniel Templeton
47f0a0e15b HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-28 07:03:51 -08:00
Sunil G
b3231b480d YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b65fc127b35368d0460b786ece88b5b8)
 2016-12-28 16:00:52 +05:30
Rohith Sharma K S
cb40e3cffa YARN-6001. Improve moveApplicationQueues command line. Contributed by Sunil G. 
(cherry picked from commit e297be74ed4c4334bf48dd01d3b10e302147041d)
 2016-12-28 12:17:38 +05:30
Wangda Tan
42f571728c YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda) 
(cherry picked from commit 0840b4329b2428b20b862f70d72cbdcd6d1618ed)
 2016-12-27 21:19:10 -08:00
Brahma Reddy Battula
5d46d5d4c6 HDFS-11252. TestFileTruncate#testTruncateWithDataNodesRestartImmediately can fail with BindException. Contributed by Yiqun Lin. 
(cherry picked from commit 0ddb8defad6a7fd5eb69847d1789ba51952c0cf0)
 2016-12-28 10:37:21 +05:30
Robert Kanter
88597991b0 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 
(cherry picked from commit 9262797e86453fc04b7ca3710b73b21fcdf9e6b4)
 2016-12-27 14:14:41 -08:00
Robert Kanter
8fbd36c60f YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter) 
(cherry picked from commit 1bbd023275db535ab80fcb60e022151e9679d468)
 2016-12-27 13:35:55 -08:00
Karthik Kambatla
10a154cedd YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha) 
(cherry picked from commit c3973e7080bf71b57ace4a6adf4bb43f3c5d35b5)
 2016-12-27 12:30:50 -08:00
Mingliang Liu
74ee21a7b1 HDFS-11270. Document the missing options of NameNode bootstrap command. Contributed by Yiqun Lin 
(cherry picked from commit c0e0ef29696109af9a018462059f08fd99ee3121)
 2016-12-27 11:38:31 -08:00
Akira Ajisaka
74dd2770cc HADOOP-13943. TestCommonConfigurationFields#testCompareXmlAgainstConfigurationClass fails after HADOOP-13863. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0665c5f09afb2d6f59b6c4d4980f8b1ff9cbe620)
 2016-12-28 01:41:56 +09:00
Daniel Templeton
28b605dd1a HADOOP-13940. Document the missing envvars commands (Contributed by Yiqun Lin via Daniel Templeton) 
(cherry picked from commit d432173737a532b7106682116013b751ba162ff0)
 2016-12-27 07:19:37 -08:00
Akira Ajisaka
cd331556f6 HDFS-11272. Refine the assert messages in TestFSDirAttrOp. Contributed by Jimmy Xiang. 
(cherry picked from commit ea547529cb17c931a756ebffa6942f71f761ad13)
 2016-12-26 17:17:14 +09:00
Akira Ajisaka
1b39268233 HDFS-11250. Fix a typo in ReplicaUnderRecovery#setRecoveryID. Contributed by Yiqun Lin. 
(cherry picked from commit 8f218ea2849477bdcb4918586aaefed0cd118341)
 2016-12-26 16:52:43 +09:00
Akira Ajisaka
a779b4ab8f HDFS-11271. Typo in NameNode UI. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 483cd06ad43b85e00a782fe225f1f40657bab204)
 2016-12-26 16:37:52 +09:00
Naganarasimha
4a9f965314 YARN-6026. A couple of spelling errors in the docs. Contributed by Grant Sohn. 2016-12-26 06:50:06 +05:30
Mingliang Liu
edac82737a HADOOP-13863. Azure: Add a new SAS key mode for WASB. Contributed by Dushyanth 
(cherry picked from commit e92a77099b91620cee84513cc879089907468075)
 2016-12-22 20:33:51 -08:00
Akira Ajisaka
522bc98026 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539eaf7745ec8690f9281726fb4cdcdbe94)
 2016-12-23 04:06:53 +09:00
Akira Ajisaka
16c4dcad85 HDFS-11258. Addendum patch to fix compile issue. Contributed by Jimmy Xiang. 2016-12-23 03:56:12 +09:00
Varun Saxena
3ba589cd1e YARN-5903. Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup method (Haibo Chen via Varun Saxena) 
(cherry picked from commit 38e66d4d64f3c2e2bb43d8e5dca3866d672322b6)
 2016-12-22 23:11:18 +05:30
Haohui Mai
de926f93c2 HDFS-11258. File mtime change could not save to editlog. Contributed by Jimmy Xiang. 2016-12-21 14:22:09 -08:00
Junping Du
cc42dbb703 YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 
(cherry picked from commit 736f54b727c3f0ecc8fb9a594f2281c240c89cb8)
 2016-12-21 14:16:19 -08:00
Akira Ajisaka
897b4a6a77 HDFS-11261. Document missing NameNode metrics. Contributed by Yiqun Lin. 
(cherry picked from commit f6e2521eb216dae820846cab31397e9a88ba2f88)
 2016-12-21 14:20:15 +09:00
Xiao Chen
1ab279c311 HDFS-11195. Return error when appending files by webhdfs rest api fails. Contributed by Yuanbo Liu. 
(cherry picked from commit 5b7acdd206f5a7d1b7af29b68adaa7587d7d8c43)
 2016-12-20 12:27:40 -08:00