Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,626 Commits 326 Branches 371 Tags
Commit Graph

4321 Commits

Author SHA1 Message Date
Xiaoyu Yao
eae5bde007 HDFS-10913. Introduce fault injectors to simulate slow mirrors. Contributed by Xiaobing Zhou. 2017-01-17 10:30:34 -08:00
Arpit Agarwal
be8e001f1f HDFS-11339. Support File IO sampling for Datanode IO profiling hooks. Contributed by Hanisha Koneru. 2017-01-16 22:49:14 -08:00
Arpit Agarwal
52fdd387ec HDFS-11299. Support multiple Datanode File IO hooks. Contributed by Hanisha Koneru. 2017-01-16 22:42:13 -08:00
Arpit Agarwal
8ec9dca2e1 HDFS-11279. Cleanup unused DataNode#checkDiskErrorAsync(). Contributed by Hanisha Koneru. 2017-01-16 22:14:06 -08:00
Xiaoyu Yao
eafaddca1a HDFS-11274. Datanode should only check the failed volume upon IO errors. Contributed by Xiaoyu Yao. 2017-01-16 22:14:04 -08:00
Arpit Agarwal
ec80de3ccc HDFS-11182. Update DataNode to use DatasetVolumeChecker. 2017-01-16 22:14:00 -08:00
Arpit Agarwal
413dccaf78 HDFS-10959. Update DataNode to use DatasetVolumeChecker. Contributed by Xiaoyu Yao. 2017-01-16 22:13:58 -08:00
Akira Ajisaka
2d6be7ea23 HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb52c36b74358f22db498424e82ec4a7)
 2017-01-17 10:22:29 +09:00
Arpit Agarwal
954dae26cd HDFS-11337. (HDFS-10958 backport). Add instrumentation hooks around Datanode disk IO. 2017-01-16 13:53:30 -08:00
Wei-Chiu Chuang
914eeb997b HDFS-11306. Print remaining edit logs from buffer if edit log can't be rolled. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 1cde954a4fe7760a09b680413ad763cbe4a8feb3)
 2017-01-13 11:46:57 -08:00
Ming Ma
b6258e2b15 HDFS-9391. Update webUI/JMX to display maintenance state info. (Manoj Govindassamy via mingma) 2017-01-10 20:20:13 -08:00
Mingliang Liu
ce5ad0e88f HDFS-11253. FileInputStream leak on failure path in BlockSender. Contributed by Arpit Agarwal 2017-01-10 14:48:28 -08:00
Xiaoyu Yao
48fb796fcb HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao. 2017-01-10 14:48:23 -08:00
Yongjun Zhang
137547579b HDFS-11292. log lastWrittenTxId etc info in logSyncAll. Contributed by Yongjun Zhang. 
(cherry picked from commit 603cbcd513a74c29e0e4ec9dc181ff08887d64a4)
 2017-01-09 16:16:42 -08:00
Arpit Agarwal
6f35700f04 HDFS-11149. Support for parallel checking of FsVolumes. 
Change-Id: I6e7ada81562f380a260f8f561d0fa6a4cb102560
 2017-01-06 07:55:19 -08:00
Arpit Agarwal
fd3b1ca26c HDFS-11148. Update DataNode to use StorageLocationChecker at startup. 
Change-Id: I61dfae840e767f9a33c0bb45a058f2b866cbffa4
 2017-01-05 20:49:34 -08:00
Arpit Agarwal
76f1ab524c HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup. 
Change-Id: Iddedbeefd056af165be3557ccad199fe2878b591
 2017-01-05 20:49:30 -08:00
Chris Nauroth
9c61858ee7 HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 4c8f9e130230457fc897ed7a2a09e14d078be90a)
 2017-01-05 15:05:01 -08:00
Andrew Wang
d4ca1c5226 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 2017-01-04 15:10:15 -08:00
Andrew Wang
2b5b62f9cc Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 80e664084894bbff9b1cc3102e71b161873fb6ef.
 2017-01-03 11:12:53 -08:00
Andrew Wang
80e6640848 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
(cherry picked from commit 7fcc73fc0d248aae1edbd4e1514c5818f6198928)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
 2017-01-03 09:58:45 -08:00
Lei Xu
db81cc3add HDFS-11267. Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage. (Manoj Govindassamy via lei) 
(cherry picked from commit a4f66655ec22ca8c960f971f2b0cdafbd3430ad7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
 2016-12-29 17:06:21 +08:00
Lei Xu
f349c7692d HDFS-11251. ConcurrentModificationException during DataNode#refreshVolumes. (Manoj Govindassamy via lei) 
(cherry picked from commit e9f1396834174646a8d7aa8fc6c4a4f724ca5b28)
 2016-12-29 15:35:17 +08:00
Daniel Templeton
af266c8c85 HDFS-11278. Add missing @Test annotation for TestSafeMode.testSafeModeUtils() (Contributed by Lukas Majercak via Daniel Templeton) 
(cherry picked from commit ce3613c9656e57efdc73edd1787bd9622d1b716e)
 2016-12-28 20:11:09 -08:00
Daniel Templeton
47f0a0e15b HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-28 07:03:51 -08:00
Brahma Reddy Battula
5d46d5d4c6 HDFS-11252. TestFileTruncate#testTruncateWithDataNodesRestartImmediately can fail with BindException. Contributed by Yiqun Lin. 
(cherry picked from commit 0ddb8defad6a7fd5eb69847d1789ba51952c0cf0)
 2016-12-28 10:37:21 +05:30
Mingliang Liu
74ee21a7b1 HDFS-11270. Document the missing options of NameNode bootstrap command. Contributed by Yiqun Lin 
(cherry picked from commit c0e0ef29696109af9a018462059f08fd99ee3121)
 2016-12-27 11:38:31 -08:00
Akira Ajisaka
cd331556f6 HDFS-11272. Refine the assert messages in TestFSDirAttrOp. Contributed by Jimmy Xiang. 
(cherry picked from commit ea547529cb17c931a756ebffa6942f71f761ad13)
 2016-12-26 17:17:14 +09:00
Akira Ajisaka
1b39268233 HDFS-11250. Fix a typo in ReplicaUnderRecovery#setRecoveryID. Contributed by Yiqun Lin. 
(cherry picked from commit 8f218ea2849477bdcb4918586aaefed0cd118341)
 2016-12-26 16:52:43 +09:00
Akira Ajisaka
a779b4ab8f HDFS-11271. Typo in NameNode UI. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 483cd06ad43b85e00a782fe225f1f40657bab204)
 2016-12-26 16:37:52 +09:00
Akira Ajisaka
16c4dcad85 HDFS-11258. Addendum patch to fix compile issue. Contributed by Jimmy Xiang. 2016-12-23 03:56:12 +09:00
Haohui Mai
de926f93c2 HDFS-11258. File mtime change could not save to editlog. Contributed by Jimmy Xiang. 2016-12-21 14:22:09 -08:00
Xiao Chen
1ab279c311 HDFS-11195. Return error when appending files by webhdfs rest api fails. Contributed by Yuanbo Liu. 
(cherry picked from commit 5b7acdd206f5a7d1b7af29b68adaa7587d7d8c43)
 2016-12-20 12:27:40 -08:00
Brahma Reddy Battula
7a92345f16 HDFS-11263. ClassCastException when we use Bzipcodec for Fsimage compression. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1b401f6a734df4e23a79b3bd89c816a1fc0de574)
 2016-12-20 21:02:16 +05:30
Akira Ajisaka
a95acbcb6e HDFS-11262. Remove unused variables in FSImage.java. Contributed by Jagadesh Kiran N. 
(cherry picked from commit d2026ae9e6c86f4fd9e48b80e71233181c1853d9)
 2016-12-20 14:36:03 +09:00
Wei-Chiu Chuang
0cb99db9d9 HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang. 2016-12-15 16:35:37 -08:00
Mingliang Liu
012d28a1fa HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger 2016-12-15 14:54:40 -08:00
Akira Ajisaka
eaa50a1087 HDFS-11249. Redundant toString() in DFSConfigKeys.java. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 36947f79ba66bf8ce150813e5dc1d06c789f2b2a)
 2016-12-16 00:34:43 +09:00
Akira Ajisaka
190b2dca62 HDFS-11204. Document the missing options of hdfs zkfc command. Contributed by Yiqun Lin. 
(cherry picked from commit 72bff192cd37ff97442e0f8dd477fbc2e58fc12d)
 2016-12-14 18:53:24 +09:00
Andrew Wang
6b795c34d0 HDFS-10684. WebHDFS DataNode calls fail without parameter createparent. Contributed by John Zhuge. 
(cherry picked from commit fbdbbd57cdc3d8c778fca9266a7cadf298c8ff6c)
 2016-12-13 18:01:35 -08:00
Mingliang Liu
282a562437 HDFS-11226. cacheadmin, cryptoadmin and storagepolicyadmin should support generic options. Contributed by Brahma Reddy Battula 
(cherry picked from commit e91494c27a0b9a4a456d92fe7d632a608c4f11f3)
 2016-12-12 18:17:54 -08:00
Xiaoyu Yao
0bdaa23334 Revert "HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao." 
This reverts commit 4c9ca47386845f55c43c684c7a8250536101ddc3.
 2016-12-09 21:37:13 -08:00
Xiaoyu Yao
4c9ca47386 HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao. 
(cherry picked from commit df983b524ab68ea0c70cee9033bfff2d28052cbf)
 2016-12-09 21:31:45 -08:00
Wei-Chiu Chuang
292bd78b44 HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang. 2016-12-09 16:07:56 -08:00
Andrew Wang
215abbdf5f HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore. 2016-12-09 14:29:22 -08:00
Andrew Wang
85adc6677b HDFS-11217. Annotate NameNode and DataNode MXBean interfaces as Private/Stable. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 5bd7dece920b20da95755cf7fbba9b3f39c7da69)
 2016-12-09 12:05:45 -08:00
Vinayakumar B
199d8b5594 HDFS-11224. Lifeline message should be ignored for dead nodes (Contributed by Vinayakumar B) 
(cherry picked from commit d1d4aba71b21871140b162583a4b94ce118e1fb3)
 2016-12-09 14:54:37 +05:30
Xiao Chen
e51f32f74c HDFS-11197. Listing encryption zones fails when deleting a EZ that is on a snapshotted directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 401c7318723d8d62c7fc29728f7f4e8d336b4d2f)
 2016-12-08 12:41:17 -08:00
Akira Ajisaka
e00a124cb5 HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin. 
(cherry picked from commit 9ef89ede2f18c76c601fd585cb9d47511f5fc3ed)
 2016-12-08 13:39:58 +09:00
Vinayakumar B
4b21b47f56 HDFS-9057. allow/disallow snapshots via webhdfs (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 6d2332ae375e26d024358c6e75fdb3c68a781a66)
 2016-12-07 15:58:15 -08:00