Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,892 Commits 326 Branches 371 Tags
Commit Graph

6988 Commits

Author SHA1 Message Date
Hanisha Koneru
bd2ebf031c HDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contributed by Bharat Viswanadham. 
(cherry picked from commit 4eeff62f6925991bca725b1ede5308055817de80)
 2018-03-09 15:49:09 -08:00
Lokesh Jain
80aabd72c2
HDFS-13190. Document WebHDFS support for snapshot diff 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 7b0dc310208ee5bc191c9accb3d1312513145653)
 2018-03-09 15:07:34 -08:00
Hanisha Koneru
dec53bb9e1 HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. 
(cherry picked from commit 9a082fbe6e302df7139b65a23be9a39acd87715d)
 2018-03-09 13:39:49 -08:00
Yiqun Lin
f879504fe1 HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 2018-03-09 15:48:36 +08:00
Inigo Goiri
68cff8c235 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 
(cherry picked from commit 0c2b969e0161a068bf9ae013c4b95508dfb90a8a)
 2018-03-08 09:33:24 -08:00
Hanisha Koneru
0b1e9665df HDFS-13225. StripeReader#checkMissingBlocks() 's IOException info is incomplete. Contributed by lufei. 
(cherry picked from commit 19ae4429f9a2807548bb4b9ff296521fbdbbb2bc)
 2018-03-07 14:22:01 -08:00
Yiqun Lin
253ef33f80 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 2018-03-07 16:07:05 +08:00
Xiaoyu Yao
b74df5523c HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 
(cherry picked from commit edf9445708ffb7a9e59cb933e049b540f99add1e)
 2018-03-06 18:30:10 -08:00
Xiao Chen
283d48a4f7 HDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Stephen O'Donnell. 
(cherry picked from commit 92c15e19272a8bb59f8704c44d4cd713d3a58bcb)
 2018-03-06 09:58:09 -08:00
Xiao Chen
d28db463c2 HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 
(cherry picked from commit c75105f07b4cdbc2773435fc1125446233113c15)
 2018-03-02 16:47:36 -08:00
Xiaoyu Yao
908ca63c03 HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 
(cherry picked from commit f20e10b2dd59f99d9af009bbbb60a067b9893e69)
 2018-02-28 09:48:12 -08:00
Xiaoyu Yao
ce356dcbf6 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 
(cherry picked from commit 90fbe5c6834a6685e65542fa69e396fad2678927)
 2018-02-28 08:44:14 -08:00
Inigo Goiri
0eff245d3d HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d464683f8d392dad50e83f50d823e862e)
 2018-02-27 17:41:42 -08:00
Yiqun Lin
09017aee90 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1c7fe3dbf4b36a481feb5fdeeb6015b)
 2018-02-27 10:52:49 +08:00
Chao Sun
26395aef67 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 
(cherry picked from commit ae290a4bb4e514e2fe9b40d28426a7589afe2a3f)
 2018-02-26 16:18:51 -08:00
Wei Yan
1fb87df87e HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9b2107e5953a39d05b22966aff0b7ff)
 2018-02-26 15:29:11 -08:00
Kihwal Lee
1087b9af8c HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 451265a83d8798624ae2a144bc58fa41db826704)
 2018-02-26 10:30:50 -06:00
Xiaoyu Yao
21d4b5fd24 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 
(cherry picked from commit 1e84e46f1621fe694f806bfc41d3b2a06c9500b6)
 2018-02-23 19:41:28 -08:00
Xiao Chen
1fbd419957 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d323359587dca7831f74c9d065c2fccc60d)
 2018-02-23 14:00:25 -08:00
Xiaoyu Yao
ec1b445c62 HDFS-13136. Avoid taking FSN lock while doing group member lookup for FSD permission check. Contributed by Xiaoyu Yao. 
(cherry picked from commit e978e1d9064fc794ea83217d0af2315d4167211e)
 2018-02-22 20:18:02 -08:00
Steve Loughran
c2bbe22c5a HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-21 11:11:28 +00:00
Anu Engineer
c0d8103ba2 HDFS-13175. Add more information for checking argument in DiskBalancerVolume. 
Contributed by  Lei (Eddy) Xu.

(cherry picked from commit 121e1e1280c7b019f6d2cc3ba9eae1ead0dd8408)
 2018-02-20 19:59:31 -08:00
Yiqun Lin
7601489995 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 
(cherry picked from commit 8896d20b91520053a6bbfb680adb345cd24f4142)
 2018-02-20 13:06:26 -08:00
Kihwal Lee
7b83fca453 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e56b0380147d42f4110ad03c2276c961)
 2018-02-15 15:35:27 -06:00
Kihwal Lee
11ff6327af xattr api cleanup 
(cherry picked from commit da59acd8ca9ab5b49b988ffca64e8cce91c5f741)
 2018-02-15 11:14:42 -06:00
Brahma Reddy Battula
055d5e0b85 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
 2018-02-12 22:17:35 +05:30
Arpit Agarwal
41cf2b8e7c HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:16:48 -08:00
Yiqun Lin
38febc0ae2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 13:59:38 +08:00
Xiaoyu Yao
49ab09f00b HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d435039f69ea35f592856ca04d378809)
 2018-02-08 10:18:35 -08:00
Manoj Govindassamy
bef769e436 HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 
(cherry picked from commit 73ff09b79a5cf9932edc21c58f3a730f7379086b)

(cherry picked from commit 821729905ed8439ee5f15372faaf6b40aff7f155)
 2018-02-07 19:49:27 -08:00
Yongjun Zhang
5105b60614 HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java

(cherry picked from commit 60d82ac61f2afa2a61c8eeabf20e45b5a89311d0)
 2018-02-07 16:49:26 -08:00
Anu Engineer
2d6ab20007 HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 
(cherry picked from commit 4304fcd5bdf9fb7aa9181e866eea383f89bf171f)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java

(cherry picked from commit 9264f10bb35dbe30c75c648bf759e8aeb715834a)
 2018-02-07 11:16:53 -08:00
Brahma Reddy Battula
c6349c3204 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 
(cherry picked from commit 01bd6ab18fa48f4c7cac1497905b52e547962599)
 2018-02-07 23:27:24 +05:30
Xiao Chen
49de91446f HDFS-12933. Improve logging when DFSStripedOutputStream failed to write some blocks. Contributed by chencan. 
(cherry picked from commit e5c2fdddd69ceca21d641de56904648acedc74bf)
 2018-02-06 22:57:15 -08:00
Wei-Chiu Chuang
29a3b64ec0 HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 2021f4bdce3b27c46edaad198f0007a26a8a1391)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ReplicaBuilder.java
 2018-02-06 22:57:15 -08:00
Lei Xu
7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Yongjun Zhang
35247cca87 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16e81b4bd6fae443a997ef24d588a6e8)

(cherry picked from commit 0524fac21ef8a8e440a2fe759936e6fb6a63cbf1)
 2018-02-04 22:53:09 -08:00
Yiqun Lin
712e9381d2 HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 2018-02-02 17:34:12 +08:00
Akira Ajisaka
01661d4f90
HDFS-13048. LowRedundancyReplicatedBlocks metric can be negative 
(cherry picked from commit 4aef8bd2efd68bf96c077ddda1538dcd5691b437)
 2018-02-02 14:35:11 +09:00
Hanisha Koneru
992d7b5739 HDFS-13062. Provide support for JN to use separate journal disk per namespace. Contributed by Bharat Viswanadham. 
(cherry picked from commit dd50f53997239bf9078481cf46592ca3e41520b5)
 2018-02-01 12:32:47 -08:00
Xiaoyu Yao
784b454122 HDFS-12997. Move logging to slf4j in BlockPoolSliceStorage and Storage. Contributed by Ajay Kumar. 
(cherry picked from commit b3ae11d59790bb08b81848e9f944db7d3afbbd8a)
 2018-02-01 12:21:29 -08:00
Inigo Goiri
7491aaeb8f HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204cebf4b7060696a07b1777723a6dfa7d9d)
 2018-02-01 10:37:56 -08:00
Xiao Chen
4010262ae0 HDFS-12897. getErasureCodingPolicy should handle .snapshot dir better. Contributed by LiXin Ge. 
(cherry picked from commit ae2177d296a322d13708b85aaa8a971b8dcce128)
 2018-01-31 22:50:52 -08:00
Xiaoyu Yao
e260a0af24 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 
(cherry picked from commit b3ac6f7ce3f5e014af7ad6bf882431a97716e020)
 2018-01-31 22:40:13 -08:00
Hanisha Koneru
b71d775f57 HDFS-13092. Reduce verbosity for ThrottledAsyncChecker#schedule. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 3ce2190b581526ad2d49e8c3a47be1547037310c)
 2018-01-31 11:07:39 -08:00
Xiaoyu Yao
02790eb633 HDFS-13061. SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel. Contributed by Ajay Kumar. 
(cherry picked from commit f4bcae10d306ef2b0083e1de17253ec732b5b5c1)
 2018-01-31 10:47:46 -08:00
Inigo Goiri
961dcb693d HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 2018-01-31 09:08:47 -08:00
Inigo Goiri
0ce4851ee2 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca479dd53f614d75bab594043a9866e1)
 2018-01-30 18:27:51 -08:00
Weiwei Yang
7f62d2a473 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:35:54 +08:00
Kihwal Lee
673200ac1e HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 
(cherry picked from commit fde95d463c3123b315b3d07cb5b7b7dc19f7cb73)
 2018-01-29 17:25:30 -06:00