Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,503 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4885 Commits

Author SHA1 Message Date
Anu Engineer 88c749a903 HDFS-10618. TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition. Contributed by Eric Badger. 2018-03-12 12:14:21 -07:00
Inigo Goiri 457895d50f Revert "HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin." 
This reverts commit ab0f127084.
 2018-03-12 11:44:06 -07:00
Inigo Goiri ab0f127084 HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 
(cherry picked from commit ac627f561f)
 2018-03-12 10:04:51 -07:00
Yiqun Lin 3a34924896 HDFS-13240. RBF: Update some inaccurate document descriptions. Contributed by Yiqun Lin. 
(cherry picked from commit 4743d4a2c7)
 2018-03-10 11:33:44 +08:00
Inigo Goiri 0dadcf9ea3 HDFS-13232. RBF: ConnectionPool should return first usable connection. Contributed by Ekanth S. 
(cherry picked from commit 8133cd5305)
 2018-03-09 18:28:59 -08:00
Inigo Goiri 4b6b07690e HDFS-13212. RBF: Fix router location cache issue. Contributed by Weiwei Wu. 
(cherry picked from commit afe1a3ccd5)
 2018-03-09 17:21:10 -08:00
Yiqun Lin aa748c6bca HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 
(cherry picked from commit 122805b43a)
 2018-03-09 15:51:56 +08:00
Kihwal Lee 0aa52d4085 HDFS-12614. FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured. Contributed by Manoj Govindassamy. 2018-03-08 14:22:17 -06:00
Inigo Goiri 95f34ce462 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 
(cherry picked from commit 0c2b969e01)
(cherry picked from commit 44abf77d9b)
 2018-03-08 10:24:41 -08:00
Yiqun Lin a5d9691b54 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 
(cherry picked from commit 58ea2d7a65)
 2018-03-07 15:51:42 +08:00
Xiaoyu Yao 313fdb9968 HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 
(cherry picked from commit edf9445708)
 2018-03-06 20:01:51 -08:00
Xiaoyu Yao 4f3c4e4514 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d4)
 2018-03-06 18:49:13 -08:00
Xiao Chen 0882725c88 HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 2018-03-05 09:44:42 -08:00
Yiqun Lin 08cd911b69 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 
(cherry picked from commit a9c14b1119)
 2018-02-28 10:25:49 +08:00
Inigo Goiri 2f18785f39 HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d46)
 2018-02-27 17:42:28 -08:00
Yiqun Lin 1b0aa0e87a HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1)
 2018-02-27 10:54:37 +08:00
Wei Yan 54803ebe49 HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9)
 2018-02-26 15:32:27 -08:00
Kihwal Lee 4b43f2aa56 HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 451265a83d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockRecoveryWorker.java
 2018-02-26 10:58:07 -06:00
Arpit Agarwal 79af42f095 HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-24 15:12:36 -08:00
Xiao Chen 80f7165374 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d3233)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
 2018-02-23 14:00:33 -08:00
Gabor Bota 696e8e0299
HADOOP-15236. Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c36b4aa31c)
(cherry picked from commit b8942cb3bf)
 2018-02-23 13:58:08 +09:00
Xiao Chen 49ed7d7fc9 HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Gabor Bota, Wei-Chiu Chuang. 2018-02-19 19:15:02 -08:00
Yiqun Lin a278ad6b91 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 2018-02-20 09:58:28 +08:00
Kihwal Lee fe044e69c3 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e5)
 2018-02-15 15:38:49 -06:00
Kihwal Lee b725fd6924 xattr api cleanup 
(cherry picked from commit da59acd8ca)
 2018-02-15 11:28:45 -06:00
Brahma Reddy Battula 24af0bc144 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351ca)
 2018-02-12 22:25:44 +05:30
Arpit Agarwal 82530c966a HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:02:42 -08:00
Yiqun Lin d49074f0a2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee)
 2018-02-09 14:00:55 +08:00
Brahma Reddy Battula 9873eb63a7 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-08 18:33:11 +05:30
Yongjun Zhang b7f7fb003e HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
(cherry picked from commit f491f717e9)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
(cherry picked from commit 60d82ac61f)
 2018-02-07 15:35:41 -08:00
Yongjun Zhang 6dfea05049 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16)
 2018-02-02 23:33:10 -08:00
Yiqun Lin 7cd3770fee HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit b0627c891b)
 2018-02-02 17:38:32 +08:00
Inigo Goiri 7b8cc048cc HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204ceb)
 2018-02-01 10:38:39 -08:00
Yiqun Lin e5cac88c82 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit dbb9dded33)
 2018-01-31 09:14:31 -08:00
Weiwei Yang 0c2c4c20cb HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 
(cherry picked from commit 2e7331ca26)
 2018-01-30 21:40:07 -08:00
Inigo Goiri fffa7dc377 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca)
 2018-01-30 18:28:21 -08:00
Kihwal Lee eda786ea12 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 2018-01-29 17:52:04 -06:00
Xiao Chen 0922ea0aaf HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 
(cherry picked from commit b63dcd583f)
(cherry picked from commit 75be15cf02)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionFaultInjector.java
 2018-01-28 22:30:14 -08:00
Arpit Agarwal 7be5a46a41 HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:36 -08:00
Wei Yan cefe4b7bb6 HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 12:58:16 -08:00
Yiqun Lin 9ce24d2d50 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff744)
 2018-01-25 15:54:20 +08:00
Jianfei Jiang f4656e2150
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd08688)
 2018-01-25 15:32:13 +09:00
Inigo Goiri 1534936ac1 HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 
(cherry picked from commit 95743c672e)
 2018-01-23 19:17:22 -08:00
Inigo Goiri c6139332f9 HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 
(cherry picked from commit 130f8bce00)
 2018-01-19 09:52:29 -08:00
Yiqun Lin 65507053b4 HDFS-12973. RBF: Document global quota supporting in federation. Contributed by Yiqun Lin. 
(cherry picked from commit 9e4f52d323)
 2018-01-19 14:21:07 +08:00
Vinayakumar B b3eeea0b6e HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e1)
 2018-01-17 14:22:29 +05:30
Brahma Reddy Battula 8e7ce0eb4c HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 
(cherry picked from commit 880b9d24ff)
 2018-01-16 16:29:13 +05:30
Yiqun Lin e5612f0389 HDFS-12972. RBF: Display mount table quota info in Web UI and admin command. Contributed by Yiqun Lin. 
(cherry picked from commit 9afb8025d6)
 2018-01-13 13:20:07 +08:00
Wei-Chiu Chuang 2273499aef HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 2018-01-12 10:01:57 -08:00
Yiqun Lin 66d58d2b9a HDFS-12934. RBF: Federation supports global quota. Contributed by Yiqun Lin. 2018-01-10 19:48:47 +08:00
Inigo Goiri 215a3d800a HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 
(cherry picked from commit d9006d8a4e)
 2018-01-09 19:01:10 -08:00
Manoj Govindassamy eadd3cecf9 HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 2018-01-08 16:21:21 -08:00
Wei Yan 223bb935e2 HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 
(cherry picked from commit 2ee0d64ace)
 2018-01-08 15:23:57 -08:00
Inigo Goiri f7fedac6cd HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 
(cherry picked from commit 83b513ac6d)
 2018-01-05 09:13:52 -08:00
Xiao Chen 50b233d356 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 2017-12-28 11:55:24 -08:00
Yiqun Lin df3212aad5 HDFS-12937. RBF: Add more unit tests for router admin commands. Contributed by Yiqun Lin. 
(cherry picked from commit e040c97b77)
 2017-12-19 15:35:40 +08:00
Tsz-Wo Nicholas Sze c4e58ea823 HDFS-12347. TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently. Contributed by Bharat Viswanadham 2017-12-19 10:04:00 +08:00
Erik Krogen da4e2f38e1 HDFS-12818. Support multiple storages in DataNodeCluster / SimulatedFSDataset. Contributed by Erik Krogen. 
(cherry picked from commit 94576b17fb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/SimulatedFSDataset.java
 2017-12-18 12:23:20 -08:00
Jason Lowe 9ae8d1a8de HDFS-12881. Output streams closed with IOUtils suppressing write errors. Contributed by Ajay Kumar 2017-12-15 14:47:54 -06:00
Yiqun Lin 24c4d2a7a2 HDFS-12895. RBF: Add ACL support for mount table. Contributed by Yiqun Lin. 
(cherry picked from commit ee028bfdf1)
 2017-12-15 14:13:54 +08:00
Kihwal Lee 08a50da95f HDFS-12907. Allow read-only access to reserved raw for non-superusers. Contributed by Rushabh S Shah. 2017-12-14 15:03:47 -06:00
Yiqun Lin 22bc62058b HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 
(cherry picked from commit 91c96bdf8e)
 2017-12-14 16:03:34 +08:00
Inigo Goiri 67c2826941 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 
(cherry picked from commit 5cd1056ad7)
 2017-12-11 15:17:17 -08:00
Xiao Chen 31436a12d5 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 
(cherry picked from commit 19e0894209)
(cherry picked from commit 1d1401ad38)
 2017-12-07 21:20:33 -08:00
Xiao Chen c9d48352c2 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-12-07 21:20:33 -08:00
Chris Douglas afc606c3aa HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-07 00:43:21 -08:00
Akira Ajisaka 046424cf8a
HDFS-12889. Addendum patch to add missing file. 
(cherry picked from commit 6cca5b3bcb)
 2017-12-07 13:24:45 +09:00
Akira Ajisaka 4e92b1e89e
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 
(cherry picked from commit 44b06d34a5)
 2017-12-06 11:42:51 +09:00
Brahma Reddy Battula ceb4043ae1 HDFS-11751. DFSZKFailoverController daemon exits with wrong status code. Contributed by Bharat Viswanadham 
(cherry picked from commit 6d16a99ae8)
 2017-12-05 20:01:33 +05:30
Xiao Chen 274bbb193f HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:22 -08:00
Konstantin V Shvachko 3c57defaa4 HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 
(cherry picked from commit 60fd0d7fd7)
 2017-11-30 19:26:18 -08:00
Kihwal Lee 6fd9d11173 HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b)
 2017-11-29 15:15:41 -06:00
Wei Yan 9452cac210 HDFS-12835. Fix the javadoc errors in Router-based federation. 
(cherry picked from commit 301641811d)
 2017-11-29 10:12:15 -08:00
Konstantin V Shvachko 9d406e5dc8 HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 
(cherry picked from commit d331762f24)
 2017-11-28 17:52:08 -08:00
Yiqun Lin 3d73d2f097 HDFS-12858. RBF: Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 2017-11-28 13:42:41 +08:00
Kihwal Lee 45f59bde60 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 2017-11-27 16:52:18 -06:00
Tsz-Wo Nicholas Sze ed879d4a72 HDFS-12813. RequestHedgingProxyProvider can hide Exception thrown from the Namenode for proxy size of 1. Contributed by Mukul Kumar Singh 2017-11-20 17:11:18 -08:00
Inigo Goiri b57ea38fcf HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 
(cherry picked from commit e182e77794)
 2017-11-17 10:05:45 -08:00
Arpit Agarwal f671c22e3e HDFS-12705. WebHdfsFileSystem exceptions should retain the caused by exception. Contributed by Hanisha Koneru. 2017-11-13 13:56:13 -08:00
John Zhuge a42a19e139 HDFS-12052. Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS. Contributed by Zoran Dimitrijevic. 
(cherry picked from commit 12c8fdceaf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSServerWebServer.java
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServer.java
 2017-11-09 09:47:11 -08:00
John Zhuge d946307faa HDFS-12318. Fix IOException condition for openInfo in DFSInputStream. Contributed by legend. 
(cherry picked from commit 3123895db0)
 2017-11-09 08:58:03 -08:00
Brahma Reddy Battula eca6385fdd HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula 
(cherry picked from commit 410d0319cf)
 2017-11-08 14:43:13 +05:30
Weiwei Yang 0f20434e18 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang. 
(cherry picked from commit 56b88b0670)
 2017-11-01 17:01:23 +08:00
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Inigo Goiri 15b839c446 HDFS-12699. TestMountTable fails with Java 7. Contributed by Inigo Goiri. 
(cherry picked from commit 982bd2a5bf)
 2017-10-31 10:24:12 -07:00
Chris Douglas 9e721775a4 HADOOP-14986. Enforce JDK limitations. 
(cherry picked from commit b50def3697)
(cherry picked from commit 1660a638f0)
 2017-10-27 21:42:42 -07:00
Arpit Agarwal 8f28af4fac HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by Xiaoyu Yao. 2017-10-27 07:45:11 -07:00
Junping Du 1f2a2ba2be Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf)
 2017-10-26 23:20:24 -07:00
John Zhuge 952aa3f168 HDFS-12710. HTTPFS HTTP max header size env variable is not respected in branch-2. Contributed by Istvan Fajth. 2017-10-26 17:43:01 -07:00
Zhe Zhang 736fb3b66c Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit 86c2adc452.
 2017-10-24 15:05:26 -07:00
Arpit Agarwal a373094bf9 HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:35 -07:00
Inigo Goiri 12c81c67d7 HDFS-12620. Backporting HDFS-10467 to branch-2. Contributed by Inigo Goiri. 2017-10-19 17:40:42 -07:00
Inigo Goiri 629b88b4dd HDFS-12577. Rename Router tooling. Contributed by Inigo Goiri. 
(cherry picked from commit 53e8d0d030)
(cherry picked from commit 5d63a388d1)
 2017-10-19 17:39:27 -07:00
Inigo Goiri 27295eecdd HDFS-12273. Federation UI. Contributed by Inigo Goiri. 
(cherry picked from commit adbb2e00c7)
(cherry picked from commit 81601dac8e)
 2017-10-19 17:39:19 -07:00
Inigo Goiri 7d795989e1 HDFS-12580. Rebasing HDFS-10467 after HDFS-12447. Contributed by Inigo Goiri. 
(cherry picked from commit b12440d347)
(cherry picked from commit 6c69e23dcd)
 2017-10-19 17:39:18 -07:00
Inigo Goiri f8a71e2f10 HDFS-12381. [Documentation] Adding configuration keys for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit c2d6aa7905)
(cherry picked from commit ad41c81559)
 2017-10-19 17:39:17 -07:00
Inigo Goiri 9ad1f90b9d HDFS-12450. Fixing TestNamenodeHeartbeat and support non-HA. Contributed by Inigo Goiri. 
(cherry picked from commit 928d1e87f9)
(cherry picked from commit 67785fe006)
 2017-10-19 17:39:16 -07:00
Inigo Goiri ec84369538 HDFS-12430. Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218. Contributed by Inigo Goiri. 
(cherry picked from commit 3302e792d4)
(cherry picked from commit 1f06b81ecb)
 2017-10-19 17:39:15 -07:00
Inigo Goiri 550ce086a1 HDFS-12335. Federation Metrics. Contributed by Inigo Goiri. 
(cherry picked from commit 3b19e77752)
(cherry picked from commit bc9e588a19)
 2017-10-19 17:39:14 -07:00
Inigo Goiri 23753ff3c5 HDFS-12384. Fixing compilation issue with BanDuplicateClasses. Contributed by Inigo Goiri. 
(cherry picked from commit fabe02c8fa)
(cherry picked from commit cc58e7a983)
 2017-10-19 17:39:12 -07:00
Inigo Goiri a100f63a16 HDFS-10631. Federation State Store ZooKeeper implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 23c4ddee11)
(cherry picked from commit 7cb6bdf09e)
 2017-10-19 17:39:11 -07:00
Inigo Goiri 36c3515fbb HDFS-12312. Rebasing HDFS-10467 (2). Contributed by Inigo Goiri. 
(cherry picked from commit 90ba6843fb)
(cherry picked from commit 346c9fce43)
 2017-10-19 17:39:08 -07:00
Inigo Goiri 0b599cdc87 HDFS-11554. [Documentation] Router-based federation documentation. Contributed by Inigo Goiri. 
(cherry picked from commit ee3260211d)
(cherry picked from commit 67d10087af)
 2017-10-19 17:38:57 -07:00
Inigo Goiri 78abcb8f52 HDFS-10646. Federation admin tool. Contributed by Inigo Goiri. 
(cherry picked from commit ae27e31fbc)
(cherry picked from commit b3e6bd22e3)
 2017-10-19 17:38:54 -07:00
Inigo Goiri cc7c12ff18 HDFS-10880. Federation Mount Table State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 58b97df661)
(cherry picked from commit 6f0de27318)
 2017-10-19 17:38:33 -07:00
Inigo Goiri e700db60a6 HDFS-11826. Federation Namenode Heartbeat. Contributed by Inigo Goiri. 
(cherry picked from commit 928f8dab52)
(cherry picked from commit d8c8107332)
 2017-10-19 17:38:32 -07:00
Inigo Goiri 0228ead4d1 HDFS-10687. Federation Membership State Store internal API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 95cae08849)
(cherry picked from commit 55da7fd7eb)
 2017-10-19 17:38:30 -07:00
Inigo Goiri a09d7219c9 HDFS-12223. Rebasing HDFS-10467. Contributed by Inigo Goiri. 
(cherry picked from commit 47db6e9d8e)
(cherry picked from commit 0ec82b8cdf)
 2017-10-19 17:38:29 -07:00
Inigo Goiri 2f95eac8f6 HDFS-11546. Federation Router RPC server. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 8a9cdebebf)
(cherry picked from commit ca4f209b49)
 2017-10-19 17:38:28 -07:00
Inigo Goiri 7bd91fa2ed HDFS-10630. Federation State Store FS Implementation. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit c6e0bd640c)
(cherry picked from commit 4bf877b03f)
 2017-10-19 17:38:27 -07:00
Inigo 3f6d18ddf6 HDFS-10882. Federation State Store Interface API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6d94c90ece)
(cherry picked from commit 2c740a684a)
 2017-10-19 17:38:25 -07:00
Inigo c3b8b5e49f HDFS-10881. Federation State Store Driver API. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 0f88e04915)
(cherry picked from commit 533b986633)
 2017-10-19 17:38:24 -07:00
Inigo 096e8c4f35 HDFS-10629. Federation Roter. Contributed by Jason Kace and Inigo Goiri. 
(cherry picked from commit 6821e80172)
(cherry picked from commit 2761bbc91a)
 2017-10-19 17:38:21 -07:00
Wei-Chiu Chuang ce93a2b84b HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4ab0c8f96a)
 2017-10-19 06:18:40 -07:00
Zhe Zhang 87f4407396 HDFS-12502. nntop should support a category based on FilesInGetListingOps. 
(cherry picked from commit 60bfee270e)
(cherry picked from commit aecf6c50d3)
 2017-10-19 00:05:34 -07:00
Arpit Agarwal 0aa1b62b39 HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 
(cherry picked from commit b6942cbe9b)
(cherry picked from commit 5897095d53)
 2017-10-17 23:44:13 -07:00
Wei-Chiu Chuang a596c6f5eb HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c9)
 2017-10-16 12:59:41 -07:00
Xiao Chen b2f27641dd HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 
(cherry picked from commit 971b3ca1acc15d9a1f8e774291813dafd16ef704)
 2017-10-16 10:34:24 -07:00
Xiao Chen d82edb49c7 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 
(cherry picked from commit d0311dfc5fe92a0e0f3fb8ae92e412edce609740)
 2017-10-16 09:51:27 -07:00
Xiao Chen 4b734fe386 HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 
(cherry picked from commit 11a7729c1474aa379a62c0440c8c1a11be261a75)
 2017-10-15 19:28:38 -07:00
Andrew Wang 1ddc49527a Revert "HDFS-12603. Enable async edit logging by default." 
This reverts commit 42d3229e6f.
 2017-10-09 13:29:52 -07:00
Andrew Wang 42d3229e6f HDFS-12603. Enable async edit logging by default. 2017-10-09 11:21:43 -07:00
Xiao Chen dd0113d777 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)
 2017-10-05 19:47:04 -07:00
Yiqun Lin 2e2f8e42cf HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9d)
 2017-09-27 10:45:44 +08:00
Daryn Sharp 7f3ed5f61e [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:21:37 -05:00
Anu Engineer de5744c903 HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 2017-09-23 11:16:50 -07:00
Arpit Agarwal 09290659e7 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 2017-09-22 09:48:10 -07:00
Arpit Agarwal 18e5f2068b HDFS-12496. Make QuorumJournalManager timeout properties configurable. Contributed by Ajay Kumar. 2017-09-21 08:53:27 -07:00
Kihwal Lee 9640bfb808 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3)
 2017-09-21 08:48:04 -05:00
Akira Ajisaka 3527327f34
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb)
 2017-09-21 19:28:49 +09:00
Ming Ma 3bb23f4be9 HDFS-11035. Better documentation for maintenace mode and upgrade domain. 
(cherry picked from commit ce943eb17a)
 2017-09-20 09:42:22 -07:00
Ming Ma 7dd662eafd HDFS-12473. Change hosts JSON file format. 2017-09-20 09:08:41 -07:00
Brahma Reddy Battula eddc581860 HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55)
 2017-09-19 11:31:22 +05:30
Arpit Agarwal f42705b957 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:14:39 -07:00
Erik Krogen 791e20c244 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262)
 2017-09-15 14:03:10 -07:00
Nathan Roberts bd14e8d8b0 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc156)
 2017-09-15 14:47:07 -05:00
Kihwal Lee 07dee7fd63 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae4)
 2017-09-15 09:55:59 -05:00
John Zhuge eaef8446f6 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065)
 2017-09-14 11:29:34 -07:00
Arpit Agarwal ecf352d645 HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:21 -07:00
Xiao Chen 0a8ed025e4 Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 5337c2511d.
 2017-09-12 19:53:14 -07:00
Arpit Agarwal ac20d7e246 HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:18:56 -07:00
Manoj Govindassamy 796404c3ed HDFS-12404. Rename hdfs config authorization.provider.bypass.users to attributes.provider.bypass.users. 
(cherry picked from commit 3b3be355b3)
 2017-09-07 17:21:33 -07:00
Xiao Chen c6a1e3facb HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)
 2017-09-07 16:30:38 -07:00
Yongjun Zhang 32e8c84614 HDFS-12357. Let NameNode to bypass external attribute provider for configured users. Contributed by Yongjun Zhang, Arun Suresh. 
(cherry picked from commit d77ed238a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2017-09-07 10:22:37 -07:00
Wei-Chiu Chuang 3a2f3e78ff HDFS-12182. BlockManager.metaSave does not distinguish between "under replicated" and "missing" blocks. Contributed by Wellington Chevreuil. 2017-09-05 14:58:08 -07:00
Xiao Chen 858715d054 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 2017-08-30 09:43:12 -07:00
Yongjun Zhang 870ef0b2fd HDFS-12191. Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done to this file. Contributed by Yongjun Zhang. 
(cherry picked from commit cf93d60d3f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSDirAttrOp.java
 2017-08-29 21:57:25 -07:00
Arpit Agarwal e19f004f65 HDFS-11998. Enable DFSNetworkTopology as default. Contributed by Chen Liang. 2017-08-28 16:22:42 -07:00
Arpit Agarwal 98ba68b05d HDFS-12293. DataNode should log file name on disk error. Contributed by Ajay Kumar. 2017-08-28 15:50:06 -07:00
Yiqun Lin d63e98ad5f HDFS-11530. Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault. Contributed by Chen Liang. 2017-08-28 20:47:38 +08:00