Commit Graph

4242 Commits

Author SHA1 Message Date
Lei Xu 5b8564f986 HDFS-11687. Add new public encryption APIs required by Hive. (lei)
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5e)
(cherry picked from commit 1f30a8a105)
2017-05-09 12:11:48 -07:00
Kihwal Lee 9d8d3a84bd HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah.
(cherry picked from commit cef2815cf4)
2017-05-08 08:31:18 -05:00
Xiaoyu Yao a80ac0822b HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore.
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
(cherry picked from commit 81883e3150c42e866ba9e68d68b734f2772344b2)
2017-05-04 16:03:39 -07:00
Erik Krogen bcafaf2c61 HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen.
(cherry picked from commit d9014bda93)
(cherry picked from commit 9d6b873d3c)
2017-05-02 18:44:26 -07:00
Ming Ma d1ae1fb449 HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma)
(cherry picked from commit b48f27e794)
(cherry picked from commit 32b115da1d)
2017-05-02 12:48:51 -07:00
Ming Ma 305a9d886a HDFS-9016. Display upgrade domain information in fsck. 2017-05-02 09:52:16 -07:00
Ming Ma c4c5533216 HDFS-9005. Provide configuration support for upgrade domain. 2017-05-02 06:53:32 -07:00
Kihwal Lee c9bf21b0f3 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 2017-05-01 17:34:19 -05:00
Kihwal Lee d1e6b6db7c HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:28:27 -05:00
Konstantin V Shvachko cb12ada03f HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko.
(cherry picked from commit 28eb2aabeb)
2017-04-27 11:42:48 -07:00
Zhe Zhang 70f18911c7 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen.
(cherry picked from commit d8a3309830)
(cherry picked from commit fc46f398d9)
2017-04-27 09:54:10 -07:00
Jason Lowe d85dbd11ae HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee
(cherry picked from commit e4321ec843)
2017-04-25 17:32:07 -05:00
Yiqun Lin daa532b902 HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin.
(cherry picked from commit 724bb86351)
2017-04-19 12:31:14 -05:00
Konstantin V Shvachko 8b3b3aac10 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko.
(cherry picked from commit 8dfcd95d58)
2017-04-17 17:09:38 -07:00
Zhe Zhang 07047335a3 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen.
(cherry picked from commit ad49098eb3)
(cherry picked from commit 051ab77806)
2017-04-17 16:36:21 -07:00
Wei-Chiu Chuang ecaedca5c4
HDFS-10506. OIV's ReverseXML processor cannot reconstruct some snapshot details. Contributed by Akira Ajisaka.
(cherry picked from commit 14414705f7)
2017-04-14 21:11:31 +09:00
Mingliang Liu 9eebbcf459 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-12 13:39:20 -07:00
Kihwal Lee 9dfe0b3515 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp.
(cherry picked from commit 8ed230c805)
2017-04-12 13:30:41 -05:00
Chris Nauroth c4bf504395 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore.
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
(cherry picked from commit d5e2bd4096)
2017-04-11 21:55:28 -07:00
Junping Du dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Xiaoyu Yao 6367bc1f09 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou.
(cherry picked from commit 0eacd4c13b)
(cherry picked from commit 0391c92022)
2017-04-06 16:26:57 -07:00
Andrew Wang b1dfdea5b4 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:39:56 -07:00
Andrew Wang d12a0a25b4 HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang.
(cherry picked from commit e154893d3b)
2017-04-05 16:36:07 -07:00
Wei-Chiu Chuang eff4b2fa07 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge.
(cherry picked from commit c5ca216915)
(cherry picked from commit 434403a2a0)
2017-04-05 16:35:49 -07:00
Xiao Chen 7ad5b278a7 HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge
(cherry picked from commit d9aae22fdf)
(cherry picked from commit 24d464a150)
2017-04-05 16:35:44 -07:00
Jing Zhao e216c15625 HDFS-11395. RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode. Contributed by Nandakumar.
(cherry picked from commit 55796a0946)
(cherry picked from commit e03d8ff488)
2017-04-05 16:29:04 -07:00
Xiaobing Zhou b9b6bd2932 HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:34:27 -07:00
Wei-Chiu Chuang f9d3bb3b83 HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang.
(cherry picked from commit bc7aff7cec)
(cherry picked from commit 4f1f16f1a6)
2017-04-03 07:39:41 -07:00
Mingliang Liu 14e5a8ed28 HDFS-11592. Closing a file has a wasteful preconditions in NameNode. Contributed by Eric Badger
(cherry picked from commit e5f0622a6f)
2017-03-30 15:47:21 -07:00
Andrew Wang 7b8f87eb78 HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by SammiChen and Kai Zheng.
(cherry picked from commit 82b4a9c3d0)
2017-03-28 21:41:10 -07:00
Masatake Iwasaki ca3ede5fda HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 2017-03-28 18:15:09 +09:00
Junping Du a4fcf1c7ca Add HDFS jdiff for 2.8.0 release.
(cherry picked from commit d174c06b01)
(cherry picked from commit 2fefa1fa2e)
2017-03-25 14:34:55 -07:00
Kihwal Lee 66824b39fd HDFS-10601. Improve log message to include hostname when the NameNode is in safemode. Contributed by Kuhu Shukla.
(cherry picked from commit ba62b50eba)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
2017-03-16 11:07:13 -05:00
Wei-Chiu Chuang 851ba7d9d1 HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy.
(cherry picked from commit 385d2cb777)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 60be2e5d8a)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
2017-03-13 13:43:00 -07:00
Yiqun Lin 1f2da39d05 HDFS-11512. Increase timeout on TestShortCircuitLocalRead#testSkipWithVerifyChecksum. Contributed by Eric Badger. 2017-03-13 18:29:57 +08:00
Andrew Wang 3f23d50c9a HDFS-11498. Make RestCsrfPreventionHandler and WebHdfsHandler compatible with Netty 4.0.
(cherry picked from commit 5e74196ede)
(cherry picked from commit 6e75c1e2f0)
2017-03-06 15:04:34 -08:00
Xiao Chen 83f1e3b10a Revert "HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang."
This reverts commit f70b9d9241.

(cherry picked from commit 67d2875f8f)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
2017-03-06 14:59:07 -08:00
Brahma Reddy Battula 31bec2c16b HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula.
(cherry picked from commit dcd03df9f9)
2017-03-01 10:49:36 +05:30
Zhe Zhang 265ddb20c3 HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang.
(cherry picked from commit d269b488a7)
(cherry picked from commit 5fb785e154)
2017-02-28 09:04:07 -08:00
Wei-Chiu Chuang 295e8685dd HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao.
(cherry picked from commit f3cdf29af4)
2017-02-25 21:15:07 -08:00
Eric Payne 10d54eb978 HDFS-11404. Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc. Contributed by Eric Badger
(cherry picked from commit a07ddef101)
2017-02-21 12:13:58 -06:00
Vinayakumar B 2b3e8b75c4 HDFS-11177. 'storagepolicies -getStoragePolicy' command should accept URI based path. (Contributed by Surendra Singh Lilhore)
(cherry picked from commit 4804050630)
(cherry picked from commit 00ca9f1792)
2017-02-21 18:17:38 +05:30
Wei-Chiu Chuang 4c47cb68e8 HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang.
(cherry picked from commit 859bd159ae)
(cherry picked from commit a8531d5d52)
2017-02-15 03:02:41 -08:00
Yongjun Zhang 5bb1350922 HDFS-11391. Numeric usernames do no work with WebHDFS FS write access. (Pierre Villard via Yongjun Zhang)
(cherry picked from commit 8e53f2b9b0)
2017-02-14 14:14:58 -08:00
Wei-Chiu Chuang eca70e4bff HDFS-11084. Add a regression test for sticky bit support of OIV ReverseXML processor. Contributed by Wei-Chiu Chuang.
(cherry picked from commit 89cadb42111e4ffbd3f4bde8250013bba23eb51e)
(cherry picked from commit 8e291e1015)
2017-02-14 09:09:01 -08:00
Wei-Chiu Chuang 2463923492 HDFS-11306. Print remaining edit logs from buffer if edit log can't be rolled. Contributed by Wei-Chiu Chuang.
(cherry picked from commit 1cde954a4f)
(cherry picked from commit 914eeb997b)
2017-02-13 10:18:04 -08:00
Kihwal Lee 22f8b66137 HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp.
(cherry picked from commit 33c62d2d19)
2017-02-10 12:56:44 -06:00
Eric Payne 4e423edd95 HDFS-9300. TestDirectoryScanner.testThrottle() is still a little flakey. Contributed by Daniel Templeton.
(cherry picked from commit 3c0adac88c)
2017-02-02 15:34:59 -05:00
Anu Engineer 4f135647fd HDFS-9911. TestDataNodeLifeline Fails intermittently. Contributed by Yiqun Lin
(cherry picked from commit a95639068c)
2017-01-27 21:09:03 +00:00
Zhe Zhang 1a6b6e249f HDFS-10534. NameNode WebUI should display DataNode usage histogram. Contributed by Kai Sasaki.
(cherry picked from commit 18e1d68209)
(cherry picked from commit 558fee285a)
2017-01-25 10:53:06 -08:00