Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,092 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4519 Commits

Author SHA1 Message Date
Kihwal Lee 0d9322aa5b HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah. 
(cherry picked from commit cef2815cf4)
 2017-05-08 08:29:03 -05:00
Lei Xu 1f30a8a105 HDFS-11687. Add new public encryption APIs required by Hive. (lei) 
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5e)
 2017-05-04 12:07:29 -07:00
Arpit Agarwal 89baea96d5 HDFS-11722. Change Datanode file IO profiling sampling to percentage. Contributed by Hanisha Koneru. 2017-05-03 16:30:01 -07:00
Arpit Agarwal 4b7d1ff326 Revert "HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru." 
This reverts commit dfc6ef42c4.
 2017-05-03 16:29:44 -07:00
Arpit Agarwal dfc6ef42c4 HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru. 2017-05-03 15:53:43 -07:00
Mingliang Liu 616413ecb2 HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran 
(cherry picked from commit 83dded556d)
 2017-05-03 11:23:15 -07:00
Erik Krogen 9d6b873d3c HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d9014bda93)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
 2017-05-02 18:39:37 -07:00
Xiaoyu Yao 3e7f97fb0b HDFS-11593. Change SimpleHttpProxyHandler#exceptionCaught log level from info to debug. Contributed by Xiaobing Zhou. 
(cherry picked from commit 1058b40843)
 2017-05-01 16:13:30 -07:00
Kihwal Lee f77d138725 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 2017-05-01 17:33:24 -05:00
Kihwal Lee 6a6d0ece99 HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:20:37 -05:00
Zhe Zhang fc46f398d9 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a3309830)
 2017-04-27 09:51:04 -07:00
Konstantin V Shvachko 4cbf5c5c41 HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabeb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2017-04-26 19:02:48 -07:00
Akira Ajisaka cc66153e12
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 93fa48fcf2)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
 2017-04-26 18:08:12 +09:00
Xiaoyu Yao fc0e6de929 HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
 2017-04-25 18:49:44 -07:00
Jason Lowe 31af85c18e HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee 
(cherry picked from commit e4321ec843)
 2017-04-25 17:31:31 -05:00
Yongjun Zhang 31d7d47070 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be3)
 2017-04-21 14:01:47 -07:00
John Zhuge 5046a45507 HDFS-11685. TestDistributedFileSystem.java fails to compile. Contributed by John Zhuge. 2017-04-19 22:10:18 -07:00
Andrew Wang efb790b0d6 HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with AssertionError. 
(cherry picked from commit 74a723852d)
 2017-04-19 18:10:16 -07:00
Yiqun Lin e278eb3ab5 HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 724bb86351)
 2017-04-19 12:15:52 -05:00
Arpit Agarwal 5c092097b6 HDFS-11603. Improve slow mirror/disk warnings in BlockReceiver. 2017-04-18 16:20:43 -07:00
Konstantin V Shvachko c3840bde54 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d58)
 2017-04-17 17:06:18 -07:00
Zhe Zhang 051ab77806 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb3)
 2017-04-17 16:28:58 -07:00
Yongjun Zhang c411adac32 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-04-14 10:14:02 -07:00
Hanisha Koneru 107f685f64 HDFS-11630. TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds. Contributed by Hanisha Koneru. 2017-04-12 17:11:36 -07:00
Hanisha Koneru ed8bb51f59 HDFS-11560. Expose slow disks via NameNode JMX. Contributed by Hanisha Koneru. 2017-04-12 16:59:37 -07:00
Kihwal Lee 8ed230c805 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 2017-04-12 13:28:26 -05:00
Chris Nauroth d5e2bd4096 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
 2017-04-11 21:18:47 -07:00
Mingliang Liu f1a85b2be3 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-11 18:09:43 -07:00
Hanisha Koneru f16bb79957 HDFS-11551. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 2017-04-11 16:54:28 -07:00
Xiaoyu Yao 0391c92022 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13b)
 2017-04-06 16:15:56 -07:00
Andrew Wang ed8f738be5 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:38:22 -07:00
Xiaoyu Yao 325be6e720 HDFS-11362. StorageDirectory should initialize a non-null default StorageDirType. Contribute by Hanisha Koneru. 
(cherry picked from commit dde3bbf351ef3121ae80a96d06668d8fe33c788f)
 2017-04-06 14:33:30 -07:00
Andrew Wang 832c2f52f8 HADOOP-13688. Stop bundling HTML source code in javadoc JARs. 
(cherry picked from commit 72a2ae6452)
 2017-04-06 13:14:59 -07:00
Arpit Agarwal 75e60b6bb7 HDFS-11131. TestThrottledAsyncChecker#testCancellation is flaky. 2017-04-05 17:54:27 -07:00
Xiaobing Zhou 9f20532ef5 HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:33:19 -07:00
Andrew Wang e154893d3b HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang. 2017-04-05 12:52:34 -07:00
Hanisha Koneru 4cdda4ba87 HDFS-11545. Propagate DataNode's slow disks info to the NameNode via Heartbeat. Contributed by Hanisha Koneru. 2017-04-05 07:35:09 -07:00
Akira Ajisaka 88e1799034
HDFS-9651. Add missing robots.txt files. 
(cherry picked from commit 8cb7325527)
 2017-04-05 12:48:12 +09:00
Akira Ajisaka a563fa8ee9
HDFS-9651. All web UIs should include a robots.txt file. Contributed by Lars Francke. 
(cherry picked from commit 09e6caec3d)
 2017-04-05 12:46:09 +09:00
Hanisha Koneru fbe86471cf HDFS-11551. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 2017-04-04 10:49:44 -07:00
Wei-Chiu Chuang 4f1f16f1a6 HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang. 
(cherry picked from commit bc7aff7cec)
 2017-04-03 07:36:08 -07:00
Mingliang Liu 9e7f8cbf9a HDFS-11592. Closing a file has a wasteful preconditions in NameNode. Contributed by Eric Badger 
(cherry picked from commit e5f0622a6f)
 2017-03-30 15:44:44 -07:00
Daniel Templeton e00ad6fca5 HDFS-11571. Typo in DataStorage exception message (Contributed by Anna Budai via Daniel Templeton) 
(cherry picked from commit 15e3873dc3)
 2017-03-29 10:57:52 -07:00
Andrew Wang 82b4a9c3d0 HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by SammiChen and Kai Zheng. 2017-03-28 21:40:53 -07:00
Masatake Iwasaki b7b9d74521 HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 
(cherry picked from commit 64ea62c1cc)
 2017-03-28 09:19:42 +09:00
Andrew Wang 10deafee82 HDFS-11170. Add builder-based create API to FileSystem. Contributed by SammiChen and Wei Zhou. 2017-03-27 14:35:03 -07:00
Junping Du 2fefa1fa2e Add HDFS jdiff for 2.8.0 release. 
(cherry picked from commit d174c06b01)
 2017-03-25 14:33:06 -07:00
Hanisha Koneru a79ad2d122 HDFS-11570. Unit test for NameNodeStatusMXBean. Contributed by Hanisha Koneru. 2017-03-24 14:44:33 -07:00
Xiaobing Zhou 757d9ebcc5 HDFS-11534. Add counters for number of blocks in pending IBR. Contributed by Xiaobing Zhou. 2017-03-24 14:40:00 -07:00
Wei-Chiu Chuang 14414705f7 HDFS-10506. OIV's ReverseXML processor cannot reconstruct some snapshot details. Contributed by Akira Ajisaka. 2017-03-24 08:43:14 -07:00