Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,204 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4326 Commits

Author SHA1 Message Date
Inigo Goiri a20789f299 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6)
 2017-05-26 13:42:55 -07:00
Kihwal Lee bbef16b84e HDFS-11817. A faulty node can cause a lease leak and NPE on accessing data. Contributed by Kihwal Lee. 
(Updated TestBlockUnderConstruction for nextGenerationStamp method)
(cherry picked from commit 2cb63433ab)
 2017-05-25 17:34:56 -05:00
Brahma Reddy Battula 4f5846f1e3 HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 7bad748091)
 2017-05-25 22:40:11 +08:00
Wei-Chiu Chuang caad191c0e Revert "HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory." 
This reverts commit 998e44a972.
 2017-05-24 18:12:03 -07:00
Wei-Chiu Chuang 6c50341ea8 Revert "HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang." 
This reverts commit f9d3bb3b83.
 2017-05-24 18:11:42 -07:00
Brahma Reddy Battula a7f063576c HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore. 
(cherry picked from commit 9cab42cc79)
 2017-05-24 23:45:57 +08:00
Yiqun Lin 94a80974d3 HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao. 
(cherry picked from commit 9cbbd1eae8)
 2017-05-22 17:11:52 -07:00
Kihwal Lee 95e8f4c80d HDFS-11641. Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus. Contributed by Daryn Sharp. 
(cherry picked from commit 82ea3f4545)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-05-16 15:55:53 -05:00
Akira Ajisaka 9b595ba40b
HDFS-11833. HDFS architecture documentation descibes outdated placement policy. Contributed by Chen Liang. 
(cherry picked from commit 1d1c52b42f)
(cherry picked from commit c17cb03a23)
 2017-05-16 11:54:27 -04:00
Jason Lowe f42222a75b HDFS-11818. TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently. Contributed by Nathan Roberts 
(cherry picked from commit aeca471ec1)
 2017-05-12 17:48:00 -05:00
Vinayakumar B b121e047b4 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 
(cherry picked from commit 1411612aa4)
(cherry picked from commit 53d9f56a18)
 2017-05-12 07:47:57 +05:30
Kihwal Lee a8f209dee7 HDFS-11755. Underconstruction blocks can be considered missing. Contributed by Nathan Roberts. 2017-05-10 14:22:17 -05:00
Jason Lowe ef120bed7b HDFS-11745. Increase HDFS test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit fab73748a1)
 2017-05-10 13:03:48 -05:00
Lei Xu 5b8564f986 HDFS-11687. Add new public encryption APIs required by Hive. (lei) 
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5e)
(cherry picked from commit 1f30a8a105)
 2017-05-09 12:11:48 -07:00
Kihwal Lee 9d8d3a84bd HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah. 
(cherry picked from commit cef2815cf4)
 2017-05-08 08:31:18 -05:00
Xiaoyu Yao a80ac0822b HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
(cherry picked from commit 81883e3150c42e866ba9e68d68b734f2772344b2)
 2017-05-04 16:03:39 -07:00
Erik Krogen bcafaf2c61 HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d9014bda93)
(cherry picked from commit 9d6b873d3c)
 2017-05-02 18:44:26 -07:00
Ming Ma d1ae1fb449 HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma) 
(cherry picked from commit b48f27e794)
(cherry picked from commit 32b115da1d)
 2017-05-02 12:48:51 -07:00
Ming Ma 305a9d886a HDFS-9016. Display upgrade domain information in fsck. 2017-05-02 09:52:16 -07:00
Ming Ma c4c5533216 HDFS-9005. Provide configuration support for upgrade domain. 2017-05-02 06:53:32 -07:00
Kihwal Lee c9bf21b0f3 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 2017-05-01 17:34:19 -05:00
Kihwal Lee d1e6b6db7c HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:28:27 -05:00
Konstantin V Shvachko cb12ada03f HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabeb)
 2017-04-27 11:42:48 -07:00
Zhe Zhang 70f18911c7 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a3309830)
(cherry picked from commit fc46f398d9)
 2017-04-27 09:54:10 -07:00
Jason Lowe d85dbd11ae HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee 
(cherry picked from commit e4321ec843)
 2017-04-25 17:32:07 -05:00
Yongjun Zhang c88e5e1836 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be3)
 2017-04-21 14:12:04 -07:00
Yiqun Lin daa532b902 HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 724bb86351)
 2017-04-19 12:31:14 -05:00
Konstantin V Shvachko 8b3b3aac10 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d58)
 2017-04-17 17:09:38 -07:00
Zhe Zhang 07047335a3 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb3)
(cherry picked from commit 051ab77806)
 2017-04-17 16:36:21 -07:00
Wei-Chiu Chuang ecaedca5c4
HDFS-10506. OIV's ReverseXML processor cannot reconstruct some snapshot details. Contributed by Akira Ajisaka. 
(cherry picked from commit 14414705f7)
 2017-04-14 21:11:31 +09:00
Mingliang Liu 9eebbcf459 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-12 13:39:20 -07:00
Kihwal Lee 9dfe0b3515 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 
(cherry picked from commit 8ed230c805)
 2017-04-12 13:30:41 -05:00
Chris Nauroth c4bf504395 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
(cherry picked from commit d5e2bd4096)
 2017-04-11 21:55:28 -07:00
Junping Du dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Xiaoyu Yao 6367bc1f09 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13b)
(cherry picked from commit 0391c92022)
 2017-04-06 16:26:57 -07:00
Andrew Wang b1dfdea5b4 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:39:56 -07:00
Andrew Wang ab91ca7338 HADOOP-13688. Stop bundling HTML source code in javadoc JARs. 
(cherry picked from commit 72a2ae6452)
(cherry picked from commit 832c2f52f8)
 2017-04-06 13:16:15 -07:00
Andrew Wang d12a0a25b4 HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang. 
(cherry picked from commit e154893d3b)
 2017-04-05 16:36:07 -07:00
Wei-Chiu Chuang eff4b2fa07 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge. 
(cherry picked from commit c5ca216915)
(cherry picked from commit 434403a2a0)
 2017-04-05 16:35:49 -07:00
Xiao Chen 7ad5b278a7 HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge 
(cherry picked from commit d9aae22fdf)
(cherry picked from commit 24d464a150)
 2017-04-05 16:35:44 -07:00
Jing Zhao e216c15625 HDFS-11395. RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode. Contributed by Nandakumar. 
(cherry picked from commit 55796a0946)
(cherry picked from commit e03d8ff488)
 2017-04-05 16:29:04 -07:00
Xiaobing Zhou b9b6bd2932 HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:34:27 -07:00
Wei-Chiu Chuang f9d3bb3b83 HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang. 
(cherry picked from commit bc7aff7cec)
(cherry picked from commit 4f1f16f1a6)
 2017-04-03 07:39:41 -07:00
Mingliang Liu 14e5a8ed28 HDFS-11592. Closing a file has a wasteful preconditions in NameNode. Contributed by Eric Badger 
(cherry picked from commit e5f0622a6f)
 2017-03-30 15:47:21 -07:00
Andrew Wang 7b8f87eb78 HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by SammiChen and Kai Zheng. 
(cherry picked from commit 82b4a9c3d0)
 2017-03-28 21:41:10 -07:00
Masatake Iwasaki ca3ede5fda HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 2017-03-28 18:15:09 +09:00
Junping Du a4fcf1c7ca Add HDFS jdiff for 2.8.0 release. 
(cherry picked from commit d174c06b01)
(cherry picked from commit 2fefa1fa2e)
 2017-03-25 14:34:55 -07:00
John Zhuge be58711e91 HDFS-11561. HttpFS doc errors. Contributed by Yuanbo Liu. 
(cherry picked from commit ed2d03f2a5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/markdown/index.md
 2017-03-22 22:42:48 -07:00
Zhe Zhang 8ca9915d0f HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a8)
(cherry picked from commit f98f4bb6ca)
 2017-03-22 22:32:41 -07:00
Kihwal Lee 66824b39fd HDFS-10601. Improve log message to include hostname when the NameNode is in safemode. Contributed by Kuhu Shukla. 
(cherry picked from commit ba62b50eba)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-03-16 11:07:13 -05:00