Commit Graph

4354 Commits

Author SHA1 Message Date
Xiaoyu Yao c9adca70e6 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu.
(cherry picked from commit e15e2713e1)
2017-07-12 16:39:22 -07:00
Akira Ajisaka d1fb4948d2
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor.
(cherry picked from commit 9144fd9e9b)
(cherry picked from commit 2653cf4466)
2017-07-12 11:44:15 +09:00
Mingliang Liu db6e8ff747 HDFS-12089. Fix ambiguous NN retry log message in WebHDFS. Contributed by Eric Badger
(cherry picked from commit 6436768baf)
2017-07-05 11:12:41 -07:00
Akira Ajisaka 9f9d554edf
HDFS-12078. Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml. Contributed by Weiwei Yang.
(cherry picked from commit 186650d21d)
(cherry picked from commit 063f2132ef)
2017-07-04 14:53:02 +09:00
Wei-Chiu Chuang 8027246306 HDFS-12042. Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption. Contributed by Misha Dmitriev.
(cherry picked from commit bcba844d11)
(cherry picked from commit 94bc5cdbb3)
2017-06-30 11:43:41 -07:00
Wei-Chiu Chuang 7c076f917f HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy.
(cherry picked from commit 16c8dbde57)
(cherry picked from commit 6917c19167)
2017-06-29 06:44:37 -07:00
Kihwal Lee a560104fab HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee.
(cherry picked from commit 8f533ca76b)
2017-06-20 12:12:33 -05:00
Kihwal Lee e87bf17129 HDFS-9754. Avoid unnecessary getBlockCollection calls in BlockManager. Contributed by Jing Zhao.
(cherry picked from commit 972782d956)
(cherry picked from commit f8c9c0ff0e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetadataConsistency.java
2017-06-20 12:10:43 -05:00
Kihwal Lee 1f9255672d Revert "HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee."
This reverts commit 40e3cd24b0.
2017-06-20 09:22:25 -05:00
Kihwal Lee 40e3cd24b0 HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee.
(cherry picked from commit 8f533ca76b)
2017-06-20 09:20:02 -05:00
Brahma Reddy Battula a526c3b0bf HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han.
(cherry picked from commit 2e9daa2e27)
2017-06-20 20:46:21 +08:00
Akira Ajisaka 189e39a90d
HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen.
(cherry picked from commit 2c769167db)
(cherry picked from commit 6b3c13d543)
2017-06-20 16:41:10 +09:00
Akira Ajisaka 6a4a63b0fd
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin.
(cherry picked from commit 71626fd0b9)
2017-06-20 13:56:20 +09:00
Masatake Iwasaki 6032511239 HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar.
(cherry picked from commit d954a64730)
2017-06-20 09:25:05 +09:00
Akira Ajisaka 4eb577c9f0
Revert "HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin."
This reverts commit a52315d2c3.
2017-06-16 00:22:24 +09:00
Akira Ajisaka a52315d2c3
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin.
(cherry picked from commit 315f07700d)
(cherry picked from commit 88fa17c238)
2017-06-15 13:47:40 +09:00
Arpit Agarwal 6ca195997b HDFS-11967. TestJMXGet fails occasionally. Contributed by Arpit Agarwal. 2017-06-12 16:04:02 -07:00
Mingliang Liu 72d895a11f HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee
(cherry picked from commit facb6812d9)
2017-06-08 14:49:37 -07:00
wenxin he 2c12102edd
HADOOP-14208. Fix typo in the top page in branch-2.8
This closes #230

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-06-08 11:36:24 +09:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Brahma Reddy Battula 8a16846f5d HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula.
(cherry picked from commit fca08f8362)
2017-06-07 16:59:26 +08:00
Vinayakumar B 4a391c72d4 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:59:04 +05:30
Arpit Agarwal 5fc4b8567b HDFS-11932. BPServiceActor thread name is not correctly set. Contributed by Chen Liang. 2017-06-06 13:52:20 -07:00
Kihwal Lee 1d3fa87011 HDFS-10816. TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor. Contributed by Eric Badger.
(cherry picked from commit e2ea601231)
2017-06-05 15:28:01 -05:00
Xiao Chen ec10a81bdb HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen.
(cherry picked from commit 068e23b896c63b0f817e6b91d73c994be1551eb2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
(cherry picked from commit 6f2391e0ad6fbaa6786cc9f0e3a09b955e0cb21c)
2017-06-01 14:05:31 -07:00
Kihwal Lee 89d59c292f HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-06-01 12:25:27 -05:00
Brahma Reddy Battula 0cba282261 HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula.
(cherry picked from commit 0dcf843c00)
2017-06-01 22:40:43 +08:00
Kihwal Lee fe6de5e594 HDFS-11856. Ability to re-add upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 13:08:51 -05:00
Inigo Goiri a20789f299 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri.
(cherry picked from commit 303c8dc9b6)
2017-05-26 13:42:55 -07:00
Kihwal Lee bbef16b84e HDFS-11817. A faulty node can cause a lease leak and NPE on accessing data. Contributed by Kihwal Lee.
(Updated TestBlockUnderConstruction for nextGenerationStamp method)
(cherry picked from commit 2cb63433ab)
2017-05-25 17:34:56 -05:00
Brahma Reddy Battula 4f5846f1e3 HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula.
(cherry picked from commit 7bad748091)
2017-05-25 22:40:11 +08:00
Wei-Chiu Chuang caad191c0e Revert "HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory."
This reverts commit 998e44a972.
2017-05-24 18:12:03 -07:00
Wei-Chiu Chuang 6c50341ea8 Revert "HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang."
This reverts commit f9d3bb3b83.
2017-05-24 18:11:42 -07:00
Brahma Reddy Battula a7f063576c HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore.
(cherry picked from commit 9cab42cc79)
2017-05-24 23:45:57 +08:00
Yiqun Lin 94a80974d3 HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao.
(cherry picked from commit 9cbbd1eae8)
2017-05-22 17:11:52 -07:00
Kihwal Lee 95e8f4c80d HDFS-11641. Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus. Contributed by Daryn Sharp.
(cherry picked from commit 82ea3f4545)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
2017-05-16 15:55:53 -05:00
Akira Ajisaka 9b595ba40b
HDFS-11833. HDFS architecture documentation descibes outdated placement policy. Contributed by Chen Liang.
(cherry picked from commit 1d1c52b42f)
(cherry picked from commit c17cb03a23)
2017-05-16 11:54:27 -04:00
Jason Lowe f42222a75b HDFS-11818. TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently. Contributed by Nathan Roberts
(cherry picked from commit aeca471ec1)
2017-05-12 17:48:00 -05:00
Vinayakumar B b121e047b4 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B)
(cherry picked from commit 1411612aa4)
(cherry picked from commit 53d9f56a18)
2017-05-12 07:47:57 +05:30
Kihwal Lee a8f209dee7 HDFS-11755. Underconstruction blocks can be considered missing. Contributed by Nathan Roberts. 2017-05-10 14:22:17 -05:00
Jason Lowe ef120bed7b HDFS-11745. Increase HDFS test timeouts from 1 second to 10 seconds. Contributed by Eric Badger
(cherry picked from commit fab73748a1)
2017-05-10 13:03:48 -05:00
Lei Xu 5b8564f986 HDFS-11687. Add new public encryption APIs required by Hive. (lei)
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5e)
(cherry picked from commit 1f30a8a105)
2017-05-09 12:11:48 -07:00
Kihwal Lee 9d8d3a84bd HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah.
(cherry picked from commit cef2815cf4)
2017-05-08 08:31:18 -05:00
Xiaoyu Yao a80ac0822b HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore.
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
(cherry picked from commit 81883e3150c42e866ba9e68d68b734f2772344b2)
2017-05-04 16:03:39 -07:00
Erik Krogen bcafaf2c61 HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen.
(cherry picked from commit d9014bda93)
(cherry picked from commit 9d6b873d3c)
2017-05-02 18:44:26 -07:00
Ming Ma d1ae1fb449 HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma)
(cherry picked from commit b48f27e794)
(cherry picked from commit 32b115da1d)
2017-05-02 12:48:51 -07:00
Ming Ma 305a9d886a HDFS-9016. Display upgrade domain information in fsck. 2017-05-02 09:52:16 -07:00
Ming Ma c4c5533216 HDFS-9005. Provide configuration support for upgrade domain. 2017-05-02 06:53:32 -07:00
Kihwal Lee c9bf21b0f3 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 2017-05-01 17:34:19 -05:00
Kihwal Lee d1e6b6db7c HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:28:27 -05:00