Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,994 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2979 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli 09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Kihwal Lee 2c51bbda51 HDFS-9221. HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array. (Staffan Friberg via yliu) 
(cherry picked from commit 0ff1216100)
 2015-11-02 09:07:16 -06:00
yliu e1798e8f21 Revert "HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee." 
This reverts commit c250b21c23.
 2015-10-31 16:25:53 +08:00
Sangjin Lee c8767fb68c Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b4)
 2015-10-30 18:53:10 -07:00
Jason Lowe e894be93f6 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-10-30 14:55:33 +00:00
Kihwal Lee c250b21c23 HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit 43539b5ff4)
 2015-10-30 09:39:49 -05:00
Kihwal Lee 13b256ed22 HDFS-9083. Replication violates block placement policy. Contributed by Rushabh Shah. 2015-10-28 14:49:41 -05:00
Akira Ajisaka ce01a5951b HDFS-9317. Document fsck -blockId and -storagepolicy options in branch-2.7. (aajisaka) 2015-10-28 13:14:58 +09:00
Kihwal Lee 59a2072135 HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list. Contributed by Daniel Templeton. 2015-10-27 16:42:33 -05:00
Kihwal Lee 1d23e1ec07 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 
(cherry picked from commit 8104d52269)
 2015-10-27 11:09:05 -05:00
Arpit Agarwal 653ef52ef2 HDFS-9305. Delayed heartbeat processing causes storm of subsequent heartbeats. (Contributed by Arpit Agarwal) 2015-10-26 15:47:37 -07:00
Kihwal Lee f2fd011cc1 HDFS-9290. DFSClient#callAppend() is not backward compatible for slightly older NameNodes. Contributed by Tony Wu. 
(cherry picked from commit b9e0417bdf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2015-10-23 16:46:14 -05:00
Kihwal Lee 4cf7f8441a HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd62)
 2015-10-15 11:26:14 -05:00
Kihwal Lee efba749760 HDFS-8676. Delayed rolling upgrade finalization can cause heartbeat expiration. Contributed by Walter Su. 
(cherry picked from commit 5b43db47a3)
 2015-10-13 13:25:10 -05:00
Kihwal Lee 61388b41e7 HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 
(cherry picked from commit 60b858bfa6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java
 2015-10-13 10:46:20 -05:00
Kihwal Lee 3a5b9f49df HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 
(cherry picked from commit abf3ad988d)
 2015-10-13 10:29:42 -05:00
Kihwal Lee 5cb80992fb HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 
(cherry picked from commit 99e5204ff5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
 2015-10-07 10:24:55 -05:00
Kihwal Lee b2bc72a051 HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) 
(cherry picked from commit d2b941f94a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-10-02 14:59:29 -05:00
Kihwal Lee 6e7c76a5bd HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 2015-09-28 15:19:57 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 2b4c7fcc67 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd)
(cherry picked from commit f48af332a7)
 2015-09-23 15:13:09 -07:00
Vinayakumar B d800e2e9ed HDFS-9043. Doc updation for commands in HDFS Federation (Contributed by J.Andreina) 
(cherry picked from commit 57003fa971)
(cherry picked from commit 96e3fbf401)
 2015-09-22 16:40:19 +05:30
Vinayakumar B 7d3e882a49 HDFS-9042. Update document for the Storage policy name (Contributed by J.Andreina) 
(cherry picked from commit d8455479b8)
 2015-09-12 12:37:01 +05:30
Kihwal Lee f2cc6b5fdf HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit 30acb7372a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-09-11 15:59:29 -05:00
Vinod Kumar Vavilapalli fb15053124 HDFS-7314. Moving to 2.6.1 CHANGES.txt section. 
(cherry picked from commit f103a70af5)
 2015-09-10 16:48:53 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) c05ba079e3 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed)
(cherry picked from commit cbf90f93f9)
 2015-09-10 16:10:56 -07:00
Colin Patrick Mccabe ee830b0d27 HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit a4d9acc51d)
(cherry picked from commit 9264b7e119)
 2015-09-10 11:56:30 -07:00
Ming Ma decf8a6653 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062)
(cherry picked from commit 516bbf1c20)
 2015-09-10 11:56:29 -07:00
Jing Zhao 6c127b44ca HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a)
(cherry picked from commit 17fb442a4c)
 2015-09-10 11:56:28 -07:00
cnauroth 188096f1fe HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. 
(cherry picked from commit 50eeea1300)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd

(cherry picked from commit 25db341278)
 2015-09-10 11:56:27 -07:00
Xiaoyu Yao d3562b4e1d HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao) 
(cherry picked from commit 0100b15501)
(cherry picked from commit e68e8b3b5c)
 2015-09-10 11:56:22 -07:00
Kihwal Lee cfb27d7f78 HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee. 
(cherry picked from commit 285b31e75e)

(cherry picked from commit 7e622076d4)
 2015-09-10 10:53:19 -07:00
Akira Ajisaka c5a74c5c37 HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0113e4528d)
(cherry picked from commit eb9e06f8e3)
 2015-09-09 17:23:25 +09:00
Jing Zhao fd100ccbae HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. 2015-09-08 10:52:42 -07:00
Tsuyoshi Ozawa 139fd21c83 HADOOP-5323. Trash documentation should describe its directory structure and configurations. Contributed by Weiwei Yang. 
(cherry picked from commit 9a87f81ce6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
 2015-09-03 22:39:12 +09:00
yliu 3ab43accaf HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect. (Kihwal Lee via yliu) 2015-09-02 09:00:22 +08:00
Kihwal Lee 285dad32eb fix merge error in CHANGES.TXT 2015-08-28 15:37:07 -05:00
Kihwal Lee 42aa880df3 HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao. 
(cherry picked from commit 3e715a4f4c)
 2015-08-28 13:27:17 -05:00
Jing Zhao ba0f66984c HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang. 
(cherry picked from commit dc7a061668)
(cherry picked from commit 2edfc3ce71)
 2015-08-20 20:42:20 -07:00
Vinod Kumar Vavilapalli 9faee040f9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a0)
 2015-08-20 17:39:29 -07:00
yliu d4b6c4a3cd HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu) 2015-08-20 20:43:26 +08:00
Jing Zhao 4c005a4e16 HDFS-8867. Enable optimized block reports. Contributed by Daryn Sharp. 
(cherry picked from commit f61120d964)
(cherry picked from commit c0a4cd978a)
 2015-08-19 10:41:30 -07:00
Akira Ajisaka f98b5ae3bb HDFS-8852. HDFS architecture documentation of version 2.x is outdated about append write support. Contributed by Ajith S. 
(cherry picked from commit fc509f66d8)
(cherry picked from commit 38b344adfa)
 2015-08-18 23:33:28 +09:00
Vinayakumar B 4d94bc1d95 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
Moved to 2.6.1

(cherry picked from commit 84bf71295a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 16:15:10 +05:30
Vinayakumar B 90f3641728 HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
Moved to 2.6.1

(cherry picked from commit 08bd4edf40)
 2015-08-14 12:38:59 +05:30
Vinayakumar B b06d342749 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit fa2641143c)
 2015-08-14 11:46:58 +05:30
Vinayakumar B f40714f8d8 HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit f2b4bc9b6a)
 2015-08-14 11:38:28 +05:30
Vinayakumar B 5ba4065345 HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne. 
Moved CHANGES.TXT entry to 2.6.1

(cherry picked from commit d25cb8fe12)
(cherry picked from commit 2c0063d420)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 11:27:46 +05:30
Akira Ajisaka b919c1c044 HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang. 
(cherry picked from commit 206d4933a5)
(cherry picked from commit b2a532f61c)
 2015-07-24 18:38:44 +09:00
Akira Ajisaka 3e793224f9 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df)
(cherry picked from commit b85bbca745)
 2015-07-21 11:42:21 +09:00
Kihwal Lee e263219829 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee 
(cherry picked from commit 59388a8015)
 2015-07-14 14:08:15 -05:00