Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,726 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2481 Commits

Author SHA1 Message Date
Kihwal Lee a1c3e6a058 HDFS-7707. Edit log corruption due to delayed block removal again. Contributed by Yongjun Zhang 
(cherry picked from commit 843806d03a)

(cherry picked from commit c428d303f6)
(cherry picked from commit 21d8b22feef2fc9f6793d9c157391b142368e588)
 2015-08-31 15:55:07 -07:00
Arpit Agarwal f81f97eb7c HDFS-7575. Upgrade should generate a unique storage ID for each volume. (Contributed by Arpit Agarwal) 
(cherry picked from commit 1d9d166c0b)
(cherry picked from commit ca8e1b0739b6653833f9bc8990ab126420703f66)
 2015-08-31 15:08:42 -07:00
Kihwal Lee ae288f7291 HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade shutdown command. Contributed by Eric Payne. 
(cherry picked from commit 6bbf9fdd04)

(cherry picked from commit 33534a0c9a)
(cherry picked from commit e9a28251ee46e64e1b99b2dd54b0432bdc0b9578)
 2015-08-31 11:53:24 -07:00
cnauroth 8382e52c94 HDFS-7596. NameNode should prune dead storages from storageMap. Contributed by Arpit Agarwal. 
(cherry picked from commit ef3c3a832c)
(cherry picked from commit 75e4e55e12)
(cherry picked from commit cc637d6ece64dfeb89e78c7e9766836149e098be)
 2015-08-31 11:26:48 -07:00
Jing Zhao c8ce11d067 HDFS-7182. JMX metrics aren't accessible when NN is busy. Contributed by Ming Ma. 
(cherry picked from commit 4b589e7cfa)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystemMBean.java

(cherry picked from commit 96f0813c5d6140aabe7b2837f30971936276e689)
 2015-08-30 21:21:05 -07:00
cnauroth e6b52fc3e9 HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. 
(cherry picked from commit 7e2d9a3242)
(cherry picked from commit f0acb7c2a2)
(cherry picked from commit 33fb7b45195a89b6464e4a1cb3fbf6bbad2bcecb)
 2015-08-30 21:17:34 -07:00
Colin Patrick Mccabe c2e00e0012 HDFS-7531. Improve the concurrent access on FsVolumeList (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 3b173d9517)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

(cherry picked from commit dda1fc169d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeList.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java

(cherry picked from commit ba28192f9d5a8385283bd717bca494e6981d378f)
 2015-08-30 20:11:41 -07:00
Colin Patrick Mccabe 70f33f5eff HDFS-7035. Make adding a new data directory to the DataNode an atomic operation and improve error handling (Lei Xu via Colin P. McCabe) 
(cherry picked from commit a9331fe9b0)

(cherry picked from commit ec2621e907)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java

(cherry picked from commit d79a5849999cdb0bc315938b80ed71b4f2dcb720)
 2015-08-30 12:13:33 -07:00
Arpit Agarwal 46e9926ce3 HDFS-8486. Addendum patch to fix backporting bug. 
(cherry picked from commit 6c9e26609ceac5d16434cc7f4a1013e6faacd92f)
 2015-08-30 12:02:23 -07:00
Kihwal Lee 8549891cb3 HDFS-7213. processIncrementalBlockReport performance degradation. 
Contributed by Eric Payne.
(cherry picked from commit e226b5b40d)

(cherry picked from commit 946463efef)
(cherry picked from commit f94aa4d25c2f96faf5164e807c2c3eb031e9a1fe)
 2015-08-30 11:46:16 -07:00
Andrew Wang 0898c21e24 HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
(cherry picked from commit 406c09ad11)
(cherry picked from commit 2e15754a92)
(cherry picked from commit 014d07de2e9b39be4b6793f0e09fcf8548570ad5)
 2015-08-27 18:48:17 -07:00
Tao Luo 6d2f80107f HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
(cherry picked from commit 8bfef59029)

(cherry picked from commit 3827a1acdbc4f9fec3179dcafa614734b5fa31bc)
 2015-08-27 17:53:06 -07:00
Colin Patrick Mccabe d3a7769bff HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
(cherry picked from commit ac9ab037e9)

(cherry picked from commit 842a54a5f6)
(cherry picked from commit 1aa9e34c5106c496ffd390f6b2c822d387fb1908)
 2015-08-27 17:47:39 -07:00
Ming Ma dda8e0e328 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062)
(cherry picked from commit 516bbf1c20)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java

(cherry picked from commit fb1bf424bdad20fff7ab390ce75c4bec558e7e6d)
 2015-08-27 17:38:06 -07:00
Arpit Agarwal aa2303088f HDFS-8486. DN startup may cause severe data loss. (Contributed by Daryn Sharp) 2015-08-06 15:17:45 -07:00
Akira Ajisaka 0daf9dd552 Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:40:31 -05:00
Arpit Agarwal ef212fbe7e HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. (Contributed by Arpit Agarwal) 2015-02-04 16:26:00 -08:00
Tsz-Wo Nicholas Sze 4f5d330623 HDFS-3443. Fix NPE when namenode transition to active during startup by adding checkNNStartup() in NameNodeRpcServer. Contributed by Vinayakumar B 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
 2015-01-21 11:54:26 -08:00
Colin Patrick Mccabe 51d49ffffd HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume (cmccabe) 
(cherry picked from commit 8fa265a290)
(cherry picked from commit bcaf15e2fa)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
 2014-12-19 13:25:33 -08:00
arp 91042c950f HDFS-7503. Namenode restart after large deletions can cause slow processReport (Arpit Agarwal) 2014-12-10 23:38:55 -08:00
Colin Patrick Mccabe d93bfbace4 HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes (Noah Lorang via Colin P. McCabe) 
(cherry picked from commit d8352b9b2b)
(cherry picked from commit a037d6030b)
 2014-12-09 10:59:49 -08:00
Colin Patrick Mccabe 575d7bd5b0 HDFS-4882. Prevent the Namenode's LeaseManager from looping forever in checkLeases (Ravi Prakash via Colin P. McCabe) 
(cherry picked from commit daacbc18d7)
 2014-11-24 11:42:46 -08:00
cnauroth 2379932968 HDFS-7425. NameNode block deletion logging uses incorrect appender. Contributed by Chris Nauroth. 
(cherry picked from commit b8c094b075)
(cherry picked from commit 8d8eb8dcec)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-21 16:58:06 -08:00
Karthik Kambatla 5913c5143b Update release date for 2.5.2 
(cherry picked from commit 1e9a3f42ff)
 2014-11-21 14:35:06 -08:00
Arun C. Murthy 98a4538cf4 Preparing to release hadoop-2.6.0-rc1. 
(cherry picked from commit 9ff7315cbe)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2014-11-13 13:05:50 -08:00
cnauroth ed2a4db6d4 HDFS-7385. ThreadLocal used in FSEditLog class causes FSImage permission mess up. Contributed by jiangyu. 
(cherry picked from commit b0a41de68c)
(cherry picked from commit 4e9001530f)
 2014-11-13 12:37:48 -08:00
Haohui Mai 59b2a74ea9 Fix CHANGES.txt to keep it consistent with trunk. 2014-11-12 20:13:38 -08:00
Arun C. Murthy 797e62defa HDFS-7391. Renable SSLv2Hello in HttpFS. Contributed by Robert Kanter. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-12 19:00:50 -08:00
Arun C. Murthy 42849e5077 HDFS-7387. Merging to branch-2.6 for hadoop-2.6.0-rc1. 
(cherry picked from commit 09f36b1688)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-12 11:51:11 -08:00
Karthik Kambatla ba01844260 Set the release date for 2.5.2 
(cherry picked from commit 68a0508728)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 92de44c2c5)
 2014-11-10 15:24:14 -08:00
Arun C. Murthy 60584c732f Preparing to release hadoop-2.6.0: Set version in branch-2.6 to 2.6.1-SNAPSHOT. 2014-11-09 19:21:11 -08:00
Arun C. Murthy be31db82f7 Updated CHANGES.txt to reflect hadoop-2.6.0 release dates. 
(cherry picked from commit 770cc14442)
 2014-11-09 18:53:55 -08:00
Suresh Srinivas c98fcd1d3b HDFS-7383. DataNode.requestShortCircuitFdsForRead may throw NullPointerException. Contributed by Tsz Wo Nicholas Sze. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-09 18:18:58 -08:00
Arun C. Murthy 85531834b3 HADOOP-11286. Copied LimitInputStream from guava-0.14 to hadoop to avoid issues with newer versions of guava in applications. Contributed by Christopher Tubbs. 
(cherry picked from commit 6caa8100d5)
 2014-11-09 05:25:01 -08:00
cnauroth 539ef7fc13 HDFS-7382. DataNode in secure mode may throw NullPointerException if client connects before DataNode registers itself with NameNode. Contributed by Chris Nauroth. 
(cherry picked from commit 9ba8d8c7eb)
(cherry picked from commit 98b81c6624)
 2014-11-08 22:31:22 -08:00
Haohui Mai 9fe9ee1c16 HDFS-7379. TestBalancer#testBalancerWithRamDisk creates test files incorrectly. Contributed by Xiaoyu Yao. 2014-11-07 13:39:33 -08:00
Jing Zhao 92337e1dce HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. 
(cherry picked from commit 75a9c894b3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-06 23:12:08 -08:00
Andrew Wang a2b46a7340 HDFS-7221. TestDNFencingWithReplication fails consistently. Contributed by Charles Lamb. 
(cherry picked from commit ac56b0637e)
(cherry picked from commit 2d0996ae6d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-06 22:49:28 -08:00
Jing Zhao a9b0436132 HDFS-7364. Balancer always shows zero Bytes Already Moved. Contributed by Tsz Wo Nicholas Sze. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
 2014-11-06 17:56:15 -08:00
Haohui Mai 83ed42fa6b HDFS-7367. HDFS short-circuit read cannot negotiate shared memory slot and file descriptors when SASL is enabled on DataTransferProtocol. Contributed by Chris Nauroth. 2014-11-05 20:34:25 -08:00
cnauroth e041de4492 HDFS-7359. NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address. Contributed by Chris Nauroth. 
(cherry picked from commit ba1d4ad25b)
(cherry picked from commit f172a08238)
 2014-11-05 15:29:40 -08:00
Colin Patrick Mccabe 8702cc2360 HDFS-7199. DFSOutputStream should not silently drop data if DataStreamer crashes with an unchecked exception (rushabhs via cmccabe) 
(cherry picked from commit 56257fab1d)
(cherry picked from commit 6f7ce842c0)
 2014-11-05 10:54:56 -08:00
yliu 3faabf2052 HDFS-7218. FSNamesystem ACL operations should write to audit log on failure. (clamb via yliu) 2014-11-05 15:55:56 +08:00
Karthik Kambatla d76756dfe1 Adding release 2.5.2 to CHANGES.txt 
(cherry picked from commit 4de56d27fb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-04 16:52:56 -08:00
Jing Zhao 1ca1a0358e HDFS-7233. NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException. Contributed by Rushabh S Shah. 2014-11-04 16:05:39 -08:00
Haohui Mai 0fde606e3f HDFS-7355. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. Contributed by Chris Nauroth. 2014-11-04 15:19:18 -08:00
Haohui Mai 1b54fa6e27 HDFS-7334. Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures. Contributed by Charles Lamb. 2014-11-04 14:35:22 -08:00
Jing Zhao f354b0fe88 HDFS-7340. Make rollingUpgrade start/finalize idempotent. Contributed by Jing Zhao. 2014-11-04 10:17:55 -08:00
cnauroth ff361e7c31 HDFS-7328. TestTraceAdmin assumes Unix line endings. Contributed by Chris Nauroth. 
(cherry picked from commit 2bb327eb93)
(cherry picked from commit 5720cc9d2d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTraceAdmin.java
 2014-11-03 20:09:58 -08:00
Haohui Mai e94a044d18 HDFS-7147. Update archival storage user documentation. Contributed by Tsz Wo Nicholas Sze. 2014-11-03 15:13:19 -08:00