Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,682 Commits 326 Branches 371 Tags
Commit Graph

2161 Commits

Author SHA1 Message Date
Ming Ma
dda8e0e328 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062f3c4b208724d208e3f501eb196dfab)
(cherry picked from commit 516bbf1c20547dc513126df0d9f0934bb65c10c7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java

(cherry picked from commit fb1bf424bdad20fff7ab390ce75c4bec558e7e6d)
 2015-08-27 17:38:06 -07:00
Arpit Agarwal
aa2303088f HDFS-8486. DN startup may cause severe data loss. (Contributed by Daryn Sharp) 2015-08-06 15:17:45 -07:00
Akira Ajisaka
0daf9dd552 Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0febe546e350a5fd4f37e717f2b8e52d6f.
 2015-04-16 16:40:31 -05:00
Tsz-Wo Nicholas Sze
4f5d330623 HDFS-3443. Fix NPE when namenode transition to active during startup by adding checkNNStartup() in NameNodeRpcServer. Contributed by Vinayakumar B 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
 2015-01-21 11:54:26 -08:00
Colin Patrick Mccabe
51d49ffffd HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume (cmccabe) 
(cherry picked from commit 8fa265a290792ff42635ff9b42416c634f88bdf3)
(cherry picked from commit bcaf15e2fa94db929b8cd11ed7c07085161bf950)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
 2014-12-19 13:25:33 -08:00
arp
91042c950f HDFS-7503. Namenode restart after large deletions can cause slow processReport (Arpit Agarwal) 2014-12-10 23:38:55 -08:00
Colin Patrick Mccabe
d93bfbace4 HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes (Noah Lorang via Colin P. McCabe) 
(cherry picked from commit d8352b9b2b99aa46679c5880a724ba3f0ceb41ff)
(cherry picked from commit a037d6030b5ae9422fdb265f5e4880d515be9e37)
 2014-12-09 10:59:49 -08:00
Colin Patrick Mccabe
575d7bd5b0 HDFS-4882. Prevent the Namenode's LeaseManager from looping forever in checkLeases (Ravi Prakash via Colin P. McCabe) 
(cherry picked from commit daacbc18d739d030822df0b75205eeb067f89850)
 2014-11-24 11:42:46 -08:00
cnauroth
2379932968 HDFS-7425. NameNode block deletion logging uses incorrect appender. Contributed by Chris Nauroth. 
(cherry picked from commit b8c094b0754099f13d668081b4afcf0644e325c3)
(cherry picked from commit 8d8eb8dcec94e92d94eedef883cdece8ba333087)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-21 16:58:06 -08:00
cnauroth
ed2a4db6d4 HDFS-7385. ThreadLocal used in FSEditLog class causes FSImage permission mess up. Contributed by jiangyu. 
(cherry picked from commit b0a41de68c5b08f534ca231293de053c0b0cbd5d)
(cherry picked from commit 4e9001530f7bc414b43723858cf7984f707b30e0)
 2014-11-13 12:37:48 -08:00
Arun C. Murthy
60584c732f Preparing to release hadoop-2.6.0: Set version in branch-2.6 to 2.6.1-SNAPSHOT. 2014-11-09 19:21:11 -08:00
Suresh Srinivas
c98fcd1d3b HDFS-7383. DataNode.requestShortCircuitFdsForRead may throw NullPointerException. Contributed by Tsz Wo Nicholas Sze. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-09 18:18:58 -08:00
Arun C. Murthy
85531834b3 HADOOP-11286. Copied LimitInputStream from guava-0.14 to hadoop to avoid issues with newer versions of guava in applications. Contributed by Christopher Tubbs. 
(cherry picked from commit 6caa8100d5d2547e34356dc279fd5e65b81a925a)
 2014-11-09 05:25:01 -08:00
cnauroth
539ef7fc13 HDFS-7382. DataNode in secure mode may throw NullPointerException if client connects before DataNode registers itself with NameNode. Contributed by Chris Nauroth. 
(cherry picked from commit 9ba8d8c7eb65eeb6fe673f04e493d9eedd95a822)
(cherry picked from commit 98b81c662419d5e136fddee87d18b48b4b969ef2)
 2014-11-08 22:31:22 -08:00
Haohui Mai
9fe9ee1c16 HDFS-7379. TestBalancer#testBalancerWithRamDisk creates test files incorrectly. Contributed by Xiaoyu Yao. 2014-11-07 13:39:33 -08:00
Jing Zhao
92337e1dce HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. 
(cherry picked from commit 75a9c894b3e53d34341e918c70ba7868d074e0fb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-06 23:12:08 -08:00
Andrew Wang
a2b46a7340 HDFS-7221. TestDNFencingWithReplication fails consistently. Contributed by Charles Lamb. 
(cherry picked from commit ac56b0637e55465d3b7f7719c8689bff2a572dc0)
(cherry picked from commit 2d0996ae6d6bdfa01cff790f5ed7198d791de62b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-06 22:49:28 -08:00
Jing Zhao
a9b0436132 HDFS-7364. Balancer always shows zero Bytes Already Moved. Contributed by Tsz Wo Nicholas Sze. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
 2014-11-06 17:56:15 -08:00
Haohui Mai
83ed42fa6b HDFS-7367. HDFS short-circuit read cannot negotiate shared memory slot and file descriptors when SASL is enabled on DataTransferProtocol. Contributed by Chris Nauroth. 2014-11-05 20:34:25 -08:00
cnauroth
e041de4492 HDFS-7359. NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address. Contributed by Chris Nauroth. 
(cherry picked from commit ba1d4ad25b301f7247f3f23df15e7f800e50feed)
(cherry picked from commit f172a0823844cdf7fbb0f821cba23fc67ba4ff43)
 2014-11-05 15:29:40 -08:00
Colin Patrick Mccabe
8702cc2360 HDFS-7199. DFSOutputStream should not silently drop data if DataStreamer crashes with an unchecked exception (rushabhs via cmccabe) 
(cherry picked from commit 56257fab1d5a7f66bebd9149c7df0436c0a57adb)
(cherry picked from commit 6f7ce842c015fedb30ec762caeb37a64e8258c4a)
 2014-11-05 10:54:56 -08:00
yliu
3faabf2052 HDFS-7218. FSNamesystem ACL operations should write to audit log on failure. (clamb via yliu) 2014-11-05 15:55:56 +08:00
Jing Zhao
1ca1a0358e HDFS-7233. NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException. Contributed by Rushabh S Shah. 2014-11-04 16:05:39 -08:00
Haohui Mai
0fde606e3f HDFS-7355. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. Contributed by Chris Nauroth. 2014-11-04 15:19:18 -08:00
Haohui Mai
1b54fa6e27 HDFS-7334. Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures. Contributed by Charles Lamb. 2014-11-04 14:35:22 -08:00
Jing Zhao
f354b0fe88 HDFS-7340. Make rollingUpgrade start/finalize idempotent. Contributed by Jing Zhao. 2014-11-04 10:17:55 -08:00
cnauroth
ff361e7c31 HDFS-7328. TestTraceAdmin assumes Unix line endings. Contributed by Chris Nauroth. 
(cherry picked from commit 2bb327eb939f57626d3dac10f7016ed634375d94)
(cherry picked from commit 5720cc9d2d4076ea67400ee337b0ad8cbad6cbe7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTraceAdmin.java
 2014-11-03 20:09:58 -08:00
Haohui Mai
e94a044d18 HDFS-7147. Update archival storage user documentation. Contributed by Tsz Wo Nicholas Sze. 2014-11-03 15:13:19 -08:00
Tsz-Wo Nicholas Sze
a4dca48676 HDFS-7276. Limit the number of byte arrays used by DFSOutputStream and provide a mechanism for recycling arrays. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2014-11-01 11:29:23 -07:00
Ravi Prakash
b7296ee84f HDFS-7309. XMLUtils.mangleXmlString doesn't seem to handle less than sign. (Colin Patrick McCabe via raviprak) 
(cherry picked from commit c7f81dad30c391822eed7273278cf5885fa59264)
 2014-10-31 11:24:38 -07:00
cnauroth
d3dd1ac1c2 HDFS-7313. Support optional configuration of AES cipher suite on DataTransferProtocol. Contributed by Chris Nauroth. 
(cherry picked from commit 5573b3476a5a6fce0ac99c654a9a9ec90f744a20)
(cherry picked from commit 57fa43ab0acffb328b9153fccc9d384c7c5c566e)
 2014-10-30 22:55:45 -07:00
cnauroth
3283268d8b HDFS-6385. Show when block deletion will start after NameNode startup in WebUI. Contributed by Chris Nauroth. 
(cherry picked from commit c637d6d9d5cb36b8cd5180787c5d25b68f2aa370)
(cherry picked from commit caded2545c3c773cf6e9cd21e49dcb6858a4de04)
 2014-10-30 22:31:26 -07:00
Jing Zhao
b2a7bbf4f6 HDFS-7230. Add rolling downgrade documentation. Contributed by Tsz Wo Nicholas Sze. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-30 10:49:30 -07:00
Jing Zhao
c8dfc9126a HDFS-7305. NPE seen in wbhdfs FS while running SLive. Contributed by Jing Zhao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
 2014-10-29 16:28:50 -07:00
Kihwal Lee
1354ec1c74 HDFS-7300. The getMaxNodesPerRack() method in 
BlockPlacementPolicyDefault is flawed. contributed by Kihwal Lee
(cherry picked from commit 3ae84e1ba8928879b3eda90e79667ba5a45d60f8)
 2014-10-29 17:25:51 -05:00
Jason Lowe
ba86f06cf7 HADOOP-9740. Fix FsShell '-text' command to be able to read Avro files stored in HDFS and other filesystems. Contributed by Allan Yan 
(cherry picked from commit 0c15130db2a127ffe2ddc767dc9519567ae8c756)
 2014-10-29 20:25:25 +00:00
Colin Patrick Mccabe
5a0285fa9a HDFS-7287. The OfflineImageViewer (OIV) can output invalid XML depending on the filename (Ravi Prakash via Colin P. McCabe) 
(cherry picked from commit d33e07dc49e00db138921fb3aa52c4ef00510161)
(cherry picked from commit 112d69408630068f297616f7bac4b106f5eaa402)
 2014-10-29 12:40:18 -07:00
Brandon Li
19ec4212d6 HADOOP-11195. Move Id-Name mapping in NFS to the hadoop-common area for better maintenance. Contributed by Yongjun Zhang 
(cherry picked from commit 72a556d3b0def0ab4e4509528cc513f6df06b084)
(cherry picked from commit 6caf36f9eb979bcf3fefb285def0480e2c45192a)
 2014-10-29 11:10:10 -07:00
cnauroth
905b339a3f HDFS-7291. Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit c6f04f391b54e57467b3ae5d0a95bd96e7feaa82)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit c2790932bd01c0a8a7c32a38ea99ed47c0fe05f8)
 2014-10-28 14:18:24 -07:00
yliu
9465da32e8 HDFS-6606. Optimize HDFS Encrypted Transport performance. (yliu) 2014-10-28 21:19:26 +08:00
Vinayakumar B
3e4e6e3da7 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 
Adding missed test file

(cherry picked from commit cba1f9e3896c0526fa748cd1bb13470d5fae584a)
(cherry picked from commit fea721daf95cfaa487ce9877e5355c6672cb3cd3)
 2014-10-28 10:06:06 +05:30
Vinayakumar B
043364ffa8 HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb) 
(cherry picked from commit ed841dd9a96e54cb84d9cae5507e47ff1c8cdf6e)
(cherry picked from commit 5f58886770958fe9a0d1499f1d866c8534c6b90e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-28 10:04:56 +05:30
cnauroth
c03cca7e23 HDFS-7128. Decommission slows way down when it gets towards the end. Contributed by Ming Ma. 
(cherry picked from commit 80cb472620f0a7889e311ca32212a42b95c6794e)
(cherry picked from commit 7e949744b14e540cc2824eae8e01a3d99714d662)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
 2014-10-27 16:41:55 -07:00
cnauroth
a9f31af29c HDFS-6934. Move checksum computation off the hot path when writing to RAM disk. Contributed by Chris Nauroth. 
(cherry picked from commit 463aec11718e47d4aabb86a7a539cb973460aae6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java

(cherry picked from commit 3d67da502aa21d314d672f8b465d5415d77b5df0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderLocal.java
 2014-10-27 12:01:47 -07:00
Jing Zhao
6724c2f7e4 HDFS-7296. HdfsConstants#MEMORY_STORAGE_POLICY_ID and HdfsConstants#MEMORY_STORAGE_POLICY_ID are missing in branch-2. Contributed by Jing Zhao. 2014-10-27 11:21:48 -07:00
Colin Patrick Mccabe
62f23d0b7d HDFS-6988. Improve HDFS-6581 eviction configuration (Xiaoyu Yao via Colin P. McCabe) 
(cherry picked from commit a52eb4bc5fb21574859f779001ea9d95bf5207fe)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit 2bcda17ee7dd6af06fd880e13dde2517aa724c12)
 2014-10-24 13:22:26 -07:00
Jitendra Pandey
abb28fbb46 HDFS-6904. Added files missing in previous commit. 2014-10-24 11:59:55 -07:00
Jitendra Pandey
327db15362 HDFS-6904. YARN unable to renew delegation token fetched via webhdfs due to incorrect service port. 2014-10-24 11:44:22 -07:00
yliu
82ac5c901e HDFS-7243. HDFS concat operation should not be allowed in Encryption Zone. (clamb via yliu) 2014-10-24 11:35:13 +08:00
Colin Patrick Mccabe
1ec2dbf093 HDFS-7154. Fix returning value of starting reconfiguration task (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 7aab5fa1bd9386b036af45cd8206622a4555d74a)
(cherry picked from commit 8c3b007236e01c7ad4922d1bf611e68d09adcfa8)
 2014-10-21 15:42:39 -07:00