Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,799 Commits 326 Branches 371 Tags
Commit Graph

3756 Commits

Author SHA1 Message Date
Vinayakumar B
8ca8d218a4 HDFS-7452. skip StandbyException log for getCorruptFiles() (Contributed by Bramha Reddy Battula) 
(cherry picked from commit f313516731d787f6be64c9406ca83d941d47ee99)
(cherry picked from commit 3e76768f216772dbb7e1ec2e9055eed57f45fee9)
 2016-02-22 09:22:45 +05:30
Arpit Agarwal
4823fbb490 HDFS-9839. Reduce verbosity of processReport logging. (Contributed by Arpit Agarwal) 
This closes #78
 2016-02-20 23:20:28 -08:00
Yongjun Zhang
5a5f9ca1f0 HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit ba1c9d484a8d6a18b4b7365c49d09635d24e579a)
 2016-02-19 10:29:46 -08:00
Tsz-Wo Nicholas Sze
b30d71d450 Move HDFS-8923 to Release 2.8.0 in CHANGES.txt. 2016-02-18 18:44:23 -08:00
Tsz-Wo Nicholas Sze
11c6930a17 HDFS-8923. Add -source flag to balancer usage message. Contributed by Chris Trezzo 2016-02-18 18:39:45 -08:00
Tsz-Wo Nicholas Sze
207ed9601f HADOOP-10865. Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases. 2016-02-18 11:59:58 -08:00
Kihwal Lee
bb8079e553 HDFS-4946. Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable. Contributed by James Kinley and Nathan Roberts. 
(cherry picked from commit ba6d5ed955aa34cdcb7aaa4b1025da6ee5e91355)
 2016-02-18 13:28:17 -06:00
cnauroth
e5c8a34446 HDFS-9711. Integrate CSRF prevention filter in WebHDFS. Contributed by Chris Nauroth. 
(cherry picked from commit 5d1889a66d91608d34ca9411fb6e9161e637e9d3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

(cherry picked from commit 779a69e28dfb6e520e7ebe08cfa76f2b03a0cee7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-02-18 10:16:06 -08:00
Akira Ajisaka
a06036bb1e HDFS-6832. Fix the usage of 'hdfs namenode' command. Contributed by Manjunath Ballur. 
(cherry picked from commit e78d8e66f748c68d2d88ee748facbfa35817d5e3)
(cherry picked from commit c183f235f1ae94d5b890bf010f7c9b7fa8cf92d2)
 2016-02-19 01:35:56 +09:00
Xiaoyu Yao
5ff2012f64 HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou. 
(cherry picked from commit fe5624b85d71720ae9da90a01cad9a3d1ea41160)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java

(cherry picked from commit ad4fcd1f9f84cfef8dc8bb48ef7fbdd5c7c20c6a)
 2016-02-17 16:05:24 -08:00
Zhe Zhang
64d39b83cc HDFS-9799. Reimplement getCurrentTrashDir to remove incompatibility. (zhz) 
Change-Id: I7834bcebffed38cb384db5395ddb8b6dd9e79a0b
 2016-02-17 13:32:18 -08:00
Haohui Mai
b318429ac9 HDFS-9815. Move o.a.h.fs.Hdfs to hadoop-hdfs-client. Contributed by Vinayakumar B. 2016-02-17 10:59:50 -08:00
Junping Du
c7be3deff7 HDFS-9456. BlockPlacementPolicyWithNodeGroup should override verifyBlockPlacement(). Contributed by Xiaobing Zhou. 
(cherry picked from commit 77ba5add0d9cb10d45ca9122bca48baa7c8fb3b8)
(cherry picked from commit 6874a142d8a9be25801636db7ade3d3216fd6df8)
 2016-02-16 19:09:02 -08:00
Akira Ajisaka
19c149cdb9 HDFS-9765. TestBlockScanner#testVolumeIteratorWithCaching fails intermittently. (aajisaka) 
(cherry picked from commit 8ed07bd3e6b6a8af3b82e9a085c74be29f8dfd87)
(cherry picked from commit 8dec72bfd73174bdc2dfa8ce845b28af48a6bca4)
 2016-02-15 19:09:51 +09:00
Akira Ajisaka
4d1c23f55c HDFS-9773. Remove dead code related to SimulatedFSDataset in tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b0738ae673e4a95060b33498c049cd7790d6e371)
(cherry picked from commit 5e78264cde3e141ed9809b9d2d4fc2e3f99b1788)
 2016-02-15 15:35:35 +09:00
Junping Du
aed60ae0bf Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48944c38271399a6e452aa2c2a9f5b80)
(cherry picked from commit a28be9edc72cc4fe42fab107c17bcac6789af4e5)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-13 01:48:50 -08:00
Arpit Agarwal
8c1cf718aa HDFS-9801. ReconfigurableBase should update the cached configuration. (Arpit Agarwal) 2016-02-12 12:41:20 -08:00
Arun Suresh
4d7da8907f HDFS-9797. Log Standby exceptions thrown by RequestHedgingProxyProvider at DEBUG Level (Inigo Goiri via asuresh) 
(cherry picked from commit 9fdfb546fb67526ba261da5cbd005f33e0f1d9e1)
 2016-02-12 11:21:16 -08:00
Zhe Zhang
f72cfc416a HDFS-9644. Update encryption documentation to reflect nested EZs. (zhz) 
Change-Id: I84ae96f4711a16ded46d501c51ca322bfca5151d
 2016-02-12 10:49:41 -08:00
Masatake Iwasaki
2dd80938a2 HDFS-9700. DFSClient and DFSOutputStream should set TCP_NODELAY on sockets for DataTransferProtocol (Gary Helmling via iwasakims) 
(cherry picked from commit 372d1302c63c6f49f99be5766c5da9647ebd9ca6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java

(cherry picked from commit f71fa500f9a43d928fbd27c873544898bff50dd4)
 2016-02-13 03:34:07 +09:00
Akira Ajisaka
df23f9c49d HDFS-9768. Reuse ObjectMapper instance in HDFS to improve the performance. Contributed by Lin Yiqun. 
(cherry picked from commit e6a7044b8530afded8f8e86ff309dd0e4d39238a)
(cherry picked from commit a714d04f5d8eaba34a24ee9454ceaec68ab00a15)
 2016-02-13 02:04:22 +09:00
Arpit Agarwal
58d4c045e6 HDFS-9790. HDFS Balancer should exit with a proper message if upgrade is not finalized (Contributed by Xiaobing Zhou) 2016-02-11 20:40:49 -08:00
Andrew Wang
47746f65b0 HDFS-9788. Incompatible tag renumbering in HeartbeatResponseProto. 
(cherry picked from commit aeb13ef2ab657a1c2f9d9ed5f80783f788c4953a)
(cherry picked from commit d48957075591e0c308da79eaa5412ca8f6f73c8a)
 2016-02-10 15:26:34 -08:00
Akira Ajisaka
811ab1bf94 HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 802a7edfaed778daaa471ee96466c6d69b35934c)
(cherry picked from commit a60a0a7128d23523cde2a80bc8576ae194162fc5)
 2016-02-10 23:50:16 +09:00
Akira Ajisaka
321da0604a Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta." 
This reverts commit 3f8a30c956d464b86f91be3f0136ed7e15604f02.
 2016-02-10 23:45:44 +09:00
Uma Maheswara Rao G
de24588ca2 HDFS-9779 . TestReplicationPolicyWithNodeGroup NODE variable picks wrong rack value. Contributed by Kuhu Shukla. 
(cherry picked from commit a7fce9ab415fcc834ca1d91f913157ffd103fa5f)
 2016-02-09 15:18:07 -08:00
Allen Wittenauer
dc325ee550 HDFS-9760. WebHDFS AuthFilter cannot be configured with custom AltKerberos auth handler (Ryan Sasson via aw) 2016-02-09 14:16:02 -08:00
Akira Ajisaka
e4769775b0 HDFS-9784. Example usage is not correct in Transparent Encryption document. Contributed by Ohnishi Takashi. 
(cherry picked from commit 60d2011b7c0fe55b8bc44a141660d3e2df37a68d)
(cherry picked from commit cb53dfcc95ca9f6c45482600b52758bc096810b9)
 2016-02-10 03:30:56 +09:00
zhezhang
3257ce73bd HDFS-9244. Support nested encryption zones. 
Change-Id: I43a13035a8b27956e90967ee82058efb647c3415
 2016-02-08 16:32:11 -08:00
Uma Maheswara Rao G
3399119e72 HDFS-9777. Fix typos in DFSAdmin command line and documentation.(Wei-Chiu Chuang via umamahesh) 
(cherry picked from commit bffaa38a91b26920206a3350bf1cd60ec950aa74)
 2016-02-08 14:12:52 -08:00
Kihwal Lee
aae0f183af HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. Contributed by Walter Su. 
(cherry picked from commit 193d27de0a5d23a61cabd41162ebc3292d8526d1)
 2016-02-08 12:17:39 -06:00
Uma Maheswara Rao G
5690e11206 HDFS-9713. DataXceiver#copyBlock should return if block is pinned. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit aea31eee78a287f251447686ba3f5bbcfe9c60a3)
 2016-02-07 20:59:35 -08:00
Akira Ajisaka
3f8a30c956 HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit fe124da5ffc16e4795c3dd5542accd58361e1b08)
(cherry picked from commit e4c01b8b1ce511de24db00afacef1e737845a4f7)
 2016-02-06 19:33:11 +09:00
cnauroth
bad8006da1 HDFS-9761. Rebalancer sleeps too long between iterations. Contributed by Mingliang Liu. 
(cherry picked from commit c6497949e866594050153b953a85c0a1db59d2f8)
(cherry picked from commit a7232f5e1d20faf210f3718c551cbae4edb5933c)
 2016-02-05 15:08:07 -08:00
Tsz-Wo Nicholas Sze
494b6c7c4b HDFS-9726. Refactor IBR code to a new class. 2016-02-05 23:22:17 +08:00
Haohui Mai
fdb89ac0ae HDFS-9724. Degraded performance in WebHDFS listing as it does not reuse ObjectMapper. Contributed by Akira Ajisaka. 2016-02-04 13:41:14 -08:00
Kihwal Lee
6090211872 HDFS-9730. Storage ID update does not happen when there is a layout change. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 496f33de0ce80dc455cfd51f19612da6f9b914f9)
 2016-02-04 13:22:05 -06:00
Vinayakumar B
adadc220fa HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B) 
(cherry picked from commit ddcd6954d03d3caadc8b69f7b886a3a17c3c9d0c)
(cherry picked from commit 8cdfdbf815bf89eee3184ec7cd00e1924c1ea316)
 2016-02-04 13:19:53 +05:30
Konstantin V Shvachko
3f178be0bd HDFS-9503. Use generic option -fs for NNThroughputBenchmark instead of -namenode. Contributed by Mingliang Liu. 2016-02-03 14:45:09 -08:00
Masatake Iwasaki
5fd4bd1d8c HDFS-9601. NNThroughputBenchmark.BlockReportStats should handle NotReplicatedYetException on adding block (iwasakims) 2016-02-03 14:29:06 -08:00
Jing Zhao
a79f570b6c HDFS-9748. Avoid duplication in pendingReplications when addExpectedReplicasToPending is called twice. Contributed by Walter Su. 
(cherry picked from commit 7badf156049b78cabf8537fff9846a0f9924a090)
(cherry picked from commit 459c31de4a0db322f7a27a9b80dc3b2eecfa470d)
 2016-02-03 13:33:59 -08:00
Vinayakumar B
c2ec241989 HDFS-9740. Use a reasonable limit in DFSTestUtil.waitForMetric() (Contributed by Chang Li) 
(cherry picked from commit eb2fb943fd97c191fd7c4f5333087a28ee5c87d8)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

(cherry picked from commit 6941e2ccbf6f4c7c8663a68c7a7c9ed59bb49fbf)
 2016-02-03 11:21:42 +05:30
Vinayakumar B
e045856eda HDFS-9739. DatanodeStorage.isValidStorageId() is broken (Contributed by Mingliang Liu) 
(cherry picked from commit d6b1acb940180befeb4c855d0e4a339dbc035e7d)
(cherry picked from commit e8a9a69a2756f998f08c7b8452b84fd309921443)
 2016-02-03 10:34:35 +05:30
Junping Du
847158811b Adding release 2.6.5 to CHANGES.txt 2016-02-02 18:36:39 -08:00
Vinayakumar B
bf2936fd38 HDFS-9715. Check storage ID uniqueness on datanode startup (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 04375756a5ed6e907ee7548469c2c508aebbafb7)
(cherry picked from commit 8577c0b6a2bc7e4966866eac2bd1f2a112d0a7c1)
 2016-02-03 07:42:18 +05:30
Tsz-Wo Nicholas Sze
0ad3c51dfb HDFS-8999. Allow a file to be closed with COMMITTED but not yet COMPLETE blocks. 
(cherry picked from commit b10d8ced21a860390c46e7729a02b81d9f7b88e6)
 2016-02-03 07:41:06 +05:30
Colin Patrick Mccabe
c4c94e1cf2 HDFS-9669. TcpPeerServer should respect ipc.server.listen.queue.size (Elliot Clark via cmccabe) 
(cherry picked from commit 2da03b48eba53d4dec2a77209ad9835d808171d1)
(cherry picked from commit 60d3a3c30b05fdeec5cc369a698297ee309d620e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
 2016-02-02 11:05:41 -08:00
Lei Xu
dfbd0d4df4 HDFS-9721. Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference. (Xiao Chen via lei) 
(cherry picked from commit 9d494f0c0eaa05417f3a3e88487d878d1731da36)
 2016-02-02 10:43:06 -08:00
Vinayakumar B
94950c15c2 HDFS-9718. HAUtil#getConfForOtherNodes should unset independent generic keys before initialize (Contributed by DENG FEI) 
(cherry picked from commit 4d7055e0c8d1936d6c87cec14dbbfdff77b837f3)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HAUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHAConfiguration.java

(cherry picked from commit 80dd039b6e78d284d09522e61d7ea1fee9425345)
 2016-02-02 12:57:42 +05:30
Yongjun Zhang
d7b01ebcbe HDFS-9406. FSImage may get corrupted after deleting snapshot. (Contributed by Jing Zhao, Stanislav Antic, Vinayakumar B, Yongjun Zhang) 
(cherry picked from commit 34ab50ea92370cc7440a8f7649286b148c2fde65)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java

(cherry picked from commit fc8d9cc758d4119064d67008432f63a590b5f67a)
 2016-02-01 14:22:35 -08:00