Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,120 Commits 326 Branches 371 Tags
Commit Graph

4297 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
ecaedca5c4
HDFS-10506. OIV's ReverseXML processor cannot reconstruct some snapshot details. Contributed by Akira Ajisaka. 
(cherry picked from commit 14414705f79495eda11e302f38c792128fe0182b)
 2017-04-14 21:11:31 +09:00
Mingliang Liu
9eebbcf459 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-12 13:39:20 -07:00
Kihwal Lee
9dfe0b3515 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 
(cherry picked from commit 8ed230c805625549b1cecc830e909a7027bb4961)
 2017-04-12 13:30:41 -05:00
Chris Nauroth
c4bf504395 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
(cherry picked from commit d5e2bd4096bf2b4d8a5a22042145a08905f93cd4)
 2017-04-11 21:55:28 -07:00
Junping Du
dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Xiaoyu Yao
6367bc1f09 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13be9bad0fbed9421a6539c64bbda4df1)
(cherry picked from commit 0391c920221f4fbfede44fdfd06e678cd1230f1c)
 2017-04-06 16:26:57 -07:00
Andrew Wang
b1dfdea5b4 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:39:56 -07:00
Andrew Wang
ab91ca7338 HADOOP-13688. Stop bundling HTML source code in javadoc JARs. 
(cherry picked from commit 72a2ae6452e615c66d10829da38737896814e02b)
(cherry picked from commit 832c2f52f8543bb966596d0626af99fe995b6aaf)
 2017-04-06 13:16:15 -07:00
Andrew Wang
d12a0a25b4 HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang. 
(cherry picked from commit e154893d3b04d84579a45f0906fa674bda8b4952)
 2017-04-05 16:36:07 -07:00
Wei-Chiu Chuang
eff4b2fa07 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge. 
(cherry picked from commit c5ca2169151a5eec57152775789b6f53664e102c)
(cherry picked from commit 434403a2a0863e4d572e04e05e85d4f333bd6339)
 2017-04-05 16:35:49 -07:00
Xiao Chen
7ad5b278a7 HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge 
(cherry picked from commit d9aae22fdf2ab22ae8ce4a9d32ac71b3dde084d3)
(cherry picked from commit 24d464a15015a7bf6d02d259568fedab1cb97b84)
 2017-04-05 16:35:44 -07:00
Jing Zhao
e216c15625 HDFS-11395. RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode. Contributed by Nandakumar. 
(cherry picked from commit 55796a0946f80a35055701a34379e374399009c5)
(cherry picked from commit e03d8ff488c5818fbe167fa0143c252d7b6287f0)
 2017-04-05 16:29:04 -07:00
Xiaobing Zhou
b9b6bd2932 HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:34:27 -07:00
Wei-Chiu Chuang
f9d3bb3b83 HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang. 
(cherry picked from commit bc7aff7cec07bbc3fed63a00c8f1584c34670998)
(cherry picked from commit 4f1f16f1a67dcb6db705a73bf7fcf3f0f9d7094c)
 2017-04-03 07:39:41 -07:00
Mingliang Liu
14e5a8ed28 HDFS-11592. Closing a file has a wasteful preconditions in NameNode. Contributed by Eric Badger 
(cherry picked from commit e5f0622a6f40706d360d45200c8f259c79046438)
 2017-03-30 15:47:21 -07:00
Andrew Wang
7b8f87eb78 HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by SammiChen and Kai Zheng. 
(cherry picked from commit 82b4a9c3d058bcf54051d3075ca14f30988f5cb4)
 2017-03-28 21:41:10 -07:00
Masatake Iwasaki
ca3ede5fda HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 2017-03-28 18:15:09 +09:00
Junping Du
a4fcf1c7ca Add HDFS jdiff for 2.8.0 release. 
(cherry picked from commit d174c06b01e1f743d3111b9b760a9824d8106b86)
(cherry picked from commit 2fefa1fa2ec8b5523c81e25b8db4a6e5928e8996)
 2017-03-25 14:34:55 -07:00
John Zhuge
be58711e91 HDFS-11561. HttpFS doc errors. Contributed by Yuanbo Liu. 
(cherry picked from commit ed2d03f2a5318a88d0eced8476127f303eb1d1d9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/markdown/index.md
 2017-03-22 22:42:48 -07:00
Zhe Zhang
8ca9915d0f HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a888347f0fb9c51bb000afc986b64f98)
(cherry picked from commit f98f4bb6ca45e6f807ca747d673690232e39da52)
 2017-03-22 22:32:41 -07:00
Kihwal Lee
66824b39fd HDFS-10601. Improve log message to include hostname when the NameNode is in safemode. Contributed by Kuhu Shukla. 
(cherry picked from commit ba62b50ebacd33b55eafc9db55a2fe5b4c80207a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-03-16 11:07:13 -05:00
Wei-Chiu Chuang
851ba7d9d1 HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy. 
(cherry picked from commit 385d2cb777a0272ac20c62336c944fad295d5d12)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 60be2e5d8a1a6a8921c68f8b0f428b55152d05db)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-03-13 13:43:00 -07:00
Yiqun Lin
1f2da39d05 HDFS-11512. Increase timeout on TestShortCircuitLocalRead#testSkipWithVerifyChecksum. Contributed by Eric Badger. 2017-03-13 18:29:57 +08:00
Andrew Wang
3f23d50c9a HDFS-11498. Make RestCsrfPreventionHandler and WebHdfsHandler compatible with Netty 4.0. 
(cherry picked from commit 5e74196ede9bfc20eb6d6fe3aa6a0e5c47a40fdd)
(cherry picked from commit 6e75c1e2f04be03410dd942df4dc3e94bf7b4b24)
 2017-03-06 15:04:34 -08:00
Xiao Chen
83f1e3b10a Revert "HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang." 
This reverts commit f70b9d92413940739ed6f6cd468e6d09702939d1.

(cherry picked from commit 67d2875f8fe83cdeb621856031eb0892a65ec946)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2017-03-06 14:59:07 -08:00
Brahma Reddy Battula
31bec2c16b HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula. 
(cherry picked from commit dcd03df9f9e0080d7e179060ffc8148336c31b3e)
 2017-03-01 10:49:36 +05:30
Zhe Zhang
265ddb20c3 HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang. 
(cherry picked from commit d269b488a71a158d3ddcbdea96992abe29725c69)
(cherry picked from commit 5fb785e154b429beb0b2e392c39be91dae65bb5c)
 2017-02-28 09:04:07 -08:00
Wei-Chiu Chuang
295e8685dd HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. 
(cherry picked from commit f3cdf29af4c67a1963f51f02bf88075bf6dce679)
 2017-02-25 21:15:07 -08:00
Eric Payne
10d54eb978 HDFS-11404. Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc. Contributed by Eric Badger 
(cherry picked from commit a07ddef10115fd0082832f1c338b2484507a8f49)
 2017-02-21 12:13:58 -06:00
Vinayakumar B
2b3e8b75c4 HDFS-11177. 'storagepolicies -getStoragePolicy' command should accept URI based path. (Contributed by Surendra Singh Lilhore) 
(cherry picked from commit 480405063063f564ae0cdb34e0757ac3990569aa)
(cherry picked from commit 00ca9f1792b8d2d572559b656ffd8ad7453ef319)
 2017-02-21 18:17:38 +05:30
Wei-Chiu Chuang
4c47cb68e8 HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 859bd159ae554174200334b5eb1d7e8dbef958ad)
(cherry picked from commit a8531d5d52fb219c2adb5b6025cfc3fad264e8aa)
 2017-02-15 03:02:41 -08:00
Yongjun Zhang
5bb1350922 HDFS-11391. Numeric usernames do no work with WebHDFS FS write access. (Pierre Villard via Yongjun Zhang) 
(cherry picked from commit 8e53f2b9b08560bf4f8e81e697063277dbdc68f9)
 2017-02-14 14:14:58 -08:00
Xiao Chen
8f9aae0193 HDFS-11390. Add process name to httpfs process. Contributed by Weiwei Yang. 
(cherry picked from commit 92dbdbc490a527054388dc2fc6751aa8268856ef)
 2017-02-14 09:47:41 -08:00
Wei-Chiu Chuang
eca70e4bff HDFS-11084. Add a regression test for sticky bit support of OIV ReverseXML processor. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 89cadb42111e4ffbd3f4bde8250013bba23eb51e)
(cherry picked from commit 8e291e1015c4694f7b11cd97a6308d0d92a884cb)
 2017-02-14 09:09:01 -08:00
Wei-Chiu Chuang
2463923492 HDFS-11306. Print remaining edit logs from buffer if edit log can't be rolled. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 1cde954a4fe7760a09b680413ad763cbe4a8feb3)
(cherry picked from commit 914eeb997b798f526eb79b4ce475ba8ced2be55f)
 2017-02-13 10:18:04 -08:00
Kihwal Lee
22f8b66137 HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp. 
(cherry picked from commit 33c62d2d19cd80b9c0cb9f46a635f37080dbb27c)
 2017-02-10 12:56:44 -06:00
Eric Payne
4e423edd95 HDFS-9300. TestDirectoryScanner.testThrottle() is still a little flakey. Contributed by Daniel Templeton. 
(cherry picked from commit 3c0adac88c3cddb821a732aea02bc068cafd7ea8)
 2017-02-02 15:34:59 -05:00
Anu Engineer
4f135647fd HDFS-9911. TestDataNodeLifeline Fails intermittently. Contributed by Yiqun Lin 
(cherry picked from commit a95639068c99ebcaefe8b6c4268449d12a6577d6)
 2017-01-27 21:09:03 +00:00
Zhe Zhang
1a6b6e249f HDFS-10534. NameNode WebUI should display DataNode usage histogram. Contributed by Kai Sasaki. 
(cherry picked from commit 18e1d6820926646999e7ec248c504b4145cf1a76)
(cherry picked from commit 558fee285a00ba39f4d2bf3d81d2eab3c346bd57)
 2017-01-25 10:53:06 -08:00
Akira Ajisaka
71cbc4f6b5 HDFS-11290. TestFSNameSystemMBean should wait until JMX cache is cleared. Contributed by Erik Krogen. 
(cherry picked from commit b1a9ec856b572894e769f052aea2340fc3f23c78)
(cherry picked from commit db47bd15dd80a4a5727607832c16635cb0ec7726)
 2017-01-24 14:10:54 +09:00
Vinitha Reddy Gankidi
e7b4f88acd HDFS-10733. NameNode terminated after full GC thinking QJM is unresponsive. Contributed by Vinitha Gankidi. 2017-01-18 12:57:09 -08:00
Andrew Wang
b7b8671c0e HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory. 
(cherry picked from commit d51f8ba80816c20cd97ba5aa839a84b0bb6a7291)
(cherry picked from commit b089e122b1b01c3d877566d1f5ede5e338170432)
 2017-01-11 15:13:43 -08:00
Junping Du
7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Yongjun Zhang
f5e837e300 HDFS-11292. log lastWrittenTxId etc info in logSyncAll. Contributed by Yongjun Zhang. 
(cherry picked from commit 603cbcd513a74c29e0e4ec9dc181ff08887d64a4)
 2017-01-09 16:25:18 -08:00
Chris Nauroth
15e89634a5 HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 4c8f9e130230457fc897ed7a2a09e14d078be90a)
(cherry picked from commit 9c61858ee754b167a90533375e85ca5bde3cdce1)
 2017-01-05 15:05:12 -08:00
Wei-Chiu Chuang
162c6cc9f4 HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang. 
(cherry picked from commit 0cb99db9d91d113e7fbe229f90a61a33433cecb9)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockScanner.java
 2017-01-05 13:29:43 -08:00
Haohui Mai
e690cd5011 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2017-01-04 21:15:01 -08:00
Brahma Reddy Battula
7392a5a72c Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao." 
This reverts commit 0ff9efcf37e7e86c6b7e187df8e676c1e93fe0e7.
 2017-01-03 19:30:45 +05:30
Haohui Mai
0ff9efcf37 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2016-12-31 01:07:41 -08:00
Daniel Templeton
747be79b22 HDFS-11278. Add missing @Test annotation for TestSafeMode.testSafeModeUtils() (Contributed by Lukas Majercak via Daniel Templeton) 
(cherry picked from commit ce3613c9656e57efdc73edd1787bd9622d1b716e)
(cherry picked from commit af266c8c85971da18c2675c5aa638907b5016fc5)
 2016-12-28 20:11:56 -08:00