Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,748 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3296 Commits

Author SHA1 Message Date
Andrew Wang 086eb49424 HDFS-8573. Move creation of restartMeta file logic from BlockReceiver to ReplicaInPipeline. Contributed by Eddy Xu. 
(cherry picked from commit b258b344bb)
 2015-06-11 10:12:40 -07:00
Andrew Wang 9523e2c098 HDFS-8549. Abort the balancer if an upgrade is in progress. 
(cherry picked from commit a7a7768341)
 2015-06-10 13:43:00 -07:00
Arpit Agarwal 2d67dff15f HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:46 -07:00
Xiaoyu Yao d8d9041187 HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing. Contributed by Rakesh R. 
(cherry picked from commit 3c2397cb04)
 2015-06-09 12:21:53 -07:00
Xiaoyu Yao 116a72096d HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc. Contributed by Brahma Reddy Battula 
(cherry picked from commit 927577c87c)
 2015-06-08 22:03:07 -07:00
cnauroth cc960734e4 HDFS-8553. Document hdfs class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d2832b3d42)
 2015-06-08 16:15:53 -07:00
cnauroth 8ee50d8ca7 HDFS-8554. TestDatanodeLayoutUpgrade fails on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit 0e80d51983)
 2015-06-08 08:39:14 -07:00
cnauroth 953c4a049a HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu Engineer. 
(cherry picked from commit a6cb4894b4)
 2015-06-07 13:02:01 -07:00
Xiaoyu Yao c8dc113943 HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b61b489995)
 2015-06-06 18:48:52 -07:00
cnauroth f99744b7a0 HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. 
(cherry picked from commit 71de367c5e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
 2015-06-06 09:45:46 -07:00
cnauroth 1d1841d40f HADOOP-12055. Deprecate usage of NativeIO#link. Contributed by Andrew Wang. 
(cherry picked from commit e46cb80002)
 2015-06-06 09:17:11 -07:00
Xiaoyu Yao 1a6cdf97bb HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:33:06 -07:00
Andrew Wang 320e4cc0e7 HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu. 
(cherry picked from commit b2540f486e)
 2015-06-04 15:35:12 -07:00
Andrew Wang ccc834395b HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R. 
(cherry picked from commit 5149dc7b97)
 2015-06-04 11:09:28 -07:00
Kihwal Lee f0f277ddd0 HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. 
(cherry picked from commit ade6d9a61e)
 2015-06-04 12:52:49 -05:00
Vinayakumar B 539ff92c2a HDFS-3716. Purger should remove stale fsimage ckpt files (Contributed by J.Andreina) 
(cherry picked from commit c59e745630)
 2015-06-03 15:31:01 +05:30
Vinayakumar B 0ed9c2d8fe HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 2015-06-03 15:15:44 +05:30
Vinayakumar B 4bb725b4dd HDFS-8470. fsimage loading progress should update inode, delegation token and cache pool count. (Contributed by surendra singh lilhore) 
(cherry picked from commit e965dcec37)
 2015-06-03 14:25:12 +05:30
Vinayakumar B 066e45bcb6 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd9)
 2015-06-03 12:12:22 +05:30
Colin Patrick Mccabe 12497b0543 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8)
 2015-06-02 20:06:53 -07:00
Andrew Wang a80a68c298 HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant. (wang) 
(cherry picked from commit c1d50a91f7)
 2015-06-02 15:48:31 -07:00
Andrew Wang 379ece15fe HDFS-8386. Improve synchronization of 'streamer' reference in DFSOutputStream. Contributed by Rakesh R. 
(cherry picked from commit efc510a570)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-06-02 15:42:27 -07:00
Colin Patrick Mccabe 17e369511d HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c6425)
 2015-06-02 11:42:08 -07:00
Vinayakumar B d1da842f86 HDFS-8256. '-storagepolicies , -blockId ,-replicaDetails ' options are missed out in usage and from documentation (Contributed by J.Andreina) 
(cherry picked from commit a2bd6217eb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2015-06-02 16:25:53 +05:30
Jing Zhao 640ea994ed HDFS-8489. Subclass BlockInfo to represent contiguous blocks. Contributed by Zhe Zhang. 
(cherry picked from commit cdc13efb1a)
 2015-06-01 11:45:04 -07:00
Tsuyoshi Ozawa b91cadcafd HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS. Contributed by Archana T. 
(cherry picked from commit c2356cb0bc)
 2015-06-01 00:42:11 +09:00
Andrew Wang bdaa0fed13 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 
(cherry picked from commit 7673d4f205)

 Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
	hadoop-mapreduce-project/pom.xml
 2015-05-29 14:55:52 -07:00
Jing Zhao 17fb442a4c HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a)
 2015-05-29 11:11:01 -07:00
Vinayakumar B 983c1d3969 HDFS-7401. Add block info to DFSInputStream' WARN message when it adds node to deadNodes (Contributed by Arshad Mohammad) 
(cherry picked from commit b75df697e0)
 2015-05-29 16:36:39 +05:30
Akira Ajisaka e7b81710dd HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. 
(cherry picked from commit d725dd8af6)
 2015-05-29 11:36:59 +09:00
Colin Patrick Mccabe 4b108a441d HADOOP-11894. Bump the version of Apache HTrace to 3.2.0-incubating (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit a1140959da)
 2015-05-28 12:03:07 -07:00
Colin Patrick Mccabe b9752f24a6 HDFS-8407. libhdfs hdfsListDirectory must set errno to 0 on success (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit d2d95bfe88)
 2015-05-28 11:38:44 -07:00
cnauroth 25db341278 HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. 
(cherry picked from commit 50eeea1300)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd
 2015-05-27 22:56:04 -07:00
Andrew Wang a634809895 HDFS-8482. Rename BlockInfoContiguous to BlockInfo. Contributed by Zhe Zhang. 
(cherry picked from commit ff83459f192f7cc4ce864f0b1f01e1e08243be83)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
 2015-05-27 15:42:12 -07:00
Haohui Mai 3f321be24d Addendum of HDFS-8135. 2015-05-27 13:14:51 -07:00
Haohui Mai cc348fead7 Revert HDFS-8135 in branch-2. 2015-05-27 11:06:17 -07:00
Akira Ajisaka 9670762a82 HADOOP-11242. Record the time of calling in tracing span of IPC server. Contributed by Mastake Iwasaki. 
(cherry picked from commit bb1816328a)
 2015-05-27 18:05:08 +09:00
Colin Patrick Mccabe 3dec58dd78 HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 7dba7005b7)
 2015-05-26 12:40:41 -07:00
Haohui Mai f70b9d9241 HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang. 2015-05-24 22:30:50 -07:00
Vinayakumar B 19895a41ad HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 
(cherry picked from commit 55ed6558ba)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
 2015-05-22 16:46:50 +05:30
Vinayakumar B 658f5cf985 HDFS-8268. Port conflict log for data node server is not sufficient (Contributed by Mohammad Shahid Khan) 
(cherry picked from commit 0c6638c2ea)
 2015-05-22 16:15:39 +05:30
Arpit Agarwal a073e5bd31 HDFS-8454. Remove unnecessary throttling in TestDatanodeDeath. (Arpit Agarwal) 2015-05-21 20:44:07 -07:00
Xiaoyu Yao 61d7ccb27e HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f2)
 2015-05-21 11:59:11 -07:00
Haohui Mai 0b909d028f HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp. Contributed by Haohui Mai. 2015-05-21 08:06:13 -07:00
Akira Ajisaka b578f76bed HADOOP-10366. Add whitespaces between classes for values in core-default.xml to fit better in browser. Contributed by kanaka kumar avvaru. 
(cherry picked from commit 0e4f1081c7)
 2015-05-21 17:53:18 +09:00
Akira Ajisaka 185cddfd6a HDFS-4383. Document the lease limits. Contributed by Arshad Mohammad. 
(cherry picked from commit dc8434ab2b)
 2015-05-21 17:31:47 +09:00
Colin Patrick Mccabe 9534e533dc HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 470c87dbc6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
 2015-05-19 11:15:04 -07:00
Kihwal Lee 536b9ee6d6 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 
(cherry picked from commit 8860e352c3)
 2015-05-19 13:06:48 -05:00
Kihwal Lee 9e656bfa94 HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui. 
(cherry picked from commit de30d66b26)
 2015-05-19 08:06:07 -05:00
Vinayakumar B 5bbf157cb2 HADOOP-11103. Clean up RemoteException (Contributed by Sean Busbey) 
(cherry picked from commit d4a2830b63)
 2015-05-19 14:41:34 +05:30
Allen Wittenauer 3d0e2e54a1 HADOOP-8934. Shell command ls should include sort options (Jonathan Allen via aw) 
(cherry picked from commit 30b797ee9d)

Conflicts:
	hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
 2015-05-19 17:38:22 +09:00
Vinayakumar B 6dee42f6db HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions (Contributed by Rakesh R) 
(cherry picked from commit 93972a332a)
 2015-05-19 12:25:00 +05:30
Ravi Prakash e5b805d361 HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak) 
(cherry picked from commit cdfae446ad)
 2015-05-18 12:38:32 -07:00
Tsz-Wo Nicholas Sze 7bd4db968d HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 03:01:07 +08:00
Arpit Agarwal 9bcb740049 HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) 2015-05-18 11:49:01 -07:00
Uma Maheswara Rao G ea875939f1 Revert from branch-2: HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R. 2015-05-18 14:38:30 +05:30
Arpit Agarwal 151f6d6dcc HDFS-8157. Writes to RAM DISK reserve locked memory for block files. (Arpit Agarwal) 2015-05-16 09:05:50 -07:00
Haohui Mai 46d62e7016 HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal. 2015-05-15 19:12:46 -07:00
Haohui Mai 1796d94d62 HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai. 2015-05-15 19:10:05 -07:00
Jing Zhao d6aa65d037 HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 8f37873342)
 2015-05-15 16:32:25 -07:00
Vinayakumar B d9455c790f HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) 
(cherry picked from commit 7f2e89fa7082840bfa3e8e593c93db050a80d04f)
 2015-05-15 11:05:26 +05:30
Akira Ajisaka 7fb3486eb3 HDFS-8350. Remove old webhdfs.xml and other outdated documentation stuff. Contributed by Brahma Reddy Battula. 2015-05-15 13:57:58 +09:00
Akira Ajisaka 2e5b7f24ab HDFS-8371. Fix test failure in TestHdfsConfigFields for spanreceiver properties. Contributed by Ray Chiang. 
(cherry picked from commit cbc01ed08e)
 2015-05-15 12:15:26 +09:00
Vinayakumar B 4bbcffa510 HDFS-8150. Make getFileChecksum fail for blocks under construction (Contributed by J.Andreina) 
(cherry picked from commit def9136e02)
 2015-05-14 15:55:20 +05:30
Haohui Mai 4b5915be9f HDFS-7728. Avoid updating quota usage while loading edits. Contributed by Jing Zhao. 2015-05-13 21:50:41 -07:00
Arpit Agarwal 33f6136ee3 HDFS-8243. Files written by TestHostsFiles and TestNameNodeMXBean are causing Release Audit Warnings. (Contributed by Ruth Wisniewski) 2015-05-13 19:46:34 -07:00
Colin Patrick Mccabe 52da75180a HDFS-8380. Always call addStoredBlock on blocks which have been shifted from one storage to another (cmccabe) 
(cherry picked from commit 281d47a969)
 2015-05-13 14:45:37 -07:00
cnauroth ddc4afeff2 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 2015-05-13 12:25:49 -07:00
Kihwal Lee 2e9392c157 HDFS-8358. TestTraceAdmin fails. Contributed by Masatake Iwasaki. 
(cherry picked from commit f9a46a00d2)
 2015-05-13 14:16:37 -05:00
Tsz-Wo Nicholas Sze bdf8f64be8 HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore 2015-05-13 11:58:35 -07:00
Vinayakumar B 2adb1257b1 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 
(cherry picked from commit 065d8f2a34)
 2015-05-13 17:30:40 +05:30
Akira Ajisaka 7596c6e731 HDFS-6184. Capture NN's thread dump when it fails over. Contributed by Ming Ma. 
(cherry picked from commit 2463666ecb)
 2015-05-13 11:37:59 +09:00
yliu c31c6fbda7 HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication. (Contributed by Zhe Zhang) 2015-05-12 21:26:57 +08:00
Vinayakumar B e1b8bb9d84 HDFS-8362. Java Compilation Error in TestHdfsConfigFields.java (Contributed by Arshad Mohammad) 
(cherry picked from commit 987abc99b0)
 2015-05-12 12:10:14 +05:30
Kihwal Lee 50778f9d45 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 
(cherry picked from commit ea11590aad)
 2015-05-11 14:33:45 -05:00
Junping Du a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 
(cherry picked from commit 7e543c27fa)
 2015-05-11 05:28:25 -07:00
Akira Ajisaka 703f1e9d16 HDFS-8351. Remove namenode -finalize option from document. (aajisaka) 
(cherry picked from commit 3fa2efc09f)
 2015-05-11 15:37:12 +09:00
Haohui Mai 1f499a61dc HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter objects. Contributed by Li Lu. 2015-05-09 22:52:07 -07:00
Haohui Mai 300d36cee7 HDFS-8327. Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate(). Contributed by Haohui Mai. 2015-05-08 23:09:57 -07:00
Haohui Mai 51bfb66ec8 Add missing entry in CHANGES.txt for HDFS-6757. 2015-05-08 23:09:55 -07:00
Haohui Mai 73d61f7ae7 HDFS-6757. Simplify lease manager with INodeID. Contributed by Haohui Mai. 2015-05-08 23:04:57 -07:00
Arpit Agarwal 969a8f6c96 HDFS-8097. TestFileTruncate is failing intermittently. (Contributed by Rakesh R) 2015-05-08 16:29:01 -07:00
Kihwal Lee f264a5aeed HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 
(cherry picked from commit 2d4ae3d18b)
 2015-05-08 16:37:26 -05:00
Colin Patrick Mccabe ab80a572be HDFS-8113. Add check for null BlockCollection pointers in BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe) 
(cherry picked from commit f523e963e4)
 2015-05-08 14:00:13 -07:00
Yongjun Zhang fbb2cf54bc HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream. (Esteban Gutierrez via Yongjun Zhang) 
(cherry picked from commit 730f9930a4)
 2015-05-08 01:38:01 -07:00
Kihwal Lee 30cb540b4d HDFS-7433. Optimize performance of DatanodeManager's node map. Contributed by Daryn Sharp. 
(cherry picked from commit 7a7960be41)
 2015-05-08 15:20:14 -05:00
Colin Patrick Mccabe d83ae68bb0 HDFS-8284. Update documentation about how to use HTrace with HDFS (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 8f7c2364d7)
 2015-05-08 12:30:34 -07:00
cnauroth 5c188908fe HDFS-5640. Add snapshot methods to FileContext. Contributed by Rakesh R. 
(cherry picked from commit 26f61d41df)
 2015-05-08 11:21:02 -07:00
Arun Suresh 2124742e07 HDFS-7559. Create unit test to automatically compare HDFS related classes and hdfs-default.xml. (Ray Chiang via asuresh) 
(cherry picked from commit 3cefc02af7)
 2015-05-08 10:31:23 -07:00
Arpit Agarwal 68d85e92bf HDFS-8340. Fix NFS documentation of nfs.wtmax. (Contributed by Ajith S) 2015-05-08 10:15:35 -07:00
Arpit Agarwal a0905ae57a HDFS-8274. NFS configuration nfs.dump.dir not working (Contributed by Ajith S) 2015-05-08 10:04:11 -07:00
Haohui Mai d4d7d7bf22 HDFS-8346. libwebhdfs build fails during link due to unresolved external symbols. Contributed by Chris Nauroth. 2015-05-08 09:58:00 -07:00
Kihwal Lee 802a5775f3 HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6f622672b6)
 2015-05-08 09:32:07 -05:00
Uma Maheswara Rao G faf26f081c HDFS-6285. tidy an error log inside BlockReceiver. Contributed by Liang Xie. 
(cherry picked from commit 7b1ea9c481)
 2015-05-08 17:25:07 +05:30
Vinayakumar B d129bbbb37 HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) 
(cherry picked from commit c648317a68)
 2015-05-08 17:18:59 +05:30
Vinayakumar B a87ab77fec HDFS-6576. Datanode log is generating at root directory in security mode (Contributed by surendra singh lilhore) 2015-05-08 16:37:36 +05:30
Vinayakumar B 3ee1b50dad HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
Added one missed change, missed while resolving conflict
 2015-05-08 15:39:47 +05:30
Vinayakumar B 3a894b4563 HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
(cherry picked from commit e01c1a979c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md
 2015-05-08 15:34:17 +05:30
Vinayakumar B eaa5f3e448 HDFS-8209. Support different number of datanode directories in MiniDFSCluster. (Contributed by surendra singh lilhore) 
(cherry picked from commit 4c6816faf8)
 2015-05-08 15:04:58 +05:30
Tsuyoshi Ozawa 8109f29ee2 HDFS-8207. Improper log message when blockreport interval compared with initial delay. Contributed by Brahma Reddy Battula and Ashish Singhi. 
(cherry picked from commit 888e630990)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-05-08 18:14:08 +09:00
Vinayakumar B 595b92e2c2 HDFS-8175. Provide information on snapshotDiff for supporting the comparison between snapshot and current status (Contributed by J.Andreina) 
(cherry picked from commit 6232235df3)
 2015-05-08 14:38:53 +05:30
Vinayakumar B 10999ffd7a HDFS-8187. Remove usage of '-setStoragePolicy' and '-getStoragePolicy' using dfsadmin cmd (as it is not been supported) (Contributed by J.Andreina) 
(cherry picked from commit e642eff0e0)
 2015-05-08 14:26:58 +05:30
Uma Maheswara Rao G a56f57b421 HDFS-8108. Fsck should provide the info on mandatory option to be used along with -blocks ,-locations and -racks. Contributed by J.Andreina. 
(cherry picked from commit 4b00ae5c11)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2015-05-08 14:23:42 +05:30
Akira Ajisaka 143c9a8ff2 HDFS-8222. Remove usage of "dfsadmin -upgradeProgress" from document which is no longer supported. Contributed by J.Andreina. 
(cherry picked from commit fd605997ee)
 2015-05-08 17:23:25 +09:00
Vinayakumar B 1f78502416 HDFS-7998. HDFS Federation : Command mentioned to add a NN to existing federated cluster is wrong (Contributed by Ajith S) 
(cherry picked from commit 26da2ddf38)
 2015-05-08 13:47:06 +05:30
Uma Maheswara Rao G b7a5ebd566 HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R. Merge from trunk 2015-05-08 13:46:04 +05:30
Vinayakumar B 089d420a82 HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun) 
(cherry picked from commit e5e492a963)
 2015-05-08 11:26:56 +05:30
Uma Maheswara Rao G 7fe1d74005 HDFS-8174. Update replication count to live rep count in fsck report. Contributed by J.Andreina 
(cherry picked from commit 2ea0f2fc93)
 2015-05-08 11:07:58 +05:30
Vinayakumar B d0fff5d328 HDFS-8067. haadmin prints out stale help messages (Contributed by Ajith S) 
(cherry picked from commit 66988476d0)
 2015-05-08 10:47:23 +05:30
Vinayakumar B 00954714a6 HDFS-8257. Namenode rollingUpgrade option is incorrect in document (Contributed by J.Andreina) 
(cherry picked from commit c7c26a1e4a)
 2015-05-08 10:36:09 +05:30
Haohui Mai a3abe8d7e4 HDFS-8037. CheckAccess in WebHDFS silently accepts malformed FsActions parameters. Contributed by Walter Su. 2015-05-07 15:25:36 -07:00
Haohui Mai f6b908ab07 HDFS-8321. CacheDirectives and CachePool operations should throw RetriableException in safemode. Contributed by Haohui Mai. 2015-05-07 15:17:02 -07:00
Tsz-Wo Nicholas Sze 4e1f2eb395 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:37:33 -07:00
Akira Ajisaka ab7dab55ea HDFS-8325. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 449e4426a5)
 2015-05-07 15:15:12 +09:00
cnauroth 276e2d3390 HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6633a8474d)
 2015-05-06 21:11:44 -07:00
Konstantin V Shvachko 193615c4b2 HDFS-2484. checkLease should throw FileNotFoundException when file does not exist. Contributed by Rakesh R. 2015-05-06 16:11:12 -07:00
Colin Patrick Mccabe f79b1f0190 HDFS-7847. Modify NNThroughputBenchmark to be able to operate on a remote NameNode (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit ffce9a3413)
 2015-05-06 10:50:33 -07:00
Xiaoyu Yao bd88215393 HDFS-8310. Fix TestCLI.testAll 'help: help for find' on Windows. (Kiran Kumar M R via Xiaoyu Yao) 
(cherry picked from commit 7a26d174af)
 2015-05-06 10:27:07 -07:00
Haohui Mai a4f868f0b8 HDFS-8314. Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users. Contributed by Li Lu. 2015-05-05 15:56:08 -07:00
Xiaoyu Yao e68e8b3b5c HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao) 
(cherry picked from commit 0100b15501)
 2015-05-05 13:42:22 -07:00
Colin Patrick Mccabe 1f6bcf94cc HDFS-7758. Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 24d3a2d4fd)
 2015-05-05 11:09:28 -07:00
Colin Patrick Mccabe 1f01d8347a HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 
(cherry picked from commit fcd4cb7516)
 2015-05-05 11:06:47 -07:00
Vinayakumar B 01bdfd794c HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit

(cherry picked from commit 318081ccd7)
 2015-05-05 11:06:44 +05:30
Haohui Mai db0bd6dca8 HDFS-8237. Move all protocol classes used by ClientProtocol to hdfs-client. Contributed by Haohui Mai. 2015-05-04 15:02:49 -07:00
Colin Patrick Mccabe 8efcf5bdbd HDFS-7397. Add more detail to the documentation for the conf key "dfs.client.read.shortcircuit.streams.cache.size" (Brahma Reddy Battula via Colin P. McCabe) 
(cherry picked from commit 3fe79e1db8)
 2015-05-04 12:50:59 -07:00
cnauroth a43080a2cf HDFS-8290. WebHDFS calls before namesystem initialization can cause NullPointerException. Contributed by Chris Nauroth. 
(cherry picked from commit c4578760b6)
 2015-05-04 11:35:18 -07:00
Xiaoyu Yao 5b27d9e076 HDFS-8309. Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows. (xyao) 
(cherry picked from commit a319771d1d)
 2015-05-02 22:32:28 -07:00
Haohui Mai 65a19fbc30 HDFS-8249. Separate HdfsConstants into the client and the server side class. Contributed by Haohui Mai. 2015-05-02 10:03:04 -07:00
Jonathan Eagles 9ed6b77da1 HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles) 
(cherry picked from commit 2d7363b273)

Conflicts:
	pom.xml
 2015-05-01 19:04:41 -07:00
Arpit Agarwal 8e3fdd0a40 HDFS-8229. LAZY_PERSIST file gets deleted after NameNode restart. (Contributed by Surendra Singh Lilhore) 2015-05-01 16:42:35 -07:00
Tsz-Wo Nicholas Sze 72751b957c HDFS-8086. Move LeaseRenewer to the hdfs.client.impl package. Contributed by Takanobu 2015-05-01 15:14:01 -07:00
Arun Suresh 8b6fc83ef4 Updating CHANGES.txt - Pulling in HDFS-8091 to branch-2.7 (for 2.7.1) 
(cherry picked from commit 3393461197)
 2015-05-01 11:49:38 -07:00
Colin Patrick Mccabe 9edea9507d HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 
(cherry picked from commit b82567d455)
 2015-05-01 11:47:28 -07:00
Arpit Agarwal aa81b86898 HDFS-8276. LazyPersistFileScrubber should be disabled if scrubber interval configured zero. (Contributed by Surendra Singh Lilhore) 2015-05-01 11:11:56 -07:00
Jing Zhao ec7b392965 HDFS-8300. Fix unit test failures and findbugs warning caused by HDFS-8283. Contributed by Jing Zhao. 
(cherry picked from commit 98a6176628)
 2015-04-30 19:29:11 -07:00
Andrew Wang 34d686e6e9 HDFS-8292. Move conditional in fmt_time from dfs-dust.js to status.html. Contributed by Charles Lamb. 
(cherry picked from commit 87e9978235)
 2015-04-30 18:23:21 -07:00
Haohui Mai 01acf3a3af HDFS-8200. Refactor FSDirStatAndListingOp. Contributed by Haohui Mai. 2015-04-30 13:41:55 -07:00
Akira Ajisaka fdb886d656 HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao. 
(cherry picked from commit de9404f02f)
 2015-04-30 20:13:18 +09:00
Akira Ajisaka 2f2d0aa990 HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. 
(cherry picked from commit fd06186071bb369ee045b57a82419566e87257ec)
 2015-04-30 19:11:46 +09:00
Andrew Wang 9f8412ac6e HDFS-8214. Secondary NN Web UI shows wrong date for Last Checkpoint. Contributed by Charles Lamb. 
(cherry picked from commit aa22450442)
 2015-04-29 17:38:09 -07:00
Haohui Mai 460127e6f2 HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. 2015-04-29 11:12:54 -07:00
Jing Zhao ecdebb7369 HDFS-8283. DataStreamer cleanup and some minor improvement. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 7947e5b53b)
 2015-04-29 10:43:52 -07:00
Haohui Mai fc9e7d7a24 HDFS-8280. Code Cleanup in DFSInputStream. Contributed by Jing Zhao. 2015-04-28 18:12:05 -07:00
Haohui Mai 7bbe9495c4 HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai. 2015-04-28 18:06:08 -07:00
Tsz-Wo Nicholas Sze 9b62a8ed77 HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su 2015-04-28 13:06:30 -07:00
cnauroth a82addd614 HDFS-8232. Missing datanode counters when using Metrics2 sink interface. Contributed by Anu Engineer. 
(cherry picked from commit feb68cb547)
 2015-04-27 16:48:25 -07:00
Arpit Agarwal de85ff58dc HDFS-8205. Fix CHANGES.txt 2015-04-27 12:30:55 -07:00
Arpit Agarwal c254a2a671 HDFS-8205. CommandFormat#parse() should not parse option as value of option. (Contributed by Peter Shi and Xiaoyu Yao) 2015-04-27 12:25:45 -07:00
Xiaoyu Yao 7aa3399cc5 HDFS-8206. Fix the typos in hadoop-hdfs-httpfs. (Brahma Reddy Battula via xyao) 2015-04-25 22:24:38 -07:00
cnauroth 8d256c56db HDFS-8247. TestDiskspaceQuotaUpdate#testAppendOverTypeQuota is failing. Contributed by Xiaoyu Yao. 
(cherry picked from commit a00e001a1a)
 2015-04-25 11:37:41 -07:00
Arpit Agarwal 932cff610a HDFS-8211. DataNode UUID is always null in the JMX counter. (Contributed by Anu Engineer) 2015-04-24 16:47:56 -07:00
Andrew Wang 89a15d6074 HDFS-8191. Fix byte to integer casting in SimulatedFSDataset#simulatedByte. Contributed by Zhe Zhang. 
(cherry picked from commit c7d9ad68e3)
 2015-04-24 11:54:48 -07:00
Jing Zhao 5870d504e1 HDFS-8176. Record from/to snapshots in audit log for snapshot diff report. Contributed by J. Andreina. 
(cherry picked from commit cf6c8a1b4e)
 2015-04-24 10:24:12 -07:00
Vinayakumar B 2ec356fcdb HDFS-8231. StackTrace displayed at client while QuotaByStorageType exceeds (Contributed by J.Andreina and Xiaoyu Yao) 
(cherry picked from commit c8d72907ff)
 2015-04-24 12:51:56 +05:30
Vinayakumar B 68063cac3e HDFS-8217. During block recovery for truncate Log new Block Id in case of copy-on-truncate is true. (Contributed by Vinayakumar B) 
(cherry picked from commit 262c1bc339)
 2015-04-24 12:18:04 +05:30
Colin Patrick Mccabe 3d0385c3c8 HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) 
(cherry picked from commit a8898445dc)
 2015-04-23 19:15:33 -07:00
Colin Patrick Mccabe 788b76761d HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) 
(cherry picked from commit a0e0a63209)
 2015-04-23 19:08:20 -07:00
Haohui Mai b64bb2b9b4 HDFS-8052. Move WebHdfsFileSystem into hadoop-hdfs-client. Contributed by Haohui Mai. 2015-04-23 17:44:21 -07:00
Tsz-Wo Nicholas Sze f27a976a2f HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma 2015-04-23 14:23:02 -07:00
Tsz-Wo Nicholas Sze d6c8609663 HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore 2015-04-23 11:56:09 -07:00
Arun Suresh 889b92fa40 HDFS-4448. Allow HA NN to start in secure mode with wildcard address configured (atm via asuresh) 
(cherry picked from commit baf8bc6c48)
 2015-04-23 01:45:33 -07:00
Haohui Mai 71c9f2f4e7 HDFS-8218. Move classes that used by ClientProtocol into hdfs-client. Contributed by Haohui Mai. 2015-04-22 11:33:32 -07:00
Haohui Mai f02ca4ab15 HDFS-8185. Separate client related routines in HAUtil into a new class. Contributed by Haohui Mai. 2015-04-21 22:09:13 -07:00
Colin Patrick Mccabe 447f2f699e HDFS-8133. Improve readability of deleted block check (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 997408eaac)
 2015-04-21 12:06:33 -07:00
Arpit Agarwal 7b3acc5c9d HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal) 2015-04-21 10:58:12 -07:00
Vinayakumar B e7bb0fc922 HDFS-7993. Provide each Replica details in fsck (Contributed by J.Andreina) 
(cherry picked from commit 8ddbb8dd43)
 2015-04-21 15:27:15 +05:30
Vinod Kumar Vavilapalli 27fc4fb998 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:01 -07:00
Vinayakumar B 961051e569 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Updated CHANGES.TXT for correct version

(cherry picked from commit ed4137cebf)
 2015-04-21 08:00:37 +05:30
Arpit Agarwal 95a8d452c5 HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao) 2015-04-20 15:42:49 -07:00
Vinayakumar B c562e1b19b HDFS-8173. NPE thrown at DataNode shutdown when HTTP server was not able to create (Contributed by surendra singh lilhore) 
(cherry picked from commit c17cd4f7c1)
 2015-04-20 14:58:59 +05:30
Haohui Mai 5d3a4d51bd HDFS-8169. Move LocatedBlocks and related classes to hdfs-client. Contributed by Haohui Mai. 2015-04-20 00:36:19 -07:00
Arpit Agarwal 490fac391c HDFS-8152. Refactoring of lazy persist storage cases. (Arpit Agarwal) 2015-04-19 16:09:12 -07:00
Tsuyoshi Ozawa 890238b496 HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5459b241c8)
 2015-04-19 18:23:57 +09:00
Tsuyoshi Ozawa e083c07913 HDFS-7863. Missing description of some methods and parameters in javadoc of FSDirDeleteOp. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5112477d9e)
 2015-04-19 18:07:19 +09:00
Haohui Mai a825243280 HDFS-8165. Move GRANDFATHER_GENERATION_STAMP and GRANDFATER_INODE_ID to hdfs-client. Contributed by Haohui Mai. 2015-04-16 22:54:33 -07:00
Jitendra Pandey 04962efe43 HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer. 
cherry picked from 369ddc67bd
 2015-04-16 22:15:57 -07:00
Tsz-Wo Nicholas Sze fa7b18e387 HDFS-7891. A block placement policy with best rack failure tolerance. Contributed by Walter Su 2015-04-16 18:26:46 -07:00
Akira Ajisaka d363f853ab Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:50:04 -05:00
Tsz-Wo Nicholas Sze bad547d969 HDFS-8082. Move dfs.client.read.*, dfs.client.short.circuit.*, dfs.client.mmap.* and dfs.client.hedged.read.* conf from DFSConfigKeys to HdfsClientConfigKeys. 2015-04-16 13:27:15 -07:00
Haohui Mai e1ebad8203 HDFS-8135. Remove the deprecated FSConstants class. Contributed by Li Lu. 2015-04-16 11:26:13 -07:00
Andrew Wang 9123f26c73 HDFS-8142. DistributedFileSystem encryption zone commands should resolve relative paths. Contributed by Rakesh R. 
(cherry picked from commit 2e8ea780a4)
 2015-04-16 07:14:53 -07:00
Akira Ajisaka 6d183561ff HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula. 
(cherry picked from commit de0f1700c1)
 2015-04-15 15:32:08 -05:00
Jing Zhao 725621d95a HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595)
 2015-04-15 13:11:15 -07:00
Jing Zhao d4dd97eabd HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473b)
 2015-04-15 12:39:09 -07:00
Arpit Agarwal 96e12fa46a HDFS-8144. Split TestLazyPersistFiles into multiple tests. (Arpit Agarwal) 2015-04-15 10:25:16 -07:00
Andrew Wang 871bf6a765 HDFS-8117. More accurate verification in SimulatedFSDataset: replace DEFAULT_DATABYTE with patterned data. Contributed by Zhe Zhang. 2015-04-15 08:43:42 -07:00
Jing Zhao 38b031d6ba HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d)
 2015-04-14 15:12:37 -07:00
cnauroth c7ebecfff5 HDFS-8055. NullPointerException when topology script is missing. Contributed by Anu Engineer. 
(cherry picked from commit fef596df03)
 2015-04-14 10:19:45 -07:00
cnauroth 9779b79ed8 HDFS-6666. Abort NameNode and DataNode startup if security is enabled but block access token is not enabled. Contributed by Vijay Bhat. 
(cherry picked from commit d45aa7647b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNode.java
 2015-04-14 10:03:53 -07:00
Arpit Agarwal d77705c980 HDFS-7701. Support reporting per storage type quota and usage with hadoop/hdfs shell. (Contributed by Peter Shi) 2015-04-13 21:20:30 -07:00
Allen Wittenauer b7a052c84a HADOOP-7713. dfs -count -q should label output column (Jonathan Allen via aw) 2015-04-13 21:18:59 -07:00
Tsz-Wo Nicholas Sze e4fe5fa2ee HDFS-8083. Move dfs.client.write.* conf from DFSConfigKeys to HdfsClientConfigKeys.Write. 2015-04-13 11:45:03 -07:00
Vinayakumar B 1071a57b20 HDFS-8111. NPE thrown when invalid FSImage filename given for 'hdfs oiv_legacy' cmd ( Contributed by surendra singh lilhore ) 
(cherry picked from commit 14384f5b51)
 2015-04-13 17:19:18 +05:30
cnauroth 293c763e82 HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. 
(cherry picked from commit f8f5887209)
 2015-04-11 13:24:06 -07:00
Tsz-Wo Nicholas Sze ef2bd4bd11 HDFS-8084. Move dfs.client.failover.* confs from DFSConfigKeys to HdfsClientConfigKeys.Failover and fix typos in the dfs.http.client.* configuration keys. 2015-04-10 19:58:42 -07:00
Haohui Mai 6f97b780df HDFS-8103. Move BlockTokenSecretManager.AccessMode into BlockTokenIdentifier. Contributed by Haohui Mai. 2015-04-10 16:36:57 -07:00
Tsz-Wo Nicholas Sze 1113aca7f8 HDFS-8100. Refactor DFSClient.Conf to a standalone class and separates short-circuit related conf to ShortCircuitConf. 2015-04-10 14:49:32 -07:00
Arun Suresh 3d17c50176 HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 
(cherry picked from commit 7660da95cb)
 2015-04-10 11:42:18 -07:00
Vinayakumar B 4aba069b37 HDFS-7939. Two fsimage_rollback_* files are created which are not deleted after rollback. (Contributed by J.Andreina) 
(cherry picked from commit 987c9e12e1)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-10 11:27:20 +05:30
Konstantin V Shvachko 0e99da620d HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:02:14 -07:00