Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,811 Commits 326 Branches 371 Tags
Commit Graph

2916 Commits

Author SHA1 Message Date
Yongjun Zhang
f4ae56d605 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit b1d3ed02c16113f2f6bd74f56e3d293c0daf64e2)
 2015-06-15 21:51:21 -07:00
Tsz-Wo Nicholas Sze
632cb542f6 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:29:33 -07:00
Tsz-Wo Nicholas Sze
7a248f2e9e HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:13:56 -07:00
Arpit Agarwal
a1810d75f0 HDFS-8600. TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7. (Contributed by Arpit Agarwal) 2015-06-14 20:18:31 -07:00
Arpit Agarwal
1145d09e0a HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:42 -07:00
Yongjun Zhang
6dda57a817 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961410b6ac2b275cdcff4b95d75727e8d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java

(cherry picked from commit cd578fa289252b0ae1e27f7e7b097cceeb2c30ff)
 2015-06-14 12:04:16 -07:00
Haohui Mai
3651feb49d HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 19:14:48 -07:00
Arpit Agarwal
3ec9b63586 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:15 -07:00
Colin Patrick Mccabe
0bcc68337b HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbbadd4db41be83347e8c454049f4a7561)
(cherry picked from commit 754dd3496ce166f3743a8512f555ade6a7c2135d)
 2015-06-11 14:30:21 -07:00
Arpit Agarwal
105cadb421 HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:55 -07:00
Xiaoyu Yao
8c1a46ca4f HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:13:40 -07:00
Vinayakumar B
dd98cfd328 HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 
(cherry picked from commit 0ed9c2d8fec93b5dac9c305eda272ad8dfd869a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2015-06-03 15:18:42 +05:30
Vinayakumar B
8090a6ee63 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd917e8641e902c5f0695e65ded472f9a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
 2015-06-03 12:20:54 +05:30
Colin Patrick Mccabe
a7058caa72 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8896c4e669dba1fcf86fa5d6fee7c6d)
(cherry picked from commit 12497b0543176306edbd5cd8660a59b1e2e3a1c8)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2015-06-02 20:07:35 -07:00
Colin Patrick Mccabe
323ffccfe9 HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c642589dec4e663479771d0ae1782038b63)
(cherry picked from commit 17e369511dd18d6e25e10ddb0cd9511f7ec8f469)
 2015-06-02 11:46:12 -07:00
Kihwal Lee
1ff3f16ed0 HDFS-5215. dfs.datanode.du.reserved is not considered while computing 
available space ( Brahma Reddy Battula via Yongjun Zhang)
(cherry picked from commit 66763bb06f107f0e072c773a5feb25903c688ddc)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-02 08:14:57 -05:00
Xiaoyu Yao
c550825673 HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f27149537fdb89f46af26bee14531ca4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
 2015-05-21 12:49:49 -07:00
Kihwal Lee
7ac66f8712 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 
(cherry picked from commit 8860e352c394372e4eb3ebdf82ea899567f34e4e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2015-05-19 13:17:45 -05:00
Tsz-Wo Nicholas Sze
59d1b4a323 HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 03:17:51 +08:00
cnauroth
82530ba0f1 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 
(cherry picked from commit ddc4afeff256e124ce4291c7a2bab37ad55c8e5c)
 2015-05-13 12:26:02 -07:00
Tsz-Wo Nicholas Sze
2113e0a3f2 HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore 2015-05-13 11:59:01 -07:00
Vinayakumar B
ef2488e7f0 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 
(cherry picked from commit 065d8f2a34296b566e7ca541a284f7991212f14c)
 2015-05-13 17:30:03 +05:30
Kihwal Lee
1e4a686ea9 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 
(cherry picked from commit ea11590aad952b5b560a5101d064adf27d8656db)
 2015-05-11 17:11:51 -05:00
Kihwal Lee
f57b1bfbdd HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 
(cherry picked from commit 2d4ae3d18bc530fa9f81ee616db8af3395705fb9)
 2015-05-08 16:38:43 -05:00
Kihwal Lee
bbb9f6aafe HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6f622672b62aa8d719060063ef0e47480cdc8655)
 2015-05-08 09:32:44 -05:00
Vinayakumar B
1fbefe586b HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
Added one missed change, missed while resolving conflict

(cherry picked from commit 3ee1b50dad9604bd959adbae472813f4bdf2fec7)
 2015-05-08 15:42:17 +05:30
Vinayakumar B
99309a9fec HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
(cherry picked from commit e01c1a979cbc8fb398d5b7a46693657bc2668a58)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md

(cherry picked from commit 3a894b45635d6528bb4c29953f57bd418a7c8a80)
 2015-05-08 15:42:02 +05:30
Tsz-Wo Nicholas Sze
74bd9666d1 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:38:15 -07:00
Colin Patrick Mccabe
e056e0a3c1 HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 
(cherry picked from commit fcd4cb751665adb241081e42b3403c3856b6c6fe)
(cherry picked from commit 1f01d8347a57bb95936f6d1fa5dc33b20239d6b2)
 2015-05-05 11:07:13 -07:00
Vinayakumar B
69d837428c HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit

(cherry picked from commit 318081ccd7af1ec02ec18f35ea95c579326be728)
 2015-05-05 11:07:59 +05:30
Colin Patrick Mccabe
472fd563e4 HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 
(cherry picked from commit b82567d45507c50d2f28eff4bbdf3b1a69d4bf1b)
(cherry picked from commit 9edea9507db2566cf465a77953320b1349859486)
 2015-05-01 11:52:08 -07:00
Arun Suresh
4e6ff8c78b HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh) 
(cherry picked from commit 922b7ed21d1f1460263ca42f709bb9f415d189c5)
 2015-05-01 11:42:29 -07:00
Akira Ajisaka
185a1ff275 HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao. 
(cherry picked from commit de9404f02f36bf9a1100c67f41db907d494bb9ed)
(cherry picked from commit fdb886d65642f170e65c619e49324aa208237519)
 2015-04-30 20:13:47 +09:00
Haohui Mai
6ac2b5712b HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. 2015-04-29 11:13:03 -07:00
Haohui Mai
4b9147ef8c HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai. 2015-04-28 18:06:22 -07:00
Tsz-Wo Nicholas Sze
6b20b325f4 HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su 2015-04-28 13:07:20 -07:00
Colin Patrick Mccabe
176d22b440 HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) 
(cherry picked from commit a8898445dc9b5cdb7230e2e23a57393c9f378ff0)
(cherry picked from commit 3d0385c3c8fab72be02453a158c7692212ab3b5e)
 2015-04-23 19:28:38 -07:00
Colin Patrick Mccabe
c2e6d2589a HADOOP-11802. Fix renamed constant in test file in backport to 2.7.1 (cmccabe) 2015-04-23 19:16:28 -07:00
Colin Patrick Mccabe
496afb5e1a HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) 
(cherry picked from commit a0e0a63209b5eb17dca5cc503be36aa52defeabd)
(cherry picked from commit 788b76761d5dfadf688406d50169e95401fe5d33)
 2015-04-23 19:09:50 -07:00
Tsz-Wo Nicholas Sze
5b4eda7573 HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore 2015-04-23 11:57:16 -07:00
Arpit Agarwal
b17d365fa1 HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal) 2015-04-21 11:17:27 -07:00
Vinod Kumar Vavilapalli
7dd5f42a76 Set the release date for 2.7.0 
(cherry picked from commit d52de61544060a04a273114f55979e7e929a576c)
 2015-04-20 20:18:38 -07:00
Vinayakumar B
1c8d54bc04 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675b8fb73c40fac1e581b02b005459d95)
 2015-04-21 07:45:59 +05:30
Arpit Agarwal
cc42ad03d9 HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao) 2015-04-20 15:42:58 -07:00
Jitendra Pandey
0743f128a0 HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer. 
cherry picked from 369ddc67bdaf61cca3f2f766ab504e2932f6fb72
 2015-04-16 22:23:35 -07:00
Akira Ajisaka
cffc801ecb Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0febe546e350a5fd4f37e717f2b8e52d6f.
 2015-04-16 16:46:19 -05:00
Akira Ajisaka
35e1936f84 HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula. 
(cherry picked from commit de0f1700c150a819b38028c44ef1926507086e6c)
(cherry picked from commit 6d183561ffbbd87ebaca9b67fe9bda727724aff7)
 2015-04-15 15:32:29 -05:00
Jing Zhao
e1ce0700eb HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595d1591e7f542791224607d8c5fce3e2)
(cherry picked from commit 725621d95a833f813e2164e32b33984e1c1ee29a)
 2015-04-15 13:11:45 -07:00
Jing Zhao
9e11ae684a HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473bb1f18d1510deb61bae2bcb8c156f18)
(cherry picked from commit d4dd97eabd6691eedeeb9fb7685060dfb192ff21)
 2015-04-15 12:41:34 -07:00
Jing Zhao
50c8d06314 HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d0bdd08b3b40aba6fe2ded1d2e0d846)
(cherry picked from commit 38b031d6bab8527698bd186887d301bd6a63cf01)
 2015-04-14 15:15:36 -07:00