Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,508 Commits 326 Branches 371 Tags
Commit Graph

3190 Commits

Author SHA1 Message Date
Kihwal Lee
d2d038b0a0 HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 
(cherry picked from commit c74517c46bf00af408ed866b6577623cdec02de1)
 2015-06-16 15:42:00 -05:00
Xiaoyu Yao
d7c8a6feaa HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4e88ff5b27cc33d311ab7a7248c3cf6303997ddd)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-16 12:47:05 -07:00
Haohui Mai
be8e4ed1e1 HDFS-8592. SafeModeException never get unwrapped. Contributed by Haohui Mai. 2015-06-15 22:07:42 -07:00
Yongjun Zhang
b1d3ed02c1 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 2015-06-15 21:45:30 -07:00
Tsz-Wo Nicholas Sze
10b97f6655 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:18:09 -07:00
Tsz-Wo Nicholas Sze
95483513ed Move HDFS-8540 to 2.8 in CHANGES.txt. 2015-06-15 17:02:45 -07:00
Tsz-Wo Nicholas Sze
9ba29f081b HDFS-8540. Mover should exit with NO_MOVE_BLOCK if no block can be moved. Contributed by surendra singh lilhore 2015-06-15 16:28:37 -07:00
Tsz-Wo Nicholas Sze
5c111a5192 HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:09:32 -07:00
Arpit Agarwal
ea9a9b3d7e HDFS-8607. TestFileCorruption doesn't work as expected. (Contributed by Walter Su) 2015-06-15 10:12:07 -07:00
Arpit Agarwal
d07f3804b0 HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:36 -07:00
Yongjun Zhang
cd578fa289 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961410b6ac2b275cdcff4b95d75727e8d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
 2015-06-14 12:01:28 -07:00
cnauroth
92144ca75c HDFS-8593. Calculation of effective layout version mishandles comparison to current layout version in storage. Contributed by Chris Nauroth. 
(cherry picked from commit b8341f1cd89791c51b396ad531ec7fcc631be149)
 2015-06-13 10:38:03 -07:00
Andrew Wang
2776255ed6 HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang. 
(cherry picked from commit 355e98ab7c7dd997378a8624f642199713467fb8)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
 2015-06-12 11:39:02 -07:00
Colin Patrick Mccabe
378bb484bb HDFS-7923. The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages (cmccabe) 
(cherry picked from commit 12b5b06c063d93e6c683c9b6fac9a96912f59e59)
 2015-06-12 11:29:05 -07:00
Akira Ajisaka
e397cca456 HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to hadoop-common. Contributed by Masatake Iwasaki. 
(cherry picked from commit e4489d97e5f0cec601b93e1b883fa194a7353229)
 2015-06-12 07:26:05 -07:00
Haohui Mai
f1ee2eaf86 HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 18:59:53 -07:00
Arpit Agarwal
a560e654d9 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:50 -07:00
Colin Patrick Mccabe
754dd3496c HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbbadd4db41be83347e8c454049f4a7561)
 2015-06-11 14:28:41 -07:00
Andrew Wang
086eb49424 HDFS-8573. Move creation of restartMeta file logic from BlockReceiver to ReplicaInPipeline. Contributed by Eddy Xu. 
(cherry picked from commit b258b344bb76af6492828201959e36b45f0f75b8)
 2015-06-11 10:12:40 -07:00
Andrew Wang
9523e2c098 HDFS-8549. Abort the balancer if an upgrade is in progress. 
(cherry picked from commit a7a7768341f1b7d3a8f2686e2f4d00c57f2e1d4f)
 2015-06-10 13:43:00 -07:00
Arpit Agarwal
2d67dff15f HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:46 -07:00
Xiaoyu Yao
d8d9041187 HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing. Contributed by Rakesh R. 
(cherry picked from commit 3c2397cb041068172c8ad209996414d7371d2eca)
 2015-06-09 12:21:53 -07:00
Xiaoyu Yao
116a72096d HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc. Contributed by Brahma Reddy Battula 
(cherry picked from commit 927577c87ca19e8b5b75722f78e2def6d9386576)
 2015-06-08 22:03:07 -07:00
cnauroth
cc960734e4 HDFS-8553. Document hdfs class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d2832b3d4243c6c470c774bc33fd13f70b3e7b72)
 2015-06-08 16:15:53 -07:00
cnauroth
8ee50d8ca7 HDFS-8554. TestDatanodeLayoutUpgrade fails on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit 0e80d51983942dca3348c8a8401bad3ecbaab010)
 2015-06-08 08:39:14 -07:00
cnauroth
953c4a049a HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu Engineer. 
(cherry picked from commit a6cb4894b45b0b909aca716058590dd6f0e3a6f9)
 2015-06-07 13:02:01 -07:00
Xiaoyu Yao
c8dc113943 HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b61b48999530851da177db2e8998858ffc01f77e)
 2015-06-06 18:48:52 -07:00
cnauroth
f99744b7a0 HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. 
(cherry picked from commit 71de367c5e80ea76d1e8d21f0216cd6b879dcee5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
 2015-06-06 09:45:46 -07:00
cnauroth
1d1841d40f HADOOP-12055. Deprecate usage of NativeIO#link. Contributed by Andrew Wang. 
(cherry picked from commit e46cb800028c95f9bce575d05268cd10d0913222)
 2015-06-06 09:17:11 -07:00
Xiaoyu Yao
1a6cdf97bb HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:33:06 -07:00
Andrew Wang
320e4cc0e7 HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu. 
(cherry picked from commit b2540f486ed99e1433d4e5118608da8dd365a934)
 2015-06-04 15:35:12 -07:00
Andrew Wang
ccc834395b HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R. 
(cherry picked from commit 5149dc7b975f0e90a14e3da02685594028534805)
 2015-06-04 11:09:28 -07:00
Kihwal Lee
f0f277ddd0 HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. 
(cherry picked from commit ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)
 2015-06-04 12:52:49 -05:00
Vinayakumar B
539ff92c2a HDFS-3716. Purger should remove stale fsimage ckpt files (Contributed by J.Andreina) 
(cherry picked from commit c59e7456304def867afe35d6ee009ab572f8fdeb)
 2015-06-03 15:31:01 +05:30
Vinayakumar B
0ed9c2d8fe HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 2015-06-03 15:15:44 +05:30
Vinayakumar B
4bb725b4dd HDFS-8470. fsimage loading progress should update inode, delegation token and cache pool count. (Contributed by surendra singh lilhore) 
(cherry picked from commit e965dcec378cb807856372425307598792977604)
 2015-06-03 14:25:12 +05:30
Vinayakumar B
066e45bcb6 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd917e8641e902c5f0695e65ded472f9a)
 2015-06-03 12:12:22 +05:30
Colin Patrick Mccabe
12497b0543 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8896c4e669dba1fcf86fa5d6fee7c6d)
 2015-06-02 20:06:53 -07:00
Andrew Wang
a80a68c298 HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant. (wang) 
(cherry picked from commit c1d50a91f7c05e4aaf4655380c8dcd11703ff158)
 2015-06-02 15:48:31 -07:00
Andrew Wang
379ece15fe HDFS-8386. Improve synchronization of 'streamer' reference in DFSOutputStream. Contributed by Rakesh R. 
(cherry picked from commit efc510a570cf880e7df1b69932aa41932658ee51)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-06-02 15:42:27 -07:00
Colin Patrick Mccabe
17e369511d HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c642589dec4e663479771d0ae1782038b63)
 2015-06-02 11:42:08 -07:00
Vinayakumar B
d1da842f86 HDFS-8256. '-storagepolicies , -blockId ,-replicaDetails ' options are missed out in usage and from documentation (Contributed by J.Andreina) 
(cherry picked from commit a2bd6217ebd68ae8cdd7814722659eebcf53004b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2015-06-02 16:25:53 +05:30
Jing Zhao
640ea994ed HDFS-8489. Subclass BlockInfo to represent contiguous blocks. Contributed by Zhe Zhang. 
(cherry picked from commit cdc13efb1af54d931585d25c5ba696a012412828)
 2015-06-01 11:45:04 -07:00
Tsuyoshi Ozawa
b91cadcafd HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS. Contributed by Archana T. 
(cherry picked from commit c2356cb0bc3a0ad7d15c03fc63591608bfd5cbe9)
 2015-06-01 00:42:11 +09:00
Andrew Wang
bdaa0fed13 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 
(cherry picked from commit 7673d4f205b26a6a26cfc47d999ece96f3c42782)

 Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
	hadoop-mapreduce-project/pom.xml
 2015-05-29 14:55:52 -07:00
Jing Zhao
17fb442a4c HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a4d097b647dd77f1345787dd376d5ea)
 2015-05-29 11:11:01 -07:00
Vinayakumar B
983c1d3969 HDFS-7401. Add block info to DFSInputStream' WARN message when it adds node to deadNodes (Contributed by Arshad Mohammad) 
(cherry picked from commit b75df697e0f101f86788ad23a338ab3545b8d702)
 2015-05-29 16:36:39 +05:30
Akira Ajisaka
e7b81710dd HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. 
(cherry picked from commit d725dd8af682f0877cf523744d9801174b727f4e)
 2015-05-29 11:36:59 +09:00
Colin Patrick Mccabe
4b108a441d HADOOP-11894. Bump the version of Apache HTrace to 3.2.0-incubating (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit a1140959dab3f35accbd6c66abfa14f94ff7dcec)
 2015-05-28 12:03:07 -07:00
Colin Patrick Mccabe
b9752f24a6 HDFS-8407. libhdfs hdfsListDirectory must set errno to 0 on success (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit d2d95bfe886a7fdf9d58fd5c47ec7c0158393afb)
 2015-05-28 11:38:44 -07:00