Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,807 Commits 326 Branches 371 Tags
Commit Graph

3334 Commits

Author SHA1 Message Date
Andrew Wang
320e4cc0e7 HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu. 
(cherry picked from commit b2540f486ed99e1433d4e5118608da8dd365a934)
 2015-06-04 15:35:12 -07:00
Andrew Wang
ccc834395b HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R. 
(cherry picked from commit 5149dc7b975f0e90a14e3da02685594028534805)
 2015-06-04 11:09:28 -07:00
Kihwal Lee
f0f277ddd0 HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. 
(cherry picked from commit ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)
 2015-06-04 12:52:49 -05:00
Vinayakumar B
539ff92c2a HDFS-3716. Purger should remove stale fsimage ckpt files (Contributed by J.Andreina) 
(cherry picked from commit c59e7456304def867afe35d6ee009ab572f8fdeb)
 2015-06-03 15:31:01 +05:30
Vinayakumar B
0ed9c2d8fe HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 2015-06-03 15:15:44 +05:30
Vinayakumar B
4bb725b4dd HDFS-8470. fsimage loading progress should update inode, delegation token and cache pool count. (Contributed by surendra singh lilhore) 
(cherry picked from commit e965dcec378cb807856372425307598792977604)
 2015-06-03 14:25:12 +05:30
Vinayakumar B
066e45bcb6 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd917e8641e902c5f0695e65ded472f9a)
 2015-06-03 12:12:22 +05:30
Colin Patrick Mccabe
12497b0543 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8896c4e669dba1fcf86fa5d6fee7c6d)
 2015-06-02 20:06:53 -07:00
Andrew Wang
a80a68c298 HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant. (wang) 
(cherry picked from commit c1d50a91f7c05e4aaf4655380c8dcd11703ff158)
 2015-06-02 15:48:31 -07:00
Andrew Wang
379ece15fe HDFS-8386. Improve synchronization of 'streamer' reference in DFSOutputStream. Contributed by Rakesh R. 
(cherry picked from commit efc510a570cf880e7df1b69932aa41932658ee51)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-06-02 15:42:27 -07:00
Colin Patrick Mccabe
17e369511d HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c642589dec4e663479771d0ae1782038b63)
 2015-06-02 11:42:08 -07:00
Vinayakumar B
d1da842f86 HDFS-8256. '-storagepolicies , -blockId ,-replicaDetails ' options are missed out in usage and from documentation (Contributed by J.Andreina) 
(cherry picked from commit a2bd6217ebd68ae8cdd7814722659eebcf53004b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2015-06-02 16:25:53 +05:30
Jing Zhao
640ea994ed HDFS-8489. Subclass BlockInfo to represent contiguous blocks. Contributed by Zhe Zhang. 
(cherry picked from commit cdc13efb1af54d931585d25c5ba696a012412828)
 2015-06-01 11:45:04 -07:00
Tsuyoshi Ozawa
b91cadcafd HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS. Contributed by Archana T. 
(cherry picked from commit c2356cb0bc3a0ad7d15c03fc63591608bfd5cbe9)
 2015-06-01 00:42:11 +09:00
Andrew Wang
bdaa0fed13 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 
(cherry picked from commit 7673d4f205b26a6a26cfc47d999ece96f3c42782)

 Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
	hadoop-mapreduce-project/pom.xml
 2015-05-29 14:55:52 -07:00
Jing Zhao
17fb442a4c HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a4d097b647dd77f1345787dd376d5ea)
 2015-05-29 11:11:01 -07:00
Vinayakumar B
983c1d3969 HDFS-7401. Add block info to DFSInputStream' WARN message when it adds node to deadNodes (Contributed by Arshad Mohammad) 
(cherry picked from commit b75df697e0f101f86788ad23a338ab3545b8d702)
 2015-05-29 16:36:39 +05:30
Akira Ajisaka
e7b81710dd HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. 
(cherry picked from commit d725dd8af682f0877cf523744d9801174b727f4e)
 2015-05-29 11:36:59 +09:00
Colin Patrick Mccabe
4b108a441d HADOOP-11894. Bump the version of Apache HTrace to 3.2.0-incubating (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit a1140959dab3f35accbd6c66abfa14f94ff7dcec)
 2015-05-28 12:03:07 -07:00
Colin Patrick Mccabe
b9752f24a6 HDFS-8407. libhdfs hdfsListDirectory must set errno to 0 on success (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit d2d95bfe886a7fdf9d58fd5c47ec7c0158393afb)
 2015-05-28 11:38:44 -07:00
cnauroth
25db341278 HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. 
(cherry picked from commit 50eeea13000f0c82e0567410f0f8b611248f8c1b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd
 2015-05-27 22:56:04 -07:00
Andrew Wang
a634809895 HDFS-8482. Rename BlockInfoContiguous to BlockInfo. Contributed by Zhe Zhang. 
(cherry picked from commit ff83459f192f7cc4ce864f0b1f01e1e08243be83)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
 2015-05-27 15:42:12 -07:00
Haohui Mai
3f321be24d Addendum of HDFS-8135. 2015-05-27 13:14:51 -07:00
Haohui Mai
cc348fead7 Revert HDFS-8135 in branch-2. 2015-05-27 11:06:17 -07:00
Akira Ajisaka
9670762a82 HADOOP-11242. Record the time of calling in tracing span of IPC server. Contributed by Mastake Iwasaki. 
(cherry picked from commit bb1816328a36ec3f8c6bd9fdb950d9a4ec8388c8)
 2015-05-27 18:05:08 +09:00
Colin Patrick Mccabe
3dec58dd78 HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 7dba7005b79994106321b0f86bc8f4ea51a3c185)
 2015-05-26 12:40:41 -07:00
Haohui Mai
f70b9d9241 HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang. 2015-05-24 22:30:50 -07:00
Vinayakumar B
19895a41ad HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 
(cherry picked from commit 55ed6558bad5d76ce4ebf958fc101a48032a0cd1)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
 2015-05-22 16:46:50 +05:30
Vinayakumar B
658f5cf985 HDFS-8268. Port conflict log for data node server is not sufficient (Contributed by Mohammad Shahid Khan) 
(cherry picked from commit 0c6638c2ea278bd460df88e7118945e461266a8b)
 2015-05-22 16:15:39 +05:30
Arpit Agarwal
a073e5bd31 HDFS-8454. Remove unnecessary throttling in TestDatanodeDeath. (Arpit Agarwal) 2015-05-21 20:44:07 -07:00
Xiaoyu Yao
61d7ccb27e HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f27149537fdb89f46af26bee14531ca4)
 2015-05-21 11:59:11 -07:00
Haohui Mai
0b909d028f HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp. Contributed by Haohui Mai. 2015-05-21 08:06:13 -07:00
Akira Ajisaka
b578f76bed HADOOP-10366. Add whitespaces between classes for values in core-default.xml to fit better in browser. Contributed by kanaka kumar avvaru. 
(cherry picked from commit 0e4f1081c7a98e1c0c4f922f5e2afe467a0d763f)
 2015-05-21 17:53:18 +09:00
Akira Ajisaka
185cddfd6a HDFS-4383. Document the lease limits. Contributed by Arshad Mohammad. 
(cherry picked from commit dc8434ab2b177ca9673bd8eecf7b185d4c4ffb31)
 2015-05-21 17:31:47 +09:00
Colin Patrick Mccabe
9534e533dc HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 470c87dbc6c24dd3b370f1ad9e7ab1f6dabd2080)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
 2015-05-19 11:15:04 -07:00
Kihwal Lee
536b9ee6d6 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 
(cherry picked from commit 8860e352c394372e4eb3ebdf82ea899567f34e4e)
 2015-05-19 13:06:48 -05:00
Kihwal Lee
9e656bfa94 HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui. 
(cherry picked from commit de30d66b2673d0344346fb985e786247ca682317)
 2015-05-19 08:06:07 -05:00
Vinayakumar B
5bbf157cb2 HADOOP-11103. Clean up RemoteException (Contributed by Sean Busbey) 
(cherry picked from commit d4a2830b63f0819979b592f4ea6ea3abd5885b71)
 2015-05-19 14:41:34 +05:30
Allen Wittenauer
3d0e2e54a1 HADOOP-8934. Shell command ls should include sort options (Jonathan Allen via aw) 
(cherry picked from commit 30b797ee9df30260314eeadffc7d51492871b352)

Conflicts:
	hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
 2015-05-19 17:38:22 +09:00
Vinayakumar B
6dee42f6db HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions (Contributed by Rakesh R) 
(cherry picked from commit 93972a332a9fc6390447fc5fc9785c98fb4c3344)
 2015-05-19 12:25:00 +05:30
Ravi Prakash
e5b805d361 HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak) 
(cherry picked from commit cdfae446ad285db979a79bf55665363fd943702c)
 2015-05-18 12:38:32 -07:00
Tsz-Wo Nicholas Sze
7bd4db968d HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 03:01:07 +08:00
Arpit Agarwal
9bcb740049 HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) 2015-05-18 11:49:01 -07:00
Uma Maheswara Rao G
ea875939f1 Revert from branch-2: HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R. 2015-05-18 14:38:30 +05:30
Arpit Agarwal
151f6d6dcc HDFS-8157. Writes to RAM DISK reserve locked memory for block files. (Arpit Agarwal) 2015-05-16 09:05:50 -07:00
Haohui Mai
46d62e7016 HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal. 2015-05-15 19:12:46 -07:00
Haohui Mai
1796d94d62 HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai. 2015-05-15 19:10:05 -07:00
Jing Zhao
d6aa65d037 HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 8f378733423a5244461df79a92c00239514b8b93)
 2015-05-15 16:32:25 -07:00
Vinayakumar B
d9455c790f HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) 
(cherry picked from commit 7f2e89fa7082840bfa3e8e593c93db050a80d04f)
 2015-05-15 11:05:26 +05:30
Akira Ajisaka
7fb3486eb3 HDFS-8350. Remove old webhdfs.xml and other outdated documentation stuff. Contributed by Brahma Reddy Battula. 2015-05-15 13:57:58 +09:00