3003 Commits

Author SHA1 Message Date
Haohui Mai
67bce1e827 HDFS-8984. Move replication queues related methods in FSNamesystem to BlockManager. Contributed by Haohui Mai. 2015-09-04 11:45:01 -07:00
Jing Zhao
21178f0b1b HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao.
(cherry picked from commit 8928729c80af0a154524e06fb13ed9b191986a78)
2015-09-04 11:42:53 -07:00
Haohui Mai
8f712dd2f0 HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to hadoop-hdfs-client module. Contributed by Mingliang Liu. 2015-09-04 10:41:28 -07:00
Andrew Wang
267bb22aaa HDFS-9021. Use a yellow elephant rather than a blue one in diagram.
(cherry picked from commit c83d13c64993c3a7f0f35142cddac19e1074976e)
2015-09-03 16:27:23 -07:00
Haohui Mai
d99018d6bc HDFS-9002. Move o.a.h.hdfs.net/*Peer classes to hdfs-client. Contributed by Mingliang Liu. 2015-09-03 15:33:22 -07:00
Jakob Homan
9871f57d41 HDFS-8939. Test(S)WebHdfsFileContextMainOperations failing on branch-2. Contributed by Chris Nauroth. 2015-09-03 13:25:53 -07:00
Colin Patrick Mccabe
9310745d22 HDFS-8964. When validating the edit log, do not read at or beyond the file offset that is being written (Zhe Zhang via Colin P. McCabe)
(cherry picked from commit 53c38cc89ab979ec47557dcfa7affbad20578c0a)
2015-09-03 11:25:33 -07:00
Arpit Agarwal
e1842ce678 HDFS-9009. Send metrics logs to NullAppender by default. (Arpit Agarwal) 2015-09-03 11:14:53 -07:00
Tsuyoshi Ozawa
f4ee0bf19f HADOOP-5323. Trash documentation should describe its directory structure and configurations. Contributed by Weiwei Yang.
(cherry picked from commit 9a87f81ce6839d1b5ab3d663c1e82c7e572bdeb8)
2015-09-03 22:34:40 +09:00
Akira Ajisaka
772fe0f478 HDFS-8885. ByteRangeInputStream used in webhdfs does not override available(). Contributed by Shradha Revankar.
(cherry picked from commit c92e31bd659e95c8baa0f3b2bf0cd7f6f72278e6)
2015-09-03 19:41:36 +09:00
Ming Ma
f81d12668f HDFS-8890. Allow admin to specify which blockpools the balancer should run on. (Chris Trezzo via mingma)
(cherry picked from commit d31a41c35927f02f2fb40d19380b5df4bb2b6d57)
2015-09-02 15:57:55 -07:00
Jing Zhao
1d56325a80 HDFS-9003. ForkJoin thread pool leaks. Contributed by Kihwal Lee.
(cherry picked from commit de928d566a119f0b7fa5f171719642cd86be0af7)
2015-09-02 15:19:29 -07:00
Andrew Wang
e1943e4471 HDFS-328. Improve fs -setrep error message for invalid replication factors. Contributed by Daniel Templeton.
(cherry picked from commit afc88b396f06488c331564e0f6987013bb920d3e)
2015-09-02 13:49:23 -07:00
Colin Patrick Mccabe
5e4bc2640d HDFS-2070. Add more unit tests for FsShell getmerge (Daniel Templeton via Colin P. McCabe)
(cherry picked from commit c006c3a1e6d3631c5724e1eb0a0b620d9f7c8dcc)
2015-09-02 12:47:00 -07:00
Akira Ajisaka
25efccba01 HDFS-8388. Time and Date format need to be in sync in NameNode UI page. Contributed by Surendra Singh Lilhore.
(cherry picked from commit 65ccf2b1252a5c83755fa24a93cf1d30ee59b2c3)
2015-09-02 14:30:23 +09:00
yliu
0fb6b68101 HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect. (Kihwal Lee via yliu) 2015-09-02 09:01:31 +08:00
Colin Patrick Mccabe
b52ae6140b HDFS-8965. Harden edit log reading code against out of memory errors (cmccabe)
(cherry picked from commit 24f6a7c9563757234f53ca23e12f9c9208b53082)
2015-08-31 18:06:48 -07:00
yliu
d852ec1f77 HDFS-8946. Improve choosing datanode storage for block placement. (yliu) 2015-09-01 08:51:32 +08:00
Haohui Mai
94ab3b9e8f HDFS-8990. Move RemoteBlockReader to hdfs-client module. Contributed by Mingliang Liu. 2015-08-31 13:54:42 -07:00
Jing Zhao
f39c749f2a HDFS-8980. Remove unnecessary block replacement in INodeFile. Contributed by Jing Zhao.
(cherry picked from commit caa04de149030691b7bc952b534c6128db217ed2)
2015-08-31 11:48:32 -07:00
Jakob Homan
94944ba4f6 HDFS-8155. Support OAuth2 in WebHDFS.
(cherry picked from commit 837fb75e8e03b2f016bcea2f4605106a5022491c)
2015-08-29 18:38:21 -07:00
Arpit Agarwal
f1d5f1417e HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) 2015-08-29 09:57:53 -07:00
Haohui Mai
a3d0534558 HDFS-8925. Move BlockReaderLocal to hdfs-client. Contributed by Mingliang Liu. 2015-08-28 14:38:23 -07:00
Colin Patrick Mccabe
159969f658 HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list (Daniel Templeton)
(cherry picked from commit b94b56806d3d6e04984e229b479f7ac15b62bbfa)
2015-08-28 14:22:05 -07:00
Haohui Mai
dea531a3d5 HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu. 2015-08-28 14:13:59 -07:00
Kihwal Lee
5184779418 HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee.
(cherry picked from commit b6ceee9bf42eec15891f60a014bbfa47e03f563c)
2015-08-28 13:19:28 -05:00
yliu
5f9b323eb0 HDFS-8963. Fix incorrect sign extension of xattr length in HDFS-8900. (Colin Patrick McCabe via yliu) 2015-08-28 10:53:53 +08:00
Haohui Mai
07a19325c4 Revert "HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu."
This reverts commit 0ae19f76f428264285073b151bf1e340a5d253ef.
2015-08-27 16:09:51 -07:00
Haohui Mai
0ae19f76f4 HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu. 2015-08-27 15:49:26 -07:00
Haohui Mai
1b01d163a2 HDFS-8969. Clean up findbugs warnings for HDFS-8823 and HDFS-8932. Contributed by Anu Engineer. 2015-08-27 13:03:29 -07:00
Haohui Mai
f4b2348503 HDFS-8961. Investigate lock issue in o.a.h.hdfs.shortcircuit.DfsClientShmManager.EndpointShmManager. Contributed by Mingliang Liu. 2015-08-27 09:13:37 -07:00
Haohui Mai
af1e2993c7 HDFS-8962. Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf. Contributed by Mingliang Liu. 2015-08-27 09:05:28 -07:00
Harsh J
574f3a98ce HDFS-2390. dfsadmin -setBalancerBandwidth does not validate -ve value. Contributed by Gautam Gopalakrishnan.
(cherry picked from commit 0bf285413f8fcaadbb2d5817fe8090f5fb0d37d9)
2015-08-27 16:23:22 +05:30
Vinayakumar B
d63122979f HDFS-8682. Should not remove decommissioned node,while calculating the number of live/dead decommissioned node. (Contributed by J. Andreina)
(cherry picked from commit fdb56f74f38cabb0f94e0781fcedb1594904c026)
2015-08-27 13:07:35 +05:30
Haohui Mai
2f3faf39ef HDFS-8248. Store INodeId instead of the INodeFile object in BlockInfoContiguous. Contributed by Haohui Mai. 2015-08-26 18:14:49 -07:00
Jing Zhao
10c4d6df61 HDFS-8896. DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager. Contributed by Walter Su.
(cherry picked from commit f44b599003bb79b1ec9b92e7546546523ec01676)
2015-08-26 16:28:38 -07:00
Haohui Mai
f0f6f1c7e9 HDFS-8951. Move the shortcircuit package to hdfs-client. Contributed by Mingliang Liu. 2015-08-26 14:22:33 -07:00
Colin Patrick Mccabe
9264b7e119 HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe)
(cherry picked from commit a4d9acc51d1a977bc333da17780c00c72e8546f1)
2015-08-25 14:30:15 -07:00
yliu
df5dbf317d HDFS-8900. Compact XAttrs to optimize memory footprint. (yliu) 2015-08-25 16:14:11 +08:00
Xiaoyu Yao
137bde0755 HDFS-8932. NPE thrown in NameNode when try to get TotalSyncCount metric before editLogStream initialization. Contributed by Surendra Singh Lilhore
(cherry picked from commit 3b00eaea256d252be3361a7d9106b88756fcb9ba)
2015-08-24 16:57:02 -07:00
Haohui Mai
ef8437a382 HDFS-8948. Use GenericTestUtils to set log levels in TestPread and TestReplaceDatanodeOnFailure. Contributed by Mingliang Liu. 2015-08-24 16:16:33 -07:00
Jing Zhao
eefc1c174b HDFS-8928. Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array. Contributed by Jing Zhao.
(cherry picked from commit bdd79388f39f4f35af7decd5703eff587b0ddfb7)
2015-08-24 15:55:58 -07:00
Haohui Mai
95f8e93691 HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu. 2015-08-24 13:40:30 -07:00
Andrew Wang
a727c6db05 Revert "HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu."
This reverts commit 8e4afa3a671583c95263218b85cf6bfbc1e43635.
2015-08-24 11:51:46 -07:00
Colin Patrick Mccabe
87d0133703 HDFS-8930. Block report lease may leak if the 2nd full block report comes when NN is still in safemode (Colin P. McCabe via Jing Zhao)
(cherry picked from commit b5ce87f84d9de0a5347ab38c0567a5a70d1fbfd7)
2015-08-24 11:33:21 -07:00
Akira Ajisaka
f7ee225052 HDFS-8942. Update hyperlink to rack awareness page in HDFS Architecture documentation. Contributed by Masatake Iwasaki.
(cherry picked from commit bcaf83902aa4d1e3e2cd26442df0a253eae7f633)
2015-08-24 13:53:22 +09:00
Haohui Mai
8e4afa3a67 HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu. 2015-08-22 13:30:48 -07:00
Haohui Mai
8c205267c8 HDFS-8823. Move replication factor into individual blocks. Contributed by Haohui Mai. 2015-08-22 00:09:59 -07:00
Chris Douglas
c8f1097989 HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang.
Moved CHANGES.txt entry to 2.7.2

(cherry picked from commit b0564c9f3c501bf7806f07649929038624dea10f)
2015-08-20 20:39:59 -07:00
Vinod Kumar Vavilapalli
45be1240d9 Creating 2.6.2 entries in CHANGES.txt files.
(cherry picked from commit 33474939a048ac7bbe07df772f0000c84979722f)
2015-08-20 17:23:54 -07:00