Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,567 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4025 Commits

Author SHA1 Message Date
Steve Loughran 9a8d09dada HDFS-9073. Fix failures in TestLazyPersistLockedMemory testReleaseOnEviction(). (J.Andreina via stevel) 2015-09-16 10:11:59 +01:00
Steve Loughran 1f28703d93 HDFS-9072. Fix random failures in TestJMXGet. (J.Andreina via stevel) 2015-09-16 10:08:18 +01:00
Haohui Mai 08c2c402cf HDFS-7986. Allow files / directories to be deleted from the NameNode UI. Contributed by Ravi Prakash. 2015-09-15 16:48:33 -07:00
cnauroth b297a9c95c HDFS-9082. Change the log level in WebHdfsFileSystem.initialize() from INFO to DEBUG. Contributed by Santhosh Nayak. 
(cherry picked from commit 559c09dc0e)
 2015-09-15 15:13:56 -07:00
Vinayakumar B 2b78714c68 HDFS-8953. DataNode Metrics logging (Contributed by Kanaka Kumar Avvaru) 
(cherry picked from commit ce69c9b54c)
 2015-09-16 00:18:59 +05:30
Ming Ma 3531823fcc HDFS-9008. Balancer#Parameters class could use a builder pattern. (Chris Trezzo via mingma) 
(cherry picked from commit 083b44c136)
 2015-09-15 10:18:44 -07:00
Vinayakumar B aa963ab441 HDFS-9067. o.a.h.hdfs.server.datanode.fsdataset.impl.TestLazyWriter is failing in trunk (Contributed by Surendra Singh Lilhore) 
(cherry picked from commit a440567491)
 2015-09-15 17:33:15 +05:30
Haohui Mai df9bb7449c HDFS-9065. Include commas on # of files, blocks, total filesystem objects in NN Web UI. Contributed by Daniel Templeton. 2015-09-14 20:27:24 -07:00
Haohui Mai 5546df0020 HDFS-9010. Replace NameNode.DEFAULT_PORT with HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT config key. Contributed by Mingliang Liu. 2015-09-14 18:23:10 -07:00
Colin Patrick Mccabe 149d731559 HDFS-8829. Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning (He Tianyi via Colin P. McCabe) 
Change-Id: I77dc71aaf9e14ef743f2a2cbebeec04a4f628c78
(cherry picked from commit 7b5cf5352e)
 2015-09-14 16:02:26 -07:00
Colin Patrick Mccabe 2596695654 HDFS-8996. Consolidate validateLog and scanLog in FJM#EditLogFile (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 53bad4eb00)
 2015-09-14 15:52:39 -07:00
Steve Loughran c951d56556 HDFS-9069. TestNameNodeMetricsLogger failing -port in use. (stevel) 2015-09-14 10:36:34 +01:00
Haohui Mai 323821b174 HDFS-9041. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu. 2015-09-12 11:02:27 -07:00
Vinayakumar B 7545488b12 HDFS-9036. In BlockPlacementPolicyWithNodeGroup#chooseLocalStorage , random node is selected eventhough fallbackToLocalRack is true. (Contributed by J.Andreina) 
(cherry picked from commit c715650385)
 2015-09-12 17:40:41 +05:30
Vinayakumar B 1857621671 HDFS-9042. Update document for the Storage policy name (Contributed by J.Andreina) 
(cherry picked from commit d8455479b8)
 2015-09-12 12:36:37 +05:30
Kihwal Lee 50a367da20 Fix up CHANGES.txt 2015-09-11 16:02:05 -05:00
Arpit Agarwal d732c7b9de HDFS-9027. Refactor o.a.h.hdfs.DataStreamer#isLazyPersist() method. (Contributed by Mingliang Liu) 2015-09-11 10:12:02 -07:00
Vinod Kumar Vavilapalli d5823452b9 HDFS-7314. Moving to 2.6.1 CHANGES.txt section. 
(cherry picked from commit f103a70af5)
 2015-09-10 16:47:00 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) cbf90f93f9 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed)
 2015-09-10 14:01:15 -07:00
Kihwal Lee 633344aa57 HDFS-6763. Initialize file system-wide quota once on transitioning to active. Contributed by Kihwal Lee 
(cherry picked from commit a40342b0da)
 2015-09-10 09:18:10 -05:00
Akira Ajisaka c260a8aa93 HDFS-8974. Convert docs in xdoc format to markdown. Contributed by Masatake Iwasaki. 2015-09-10 16:50:57 +09:00
Vinayakumar B cd256c1fda HDFS-8581. ContentSummary on / skips further counts on yielding lock (contributed by J.Andreina) 
(cherry picked from commit 4014ce5990)
 2015-09-10 00:09:27 +05:30
Vinayakumar B 03f50de48d HDFS-7116. Add a command to get the balancer bandwidth (Contributed by Rakesh R) 
(cherry picked from commit 0f0e897bf1)
 2015-09-09 22:59:55 +05:30
Kihwal Lee d003797098 HDFS-8716. Introduce a new config specifically for safe mode block count. Contributed by Chang Li. 
(cherry picked from commit 4d13335fc9)
 2015-09-09 11:10:02 -05:00
Akira Ajisaka eb9e06f8e3 HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0113e4528d)
 2015-09-09 17:21:22 +09:00
Vinayakumar B ddecd86818 HDFS-8929. Add a metric to expose the timestamp of the last journal (Contributed by surendra singh lilhore) 
(cherry picked from commit 94cf7ab9d2)
 2015-09-09 11:06:06 +05:30
Colin Patrick Mccabe b9bf1a9391 HDFS-8860. Remove unused Replica copyOnWrite code (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit a153b9601a)
 2015-09-08 18:37:42 -07:00
Xiaoyu Yao b2465690a1 HDFS-9019. Adding informative message to sticky bit permission denied exception. Contributed by Xiaoyu Yao. 
(cherry picked from commit 970daaa5e4)
 2015-09-08 09:58:29 -07:00
Andrew Wang 355398e441 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao. 
(cherry picked from commit e1feaf6db0)
 2015-09-04 13:48:42 -07:00
Haohui Mai 67bce1e827 HDFS-8984. Move replication queues related methods in FSNamesystem to BlockManager. Contributed by Haohui Mai. 2015-09-04 11:45:01 -07:00
Jing Zhao 21178f0b1b HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. 
(cherry picked from commit 8928729c80)
 2015-09-04 11:42:53 -07:00
Haohui Mai 8f712dd2f0 HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to hadoop-hdfs-client module. Contributed by Mingliang Liu. 2015-09-04 10:41:28 -07:00
Andrew Wang 267bb22aaa HDFS-9021. Use a yellow elephant rather than a blue one in diagram. 
(cherry picked from commit c83d13c649)
 2015-09-03 16:27:23 -07:00
Haohui Mai d99018d6bc HDFS-9002. Move o.a.h.hdfs.net/*Peer classes to hdfs-client. Contributed by Mingliang Liu. 2015-09-03 15:33:22 -07:00
Jakob Homan 9871f57d41 HDFS-8939. Test(S)WebHdfsFileContextMainOperations failing on branch-2. Contributed by Chris Nauroth. 2015-09-03 13:25:53 -07:00
Colin Patrick Mccabe 9310745d22 HDFS-8964. When validating the edit log, do not read at or beyond the file offset that is being written (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 53c38cc89a)
 2015-09-03 11:25:33 -07:00
Arpit Agarwal e1842ce678 HDFS-9009. Send metrics logs to NullAppender by default. (Arpit Agarwal) 2015-09-03 11:14:53 -07:00
Tsuyoshi Ozawa f4ee0bf19f HADOOP-5323. Trash documentation should describe its directory structure and configurations. Contributed by Weiwei Yang. 
(cherry picked from commit 9a87f81ce6)
 2015-09-03 22:34:40 +09:00
Akira Ajisaka 772fe0f478 HDFS-8885. ByteRangeInputStream used in webhdfs does not override available(). Contributed by Shradha Revankar. 
(cherry picked from commit c92e31bd65)
 2015-09-03 19:41:36 +09:00
Ming Ma f81d12668f HDFS-8890. Allow admin to specify which blockpools the balancer should run on. (Chris Trezzo via mingma) 
(cherry picked from commit d31a41c359)
 2015-09-02 15:57:55 -07:00
Jing Zhao 1d56325a80 HDFS-9003. ForkJoin thread pool leaks. Contributed by Kihwal Lee. 
(cherry picked from commit de928d566a)
 2015-09-02 15:19:29 -07:00
Andrew Wang e1943e4471 HDFS-328. Improve fs -setrep error message for invalid replication factors. Contributed by Daniel Templeton. 
(cherry picked from commit afc88b396f)
 2015-09-02 13:49:23 -07:00
Colin Patrick Mccabe 5e4bc2640d HDFS-2070. Add more unit tests for FsShell getmerge (Daniel Templeton via Colin P. McCabe) 
(cherry picked from commit c006c3a1e6)
 2015-09-02 12:47:00 -07:00
Akira Ajisaka 25efccba01 HDFS-8388. Time and Date format need to be in sync in NameNode UI page. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 65ccf2b125)
 2015-09-02 14:30:23 +09:00
yliu 0fb6b68101 HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect. (Kihwal Lee via yliu) 2015-09-02 09:01:31 +08:00
Colin Patrick Mccabe b52ae6140b HDFS-8965. Harden edit log reading code against out of memory errors (cmccabe) 
(cherry picked from commit 24f6a7c956)
 2015-08-31 18:06:48 -07:00
yliu d852ec1f77 HDFS-8946. Improve choosing datanode storage for block placement. (yliu) 2015-09-01 08:51:32 +08:00
Haohui Mai 94ab3b9e8f HDFS-8990. Move RemoteBlockReader to hdfs-client module. Contributed by Mingliang Liu. 2015-08-31 13:54:42 -07:00
Jing Zhao f39c749f2a HDFS-8980. Remove unnecessary block replacement in INodeFile. Contributed by Jing Zhao. 
(cherry picked from commit caa04de149)
 2015-08-31 11:48:32 -07:00
Jakob Homan 94944ba4f6 HDFS-8155. Support OAuth2 in WebHDFS. 
(cherry picked from commit 837fb75e8e)
 2015-08-29 18:38:21 -07:00
Arpit Agarwal f1d5f1417e HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) 2015-08-29 09:57:53 -07:00
Haohui Mai a3d0534558 HDFS-8925. Move BlockReaderLocal to hdfs-client. Contributed by Mingliang Liu. 2015-08-28 14:38:23 -07:00
Colin Patrick Mccabe 159969f658 HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list (Daniel Templeton) 
(cherry picked from commit b94b56806d)
 2015-08-28 14:22:05 -07:00
Haohui Mai dea531a3d5 HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu. 2015-08-28 14:13:59 -07:00
Kihwal Lee 5184779418 HDFS-8865. Improve quota initialization performance. Contributed by Kihwal Lee. 
(cherry picked from commit b6ceee9bf4)
 2015-08-28 13:19:28 -05:00
yliu 5f9b323eb0 HDFS-8963. Fix incorrect sign extension of xattr length in HDFS-8900. (Colin Patrick McCabe via yliu) 2015-08-28 10:53:53 +08:00
Haohui Mai 07a19325c4 Revert "HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu." 
This reverts commit 0ae19f76f4.
 2015-08-27 16:09:51 -07:00
Haohui Mai 0ae19f76f4 HDFS-8938. Extract BlockToMarkCorrupt and ReplicationWork as standalone classes from BlockManager. Contributed by Mingliang Liu. 2015-08-27 15:49:26 -07:00
Haohui Mai 1b01d163a2 HDFS-8969. Clean up findbugs warnings for HDFS-8823 and HDFS-8932. Contributed by Anu Engineer. 2015-08-27 13:03:29 -07:00
Haohui Mai f4b2348503 HDFS-8961. Investigate lock issue in o.a.h.hdfs.shortcircuit.DfsClientShmManager.EndpointShmManager. Contributed by Mingliang Liu. 2015-08-27 09:13:37 -07:00
Haohui Mai af1e2993c7 HDFS-8962. Clean up checkstyle warnings in o.a.h.hdfs.DfsClientConf. Contributed by Mingliang Liu. 2015-08-27 09:05:28 -07:00
Harsh J 574f3a98ce HDFS-2390. dfsadmin -setBalancerBandwidth does not validate -ve value. Contributed by Gautam Gopalakrishnan. 
(cherry picked from commit 0bf285413f)
 2015-08-27 16:23:22 +05:30
Vinayakumar B d63122979f HDFS-8682. Should not remove decommissioned node,while calculating the number of live/dead decommissioned node. (Contributed by J. Andreina) 
(cherry picked from commit fdb56f74f3)
 2015-08-27 13:07:35 +05:30
Haohui Mai 2f3faf39ef HDFS-8248. Store INodeId instead of the INodeFile object in BlockInfoContiguous. Contributed by Haohui Mai. 2015-08-26 18:14:49 -07:00
Jing Zhao 10c4d6df61 HDFS-8896. DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager. Contributed by Walter Su. 
(cherry picked from commit f44b599003)
 2015-08-26 16:28:38 -07:00
Haohui Mai f0f6f1c7e9 HDFS-8951. Move the shortcircuit package to hdfs-client. Contributed by Mingliang Liu. 2015-08-26 14:22:33 -07:00
Colin Patrick Mccabe 9264b7e119 HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit a4d9acc51d)
 2015-08-25 14:30:15 -07:00
yliu df5dbf317d HDFS-8900. Compact XAttrs to optimize memory footprint. (yliu) 2015-08-25 16:14:11 +08:00
Xiaoyu Yao 137bde0755 HDFS-8932. NPE thrown in NameNode when try to get TotalSyncCount metric before editLogStream initialization. Contributed by Surendra Singh Lilhore 
(cherry picked from commit 3b00eaea25)
 2015-08-24 16:57:02 -07:00
Haohui Mai ef8437a382 HDFS-8948. Use GenericTestUtils to set log levels in TestPread and TestReplaceDatanodeOnFailure. Contributed by Mingliang Liu. 2015-08-24 16:16:33 -07:00
Jing Zhao eefc1c174b HDFS-8928. Improvements for BlockUnderConstructionFeature: ReplicaUnderConstruction as a separate class and replicas as an array. Contributed by Jing Zhao. 
(cherry picked from commit bdd79388f3)
 2015-08-24 15:55:58 -07:00
Haohui Mai 95f8e93691 HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu. 2015-08-24 13:40:30 -07:00
Andrew Wang a727c6db05 Revert "HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu." 
This reverts commit 8e4afa3a67.
 2015-08-24 11:51:46 -07:00
Colin Patrick Mccabe 87d0133703 HDFS-8930. Block report lease may leak if the 2nd full block report comes when NN is still in safemode (Colin P. McCabe via Jing Zhao) 
(cherry picked from commit b5ce87f84d)
 2015-08-24 11:33:21 -07:00
Akira Ajisaka f7ee225052 HDFS-8942. Update hyperlink to rack awareness page in HDFS Architecture documentation. Contributed by Masatake Iwasaki. 
(cherry picked from commit bcaf83902a)
 2015-08-24 13:53:22 +09:00
Haohui Mai 8e4afa3a67 HDFS-8934. Move ShortCircuitShm to hdfs-client. Contributed by Mingliang Liu. 2015-08-22 13:30:48 -07:00
Haohui Mai 8c205267c8 HDFS-8823. Move replication factor into individual blocks. Contributed by Haohui Mai. 2015-08-22 00:09:59 -07:00
Lei Xu 3a4d614f68 HDFS-8924. Add pluggable interface for reading replicas in DFSClient. (Colin Patrick McCabe via Lei Xu) 
(cherry picked from 7087e700e0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java
 2015-08-21 17:17:19 -07:00
Chris Douglas c8f1097989 HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang. 
Moved CHANGES.txt entry to 2.7.2

(cherry picked from commit b0564c9f3c)
 2015-08-20 20:39:59 -07:00
Vinod Kumar Vavilapalli 45be1240d9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a0)
 2015-08-20 17:23:54 -07:00
Jing Zhao cd0c6b5789 HDFS-8809. HDFS fsck reports under construction blocks as CORRUPT. Contributed by Jing Zhao. 
(cherry picked from commit c8bca62718)
 2015-08-20 16:32:04 -07:00
Colin Patrick Mccabe a0450b96a2 HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) 
(cherry picked from commit 7642f64c24)
 2015-08-20 14:40:37 -07:00
Yongjun Zhang b085c5ef6f HDFS-8828. Utilize Snapshot diff report to build diff copy list in distcp. (Yufei Gu via Yongjun Zhang) 
(cherry picked from commit 0bc15cb6e6)
 2015-08-20 08:10:34 -07:00
yliu 146db49f7f HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu) 2015-08-20 20:46:45 +08:00
yliu 63bdbb7793 HDFS-8884. Fail-fast check in BlockPlacementPolicyDefault#chooseTarget. (yliu) 2015-08-20 20:08:20 +08:00
Jing Zhao 62dfe1b7e4 HDFS-8917. Cleanup BlockInfoUnderConstruction from comments and tests. Contributed by Zhe Zhang. 
(cherry picked from commit 4e14f7982a)
 2015-08-19 15:16:35 -07:00
Haohui Mai 535b6db74c HDFS-8803. Move DfsClientConf to hdfs-client. Contributed by Mingliang Liu. 2015-08-19 11:43:32 -07:00
Jing Zhao c0a4cd978a HDFS-8867. Enable optimized block reports. Contributed by Daryn Sharp. 
(cherry picked from commit f61120d964)
 2015-08-19 10:39:18 -07:00
Arpit Agarwal 10a0ddbac3 HDFS-8911. NameNode Metric : Add Editlog counters as a JMX metric. (Contributed by Anu Engineer) 2015-08-19 09:42:48 -07:00
yliu 0824fa1c4f HDFS-8908. TestAppendSnapshotTruncate may fail with IOException: Failed to replace a bad datanode. (Tsz Wo Nicholas Sze via yliu) 2015-08-19 21:56:56 +08:00
Tsz-Wo Nicholas Sze 7245f7fad2 HDFS-8826. In Balancer, add an option to specify the source node list so that balancer only selects blocks to move from those nodes. 2015-08-18 19:26:58 -07:00
Chris Douglas 79af15fd37 HDFS-8435. Support CreateFlag in WebHDFS. Contributed by Jakob Homan 2015-08-18 17:33:21 -07:00
Akira Ajisaka 38b344adfa HDFS-8852. HDFS architecture documentation of version 2.x is outdated about append write support. Contributed by Ajith S. 
(cherry picked from commit fc509f66d8)
 2015-08-18 23:32:57 +09:00
Tsz-Wo Nicholas Sze 9ab89a9266 HDFS-8278. When computing max-size-to-move in Balancer, count only the storage with remaining >= default block size. 2015-08-17 18:42:41 -07:00
yliu 076c688780 HDFS-8862. BlockManager#excessReplicateMap should use a HashMap. (yliu) 2015-08-18 09:25:03 +08:00
Arpit Agarwal 85cb596e2e HDFS-8880. NameNode metrics logging. (Arpit Agarwal) 2015-08-17 16:55:24 -07:00
Colin Patrick Mccabe 39326ba71c HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) 2015-08-17 15:24:56 -07:00
Colin Patrick Mccabe 08508ca1c8 HDFS-8792. BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory (Yi Liu via Colin P. McCabe) 
(cherry picked from commit c77bd6af16)
 2015-08-17 12:07:16 -07:00
Haohui Mai 55f7097900 HDFS-8801. Convert BlockInfoUnderConstruction as a feature. Contributed by Jing Zhao. 2015-08-17 11:28:51 -07:00
Haohui Mai 4e712faac2 HDFS-6407. Add sorting and pagination in the datanode tab of the NN Web UI. Contributed by Haohui Mai. 2015-08-17 11:04:17 -07:00
Xiaoyu Yao 27ccbd51f6 HDFS-8883. NameNode Metrics : Add FSNameSystem lock Queue Length. Contributed by Anu Engineer. 
(cherry picked from commit a7862d5fe4)
 2015-08-17 10:23:56 -07:00
Andrew Wang 9dcb426819 HDFS-8713. Convert DatanodeDescriptor to use SLF4J logging. 
(cherry picked from commit 3fe758436041d55da1ccde4d5109da38d1f5110f)
 2015-08-17 10:16:37 -07:00
Jing Zhao 2edfc3ce71 HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang. 
(cherry picked from commit dc7a061668)
 2015-08-14 14:43:33 -07:00
Tsz-Wo Nicholas Sze 61b9e5f7ff HDFS-8824. Do not use small blocks for balancing the cluster. 2015-08-14 13:08:57 -07:00
Arpit Agarwal b094f8b589 HDFS-7649. Multihoming docs should emphasize using hostnames in configurations. (Contributed by Brahma Reddy Battula) 2015-08-14 13:08:19 -07:00
Xiaoyu Yao 17256e2ae8 HDFS-8565. Typo in dfshealth.html - Decomissioning. (nijel via xyao) 
(cherry picked from commit 1569228ec9)
 2015-08-14 12:35:17 -07:00
Vinayakumar B 0b6574380c HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
Moved to 2.6.1

(cherry picked from commit 84bf71295a)
 2015-08-14 16:13:47 +05:30
Vinayakumar B 15760a148b HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
Moved to 2.6.1

(cherry picked from commit 08bd4edf40)
 2015-08-14 12:38:23 +05:30
Vinayakumar B 9839abe1a8 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit fa2641143c)
 2015-08-14 11:46:30 +05:30
Vinayakumar B c2a9c3929e HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit f2b4bc9b6a)
 2015-08-14 11:38:03 +05:30
Vinayakumar B 2c0063d420 HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne. 
Moved CHANGES.TXT entry to 2.6.1

(cherry picked from commit d25cb8fe12)
 2015-08-14 11:24:12 +05:30
yliu 073cb16d81 HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu) 2015-08-13 16:42:40 +08:00
Akira Ajisaka 200fadedc0 HDFS-8622. Implement GETCONTENTSUMMARY operation for WebImageViewer. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 40f815131e)
 2015-08-13 12:23:03 +09:00
cnauroth bf8b09d4b8 HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu. 
(cherry picked from commit 6cc8e38db5)
 2015-08-12 16:45:05 -07:00
Xiaoyu Yao 458b684941 HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao. 
(cherry picked from commit 3e715a4f4c)
 2015-08-12 15:25:20 -07:00
Andrew Wang d53296281a HDFS-8887. Expose storage type and storage ID in BlockLocation. 
(cherry picked from commit 1ea1a8334e)
 2015-08-11 23:33:06 -07:00
Jing Zhao f0e13c5499 HDFS-8805. Archival Storage: getStoragePolicy should not need superuser privilege. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1fc3c779a4)
 2015-08-11 10:28:48 -07:00
Tsz-Wo Nicholas Sze ac8d153046 HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. 2015-08-10 16:53:15 -07:00
Jakob Homan e771dbefe9 HDFS-8866. Typo in docs: Rumtime -> Runtime. Contributed by Gabor Liptak. 
(cherry picked from commit 8f73bdd06b)
 2015-08-07 11:39:54 -07:00
Andrew Wang 0c4c7b3b62 HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. 2015-08-07 09:53:04 -07:00
Arpit Agarwal 42a05d29ee HDFS-8856. Make LeaseManager#countPath O(1). (Contributed by Arpit Agarwal) 2015-08-06 18:52:07 -07:00
Jing Zhao d7d804623e Revert "HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang." 
This reverts commit c17439c2dd.

(cherry picked from commit f4c523b69b)
 2015-08-06 10:26:09 -07:00
Jing Zhao 2c694286b4 Revert "HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang." 
This reverts commit de480d6c89.

(cherry picked from commit 663eba0ab1)
 2015-08-06 10:25:54 -07:00
Vinayakumar B f8932d478f HDFS-8815. DFS getStoragePolicy implementation using single RPC call (Contributed by Surendra Singh Lilhore) 
(cherry picked from commit cc71ad80e1)
 2015-08-06 11:11:59 +05:30
Colin Patrick Mccabe b64d2ac478 HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit c95993cbaf)
 2015-08-04 12:22:40 -07:00
yliu d2b941f94a HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) 2015-08-04 15:51:37 +08:00
Zhihai Xu c6e79178d4 HDFS-8847. change TestHDFSContractAppend to not override testRenameFileBeingAppended method. Contributed by Zhihai Xu 
(cherry picked from commit 90b510405a)
 2015-07-31 21:48:03 -07:00
Xiaoyu Yao 52ee3ecdf6 HDFS-6860. BlockStateChange logs are too noisy. Contributed by Chang Li and Xiaoyu Yao. 
(cherry picked from commit d311a38a6b)
 2015-07-31 16:19:11 -07:00
Colin Patrick Mccabe 07a042aa3c HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 
(cherry picked from commit 21d10ccc6e)
 2015-07-31 15:02:01 -07:00
Harsh J 212b8b919d HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. 
(cherry picked from commit c5caa25b8f)
 2015-07-31 08:59:22 +05:30
Arpit Agarwal 95e71772cf HDFS-7192. DN should ignore lazyPersist hint if the writer is not local. (Contributed by Arpit Agarwal) 2015-07-30 13:16:52 -07:00
Haohui Mai 3faf7bc3fe HDFS-8816. Improve visualization for the Datanode tab in the NN UI. Contributed by Haohui Mai. 2015-07-29 17:14:20 -07:00
Jason Lowe e6b4714d0b HADOOP-10945. 4-digit octal umask permissions throws a parse error. Contributed by Chang Li 
(cherry picked from commit 6f0a35724f)
 2015-07-29 15:25:59 +00:00
Vinayakumar B a0238d85f6 HDFS-8670. Better to exclude decommissioned nodes for namenode NodeUsage JMX (Contributed by J.Andreina) 
(cherry picked from commit 6374ee0db4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-07-29 14:49:16 +05:30
Akira Ajisaka dff49aee08 Revert "HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)" 
This reverts commit 17319c4a65.
 2015-07-29 16:52:54 +09:00
Lei Xu 9d81916040 HDFS-8834. TestReplication is not valid after HDFS-6482. (Contributed by Lei Xu) 
(cherry picked from commit f4f1b8b267)
 2015-07-28 23:41:44 -07:00
Vinayakumar B dcf1f1fdf8 HDFS-8822. Add SSD storagepolicy tests in TestBlockStoragePolicy#testDefaultPolicies (Contributed by Vinayakumar B) 
(cherry picked from commit 975e138df3)
 2015-07-29 11:51:18 +05:30
Vinayakumar B 0c09597942 HDFS-8811. Move BlockStoragePolicy name's constants from HdfsServerConstants.java to HdfsConstants.java (Contributed by Vinayakumar B) 
(cherry picked from commit 50887e5b07)
 2015-07-29 11:46:19 +05:30
Jakob Homan 71a5aa267d HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak. 
(cherry picked from commit 0712a8103f)
 2015-07-28 21:11:02 -07:00
Arun Suresh ffe6a34d5b HDFS-7858. Improve HA Namenode Failover detection on the client. (asuresh) 
(cherry picked from commit 030fcfa99c)
 2015-07-27 23:03:34 -07:00
Akira Ajisaka 1cf5e40838 HDFS-8810. Correct assertions in TestDFSInotifyEventInputStream class. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 1df78688c6)
 2015-07-27 13:18:02 +09:00
Akira Ajisaka 7bdd32ce3c HDFS-8735. Inotify: All events classes should implement toString() API. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit f8f6091823)
 2015-07-25 02:57:33 +09:00
Akira Ajisaka b2a532f61c HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang. 
(cherry picked from commit 206d4933a5)
 2015-07-24 18:29:24 +09:00
Akira Ajisaka 17319c4a65 HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka) 
(cherry picked from commit 02c01815ec)
 2015-07-24 11:39:16 +09:00
Haohui Mai d3e6b40b44 HDFS-8730. Clean up the import statements in ClientProtocol. Contributed by Takanobu Asanuma. 2015-07-23 10:30:43 -07:00
Arpit Agarwal 601b503895 HADOOP-12161. Add getStoragePolicy API to the FileSystem interface. (Contributed by Brahma Reddy Battula) 2015-07-23 10:14:01 -07:00
Jing Zhao 71764a92c6 HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. 
(cherry picked from commit e91ccfad07)
 2015-07-22 18:03:45 -07:00
yliu 930e344d78 HDFS-8795. Improve InvalidateBlocks#node2blocks. (yliu) 2015-07-22 15:15:08 +08:00
Haohui Mai 53ed25c3d1 HDFS-8495. Consolidate append() related implementation into a single class. Contributed by Rakesh R. 2015-07-21 17:28:28 -07:00
cnauroth aab1c14df6 HDFS-8773. Few FSNamesystem metrics are not documented in the Metrics page. Contributed by Rakesh R. 
(cherry picked from commit a26cc66f38)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-07-21 14:13:15 -07:00
cnauroth 57d55d40dd HDFS-8721. Add a metric for number of encryption zones. Contributed by Rakesh R. 
(cherry picked from commit cb03768b1b)
 2015-07-21 13:56:07 -07:00
Ravi Prakash b728edce0f Revert "HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak)" 
This reverts commit 3d58c7a700.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-07-21 11:30:41 -07:00
Vinayakumar B cd3692fe4b HDFS-7582. Enforce maximum number of ACL entries separately per access and default. (Contributed by Vinayakumar B) 
(cherry picked from commit 29cf887b22)
 2015-07-21 15:17:26 +05:30
Haohui Mai 4283831348 HDFS-7483. Display information per tier on the Namenode UI. Contributed by Benoy Antony and Haohui Mai. 2015-07-20 20:13:16 -07:00
Akira Ajisaka e78ee18642 Move HDFS-6945 to 2.7.2 section in CHANGES.txt. 
(cherry picked from commit a628f67590)
 2015-07-21 11:45:53 +09:00
yliu cadd02ad1d HDFS-8794. Improve CorruptReplicasMap#corruptReplicasMap. (yliu) 2015-07-21 09:18:43 +08:00
Ravi Prakash 3d58c7a700 HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak) 
(cherry picked from commit e4f756260f)
 2015-07-20 14:04:28 -07:00
Ming Ma 516bbf1c20 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062)
 2015-07-16 12:55:29 -07:00
Arpit Agarwal dc211ca658 HDFS-8778. TestBlockReportRateLimiting#testLeaseExpiration can deadlock. (Contributed by Arpit Agarwal) 2015-07-15 14:09:06 -07:00
Akira Ajisaka bad5d288ac HDFS-8742. Inotify: Support event for OP_TRUNCATE. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 979c9ca2ca)
 2015-07-15 04:43:03 +09:00
Kihwal Lee b0a2dc9c84 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee 
(cherry picked from commit 59388a8015)
 2015-07-14 14:07:38 -05:00
Colin Patrick Mccabe 471037883f HDFS-7608: hdfs dfsclient newConnectedPeer has no write timeout (Xiaoyu Yao via Colin P. McCabe) 
(cherry picked from commit 1d74ccecec)
 2015-07-14 11:00:45 -07:00
Tsz-Wo Nicholas Sze d8d6d69b09 HDFS-8541. Mover should exit with NO_MOVE_PROGRESS if there is no move progress. Contributed by Surendra Singh Lilhore 2015-07-13 15:13:14 -07:00
Tsz-Wo Nicholas Sze 2e455d4d8b Add HDFS-8143 to CHANGES.txt. 2015-07-13 15:07:28 -07:00
Jing Zhao 0e8c335e7c HDFS-8751. Remove setBlocks API from INodeFile and misc code cleanup. Contributed by Zhe Zhang 
(cherry picked from commit 47f4c54106)
 2015-07-10 14:18:09 -07:00
Vinayakumar B c1447e654d HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B)HDFS-2956. calling fetchdt without a --renewer argument throws NPE (Contributed by Vinayakumar B) 
(cherry picked from commit b48908033f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDelegationTokenFetcher.java
 2015-07-10 15:52:35 +05:30
Akira Ajisaka 3a0318ea93 HDFS-8749. Fix findbugs warnings in BlockManager.java. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d66302ed9b)
 2015-07-10 15:04:44 +09:00
Jing Zhao ae0d5d6917 HDFS-8729. Fix TestFileTruncate#testTruncateWithDataNodesRestartImmediately which occasionally failed. Contributed by Walter Su. 
(cherry picked from commit f4ca530c1c)
 2015-07-09 13:18:35 -07:00
Haohui Mai 981fe4cfaa HDFS-8726. Move protobuf files that define the client-sever protocols to hdfs-client. Contributed by Haohui Mai. 2015-07-08 10:37:31 -07:00
Arpit Agarwal e519df47b2 HDFS-8642. Make TestFileTruncate more reliable. (Contributed by Rakesh R) 2015-07-08 09:31:17 -07:00
Vinayakumar B 6fe2e7c3f3 HDFS-8712. Remove 'public' and 'abstract' modifiers in FsVolumeSpi and FsDatasetSpi (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit bd4e10900c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
 2015-07-08 17:03:18 +05:30
Vinayakumar B 01e12b7d69 Revert "HDFS-8712. Remove 'public' and 'abstract' modifiers in FsVolumeSpi and FsDatasetSpi (Contributed by Lei (Eddy) Xu)" 
This reverts commit 38ee9247cd.
 2015-07-08 16:59:22 +05:30
Vinayakumar B 38ee9247cd HDFS-8712. Remove 'public' and 'abstract' modifiers in FsVolumeSpi and FsDatasetSpi (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit bd4e10900c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
 2015-07-08 16:36:12 +05:30
Xiaoyu Yao 907abdca73 HDFS-8711. setSpaceQuota command should print the available storage type when input storage type is wrong. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b68701b7b2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
 2015-07-08 16:25:47 +05:30
Haohui Mai b181c13cbf HDFS-8620. Clean up the checkstyle warinings about ClientProtocol. Contributed by Takanobu Asanuma. 2015-07-07 14:04:12 -07:00
Jing Zhao bed74b620a Revert "HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao." 
This reverts commit d62b63d297.

(cherry picked from commit bc99aaffe7)
 2015-07-07 10:14:17 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) a8bf34cad8 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8)
 2015-07-06 16:41:00 -07:00
Jing Zhao 3d73d2027f HDFS-8652. Track BlockInfo instead of Block in CorruptReplicasMap. Contributed by Jing Zhao. 
(cherry picked from commit d62b63d297)
 2015-07-06 15:55:03 -07:00
Vinayakumar B 9a774dfae5 HDFS-8686. WebHdfsFileSystem#getXAttr(Path p, final String name) doesn't work if namespace is in capitals (Contributed by kanaka kumar avvaru) 
(cherry picked from commit fc92d3e651)
 2015-07-06 16:09:42 +05:30
Vinayakumar B b963ca84bc HDFS-8577. Avoid retrying to recover lease on a file which does not exist (Contributed by J.Andreina) 
(cherry picked from commit 2eae130ab9)
 2015-07-03 13:36:07 +05:30
Andrew Wang efe9ae9fc6 HDFS-8709. Clarify automatic sync in FSEditLog#logEdit. 
(cherry picked from commit 5fddc5177d)
 2015-07-02 10:26:43 -07:00
Vinayakumar B 55bf014a07 HDFS-8703. Merge refactor of DFSInputStream from ErasureCoding branch (Contributed by Vinayakumar B) 
(cherry picked from commit bff5999d07)
 2015-07-02 16:12:13 +05:30
Colin Patrick Mccabe 490bef0856 HADOOP-12171. Shorten overly-long htrace span names for server (cmccabe) 
(cherry picked from commit a78d5074fb)
 2015-07-01 17:57:33 -07:00
Jing Zhao ff7562cc99 HDFS-8666. Speedup the TestMover test. Contributed by Walter Su. 
(cherry picked from commit 152e5df3b6)
 2015-07-01 15:00:16 -07:00
Arpit Agarwal d38f377cd3 HDFS-8706. Fix typo in datanode startup options in HDFSCommands.html. (Controibuted by Brahma Reddy Battula) 2015-07-01 10:36:10 -07:00
cnauroth f7ba3c0045 HADOOP-12116. Fix unrecommended syntax usages in hadoop/hdfs/yarn script for cygwin in branch-2. Contributed by Li Lu. 2015-06-30 16:55:10 -07:00
Colin Patrick Mccabe 7c698bc350 HADOOP-10798. globStatus() should always return a sorted list of files (cmccabe) 
(cherry picked from commit 68e588cbee)
 2015-06-30 16:42:36 -07:00
Colin Patrick Mccabe 604d1bbdb7 HDFS-8635. Migrate HDFS native build to new CMake framework (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit 29df73dc05)
 2015-06-30 16:01:48 -07:00
Vinayakumar B bc06d5b041 HADOOP-12154. FileSystem#getUsed() returns the file length only from root '/' (Contributed by J.Andreina) 
(cherry picked from commit 6d99017f38)
 2015-06-30 15:25:58 +05:30
Vinayakumar B 3c632c80a2 HDFS-8579. Update HDFS usage with missing options (Contributed by J.Andreina) 
(cherry picked from commit 77588e1d32)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
 2015-06-30 15:02:46 +05:30
Haohui Mai eccd4f273e HDFS-8493. Consolidate truncate() related implementation in a single class. Contributed by Rakesh R. 2015-06-29 16:44:58 -07:00
Arpit Agarwal ff78e611c0 HDFS-8687. Remove the duplicate usage message from Dfsck.java. (Contributed by Brahma Reddy Battula) 2015-06-29 14:56:49 -07:00
Yongjun Zhang 11bdcb754a HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang. 
(cherry picked from commit 43a1288fff)
 2015-06-29 14:48:17 -07:00
Andrew Wang ada7f66e12 HDFS-8653. Code cleanup for DatanodeManager, DatanodeDescriptor and DatanodeStorageInfo. Contributed by Zhe Zhang. 
(cherry picked from commit 2ffd84273a)
 2015-06-29 12:12:54 -07:00
Benoy Antony e40e9fc7f4 HDFS-7390. Provide JMX metrics per storage type. (Benoy Antony) 
(cherry picked from commit d3fed8e653)
 2015-06-29 11:02:43 -07:00
Vinayakumar B 438bb25d43 HDFS-8628. Update missing command option for fetchdt (Contributed by J.Andreina) 
(cherry picked from commit fde20ffcef)
 2015-06-29 16:51:15 +05:30
Vinayakumar B beab746177 HDFS-8586. Dead Datanode is allocated for write when client is from deadnode (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 88ceb382ef)
 2015-06-29 15:25:54 +05:30
Vinod Kumar Vavilapalli 4155bb565e Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5db)
 2015-06-28 16:30:27 -07:00
Arpit Agarwal 0fd47fa809 HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal) 2015-06-28 14:51:30 -07:00
Andrew Wang dd7776b2fe HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 
(cherry picked from commit 60b858bfa6)
 2015-06-26 11:33:40 -07:00
Jing Zhao 83d76151e2 HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang. 
(cherry picked from commit de480d6c89)
 2015-06-26 11:04:05 -07:00
Colin Patrick Mccabe 8552af91f4 HDFS-8651. Make hadoop-hdfs-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit 1b764a01fd)
 2015-06-26 10:28:14 -07:00
Andrew Wang 84fdd4a3a0 HDFS-8546. Use try with resources in DataStorage and Storage. 
(cherry picked from commit 1403b84b12)
 2015-06-25 17:50:39 -07:00
Andrew Wang ba406b723d HDFS-8665. Fix replication check in DFSTestUtils#waitForReplication. 
(cherry picked from commit ff0e5e572f)
 2015-06-25 17:29:30 -07:00
Arpit Agarwal 05e9ffdd62 HDFS-8640. Make reserved RBW space visible through JMX. (Contributed by kanaka kumar avvaru) 2015-06-25 10:13:31 -07:00
Akira Ajisaka 44dbddb022 HDFS-8462. Implement GETXATTRS and LISTXATTRS operations for WebImageViewer. Contributed by Jagadesh Kiran N. 
(cherry picked from commit bc433908d3)
 2015-06-26 00:22:30 +09:00
Andrew Wang 4b8593caf8 HDFS-8645. Resolve inconsistent code in TestReplicationPolicy between trunk and branch-2. Contributed by Zhe Zhang. 2015-06-24 18:26:14 -07:00
Andrew Wang 431f685300 HDFS-8646. Prune cached replicas from DatanodeDescriptor state on replica invalidation. 
(cherry picked from commit afe9ea3c12)
 2015-06-24 14:46:13 -07:00
Haohui Mai 0cd7603bd6 Addendum patch for HDFS-6564. 2015-06-23 22:30:27 -07:00
Jing Zhao 77609e6465 HDFS-8639. Add Option for NameNode HTTP port in MiniDFSClusterManager. Contributed by Kai Sasaki. 
(cherry picked from commit 2ba6465721)
 2015-06-23 22:21:01 -07:00
Haohui Mai 3fb192bfb7 HDFS-6564. Use slf4j instead of common-logging in hdfs-client. Contributed by Rakesh R. 2015-06-23 11:41:32 -07:00
Jakob Homan f6157a4a08 HDFS-8542. WebHDFS getHomeDirectory behavior does not match specification. Contributed by Kanaka Kumar Avvaru. 
(cherry picked from commit fac4e04dd3)
 2015-06-22 16:38:07 -07:00
Colin Patrick Mccabe cbd11681ce HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 7b424f938c)
 2015-06-22 14:37:35 -07:00
Andrew Wang c71f57b2a0 HDFS-8608. Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks). Contributed by Zhe Zhang. 
(cherry picked from commit 6e3fcffe29)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/UnderReplicatedBlocks.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
 2015-06-22 14:22:48 -07:00
Andrew Wang cd1e0930dc HDFS-4366. Block Replication Policy Implementation May Skip Higher-Priority Blocks for Lower-Priority Blocks. (Derek Dagit and Zhe Zhang via wang) 2015-06-22 14:18:33 -07:00
Arpit Agarwal 175c5829f0 HDFS-8192. Eviction should key off used locked memory instead of ram disk free space. (Contributed by Arpit Agarwal) 2015-06-20 13:27:58 -07:00
Arpit Agarwal 5dcda16859 HDFS-7164. Update CHANGES.txt 2015-06-20 13:09:21 -07:00
Yongjun Zhang 5774b1ff02 Move HDFS-7546 to release 2.7.1 in CHANGES.txt. 
(cherry picked from commit bcb3c40bed)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-20 00:49:17 -07:00
Allen Wittenauer 92cff5663e HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) 
(cherry picked from commit 63613c79c1)
 2015-06-20 00:40:44 -07:00
Yongjun Zhang 87c3d3739f Move HDFS-8337 to branch 2.8 in CHANGES.txt. 
(cherry picked from commit 20c03c9606)
 2015-06-20 00:28:48 -07:00
Yongjun Zhang d657e378ad HDFS-8337. Accessing httpfs via webhdfs doesn't work from a jar with kerberos. Contributed by Yongjun Zhang. 
(cherry picked from commit 971dc838ec)
 2015-06-19 23:44:48 -07:00
Colin Patrick Mccabe c94579f845 HDFS-8582. Support getting a list of reconfigurable config properties and do not generate spurious reconfig warnings (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit d112d18324)
 2015-06-19 19:14:38 -07:00
Arpit Agarwal fcd1824b61 HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru) 2015-06-19 18:23:44 -07:00
Yongjun Zhang 2e13e5b253 HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang. 
(cherry picked from commit b42f1ec13d)
 2015-06-19 09:50:14 -07:00
Andrew Wang 7f62e41b2e HDFS-8605. Merge Refactor of DFSOutputStream from HDFS-7285 branch. (vinayakumarb) 
(cherry picked from commit 1c13519e1e)
 2015-06-18 08:48:23 -07:00
Akira Ajisaka 998027652f HDFS-6249. Output AclEntry in PBImageXmlWriter. Contributed by surendra singh lilhore. 
(cherry picked from commit cc432885ad)
 2015-06-17 17:41:55 -07:00
Akira Ajisaka 9f9315008e HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1a169a26bc)
 2015-06-17 17:13:32 -07:00
Colin Patrick Mccabe 8939e3cc2d HDFS-8589. Fix unused imports in BPServiceActor and BlockReportLeaseManager (cmccabe) 
(cherry picked from commit 45ced38f10)
 2015-06-17 17:02:08 -07:00
Haohui Mai 38a16e1a45 HDFS-8446. Separate safemode related operations in GetBlockLocations(). Contributed by Haohui Mai. 2015-06-17 16:38:19 -07:00
Haohui Mai 1270cb47e1 Update CHANGES.txt for HDFS-8238. 2015-06-17 16:20:40 -07:00
Haohui Mai 09941e6f8d HDFS-8238. Move ClientProtocol to the hdfs-client. Contributed by Takanobu Asanuma. 2015-06-17 16:16:55 -07:00
Arpit Agarwal acc90ca175 HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal) 2015-06-16 19:06:39 -07:00
Tsz-Wo Nicholas Sze 64389f727c HDFS-8606. Cleanup DFSOutputStream by removing unwanted changes from HDFS-8386. Contributed by Rakesh R 2015-06-16 18:09:30 -07:00
Xiaoyu Yao 4bfbc6b1d4 HDFS-8551. Fix hdfs datanode CLI usage message. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 9cd5ad9d84)
 2015-06-16 14:17:51 -07:00
Xiaoyu Yao 8ab23d9ec1 HDFS-8548. Minicluster throws NPE on shutdown. Contributed by surendra singh lilhore. 
(cherry picked from commit 6a76250b39)
 2015-06-16 14:00:33 -07:00
Kihwal Lee d2d038b0a0 HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 
(cherry picked from commit c74517c46b)
 2015-06-16 15:42:00 -05:00
Xiaoyu Yao d7c8a6feaa HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4e88ff5b27)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-16 12:47:05 -07:00
Haohui Mai be8e4ed1e1 HDFS-8592. SafeModeException never get unwrapped. Contributed by Haohui Mai. 2015-06-15 22:07:42 -07:00
Yongjun Zhang b1d3ed02c1 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 2015-06-15 21:45:30 -07:00
Tsz-Wo Nicholas Sze 10b97f6655 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:18:09 -07:00
Tsz-Wo Nicholas Sze 95483513ed Move HDFS-8540 to 2.8 in CHANGES.txt. 2015-06-15 17:02:45 -07:00
Tsz-Wo Nicholas Sze 9ba29f081b HDFS-8540. Mover should exit with NO_MOVE_BLOCK if no block can be moved. Contributed by surendra singh lilhore 2015-06-15 16:28:37 -07:00
Tsz-Wo Nicholas Sze 5c111a5192 HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:09:32 -07:00
Arpit Agarwal ea9a9b3d7e HDFS-8607. TestFileCorruption doesn't work as expected. (Contributed by Walter Su) 2015-06-15 10:12:07 -07:00
Arpit Agarwal d07f3804b0 HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:36 -07:00
Yongjun Zhang cd578fa289 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
 2015-06-14 12:01:28 -07:00
cnauroth 92144ca75c HDFS-8593. Calculation of effective layout version mishandles comparison to current layout version in storage. Contributed by Chris Nauroth. 
(cherry picked from commit b8341f1cd8)
 2015-06-13 10:38:03 -07:00
Andrew Wang 2776255ed6 HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang. 
(cherry picked from commit 355e98ab7c7dd997378a8624f642199713467fb8)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
 2015-06-12 11:39:02 -07:00
Colin Patrick Mccabe 378bb484bb HDFS-7923. The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages (cmccabe) 
(cherry picked from commit 12b5b06c06)
 2015-06-12 11:29:05 -07:00
Akira Ajisaka e397cca456 HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to hadoop-common. Contributed by Masatake Iwasaki. 
(cherry picked from commit e4489d97e5)
 2015-06-12 07:26:05 -07:00
Haohui Mai f1ee2eaf86 HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 18:59:53 -07:00
Arpit Agarwal a560e654d9 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:50 -07:00
Colin Patrick Mccabe 754dd3496c HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbba)
 2015-06-11 14:28:41 -07:00
Andrew Wang 086eb49424 HDFS-8573. Move creation of restartMeta file logic from BlockReceiver to ReplicaInPipeline. Contributed by Eddy Xu. 
(cherry picked from commit b258b344bb)
 2015-06-11 10:12:40 -07:00
Andrew Wang 9523e2c098 HDFS-8549. Abort the balancer if an upgrade is in progress. 
(cherry picked from commit a7a7768341)
 2015-06-10 13:43:00 -07:00
Arpit Agarwal 2d67dff15f HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:46 -07:00
Xiaoyu Yao d8d9041187 HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing. Contributed by Rakesh R. 
(cherry picked from commit 3c2397cb04)
 2015-06-09 12:21:53 -07:00
Xiaoyu Yao 116a72096d HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc. Contributed by Brahma Reddy Battula 
(cherry picked from commit 927577c87c)
 2015-06-08 22:03:07 -07:00
cnauroth cc960734e4 HDFS-8553. Document hdfs class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d2832b3d42)
 2015-06-08 16:15:53 -07:00
cnauroth 8ee50d8ca7 HDFS-8554. TestDatanodeLayoutUpgrade fails on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit 0e80d51983)
 2015-06-08 08:39:14 -07:00
cnauroth 953c4a049a HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu Engineer. 
(cherry picked from commit a6cb4894b4)
 2015-06-07 13:02:01 -07:00
Xiaoyu Yao c8dc113943 HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b61b489995)
 2015-06-06 18:48:52 -07:00
cnauroth f99744b7a0 HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. 
(cherry picked from commit 71de367c5e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
 2015-06-06 09:45:46 -07:00
cnauroth 1d1841d40f HADOOP-12055. Deprecate usage of NativeIO#link. Contributed by Andrew Wang. 
(cherry picked from commit e46cb80002)
 2015-06-06 09:17:11 -07:00
Xiaoyu Yao 1a6cdf97bb HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:33:06 -07:00
Andrew Wang 320e4cc0e7 HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu. 
(cherry picked from commit b2540f486e)
 2015-06-04 15:35:12 -07:00
Andrew Wang ccc834395b HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R. 
(cherry picked from commit 5149dc7b97)
 2015-06-04 11:09:28 -07:00
Kihwal Lee f0f277ddd0 HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. 
(cherry picked from commit ade6d9a61e)
 2015-06-04 12:52:49 -05:00
Vinayakumar B 539ff92c2a HDFS-3716. Purger should remove stale fsimage ckpt files (Contributed by J.Andreina) 
(cherry picked from commit c59e745630)
 2015-06-03 15:31:01 +05:30
Vinayakumar B 0ed9c2d8fe HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 2015-06-03 15:15:44 +05:30
Vinayakumar B 4bb725b4dd HDFS-8470. fsimage loading progress should update inode, delegation token and cache pool count. (Contributed by surendra singh lilhore) 
(cherry picked from commit e965dcec37)
 2015-06-03 14:25:12 +05:30
Vinayakumar B 066e45bcb6 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd9)
 2015-06-03 12:12:22 +05:30
Colin Patrick Mccabe 12497b0543 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8)
 2015-06-02 20:06:53 -07:00
Andrew Wang a80a68c298 HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant. (wang) 
(cherry picked from commit c1d50a91f7)
 2015-06-02 15:48:31 -07:00
Andrew Wang 379ece15fe HDFS-8386. Improve synchronization of 'streamer' reference in DFSOutputStream. Contributed by Rakesh R. 
(cherry picked from commit efc510a570)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-06-02 15:42:27 -07:00
Colin Patrick Mccabe 17e369511d HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c6425)
 2015-06-02 11:42:08 -07:00
Vinayakumar B d1da842f86 HDFS-8256. '-storagepolicies , -blockId ,-replicaDetails ' options are missed out in usage and from documentation (Contributed by J.Andreina) 
(cherry picked from commit a2bd6217eb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2015-06-02 16:25:53 +05:30
Jing Zhao 640ea994ed HDFS-8489. Subclass BlockInfo to represent contiguous blocks. Contributed by Zhe Zhang. 
(cherry picked from commit cdc13efb1a)
 2015-06-01 11:45:04 -07:00
Tsuyoshi Ozawa b91cadcafd HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS. Contributed by Archana T. 
(cherry picked from commit c2356cb0bc)
 2015-06-01 00:42:11 +09:00
Andrew Wang bdaa0fed13 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 
(cherry picked from commit 7673d4f205)

 Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
	hadoop-mapreduce-project/pom.xml
 2015-05-29 14:55:52 -07:00
Jing Zhao 17fb442a4c HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a)
 2015-05-29 11:11:01 -07:00
Vinayakumar B 983c1d3969 HDFS-7401. Add block info to DFSInputStream' WARN message when it adds node to deadNodes (Contributed by Arshad Mohammad) 
(cherry picked from commit b75df697e0)
 2015-05-29 16:36:39 +05:30
Akira Ajisaka e7b81710dd HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. 
(cherry picked from commit d725dd8af6)
 2015-05-29 11:36:59 +09:00
Colin Patrick Mccabe 4b108a441d HADOOP-11894. Bump the version of Apache HTrace to 3.2.0-incubating (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit a1140959da)
 2015-05-28 12:03:07 -07:00
Colin Patrick Mccabe b9752f24a6 HDFS-8407. libhdfs hdfsListDirectory must set errno to 0 on success (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit d2d95bfe88)
 2015-05-28 11:38:44 -07:00
cnauroth 25db341278 HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. 
(cherry picked from commit 50eeea1300)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd
 2015-05-27 22:56:04 -07:00
Andrew Wang a634809895 HDFS-8482. Rename BlockInfoContiguous to BlockInfo. Contributed by Zhe Zhang. 
(cherry picked from commit ff83459f192f7cc4ce864f0b1f01e1e08243be83)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
 2015-05-27 15:42:12 -07:00
Haohui Mai 3f321be24d Addendum of HDFS-8135. 2015-05-27 13:14:51 -07:00
Haohui Mai cc348fead7 Revert HDFS-8135 in branch-2. 2015-05-27 11:06:17 -07:00
Akira Ajisaka 9670762a82 HADOOP-11242. Record the time of calling in tracing span of IPC server. Contributed by Mastake Iwasaki. 
(cherry picked from commit bb1816328a)
 2015-05-27 18:05:08 +09:00
Haohui Mai f70b9d9241 HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang. 2015-05-24 22:30:50 -07:00
Vinayakumar B 19895a41ad HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 
(cherry picked from commit 55ed6558ba)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
 2015-05-22 16:46:50 +05:30
Vinayakumar B 658f5cf985 HDFS-8268. Port conflict log for data node server is not sufficient (Contributed by Mohammad Shahid Khan) 
(cherry picked from commit 0c6638c2ea)
 2015-05-22 16:15:39 +05:30
Arpit Agarwal a073e5bd31 HDFS-8454. Remove unnecessary throttling in TestDatanodeDeath. (Arpit Agarwal) 2015-05-21 20:44:07 -07:00
Xiaoyu Yao 61d7ccb27e HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f2)
 2015-05-21 11:59:11 -07:00
Haohui Mai 0b909d028f HDFS-8421. Move startFile() and related functions into FSDirWriteFileOp. Contributed by Haohui Mai. 2015-05-21 08:06:13 -07:00
Akira Ajisaka b578f76bed HADOOP-10366. Add whitespaces between classes for values in core-default.xml to fit better in browser. Contributed by kanaka kumar avvaru. 
(cherry picked from commit 0e4f1081c7)
 2015-05-21 17:53:18 +09:00
Akira Ajisaka 185cddfd6a HDFS-4383. Document the lease limits. Contributed by Arshad Mohammad. 
(cherry picked from commit dc8434ab2b)
 2015-05-21 17:31:47 +09:00
Colin Patrick Mccabe 9534e533dc HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 470c87dbc6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
 2015-05-19 11:15:04 -07:00
Kihwal Lee 536b9ee6d6 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 
(cherry picked from commit 8860e352c3)
 2015-05-19 13:06:48 -05:00
Kihwal Lee 9e656bfa94 HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui. 
(cherry picked from commit de30d66b26)
 2015-05-19 08:06:07 -05:00
Vinayakumar B 5bbf157cb2 HADOOP-11103. Clean up RemoteException (Contributed by Sean Busbey) 
(cherry picked from commit d4a2830b63)
 2015-05-19 14:41:34 +05:30
Allen Wittenauer 3d0e2e54a1 HADOOP-8934. Shell command ls should include sort options (Jonathan Allen via aw) 
(cherry picked from commit 30b797ee9d)

Conflicts:
	hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
 2015-05-19 17:38:22 +09:00
Vinayakumar B 6dee42f6db HDFS-6348. SecondaryNameNode not terminating properly on runtime exceptions (Contributed by Rakesh R) 
(cherry picked from commit 93972a332a)
 2015-05-19 12:25:00 +05:30
Ravi Prakash e5b805d361 HDFS-4185. Add a metric for number of active leases (Rakesh R via raviprak) 
(cherry picked from commit cdfae446ad)
 2015-05-18 12:38:32 -07:00
Tsz-Wo Nicholas Sze 7bd4db968d HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 03:01:07 +08:00
Arpit Agarwal 9bcb740049 HDFS-8345. Storage policy APIs must be exposed via the FileSystem interface. (Arpit Agarwal) 2015-05-18 11:49:01 -07:00
Uma Maheswara Rao G ea875939f1 Revert from branch-2: HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R. 2015-05-18 14:38:30 +05:30
Arpit Agarwal 151f6d6dcc HDFS-8157. Writes to RAM DISK reserve locked memory for block files. (Arpit Agarwal) 2015-05-16 09:05:50 -07:00
Haohui Mai 46d62e7016 HDFS-8403. Eliminate retries in TestFileCreation#testOverwriteOpenForWrite. Contributed by Arpit Agarwal. 2015-05-15 19:12:46 -07:00
Haohui Mai 1796d94d62 HDFS-8394. Move getAdditionalBlock() and related functionalities into a separate class. Contributed by Haohui Mai. 2015-05-15 19:10:05 -07:00
Jing Zhao d6aa65d037 HDFS-8397. Refactor the error handling code in DataStreamer. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 8f37873342)
 2015-05-15 16:32:25 -07:00
Vinayakumar B d9455c790f HDFS-6888. Allow selectively audit logging ops (Contributed by Chen He) 
(cherry picked from commit 7f2e89fa7082840bfa3e8e593c93db050a80d04f)
 2015-05-15 11:05:26 +05:30
Akira Ajisaka 7fb3486eb3 HDFS-8350. Remove old webhdfs.xml and other outdated documentation stuff. Contributed by Brahma Reddy Battula. 2015-05-15 13:57:58 +09:00
Akira Ajisaka 2e5b7f24ab HDFS-8371. Fix test failure in TestHdfsConfigFields for spanreceiver properties. Contributed by Ray Chiang. 
(cherry picked from commit cbc01ed08e)
 2015-05-15 12:15:26 +09:00
Vinayakumar B 4bbcffa510 HDFS-8150. Make getFileChecksum fail for blocks under construction (Contributed by J.Andreina) 
(cherry picked from commit def9136e02)
 2015-05-14 15:55:20 +05:30
Haohui Mai 4b5915be9f HDFS-7728. Avoid updating quota usage while loading edits. Contributed by Jing Zhao. 2015-05-13 21:50:41 -07:00
Arpit Agarwal 33f6136ee3 HDFS-8243. Files written by TestHostsFiles and TestNameNodeMXBean are causing Release Audit Warnings. (Contributed by Ruth Wisniewski) 2015-05-13 19:46:34 -07:00
Colin Patrick Mccabe 52da75180a HDFS-8380. Always call addStoredBlock on blocks which have been shifted from one storage to another (cmccabe) 
(cherry picked from commit 281d47a969)
 2015-05-13 14:45:37 -07:00
cnauroth ddc4afeff2 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 2015-05-13 12:25:49 -07:00
Kihwal Lee 2e9392c157 HDFS-8358. TestTraceAdmin fails. Contributed by Masatake Iwasaki. 
(cherry picked from commit f9a46a00d2)
 2015-05-13 14:16:37 -05:00
Tsz-Wo Nicholas Sze bdf8f64be8 HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore 2015-05-13 11:58:35 -07:00
Vinayakumar B 2adb1257b1 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 
(cherry picked from commit 065d8f2a34)
 2015-05-13 17:30:40 +05:30
Akira Ajisaka 7596c6e731 HDFS-6184. Capture NN's thread dump when it fails over. Contributed by Ming Ma. 
(cherry picked from commit 2463666ecb)
 2015-05-13 11:37:59 +09:00
yliu c31c6fbda7 HDFS-8255. Rename getBlockReplication to getPreferredBlockReplication. (Contributed by Zhe Zhang) 2015-05-12 21:26:57 +08:00
Vinayakumar B e1b8bb9d84 HDFS-8362. Java Compilation Error in TestHdfsConfigFields.java (Contributed by Arshad Mohammad) 
(cherry picked from commit 987abc99b0)
 2015-05-12 12:10:14 +05:30
Kihwal Lee 50778f9d45 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 
(cherry picked from commit ea11590aad)
 2015-05-11 14:33:45 -05:00
Junping Du a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 
(cherry picked from commit 7e543c27fa)
 2015-05-11 05:28:25 -07:00
Akira Ajisaka 703f1e9d16 HDFS-8351. Remove namenode -finalize option from document. (aajisaka) 
(cherry picked from commit 3fa2efc09f)
 2015-05-11 15:37:12 +09:00
Haohui Mai 1f499a61dc HDFS-8357. Consolidate parameters of INode.CleanSubtree() into a parameter objects. Contributed by Li Lu. 2015-05-09 22:52:07 -07:00
Haohui Mai 300d36cee7 HDFS-8327. Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate(). Contributed by Haohui Mai. 2015-05-08 23:09:57 -07:00
Haohui Mai 51bfb66ec8 Add missing entry in CHANGES.txt for HDFS-6757. 2015-05-08 23:09:55 -07:00
Haohui Mai 73d61f7ae7 HDFS-6757. Simplify lease manager with INodeID. Contributed by Haohui Mai. 2015-05-08 23:04:57 -07:00
Arpit Agarwal 969a8f6c96 HDFS-8097. TestFileTruncate is failing intermittently. (Contributed by Rakesh R) 2015-05-08 16:29:01 -07:00
Kihwal Lee f264a5aeed HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 
(cherry picked from commit 2d4ae3d18b)
 2015-05-08 16:37:26 -05:00
Colin Patrick Mccabe ab80a572be HDFS-8113. Add check for null BlockCollection pointers in BlockInfoContiguous structures (Chengbing Liu via Colin P. McCabe) 
(cherry picked from commit f523e963e4)
 2015-05-08 14:00:13 -07:00
Yongjun Zhang fbb2cf54bc HDFS-8311. DataStreamer.transfer() should timeout the socket InputStream. (Esteban Gutierrez via Yongjun Zhang) 
(cherry picked from commit 730f9930a4)
 2015-05-08 01:38:01 -07:00
Kihwal Lee 30cb540b4d HDFS-7433. Optimize performance of DatanodeManager's node map. Contributed by Daryn Sharp. 
(cherry picked from commit 7a7960be41)
 2015-05-08 15:20:14 -05:00
Colin Patrick Mccabe d83ae68bb0 HDFS-8284. Update documentation about how to use HTrace with HDFS (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 8f7c2364d7)
 2015-05-08 12:30:34 -07:00
cnauroth 5c188908fe HDFS-5640. Add snapshot methods to FileContext. Contributed by Rakesh R. 
(cherry picked from commit 26f61d41df)
 2015-05-08 11:21:02 -07:00
Arun Suresh 2124742e07 HDFS-7559. Create unit test to automatically compare HDFS related classes and hdfs-default.xml. (Ray Chiang via asuresh) 
(cherry picked from commit 3cefc02af7)
 2015-05-08 10:31:23 -07:00
Arpit Agarwal 68d85e92bf HDFS-8340. Fix NFS documentation of nfs.wtmax. (Contributed by Ajith S) 2015-05-08 10:15:35 -07:00
Arpit Agarwal a0905ae57a HDFS-8274. NFS configuration nfs.dump.dir not working (Contributed by Ajith S) 2015-05-08 10:04:11 -07:00
Haohui Mai d4d7d7bf22 HDFS-8346. libwebhdfs build fails during link due to unresolved external symbols. Contributed by Chris Nauroth. 2015-05-08 09:58:00 -07:00
Kihwal Lee 802a5775f3 HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6f622672b6)
 2015-05-08 09:32:07 -05:00
Uma Maheswara Rao G faf26f081c HDFS-6285. tidy an error log inside BlockReceiver. Contributed by Liang Xie. 
(cherry picked from commit 7b1ea9c481)
 2015-05-08 17:25:07 +05:30
Vinayakumar B d129bbbb37 HDFS-3384. DataStreamer thread should be closed immediatly when failed to setup a PipelineForAppendOrRecovery (Contributed by Uma Maheswara Rao G) 
(cherry picked from commit c648317a68)
 2015-05-08 17:18:59 +05:30
Vinayakumar B a87ab77fec HDFS-6576. Datanode log is generating at root directory in security mode (Contributed by surendra singh lilhore) 2015-05-08 16:37:36 +05:30
Vinayakumar B 3ee1b50dad HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
Added one missed change, missed while resolving conflict
 2015-05-08 15:39:47 +05:30
Vinayakumar B 3a894b4563 HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
(cherry picked from commit e01c1a979c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md
 2015-05-08 15:34:17 +05:30
Vinayakumar B eaa5f3e448 HDFS-8209. Support different number of datanode directories in MiniDFSCluster. (Contributed by surendra singh lilhore) 
(cherry picked from commit 4c6816faf8)
 2015-05-08 15:04:58 +05:30
Tsuyoshi Ozawa 8109f29ee2 HDFS-8207. Improper log message when blockreport interval compared with initial delay. Contributed by Brahma Reddy Battula and Ashish Singhi. 
(cherry picked from commit 888e630990)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-05-08 18:14:08 +09:00
Vinayakumar B 595b92e2c2 HDFS-8175. Provide information on snapshotDiff for supporting the comparison between snapshot and current status (Contributed by J.Andreina) 
(cherry picked from commit 6232235df3)
 2015-05-08 14:38:53 +05:30
Vinayakumar B 10999ffd7a HDFS-8187. Remove usage of '-setStoragePolicy' and '-getStoragePolicy' using dfsadmin cmd (as it is not been supported) (Contributed by J.Andreina) 
(cherry picked from commit e642eff0e0)
 2015-05-08 14:26:58 +05:30
Uma Maheswara Rao G a56f57b421 HDFS-8108. Fsck should provide the info on mandatory option to be used along with -blocks ,-locations and -racks. Contributed by J.Andreina. 
(cherry picked from commit 4b00ae5c11)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2015-05-08 14:23:42 +05:30
Akira Ajisaka 143c9a8ff2 HDFS-8222. Remove usage of "dfsadmin -upgradeProgress" from document which is no longer supported. Contributed by J.Andreina. 
(cherry picked from commit fd605997ee)
 2015-05-08 17:23:25 +09:00
Vinayakumar B 1f78502416 HDFS-7998. HDFS Federation : Command mentioned to add a NN to existing federated cluster is wrong (Contributed by Ajith S) 
(cherry picked from commit 26da2ddf38)
 2015-05-08 13:47:06 +05:30
Uma Maheswara Rao G b7a5ebd566 HDFS-8332. DFS client API calls should check filesystem closed. Contributed by Rakesh R. Merge from trunk 2015-05-08 13:46:04 +05:30
Vinayakumar B 089d420a82 HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun) 
(cherry picked from commit e5e492a963)
 2015-05-08 11:26:56 +05:30
Uma Maheswara Rao G 7fe1d74005 HDFS-8174. Update replication count to live rep count in fsck report. Contributed by J.Andreina 
(cherry picked from commit 2ea0f2fc93)
 2015-05-08 11:07:58 +05:30
Vinayakumar B d0fff5d328 HDFS-8067. haadmin prints out stale help messages (Contributed by Ajith S) 
(cherry picked from commit 66988476d0)
 2015-05-08 10:47:23 +05:30
Vinayakumar B 00954714a6 HDFS-8257. Namenode rollingUpgrade option is incorrect in document (Contributed by J.Andreina) 
(cherry picked from commit c7c26a1e4a)
 2015-05-08 10:36:09 +05:30
Haohui Mai a3abe8d7e4 HDFS-8037. CheckAccess in WebHDFS silently accepts malformed FsActions parameters. Contributed by Walter Su. 2015-05-07 15:25:36 -07:00
Haohui Mai f6b908ab07 HDFS-8321. CacheDirectives and CachePool operations should throw RetriableException in safemode. Contributed by Haohui Mai. 2015-05-07 15:17:02 -07:00
Tsz-Wo Nicholas Sze 4e1f2eb395 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:37:33 -07:00
Akira Ajisaka ab7dab55ea HDFS-8325. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 449e4426a5)
 2015-05-07 15:15:12 +09:00
cnauroth 276e2d3390 HDFS-7833. DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6633a8474d)
 2015-05-06 21:11:44 -07:00
Konstantin V Shvachko 193615c4b2 HDFS-2484. checkLease should throw FileNotFoundException when file does not exist. Contributed by Rakesh R. 2015-05-06 16:11:12 -07:00
Colin Patrick Mccabe f79b1f0190 HDFS-7847. Modify NNThroughputBenchmark to be able to operate on a remote NameNode (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit ffce9a3413)
 2015-05-06 10:50:33 -07:00
Xiaoyu Yao bd88215393 HDFS-8310. Fix TestCLI.testAll 'help: help for find' on Windows. (Kiran Kumar M R via Xiaoyu Yao) 
(cherry picked from commit 7a26d174af)
 2015-05-06 10:27:07 -07:00
Haohui Mai a4f868f0b8 HDFS-8314. Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users. Contributed by Li Lu. 2015-05-05 15:56:08 -07:00
Xiaoyu Yao e68e8b3b5c HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao) 
(cherry picked from commit 0100b15501)
 2015-05-05 13:42:22 -07:00
Colin Patrick Mccabe 1f6bcf94cc HDFS-7758. Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 24d3a2d4fd)
 2015-05-05 11:09:28 -07:00
Colin Patrick Mccabe 1f01d8347a HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 
(cherry picked from commit fcd4cb7516)
 2015-05-05 11:06:47 -07:00
Vinayakumar B 01bdfd794c HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit

(cherry picked from commit 318081ccd7)
 2015-05-05 11:06:44 +05:30
Haohui Mai db0bd6dca8 HDFS-8237. Move all protocol classes used by ClientProtocol to hdfs-client. Contributed by Haohui Mai. 2015-05-04 15:02:49 -07:00
Colin Patrick Mccabe 8efcf5bdbd HDFS-7397. Add more detail to the documentation for the conf key "dfs.client.read.shortcircuit.streams.cache.size" (Brahma Reddy Battula via Colin P. McCabe) 
(cherry picked from commit 3fe79e1db8)
 2015-05-04 12:50:59 -07:00
cnauroth a43080a2cf HDFS-8290. WebHDFS calls before namesystem initialization can cause NullPointerException. Contributed by Chris Nauroth. 
(cherry picked from commit c4578760b6)
 2015-05-04 11:35:18 -07:00
Xiaoyu Yao 5b27d9e076 HDFS-8309. Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows. (xyao) 
(cherry picked from commit a319771d1d)
 2015-05-02 22:32:28 -07:00
Haohui Mai 65a19fbc30 HDFS-8249. Separate HdfsConstants into the client and the server side class. Contributed by Haohui Mai. 2015-05-02 10:03:04 -07:00
Jonathan Eagles 9ed6b77da1 HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles) 
(cherry picked from commit 2d7363b273)

Conflicts:
	pom.xml
 2015-05-01 19:04:41 -07:00
Arpit Agarwal 8e3fdd0a40 HDFS-8229. LAZY_PERSIST file gets deleted after NameNode restart. (Contributed by Surendra Singh Lilhore) 2015-05-01 16:42:35 -07:00
Tsz-Wo Nicholas Sze 72751b957c HDFS-8086. Move LeaseRenewer to the hdfs.client.impl package. Contributed by Takanobu 2015-05-01 15:14:01 -07:00
Arun Suresh 8b6fc83ef4 Updating CHANGES.txt - Pulling in HDFS-8091 to branch-2.7 (for 2.7.1) 
(cherry picked from commit 3393461197)
 2015-05-01 11:49:38 -07:00
Colin Patrick Mccabe 9edea9507d HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 
(cherry picked from commit b82567d455)
 2015-05-01 11:47:28 -07:00
Arpit Agarwal aa81b86898 HDFS-8276. LazyPersistFileScrubber should be disabled if scrubber interval configured zero. (Contributed by Surendra Singh Lilhore) 2015-05-01 11:11:56 -07:00
Jing Zhao ec7b392965 HDFS-8300. Fix unit test failures and findbugs warning caused by HDFS-8283. Contributed by Jing Zhao. 
(cherry picked from commit 98a6176628)
 2015-04-30 19:29:11 -07:00
Andrew Wang 34d686e6e9 HDFS-8292. Move conditional in fmt_time from dfs-dust.js to status.html. Contributed by Charles Lamb. 
(cherry picked from commit 87e9978235)
 2015-04-30 18:23:21 -07:00
Haohui Mai 01acf3a3af HDFS-8200. Refactor FSDirStatAndListingOp. Contributed by Haohui Mai. 2015-04-30 13:41:55 -07:00
Akira Ajisaka fdb886d656 HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao. 
(cherry picked from commit de9404f02f)
 2015-04-30 20:13:18 +09:00
Akira Ajisaka 2f2d0aa990 HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. 
(cherry picked from commit fd06186071bb369ee045b57a82419566e87257ec)
 2015-04-30 19:11:46 +09:00
Andrew Wang 9f8412ac6e HDFS-8214. Secondary NN Web UI shows wrong date for Last Checkpoint. Contributed by Charles Lamb. 
(cherry picked from commit aa22450442)
 2015-04-29 17:38:09 -07:00
Haohui Mai 460127e6f2 HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. 2015-04-29 11:12:54 -07:00
Jing Zhao ecdebb7369 HDFS-8283. DataStreamer cleanup and some minor improvement. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 7947e5b53b)
 2015-04-29 10:43:52 -07:00
Haohui Mai fc9e7d7a24 HDFS-8280. Code Cleanup in DFSInputStream. Contributed by Jing Zhao. 2015-04-28 18:12:05 -07:00
Haohui Mai 7bbe9495c4 HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai. 2015-04-28 18:06:08 -07:00
Tsz-Wo Nicholas Sze 9b62a8ed77 HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su 2015-04-28 13:06:30 -07:00
cnauroth a82addd614 HDFS-8232. Missing datanode counters when using Metrics2 sink interface. Contributed by Anu Engineer. 
(cherry picked from commit feb68cb547)
 2015-04-27 16:48:25 -07:00
Arpit Agarwal de85ff58dc HDFS-8205. Fix CHANGES.txt 2015-04-27 12:30:55 -07:00
Arpit Agarwal c254a2a671 HDFS-8205. CommandFormat#parse() should not parse option as value of option. (Contributed by Peter Shi and Xiaoyu Yao) 2015-04-27 12:25:45 -07:00
Xiaoyu Yao 7aa3399cc5 HDFS-8206. Fix the typos in hadoop-hdfs-httpfs. (Brahma Reddy Battula via xyao) 2015-04-25 22:24:38 -07:00
cnauroth 8d256c56db HDFS-8247. TestDiskspaceQuotaUpdate#testAppendOverTypeQuota is failing. Contributed by Xiaoyu Yao. 
(cherry picked from commit a00e001a1a)
 2015-04-25 11:37:41 -07:00
Arpit Agarwal 932cff610a HDFS-8211. DataNode UUID is always null in the JMX counter. (Contributed by Anu Engineer) 2015-04-24 16:47:56 -07:00
Andrew Wang 89a15d6074 HDFS-8191. Fix byte to integer casting in SimulatedFSDataset#simulatedByte. Contributed by Zhe Zhang. 
(cherry picked from commit c7d9ad68e3)
 2015-04-24 11:54:48 -07:00
Jing Zhao 5870d504e1 HDFS-8176. Record from/to snapshots in audit log for snapshot diff report. Contributed by J. Andreina. 
(cherry picked from commit cf6c8a1b4e)
 2015-04-24 10:24:12 -07:00
Vinayakumar B 2ec356fcdb HDFS-8231. StackTrace displayed at client while QuotaByStorageType exceeds (Contributed by J.Andreina and Xiaoyu Yao) 
(cherry picked from commit c8d72907ff)
 2015-04-24 12:51:56 +05:30
Vinayakumar B 68063cac3e HDFS-8217. During block recovery for truncate Log new Block Id in case of copy-on-truncate is true. (Contributed by Vinayakumar B) 
(cherry picked from commit 262c1bc339)
 2015-04-24 12:18:04 +05:30
Colin Patrick Mccabe 3d0385c3c8 HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) 
(cherry picked from commit a8898445dc)
 2015-04-23 19:15:33 -07:00
Colin Patrick Mccabe 788b76761d HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) 
(cherry picked from commit a0e0a63209)
 2015-04-23 19:08:20 -07:00
Haohui Mai b64bb2b9b4 HDFS-8052. Move WebHdfsFileSystem into hadoop-hdfs-client. Contributed by Haohui Mai. 2015-04-23 17:44:21 -07:00
Tsz-Wo Nicholas Sze f27a976a2f HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma 2015-04-23 14:23:02 -07:00
Tsz-Wo Nicholas Sze d6c8609663 HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore 2015-04-23 11:56:09 -07:00
Arun Suresh 889b92fa40 HDFS-4448. Allow HA NN to start in secure mode with wildcard address configured (atm via asuresh) 
(cherry picked from commit baf8bc6c48)
 2015-04-23 01:45:33 -07:00
Haohui Mai 71c9f2f4e7 HDFS-8218. Move classes that used by ClientProtocol into hdfs-client. Contributed by Haohui Mai. 2015-04-22 11:33:32 -07:00
Haohui Mai f02ca4ab15 HDFS-8185. Separate client related routines in HAUtil into a new class. Contributed by Haohui Mai. 2015-04-21 22:09:13 -07:00
Colin Patrick Mccabe 447f2f699e HDFS-8133. Improve readability of deleted block check (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 997408eaac)
 2015-04-21 12:06:33 -07:00
Arpit Agarwal 7b3acc5c9d HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal) 2015-04-21 10:58:12 -07:00
Vinayakumar B e7bb0fc922 HDFS-7993. Provide each Replica details in fsck (Contributed by J.Andreina) 
(cherry picked from commit 8ddbb8dd43)
 2015-04-21 15:27:15 +05:30
Vinod Kumar Vavilapalli 27fc4fb998 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:01 -07:00
Vinayakumar B 961051e569 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Updated CHANGES.TXT for correct version

(cherry picked from commit ed4137cebf)
 2015-04-21 08:00:37 +05:30
Arpit Agarwal 95a8d452c5 HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao) 2015-04-20 15:42:49 -07:00
Vinayakumar B c562e1b19b HDFS-8173. NPE thrown at DataNode shutdown when HTTP server was not able to create (Contributed by surendra singh lilhore) 
(cherry picked from commit c17cd4f7c1)
 2015-04-20 14:58:59 +05:30
Haohui Mai 5d3a4d51bd HDFS-8169. Move LocatedBlocks and related classes to hdfs-client. Contributed by Haohui Mai. 2015-04-20 00:36:19 -07:00
Arpit Agarwal 490fac391c HDFS-8152. Refactoring of lazy persist storage cases. (Arpit Agarwal) 2015-04-19 16:09:12 -07:00
Tsuyoshi Ozawa 890238b496 HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5459b241c8)
 2015-04-19 18:23:57 +09:00
Tsuyoshi Ozawa e083c07913 HDFS-7863. Missing description of some methods and parameters in javadoc of FSDirDeleteOp. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5112477d9e)
 2015-04-19 18:07:19 +09:00
Haohui Mai a825243280 HDFS-8165. Move GRANDFATHER_GENERATION_STAMP and GRANDFATER_INODE_ID to hdfs-client. Contributed by Haohui Mai. 2015-04-16 22:54:33 -07:00
Jitendra Pandey 04962efe43 HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer. 
cherry picked from 369ddc67bd
 2015-04-16 22:15:57 -07:00
Tsz-Wo Nicholas Sze fa7b18e387 HDFS-7891. A block placement policy with best rack failure tolerance. Contributed by Walter Su 2015-04-16 18:26:46 -07:00
Akira Ajisaka d363f853ab Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:50:04 -05:00
Tsz-Wo Nicholas Sze bad547d969 HDFS-8082. Move dfs.client.read.*, dfs.client.short.circuit.*, dfs.client.mmap.* and dfs.client.hedged.read.* conf from DFSConfigKeys to HdfsClientConfigKeys. 2015-04-16 13:27:15 -07:00
Haohui Mai e1ebad8203 HDFS-8135. Remove the deprecated FSConstants class. Contributed by Li Lu. 2015-04-16 11:26:13 -07:00
Andrew Wang 9123f26c73 HDFS-8142. DistributedFileSystem encryption zone commands should resolve relative paths. Contributed by Rakesh R. 
(cherry picked from commit 2e8ea780a4)
 2015-04-16 07:14:53 -07:00
Akira Ajisaka 6d183561ff HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula. 
(cherry picked from commit de0f1700c1)
 2015-04-15 15:32:08 -05:00
Jing Zhao 725621d95a HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595)
 2015-04-15 13:11:15 -07:00
Jing Zhao d4dd97eabd HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473b)
 2015-04-15 12:39:09 -07:00
Arpit Agarwal 96e12fa46a HDFS-8144. Split TestLazyPersistFiles into multiple tests. (Arpit Agarwal) 2015-04-15 10:25:16 -07:00
Andrew Wang 871bf6a765 HDFS-8117. More accurate verification in SimulatedFSDataset: replace DEFAULT_DATABYTE with patterned data. Contributed by Zhe Zhang. 2015-04-15 08:43:42 -07:00
Jing Zhao 38b031d6ba HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d)
 2015-04-14 15:12:37 -07:00
cnauroth c7ebecfff5 HDFS-8055. NullPointerException when topology script is missing. Contributed by Anu Engineer. 
(cherry picked from commit fef596df03)
 2015-04-14 10:19:45 -07:00
cnauroth 9779b79ed8 HDFS-6666. Abort NameNode and DataNode startup if security is enabled but block access token is not enabled. Contributed by Vijay Bhat. 
(cherry picked from commit d45aa7647b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSecureNameNode.java
 2015-04-14 10:03:53 -07:00
Arpit Agarwal d77705c980 HDFS-7701. Support reporting per storage type quota and usage with hadoop/hdfs shell. (Contributed by Peter Shi) 2015-04-13 21:20:30 -07:00
Allen Wittenauer b7a052c84a HADOOP-7713. dfs -count -q should label output column (Jonathan Allen via aw) 2015-04-13 21:18:59 -07:00
Tsz-Wo Nicholas Sze e4fe5fa2ee HDFS-8083. Move dfs.client.write.* conf from DFSConfigKeys to HdfsClientConfigKeys.Write. 2015-04-13 11:45:03 -07:00
Vinayakumar B 1071a57b20 HDFS-8111. NPE thrown when invalid FSImage filename given for 'hdfs oiv_legacy' cmd ( Contributed by surendra singh lilhore ) 
(cherry picked from commit 14384f5b51)
 2015-04-13 17:19:18 +05:30
cnauroth 293c763e82 HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. 
(cherry picked from commit f8f5887209)
 2015-04-11 13:24:06 -07:00
Tsz-Wo Nicholas Sze ef2bd4bd11 HDFS-8084. Move dfs.client.failover.* confs from DFSConfigKeys to HdfsClientConfigKeys.Failover and fix typos in the dfs.http.client.* configuration keys. 2015-04-10 19:58:42 -07:00
Haohui Mai 6f97b780df HDFS-8103. Move BlockTokenSecretManager.AccessMode into BlockTokenIdentifier. Contributed by Haohui Mai. 2015-04-10 16:36:57 -07:00
Tsz-Wo Nicholas Sze 1113aca7f8 HDFS-8100. Refactor DFSClient.Conf to a standalone class and separates short-circuit related conf to ShortCircuitConf. 2015-04-10 14:49:32 -07:00
Arun Suresh 3d17c50176 HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 
(cherry picked from commit 7660da95cb)
 2015-04-10 11:42:18 -07:00
Vinayakumar B 4aba069b37 HDFS-7939. Two fsimage_rollback_* files are created which are not deleted after rollback. (Contributed by J.Andreina) 
(cherry picked from commit 987c9e12e1)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-10 11:27:20 +05:30
Konstantin V Shvachko 0e99da620d HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:02:14 -07:00
Haohui Mai e34afb48d4 HDFS-8102. Separate webhdfs retry configuration keys from DFSConfigKeys. Contributed by Haohui Mai. 2015-04-09 14:36:35 -07:00
Vinod Kumar Vavilapalli aec1c5ec7d Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1e)
 2015-04-09 13:56:14 -07:00
Arun Suresh c846e84d55 HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh) 
(cherry picked from commit 922b7ed21d)
 2015-04-09 13:13:23 -07:00
Colin Patrick Mccabe 8dac245920 HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 61dc2ea3fe)
 2015-04-09 11:42:00 -07:00
Colin Patrick Mccabe ca12088253 HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit 30acb7372a)
 2015-04-09 11:23:01 -07:00
Aaron T. Myers edf2f52d6d HDFS-8101. DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime. Contributed by Sean Busbey. 
(cherry picked from commit 3fe61e0bb0)
 2015-04-09 09:45:02 -07:00
Kihwal Lee 6d1cb34221 Fix CHANGES.txt for branch-2 2015-04-09 10:05:43 -05:00
Vinayakumar B 4e099c113b HDFS-8096. DatanodeMetrics#blocksReplicated will get incremented early and even for failed transfers (Contributed by Vinayakumar B) 
(cherry picked from commit 9d8952f97f)
 2015-04-09 11:59:05 +05:30
Andrew Wang 351fac25a9 HDFS-7979. Initialize block report IDs with a random number. 
(cherry picked from commit b1e059089d)
 2015-04-08 21:43:57 -07:00
Haohui Mai 88ef75f167 HDFS-8089. Move o.a.h.hdfs.web.resources.* to the client jars. Contributed by Haohui Mai. 2015-04-08 16:30:57 -07:00
Andrew Wang 069366e1be HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma. 2015-04-08 16:09:28 -07:00
Haohui Mai 6d2eca081d Revert HDFS-7813. 2015-04-08 16:03:20 -07:00
Haohui Mai 35816c438d Revert HDFS-7808. 2015-04-08 16:00:10 -07:00
Andrew Wang 8104d52269 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 2015-04-08 15:52:15 -07:00
Andrew Wang 9792500c54 HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. 
(cherry picked from commit a42bb1cd91)
 2015-04-08 15:41:54 -07:00
Kihwal Lee 7e622076d4 HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee. 
(cherry picked from commit 285b31e75e)
 2015-04-08 15:39:25 -05:00
Tsz-Wo Nicholas Sze de7f9a8bcc HDFS-8079. Move CorruptFileBlockIterator to a new hdfs.client.impl package. 2015-04-08 11:51:43 -07:00
Arpit Agarwal f0324738c9 HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal) 2015-04-08 11:38:30 -07:00
Haohui Mai f9fbde3074 HDFS-8080. Separate JSON related routines used by WebHdfsFileSystem to a package local class. Contributed by Haohui Mai. 2015-04-07 21:30:23 -07:00
Vinayakumar B 3cf7ac181b HDFS-8073. Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided. (Contributed by Walter Su) 
(cherry picked from commit d505c8acd3)
 2015-04-08 09:57:49 +05:30
Tsz-Wo Nicholas Sze eb3ca514fb HDFS-8079. Move dfs.client.retry.* confs from DFSConfigKeys to HdfsClientConfigKeys.Retry. 2015-04-07 19:53:30 -07:00
Yongjun Zhang b76c321e94 HDFS-5215. dfs.datanode.du.reserved is not considered while computing 
available space ( Brahma Reddy Battula via Yongjun Zhang)

(cherry picked from commit 66763bb06f)
 2015-04-07 19:36:02 -07:00
cnauroth 8921526440 HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao. 
(cherry picked from commit 672ed462965ea7da10ce3df48c02e2a3bd13f0ae)
 2015-04-07 14:23:29 -07:00
Tsz-Wo Nicholas Sze a4f0eea261 HDFS-8049. Add @InterfaceAudience.Private annotation to hdfs client implementation. Contributed by Takuya Fukudome 2015-04-07 14:01:55 -07:00
Konstantin V Shvachko 81b522790b HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko. 2015-04-06 22:23:33 -07:00
Colin Patrick Mccabe a827089905 HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 
(cherry picked from commit 28bebc81db)
 2015-04-06 09:10:15 -07:00
Vinod Kumar Vavilapalli 00854d2c2f HADOOP-11776. Fixed the broken JDiff support in Hadoop 2. Contributed by Li Lu. 
(cherry picked from commit 4b3948ea36)
 2015-04-04 13:52:51 -07:00
Vinod Kumar Vavilapalli 4eec6e09d0 HADOOP-11377. Added Null.java without which jdiff completely flops. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit 5370e7128b)
 2015-04-04 13:48:14 -07:00
Colin Patrick Mccabe b26ba22a90 HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a)
 2015-04-03 16:56:25 -07:00
Colin Patrick Mccabe 0c5069c432 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9)
 2015-04-03 14:26:45 -07:00
cnauroth 28e0602e7b HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e42891)
 2015-04-03 10:44:32 -07:00
cnauroth 0dde8c079d HDFS-8034. Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit dc5596c36a)
 2015-04-03 10:10:29 -07:00
Jing Zhao 2c1f33d178 HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:22:43 -07:00
Haohui Mai 1278310c21 HDFS-7893. Update the POM to create a separate hdfs-client jar. Contributed by Haohui Mai. 2015-04-02 15:39:40 -07:00
Haohui Mai 6e36dbf03a HDFS-8035. Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager. Contributed by Haohui Mai. 2015-04-02 11:38:02 -07:00
Tsz-Wo Nicholas Sze 81ed3a2bb8 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:12:27 -07:00
Tsz-Wo Nicholas Sze ce3f32590d HDFS-7888. Change DFSOutputStream and DataStreamer for convenience of subclassing. Contributed by Li Bo 2015-04-02 11:00:47 -07:00
Vinayakumar B beb0fd0d60 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675)
 2015-04-02 08:12:51 +05:30
Brandon Li 27970b3f46 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c26)
 2015-04-01 17:25:28 -07:00
Haohui Mai cfcf795492 HDFS-8008. Support client-side back off when the datanodes are congested. Contributed by Haohui Mai. 2015-04-01 16:54:53 -07:00
Haohui Mai 9e114ee607 HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:09 -07:00
Colin Patrick Mccabe 6f753da4a9 HDFS-7922. ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors (Rakesh R via Colin P. McCabe) 
(cherry picked from commit 3c7adaaf35)
 2015-04-01 16:11:44 -07:00
Colin Patrick Mccabe 4b74aa7182 HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk (cmccabe) 
(cherry picked from commit c94d594a57)
 2015-04-01 14:10:25 -07:00
Andrew Wang b17d8a53fa HDFS-7978. Add LOG.isDebugEnabled() guard for some LOG.debug(..). Contributed by Walter Su. 
(cherry picked from commit ed72daa5df)
 2015-04-01 12:54:46 -07:00
Haohui Mai 2ca2515b33 Add the missing files for HDFS-8009. 2015-04-01 10:58:41 -07:00
Haohui Mai a5bcfe0d33 HDFS-8009. Signal congestion on the DataNode. Contributed by Haohui Mai. 2015-04-01 10:57:07 -07:00
Akira Ajisaka b85bbca745 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df)
 2015-04-01 09:08:22 +09:00
Akira Ajisaka 7f6811ad73 HDFS-7671. hdfs user guide should point to the common rack awareness doc. Contributed by Kai Sasaki. 
(cherry picked from commit 859cab2f22)
 2015-04-01 00:26:54 +09:00
yliu e4cd67e9b0 HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) 2015-03-31 21:16:13 +08:00
Arpit Agarwal 312e50f9c4 HDFS-7944. Minor cleanup of BlockPoolManager#getAllNamenodeThreads. (Arpit Agarwal) 2015-03-30 21:24:59 -07:00
Harsh J 32766b6563 HDFS-3918. EditLogTailer shouldn't log WARN when other node is in standby mode. Contributed by Todd Lipcon. 
(cherry picked from commit cce66ba3c9)
 2015-03-31 08:17:22 +05:30
Arpit Agarwal abf3ad988d HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 2015-03-30 15:25:26 -07:00
Haohui Mai 02fcf622fb Addendum for HDFS-7748. 2015-03-30 13:11:24 -07:00
Haohui Mai dd5b2dac5a HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 11:59:32 -07:00
Colin Patrick Mccabe 02ed22cd2d HDFS-7261. storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState() (Brahma Reddy Battula via Colin P. McCabe) 
(cherry picked from commit 1feb9569f3)
 2015-03-30 10:53:57 -07:00
Akira Ajisaka d4bb9b2146 HDFS-8002. Website refers to /trash directory. Contributd by Brahma Reddy Battula. 
(cherry picked from commit e7ea2a8e8f)
 2015-03-31 00:28:27 +09:00
Kihwal Lee c4cedfc1d6 HDFS-7742. Favoring decommissioning node for replication can cause a block to stay 
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48)
 2015-03-30 10:11:25 -05:00
Harsh J c58357939f HDFS-4396. Add START_MSG/SHUTDOWN_MSG for ZKFC. Contributed by Liang Xie. 
(cherry picked from commit ae3e8c61ff)
 2015-03-30 15:22:57 +05:30
Vinayakumar B 9f49b3e93d HDFS-7890. Improve information on Top users for metrics in RollingWindowsManager and lower log level (Contributed by J.Andreina) 
(cherry picked from commit 1ed9fb7664)
 2015-03-30 10:04:11 +05:30
Akira Ajisaka 7aded71f3a HDFS-6408. Remove redundant definitions in log4j.properties. Contributed by Abhiraj Butala. 
(cherry picked from commit 232eca944a)
 2015-03-30 11:26:11 +09:00
Akira Ajisaka a79a38d096 HDFS-6263. Remove DRFA.MaxBackupIndex config from log4j.properties. Contributed by Abhiraj Butala. 
(cherry picked from commit 257c77f895)
 2015-03-30 10:52:45 +09:00
Harsh J b679dc5a8e HDFS-7501. TransactionsSinceLastCheckpoint can be negative on SBNs. Contributed by Gautam Gopalakrishnan. 
(cherry picked from commit 3d9132d434)
 2015-03-29 00:46:22 +05:30
Arpit Agarwal d0bc2eff1a HDFS-7700. Document quota support for storage types. (Contributed by Xiaoyu Yao) 2015-03-27 19:49:33 -07:00
Arun Suresh dadda2edca HDFS-8004. Use KeyProviderCryptoExtension#warmUpEncryptedKeys when creating an encryption zone. (awang via asuresh) 
(cherry picked from commit e97f8e44af)
 2015-03-27 19:25:32 -07:00
Kihwal Lee 577ea08865 HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp. 
(cherry picked from commit 60882ab26d)
 2015-03-27 09:06:23 -05:00
Colin Patrick Mccabe 2a750c9aa6 HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe) 
(cherry picked from commit 61df1b27a7)
 2015-03-26 13:28:58 -07:00
Arpit Agarwal c035106c05 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:47:43 -07:00
Kihwal Lee 071277b9ce HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki. 2015-03-26 08:42:45 -05:00
Haohui Mai 6e9202c802 HDFS-7918. Clean up unused code for old web UI in branch-2. Contributed by Brahma Reddy Battula. 2015-03-25 17:46:36 -07:00
Kihwal Lee be4eabdcd4 HDFS-7928. Scanning blocks from disk during rolling upgrade startup takes a lot of time if disks are busy. Contributed by Rushabh Shah. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2015-03-25 14:51:00 -05:00
Jitendra Pandey 456cec127b HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2015-03-24 16:02:21 -07:00
Haohui Mai 161dae898f HDFS-7713. Implement mkdirs in the HDFS Web UI. Contributed by Ravi Prakash. 2015-03-24 15:49:08 -07:00
Jing Zhao 483f77b75b HDFS-7854. Separate class DataStreamer out of DFSOutputStream. Contributed by Li Bo. 
(cherry picked from commit a16bfff71b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-24 11:21:10 -07:00
Brandon Li 260dbe96c3 HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li 
(cherry picked from commit 570a83ae80)
 2015-03-24 10:50:50 -07:00
Harsh J 608ad6c2cd HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. 
(cherry picked from commit 2da3d2ed2ff2e9b48dbda7e029aa58261c729d35)
 2015-03-24 23:03:56 +05:30
Brandon Li 95bde88986 HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li 
(cherry picked from commit 51f1f49374)
 2015-03-24 10:29:27 -07:00
Andrew Wang e39c58fd9d HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu. 
(cherry picked from commit 6413d34986)
 2015-03-24 09:07:24 -07:00
Vinayakumar B c3e2c20c99 HDFS-3325. When configuring 'dfs.namenode.safemode.threshold-pct' to a value greater or equal to 1 there is mismatch in the UI report (Contributed by J.Andreina) 
(cherry picked from commit c6c396fcd6)
 2015-03-24 12:13:21 +05:30
Plamen Jeliazkov 75d6d9e022 HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:07:01 -07:00
Andrew Wang 2f46ee50bd HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 
(cherry picked from commit 50ee8f4e67)
 2015-03-23 22:00:44 -07:00
Tsz-Wo Nicholas Sze fe693b72de HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:51:31 +09:00
cnauroth 01c0bcb176 HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2c238ae4e0)
 2015-03-23 16:30:33 -07:00
Akira Ajisaka fad8c78173 HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2. Contributed by Brahma Reddy Battula. 2015-03-24 06:24:29 +09:00
Brandon Li 503d8e4164 HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li 
(cherry picked from commit 36af4a913c)
 2015-03-23 10:09:43 -07:00
Kihwal Lee 29642b33cb HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 
(cherry picked from commit 99a8dcd19528b265d4fda9ae09a17e4af52f2782)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
 2015-03-20 14:05:02 -05:00
Jing Zhao 6be52e42a9 HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao. 
(cherry picked from commit d368d3647a)
 2015-03-20 11:54:35 -07:00
Jing Zhao a3e16f414c HDFS-7829. Code clean up for LocatedBlock. Contributed by Takanobu Asanuma. 
(cherry picked from commit a6a5aae472)
 2015-03-20 10:51:06 -07:00
Yongjun Zhang 0a41b1bdca HDFS-7835. make initial sleeptime in locateFollowingBlock configurable for DFSClient. Contributed by Zhihai Xu. 
(cherry-picked from commit 15612313f5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
 2015-03-20 09:25:59 -07:00
yliu 3fac95f48d HDFS-7962. Remove duplicated logs in BlockManager. (yliu) 2015-03-20 05:31:08 +08:00
yliu 90164ffd84 HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) 2015-03-19 23:24:55 +08:00
Kihwal Lee 2b9173059d HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee. 
(cherry picked from commit 61a4c7fc98)
 2015-03-19 12:28:36 -05:00
Vinayakumar B 786ba9784d HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 5be5129d93)
 2015-03-19 19:06:47 +05:30
Kihwal Lee ceb39c1cc6 HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai 
(cherry picked from commit e79be0ee12)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-19 08:03:47 -05:00
Vinayakumar B 25515d1421 HDFS-7867. Update action param from 'start' to 'prepare' in rolling upgrade ( Contributed by J.Andreina) Updated JIRA id. 
(cherry picked from commit 355e008ffb)
 2015-03-19 12:11:34 +05:30
Vinayakumar B 4527aa6f77 HDFS-7869. Update action param from 'start' to 'prepare' in rolling upgrade ( Contributed by J.Andreina) 
(cherry picked from commit 338b477ff7)
 2015-03-19 12:01:05 +05:30
Colin Patrick Mccabe 219eb22c15 HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 43b41f2241)
 2015-03-18 19:08:30 -07:00
Jing Zhao 5a5b244648 HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao. 
(cherry picked from commit c7c71cdba5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-03-18 18:56:56 -07:00
Jing Zhao 6dcc79507d HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao. 
(cherry picked from commit bee5a6a64a)
 2015-03-18 18:45:01 -07:00
Colin Patrick Mccabe 79c07bbaca HDFS-7054. Make DFSOutputStream tracing more fine-grained (cmccabe) 
(cherry picked from commit 8234fd0e10)
 2015-03-18 18:15:17 -07:00
Haohui Mai 8eb97ea14e HDFS-7945. The WebHdfs system on DN does not honor the length parameter. Contributed by Haohui Mai. 2015-03-18 16:19:57 -07:00
Arpit Agarwal ca629f4579 HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao) 2015-03-18 14:05:24 -07:00
Haohui Mai 572555c9f9 HDFS-7697. Mark the PB OIV tool as experimental. Contributed by Lei (Eddy) Xu. 2015-03-18 13:34:03 -07:00
Arpit Agarwal e7ceb434d0 Fix CHANGES.txt for HDFS-7722. 2015-03-18 12:52:50 -07:00
Arpit Agarwal 6b7cf6e096 HDFS-7951. Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux. (Contributed by Xiaoyu Yao) 2015-03-18 12:46:09 -07:00
Arpit Agarwal 97341a4a1d HDFS-7950. Fix TestFsDatasetImpl#testAddVolumes failure on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:35:35 -07:00
Arpit Agarwal a08b1b5703 HDFS-7948. TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows. (Contributed by Xiaoyu Yao) 2015-03-18 12:26:11 -07:00
Haohui Mai b547e6d6e0 HDFS-7953. NN Web UI fails to navigate to paths that contain #. Contributed by kanaka kumar avvaru. 2015-03-18 11:08:29 -07:00
Arpit Agarwal 7d0f84bd8d HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Contributed by Xiaoyu Yao) 2015-03-17 21:34:35 -07:00
Colin Patrick Mccabe 455d4aa8a1 HDFS-7940. Add tracing to DFSClient#setQuotaByStorageType (Rakesh R via Colin P. McCabe) 
(cherry picked from commit d8846707c5)
 2015-03-17 11:01:42 -07:00
Vinayakumar B 6ddb1bc857 HDFS-5356. MiniDFSCluster should close all open FileSystems when shutdown() (Contributed by Rakesh R) 
(cherry picked from commit 018893e81e)
 2015-03-17 15:34:48 +05:30
yliu 991ac04afc HDFS-7838. Expose truncate API for libhdfs. (yliu) 2015-03-17 07:24:20 +08:00
Harsh J cbb8858367 HDFS-2360. Ugly stacktrce when quota exceeds. (harsh) 
(cherry picked from commit 046521cd65)
 2015-03-17 10:28:52 +05:30
Konstantin V Shvachko 4568acaa62 HDFS-7886. Fix TestFileTruncate falures. Contributed by Plamen Jeliazkov and Konstantin Shvachko. 2015-03-16 12:55:16 -07:00
Colin Patrick Mccabe c71b54fa32 HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe) 
(cherry picked from commit bc9cb3e271)
 2015-03-14 22:41:50 -07:00
Colin Patrick Mccabe f93a2dd94b Revert "HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe)" (Jenkins didn't run yet) 
This reverts commit 7550052b85.
 2015-03-13 18:39:43 -07:00
Colin Patrick Mccabe 7550052b85 HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe) 
(cherry picked from commit 5aa892ed48)
 2015-03-13 18:30:36 -07:00
Konstantin V Shvachko 8afbfed5d3 HDFS-2605. Remove redundant "Release 0.21.1" section from CHANGES.txt. Contributed by Allen Wittenauer. 2015-03-13 13:45:13 -07:00
Konstantin V Shvachko b199081f54 HDFS-7903. Cannot recover block after truncate and delete snapshot. Contributed by Plamen Jeliazkov. 2015-03-13 12:40:38 -07:00
Kihwal Lee 464271a5ed HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp. 
(cherry picked from commit d324164a51)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2015-03-13 14:36:34 -05:00
Brandon Li af80a98ace HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contributed by Tsz Wo Nicholas Sze 
(cherry picked from commit f446669afb)
 2015-03-13 10:43:46 -07:00
Colin Patrick Mccabe 7455412a24 HDFS-7722. DataNode#checkDiskError should also remove Storage when error is found. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit b49c3a1813)
 2015-03-12 12:40:38 -07:00
Tsz-Wo Nicholas Sze 981a1fa815 HDFS-6833. DirectoryScanner should not register a deleting block with memory of DataNode. Contributed by Shinichi Yamashita 2015-03-13 02:27:37 +08:00
Akira Ajisaka 6c80a3d34a HDFS-7880. Remove the tests for legacy Web UI in branch-2. Contributed by Brahma Reddy Battula. 2015-03-12 10:41:34 +09:00
cnauroth 5e99b7cd2f HDFS-7491. Add incremental blockreport latency to DN metrics. Contributed by Ming Ma. 
(cherry picked from commit fb34f45727)
 2015-03-11 14:11:39 -07:00
Colin Patrick Mccabe eefca23e8c HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 5c1036d598)
 2015-03-10 19:45:18 -07:00
Akira Ajisaka 2d3c5507ce HDFS-6806. HDFS Rolling upgrade document should mention the versions available. Contributed by J.Andreina. 
(cherry picked from commit 82db3341bf)
 2015-03-10 11:23:06 +09:00
Jing Zhao c7105fcff0 HDFS-7898. Change TestAppendSnapshotTruncate to fail-fast. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit e43882e84a)
 2015-03-09 10:52:39 -07:00
Tsuyoshi Ozawa b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Vinod Kumar Vavilapalli 8e6f8e9a04 Adding 2.8 section in CHANGES.txt 2015-03-08 20:26:57 -07:00
Chris Douglas 3bc4f3502a HDFS-7411. Change decommission logic to throttle by blocks rather 
than nodes in each interval. Contributed by Andrew Wang
 2015-03-08 18:32:20 -07:00
Brandon Li c8af3b2768 HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li 
(cherry picked from commit 0f8ecb1d0c)
 2015-03-06 15:22:20 -08:00
Haohui Mai ab251fd355 HDFS-7818. OffsetParam should return the default value instead of throwing NPE when the value is unspecified. Contributed by Eric Payne. 2015-03-06 14:26:47 -08:00
Jing Zhao 994dadb9ba HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 24db0812be)
 2015-03-06 10:57:38 -08:00
Jing Zhao c5d52c7b52 HDFS-7855. Separate class Packet from DFSOutputStream. Contributed by Li Bo. 
(cherry picked from commit 952640fa4c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-05 11:11:08 -08:00
Tsuyoshi Ozawa f805d48b19 HADOOP-11648. Set DomainSocketWatcher thread name explicitly. Contributed by Liang Xie. 
(cherry picked from commit 74a4754d1c)
 2015-03-05 16:06:37 +09:00
Tsz-Wo Nicholas Sze f9a2007aff HDFS-7746. Add a test randomly mixing append, truncate and snapshot operations. 2015-03-05 10:23:14 +08:00
Dongming Liang ada545d3b7 HDFS-1522. Combine two BLOCK_FILE_PREFIX constants into one. Contributed by Dongming Liang. 2015-03-04 17:52:25 -08:00
Kihwal Lee f85530f649 HDFS-7434. DatanodeID hashCode should not be mutable. Contributed by Daryn Sharp. 
(cherry picked from commit 722b479469)
 2015-03-04 17:23:00 -06:00
Jing Zhao b2ccf54c14 HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao. 
(cherry picked from commit ed70fa142c)
 2015-03-04 10:32:39 -08:00
Haohui Mai d020544ef6 HDFS-7879. hdfs.dll does not export functions of the public libhdfs API. Contributed by Chris Nauroth. 2015-03-04 09:17:29 -08:00
Vinayakumar B c24448c27b HDFS-7869. Inconsistency in the return information while performing rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 3560180b6e)
 2015-03-04 14:39:51 +05:30
Akira Ajisaka 31b3f84601 HADOOP-11666. Revert the format change of du output introduced by HADOOP-6857. Contributed by Byron Wong. 2015-03-03 21:46:28 -08:00
Aaron T. Myers bce3d442ff HDFS-7682. {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content. Contributed by Charles Lamb. 
(cherry picked from commit f2d7a67a2c)
 2015-03-03 18:09:31 -08:00
Haohui Mai 65bfde552a HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA. 2015-03-03 17:58:21 -08:00
Arpit Agarwal 7e6624c218 HDFS-7757. Misleading error messages in FSImage.java. (Contributed by Brahma Reddy Battula) 2015-03-03 10:55:32 -08:00
Jing Zhao 6090f51725 HDFS-7871. NameNodeEditLogRoller can keep printing 'Swallowing exception' message. Contributed by Jing Zhao. 
(cherry picked from commit b442aeec95)
 2015-03-02 20:23:10 -08:00
Haohui Mai 651aa8761e HDFS-7785. Improve diagnostics information for HttpPutFailedException. Contributed by Chengbing Liu. 2015-03-02 15:35:19 -08:00
Gera Shegalov 52d0724a20 HDFS-7789. DFSck should resolve the path to support cross-FS symlinks. (gera) 
(cherry picked from commit cbb492578e)
 2015-03-02 01:05:15 -08:00
Tsz-Wo Nicholas Sze a5f3156b30 HDFS-7439. Add BlockOpResponseProto's message to the exception messages. Contributed by Takanobu Asanuma 2015-03-02 15:05:06 +08:00
Akira Ajisaka d664705fd1 HDFS-5853. Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml (aajisaka) 
(cherry picked from commit aa55fd3096)
 2015-03-01 01:18:36 -08:00
Tsz-Wo Nicholas Sze c52636df3f recommit "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
(cherry picked from commit acc172e371)
 2015-02-27 14:20:26 -08:00
Akira Ajisaka 0f9289e848 HDFS-7685. Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml. Contributed by Kai Sasaki. 
(cherry picked from commit 8719cdd4f6)
 2015-02-27 12:18:46 -08:00
Tsz-Wo Nicholas Sze 657b027bb2 HDFS-7308. Change the packet chunk size computation in DFSOutputStream in order to enforce packet size <= 64kB. Contributed by Takuya Fukudome 2015-02-27 23:46:42 +08:00
Vinayakumar B bc60404eaf HDFS-6753. Initialize checkDisk when DirectoryScanner not able to get files list for scanning (Contributed by J.Andreina) 
(cherry picked from commit 4f75b15628)
 2015-02-27 16:37:03 +05:30
Tsz-Wo Nicholas Sze d223a4a594 Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
This reverts commit acc172e371.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-27 18:27:19 +08:00
cnauroth 5dfb793b05 HDFS-7774. Unresolved symbols error while compiling HDFS on Windows 7/32 bit. Contributed by Kiran Kumar M R. 
(cherry picked from commit c6d5b37a1c)
 2015-02-26 13:37:58 -08:00
Colin Patrick Mccabe 8b3b9568b6 HDFS-7819. Log WARN message for the blocks which are not in Block ID based layout (Rakesh R via Colin P. McCabe) 
(cherry picked from commit f0c980abed)
 2015-02-26 12:03:11 -08:00
Vinayakumar B f0f3d1937d HDFS-7832. Show 'Last Modified' in Namenode's 'Browse Filesystem' (Contributed by Vinayakumar B) 
(cherry picked from commit 166eecf687)
 2015-02-26 14:37:03 +05:30
Tsz-Wo Nicholas Sze 22b13a0ddb HDFS-7537. Add "UNDER MIN REPL'D BLOCKS" count to fsck. Contributed by GAO Rui 2015-02-26 11:47:04 +08:00
Tsz-Wo Nicholas Sze 12fe3afcd3 HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade. 2015-02-26 10:15:49 +08:00
Benoy Antony 4e400030f6 HDFS-7467. Provide storage tier information for a directory via fsck. (Benoy Antony) 2015-02-25 16:46:41 -08:00
Colin Patrick Mccabe d389a1ae98 HDFS-7495. Remove updatePosition argument from DFSInputStream#getBlockAt() (cmccabe) 
(cherry picked from commit caa42adf20)
 2015-02-25 13:39:02 -08:00
Colin Patrick Mccabe 5c0073e706 HDFS-7668. Backport "Convert site documentation from apt to markdown" to branch-2 (Masatake Iwasaki via Colin P. McCabe) 
(backported from commit 2f1e5dc628)

Conflicts:
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithNFS.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithQJM.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsDesign.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsQuotaAdminGuide.apt.vm
 2015-02-24 16:18:18 -08:00
Andrew Wang efb7e287f4 HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie. 
(cherry picked from commit 7105ebaa9f)
 2015-02-24 15:31:18 -08:00
Jing Zhao 8346427929 HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. 
(cherry picked from commit 73bcfa99af)
 2015-02-24 10:34:42 -08:00
Tsuyoshi Ozawa 280dda5bcd HDFS-7008. xlator should be closed upon exit from DFSAdmin#genericRefresh(). (ozawa) 
(cherry picked from commit b53fd7163b)
 2015-02-24 23:59:59 +09:00
Colin Patrick Mccabe 453b210d0a HDFS-7805. NameNode recovery prompt should be printed on console (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit faaddb6ecb)
 2015-02-23 19:06:01 -08:00
Colin Patrick Mccabe f4da6f7193 HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe) 
(cherry picked from commit 585768667e)
 2015-02-23 18:45:44 -08:00
cnauroth 657a6e389b HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma. 
(cherry picked from commit 769507bd7a)
 2015-02-23 15:12:36 -08:00
Arpit Agarwal 83f779e49a HDFS-7806. Refactor: move StorageType from hadoop-hdfs to hadoop-common. (Contributed by Xiaoyu Yao) 2015-02-21 15:48:59 -08:00
yliu eca1588db8 HDFS-7740. Test truncate with DataNodes restarting. (yliu) 2015-02-21 06:33:58 +08:00
cnauroth 9f06fb3aef HDFS-7813. TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently. Contributed by Rakesh R. 
(cherry picked from commit 0d6af574e0)
 2015-02-20 17:01:22 -08:00
cnauroth e09ba94f28 HDFS-7773. Additional metrics in HDFS to be accessed via jmx. Contributed by Anu Engineer. 2015-02-20 12:38:34 -08:00
cnauroth ad77dfc3e4 HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao. 
(cherry picked from commit 8c6ae0d619)
 2015-02-20 12:22:00 -08:00
Kihwal Lee b9157f92fc HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah. 
(cherry picked from commit 7ae5255a16)
 2015-02-20 09:09:56 -06:00
Harsh J 5c2c6b00dd HDFS-7752. Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml. Contributed by Wellington Chevreuil. 
(cherry picked from commit b9a17909ba)
 2015-02-20 19:21:34 +05:30
yliu fee29e4a4f HDFS-7656. Expose truncate API for HDFS httpfs. (yliu) 2015-02-19 08:38:25 +08:00
Haohui Mai 3605928bd0 HDFS-7808. Remove obsolete -ns options in in DFSHAAdmin.java. Contributed by Arshad Mohammad. 2015-02-18 15:14:53 -08:00
cnauroth b1b1e05e18 HDFS-7772. Document hdfs balancer -exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao. 2015-02-18 12:03:00 -08:00
Akira Ajisaka 6d6c684567 HDFS-7780. Update use of Iterator to Iterable in DataXceiverServer and SnapshotDiffInfo. Contributed by Ray Chiang. 
(cherry picked from commit 6dc8812a95)
 2015-02-17 14:49:03 -08:00
Haohui Mai f92a4904bf HDFS-6662. WebHDFS cannot open a file if its path contains "%". Contributed by Gerson Carlos. 2015-02-17 13:05:00 -08:00
Kihwal Lee 8b37b4a78b HDFS-7795. Show warning if not all favored nodes were chosen by namenode. Contributed by Kihwal Lee. 
(cherry picked from commit db6606223c)
 2015-02-17 13:06:41 -06:00
Uma Maheswara Rao G 2cbac36fd3 HDFS-4266. BKJM: Separate write and ack quorum (Rakesh R via umamahesh) 
(cherry picked from commit f0412de1c1)
 2015-02-17 21:31:43 +05:30
Uma Maheswara Rao G 35fecb5306 HDFS-7797. Add audit log for setQuota operation (Rakesh R via umamahesh) 
(cherry picked from commit f24a56787a)
 2015-02-17 20:16:34 +05:30
Tsuyoshi Ozawa 00fb0710b6 HADOOP-11600. Fix up source codes to be compiled with Guava 17.0. (ozawa) 
(cherry picked from commit 2f0f756b26)
 2015-02-17 21:57:58 +09:00
Vinayakumar B 91a5d92916 HDFS-7703. Support favouredNodes for the append for new blocks ( Contributed by Vinayakumar B) 
(cherry picked from commit 89a5449280)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-17 15:19:03 +05:30
yliu 54e33baaf6 HDFS-7798. Checkpointing failure caused by shared KerberosAuthenticator. (Chengbing Liu via yliu) 2015-02-17 07:50:14 +08:00
cnauroth 441dfa4867 HDFS-7604. Track and display failed DataNode storage locations in NameNode. Contributed by Chris Nauroth. 
(cherry picked from commit 9729b244de)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
 2015-02-16 14:44:49 -08:00
Arpit Agarwal c6aa6cbfe5 HDFS-7775. Use consistent naming for NN-internal quota related types and functions. (Contributed bu Xiaoyu Yao) 2015-02-13 21:01:54 -08:00
cnauroth 751b834ad1 HDFS-7778. Rename FsVolumeListTest to TestFsVolumeList and commit it to branch-2. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2efb2347a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsVolumeList.java
 2015-02-13 16:30:53 -08:00
Akira Ajisaka 3ad1d8d533 HDFS-4265. BKJM doesn't take advantage of speculative reads. Contributed by Rakesh R. 
(cherry picked from commit 0d521e3326)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-13 15:25:13 -08:00
Colin Patrick Mccabe 7b9c978df2 HDFS-7686. Re-add rapid rescan of possibly corrupt block feature to the block scanner (cmccabe) 
(cherry picked from commit 8bb9a5000e)
 2015-02-13 15:19:04 -08:00
Colin Patrick Mccabe 6edcb9a503 HDFS-7721. The HDFS BlockScanner may run fast during the first hour (cmccabe) 
(cherry picked from commit 115428176e)
 2015-02-13 15:18:10 -08:00
Colin Patrick Mccabe a54b4a7fc6 HDFS-7430 backport addendum. Add missing change. (cmccabe) 2015-02-13 15:11:14 -08:00
Colin Patrick Mccabe a8c6a96f3b HDFS-7430. Refactor the BlockScanner to use O(1) memory and use multiple threads (cmccabe) 
(cherry picked from commit 6e62a1a672)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeListTest.java
 2015-02-13 15:04:06 -08:00
Arpit Agarwal f507791554 HDFS-7776. Adding additional unit tests for Quota By Storage Type. (Contributed by Xiaoyu Yao) 2015-02-13 12:02:54 -08:00
Allen Wittenauer 730597c20c HDFS-316. Balancer should run for a configurable # of iterations (Xiaoyu Yao via aw) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
 2015-02-12 18:02:23 -08:00
Colin Patrick Mccabe a9a361c549 HDFS-7790. Do not create optional fields in DFSInputStream unless they are needed (cmccabe) 
(cherry picked from commit 871cb56152)
 2015-02-12 17:49:11 -08:00
Akira Ajisaka 5696376e29 HDFS-7684. The host:port settings of the deamons should be trimmed before use. Contributed by Anu Engineer. 
(cherry picked from commit 9235e2fb4e94a86671e1494f293c973cf572281c)
 2015-02-12 17:40:48 -08:00
Arpit Agarwal 676fc2d1dc HDFS-7723. Quota By Storage Type namenode implemenation. (Contributed by Xiaoyu Yao) 2015-02-12 10:50:14 -08:00
Colin Patrick Mccabe e35788aa5a HDFS-7694. FSDataInputStream should support "unbuffer" (cmccabe) 
(cherry picked from commit 6b39ad0865)
 2015-02-12 10:42:14 -08:00
Jing Zhao 2156e38d5e HDFS-3689. Add support for variable length block. Contributed by Jing Zhao. 
(cherry picked from commit 2848db814a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirConcatOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
 2015-02-12 10:23:13 -08:00
Kihwal Lee 146f559097 Adding missing files from HDFS-7404 
(cherry picked from commit 8d7215d40f)
 2015-02-12 11:18:41 -06:00
Kihwal Lee 39594301e1 HDFS-7704. DN heartbeat to Active NN may be blocked and expire if connection to Standby NN continues to time out. Contributed by Rushabh Shah. 
(cherry picked from commit 38262779bb)
 2015-02-12 09:17:10 -06:00
yliu f8d0825f39 HDFS-7761. cleanup unnecssary code logic in LocatedBlock. (yliu) 2015-02-12 03:09:38 +08:00
Tsz-Wo Nicholas Sze 65a6cf47ec HDFS-6133. Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover. Contributed by zhaoyunjiong 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
 2015-02-11 15:16:34 -08:00
Colin Patrick Mccabe 2ca76df21a HDFS-7771. fuse_dfs should permit FILE: on the front of KRB5CCNAME (cmccabe) 
(cherry picked from commit 50625e660a)
 2015-02-11 12:14:56 -08:00
Haohui Mai 744db47872 HDFS-7736. Fix typos in dfsadmin/fsck/snapshotDiff usage messages. Contributed by Brahma Reddy Battula. 2015-02-11 11:24:44 -08:00
Tsz-Wo Nicholas Sze b49eeddbaf HDFS-7751. Fix TestHDFSCLI for quota with storage type. Contributed by Xiaoyu Yao 2015-02-11 11:10:05 -08:00
Arpit Agarwal 8fa0fc9574 HDFS-7720. Quota by Storage Type API, tools and ClientNameNode Protocol changes. (Contributed by Xiaoyu Yao) 2015-02-11 11:07:53 -08:00
Konstantin V Shvachko 04789db74c HDFS-7760. Document truncate for WebHDFS. Contributed by Konstantin Shvachko. 2015-02-11 10:05:53 -08:00
Konstantin V Shvachko 7f18176cd7 HDFS-7753. Fix Multithreaded correctness Warnings in BackupImage. Contributed by Rakesh R and Konstantin Shvachko. 2015-02-11 00:52:12 -08:00
Tsz-Wo Nicholas Sze acc172e371 HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir. 2015-02-10 17:50:00 -08:00
Konstantin V Shvachko 2b722b904e HDFS-7058. Tests for truncate CLI. Contributed by Dasha Boudnik. 2015-02-09 21:31:02 -08:00
Vinayakumar B a1bf7aecf7 HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode.(Contributed by Vinayakumar B) 
(cherry picked from commit 3d15728ff5)
 2015-02-10 10:45:14 +05:30
Colin Patrick Mccabe 643a8892d1 HDFS-7718. Store KeyProvider in ClientContext to avoid leaking key provider threads when using FileContext (Arun Suresh via Colin P. McCabe) 
(cherry picked from commit 02340a24f2)
 2015-02-09 20:25:31 -08:00
Colin Patrick Mccabe 1465393d14 HDFS-7744. Fix potential NPE in DFSInputStream after setDropBehind or setReadahead is called (cmccabe) 
(cherry picked from commit a9dc5cd706)
 2015-02-09 20:19:48 -08:00
yliu cc0668ebe6 HDFS-7756. Restore method signature for LocatedBlock#getLocations(). (Ted Yu via yliu) 2015-02-10 02:00:32 +08:00
yliu ae316705bb HADOOP-11510. Expose truncate API via FileContext. (yliu) 2015-02-10 01:43:08 +08:00
Arpit Agarwal ff900eb64a HDFS-7647. DatanodeManager.sortLocatedBlocks sorts DatanodeInfos but not StorageIDs. (Contributed by Milan Desai) 2015-02-09 12:17:53 -08:00
Jing Zhao 1e49b40f1f HDFS-7743. Code cleanup of BlockInfo and rename BlockInfo to BlockInfoContiguous. Contributed by Jing Zhao. 
(cherry picked from commit 1382ae525c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockCollection.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
 2015-02-08 12:16:09 -08:00
Tsz-Wo Nicholas Sze c5f18ba65b HDFS-7738. Revise the exception message for recover lease; add more truncate tests such as truncate with HA setup, negative tests, truncate with other operations and multiple truncates. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHAAppend.java
 2015-02-07 15:26:06 -08:00
Akira Ajisaka 964a3b0c3e HDFS-7710. Remove dead code in BackupImage.java. Contributed by Xiaoyu Yao. 
(cherry picked from commit cfb829ecd5)
 2015-02-06 19:00:58 -08:00
Akira Ajisaka 2ec796ba80 HDFS-7732. Fix the order of the parameters in DFSConfigKeys. Contributed by Brahma Reddy Battula. 
(cherry picked from commit eaab959a3e)
 2015-02-06 11:28:18 -08:00
yliu e2803f5a51 HDFS-7741. Remove unnecessary synchronized in FSDataInputStream and HdfsDataInputStream. (yliu) 2015-02-06 06:29:52 +08:00
yliu 44d800b353 HDFS-7698. Fix locking on HDFS read statistics and add a method for clearing them. (Colin P. McCabe via yliu) 2015-02-05 23:57:36 +08:00
yliu 3ffe5a1ed2 HDFS-7655. Expose truncate API for Web HDFS. (yliu) 2015-02-05 23:48:55 +08:00
Haohui Mai 67002a5fc0 HDFS-7270. Add congestion signaling capability to DataNode write protocol. Contributed by Haohui Mai. 2015-02-05 11:03:43 -08:00
Tsuyoshi Ozawa 970fdc3ad9 HDFS-7709. Fix findbug warnings in httpfs. Contributed by Rakesh R. 
(cherry picked from commit 20660b7a67)
 2015-02-05 19:57:40 +09:00
Arpit Agarwal 5807afed0a HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. (Contributed by Arpit Agarwal) 2015-02-04 16:25:40 -08:00
Colin Patrick Mccabe 76c0757c6e HDFS-7719. BlockPoolSliceStorage#removeVolumes fails to remove some in-memory state associated with volumes. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 40a415799b)
 2015-02-04 12:46:39 -08:00
Andrew Wang 696266e99c HDFS-7734. Class cast exception in NameNode#main. Contributed by Yi Liu. 
(cherry picked from commit 9175105eea)
 2015-02-04 11:31:49 -08:00
Tsuyoshi Ozawa f92d4fa20f HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa) 
(cherry picked from commit ba58da29f4)
 2015-02-05 00:51:40 +09:00
Andrew Wang debd63f0b0 HDFS-7712. Switch blockStateChangeLog to use slf4j. 
(cherry picked from commit 3ae38ec7df)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
 2015-02-03 15:04:16 -08:00
Kihwal Lee c428d303f6 HDFS-7707. Edit log corruption due to delayed block removal again. Contributed by Yongjun Zhang 
(cherry picked from commit 843806d03a)
 2015-02-03 14:46:24 -06:00
Haohui Mai 7cb70d3447 HDFS-6651. Deletion failure can leak inodes permanently. Contributed by Jing Zhao. 2015-02-02 16:32:56 -08:00
Tsz-Wo Nicholas Sze 8426c7d806 HDFS-7696. In FsDatasetImpl, the getBlockInputStream(..) and getTmpInputStreams(..) methods may leak file descriptors. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2015-02-02 13:42:15 -08:00
Tsz-Wo Nicholas Sze a5568a276d HDFS-7681. Change ReplicaInputStreams constructor to take InputStream(s) instead of FileDescriptor(s). Contributed by Joe Pallas 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-02 13:30:07 -08:00
Tsz-Wo Nicholas Sze 2ecfd018f4 HDFS-5782. Change BlockListAsLongs constructor to take Replica as parameter type instead of concrete classes Block and ReplicaInfo. Contributed by David Powell and Joe Pallas 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-02 13:24:02 -08:00
Tsz-Wo Nicholas Sze e43b1428b4 HDFS-5631. Change BlockMetadataHeader.readHeader(..), ChunkChecksum class and constructor to public; and fix FsDatasetSpi to use generic type instead of FsVolumeImpl. Contributed by David Powell and Joe Pallas 2015-02-02 12:59:22 -08:00
Colin Patrick Mccabe 4bf14876b3 HADOOP-11498 addendum. Fix extra "import" statements in branch-2.7 patch backport 2015-01-30 16:07:12 -08:00
Colin Patrick Mccabe 80381b62aa HADOOP-11498. Bump the version of HTrace to 3.1.0-incubating (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 09ad9a868a)
 2015-01-30 16:02:36 -08:00
Andrew Wang fcd23b61e3 HDFS-7706. Switch BlockManager logging to use slf4j. 
(cherry picked from commit 951b3608a8)
 2015-01-30 11:36:02 -08:00
Colin Patrick Mccabe e8d154baea HADOOP-11403. Avoid using sys_errlist on Solaris, which lacks support for it (Malcolm Kavalsky via Colin P. McCabe) 
(cherry picked from commit e36ef3b402)
 2015-01-29 15:48:59 -08:00
Kihwal Lee 3b030731a0 HDFS-7603. The background replication queue initialization may not let others run. Contributed by Kihwal Lee. 
(cherry picked from commit 89b07490f8)
 2015-01-29 12:37:34 -06:00
yliu dc7950ef18 HDFS-7423. various typos and message formatting fixes in nfs daemon and doc. (Charles Lamb via yliu) 2015-01-29 04:20:14 +08:00
Jing Zhao e8300957a7 HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. Contributed by Jing Zhao and Byron Wong. 
(cherry picked from commit d244574d03)
 2015-01-28 15:25:42 -08:00