Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,824 Commits 326 Branches 371 Tags
Commit Graph

3347 Commits

Author SHA1 Message Date
Tsz-Wo Nicholas Sze
61b9e5f7ff HDFS-8824. Do not use small blocks for balancing the cluster. 2015-08-14 13:08:57 -07:00
Arpit Agarwal
b094f8b589 HDFS-7649. Multihoming docs should emphasize using hostnames in configurations. (Contributed by Brahma Reddy Battula) 2015-08-14 13:08:19 -07:00
Xiaoyu Yao
17256e2ae8 HDFS-8565. Typo in dfshealth.html - Decomissioning. (nijel via xyao) 
(cherry picked from commit 1569228ec9090823186f062257fdf1beb5ee1781)
 2015-08-14 12:35:17 -07:00
Vinayakumar B
0b6574380c HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
Moved to 2.6.1

(cherry picked from commit 84bf71295a5e52b2a7bb69440a885a25bc75f544)
 2015-08-14 16:13:47 +05:30
Vinayakumar B
15760a148b HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
Moved to 2.6.1

(cherry picked from commit 08bd4edf4092901273da0d73a5cc760fdc11052b)
 2015-08-14 12:38:23 +05:30
Vinayakumar B
9839abe1a8 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit fa2641143c0d74c4fef122d79f27791e15d3b43f)
 2015-08-14 11:46:30 +05:30
Vinayakumar B
c2a9c3929e HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)
 2015-08-14 11:38:03 +05:30
Vinayakumar B
2c0063d420 HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne. 
Moved CHANGES.TXT entry to 2.6.1

(cherry picked from commit d25cb8fe12d00faf3e8f3bfd23fd1b01981a340f)
 2015-08-14 11:24:12 +05:30
yliu
073cb16d81 HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu) 2015-08-13 16:42:40 +08:00
Akira Ajisaka
200fadedc0 HDFS-8622. Implement GETCONTENTSUMMARY operation for WebImageViewer. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 40f815131e822f5b7a8e6a6827f4b85b31220c43)
 2015-08-13 12:23:03 +09:00
cnauroth
bf8b09d4b8 HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu. 
(cherry picked from commit 6cc8e38db5b26bdd02bc6bc1c9684db2593eec25)
 2015-08-12 16:45:05 -07:00
Xiaoyu Yao
458b684941 HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao. 
(cherry picked from commit 3e715a4f4c46bcd8b3054cb0566e526c46bd5d66)
 2015-08-12 15:25:20 -07:00
Andrew Wang
d53296281a HDFS-8887. Expose storage type and storage ID in BlockLocation. 
(cherry picked from commit 1ea1a8334ea01814121490a5bfd2a0205c66d6e4)
 2015-08-11 23:33:06 -07:00
Jing Zhao
f0e13c5499 HDFS-8805. Archival Storage: getStoragePolicy should not need superuser privilege. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1fc3c779a422bafdb86ad1a5b2349802dda1cb62)
 2015-08-11 10:28:48 -07:00
Tsz-Wo Nicholas Sze
ac8d153046 HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. 2015-08-10 16:53:15 -07:00
Jakob Homan
e771dbefe9 HDFS-8866. Typo in docs: Rumtime -> Runtime. Contributed by Gabor Liptak. 
(cherry picked from commit 8f73bdd06b16d5048ffb6071bbcecf849c6225db)
 2015-08-07 11:39:54 -07:00
Andrew Wang
0c4c7b3b62 HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. 2015-08-07 09:53:04 -07:00
Arpit Agarwal
42a05d29ee HDFS-8856. Make LeaseManager#countPath O(1). (Contributed by Arpit Agarwal) 2015-08-06 18:52:07 -07:00
Jing Zhao
d7d804623e Revert "HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang." 
This reverts commit c17439c2ddd921b63b1635e6f1cba634b8da8557.

(cherry picked from commit f4c523b69ba55b1fd35e8995c3011a9f546ac835)
 2015-08-06 10:26:09 -07:00
Jing Zhao
2c694286b4 Revert "HDFS-8623. Refactor NameNode handling of invalid, corrupt, and under-recovery blocks. Contributed by Zhe Zhang." 
This reverts commit de480d6c8945bd8b5b00e8657b7a72ce8dd9b6b5.

(cherry picked from commit 663eba0ab1c73b45f98e46ffc87ad8fd91584046)
 2015-08-06 10:25:54 -07:00
Vinayakumar B
f8932d478f HDFS-8815. DFS getStoragePolicy implementation using single RPC call (Contributed by Surendra Singh Lilhore) 
(cherry picked from commit cc71ad80e184fc6e5043729e8cfcf6a62ca3e71f)
 2015-08-06 11:11:59 +05:30
Lei Xu
f7b0f292e7 HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu) 
(cherry picked from commit d5403747b57b1e294e533ce17f197e7be8f5339c)
 2015-08-04 18:56:07 -07:00
Colin Patrick Mccabe
b64d2ac478 HDFS-8844. TestHDFSCLI does not cleanup the test directory (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit c95993cbaf51e2925ea9b1b95cf4f0d879e66489)
 2015-08-04 12:22:40 -07:00
yliu
d2b941f94a HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) 2015-08-04 15:51:37 +08:00
Zhihai Xu
c6e79178d4 HDFS-8847. change TestHDFSContractAppend to not override testRenameFileBeingAppended method. Contributed by Zhihai Xu 
(cherry picked from commit 90b510405af5ac9d4947c4201065b64994f7ef63)
 2015-07-31 21:48:03 -07:00
Xiaoyu Yao
52ee3ecdf6 HDFS-6860. BlockStateChange logs are too noisy. Contributed by Chang Li and Xiaoyu Yao. 
(cherry picked from commit d311a38a6b32bbb210bd8748cfb65463e9c0740e)
 2015-07-31 16:19:11 -07:00
Colin Patrick Mccabe
07a042aa3c HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 
(cherry picked from commit 21d10ccc6e463cf250414264c78acb4a6e7c83e3)
 2015-07-31 15:02:01 -07:00
Harsh J
212b8b919d HDFS-8821. Explain message "Operation category X is not supported in state standby". Contributed by Gautam Gopalakrishnan. 
(cherry picked from commit c5caa25b8f2953e2b7a9d2c9dcbdbf1fed95c10b)
 2015-07-31 08:59:22 +05:30
Arpit Agarwal
95e71772cf HDFS-7192. DN should ignore lazyPersist hint if the writer is not local. (Contributed by Arpit Agarwal) 2015-07-30 13:16:52 -07:00
Haohui Mai
3faf7bc3fe HDFS-8816. Improve visualization for the Datanode tab in the NN UI. Contributed by Haohui Mai. 2015-07-29 17:14:20 -07:00
Jason Lowe
e6b4714d0b HADOOP-10945. 4-digit octal umask permissions throws a parse error. Contributed by Chang Li 
(cherry picked from commit 6f0a35724f0da80146dbae4b6f6c341e1d3101f5)
 2015-07-29 15:25:59 +00:00
Vinayakumar B
a0238d85f6 HDFS-8670. Better to exclude decommissioned nodes for namenode NodeUsage JMX (Contributed by J.Andreina) 
(cherry picked from commit 6374ee0db445e0a1c3462c19ddee345df740cfb3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-07-29 14:49:16 +05:30
Akira Ajisaka
dff49aee08 Revert "HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)" 
This reverts commit 17319c4a659dd150ccfe80da81fbaee69ca8ef1e.
 2015-07-29 16:52:54 +09:00
Lei Xu
9d81916040 HDFS-8834. TestReplication is not valid after HDFS-6482. (Contributed by Lei Xu) 
(cherry picked from commit f4f1b8b267703b8bebab06e17e69a4a4de611592)
 2015-07-28 23:41:44 -07:00
Vinayakumar B
dcf1f1fdf8 HDFS-8822. Add SSD storagepolicy tests in TestBlockStoragePolicy#testDefaultPolicies (Contributed by Vinayakumar B) 
(cherry picked from commit 975e138df316f59e8bb0642e138d4b1170fb8184)
 2015-07-29 11:51:18 +05:30
Vinayakumar B
0c09597942 HDFS-8811. Move BlockStoragePolicy name's constants from HdfsServerConstants.java to HdfsConstants.java (Contributed by Vinayakumar B) 
(cherry picked from commit 50887e5b07b6abb20c0edd74211e5612dc7b16da)
 2015-07-29 11:46:19 +05:30
Jakob Homan
71a5aa267d HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak. 
(cherry picked from commit 0712a8103fec6e9a9ceba335e3c3800b85b2c7ca)
 2015-07-28 21:11:02 -07:00
Arun Suresh
ffe6a34d5b HDFS-7858. Improve HA Namenode Failover detection on the client. (asuresh) 
(cherry picked from commit 030fcfa99c345ad57625486eeabedebf2fd4411f)
 2015-07-27 23:03:34 -07:00
Akira Ajisaka
1cf5e40838 HDFS-8810. Correct assertions in TestDFSInotifyEventInputStream class. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 1df78688c69476f89d16f93bc74a4f05d0b1a3da)
 2015-07-27 13:18:02 +09:00
Akira Ajisaka
7bdd32ce3c HDFS-8735. Inotify: All events classes should implement toString() API. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit f8f60918230dd466ae8dda1fbc28878e19273232)
 2015-07-25 02:57:33 +09:00
Akira Ajisaka
b2a532f61c HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang. 
(cherry picked from commit 206d4933a567147b62f463c2daa3d063ad40822b)
 2015-07-24 18:29:24 +09:00
Akira Ajisaka
17319c4a65 HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka) 
(cherry picked from commit 02c01815eca656814febcdaca6115e5f53b9c746)
 2015-07-24 11:39:16 +09:00
Haohui Mai
d3e6b40b44 HDFS-8730. Clean up the import statements in ClientProtocol. Contributed by Takanobu Asanuma. 2015-07-23 10:30:43 -07:00
Arpit Agarwal
601b503895 HADOOP-12161. Add getStoragePolicy API to the FileSystem interface. (Contributed by Brahma Reddy Battula) 2015-07-23 10:14:01 -07:00
Jing Zhao
71764a92c6 HDFS-8797. WebHdfsFileSystem creates too many connections for pread. Contributed by Jing Zhao. 
(cherry picked from commit e91ccfad07ec5b5674a84009772dd31a82b4e4de)
 2015-07-22 18:03:45 -07:00
yliu
930e344d78 HDFS-8795. Improve InvalidateBlocks#node2blocks. (yliu) 2015-07-22 15:15:08 +08:00
Haohui Mai
53ed25c3d1 HDFS-8495. Consolidate append() related implementation into a single class. Contributed by Rakesh R. 2015-07-21 17:28:28 -07:00
cnauroth
aab1c14df6 HDFS-8773. Few FSNamesystem metrics are not documented in the Metrics page. Contributed by Rakesh R. 
(cherry picked from commit a26cc66f38daec2342215a66b599bf59cee1112c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-07-21 14:13:15 -07:00
cnauroth
57d55d40dd HDFS-8721. Add a metric for number of encryption zones. Contributed by Rakesh R. 
(cherry picked from commit cb03768b1b2250b9b5a7944cf6ef918e8a974e20)
 2015-07-21 13:56:07 -07:00
Ravi Prakash
b728edce0f Revert "HDFS-8344. NameNode doesn't recover lease for files with missing blocks (raviprak)" 
This reverts commit 3d58c7a7006991b46efe8a8f60b244f4f85b481a.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-07-21 11:30:41 -07:00