Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,431 Commits 326 Branches 371 Tags
Commit Graph

4331 Commits

Author SHA1 Message Date
Zhe Zhang
239dadbd46 Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit f1ff21e00c21ba2514b81779389141db150882cc.
 2017-10-24 15:04:37 -07:00
Wei-Chiu Chuang
a83f87e170 HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4ab0c8f96a41c573cc1f1e71c18871d243f952b9)
(cherry picked from commit ce93a2b84badacb6d0b19bbb149afbcf8f38c665)
 2017-10-19 06:19:11 -07:00
Zhe Zhang
f1ff21e00c HDFS-12502. nntop should support a category based on FilesInGetListingOps. 
(cherry picked from commit 60bfee270ed3a653c44c0bc92396167b5022df6e)
(cherry picked from commit aecf6c50d351e7aed2008a682677a1edb104585d)
(cherry picked from commit 87f440739604f97261da9624afb36920a6a4c28f)
 2017-10-19 00:09:33 -07:00
Arpit Agarwal
dcdb3f250c HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 
(cherry picked from commit b6942cbe9b8c9469e8c2b64c3268d671f5a43e75)
(cherry picked from commit 5897095d539be086ed37df011f024e37eb37b0cd)
(cherry picked from commit 0aa1b62b396f6fa43f15900c26dd28b95c4e40ce)
 2017-10-17 23:46:14 -07:00
Wei-Chiu Chuang
843b8b7b96 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
(cherry picked from commit a596c6f5eb553cf38123d61b844696ddbec99be0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-10-16 13:04:36 -07:00
Xiao Chen
6a55d0e671 HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 
(cherry picked from commit 971b3ca1acc15d9a1f8e774291813dafd16ef704)
 2017-10-16 10:34:31 -07:00
Xiao Chen
db726c4483 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 
(cherry picked from commit d0311dfc5fe92a0e0f3fb8ae92e412edce609740)
(cherry picked from commit f6e768104b0e9a087584c25e28bf8cd29802c719)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2017-10-16 09:51:34 -07:00
Xiao Chen
27badf6df9 HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 
(cherry picked from commit 11a7729c1474aa379a62c0440c8c1a11be261a75)
 2017-10-15 19:28:45 -07:00
Xiao Chen
99b7ced3bf HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/LoadBalancingKMSClientProvider.java
 2017-10-05 19:47:26 -07:00
Yiqun Lin
fb5de93c46 HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9de7c3fbe451bcdc3fd928a27a0d8b4e)
 2017-09-27 10:51:00 +08:00
Daryn Sharp
438291787a [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:25:12 -05:00
Anu Engineer
3ecc1431fc
HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 
(cherry picked from commit de5744c9035308654c7096bf5bd04e769a3a19e3)
 2017-09-25 13:12:27 +09:00
Kihwal Lee
c0bb242360 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3152a307b5c0903056d73ce07775a08)
 2017-09-21 09:00:26 -05:00
Akira Ajisaka
4ac3fc10c2
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb9bad90586a95d2b2380ee6e8724d6b)
(cherry picked from commit 3527327f349264ae62f3673ab368505a16cc0a34)
 2017-09-21 19:29:18 +09:00
Ming Ma
c54310a638 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:21:32 -07:00
Brahma Reddy Battula
a81167e2ec HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:34:52 +05:30
Erik Krogen
b8d86bb535 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-15 14:06:14 -07:00
Nathan Roberts
11b325d17d HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 14:54:37 -05:00
Kihwal Lee
9e9e36fa70 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 2017-09-15 13:16:43 -05:00
John Zhuge
584cd134e8 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:35:25 -07:00
Arpit Agarwal
2cddf1cc0f HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:55:30 -07:00
Xiao Chen
9d4c4b2f6e Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 8b7d1df766002ae033d5fca8017cfe7141803c58.
 2017-09-12 19:53:26 -07:00
Xiao Chen
69d7b1ae50 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-09-07 16:30:53 -07:00
Xiao Chen
98436a2eb7 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 441c472552eadf66c748ab9493fd02d2c914994a)
 2017-08-30 09:43:27 -07:00
Kihwal Lee
b0e951725f HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396089972e23c533eca33c9cba231c45a)
 2017-08-25 15:04:37 -05:00
Arpit Agarwal
b07eef43ae HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:02:59 -07:00
Arpit Agarwal
22fb9bb843 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 11:10:25 -07:00
Colin Patrick Mccabe
945c916eec HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had an error (Rakesh R via cmccabe) 
(cherry picked from commit f67149ab08bb49381def6c535ab4c4610e0a4221)
 2017-08-25 11:02:05 -07:00
John Zhuge
bc1c8f3e5a HDFS-11738. Hedged pread takes more time when block moved from initial locations. Contributed by Vinayakumar B. 
(cherry picked from commit b6bfb2fcb2391d51b8de97c01c1290880779132e)
(cherry picked from commit c54c3500ed720e39926f4cc7ca3db0cb4854ff55)
 2017-08-23 20:05:54 +05:30
Ravi Prakash
a611922de4 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
(cherry picked from commit 895a35eac0422c1845ebe3c5bed1ac2bda1ac807)
 2017-08-15 15:46:22 -07:00
Kihwal Lee
a54c3437af HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:51:19 -05:00
Kihwal Lee
639380efff HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 
(cherry picked from commit 74bcc8d1f213ffd18056b7d0590d0cd346abaff9)
 2017-08-09 09:33:07 -05:00
Haohui Mai
6801b898d5 HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 2017-08-08 15:28:39 -07:00
Andrew Wang
22ef7c6bf2 HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:45:47 -07:00
Manoj Govindassamy
fc66e76a4a HDFS-10480. Add an admin command to list currently open files. 2017-08-01 17:37:45 -07:00
Brahma Reddy Battula
90f3108b8e HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 8bfb9971cad1a83da852de4d12f1de4197d25d21)
 2017-07-30 14:02:25 +08:00
Brahma Reddy Battula
9a168ae884 HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)
 2017-07-27 12:37:36 -07:00
Kihwal Lee
c586b7ab7d HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb52c36b74358f22db498424e82ec4a7)
(cherry picked from commit 2d6be7ea2311deb13cbad738b4c8fc68ad25ab2a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestRMAdminCLI.java
 2017-07-26 14:15:39 -05:00
Kihwal Lee
38c56bbb46 HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 
(cherry picked from commit 3d8c7d701b8f22b3b4d67f587f7ff2318c60c541)
 2017-07-25 11:08:57 -05:00
Kihwal Lee
7be52c9c79 HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 
(cherry picked from commit 1a79dcfc457969d6a6c08ffffe4152fd7638e48a)
 2017-07-25 10:33:09 -05:00
Brahma Reddy Battula
587d47cfec HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 
(cherry picked from commit 820496fbbc00ede0484bec5511c6b12913e97356)
 2017-07-23 16:09:10 +08:00
Kihwal Lee
e229ffee64 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992eccff26a7344c3f0e719898fa97706b8c)
 2017-07-21 09:22:28 -05:00
Brahma Reddy Battula
f65dc6ee95 HDFS-12177. NameNode exits due to setting BlockPlacementPolicy loglevel to Debug. Contributed by Jiandan Yang. 2017-07-21 21:13:20 +08:00
Konstantin V Shvachko
f3bf1a6eb9 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539e211736fef12010918a60f9edad030)
 2017-07-20 16:09:56 -07:00
Anu Engineer
6ed569df21 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04eee24275257ab462133e0818f87449)
(cherry picked from commit e0297ffbc89e9f037d5f6a8c5874ce8794656e0c)
 2017-07-19 10:46:21 -07:00
Brahma Reddy Battula
6039a2a55f HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af59d96f1fac30cae14be1cabbdb74c8b)
 2017-07-15 11:12:56 +08:00
Xiao Chen
8b7d1df766 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:23:13 -07:00
Kihwal Lee
e28c74102d HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 
(cherry picked from commit e7d187a1b6a826edd5bd0f708184d48f3674d489)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
 2017-07-14 16:17:44 -05:00
Kihwal Lee
561b72970f HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 
(cherry picked from commit a47820d72a4c4aba857fd7325fa9ae9a95fe8faa)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2017-07-14 15:50:18 -05:00
Xiaoyu Yao
c9adca70e6 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 
(cherry picked from commit e15e2713e1e344b14d63726639d1c83451921515)
 2017-07-12 16:39:22 -07:00