Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,389 Commits 326 Branches 371 Tags
Commit Graph

9389 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin V Shvachko
66ee9f554e HDFS-10843. Add missing file to commit 4b5806217e6846bcecc6480af46c0e70102daac0. 2016-09-23 11:13:25 -07:00
Konstantin V Shvachko
4b5806217e HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen. 
(cherry picked from commit a5bb88c8e0fd4bd19b6d377fecbe1d2d441514f6)
 2016-09-23 10:56:15 -07:00
Mingliang Liu
2a1e48bd2f HADOOP-13601. Fix a log message typo in AbstractDelegationTokenSecretManager. Contributed by Mehran Hassani. 
(cherry picked from commit e80386d69d5fb6a08aa3366e42d2518747af569f)
 2016-09-20 13:25:01 -07:00
Jason Lowe
d6c36c2954 YARN-5540. Scheduler spends too much time looking at empty priorities. Contributed by Jason Lowe 2016-09-19 20:37:22 +00:00
Xiao Chen
81ac06039e HADOOP-13558. UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket. Contributed by Xiao Chen. 
(cherry picked from commit 680be58aac03a9ffab6b07c8fde9602ddb9dc858)
(cherry picked from commit d157733082697e67be56f516606a0127f5830c58)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java

(cherry picked from commit a7f1dc8aa8c602faf9745588cba1e337f0e59afd)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
 2016-09-19 12:56:02 -07:00
Kihwal Lee
a8b7817739 HDFS-9621. Consolidate FSDirStatAndListingOp#createFileStatus to let its INodesInPath parameter always include the target INode. Contributed by Jing Zhao. 
(cherry picked from commit 313f03bfdab32cf365bc3470c5f9b6928a24f099)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java

(cherry picked from commit c2708bc5fbf212ae6c48084442748c3265ddccd7)
 2016-09-15 11:05:52 -07:00
Haohui Mai
6f44510d87 HDFS-8200. Refactor FSDirStatAndListingOp. Contributed by Haohui Mai. 
(cherry picked from commit 01acf3a3af5727bfdc0a37f9caf4cf8e1be89224)
 2016-09-15 10:54:15 -07:00
Kihwal Lee
199ccd4915 HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp. 
(cherry picked from commit 1ef8d7a638df5150b8426755af034839d5f88ca2)
(cherry picked from commit 3dac73080a7cab3652512d56510235fdef157fa7)
 2016-09-13 21:58:58 -07:00
Tsz-Wo Nicholas Sze
9afc6615b0 In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin 
(cherry picked from commit cefa21e98a12b06602ee8000f8cef6c3b17af999)
 2016-09-12 23:28:34 -07:00
Kihwal Lee
a39a9fc46b HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp. 
(cherry picked from commit 438a9f047eb6af2a4b916a4f6ef6f68adeab8068)
(cherry picked from commit 09b06a6e9e873ad05fc2ebfaf0e9e9eba71516d2)
(cherry picked from commit f09b04b3f5252b0482e7ead5ed2661578cd4c56c)
 2016-09-12 15:57:48 -07:00
Tsz-Wo Nicholas Sze
32d810133c HDFS-8818. Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable. 
(cherry picked from commit ac8d153046d542e9d13cb3f67c0461764eaf0a26)
 2016-09-12 15:01:58 -07:00
Xiaoyu Yao
d7da70332f HDFS-9019. Adding informative message to sticky bit permission denied exception. Contributed by Xiaoyu Yao. 
(cherry picked from commit 970daaa5e44d3c09afd46d1c8e923a5096708c44)
(cherry picked from commit b2465690a15ee4257b381a8d186ffa6981ddc537)
 2016-09-12 13:16:31 -07:00
Yongjun Zhang
e6c162a394 HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang) 
(cherry picked from commit 5ea6fd85c7aff6df28b87789f607bb57ee920639)
(cherry picked from commit 3e4c7906c2cf5c3e4c708fc56b670fa788e8cec7)
 2016-09-12 13:02:56 -07:00
Yongjun Zhang
97e4b13038 HDFS-5802. NameNode does not check for inode type before traversing down a path. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 9859fd006b638d94c9842a719f12c51159746fc8)
 2016-09-12 12:58:00 -07:00
Akira Ajisaka
e807fde448 HADOOP-13579. Fix source-level compatibility after HADOOP-11252. Contributed by Tsuyoshi Ozawa. 2016-09-09 14:27:57 +09:00
Arpit Agarwal
607801b2ff HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-08 13:40:01 -07:00
Kihwal Lee
67204f2834 HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp. 
(cherry picked from commit 6ae39199dac6ac7be6802b31452552c76da16e24)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java

(cherry picked from commit 77b61d1f4e34de9fed3dd6167fa93bb35e85e038)
(cherry picked from commit 58df27b87388dbb9729862a58dcdbe0b57e8b1ab)
 2016-09-01 15:09:02 -07:00
Zhe Zhang
6793ef0de6 HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen. 
(cherry picked from commit 6f4b0d33ca339e3724623a1d23c101f8cfd3cdd5)
(cherry picked from commit f87e2f5d05bdb54ce5028d2651c8947fa0f80539)
(cherry picked from commit 7adbd569a6715e29205865b27c4d9bd3af56284f)
 2016-08-31 23:57:38 -07:00
Kihwal Lee
73ba5a0170 HDFS-10674. Optimize creating a full path from an inode. Contributed by Daryn Sharp. 
(cherry picked from commit 22ef5286bc8511ddee9594b7cecc598bf41a850b)
(cherry picked from commit a5d12d9c1f2c0e6fcd918ee8e614dcaf203e77de)
(cherry picked from commit e53f6fde465e5a1a97d8c4eebe7f7897b0875f1c)
 2016-08-31 15:30:45 -07:00
Kihwal Lee
d3c6120576 HDFS-10656. Optimize conversion of byte arrays back to path string. Contributed by Daryn Sharp. 
(cherry picked from commit bebf10d2455cad1aa8985553417d4d74a61150ee)
(cherry picked from commit ab7072838f8bb848b5f7b3b10ed36f7480da8683)
(cherry picked from commit 7b5e12229d72f87667ba91c9259d35abd8f1b5c6)
 2016-08-31 15:29:26 -07:00
Daryn Sharp
319a6005e6 HDFS-10655. Fix path related byte array conversion bugs. (daryn) 
(cherry picked from commit 9f473cf903e586c556154abd56b3a3d820c6b028)
(cherry picked from commit ecb143ed21f4971cc3e3d4827ba8becb1864f171)
 2016-08-30 15:52:18 -07:00
Zhe Zhang
85a067fd6b HDFS-10809. getNumEncryptionZones causes NPE in branch-2.7. Contributed by Vinitha Reddy Gankidi. 2016-08-30 11:46:52 -07:00
Jason Lowe
d499a8cccf MAPREDUCE-4784. TestRecovery occasionally fails. Contributed by Haibo Chen 
(cherry picked from commit f70f522e805b2d8dd49bae07fc65d37a78ad2fca)
 2016-08-30 14:11:49 +00:00
Kihwal Lee
a9479f8f2c HADOOP-12418. TestRPC.testRPCInterruptedSimple fails intermittently. Contributed Kihwal Lee. 
Cherry-picked from 01b103f4ff2e8ee7e71d082885436c5cb7c6be0b
 2016-08-30 12:52:31 +00:00
Zhe Zhang
32a86f199c YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung. 
(cherry picked from commit ed6ff5cd789cab621fbfb88c8e8de4f215cd2961)
(cherry picked from commit 3ad4f0823c22b686120e8fa06ac52d705defcc61)
(cherry picked from commit 765139e4dadff707411fc50c9a0799aef340d742)
 2016-08-29 14:52:09 -07:00
Yongjun Zhang
4fd2adbc5f HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin. 
(cherry picked from commit 5d1609ddf275e4907bd224bf618e2aad4b262888)
 2016-08-29 14:25:49 -07:00
Jason Lowe
a852800d2f MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen 
(cherry picked from commit 44d0af2611cb565d8ec68f92e27937342646cca5)
 2016-08-29 20:02:47 +00:00
Mingliang Liu
3112a2447f HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu) 2016-08-29 10:22:55 -07:00
Zhe Zhang
1d01704060 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. Branch-2 patch contributed by Wei-Chiu Chuang. 
(cherry picked from commit dfcbc12026b591745a7d7279f2b840152cb53a91)
(cherry picked from commit 8bc33bf343b7e9005e04f0dc6078bfb06fb22815)
 2016-08-29 10:10:47 -07:00
Zhe Zhang
4f43607311 HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen. 
(cherry picked from commit 407b519fb14f79f19ebc4fbdf08204336a7acf77)
(cherry picked from commit f3a4cbcc60572c73e0d15580bae42a36c92fe247)
(cherry picked from commit 96cf681b5d9aca71548390cb258f8d0ca2131082)
 2016-08-26 14:34:28 -07:00
Kihwal Lee
8593225e85 HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki. 
(cherry picked from commit 13fb1b50e608558b2970184908ee5b9fcd7eb7b6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystem.java
Updated CHANGES.txt
 2016-08-26 10:09:57 -05:00
Jing Zhao
8aa18534d9 HDFS-9467. Fix data race accessing writeLockHeldTimeStamp in FSNamesystem. Contributed by Mingliang Liu. 
(cherry picked from commit e556c35b0596700f9ec9d0a51cf5027259d531b5)
(cherry picked from commit 288cf8437b7e03f071e95eb05e83a26e58fff26b)
 2016-08-25 10:34:43 -07:00
Haohui Mai
0d8a1636ef HDFS-9145. Tracking methods that hold FSNamesytemLock for too long. Contributed by Mingliang Liu. 
(cherry picked from commit 5c5362980c429a5ad0b58e0dd1933de9cde7f369)
 2016-08-25 10:18:50 -07:00
Xiaoyu Yao
e5cb9d9e18 HDFS-8883. NameNode Metrics : Add FSNameSystem lock Queue Length. Contributed by Anu Engineer. 
(cherry picked from commit a7862d5fe4c505f5d4b0c675438a971733f1f53a)
(cherry picked from commit 27ccbd51f6827d4e4a26c2026c024a3400ab96f2)
 2016-08-25 10:10:18 -07:00
cnauroth
a1d8e421f6 HDFS-8721. Add a metric for number of encryption zones. Contributed by Rakesh R. 
(cherry picked from commit cb03768b1b2250b9b5a7944cf6ef918e8a974e20)
(cherry picked from commit 57d55d40dd41ba4449b91c4676131b6b840052c4)
 2016-08-25 09:39:27 -07:00
Zhe Zhang
5fe29062eb HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh. 
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
(cherry picked from commit 6d4a4e785b254a0761a6373ff1376fe64ecf6398)
 2016-08-23 15:09:05 -07:00
Jason Lowe
cec6083055 HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen 
(cherry picked from commit 126d165efd80e266a8309241f3cf059e358f5019)
 2016-08-23 14:51:29 +00:00
Wei-Chiu Chuang
cdc48c5d5e HDFS-8224. Schedule a block for scanning if its metadata file is 
corrupt. Contributed by Rushabh S Shah.
 2016-08-23 07:28:21 -07:00
Varun Saxena
30a4c0bb63 MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:18:25 +05:30
Kihwal Lee
d04fd76956 Revert "HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla." 
This reverts commit 25b3531eb40a32a602574e5cfc1ffe028044bcc9.
Reverting HDFS-9395 from branch-2.7 as it is an incompatible change.
 2016-08-18 17:20:20 -05:00
Kihwal Lee
e3b55237b0 Fix misplaced entry in CHANGES.txt 2016-08-18 16:49:53 -05:00
Kihwal Lee
61e1348fe4 HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee. 2016-08-18 16:39:35 -05:00
Mingliang Liu
eae9b691ca HADOOP-13512. ReloadingX509TrustManager should keep reloading in case of exception. (Contributed by Mingliang Liu) 
(cherry picked from commit 0f51eae0c085ded38216824377acf8122638c3a5)
 2016-08-18 14:23:22 -07:00
Kihwal Lee
6593851fa6 Revert "HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee." 
This reverts commit 71d0e4fca79d4305fe35e44b614703d3b9883017.
 2016-08-18 13:04:20 -05:00
Andrew Wang
d59f688992 HADOOP-13494. ReconfigurableBase can log sensitive information. Contributed by Sean Mackrory. 2016-08-17 13:17:52 -07:00
Zhe Zhang
040a1b7b90 HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. 
(cherry picked from commit 293c763e82586b4f3230771a86734aae593f468d)
 2016-08-16 10:35:52 -07:00
Kihwal Lee
9f5a22ff22 HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 
(cherry picked from commit d2f3bbc29046435904ad9418073795439c71b441)
(cherry picked from commit ccaf746eeacfafc7392a994f28ee9dbf595c84af)
 2016-08-16 10:26:33 -07:00
Mingliang Liu
de96efec25 HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu) 2016-08-15 21:10:32 -07:00
Kihwal Lee
71d0e4fca7 HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee. 2016-08-15 17:40:07 -05:00
Kihwal Lee
7519956c30 HDFS-9696. Garbage snapshot records linger forever. Adding an entry to CHANGES.txt. 2016-08-15 15:50:54 -05:00