Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,691 Commits 326 Branches 371 Tags
Commit Graph

13691 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Junping Du
cae33098e6 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit e8a31f2e1c34514fba2f480e8db652f6e2ed65d8.

(cherry picked from commit fbb7d6bcbb887ce52ab1e9d5a1fed67a7f8a4be8)
 2017-07-31 14:15:06 -07:00
Junping Du
bf360da63f Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.
 2017-07-31 14:13:26 -07:00
Junping Du
c0f8fd6432 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f5290f0991f65102c2bd413f965a8ca409.
 2017-07-31 14:11:37 -07:00
Arpit Agarwal
8c63580981 HDFS-12082. BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured. Contributed by Weiwei Yang. 2017-07-31 11:34:00 -07:00
Akira Ajisaka
b19415f1a1
HADOOP-14677. mvn clean compile fails. Contributed by Andras Bokor. 
(cherry picked from commit 0fd6d0f2d361536b2baf859ddbb082eb2eadcfcf)
 2017-07-31 17:01:46 +09:00
Akira Ajisaka
9c8a296169
HADOOP-14690. RetryInvocationHandler should override toString(). Contributed by Yeliang Cang. 
(cherry picked from commit f14be0d24126747887ddc7580f4a9a70768de23d)
 2017-07-31 14:10:18 +09:00
Akira Ajisaka
98c3544e94
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
 2017-07-31 11:10:10 +09:00
Brahma Reddy Battula
8bfb9971ca HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-30 14:01:41 +08:00
Arun Suresh
7e643130f2 YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh) 
(cherry picked from commit 890e14c02a612c772cecd5dff2411060efd418a3)
 2017-07-28 16:33:42 -07:00
John Zhuge
ba94c68e0b HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 
(cherry picked from commit f735ad1b67ed82d9b11b1afd7ae39035a6aed18b)
 2017-07-28 09:51:32 -07:00
Yufei Gu
e29d1c75ea YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit 9902be72cbf7a170caa5cb1f13c227d881a39064)
 2017-07-27 23:21:30 -07:00
Yongjun Zhang
c3e8c7d56e HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 
(cherry picked from commit c6330f22a5e5c2370bab885f9bea4bf8f5e9cf44)
 2017-07-27 17:05:30 -07:00
Brahma Reddy Battula
1c5095b3cb HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)
 2017-07-27 12:30:57 -07:00
Varun Saxena
05bbbcbdbe Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit 11ece0bda1f6e5dd9d0f828b7c29acacf6087baa)
 2017-07-28 00:33:41 +05:30
Varun Saxena
0b342a644a YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit f66fd11e514fb326fd1f37a88b444a5276f0947b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
 2017-07-28 00:33:37 +05:30
Kihwal Lee
4edb849611 HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. Contributed by Daryn Sharp. 
(cherry picked from commit 27a1a5fde94d4d7ea0ed172635c146d594413781)
 2017-07-26 13:16:34 -05:00
Sunil G
08d5f3acf6 Addendum patch for YARN-6102. 2017-07-26 18:40:52 +05:30
Jitendra Pandey
554c3cd21d HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt. 2017-07-25 16:35:40 -07:00
Daniel Templeton
00f4a80e8f YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit f81a4efb8c40f99a9a6b7b42d3b6eeedf43eb27a)
 2017-07-25 13:02:00 -07:00
Kihwal Lee
3d8c7d701b HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 2017-07-25 11:07:43 -05:00
Jitendra Pandey
ab95086ed1 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-25 08:31:35 -07:00
Kihwal Lee
9fe5dd098d HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 
(cherry picked from commit 1a79dcfc457969d6a6c08ffffe4152fd7638e48a)
 2017-07-25 10:30:27 -05:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 
(cherry picked from commit cca51e916b7387ea358688e8f8188ead948fbdcc)
 2017-07-25 15:25:36 +09:00
Akira Ajisaka
09241feb13
YARN-6150. TestContainerManagerSecurity tests for Yarn Server are flakey. Contributed by Daniel Sturman and Ray Chiang. 
(cherry picked from commit 218b1b33ffe83cf2e330a2aa90685d0c14547a3d)
 2017-07-25 15:12:47 +09:00
Daniel Templeton
9b6e787230 YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue. 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit 10583625c1c803fc243adf6479cb9435af7e72da)
 2017-07-24 13:45:09 -07:00
Daniel Templeton
eb1751670a YARN-6779. DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting 
(Contributed by Yeliang Cang via Daniel Templeton)

(cherry picked from commit bb30bd3771442df253cbe55c448379580bd5ad07)
 2017-07-24 12:15:35 -07:00
Daniel Templeton
6b8e353c65 YARN-6844. AMRMClientImpl.checkNodeLabelExpression() has wrong error message 
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit 4c40cd451cbdbce5d2b94ad0e7e3cc991c3439c5)
 2017-07-24 12:06:50 -07:00
Sunil G
a657472b42 YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S. 2017-07-24 20:57:56 +05:30
Jason Lowe
db7c5636b6 YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
 2017-07-24 10:22:31 -05:00
Akira Ajisaka
c108372777
HDFS-12176. dfsadmin shows DFS Used%: NaN% if the cluster has zero block. Contributed by Weiwei Yang. 
(cherry picked from commit 770cc462281518545e3d1c0f8c21cf9ec9673200)
 2017-07-24 16:24:50 +09:00
Allen Wittenauer
daa7685344
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw) 
(cherry picked from commit d8c1fd1944160ad5b5d093731ae987d701802321)
 2017-07-24 11:57:10 +09:00
wenxinhe
7e583a3824
HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common. 
This closes #251

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-07-24 11:14:08 +09:00
Kihwal Lee
c12bf9a128 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992eccff26a7344c3f0e719898fa97706b8c)
 2017-07-21 09:19:01 -05:00
Konstantin V Shvachko
af2fb6a909 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539e211736fef12010918a60f9edad030)
 2017-07-20 15:57:34 -07:00
Kihwal Lee
a735916b2c HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp. 
(cherry picked from commit b3269f7cc119cbc51b16118b918f345f970804bc)
 2017-07-20 16:15:06 -05:00
Jason Lowe
e76e7b9f0b YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)
 2017-07-20 11:06:39 -05:00
Wangda Tan
5070c9ba39 YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda) 
Change-Id: Iaf9bb7e5ed3aa1300abdccf87ae6fcbddbd25e3e
 2017-07-19 15:25:21 -07:00
Daniel Templeton
1d629c95af HADOOP-14666. Tests use assertTrue(....equals(...)) instead of assertEquals() 
(cherry picked from commit c21c26039238f633a0d2df9670f636d026c35649)
 2017-07-19 14:01:10 -07:00
Yongjun Zhang
10a6b6b12c HDFS-12139. HTTPFS liststatus returns incorrect pathSuffix for path of file. Contributed by Yongjun Zhang. 
(cherry picked from commit 3556e36be30211f46ac38899ce11a4d4cac6d635)
 2017-07-19 11:44:35 -07:00
Jitendra Pandey
23b920cd7a HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 11:37:36 -07:00
Anu Engineer
e0297ffbc8 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04eee24275257ab462133e0818f87449)
 2017-07-19 10:42:28 -07:00
Brahma Reddy Battula
979d37ae21 HDFS-12067. Correct dfsadmin commands usage message to reflects IPC port. Contributed by steven-wugang. 
(cherry picked from commit f8cd55fe33665faf2d1b14df231516fc891118fc)
 2017-07-19 23:22:40 +08:00
Jason Lowe
057631c031 HADOOP-14669. GenericTestUtils.waitFor should use monotonic time. Contributed by Daniel Templeton 
(cherry picked from commit df180259b0cc3660e199e85447c7193bee51751c)
 2017-07-19 09:42:38 -05:00
Rohith Sharma K S
f806e8627b YARN-6819. Application report fails if app rejected due to nodesize. Contributed by Bibin A Chundatt. 
(cherry picked from commit 845c4e52bdd579a24df5dbba7477b0ebf2fa16f1)
 2017-07-19 11:16:17 +05:30
Yufei Gu
d04bb465d2 YARN-6778. In ResourceWeights, weights and setWeights() should be final. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit daaf530fce4b91cf9f568b9b0c5e8b20e6774134)
 2017-07-18 16:40:13 -07:00
Jason Lowe
9b961db569 HADOOP-14637. GenericTestUtils.waitFor needs to check condition again after max wait time. Contributed by Daniel Templeton 
(cherry picked from commit 5aa2bf231f40423865f0054ca27426ceb95ab4ba)
 2017-07-18 16:27:34 -05:00
Sunil G
487b0a3d7f Addendum patch for YARN-5731 
(cherry picked from commit 0b7afc060c2024a882bd1934d0f722bfca731742)
 2017-07-18 13:46:43 +05:30
Jitendra Pandey
b9f2effaf6 HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-17 16:25:05 -07:00
Tsz-Wo Nicholas Sze
02627f8c5f HDFS-12138. Remove redundant 'public' modifiers from BlockCollection. Contributed by Chen Liang 2017-07-17 13:59:52 -07:00
Sunil G
958835a3dc MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S. 
(cherry picked from commit fb3b5d33ffb29ee8e1ffbd2eee7a603a5777ebaf)
 2017-07-17 13:36:00 +05:30