13670 Commits

Author SHA1 Message Date
Kihwal Lee
9fe5dd098d HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp.
(cherry picked from commit 1a79dcfc457969d6a6c08ffffe4152fd7638e48a)
2017-07-25 10:30:27 -05:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor.
(cherry picked from commit cca51e916b7387ea358688e8f8188ead948fbdcc)
2017-07-25 15:25:36 +09:00
Akira Ajisaka
09241feb13
YARN-6150. TestContainerManagerSecurity tests for Yarn Server are flakey. Contributed by Daniel Sturman and Ray Chiang.
(cherry picked from commit 218b1b33ffe83cf2e330a2aa90685d0c14547a3d)
2017-07-25 15:12:47 +09:00
Daniel Templeton
9b6e787230 YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue.
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit 10583625c1c803fc243adf6479cb9435af7e72da)
2017-07-24 13:45:09 -07:00
Daniel Templeton
eb1751670a YARN-6779. DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting
(Contributed by Yeliang Cang via Daniel Templeton)

(cherry picked from commit bb30bd3771442df253cbe55c448379580bd5ad07)
2017-07-24 12:15:35 -07:00
Daniel Templeton
6b8e353c65 YARN-6844. AMRMClientImpl.checkNodeLabelExpression() has wrong error message
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit 4c40cd451cbdbce5d2b94ad0e7e3cc991c3439c5)
2017-07-24 12:06:50 -07:00
Sunil G
a657472b42 YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S. 2017-07-24 20:57:56 +05:30
Jason Lowe
db7c5636b6 YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
2017-07-24 10:22:31 -05:00
Akira Ajisaka
c108372777
HDFS-12176. dfsadmin shows DFS Used%: NaN% if the cluster has zero block. Contributed by Weiwei Yang.
(cherry picked from commit 770cc462281518545e3d1c0f8c21cf9ec9673200)
2017-07-24 16:24:50 +09:00
Allen Wittenauer
daa7685344
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw)
(cherry picked from commit d8c1fd1944160ad5b5d093731ae987d701802321)
2017-07-24 11:57:10 +09:00
wenxinhe
7e583a3824
HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.
This closes #251

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-07-24 11:14:08 +09:00
Kihwal Lee
c12bf9a128 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee
(cherry picked from commit 8e3a992eccff26a7344c3f0e719898fa97706b8c)
2017-07-21 09:19:01 -05:00
Konstantin V Shvachko
af2fb6a909 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang.
(cherry picked from commit 2a5a313539e211736fef12010918a60f9edad030)
2017-07-20 15:57:34 -07:00
Kihwal Lee
a735916b2c HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp.
(cherry picked from commit b3269f7cc119cbc51b16118b918f345f970804bc)
2017-07-20 16:15:06 -05:00
Jason Lowe
e76e7b9f0b YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)
2017-07-20 11:06:39 -05:00
Wangda Tan
5070c9ba39 YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda)
Change-Id: Iaf9bb7e5ed3aa1300abdccf87ae6fcbddbd25e3e
2017-07-19 15:25:21 -07:00
Daniel Templeton
1d629c95af HADOOP-14666. Tests use assertTrue(....equals(...)) instead of assertEquals()
(cherry picked from commit c21c26039238f633a0d2df9670f636d026c35649)
2017-07-19 14:01:10 -07:00
Yongjun Zhang
10a6b6b12c HDFS-12139. HTTPFS liststatus returns incorrect pathSuffix for path of file. Contributed by Yongjun Zhang.
(cherry picked from commit 3556e36be30211f46ac38899ce11a4d4cac6d635)
2017-07-19 11:44:35 -07:00
Jitendra Pandey
23b920cd7a HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 11:37:36 -07:00
Anu Engineer
e0297ffbc8 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh.
(cherry picked from commit 413b23eb04eee24275257ab462133e0818f87449)
2017-07-19 10:42:28 -07:00
Brahma Reddy Battula
979d37ae21 HDFS-12067. Correct dfsadmin commands usage message to reflects IPC port. Contributed by steven-wugang.
(cherry picked from commit f8cd55fe33665faf2d1b14df231516fc891118fc)
2017-07-19 23:22:40 +08:00
Jason Lowe
057631c031 HADOOP-14669. GenericTestUtils.waitFor should use monotonic time. Contributed by Daniel Templeton
(cherry picked from commit df180259b0cc3660e199e85447c7193bee51751c)
2017-07-19 09:42:38 -05:00
Rohith Sharma K S
f806e8627b YARN-6819. Application report fails if app rejected due to nodesize. Contributed by Bibin A Chundatt.
(cherry picked from commit 845c4e52bdd579a24df5dbba7477b0ebf2fa16f1)
2017-07-19 11:16:17 +05:30
Yufei Gu
d04bb465d2 YARN-6778. In ResourceWeights, weights and setWeights() should be final. (Daniel Templeton via Yufei Gu)
(cherry picked from commit daaf530fce4b91cf9f568b9b0c5e8b20e6774134)
2017-07-18 16:40:13 -07:00
Jason Lowe
9b961db569 HADOOP-14637. GenericTestUtils.waitFor needs to check condition again after max wait time. Contributed by Daniel Templeton
(cherry picked from commit 5aa2bf231f40423865f0054ca27426ceb95ab4ba)
2017-07-18 16:27:34 -05:00
Sunil G
487b0a3d7f Addendum patch for YARN-5731
(cherry picked from commit 0b7afc060c2024a882bd1934d0f722bfca731742)
2017-07-18 13:46:43 +05:30
Jitendra Pandey
b9f2effaf6 HADOOP-14640. Azure: Support affinity for service running on localhost and reuse SPNEGO hadoop.auth cookie for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-17 16:25:05 -07:00
Tsz-Wo Nicholas Sze
02627f8c5f HDFS-12138. Remove redundant 'public' modifiers from BlockCollection. Contributed by Chen Liang 2017-07-17 13:59:52 -07:00
Sunil G
958835a3dc MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S.
(cherry picked from commit fb3b5d33ffb29ee8e1ffbd2eee7a603a5777ebaf)
2017-07-17 13:36:00 +05:30
Sunil G
9b1abb448d YARN-6714. IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by Tao Yang. 2017-07-17 13:22:04 +05:30
Sunil G
d462e48336 YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin. 2017-07-17 12:25:41 +05:30
Brahma Reddy Battula
a33e89931b HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang.
(cherry picked from commit b778887af59d96f1fac30cae14be1cabbdb74c8b)
2017-07-15 10:49:11 +08:00
Xiao Chen
5337c2511d HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah.
(cherry picked from commit aa395c1ec9a170f2da732c6e6114c60412161d40)
2017-07-14 22:23:07 -07:00
Tsz-Wo Nicholas Sze
bff39ca718 HDFS-12130. Optimizing permission check for getContentSummary. Contributed by Chen Liang 2017-07-14 14:40:11 -07:00
Tsz-Wo Nicholas Sze
d51395363a Revert "HDFS-12130. Optimizing permission check for getContentSummary." to fix commit message.
This reverts commit 2ba41729abef22f9049fbd6260e8194ac1abf5cc.
2017-07-14 14:38:27 -07:00
Kihwal Lee
0fd86de128 HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp.
(cherry picked from commit e7d187a1b6a826edd5bd0f708184d48f3674d489)
2017-07-14 16:12:28 -05:00
Arun Suresh
0101973dbf YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 2017-07-14 13:57:22 -07:00
Kihwal Lee
a47820d72a HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 2017-07-14 15:45:26 -05:00
Tsz-Wo Nicholas Sze
2ba41729ab HDFS-12130. Optimizing permission check for getContentSummary. 2017-07-14 13:42:02 -07:00
Jason Lowe
9eb93c6f3f YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt
(cherry picked from commit a5ae5ac50e97cf829c41dcf01655cd9bd4d36a00)
2017-07-14 15:03:48 -05:00
bibinchundatt
af80d10e54 YARN-6759. Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure. Contributed by Naganarasimha G R
(cherry picked from commit 75c0220b4494dd4424a0c531e0bf0a763748dc62)
2017-07-14 14:05:38 +05:30
Yufei Gu
b2a34fe8d6 YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu)
(cherry picked from commit 4a574e9a84f2e997038452b22f2ad2a2d42e8ac8)
2017-07-13 23:11:27 -07:00
Sunil G
fd6935709f YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. Contributed by Wangda Tan. 2017-07-14 08:45:17 +05:30
Sunil G
f5cdee6bfd YARN-6792. Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo. Contributed by Giovanni Matteo Fumarola.
(cherry picked from commit 228ddaa31d812533b862576445494bc2cd8a2884)
2017-07-14 08:08:32 +05:30
Ravi Prakash
756a068143 MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin
(cherry picked from commit 43f0503286eccbc6bb8ae77584b635bfd0c48e50)
2017-07-13 16:19:52 -07:00
Jason Lowe
a77fb561ef YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe
(cherry picked from commit ebc048cc055d0f7d1b85bc0b6f56cd15673e837d)
2017-07-13 17:49:55 -05:00
Jason Lowe
6ce2b0e4d4 YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles
(cherry picked from commit 5f1ee72b0ebf0330417b7c0115083bc851923be4)
2017-07-13 17:28:49 -05:00
Sunil G
d83e8712f6 YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne.
(cherry picked from commit ca13b224b2feb9c44de861da9cbba8dd2a12cb35)
2017-07-13 14:50:28 -05:00
Steve Loughran
df69b13b47
HADOOP-14658. branch-2 compilation is broken in hadoop-azure
Contributed by Sunil G.

Signed-off-by: Steve Loughran <stevel@apache.org>
2017-07-13 18:25:38 +01:00
Wangda Tan
cbb5f601f1 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda)
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
(cherry picked from commit c583ab02c730be0a63d974039a78f2dc67dc2db6)
2017-07-13 10:44:07 -05:00