Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,340 Commits 326 Branches 371 Tags
Commit Graph

13340 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sunil G
d462e48336 YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin. 2017-07-17 12:25:41 +05:30
Brahma Reddy Battula
a33e89931b HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af59d96f1fac30cae14be1cabbdb74c8b)
 2017-07-15 10:49:11 +08:00
Xiao Chen
5337c2511d HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit aa395c1ec9a170f2da732c6e6114c60412161d40)
 2017-07-14 22:23:07 -07:00
Tsz-Wo Nicholas Sze
bff39ca718 HDFS-12130. Optimizing permission check for getContentSummary. Contributed by Chen Liang 2017-07-14 14:40:11 -07:00
Tsz-Wo Nicholas Sze
d51395363a Revert "HDFS-12130. Optimizing permission check for getContentSummary." to fix commit message. 
This reverts commit 2ba41729abef22f9049fbd6260e8194ac1abf5cc.
 2017-07-14 14:38:27 -07:00
Kihwal Lee
0fd86de128 HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 
(cherry picked from commit e7d187a1b6a826edd5bd0f708184d48f3674d489)
 2017-07-14 16:12:28 -05:00
Arun Suresh
0101973dbf YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 2017-07-14 13:57:22 -07:00
Kihwal Lee
a47820d72a HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 2017-07-14 15:45:26 -05:00
Tsz-Wo Nicholas Sze
2ba41729ab HDFS-12130. Optimizing permission check for getContentSummary. 2017-07-14 13:42:02 -07:00
Jason Lowe
9eb93c6f3f YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt 
(cherry picked from commit a5ae5ac50e97cf829c41dcf01655cd9bd4d36a00)
 2017-07-14 15:03:48 -05:00
bibinchundatt
af80d10e54 YARN-6759. Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure. Contributed by Naganarasimha G R 
(cherry picked from commit 75c0220b4494dd4424a0c531e0bf0a763748dc62)
 2017-07-14 14:05:38 +05:30
Yufei Gu
b2a34fe8d6 YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu) 
(cherry picked from commit 4a574e9a84f2e997038452b22f2ad2a2d42e8ac8)
 2017-07-13 23:11:27 -07:00
Sunil G
fd6935709f YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. Contributed by Wangda Tan. 2017-07-14 08:45:17 +05:30
Sunil G
f5cdee6bfd YARN-6792. Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 228ddaa31d812533b862576445494bc2cd8a2884)
 2017-07-14 08:08:32 +05:30
Ravi Prakash
756a068143 MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286eccbc6bb8ae77584b635bfd0c48e50)
 2017-07-13 16:19:52 -07:00
Jason Lowe
a77fb561ef YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe 
(cherry picked from commit ebc048cc055d0f7d1b85bc0b6f56cd15673e837d)
 2017-07-13 17:49:55 -05:00
Jason Lowe
6ce2b0e4d4 YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles 
(cherry picked from commit 5f1ee72b0ebf0330417b7c0115083bc851923be4)
 2017-07-13 17:28:49 -05:00
Sunil G
d83e8712f6 YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 
(cherry picked from commit ca13b224b2feb9c44de861da9cbba8dd2a12cb35)
 2017-07-13 14:50:28 -05:00
Steve Loughran
df69b13b47
HADOOP-14658. branch-2 compilation is broken in hadoop-azure 
Contributed by Sunil G.

Signed-off-by: Steve Loughran <stevel@apache.org>
 2017-07-13 18:25:38 +01:00
Wangda Tan
cbb5f601f1 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda) 
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
(cherry picked from commit c583ab02c730be0a63d974039a78f2dc67dc2db6)
 2017-07-13 10:44:07 -05:00
Masatake Iwasaki
956e83c100 HADOOP-14646. FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs. Contributed by Andras Bokor. 
(cherry picked from commit b61ab8573eb2f224481118004f620fe9f18db74b)
 2017-07-13 21:43:10 +09:00
Xiaoyu Yao
af8a227a68 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 
(cherry picked from commit e15e2713e1e344b14d63726639d1c83451921515)
 2017-07-12 16:38:56 -07:00
Xuan
edc0aa95f1 YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong) 2017-07-12 15:45:51 -07:00
Tsz-Wo Nicholas Sze
b4a108fa9f HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS. Contributed by Weiwei Yang 2017-07-12 14:18:22 -07:00
Jason Lowe
401b191a3c YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe 
(cherry picked from commit 655110393bf90a7dac5d85f831ea225071f5f3c2)
 2017-07-12 15:54:17 -05:00
Junping Du
ea3dba37cd MAPREDUCE-5621. mr-jobhistory-daemon.sh doesn't have to execute mkdir and chown all the time. Contributed by Shinichi Yamashita. 2017-07-12 11:54:26 -07:00
Wangda Tan
f2d440b3b3 YARN-5889. Improve and refactor user-limit calculation in Capacity Scheduler. (Sunil G via wangda) 
(cherry picked from commit 5fb723bb77722d41df6959eee23e1b0cfeb5584e)
 2017-07-12 10:06:02 -05:00
Eric Payne
e6cdf770ca Revert "YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G)" 
This reverts commit eda4ac07c1835031aca7e27cc673f1c5913813bb.
Commit eda4ac07c1835031aca7e27cc673f1c5913813bb was a separate patch from trunk rather than a cherry-pick. I will cherryp-ick dependencies and then cherry-pick the trunk commit for YARN-2113.
 2017-07-12 08:56:40 -05:00
Akira Ajisaka
3419381e98
YARN-6809. Fix typo in ResourceManagerHA.md. Contributed by Yeliang Cang. 
(cherry picked from commit b628d0da5196faa5abf4c95dc7e1bb4eb67f6707)
 2017-07-12 19:39:03 +09:00
Steve Loughran
3dfd107070
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. 
Contributed by Varada Hemeswari
(fix for java 7 compilation)
 2017-07-12 11:07:22 +01:00
Steve Loughran
1e69e52603
HADOOP-14581. Restrict setOwner to list of user when security is enabled in wasb. 
Contributed by Varada Hemeswari
 2017-07-12 10:35:59 +01:00
Akira Ajisaka
2653cf4466
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 
(cherry picked from commit 9144fd9e9b5d84d71158451428341746a6567152)
 2017-07-12 11:40:48 +09:00
Steve Loughran
41e83b2ca2
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs. 
Contributed by Thomas Marquardt

(cherry picked from commit d670c3a4da7dd80dccf6c6308603bb3bb013b3b0)
 2017-07-11 21:45:24 +01:00
wenxinhe
e494a169ed
HADOOP-14638. Replace commons-logging APIs with slf4j in StreamPumper. 
This closes #247

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fce795101461cbce37334b0799b2238825f5a5aa)
 2017-07-11 13:36:56 +09:00
Jitendra Pandey
b82485d6fe HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R. 2017-07-10 17:55:02 -07:00
Jitendra Pandey
d94b30cb03 HADOOP-14443. Azure: Support retry and client side failover for authorization, SASKey and delegation token generation. Contributed by Santhosh G Nayak. 2017-07-10 17:30:31 -07:00
Yufei Gu
20a2770d72 YARN-6764. Simplify the logic in FairScheduler#attemptScheduling. Contributed by Yufei Gu. 
(cherry picked from commit 3de47ab5ea5cb75805a03010dc72e11b0cf6c173)
 2017-07-09 16:11:26 -07:00
Naganarasimha
5f603306d8 YARN-6770. A small mistake in the example of TimelineClient. Contributed by Jinjiang Ling. 
(cherry picked from commit ba5b056e8a151fad9d7573c6e62cffc40cde7da3)
 2017-07-09 21:31:53 +05:30
Naganarasimha
b08ef44d9e YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt. 
(cherry picked from commit 626d730bfc4c1f6ea54e8a246a0a676cdb74937f)
 2017-07-09 17:27:50 +05:30
Daniel Templeton
fcef28e53b YARN-6410. FSContext.scheduler should be final (Contributed by Yeliang Cang via Daniel Templeton) 
(cherry picked from commit 9d278804425f0118b590d302985408e0794b88bb)
 2017-07-09 19:38:37 +09:00
Daniel Templeton
a6d194d720 YARN-6746. SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code 
(Contributed by Deepti Sawhney via Daniel Templeton)

(cherry picked from commit c5a0c38d8e1752031d1dd8bdd4a7a3894cf775cb)
 2017-07-09 19:31:20 +09:00
Daniel Templeton
edcfae8258 YARN-6752. Display reserved resources in web UI per application 
(Contributed by Abdullah Yousufi via Daniel Templeton)

(cherry picked from commit 06159858868a9cbeefc31bc6be5ae48a59cc8a6c)
 2017-07-09 19:19:07 +09:00
Junping Du
f823f9fd78 MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff602d48413556a1d046670e2003c71c2e)
 2017-07-07 13:27:56 -07:00
Akira Ajisaka
1c40df5bcd
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-08 03:01:18 +09:00
Wei-Chiu Chuang
730b21e9f9 HADOOP-14563. LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException. Contributed by Rushabh S Shah. 
(cherry picked from commit 8153fe2bd35fb4df0b64f93ac0046e34d4807ac3)
 2017-07-07 06:14:07 -07:00
Eric Payne
eda4ac07c1 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G) 2017-07-06 10:43:00 -05:00
Jason Lowe
53c2441c10 YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt 
(cherry picked from commit 7576a688ea84aed7206321b1f03594e43a5f216e)
 2017-07-06 09:44:02 -05:00
Jason Lowe
f0435db089 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 2017-07-06 09:22:55 -05:00
Mingliang Liu
b803641523 HDFS-12089. Fix ambiguous NN retry log message in WebHDFS. Contributed by Eric Badger 
(cherry picked from commit 6436768baf1b2ac05f6786edcd76fd3a66c03eaa)
 2017-07-05 11:12:15 -07:00
Vinayakumar B
1e8301b215 HADOOP-13414. Hide Jetty Server version header in HTTP responses. Contributed by Surendra Singth Lilhore. 2017-07-05 16:06:19 +05:30