Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,197 Commits 326 Branches 371 Tags
Commit Graph

14197 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Varun Saxena
16942ea115 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 
(cherry picked from commit a3ac1c1943518c2a242dd21dac0f17f21bc3cd7f)
 2017-10-17 23:36:33 +05:30
Sangjin Lee
e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Xiao Chen
a103b323e6 Revert "HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar." 
This reverts commit 1452ba139ea845ad9e82a5902d239b59a789bff3.
 2017-10-17 10:11:38 -07:00
Robert Kanter
8a1ed84c77 HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter) 2017-10-16 16:51:28 -07:00
Wei-Chiu Chuang
1452ba139e HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 
(cherry picked from commit e906108fc98a011630d12a43e557b81d7ef7ea5d)
 2017-10-16 16:44:35 -07:00
Nathan Roberts
b876a9314e YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe. 
(cherry picked from commit 8620140a6a3ec0117675ede06d92d830da3da551)
 2017-10-16 16:44:34 -05:00
Junping Du
3d2ef54450 YARN-7124. LogAggregationTFileController deletes/renames while file is open. Contributed by Jason Lowe. 
(cherry picked from commit 1fcbe7cf5f2f7c609cd4158912f151848268ca9e)
 2017-10-16 14:29:02 -07:00
Xiao Chen
0bddcf10ef HADOOP-14949. TestKMS#testACLs fails intermittently. 
(cherry picked from commit 152317d40ece8abfa76fb5bd29f04878c1b0d3d8)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2017-10-16 14:20:05 -07:00
Arun Suresh
53cd33cc4a YARN-7275. NM Statestore cleanup for Container updates. (Kartheek Muthyala via asuresh) 
(cherry picked from commit a50be1b8f432f50c940b66d12c7de87b95ea47c0)
 2017-10-16 13:27:08 -07:00
Wei-Chiu Chuang
a596c6f5eb HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
 2017-10-16 12:59:41 -07:00
Xiao Chen
b2f27641dd HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 
(cherry picked from commit 971b3ca1acc15d9a1f8e774291813dafd16ef704)
 2017-10-16 10:34:24 -07:00
Xiao Chen
d82edb49c7 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 
(cherry picked from commit d0311dfc5fe92a0e0f3fb8ae92e412edce609740)
 2017-10-16 09:51:27 -07:00
Steve Loughran
8902178d3e HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled. 
Contributed by Santhosh G Nayak

Cherry-picked from commit 9fcc3a1fc8cab873034f5c308ceb2d5671a954e8
 2017-10-16 16:06:41 +01:00
Xiao Chen
4b734fe386 HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 
(cherry picked from commit 11a7729c1474aa379a62c0440c8c1a11be261a75)
 2017-10-15 19:28:38 -07:00
Wangda Tan
2c1381300f YARN-7321. Backport container-executor changes from YARN-6852 to branch-2. (Varun Vasudev via wangda) 
Change-Id: Ib6434b5a5ebe591050c3949cac12ff9e2a2bbaa5
 2017-10-14 10:48:26 -07:00
Wangda Tan
64170eb53c Addendum fix for: YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (Wangda Tan) 
Change-Id: If1fe4a62e07b25e6f1b8ba803495da87e2cb2df6
(cherry picked from commit 87ea1dff9c431fb88e064e497b35fc17c65f1d53)
 2017-10-14 10:47:54 -07:00
Yufei Gu
f2350f5120 YARN-7270. Fix unsafe casting from long to int for class Resource and its sub-classes. (Yufei) 
(cherry picked from commit 7a27c2c367518e1bf6ee393391a2f9b412113819)
(cherry picked from commit dbf6c48af9aa2f3b41b1146643db6eb0140b38c5)
 2017-10-13 13:10:20 -07:00
Chris Trezzo
3e7c06af41 MAPREDUCE-5951. Add support for the YARN Shared Cache. 
(cherry picked from commit e46d5bb962b0c942f993afc505b165b1cd96e51b)
 2017-10-12 11:30:38 -07:00
Carlo Curino
b9426c0bf4 YARN-7317. Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy. (contributed by Botong Huang via curino) 
(cherry picked from commit 13fcfb3d46ee7a0d606b4bb221d1cd66ef2a5a7c)
 2017-10-12 10:42:36 -07:00
Wangda Tan
1ab6b4a08e YARN-6033. Add support for sections in container-executor configuration file. (Varun Vasudev via wangda) 
Change-Id: Id1ded3ee11e6242dfed5616903580f329f3bb26a
 2017-10-12 09:14:08 -07:00
Anu Engineer
5933068e70 HADOOP-13102. Update GroupsMapping documentation to reflect the new changes. Contributed by Esther Kundin. 
(cherry picked from commit 075358eb6fff5ae4a40ac4dfde292e2a9a4ceddf)
 2017-10-11 16:15:59 -07:00
Arun Suresh
fb8bd4d2d7 HADOOP-13556. Change Configuration.getPropsWithPrefix to use getProps instead of iterator. (Larry McCay via asuresh) 
(cherry picked from commit b6c2c9058e83116dcca46cd6934db3428f931347)
 2017-10-11 15:22:51 -07:00
Steve Loughran
ccfd1a4a56 HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. 
Contributed by Varada Hemeswari.

Cherry-picked from commit ref 3de574413c4d5554213d02bd0ad343ba82cf82aa
 2017-10-11 18:09:13 +01:00
Rohith Sharma K S
c2b8d4b9ab MAPREDUCE-6951. Improve exception message when mapreduce.jobhistory.webapp.address is in wrong format. Contributed by Prabhu Joseph. 
(cherry picked from commit 639f98cc9dc9fca034d0d7c26f30d5c588b9d7b8)
 2017-10-11 14:38:00 +05:30
Jason Lowe
63d86ee3a5 HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp 
(cherry picked from commit 1123f8f0b62292197f5433cd40e66d8620044608)
 2017-10-10 14:50:15 -05:00
Robert Kanter
759d7d58fa YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter) 
(cherry picked from commit ec8bf9e48ac4dc675516a57c06a29890b5138548)
 2017-10-10 10:19:02 -07:00
Xuan
6cdc6771ea Merge branch 'YARN-5734-branch-2' into branch-2. (xgong) 2017-10-09 13:53:24 -07:00
Andrew Wang
1ddc49527a Revert "HDFS-12603. Enable async edit logging by default." 
This reverts commit 42d3229e6f604d13a0419ac32979500dae306066.
 2017-10-09 13:29:52 -07:00
Andrew Wang
42d3229e6f HDFS-12603. Enable async edit logging by default. 2017-10-09 11:21:43 -07:00
Jonathan Hung
4bd3f0df42 YARN-7252. Removing queue then failing over results in exception 2017-10-09 11:12:08 -07:00
Jonathan Hung
06c8938ab0 YARN-7251. Misc changes to YARN-5734 2017-10-09 11:12:08 -07:00
Jonathan Hung
40bec38465 YARN-7238. Documentation for API based scheduler configuration management 2017-10-09 11:12:08 -07:00
Jonathan Hung
c8f9fe88d5 YARN-7046. Add closing logic to configuration store 2017-10-09 11:12:08 -07:00
Wangda Tan
887b366d5f YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda) 
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
 2017-10-09 11:12:07 -07:00
Xuan
36cda2ff80 YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
3a31c38885 YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
cd58f5da5a YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
16b27573ef YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao 2017-10-09 11:12:07 -07:00
Xuan
96f4cf7cde YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong) 2017-10-09 11:12:07 -07:00
Xuan
3c50084219 YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong) 2017-10-09 11:12:07 -07:00
Wangda Tan
a47823ca98 YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda) 
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
 2017-10-09 11:12:07 -07:00
Wangda Tan
53a1268125 YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda) 2017-10-09 11:12:07 -07:00
Jonathan Hung
072527b31a YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store 2017-10-09 11:12:07 -07:00
Xuan
38cf9ccd5b YARN-5946: Create YarnConfigurationStore interface and 
InMemoryConfigurationStore class. Contributed by Jonathan Hung
 2017-10-09 11:12:07 -07:00
Jonathan Hung
cd4bc54e6c YARN-5951. Changes to allow CapacityScheduler to use configuration store 2017-10-09 11:12:07 -07:00
Daniel Templeton
4d377c89bb YARN-2612 addendum: fixed javadoc error. 
(cherry picked from commit 6d6ca4c923142a893870fc8a6309c02f60dc2d24)
(cherry picked from commit bec03f7060d353e78a68bb64c88397cf7181727d)
 2017-10-09 14:16:50 +02:00
Sunil G
045526aca1 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f4367974bf1883f1982e31ab2865107)
 2017-10-09 14:44:33 +05:30
Steve Loughran
34040e8d73 HADOOP-14899. Restrict Access to setPermission operation when authorization is enabled in WASB 
Contributed by Kannapiran Srinivasan.
 2017-10-07 11:20:18 +01:00
Xiaoyu Yao
65b5e81752 HADOOP-14920. KMSClientProvider won't work with KMS delegation token retrieved from non-Java client. Contributed by Xiaoyu Yao. 
(cherry picked from commit 2b08a1fc644904a37545107666efc25b3552542d)
 2017-10-06 10:39:37 -07:00
Yufei Gu
da0a96968d YARN-7207. Cache the RM proxy server address. (Yufei Gu) 
(cherry picked from commit 72d22b753abde4d07a727479d3f3d5d84d5dd6b2)

# Conflicts:
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-06 09:56:24 -07:00