Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,768 Commits 326 Branches 371 Tags
Commit Graph

3373 Commits

Author SHA1 Message Date
Jonathan Hung
4bd3f0df42 YARN-7252. Removing queue then failing over results in exception 2017-10-09 11:12:08 -07:00
Jonathan Hung
06c8938ab0 YARN-7251. Misc changes to YARN-5734 2017-10-09 11:12:08 -07:00
Jonathan Hung
40bec38465 YARN-7238. Documentation for API based scheduler configuration management 2017-10-09 11:12:08 -07:00
Jonathan Hung
c8f9fe88d5 YARN-7046. Add closing logic to configuration store 2017-10-09 11:12:08 -07:00
Wangda Tan
887b366d5f YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda) 
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
 2017-10-09 11:12:07 -07:00
Xuan
36cda2ff80 YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
3a31c38885 YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
cd58f5da5a YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
16b27573ef YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao 2017-10-09 11:12:07 -07:00
Xuan
96f4cf7cde YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong) 2017-10-09 11:12:07 -07:00
Xuan
3c50084219 YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong) 2017-10-09 11:12:07 -07:00
Wangda Tan
a47823ca98 YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda) 
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
 2017-10-09 11:12:07 -07:00
Wangda Tan
53a1268125 YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda) 2017-10-09 11:12:07 -07:00
Jonathan Hung
072527b31a YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store 2017-10-09 11:12:07 -07:00
Xuan
38cf9ccd5b YARN-5946: Create YarnConfigurationStore interface and 
InMemoryConfigurationStore class. Contributed by Jonathan Hung
 2017-10-09 11:12:07 -07:00
Jonathan Hung
cd4bc54e6c YARN-5951. Changes to allow CapacityScheduler to use configuration store 2017-10-09 11:12:07 -07:00
Daniel Templeton
4d377c89bb YARN-2612 addendum: fixed javadoc error. 
(cherry picked from commit 6d6ca4c923142a893870fc8a6309c02f60dc2d24)
(cherry picked from commit bec03f7060d353e78a68bb64c88397cf7181727d)
 2017-10-09 14:16:50 +02:00
Sunil G
045526aca1 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f4367974bf1883f1982e31ab2865107)
 2017-10-09 14:44:33 +05:30
Yufei Gu
da0a96968d YARN-7207. Cache the RM proxy server address. (Yufei Gu) 
(cherry picked from commit 72d22b753abde4d07a727479d3f3d5d84d5dd6b2)

# Conflicts:
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-06 09:56:24 -07:00
Yufei Gu
c949e8e83a YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu) 2017-10-06 00:35:03 -07:00
Arun Suresh
cb91bb3c96 YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh). 
(cherry picked from commit b733348dde18a242e6c9074c512116a8baf1d281)
 2017-10-05 15:22:20 -07:00
Naganarasimha
5cb3bd3b06 YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe. 
(cherry picked from commit d0046bb5c6919c9620ffa5330eb37b39484151ad)
 2017-10-06 00:09:46 +05:30
Chris Trezzo
d12cea2098 YARN-2960. Add documentation for the YARN shared cache. 
(cherry picked from commit 7e76f85bc68166b01b51fcf6ba4b3fd9281d4a03)
 2017-10-05 11:23:24 -07:00
Subru Krishnan
7fd4a997f4 YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit e6e614e380ed1d746973b50f666a9c40d272073e)
 2017-10-04 19:30:57 -07:00
Subru Krishnan
7836a6b59a YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru) 2017-10-04 16:29:08 -07:00
Akira Ajisaka
5508b9b7c8
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails. 
(cherry picked from commit 2df1b2ac0509ba10fff606ada7e9b3562c12dd16)
 2017-10-05 06:08:46 +09:00
Sidharta S
3ecbfff937 YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe 2017-10-04 14:05:01 -07:00
Wangda Tan
175ff0585d YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (wangda via Jian He) 
Change-Id: I582566e09d5ef67e9cc9e9805231ccbabe57c3d0
 2017-10-03 12:19:11 -07:00
Wangda Tan
42e873d4ea YARN-6625. yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment. (Yufei Gu) 2017-10-03 12:04:56 -07:00
Wangda Tan
598329e6f8 YARN-6550. Capture launch_container.sh logs to a separate log file. (Suma Shivaprasad via wangda) 
Change-Id: I66c9b50ae3955dda0b5bf0674a9bfb728db27acb
 2017-10-03 09:46:35 -07:00
Wangda Tan
8beae14a07 YARN-7259. Add size-based rolling policy to LogAggregationIndexedFileController. (xgong via wangda) 
Change-Id: Ifaf82c0aee6b73b9b6ebf103aa72e131e3942f31
(cherry picked from commit 280080fad01304c85a9ede4d4f7b707eb36c0155)
 2017-10-02 15:36:11 -07:00
Yufei Gu
e84d508c1b YARN-6333. Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold. (Contributed by Chetna Chaudhari via Yufei Gu) 
(cherry picked from commit 66c417167ae0d24467360ae3b222a10f00fa2700)
 2017-09-29 18:08:55 -07:00
Wangda Tan
5fa5ba7f4d YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda) 
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
(cherry picked from commit ec2ae3060a807c8754826af2135a68c08b2e4f13)
(cherry picked from commit d58d75ca501c6d1bb20605c1c964c0d351012763)
 2017-09-29 16:53:16 -07:00
Eric Payne
b0ba31c518 YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918e0fbe81cb64e8b9f6959b8d66679f6)
 2017-09-29 10:50:21 -05:00
Jason Lowe
c0bfb385a8 YARN-7260. yarn.router.pipeline.cache-max-size is missing in yarn-default.xml. Contributed by Jason Lowe 2017-09-29 09:53:05 -05:00
Chris Trezzo
cba1891b61 YARN-7250. Update Shared cache client api to use URLs. 
(cherry picked from commit c114da5e64d14b1d9e614081c4171ea0391cb1aa)
 2017-09-28 15:32:18 -07:00
Subru Krishnan
ffcf5ba1ce YARN-6962. Add support for updateContainers when allocating using FederationInterceptor. (Botong Huang via Subru). 
(cherry picked from commit ca669f9f8bc7abe5b7d4648c589aa1756bd336d1)
 2017-09-28 13:11:32 -07:00
Subru Krishnan
d6da014f67 YARN-6955. Handle concurrent register AM requests in FederationInterceptor. (Botong Huang via Subru). 
(cherry picked from commit c61f2c419830e40bb47fb2b1fe1f7d6109ed29a9)
 2017-09-28 13:11:19 -07:00
Jason Lowe
913a64e4c9 YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh 
(cherry picked from commit 85d81ae58ec4361a944c84753a900460a0888b9b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-28 14:25:01 -05:00
Arun Suresh
53c1115908 YARN-6691. Update YARN daemon startup/shutdown scripts to include Router service. (Giovanni Matteo Fumarola via asuresh) 2017-09-28 11:41:43 -07:00
Xuan
c143708acf YARN-7257. AggregatedLogsBlock reports a bad 'end' value as a bad 
'start' value. Contributed by Jason Lowe

(cherry picked from commit 28c4957fccebe2d7e63ec9fe9af58313b4f21d4f)
(cherry picked from commit 81e78727938a1c0369f164849cc3a08add31be46)
 2017-09-27 16:04:06 -07:00
Rohith Sharma K S
c570ddaa66 YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G. 2017-09-27 11:49:59 +05:30
Jason Lowe
c4de765fcb Revert "YARN-6570. No logs were found for running application, running" 
This reverts commit 6aaca3d08edf02925bcc0b4521b21b6bf9aacb99.
 2017-09-26 10:44:16 -05:00
Rohith Sharma K S
c27f682032 YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R. 
(cherry picked from commit 06e5a7b5cf141420d3a411088b87acba72e68cad)
 2017-09-26 14:50:34 +05:30
Wangda Tan
179d8dc83c YARN-7118. AHS REST API can return NullPointerException. (Billie Rinaldi via wangda) 
Change-Id: I16ca93385ab80e3680d4d173b8d1b79929ea7174
 2017-09-25 21:41:00 -07:00
Arun Suresh
e7199cb494 YARN-7240. Add more states and transitions to stabilize the NM Container state machine. (Kartheek Muthyala via asuresh) 
(cherry picked from commit df800f6cf3ea663daf4081ebe784808b08d9366d)
 2017-09-25 16:36:58 -07:00
Akira Ajisaka
c0dacda546
YARN-7153. Remove duplicated code in AMRMClientAsyncImpl.java. Contributed by Sen Zhao. 
(cherry picked from commit e928ee583c5a1367e24eab34057f8d8496891452)
 2017-09-26 02:05:35 +09:00
Anu Engineer
d0c0bd573b Revert "YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi." 
This reverts commit de1d747644727ed7c469570bad56400741fb034e.
 2017-09-23 11:12:16 -07:00
Junping Du
de1d747644 YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi. 
(cherry picked from commit 4002bf0a9e7e3619a3bdcff071f0369cbf2873ad)
 2017-09-22 13:59:03 -07:00
Carlo Curino
7cd9018b13 [YARN FEDERATION BACKPORT] Fixing more Java 1.7 compilation issues 2017-09-21 18:19:36 -07:00