Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,641 Commits 326 Branches 371 Tags
Commit Graph

1927 Commits

Author SHA1 Message Date
Robert Kanter
d92dddaf7c YARN-7358. TestZKConfigurationStore and TestLeveldbConfigurationStore should explicitly set capacity scheduler (haibochen via rkanter) 
(cherry picked from commit 2da654e34a436aae266c1fbdec5c1067da8d854e)
 2017-10-26 10:45:23 -07:00
Subru Krishnan
926b188871 YARN-4827. Document configuration of ReservationSystem for FairScheduler. (Yufei Gu via Subru). 
(cherry picked from commit 3fae675383489129b3ca3c66683a1215d0c6edf0)
 2017-10-25 15:08:19 -07:00
Yufei Gu
6e953eb6a3 YARN-7045. Remove FSLeafQueue#addAppSchedulable. (Contributed by Sen Zhao via Yufei Gu) 
(cherry picked from commit a92ef030a2707182e90acee644e47c8ef7e1fd8d)
 2017-10-24 13:52:02 -07:00
Robert Kanter
c5eeb2e4ce YARN-7385. TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE (yufeigu via rkanter) 
(cherry picked from commit 03af442e7608db2f8e6eb85a15aa0ba42781edab)
 2017-10-24 13:40:06 -07:00
Robert Kanter
4bae02a97d YARN-7382. NoSuchElementException in FairScheduler after failover causes RM crash (rkanter) 
(cherry picked from commit 025c6565725c1819566377632753e8b9055617a6)
 2017-10-24 10:25:43 -07:00
Daniel Templeton
9da9023371 YARN-7357. Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations 
(Contributed by Sen Zhao via Daniel Templeton)

(cherry picked from commit 0553f5a012e14c8ceb40a16161440384f9f3a9a8)
 2017-10-23 13:51:39 -07:00
Eric Payne
3f315a526b YARN-4163: Audit getQueueInfo and getApplications calls 
(cherry picked from commit 921338cd86e7215b0c4b1efdf2daf9449fb12c7b)
 2017-10-23 11:48:41 -05:00
Sunil G
4b872e5e9f YARN-6871. Add additional deSelects params in RMWebServices#getAppReport. Contributed by Tanuj Nayak. 
(cherry picked from commit 8facf1f976d7e12a846f12baabf54be1b7a49f9d)
 2017-10-20 17:20:39 -07:00
Yufei Gu
96106b8f5f YARN-4090. Make Collections.sort() more efficient by caching resource usage. (Contributed by Yufei Gu, Shilong Zhang and Xianyin Xin) 
(cherry picked from commit 1f4cdf10681b6903207a63fb5c306c9665ed9464)
 2017-10-20 01:34:05 -07:00
Yufei Gu
b2f516118c YARN-7359. TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic. (Contributed by Haibo Chen) 
(cherry picked from commit 7b4b0187806601e33f5a88d48991e7c12ee4419f)
 2017-10-19 16:53:10 -07:00
Yufei Gu
288d2fc309 YARN-7294. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair Scheduler. (Contributed by Miklos Szegedi) 
(cherry picked from commit cbd2b73ef81a7e275c5d4f842cac5b81ff2f8c84)
 2017-10-19 16:40:36 -07:00
Subru Krishnan
81a86860bf YARN-7311. Fix TestRMWebServicesReservation parametrization for fair scheduler. (Yufei Gu via Subru). 
(cherry picked from commit 75323394fbc4211596a2c8fbb5e584f3183f742f)
 2017-10-17 12:39:40 -07:00
Robert Kanter
6971699099 YARN-7308. TestApplicationACLs fails with FairScheduler (rkanter) 
(cherry picked from commit 8a615259281511de639c1f9aa719a842e34a523f)
 2017-10-16 15:34:50 -07:00
Robert Kanter
b7cef4e934 YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter) 
(cherry picked from commit ec8bf9e48ac4dc675516a57c06a29890b5138548)
 2017-10-13 13:57:40 -07:00
Jonathan Hung
043a217017 YARN-7252. Removing queue then failing over results in exception 2017-10-09 14:54:57 -07:00
Jonathan Hung
168a3df8e6 YARN-7251. Misc changes to YARN-5734 2017-10-09 14:54:57 -07:00
Jonathan Hung
5105bc4721 YARN-7046. Add closing logic to configuration store 2017-10-09 14:54:56 -07:00
Wangda Tan
e6827a9121 YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda) 
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
 2017-10-09 14:54:56 -07:00
Xuan
4bf38062a5 YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung 2017-10-09 14:54:56 -07:00
Xuan
0b546420da YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung 2017-10-09 14:54:55 -07:00
Xuan
793476e4b7 YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung 2017-10-09 14:54:55 -07:00
Xuan
09b7c8766e YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao 2017-10-09 14:54:54 -07:00
Xuan
e09f0d64ac YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong) 2017-10-09 14:54:54 -07:00
Xuan
3f29c1585b YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong) 2017-10-09 14:54:54 -07:00
Wangda Tan
5fd1668a27 YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda) 
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
 2017-10-09 14:54:53 -07:00
Wangda Tan
d1514bacd1 YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda) 2017-10-09 14:54:53 -07:00
Jonathan Hung
28754646e5 YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store 2017-10-09 14:54:53 -07:00
Xuan
c4c2feeccf YARN-5946: Create YarnConfigurationStore interface and 
InMemoryConfigurationStore class. Contributed by Jonathan Hung
 2017-10-09 14:54:52 -07:00
Jonathan Hung
72169a0281 YARN-5951. Changes to allow CapacityScheduler to use configuration store 2017-10-09 14:54:52 -07:00
Daniel Templeton
bec03f7060 YARN-2612 addendum: fixed javadoc error. 
(cherry picked from commit 6d6ca4c923142a893870fc8a6309c02f60dc2d24)
 2017-10-09 14:15:52 +02:00
Sunil G
84e2e2a6f6 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f4367974bf1883f1982e31ab2865107)
 2017-10-09 14:40:44 +05:30
Yufei Gu
4cfc7d0af6 YARN-7207. Cache the RM proxy server address. (Yufei Gu) 
(cherry picked from commit 72d22b753abde4d07a727479d3f3d5d84d5dd6b2)

# Conflicts:
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-06 10:03:48 -07:00
Yufei Gu
9bb9831347 YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu) 2017-10-06 00:29:15 -07:00
Arun Suresh
b61d12e300 YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh). 
(cherry picked from commit b733348dde18a242e6c9074c512116a8baf1d281)
 2017-10-05 10:06:30 -07:00
Subru Krishnan
829a8e26b9 YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit e6e614e380ed1d746973b50f666a9c40d272073e)
 2017-10-04 19:29:03 -07:00
Subru Krishnan
8d59b1fbee YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit fa6137501c1499ae33f6e0e2adc31671a7e782dc)
 2017-10-04 16:41:18 -07:00
Subru Krishnan
eb0a00c9b5 YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 
(cherry picked from commit b6e7d1369690eaf50ce9ea7968f91a72ecb74de0)
 2017-10-04 16:41:18 -07:00
Subru Krishnan
b5acbfef48 YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru). 
(cherry picked from commit d4d2fd1acd2fdddf04f45e67897804eea30d79a1)
 2017-10-04 16:33:39 -07:00
Akira Ajisaka
fe0d6f5b1c
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails. 
(cherry picked from commit 2df1b2ac0509ba10fff606ada7e9b3562c12dd16)
 2017-10-05 06:08:05 +09:00
Akira Ajisaka
b658ae002f
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 
(cherry picked from commit c9b525bd943fe7e098e72fb9be5623862c7ab5ff)
 2017-10-04 07:02:31 +09:00
Eric Payne
7a9479597f YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918e0fbe81cb64e8b9f6959b8d66679f6)
 2017-09-29 10:39:01 -05:00
Jason Lowe
a1e9739730 YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh 
(cherry picked from commit 85d81ae58ec4361a944c84753a900460a0888b9b)
 2017-09-28 14:14:14 -05:00
Andrew Wang
5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Rohith Sharma K S
52571d9fa3 YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R. 
(cherry picked from commit 06e5a7b5cf141420d3a411088b87acba72e68cad)
 2017-09-26 14:46:47 +05:30
Arun Suresh
2ccfe92a4e YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 
(cherry picked from commit 56ef5279c1db93d03b2f1e04badbfe804f548918)
 2017-09-18 15:50:39 -07:00
Rohith Sharma K S
508e93e48c YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G. 2017-09-18 22:56:52 +10:00
Wangda Tan
e3a0104980 YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda) 
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
 2017-09-17 21:19:20 -07:00
Wangda Tan
03b6e95fa5 YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda) 
Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e
(cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)
 2017-09-17 21:19:20 -07:00
Robert Kanter
d3d32d3580 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:48:59 -07:00
Yufei Gu
8d49c2fc3b YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d508292ed88eda0f5356776437ba67d4c)
 2017-09-14 15:48:48 -07:00