Subru Krishnan
b9dcf9283e
YARN-5602. Utils for Federation State and Policy Store. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 326a2e6bde
)
(cherry picked from commit e1da8f0667
)
2017-09-21 16:25:10 -07:00
Carlo Curino
b40bdafe31
YARN-6370. Properly handle rack requests for non-active subclusters in LocalityMulticastAMRMProxyPolicy. (Contributed by Botong Huang via curino).
...
(cherry picked from commit ce419881c3
)
(cherry picked from commit 1c64e1709b
)
2017-09-21 16:25:10 -07:00
Subru Krishnan
61c07e4f3f
YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru)
...
(cherry picked from commit 51aeb2ce0c
)
(cherry picked from commit 86b2bec56e
)
2017-09-21 16:25:09 -07:00
Carlo Curino
6191fac914
YARN-6190. Validation and synchronization fixes in LocalityMulticastAMRMProxyPolicy. (Botong Huang via curino)
...
(cherry picked from commit 5c486961cd
)
(cherry picked from commit 8623644f45
)
2017-09-21 16:25:08 -07:00
Subru Krishnan
2dca88b924
YARN-5410. Bootstrap Router server module. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 04f111394b
)
(cherry picked from commit 193ec456d6
)
2017-09-21 16:25:02 -07:00
Subru Krishnan
ce9110ab5c
YARN-6093. Minor bugs with AMRMtoken renewal and state store availability when using FederationRMFailoverProxyProvider during RM failover. (Botong Huang via Subru).
...
(cherry picked from commit 66500f4fa6
)
(cherry picked from commit 98b45b0ed3
)
2017-09-21 16:23:47 -07:00
Subru Krishnan
002a77dcce
YARN-5872. Add AlwayReject policies for router and amrmproxy. (Carlo Curino via Subru).
...
(cherry picked from commit db26bade05
)
(cherry picked from commit 91803305e5
)
2017-09-21 16:23:46 -07:00
Subru Krishnan
e933a1790c
YARN-5905. Update the RM webapp host that is reported as part of Federation membership to current primary RM's IP.
...
(cherry picked from commit e0f0e7afed
)
(cherry picked from commit 78032e51d9
)
2017-09-21 16:23:45 -07:00
Subru Krishnan
d32ffa9e5e
YARN-5676. Add a HashBasedRouterPolicy, and small policies and test refactoring. (Carlo Curino via Subru).
...
(cherry picked from commit 575137f41c
)
(cherry picked from commit 4128c9522d
)
2017-09-21 16:23:45 -07:00
Subru Krishnan
d87d2b546e
YARN-5634. Simplify initialization/use of RouterPolicy via a RouterPolicyFacade. (Carlo Curino via Subru).
...
(cherry picked from commit d7672ce2bd
)
(cherry picked from commit 083a05bc14
)
2017-09-21 16:23:44 -07:00
Subru Krishnan
b3a1ab711c
YARN-5391. PolicyManager to tie together Router/AMRM Federation policies. (Carlo Curino via Subru).
...
(cherry picked from commit 20893682ec
)
(cherry picked from commit 93d9fdeca6
)
2017-09-21 16:23:43 -07:00
Subru Krishnan
5c5ab53f3d
YARN-5325. Stateless ARMRMProxy policies implementation. (Carlo Curino via Subru).
...
(cherry picked from commit 11c5336522
)
(cherry picked from commit 1dadd0b45a
)
2017-09-21 16:23:42 -07:00
Subru Krishnan
f792f7b068
YARN-5324. Stateless Federation router policies implementation. (Carlo Curino via Subru).
...
(cherry picked from commit 1298127bda
)
(cherry picked from commit 0662996b6a
)
2017-09-21 16:23:42 -07:00
Subru Krishnan
23c42408ba
YARN-5323. Policies APIs for Federation Router and AMRMProxy policies. (Carlo Curino via Subru).
...
(cherry picked from commit f8208fe0b5
)
(cherry picked from commit 81472778d7
)
2017-09-21 16:23:41 -07:00
Jian He
aac8755125
YARN-5601. Make the RM epoch base value configurable. Contributed by Subru Krishnan
...
(cherry picked from commit 9ca2aba9cc
)
(cherry picked from commit 2797507d51
)
2017-09-21 16:23:40 -07:00
Subru Krishnan
8409fef0d9
YARN-5612. Return SubClusterId in FederationStateStoreFacade#addApplicationHomeSubCluster for Router Failover. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit ac1ba2a304
)
(cherry picked from commit 1882bc10fb
)
2017-09-21 16:23:39 -07:00
Jian He
ea2ecdb796
YARN-3671. Integrate Federation services with ResourceManager. Contributed by Subru Krishnan
...
(cherry picked from commit 8573c286e2
)
(cherry picked from commit 94a24567d6
)
2017-09-21 16:23:33 -07:00
Jian He
615c912b61
YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan
...
(cherry picked from commit 3307564a5f
)
(cherry picked from commit bdfad4523f
)
2017-09-21 16:13:06 -07:00
Subru Krishnan
32a8618f39
YARN-5467. InputValidator for the FederationStateStore internal APIs. (Giovanni Matteo Fumarola via Subru)
...
(cherry picked from commit bd44182e70
)
(cherry picked from commit cfafd173bd
)
2017-09-21 16:13:06 -07:00
Jian He
fb3f9b48fb
YARN-3674. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan
...
(cherry picked from commit 5e976cd2b9
)
(cherry picked from commit 5c84382397
)
2017-09-21 16:12:34 -07:00
Subru Krishnan
1889dfe409
YARN-5519. Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover. (Ellen Hui via Subru)
...
(cherry picked from commit e4f928cf9c
)
(cherry picked from commit 52558df620
)
2017-09-21 16:00:40 -07:00
Subru Krishnan
cf15768f45
YARN-5407. In-memory based implementation of the FederationApplicationStateStore/FederationPolicyStateStore. (Ellen Hui via Subru)
...
(cherry picked from commit b747d59f41
)
(cherry picked from commit 0733088c29
)
2017-09-21 16:00:39 -07:00
Subru Krishnan
4c3d45a1cf
YARN-5408. Compose Federation membership/application/policy APIs into an uber FederationStateStore API. (Ellen Hui via Subru).
...
(cherry picked from commit 268389d021
)
(cherry picked from commit e0c3a44396
)
2017-09-21 16:00:38 -07:00
Subru Krishnan
d0dddb4ebd
YARN-3664. Federation PolicyStore internal APIs
...
(cherry picked from commit 2f377263a1
)
(cherry picked from commit f1a508bdef
)
2017-09-21 16:00:38 -07:00
Subru Krishnan
07834ad678
YARN-5307. Federation Application State Store internal APIs
...
(cherry picked from commit 1af5292042
)
(cherry picked from commit 047a16bf15
)
2017-09-21 16:00:37 -07:00
Subru Krishnan
31d92378ba
YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui.
...
(cherry picked from commit d3dc461a93
)
(cherry picked from commit d19b677301
)
2017-09-21 16:00:36 -07:00
Subru Krishnan
8b6a2afa6e
YARN-5406. In-memory based implementation of the FederationMembershipStateStore. Contributed by Ellen Hui.
...
(cherry picked from commit 20d1d2be91
)
(cherry picked from commit dcf8c52d80
)
2017-09-21 16:00:35 -07:00
Subru Krishnan
a939d2fdb8
YARN-3662. Federation Membership State Store internal APIs.
...
(cherry picked from commit 5424d0899c
)
(cherry picked from commit 3e22896f86
)
2017-09-21 16:00:35 -07:00
Subru Krishnan
194524bdf8
YARN-5300. Exclude generated federation protobuf sources from YARN Javadoc/findbugs build
...
(cherry picked from commit 9d23570172
)
(cherry picked from commit dd75ffb26b
)
2017-09-21 16:00:34 -07:00
Jason Lowe
fc6d9612c7
YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger.
...
(cherry picked from commit bfd1a72ba8
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
2017-09-21 17:52:11 -05:00
Junping Du
e67c8347c4
YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi.
2017-09-21 14:12:57 -07:00
Yufei Gu
c3c8b0ffab
YARN-7045. Remove FSLeafQueue#addAppSchedulable. (Contributed by Sen Zhao via Yufei Gu)
...
(cherry picked from commit a92ef030a2
)
2017-09-21 12:40:37 -07:00
Jason Lowe
ae42bf73c0
YARN-6968. Hardcoded absolute pathname in DockerLinuxContainerRuntime. Contributed by Eric Badger
...
(cherry picked from commit 10d7493587
)
2017-09-21 08:39:31 -05:00
Sangjin Lee
a79422f717
YARN-6771. Use classloader inside configuration class to make new
...
classes. Contributed by Jongyoul Lee.
(cherry picked from commit 8b336632ac
)
2017-09-20 22:46:10 -07:00
Junping Du
bdd8433d4c
YARN-7186. Fix finicky TestContainerManager tests. Contributed by Arun Suresh.
...
(cherry picked from commit 647b7527a9
)
2017-09-19 18:36:25 -07:00
Yufei Gu
64878193d5
YARN-6499. Remove the doc about Schedulable#redistributeShare(). (Contributed by Chetna Chaudhari via Yufei Gu)
...
(cherry picked from commit a9019e1fb7
)
2017-09-19 18:29:24 -07:00
Arun Suresh
8a12a914fd
YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh)
...
(cherry picked from commit 56ef5279c1
)
2017-09-18 15:51:19 -07:00
Xuan
6aaca3d08e
YARN-6570. No logs were found for running application, running
...
container. Contributed by Junping Du
(cherry picked from commit 7c732924a8
)
(cherry picked from commit e91447ccc9
)
2017-09-18 14:05:36 -07:00
Wangda Tan
74708d3817
YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda)
...
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
(cherry picked from commit 38c14ef8d8
)
2017-09-18 13:58:51 -05:00
Jason Lowe
cd6cf0caf6
YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh
...
(cherry picked from commit a4f9c7c924
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
2017-09-18 10:34:53 -05:00
Junping Du
c34c823d26
YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong.
...
(cherry picked from commit 1a84c24b0c
)
2017-09-15 15:34:00 -07:00
Robert Kanter
4e36452d02
YARN-7162. Remove XML excludes file format (rkanter)
2017-09-15 09:53:55 -07:00
Sunil G
317af87add
Addendum patch for YARN-7163 in branch-2
2017-09-15 11:34:14 +05:30
Arun Suresh
b93a23f411
YARN-6059. Update paused container state in the NM state store. (Hitesh Sharma via asuresh)
...
(cherry picked from commit 66ca0a6540
)
2017-09-14 10:30:30 -07:00
Arun Suresh
52bf458616
YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh)
...
(cherry picked from commit 4f8194430f
)
2017-09-14 10:29:54 -07:00
Arun Suresh
1af9f962c9
YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh)
...
(cherry picked from commit 864fbacd45
)
2017-09-14 10:26:34 -07:00
Junping Du
7e6cc17133
YARN-7194. Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat. Contributed by Xuan Gong.
...
(cherry picked from commit c92c1d521e
)
2017-09-13 22:18:39 -07:00
Eric Payne
fb3d53e001
YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe.
...
(cherry picked from commit 3860be7961
)
2017-09-13 15:07:27 -05:00
Nathan Roberts
1d33290a0a
YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts)
...
(cherry picked from commit d265459024
)
2017-09-13 09:47:55 -07:00
Varun Vasudev
4647761341
YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas.
...
(cherry picked from commit 972da46cb4
)
2017-09-13 18:15:02 +05:30
Robert Kanter
055bde9806
YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen via rkanter)
...
(cherry picked from commit 44fdf00964
)
2017-09-13 16:48:43 +09:00
Arun Suresh
0d32a38ac3
YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix.
...
(cherry picked from commit a1c398e0dc
)
2017-09-12 17:05:01 -07:00
Arun Suresh
d735c1f26b
YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh)
...
(cherry picked from commit 2ae72692fc
)
2017-09-12 16:33:05 -07:00
Arun Suresh
7803a93fff
YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh)
...
(cherry picked from commit 5b007921cd
)
2017-09-12 16:33:05 -07:00
Yufei Gu
1d40bc225a
YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu)
...
(cherry picked from commit 82c5dd1d50
)
2017-09-12 16:14:31 -07:00
Wangda Tan
10803bf08d
YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda)
...
Change-Id: Ia7d61e0d9df1e703bc983a31e6856e84a5a0521c
2017-09-11 20:56:17 -07:00
Daniel Templeton
fac4172e02
YARN-6622. Document Docker work as experimental (Contributed by Varun Vasudev)
...
(cherry picked from commit 6651cbcc72
)
2017-09-11 16:16:15 -07:00
Daniel Templeton
2e5a48afe3
Revert "YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev)"
...
This reverts commit 1acd004727
.
2017-09-11 16:15:06 -07:00
Daniel Templeton
1acd004727
YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev)
...
(cherry picked from commit 66be6c848b
)
2017-09-11 15:48:21 -07:00
Wangda Tan
c093e4622c
YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda)
...
Change-Id: I8fa7d70526505d4082ac9b51cd6066376a599108
2017-09-11 11:34:51 -07:00
Yufei Gu
416a44004f
YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu)
...
(cherry picked from commit 5ab21dfe9c
)
2017-09-10 23:24:14 -07:00
Xuan
02ae3f8202
YARN-7072. Add a new log aggregation file format controller (xgong)
2017-09-10 23:18:31 -07:00
Sunil G
23dc6ef8b0
YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S.
2017-09-11 10:34:59 +05:30
Subru Krishnan
2c051dbe9e
YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru).
...
(cherry picked from commit fa6137501c
)
2017-09-08 14:44:46 -07:00
Chris Douglas
2f8825588e
YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency
...
(cherry picked from commit d383bfdcd4
)
2017-09-08 14:44:34 -07:00
Nathan Roberts
b3a478832a
YARN-6219. Fix conf usage in NMWebServer unit tests. Contributed by Jason Lowe.
2017-09-08 16:01:06 -05:00
Arun Suresh
2b09cec218
YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit 1f53ae7972
)
2017-09-08 09:26:50 -07:00
Sunil G
a39829c456
YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S.
2017-09-08 19:20:52 +05:30
Jason Lowe
c004d57d47
YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf
2017-09-07 16:24:55 -05:00
Sunil G
ef342455cb
YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne.
...
(cherry picked from commit c1386a90d7
)
2017-09-07 14:47:07 -05:00
Arun Suresh
48279f7876
YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit c41118a7f8
)
2017-09-07 10:30:41 -07:00
Sunil G
8cce19241e
YARN-6992. Kill application button is visible even if the application is FINISHED in RM UI. Contributed by Suma Shivaprasad.
...
(cherry picked from commit 6f101e7df1
)
2017-09-07 13:18:30 +05:30
Subru Krishnan
5ccdd83923
YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem.
...
(cherry picked from commit b6e7d13696
)
2017-09-06 16:46:49 -07:00
Junping Du
1ef48f1d4b
YARN-7144. Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Contributed by Xuan Gong.
...
(cherry picked from commit 22de9449f8
)
2017-09-06 14:54:29 -07:00
Jason Lowe
fc444da891
YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe
...
(cherry picked from commit 704267cb49
)
2017-09-06 15:25:37 -05:00
Junping Du
eaf134c46c
YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong.
...
(cherry picked from commit 63231a2a30
)
2017-09-06 13:20:58 -07:00
Wei-Chiu Chuang
92d9ad7761
Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem."
...
This reverts commit 80516b3de7
.
2017-09-05 14:52:43 -07:00
Subru Krishnan
80516b3de7
Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem.
...
(cherry picked from commit 7996eca7dc
)
2017-09-01 15:17:27 -07:00
Oleg Danilov
b616462eef
YARN-7023. Incorrect ReservationId.compareTo() implementation.
...
This closes #262
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e1
)
2017-09-01 21:36:12 +02:00
Jason Lowe
57b5b186ce
YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne
...
(cherry picked from commit 99a7f5d451
)
2017-09-01 11:10:40 -05:00
Junping Du
119220b88f
YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong.
2017-09-01 03:04:55 -07:00
Daniel Templeton
004231dc08
YARN-6780. ResourceWeights.toString() cleanup (Contributed by weiyuan via Daniel Templeton)
...
(cherry picked from commit d4417dae4f
)
2017-08-31 09:38:31 -07:00
Sunil G
e8b4427259
YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan.
...
(cherry picked from commit f9e0cc8cdc
)
2017-08-31 18:43:17 +05:30
Robert Kanter
8523319e15
YARN-7094. Document the current known issue with server-side NM graceful decom (rkanter)
...
(cherry picked from commit ce79f7be29
)
2017-08-30 18:06:03 -07:00
Rohith Sharma K S
877d96b83d
YARN-6982. Potential issue on setting AMContainerSpec#tokenConf to null before app is completed. Contributed by Manikandan R.
...
(cherry picked from commit 4cae120c61
)
2017-08-30 10:50:57 +05:30
Jian He
036cbd7453
YARN-6756. ContainerRequest#executionTypeRequest causes NPE. Contributed by Jian He
2017-08-29 20:36:23 -07:00
Wangda Tan
9bcd64d233
YARN-5219. When an export var command fails in launch_container.sh, the full container launch should fail. (Sunil G via wangda)
...
Change-Id: Iaa6b978bb89482e9d1d77ba57f4adfdc48e39a3c
(cherry picked from commit f59332b97b
)
2017-08-29 15:29:53 -07:00
Junping Du
b906bddd95
YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang.
...
(cherry picked from commit ad45d19998
)
2017-08-29 15:17:21 -07:00
Jason Lowe
c930aa3ff9
YARN-5816. TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey. Contributed by Robert Kanter
2017-08-29 10:02:44 -05:00
Junping Du
f252e1fe28
YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He.
...
(cherry picked from commit 312b1fd9da
)
2017-08-28 18:27:38 -07:00
Wangda Tan
82e1cd08eb
YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda)
...
Change-Id: I432f4523921666ac218fcf9936f826a0d8d635e0
(cherry picked from commit 51881a863c
)
2017-08-28 11:38:05 -07:00
Yufei Gu
221c1bdf7e
YARN-7099. ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. (Contributed by Miklos Szegedi via Yufei Gu)
...
(cherry picked from commit d5bc958279
)
2017-08-28 10:11:01 -07:00
Sunil G
80570737a8
YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne.
...
(cherry picked from commit 02599bda04
)
2017-08-28 12:27:29 +05:30
Li Lu
b7f319c53c
YARN-6999. Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. (Linlin Zhou via gtcarrera9)
...
(cherry picked from commit 3bee7046c5
)
2017-08-25 22:45:20 -07:00
Eric Payne
1617744801
YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe.
...
(cherry picked from commit e864f81471
)
2017-08-25 15:52:29 -05:00
Jason Lowe
ea5bb5898f
YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne
...
(cherry picked from commit 39a9dc8e4a
)
2017-08-25 15:32:29 -05:00
Jason Lowe
82957ce0d0
YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang
...
(cherry picked from commit 3a4e861169
)
2017-08-25 09:20:50 -05:00
Yufei Gu
389b659c94
YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu)
...
(cherry picked from commit 9e2699ac2c
)
2017-08-24 17:54:00 -07:00
Junping Du
e12358c5ce
YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong.
...
(cherry picked from commit c2cb7ea1ef
)
2017-08-24 15:27:40 -07:00
Akira Ajisaka
bf903396a0
YARN-7047. Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager. Contributed by Yeliang Cang.
2017-08-25 03:28:02 +09:00
bibinchundatt
cc50ca0722
YARN-7074. Fix NM state store update comment. Contributed by Botong Huang
...
(cherry picked from commit de0cba700b
)
2017-08-24 16:40:18 +05:30
Junping Du
c98917f798
YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan.
...
(cherry picked from commit 652dd434d9
)
2017-08-23 18:07:07 -07:00
Wangda Tan
245b49332d
YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda)
...
Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb
(cherry picked from commit f49843a988
)
2017-08-23 10:35:37 -07:00
Subru Krishnan
a45ffdcddc
YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru).
...
(cherry picked from commit 4249172e14
)
2017-08-22 19:21:35 -07:00
Ray Chiang
2299c8dffa
YARN-6798. Fix NM startup failure with old state store due to version mismatch. (Botong Huang via rchiang)
...
(cherry picked from commit f5f14a2ad6
)
2017-08-22 15:24:02 -07:00
Jason Lowe
b15a393440
YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger
2017-08-22 16:28:08 -05:00
Jason Lowe
a7893033fe
YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp
...
(cherry picked from commit 657dd59cc8
)
2017-08-22 13:17:31 -05:00
Jason Lowe
fc7df25a11
YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles
...
(cherry picked from commit 3efcd51c3b
)
2017-08-22 12:59:47 -05:00
Arun Suresh
e0ba8a9ed8
YARN-6979. [Addendum patch] Fixed classname and added javadocs. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit 7a82d7bcea
)
2017-08-20 10:28:42 -07:00
Arun Suresh
f8f193c8e8
YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit 8410d862d3
)
2017-08-20 08:31:17 -07:00
Yufei Gu
4bd417a056
YARN-6969. Clean up unused code in class FairSchedulerQueueInfo. (Larry Lo via Yufei Gu)
...
(cherry picked from commit 8991f0baec
)
2017-08-18 14:45:13 -07:00
Daniel Templeton
78f9dca2cb
YARN-6952. Enable scheduling monitor in FS. (Yufei Gu via Daniel Templeton)
2017-08-18 14:40:30 -07:00
bibinchundatt
ecf1a029bb
YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su.
...
(cherry picked from commit e05fa3451d
)
2017-08-18 20:31:26 +05:30
Sunil G
5511cea86b
YARN-3254. HealthReport should include disk full information. Contributed by Suma Shivaprasad.
...
(cherry picked from commit f9a0e23381
)
2017-08-17 15:10:07 +05:30
Naganarasimha
fa3094e436
YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung.
...
(cherry picked from commit 0a3aa40fe7
)
2017-08-17 10:42:00 +05:30
Jason Lowe
cf30380d46
YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter
...
(cherry picked from commit 14553061be
)
2017-08-16 13:06:13 -05:00
bibinchundatt
67f9968407
YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R.
2017-08-16 14:52:09 +05:30
Akira Ajisaka
a28a3dc8f2
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka.
...
(cherry picked from commit 588c190afd
)
2017-08-16 14:07:08 +09:00
Subru Krishnan
ad46a90872
HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru).
...
(cherry picked from commit 75dd866bfb
)
2017-08-15 16:54:48 -07:00
Arun Suresh
7b22df3da6
YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit 4d7be1d857
)
2017-08-14 19:57:34 -07:00
Jason Lowe
56d0e757c7
YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne
...
(cherry picked from commit 5558792894
)
2017-08-14 15:34:50 -05:00
Jason Lowe
e5ba40cc98
YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles
...
(cherry picked from commit 3325ef653d
)
2017-08-14 14:41:38 -05:00
Subru Krishnan
a70efb6138
HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru).
2017-08-14 11:03:50 -07:00
Carlo Curino
394573780b
YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino)
...
(cherry picked from commit 28d97b79b6
)
2017-08-14 11:01:56 -07:00
Jian He
630ada1b4e
YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang
2017-08-14 10:56:39 -07:00
Daniel Templeton
7d3cea232d
Revert "YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator"
...
This reverts commit ce0cdc50d9
.
2017-08-11 20:52:07 -07:00
Daniel Templeton
ce0cdc50d9
YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator
...
(Contributed by Larry Lo via Daniel Templeton)
(cherry picked from commit 0996acde6c
)
2017-08-11 14:51:42 -07:00
Daniel Templeton
11e3673748
YARN-6967. Limit application attempt's diagnostic message size thoroughly
...
(Contributed by Chengbing Liu via Daniel Templeton)
(cherry picked from commit 65364defb4
)
2017-08-11 14:31:11 -07:00
Daniel Templeton
2566e5fcf7
YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces
...
(Contributed by weiyuan via Daniel Templeton)
(cherry picked from commit c7680d4cc4
)
2017-08-11 14:23:17 -07:00
Karthik Kambatla
f6c74ac5ea
YARN-5927. BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate. (Kai Sasaki via kasha)
...
(cherry picked from commit 8c4b6d16a5
)
2017-08-11 12:18:29 -07:00
Akira Ajisaka
1f80e05eb4
YARN-6958. Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice. Contributed by Yeliang Cang.
2017-08-10 22:14:44 +09:00
Jian He
8b7590057e
YARN-6804. Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi
2017-08-09 11:12:38 -07:00
Subru Krishnan
e9ec1e8f87
YARN-6897. Refactoring RMWebServices by moving some util methods to RMWebAppUtil. (Giovanni Matteo Fumarola via Subru).
2017-08-08 14:01:22 -07:00
Daniel Templeton
c5b3d80a4b
YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code
...
(Contributed by Angela Wang via Daniel Templeton)
(cherry picked from commit e0c24145d2
)
2017-08-08 13:36:49 -07:00
Junping Du
e9c21d14b4
YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S.
2017-08-08 12:45:55 -07:00
Daniel Templeton
675d368950
YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path
...
(Contributed by Miklos Szegedi via Daniel Templeton)
(cherry picked from commit 47b145b9b4
)
2017-08-08 11:12:34 -07:00
Jian He
ad1567c487
YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du
2017-08-08 11:10:29 -07:00
Akira Ajisaka
9fd5a03611
YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang.
2017-08-08 19:28:33 +09:00
Arun Suresh
3f50800d41
YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh)
...
(cherry picked from commit 8d3fd81980
)
2017-08-07 19:01:13 -07:00
Wangda Tan
e18415a350
YARN-4161. Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration. (Wei Yan via wangda)
...
Change-Id: Ic441ae4e0bf72e7232411eb54243ec143d5fd0d3
(cherry picked from commit adb84f34db
)
2017-08-07 11:34:45 -07:00
Akira Ajisaka
cfdf29785b
YARN-6957. Moving logging APIs over to slf4j in hadoop-yarn-server-sharedcachemanager. Contributed by Yeliang Cang.
...
(cherry picked from commit b0fbf17965
)
2017-08-07 19:26:18 +09:00
Sunil G
d15bac141e
YARN-6951. Fix debug log when Resource Handler chain is enabled. Contributed by Yang Wang.
...
(cherry picked from commit 46b7054fa7
)
2017-08-07 13:17:04 +05:30
Yufei Gu
b9549e108e
YARN-6802. Add max AM resource and AM resource usage to leaf queue view in FairScheduler WebUI. (YunFan Zhou via Yufei Gu)
2017-08-04 17:23:44 -07:00
Carlo Curino
7654b52167
YARN-6634. [API] Refactor ResourceManager WebServices to make API explicit (Contributed by Giovanni Matteo Fumarola via curino).
...
(Edited cherry-pick of 547cb53e8a
).
2017-08-04 17:04:20 -07:00
Arun Suresh
4cfd248a72
YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh)
...
(cherry picked from commit 077fcf6a96
)
2017-08-04 16:50:21 -07:00
Arun Suresh
043b7d133e
YARN-6776. Refactor ApplicaitonMasterService to move actual processing logic to a separate class. (asuresh)
...
(cherry picked from commit 5496a34c0c
)
2017-08-04 16:43:35 -07:00
Arun Suresh
2d62af6545
YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh)
...
(cherry picked from commit aaf106fde3
)
2017-08-04 16:19:44 -07:00
Arun Suresh
88d6adf419
YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit 35dc782923
)
2017-08-03 21:18:43 -07:00
Yufei Gu
b6729a7be5
YARN-6895. [FairScheduler] Preemption reservation may cause regular reservation leaks. (Miklos Szegedi via Yufei Gu)
2017-08-03 16:56:15 -07:00
Yufei Gu
ce609d06c0
YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu)
2017-08-03 13:52:58 -07:00
Sunil G
82ec700241
YARN-6678. Handle IllegalStateException in Async Scheduling mode of CapacityScheduler. Contributed by Tao Yang.
...
(cherry picked from commit f64cfeaf61
)
2017-08-03 19:28:16 +05:30
Eric Payne
223229b99c
YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe.
...
(cherry picked from commit 8ce8672b6b
)
2017-08-02 12:58:07 -05:00
Eric Payne
686bdc48b8
YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe.
...
(cherry picked from commit 48899134d2
)
2017-08-02 11:29:56 -05:00
Jian He
6ee0fe70c7
YARN-6872. [Addendum patch] Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G
2017-08-02 00:03:59 -07:00
Jian He
4006cc266b
YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G
2017-08-01 10:00:04 -07:00
Akira Ajisaka
72bde1dad5
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor.
...
(cherry picked from commit b38a1eea8e
)
2017-08-01 15:16:21 +09:00
Jason Lowe
b9640c02bf
YARN-6628. Unexpected jackson-core-2.2.3 dependency introduced. Contributed by Jonathan Eagles
2017-07-31 16:20:54 -05:00
Akira Ajisaka
98c3544e94
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout.
...
(cherry picked from commit f8bed5e9a7
)
2017-07-31 11:10:10 +09:00
Arun Suresh
7e643130f2
YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh)
...
(cherry picked from commit 890e14c02a
)
2017-07-28 16:33:42 -07:00
Yufei Gu
e29d1c75ea
YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu)
...
(cherry picked from commit 9902be72cb
)
2017-07-27 23:21:30 -07:00
Varun Saxena
05bbbcbdbe
Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)
...
(cherry picked from commit 11ece0bda1
)
2017-07-28 00:33:41 +05:30
Varun Saxena
0b342a644a
YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)
...
(cherry picked from commit f66fd11e51
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
2017-07-28 00:33:37 +05:30
Sunil G
08d5f3acf6
Addendum patch for YARN-6102.
2017-07-26 18:40:52 +05:30
Daniel Templeton
00f4a80e8f
YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton)
...
(cherry picked from commit f81a4efb8c
)
2017-07-25 13:02:00 -07:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor.
...
(cherry picked from commit cca51e916b
)
2017-07-25 15:25:36 +09:00
Akira Ajisaka
09241feb13
YARN-6150. TestContainerManagerSecurity tests for Yarn Server are flakey. Contributed by Daniel Sturman and Ray Chiang.
...
(cherry picked from commit 218b1b33ff
)
2017-07-25 15:12:47 +09:00
Daniel Templeton
9b6e787230
YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue.
...
(Contributed by Yufei Gu via Daniel Templeton)
(cherry picked from commit 10583625c1
)
2017-07-24 13:45:09 -07:00
Daniel Templeton
eb1751670a
YARN-6779. DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting
...
(Contributed by Yeliang Cang via Daniel Templeton)
(cherry picked from commit bb30bd3771
)
2017-07-24 12:15:35 -07:00
Daniel Templeton
6b8e353c65
YARN-6844. AMRMClientImpl.checkNodeLabelExpression() has wrong error message
...
(Contributed by Manikandan R via Daniel Templeton)
(cherry picked from commit 4c40cd451c
)
2017-07-24 12:06:50 -07:00
Sunil G
a657472b42
YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S.
2017-07-24 20:57:56 +05:30
Jason Lowe
db7c5636b6
YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles
...
(cherry picked from commit 24853bf32a
)
2017-07-24 10:22:31 -05:00
Jason Lowe
e76e7b9f0b
YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling
...
(cherry picked from commit c8df3668ec
)
2017-07-20 11:06:39 -05:00
Wangda Tan
5070c9ba39
YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda)
...
Change-Id: Iaf9bb7e5ed3aa1300abdccf87ae6fcbddbd25e3e
2017-07-19 15:25:21 -07:00
Rohith Sharma K S
f806e8627b
YARN-6819. Application report fails if app rejected due to nodesize. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 845c4e52bd
)
2017-07-19 11:16:17 +05:30
Yufei Gu
d04bb465d2
YARN-6778. In ResourceWeights, weights and setWeights() should be final. (Daniel Templeton via Yufei Gu)
...
(cherry picked from commit daaf530fce
)
2017-07-18 16:40:13 -07:00
Sunil G
487b0a3d7f
Addendum patch for YARN-5731
...
(cherry picked from commit 0b7afc060c
)
2017-07-18 13:46:43 +05:30
Sunil G
9b1abb448d
YARN-6714. IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by Tao Yang.
2017-07-17 13:22:04 +05:30
Sunil G
d462e48336
YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin.
2017-07-17 12:25:41 +05:30
Arun Suresh
0101973dbf
YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh)
2017-07-14 13:57:22 -07:00
Jason Lowe
9eb93c6f3f
YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt
...
(cherry picked from commit a5ae5ac50e
)
2017-07-14 15:03:48 -05:00
bibinchundatt
af80d10e54
YARN-6759. Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure. Contributed by Naganarasimha G R
...
(cherry picked from commit 75c0220b44
)
2017-07-14 14:05:38 +05:30
Yufei Gu
b2a34fe8d6
YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu)
...
(cherry picked from commit 4a574e9a84
)
2017-07-13 23:11:27 -07:00
Sunil G
fd6935709f
YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. Contributed by Wangda Tan.
2017-07-14 08:45:17 +05:30
Sunil G
f5cdee6bfd
YARN-6792. Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo. Contributed by Giovanni Matteo Fumarola.
...
(cherry picked from commit 228ddaa31d
)
2017-07-14 08:08:32 +05:30
Jason Lowe
a77fb561ef
YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe
...
(cherry picked from commit ebc048cc05
)
2017-07-13 17:49:55 -05:00
Jason Lowe
6ce2b0e4d4
YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles
...
(cherry picked from commit 5f1ee72b0e
)
2017-07-13 17:28:49 -05:00
Sunil G
d83e8712f6
YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne.
...
(cherry picked from commit ca13b224b2
)
2017-07-13 14:50:28 -05:00
Wangda Tan
cbb5f601f1
YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda)
...
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
(cherry picked from commit c583ab02c7
)
2017-07-13 10:44:07 -05:00
Xuan
edc0aa95f1
YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong)
2017-07-12 15:45:51 -07:00
Jason Lowe
401b191a3c
YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe
...
(cherry picked from commit 655110393b
)
2017-07-12 15:54:17 -05:00
Wangda Tan
f2d440b3b3
YARN-5889. Improve and refactor user-limit calculation in Capacity Scheduler. (Sunil G via wangda)
...
(cherry picked from commit 5fb723bb77
)
2017-07-12 10:06:02 -05:00
Eric Payne
e6cdf770ca
Revert "YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G)"
...
This reverts commit eda4ac07c1
.
Commit eda4ac07c1
was a separate patch from trunk rather than a cherry-pick. I will cherryp-ick dependencies and then cherry-pick the trunk commit for YARN-2113.
2017-07-12 08:56:40 -05:00
Akira Ajisaka
3419381e98
YARN-6809. Fix typo in ResourceManagerHA.md. Contributed by Yeliang Cang.
...
(cherry picked from commit b628d0da51
)
2017-07-12 19:39:03 +09:00
Yufei Gu
20a2770d72
YARN-6764. Simplify the logic in FairScheduler#attemptScheduling. Contributed by Yufei Gu.
...
(cherry picked from commit 3de47ab5ea
)
2017-07-09 16:11:26 -07:00
Naganarasimha
5f603306d8
YARN-6770. A small mistake in the example of TimelineClient. Contributed by Jinjiang Ling.
...
(cherry picked from commit ba5b056e8a
)
2017-07-09 21:31:53 +05:30
Naganarasimha
b08ef44d9e
YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 626d730bfc
)
2017-07-09 17:27:50 +05:30
Daniel Templeton
fcef28e53b
YARN-6410. FSContext.scheduler should be final (Contributed by Yeliang Cang via Daniel Templeton)
...
(cherry picked from commit 9d27880442
)
2017-07-09 19:38:37 +09:00
Daniel Templeton
a6d194d720
YARN-6746. SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code
...
(Contributed by Deepti Sawhney via Daniel Templeton)
(cherry picked from commit c5a0c38d8e
)
2017-07-09 19:31:20 +09:00
Daniel Templeton
edcfae8258
YARN-6752. Display reserved resources in web UI per application
...
(Contributed by Abdullah Yousufi via Daniel Templeton)
(cherry picked from commit 0615985886
)
2017-07-09 19:19:07 +09:00
Eric Payne
eda4ac07c1
YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G)
2017-07-06 10:43:00 -05:00
Jason Lowe
53c2441c10
YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt
...
(cherry picked from commit 7576a688ea
)
2017-07-06 09:44:02 -05:00
Naganarasimha
87c81cfad3
YARN-6681. Eliminate double-copy of child queues in canAssignToThisQueue. Contributed by Daryn Sharp.
...
(cherry picked from commit fa1aaee87b
)
2017-07-01 12:32:01 +05:30
Daniel Templeton
93c5421a43
YARN-6751. Display reserved resources in web UI per queue
...
(Contributed by Abdullah Yousufi via Daniel Templeton)
(cherry picked from commit ec97519779
)
2017-06-29 16:57:23 -07:00
Daniel Templeton
15242786b6
YARN-6743. yarn.resourcemanager.zk-max-znode-size.bytes description needs spaces in yarn-default.xml
...
(Contributed by Lori Loberg via Daniel Templeton)
(cherry picked from commit 25d891a784
)
2017-06-28 17:17:56 -07:00
Junping Du
c5f43feb7c
YARN-5311. Document graceful decommission CLI and usage. Contributed by Elek, Marton.
...
(cherry picked from commit 4e3eebc943
)
2017-06-28 15:32:49 -07:00
Naganarasimha
a5ae7c0cae
YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R.
2017-06-29 00:09:30 +05:30
Jason Lowe
44b3c6ffaa
YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper instances. Contributed by Zoltan Haindrich
...
(cherry picked from commit 63ce1593c5
)
2017-06-27 17:13:49 -05:00
Naganarasimha
92e266c2b6
YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt.
2017-06-24 23:14:51 +05:30
Yufei Gu
4c59b446e2
YARN-5876. TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk. (Robert Kanter via Yufei Gu)
...
(cherry picked from commit 0b77262890
)
2017-06-23 13:30:20 -07:00
Arun Suresh
66f780e6ab
YARN-6127. Add support for work preserving NM restart when AMRMProxy is enabled. (Botong Huang via asuresh).
2017-06-22 15:07:38 -07:00
Akira Ajisaka
35f6c7c9c7
YARN-6713. Fix dead link in the Javadoc of FairSchedulerEventLog.java. Contributed by Weiwei Yang.
...
(cherry picked from commit 099dfe92a9
)
2017-06-20 17:23:06 +09:00
Naganarasimha
578318473c
YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp
...
(cherry picked from commit ee89ac84e6
)
2017-06-19 22:45:21 +05:30
Naganarasimha
3106f256e2
YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common(addendum). Contributed by Weiwei Yang.
...
(cherry picked from commit 7582dedad1
)
2017-06-18 19:56:11 +05:30
bibinchundatt
3f9acf65fb
YARN-6703. RM startup failure with old state store due to version mismatch. Contributed by Varun Saxena
...
(cherry picked from commit d64c842743
)
2017-06-12 18:32:23 +05:30
Eric Payne
33605e0896
YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G.
...
(cherry picked from commit 5578af8603
)
2017-06-09 16:18:16 -05:00
Naganarasimha
85d561a5b3
YARN-6681. Improve performance of AssignmentInformation datastructures. Contributed by Daryn sharp.
2017-06-09 11:46:51 +05:30
Daniel Templeton
c5b28c03a1
YARN-6679. Reduce Resource instance overhead via non-PBImpl
...
(Contributed by Daryn Sharp via Daniel Templeton)
2017-06-08 10:41:32 -07:00
Daniel Templeton
0bcbf2a09b
YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application
...
(Contributed by Akira Ajisaka via Daniel Templeton)
(cherry picked from commit 73ecb19312
)
2017-06-02 08:52:27 -07:00
Daniel Templeton
10b7a5fa98
YARN-6246. Identifying starved apps does not need the scheduler writelock
...
(Contributed by Karthik Kambatla via Daniel Templeton)
(cherry picked from commit d5b71e4175
)
2017-05-31 15:50:11 -07:00
Nathan Roberts
c112bf6838
YARN-6649. RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception. Contributed by Jon Eagles.
...
(cherry picked from commit 4369690ce6
)
2017-05-31 12:35:50 -05:00
Jonathan Eagles
f5f12b576e
YARN-6497. Method length of ResourceManager#serviceInit() is too long (Gergely Novák via jeagles)
...
(cherry picked from commit cbfed0e82f
)
2017-05-31 10:19:51 -05:00
Varun Vasudev
1eecde3355
YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf.
...
(cherry picked from commit 547f18cb96
)
2017-05-31 16:16:33 +05:30
Jason Lowe
3c0eb9c415
YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla
...
(cherry picked from commit aea42930bb
)
2017-05-26 09:39:22 -05:00
Yufei Gu
fc6cb4b2dd
YARN-6582. FSAppAttempt demand can be updated atomically in updateDemand(). (Karthik Kambatla via Yufei Gu)
...
(cherry picked from commit 87590090c8
)
2017-05-25 14:25:49 -07:00
Jason Lowe
4c44ff69df
YARN-6643. TestRMFailover fails rarely due to port conflict. Contributed by Robert Kanter
...
(cherry picked from commit 3fd6a2da4e
)
2017-05-25 16:09:08 -05:00
Varun Vasudev
1a6c532301
Addendum patch to fix Docker sanitization.
...
(cherry picked from commit 2ff2a1f50e8c7c0f33676b010b256d6c8daf912d)
2017-05-25 14:54:08 +05:30
Akira Ajisaka
ca1c0cbc62
YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan.
...
(cherry picked from commit bc28da65fb
)
2017-05-25 17:07:50 +09:00
Subru Krishnan
d1724ef919
YARN-6587. Refactor of ResourceManager#startWebApp in a Util class. (Giovanni Matteo Fumarola via Subru).
2017-05-24 13:40:49 -07:00
Jason Lowe
96a97e7eb2
YARN-6615. AmIpFilter drops query parameters on redirect. Contributed by Wilfred Spiegelenburg
...
(cherry picked from commit 8bf1949c0e
)
2017-05-24 11:24:00 -05:00
Sunil G
ec346d6377
YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.
2017-05-23 07:57:36 +05:30
Wangda Tan
9a8fdfa385
YARN-6493. Print requested node partition in assignContainer logs. (Jonathan Hung via wangda)
...
Change-Id: Ib20326daac10acad363befb0ae0998441fa5b32d
(cherry picked from commit 8e0f83e49a
)
2017-05-22 14:43:44 -07:00
Robert Kanter
2719cc0388
YARN-6602. Impersonation does not work if standby RM is contacted first (rkanter)
...
(cherry picked from commit 9855225a79
)
2017-05-19 15:43:43 -07:00
Yufei Gu
50e263a316
YARN-6249. TestFairSchedulerPreemption fails inconsistently. (Tao Jie via Yufei Gu)
...
(cherry picked from commit 66bba8c024
)
2017-05-19 15:22:14 -07:00
Eric Payne
46c54bf200
YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe.
...
(cherry picked from commit 19482e0d09
)
2017-05-19 11:29:02 -05:00
Varun Vasudev
cd1a04e3c6
Sanitize arguments before launching Docker containers.
2017-05-18 11:46:03 +05:30
Varun Vasudev
603a0a316c
Validate docker image name before launching container.
2017-05-18 11:31:12 +05:30
Chris Douglas
3d2afb209c
YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin
...
(cherry picked from commit b23fcc86c6
)
2017-05-17 20:40:20 -07:00
Rohith Sharma K S
379e5bd26f
HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe.
2017-05-17 08:27:50 +05:30
Jian He
82751ff0cb
YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh
2017-05-16 10:49:12 -07:00
Jason Lowe
feb7e92121
YARN-6603. NPE in RMAppsBlock. Contributed by Jason Lowe
...
(cherry picked from commit 489f85933c
)
2017-05-16 09:28:42 -05:00
Jonathan Eagles
d370edc6aa
YARN-6598. History server getApplicationReport NPE when fetching report for pre-2.8 job (Jason Lowe via jeagles)
...
(cherry picked from commit c48f2976a3
)
2017-05-15 10:32:47 -05:00
Haibo Chen
6284cc4663
YARN-6580. Incorrect logger for FairSharePolicy. (Vrushali C via Haibo Chen)
...
(cherry picked from commit 6c35001b9f
)
2017-05-12 13:07:17 -07:00
Daniel Templeton
717f03ecf7
YARN-6380. FSAppAttempt keeps redundant copy of the queue
...
(cherry picked from commit 90cb5b4635
)
2017-05-11 13:48:46 -07:00
Min Shen
9310d24b08
YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen.
...
(cherry picked from commit 2ada100da7
)
2017-05-11 13:29:02 -07:00
Brahma Reddy Battula
3766e54893
HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang
...
(cherry picked from commit 4aae2d40a3
)
2017-05-11 23:38:18 +05:30
Daniel Templeton
df3d7415c0
YARN-6571. Fix JavaDoc issues in SchedulingPolicy
...
(Contributed by Weiwei Yang via Daniel Templeton)
(cherry picked from commit 18d54b5ae2f76513548c453e05cdd6d87777cd06)
2017-05-10 14:37:36 -07:00
Jason Lowe
3230dd1ab6
YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger
...
(cherry picked from commit 6099deebcb
)
2017-05-10 08:48:47 -05:00
Karthik Kambatla
3403ed3148
YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha)
...
(cherry picked from commit 166be0ee95
)
2017-05-09 14:44:39 -07:00
Daniel Templeton
486a160735
YARN-5301. NM mount cpu cgroups failed on some systems
...
(Contributed by Miklos Szegedi via Daniel Templeton)
(cherry picked from commit a2f680493f
)
2017-05-09 12:56:28 -07:00
Daniel Templeton
920a3078f6
YARN-6302. Fail the node if Linux Container Executor is not configured properly
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-05-09 10:08:07 -07:00