1c5c2b5dde
YARN-7339. LocalityMulticastAMRMProxyPolicy should handle cancel request properly. (Botong Huang via curino)
2017-10-24 10:39:04 -07:00
025c656572
YARN-7382. NoSuchElementException in FairScheduler after failover causes RM crash (rkanter)
2017-10-24 10:21:44 -07:00
5da295a34e
YARN-7320. Duplicate LiteralByteStrings in SystemCredentialsForAppsProto.credentialsForApp_ (misha@cloudera.com via rkanter)
2017-10-23 17:56:56 -07:00
9e77dc2bd1
YARN-7357. Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations
...
(Contributed by Sen Zhao via Daniel Templeton)
2017-10-23 13:51:19 -07:00
921338cd86
YARN-4163: Audit getQueueInfo and getApplications calls
2017-10-23 11:43:41 -05:00
480187aebb
YARN-7372. TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic is flaky.
2017-10-20 14:24:17 -07:00
0799fde35e
YARN-7261. Add debug message for better download latency monitoring. (Yufei Gu)
2017-10-20 10:00:13 -07:00
b61144a93d
YARN-7353. Improved volume mount check for directories and unit test compatibility on RHEL7. Contributed by Eric Badger.
2017-10-20 12:02:06 -04:00
1f4cdf1068
YARN-4090. Make Collections.sort() more efficient by caching resource usage. (Contributed by Yufei Gu, Shilong Zhang and Xianyin Xin)
2017-10-20 01:32:20 -07:00
7b4b018780
YARN-7359. TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic. (Contributed by Haibo Chen)
2017-10-19 16:51:47 -07:00
cbd2b73ef8
YARN-7294. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair Scheduler. (Contributed by Miklos Szegedi)
2017-10-19 16:39:25 -07:00
c1b08ba720
YARN-7345. GPU Isolation: Incorrect minor device numbers written to devices.deny file. (Jonathan Hung via wangda)
2017-10-19 14:45:44 -07:00
75323394fb
YARN-7311. Fix TestRMWebServicesReservation parametrization for fair scheduler. (Yufei Gu via Subru).
2017-10-17 12:38:06 -07:00
acabc657ff
YARN-7341. TestRouterWebServiceUtil#testMergeMetrics is flakey. (Robert Kanter via Haibo Chen)
2017-10-17 10:15:53 -07:00
8a61525928
YARN-7308. TestApplicationACLs fails with FairScheduler (rkanter)
2017-10-16 15:34:32 -07:00
4540ffd15f
YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe.
2017-10-16 17:00:38 -05:00
a50be1b8f4
YARN-7275. NM Statestore cleanup for Container updates. (Kartheek Muthyala via asuresh)
2017-10-16 13:12:15 -07:00
87ea1dff9c
Addendum fix for: YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (Wangda Tan)
...
Change-Id: If1fe4a62e07b25e6f1b8ba803495da87e2cb2df6
2017-10-14 10:41:58 -07:00
f4fb6695a3
YARN-7180. Remove class ResourceType. Contributed by Sunil G.
2017-10-13 15:17:05 +05:30
13fcfb3d46
YARN-7317. Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy. (contributed by Botong Huang via curino)
2017-10-12 10:38:58 -07:00
fa5cfc68f3
YARN-6620. Add support in NodeManager to isolate GPU devices by using CGroups. Contributed by Wangda Tan.
2017-10-11 23:44:33 +05:30
ebb34c7053
YARN-7082. TestContainerManagerSecurity failing in trunk. Contributed by Akira Ajisaka
2017-10-11 10:31:02 -05:00
ec8bf9e48a
YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter)
2017-10-10 10:18:42 -07:00
5cfbad68c0
YARN-7237. Cleanup usages of ResourceProfiles. (Wangda Tan)
...
Change-Id: I959c8531bd48231956068cb7d2312f3235549f9c
(cherry picked from commit 8536db5a7549fa44d9f9e444b41bd5978b5c6ae1)
2017-10-09 20:36:57 -07:00
1d36b53ab6
YARN-7252. Removing queue then failing over results in exception
2017-10-09 11:11:23 -07:00
09c5dfe937
YARN-7251. Misc changes to YARN-5734
2017-10-09 11:11:23 -07:00
d6622daaa3
YARN-7046. Add closing logic to configuration store
2017-10-09 11:11:23 -07:00
ff39c0de20
YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda)
...
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
2017-10-09 11:11:23 -07:00
4d8abd84f4
YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung
2017-10-09 11:11:23 -07:00
4c8b208adb
YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung
2017-10-09 11:11:23 -07:00
74ba6ffa0b
YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung
2017-10-09 11:11:23 -07:00
04f9e80bb2
YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao
2017-10-09 11:11:23 -07:00
916bdbd6be
YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong)
2017-10-09 11:11:23 -07:00
e566fd8b58
YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong)
2017-10-09 11:11:23 -07:00
a4e6253046
YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda)
...
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
2017-10-09 11:11:23 -07:00
d8a5136b1f
YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda)
2017-10-09 11:11:23 -07:00
ef686635c1
YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store
2017-10-09 11:11:23 -07:00
e3579a8c3b
YARN-5946: Create YarnConfigurationStore interface and
...
InMemoryConfigurationStore class. Contributed by Jonathan Hung
2017-10-09 11:11:23 -07:00
ef59cbe08a
YARN-5951. Changes to allow CapacityScheduler to use configuration store
2017-10-09 11:11:22 -07:00
6d6ca4c923
YARN-2612 addendum: fixed javadoc error.
2017-10-09 14:12:13 +02:00
8137254a1f
YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne.
2017-10-09 14:39:41 +05:30
6fe6a72352
Revert "YARN-7180. Remove class ResourceType. Contributed by Sunil G."
...
This reverts commit 6bd81ab054
2017-10-09 13:21:13 +05:30
6bd81ab054
YARN-7180. Remove class ResourceType. Contributed by Sunil G.
2017-10-09 10:20:40 +05:30
72d22b753a
YARN-7207. Cache the RM proxy server address. (Yufei Gu)
2017-10-06 09:49:11 -07:00
49ae538164
YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage(addendum). (Yufei Gu)
2017-10-06 00:08:25 -07:00
99292adcef
YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu)
2017-10-05 23:51:58 -07:00
c071aad5da
YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen)
2017-10-05 15:50:13 -07:00
d0046bb5c6
YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe.
2017-10-06 00:07:53 +05:30
b733348dde
YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh).
2017-10-05 09:58:04 -07:00
592bf2d550
YARN-7279. Fix typo in helper message of ContainerLauncher. Contributed by Elek, Marton.
2017-10-05 10:52:00 +05:30
e6e614e380
YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru).
2017-10-04 19:28:27 -07:00
2df1b2ac05
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails.
2017-10-05 06:07:32 +09:00
acf5b880d8
YARN-6975. Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router. Contributed by Yeliang Cang.
2017-10-04 14:53:41 +09:00
c9b525bd94
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.
2017-10-04 07:01:54 +09:00
4a87773718
YARN-6916. Moving logging APIs over to slf4j in hadoop-yarn-server-common. Contributed by Bibin A Chundatt and Akira Ajisaka.
2017-10-04 06:06:39 +09:00
4111e6c781
YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. Contributed by Wangda Tan
2017-10-03 09:54:38 -07:00
7eb8499996
YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe
2017-10-02 19:04:49 -07:00
d4d2fd1acd
YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru).
2017-10-02 18:14:44 -07:00
ceca9694f9
YARN-3661. Basic Federation UI. (Contributed by Inigo Goiri via curino)
2017-10-02 13:03:32 -07:00
febeead5f9
YARN-6550. Capture launch_container.sh logs to a separate log file. (Suma Shivaprasad via wangda)
...
Change-Id: I0ee0b1bb459437432a22cf68861a6354f0decabb
2017-09-29 16:39:46 -07:00
7f6118f918
YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe.
2017-09-29 10:33:57 -05:00
d3b1c63195
YARN-6623. Add support to turn off launching privileged containers in the container-executor. (Varun Vasudev via wangda)
...
Change-Id: I76aec68cbfb42d239279d7245a03290d691e87a4
2017-09-28 16:41:09 -07:00
ca669f9f8b
YARN-6962. Add support for updateContainers when allocating using FederationInterceptor. (Botong Huang via Subru).
2017-09-28 13:04:03 -07:00
85d81ae58e
YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh
2017-09-28 14:10:15 -05:00
8facf1f976
YARN-6871. Add additional deSelects params in RMWebServices#getAppReport. Contributed by Tanuj Nayak.
2017-09-27 14:37:32 +05:30
14fec04ed3
Revert "YARN-6570. No logs were found for running application, running"
...
This reverts commit 7c732924a8
2017-09-26 10:38:03 -05:00
06e5a7b5cf
YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R.
2017-09-26 14:34:09 +05:30
df800f6cf3
YARN-7240. Add more states and transitions to stabilize the NM Container state machine. (Kartheek Muthyala via asuresh)
2017-09-25 16:02:55 -07:00
164a063e74
YARN-6878. TestCapacityScheduler.testDefaultNodeLabelExpressionQueueConfig() has the args to assertEqual() in the wrong order
...
(Contributed by Sen Zhao via Daniel Templeton)
2017-09-22 23:53:32 +02:00
4002bf0a9e
YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi.
2017-09-22 13:58:09 -07:00
bfd1a72ba8
YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger.
2017-09-21 17:41:34 -05:00
e5e1851d80
YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi.
2017-09-21 14:01:16 -07:00
a92ef030a2
YARN-7045. Remove FSLeafQueue#addAppSchedulable. (Contributed by Sen Zhao via Yufei Gu)
2017-09-21 12:31:07 -07:00
263e2c692a
YARN-6991. "Kill application" button does not show error if other user tries to kill the application for secure cluster. (Suma Shivaprasad via wangda)
...
Change-Id: I7e7894b24609709f89064ee5882f055dbb09080b
2017-09-21 12:00:53 -07:00
10d7493587
YARN-6968. Hardcoded absolute pathname in DockerLinuxContainerRuntime. Contributed by Eric Badger
2017-09-21 08:38:08 -05:00
a9019e1fb7
YARN-6499. Remove the doc about Schedulable#redistributeShare(). (Contributed by Chetna Chaudhari via Yufei Gu)
2017-09-19 18:28:31 -07:00
647b7527a9
YARN-7186. Fix finicky TestContainerManager tests. Contributed by Arun Suresh.
2017-09-19 18:31:15 -07:00
56ef5279c1
YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh)
2017-09-18 15:49:31 -07:00
7c732924a8
YARN-6570. No logs were found for running application, running
...
container. Contributed by Junping Du
2017-09-18 14:04:05 -07:00
a4f9c7c924
YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh
2017-09-18 10:16:09 -05:00
e81596d06d
YARN-7172. ResourceCalculator.fitsIn() should not take a cluster resource parameter. (Sen Zhao via wangda)
...
Change-Id: Icc3670c9381ce7591ca69ec12da5aa52d3612d34
2017-09-17 21:20:43 -07:00
38c14ef8d8
YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda)
...
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
2017-09-15 21:29:39 -07:00
958e8c0e25
YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda)
...
Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e
(cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)
2017-09-15 21:29:31 -07:00
09b476e6da
YARN-6612. Update fair scheduler policies to be aware of resource types. (Contributed by Daniel Templeton via Yufei Gu)
2017-09-14 11:23:37 -07:00
66ca0a6540
YARN-6059. Update paused container state in the NM state store. (Hitesh Sharma via asuresh)
2017-09-14 08:51:26 -07:00
4f8194430f
YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh)
2017-09-14 08:51:26 -07:00
864fbacd45
YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh)
2017-09-14 08:48:39 -07:00
bb34ae9554
YARN-7146. Many RM unit tests failing with FairScheduler (rkanter)
2017-09-13 15:44:54 -07:00
3860be7961
YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe.
2017-09-13 14:41:55 -05:00
a3c44195be
Revert 'YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe.'
...
This reverts commit 729d05f529
2017-09-13 14:38:58 -05:00
729d05f529
YARN-4727. Unable to override the /home/ericp/run/conf/ env variable for container. Contributed by Jason Lowe.
2017-09-13 14:29:08 -05:00
5324388cf2
YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G.
2017-09-13 23:36:47 +05:30
a1c398e0dc
YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix.
2017-09-12 17:03:32 -07:00
82c5dd1d50
YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu)
2017-09-12 16:13:39 -07:00
2ae72692fc
YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh)
2017-09-12 16:10:08 -07:00
da0b6a354b
YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan.
2017-09-12 09:19:12 -07:00
bf2b687412
YARN-7136. Additional Performance Improvement for Resource Profile Feature
...
(Contributed by Wangda Tan via Daniel Templeton)
2017-09-12 09:19:12 -07:00
e490602e9b
YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda)
...
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
2017-09-12 09:19:12 -07:00
a333ba54e3
YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan.
2017-09-12 09:19:12 -07:00
39240b61a1
YARN-7043. Cleanup ResourceProfileManager. (wangda)
...
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
2017-09-12 09:19:12 -07:00
df3855541a
YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda)
...
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
2017-09-12 09:19:12 -07:00
6746f8cadb
YARN-6908. ResourceProfilesManagerImpl is missing @Overrides on methods
...
(Contributed by Sunil G. via Daniel Templeton)
2017-09-12 09:19:12 -07:00
5c4ab4c291
YARN-6935. [YARN-3926] ResourceProfilesManagerImpl.parseResource() has no need of the key parameter
...
(Contributed by Manikandan R via Daniel Templeton)
2017-09-12 09:19:11 -07:00
3aeaafecb8
YARN-6788. [YARN-3926] Improve performance of resource profile branch
...
(Contributed by Sunil Govindan via Daniel Templeton)
2017-09-12 09:19:11 -07:00
58da54640e
YARN-6445. [YARN-3926] Performance improvements in resource profile branch with respect to SLS. Contributed by Varun Vasudev.
2017-09-12 09:19:11 -07:00
dae65f3bef
YARN-6232. Update resource usage and preempted resource calculations to take into account all resource types. Contributed by Varun Vasudev.
2017-09-12 09:19:11 -07:00
7805deed48
YARN-5588. [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev.
2017-09-12 09:19:11 -07:00
6708ac3301
YARN-5587. Add support for resource profiles. (vvasudev via asuresh)
2017-09-12 09:19:10 -07:00
c2032e251e
YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
a9b4426302
YARN-5707. Add manager class for resource profiles. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
239c1824a0
YARN-5586. Update the Resources class to consider all resource types. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
759114b006
YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
e74d1be04b
YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda)
...
Change-Id: I1c39ed5c59dee739ba5044b61b3ef5ed203b79c1
2017-09-11 20:52:08 -07:00
fa531788fd
YARN-7132. FairScheduler.initScheduler() contains a surprising unary plus. (Yeliang Cang via Haibo Chen)
2017-09-11 20:42:49 -07:00
661f5eb0c6
YARN-7128. The error message in TimelineSchemaCreator is not enough to find out the error. (Jinjiang Ling via Haibo Chen)
2017-09-11 16:20:20 -07:00
5ab21dfe9c
YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu)
2017-09-10 23:17:59 -07:00
722ee84194
YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S.
2017-09-11 10:28:10 +05:30
3fddabc2fe
YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong.
2017-09-08 15:16:19 -07:00
1f53ae7972
YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh)
2017-09-08 09:24:05 -07:00
fa6137501c
YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru).
2017-09-07 19:07:17 -07:00
56d93d2e39
YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S.
2017-09-08 07:15:17 +05:30
b0b535d9d5
YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf
2017-09-07 16:17:03 -05:00
f155ab7cfa
YARN-7033. Add support for NM Recovery of assigned resources (e.g. GPU's, NUMA, FPGA's) to container. (Devaraj K and Wangda Tan)
...
Change-Id: Iffd18bb95debe1c8cc55e30abc1d8f663e9d0e30
2017-09-07 14:13:37 -07:00
6f101e7df1
YARN-6992. Kill application button is visible even if the application is FINISHED in RM UI. Contributed by Suma Shivaprasad.
2017-09-07 13:16:20 +05:30
b6e7d13696
YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem.
2017-09-06 16:46:01 -07:00
e3345e985b
Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem."
...
This reverts commit 7996eca7dc
2017-09-06 16:39:23 -07:00
7996eca7dc
Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem.
2017-09-01 15:16:40 -07:00
0d419c984f
Preparing for 3.1.0 development
2017-09-01 11:53:48 -07:00
99a7f5d451
YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne
2017-09-01 11:09:14 -05:00
dcd0bedcc8
YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena
2017-09-01 11:29:16 +05:30
0adc3a0533
YARN-6721. container-executor should have stack checking
...
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-31 19:39:31 -07:00
91cc070d67
YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong.
2017-08-31 16:41:43 -07:00
bac4e8cca8
YARN-7095. Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru).
2017-08-31 15:05:41 -07:00
d4417dae4f
YARN-6780. ResourceWeights.toString() cleanup (Contributed by weiyuan via Daniel Templeton)
2017-08-31 09:35:01 -07:00
f9e0cc8cdc
YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan.
2017-08-31 18:42:01 +05:30
71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms
...
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-30 21:47:24 -07:00
cc23514aba
YARN-7115. Move BoundedAppender to org.hadoop.yarn.util pacakge
...
(Contributed by Jian He via Daniel Templeton)
2017-08-30 17:26:13 -07:00
a20e7105ea
YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen)
2017-08-30 09:14:59 -07:00
3d00c8f394
Made fixes for whitespace errors and checstyle warnings before merge.
2017-08-30 11:29:54 +05:30
512068a01f
YARN-6905. Addendum to fix TestTimelineReaderWebServicesHBaseStorage due to missing FastNumberFormat
2017-08-30 11:29:54 +05:30
08f40bcc7f
MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena
2017-08-30 11:29:54 +05:30
9b08f365d7
YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen)
2017-08-30 11:29:54 +05:30
b2efebdd07
YARN-6861. Reader API for sub application entities (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:54 +05:30
7fd6ae2479
YARN-6134. [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active. Contributed by Varun Saxena
2017-08-30 11:29:54 +05:30
e276c75ec1
YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena.
2017-08-30 11:29:54 +05:30
32188d3295
YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena.
2017-08-30 11:29:54 +05:30
b664569586
YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena
2017-08-30 11:29:53 +05:30
d5ff965fee
YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C
2017-08-30 11:29:53 +05:30
60765aff9b
YARN-6874. Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite (Vrushali C via Varun Saxena)
2017-08-30 11:29:53 +05:30
354be99dbf
YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena.
2017-08-30 11:29:53 +05:30
7594d1de7b
YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena.
2017-08-30 11:29:53 +05:30
9f6540535d
YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:53 +05:30
3fb71b1393
YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C.
2017-08-30 11:29:53 +05:30
a990ff70c2
YARN-6733. Add table for storing sub-application entities. Contributed by Vrushali C.
2017-08-30 11:29:53 +05:30
61136d03f2
YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics (Contributed by Varun Saxena via Vrushali C)
2017-08-30 11:29:53 +05:30
70078e91e3
YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena.
2017-08-30 11:29:53 +05:30
660413165a
YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C.
2017-08-30 11:29:53 +05:30
ac7f52df83
YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena
2017-08-30 11:29:53 +05:30
d3f11e3f13
YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena
2017-08-30 11:29:53 +05:30
879de51206
YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena
2017-08-30 11:29:53 +05:30
f6a51dce18
YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena)
2017-08-30 11:29:52 +05:30
a8f082a180
YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen)
...
(cherry picked from commit 47474fffac
2017-08-30 11:29:52 +05:30
b87b72b40a
YARN-6146. Add Builder methods for TimelineEntityFilters (Haibo Chen via Varun Saxena)
2017-08-30 11:29:52 +05:30
44999aabc2
YARN-6318. timeline service schema creator fails if executed from a remote machine (Sangjin Lee via Varun Saxena)
2017-08-30 11:29:52 +05:30
18b3a80df7
YARN-6237. Move UID constant to TimelineReaderUtils (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:52 +05:30
c3bd8d6ad3
YARN-6256. Add FROM_ID info key for timeline entities in reader response (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:52 +05:30
8bb2646595
YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:52 +05:30
cf30b3b914
YARN-6253. FlowAcitivityColumnPrefix.store(byte[] rowKey, ...) drops timestamp. Contributed by Haibo Chen.
2017-08-30 11:29:52 +05:30
ccb38c1991
YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen.
2017-08-30 11:29:52 +05:30
580d884913
YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C.
2017-08-30 11:29:52 +05:30
5e0acee75e
Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's
2017-08-30 11:29:52 +05:30
6f65cf27bb
YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:51 +05:30
71847ed44d
YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena)
2017-08-30 11:29:51 +05:30
02a9710a09
YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:51 +05:30
2556c012e3
YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.
2017-08-30 11:29:51 +05:30
4481561e4a
YARN-5739. Provide timeline reader API to list available timeline entity types for one application. Contributed by Li Lu.
2017-08-30 11:29:51 +05:30
092fead5d9
YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena.
2017-08-30 11:29:51 +05:30
c92a7ab31c
YARN-3359. Addendum for Recover collector list when RM fails over (Li Lu via Varun Saxena)
2017-08-30 11:29:51 +05:30
91d3443806
YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena)
2017-08-30 11:29:51 +05:30
643a20a358
YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis.
2017-08-30 11:29:51 +05:30
05ff04439e
YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S.
2017-08-30 11:29:51 +05:30
9bb5cc3d49
YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena)
2017-08-30 11:29:50 +05:30
78b7e070d8
YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu.
2017-08-30 11:29:50 +05:30
e72e6260da
YARN-5229. Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl. (Vrushali C via gtcarrera9)
2017-08-30 11:29:50 +05:30
278bd6fcff
YARN-5156. YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state (Vrushali C via sjlee)
2017-08-30 11:29:50 +05:30
4cae120c61
YARN-6982. Potential issue on setting AMContainerSpec#tokenConf to null before app is completed. Contributed by Manikandan R.
2017-08-30 10:45:11 +05:30
f59332b97b
YARN-5219. When an export var command fails in launch_container.sh, the full container launch should fail. (Sunil G via wangda)
...
Change-Id: Iaa6b978bb89482e9d1d77ba57f4adfdc48e39a3c
2017-08-29 15:29:22 -07:00
ad45d19998
YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang.
2017-08-29 15:16:23 -07:00
cc8893edc0
YARN-7010. Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps). (Contributed by Giovanni Matteo Fumarola via curino)
2017-08-29 14:53:09 -07:00
1f6dc4ee9a
YARN-5816. TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey. Contributed by Robert Kanter
2017-08-29 09:52:24 -05:00
312b1fd9da
YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He.
2017-08-28 18:26:51 -07:00
d5bc958279
YARN-7099. ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. (Contributed by Miklos Szegedi via Yufei Gu)
2017-08-28 10:09:46 -07:00
02599bda04
YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne.
2017-08-28 12:22:56 +05:30
3bee7046c5
YARN-6999. Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. (Linlin Zhou via gtcarrera9)
2017-08-25 22:43:25 -07:00
e864f81471
YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe.
2017-08-25 15:37:54 -05:00
39a9dc8e4a
YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne
2017-08-25 15:31:29 -05:00
8b7cbe3840
HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar.
2017-08-25 09:55:46 -07:00
3a4e861169
YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang
2017-08-25 09:16:17 -05:00
9e2699ac2c
YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu)
2017-08-24 17:52:41 -07:00
c2cb7ea1ef
YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong.
2017-08-24 13:36:49 -07:00
de0cba700b
YARN-7074. Fix NM state store update comment. Contributed by Botong Huang
2017-08-24 16:34:38 +05:30
652dd434d9
YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan.
2017-08-23 18:06:29 -07:00
f49843a988
YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda)
...
Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb
2017-08-23 09:56:20 -07:00
4249172e14
YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru).
2017-08-22 19:20:57 -07:00
657dd59cc8
YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp
2017-08-22 13:16:24 -05:00
3efcd51c3b
YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles
2017-08-22 12:56:09 -05:00
d5ff57a08f
YARN-7047. Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager. Contributed by Yeliang Cang.
2017-08-22 17:14:12 +09:00
ae8fb13b31
YARN-6923. Metrics for Federation Router. (Giovanni Matteo Fumarola via asuresh)
2017-08-21 22:50:24 -07:00
75abc9a8e2
YARN-5603. Metrics for Federation StateStore. (Ellen Hui via asuresh)
2017-08-21 22:43:08 -07:00
7a82d7bcea
YARN-6979. [Addendum patch] Fixed classname and added javadocs. (Kartheek Muthyala via asuresh)
2017-08-20 10:24:05 -07:00
8410d862d3
YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)
2017-08-20 07:54:09 -07:00
436c2638f9
YARN-6852. Native code changes to support isolate GPU devices by using CGroups. (wangda)
...
Change-Id: I4869cc4d8ad539539ccba4bea5a178cacdb741ab
2017-08-18 18:26:36 -07:00
8991f0baec
YARN-6969. Clean up unused code in class FairSchedulerQueueInfo. (Larry Lo via Yufei Gu)
2017-08-18 14:38:44 -07:00
e05fa3451d
YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su.
2017-08-18 20:28:50 +05:30
ab1a8ae85f
YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger
2017-08-17 15:50:14 -05:00
f9a0e23381
YARN-3254. HealthReport should include disk full information. Contributed by Suma Shivaprasad.
2017-08-17 15:07:15 +05:30
de462da04e
YARN-6900. ZooKeeper based implementation of the FederationStateStore. (Íñigo Goiri via Subru).
2017-08-16 11:43:24 -07:00
588c190afd
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka.
2017-08-16 14:06:22 +09:00
75dd866bfb
HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru).
2017-08-15 16:53:59 -07:00
d265459024
YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts)
2017-08-15 15:52:48 -05:00
4d7be1d857
YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh)
2017-08-14 19:46:17 -07:00
5558792894
YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne
2017-08-14 15:31:34 -05:00
3325ef653d
YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles
2017-08-14 14:40:08 -05:00
6b09c32705
YARN-6881. LOG is unused in AllocationConfiguration (Contributed by weiyuan via Daniel Templeton)
2017-08-14 11:55:33 -07:00
608a06cca5
YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen)
2017-08-14 11:41:11 -07:00
18f3603bce
YARN-6996. Change javax.cache library implementation from JSR107 to Apache Geronimo. (Ray Chiang via Subru).
2017-08-14 11:10:00 -07:00
e2f6299f6f
YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang
2017-08-14 10:51:30 -07:00
d8f74c3964
YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R.
2017-08-14 09:39:00 +05:30
28d97b79b6
YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino)
2017-08-11 16:58:04 -07:00
cc59b5fb26
YARN-6896. Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution). (Contributed by Giovanni Matteo Fumarola via curino)
2017-08-11 15:58:01 -07:00
0996acde6c
YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator
...
(Contributed by Larry Lo via Daniel Templeton)
2017-08-11 14:50:46 -07:00
65364defb4
YARN-6967. Limit application attempt's diagnostic message size thoroughly
...
(Contributed by Chengbing Liu via Daniel Templeton)
2017-08-11 14:28:55 -07:00
c7680d4cc4
YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces
...
(Contributed by weiyuan via Daniel Templeton)
2017-08-11 14:22:02 -07:00
218588be77
YARN-6952. Enable scheduling monitor in FS (Contributed by Yufei Gu via Daniel Templeton)
2017-08-11 14:04:19 -07:00
bbbf0e2a41
HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru).
2017-08-11 13:58:45 -07:00
8c4b6d16a5
YARN-5927. BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate. (Kai Sasaki via kasha)
2017-08-11 12:15:43 -07:00
ec694145cf
YARN-6033. Add support for sections in container-executor configuration file. (Varun Vasudev via wandga)
...
Change-Id: Ibc6d2a959debe5d8ff2b51504149742449d1f1da
2017-08-09 10:51:29 -07:00
63cfcb90ac
YARN-6958. Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice. Contributed by Yeliang Cang.
2017-08-09 23:58:22 +09:00
1a18d5e514
YARN-6515. Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager. Contributed by Naganarasimha G R.
2017-08-09 21:56:43 +09:00
ad2a350662
YARN-6970. Add PoolInitializationException as retriable exception in FederationFacade. (Giovanni Matteo Fumarola via Subru).
2017-08-08 16:48:29 -07:00
e0c24145d2
YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code
...
(Contributed by Angela Wang via Daniel Templeton)
2017-08-08 13:35:22 -07:00
1794de3ea4
YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda)
...
Change-Id: If1b1827345f98f0a49cc7e39d1ba41fbeed5e911
2017-08-08 12:56:29 -07:00
acf9bd8b1d
YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du
2017-08-08 11:09:38 -07:00
47b145b9b4
YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-08-08 10:33:26 -07:00
98912950b6
YARN-6961. Remove commons-logging dependency from hadoop-yarn-server-applicationhistoryservice module. Contributed by Yeliang Cang.
2017-08-08 19:38:58 +09:00
8d3fd81980
YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh)
2017-08-07 18:59:25 -07:00
c61f2c4198
YARN-6955. Handle concurrent register AM requests in FederationInterceptor. (Botong Huang via Subru).
2017-08-07 16:58:29 -07:00
adb84f34db
YARN-4161. Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration. (Wei Yan via wangda)
...
Change-Id: Ic441ae4e0bf72e7232411eb54243ec143d5fd0d3
2017-08-07 11:32:21 -07:00
b0fbf17965
YARN-6957. Moving logging APIs over to slf4j in hadoop-yarn-server-sharedcachemanager. Contributed by Yeliang Cang.
2017-08-07 19:25:40 +09:00
839e077faf
YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang.
2017-08-07 18:56:00 +09:00
46b7054fa7
YARN-6951. Fix debug log when Resource Handler chain is enabled. Contributed by Yang Wang.
2017-08-07 13:15:46 +05:30
f44b349b81
YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S.
2017-08-04 16:03:56 -07:00
35dc782923
YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh)
2017-08-03 21:15:40 -07:00
0542e6f86e
YARN-6932. Fix TestFederationRMFailoverProxyProvider test case failure.
2017-08-03 14:18:03 -07:00
c617fe02b3
YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu)
2017-08-03 11:33:37 -07:00
293c74a81b
YARN-6673 Add cpu cgroup configurations for opportunistic containers. (Miklos Szegedi via Haibo Chen)
2017-08-03 09:56:51 -07:00
c5d256c760
YARN-6674 Add memory cgroup settings for opportunistic containers. (Miklos Szegedi via Haibo Chen)
2017-08-03 09:52:35 -07:00
f64cfeaf61
YARN-6678. Handle IllegalStateException in Async Scheduling mode of CapacityScheduler. Contributed by Tao Yang.
2017-08-03 19:27:10 +05:30
8ce8672b6b
YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe.
2017-08-02 12:53:22 -05:00
45535f8afa
YARN-6895. [FairScheduler] Preemption reservation may cause regular reservation leaks. (Miklos Szegedi via Yufei Gu)
2017-08-02 09:25:19 -07:00
48899134d2
YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe.
2017-08-02 10:59:33 -05:00
f9139ac8f6
YARN-6872. [Addendum patch] Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G
2017-08-02 00:03:25 -07:00
acda6b96a4
YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino)
...
(cherry picked from commit b6240b92ab
2017-08-01 17:28:28 -07:00
86f05c3312
YARN-6866. Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk. (Botong Huang via Subru).
...
(cherry picked from commit 40453879ec
2017-08-01 17:28:28 -07:00
8737c2a8e9
Bumping up hadoop-yarn-server-router module to 3.0.0-beta1 post rebase.
...
(cherry picked from commit 69e3ed2680
2017-08-01 17:28:27 -07:00
712e97d4cf
Bumping up yarn-server-router (new) module pom to beta1 and fixing imports post rebase.
...
(cherry picked from commit f1eff212c5
2017-08-01 17:28:27 -07:00
3e1dc7ece0
YARN-6821. Move FederationStateStore SQL DDL files from test resource to sbin.
...
(cherry picked from commit cd9db822f0
2017-08-01 17:28:27 -07:00
825108d08a
YARN-6815. FederationStateStoreFacade return behavior should be consistent irrespective of whether caching is enabled or not.
...
(cherry picked from commit 8820693cd7
2017-08-01 17:28:27 -07:00
43a97174fe
YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 52daa6d971
2017-08-01 17:28:26 -07:00
4cfec943b1
YARN-6724. Add ability to blacklist sub-clusters when invoking Routing policies. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit f8e5de5969
2017-08-01 17:28:26 -07:00
70b1a757f1
YARN-6511. Federation: transparently spanning application across multiple sub-clusters. (Botong Huang via Subru).
...
(cherry picked from commit 8c988d235e
2017-08-01 17:28:26 -07:00
bed1832c93
YARN-3666. Federation Intercepting and propagating AM- home RM communications. (Botong Huang via Subru).
...
(cherry picked from commit 2399eb8200
2017-08-01 17:28:25 -07:00
42949a6e01
YARN-6666. Fix unit test failure in TestRouterClientRMService. (Botong Huang via Subru).
...
(cherry picked from commit e750907d0a
2017-08-01 17:28:25 -07:00
f317e0cb48
Bumping up hadoop-yarn-server-router module to 3.0.0-alpha4 post rebase.
...
(cherry picked from commit bd9c7b1bec
2017-08-01 17:28:25 -07:00
859aa1f9d6
YARN-5531. UnmanagedAM pool manager for federating application across clusters. (Botong Huang via Subru).
...
(cherry picked from commit 73bb2102ce
2017-08-01 17:28:24 -07:00
7444406d6d
YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 67846a5519
2017-08-01 17:28:24 -07:00
80e1904000
YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 4846069061
2017-08-01 17:28:24 -07:00
5272af8c7e
YARN-3663. Federation State and Policy Store (DBMS implementation). (Giovanni Matteo Fumarola via curino).
...
(cherry picked from commit be99c1fe2e
2017-08-01 17:28:24 -07:00
c4bbdfeb66
YARN-6203: Occasional test failure in TestWeightedRandomRouterPolicy (curino)
...
(cherry picked from commit 121d55bd29
2017-08-01 17:28:23 -07:00
e1da8f0667
YARN-5602. Utils for Federation State and Policy Store. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 326a2e6bde
2017-08-01 17:28:23 -07:00
1c64e1709b
YARN-6370. Properly handle rack requests for non-active subclusters in LocalityMulticastAMRMProxyPolicy. (Contributed by Botong Huang via curino).
...
(cherry picked from commit ce419881c3
2017-08-01 17:28:23 -07:00
86b2bec56e
YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru)
...
(cherry picked from commit 51aeb2ce0c
2017-08-01 17:28:23 -07:00
8623644f45
YARN-6190. Validation and synchronization fixes in LocalityMulticastAMRMProxyPolicy. (Botong Huang via curino)
...
(cherry picked from commit 5c486961cd
2017-08-01 17:28:22 -07:00
193ec456d6
YARN-5410. Bootstrap Router server module. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 04f111394b
2017-08-01 17:28:22 -07:00
98b45b0ed3
YARN-6093. Minor bugs with AMRMtoken renewal and state store availability when using FederationRMFailoverProxyProvider during RM failover. (Botong Huang via Subru).
...
(cherry picked from commit 66500f4fa6
2017-08-01 17:28:22 -07:00
91803305e5
YARN-5872. Add AlwayReject policies for router and amrmproxy. (Carlo Curino via Subru).
...
(cherry picked from commit db26bade05
2017-08-01 17:28:22 -07:00
78032e51d9
YARN-5905. Update the RM webapp host that is reported as part of Federation membership to current primary RM's IP.
...
(cherry picked from commit e0f0e7afed
2017-08-01 17:28:22 -07:00
4128c9522d
YARN-5676. Add a HashBasedRouterPolicy, and small policies and test refactoring. (Carlo Curino via Subru).
...
(cherry picked from commit 575137f41c
2017-08-01 17:28:21 -07:00
083a05bc14
YARN-5634. Simplify initialization/use of RouterPolicy via a RouterPolicyFacade. (Carlo Curino via Subru).
...
(cherry picked from commit d7672ce2bd
2017-08-01 17:28:21 -07:00
93d9fdeca6
YARN-5391. PolicyManager to tie together Router/AMRM Federation policies. (Carlo Curino via Subru).
...
(cherry picked from commit 20893682ec
2017-08-01 17:28:21 -07:00
1dadd0b45a
YARN-5325. Stateless ARMRMProxy policies implementation. (Carlo Curino via Subru).
...
(cherry picked from commit 11c5336522
2017-08-01 17:28:21 -07:00
0662996b6a
YARN-5324. Stateless Federation router policies implementation. (Carlo Curino via Subru).
...
(cherry picked from commit 1298127bda
2017-08-01 17:28:21 -07:00
81472778d7
YARN-5323. Policies APIs for Federation Router and AMRMProxy policies. (Carlo Curino via Subru).
...
(cherry picked from commit f8208fe0b5
2017-08-01 17:28:20 -07:00
2797507d51
YARN-5601. Make the RM epoch base value configurable. Contributed by Subru Krishnan
...
(cherry picked from commit 9ca2aba9cc
2017-08-01 17:28:20 -07:00
1882bc10fb
YARN-5612. Return SubClusterId in FederationStateStoreFacade#addApplicationHomeSubCluster for Router Failover. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit ac1ba2a304
2017-08-01 17:28:20 -07:00
94a24567d6
YARN-3671. Integrate Federation services with ResourceManager. Contributed by Subru Krishnan
...
(cherry picked from commit 8573c286e2
2017-08-01 17:28:20 -07:00
bdfad4523f
YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan
...
(cherry picked from commit 3307564a5f
2017-08-01 17:28:20 -07:00
cfafd173bd
YARN-5467. InputValidator for the FederationStateStore internal APIs. (Giovanni Matteo Fumarola via Subru)
...
(cherry picked from commit bd44182e70
2017-08-01 17:28:19 -07:00
5c84382397
YARN-3672. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan
...
(cherry picked from commit 5e976cd2b9
2017-08-01 17:28:19 -07:00
52558df620
YARN-5519. Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover. (Ellen Hui via Subru)
...
(cherry picked from commit e4f928cf9c
2017-08-01 17:28:19 -07:00
0733088c29
YARN-5407. In-memory based implementation of the FederationApplicationStateStore/FederationPolicyStateStore. (Ellen Hui via Subru)
...
(cherry picked from commit b747d59f41
2017-08-01 17:28:19 -07:00
e0c3a44396
YARN-5408. Compose Federation membership/application/policy APIs into an uber FederationStateStore API. (Ellen Hui via Subru).
...
(cherry picked from commit 268389d021
2017-08-01 17:28:19 -07:00
f1a508bdef
YARN-3664. Federation PolicyStore internal APIs
...
(cherry picked from commit 2f377263a1
2017-08-01 17:28:18 -07:00
047a16bf15
YARN-5307. Federation Application State Store internal APIs
...
(cherry picked from commit 1af5292042
2017-08-01 17:28:18 -07:00
d19b677301
YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui.
...
(cherry picked from commit d3dc461a93
2017-08-01 17:28:18 -07:00
dcf8c52d80
YARN-5406. In-memory based implementation of the FederationMembershipStateStore. Contributed by Ellen Hui.
...
(cherry picked from commit 20d1d2be91
2017-08-01 17:28:18 -07:00
3e22896f86
YARN-3662. Federation Membership State Store internal APIs.
...
(cherry picked from commit 5424d0899c
2017-08-01 17:28:17 -07:00
91f120f743
YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G
2017-08-01 09:56:33 -07:00
b38a1eea8e
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor.
2017-08-01 15:15:43 +09:00
a4aa1cb405
Revert "YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang."
...
This reverts commit 1a78c0ff01
2017-08-01 12:15:03 +09:00
1a78c0ff01
YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang.
2017-08-01 10:53:32 +09:00
f8bed5e9a7
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout.
2017-07-31 11:09:13 +09:00
890e14c02a
YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh)
2017-07-28 16:32:43 -07:00
bcde66bed1
YARN-6897. Refactoring RMWebServices by moving some util methods to RMWebAppUtil. (Giovanni Matteo Fumarola via Subru).
2017-07-28 15:44:36 -07:00
9ea01fd956
YARN-6802. Add max AM resource and AM resource usage to leaf queue view in FairScheduler WebUI. (YunFan Zhou via Yufei Gu)
2017-07-28 09:52:53 -07:00
9902be72cb
YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu)
2017-07-27 23:19:39 -07:00
38c6fa5c7a
HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier.
2017-07-28 09:10:52 +09:00
11ece0bda1
Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)
2017-07-28 00:10:45 +05:30
f66fd11e51
YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)
2017-07-27 23:14:50 +05:30
f81a4efb8c
YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton)
2017-07-25 13:00:31 -07:00
ac9489f7fc
YARN-6804. Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi
2017-07-25 09:57:14 -07:00
cca51e916b
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor.
2017-07-25 15:24:56 +09:00
218b1b33ff
YARN-6150. TestContainerManagerSecurity tests for Yarn Server are flakey. Contributed by Daniel Sturman and Ray Chiang.
2017-07-25 15:12:08 +09:00
10583625c1
YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue.
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-07-24 13:44:00 -07:00
bb30bd3771
YARN-6779. DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting
...
(Contributed by Yeliang Cang via Daniel Templeton)
2017-07-24 12:13:50 -07:00
e315328428
YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S.
2017-07-24 11:39:03 +05:30
2054324d47
Revert "YARN-6804. [YARN core changes] Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi"
...
This reverts commit 4a771d9010
2017-07-23 12:56:18 +08:00
4a771d9010
YARN-6804. [YARN core changes] Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi
2017-07-21 11:31:33 -07:00
c8df3668ec
YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling
2017-07-20 11:03:04 -05:00
077fcf6a96
YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh)
2017-07-19 12:26:40 -07:00
845c4e52bd
YARN-6819. Application report fails if app rejected due to nodesize. Contributed by Bibin A Chundatt.
2017-07-19 11:10:52 +05:30
daaf530fce
YARN-6778. In ResourceWeights, weights and setWeights() should be final. (Daniel Templeton via Yufei Gu)
2017-07-18 16:38:07 -07:00
f5f14a2ad6
YARN-6798. Fix NM startup failure with old state store due to version mismatch. (Botong Huang via rchiang)
2017-07-18 12:35:08 -07:00
0b7afc060c
Addendum patch for YARN-5731
2017-07-18 11:49:09 +05:30
5b007921cd
YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh)
2017-07-17 14:11:14 -07:00
9e0cde1469
YARN-6625. yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment. (Yufei Gu)
2017-07-14 14:10:45 -07:00
a5ae5ac50e
YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt
2017-07-14 14:56:00 -05:00
75c0220b44
YARN-6759. Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure. Contributed by Naganarasimha G R
2017-07-14 13:53:39 +05:30
4a574e9a84
YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu)
2017-07-13 23:10:10 -07:00
228ddaa31d
YARN-6792. Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo. Contributed by Giovanni Matteo Fumarola.
2017-07-14 08:07:05 +05:30
ebc048cc05
YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe
2017-07-13 17:44:47 -05:00
0ffca5d347
Revert "YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe"
...
This reverts commit f76f5c0919
2017-07-13 17:42:38 -05:00
f76f5c0919
YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe
2017-07-13 17:38:17 -05:00
5f1ee72b0e
YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles
2017-07-13 17:27:40 -05:00
945c0958bb
YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda)
...
Change-Id: I84ccd54200ccbaae23018ef320028e42b4c3509a
2017-07-13 10:30:15 -07:00
cf0d0844d6
YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. Contributed by Wangda Tan.
2017-07-13 16:48:29 +05:30
9845bea59c
YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong)
2017-07-12 15:40:45 -07:00
34f113df5c
YARN-6714. IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by Tao Yang.
2017-07-11 14:52:44 +05:30
5496a34c0c
YARN-6776. Refactor ApplicaitonMasterService to move actual processing logic to a separate class. (asuresh)
2017-07-10 14:34:58 -07:00
09653ea098
HADOOP-14634. Remove jline from main Hadoop pom.xml.
...
Contributed by Ray Chiang.
2017-07-10 11:53:13 +01:00
3de47ab5ea
YARN-6764. Simplify the logic in FairScheduler#attemptScheduling. Contributed by Yufei Gu.
2017-07-09 16:09:12 -07:00
626d730bfc
YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt.
2017-07-09 17:26:09 +05:30
9d27880442
YARN-6410. FSContext.scheduler should be final (Contributed by Yeliang Cang via Daniel Templeton)
2017-07-09 19:34:35 +09:00
c5a0c38d8e
YARN-6746. SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code
...
(Contributed by Deepti Sawhney via Daniel Templeton)
2017-07-09 19:27:32 +09:00
0615985886
YARN-6752. Display reserved resources in web UI per application
...
(Contributed by Abdullah Yousufi via Daniel Templeton)
2017-07-09 19:02:31 +09:00
7576a688ea
YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt
2017-07-06 09:40:09 -05:00
fa1aaee87b
YARN-6681. Eliminate double-copy of child queues in canAssignToThisQueue. Contributed by Daryn Sharp.
2017-07-01 12:29:39 +05:30
af2773f609
Updating version for 3.0.0-beta1 development
2017-06-29 17:57:40 -07:00
ec97519779
YARN-6751. Display reserved resources in web UI per queue
...
(Contributed by Abdullah Yousufi via Daniel Templeton)
2017-06-29 16:53:50 -07:00
c1edca101c
YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin.
2017-06-28 15:40:58 -07:00
63ce1593c5
YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper instances. Contributed by Zoltan Haindrich
2017-06-27 17:12:42 -05:00
0b77262890
YARN-5876. TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk. (Robert Kanter via Yufei Gu)
2017-06-23 13:26:03 -07:00
ca13b224b2
YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne.
2017-06-22 23:50:57 -07:00
740204b292
YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt.
2017-06-23 07:52:41 +05:30
49aa60e50d
YARN-6127. Add support for work preserving NM restart when AMRMProxy is enabled. (Botong Huang via asuresh).
2017-06-22 11:27:13 -07:00
099dfe92a9
YARN-6713. Fix dead link in the Javadoc of FairSchedulerEventLog.java. Contributed by Weiwei Yang.
2017-06-20 17:20:27 +09:00
ee89ac84e6
YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp
2017-06-19 22:42:45 +05:30
e9c2aa1bc3
YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R.
2017-06-19 21:31:40 +05:30
d64c842743
YARN-6703. RM startup failure with old state store due to version mismatch. Contributed by Varun Saxena
2017-06-12 18:07:43 +05:30
5578af8603
YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G.
2017-06-09 16:16:34 -05:00
99634d10ed
YARN-6682. Improve performance of AssignmentInformation datastructures. Contributed by Daryn Sharp.
2017-06-09 11:10:13 +05:30
a062374c39
YARN-6679. Reduce Resource instance overhead via non-PBImpl
...
(Contributed by Daryn Sharp via Daniel Templeton)
2017-06-08 10:08:08 -07:00
547cb53e8a
YARN-6634. [API] Refactor ResourceManager WebServices to make API explicit. (Giovanni Matteo Fumarola via curino)
2017-06-07 13:41:06 -07:00
0b7bff706e
YARN-6604. Allow metric TTL for Application table to be specified through cmd (Haibo Chen via Varun Saxena)
2017-06-07 21:51:07 +05:30
7311015ace
YARN-6683. Invalid event: COLLECTOR_UPDATE at KILLED. Contributed by Rohith Sharma K S
2017-06-05 13:16:57 -07:00
73ecb19312
YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application
...
(Contributed by Akira Ajisaka via Daniel Templeton)
2017-06-02 08:50:19 -07:00
d95c82cb79
YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C)
2017-06-01 18:30:23 -07:00
d5b71e4175
YARN-6246. Identifying starved apps does not need the scheduler writelock
...
(Contributed by Karthik Kambatla via Daniel Templeton)
2017-05-31 15:48:04 -07:00
4369690ce6
YARN-6649. RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception. Contributed by Jon Eagles.
2017-05-31 11:32:32 -05:00
cbfed0e82f
YARN-6497. Method length of ResourceManager#serviceInit() is too long (Gergely Novák via jeagles)
2017-05-31 10:18:42 -05:00
547f18cb96
YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf.
2017-05-31 16:15:35 +05:30
16ad896d5c
Update maven version for 3.0.0-alpha4 development
2017-05-26 14:09:44 -07:00
d81372dfad
YARN-6646. Modifier 'static' is redundant for inner enums
...
(Contributed by ZhangBing Lin via Daniel Templeton)
2017-05-26 12:05:48 -07:00
aea42930bb
YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla
2017-05-26 09:37:56 -05:00
47474fffac
YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen)
2017-05-25 21:15:27 -07:00
87590090c8
YARN-6582. FSAppAttempt demand can be updated atomically in updateDemand(). (Karthik Kambatla via Yufei Gu)
2017-05-25 14:22:13 -07:00
3fd6a2da4e
YARN-6643. TestRMFailover fails rarely due to port conflict. Contributed by Robert Kanter
2017-05-25 16:07:52 -05:00
1a56a3db59
Addendum patch to fix Docker sanitization.
2017-05-25 14:53:57 +05:30
bc28da65fb
YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan.
2017-05-25 17:06:26 +09:00
8bf1949c0e
YARN-6615. AmIpFilter drops query parameters on redirect. Contributed by Wilfred Spiegelenburg
2017-05-24 11:22:42 -05:00
8e0f83e49a
YARN-6493. Print requested node partition in assignContainer logs. (Jonathan Hung via wangda)
...
Change-Id: Ib20326daac10acad363befb0ae0998441fa5b32d
2017-05-22 14:28:55 -07:00
c583ab02c7
YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda)
...
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
2017-05-22 14:26:13 -07:00
b6f66b0da1
YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.
2017-05-22 14:10:06 +05:30
e135c0b207
YARN-6627. Use deployed webapp folder to launch new YARN UI. Contributed by Sunil G
2017-05-19 20:14:03 -07:00
9855225a79
YARN-6602. Impersonation does not work if standby RM is contacted first (rkanter)
2017-05-19 15:43:22 -07:00
66bba8c024
YARN-6249. TestFairSchedulerPreemption fails inconsistently. (Tao Jie via Yufei Gu)
2017-05-19 14:53:27 -07:00
19482e0d09
YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe.
2017-05-19 11:24:54 -05:00
b46cd315f7
Sanitize arguments before launching Docker containers.
2017-05-18 10:37:16 +05:30
e120ee865a
Validate docker image name before launching container.
2017-05-18 10:29:34 +05:30
b23fcc86c6
YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin
2017-05-17 20:40:06 -07:00
d87a63a901
HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe.
2017-05-17 08:27:45 +05:30
18c494a00c
YARN-6447. Provide container sandbox policies for groups (gphillips via rkanter)
2017-05-16 18:02:39 -07:00
8236130b2c
YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh
2017-05-16 10:48:46 -07:00
489f85933c
YARN-6603. NPE in RMAppsBlock. Contributed by Jason Lowe
2017-05-16 09:26:44 -05:00
c48f2976a3
YARN-6598. History server getApplicationReport NPE when fetching report for pre-2.8 job (Jason Lowe via jeagles)
2017-05-15 10:32:01 -05:00
6600abbb5c
YARN-6587. Refactor of ResourceManager#startWebApp in a Util class. (Giovanni Matteo Fumarola via curino).
2017-05-12 17:48:55 -07:00
6c35001b9f
YARN-6580. Incorrect logger for FairSharePolicy. (Vrushali C via Haibo Chen)
2017-05-12 13:05:40 -07:00
90cb5b4635
YARN-6380. FSAppAttempt keeps redundant copy of the queue
2017-05-11 13:47:02 -07:00
2ada100da7
YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen.
2017-05-11 13:25:31 -07:00
4aae2d40a3
HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang
2017-05-11 23:33:45 +05:30
5cb6e3e082
YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.
2017-05-10 14:49:25 -07:00
e7654c4a1f
YARN-6571. Fix JavaDoc issues in SchedulingPolicy
...
(Contributed by Weiwei Yang via Daniel Templeton)
2017-05-10 14:34:13 -07:00
74a61438ca
YARN-6475. Fix some long function checkstyle issues
...
(Contributed by Soumabrata Chakraborty via Daniel Templeton)
2017-05-10 10:46:50 -07:00
6099deebcb
YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger
2017-05-10 08:47:48 -05:00
ab2bb93a2a
YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen)
2017-05-09 21:37:30 -07:00
461ee44d28
YARN-6435. [ATSv2] Can't retrieve more than 1000 versions of metrics in time series. (Rohith Sharma K S via Haibo Chen)
2017-05-09 21:12:57 -07:00
166be0ee95
YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha)
2017-05-09 14:44:16 -07:00
a2f680493f
YARN-5301. NM mount cpu cgroups failed on some systems
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-05-09 12:05:46 -07:00
7dd258d8f4
YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C)
2017-05-09 11:18:12 -07:00
543aac9f28
HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2.
2017-05-09 09:22:53 -07:00
749e5c09b9
YARN-6281. Cleanup when AMRMProxy fails to initialize a new interceptor chain. (Botong Huang via Subru)
...
(cherry picked from commit 57a9afbd45b7ef8e6021cc58f96bd0074bf1389d)
2017-05-08 16:55:47 -07:00
cd9ff27ffc
YARN-6234. Support multiple attempts on the node when AMRMProxy is enabled. (Giovanni Matteo Fumarola via Subru).
2017-05-08 16:41:30 -07:00
424887ecb7
YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R
2017-05-08 17:14:37 -05:00
d6eed5acca
YARN-6565. Fix memory leak and finish app trigger in AMRMProxy. (Botong Huang via Subru).
2017-05-05 16:27:49 -07:00
e4f34ecb04
HADOOP-14390. Correct spelling of 'succeed' and variants. Contributed by Dongtao Zhang
2017-05-05 12:10:50 -07:00
54e2b9e876
YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C)
2017-05-04 15:25:56 -07:00
fd5cb2c946
YARN-6374. Improve test coverage and add utility classes for common Docker operations. Contributed by Shane Kumpf
2017-05-03 14:49:30 -07:00
9f0aea0ee2
YARN-6481. Yarn top shows negative container number in FS
...
(Contributed by Tao Jie via Daniel Templeton)
2017-05-02 13:06:47 -07:00
6bf42e48ef
YARN-5331. Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru).
2017-05-01 18:48:36 -07:00
e514fc432a
YARN-6542. Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent.
2017-05-01 18:10:07 -07:00
a3a615eeab
YARN-4359. Update LowCost agents logic to take advantage of YARN-4358. (Jonathan Yaniv and Ishai Menache via Subru).
2017-05-01 16:01:07 -07:00
30fc580196
YARN-6519. Fix warnings from Spotbugs in hadoop-yarn-server-resourcemanager. Contributed by Weiwei Yang.
2017-05-01 20:15:27 +05:30
0f1af3178e
YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang.
2017-05-01 16:49:30 +05:30
2e52789edf
YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton)
2017-04-28 13:30:28 -07:00
68e45f554b
YARN-6472. Improve Java sandbox regex (gphillips via rkanter)
2017-04-28 11:01:50 -07:00
371b6467dc
YARN-5894. fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 (haibochen via rkanter)
2017-04-27 17:49:38 -07:00
793bbf216d
YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena)
2017-04-27 15:01:58 +05:30
8a99eba96d
Revert "YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor."
...
This reverts commit 34ab8e73d4
2017-04-26 15:34:23 +09:00
2ba21d6376
YARN-6392. Add submit time to Application Summary log. (Zhihai Xu via wangda)
2017-04-24 13:17:35 -07:00
8ac50e1322
YARN-6500. Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler. (Miklos Szegedi via Haibo Chen)
2017-04-24 11:37:52 -07:00
52adf71914
YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe.
2017-04-24 12:03:28 -05:00
46940d92e2
YARN-6302. Fail the node if Linux Container Executor is not configured properly
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-04-19 12:23:49 -07:00
dd43b895c2
YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-04-19 11:44:55 -07:00
41ac190dba
YARN-6438. Code can be improved in ContainersMonitorImpl.java
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-04-19 10:53:08 -07:00
9ff25408bc
YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu.
2017-04-19 20:39:09 +05:30
af8e9842d2
YARN-6451. Add RM monitor validating metrics invariants. Contributed by Carlo Curino
2017-04-18 10:29:44 -07:00
6ed9d36224
YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S.
2017-04-17 15:51:10 -07:00
4168805502
YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger
2017-04-14 17:15:48 -05:00
8a1d7480f7
YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha)
2017-04-14 15:07:14 -07:00
a41f8dd58e
YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger
2017-04-14 10:53:09 -05:00
c3375175d6
YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha)
2017-04-12 14:21:20 -07:00
a16ab2be91
YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe.
2017-04-12 10:20:08 -05:00
4d4ad0ebb7
YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda)
2017-04-11 14:56:18 -07:00
7d873c465b
YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang.
2017-04-11 22:42:08 +05:30
0e065f2ede
YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu
2017-04-11 08:44:18 -05:00
7999318af1
YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos)
2017-04-10 15:34:44 -07:00
e9ac61cc0e
YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton)
2017-04-10 14:56:42 -07:00
2aa8967809
YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha)
2017-04-07 17:31:59 -07:00
63f7322522
YARN-6368. Decommissioning an NM results in a -1 exit code (miklos.szegedi@cloudera.com via rkanter)
2017-04-07 14:28:07 -07:00
0116c3c957
YARN-5797. Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches. (Chris Trezzo via mingma)
2017-04-06 16:54:43 -07:00
1b081ca27e
YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka
2017-04-06 16:24:36 -05:00
1a9439e299
YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena.
2017-04-06 10:15:22 +05:30
a2c57bb70d
YARN-6381. FSAppAttempt has several variables that should be final
...
(Contributed by Ameet Zaveri via Daniel Templeton)
2017-04-05 16:07:24 -07:00
539ef5aa2e
HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu.
2017-04-05 16:04:09 -07:00
b4c4f36594
YARN-6448. Continuous scheduling thread crashes while sorting nodes. (Yufei Gu via kasha)
2017-04-05 15:43:14 -07:00
7d963c477a
YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena)
2017-04-06 01:20:41 +05:30
87e2ef8c98
YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda)
2017-04-05 11:28:30 -07:00
e8071aa249
YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang
2017-04-05 12:30:45 -05:00
34ab8e73d4
YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor.
2017-04-05 17:33:07 +09:00
19b89c4c7b
YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena)
2017-04-05 09:38:35 +05:30
2d5c09b848
YARN-6004. Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines. (Chris Trezzo via mingma)
2017-04-04 17:56:21 -07:00
9cc04b4708
YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha)
2017-04-04 17:08:33 -07:00
3fdae0a2b6
YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda)
2017-04-04 14:39:01 -07:00
6eba79232f
HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang
2017-04-03 20:13:14 -07:00
a4b5aa8493
YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena)
2017-04-02 04:54:12 +05:30
b8838578ce
YARN-6414. ATSv2 HBase related tests fail due to guava version upgrade (Haibo Chen via Varun Saxena)
2017-04-02 04:37:34 +05:30
318bfb01bc
YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe
2017-03-31 12:30:35 -05:00
4d1fac5df2
YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu
2017-03-31 10:05:34 -05:00
b58777a9c9
YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena)
2017-03-31 02:17:20 +05:30
640ba1d23f
YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled
2017-03-29 12:38:11 -07:00
82fb9ce8df
YARN-6329. Remove unnecessary TODO comment from AppLogAggregatorImpl.java. Contributed by victor bertschinger.
2017-03-29 10:36:24 +09:00
063b513b1c
YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena)
2017-03-29 03:48:03 +05:30
01aca54a22
YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena)
2017-03-29 01:53:20 +05:30
fdf8f8ebca
YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter
2017-03-28 09:33:26 -05:00
9bae6720cb
YARN-6050. AMs can't be scheduled on racks or nodes (rkanter)
2017-03-27 17:23:13 -07:00
cd014d57aa
YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda)
2017-03-27 13:29:09 -07:00
b63ae854f9
YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki.
2017-03-27 11:23:41 +09:00
cffea251be
YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki.
2017-03-25 22:26:19 +05:30
84ddedc0b2
HADOOP-10101. Update guava dependency to the latest version. (ozawa)
2017-03-25 10:06:52 +09:00
d4f73e7e27
HDFS-11574. Spelling mistakes in the Java source. Contributed by Hu Xiaodong.
2017-03-24 09:38:17 -07:00
f462e1ff68
YARN-5934. Fix TestTimelineWebServices.testPrimaryFilterNumericString (Akira Ajisaka via Varun Saxena)
2017-03-22 04:11:53 +05:30
0a05c5c598
YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable
...
(Contributed by Laura Adams via Daniel Templeton)
2017-03-21 14:15:40 -07:00
35034653d0
YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena)
2017-03-20 23:29:09 +05:30
4a8e304502
YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi
2017-03-17 14:06:35 -05:00
7114baddb6
YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee
2017-03-16 09:30:10 -05:00
2d0e24eb01
YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202
2017-03-15 14:48:05 +05:30
4c66a8d19b
YARN-6331. Fix flakiness in TestFairScheduler#testDumpState. (Yufei Gu via rchiang)
2017-03-14 15:09:47 -07:00
0a3aa40fe7
YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung.
2017-03-15 01:22:25 +05:30
34424e98a6
YARN-6314. Potential infinite redirection on YARN log redirection web service. Contributed by Xuan Gong.
...
(cherry picked from commit 5a9dda796f0e73060ada794ad5752cc6a237ab2e)
2017-03-14 02:58:07 -07:00
4db9cc70d0
YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang)
2017-03-10 16:13:31 -08:00
229c7c9f89
YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena)
2017-03-11 04:09:01 +05:30
9649c27864
YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger
2017-03-10 13:06:54 -06:00
a96afae125
YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha)
2017-03-09 23:11:54 -08:00
846a0cd678
YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton)
2017-03-09 17:51:47 -08:00
e96a0b8c92
YARN-5669. Add support for docker pull command (Contribtued by luhuichun)
2017-03-09 16:22:19 -08:00
822a74f2ae
YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton)
2017-03-09 12:14:33 -08:00
d7762a5511
YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne
2017-03-08 16:46:09 -06:00
287ba4ffa6
YARN-6297. TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected (haibochen via rkanter)
2017-03-08 10:45:33 -08:00
1eb8186703
YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt.
2017-03-08 12:04:30 +05:30
e0c239cdbd
YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton)
2017-03-07 11:58:48 -08:00
c1386a90d7
YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne.
2017-03-04 13:46:35 +05:30
8db7a8c3ae
YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He.
2017-03-03 14:39:31 -08:00
2148b83993
YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang)
2017-03-03 13:04:51 -08:00
e58fc76030
YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen
2017-03-03 08:23:08 -06:00
6f6dfe0202
YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter)
2017-03-01 14:53:47 -08:00
235203dffd
YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam
2017-02-28 13:23:36 -08:00
e0bb867c3f
YARN-6189: Improve application status log message when RM restarted when
...
app is in NEW state. Contributed by Junping Du
2017-02-28 11:04:56 -08:00
eac6b4c35c
YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda)
2017-02-28 10:35:50 -08:00
0f35443bf2
YARN-6239. Fix javadoc warnings in YARN that caused by deprecated FileSystem APIs. Contributed by Yiqun Lin.
2017-02-28 18:51:56 +08:00
f187d63816
YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha)
2017-02-27 22:10:57 -08:00
5f5b031d1f
YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R.
2017-02-27 20:38:29 +05:30
fbfe86deea
YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha)
2017-02-26 20:36:33 -08:00
815d53506f
YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha)
2017-02-26 20:16:36 -08:00
53d372a255
YARN-6228: EntityGroupFSTimelineStore should allow configurable cache
...
stores. Contributed by Li Lu
2017-02-24 10:59:35 -08:00
694e680d20
YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha)
2017-02-23 15:21:52 -08:00
a207aa9930
YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt.
2017-02-23 14:19:07 +05:30
b10e962224
YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha)
2017-02-22 15:58:49 -08:00
718ad9f6ee
YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha)
2017-02-22 15:46:07 -08:00
1a6ca75f38
YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda)
2017-02-22 11:17:09 -08:00
172b23af33
HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu.
2017-02-18 18:34:13 -08:00
dbbfcf74ab
YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha)
2017-02-17 14:07:31 -08:00
c7a36e6130
YARN-6125. The application attempt's diagnostic message should have a maximum size
...
(Contributed by Andras Piros via Daniel Templeton)
2017-02-17 13:40:58 -08:00
9a92837786
YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with
...
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)
2017-02-17 13:13:46 -08:00
a77f432449
YARN-6171. ConcurrentModificationException on FSAppAttempt.containersToPreempt. (Miklos Szegedi via kasha)
2017-02-16 14:54:58 -08:00
4fa1afdb88
YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R.
2017-02-16 11:41:04 -08:00
e63a7814d2
YARN-6200. addendum to fix compilation error caused by reverting YARN-5068.
2017-02-16 18:19:57 +05:30
a393e84c6f
YARN-6200. Reverting since the same functionality achieved by YARN-1623.
...
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"
This reverts commit b7ac85259c
2017-02-16 18:00:25 +05:30
74dd142250
YARN-5798. Set UncaughtExceptionHandler for all FairScheduler threads. (Yufei Gu via kasha)
2017-02-16 00:03:09 -08:00
11be3f70e0
YARN-4212. FairScheduler: Can't create a DRF queue under a FAIR policy queue. (Yufei Gu via kasha)
2017-02-15 23:51:22 -08:00
6c25dbcdc0
YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha)
2017-02-15 23:16:12 -08:00
ce2d5bfa5f
YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong.
2017-02-15 09:05:14 -08:00
b7613e0f40
YARN-6156. AM blacklisting to consider node label partition (Bibin A Chundatt via Varun Saxena)
2017-02-15 14:48:17 +05:30
353a9b2d91
YARN-6061. Addendum. Remove extraneous change.
2017-02-14 15:19:52 -08:00
652679aa8a
YARN-6061. Add an UncaughtExceptionHandler for critical threads in RM. (Yufei Gu via kasha)
2017-02-14 13:39:41 -08:00
aaf106fde3
YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh)
2017-02-14 06:09:10 -08:00
646c6d6509
YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha)
2017-02-13 11:26:30 -08:00
464ff479ce
YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong.
2017-02-13 06:12:54 -08:00
649deb72fb
YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena)
2017-02-11 19:21:45 +05:30
08f93978f3
YARN-6144. FairScheduler: preempted resources can become negative. (Miklos Szegedi via kasha)
2017-02-09 14:45:04 -08:00
9b85053583
YARN-6112. UpdateCallDuration is calculated only when debug logging is enabled. (Yufei Gu via kasha)
2017-02-09 13:09:32 -08:00
5fb723bb77
YARN-5889. Improve and refactor user-limit calculation in Capacity Scheduler. (Sunil G via wangda)
2017-02-09 10:23:50 -08:00
9dbfab1284
YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong.
2017-02-07 02:13:14 -08:00
327c9980aa
YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong.
2017-02-02 00:41:18 -08:00
3e06475307
YARN-4658. Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment (Contributed by Udai Potluri via Daniel Templeton)
2017-01-31 16:25:33 -08:00
87852b6ef4
YARN-6103. Log updates for ZKRMStateStore (Contributed by Daniel Sturman via Daniel Templeton)
2017-01-30 21:44:48 -08:00
11e44bdda7
YARN-6131. FairScheduler: Lower update interval for faster tests. (kasha)
2017-01-30 13:35:23 -08:00
165f07f51a
YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan.
2017-01-27 18:37:51 +05:30
f85b74ccf9
YARN-4975. Fair Scheduler: exception thrown when a parent queue marked 'parent' has configured child queues
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-01-26 10:31:09 -08:00
9e19f758c1
YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger
2017-01-25 21:41:43 +00:00
abedb8a9d8
YARN-5830. FairScheduler: Avoid preempting AM containers. (Yufei Gu via kasha)
2017-01-25 12:17:28 -08:00
8528d85a68
YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong.
2017-01-24 15:26:53 -08:00
a33ce45e35
YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S
2017-01-24 16:17:36 +00:00
0101267d9d
YARN-6082. Invalid REST api response for getApps since queueUsagePercentage is coming as INF. Contributed by Sunil G.
2017-01-24 10:59:55 +05:30
ce832059db
YARN-5864. Capacity Scheduler - Queue Priorities. (wangda)
2017-01-23 10:52:14 -08:00
69fa81679f
YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He
2017-01-23 17:12:51 +00:00
3fa0d540df
YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang.
2017-01-23 12:19:35 +05:30
b01514f65b
YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen.
2017-01-19 20:52:55 -08:00
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
72054a817d
YARN-5556. CapacityScheduler: Support deleting queues without requiring a RM restart. (Naganarasimha G R via wangda)
2017-01-18 13:31:33 -08:00
e224c96234
YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way down to the app. (Yufei Gu via kasha)
2017-01-17 17:01:31 -08:00
4d1f3d9020
YARN-6016. Fix minor bugs in handling of local AMRMToken in AMRMProxy. (Botong Huang via Subru).
2017-01-17 14:48:03 -08:00
695f110f93
YARN-6071. Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only)
...
(Contributed by Wangda Tan via Daniel Templeton)
2017-01-17 12:55:47 -08:00
cf695577aa
YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong.
2017-01-16 16:20:24 -08:00
d3170f9eba
YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan.
2017-01-13 18:22:29 +05:30
a6b06f7179
YARN-6072. RM unable to start in secure mode. Contributed by Ajith S.
2017-01-12 06:40:41 +05:30
e6f13fe5d1
YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton)
2017-01-11 16:00:01 -08:00
7979939428
YARN-5554. MoveApplicationAcrossQueues does not check user permission on the target queue
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2017-01-11 14:34:10 -08:00
357eab9566
YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du
2017-01-11 17:13:37 +00:00
4db119b7b5
YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay.
2017-01-10 18:20:27 -08:00
4c431a6940
YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash.
2017-01-10 15:54:16 +05:30
9594c35dcb
YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang.
2017-01-10 14:08:01 +05:30
945db55f2e
YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe.
2017-01-09 18:14:46 -08:00
287d3d6804
YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)
2017-01-09 08:40:39 -08:00
f59e36b4ce
YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena)
2017-01-09 11:17:19 +05:30
ac16400e1f
YARN-6022. Revert changes of AbstractResourceRequest (Contributed by Wangda Tan via Daniel Templeton)
2017-01-07 08:24:30 -08:00
a59df15757
YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S.
2017-01-07 08:11:58 +05:30
020316458d
YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S
2017-01-06 18:19:09 -08:00
2977bc6a14
YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (Wangda Tan via asuresh)
2017-01-06 09:59:57 -08:00
0a55bd841e
YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)
2017-01-05 10:31:05 -08:00
e49e0a6e37
YARN-5988. RM unable to start in secure setup. Contributed by Ajith S.
2017-01-04 12:09:29 +05:30
8fadd69047
YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong.
2017-01-03 15:03:38 -08:00
f69a107aec
YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
...
(cherry picked from commit e0f2379312
2017-01-03 14:53:36 -08:00
ab1faa4ba8
Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" to add correct JIRA number
...
This reverts commit e0f2379312
2017-01-03 14:53:13 -08:00
e0f2379312
Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
2017-01-03 14:46:41 -08:00
165d01a73e
YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton)
2016-12-30 08:41:52 -08:00
f216276d21
YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter)
2016-12-28 15:21:52 -08:00
9ca54f4810
YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan.
2016-12-28 22:48:19 +05:30
972da46cb4
YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas.
2016-12-28 14:59:57 +05:30
f6715b26b6
YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan.
2016-12-28 12:33:02 +05:30
0840b4329b
YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda)
2016-12-27 21:18:33 -08:00
9262797e86
YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter)
2016-12-27 14:14:08 -08:00
1bbd023275
YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter)
2016-12-27 13:35:34 -08:00
e216e8e233
HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton)
2016-12-27 13:03:16 -08:00
ac1e5d4f77
YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh)
2016-12-27 12:40:44 -08:00
c3973e7080
YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha)
2016-12-27 12:30:35 -08:00
4d3f73acc0
YARN-6000. Make AllocationFileLoaderService.Listener public. (Tao Jie via kasha)
2016-12-23 11:40:56 -08:00
ae401539ea
YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.
2016-12-22 14:32:24 +09:00
736f54b727
YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong.
2016-12-21 14:14:42 -08:00
8b042bc1e6
YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C.
2016-12-21 09:43:17 -08:00
575773a357
YARN-5877. Allow all env's from yarn.nodemanager.env-whitelist to get overridden during launch. Contributed by Bibin A Chundatt.
2016-12-20 11:39:06 +05:30
ef2dd7b78c
YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB.
2016-12-19 11:44:03 +05:30
2273a74c1f
YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh)
2016-12-16 08:14:34 -08:00
ada876cd1d
Revert YARN-4126. RM should not issue delegation tokens in unsecure mode.
2016-12-13 20:49:54 -08:00
4b149a1e77
YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter)
2016-12-09 17:28:25 -08:00
a6410a542e
YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla
2016-12-09 17:00:37 -08:00
b0aace21b1
YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh)
2016-12-09 16:41:25 -08:00
55f5886ea2
YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.
2016-12-09 16:17:24 -08:00
a5a55a54ab
YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen.
2016-12-08 12:31:12 -08:00
74d0066d33
YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S.
2016-12-08 15:53:56 +05:30
72fe546841
YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter)
2016-12-07 15:07:25 -08:00
9f5d2c4fff
YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-12-07 11:12:14 -08:00
ab923a53fc
YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S
2016-12-08 00:27:25 +05:30
563480dccd
YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G.
2016-12-07 10:39:14 +05:30
f3b8ff54ab
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena
2016-12-06 06:53:38 +05:30
43ebff2e35
YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan
2016-12-05 11:39:34 -08:00
f885160f4a
YARN-5746. The state of the parentQueue and its childQueues should be synchronized. Contributed by Xuan Gong
2016-12-02 16:17:31 -08:00
5bd18c49bd
YARN-5929. Missing scheduling policy in the FS queue metric. (Contributed by Yufei Gu via Daniel Templeton)
2016-12-02 13:55:42 -08:00
69fb70c31a
YARN-5761. Separate QueueManager from Scheduler. (Xuan Gong via gtcarrera9)
2016-11-30 13:38:42 -08:00
b3befc021b
YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton)
2016-11-30 09:50:33 -08:00
cfd8076f81
Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen"
...
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
2016-11-30 15:44:52 +08:00
dd9a96cb17
YARN-5851. TestContainerManagerSecurity testContainerManager[1] failed (Contributed by Haibo Chen via Daniel Templeton)
2016-11-29 14:22:19 -08:00
3b9d3acd20
YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues
...
(Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 12:46:05 -08:00
25f9872be6
YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler
...
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 09:40:49 -08:00
62b42ef5dd
YARN-5725. Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange
...
when setting IP and host (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-28 15:19:52 -08:00
5d5614f847
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-28 14:34:44 +08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
c06114d6a3
YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena.
2016-11-27 23:35:53 +05:30
e2f77ca77b
Revert "YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena."
...
This reverts commit 9f2960be19
2016-11-27 23:32:59 +05:30
9f2960be19
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena.
2016-11-27 23:31:27 +05:30
eb0a483ed0
YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S.
2016-11-24 18:18:42 +05:30
e15c20edba
YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena.
2016-11-24 12:18:38 +05:30
10468529a9
YARN-4752. Improved preemption in FairScheduler. (kasha)
...
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge
Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
2016-11-23 19:48:59 -10:00
1f12867a69
YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S
2016-11-23 16:25:39 -08:00
005850b28f
YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh)
2016-11-23 09:53:31 -08:00
3541ed8068
YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena
2016-11-23 14:12:23 +05:30
83cc7263af
YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-11-22 11:16:00 -08:00
a926f895c1
YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G.
2016-11-22 14:49:15 +05:30
6f8074298d
YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn.
2016-11-22 10:20:31 +09:00
009452bb6d
YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger
2016-11-21 16:36:08 +00:00
f05a9ceb4a
YARN-3538. TimelineWebService doesn't catch runtime exception. Contributed by Steve Loughran and Tsuyoshi Ozawa.
2016-11-17 16:55:39 -08:00
59bfcbf357
YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang
2016-11-16 22:21:03 +00:00
d65603517e
YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee.
2016-11-16 15:14:00 +05:30
f121d0b036
YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák.
2016-11-15 13:58:11 -08:00
264ddb13ff
YARN-5736 Addendum. Fixes segfault due to unterminated string. (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-15 13:01:57 -08:00
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
3219b7b4ac
YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh)
2016-11-15 07:56:25 -08:00
7ffb9943b8
YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung.
2016-11-15 15:41:56 +05:30
43aef303bf
Reverted due to issue YARN-5765. Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang"
...
This reverts commit 131d58a24e
2016-11-15 13:27:37 +05:30
b7070f3308
YARN-5874. RM -format-state-store and -remove-application-from-state-store commands fail with NPE. Contributed by Varun Saxena.
2016-11-15 10:58:25 +05:30
296c5de0cf
YARN-5873. RM crashes with NPE if generic application history is enabled. Contributed by Varun Saxena.
2016-11-15 10:56:28 +05:30
79448d4ab1
YARN-4033. In FairScheduler, parent queues should also display queue status. (Siqi Li via kasha)
2016-11-13 21:38:36 -08:00
fad9609d13
YARN-5825. ProportionalPreemptionalPolicy should use readLock over LeafQueue instead of synchronized block. Contributed by Sunil G
2016-11-11 15:16:21 -08:00
ede1a473f5
YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda)
2016-11-11 13:34:56 -08:00
503e73e849
YARN-5545. Fix issues related to Max App in capacity scheduler. Contributed by Bibin A Chundatt
2016-11-11 20:48:31 +05:30
93eeb13164
YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li.
2016-11-10 22:35:12 +00:00
3a98419532
YARN-5834. TestNodeStatusUpdater.testNMRMConnectionConf compares nodemanager wait time to the incorrect value. (Chang Li via kasha)
2016-11-10 14:08:51 -08:00
86ac1ad9fd
YARN-5453. FairScheduler#update may skip update demand resource of child queue/app if current demand reached maxResource. (sandflee via kasha)
2016-11-09 23:44:02 -08:00
c202a10923
YARN-5862. TestDiskFailures.testLocalDirsFailures failed (Yufei Gu via Varun Saxena)
2016-11-10 11:41:34 +05:30
de3a5f8d08
YARN-5856. Unnecessary duplicate start container request sent to NM State store. Contributed by Varun Saxena.
2016-11-10 05:42:30 +05:30
bcc15c6290
YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S
2016-11-09 16:08:05 -08:00
edbee9e609
YARN-4498. Application level node labels stats to be available in REST (addendum patch). Contributed by Bibin A Chundatt.
2016-11-10 05:00:05 +05:30
59ee8b7a88
YARN-4329. [YARN-5437] Allow fetching exact reason as to why a submitted app
...
is in ACCEPTED state in Fair Scheduler (Contributed by Yufei Gu)
2016-11-09 13:11:37 -08:00
09f43fa9c0
YARN-5736. YARN container executor config does not handle white space (miklos.szegedi@cloudera.com via rkanter)
2016-11-09 13:34:40 +01:00
283fa33feb
YARN-5823. Update NMTokens in case of requests with only opportunistic containers. (Konstantinos Karanasos via asuresh)
2016-11-09 00:11:25 -08:00
e1c6ef2efa
HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey.
2016-11-08 16:33:55 -08:00
3f93ac0733
YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri
2016-11-08 22:01:26 +00:00
f38a6d03a1
YARN-5377. Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers. (Konstantinos Karanasos via asuresh)
2016-11-07 22:10:03 -08:00
de3b4aac56
YARN-5716. Add global scheduler interface definition and update CapacityScheduler to use it. Contributed by Wangda Tan
2016-11-07 10:14:39 -08:00
c00b5d1e51
YARN-5804. New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075. (Sunil G via wangda)
...
(cherry picked from commit ddebaa6b7ae1031899139d153d57df5387d60542)
2016-11-06 13:13:31 -08:00
3de0da2a76
YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
817d5f4c96
YARN-4849. Addendum patch to fix javadocs. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
c85cc3b56e
YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda)
...
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
6bb741ff0e
YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter
2016-11-04 22:20:21 +00:00
0aafc122d4
YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh)
2016-11-04 07:31:54 -07:00
19b3779ae7
YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributred by Bibin A Chundatt
2016-11-04 16:07:28 +05:30
9ee0e3172e
YARN-5822. Log ContainerRuntime initialization error in LinuxContainerExecutor. Contributed by Sidharta Seethana.
2016-11-03 22:33:43 +05:30
352cbaa7a5
YARN-4862. Handle duplicate completed containers in RMNodeImpl. Contributed by Rohith Sharma K S
2016-11-03 13:54:31 +00:00
377919010b
YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena)
2016-11-03 00:37:09 +05:30
76893a4100
YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen
2016-11-01 20:23:24 +00:00
9d0c2ad5a5
YARN-3732. Change NodeHeartbeatResponse.java and RegisterNodeManagerResponse.java as abstract classes. Contributed by Devaraj K.
2016-11-01 16:25:45 +05:30
7d2d8d25ba
YARN-5788. Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode (Bibin A Chundatt via Varun Saxena)
2016-11-01 15:32:04 +05:30
310aa4686e
YARN-5805. Add isDebugEnabled check for debug logs in nodemanager (Bibin A Chundatt via Varun Saxena)
2016-11-01 14:51:31 +05:30
733aa99313
YARN-4998. Minor cleanup to UGI use in AdminService. (Daniel Templeton via kasha)
2016-10-31 16:26:01 -07:00
90dd3a8148
YARN-2009. CapacityScheduler: Add intra-queue preemption for app priority support. (Sunil G via wangda)
2016-10-31 15:18:31 -07:00
f3eb4c3c73
YARN-5793. Trim configuration values in DockerLinuxContainerRuntime (Contributed by Tianyin Xu via Daniel Templeton)
2016-10-31 13:51:07 -07:00
2528bea67f
YARN-4396. Log the trace information on FSAppAttempt#assignContainer (Contributed by Yiqun Li via Daniel Templeton)
2016-10-31 13:34:25 -07:00
cc2c993a8a
YARN-4907. Make all MockRM#waitForState consistent. (Contributed by Yufei Gu via Daniel Templeton)
2016-10-31 13:20:56 -07:00
e0bebbbcdd
YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt
2016-10-31 04:38:20 +05:30
aa3cab1eb2
YARN-5799. Fix Opportunistic Allocation to set the correct value of Node Http Address. (asuresh)
2016-10-29 02:03:57 -07:00
1c8ab41e8b
YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena)
2016-10-29 13:47:39 +05:30
Carlo Curino
Robert Kanter
Daniel Templeton
Eric Payne
Haibo Chen
Yufei Gu
Eric Yang
Wangda Tan
Subru Krishnan
Nathan Roberts
Arun Suresh
Rohith Sharma K S
Sunil G
Jason Lowe
Jonathan Hung
Xuan
Naganarasimha
Subru Krishnan
Akira Ajisaka
Jian He
Sidharta S
Junping Du
Varun Vasudev
Andrew Wang
bibinchundatt
Allen Wittenauer
Varun Saxena
Jason Lowe
Vrushali C
Sangjin Lee
Li Lu
Arpit Agarwal
Karthik Kambatla
Brahma Reddy Battula
Ray Chiang
Steve Loughran
Daniel Templeton
Jonathan Eagles
Yufei Gu
Chris Douglas
Haibo Chen
Min Shen
Karthik Kambatla
Konstantinos Karanasos
Ming Ma
Tsuyoshi Ozawa
Ravi Prakash
Yiqun Lin
Junping Du
Kai Zheng
Xiao Chen
Mingliang Liu