3562 Commits

Author	SHA1	Message	Date
Sunil G	8bb83a8f62	Queue ACL validations should validate parent queue ACLs before auto-creating leaf queues. Contributed by Suma Shivaprasad.	2017-12-12 15:20:59 +05:30
Sunil G	5c87fb2f62	YARN-7635. TestRMWebServicesSchedulerActivities fails in trunk. Contributed by Sunil G.	2017-12-12 15:08:18 +05:30
Sunil G	312ceebde8	YARN-7632. Effective min and max resource need to be set for auto created leaf queues upon creation and capacity management. Contributed by Suma Shivaprasad.	2017-12-11 19:20:02 +05:30
Weiwei Yang	a2edc4cbf5	YARN-7608. Incorrect sTarget column causing DataTable warning on RM application and scheduler web page. Contributed by Gergely Novák.	2017-12-11 10:31:46 +08:00
Subru Krishnan	670e8d4ec7	YARN-6704. Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService. (Botong Huang via Subru).	2017-12-08 15:39:18 -08:00
Wangda Tan	04b84da245	YARN-7443. Add native FPGA module support to do isolation with cgroups. (Zhankun Tang via wangda) ... Change-Id: Ic4b7f9f3e032986b8f955139c9fe4d3a6c818a53	2017-12-08 15:18:22 -08:00
Wangda Tan	adca1a72e4	YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) ... Change-Id: I46689e530550ee0a6ac7a29786aab2cc1bdf314f	2017-12-08 15:17:02 -08:00
Wangda Tan	a8316df8c0	YARN-7520. Queue Ordering policy changes for ordering auto created leaf queues within Managed parent Queues. (Suma Shivaprasad via wangda) ... Change-Id: I482f086945bd448d512cb5b3879d7371e37ee134	2017-12-08 15:11:28 -08:00
Wangda Tan	f548bfffbd	YARN-7420. YARN UI changes to depict auto created queues. (Suma Shivaprasad via wangda) ... Change-Id: I8039d3772a191ddede132cd1f8b08a8ca2e275b7	2017-12-08 15:10:47 -08:00
Wangda Tan	b38643c9a8	YARN-7473. Implement Framework and policy for capacity management of auto created queues. (Suma Shivaprasad via wangda) ... Change-Id: Icca7805fe12f6f7fb335effff4b121b6f7f6337b	2017-12-08 15:10:16 -08:00
Wangda Tan	74665e3a7d	YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda) ... Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2	2017-12-08 15:07:56 -08:00
Sunil G	4db4a4a165	YARN-7575. NPE in scheduler UI when max-capacity is not configured. Contributed by Sunil G.	2017-12-07 18:56:54 -08:00
Sunil G	daa1cdd062	YARN-7564. Cleanup to fix checkstyle issues of YARN-5881 branch. Contributed by Sunil G.	2017-12-07 18:56:54 -08:00
Wangda Tan	1012b901c8	YARN-7544. Use queue-path.capacity/maximum-capacity to specify absolute min/max resources. (Sunil G via wangda) ... Change-Id: I685341be213eee500f51e02f01c91def89391c17	2017-12-07 18:56:54 -08:00
Wangda Tan	b7b8cd5324	YARN-7538. Fix performance regression introduced by Capacity Scheduler absolute min/max resource refactoring. (Sunil G via wangda) ... Change-Id: Ic9bd7e599c56970fe01cb0e1bba6df7d1f77eb29	2017-12-07 18:56:54 -08:00
Wangda Tan	7462c38277	YARN-7483. CapacityScheduler test cases cleanup post YARN-5881. (Sunil G via wangda) ... Change-Id: I9741a6baf5cb7352d05636efb6c0b24790e7589a	2017-12-07 18:56:54 -08:00
Rohith Sharma K S	e65ca92fb6	YARN-7482. Max applications calculation per queue has to be retrospected with absolute resource support. Contributed by Sunil G.	2017-12-07 18:56:54 -08:00
Wangda Tan	034b312d9f	YARN-7411. Inter-Queue preemption's computeFixpointAllocation need to handle absolute resources while computing normalizedGuarantee. (Sunil G via wangda) ... Change-Id: I41b1d7558c20fc4eb2050d40134175a2ef6330cb	2017-12-07 18:56:54 -08:00
Wangda Tan	aa3f62740f	YARN-7332. Compute effectiveCapacity per each resource vector. (Sunil G via wangda)	2017-12-07 18:56:54 -08:00
Wangda Tan	d52627a7cb	YARN-7254. UI and metrics changes related to absolute resource configuration. (Sunil G via wangda)	2017-12-07 18:56:54 -08:00
Wangda Tan	5e798b1a0d	YARN-6471. Support to add min/max resource configuration for a queue. (Sunil G via wangda) ... Change-Id: I9213f5297a6841fab5c573e85ee4c4e5f4a0b7ff	2017-12-07 18:56:54 -08:00
Weiwei Yang	e411dd6666	YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang.	2017-12-07 17:29:40 +08:00
Weiwei Yang	05c347fe51	YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang.	2017-12-06 12:21:52 +08:00
Sunil G	a957f1c60e	YARN-7438. Additional changes to make SchedulingPlacementSet agnostic to ResourceRequest / placement algorithm. Contributed by Wangda Tan	2017-12-05 22:50:07 +05:30
Sunil G	f9f317b702	YARN-7586. Application Placement should be done before ACL checks in ResourceManager. Contributed by Suma Shivaprasad.	2017-12-05 18:28:31 +05:30
Robert Kanter	d8863fc16f	YARN-5594. Handle old RMDelegationToken format when recovering RM (rkanter)	2017-12-04 13:14:55 -08:00
Arun Suresh	37ca416950	YARN-7587. Skip dispatching opportunistic containers to nodes whose queue is already full. (Weiwei Yang via asuresh)	2017-12-03 22:22:01 -08:00
Sunil G	81f6e46b2f	YARN-6907. Node information page in the old web UI should report resource types. Contributed by Gergely Novák.	2017-12-04 11:27:23 +05:30
Sunil G	30f2646b15	YARN-7594. TestNMWebServices#testGetNMResourceInfo fails on trunk. Contributed by Gergely Novák.	2017-12-04 10:45:07 +05:30
Jason Lowe	60f95fb719	YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan	2017-12-01 15:47:01 -06:00
Robert Kanter	c83fe44917	YARN-4813. TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently (grepas via rkanter)	2017-12-01 12:18:13 -08:00
Wangda Tan	7225ec0ceb	YARN-6507. Add support in NodeManager to isolate FPGA devices with CGroups. (Zhankun Tang via wangda) ... Change-Id: Ic9afd841805f1035423915a0b0add5f3ba96cf9d	2017-12-01 10:50:49 -08:00
Sunil G	556aea3f36	YARN-7487. Ensure volume to include GPU base libraries after created by plugin. Contributed by Wangda Tan.	2017-12-01 13:36:28 +05:30
Wangda Tan	a63d19d365	YARN-6124. Make SchedulingEditPolicy can be enabled / disabled / updated with RMAdmin -refreshQueues. (Zian Chen via wangda) ... Change-Id: Id93656f3af7dcd78cafa94e33663c78d410d43c2	2017-11-30 15:57:22 -08:00
Wangda Tan	c9a54aab6b	YARN-7573. Gpu Information page could be empty for nodes without GPU. (Sunil G via wangda) ... Change-Id: I7f614e5a589a09ce4e4286c84b706e05c29abd14	2017-11-29 17:46:16 -08:00
Daniel Templeton	8498d287cd	YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory	2017-11-29 11:11:36 -08:00
Jason Lowe	a2c7a73e33	YARN-6647. RM can crash during transitionToStandby due to InterruptedException. Contributed by Bibin A Chundatt	2017-11-28 11:15:44 -06:00
Yufei Gu	d8923cdbf1	YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu)	2017-11-27 14:31:52 -08:00
Jian He	fedabcad42	YARN-6168. Restarted RM may not inform AM about all existing containers. Contributed by Chandni Singh	2017-11-27 10:19:58 -08:00
Yufei Gu	2bde3aedf1	YARN-7290. Method canContainerBePreempted can return true when it shouldn't. (Contributed by Steven Rand)	2017-11-24 23:32:46 -08:00
Wangda Tan	834e91ee91	YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) ... Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748	2017-11-23 19:59:03 -08:00
Arun Suresh	b46ca7e73b	YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh)	2017-11-22 19:18:30 -08:00
Sunil G	aab439593b	YARN-7330. Add support to show GPU in UI including metrics. Contributed by Wangda Tan.	2017-11-23 07:54:20 +05:30
Yufei Gu	4cc9479dae	YARN-7524. Remove unused FairSchedulerEventLog. (Contributed by Wilfred Spiegelenburg)	2017-11-22 14:18:36 -08:00
Eric Yang	d42a336cfa	YARN-5534. Allow user provided Docker volume mount list. (Contributed by Shane Kumpf)	2017-11-22 13:05:34 -05:00
yufei	03c311eae3	YARN-7513. Remove the scheduler lock in FSAppAttempt.getWeight() (Contributed by Wilfred Spiegelenburg)	2017-11-21 10:33:34 -08:00
Wangda Tan	0d781dd03b	YARN-7527. Over-allocate node resource in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) ... Change-Id: I51ae6c2ab7a3d1febdd7d8d0519b63a13295ac7d	2017-11-20 11:48:15 -08:00
bibinchundatt	b5b81a4f08	YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang.	2017-11-18 19:25:29 +05:30
Subru Krishnan	d5f66888b8	YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru).	2017-11-17 17:39:06 -08:00
Eric Yang	0940e4f692	YARN-7218. Decouple YARN Services REST API namespace from RM. (Contributed by Eric Yang)	2017-11-17 12:28:12 -05:00
Wangda Tan	0987a7b8cb	YARN-7419. CapacityScheduler: Allow auto leaf queue creation after queue mapping. (Suma Shivaprasad via wangda) ... Change-Id: Ia1704bb8cb5070e5b180b5a85787d7b9ca57ebc6	2017-11-16 11:25:52 -08:00
Sunil G	61ace174cd	YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne.	2017-11-16 22:34:23 +05:30
Daniel Templeton	b246c54749	YARN-7414. FairScheduler#getAppWeight() should be moved into FSAppAttempt#getWeight() ... (Contributed by Soumabrata Chakraborty via Daniel Templeton)	2017-11-15 10:03:29 -08:00
Junping Du	e14f03dfbf	YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi.	2017-11-13 15:27:37 -08:00
Wangda Tan	dd07038ffa	YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) ... Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595	2017-11-09 14:51:15 -08:00
Konstantinos Karanasos	ac4d2b1081	YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos)	2017-11-09 13:01:24 -08:00
Robert Kanter	a2c150a736	YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter)	2017-11-09 12:12:52 -08:00
Haibo Chen	a1382a18df	YARN-7388. TestAMRestart should be scheduler agnostic.	2017-11-09 10:49:50 -08:00
bibinchundatt	0a72c2f56c	YARN-7454. RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup. Contributed by Jason Lowe.	2017-11-09 21:01:19 +05:30
Daniel Templeton	49b4c0b334	YARN-7458. TestContainerManagerSecurity is still flakey ... (Contributed by Robert Kanter via Daniel Templeton) Change-Id: Ibb1975ad086c3a33f8af0b4f8b9a13c3cdca3f7d	2017-11-08 17:31:14 -08:00
Daniel Templeton	0de10680b7	YARN-7166. Container REST endpoints should report resource types ... Change-Id: If9c2fe58d4cf758bb6b6cf363dc01f35f8720987	2017-11-08 16:43:49 -08:00
Arun Suresh	cb35a59589	YARN-7343. Add a junit test for ContainerScheduler recovery. (Sampada Dehankar via asuresh)	2017-11-08 08:14:02 -08:00
Arun Suresh	a9c70b0e84	YARN-7453. Fix issue where RM fails to switch to active after first successful start. (Rohith Sharma K S via asuresh)	2017-11-08 08:00:53 -08:00
Daniel Templeton	8db9d61ac2	YARN-7401. Reduce lock contention in ClusterNodeTracker#getClusterCapacity()	2017-11-07 14:53:48 -08:00
Wangda Tan	13fa2d4e3e	YARN-7394. Merge code paths for Reservation/Plan queues and Auto Created queues. (Suma Shivaprasad via wangda)	2017-11-06 21:38:24 -08:00
Haibo Chen	8f214dc4f8	YARN-7360. TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic.	2017-11-06 15:45:37 -08:00
Jian He	a55d0738f1	YARN-7371. Added allocateRequestId in NMContainerStatus for recovery. Contributed by Chandni Singh	2017-11-06 13:30:20 -08:00
Jian He	c723021579	YARN-6626. Embed REST API service into RM. Contributed by Eric Yang	2017-11-06 13:30:17 -08:00
Jian He	673c0db43c	Revert "YARN-6626. Embed REST API service into RM. Contributed by Eric Yang" ... This reverts commit 63d1084e9781e0fee876916190b69f6242dd00e4.	2017-11-06 13:30:17 -08:00
Jian He	9e677fa05c	YARN-6626. Embed REST API service into RM. Contributed by Eric Yang	2017-11-06 13:30:17 -08:00
Billie Rinaldi	ce74e64363	YARN-7210. Some NPE fixes in Registry DNS. Contributed by Jian He	2017-11-06 13:30:16 -08:00
Jian He	bd96c4c235	Rebase onto latest trunk. minor conflicts	2017-11-06 13:30:13 -08:00
Billie Rinaldi	1888318c89	YARN-6903. Yarn-native-service framework core rewrite. Contributed by Jian He	2017-11-06 13:30:11 -08:00
Jian He	8d335e59cf	YARN-6804. [yarn-native-services changes] Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi	2017-11-06 13:30:10 -08:00
Billie Rinaldi	ce05c6e981	YARN-6545. Followup fix for YARN-6405. Contributed by Jian He	2017-11-06 13:30:07 -08:00
bibinchundatt	dcd99c4b9a	Add containerId to Localizer failed logs. Contributed by Prabhu Joseph	2017-11-06 22:39:10 +05:30
Inigo Goiri	6fc09beac4	YARN-7434. Router getApps REST invocation fails with multiple RMs. Contributed by Inigo Goiri.	2017-11-02 21:29:53 -07:00
Eric Payne	e6ec02001f	YARN-7370: Preemption properties should be refreshable. Contrubted by Gergely Novák.	2017-11-02 12:37:33 -05:00
Jason Lowe	d00b6f7c1f	YARN-7286. Add support for docker to have no capabilities. Contributed by Eric Badger	2017-11-02 09:37:17 -05:00
Rohith Sharma K S	940ffe3f9c	addendum patch for YARN-7289.	2017-11-02 13:55:19 +05:30
Jian He	0cc98ae0ec	YARN-7396. NPE when accessing container logs due to null dirsHandler. Contributed by Jonathan Hung	2017-11-01 17:00:32 -07:00
Eric Yang	7a49ddfdde	YARN-7412. Fix unit test for docker mount check on ubuntu. (Contributed by Eric Badger)	2017-11-01 18:39:56 -04:00
Inigo Goiri	70f1a9470c	YARN-7276 addendum to add timeline service depencies. Contributed by Inigo Goiri.	2017-11-01 13:26:37 -07:00
Daniel Templeton	9711b78998	YARN-7374. Improve performance of DRF comparisons for resource types in fair scheduler	2017-10-29 18:54:33 -07:00
Yufei Gu	d4811c8cfa	YARN-6747. TestFSAppStarvation.testPreemptionEnable fails intermittently. (Contributed by Miklos Szegedi)	2017-10-29 16:44:16 -07:00
Sunil G	9114d7a5a0	YARN-7224. Support GPU isolation for docker container. Contributed by Wangda Tan.	2017-10-29 11:08:44 +05:30
Daniel Templeton	e62bbbca7a	YARN-7397. Reduce lock contention in FairScheduler#getAppWeight()	2017-10-28 09:13:13 -07:00
Arun Suresh	9c5c68745e	YARN-7299. Fix TestDistributedScheduler. (asuresh)	2017-10-27 23:08:18 -07:00
Inigo Goiri	8be5707067	YARN-7276. Federation Router Web Service fixes. Contributed by Inigo Goiri.	2017-10-27 16:46:05 -07:00
Jason Lowe	665bb147aa	YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla	2017-10-27 16:56:05 -05:00
Rohith Sharma K S	5c799ecf09	YARN-7289. Application lifetime does not work with FairScheduler. Contributed by Miklos Szegedi.	2017-10-27 22:46:38 +05:30
Sunil G	792388e1c0	YARN-7375. Possible NPE in RMWebapp when HA is enabled and the active RM fails. Contributed by Chandni Singh.	2017-10-27 20:53:57 +05:30
Wangda Tan	36e158ae98	YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda) ... Change-Id: I14c5ea7252b7c17e86ab38f692b5f9d43196dbe0	2017-10-26 20:15:19 -07:00
Robert Kanter	b1de78619f	YARN-7262. Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow (rkanter)	2017-10-26 17:47:32 -07:00
Robert Kanter	088ffee716	YARN-7320. Duplicate LiteralByteStrings in SystemCredentialsForAppsProto.credentialsForApp_ Addendum (misha@cloudera.com via rkanter)	2017-10-26 15:50:14 -07:00
Subru Krishnan	25932da6d1	YARN-5516. Add REST API for supporting recurring reservations. (Sean Po via Subru).	2017-10-26 12:10:14 -07:00
Robert Kanter	2da654e34a	YARN-7358. TestZKConfigurationStore and TestLeveldbConfigurationStore should explicitly set capacity scheduler (haibochen via rkanter)	2017-10-26 10:25:10 -07:00
Subru Krishnan	3fae675383	YARN-4827. Document configuration of ReservationSystem for FairScheduler. (Yufei Gu via Subru).	2017-10-25 15:07:50 -07:00
Haibo Chen	d7f3737f3b	YARN-7389. Make TestResourceManager Scheduler agnostic. (Robert Kanter via Haibo Chen)	2017-10-24 22:17:56 -07:00
Robert Kanter	03af442e76	YARN-7385. TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE (yufeigu via rkanter)	2017-10-24 13:36:50 -07:00
Carlo Curino	1c5c2b5dde	YARN-7339. LocalityMulticastAMRMProxyPolicy should handle cancel request properly. (Botong Huang via curino)	2017-10-24 10:39:04 -07:00
Robert Kanter	025c656572	YARN-7382. NoSuchElementException in FairScheduler after failover causes RM crash (rkanter)	2017-10-24 10:21:44 -07:00
Robert Kanter	5da295a34e	YARN-7320. Duplicate LiteralByteStrings in SystemCredentialsForAppsProto.credentialsForApp_ (misha@cloudera.com via rkanter)	2017-10-23 17:56:56 -07:00
Daniel Templeton	9e77dc2bd1	YARN-7357. Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations ... (Contributed by Sen Zhao via Daniel Templeton)	2017-10-23 13:51:19 -07:00
Eric Payne	921338cd86	YARN-4163: Audit getQueueInfo and getApplications calls	2017-10-23 11:43:41 -05:00
Haibo Chen	480187aebb	YARN-7372. TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic is flaky.	2017-10-20 14:24:17 -07:00
Yufei Gu	0799fde35e	YARN-7261. Add debug message for better download latency monitoring. (Yufei Gu)	2017-10-20 10:00:13 -07:00
Eric Yang	b61144a93d	YARN-7353. Improved volume mount check for directories and unit test compatibility on RHEL7. Contributed by Eric Badger.	2017-10-20 12:02:06 -04:00
Yufei Gu	1f4cdf1068	YARN-4090. Make Collections.sort() more efficient by caching resource usage. (Contributed by Yufei Gu, Shilong Zhang and Xianyin Xin)	2017-10-20 01:32:20 -07:00
Yufei Gu	7b4b018780	YARN-7359. TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic. (Contributed by Haibo Chen)	2017-10-19 16:51:47 -07:00
Yufei Gu	cbd2b73ef8	YARN-7294. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair Scheduler. (Contributed by Miklos Szegedi)	2017-10-19 16:39:25 -07:00
Wangda Tan	c1b08ba720	YARN-7345. GPU Isolation: Incorrect minor device numbers written to devices.deny file. (Jonathan Hung via wangda)	2017-10-19 14:45:44 -07:00
Subru Krishnan	75323394fb	YARN-7311. Fix TestRMWebServicesReservation parametrization for fair scheduler. (Yufei Gu via Subru).	2017-10-17 12:38:06 -07:00
Haibo Chen	acabc657ff	YARN-7341. TestRouterWebServiceUtil#testMergeMetrics is flakey. (Robert Kanter via Haibo Chen)	2017-10-17 10:15:53 -07:00
Robert Kanter	8a61525928	YARN-7308. TestApplicationACLs fails with FairScheduler (rkanter)	2017-10-16 15:34:32 -07:00
Nathan Roberts	4540ffd15f	YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe.	2017-10-16 17:00:38 -05:00
Arun Suresh	a50be1b8f4	YARN-7275. NM Statestore cleanup for Container updates. (Kartheek Muthyala via asuresh)	2017-10-16 13:12:15 -07:00
Wangda Tan	87ea1dff9c	Addendum fix for: YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (Wangda Tan) ... Change-Id: If1fe4a62e07b25e6f1b8ba803495da87e2cb2df6	2017-10-14 10:41:58 -07:00
Rohith Sharma K S	f4fb6695a3	YARN-7180. Remove class ResourceType. Contributed by Sunil G.	2017-10-13 15:17:05 +05:30
Carlo Curino	13fcfb3d46	YARN-7317. Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy. (contributed by Botong Huang via curino)	2017-10-12 10:38:58 -07:00
Sunil G	fa5cfc68f3	YARN-6620. Add support in NodeManager to isolate GPU devices by using CGroups. Contributed by Wangda Tan.	2017-10-11 23:44:33 +05:30
Jason Lowe	ebb34c7053	YARN-7082. TestContainerManagerSecurity failing in trunk. Contributed by Akira Ajisaka	2017-10-11 10:31:02 -05:00
Robert Kanter	ec8bf9e48a	YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter)	2017-10-10 10:18:42 -07:00
Wangda Tan	5cfbad68c0	YARN-7237. Cleanup usages of ResourceProfiles. (Wangda Tan) ... Change-Id: I959c8531bd48231956068cb7d2312f3235549f9c (cherry picked from commit 8536db5a7549fa44d9f9e444b41bd5978b5c6ae1)	2017-10-09 20:36:57 -07:00
Jonathan Hung	1d36b53ab6	YARN-7252. Removing queue then failing over results in exception	2017-10-09 11:11:23 -07:00
Jonathan Hung	09c5dfe937	YARN-7251. Misc changes to YARN-5734	2017-10-09 11:11:23 -07:00
Jonathan Hung	d6622daaa3	YARN-7046. Add closing logic to configuration store	2017-10-09 11:11:23 -07:00
Wangda Tan	ff39c0de20	YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda) ... Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc	2017-10-09 11:11:23 -07:00
Xuan	4d8abd84f4	YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung	2017-10-09 11:11:23 -07:00
Xuan	4c8b208adb	YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung	2017-10-09 11:11:23 -07:00
Xuan	74ba6ffa0b	YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung	2017-10-09 11:11:23 -07:00
Xuan	04f9e80bb2	YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao	2017-10-09 11:11:23 -07:00
Xuan	916bdbd6be	YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong)	2017-10-09 11:11:23 -07:00
Xuan	e566fd8b58	YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong)	2017-10-09 11:11:23 -07:00
Wangda Tan	a4e6253046	YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda) ... Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f	2017-10-09 11:11:23 -07:00
Wangda Tan	d8a5136b1f	YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda)	2017-10-09 11:11:23 -07:00
Jonathan Hung	ef686635c1	YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store	2017-10-09 11:11:23 -07:00
Xuan	e3579a8c3b	YARN-5946: Create YarnConfigurationStore interface and ... InMemoryConfigurationStore class. Contributed by Jonathan Hung	2017-10-09 11:11:23 -07:00
Jonathan Hung	ef59cbe08a	YARN-5951. Changes to allow CapacityScheduler to use configuration store	2017-10-09 11:11:22 -07:00
Daniel Templeton	6d6ca4c923	YARN-2612 addendum: fixed javadoc error.	2017-10-09 14:12:13 +02:00
Sunil G	8137254a1f	YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne.	2017-10-09 14:39:41 +05:30
Rohith Sharma K S	6fe6a72352	Revert "YARN-7180. Remove class ResourceType. Contributed by Sunil G." ... This reverts commit 6bd81ab054.	2017-10-09 13:21:13 +05:30
Rohith Sharma K S	6bd81ab054	YARN-7180. Remove class ResourceType. Contributed by Sunil G.	2017-10-09 10:20:40 +05:30
Yufei Gu	72d22b753a	YARN-7207. Cache the RM proxy server address. (Yufei Gu)	2017-10-06 09:49:11 -07:00
Yufei Gu	49ae538164	YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage(addendum). (Yufei Gu)	2017-10-06 00:08:25 -07:00
Yufei Gu	99292adcef	YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu)	2017-10-05 23:51:58 -07:00
Haibo Chen	c071aad5da	YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen)	2017-10-05 15:50:13 -07:00
Naganarasimha	d0046bb5c6	YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe.	2017-10-06 00:07:53 +05:30
Arun Suresh	b733348dde	YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh).	2017-10-05 09:58:04 -07:00
Sunil G	592bf2d550	YARN-7279. Fix typo in helper message of ContainerLauncher. Contributed by Elek, Marton.	2017-10-05 10:52:00 +05:30
Subru Krishnan	e6e614e380	YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru).	2017-10-04 19:28:27 -07:00
Akira Ajisaka	2df1b2ac05	YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails.	2017-10-05 06:07:32 +09:00
Akira Ajisaka	acf5b880d8	YARN-6975. Moving logging APIs over to slf4j in hadoop-yarn-server-tests, hadoop-yarn-server-web-proxy and hadoop-yarn-server-router. Contributed by Yeliang Cang.	2017-10-04 14:53:41 +09:00
Akira Ajisaka	c9b525bd94	YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.	2017-10-04 07:01:54 +09:00
Akira Ajisaka	4a87773718	YARN-6916. Moving logging APIs over to slf4j in hadoop-yarn-server-common. Contributed by Bibin A Chundatt and Akira Ajisaka.	2017-10-04 06:06:39 +09:00
Jian He	4111e6c781	YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. Contributed by Wangda Tan	2017-10-03 09:54:38 -07:00
Sidharta S	7eb8499996	YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe	2017-10-02 19:04:49 -07:00
Subru Krishnan	d4d2fd1acd	YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru).	2017-10-02 18:14:44 -07:00
Carlo Curino	ceca9694f9	YARN-3661. Basic Federation UI. (Contributed by Inigo Goiri via curino)	2017-10-02 13:03:32 -07:00
Wangda Tan	febeead5f9	YARN-6550. Capture launch_container.sh logs to a separate log file. (Suma Shivaprasad via wangda) ... Change-Id: I0ee0b1bb459437432a22cf68861a6354f0decabb	2017-09-29 16:39:46 -07:00
Eric Payne	7f6118f918	YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe.	2017-09-29 10:33:57 -05:00
Wangda Tan	d3b1c63195	YARN-6623. Add support to turn off launching privileged containers in the container-executor. (Varun Vasudev via wangda) ... Change-Id: I76aec68cbfb42d239279d7245a03290d691e87a4	2017-09-28 16:41:09 -07:00
Subru Krishnan	ca669f9f8b	YARN-6962. Add support for updateContainers when allocating using FederationInterceptor. (Botong Huang via Subru).	2017-09-28 13:04:03 -07:00
Jason Lowe	85d81ae58e	YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh	2017-09-28 14:10:15 -05:00
Sunil G	8facf1f976	YARN-6871. Add additional deSelects params in RMWebServices#getAppReport. Contributed by Tanuj Nayak.	2017-09-27 14:37:32 +05:30
Jason Lowe	14fec04ed3	Revert "YARN-6570. No logs were found for running application, running" ... This reverts commit 7c732924a8.	2017-09-26 10:38:03 -05:00
Rohith Sharma K S	06e5a7b5cf	YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R.	2017-09-26 14:34:09 +05:30
Arun Suresh	df800f6cf3	YARN-7240. Add more states and transitions to stabilize the NM Container state machine. (Kartheek Muthyala via asuresh)	2017-09-25 16:02:55 -07:00
Daniel Templeton	164a063e74	YARN-6878. TestCapacityScheduler.testDefaultNodeLabelExpressionQueueConfig() has the args to assertEqual() in the wrong order ... (Contributed by Sen Zhao via Daniel Templeton)	2017-09-22 23:53:32 +02:00
Junping Du	4002bf0a9e	YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi.	2017-09-22 13:58:09 -07:00
Jason Lowe	bfd1a72ba8	YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger.	2017-09-21 17:41:34 -05:00
Junping Du	e5e1851d80	YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi.	2017-09-21 14:01:16 -07:00
Yufei Gu	a92ef030a2	YARN-7045. Remove FSLeafQueue#addAppSchedulable. (Contributed by Sen Zhao via Yufei Gu)	2017-09-21 12:31:07 -07:00
Wangda Tan	263e2c692a	YARN-6991. "Kill application" button does not show error if other user tries to kill the application for secure cluster. (Suma Shivaprasad via wangda) ... Change-Id: I7e7894b24609709f89064ee5882f055dbb09080b	2017-09-21 12:00:53 -07:00
Jason Lowe	10d7493587	YARN-6968. Hardcoded absolute pathname in DockerLinuxContainerRuntime. Contributed by Eric Badger	2017-09-21 08:38:08 -05:00
Yufei Gu	a9019e1fb7	YARN-6499. Remove the doc about Schedulable#redistributeShare(). (Contributed by Chetna Chaudhari via Yufei Gu)	2017-09-19 18:28:31 -07:00
Junping Du	647b7527a9	YARN-7186. Fix finicky TestContainerManager tests. Contributed by Arun Suresh.	2017-09-19 18:31:15 -07:00
Arun Suresh	56ef5279c1	YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh)	2017-09-18 15:49:31 -07:00
Xuan	7c732924a8	YARN-6570. No logs were found for running application, running ... container. Contributed by Junping Du	2017-09-18 14:04:05 -07:00
Jason Lowe	a4f9c7c924	YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh	2017-09-18 10:16:09 -05:00
Wangda Tan	e81596d06d	YARN-7172. ResourceCalculator.fitsIn() should not take a cluster resource parameter. (Sen Zhao via wangda) ... Change-Id: Icc3670c9381ce7591ca69ec12da5aa52d3612d34	2017-09-17 21:20:43 -07:00
Wangda Tan	38c14ef8d8	YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda) ... Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6 (cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)	2017-09-15 21:29:39 -07:00
Wangda Tan	958e8c0e25	YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda) ... Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e (cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)	2017-09-15 21:29:31 -07:00
Yufei Gu	09b476e6da	YARN-6612. Update fair scheduler policies to be aware of resource types. (Contributed by Daniel Templeton via Yufei Gu)	2017-09-14 11:23:37 -07:00
Arun Suresh	66ca0a6540	YARN-6059. Update paused container state in the NM state store. (Hitesh Sharma via asuresh)	2017-09-14 08:51:26 -07:00
Arun Suresh	4f8194430f	YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh)	2017-09-14 08:51:26 -07:00
Arun Suresh	864fbacd45	YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh)	2017-09-14 08:48:39 -07:00
Robert Kanter	bb34ae9554	YARN-7146. Many RM unit tests failing with FairScheduler (rkanter)	2017-09-13 15:44:54 -07:00
Eric Payne	3860be7961	YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe.	2017-09-13 14:41:55 -05:00
Eric Payne	a3c44195be	Revert 'YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe.' ... This reverts commit 729d05f529.	2017-09-13 14:38:58 -05:00
Eric Payne	729d05f529	YARN-4727. Unable to override the /home/ericp/run/conf/ env variable for container. Contributed by Jason Lowe.	2017-09-13 14:29:08 -05:00
Rohith Sharma K S	5324388cf2	YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G.	2017-09-13 23:36:47 +05:30
Arun Suresh	a1c398e0dc	YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix.	2017-09-12 17:03:32 -07:00
Yufei Gu	82c5dd1d50	YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu)	2017-09-12 16:13:39 -07:00
Arun Suresh	2ae72692fc	YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh)	2017-09-12 16:10:08 -07:00
Sunil G	da0b6a354b	YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan.	2017-09-12 09:19:12 -07:00
Daniel Templeton	bf2b687412	YARN-7136. Additional Performance Improvement for Resource Profile Feature ... (Contributed by Wangda Tan via Daniel Templeton)	2017-09-12 09:19:12 -07:00
Wangda Tan	e490602e9b	YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) ... Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d	2017-09-12 09:19:12 -07:00
Sunil G	a333ba54e3	YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan.	2017-09-12 09:19:12 -07:00
Wangda Tan	39240b61a1	YARN-7043. Cleanup ResourceProfileManager. (wangda) ... Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913	2017-09-12 09:19:12 -07:00
Wangda Tan	df3855541a	YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) ... Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d	2017-09-12 09:19:12 -07:00
Daniel Templeton	6746f8cadb	YARN-6908. ResourceProfilesManagerImpl is missing @Overrides on methods ... (Contributed by Sunil G. via Daniel Templeton)	2017-09-12 09:19:12 -07:00
Daniel Templeton	5c4ab4c291	YARN-6935. [YARN-3926] ResourceProfilesManagerImpl.parseResource() has no need of the key parameter ... (Contributed by Manikandan R via Daniel Templeton)	2017-09-12 09:19:11 -07:00
Daniel Templeton	3aeaafecb8	YARN-6788. [YARN-3926] Improve performance of resource profile branch ... (Contributed by Sunil Govindan via Daniel Templeton)	2017-09-12 09:19:11 -07:00
Sunil G	58da54640e	YARN-6445. [YARN-3926] Performance improvements in resource profile branch with respect to SLS. Contributed by Varun Vasudev.	2017-09-12 09:19:11 -07:00
Sunil G	dae65f3bef	YARN-6232. Update resource usage and preempted resource calculations to take into account all resource types. Contributed by Varun Vasudev.	2017-09-12 09:19:11 -07:00
Sunil G	7805deed48	YARN-5588. [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev.	2017-09-12 09:19:11 -07:00
Arun Suresh	6708ac3301	YARN-5587. Add support for resource profiles. (vvasudev via asuresh)	2017-09-12 09:19:10 -07:00
Varun Vasudev	c2032e251e	YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev.	2017-09-12 09:19:10 -07:00
Varun Vasudev	a9b4426302	YARN-5707. Add manager class for resource profiles. Contributed by Varun Vasudev.	2017-09-12 09:19:10 -07:00
Rohith Sharma K S	239c1824a0	YARN-5586. Update the Resources class to consider all resource types. Contributed by Varun Vasudev.	2017-09-12 09:19:10 -07:00
Varun Vasudev	759114b006	YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev.	2017-09-12 09:19:10 -07:00
Wangda Tan	e74d1be04b	YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) ... Change-Id: I1c39ed5c59dee739ba5044b61b3ef5ed203b79c1	2017-09-11 20:52:08 -07:00
Haibo Chen	fa531788fd	YARN-7132. FairScheduler.initScheduler() contains a surprising unary plus. (Yeliang Cang via Haibo Chen)	2017-09-11 20:42:49 -07:00
Haibo Chen	661f5eb0c6	YARN-7128. The error message in TimelineSchemaCreator is not enough to find out the error. (Jinjiang Ling via Haibo Chen)	2017-09-11 16:20:20 -07:00
Yufei Gu	5ab21dfe9c	YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu)	2017-09-10 23:17:59 -07:00
Sunil G	722ee84194	YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S.	2017-09-11 10:28:10 +05:30
Junping Du	3fddabc2fe	YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong.	2017-09-08 15:16:19 -07:00
Arun Suresh	1f53ae7972	YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh)	2017-09-08 09:24:05 -07:00
Subru Krishnan	fa6137501c	YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru).	2017-09-07 19:07:17 -07:00
Sunil G	56d93d2e39	YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S.	2017-09-08 07:15:17 +05:30
Jason Lowe	b0b535d9d5	YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf	2017-09-07 16:17:03 -05:00
Wangda Tan	f155ab7cfa	YARN-7033. Add support for NM Recovery of assigned resources (e.g. GPU's, NUMA, FPGA's) to container. (Devaraj K and Wangda Tan) ... Change-Id: Iffd18bb95debe1c8cc55e30abc1d8f663e9d0e30	2017-09-07 14:13:37 -07:00
Sunil G	6f101e7df1	YARN-6992. Kill application button is visible even if the application is FINISHED in RM UI. Contributed by Suma Shivaprasad.	2017-09-07 13:16:20 +05:30
Subru Krishnan	b6e7d13696	YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem.	2017-09-06 16:46:01 -07:00
Subru Krishnan	e3345e985b	Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem." ... This reverts commit 7996eca7dc.	2017-09-06 16:39:23 -07:00
Subru Krishnan	7996eca7dc	Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem.	2017-09-01 15:16:40 -07:00
Andrew Wang	0d419c984f	Preparing for 3.1.0 development	2017-09-01 11:53:48 -07:00
Jason Lowe	99a7f5d451	YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne	2017-09-01 11:09:14 -05:00
bibinchundatt	dcd0bedcc8	YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena	2017-09-01 11:29:16 +05:30
Allen Wittenauer	0adc3a0533	YARN-6721. container-executor should have stack checking ... Signed-off-by: Chris Douglas <cdouglas@apache.org>	2017-08-31 19:39:31 -07:00
Junping Du	91cc070d67	YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong.	2017-08-31 16:41:43 -07:00
Subru Krishnan	bac4e8cca8	YARN-7095. Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru).	2017-08-31 15:05:41 -07:00
Daniel Templeton	d4417dae4f	YARN-6780. ResourceWeights.toString() cleanup (Contributed by weiyuan via Daniel Templeton)	2017-08-31 09:35:01 -07:00
Sunil G	f9e0cc8cdc	YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan.	2017-08-31 18:42:01 +05:30
Allen Wittenauer	71bbb86d69	HADOOP-14670. Increase minimum cmake version for all platforms ... Signed-off-by: Chris Douglas <cdouglas@apache.org>	2017-08-30 21:47:24 -07:00
Daniel Templeton	cc23514aba	YARN-7115. Move BoundedAppender to org.hadoop.yarn.util pacakge ... (Contributed by Jian He via Daniel Templeton)	2017-08-30 17:26:13 -07:00
Haibo Chen	a20e7105ea	YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen)	2017-08-30 09:14:59 -07:00
Varun Saxena	3d00c8f394	Made fixes for whitespace errors and checstyle warnings before merge.	2017-08-30 11:29:54 +05:30
Varun Saxena	512068a01f	YARN-6905. Addendum to fix TestTimelineReaderWebServicesHBaseStorage due to missing FastNumberFormat	2017-08-30 11:29:54 +05:30
Jian He	08f40bcc7f	MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena	2017-08-30 11:29:54 +05:30
Haibo Chen	9b08f365d7	YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen)	2017-08-30 11:29:54 +05:30
Varun Saxena	b2efebdd07	YARN-6861. Reader API for sub application entities (Rohith Sharma K S via Varun Saxena)	2017-08-30 11:29:54 +05:30
Jian He	7fd6ae2479	YARN-6134. [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active. Contributed by Varun Saxena	2017-08-30 11:29:54 +05:30
Rohith Sharma K S	e276c75ec1	YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena.	2017-08-30 11:29:54 +05:30
Rohith Sharma K S	32188d3295	YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena.	2017-08-30 11:29:54 +05:30
Jian He	b664569586	YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena	2017-08-30 11:29:53 +05:30
Jason Lowe	d5ff965fee	YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C	2017-08-30 11:29:53 +05:30
Varun Saxena	60765aff9b	YARN-6874. Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite (Vrushali C via Varun Saxena)	2017-08-30 11:29:53 +05:30
Rohith Sharma K S	354be99dbf	YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena.	2017-08-30 11:29:53 +05:30
Rohith Sharma K S	7594d1de7b	YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena.	2017-08-30 11:29:53 +05:30
Varun Saxena	9f6540535d	YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena)	2017-08-30 11:29:53 +05:30
Rohith Sharma K S	3fb71b1393	YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C.	2017-08-30 11:29:53 +05:30
Rohith Sharma K S	a990ff70c2	YARN-6733. Add table for storing sub-application entities. Contributed by Vrushali C.	2017-08-30 11:29:53 +05:30
Vrushali C	61136d03f2	YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics (Contributed by Varun Saxena via Vrushali C)	2017-08-30 11:29:53 +05:30
Rohith Sharma K S	70078e91e3	YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena.	2017-08-30 11:29:53 +05:30
Rohith Sharma K S	660413165a	YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C.	2017-08-30 11:29:53 +05:30
Jian He	ac7f52df83	YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena	2017-08-30 11:29:53 +05:30
Jian He	d3f11e3f13	YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena	2017-08-30 11:29:53 +05:30
Jian He	879de51206	YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena	2017-08-30 11:29:53 +05:30
Varun Saxena	f6a51dce18	YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena)	2017-08-30 11:29:52 +05:30
Haibo Chen	a8f082a180	YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen) ... (cherry picked from commit 47474fffac)	2017-08-30 11:29:52 +05:30
Varun Saxena	b87b72b40a	YARN-6146. Add Builder methods for TimelineEntityFilters (Haibo Chen via Varun Saxena)	2017-08-30 11:29:52 +05:30
Varun Saxena	44999aabc2	YARN-6318. timeline service schema creator fails if executed from a remote machine (Sangjin Lee via Varun Saxena)	2017-08-30 11:29:52 +05:30
Varun Saxena	18b3a80df7	YARN-6237. Move UID constant to TimelineReaderUtils (Rohith Sharma K S via Varun Saxena)	2017-08-30 11:29:52 +05:30
Varun Saxena	c3bd8d6ad3	YARN-6256. Add FROM_ID info key for timeline entities in reader response (Rohith Sharma K S via Varun Saxena)	2017-08-30 11:29:52 +05:30
Varun Saxena	8bb2646595	YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena)	2017-08-30 11:29:52 +05:30
Sangjin Lee	cf30b3b914	YARN-6253. FlowAcitivityColumnPrefix.store(byte[] rowKey, ...) drops timestamp. Contributed by Haibo Chen.	2017-08-30 11:29:52 +05:30
Sangjin Lee	ccb38c1991	YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen.	2017-08-30 11:29:52 +05:30
Sangjin Lee	580d884913	YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C.	2017-08-30 11:29:52 +05:30
Varun Saxena	5e0acee75e	Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's	2017-08-30 11:29:52 +05:30
Varun Saxena	6f65cf27bb	YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena)	2017-08-30 11:29:51 +05:30
Varun Saxena	71847ed44d	YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena)	2017-08-30 11:29:51 +05:30
Varun Saxena	02a9710a09	YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena)	2017-08-30 11:29:51 +05:30
Sangjin Lee	2556c012e3	YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.	2017-08-30 11:29:51 +05:30
Sangjin Lee	4481561e4a	YARN-5739. Provide timeline reader API to list available timeline entity types for one application. Contributed by Li Lu.	2017-08-30 11:29:51 +05:30
Sangjin Lee	092fead5d9	YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena.	2017-08-30 11:29:51 +05:30
Varun Saxena	c92a7ab31c	YARN-3359. Addendum for Recover collector list when RM fails over (Li Lu via Varun Saxena)	2017-08-30 11:29:51 +05:30
Varun Saxena	91d3443806	YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena)	2017-08-30 11:29:51 +05:30
Sangjin Lee	643a20a358	YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis.	2017-08-30 11:29:51 +05:30
Sangjin Lee	05ff04439e	YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S.	2017-08-30 11:29:51 +05:30
Varun Saxena	9bb5cc3d49	YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena)	2017-08-30 11:29:50 +05:30
Sangjin Lee	78b7e070d8	YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu.	2017-08-30 11:29:50 +05:30
Li Lu	e72e6260da	YARN-5229. Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl. (Vrushali C via gtcarrera9)	2017-08-30 11:29:50 +05:30
Sangjin Lee	278bd6fcff	YARN-5156. YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state (Vrushali C via sjlee)	2017-08-30 11:29:50 +05:30
Rohith Sharma K S	4cae120c61	YARN-6982. Potential issue on setting AMContainerSpec#tokenConf to null before app is completed. Contributed by Manikandan R.	2017-08-30 10:45:11 +05:30
Wangda Tan	f59332b97b	YARN-5219. When an export var command fails in launch_container.sh, the full container launch should fail. (Sunil G via wangda) ... Change-Id: Iaa6b978bb89482e9d1d77ba57f4adfdc48e39a3c	2017-08-29 15:29:22 -07:00
Junping Du	ad45d19998	YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang.	2017-08-29 15:16:23 -07:00
Carlo Curino	cc8893edc0	YARN-7010. Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps). (Contributed by Giovanni Matteo Fumarola via curino)	2017-08-29 14:53:09 -07:00
Jason Lowe	1f6dc4ee9a	YARN-5816. TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey. Contributed by Robert Kanter	2017-08-29 09:52:24 -05:00
Junping Du	312b1fd9da	YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He.	2017-08-28 18:26:51 -07:00
Yufei Gu	d5bc958279	YARN-7099. ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. (Contributed by Miklos Szegedi via Yufei Gu)	2017-08-28 10:09:46 -07:00
Sunil G	02599bda04	YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne.	2017-08-28 12:22:56 +05:30
Li Lu	3bee7046c5	YARN-6999. Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. (Linlin Zhou via gtcarrera9)	2017-08-25 22:43:25 -07:00
Eric Payne	e864f81471	YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe.	2017-08-25 15:37:54 -05:00
Jason Lowe	39a9dc8e4a	YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne	2017-08-25 15:31:29 -05:00
Arpit Agarwal	8b7cbe3840	HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar.	2017-08-25 09:55:46 -07:00
Jason Lowe	3a4e861169	YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang	2017-08-25 09:16:17 -05:00
Yufei Gu	9e2699ac2c	YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu)	2017-08-24 17:52:41 -07:00
Junping Du	c2cb7ea1ef	YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong.	2017-08-24 13:36:49 -07:00
bibinchundatt	de0cba700b	YARN-7074. Fix NM state store update comment. Contributed by Botong Huang	2017-08-24 16:34:38 +05:30
Junping Du	652dd434d9	YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan.	2017-08-23 18:06:29 -07:00
Wangda Tan	f49843a988	YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda) ... Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb	2017-08-23 09:56:20 -07:00
Subru Krishnan	4249172e14	YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru).	2017-08-22 19:20:57 -07:00
Jason Lowe	657dd59cc8	YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp	2017-08-22 13:16:24 -05:00
Jason Lowe	3efcd51c3b	YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles	2017-08-22 12:56:09 -05:00
Akira Ajisaka	d5ff57a08f	YARN-7047. Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager. Contributed by Yeliang Cang.	2017-08-22 17:14:12 +09:00
Arun Suresh	ae8fb13b31	YARN-6923. Metrics for Federation Router. (Giovanni Matteo Fumarola via asuresh)	2017-08-21 22:50:24 -07:00
Arun Suresh	75abc9a8e2	YARN-5603. Metrics for Federation StateStore. (Ellen Hui via asuresh)	2017-08-21 22:43:08 -07:00
Arun Suresh	7a82d7bcea	YARN-6979. [Addendum patch] Fixed classname and added javadocs. (Kartheek Muthyala via asuresh)	2017-08-20 10:24:05 -07:00
Arun Suresh	8410d862d3	YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)	2017-08-20 07:54:09 -07:00
Wangda Tan	436c2638f9	YARN-6852. Native code changes to support isolate GPU devices by using CGroups. (wangda) ... Change-Id: I4869cc4d8ad539539ccba4bea5a178cacdb741ab	2017-08-18 18:26:36 -07:00
Yufei Gu	8991f0baec	YARN-6969. Clean up unused code in class FairSchedulerQueueInfo. (Larry Lo via Yufei Gu)	2017-08-18 14:38:44 -07:00
bibinchundatt	e05fa3451d	YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su.	2017-08-18 20:28:50 +05:30
Jason Lowe	ab1a8ae85f	YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger	2017-08-17 15:50:14 -05:00
Sunil G	f9a0e23381	YARN-3254. HealthReport should include disk full information. Contributed by Suma Shivaprasad.	2017-08-17 15:07:15 +05:30
Subru Krishnan	de462da04e	YARN-6900. ZooKeeper based implementation of the FederationStateStore. (Íñigo Goiri via Subru).	2017-08-16 11:43:24 -07:00
Akira Ajisaka	588c190afd	YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka.	2017-08-16 14:06:22 +09:00
Subru Krishnan	75dd866bfb	HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru).	2017-08-15 16:53:59 -07:00
Nathan Roberts	d265459024	YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts)	2017-08-15 15:52:48 -05:00
Arun Suresh	4d7be1d857	YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh)	2017-08-14 19:46:17 -07:00
Jason Lowe	5558792894	YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne	2017-08-14 15:31:34 -05:00
Jason Lowe	3325ef653d	YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles	2017-08-14 14:40:08 -05:00
Daniel Templeton	6b09c32705	YARN-6881. LOG is unused in AllocationConfiguration (Contributed by weiyuan via Daniel Templeton)	2017-08-14 11:55:33 -07:00
Vrushali C	608a06cca5	YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen)	2017-08-14 11:41:11 -07:00
Subru Krishnan	18f3603bce	YARN-6996. Change javax.cache library implementation from JSR107 to Apache Geronimo. (Ray Chiang via Subru).	2017-08-14 11:10:00 -07:00
Jian He	e2f6299f6f	YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang	2017-08-14 10:51:30 -07:00
bibinchundatt	d8f74c3964	YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R.	2017-08-14 09:39:00 +05:30
Carlo Curino	28d97b79b6	YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino)	2017-08-11 16:58:04 -07:00
Carlo Curino	cc59b5fb26	YARN-6896. Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution). (Contributed by Giovanni Matteo Fumarola via curino)	2017-08-11 15:58:01 -07:00
Daniel Templeton	0996acde6c	YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator ... (Contributed by Larry Lo via Daniel Templeton)	2017-08-11 14:50:46 -07:00
Daniel Templeton	65364defb4	YARN-6967. Limit application attempt's diagnostic message size thoroughly ... (Contributed by Chengbing Liu via Daniel Templeton)	2017-08-11 14:28:55 -07:00
Daniel Templeton	c7680d4cc4	YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces ... (Contributed by weiyuan via Daniel Templeton)	2017-08-11 14:22:02 -07:00
Daniel Templeton	218588be77	YARN-6952. Enable scheduling monitor in FS (Contributed by Yufei Gu via Daniel Templeton)	2017-08-11 14:04:19 -07:00
Subru Krishnan	bbbf0e2a41	HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru).	2017-08-11 13:58:45 -07:00
Karthik Kambatla	8c4b6d16a5	YARN-5927. BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate. (Kai Sasaki via kasha)	2017-08-11 12:15:43 -07:00
Wangda Tan	ec694145cf	YARN-6033. Add support for sections in container-executor configuration file. (Varun Vasudev via wandga) ... Change-Id: Ibc6d2a959debe5d8ff2b51504149742449d1f1da	2017-08-09 10:51:29 -07:00
Akira Ajisaka	63cfcb90ac	YARN-6958. Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice. Contributed by Yeliang Cang.	2017-08-09 23:58:22 +09:00
Akira Ajisaka	1a18d5e514	YARN-6515. Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager. Contributed by Naganarasimha G R.	2017-08-09 21:56:43 +09:00
Subru Krishnan	ad2a350662	YARN-6970. Add PoolInitializationException as retriable exception in FederationFacade. (Giovanni Matteo Fumarola via Subru).	2017-08-08 16:48:29 -07:00
Daniel Templeton	e0c24145d2	YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code ... (Contributed by Angela Wang via Daniel Templeton)	2017-08-08 13:35:22 -07:00
Wangda Tan	1794de3ea4	YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda) ... Change-Id: If1b1827345f98f0a49cc7e39d1ba41fbeed5e911	2017-08-08 12:56:29 -07:00
Jian He	acf9bd8b1d	YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du	2017-08-08 11:09:38 -07:00
Daniel Templeton	47b145b9b4	YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path ... (Contributed by Miklos Szegedi via Daniel Templeton)	2017-08-08 10:33:26 -07:00
Akira Ajisaka	98912950b6	YARN-6961. Remove commons-logging dependency from hadoop-yarn-server-applicationhistoryservice module. Contributed by Yeliang Cang.	2017-08-08 19:38:58 +09:00
Arun Suresh	8d3fd81980	YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh)	2017-08-07 18:59:25 -07:00
Subru Krishnan	c61f2c4198	YARN-6955. Handle concurrent register AM requests in FederationInterceptor. (Botong Huang via Subru).	2017-08-07 16:58:29 -07:00
Wangda Tan	adb84f34db	YARN-4161. Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration. (Wei Yan via wangda) ... Change-Id: Ic441ae4e0bf72e7232411eb54243ec143d5fd0d3	2017-08-07 11:32:21 -07:00
Akira Ajisaka	b0fbf17965	YARN-6957. Moving logging APIs over to slf4j in hadoop-yarn-server-sharedcachemanager. Contributed by Yeliang Cang.	2017-08-07 19:25:40 +09:00
Akira Ajisaka	839e077faf	YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang.	2017-08-07 18:56:00 +09:00
Sunil G	46b7054fa7	YARN-6951. Fix debug log when Resource Handler chain is enabled. Contributed by Yang Wang.	2017-08-07 13:15:46 +05:30
Junping Du	f44b349b81	YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S.	2017-08-04 16:03:56 -07:00
Arun Suresh	35dc782923	YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh)	2017-08-03 21:15:40 -07:00
Subru Krishnan	0542e6f86e	YARN-6932. Fix TestFederationRMFailoverProxyProvider test case failure.	2017-08-03 14:18:03 -07:00
Yufei Gu	c617fe02b3	YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu)	2017-08-03 11:33:37 -07:00
Haibo Chen	293c74a81b	YARN-6673 Add cpu cgroup configurations for opportunistic containers. (Miklos Szegedi via Haibo Chen)	2017-08-03 09:56:51 -07:00
Haibo Chen	c5d256c760	YARN-6674 Add memory cgroup settings for opportunistic containers. (Miklos Szegedi via Haibo Chen)	2017-08-03 09:52:35 -07:00
Sunil G	f64cfeaf61	YARN-6678. Handle IllegalStateException in Async Scheduling mode of CapacityScheduler. Contributed by Tao Yang.	2017-08-03 19:27:10 +05:30
Eric Payne	8ce8672b6b	YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe.	2017-08-02 12:53:22 -05:00
Yufei Gu	45535f8afa	YARN-6895. [FairScheduler] Preemption reservation may cause regular reservation leaks. (Miklos Szegedi via Yufei Gu)	2017-08-02 09:25:19 -07:00
Eric Payne	48899134d2	YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe.	2017-08-02 10:59:33 -05:00
Jian He	f9139ac8f6	YARN-6872. [Addendum patch] Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G	2017-08-02 00:03:25 -07:00
Carlo Curino	acda6b96a4	YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino) ... (cherry picked from commit b6240b92ab)	2017-08-01 17:28:28 -07:00
Subru Krishnan	86f05c3312	YARN-6866. Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk. (Botong Huang via Subru). ... (cherry picked from commit 40453879ec)	2017-08-01 17:28:28 -07:00
Subru Krishnan	8737c2a8e9	Bumping up hadoop-yarn-server-router module to 3.0.0-beta1 post rebase. ... (cherry picked from commit 69e3ed2680)	2017-08-01 17:28:27 -07:00
Subru Krishnan	712e97d4cf	Bumping up yarn-server-router (new) module pom to beta1 and fixing imports post rebase. ... (cherry picked from commit f1eff212c5)	2017-08-01 17:28:27 -07:00
Subru Krishnan	3e1dc7ece0	YARN-6821. Move FederationStateStore SQL DDL files from test resource to sbin. ... (cherry picked from commit cd9db822f0)	2017-08-01 17:28:27 -07:00
Subru Krishnan	825108d08a	YARN-6815. FederationStateStoreFacade return behavior should be consistent irrespective of whether caching is enabled or not. ... (cherry picked from commit 8820693cd7)	2017-08-01 17:28:27 -07:00
Subru Krishnan	43a97174fe	YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru). ... (cherry picked from commit 52daa6d971)	2017-08-01 17:28:26 -07:00
Subru Krishnan	4cfec943b1	YARN-6724. Add ability to blacklist sub-clusters when invoking Routing policies. (Giovanni Matteo Fumarola via Subru). ... (cherry picked from commit f8e5de5969)	2017-08-01 17:28:26 -07:00
Subru Krishnan	70b1a757f1	YARN-6511. Federation: transparently spanning application across multiple sub-clusters. (Botong Huang via Subru). ... (cherry picked from commit 8c988d235e)	2017-08-01 17:28:26 -07:00
Subru Krishnan	bed1832c93	YARN-3666. Federation Intercepting and propagating AM- home RM communications. (Botong Huang via Subru). ... (cherry picked from commit 2399eb8200)	2017-08-01 17:28:25 -07:00
Subru Krishnan	42949a6e01	YARN-6666. Fix unit test failure in TestRouterClientRMService. (Botong Huang via Subru). ... (cherry picked from commit e750907d0a)	2017-08-01 17:28:25 -07:00
Subru Krishnan	f317e0cb48	Bumping up hadoop-yarn-server-router module to 3.0.0-alpha4 post rebase. ... (cherry picked from commit bd9c7b1bec)	2017-08-01 17:28:25 -07:00
Subru Krishnan	859aa1f9d6	YARN-5531. UnmanagedAM pool manager for federating application across clusters. (Botong Huang via Subru). ... (cherry picked from commit 73bb2102ce)	2017-08-01 17:28:24 -07:00
Subru Krishnan	7444406d6d	YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru). ... (cherry picked from commit 67846a5519)	2017-08-01 17:28:24 -07:00
Subru Krishnan	80e1904000	YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru). ... (cherry picked from commit 4846069061)	2017-08-01 17:28:24 -07:00
Carlo Curino	5272af8c7e	YARN-3663. Federation State and Policy Store (DBMS implementation). (Giovanni Matteo Fumarola via curino). ... (cherry picked from commit be99c1fe2e)	2017-08-01 17:28:24 -07:00
Carlo Curino	c4bbdfeb66	YARN-6203: Occasional test failure in TestWeightedRandomRouterPolicy (curino) ... (cherry picked from commit 121d55bd29)	2017-08-01 17:28:23 -07:00
Subru Krishnan	e1da8f0667	YARN-5602. Utils for Federation State and Policy Store. (Giovanni Matteo Fumarola via Subru). ... (cherry picked from commit 326a2e6bde)	2017-08-01 17:28:23 -07:00
Carlo Curino	1c64e1709b	YARN-6370. Properly handle rack requests for non-active subclusters in LocalityMulticastAMRMProxyPolicy. (Contributed by Botong Huang via curino). ... (cherry picked from commit ce419881c3)	2017-08-01 17:28:23 -07:00
Subru Krishnan	86b2bec56e	YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru) ... (cherry picked from commit 51aeb2ce0c)	2017-08-01 17:28:23 -07:00
Carlo Curino	8623644f45	YARN-6190. Validation and synchronization fixes in LocalityMulticastAMRMProxyPolicy. (Botong Huang via curino) ... (cherry picked from commit 5c486961cd)	2017-08-01 17:28:22 -07:00
Subru Krishnan	193ec456d6	YARN-5410. Bootstrap Router server module. (Giovanni Matteo Fumarola via Subru). ... (cherry picked from commit 04f111394b)	2017-08-01 17:28:22 -07:00
Subru Krishnan	98b45b0ed3	YARN-6093. Minor bugs with AMRMtoken renewal and state store availability when using FederationRMFailoverProxyProvider during RM failover. (Botong Huang via Subru). ... (cherry picked from commit 66500f4fa6)	2017-08-01 17:28:22 -07:00
Subru Krishnan	91803305e5	YARN-5872. Add AlwayReject policies for router and amrmproxy. (Carlo Curino via Subru). ... (cherry picked from commit db26bade05)	2017-08-01 17:28:22 -07:00
Subru Krishnan	78032e51d9	YARN-5905. Update the RM webapp host that is reported as part of Federation membership to current primary RM's IP. ... (cherry picked from commit e0f0e7afed)	2017-08-01 17:28:22 -07:00
Subru Krishnan	4128c9522d	YARN-5676. Add a HashBasedRouterPolicy, and small policies and test refactoring. (Carlo Curino via Subru). ... (cherry picked from commit 575137f41c)	2017-08-01 17:28:21 -07:00
Subru Krishnan	083a05bc14	YARN-5634. Simplify initialization/use of RouterPolicy via a RouterPolicyFacade. (Carlo Curino via Subru). ... (cherry picked from commit d7672ce2bd)	2017-08-01 17:28:21 -07:00
Subru Krishnan	93d9fdeca6	YARN-5391. PolicyManager to tie together Router/AMRM Federation policies. (Carlo Curino via Subru). ... (cherry picked from commit 20893682ec)	2017-08-01 17:28:21 -07:00
Subru Krishnan	1dadd0b45a	YARN-5325. Stateless ARMRMProxy policies implementation. (Carlo Curino via Subru). ... (cherry picked from commit 11c5336522)	2017-08-01 17:28:21 -07:00
Subru Krishnan	0662996b6a	YARN-5324. Stateless Federation router policies implementation. (Carlo Curino via Subru). ... (cherry picked from commit 1298127bda)	2017-08-01 17:28:21 -07:00
Subru Krishnan	81472778d7	YARN-5323. Policies APIs for Federation Router and AMRMProxy policies. (Carlo Curino via Subru). ... (cherry picked from commit f8208fe0b5)	2017-08-01 17:28:20 -07:00
Jian He	2797507d51	YARN-5601. Make the RM epoch base value configurable. Contributed by Subru Krishnan ... (cherry picked from commit 9ca2aba9cc)	2017-08-01 17:28:20 -07:00
Subru Krishnan	1882bc10fb	YARN-5612. Return SubClusterId in FederationStateStoreFacade#addApplicationHomeSubCluster for Router Failover. (Giovanni Matteo Fumarola via Subru). ... (cherry picked from commit ac1ba2a304)	2017-08-01 17:28:20 -07:00
Jian He	94a24567d6	YARN-3671. Integrate Federation services with ResourceManager. Contributed by Subru Krishnan ... (cherry picked from commit 8573c286e2)	2017-08-01 17:28:20 -07:00
Jian He	bdfad4523f	YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan ... (cherry picked from commit 3307564a5f)	2017-08-01 17:28:20 -07:00
Subru Krishnan	cfafd173bd	YARN-5467. InputValidator for the FederationStateStore internal APIs. (Giovanni Matteo Fumarola via Subru) ... (cherry picked from commit bd44182e70)	2017-08-01 17:28:19 -07:00
Jian He	5c84382397	YARN-3672. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan ... (cherry picked from commit 5e976cd2b9)	2017-08-01 17:28:19 -07:00
Subru Krishnan	52558df620	YARN-5519. Add SubClusterId in AddApplicationHomeSubClusterResponse for Router Failover. (Ellen Hui via Subru) ... (cherry picked from commit e4f928cf9c)	2017-08-01 17:28:19 -07:00
Subru Krishnan	0733088c29	YARN-5407. In-memory based implementation of the FederationApplicationStateStore/FederationPolicyStateStore. (Ellen Hui via Subru) ... (cherry picked from commit b747d59f41)	2017-08-01 17:28:19 -07:00
Subru Krishnan	e0c3a44396	YARN-5408. Compose Federation membership/application/policy APIs into an uber FederationStateStore API. (Ellen Hui via Subru). ... (cherry picked from commit 268389d021)	2017-08-01 17:28:19 -07:00
Subru Krishnan	f1a508bdef	YARN-3664. Federation PolicyStore internal APIs ... (cherry picked from commit 2f377263a1)	2017-08-01 17:28:18 -07:00
Subru Krishnan	047a16bf15	YARN-5307. Federation Application State Store internal APIs ... (cherry picked from commit 1af5292042)	2017-08-01 17:28:18 -07:00
Subru Krishnan	d19b677301	YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui. ... (cherry picked from commit d3dc461a93)	2017-08-01 17:28:18 -07:00
Subru Krishnan	dcf8c52d80	YARN-5406. In-memory based implementation of the FederationMembershipStateStore. Contributed by Ellen Hui. ... (cherry picked from commit 20d1d2be91)	2017-08-01 17:28:18 -07:00
Subru Krishnan	3e22896f86	YARN-3662. Federation Membership State Store internal APIs. ... (cherry picked from commit 5424d0899c)	2017-08-01 17:28:17 -07:00
Jian He	91f120f743	YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G	2017-08-01 09:56:33 -07:00
Akira Ajisaka	b38a1eea8e	HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor.	2017-08-01 15:15:43 +09:00
Akira Ajisaka	a4aa1cb405	Revert "YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang." ... This reverts commit 1a78c0ff01.	2017-08-01 12:15:03 +09:00
Akira Ajisaka	1a78c0ff01	YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang.	2017-08-01 10:53:32 +09:00
Akira Ajisaka	f8bed5e9a7	YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout.	2017-07-31 11:09:13 +09:00
Arun Suresh	890e14c02a	YARN-6870. Fix floating point inaccuracies in resource availability check in AllocationBasedResourceUtilizationTracker. (Brook Zhou via asuresh)	2017-07-28 16:32:43 -07:00
Subru Krishnan	bcde66bed1	YARN-6897. Refactoring RMWebServices by moving some util methods to RMWebAppUtil. (Giovanni Matteo Fumarola via Subru).	2017-07-28 15:44:36 -07:00
Yufei Gu	9ea01fd956	YARN-6802. Add max AM resource and AM resource usage to leaf queue view in FairScheduler WebUI. (YunFan Zhou via Yufei Gu)	2017-07-28 09:52:53 -07:00
Yufei Gu	9902be72cb	YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu)	2017-07-27 23:19:39 -07:00
Akira Ajisaka	38c6fa5c7a	HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier.	2017-07-28 09:10:52 +09:00
Varun Saxena	11ece0bda1	Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)	2017-07-28 00:10:45 +05:30
Varun Saxena	f66fd11e51	YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)	2017-07-27 23:14:50 +05:30
Daniel Templeton	f81a4efb8c	YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton)	2017-07-25 13:00:31 -07:00
Jian He	ac9489f7fc	YARN-6804. Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi	2017-07-25 09:57:14 -07:00
Akira Ajisaka	cca51e916b	HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor.	2017-07-25 15:24:56 +09:00
Akira Ajisaka	218b1b33ff	YARN-6150. TestContainerManagerSecurity tests for Yarn Server are flakey. Contributed by Daniel Sturman and Ray Chiang.	2017-07-25 15:12:08 +09:00
Daniel Templeton	10583625c1	YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue. ... (Contributed by Yufei Gu via Daniel Templeton)	2017-07-24 13:44:00 -07:00
Daniel Templeton	bb30bd3771	YARN-6779. DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting ... (Contributed by Yeliang Cang via Daniel Templeton)	2017-07-24 12:13:50 -07:00
Sunil G	e315328428	YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S.	2017-07-24 11:39:03 +05:30
Brahma Reddy Battula	2054324d47	Revert "YARN-6804. [YARN core changes] Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi" ... This reverts commit 4a771d9010.	2017-07-23 12:56:18 +08:00
Jian He	4a771d9010	YARN-6804. [YARN core changes] Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi	2017-07-21 11:31:33 -07:00
Jason Lowe	c8df3668ec	YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling	2017-07-20 11:03:04 -05:00
Arun Suresh	077fcf6a96	YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh)	2017-07-19 12:26:40 -07:00
Rohith Sharma K S	845c4e52bd	YARN-6819. Application report fails if app rejected due to nodesize. Contributed by Bibin A Chundatt.	2017-07-19 11:10:52 +05:30
Yufei Gu	daaf530fce	YARN-6778. In ResourceWeights, weights and setWeights() should be final. (Daniel Templeton via Yufei Gu)	2017-07-18 16:38:07 -07:00
Ray Chiang	f5f14a2ad6	YARN-6798. Fix NM startup failure with old state store due to version mismatch. (Botong Huang via rchiang)	2017-07-18 12:35:08 -07:00
Sunil G	0b7afc060c	Addendum patch for YARN-5731	2017-07-18 11:49:09 +05:30
Arun Suresh	5b007921cd	YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh)	2017-07-17 14:11:14 -07:00
Yufei Gu	9e0cde1469	YARN-6625. yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment. (Yufei Gu)	2017-07-14 14:10:45 -07:00
Jason Lowe	a5ae5ac50e	YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt	2017-07-14 14:56:00 -05:00
bibinchundatt	75c0220b44	YARN-6759. Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure. Contributed by Naganarasimha G R	2017-07-14 13:53:39 +05:30
Yufei Gu	4a574e9a84	YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu)	2017-07-13 23:10:10 -07:00
Sunil G	228ddaa31d	YARN-6792. Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo. Contributed by Giovanni Matteo Fumarola.	2017-07-14 08:07:05 +05:30
Jason Lowe	ebc048cc05	YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe	2017-07-13 17:44:47 -05:00
Jason Lowe	0ffca5d347	Revert "YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe" ... This reverts commit f76f5c0919.	2017-07-13 17:42:38 -05:00
Jason Lowe	f76f5c0919	YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe	2017-07-13 17:38:17 -05:00
Jason Lowe	5f1ee72b0e	YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles	2017-07-13 17:27:40 -05:00
Wangda Tan	945c0958bb	YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda) ... Change-Id: I84ccd54200ccbaae23018ef320028e42b4c3509a	2017-07-13 10:30:15 -07:00
Sunil G	cf0d0844d6	YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. Contributed by Wangda Tan.	2017-07-13 16:48:29 +05:30
Xuan	9845bea59c	YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong)	2017-07-12 15:40:45 -07:00
Sunil G	34f113df5c	YARN-6714. IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by Tao Yang.	2017-07-11 14:52:44 +05:30
Arun Suresh	5496a34c0c	YARN-6776. Refactor ApplicaitonMasterService to move actual processing logic to a separate class. (asuresh)	2017-07-10 14:34:58 -07:00
Steve Loughran	09653ea098	HADOOP-14634. Remove jline from main Hadoop pom.xml. ... Contributed by Ray Chiang.	2017-07-10 11:53:13 +01:00
Yufei Gu	3de47ab5ea	YARN-6764. Simplify the logic in FairScheduler#attemptScheduling. Contributed by Yufei Gu.	2017-07-09 16:09:12 -07:00
Naganarasimha	626d730bfc	YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt.	2017-07-09 17:26:09 +05:30
Daniel Templeton	9d27880442	YARN-6410. FSContext.scheduler should be final (Contributed by Yeliang Cang via Daniel Templeton)	2017-07-09 19:34:35 +09:00
Daniel Templeton	c5a0c38d8e	YARN-6746. SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code ... (Contributed by Deepti Sawhney via Daniel Templeton)	2017-07-09 19:27:32 +09:00
Daniel Templeton	0615985886	YARN-6752. Display reserved resources in web UI per application ... (Contributed by Abdullah Yousufi via Daniel Templeton)	2017-07-09 19:02:31 +09:00
Jason Lowe	7576a688ea	YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt	2017-07-06 09:40:09 -05:00
Naganarasimha	fa1aaee87b	YARN-6681. Eliminate double-copy of child queues in canAssignToThisQueue. Contributed by Daryn Sharp.	2017-07-01 12:29:39 +05:30
Andrew Wang	af2773f609	Updating version for 3.0.0-beta1 development	2017-06-29 17:57:40 -07:00
Daniel Templeton	ec97519779	YARN-6751. Display reserved resources in web UI per queue ... (Contributed by Abdullah Yousufi via Daniel Templeton)	2017-06-29 16:53:50 -07:00
Sunil G	c1edca101c	YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin.	2017-06-28 15:40:58 -07:00
Jason Lowe	63ce1593c5	YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper instances. Contributed by Zoltan Haindrich	2017-06-27 17:12:42 -05:00
Yufei Gu	0b77262890	YARN-5876. TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk. (Robert Kanter via Yufei Gu)	2017-06-23 13:26:03 -07:00
Sunil G	ca13b224b2	YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne.	2017-06-22 23:50:57 -07:00
Naganarasimha	740204b292	YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt.	2017-06-23 07:52:41 +05:30
Arun Suresh	49aa60e50d	YARN-6127. Add support for work preserving NM restart when AMRMProxy is enabled. (Botong Huang via asuresh).	2017-06-22 11:27:13 -07:00
Akira Ajisaka	099dfe92a9	YARN-6713. Fix dead link in the Javadoc of FairSchedulerEventLog.java. Contributed by Weiwei Yang.	2017-06-20 17:20:27 +09:00
Naganarasimha	ee89ac84e6	YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp	2017-06-19 22:42:45 +05:30
Naganarasimha	e9c2aa1bc3	YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R.	2017-06-19 21:31:40 +05:30
bibinchundatt	d64c842743	YARN-6703. RM startup failure with old state store due to version mismatch. Contributed by Varun Saxena	2017-06-12 18:07:43 +05:30
Eric Payne	5578af8603	YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G.	2017-06-09 16:16:34 -05:00
Naganarasimha	99634d10ed	YARN-6682. Improve performance of AssignmentInformation datastructures. Contributed by Daryn Sharp.	2017-06-09 11:10:13 +05:30
Daniel Templeton	a062374c39	YARN-6679. Reduce Resource instance overhead via non-PBImpl ... (Contributed by Daryn Sharp via Daniel Templeton)	2017-06-08 10:08:08 -07:00
Carlo Curino	547cb53e8a	YARN-6634. [API] Refactor ResourceManager WebServices to make API explicit. (Giovanni Matteo Fumarola via curino)	2017-06-07 13:41:06 -07:00
Varun Saxena	0b7bff706e	YARN-6604. Allow metric TTL for Application table to be specified through cmd (Haibo Chen via Varun Saxena)	2017-06-07 21:51:07 +05:30
Jian He	7311015ace	YARN-6683. Invalid event: COLLECTOR_UPDATE at KILLED. Contributed by Rohith Sharma K S	2017-06-05 13:16:57 -07:00
Daniel Templeton	73ecb19312	YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application ... (Contributed by Akira Ajisaka via Daniel Templeton)	2017-06-02 08:50:19 -07:00
vrushali	d95c82cb79	YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C)	2017-06-01 18:30:23 -07:00
Daniel Templeton	d5b71e4175	YARN-6246. Identifying starved apps does not need the scheduler writelock ... (Contributed by Karthik Kambatla via Daniel Templeton)	2017-05-31 15:48:04 -07:00
Nathan Roberts	4369690ce6	YARN-6649. RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception. Contributed by Jon Eagles.	2017-05-31 11:32:32 -05:00
Jonathan Eagles	cbfed0e82f	YARN-6497. Method length of ResourceManager#serviceInit() is too long (Gergely Novák via jeagles)	2017-05-31 10:18:42 -05:00
Varun Vasudev	547f18cb96	YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf.	2017-05-31 16:15:35 +05:30
Andrew Wang	16ad896d5c	Update maven version for 3.0.0-alpha4 development	2017-05-26 14:09:44 -07:00
Daniel Templeton	d81372dfad	YARN-6646. Modifier 'static' is redundant for inner enums ... (Contributed by ZhangBing Lin via Daniel Templeton)	2017-05-26 12:05:48 -07:00
Jason Lowe	aea42930bb	YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla	2017-05-26 09:37:56 -05:00
Haibo Chen	47474fffac	YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen)	2017-05-25 21:15:27 -07:00
Yufei Gu	87590090c8	YARN-6582. FSAppAttempt demand can be updated atomically in updateDemand(). (Karthik Kambatla via Yufei Gu)	2017-05-25 14:22:13 -07:00
Jason Lowe	3fd6a2da4e	YARN-6643. TestRMFailover fails rarely due to port conflict. Contributed by Robert Kanter	2017-05-25 16:07:52 -05:00
Varun Vasudev	1a56a3db59	Addendum patch to fix Docker sanitization.	2017-05-25 14:53:57 +05:30
Akira Ajisaka	bc28da65fb	YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan.	2017-05-25 17:06:26 +09:00
Jason Lowe	8bf1949c0e	YARN-6615. AmIpFilter drops query parameters on redirect. Contributed by Wilfred Spiegelenburg	2017-05-24 11:22:42 -05:00
Wangda Tan	8e0f83e49a	YARN-6493. Print requested node partition in assignContainer logs. (Jonathan Hung via wangda) ... Change-Id: Ib20326daac10acad363befb0ae0998441fa5b32d	2017-05-22 14:28:55 -07:00
Wangda Tan	c583ab02c7	YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda) ... Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c	2017-05-22 14:26:13 -07:00
Sunil G	b6f66b0da1	YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.	2017-05-22 14:10:06 +05:30
Jian He	e135c0b207	YARN-6627. Use deployed webapp folder to launch new YARN UI. Contributed by Sunil G	2017-05-19 20:14:03 -07:00
Robert Kanter	9855225a79	YARN-6602. Impersonation does not work if standby RM is contacted first (rkanter)	2017-05-19 15:43:22 -07:00
Yufei Gu	66bba8c024	YARN-6249. TestFairSchedulerPreemption fails inconsistently. (Tao Jie via Yufei Gu)	2017-05-19 14:53:27 -07:00
Eric Payne	19482e0d09	YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe.	2017-05-19 11:24:54 -05:00
Varun Vasudev	b46cd315f7	Sanitize arguments before launching Docker containers.	2017-05-18 10:37:16 +05:30
Varun Vasudev	e120ee865a	Validate docker image name before launching container.	2017-05-18 10:29:34 +05:30
Chris Douglas	b23fcc86c6	YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin	2017-05-17 20:40:06 -07:00
Rohith Sharma K S	d87a63a901	HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe.	2017-05-17 08:27:45 +05:30
Robert Kanter	18c494a00c	YARN-6447. Provide container sandbox policies for groups (gphillips via rkanter)	2017-05-16 18:02:39 -07:00
Jian He	8236130b2c	YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh	2017-05-16 10:48:46 -07:00
Jason Lowe	489f85933c	YARN-6603. NPE in RMAppsBlock. Contributed by Jason Lowe	2017-05-16 09:26:44 -05:00
Jonathan Eagles	c48f2976a3	YARN-6598. History server getApplicationReport NPE when fetching report for pre-2.8 job (Jason Lowe via jeagles)	2017-05-15 10:32:01 -05:00
Carlo Curino	6600abbb5c	YARN-6587. Refactor of ResourceManager#startWebApp in a Util class. (Giovanni Matteo Fumarola via curino).	2017-05-12 17:48:55 -07:00
Haibo Chen	6c35001b9f	YARN-6580. Incorrect logger for FairSharePolicy. (Vrushali C via Haibo Chen)	2017-05-12 13:05:40 -07:00
Daniel Templeton	90cb5b4635	YARN-6380. FSAppAttempt keeps redundant copy of the queue	2017-05-11 13:47:02 -07:00
Min Shen	2ada100da7	YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen.	2017-05-11 13:25:31 -07:00
Brahma Reddy Battula	4aae2d40a3	HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang	2017-05-11 23:33:45 +05:30
Carlo Curino	5cb6e3e082	YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.	2017-05-10 14:49:25 -07:00
Daniel Templeton	e7654c4a1f	YARN-6571. Fix JavaDoc issues in SchedulingPolicy ... (Contributed by Weiwei Yang via Daniel Templeton)	2017-05-10 14:34:13 -07:00
Daniel Templeton	74a61438ca	YARN-6475. Fix some long function checkstyle issues ... (Contributed by Soumabrata Chakraborty via Daniel Templeton)	2017-05-10 10:46:50 -07:00
Jason Lowe	6099deebcb	YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger	2017-05-10 08:47:48 -05:00
Haibo Chen	ab2bb93a2a	YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen)	2017-05-09 21:37:30 -07:00
Haibo Chen	461ee44d28	YARN-6435. [ATSv2] Can't retrieve more than 1000 versions of metrics in time series. (Rohith Sharma K S via Haibo Chen)	2017-05-09 21:12:57 -07:00
Karthik Kambatla	166be0ee95	YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha)	2017-05-09 14:44:16 -07:00
Daniel Templeton	a2f680493f	YARN-5301. NM mount cpu cgroups failed on some systems ... (Contributed by Miklos Szegedi via Daniel Templeton)	2017-05-09 12:05:46 -07:00
Vrushali Channapattan	7dd258d8f4	YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C)	2017-05-09 11:18:12 -07:00
Andrew Wang	543aac9f28	HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2.	2017-05-09 09:22:53 -07:00
Subru Krishnan	749e5c09b9	YARN-6281. Cleanup when AMRMProxy fails to initialize a new interceptor chain. (Botong Huang via Subru) ... (cherry picked from commit 57a9afbd45b7ef8e6021cc58f96bd0074bf1389d)	2017-05-08 16:55:47 -07:00
Subru Krishnan	cd9ff27ffc	YARN-6234. Support multiple attempts on the node when AMRMProxy is enabled. (Giovanni Matteo Fumarola via Subru).	2017-05-08 16:41:30 -07:00
Jason Lowe	424887ecb7	YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R	2017-05-08 17:14:37 -05:00
Subru Krishnan	d6eed5acca	YARN-6565. Fix memory leak and finish app trigger in AMRMProxy. (Botong Huang via Subru).	2017-05-05 16:27:49 -07:00
Chris Douglas	e4f34ecb04	HADOOP-14390. Correct spelling of 'succeed' and variants. Contributed by Dongtao Zhang	2017-05-05 12:10:50 -07:00
Vrushali Channapattan	54e2b9e876	YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C)	2017-05-04 15:25:56 -07:00
Sidharta S	fd5cb2c946	YARN-6374. Improve test coverage and add utility classes for common Docker operations. Contributed by Shane Kumpf	2017-05-03 14:49:30 -07:00
Daniel Templeton	9f0aea0ee2	YARN-6481. Yarn top shows negative container number in FS ... (Contributed by Tao Jie via Daniel Templeton)	2017-05-02 13:06:47 -07:00
Subru Krishnan	6bf42e48ef	YARN-5331. Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru).	2017-05-01 18:48:36 -07:00
Subru Krishnan	e514fc432a	YARN-6542. Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent.	2017-05-01 18:10:07 -07:00
Subru Krishnan	a3a615eeab	YARN-4359. Update LowCost agents logic to take advantage of YARN-4358. (Jonathan Yaniv and Ishai Menache via Subru).	2017-05-01 16:01:07 -07:00
Naganarasimha	30fc580196	YARN-6519. Fix warnings from Spotbugs in hadoop-yarn-server-resourcemanager. Contributed by Weiwei Yang.	2017-05-01 20:15:27 +05:30
Naganarasimha	0f1af3178e	YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang.	2017-05-01 16:49:30 +05:30
Daniel Templeton	2e52789edf	YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton)	2017-04-28 13:30:28 -07:00
Robert Kanter	68e45f554b	YARN-6472. Improve Java sandbox regex (gphillips via rkanter)	2017-04-28 11:01:50 -07:00
Robert Kanter	371b6467dc	YARN-5894. fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 (haibochen via rkanter)	2017-04-27 17:49:38 -07:00
Varun Saxena	793bbf216d	YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena)	2017-04-27 15:01:58 +05:30
Akira Ajisaka	8a99eba96d	Revert "YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor." ... This reverts commit 34ab8e73d4.	2017-04-26 15:34:23 +09:00
Wangda Tan	2ba21d6376	YARN-6392. Add submit time to Application Summary log. (Zhihai Xu via wangda)	2017-04-24 13:17:35 -07:00
Haibo Chen	8ac50e1322	YARN-6500. Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler. (Miklos Szegedi via Haibo Chen)	2017-04-24 11:37:52 -07:00
Eric Payne	52adf71914	YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe.	2017-04-24 12:03:28 -05:00
Daniel Templeton	46940d92e2	YARN-6302. Fail the node if Linux Container Executor is not configured properly ... (Contributed by Miklos Szegedi via Daniel Templeton)	2017-04-19 12:23:49 -07:00
Daniel Templeton	dd43b895c2	YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded ... (Contributed by Yufei Gu via Daniel Templeton)	2017-04-19 11:44:55 -07:00
Daniel Templeton	41ac190dba	YARN-6438. Code can be improved in ContainersMonitorImpl.java ... (Contributed by Miklos Szegedi via Daniel Templeton)	2017-04-19 10:53:08 -07:00
Sunil G	9ff25408bc	YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu.	2017-04-19 20:39:09 +05:30
Chris Douglas	af8e9842d2	YARN-6451. Add RM monitor validating metrics invariants. Contributed by Carlo Curino	2017-04-18 10:29:44 -07:00
Junping Du	6ed9d36224	YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S.	2017-04-17 15:51:10 -07:00
Jason Lowe	4168805502	YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger	2017-04-14 17:15:48 -05:00
Karthik Kambatla	8a1d7480f7	YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha)	2017-04-14 15:07:14 -07:00
Eric Payne	a41f8dd58e	YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger	2017-04-14 10:53:09 -05:00
Karthik Kambatla	c3375175d6	YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha)	2017-04-12 14:21:20 -07:00
Eric Payne	a16ab2be91	YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe.	2017-04-12 10:20:08 -05:00
Wangda Tan	4d4ad0ebb7	YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda)	2017-04-11 14:56:18 -07:00
Naganarasimha	7d873c465b	YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang.	2017-04-11 22:42:08 +05:30
Jason Lowe	0e065f2ede	YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu	2017-04-11 08:44:18 -05:00
Konstantinos Karanasos	7999318af1	YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos)	2017-04-10 15:34:44 -07:00
Daniel Templeton	e9ac61cc0e	YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton)	2017-04-10 14:56:42 -07:00
Karthik Kambatla	2aa8967809	YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha)	2017-04-07 17:31:59 -07:00
Robert Kanter	63f7322522	YARN-6368. Decommissioning an NM results in a -1 exit code (miklos.szegedi@cloudera.com via rkanter)	2017-04-07 14:28:07 -07:00
Ming Ma	0116c3c957	YARN-5797. Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches. (Chris Trezzo via mingma)	2017-04-06 16:54:43 -07:00
Jason Lowe	1b081ca27e	YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka	2017-04-06 16:24:36 -05:00
Rohith Sharma K S	1a9439e299	YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena.	2017-04-06 10:15:22 +05:30
Daniel Templeton	a2c57bb70d	YARN-6381. FSAppAttempt has several variables that should be final ... (Contributed by Ameet Zaveri via Daniel Templeton)	2017-04-05 16:07:24 -07:00
Andrew Wang	539ef5aa2e	HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu.	2017-04-05 16:04:09 -07:00
Karthik Kambatla	b4c4f36594	YARN-6448. Continuous scheduling thread crashes while sorting nodes. (Yufei Gu via kasha)	2017-04-05 15:43:14 -07:00
Varun Saxena	7d963c477a	YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena)	2017-04-06 01:20:41 +05:30
Wangda Tan	87e2ef8c98	YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda)	2017-04-05 11:28:30 -07:00
Jason Lowe	e8071aa249	YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang	2017-04-05 12:30:45 -05:00
Akira Ajisaka	34ab8e73d4	YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor.	2017-04-05 17:33:07 +09:00
Varun Saxena	19b89c4c7b	YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena)	2017-04-05 09:38:35 +05:30
Ming Ma	2d5c09b848	YARN-6004. Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines. (Chris Trezzo via mingma)	2017-04-04 17:56:21 -07:00
Karthik Kambatla	9cc04b4708	YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha)	2017-04-04 17:08:33 -07:00
Wangda Tan	3fdae0a2b6	YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda)	2017-04-04 14:39:01 -07:00
Chris Douglas	6eba79232f	HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang	2017-04-03 20:13:14 -07:00
Varun Saxena	a4b5aa8493	YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena)	2017-04-02 04:54:12 +05:30
Varun Saxena	b8838578ce	YARN-6414. ATSv2 HBase related tests fail due to guava version upgrade (Haibo Chen via Varun Saxena)	2017-04-02 04:37:34 +05:30
Eric Payne	318bfb01bc	YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe	2017-03-31 12:30:35 -05:00
Jason Lowe	4d1fac5df2	YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu	2017-03-31 10:05:34 -05:00
Varun Saxena	b58777a9c9	YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena)	2017-03-31 02:17:20 +05:30
Daniel Templeton	640ba1d23f	YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled	2017-03-29 12:38:11 -07:00
Akira Ajisaka	82fb9ce8df	YARN-6329. Remove unnecessary TODO comment from AppLogAggregatorImpl.java. Contributed by victor bertschinger.	2017-03-29 10:36:24 +09:00
Varun Saxena	063b513b1c	YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena)	2017-03-29 03:48:03 +05:30
Varun Saxena	01aca54a22	YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena)	2017-03-29 01:53:20 +05:30
Jason Lowe	fdf8f8ebca	YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter	2017-03-28 09:33:26 -05:00
Robert Kanter	9bae6720cb	YARN-6050. AMs can't be scheduled on racks or nodes (rkanter)	2017-03-27 17:23:13 -07:00
Wangda Tan	cd014d57aa	YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda)	2017-03-27 13:29:09 -07:00
Akira Ajisaka	b63ae854f9	YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki.	2017-03-27 11:23:41 +09:00
Sunil G	cffea251be	YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki.	2017-03-25 22:26:19 +05:30
Tsuyoshi Ozawa	84ddedc0b2	HADOOP-10101. Update guava dependency to the latest version. (ozawa)	2017-03-25 10:06:52 +09:00
Ravi Prakash	d4f73e7e27	HDFS-11574. Spelling mistakes in the Java source. Contributed by Hu Xiaodong.	2017-03-24 09:38:17 -07:00
Varun Saxena	f462e1ff68	YARN-5934. Fix TestTimelineWebServices.testPrimaryFilterNumericString (Akira Ajisaka via Varun Saxena)	2017-03-22 04:11:53 +05:30
Daniel Templeton	0a05c5c598	YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable ... (Contributed by Laura Adams via Daniel Templeton)	2017-03-21 14:15:40 -07:00
Varun Saxena	35034653d0	YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena)	2017-03-20 23:29:09 +05:30
Jason Lowe	4a8e304502	YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi	2017-03-17 14:06:35 -05:00
Jason Lowe	7114baddb6	YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee	2017-03-16 09:30:10 -05:00
Sunil G	2d0e24eb01	YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202	2017-03-15 14:48:05 +05:30
Ray Chiang	4c66a8d19b	YARN-6331. Fix flakiness in TestFairScheduler#testDumpState. (Yufei Gu via rchiang)	2017-03-14 15:09:47 -07:00
Naganarasimha	0a3aa40fe7	YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung.	2017-03-15 01:22:25 +05:30
Junping Du	34424e98a6	YARN-6314. Potential infinite redirection on YARN log redirection web service. Contributed by Xuan Gong. ... (cherry picked from commit 5a9dda796f0e73060ada794ad5752cc6a237ab2e)	2017-03-14 02:58:07 -07:00
Ray Chiang	4db9cc70d0	YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang)	2017-03-10 16:13:31 -08:00