0cb1b178ec
YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan.
...
(cherry picked from commit e0c9653166
2020-07-30 16:39:56 +00:00
d3c7f7cdbd
YARN-10343. Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. Contributed by Eric Payne
...
(cherry picked from commit 3eaf62726f
2020-07-28 13:43:44 -07:00
7ec692aa83
YARN-4771. Some containers can be skipped during log aggregation after NM
...
restart. Contributed by Jason Lowe and Jim Brennan.
(cherry picked from commit ac5f21dbef
2020-07-24 22:46:57 +00:00
e3b8d4eb05
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-22 18:21:14 +05:30
9e7266df6c
HADOOP-17099. Replace Guava Predicate with Java8+ Predicate
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 1f71c4ae71
2020-07-15 11:40:13 -05:00
5969922305
HADOOP-17101. Replace Guava Function with Java8+ Function
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 98fcffe93f
2020-07-15 09:57:36 -05:00
41bcef9486
YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by
...
Jim Brennan
(cherry picked from commit 48f90115b5
2020-07-13 23:12:18 +00:00
7044a007b3
YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan)
...
(cherry picked from commit 0427100b75
2020-07-13 19:02:40 +00:00
7b175739a9
YARN-10341. Yarn Service Container Completed event doesn't get processed. Contributed by Bilwa S T.
...
(cherry picked from commit dfe60392c9
2020-07-09 12:36:21 +05:30
0aa2d7d506
YARN-10344. Sync netty versions in hadoop-yarn-csi. ( #2126 )
...
(cherry picked from commit 10d218934c
2020-07-09 15:07:44 +09:00
20df70a895
HADOOP-17111. Replace Guava Optional with Java8+ Optional. Contributed by Ahmed Hussein.
...
(cherry picked from commit 639acb6d89
2020-07-06 16:09:37 +09:00
439c51425e
YARN-10330. Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule. Contributed by Peter Bacsko
2020-07-01 17:42:45 +02:00
cfb2084cba
YARN-10325. Document max-parallel-apps for Capacity Scheduler. Contributed by Peter Bacsko
2020-07-01 13:40:36 +02:00
d88a6eebf2
YARN-10318. ApplicationHistory Web UI incorrect column indexing. Contributed by Andras Gyori
2020-07-01 13:31:35 +02:00
8b482744e9
YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy. Contributed by Szilard Nemeth
2020-06-30 11:32:59 +02:00
aa283fc2c2
YARN-10331. Upgrade node.js to 10.21.0. ( #2106 )
...
(cherry picked from commit cd188ea9f0
2020-06-30 16:54:59 +09:00
d7696453a0
YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan).
...
(cherry picked from commit 74fc13cf91
2020-06-29 18:59:52 +00:00
fa41e38450
YARN-10279. Avoid unnecessary QueueMappingEntity creations. Contributed by Marton Hudaky
...
(cherry picked from commit 6a8fd73b27
2020-06-25 17:28:48 +02:00
480919e42d
YARN-10316. FS-CS converter: convert maxAppsDefault, maxRunningApps settings. Contributed by Peter Bacsko
2020-06-23 16:25:33 +02:00
8f1b70e367
YARN-9930. Support max running app logic for CapacityScheduler. Contributed by Peter Bacsko
2020-06-22 12:00:06 +02:00
ec913398a9
YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak
2020-06-17 14:36:08 +02:00
8be302a3b8
YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak
2020-06-16 18:25:47 +02:00
52efe48d79
YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke
2020-06-16 18:01:39 +02:00
534b15caf9
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:20:10 +05:30
fcd7ce53b5
YARN-10312. Add support for yarn logs -logFile to retain backward compatibility.
...
Contributed by Jim Brennan.
(cherry picked from commit fed6fecd3a
2020-06-11 21:11:20 +00:00
e35f619841
YARN-10296. Make ContainerPBImpl#getId/setId synchronized. Contributed by Benjamin Teke
2020-06-10 18:00:21 +02:00
a7526ba9f7
YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger).
...
(cherry picked from commit 56247db302
2020-06-09 18:51:46 +00:00
ac307fe20d
YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal
2020-06-05 12:40:57 +02:00
a266e32d82
YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori
2020-06-05 09:50:43 +02:00
f65f64e8ae
YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak
2020-06-02 18:32:06 +02:00
07b8963aa3
YARN-10259. Fix reservation logic in Multi Node Placement.
...
Reviewed by Wangda Tan.
(cherry picked from commit 6ce295b787
2020-06-01 18:24:38 +05:30
7a323a45aa
YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R
...
(cherry picked from commit c30c23cb66
2020-05-29 09:25:08 -07:00
94358dbc3c
YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
...
(cherry picked from commit ac4540dd8e
2020-05-27 13:07:17 +02:00
2c60bcb1c2
Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak"
...
This reverts commit 9ce913a7bf
2020-05-27 13:07:01 +02:00
9ce913a7bf
FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
...
(cherry picked from commit ac4540dd8e
2020-05-22 14:48:53 +02:00
d7cf19d7c0
YARN-10228. Relax restriction of file path character in yarn.service.am.java.opts.
...
Contributed by Bilwa S T via eyang
2020-05-20 09:31:40 -07:00
7cef7109de
YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
2020-05-14 15:17:13 +05:30
74887ab2ad
YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
2020-05-14 00:37:09 +05:30
564d3211f2
YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T
...
(cherry picked from commit fff1d2c122
2020-05-12 10:49:10 -07:00
95e4ed18a6
YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2020-05-12 09:03:50 +02:00
7aade932c7
YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 92e3ebb401
2020-05-06 12:28:04 -05:00
1d8350055e
YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit f5081a9a5d
2020-05-04 10:24:01 -05:00
dfa7f160a5
Preparing for 3.3.1 development
2020-04-30 13:33:42 +09:00
8ffe1f313c
YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
...
(cherry picked from commit 410c605aec
2020-04-29 15:54:32 +02:00
75433a658d
YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
2020-04-29 09:36:37 +02:00
e45faae1d4
YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori
...
(cherry picked from commit ab36429559
2020-04-28 18:56:29 +02:00
62ee5970f8
YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
...
(cherry picked from commit 6ef01646ba
2020-04-28 05:45:37 +09:00
c0b7b38e22
YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph
...
(cherry picked from commit f91e21ac10
2020-04-27 08:26:46 +02:00
7a3f190d89
YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke
2020-04-24 17:09:35 +02:00
61ca459c74
YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
2020-04-24 11:28:52 +02:00
497c7a1680
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. ( #1939 )
...
(cherry picked from commit 9827ff2961
2020-04-24 10:27:39 +09:00
1340518cd8
YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja
2020-04-23 14:55:50 +02:00
49ae9b2137
YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T
2020-04-17 10:34:59 -07:00
4bd37f2283
YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja
2020-04-15 07:54:53 +02:00
f473473355
YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko
...
(cherry picked from commit 55fcbcb5c2
2020-04-15 07:27:47 +02:00
e1dd78143b
YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T
...
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
2020-04-09 11:06:49 -07:00
061a7a8a41
YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T.
...
(cherry picked from commit c04a484aed
2020-04-09 15:47:57 +02:00
94df6b513b
YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.
...
(cherry picked from commit 6fb29d0f17
2020-04-09 18:52:52 +05:30
e8520fd132
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 ( #1933 )
...
(cherry picked from commit e53d472bb0
2020-04-09 01:49:01 +09:00
618a9208f7
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja
...
(cherry picked from commit 2214005c0f
2020-04-08 13:02:51 +10:00
d293e120eb
YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja
...
(cherry picked from commit bffb43b00e
2020-04-07 17:04:01 +02:00
7abc6221a3
YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja
...
(cherry picked from commit 45362a9f4c
2020-04-07 16:01:47 +02:00
a2f4dcb470
YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao.
...
(cherry picked from commit c162648aff
2020-04-01 16:06:16 +09:00
50f7f6dfd1
YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover.
2020-03-26 08:14:30 -07:00
3d5ade1839
YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R
2020-03-26 13:23:43 +01:00
9a297ff31d
YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko
2020-03-26 12:35:10 +01:00
348685dcb9
YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke
2020-03-26 12:18:15 +01:00
6ce189c621
YARN-10200. Add number of containers to RMAppManager summary
2020-03-25 10:41:59 -07:00
2d294bd575
HADOOP-16938. Make non-HA proxy providers pluggable
2020-03-25 08:06:58 -07:00
cdb2107066
YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.
2020-03-25 16:50:19 +05:30
d312991ead
YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue
...
Contributed by Peter Bacsko. Reviewed by Manikandan R.
2020-03-23 18:01:14 +05:30
f2d3ac2a3f
YARN-10034. Remove Allocation Tags from released container from Decommission node
...
Contributed by Kyungwan Nam. Reviewed by Adam Antal.
2020-03-19 12:55:14 +05:30
1975479285
YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed
...
Hussein.
2020-03-16 22:24:02 +00:00
b9d825f178
YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.
2020-03-13 14:39:36 -07:00
d273f6a2b4
YARN-9538. Document scheduler/app activities and REST APIs. Contributed by Tao Yang.
2020-03-13 14:36:52 -07:00
c83644deac
YARN-10110. Adding RouterPolicyProvider for RM Federation.
...
Contributed by Bilwa S T.
2020-03-13 16:56:44 +05:30
5ead9c15ca
YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori
2020-03-12 12:29:03 +01:00
38d87883b6
YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko
2020-03-12 12:18:37 +01:00
0fd8bf5f6b
YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.
2020-03-12 15:03:12 +05:30
cf9cf83a43
YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori
2020-03-10 16:44:48 +01:00
61f4cf3055
YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke
2020-03-10 16:35:04 +01:00
9314ef947f
YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko
2020-03-10 16:07:46 +01:00
ede05b19d1
YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein)
2020-03-10 14:17:45 +00:00
44afe1154d
YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori
2020-03-09 16:08:24 +01:00
3859fa76d0
YARN-6924. Metrics for Federation AMRMProxy. Contributed by Young Chen
2020-03-07 09:34:42 +05:30
ea0444851d
YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke
2020-03-06 17:49:45 +01:00
004e955348
YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko
2020-03-05 09:56:42 +01:00
2649f8b327
YARN-10173. Make pid file generation timeout configurable in case of reacquired
...
container. Contributed by Adam Antal.
2020-03-04 23:31:57 +00:00
c0d084247c
YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko
2020-03-03 09:46:32 +01:00
e9eecedf69
YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton
2020-03-02 16:25:14 +01:00
a43510e21d
YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan.
2020-02-27 13:18:30 -08:00
b420ddeada
YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R.
2020-02-27 12:53:20 -08:00
10461e0193
YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton
2020-02-27 20:10:32 +01:00
7dfa37e8f0
YARN-10152. Fix findbugs warnings in hadoop-yarn-applications-mawo-core module ( #1852 )
2020-02-27 10:36:49 +09:00
3a9ccf7f6d
YARN-10141.Interceptor in FederationInterceptorREST doesnt update on RM switchover. Contributed by D M Murali Krishna Reddy.
2020-02-26 23:24:00 +05:30
b5df1da3a1
YARN-9593. Support Comma in the value of Scheduler Configuration Mutation API.
2020-02-26 15:04:29 +05:30
d608e94f92
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md ( #1856 )
2020-02-26 17:29:26 +09:00
d68616b045
YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal
2020-02-25 21:30:04 +01:00
dda00d3ff5
YARN-10074. Update netty to 4.1.42Final in yarn-csi. Contributed by Wei-Chiu Chuang. ( #1807 )
2020-02-25 13:47:52 +09:00
72fa9c7f1b
YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko
2020-02-24 21:54:07 +01:00
34cf63c7d0
YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko
2020-02-24 21:45:49 +01:00
181e6d033e
YARN-10119. Option to reset AM failure count for YARN Service
...
Contributed by kyungwan nam.
2020-02-20 20:34:06 +05:30
4af2556b48
YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal
2020-02-20 15:27:15 +01:00
3f1aad05f0
YARN-10147. FPGA plugin can't find the localized aocx file. Contributed by Peter Bacsko
2020-02-19 15:54:25 +01:00
6526f95bd2
YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph.
2020-02-19 11:17:22 +05:30
d6d7f8d8c5
YARN-8374. Upgrade objenesis to 2.6 ( #1798 )
2020-02-19 09:50:37 +09:00
20add89718
YARN-10136. [Router] : Application metrics are hardcode as N/A in UI. Contributed by Bilwa S T.
2020-02-14 16:50:29 +05:30
da99ac7e93
YARN-10137. UIv2 build is broken in trunk. Contributed by Adam Antal
2020-02-13 16:31:35 +01:00
f1b1b332f5
YARN-10029. Add option to UIv2 to get container logs from the new JHS API. Contributed by Adam Antal
2020-02-13 12:08:54 +01:00
fe7d67a8a2
YARN-9521. Handle FileSystem close in ApiServiceClient
...
Contributed by kyungwan nam. Reviewed by Eric Yang.
2020-02-13 09:39:13 +05:30
e637797211
YARN-10127. Remove setting App Ordering Policy to ParentQueue in FSQueueConverter
...
Contributed by Peter Bacsko.
2020-02-11 22:01:58 +05:30
28f730b317
YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph
2020-02-09 21:14:53 +05:30
3f0a7cd17a
YARN-9624. Use switch case for ProtoUtils#convertFromProtoFormat containerState. Contributed by Bilwa S T
2020-02-09 19:14:18 +05:30
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency ( #1635 ). Contributed by Vinayakumar B.
2020-02-07 14:51:24 +05:30
71b2c2ffe9
YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal
2020-02-06 12:25:06 +01:00
314e2f9d2e
YARN-10116. Expose diagnostics in RMAppManager summary
2020-02-04 17:44:05 -08:00
bf8686f43f
YARN-8982. [Router] Add locality policy. Contributed by Young Chen.
2020-01-30 16:59:36 -08:00
a7d72c523a
YARN-10099. FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues. Contributed by Peter Bacsko
2020-01-30 16:03:38 +01:00
a5ef08b619
YARN-9743. [JDK11] TestTimelineWebServices.testContextFactory fails. ( #1824 ) Contributed by Akira Ajisaka and Kinga Marton.
2020-01-30 14:10:31 +09:00
825db8fe2a
YARN-10107. Fix GpuResourcePlugin#getNMResourceInfo to honor Auto Discovery Enabled
...
Contributed by Szilard Nemeth.
2020-01-29 13:30:00 +05:30
e578e52aae
YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne.
2020-01-29 03:54:43 +00:00
5abd0148eb
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-28 10:06:37 -08:00
1ab9c692fa
YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change
...
Contributed by Kinga Marton.
2020-01-28 23:16:04 +05:30
94f060242e
YARN-10104. FS-CS converter: dry run should work without output defined. Contributed by Peter Bacsko
2020-01-28 15:36:04 +01:00
ca29768035
YARN-10085. FS-CS converter: remove mixed ordering policy check. Contributed by Peter Bacsko
2020-01-28 15:22:48 +01:00
9520b2ad79
YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal
2020-01-22 16:25:07 +01:00
b4870bce3a
Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R."
...
This reverts commit 0696828a09
2020-01-21 17:45:17 -08:00
0696828a09
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-21 13:41:01 -08:00
1defe3a65a
HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.
2020-01-21 10:58:32 +09:00
581072a8f0
YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg
2020-01-20 13:10:32 +01:00
6d52bbbfcf
YARN-9525. IFile format is not working against s3a remote folder. Contributed by Adam Antal
2020-01-20 12:36:55 +01:00
8b3ee2f7e9
YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph
2020-01-20 09:54:22 +01:00
57aad0f43a
YARN-10081. Exception message from ClientRMProxy#getRMAddress is misleading. Contributed by Ravuri Sushma Sree
2020-01-20 09:41:06 +01:00
14d0f9a775
YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph
2020-01-20 09:28:23 +01:00
a0ff42d761
YARN-10070. Fix NPE if no queue mapping defined for proxy user when using application tag based placement.
...
Contributed by Kinga Marton. Reviewed by Adam Antal.
2020-01-16 12:55:27 +05:30
14c2c3d69d
YARN-9512. [JDK11] TestAuxServices#testCustomizedAuxServiceClassPath fails because of ClassCastException. ( #1806 )
2020-01-16 12:21:50 +09:00
2aa065d98f
YARN-10082. FS-CS converter: disable terminal placement rule checking. Contributed by Peter Bacsko
2020-01-15 12:42:13 +01:00
7c5cecc3b3
YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R
2020-01-15 09:55:41 +01:00
1c51f36be7
YARN-9788. Queue Management API does not support parallel updates. Contributed by Prabhu Joseph
2020-01-14 12:26:03 +01:00
13cea0412c
YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal
2020-01-14 11:00:08 +01:00
621c5eac38
YARN-9912. Capacity scheduler: support u:user2:%secondary_group queue mapping. Contributed by Manikandan R
2020-01-13 17:23:00 +01:00
b7ef8a333f
YARN-9868. Validate %primary_group queue in CS queue manager. Contributed by Manikandan R
2020-01-13 17:15:09 +01:00
2576c31644
YARN-9989. Typo in CapacityScheduler documentation: Runtime Configuration. Contributed by Kevin Su
2020-01-13 16:50:07 +01:00
d842dfffa5
YARN-9866. u:user2:%primary_group is not working as expected. Contributed by Manikandan R
2020-01-12 14:04:15 +01:00
24e6a9e43a
YARN-10067. Add dry-run feature to FS-CS converter tool. Contributed by Peter Bacsko
2020-01-12 13:48:39 +01:00
2301b25899
YARN-10019. Improved container-executor exec() calls.
...
Contributed by Peter Bacsko
2020-01-10 19:04:04 -05:00
20a90c0b5b
YARN-10071. Sync Mockito version with other modules
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-01-10 17:41:04 +09:00
93233a7d6e
YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla)
2020-01-09 17:18:44 +00:00
b1e07d27cc
YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan)
2020-01-08 19:26:01 +00:00
6899be5a17
YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan)
2020-01-08 17:29:56 +00:00
571795cd18
YARN-10068. Fix TimelineV2Client leaking File Descriptors.
...
Contributed by Anand Srinivasan. Reviewed by Adam Antal.
2020-01-08 12:01:30 +05:30
d81d45ff2f
YARN-9956. Improved connection error message for YARN ApiServerClient.
...
Contributed by Prabhu Joseph
2020-01-06 13:24:16 -05:00
dd2607e3ec
YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal
2020-01-06 17:16:11 +01:00
768ee22e9e
YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal
2020-01-06 16:26:33 +01:00
eca7e14c2f
YARN-10053. Use Shared Group Mapping Service in Placement Rules.
...
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:13:57 +05:30
21ada4d1b0
Revert "YARN-10053. Use Shared Group Mapping Service in Placement Rules."
...
This reverts commit 217b56ffdd
2020-01-02 14:12:43 +05:30
217b56ffdd
YARN-10053. Use Shared Group Mapping Service in Placement Rules.
...
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:07:49 +05:30
0fed874adf
YARN-10041. Should not use AbstractPath to create unix domain socket ( #1771 )
2019-12-27 16:50:15 +05:30
40887c9b12
YARN-10057. Upgrade the dependencies managed by yarnpkg. ( #1780 )
2019-12-25 14:10:44 +09:00
34ff7dbaf5
YARN-10055. bower install fails. ( #1778 )
2019-12-23 19:45:47 +09:00
412035b47a
YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne.
2019-12-20 18:07:00 +00:00
12722ab0c7
YARN-10042. Upgrade grpc-xxx depdencies to 1.26.0. Contributed by Sheng Liu.
2019-12-20 11:10:27 +08:00
ef59ffd362
YARN-10038. [UI] Finish Time is not correctly parsed in the RM Apps page. Contributed by Inigo Goiri.
2019-12-19 11:37:17 -08:00
7868da894a
YARN-10037. Upgrade build tools for YARN Web UI v2.
2019-12-19 19:27:31 +09:00
7b93575b92
YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne
2019-12-18 13:18:11 -08:00
fddc3d55c3
YARN-10039. Allow disabling app submission from REST endpoints
2019-12-18 10:48:05 -08:00
92c8962f98
YARN-10020. Fix build instruction of hadoop-yarn-ui. ( #1750 )
2019-12-18 13:34:12 +09:00
f47dcf2d4c
YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne.
2019-12-17 17:11:23 +00:00
631dbbc6f2
YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal
2019-12-15 17:28:04 +01:00
d4bde134e3
YARN-7721. Fix TestContinuousScheduling fails sporadically with NPE.
...
Contributed by Wilfred Spiegelenburg.
2019-12-10 15:00:23 +05:30
dc66de7448
YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena.
2019-12-09 18:37:34 +09:00
e9c5bb8078
Bump bower from 1.7.7 to 1.8.8 ( #1683 )
...
Bumps [bower](https://github.com/bower/bower ) from 1.7.7 to 1.8.8.
- [Release notes](https://github.com/bower/bower/releases )
- [Changelog](https://github.com/bower/bower/blob/master/CHANGELOG.md )
- [Commits](https://github.com/bower/bower/compare/v1.7.7...v1.8.8 )
Signed-off-by: dependabot[bot] <support@github.com>
2019-12-09 15:46:08 +09:00
8dffd8dc89
YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger
2019-12-09 01:25:10 +00:00
ab2cc5ac92
Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger"
...
This reverts commit 76bb297db9
2019-12-09 01:25:10 +00:00
6d970d8100
YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R
2019-12-08 16:35:02 -08:00
cd4779f317
Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources"
...
This reverts commit 92bce918dc
2019-12-08 16:34:46 -08:00
92bce918dc
YARN-10012. Guaranteed and max capacity queue metrics for custom resources
2019-12-08 15:52:17 -08:00
76bb297db9
YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger
2019-12-06 18:33:58 +00:00
e575df5ca6
YARN-10006. IOException used in place of YARNException in CapaitySheduler
...
Contributed by Adam Antal. Reviewed by Szilard Nemeth.
2019-12-06 12:51:28 +05:30
4f758dd4c6
YARN-9607. Auto-configuring rollover-size of IFile format for non-appendable filesystems. Contributed by Adam Antal
2019-12-05 21:49:00 +01:00
cf68857631
YARN-9938. Validate Parent Queue for QueueMapping contains dynamic group as parent queue. Contributed by Manikandan R
2019-12-05 21:04:42 +01:00
0ab9c0398d
YARN-9969. Improve yarn.scheduler.capacity.queue-mappings documentation. Contributed by Manikandan R
2019-12-05 20:53:43 +01:00
5cc6f945da
YARN-9789. Disable Option for Write Ahead Logs of LogMutation. Contributed by Prabhu Joseph
2019-12-05 20:43:26 +01:00
4627dd6708
YARN-9780. SchedulerConf Mutation API does not Allow Stop and Remove Queue in a single call. Contributed by Prabhu Joseph
2019-12-05 20:38:37 +01:00
c71befaf8f
YARN-9781. SchedConfCli to get current stored scheduler configuration. Contributed by Prabhu Joseph
2019-12-05 20:31:24 +01:00
520fe2c99b
YARN-5106. Provide a builder interface for FairScheduler allocations for use in tests. Contributed by Adam Antal
2019-12-05 17:37:40 +01:00
682e6fdeda
YARN-9052. Replace all MockRM submit method definitions with a builder. Contributed by Szilard Nemeth.
2019-12-05 13:26:23 +05:30
c48de9aa2d
YARN-9958. Remove the invalid lock in ContainerExecutor ( #1704 )
2019-12-04 14:35:41 +09:00
a2dadac790
YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph.
2019-11-29 11:54:33 +05:30
ef950b0863
YARN-9290. Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler. Contributed by Prabhu Joseph
2019-11-26 22:04:07 +01:00
828ab400ee
YARN-9362. Code cleanup in TestNMLeveldbStateStoreService. Contributed by Denes Gerencser
2019-11-26 21:45:12 +01:00
aa7ab2719f
YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth.
2019-11-27 01:59:12 +05:30
8c9018d5c7
YARN-9899. Migration tool that help to generate CS config based on FS config [Phase 2]. Contributed by Peter Bacsko
2019-11-26 21:22:35 +01:00
52e9ee39a1
YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak
2019-11-26 16:42:59 +01:00
448ffb12ec
YARN-9937. addendum: Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2019-11-26 13:41:41 +01:00
6c20512b68
Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger"
...
This reverts commit 289bbca870
2019-11-25 17:46:42 +00:00
f8e36e03b4
YARN-9966. Code duplication in UserGroupMappingPlacementRule ( #1709 )
2019-11-25 15:28:53 +09:00
4c1a1287bc
YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi
2019-11-22 12:58:10 +01:00
8afabe41e4
YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu.
2019-11-22 16:06:14 +08:00
dfdc6d6dd9
YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton
2019-11-19 21:14:15 +01:00
ea8ffac121
YARN-9983. Fixed typo in YARN Service overview.
...
Contributed by Denes Gerencser
2019-11-19 14:16:51 -05:00
30b2096933
YARN-9986. Allow app owner to signal container via RMWebServices. Contributed by kyungwan nam.
2019-11-19 11:35:00 +00:00
ea68756c0c
YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg.
2019-11-19 14:10:41 +05:30
215f2052fc
YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg.
2019-11-19 14:03:02 +05:30
dc3f4fc2f4
YARN-9965. Fix NodeManager failing to start on subsequent times when Hdfs Auxillary Jar is set (addendum). Contributed by Prabhu Joseph.
2019-11-19 08:08:40 +05:30
0e22e9ab83
YARN-9562. Add Java changes for the new RuncContainerRuntime. Contributed by Eric Badger
2019-11-18 21:56:31 +00:00
289bbca870
YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger
2019-11-18 21:54:16 +00:00
27642367ef
YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko
2019-11-18 16:29:39 +01:00
bd454348b0
YARN-9982. Fix Container API example link in NodeManager REST API doc. Contributed by Charan Hebri.
2019-11-18 15:19:47 +05:30
b83b9ab418
YARN-9537. Add configuration to disable AM preemption. Contributed by ZhouKang
2019-11-12 09:40:16 -08:00
fb512f5087
YARN-9697. Efficient allocation of Opportunistic containers. Contributed by Abhishek Modi.
2019-11-12 16:34:04 +05:30
30b93f914b
YARN-9865. Capacity scheduler: add support for combined %user + %secondary_group mapping. Contributed by Manikandan R
2019-11-11 13:28:02 +01:00
516377bfa6
YARN-9965. Fix NodeManager failing to start when Hdfs Auxillary Jar is set. Contributed by Prabhu Joseph.
2019-11-11 16:01:44 +05:30
ceb9c6175e
YARN-9890. [UI2] Add Application tag to the app table and app detail page. Contributed by Julia Kinga Marton
2019-11-09 14:12:04 +01:00
31f172fd96
YARN-9677. Make FpgaDevice and GpuDevice classes more similar to each other. Contributed by kevin su
2019-11-09 14:04:14 +01:00
7d0adddf09
YARN-9689: Support proxy user for Router to support kerberos ( #1654 )
2019-11-04 17:13:29 -08:00
51e7d1b37e
YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph.
2019-11-04 23:27:49 +05:30
d462308e04
YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. Contributed by Prabhu Joseph.
2019-11-03 08:47:27 +05:30
e6137d0e50
Make upstream aware of 2.10.0 release
2019-10-30 20:58:32 -07:00
9a2e43e29e
YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph.
2019-10-31 00:34:27 +05:30
e3e7daa4f5
YARN-9889. [UI] Add Application Tag column to RM All Applications table. Contributed by Kinga Marton
2019-10-30 16:26:56 +01:00
ed45c13f67
YARN-2442. ResourceManager JMX UI does not give HA State. Contributed by Rohith Sharma and Cyrus Jackson.
2019-10-29 15:54:55 +05:30
eef34f2d87
YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan
2019-10-25 17:13:04 +00:00
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
fd84ca5161
YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi
2019-10-24 10:05:00 +08:00
83d148074f
YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph.
2019-10-21 20:30:50 +00:00
3f7756dc6c
YARN-9922. Fixed YARN javadoc errors from YARN-9699.
...
Contributed by Peter Bacsko
2019-10-21 16:16:49 -04:00
155864da00
YARN-9875. Improve fair scheduler configuration store on HDFS.
...
Contributed by Prabhu Joseph
2019-10-18 20:30:11 -04:00
72b1bed998
YARN-9884. Make container-executor mount logic modular
...
Contributed by Eric Badger
2019-10-18 19:10:32 -04:00
6d92aa7c30
HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 ( #1656 ). Contributed by Norbert Kalmár, Mate Szalay-Beko
...
* HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5
- Add a static initializer for the unit tests using ZooKeeper to enable
the four-letter-words diagnostic telnet commands. (this is an interface
that become disabled by default, so to keep the ZooKeeper 3.4.x behavior
we enabled it for the tests)
- Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt
records. The new ZooKeeper seems to respond quicker during the fail-over
tests than the ZooKeeper, so we made sure to catch all the relevant records
by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt().
Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>
2019-10-18 13:26:20 -07:00
3d41f33018
HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng.
...
Co-authored-By: Siyao Meng <smeng@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-18 08:25:02 -07:00
f0699a7406
YARN-9841. Capacity scheduler: add support for combined %user + %primary_group mapping. Contributed by Manikandan R
2019-10-18 11:19:49 +02:00
3990ffa083
YARN-9909. Offline format of YarnConfigurationStore. Contributed by Prabhu Joseph.
2019-10-17 06:43:06 +05:30
a5034c7988
YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R.
2019-10-16 21:10:08 +00:00
090f73a9aa
YARN-9900. Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format. Contributed by Prabhu Joseph.
2019-10-16 18:14:31 +05:30
a8ef03e961
YARN-9840. Capacity scheduler: add support for Secondary Group rule mapping. Contributed by Manikandan R
2019-10-15 17:22:52 +02:00
5cc7873a47
YARN-9699. Migration tool that help to generate CS config based on FS config [Phase 1]. Contributed by Peter Bacsko
2019-10-14 17:19:39 +02:00
ec86f42e40
YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal
2019-10-11 14:01:19 +02:00
31e0122f4d
YARN-9860. Enable service mode for Docker containers on YARN
...
Contributed by Prabhu Joseph and Shane Kumpf
2019-10-10 19:02:02 -04:00
6f1ab95168
YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl
2019-10-09 13:26:26 +02:00
35f093f5b3
YARN-9356. Add more tests to ratio method in TestResourceCalculator. Contributed by Zoltan Siegl
2019-10-09 13:09:09 +02:00
be901f4962
YARN-9873. Mutation API Config Change need to update Version Number. Contributed by Prabhu Joseph
2019-10-09 15:53:14 +05:30
5462d2176f
YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena
2019-10-08 11:20:13 -07:00
fb1ecff6a2
Revert "YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph"
...
This reverts commit 4510970e2f
2019-10-05 09:15:17 +05:30
4510970e2f
YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph
2019-10-04 21:49:07 +05:30
0d2d6f9267
YARN-9792. Document examples of SchedulerConf with Node Labels. Contributed by Prabhu Joseph
2019-10-02 23:43:47 +05:30
61a8436004
YARN-9870. Remove unused function from OpportunisticContainerAllocatorAMService. Contributed by Abhishek Modi.
2019-10-02 17:55:07 +05:30
d1ddba60e5
YARN-9862. Increase yarn-services-core test timeout value.
...
Contributed by Prabhu Joseph
2019-10-01 21:15:26 -04:00
99cd7572f1
YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph
2019-10-01 20:06:21 +05:30
425a6c89ae
YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung.
2019-10-01 16:02:26 +05:30
Eric E Payne
Jonathan Hung
Eric Badger
Ayush Saxena
Ahmed Hussein
Brahma Reddy Battula
Akira Ajisaka
Szilard Nemeth
Vinayakumar B
Prabhu Joseph
Eric Yang
Surendra Singh Lilhore
Sunil G
Wilfred Spiegelenburg
RogPodge
Prabhu Joseph
Weiwei Yang
Bilwa
bibinchundatt
Inigo Goiri
tanu.ajmera
Elixir Kook
Wei-Chiu Chuang
Giovanni Matteo Fumarola
Adam Antal
Prabhu Joseph
Liu sheng
Zhankun Tang
Masatake Iwasaki
dependabot[bot]
Wanqiang Ji
Abhishek Modi
HUAN-PING SU
Tao Yang
Yufei Gu
zhoukang
Jonathan Hung
Mate Szalay-Beko
Yuming Wang