e4249c3202
YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph
...
(cherry picked from commit 8660e48ca1
2019-08-26 15:51:38 -07:00
7f20c31e31
YARN-9642. Fix Memory Leak in AbstractYarnScheduler caused by timer. Contributed by Bibin A Chundatt.
...
(cherry picked from commit d3ce53e507
2019-08-26 23:23:49 +05:30
ab98f91638
YARN-8917. Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource. Contributed by Tao Yang.
2019-08-26 21:13:02 +05:30
6980f1740f
YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko
2019-08-21 16:49:34 +02:00
a83718f130
YARN-9100. Add tests for GpuResourceAllocator and do minor code cleanup. Contributed by Peter Bacsko
2019-08-16 15:24:44 +02:00
df616370f0
YARN-8586. Extract log aggregation related fields and methods from RMAppImpl. Contributed by Peter Bacsko
2019-08-16 11:52:51 +02:00
8fee3808c5
YARN-9749. TestAppLogAggregatorImpl#testDFSQuotaExceeded fails on trunk. Contributed by Adam Antal
...
(cherry picked from commit 2a05e0ff3b
2019-08-16 08:52:34 +02:00
e616037d1f
YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph
...
(cherry picked from commit 1845a83cec
2019-08-15 17:16:06 +02:00
d5446b3a23
YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl… ( #1261 )
...
* YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl and connected classes
* Using {} placeholder, and increasing loglevel if log aggregation failed.
(cherry picked from commit c89bdfacc8
2019-08-14 17:36:41 +02:00
4bb238c480
YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko
2019-08-14 17:16:54 +02:00
4dc477b606
YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko
2019-08-14 17:01:41 +02:00
9a87e74e54
YARN-9134. No test coverage for redefining FPGA / GPU resource types in TestResourceUtils. Contributed by Peter Bacsko
2019-08-14 16:46:34 +02:00
cec71691be
YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.
...
(cherry picked from commit 2ac029b949
2019-08-13 16:34:29 +00:00
c5aea8ca56
YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph
...
(cherry picked from commit e4b538bbda
2019-08-12 15:16:18 +02:00
844259203f
YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph
...
(cherry picked from commit b91099efd6
2019-08-12 15:06:16 +02:00
b20fd9e212
YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko
2019-08-12 14:02:17 +02:00
02b4635ff0
YARN-9729. [UI2] Fix error message for logs when ATSv2 is offline. Contributed by Zoltan Siegl.
...
(cherry picked from commit 1c5b28659f
2019-08-11 11:49:25 +05:30
2e6beb1550
Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph
...
(cherry picked from commit 54ac80176e
2019-08-09 13:20:10 +02:00
9fb6c6e2a1
YARN-9715. [UI2] yarn-container-log URI need to be encoded to avoid potential misuses. Contributed by Akhil PB.
...
(cherry picked from commit acffec7a92
2019-08-09 16:07:04 +05:30
3e9071207a
SUBMARINE-57. Add more elaborate message if submarine command is not recognized. Contributed by Adam Antal
...
(cherry picked from commit e5f4cd0fda
2019-08-09 12:14:49 +02:00
4c4f7d9c80
YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently ( #1121 )
...
(cherry picked from commit cbcada804d
2019-08-09 11:43:30 +02:00
02d0e54596
YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak
...
(cherry picked from commit e0c21c6da9
2019-08-09 10:23:10 +02:00
f0dfb8b832
YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak
...
(cherry picked from commit 742e30b473
2019-08-09 10:02:35 +02:00
3bcf44f070
YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak
...
(cherry picked from commit 72d7e570a7
2019-08-09 09:50:32 +02:00
e47c483d9f
YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang.
...
(cherry picked from commit 3b38f2019e
2019-08-08 12:54:31 +00:00
8d357343c4
YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)
...
(cherry picked from commit f51702d539
2019-08-06 14:59:49 -07:00
168dc3f258
YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
...
(cherry picked from commit 42683aef1a
2019-07-30 19:19:33 +00:00
15344006bc
YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung)
2019-07-22 10:46:45 -07:00
bf3d9f6282
YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam.
...
(cherry picked from commit 85d9111a88
2019-07-17 10:47:25 +08:00
4866735cde
YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T.
...
(cherry picked from commit 7a93be0f60
2019-07-16 14:06:36 +05:30
7c9cfc0996
YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal
...
(cherry picked from commit 5446308360
2019-07-15 13:30:58 +02:00
28d6a453a9
YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko
...
(cherry picked from commit 18ee1092b4
2019-07-15 12:02:39 +02:00
2fcbdf4131
YARN-9337. Addendum to fix compilation error due to mockito spy call
...
(cherry picked from commit bb37c6cb7f
2019-07-13 00:45:38 +02:00
4fa0de9f04
YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl
...
(cherry picked from commit 557056e18e
2019-07-12 17:40:57 +02:00
0ede873090
YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal
...
(cherry picked from commit 61b0c2bb7c
2019-07-12 17:29:47 +02:00
c61c969668
YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal
...
(cherry picked from commit c416284bb7
2019-07-12 16:53:26 +02:00
3e3bbb7f5e
YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl
...
(cherry picked from commit 9cec023186
2019-07-11 20:01:52 +02:00
4216090f19
YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal.
2019-07-11 19:24:11 +02:00
5f8395f393
YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T.
2019-07-10 10:34:39 +05:30
4638fa00fc
YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal.
...
(cherry picked from commit a2a8be18cb
2019-07-04 10:26:29 +02:00
d18986e4e8
YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt.
2019-07-04 11:05:54 +05:30
bea79e7645
YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 0c8813f135
2019-07-02 12:15:05 +05:30
c9bccaf148
YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi.
...
(cherry picked from commit 570eee30e5
2019-07-02 10:05:22 +08:00
49d7bb6a92
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2019-06-28 14:20:01 -07:00
860606fc67
YARN-9581. Add support for get multiple RM webapp URLs.
...
Contributed by Prabhu Joseph
(cherry picked from commit f02b0e1994
2019-06-28 14:57:50 -04:00
a2f4e4698b
YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T.
...
(cherry picked from commit be80334cdf
2019-06-27 10:04:40 +05:30
1944a7d844
YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
...
(cherry picked from commit 83dcb9d87e
2019-06-21 17:52:22 +08:00
f148b29508
YARN-9630. [UI2] Add a link in docs's top page
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit eb6be4643f
2019-06-18 14:57:01 +09:00
1e7201f9aa
YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji.
...
(cherry picked from commit 67414a1a80
2019-06-18 13:18:27 +08:00
65f7ec2f39
YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError. Contributed by Sushil Ks.
...
(cherry picked from commit eeaf8edaa7
2019-06-13 14:22:16 -07:00
e0b3cbd221
HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota.
2019-06-13 07:53:40 -06:00
253dcde517
YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B.
...
(cherry picked from commit 52128e352a
2019-06-12 19:25:02 +05:30
72203f7a12
YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl.
2019-06-12 19:23:40 +05:30
f1ead03672
Revert "YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl."
...
This reverts commit f1d3a17d3e
2019-06-12 19:10:23 +05:30
3303723f55
YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T.
2019-06-11 23:42:29 +05:30
d9284d4a57
YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
...
(cherry picked from commit 60c95e9b6a
2019-06-11 23:13:18 +05:30
a37011bd5e
YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie.
...
(cherry picked from commit 6d80b9bc3f
2019-06-11 23:01:03 +05:30
68aec0a98d
YARN-9581. Fixed yarn logs cli to access RM2.
...
Contributed by Prabhu Joseph
(cherry picked from commit cb9bc6e64c
2019-06-06 16:43:25 -04:00
f1d3a17d3e
YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl.
2019-06-06 06:24:01 +05:30
6e2b091515
YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
2019-06-04 15:24:37 +08:00
2f012044ff
YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB.
...
(cherry picked from commit 59719dc560
2019-06-03 15:54:07 +05:30
58042dadc3
YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB.
...
(cherry picked from commit 7f46dda513
2019-06-03 12:25:16 +05:30
e027c87da2
YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T.
...
(cherry picked from commit 4530f4500d
2019-06-03 14:15:20 +08:00
b2a39e8883
YARN-9542. Fix LogsCLI guessAppOwner ignores custome file format suffix.
...
Contributed by Prabhu Joseph
2019-05-29 18:04:13 -04:00
2e561cef47
YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph.
...
(cherry picked from commit 3c63551101
2019-05-29 18:43:13 +00:00
777f7345ef
YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit abf76ac371
2019-05-29 12:14:01 -05:00
a9a3450560
HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:34:16 +09:00
c917ba782e
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang.
...
(cherry picked from commit 4a692bc3be
2019-05-28 16:54:43 +09:00
855dc997d6
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:27:33 +09:00
71f5bfb822
YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T.
...
(cherry picked from commit 570fa2da20
2019-05-15 13:31:07 +05:30
f4ee38df29
YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal.
...
(cherry picked from commit 7d831eca64
2019-05-14 10:49:09 -07:00
b5b152aded
YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl.
...
(cherry picked from commit 64c7f36ab1
2019-05-10 14:28:33 +05:30
c6573562cb
YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung)
...
(cherry picked from commit 597fa47ad1
2019-05-06 11:58:31 -07:00
6fce24fb40
YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein.
...
(cherry picked from commit b094b94d43
2019-05-02 19:48:06 +00:00
cc0c85f04a
YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph.
...
(cherry picked from commit 1c8046d67e
2019-04-23 14:24:15 +08:00
ac85aa80d9
YARN-8587. Added retries for fetching docker exit code.
...
Contributed by Charo Zhang
(cherry picked from commit c16c49b8c3
2019-04-19 15:40:23 -04:00
4a64dab0dd
YARN-8622. Fixed container-executor compilation on MacOSX.
...
Contributed by Siyao Meng
(cherry picked from commit ef97a20831
2019-04-18 19:01:11 -04:00
2503409977
YARN-6695. Fixed NPE in publishing appFinished events to ATSv2.
...
Contributed by Prabhu Joseph
(cherry picked from commit df76cdc895
2019-04-18 12:30:55 -04:00
742a3ad24b
YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6e4399ea61
2019-04-17 22:57:33 -07:00
db185de31c
YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu.
...
(cherry picked from commit 8c1bba375b
2019-04-10 23:02:24 +08:00
7a80b1b481
YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang.
...
(cherry picked from commit ec143cbf67
2019-04-06 20:19:03 +08:00
06cf8ce30f
YARN-9394. Use new API of RackResolver to get better performance. Contributed by Lantao Jin.
...
(cherry picked from commit 945e8c6064
2019-04-04 18:41:04 +08:00
10642a6205
YARN-9391. Fixed node manager environment leaks into Docker containers.
...
Contributed by Jim Brennan
(cherry picked from commit 3c45762a0b
2019-03-25 15:54:52 -04:00
aff5973401
YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB.
...
(cherry picked from commit c79f139519
2019-03-06 16:49:49 +05:30
d721634fea
YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth.
...
(cherry picked from commit 46045c5cb3
2019-03-06 16:01:56 +05:30
63ed16e076
Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph"
...
This reverts commit cf1944eb6e
2019-03-04 17:01:40 +05:30
4ceb4e4f05
YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin.
...
(cherry picked from commit fe6b2b2f23e69f0643e870d9c500117088983209)
2019-03-02 16:04:24 +00:00
d045f02a8d
YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth.
2019-03-01 19:27:03 +05:30
3f3548b66a
YARN-9334. Allow YARN Service client to send SPNEGO challenge header when authentication type is not simple.
...
Contributed by Billie Rinaldi
(cherry picked from commit 04b228e43b
2019-02-28 09:33:05 -08:00
7575e3090d
YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph.
2019-02-28 09:32:07 +08:00
7fa5373ec4
YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie.
...
(cherry picked from commit 8c30114b00
2019-02-27 17:35:09 +08:00
809e3f2453
YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth.
...
(cherry picked from commit 5e91ebd91a
2019-02-27 12:03:58 +05:30
a95a0cbf2f
YARN-9087. Improve logging for initialization of Resource plugins. Contributed by Szilard Nemeth.
2019-02-27 11:54:43 +05:30
bdde6a612e
YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang.
...
(cherry picked from commit 3e1739d589
2019-02-26 00:18:24 +08:00
359e459df1
YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang.
...
(cherry picked from commit 6cec90653d
2019-02-25 15:29:31 +05:30
f282f9c362
YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth.
2019-02-25 11:37:42 +05:30
cdce1c17a0
YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph.
...
(cherry picked from commit 9cd5c5447f
2019-02-24 22:48:55 +08:00
604a915bab
YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie.
...
(cherry picked from commit 9c88695bcd
2019-02-24 22:21:53 +08:00
3e1bd53a37
YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph
2019-02-23 07:59:51 +05:30
cf1944eb6e
YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph
2019-02-22 20:51:47 +05:30
d75aa33612
YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth.
...
(cherry picked from commit 95fbbfed75
2019-02-22 20:23:24 +05:30
fb2735de16
YARN-9258. Support to specify allocation tags without constraint in distributed shell CLI. Contributed by Prabhu Joseph.
...
(cherry picked from commit 6c4ab0312b
2019-02-22 00:23:21 +08:00
c2ef443359
YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph.
2019-02-21 18:06:26 +08:00
e6f2b8730f
YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T.
...
(cherry picked from commit b8de78c570
2019-02-20 01:20:15 +05:30
a060e8cb51
YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB.
2019-02-17 20:19:46 +05:30
830aaac023
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 9385ec45d7
2019-02-15 18:48:21 +09:00
b4dc62a120
YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri.
...
(cherry picked from commit 5b55f3538c
2019-02-15 12:43:01 +05:30
f0b476bb31
YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka.
...
(cherry picked from commit 3a39d9a2d2
2019-02-15 15:43:32 +09:00
a8563d23f5
YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri.
...
(cherry picked from commit b66d5ae9e2
2019-02-14 23:10:21 +05:30
d321d0e747
YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB.
2019-02-14 20:55:43 +05:30
4d4968646a
YARN-9252. Allocation Tag Namespace support in Distributed Shell. Contributed by Prabhu Joseph.
...
(cherry picked from commit 2b7f828d46
2019-02-12 16:31:38 +08:00
9cfa5ce0c1
YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph.
...
(cherry picked from commit 711d22f166
2019-02-12 16:22:34 +08:00
314c559971
YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi.
...
(cherry picked from commit f738b397ae
2019-02-12 14:21:06 +08:00
fbd03543d8
YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph.
...
(cherry picked from commit 0a1637c750
2019-02-11 15:56:34 +08:00
3d552b24ba
YARN-8761. Service AM support for decommissioning component instances.
...
Contributed by Billie Rinaldi
(cherry picked from commit 4c465f5535
2019-02-10 09:26:27 -08:00
6229469574
YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri.
...
(cherry picked from commit e0ab1bdece
2019-02-09 00:28:59 +09:00
fbc08145cf
YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri.
2019-02-08 11:22:44 +05:30
55dde827e6
YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein
...
(cherry picked from commit d1ca9432dd
2019-02-07 16:47:15 +00:00
e2b91b2ccb
YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena.
2019-02-07 13:03:42 +05:30
649da5af04
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa
2019-02-05 14:05:51 -08:00
3a51e50fd8
YARN-9275. Add link to NodeAttributes doc in PlacementConstraints document. Contributed by Masatake Iwasaki.
...
(cherry picked from commit aa7ce50e55
2019-02-05 16:09:15 +08:00
b64e9df949
YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie.
...
(cherry picked from commit 28ad20a711
2019-02-04 14:00:30 +05:30
99876a5ab8
YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla.
...
(cherry picked from commit 604b2489a9
2019-02-04 12:49:06 +05:30
a0fafbc3ef
YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang.
...
(cherry picked from commit f20b043a02
2019-02-04 12:45:40 +05:30
0e7060a1d5
YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth.
...
(cherry picked from commit 71c49fa60f
2019-01-31 09:26:07 +05:30
4052b7ee60
YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph
...
(cherry picked from commit 04105bbfdb
2019-01-29 17:52:54 +00:00
6b8dd8d113
YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang.
...
(cherry picked from commit 4f63ffe444
2019-01-29 10:42:09 +08:00
8c332affa8
YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency.
...
Contributed by Suma Shivaprasad
(cherry picked from commit f5a95f7998
2019-01-28 18:12:04 -05:00
bf760e7e81
YARN-9222. Print launchTime in ApplicationSummary
...
(cherry picked from commit 6cace58e21
2019-01-25 13:23:37 -08:00
4aea4b7c9c
YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB
...
(cherry picked from commit c726445990
2019-01-24 15:02:21 +05:30
bc6374f282
YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang.
2019-01-23 18:10:28 +08:00
8ad7711605
YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph.
...
(cherry picked from commit 721d5c2a5f
2019-01-23 18:07:45 +08:00
9114489566
YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang.
...
(cherry picked from commit d43df31751
2019-01-22 10:46:37 +08:00
ac2f4b64f9
YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang.
...
(cherry picked from commit abde1e1f58
2019-01-21 21:20:01 +08:00
fe7cb2d84a
YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda)
...
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
(cherry picked from commit 6d7eedfd28
2019-01-17 15:13:42 -08:00
4aee7946dc
YARN-9203. Fix typos in yarn-default.xml.
...
This closes #437
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 0a46baecd3
2019-01-17 16:19:49 +09:00
a415446705
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma.
...
(cherry picked from commit 104ef5df36
2019-01-16 14:37:43 +09:00
1dc2b49bfd
YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda)
...
Change-Id: Ib8044307a4241f6b1b7b9b8266b9256f39b16384
2019-01-07 12:21:33 -08:00
bba76b6f31
HADOOP-16031. Fixed TestSecureLogins unit test. Contributed by Akira Ajisaka
2019-01-07 13:24:01 -05:00
2b549e32e1
YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg.
...
(cherry picked from commit 944cf87223
2019-01-07 16:05:57 +08:00
cffe5c1ba0
YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda)
...
Change-Id: I115b6710eb11501e21ab3e9e7cd1a87f322e5cf8
(cherry picked from commit 2c02aa6ec2
2019-01-06 11:10:47 -08:00
1ad1ab5d7a
YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda)
...
Change-Id: Ib2b17cf7860ddc9188f06498086de3326bc09519
(cherry picked from commit 8f004fe4e0
2019-01-06 11:10:41 -08:00
a24cca11f2
YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie.
...
(cherry picked from commit cfe89e6f96
2019-01-04 01:04:39 +08:00
9bfea98746
YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu.
...
(cherry picked from commit 742fd978b5
2018-12-26 08:27:50 +05:30
d6baf9d5bf
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena.
...
(cherry picked from commit e98e963e51
2018-12-22 13:26:41 +09:00
7deef08eb8
YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang.
2018-12-21 16:31:03 +08:00
29c9c8a893
YARN-9126. Fix container clean up for reinitialization.
...
Contributed by Chandni Singh
(cherry picked from commit e815fd9c49
2018-12-19 14:58:19 -05:00
4e1cef3625
YARN-9084. Reset container state and defer readiness check for upgrade.
...
Contributed by Chandni Singh
(cherry picked from commit ccdd982e51
2018-12-18 18:05:40 -05:00
28ca14e71b
YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator.
...
Contributed by Tarun Parimi
(cherry picked from commit 71e0b0d800
2018-12-17 12:08:09 -05:00
71d8ffb551
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang.
...
(cherry picked from commit 26e55d4ceb
2018-12-15 10:30:37 +09:00
52aafb9789
YARN-9125. Fixed Carriage Return detection in Docker container launch command.
...
Contributed by Billie Rinaldi
(cherry picked from commit b2d7204ed0
2018-12-14 17:55:10 -05:00
29e4e5f62c
YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda)
...
Change-Id: I73d73f3d631b28fb9866faa56571839b13824a97
(cherry picked from commit 9fba6cc247
2018-12-12 11:52:16 -08:00
ef0f835537
YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda)
...
Change-Id: I845131273e52a9d81dbc813ea6d4af06b205e334
(cherry picked from commit c771fe6e10
2018-12-12 11:52:07 -08:00
2b3c3d2a32
YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters.
...
(cherry picked from commit 1c09a10e96
2018-12-10 12:07:23 +08:00
3ab6ea7aca
YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics
...
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
2018-12-07 10:45:47 -08:00
8c70728f7f
YARN-9071. Improved status update for reinitialized containers.
...
Contributed by Chandni Singh
(cherry picked from commit 1b790f4dd1
2018-12-05 19:04:55 -05:00
559cb99bf6
Revert "YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)"
...
This reverts commit 920401b296
2018-12-04 14:13:39 -08:00
6b01e4d2a8
YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu
...
(cherry picked from commit 1c8bd7128c99d8215ef16438bd2ce6b1f025a966)
2018-11-30 10:16:00 -08:00
a8f67ad7c9
YARN-9067. Fixed Resource Manager resource leak via YARN service.
...
Contributed by Eric Yang
2018-11-29 13:50:06 -05:00
183ec39c4b
YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T.
...
(cherry picked from commit 07142f54a8
2018-11-29 22:16:32 +05:30
ee9deb6e9c
YARN-9030. Log aggregation changes to handle filesystems which do not support setting permissions. (Suma Shivaprasad via wangda)
...
Change-Id: I80f1e8196b8624e24d74494719fdedfd7061dced
(cherry picked from commit 9de8e8d049
2018-11-28 15:06:35 -08:00
df0e7766e4
YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko
...
(cherry picked from commit 3ce99e32f7
2018-11-28 08:50:18 -06:00
db8b2a130c
YARN-9056. Improved YARN service upgrade state logic for readiness check.
...
Contributed by Chandni Singh
(cherry picked from commit f657a2a661
2018-11-27 19:45:23 -05:00
838190482d
YARN-8986. Added port publish for Docker container running with bridge.
...
Contributed by Charo Zhang
2018-11-27 14:27:13 -05:00
01477108d9
YARN-8984. AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty. Contributed by Yang Wang.
...
(cherry picked from commit 176bb3f812
2018-11-22 17:04:16 +08:00
34387599c4
YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda)
...
Change-Id: I21b1addc9c32817650ea744a8f2e6b5602f2f4d4
(cherry picked from commit 8b23814415
2018-11-19 09:01:21 -08:00
650581a19d
YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun.
...
(cherry picked from commit d027a24f03
2018-11-18 23:23:09 +08:00
bee5bf867b
YARN-9001. [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs. (Zac Zhou via wangda)
...
Change-Id: I7e8d1c27ebd37e0907ca570c4f3d56fe7a859635
2018-11-16 10:26:50 -08:00
13e3670e7f
YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report.
...
(cherry picked from commit ee3355be3c
2018-11-16 18:37:20 +05:30
da05328824
YARN-8834 Provide Java client for fetching Yarn specific entities from TimelineReader. Contributed by Abhishek Modi
...
(cherry picked from commit a3edfddcf7
2018-11-16 18:37:01 +05:30
ea753046b0
YARN-8918. [Submarine] Correct method usage of str.subString in CliUtils. (Zhankun Tang via wangda)
...
Change-Id: Id1f11dbab3aa838dee3a0ec8b8fd5dc32f5dd946
(cherry picked from commit 076b795b2e
2018-11-13 11:35:06 -08:00
2415f8a5be
YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt.
...
(cherry picked from commit c741109522
2018-11-12 18:21:11 +08:00
5399ff8177
YARN-9002. Improve keytab loading for YARN Service.
...
Contributed by Gour Saha
(cherry picked from commit 2664248797
2018-11-10 01:54:59 -05:00
b3321d9933
YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang.
...
(cherry picked from commit e1bbf7dcdf
2018-11-08 17:50:16 +08:00
b10ec0aa14
YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji.
...
(cherry picked from commit c96cbe8659
2018-11-07 22:44:01 +08:00
02abd0f80a
YARN-8866. Fix a parsing error for crossdomain.xml.
...
(cherry picked from commit 8dc1f6dbf7
2018-11-07 18:29:28 +09:00
4ff7255d6e
YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang.
...
(cherry picked from commit 482716e5a4
2018-11-07 10:49:09 +08:00
41c4e9583d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang.
...
(cherry picked from commit 951c98f890
2018-11-07 11:18:48 +09:00
9265934201
YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg
...
(cherry picked from commit ab6aa4c726
2018-11-06 08:47:30 -06:00
f00125e2d9
YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang.
...
(cherry picked from commit 5d6554c722
2018-11-06 14:52:10 +08:00
b474239e0b
YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji.
...
(cherry picked from commit c7fcca0d7e
2018-11-06 13:17:19 +08:00
b9a3c988c0
YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne
...
(cherry picked from commit 2ab611d48b
2018-11-01 14:31:09 -07:00
7920f79049
YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0.
...
(cherry picked from commit 4ec4ec6971
2018-10-30 11:50:33 +05:30
288bc3f1eb
YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang.
...
(cherry picked from commit 78ea897b67
2018-10-29 12:05:45 +08:00
5db4272d91
YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg.
...
(cherry picked from commit 1d90a0dd23
2018-10-29 11:55:22 +08:00
3665c46176
YARN-8938. Updated YARN service upgrade document.
...
Contributed by Chandni Singh
(cherry picked from commit bbc6dcd3d0
2018-10-24 11:52:50 -04:00
7097755925
YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg
...
(cherry picked from commit 93fb3b4b9c
2018-10-23 12:54:04 -05:00
660fff3138
YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli.
...
(cherry picked from commit 0b62983c5a
2018-10-23 14:06:13 +05:30
30998fea28
YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh.
...
(cherry picked from commit 2202e00ba8
2018-10-23 09:57:26 +05:30
8724c2900f
YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad.
...
(cherry picked from commit e50334513c
2018-10-22 14:16:24 +05:30
04600cf5da
YARN-8910. Fixed misleading log statement when container max retries is infinite.
...
Contributed by Chandni Singh
(cherry picked from commit 47ad98b2e1
2018-10-19 13:49:51 -04:00
9ed9e185d7
YARN-8916. Define a constant docker string in ContainerRuntimeConstants.java for better maintainability. (Zhankun Tang via wangda)
...
Change-Id: I1349e740037f81afdbe30edbe741f20e88fd0a90
(cherry picked from commit 5e02b4915b
2018-10-19 09:50:18 -07:00
bd08554aa4
YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda)
...
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
(cherry picked from commit 071b9e19aa
2018-10-19 09:50:12 -07:00
042f2df19b
YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang.
...
(cherry picked from commit 13cc0f50ea
2018-10-19 10:01:45 +08:00
91c916e209
YARN-8618. Added detection for non-upgradable service.
...
Contributed by Chandni Singh
(cherry picked from commit 66f059ed1d
2018-10-18 20:00:50 -04:00
7252f8e117
YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda)
...
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07
2018-10-18 12:12:19 -07:00
5f7ed043c8
YARN-8456. Fix a configuration handling bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable. (Zhankun Tang via wangda)
...
Change-Id: Iff150ea98ba0c60d448474fd940eb121afce6965
(cherry picked from commit a457a8951a
2018-10-18 12:12:14 -07:00
a7785d2764
YARN-6098. Add documentation for Delete Queue. (Suma Shivaprasad via wangda)
...
Change-Id: I23bf7ca0acbe1168a7f805ceccdd62fe41bfeb35
(cherry picked from commit bfb88b10f4
2018-10-18 12:12:09 -07:00
4681e17a36
YARN-8687. Update YARN service file type in documentation.
...
Contributed by Suma Shivaprasad
2018-10-18 12:04:04 -04:00
bde4fd5ed9
Preparing for 3.2.0 release
2018-10-18 17:07:45 +05:30
6380ee5512
YARN-8759. Copy of resource-types.xml is not deleted if test fails, causes other test failures. Contributed by Antal Bálint Steinbach.
...
(cherry picked from commit 5085e5fa9e
2018-10-17 16:06:07 +05:30
fc25def7d7
YARN-8810. Fixed a YARN service bug in comparing ConfigFile object.
...
Contributed by Chandni Singh
2018-10-16 18:57:50 -04:00
80ee60c88c
YARN-8892. YARN UI2 doc changes to update security status (verified under security environment). (Sunil G via wangda)
...
Change-Id: I8bc8622936861b8d6de3e42a0b75af86ad8a3961
(cherry picked from commit 538250db26
2018-10-16 13:53:08 -07:00
aa23c8d739
YARN-8798. [Submarine] Job should not be submitted if --input_path option is missing. (Zhankun Tang via wangda)
...
Change-Id: I7ae0e44eb5179b04a6ac861ec1c65f3b18c38f0f
(cherry picked from commit 143d74775b
2018-10-16 13:53:03 -07:00
920401b296
YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)
...
Change-Id: I46e8d9fd32c7745c313030da62da41486a77b3ea
(cherry picked from commit 46d6e00166
2018-10-16 13:52:58 -07:00
8610c20724
YARN-8875. [Submarine] Add documentation for submarine installation script details. (Xun Liu via wangda)
...
Change-Id: I1c8d39c394e5a30f967ea514919835b951f2c124
(cherry picked from commit ed08dd3b0c
2018-10-16 13:52:53 -07:00
a439aa79dc
YARN-8879. Kerberos principal is needed when submitting a submarine job. Contributed by Zac Zhou.
...
(cherry picked from commit 753f149fd3
2018-10-16 22:18:30 +05:30
1ca5f974f1
YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang.
...
(cherry picked from commit fa94d370b6
2018-10-15 21:22:45 +05:30
62d329cac0
YARN-5742 Serve aggregated logs of historical apps from timeline service. Contributed by Rohith Sharma KS
...
(cherry picked from commit 8d1981806f
2018-10-12 07:18:13 +05:30
cdbca8b133
YARN-8861. executorLock is misleading in ContainerLaunch. Contributed by Chandni Singh
...
(cherry picked from commit e787d65a08
2018-10-11 10:58:48 -05:00
f29a7fdb14
YARN-8753. [UI2] Lost nodes representation missing from Nodemanagers Chart. Contributed by Yesha Vora.
...
(cherry picked from commit 8a37983990
2018-10-11 16:16:05 +05:30
0bf0fb5499
YARN-8666. [UI2] Remove application tab from YARN Queue Page. Contributed by Yesha Vora.
...
(cherry picked from commit d3da741fa2
2018-10-11 16:07:08 +05:30
145c7aa663
YARN-7644. NM gets backed up deleting docker containers. Contributed by Chandni Singh
...
(cherry picked from commit 5ce70e1211
2018-10-10 10:01:52 -05:00
eb0147a4c7
YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan.
...
(cherry picked from commit edce866489
2018-10-10 09:42:45 +08:00
614d6cf99c
YARN-8845. Removed unused hadoop.registry.rm reference.
...
Contributed by Íñigo Goiri
(cherry picked from commit bf04f19456
2018-10-09 15:56:04 -04:00
8e8b748724
YARN-8852. Add documentation for submarine installation details. (Zac Zhou via wangda)
...
Change-Id: If5681d1ef37ff5dc916735eeef15a6120173d653
(cherry picked from commit a23ea68b97
2018-10-09 10:19:16 -07:00
f6a73d181f
YARN-8853. [UI2] Application Attempts tab is not shown correctly when there are no attempts. Contributed by Akhil PB.
...
(cherry picked from commit 6f11919beb
2018-10-09 22:05:53 +05:30
21553e22b6
YARN-8843. updateNodeResource does not support units for memory. Contributed by Manikandan R.
...
(cherry picked from commit 745f64012a
2018-10-08 10:40:59 -07:00
50acba2e39
YARN-7825. [UI2] Maintain constant horizontal application info bar for Application Attempt page. Contributed by Akhil PB.
...
(cherry picked from commit 228dc192f1
2018-10-08 11:55:50 +05:30
843e7f81fe
YARN-8819. Fix findbugs warnings in YarnServiceUtils
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b85ee0d75d
2018-10-07 00:55:08 +09:00
a7711073ac
YARN-8797. [UI2] Improve error pages in new YARN UI. Contributed by Akhil PB.
...
(cherry picked from commit 94d82f4162
2018-10-05 14:33:47 +05:30
8957a61ed9
YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB.
...
(cherry picked from commit 751f626e50
2018-10-05 12:29:08 +05:30
b3ac886933
YARN-8844. TestNMProxy unit test is failing. (Eric Yang via wangda)
...
Change-Id: I241fa8701b6f1dbcad87fd2e9a429e32e7aa40f5
2018-10-04 10:48:47 -07:00
abe4a8e5d8
YARN-8758. Support getting PreemptionMessage when using AMRMClientAsyn. (Zian Chen via wangda)
...
Change-Id: Ibf5d165f49957b582eeadeb41dc285c84d2f05e7
(cherry picked from commit 6926fd0ec6
2018-10-04 10:43:16 -07:00
4de3cf1968
YARN-8840. Add missing cleanupSSLConfig() call for TestTimelineClient test. Contributed by Aki Tanaka.
...
(cherry picked from commit 39b35036ba
2018-10-04 09:52:50 +09:00
5edb9d3b97
YARN-8785. Improve the error message when a bind mount is not whitelisted. Contributed by Simon Prewo
2018-10-02 07:16:29 -06:00
d0ee6fbe28
YARN-8621. Add test coverage of custom Resource Types for the apps/<appId> REST API endpoint. (Contributed by Szilard Nemeth)
2018-10-01 14:46:42 -07:00
59d5af21b7
YARN-8760. [AMRMProxy] Fix concurrent re-register due to YarnRM failover in AMRMClientRelayer. Contributed by Botong Huang.
2018-10-01 13:12:38 -07:00
fd6be5898a
YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach.
2018-09-29 17:47:12 +08:00
19ad5be651
YARN-8800. Updated documentation of Submarine with latest examples. Contributed by Wangda Tan.
2018-09-29 00:01:04 +05:30
d1c1dde309
YARN-8734. Readiness check for remote service belonging to the same user. Contributed by Eric Yang
2018-09-28 09:40:48 -07:00
8598b498bc
YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang.
2018-09-28 15:32:07 +00:00
7093afd874
YARN-8829. Cluster metrics can fail with IndexOutOfBoundsException. Contributed by Akshay Agarwal.
2018-09-28 12:35:33 +05:30
90e2e493b3
YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks.
2018-09-27 15:53:39 -07:00
b237a0dd44
YARN-6456. Added config to set default container runtimes.
...
Contributed by Craig Condit
2018-09-27 15:31:18 -04:00
6b988d821e
YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang
2018-09-26 14:43:00 -07:00
913f87dada
YARN-8665. Added Yarn service cancel upgrade option.
...
Contributed by Chandni Singh
2018-09-26 14:51:35 -04:00
e5287a4fe0
YARN-8824. App Nodelabel missed after RM restart for finished apps. Contributed by Bibin A Chundatt.
2018-09-26 12:30:26 +05:30
44edcdfd6a
YARN-8745. Misplaced the TestRMWebServicesFairScheduler.java file. Contributed by Y. SREENIVASULU REDDY.
2018-09-26 10:09:11 +09:00
94b5c54e8a
YARN-8752. yarn-registry.md has wrong word ong-lived, it should be long-lived. Contributed by leiqiang.
2018-09-26 09:42:53 +09:00
26c94a0fd0
YARN-8623. Update Docker examples to use image which exists. Contributed by Craig Condit
2018-09-25 06:38:25 -06:00
50bc7746d7
YARN-8815. RM fails to recover finished unmanaged AM. Contributed by Bibin A Chundatt.
2018-09-25 11:31:14 +05:30
d0bc6a27fd
YARN-8817. [Submarine] In cases when user doesn't ask HDFS path while submitting job but framework requires user to set HDFS related environments. Contributed by Wangda Tan.
2018-09-25 11:23:34 +05:30
29dad7d258
YARN-8616. systemClock should be used in RMAppImpl instead of System.currentTimeMills(), to be consistent. (Contributed by Szilard Nemeth)
2018-09-24 16:04:28 -07:00
cf62ff9a6a
YARN-6338. Typos in Docker docs: contains => containers. (Contributed by Zoltan Siegl)
2018-09-24 15:54:02 -07:00
3090922805
YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang.
2018-09-24 11:37:05 -07:00
32a35dc94b
YARN-8742. [UI2] Container logs on Application / Service pages on UI2 are not available many case, improve error messages in such cases. Contributed by Akhil PB.
2018-09-24 12:20:28 +05:30
0cd6346102
YARN-8769. [Submarine] Allow user to specify customized quicklink(s) when submit Submarine job. Contributed by Wangda Tan.
2018-09-21 23:39:22 +05:30
a2752779ac
YARN-8628. [UI2] Few duplicated or inconsistent information displayed in UI2. Contributed by Akhil PB.
2018-09-21 15:47:10 +05:30
aa4bd493c3
YARN-8801. Fixed header comments for docker utility functions.
...
Contributed by Zian Chen
2018-09-20 13:08:59 -04:00
6b5838ed32
YARN-8784. DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk. Contributed by Eric Badger
2018-09-19 16:44:51 -05:00
efdea85ad1
YARN-8791. Trim docker inspect output for line feed for STOPSIGNAL parsing.
...
Contributed by Chandni Singh
2018-09-19 13:16:11 -04:00
1824d5d1c4
YARN-8757. [Submarine] Add Tensorboard component when --tensorboard is specified. Contributed by Wangda Tan.
2018-09-19 22:18:55 +05:30
0712537e79
YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang.
2018-09-19 19:31:07 +08:00
2df0a8dcb3
YARN-8648. Container cgroups are leaked when using docker. Contributed by Jim Brennan
2018-09-18 15:36:45 -05:00
78a0d173e4
YARN-8787. Fix broken list items in PlacementConstraints documentation. Contributed by Masahiro Tanaka.
2018-09-18 17:19:39 +08:00
bbeca0107e
YARN-8652. [UI2] YARN UI2 breaks if getUserInfo REST API is not available in older versions. Contributed by Akhil PB.
2018-09-18 12:44:21 +05:30
0cc6e03945
YARN-8726. [UI2] YARN UI2 is not accessible when config.env file failed to load. Contributed by Akhil PB.
2018-09-18 12:31:55 +05:30
9a265fa673
YARN-8782. Fix exception message in Resource.throwExceptionWhenArrayOutOfBound. Contributed by Gergely Pollak.
2018-09-17 22:15:24 +08:00
33d8327cff
YARN-8715. Make allocation tags in the placement spec optional for node-attributes. Contributed by Weiwei Yang.
2018-09-17 10:07:45 +05:30
144a55f0e3
YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit
2018-09-14 10:41:55 -06:00
78902f0250
YARN-8748. Javadoc warnings within the nodemanager package. Contributed by Craig Condit
2018-09-14 10:28:36 -06:00
99237607bf
YARN-8706. Allow additional flag in docker inspect call.
...
Contributed by Chandni Singh
2018-09-14 11:46:59 -04:00
f1a893fdbc
YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi.
2018-09-14 16:33:51 +08:00
568ebecdf4
YARN-8772. Annotation javax.annotation.Generated has moved
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-09-14 15:02:52 +09:00
250b50018e
YARN-8680. YARN NM: Implement Iterable Abstraction for LocalResourceTracker state. Contributed by Pradeep Ambati
2018-09-13 13:28:54 -05:00
39c1ea1ed4
YARN-8729. Node status updater thread could be lost after it is restarted. Contributed by Tao Yang.
2018-09-13 22:21:35 +08:00
f4bda5e8e9
YARN-8630. ATSv2 REST APIs should honor filter-entity-list-by-user in non-secure cluster when ACls are enabled. Contributed by Rohith Sharma K S.
2018-09-13 17:47:21 +05:30
8e9afbfb66
YARN-8768. Javadoc error in node attributes. Contributed by Sunil Govindan.
2018-09-12 15:12:28 -06:00
02b9bfdf9e
YARN-8658. [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor. Contributed by Young Chen.
2018-09-12 11:46:35 -07:00
5e64e62dee
YARN-8740. Clear node attribute path after each test run. Contributed by Bibin A Chundatt.
2018-09-12 16:01:01 +05:30
c44088ac19
YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan.
2018-09-12 16:01:01 +05:30
1e7d6e55a5
YARN-7865. Node attributes documentation. Contributed by Naganarasimha G R.
2018-09-12 16:01:01 +05:30
52194351e7
YARN-8721. Relax NE node-attribute check when attribute doesn't exist on a node. Contributed by Sunil Govindan.
2018-09-12 16:01:01 +05:30
67ae81f0e0
YARN-7863. Modify placement constraints to support node attributes. Contributed by Sunil Govindan.
2018-09-12 16:01:01 +05:30
8c947398ab
YARN-8574. Allow dot in attribute values. Contributed by Bibin A Chundatt.
2018-09-12 16:01:01 +05:30
eb08543c7a
YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt.
2018-09-12 16:01:01 +05:30
76183428b7
YARN-8351. Node attribute manager logs are flooding RM logs. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
8cf6a9a2bd
YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R.
2018-09-12 16:01:00 +05:30
5dc7d6e0f3
YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt.
2018-09-12 16:01:00 +05:30
0a01b1350d
YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt.
2018-09-12 16:01:00 +05:30
b9890d1f66
YARN-7875. Node Attribute store for storing and recovering attributes. Contributed by Bibin A Chundatt.
2018-09-12 16:01:00 +05:30
a6590c1f1f
YARN-8117. Fix TestRMWebServicesNodes test failure. Contributed by Bibin A Chundatt.
2018-09-12 16:01:00 +05:30
901e85238d
YARN-8033. CLI Integration with NodeAttributesManagerImpl. Contributed by Naganarasimha G R.
2018-09-12 16:01:00 +05:30
89b3ebd11e
YARN-8092. Expose Node Attributes info via RM nodes REST API. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
440ff7f563
YARN-8094. Support configuration based Node Attribute provider. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
6f4bc49c6d
YARN-7988. Refactor FSNodeLabelStore code for Node Attributes store support. Contributed by Bibin A Chundatt.
2018-09-12 16:01:00 +05:30
3b3b6efe21
YARN-7871. Node attributes reporting from NM to RM. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
86d024ef2a
YARN-7965. NodeAttributeManager add/get API is not working properly. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
ffcabd24c3
YARN-7856. Validate Node Attributes from NM. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
2f7712be09
YARN-6858. Attribute Manager to store and provide node attributes in RM. Contributed by Naganarasimha G R.
2018-09-12 16:01:00 +05:30
d312b5cf9f
YARN-7757. Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers. Contributed by Weiwei Yang.
2018-09-12 16:01:00 +05:30
4458b2772f
YARN-7840. Update PB for prefix support of node attributes. Contributed by Naganarasimha G R.
2018-09-12 16:01:00 +05:30
d9d93e3925
YARN-7842. PB changes to carry node-attributes in NM heartbeat. Contributed by Weiwei Yang.
2018-09-12 16:00:59 +05:30
2475fb0a1e
YARN-6856. [YARN-3409] Support CLI for Node Attributes Mapping. Contributed by Naganarasimha G R.
2018-09-12 16:00:59 +05:30
1f42ce907a
YARN-6855. [YARN-3409] CLI Proto Modifications to support Node Attributes. Contributed by Naganarasimha G R.
2018-09-12 16:00:59 +05:30
987d8191ad
YARN-8709: CS preemption monitor always fails since one under-served queue was deleted. Contributed by Tao Yang.
2018-09-10 19:55:20 +00:00
de85f841da
YARN-8698. [Submarine] Failed to reset Hadoop home environment when submitting a submarine job. (Zac Zhou via wangda)
...
Change-Id: If7fdc11fcc4e49bed2186ded76cd69164f34f502
2018-09-09 17:45:31 -07:00
bf8a1750e9
YARN-8706. Updated docker container stop logic to avoid double kill.
...
Contributed by Chandni Singh
2018-09-07 20:18:09 -04:00
7d62334387
YARN-8751. Reduce conditions that mark node manager as unhealthy.
...
Contributed by Craig Condit
2018-09-07 19:46:15 -04:00
Jonathan Hung
bibinchundatt
Rohith Sharma K S
Szilard Nemeth
Adam Antal
Eric Badger
Sunil G
Eric E Payne
Haibo Chen
Weiwei Yang
Szilard Nemeth
Erik Krogen
Eric Yang
Wanqiang Ji
Zhankun Tang
Inigo Goiri
Sean Mackrory
Ahmed Hussein
Takanobu Asanuma
Akira Ajisaka
Siyao Meng
Giovanni Matteo Fumarola
Masatake Iwasaki
Vinayakumar B
Wangda Tan
rahul3
Jonathan Hung
Jonathan Hung
Jason Lowe
Vrushali C
Vidura Mudalige
Shane Kumpf
Billie Rinaldi
Eric E Payne
Andrew Purtell
Naganarasimha