b71a7f195c
YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar
2019-09-06 11:00:07 -07:00
e7d44e48f7
YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar
2019-09-06 10:17:07 -07:00
0ccf4b0fe1
YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta
2019-09-05 14:01:42 -07:00
2e2e5401f2
YARN-9718. Fixed yarn.service.am.java.opts shell injection. Contributed by Eric Yang
2019-09-05 12:49:58 -07:00
172bcd8e01
YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi.
2019-09-05 16:20:05 +08:00
3db7184082
YARN-9804. Update ATSv2 document for latest feature supports.
2019-09-05 08:42:17 +05:30
bb26514ba9
YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T.
2019-09-03 14:55:15 +08:00
03489124ea
YARN-9797. LeafQueue#activateApplications should use resourceCalculator#fitsIn. Contributed by Bilwa S T.
2019-09-03 11:53:34 +05:30
f4d6e82075
YARN-9400. Remove unnecessary if at EntityGroupFSTimelineStore#parseApplicationId. Contributed by Prabhu Joseph.
2019-09-02 10:58:23 +05:30
b939ae98e6
YARN-8174. Add containerId to ResourceLocalizationService fetch failure log statement. Contributed by Prabhu Joseph.
2019-09-02 10:45:59 +05:30
bc27f7fb6d
YARN-7982. Do ACLs check while retrieving entity-types per application. Contribued by Prabhu Joseph.
2019-09-02 10:15:59 +05:30
751b5a1ac8
YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph.
2019-09-01 22:40:40 +05:30
fef65b4c2b
YARN-9793. Remove duplicate sentence from TimelineServiceV2.md. Contributed by Julia Kinga Marton.
2019-09-01 12:43:26 +05:30
c7ef4fb74a
YARN-8678. Queue Management API - rephrase error messages. Contributed by Prabhu Joseph.
2019-09-01 10:00:04 +05:30
d2d963f3d4
YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam.
2019-09-01 09:54:46 +05:30
7d998cb627
YARN-9800. TestRMDelegationTokens can fail in testRemoveExpiredMasterKeyInRMStateStore. Contributed by Adam Antal.
2019-08-31 10:03:09 +05:30
eeccd2f6f6
YARN-9798. ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently. Contributed by Tao Yang.
2019-08-31 09:57:45 +05:30
c929b383f8
YARN-9540. TestRMAppTransitions fails intermittently. Contributed by Tao Yang.
2019-08-30 17:17:43 +05:30
5b557de721
YARN-9714. ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby. Contributed by Tao Yang.
2019-08-30 10:35:09 +05:30
8c0759d02a
YARN-9664. Improve response of scheduler/app activities for better understanding. Contributed by Tao Yang.
2019-08-29 18:14:39 +08:00
6f2226a013
YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).
2019-08-28 20:40:34 +00:00
8ef46595da
YARN-9438. launchTime not written to state store for running applications
2019-08-27 15:53:43 -07:00
717c853873
YARN-9755. Fixed RM failing to start when FileSystemBasedConfigurationProvider is configured.
...
Contributed by Prabhu Joseph
2019-08-27 13:14:59 -04:00
d70f5231a7
YARN-9640. Slow event processing could cause too many attempt unregister events. Contributed by Bibin A Chundatt.
2019-08-27 08:38:12 +05:30
8660e48ca1
YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph
2019-08-26 15:50:33 -07:00
d3ce53e507
YARN-9642. Fix Memory Leak in AbstractYarnScheduler caused by timer. Contributed by Bibin A Chundatt.
2019-08-26 23:21:33 +05:30
6d7f01c92d
YARN-9771. Add GPU in the container-executor.cfg example. Contributed by Julia Kinga Marton
2019-08-26 16:12:35 +00:00
689d2e6105
YARN-8917. Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource. Contributed by Tao Yang.
2019-08-26 21:06:15 +05:30
7ab88dbfa6
YARN-7291. Better input parsing for resource in allocation file. Contributed by Zoltan Siegl
2019-08-21 17:01:18 +02:00
e8fa192f07
YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko
2019-08-21 16:44:22 +02:00
e684b17e6f
YARN-5857. TestLogAggregationService.testFixedSizeThreadPool fails intermittently on trunk. Contributed by Bilwa S T.
2019-08-21 17:14:42 +05:30
094d73608f
YARN-9758. Upgrade JQuery to latest version for YARN UI. Contributed by Akhil PB.
2019-08-20 18:47:44 +05:30
0e0ddfaf24
YARN-2599. Standby RM should expose jmx endpoint. Contributed by Rohith Sharma K S.
2019-08-17 15:43:19 +05:30
8d754c2c39
HADOOP-16351. Change ":" to ApplicationConstants.CLASS_PATH_SEPARATOR. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-16 17:26:09 -07:00
9b8359bb08
YARN-9461. TestRMWebServicesDelegationTokenAuthentication.testCancelledDelegationToken fails with HTTP 400. Contributed by Peter Bacsko
2019-08-16 12:31:58 +02:00
4456ea67b9
YARN-8586. Extract log aggregation related fields and methods from RMAppImpl. Contributed by Peter Bacsko
2019-08-16 11:36:14 +02:00
2216ec54e5
YARN-9100. Add tests for GpuResourceAllocator and do minor code cleanup. Contributed by Peter Bacsko
2019-08-16 09:13:20 +02:00
2a05e0ff3b
YARN-9749. TestAppLogAggregatorImpl#testDFSQuotaExceeded fails on trunk. Contributed by Adam Antal
2019-08-16 08:52:09 +02:00
22c4f38c4b
YARN-9679. Regular code cleanup in TestResourcePluginManager ( #1122 )
2019-08-15 17:32:05 +02:00
1845a83cec
YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph
2019-08-15 17:15:38 +02:00
167acd87da
YARN-9683. Remove reapDockerContainerNoPid left behind by YARN-9074 ( #1212 ) Contributed by Kevin Su.
...
Reviewed-by: Eric Yang <eyang@apache.org>
Reviewed-by: Adam Antal <adam.antal@cloudera.com>
2019-08-14 10:42:29 -07:00
c89bdfacc8
YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl… ( #1261 )
...
* YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl and connected classes
* Using {} placeholder, and increasing loglevel if log aggregation failed.
2019-08-14 17:35:16 +02:00
3e0410449f
YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko
2019-08-14 17:13:54 +02:00
e5e609384f
YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko
2019-08-14 16:58:22 +02:00
89a53c7eb4
YARN-9747. Reduce additional namenode call by EntityGroupFSTimelineStore#cleanLogs. Contributed by Prabhu Joseph.
2019-08-14 13:46:23 +05:30
2ac029b949
YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.
2019-08-13 16:21:18 +00:00
b4097b96a3
YARN-9744. RollingLevelDBTimelineStore.getEntityByTime fails with NPE. Contributed by Prabhu Joseph.
2019-08-13 19:04:00 +05:30
201dc667e9
YARN-9719. Fixed YARN service restart bug when application ID no longer exist in RM.
...
Contributed by kyungwan nam
2019-08-12 18:24:00 -04:00
e4b538bbda
YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph
2019-08-12 15:15:43 +02:00
b91099efd6
YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph
2019-08-12 15:05:28 +02:00
e0517fea33
YARN-9134. No test coverage for redefining FPGA / GPU resource types in TestResourceUtils. Contributed by Peter Bacsko
2019-08-12 14:36:07 +02:00
13a5803ccf
YARN-9464. Support pending resource metrics in RM's RESTful API. Contributed by Prabhu Joseph.
2019-08-12 14:31:24 +05:30
8fbf8b2eb0
YARN-9722. PlacementRule logs object ID in place of queue name. Contributed by Prabhu Joseph.
2019-08-12 10:44:46 +05:30
cf5d8957ee
YARN-9657. AbstractLivelinessMonitor add serviceName to PingChecker thread. Contributed by Bilwa S T.
2019-08-11 15:41:56 +05:30
1c5b28659f
YARN-9729. [UI2] Fix error message for logs when ATSv2 is offline. Contributed by Zoltan Siegl.
2019-08-11 11:47:07 +05:30
e69db45e50
Revert "YARN-9729. [UI2] Fix error message for logs without ATSv2. Contributed by Zoltan Siegl."
...
This reverts commit 50cec5a704
2019-08-10 21:10:22 +05:30
50cec5a704
YARN-9729. [UI2] Fix error message for logs without ATSv2. Contributed by Zoltan Siegl.
2019-08-10 21:06:25 +05:30
6ff0453ede
YARN-9527. Prevent rogue Localizer Runner from downloading same file repeatly.
...
Contributed by Jim Brennan
2019-08-09 14:12:17 -04:00
a79564fed0
YARN-9732. yarn.system-metrics-publisher.enabled=false is not honored by RM. Contributed by KWON BYUNGCHANG.
2019-08-09 22:25:30 +05:30
acffec7a92
YARN-9715. [UI2] yarn-container-log URI need to be encoded to avoid potential misuses. Contributed by Akhil PB.
2019-08-09 16:05:02 +05:30
e0c21c6da9
YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak
2019-08-09 10:18:34 +02:00
742e30b473
YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak
2019-08-09 09:59:19 +02:00
72d7e570a7
YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak
2019-08-09 09:49:18 +02:00
9e6519a11a
YARN-9711. Missing spaces in NMClientImpl ( #1177 ) Contributed by Charles Xu.
2019-08-08 06:41:04 -07:00
3b38f2019e
YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang.
2019-08-08 12:37:50 +00:00
22d7d1f8bf
YARN-9601.Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations ( #908 ) Contributed by hunshenshi.
2019-08-07 21:26:14 -07:00
f51702d539
YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)
2019-08-06 13:52:30 -07:00
7c2042a44d
YARN-9678. Addendum: TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 10:21:55 -07:00
b8bf09ba3d
YARN-9678. TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 09:05:53 -07:00
1c53ce0cda
YARN-9410. Typo in documentation: Using FPGA On YARN ( #1220 ) Contributed by kevin su.
2019-08-06 06:29:58 -07:00
d6697da5e8
YARN-9667. Use setbuf with line buffer to reduce fflush complexity in container-executor.
...
Contributed by Peter Bacsko
2019-08-05 13:59:12 -04:00
54ac80176e
Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph
2019-08-02 13:38:06 +02:00
1930a7bf60
YARN-9093. Remove commented code block from the beginning of Tes… ( #444 )
2019-08-02 13:16:19 +02:00
cbcada804d
YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently ( #1121 )
2019-08-02 13:04:34 +02:00
95fc38f2e9
YARN-9375. Use Configured in GpuDiscoverer and FpgaDiscoverer ( #1131 )
...
Contributed by Adam Antal
2019-08-02 11:24:09 +02:00
42683aef1a
YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
2019-07-30 18:58:36 +00:00
9b8b3acb0a
HADOOP-16451. Update jackson-databind to 2.9.9.1. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-24 17:25:25 -07:00
9838a47d44
YARN-6046. Fixed documentation error in YarnApplicationSecurity.
...
Contributed by Yousef Abu-Salah
2019-07-18 12:39:18 -04:00
c34ceb5fde
YARN-9568. Fixed NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover.
...
Contributed by Steve Loughran
2019-07-18 12:30:53 -04:00
85d9111a88
YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam.
2019-07-17 10:28:20 +08:00
5915c902aa
YARN-9646. DistributedShell tests failed to bind to a local host name. (Contributed by Ray Yang)
2019-07-16 17:36:49 -07:00
7a93be0f60
YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T.
2019-07-16 14:03:22 +05:30
5446308360
YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal
2019-07-15 13:28:01 +02:00
18ee1092b4
YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko
2019-07-15 11:59:11 +02:00
91ce09e706
YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko
2019-07-15 10:47:20 +02:00
557056e18e
YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl
2019-07-12 17:40:14 +02:00
61b0c2bb7c
YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal
2019-07-12 17:28:14 +02:00
8b3c6791b1
YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko
2019-07-12 17:20:42 +02:00
c416284bb7
YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal
2019-07-12 16:51:58 +02:00
9b54dd7186
YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonathan Hung)
2019-07-11 13:57:08 -07:00
9cec023186
YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl
2019-07-11 16:11:21 +02:00
ccaa99c923
HADOOP-16381. The JSON License is included in binary tarball via azure-documentdb:1.16.2. Contributed by Sushil Ks.
2019-07-11 13:49:42 +09:00
555dabf4f8
YARN-9660. Update support documentation for Docker on YARN.
...
Contributed by Peter Bacsko
2019-07-10 17:15:33 -04:00
a2a8be18cb
YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal.
2019-07-03 13:45:00 +02:00
15d82fcb75
YARN-9658. Fix UT failures in TestLeafQueue. Contributed by Tao Yang.
2019-07-03 12:08:45 +08:00
e966edd025
YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt.
2019-07-02 12:18:16 +05:30
0c8813f135
YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt.
2019-07-02 11:44:11 +05:30
570eee30e5
YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi.
2019-07-02 09:55:25 +08:00
b1dafc3506
YARN-9661:Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java ( #1042 )
2019-07-01 10:46:33 -07:00
29465bf169
YARN-9560. Restructure DockerLinuxContainerRuntime to extend OCIContainerRuntime.
...
Contributed by Eric Badger, Jim Brennan, Craig Condit
2019-06-28 17:18:53 -04:00
f02b0e1994
YARN-9581. Add support for get multiple RM webapp URLs.
...
Contributed by Prabhu Joseph
2019-06-28 14:54:57 -04:00
f09c31a97e
Revert "YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023"
...
This reverts commit 5e7caf1287
2019-06-29 00:29:17 +08:00
5e7caf1287
YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023
2019-06-29 00:08:40 +08:00
cbae241320
YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang.
2019-06-28 23:24:53 +08:00
be80334cdf
YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T.
2019-06-27 09:59:44 +05:30
1ac967a6b7
YARN-6055. ContainersMonitorImpl need be adjusted when NM resource changed. Contributed by Inigo Goiri.
2019-06-26 14:01:31 -07:00
062eb605ac
YARN-9477. Implement VE discovery using libudev. Contributed by Peter Bacsko.
2019-06-26 23:53:14 +08:00
b220ec6f61
YARN-9374. Improve Timeline service resilience when HBase is unavailable.
...
Contributed by Prabhu Joseph and Szilard Nemeth
2019-06-24 12:19:14 -04:00
83dcb9d87e
YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
2019-06-21 17:41:05 +08:00
5bfdf62614
YARN-9631. Added ability to select JavaScript test or skip JavaScript tests for YARN application catalog.
...
Contributed by Eric Yang
(cherry picked from commit 6002b0c5c6994965d3f7231330248c093869dba2)
2019-06-19 18:48:16 -04:00
b14f0569bb
YARN-9574. Update hadoop-yarn-applications-mawo artifactId to match directory name.
...
Contributed by Wanqiang Ji
(cherry picked from commit 5f758a69ede8fafd214857a74f5d3b46198094c4)
2019-06-18 13:51:12 -04:00
eb6be4643f
YARN-9630. [UI2] Add a link in docs's top page
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-06-18 14:54:21 +09:00
67414a1a80
YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji.
2019-06-18 12:23:52 +08:00
304a47e22c
YARN-9608. DecommissioningNodesWatcher should get lists of running applications on node from RMNode. Contributed by Abhishek Modi.
2019-06-17 17:09:56 +08:00
cda9f33745
YARN-8499 ATSv2 Generalize TimelineStorageMonitor.
...
Contributed by Prabhu Joseph
2019-06-14 18:59:14 -04:00
3ba090f436
HADOOP-16366. Fixed ProxyUserAuthenticationFilterInitializer for timeline server.
...
Contributed by Prabhu Joseph
2019-06-14 12:54:16 -04:00
bcfd228336
YARN-9599. TestContainerSchedulerQueuing#testQueueShedding fails intermittently. Contributed by Abhishek Modi.
2019-06-13 11:08:35 -07:00
970b0b0c02
YARN-9578. Add limit/actions/summarize options for app activities REST API. Contributed by Tao Yang.
2019-06-13 10:44:47 +08:00
205dd2d8e1
HADOOP-16367. Fixed MiniYarnCluster AuthenticationFilter initialization.
...
Contributed by Prabhu Joseph
2019-06-12 18:03:33 -04:00
2263ead365
YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T.
2019-06-11 23:20:28 +05:30
60c95e9b6a
YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
2019-06-11 23:11:49 +05:30
6d80b9bc3f
YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie.
2019-06-11 22:49:21 +05:30
f7df55f4a8
YARN-9602. Use logger format in Container Executor. Contributed by Abhishek Modi.
2019-06-11 22:29:00 +05:30
9191e08f0a
YARN-9569. Auto-created leaf queues do not honor cluster-wide min/max memory/vcores. Contributed by Craig Condit.
2019-06-10 14:33:24 -07:00
e94e643584
YARN-9471. Cleanup in TestLogAggregationIndexFileController. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 12:18:15 -07:00
cb9bc6e64c
YARN-9581. Fixed yarn logs cli to access RM2.
...
Contributed by Prabhu Joseph
2019-06-06 16:41:58 -04:00
0976392502
YARN-9590. Correct incompatible, incomplete and redundant activities. Contributed by Tao Yang.
2019-06-06 21:59:01 +08:00
649666e118
YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal.
2019-06-06 14:51:55 +05:30
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
433e97cd34
YARN-9600. Support self-adaption width for columns of containers table on app attempt page. Contributed by Tao Yang.
2019-06-05 13:55:30 +08:00
d45669cd3c
YARN-7537. Add ability to load hbase config from distributed file system.
...
Contributed by Prabhu Joseph
2019-06-04 19:26:06 -04:00
606061aa14
YARN-9595. FPGA plugin: NullPointerException in FpgaNodeResourceUpdateHandler.updateConfiguredResource(). Contributed by Peter Bacsko.
2019-06-04 09:56:59 +08:00
bd2590d71b
YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
2019-06-03 22:59:02 +08:00
59719dc560
YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB.
2019-06-03 15:53:23 +05:30
7f46dda513
YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB.
2019-06-03 12:24:35 +05:30
4530f4500d
YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T.
2019-06-03 14:09:37 +08:00
2210897609
YARN-9592. Use Logger format in ContainersMonitorImpl. Contributed by Inigo Goiri.
2019-05-31 17:35:49 -07:00
4cb559ea7b
YARN-9027. Fixed LevelDBCacheTimelineStore initialization.
...
Contributed by Prabhu Joseph
2019-05-31 14:31:44 -04:00
52128e352a
YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B.
2019-05-31 12:29:44 +05:30
e49162f4b3
YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl.
2019-05-31 10:28:09 +05:30
7861a5eb1a
YARN-9033. ResourceHandlerChain#bootstrap is invoked twice during NM start if LinuxContainerExecutor enabled. Contributed by Zhankun Tang.
2019-05-31 10:22:26 +05:30
f1552f6edb
YARN-9553. Fix NPE in EntityGroupFSTimelineStore#getEntityTimelines. Contributed by Prabhu Joseph.
2019-05-30 11:42:27 -07:00
30c6dd92e1
YARN-9452. Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures. Contributed by Prabhu Joseph.
2019-05-30 22:32:41 +05:30
abf76ac371
YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2019-05-29 11:24:08 -05:00
3c63551101
YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph.
2019-05-29 16:05:39 +00:00
544876fe12
YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang.
2019-05-29 16:34:48 +08:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
4a692bc3be
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang.
2019-05-28 16:53:43 +09:00
9078e28a24
YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji.
2019-05-28 15:52:39 +09:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
9f056d905f
YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang.
2019-05-26 09:56:36 -04:00
460ba7fb14
YARN-9558. Fixed LogAggregation test cases.
...
Contributed by Prabhu Joseph
2019-05-23 18:38:47 -04:00
6a0e7dd454
YARN-9482. DistributedShell job with localization fails in unsecure cluster. Contributed by Prabhu Joseph.
2019-05-23 13:27:52 -07:00
e2b79912c3
YARN-9469. Fix typo in YarnConfiguration.
2019-05-23 19:00:38 +02:00
7b03072fd4
YARN-9080. Added clean up of bucket directories.
...
Contributed by Prabhu Joseph, Peter Bacsko, Szilard Nemeth
2019-05-23 12:08:44 -04:00
a85451c8d2
YARN-9575. Fix TestYarnConfigurationFields testcase failing. Contributed by Prabhu Joseph.
2019-05-21 11:25:30 -07:00
24c53e057a
YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak.
2019-05-20 10:53:01 -04:00
12c81610e0
YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-05-17 12:03:21 -07:00
fab5b80a36
YARN-9554. Fixed TimelineEntity DAO serialization handling.
...
Contributed by Prabhu Joseph
2019-05-16 16:39:50 -04:00
55bd35921c
YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko.
2019-05-15 11:50:46 -07:00
570fa2da20
YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T.
2019-05-15 13:30:09 +05:30
2de1e30658
YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T.
2019-05-15 13:21:39 +05:30
7d831eca64
YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal.
2019-05-14 10:48:08 -07:00
206e633946
YARN-9453. Clean up code long if-else chain in ApplicationCLI#run. Contributed by Wanqiang Ji.
2019-05-13 11:11:20 -07:00
29ff7fb140
YARN-9493. Scheduler Page does not display the right page by query string. Contributed by Wanqiang Ji.
2019-05-13 10:57:12 -07:00
1a47c2b7ae
YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang.
2019-05-12 22:31:39 -07:00
64c7f36ab1
YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl.
2019-05-10 14:27:23 +05:30
1b48100a5e
YARN-9522. AppBlock ignores full qualified class name of PseudoAuthenticationHandler. Contributed by Prabhu Joseph.
2019-05-09 14:02:58 -07:00
ec36126346
YARN-9483. DistributedShell does not release container when failed to localize at launch. Contributed by Prabhu Joseph.
2019-05-09 10:59:31 -07:00
90add05caa
YARN-9489. Support filtering by request-priorities and allocation-request-ids for query results of app activities. Contributed by Tao Yang.
2019-05-09 21:54:09 +08:00
0c5fa2e7d9
YARN-9535. Fixed spell check for container-executor in Docker documentation.
...
Contributed by Charan Hebri
2019-05-08 16:56:26 -04:00
3172f6cbf9
YARN-9513. Addendum patch: Fix ASF License warnings. Contributed by Giovanni Matteo Fumarola.
2019-05-08 14:56:23 +09:00
c336af3847
YARN-9432. Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled. Contributed by Tao Yang.
2019-05-08 09:54:16 +08:00
66c2a4ef89
HADOOP-16270. [JDK 11] Remove unintentional override of the version of Maven Dependency Plugin. Contributed by Xieming Li.
2019-05-08 10:50:07 +09:00
8ecbf61cca
YARN-9513. [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8. Contributed by Adam Antal.
2019-05-07 10:59:02 -07:00
49e1292ea3
YARN-9524. Fixed TestAHSWebService and TestLogsCLI unit tests.
...
Contributed by Prabhu Joseph
2019-05-06 19:48:45 -04:00
597fa47ad1
YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung)
2019-05-06 11:56:22 -07:00
12b7059ddc
YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang.
2019-05-06 20:00:15 +08:00
d331a2a0c2
YARN-9528. Federation RMs starting up at the same time can give duplicate application IDs. Contributed by Young Chen.
2019-05-03 15:05:17 -07:00
b094b94d43
YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein.
2019-05-02 19:39:26 +00:00
accb811e57
YARN-6929. Improved partition algorithm for yarn remote-app-log-dir.
...
Contributed by Prabhu Joseph
2019-04-30 17:04:59 -04:00
7fbaa7d66f
YARN-9476. [YARN-9473] Create unit tests for VE plugin. Contributed by Peter Bacsko.
2019-04-30 11:06:44 +08:00
79d3d35398
YARN-9486. Docker container exited with failure does not get clean up correctly. Contributed by Eric Yang
2019-04-26 01:21:28 +00:00
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
3f2f4186f6
YARN-9424. Change getDeclaredMethods to getMethods in FederationClientInterceptor#invokeConcurrent. Contributed by Shen Yinjie.
2019-04-23 19:58:41 -07:00
9d400627c2
YARN-9081. Update jackson from 1.9.13 to 2.x in hadoop-yarn-services-core.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-04-24 09:50:25 +09:00
fec9bf4b0b
YARN-9501. TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:42:56 -07:00
4a0ba24959
YARN-9491. TestApplicationMasterServiceFair#ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:27:04 -07:00
c504eee0c2
YARN-9339. Apps pending metric incorrect after moving app to a new queue. Contributed by Abhishek Modi.
2019-04-23 12:40:44 -07:00
8a95ea61e1
YARN-9475. [YARN-9473] Create basic VE plugin. Contributed by Peter Bacsko.
2019-04-23 17:33:58 +08:00
1c8046d67e
YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 14:21:13 +08:00
a54c1e3ace
YARN-8551. Project setup for MaWo application.
...
Contributed by Yesha Vora
2019-04-22 16:56:26 -04:00
96e3027e46
YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi.
2019-04-22 09:49:03 -07:00
adefd37105
YARN-9495. Fix findbugs warnings in hadoop-yarn-server-resourcemanager module. Contributed by Tao Yang.
2019-04-20 21:51:00 +08:00
317fcbabe3
YARN-9254. Add support for storing application catalog data to HDFS. Contributed by Eric Yang
2019-04-19 10:29:57 -07:00
aeadb9432f
YARN-9448. Fix Opportunistic Scheduling for node local allocations. Contributed by Abhishek Modi.
2019-04-19 09:41:06 -07:00
ef97a20831
YARN-8622. Fixed container-executor compilation on MacOSX.
...
Contributed by Siyao Meng
2019-04-18 18:59:21 -04:00
df76cdc895
YARN-6695. Fixed NPE in publishing appFinished events to ATSv2.
...
Contributed by Prabhu Joseph
2019-04-18 12:29:37 -04:00
aa4c744aef
YARN-9470. Fix order of actual and expected expression in assert statements
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-18 15:40:37 +09:00
6e4399ea61
YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-17 22:56:57 -07:00
9cf7401794
YARN-9349. Improved log level practices for InvalidStateTransitionException.
...
Contributed by Anuhan Torgonshar
(cherry picked from commit fe2370e039e1ee980d74769ae85d67434e0993cf)
2019-04-16 19:53:45 -04:00
b8086aed86
YARN-9123. Clean up and split testcases in TestNMWebServices for GPU support. Contributed by Szilard Nemeth.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-16 11:06:25 -07:00
2364c7d0bf
YARN-9466. Fixed application catalog navigation bar height in Safari. Contributed by Eric Yang
2019-04-16 10:04:27 -07:00
ad865888a6
YARN-8530. Add SPNEGO filter to application catalog. Contributed by Eric Yang
2019-04-16 09:52:14 -07:00
c65218ee8e
YARN-8943. Upgrade JUnit from 4 to 5 in hadoop-yarn-api.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-04-16 12:13:49 +09:00
5583e1b6fc
YARN-7848 Force removal of docker containers that do not get removed on first try. Contributed by Eric Yang
2019-04-15 20:47:09 +00:00
254efc9358
YARN-9379. Can't specify docker runtime through environment. Contributed by caozhiqiang
2019-04-15 18:24:37 +00:00
7fa73fac26
YARN-9439. Support asynchronized scheduling mode and multi-node lookup mechanism for app activities. Contributed by Tao Yang.
2019-04-16 00:12:43 +08:00
c4c16cae7d
YARN-9481. [JDK 11] Build fails due to hard-coded target version in hadoop-yarn-applications-catalog-webapp. Contributed by Kei Kori.
2019-04-15 13:08:28 +09:00
7a68e7abd5
YARN-9474. Remove hard coded sleep from Opportunistic Scheduler tests. Contributed by Abhishek Modi.
2019-04-14 20:11:20 -07:00
b2cdf809bc
YARN-9281. Add express upgrade button to Appcatalog UI. Contributed by Eric Yang
2019-04-13 18:55:11 +03:00
1943db5571
HADOOP-16237. Fix new findbugs issues after updating guava to 27.0-jre.
...
Author: Gabor Bota <gabor.bota@cloudera.com>
2019-04-12 18:28:38 -07:00
ed3747c1cc
YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi.
2019-04-11 11:49:19 -07:00
8c1bba375b
YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu.
2019-04-10 22:51:28 +08:00
32722d2661
YARN-9433. Remove unused constants in YARN resource manager
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-10 18:37:27 +09:00
cfec455c45
YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri.
2019-04-09 10:59:43 -07:00
fc05b0e70e
YARN-9313. Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities. Contributed by Tao Yang.
2019-04-08 13:40:53 +08:00
ec143cbf67
YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang.
2019-04-06 19:59:36 +08:00
22362c876d
YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi.
2019-04-05 12:06:51 -07:00
27039a29ae
YARN-9382 Publish container killed, paused and resumed events to ATSv2. Contributed by Abhishesk Modi.
2019-04-05 12:02:43 -07:00
2bf82aee82
YARN-9441. Updated YARN app catalog name for consistency.
...
Contributed by Weiwei Yang
2019-04-04 13:21:30 -04:00
8d150067e2
YARN-9396. Fixed duplicated RM Container created event to ATS.
...
Contributed by Prabhu Joseph
2019-04-04 13:01:56 -04:00
945e8c6064
YARN-9394. Use new API of RackResolver to get better performance. Contributed by Lantao Jin.
2019-04-04 18:05:01 +08:00
eb03f7c419
YARN-9303 Username splits won't help timelineservice.app_flow table. Contributed by Prabhu Joseph.
2019-04-03 22:53:05 -07:00
d7979079ea
HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
2019-04-03 12:59:39 -06:00
002dcc4ebf
YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
2019-04-03 18:57:28 +05:30
2f752830ba
YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji.
2019-04-01 20:05:15 -07:00
ab2bda57bd
YARN-9428. Add metrics for paused containers in NodeManager. Contributed by Abhishek Modi.
2019-04-01 14:21:17 -07:00
da7f8c244d
YARN-9431. Fix flaky junit test fair.TestAppRunnability after YARN-8967. Contributed by Wilfred Spiegelenburg.
2019-04-01 11:21:31 -07:00
332cab5518
YARN-9418. ATSV2 /apps//entities/YARN_CONTAINER rest api does not show metrics. Contributed by Prabhu Joseph.
2019-04-01 11:06:51 -07:00
b0d24ef39c
YARN-9227. DistributedShell RelativePath is not removed at end. Contributed by Prabhu Joseph.
2019-04-01 10:59:14 -07:00
35b0a381e7
YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
2019-04-01 09:21:36 -07:00
d9e9e5657d
YARN-9348. Application catalog build system bug fixes. Contributed by Eric Yang
2019-03-29 13:51:21 -07:00
7ee32a2eb1
YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
2019-03-29 13:51:21 -07:00
56f1e131ec
YARN-9270. Minor cleanup in TestFpgaDiscoverer. Contributed by Peter Bacsko.
2019-03-29 10:58:56 -07:00
9cd66198ee
YARN-9411. TestYarnNativeServices fails sporadically with bind address in use. Contributed by Prabhu Joseph.
2019-03-27 10:44:54 -07:00
a4cd75e09c
YARN-9269. Minor cleanup in FpgaResourceAllocator. Contributed by Peter Bacsko.
2019-03-27 10:08:07 -07:00
5257f50abb
YARN-8967. Change FairScheduler to use PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-25 22:47:24 -07:00
710cbc9bd6
YARN-9405. Fixed flaky tests in TestYarnNativeServices.
...
Contributed by Prabhu Joseph
2019-03-25 16:34:04 -04:00
eeda6891e4
YARN-9268. General improvements in FpgaDevice. Contributed by Peter Bacsko.
2019-03-25 13:22:53 -07:00
3c45762a0b
YARN-9391. Fixed node manager environment leaks into Docker containers.
...
Contributed by Jim Brennan
2019-03-25 15:53:24 -04:00
509b20b292
YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent. Contributed by Prabhu Joseph.
2019-03-22 11:45:39 -07:00
ce5eb9cb2e
YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322
...
(Contributed by Zoltan Siegl via Daniel Templeton)
Change-Id: I92d52c0ca630e71afb26b2b7587cbdbe79254a05
2019-03-22 12:28:34 +01:00
548997d6c9
YARN-9402. Opportunistic containers should not be scheduled on Decommissioning nodes. Contributed by Abhishek Modi.
2019-03-21 12:04:05 -07:00
a99eb80659
YARN-9267. General improvements in FpgaResourceHandlerImpl. Contributed by Peter Bacsko.
2019-03-21 11:15:56 -07:00
506502bb83
YARN-9370. Added logging for recovering assigned GPU devices.
...
Contributed by Yesha Vora
2019-03-20 19:12:19 -04:00
f2b862cac6
YARN-9398. Fixed javadoc errors for FPGA related java files.
...
Contributed by Peter Bacsko
2019-03-20 15:45:37 -04:00
b3b0e332e6
YARN-9299. TestTimelineReaderWhitelistAuthorizationFilter ignores Http Errors. Contributed by Prabhu Joseph.
2019-03-20 21:24:31 +05:30
0d24684eee
YARN-9357. Modify HBase Liveness monitor log to debug. Contributed by Prabhu Joseph.
2019-03-20 21:22:54 +05:30
f6e60e966e
YARN-9387. Update document for ATS HBase Custom tablenames (-entityTableName). Contributed by Prabhu Joseph.
2019-03-20 21:20:50 +05:30
c1a4eeb7c8
YARN-9389. FlowActivity and FlowRun table prefix is wrong. Contributed by Prabhu Joseph.
2019-03-20 21:18:19 +05:30
5d8bd0e5cb
YARN-9392. Handle missing scheduler events in Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-03-19 11:00:21 -07:00
c9e50c48de
YARN-9365. Fix table creation command in TimelineServiceV2.md
2019-03-19 12:05:37 +05:30
09eabda314
YARN-9364. Remove commons-logging dependency from YARN.
...
Contributed by Prabhu Joseph
2019-03-18 19:58:42 -04:00
5f6e225166
YARN-9363. Replaced debug logging with SLF4J parameterized log message.
...
Contributed by Prabhu Joseph
2019-03-18 13:57:18 -04:00
19b22c4385
YARN-9385. Fixed ApiServiceClient to use current UGI.
...
Contributed by Eric Yang
2019-03-18 13:16:34 -04:00
0e7e9013d4
YARN-9340. [Clean-up] Remove NULL check before instanceof in ResourceRequestSetKey
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I932e29b36f086f7b7c76a250e33b473617ddbda1
2019-03-18 15:08:37 +01:00
2064ca015d
YARN-9349. Changed logging to use slf4j api.
...
Contributed by Prabhu Joseph
2019-03-15 19:20:59 -04:00
03f3c8aed2
YARN-4404. Corrected typo in javadoc.
...
Contributed by Yesha Vora
2019-03-15 18:04:04 -04:00
688b177fc6
YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang
2019-03-14 19:39:00 +00:00
f235a942d5
YARN-9016 DocumentStore as a backend for ATSv2. Contributed by Sushil Ks.
2019-03-13 16:45:23 -07:00
17a3e14d25
YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi.
2019-03-12 21:33:17 -07:00
67cc24a7a4
YARN-9381. The yarn-default.xml has two identical property named yarn.timeline-service.http-cross-origin.enabled. Contributed by Abhishek Modi.
2019-03-12 15:21:03 -07:00
8e1539eca8
YARN-9266. General improvements in IntelFpgaOpenclPlugin. Contributed by Peter Bacsko.
2019-03-13 02:45:17 +05:30
ab574ffd72
Revert "YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang"
...
This reverts commit 2576401928
2019-03-12 11:50:16 -07:00
55069bfb09
Revert "YARN-9348. Application catalog build system bug fixes."
...
This reverts commit 01ada40ea4
2019-03-12 11:49:55 -07:00
024b3bae8a
Revert "YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang"
...
This reverts commit aab7b77536
2019-03-12 11:48:55 -07:00
aab7b77536
YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
2019-03-12 10:06:09 -07:00
c24af4b0d6
YARN-9336. JobHistoryServer leaks CLOSE_WAIT tcp connections when using LogAggregationIndexedFileController. Contributed by Tarun Parimi.
2019-03-12 20:57:27 +05:30
fa7a0b269a
YARN-8805. Automatically convert the launch command to the exec form when using entrypoint support
2019-03-11 12:55:01 -05:00
de15a66d78
YARN-9265. FPGA plugin fails to recognize Intel Processing Accelerator Card. Contributed by Peter Bacsko.
2019-03-08 17:39:22 +05:30
373705fcea
YARN-9239. Document docker registry deployment with Ozone CSI driver. Contributed by Eric Yang.
2019-03-08 09:47:20 +08:00
39b4a37e02
YARN-9341. Fixed enentrant lock usage in YARN project.
...
Contributed by Prabhu Joseph
2019-03-07 16:47:45 -05:00
491313ab84
YARN-8218 Add application launch time to ATSV1. Contributed by Abhishek Modi
2019-03-06 21:47:29 -08:00
01ada40ea4
YARN-9348. Application catalog build system bug fixes.
...
Contributed by Eric Yang
2019-03-06 17:55:48 -05:00
c79f139519
YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB.
2019-03-06 16:49:08 +05:30
46045c5cb3
YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth.
2019-03-06 16:01:08 +05:30
7b42e0e32a
YARN-7266. Fixed deadlock in Timeline Server thread initialization.
...
Contributed by Prabhu Joseph
2019-03-05 12:17:01 -05:00
2576401928
YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
2019-03-05 06:14:14 -08:00
0aefe2846f
YARN-9298. Implement FS placement rules using PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-04 23:49:07 -08:00
e40e2d6ad5
YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-03-05 14:10:08 +09:00
15098df744
Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph."
...
This reverts commit a63c358b78
2019-03-04 16:57:31 +05:30
bd8d299ded
YARN-7477. Moving logging APIs over to slf4j in hadoop-yarn-common. Contributed by Prabhu Joseph.
2019-03-04 16:59:20 +09:00
e20b5ef52c
YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin.
2019-03-04 16:59:20 +09:00
cab8529ecb
YARN-7904. Privileged, trusted containers should be supported only in ENTRYPOINT mode. Contributed by Eric Yang.
2019-03-01 11:06:09 -08:00
dcaca19871
YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth.
2019-03-01 19:24:35 +05:30
538bb4880d
YARN-9323. FSLeafQueue#computeMaxAMResource does not override zero values for custom resources
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: Id844ccf09488f367c0c7de0a3b2d4aca1bba31cc
2019-02-27 19:59:48 -08:00
7b928f19a4
YARN-9322. Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: I14c12f1265999d62102f2ec5506d90015efeefe8
2019-02-27 19:43:50 -08:00
1779fc57a1
YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph.
2019-02-28 09:56:29 +08:00
04b228e43b
YARN-9334. Allow YARN Service client to send SPNEGO challenge header when authentication type is not simple.
...
Contributed by Billie Rinaldi
2019-02-27 18:47:14 -05:00
ea3cdc60b3
YARN-3841 [atsv2 Storage implementation] Adding retry semantics to HDFS backing storage. Contributed by Abhishek Modi.
2019-02-27 14:55:35 -08:00
0ec962ac8f
YARN-5336 Limit the flow name size & consider cleanup for hex chars. Contributed by Sushil Ks
2019-02-27 14:43:39 -08:00
fbc7bb315f
YARN-9245. Added query docker image command ability to node manager.
...
Contributed by Chandni Singh
2019-02-27 14:57:24 -05:00
29e27faf96
YARN-9318. Resources#multiplyAndRoundUp does not consider Resource Types
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: Ia45f528574c2b054f6f764d1d140e592bdb7e217
2019-02-27 10:29:17 -08:00
8c30114b00
YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie.
2019-02-27 17:29:02 +08:00
6c96f5e4b6
YARN-8378. ApplicationHistoryManagerImpl#getApplications doesn't honor filters. Contributed by Lantao Jin.
2019-02-27 10:32:58 +05:30
8eae260af5
YARN-9311. Fix TestRMRestart hangs. Contributed by Prabhu Joseph.
2019-02-27 10:28:16 +05:30
a106d2dc9d
YARN-8783. Improve the documentation for the docker.trusted.registries configuration. Contributed by Eric Yang.
2019-02-26 10:19:06 -08:00
c6ea28c480
YARN-9331. [YARN-8851] Fix a bug that lacking cgroup initialization when bootstrap DeviceResourceHandlerImpl. Contributed by Zhankun Tang.
2019-02-26 10:05:31 +08:00
95372657fc
YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena.
2019-02-25 11:45:37 -08:00
3e1739d589
YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang.
2019-02-26 00:08:13 +08:00
6cec90653d
YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang.
2019-02-25 15:28:45 +05:30
5e91ebd91a
YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth.
2019-02-25 11:30:46 +05:30
9cd5c5447f
YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph.
2019-02-24 22:42:27 +08:00
50094d7fef
YARN-9300. Lazy preemption should trigger an update on queue preemption metrics for CapacityScheduler. Contributed by Tao Yang.
2019-02-24 22:17:29 +08:00
dddcfa4d9f
YARN-8821. [YARN-8851] GPU hierarchy/topology scheduling support based on pluggable device framework. Contributed by Zhankun Tang.
2019-02-24 14:37:06 +05:30
05bce33d6e
YARN-9244. Document docker registry deployment with direct S3 driver.
...
Contributed by Suma Shivaprasad
2019-02-22 19:13:52 -05:00
ed13cf8406
YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph
2019-02-22 21:01:34 +05:30
95fbbfed75
YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth.
2019-02-22 20:22:17 +05:30
9636fe4114
YARN-8891. Documentation of the pluggable device framework. Contributed by Zhankun Tang.
2019-02-22 20:00:13 +05:30
9c88695bcd
YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie.
2019-02-22 20:58:33 +08:00
632d5e8a98
YARN-9156. Improve debug message in device plugin method compatibility check of ResourcePluginManager. Contributed by Zhankun Tang.
2019-02-22 11:27:30 +08:00
b17a2602d1
YARN-9319. Fix compilation issue of handling typedef an existing name by gcc compiler. (Zhankun Tang via wangda)
...
Change-Id: Ie8293cc5b710ff355fb6a1ea7201be53ed0385bf
2019-02-21 11:17:32 -08:00
6c4ab0312b
YARN-9258. Support to specify allocation tags without constraint in distributed shell CLI. Contributed by Prabhu Joseph.
2019-02-22 00:18:07 +08:00
eedcc8e26e
YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph.
2019-02-21 17:21:21 +08:00
a63c358b78
YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph.
2019-02-21 13:29:01 +05:30
b8de78c570
YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T.
2019-02-20 01:19:20 +05:30
db4d1a1e2f
YARN-9060. [YARN-8851] Phase 1 - Support device isolation and use the Nvidia GPU plugin as an example. Contributed by Zhankun Tang.
2019-02-18 15:58:04 +05:30
0f2b65c3da
HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
2019-02-18 15:21:46 +09:00
ba56bc24f5
YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth.
2019-02-17 20:08:20 +05:30
9057aa9fc5
SUBMARINE-1. Move code base of submarine from yarn-applications to top directory. Contributed by Wangda Tan.
2019-02-17 20:03:56 +05:30
7c1b561e33
YARN-8927. Added support for top level Dockerhub images to trusted registry using library keyword.
...
Contributed by Zhankun Tang
2019-02-15 11:18:07 -05:00
9385ec45d7
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-02-15 18:47:10 +09:00
5b55f3538c
YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri.
2019-02-15 12:42:21 +05:30
3a39d9a2d2
YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka.
2019-02-15 15:41:53 +09:00
b66d5ae9e2
YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri.
2019-02-14 23:08:19 +05:30
dfe0f42835
YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB.
2019-02-14 15:42:19 +05:30
3dc2523266
YARN-9184. Add a system flag to allow update to latest docker images.
...
Contributed by Zhaohui Xin
2019-02-12 16:16:35 -05:00
7536488bbd
YARN-996. REST API support for node resource configuration. Contributed by Inigo Goiri.
2019-02-11 15:27:43 -08:00
1ce2e91c4b
YARN-9229. Document docker registry deployment with NFS Gateway. Contributed by Eric Yang.
2019-02-11 14:54:30 -08:00
0a1637c750
YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph.
2019-02-11 15:53:50 +08:00
2b7f828d46
YARN-9252. Allocation Tag Namespace support in Distributed Shell. Contributed by Prabhu Joseph.
2019-02-11 14:42:21 +08:00
e0ab1bdece
YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri.
2019-02-09 00:26:50 +09:00
d1ca9432dd
YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein
2019-02-07 16:38:11 +00:00
3c96a03dea
YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena.
2019-02-07 13:02:35 +05:30
7fa62e150c
YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi.
2019-02-07 09:58:53 +05:30
1e5e08d83b
YARN-7627. [ATSv2] When passing a non-number as metricslimit, the error message is wrong. Contributed by Charan Hebri.
2019-02-07 09:58:53 +05:30
711d22f166
YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph.
2019-02-06 22:36:51 +08:00
308f3168fa
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
2044967e75
YARN-9246 NPE when executing a command yarn node -status or -states without additional arguments. Contributed by Masahiro Tanaka
2019-02-05 09:39:38 -08:00
e3ec18b0c4
YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi.
2019-02-05 13:48:04 +05:30
ba38db4f5b
YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri.
2019-02-05 13:41:44 +05:30
aa7ce50e55
YARN-9275. Add link to NodeAttributes doc in PlacementConstraints document. Contributed by Masatake Iwasaki.
2019-02-05 16:07:41 +08:00
571838920d
YARN-9149. yarn container -status misses logUrl when integrated with ATSv2. Contributed by Abhishek Modi.
2019-02-05 12:39:10 +05:30
604b2489a9
YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla.
2019-02-04 12:48:28 +05:30
69bcff380f
YARN-9231. TestDistributedShell fix timeout. Contributed by Prabhu Joseph.
2019-02-02 23:27:31 +08:00
f20b043a02
YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang.
2019-02-02 23:04:34 +08:00
28ad20a711
YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie.
2019-02-02 23:02:45 +08:00
2c13513040
YARN-8549 Adding a NoOp timeline writer and reader plugin classes for ATSv2. Contributed by Prabha Manepalli.
2019-02-01 14:26:50 -08:00
f738b397ae
YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi.
2019-01-31 11:24:15 -08:00
71c49fa60f
YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth.
2019-01-31 09:25:29 +05:30
a3a9ae3cea
YARN-9251. Build failure for -Dhbase.profile=2.0. Contributed by Rohith Sharma K S.
2019-01-30 05:17:04 -08:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
04105bbfdb
YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph
2019-01-29 17:22:24 +00:00
085f0e8ae7
YARN-9086. [CSI] Run csi-driver-adaptor as aux service. Contributed by Weiwei Yang.
2019-01-29 14:53:08 +08:00
4f63ffe444
YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang.
2019-01-29 10:34:26 +08:00
f5a95f7998
YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency.
...
Contributed by Suma Shivaprasad
2019-01-28 18:10:33 -05:00
2e636dd3c4
YARN-9074. Consolidate docker removal logic in ContainerCleanup.
...
Contributed by Zhaohui Xin
2019-01-28 18:05:53 -05:00
1ab69a9543
YARN-9221. Added flag to disable dynamic auxiliary service feature.
...
Contributed by Billie Rinaldi
2019-01-25 19:05:36 -05:00
6cace58e21
YARN-9222. Print launchTime in ApplicationSummary
2019-01-25 13:17:00 -08:00
84bb9808a2
YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB.
2019-01-25 14:00:35 +05:30
a33ef4fd31
YARN-8867. Added resource localization status to YARN service status call.
...
Contributed by Chandni Singh
2019-01-24 18:43:21 -05:00
a4bd64e724
YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu.
2019-01-24 17:53:43 +08:00
c726445990
YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB
2019-01-24 15:01:43 +05:30
721d5c2a5f
YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph.
2019-01-23 17:59:36 +08:00
e3e076dc78
YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang.
2019-01-23 14:14:49 +08:00
2fa9389c2e
YARN-9146. Added REST API to configure auxiliary service.
...
Contributed by Billie Rinaldi
2019-01-22 18:24:43 -05:00
d43df31751
YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang.
2019-01-22 10:23:32 +08:00
abde1e1f58
YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang.
2019-01-21 16:54:14 +08:00
27aa6e8899
HADOOP-16046. [JDK 11] Correct the compiler exclusion of org/apache/hadoop/yarn/webapp/hamlet/** classes for >= Java 9. Contributed by Devaraj K.
2019-01-21 10:54:58 +09:00
824dfa3b09
YARN-8489. Support "dominant" component concept in YARN service.
...
Contributed by Zac Zhou
2019-01-18 20:23:04 -05:00
8c7f6b2d4d
YARN-9197. Add safe guard against NPE for component instance failure.
...
Contributed by kyungwan nam
2019-01-18 13:49:56 -05:00
6d7eedfd28
YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda)
...
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
2019-01-17 14:20:10 -08:00
0a46baecd3
YARN-9203. Fix typos in yarn-default.xml.
...
This closes #437
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-01-17 16:18:33 +09:00
104ef5df36
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma.
2019-01-16 14:36:36 +09:00
713ded6b15
YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
2019-01-15 21:28:10 -08:00
6be39230a6
YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
2019-01-15 21:25:37 -08:00
614af50625
YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo.
2019-01-15 12:59:59 +09:00
6a923464af
YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R
2019-01-08 16:54:05 -06:00
0921b706f7
YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan.
2019-01-08 14:30:53 +08:00
489411579c
YARN-9169. Add metrics for queued opportunistic and guaranteed containers. Contributed by Abhishek Modi.
2019-01-07 15:16:55 -08:00
0a01d49917
YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda)
...
Change-Id: Id8af27134d3286a7a10d85eda9be25df9689d0e7
2019-01-07 12:07:26 -08:00
5db7c49062
YARN-9162. Fix TestRMAdminCLI#testHelp. Contributed by Ayush Saxena.
2019-01-07 17:32:28 +09:00
944cf87223
YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg.
2019-01-07 15:57:31 +08:00
d3321fbed0
Revert "YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi."
...
This reverts commit 650b9cbe16
2019-01-07 08:58:09 +08:00
2c02aa6ec2
YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda)
...
Change-Id: I115b6710eb11501e21ab3e9e7cd1a87f322e5cf8
2019-01-06 10:25:35 -08:00
8f004fe4e0
YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda)
...
Change-Id: Ib2b17cf7860ddc9188f06498086de3326bc09519
2019-01-06 10:24:29 -08:00
650b9cbe16
YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi.
2019-01-06 11:29:59 +08:00
6e35f7130f
YARN-9166. Fix logging for preemption of Opportunistic containers for Guaranteed containers. Contributed by Abhishek Modi.
2019-01-04 11:16:41 -08:00
8c6978c3ba
YARN-6149. Allow port range to be specified while starting NM Timeline collector manager. Contributed by Abhishek Modi.
2019-01-04 14:07:02 +05:30
573b158791
YARN-8567. Fetching yarn logs fails for long running application if it is not present in timeline store. Contributed by Tarun Parimi.
2019-01-04 14:01:35 +05:30
f4906ac019
YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang.
2019-01-04 12:10:00 +05:30
dfceffa70d
YARN-9147. Rmove auxiliary services when manifest file is removed.
...
Contributed by Billie Rinaldi
2019-01-03 12:57:21 -05:00
cfe89e6f96
YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie.
2019-01-03 23:56:28 +08:00
128f340f4f
YARN-8894. Improve InMemoryPlan#toString. Contributed by BELUGA BEHR.
2018-12-28 17:43:47 +09:00
15fe1b34dd
YARN-8878. Remove StringBuffer from ManagedParentQueue.java. Contributed by BELUGA BEHR.
2018-12-28 17:29:22 +09:00
742fd978b5
YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu.
2018-12-26 08:26:48 +05:30
091ad974cd
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
657aa433e2
YARN 9108. Fix FederationIntercepter merge home and secondary allocate response typo. Contributed by Abhishek Modi.
2018-12-22 12:41:49 -08:00
e98e963e51
YARN-9154. Fix itemization in YARN service quickstart document. Contributed by Ayush Saxena.
2018-12-22 13:25:39 +09:00
f82922dcfa
YARN-5168. Added exposed port information for Docker container.
...
Contributed by Xun Liu
2018-12-21 19:44:07 -05:00
ea724181d6
YARN-9132. Added file permission check for auxiliary services manifest file.
...
Contributed by Billie Rinaldi
2018-12-21 14:56:39 -05:00
f659485ee8
YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang.
2018-12-21 10:56:42 +08:00
a80d321074
YARN-9152. Improved AuxServices REST API output.
...
Contributed by Billie Rinaldi
2018-12-20 19:21:55 -05:00
7affa3053c
YARN-9131. Updated document usage for dynamic auxiliary service.
...
Contributed by Billie Rinaldi
2018-12-20 18:45:49 -05:00
5df9fb16b9
YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed by Rong Tang.
2018-12-19 18:58:00 -08:00
2e544dc921
YARN-9129. Ensure flush after printing to log plus additional cleanup. Contributed by Eric Yang
2018-12-19 16:45:23 -08:00
e815fd9c49
YARN-9126. Fix container clean up for reinitialization.
...
Contributed by Chandni Singh
2018-12-19 14:55:56 -05:00
ccdd982e51
YARN-9084. Reset container state and defer readiness check for upgrade.
...
Contributed by Chandni Singh
2018-12-18 18:02:03 -05:00
c7a5a4435e
YARN-9075. Add ability to register/remove auxiliary service without restart node manager.
...
Contributed by Billie Rinaldi
2018-12-18 17:05:51 -05:00
a6e9d27c29
YARN-9117. Add a check for insecure setup for container terminal. Contributed by Eric Yang
2018-12-18 12:49:10 -08:00
c5c73182e5
YARN-9072. Send exit command to terminate docker exec on connection close. Contributed by Eric Yang
2018-12-18 10:06:33 -08:00
71e0b0d800
YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator.
...
Contributed by Tarun Parimi
2018-12-17 12:04:25 -05:00
04c03476c9
YARN-9095. Removed Unused field from Resource: NUM_MANDATORY_RESOURCES
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-12-15 16:07:27 +09:00
4f622337ee
YARN-9091. Improve terminal message when connection is refused. Contributed by Eric Yang
2018-12-14 19:55:50 -08:00
42297f7d99
YARN-8963. Add flag to disable interactive shell. Contributed by Eric Yang
2018-12-14 18:44:36 -08:00
aab310978f
YARN-9089. Add Terminal Link to Service component instance page for UI2. Contributed by Eric Yang
2018-12-14 18:38:03 -08:00
26e55d4ceb
YARN-9122. Add table of contents to YARN Service API document. Contributed by Zhankun Tang.
2018-12-15 10:29:47 +09:00
b2d7204ed0
YARN-9125. Fixed Carriage Return detection in Docker container launch command.
...
Contributed by Billie Rinaldi
2018-12-14 17:52:26 -05:00
ca90b2480e
YARN-9032. Support sh shell for interactive container shell at command line. Contributed by Eric Yang
2018-12-14 14:08:22 -08:00
72e7c6a489
YARN-8962. Add ability to use interactive shell with normal yarn container. Contributed by Eric Yang
2018-12-12 18:18:57 -08:00
37eb919c59
YARN-8885. [DevicePlugin] Support NM APIs to query device resource allocation. (Zhankun Tang via wangda)
...
Change-Id: I2a9870709b512af1ac6c09c9701d0b3c0791ff32
2018-12-12 11:45:47 -08:00
9fba6cc247
YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda)
...
Change-Id: I73d73f3d631b28fb9866faa56571839b13824a97
2018-12-12 11:44:58 -08:00
61bdcb7b2b
YARN-9015. [DevicePlugin] Add an interface for device plugin to provide customized scheduler. (Zhankun Tang via wangda)
...
Change-Id: Ib2e4ae47a6f29bb3082c1f8520cf5a52ca720979
2018-12-12 11:44:22 -08:00
c771fe6e10
YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda)
...
Change-Id: I845131273e52a9d81dbc813ea6d4af06b205e334
2018-12-12 11:43:23 -08:00
8a5361116e
YARN-9078. [Submarine] Clean up the code of CliUtils#parseResourcesString. (Zhankun Tang via wangda)
...
Change-Id: I21ac5757b6115d55ec3157ba25db1b65bc85a37b
2018-12-12 11:42:44 -08:00
5a03c8b9ba
YARN-9114. [UI2] log service redirect url need to support user name. Contributed by Akhil PB
2018-12-12 14:19:49 +05:30
fb55e5201e
YARN-9008. Extend YARN distributed shell with file localization feature. (Contributed by Peter Bacsko)
2018-12-11 14:13:34 -08:00
881230da21
YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth)
2018-12-11 11:41:32 -08:00
ac578c0e82
YARN-9087. Improve logging for initialization of Resource plugins. (Contributed by Szilard Nemeth)
2018-12-10 11:12:54 -08:00
64411a6ff7
YARN-8738. FairScheduler should not parse negative maxResources or minResources values as positive. (Contributed by Szilard Nemeth)
2018-12-10 11:06:50 -08:00
1c09a10e96
YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters.
2018-12-10 11:55:24 +08:00
154449fbd8
YARN-8914. Add xtermjs to YARN UI2. Contributed by Eric Yang and Akhil PB
2018-12-07 10:56:17 -08:00
cc51607ccd
YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics
2018-12-07 10:45:37 -08:00
5d4a432660
YARN-9025. TestFairScheduler#testChildMaxResources is flaky. (Contributed by Szilard Nemeth)
2018-12-05 16:36:13 -08:00
912b1f9d64
YARN-9019. Ratio calculation of ResourceCalculator implementations could return NaN. (Contributed by Szilard Nemeth)
2018-12-05 15:16:41 -08:00
b3c75c1f1d
YARN-8985. Improve debug log in FSParentQueue when assigning container. (Contributed by Wilfred Spiegelenburg)
2018-12-05 15:02:57 -08:00
2bba329c14
YARN-8994. Fix race condition between move app and queue cleanup in Fair Scheduler. (Contributed by Wilfred Spiegelenburg)
2018-12-05 14:38:15 -08:00
1b790f4dd1
YARN-9071. Improved status update for reinitialized containers.
...
Contributed by Chandni Singh
2018-12-05 17:00:56 -05:00
1dabb31cdf
YARN-9057. Removed third party class bundle from CSI jar file.
...
Contributed by Weiwei Yang
2018-12-05 15:56:10 -05:00
228156cfd1
Revert "YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)"
...
This reverts commit 46d6e00166
2018-12-04 14:13:06 -08:00
e89941fdbb
YARN-9041. Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt. Contributed by Wanqiang Ji.
2018-12-04 10:08:45 -08:00
6d7b44c489
YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu
2018-11-30 10:13:52 -08:00
bad12031f6
YARN-9010. Fix the incorrect trailing slash deletion in constructor method of CGroupsHandlerImpl. (Zhankun Tang via wangda)
...
Change-Id: Iaecc66d57781cc10f19ead4647e47fc9556676da
2018-11-29 14:56:07 -08:00
07142f54a8
YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T.
2018-11-29 22:02:59 +05:30
a68d766e87
YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt.
2018-11-29 21:43:34 +05:30
efc4d91cbe
YARN-9067. YARN Resource Manager is running OOM because of leak of Configuration Object. Contributed by Eric Yang.
2018-11-29 09:34:14 +08:00
fe7dab8ef5
YARN-8989. [YARN-8851] Move DockerCommandPlugin volume related APIs' invocation from DockerLinuxContainerRuntime#prepareContainer to #launchContainer. (Zhankun Tang via wangda)
...
Change-Id: Ia6d532c687168448416dfdf46f0ac34bff20e6ca
2018-11-28 15:03:06 -08:00
897643928c
YARN-8975. [Submarine] Use predefined Charset object StandardCharsets.UTF_8 instead of String UTF-8. (Zhankun Tang via wangda)
...
Change-Id: If6c7904aa17895e543cfca245264249eb7328bdc
2018-11-28 14:39:06 -08:00
8ebeda98a9
YARN-8974. Improve the assertion message in TestGPUResourceHandler. (Zhankun Tang via wangda)
...
Change-Id: I4eb58e9d251d5f54e7feffc4fbb813b4f5ae4b1b
2018-11-28 14:36:30 -08:00
9ed87567ad
YARN-9061. Improve the GPU/FPGA module log message of container-executor. (Zhankun Tang via wangda)
...
Change-Id: Iece9b47438357077a53984a820d4d6423f480518
2018-11-28 14:31:31 -08:00
579ef4be06
YARN-8882. [YARN-8851] Add a shared device mapping manager (scheduler) for device plugins. (Zhankun Tang via wangda)
...
Change-Id: I9435136642c3d556971a357bf687f69df90bb45e
2018-11-28 14:09:52 -08:00
9de8e8d049
YARN-9030. Log aggregation changes to handle filesystems which do not support setting permissions. (Suma Shivaprasad via wangda)
...
Change-Id: I80f1e8196b8624e24d74494719fdedfd7061dced
2018-11-28 13:36:21 -08:00
3ce99e32f7
YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko
2018-11-28 08:46:11 -06:00
b3a052d199
YARN-9044. LogsCLI should contact ATSv2 for -am option. Contributed by Rohith Sharma K S
2018-11-28 00:46:53 -08:00
7dc272199f
YARN-9034. ApplicationCLI should have option to take clusterId. Contributed by Rohith Sharma K S.
2018-11-28 00:42:14 -08:00
f657a2a661
YARN-9056. Improved YARN service upgrade state logic for readiness check.
...
Contributed by Chandni Singh
2018-11-27 18:36:59 -05:00
33e0df4b35
YARN-8986. Added port publish for Docker container running with bridge network.
...
Contributed by Charo Zhang
2018-11-26 19:45:05 -05:00
0d8406135f
YARN-9054. Fix FederationStateStoreFacade#buildGetSubClustersCacheRequest. Contributed by Bibin A Chundatt.
2018-11-26 10:57:38 -08:00
e223a790a7
YARN-9042. Fix javadoc error in deviceplugin package. Contributed by Zhankun Tang.
2018-11-22 20:04:30 +05:30
176bb3f812
YARN-8984. AMRMClient#OutstandingSchedRequests leaks when AllocationTags is null or empty. Contributed by Yang Wang.
2018-11-22 16:52:29 +08:00
5fe1dbf195
YARN-8964. [UI2] YARN ui2 should use clusters/{cluster name} for all ATSv2 REST APIs. Contributed by Akhil PB.
2018-11-22 10:25:22 +05:30
a41b648e98
YARN-8992. Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue. (Contributed by Wilfred Spiegelenburg)
2018-11-20 14:12:37 -08:00
49824ed260
YARN-8838. Check that container user is same as websocket user for interactive shell. Contributed by Eric Yang
2018-11-20 11:12:24 -08:00
5fb14e0635
YARN-8953. [CSI] CSI driver adaptor module support in NodeManager. Contributed by Weiwei Yang.
2018-11-20 08:02:11 +05:30
27ffec7ba7
YARN-8778. Add command line interface to invoke interactive docker shell. Contributed by Eric Yang
2018-11-19 17:59:12 -08:00
8b23814415
YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda)
...
Change-Id: I21b1addc9c32817650ea744a8f2e6b5602f2f4d4
2018-11-19 08:55:30 -08:00
6357803645
YARN-8881. [YARN-8851] Add basic pluggable device plugin framework. (Zhankun Tang via wangda)
...
Change-Id: If9a2f68cd4713b4ec932cdeda68106f17437c3d3
2018-11-19 08:54:31 -08:00
d027a24f03
YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun.
2018-11-18 23:18:26 +08:00
abd6d48c46
YARN-8860. Federation client intercepter class contains unwanted character. Contributed by Abhishek Modi.
2018-11-16 10:23:06 -08:00
ee3355be3c
YARN-8303. YarnClient should contact TimelineReader for application/attempt/container report.
2018-11-16 17:58:19 +05:30
eeaf8edaa7
YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError. Contributed by Sushil Ks.
2018-11-15 09:25:25 -08:00
21ec4bdaef
YARN-8672. Improve token filename management for localization.
...
Contributed by Chandni Singh
2018-11-14 15:22:01 -05:00
fcbd205cc3
YARN-9001. [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs. (Zac Zhou via wangda)
...
Change-Id: Ic3d6c1e439df9cdf74448b345b925343224efe51
2018-11-13 13:13:27 -08:00
076b795b2e
YARN-8918. [Submarine] Correct method usage of str.subString in CliUtils. (Zhankun Tang via wangda)
...
Change-Id: Id1f11dbab3aa838dee3a0ec8b8fd5dc32f5dd946
2018-11-13 11:24:15 -08:00
4c465f5535
YARN-8761. Service AM support for decommissioning component instances.
...
Contributed by Billie Rinaldi
2018-11-12 19:53:10 -05:00
e269c3fb5a
YARN-8997. [Submarine] Small refactors of modifier, condition check and redundant local variables. Contributed by Zhankun Tang.
2018-11-12 15:06:43 -08:00
1f9c4f32e8
YARN-8776. Implement Container Exec feature in LinuxContainerExecutor. Contributed by Eric Yang
2018-11-12 10:42:30 -08:00
42f3a7082a
YARN-8877. [CSI] Extend service spec to allow setting resource attributes. Contributed by Weiwei Yang.
2018-11-12 21:09:30 +05:30
c741109522
YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt.
2018-11-12 18:18:23 +08:00
4e7284443e
YARN-8902. [CSI] Add volume manager that manages CSI volume lifecycle. Contributed by Weiwei Yang.
2018-11-12 11:57:02 +05:30
b5ec85d966
YARN-8933. [AMRMProxy] Fix potential empty fields in allocation response, move SubClusterTimeout to FederationInterceptor. Contributed by Botong Huang.
2018-11-11 11:12:53 -08:00
2664248797
YARN-9002. Improve keytab loading for YARN Service.
...
Contributed by Gour Saha
2018-11-10 01:52:19 -05:00
524a7523c4
YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg)
2018-11-08 16:02:48 -08:00
e1bbf7dcdf
YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang.
2018-11-08 17:47:18 +08:00
f8c72d7b3a
YARN-8880. Add configurations for pluggable plugin framework. Contributed by Zhankun Tang.
2018-11-08 12:23:00 +08:00
c96cbe8659
YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji.
2018-11-07 22:42:22 +08:00
8dc1f6dbf7
YARN-8866. Fix a parsing error for crossdomain.xml.
2018-11-07 18:26:07 +09:00
482716e5a4
YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang.
2018-11-07 10:48:07 +08:00
951c98f890
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang.
2018-11-07 11:17:35 +09:00
08d69d91f2
YARN-8957. Add Serializable interface to ComponentContainers.
...
Contributed by Zhankun Tang
2018-11-06 14:46:05 -05:00
ab6aa4c726
YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg
2018-11-06 08:40:59 -06:00
5d6554c722
YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang.
2018-11-06 14:50:09 +08:00
c7fcca0d7e
YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji.
2018-11-06 13:14:57 +08:00
f84a278baa
YARN-8905. [Router] Add JvmMetricsInfo and pause monitor. Contributed by Bilwa S T.
2018-11-03 20:35:31 +05:30
989715ec50
YARN-8893. [AMRMProxy] Fix thread leak in AMRMClientRelayer and UAM client. Contributed by Botong Huang.
2018-11-02 15:30:08 -07:00
aed836efbf
YARN-8897. LoadBasedRouterPolicy throws NPE in case of sub cluster unavailability. Contributed by Bilwa S T.
2018-11-02 11:27:11 -07:00
babc946d40
YARN-8954. Reservations list field in ReservationListInfo is not accessible. Contributed by Oleksandr Shevchenko.
2018-11-02 11:10:08 -07:00
2ab611d48b
YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne
2018-11-01 14:22:00 -07:00
d174b91635
HADOOP-15895. [JDK9+] Add missing javax.annotation-api dependency to hadoop-yarn-csi.
...
Contributed by Takanobu Asanuma.
2018-11-01 12:56:20 +00:00
Jonathan Hung
Billie Rinaldi
Tao Yang
Rohith Sharma K S
Zhankun Tang
bibinchundatt
Abhishek Modi
Rohith Sharma K S
Weiwei Yang
Eric E Payne
Eric Yang
Eric Badger
Szilard Nemeth
Sunil G
HUAN-PING SU
Adam Antal
Charles Xu
hunshenshi
Haibo Chen
Vidura Mudalige
Siyao Meng
Akira Ajisaka
Giovanni Matteo Fumarola
Wanqiang Ji
Suma Shivaprasad
Ahmed Hussein
Igor Rudenko
Sean Mackrory
Inigo Goiri
Prabhu Joseph
Szilard Nemeth
Gabor Bota
Igor Rudenko
Vrushali C
Yufei Gu
Devaraj K
Zoltan Siegl
Shweta Yakkali
Eric Badger
Yufei Gu
Wangda Tan
Masatake Iwasaki
Vinayakumar B
Takanobu Asanuma
rahul3
Jason Lowe
Erik Krogen
Botong Huang
Vidura Mudalige
Jonathan Hung
Jonathan Hung
Steve Loughran