d462308e04
YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. Contributed by Prabhu Joseph.
2019-11-03 08:47:27 +05:30
9a2e43e29e
YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph.
2019-10-31 00:34:27 +05:30
e3e7daa4f5
YARN-9889. [UI] Add Application Tag column to RM All Applications table. Contributed by Kinga Marton
2019-10-30 16:26:56 +01:00
ed45c13f67
YARN-2442. ResourceManager JMX UI does not give HA State. Contributed by Rohith Sharma and Cyrus Jackson.
2019-10-29 15:54:55 +05:30
eef34f2d87
YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan
2019-10-25 17:13:04 +00:00
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
fd84ca5161
YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi
2019-10-24 10:05:00 +08:00
83d148074f
YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph.
2019-10-21 20:30:50 +00:00
3f7756dc6c
YARN-9922. Fixed YARN javadoc errors from YARN-9699.
...
Contributed by Peter Bacsko
2019-10-21 16:16:49 -04:00
155864da00
YARN-9875. Improve fair scheduler configuration store on HDFS.
...
Contributed by Prabhu Joseph
2019-10-18 20:30:11 -04:00
72b1bed998
YARN-9884. Make container-executor mount logic modular
...
Contributed by Eric Badger
2019-10-18 19:10:32 -04:00
6d92aa7c30
HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 ( #1656 ). Contributed by Norbert Kalmár, Mate Szalay-Beko
...
* HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5
- Add a static initializer for the unit tests using ZooKeeper to enable
the four-letter-words diagnostic telnet commands. (this is an interface
that become disabled by default, so to keep the ZooKeeper 3.4.x behavior
we enabled it for the tests)
- Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt
records. The new ZooKeeper seems to respond quicker during the fail-over
tests than the ZooKeeper, so we made sure to catch all the relevant records
by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt().
Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>
2019-10-18 13:26:20 -07:00
f0699a7406
YARN-9841. Capacity scheduler: add support for combined %user + %primary_group mapping. Contributed by Manikandan R
2019-10-18 11:19:49 +02:00
3990ffa083
YARN-9909. Offline format of YarnConfigurationStore. Contributed by Prabhu Joseph.
2019-10-17 06:43:06 +05:30
a5034c7988
YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R.
2019-10-16 21:10:08 +00:00
090f73a9aa
YARN-9900. Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format. Contributed by Prabhu Joseph.
2019-10-16 18:14:31 +05:30
a8ef03e961
YARN-9840. Capacity scheduler: add support for Secondary Group rule mapping. Contributed by Manikandan R
2019-10-15 17:22:52 +02:00
5cc7873a47
YARN-9699. Migration tool that help to generate CS config based on FS config [Phase 1]. Contributed by Peter Bacsko
2019-10-14 17:19:39 +02:00
ec86f42e40
YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal
2019-10-11 14:01:19 +02:00
31e0122f4d
YARN-9860. Enable service mode for Docker containers on YARN
...
Contributed by Prabhu Joseph and Shane Kumpf
2019-10-10 19:02:02 -04:00
6f1ab95168
YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl
2019-10-09 13:26:26 +02:00
be901f4962
YARN-9873. Mutation API Config Change need to update Version Number. Contributed by Prabhu Joseph
2019-10-09 15:53:14 +05:30
5462d2176f
YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena
2019-10-08 11:20:13 -07:00
fb1ecff6a2
Revert "YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph"
...
This reverts commit 4510970e2f
2019-10-05 09:15:17 +05:30
4510970e2f
YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph
2019-10-04 21:49:07 +05:30
61a8436004
YARN-9870. Remove unused function from OpportunisticContainerAllocatorAMService. Contributed by Abhishek Modi.
2019-10-02 17:55:07 +05:30
425a6c89ae
YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung.
2019-10-01 16:02:26 +05:30
137546a78a
YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph
2019-10-01 09:08:41 +05:30
4d3c580b03
YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi.
2019-09-30 23:40:15 +05:30
18a8c2404e
YARN-9857. TestDelegationTokenRenewer throws NPE but tests pass. Contributed by Ahmed Hussein
2019-09-26 16:15:08 +00:00
606e341c1a
Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label
2019-09-25 17:54:53 -07:00
bec0864394
YARN-9808. Zero length files in container log output haven't got a header. Contributed by Adam Antal
2019-09-25 10:28:34 +02:00
c2731d4b63
YARN-9730. Support forcing configured partitions to be exclusive based on app node label
2019-09-24 13:51:32 -07:00
bfe1dac29d
HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes ( #1496 )
...
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
2019-09-24 08:50:36 +08:00
3d78b1223d
YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar
2019-09-23 13:12:16 -07:00
c474e24c0b
YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko.
2019-09-17 19:41:38 +05:30
da2b4fe204
YARN-9794. RM crashes due to runtime errors in TimelineServiceV2Publisher. Contributed by Tarun Parimi.
2019-09-15 14:16:18 +05:30
4239007349
YARN-9787. Typo in analysesErrorMsg. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org>
2019-09-13 10:36:45 -07:00
44850f6784
YARN-9816. EntityGroupFSTimelineStore#scanActiveLogs fails when undesired files are present under /ats/active. Contribued by Prabhu Joseph.
2019-09-12 12:50:10 +05:30
3b06f0bf9e
YARN-9819. Make TestOpportunisticContainerAllocatorAMService more resilient. Contribued by Abhishek Modi
2019-09-12 08:09:57 +05:30
56b7571131
YARN-9815 ReservationACLsTestBase fails with NPE. Contributed by Ahmed Hussein
2019-09-11 21:59:28 +00:00
f8f8598ea5
YARN-9824. Fall back to configured queue ordering policy class name
2019-09-10 15:19:07 -07:00
10144a580e
YARN-9728. Bugfix for escaping illegal xml characters for Resource Manager REST API.
...
Contributed by Prabhu Joseph
2019-09-10 17:04:39 -04:00
60af8793b4
YARN-9821. NM hangs at serviceStop when ATSV2 Backend Hbase is Down. Contributed by Prabhu Joseph.
2019-09-09 14:26:00 +05:30
387c332b64
YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph
2019-09-09 00:23:04 -07:00
43e389b980
YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar
2019-09-08 19:07:11 -07:00
162af6f554
YARN-9784. org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue is flaky. Contributed by Julia Kinga Marton.
2019-09-07 18:08:04 +05:30
34b82e6da0
YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda)
...
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
2019-09-06 19:18:13 -07:00
73575701ab
YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li.
2019-09-07 07:52:39 +08:00
b71a7f195c
YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar
2019-09-06 11:00:07 -07:00
e7d44e48f7
YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar
2019-09-06 10:17:07 -07:00
0ccf4b0fe1
YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta
2019-09-05 14:01:42 -07:00
03489124ea
YARN-9797. LeafQueue#activateApplications should use resourceCalculator#fitsIn. Contributed by Bilwa S T.
2019-09-03 11:53:34 +05:30
f4d6e82075
YARN-9400. Remove unnecessary if at EntityGroupFSTimelineStore#parseApplicationId. Contributed by Prabhu Joseph.
2019-09-02 10:58:23 +05:30
b939ae98e6
YARN-8174. Add containerId to ResourceLocalizationService fetch failure log statement. Contributed by Prabhu Joseph.
2019-09-02 10:45:59 +05:30
bc27f7fb6d
YARN-7982. Do ACLs check while retrieving entity-types per application. Contribued by Prabhu Joseph.
2019-09-02 10:15:59 +05:30
751b5a1ac8
YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph.
2019-09-01 22:40:40 +05:30
c7ef4fb74a
YARN-8678. Queue Management API - rephrase error messages. Contributed by Prabhu Joseph.
2019-09-01 10:00:04 +05:30
d2d963f3d4
YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam.
2019-09-01 09:54:46 +05:30
7d998cb627
YARN-9800. TestRMDelegationTokens can fail in testRemoveExpiredMasterKeyInRMStateStore. Contributed by Adam Antal.
2019-08-31 10:03:09 +05:30
eeccd2f6f6
YARN-9798. ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently. Contributed by Tao Yang.
2019-08-31 09:57:45 +05:30
c929b383f8
YARN-9540. TestRMAppTransitions fails intermittently. Contributed by Tao Yang.
2019-08-30 17:17:43 +05:30
5b557de721
YARN-9714. ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby. Contributed by Tao Yang.
2019-08-30 10:35:09 +05:30
8c0759d02a
YARN-9664. Improve response of scheduler/app activities for better understanding. Contributed by Tao Yang.
2019-08-29 18:14:39 +08:00
6f2226a013
YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).
2019-08-28 20:40:34 +00:00
8ef46595da
YARN-9438. launchTime not written to state store for running applications
2019-08-27 15:53:43 -07:00
717c853873
YARN-9755. Fixed RM failing to start when FileSystemBasedConfigurationProvider is configured.
...
Contributed by Prabhu Joseph
2019-08-27 13:14:59 -04:00
d70f5231a7
YARN-9640. Slow event processing could cause too many attempt unregister events. Contributed by Bibin A Chundatt.
2019-08-27 08:38:12 +05:30
8660e48ca1
YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph
2019-08-26 15:50:33 -07:00
d3ce53e507
YARN-9642. Fix Memory Leak in AbstractYarnScheduler caused by timer. Contributed by Bibin A Chundatt.
2019-08-26 23:21:33 +05:30
689d2e6105
YARN-8917. Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource. Contributed by Tao Yang.
2019-08-26 21:06:15 +05:30
7ab88dbfa6
YARN-7291. Better input parsing for resource in allocation file. Contributed by Zoltan Siegl
2019-08-21 17:01:18 +02:00
e8fa192f07
YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko
2019-08-21 16:44:22 +02:00
e684b17e6f
YARN-5857. TestLogAggregationService.testFixedSizeThreadPool fails intermittently on trunk. Contributed by Bilwa S T.
2019-08-21 17:14:42 +05:30
0e0ddfaf24
YARN-2599. Standby RM should expose jmx endpoint. Contributed by Rohith Sharma K S.
2019-08-17 15:43:19 +05:30
9b8359bb08
YARN-9461. TestRMWebServicesDelegationTokenAuthentication.testCancelledDelegationToken fails with HTTP 400. Contributed by Peter Bacsko
2019-08-16 12:31:58 +02:00
4456ea67b9
YARN-8586. Extract log aggregation related fields and methods from RMAppImpl. Contributed by Peter Bacsko
2019-08-16 11:36:14 +02:00
2216ec54e5
YARN-9100. Add tests for GpuResourceAllocator and do minor code cleanup. Contributed by Peter Bacsko
2019-08-16 09:13:20 +02:00
2a05e0ff3b
YARN-9749. TestAppLogAggregatorImpl#testDFSQuotaExceeded fails on trunk. Contributed by Adam Antal
2019-08-16 08:52:09 +02:00
22c4f38c4b
YARN-9679. Regular code cleanup in TestResourcePluginManager ( #1122 )
2019-08-15 17:32:05 +02:00
1845a83cec
YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph
2019-08-15 17:15:38 +02:00
167acd87da
YARN-9683. Remove reapDockerContainerNoPid left behind by YARN-9074 ( #1212 ) Contributed by Kevin Su.
...
Reviewed-by: Eric Yang <eyang@apache.org>
Reviewed-by: Adam Antal <adam.antal@cloudera.com>
2019-08-14 10:42:29 -07:00
c89bdfacc8
YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl… ( #1261 )
...
* YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl and connected classes
* Using {} placeholder, and increasing loglevel if log aggregation failed.
2019-08-14 17:35:16 +02:00
3e0410449f
YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko
2019-08-14 17:13:54 +02:00
e5e609384f
YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko
2019-08-14 16:58:22 +02:00
89a53c7eb4
YARN-9747. Reduce additional namenode call by EntityGroupFSTimelineStore#cleanLogs. Contributed by Prabhu Joseph.
2019-08-14 13:46:23 +05:30
2ac029b949
YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.
2019-08-13 16:21:18 +00:00
b4097b96a3
YARN-9744. RollingLevelDBTimelineStore.getEntityByTime fails with NPE. Contributed by Prabhu Joseph.
2019-08-13 19:04:00 +05:30
e4b538bbda
YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph
2019-08-12 15:15:43 +02:00
13a5803ccf
YARN-9464. Support pending resource metrics in RM's RESTful API. Contributed by Prabhu Joseph.
2019-08-12 14:31:24 +05:30
8fbf8b2eb0
YARN-9722. PlacementRule logs object ID in place of queue name. Contributed by Prabhu Joseph.
2019-08-12 10:44:46 +05:30
6ff0453ede
YARN-9527. Prevent rogue Localizer Runner from downloading same file repeatly.
...
Contributed by Jim Brennan
2019-08-09 14:12:17 -04:00
a79564fed0
YARN-9732. yarn.system-metrics-publisher.enabled=false is not honored by RM. Contributed by KWON BYUNGCHANG.
2019-08-09 22:25:30 +05:30
e0c21c6da9
YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak
2019-08-09 10:18:34 +02:00
742e30b473
YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak
2019-08-09 09:59:19 +02:00
72d7e570a7
YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak
2019-08-09 09:49:18 +02:00
3b38f2019e
YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang.
2019-08-08 12:37:50 +00:00
22d7d1f8bf
YARN-9601.Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations ( #908 ) Contributed by hunshenshi.
2019-08-07 21:26:14 -07:00
f51702d539
YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)
2019-08-06 13:52:30 -07:00
7c2042a44d
YARN-9678. Addendum: TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 10:21:55 -07:00
b8bf09ba3d
YARN-9678. TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 09:05:53 -07:00
d6697da5e8
YARN-9667. Use setbuf with line buffer to reduce fflush complexity in container-executor.
...
Contributed by Peter Bacsko
2019-08-05 13:59:12 -04:00
54ac80176e
Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph
2019-08-02 13:38:06 +02:00
1930a7bf60
YARN-9093. Remove commented code block from the beginning of Tes… ( #444 )
2019-08-02 13:16:19 +02:00
95fc38f2e9
YARN-9375. Use Configured in GpuDiscoverer and FpgaDiscoverer ( #1131 )
...
Contributed by Adam Antal
2019-08-02 11:24:09 +02:00
42683aef1a
YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
2019-07-30 18:58:36 +00:00
c34ceb5fde
YARN-9568. Fixed NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover.
...
Contributed by Steve Loughran
2019-07-18 12:30:53 -04:00
5915c902aa
YARN-9646. DistributedShell tests failed to bind to a local host name. (Contributed by Ray Yang)
2019-07-16 17:36:49 -07:00
7a93be0f60
YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T.
2019-07-16 14:03:22 +05:30
18ee1092b4
YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko
2019-07-15 11:59:11 +02:00
91ce09e706
YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko
2019-07-15 10:47:20 +02:00
61b0c2bb7c
YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal
2019-07-12 17:28:14 +02:00
8b3c6791b1
YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko
2019-07-12 17:20:42 +02:00
c416284bb7
YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal
2019-07-12 16:51:58 +02:00
9b54dd7186
YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonathan Hung)
2019-07-11 13:57:08 -07:00
ccaa99c923
HADOOP-16381. The JSON License is included in binary tarball via azure-documentdb:1.16.2. Contributed by Sushil Ks.
2019-07-11 13:49:42 +09:00
a2a8be18cb
YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal.
2019-07-03 13:45:00 +02:00
15d82fcb75
YARN-9658. Fix UT failures in TestLeafQueue. Contributed by Tao Yang.
2019-07-03 12:08:45 +08:00
e966edd025
YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt.
2019-07-02 12:18:16 +05:30
570eee30e5
YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi.
2019-07-02 09:55:25 +08:00
b1dafc3506
YARN-9661:Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java ( #1042 )
2019-07-01 10:46:33 -07:00
29465bf169
YARN-9560. Restructure DockerLinuxContainerRuntime to extend OCIContainerRuntime.
...
Contributed by Eric Badger, Jim Brennan, Craig Condit
2019-06-28 17:18:53 -04:00
f09c31a97e
Revert "YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023"
...
This reverts commit 5e7caf1287
2019-06-29 00:29:17 +08:00
5e7caf1287
YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023
2019-06-29 00:08:40 +08:00
cbae241320
YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang.
2019-06-28 23:24:53 +08:00
be80334cdf
YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T.
2019-06-27 09:59:44 +05:30
1ac967a6b7
YARN-6055. ContainersMonitorImpl need be adjusted when NM resource changed. Contributed by Inigo Goiri.
2019-06-26 14:01:31 -07:00
062eb605ac
YARN-9477. Implement VE discovery using libudev. Contributed by Peter Bacsko.
2019-06-26 23:53:14 +08:00
b220ec6f61
YARN-9374. Improve Timeline service resilience when HBase is unavailable.
...
Contributed by Prabhu Joseph and Szilard Nemeth
2019-06-24 12:19:14 -04:00
83dcb9d87e
YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
2019-06-21 17:41:05 +08:00
67414a1a80
YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji.
2019-06-18 12:23:52 +08:00
304a47e22c
YARN-9608. DecommissioningNodesWatcher should get lists of running applications on node from RMNode. Contributed by Abhishek Modi.
2019-06-17 17:09:56 +08:00
cda9f33745
YARN-8499 ATSv2 Generalize TimelineStorageMonitor.
...
Contributed by Prabhu Joseph
2019-06-14 18:59:14 -04:00
3ba090f436
HADOOP-16366. Fixed ProxyUserAuthenticationFilterInitializer for timeline server.
...
Contributed by Prabhu Joseph
2019-06-14 12:54:16 -04:00
bcfd228336
YARN-9599. TestContainerSchedulerQueuing#testQueueShedding fails intermittently. Contributed by Abhishek Modi.
2019-06-13 11:08:35 -07:00
970b0b0c02
YARN-9578. Add limit/actions/summarize options for app activities REST API. Contributed by Tao Yang.
2019-06-13 10:44:47 +08:00
205dd2d8e1
HADOOP-16367. Fixed MiniYarnCluster AuthenticationFilter initialization.
...
Contributed by Prabhu Joseph
2019-06-12 18:03:33 -04:00
2263ead365
YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T.
2019-06-11 23:20:28 +05:30
60c95e9b6a
YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
2019-06-11 23:11:49 +05:30
6d80b9bc3f
YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie.
2019-06-11 22:49:21 +05:30
f7df55f4a8
YARN-9602. Use logger format in Container Executor. Contributed by Abhishek Modi.
2019-06-11 22:29:00 +05:30
9191e08f0a
YARN-9569. Auto-created leaf queues do not honor cluster-wide min/max memory/vcores. Contributed by Craig Condit.
2019-06-10 14:33:24 -07:00
0976392502
YARN-9590. Correct incompatible, incomplete and redundant activities. Contributed by Tao Yang.
2019-06-06 21:59:01 +08:00
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
433e97cd34
YARN-9600. Support self-adaption width for columns of containers table on app attempt page. Contributed by Tao Yang.
2019-06-05 13:55:30 +08:00
d45669cd3c
YARN-7537. Add ability to load hbase config from distributed file system.
...
Contributed by Prabhu Joseph
2019-06-04 19:26:06 -04:00
606061aa14
YARN-9595. FPGA plugin: NullPointerException in FpgaNodeResourceUpdateHandler.updateConfiguredResource(). Contributed by Peter Bacsko.
2019-06-04 09:56:59 +08:00
bd2590d71b
YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
2019-06-03 22:59:02 +08:00
4530f4500d
YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T.
2019-06-03 14:09:37 +08:00
2210897609
YARN-9592. Use Logger format in ContainersMonitorImpl. Contributed by Inigo Goiri.
2019-05-31 17:35:49 -07:00
4cb559ea7b
YARN-9027. Fixed LevelDBCacheTimelineStore initialization.
...
Contributed by Prabhu Joseph
2019-05-31 14:31:44 -04:00
e49162f4b3
YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl.
2019-05-31 10:28:09 +05:30
7861a5eb1a
YARN-9033. ResourceHandlerChain#bootstrap is invoked twice during NM start if LinuxContainerExecutor enabled. Contributed by Zhankun Tang.
2019-05-31 10:22:26 +05:30
f1552f6edb
YARN-9553. Fix NPE in EntityGroupFSTimelineStore#getEntityTimelines. Contributed by Prabhu Joseph.
2019-05-30 11:42:27 -07:00
30c6dd92e1
YARN-9452. Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures. Contributed by Prabhu Joseph.
2019-05-30 22:32:41 +05:30
abf76ac371
YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2019-05-29 11:24:08 -05:00
3c63551101
YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph.
2019-05-29 16:05:39 +00:00
544876fe12
YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang.
2019-05-29 16:34:48 +08:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
9078e28a24
YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji.
2019-05-28 15:52:39 +09:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
9f056d905f
YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang.
2019-05-26 09:56:36 -04:00
460ba7fb14
YARN-9558. Fixed LogAggregation test cases.
...
Contributed by Prabhu Joseph
2019-05-23 18:38:47 -04:00
7b03072fd4
YARN-9080. Added clean up of bucket directories.
...
Contributed by Prabhu Joseph, Peter Bacsko, Szilard Nemeth
2019-05-23 12:08:44 -04:00
12c81610e0
YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-05-17 12:03:21 -07:00
fab5b80a36
YARN-9554. Fixed TimelineEntity DAO serialization handling.
...
Contributed by Prabhu Joseph
2019-05-16 16:39:50 -04:00
55bd35921c
YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko.
2019-05-15 11:50:46 -07:00
570fa2da20
YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T.
2019-05-15 13:30:09 +05:30
2de1e30658
YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T.
2019-05-15 13:21:39 +05:30
29ff7fb140
YARN-9493. Scheduler Page does not display the right page by query string. Contributed by Wanqiang Ji.
2019-05-13 10:57:12 -07:00
1a47c2b7ae
YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang.
2019-05-12 22:31:39 -07:00
1b48100a5e
YARN-9522. AppBlock ignores full qualified class name of PseudoAuthenticationHandler. Contributed by Prabhu Joseph.
2019-05-09 14:02:58 -07:00
90add05caa
YARN-9489. Support filtering by request-priorities and allocation-request-ids for query results of app activities. Contributed by Tao Yang.
2019-05-09 21:54:09 +08:00
3172f6cbf9
YARN-9513. Addendum patch: Fix ASF License warnings. Contributed by Giovanni Matteo Fumarola.
2019-05-08 14:56:23 +09:00
c336af3847
YARN-9432. Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled. Contributed by Tao Yang.
2019-05-08 09:54:16 +08:00
8ecbf61cca
YARN-9513. [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8. Contributed by Adam Antal.
2019-05-07 10:59:02 -07:00
597fa47ad1
YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung)
2019-05-06 11:56:22 -07:00
12b7059ddc
YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang.
2019-05-06 20:00:15 +08:00
b094b94d43
YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein.
2019-05-02 19:39:26 +00:00
accb811e57
YARN-6929. Improved partition algorithm for yarn remote-app-log-dir.
...
Contributed by Prabhu Joseph
2019-04-30 17:04:59 -04:00
7fbaa7d66f
YARN-9476. [YARN-9473] Create unit tests for VE plugin. Contributed by Peter Bacsko.
2019-04-30 11:06:44 +08:00
79d3d35398
YARN-9486. Docker container exited with failure does not get clean up correctly. Contributed by Eric Yang
2019-04-26 01:21:28 +00:00
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
3f2f4186f6
YARN-9424. Change getDeclaredMethods to getMethods in FederationClientInterceptor#invokeConcurrent. Contributed by Shen Yinjie.
2019-04-23 19:58:41 -07:00
fec9bf4b0b
YARN-9501. TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:42:56 -07:00
4a0ba24959
YARN-9491. TestApplicationMasterServiceFair#ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:27:04 -07:00
c504eee0c2
YARN-9339. Apps pending metric incorrect after moving app to a new queue. Contributed by Abhishek Modi.
2019-04-23 12:40:44 -07:00
8a95ea61e1
YARN-9475. [YARN-9473] Create basic VE plugin. Contributed by Peter Bacsko.
2019-04-23 17:33:58 +08:00
1c8046d67e
YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 14:21:13 +08:00
96e3027e46
YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi.
2019-04-22 09:49:03 -07:00
aeadb9432f
YARN-9448. Fix Opportunistic Scheduling for node local allocations. Contributed by Abhishek Modi.
2019-04-19 09:41:06 -07:00
ef97a20831
YARN-8622. Fixed container-executor compilation on MacOSX.
...
Contributed by Siyao Meng
2019-04-18 18:59:21 -04:00
df76cdc895
YARN-6695. Fixed NPE in publishing appFinished events to ATSv2.
...
Contributed by Prabhu Joseph
2019-04-18 12:29:37 -04:00
aa4c744aef
YARN-9470. Fix order of actual and expected expression in assert statements
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-18 15:40:37 +09:00
6e4399ea61
YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-17 22:56:57 -07:00
9cf7401794
YARN-9349. Improved log level practices for InvalidStateTransitionException.
...
Contributed by Anuhan Torgonshar
(cherry picked from commit fe2370e039e1ee980d74769ae85d67434e0993cf)
2019-04-16 19:53:45 -04:00
b8086aed86
YARN-9123. Clean up and split testcases in TestNMWebServices for GPU support. Contributed by Szilard Nemeth.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-16 11:06:25 -07:00
5583e1b6fc
YARN-7848 Force removal of docker containers that do not get removed on first try. Contributed by Eric Yang
2019-04-15 20:47:09 +00:00
254efc9358
YARN-9379. Can't specify docker runtime through environment. Contributed by caozhiqiang
2019-04-15 18:24:37 +00:00
7fa73fac26
YARN-9439. Support asynchronized scheduling mode and multi-node lookup mechanism for app activities. Contributed by Tao Yang.
2019-04-16 00:12:43 +08:00
7a68e7abd5
YARN-9474. Remove hard coded sleep from Opportunistic Scheduler tests. Contributed by Abhishek Modi.
2019-04-14 20:11:20 -07:00
1943db5571
HADOOP-16237. Fix new findbugs issues after updating guava to 27.0-jre.
...
Author: Gabor Bota <gabor.bota@cloudera.com>
2019-04-12 18:28:38 -07:00
ed3747c1cc
YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi.
2019-04-11 11:49:19 -07:00
8c1bba375b
YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu.
2019-04-10 22:51:28 +08:00
32722d2661
YARN-9433. Remove unused constants in YARN resource manager
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-10 18:37:27 +09:00
cfec455c45
YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri.
2019-04-09 10:59:43 -07:00
fc05b0e70e
YARN-9313. Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities. Contributed by Tao Yang.
2019-04-08 13:40:53 +08:00
ec143cbf67
YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang.
2019-04-06 19:59:36 +08:00
22362c876d
YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi.
2019-04-05 12:06:51 -07:00
27039a29ae
YARN-9382 Publish container killed, paused and resumed events to ATSv2. Contributed by Abhishesk Modi.
2019-04-05 12:02:43 -07:00
8d150067e2
YARN-9396. Fixed duplicated RM Container created event to ATS.
...
Contributed by Prabhu Joseph
2019-04-04 13:01:56 -04:00
eb03f7c419
YARN-9303 Username splits won't help timelineservice.app_flow table. Contributed by Prabhu Joseph.
2019-04-03 22:53:05 -07:00
002dcc4ebf
YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
2019-04-03 18:57:28 +05:30
2f752830ba
YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji.
2019-04-01 20:05:15 -07:00
ab2bda57bd
YARN-9428. Add metrics for paused containers in NodeManager. Contributed by Abhishek Modi.
2019-04-01 14:21:17 -07:00
da7f8c244d
YARN-9431. Fix flaky junit test fair.TestAppRunnability after YARN-8967. Contributed by Wilfred Spiegelenburg.
2019-04-01 11:21:31 -07:00
332cab5518
YARN-9418. ATSV2 /apps//entities/YARN_CONTAINER rest api does not show metrics. Contributed by Prabhu Joseph.
2019-04-01 11:06:51 -07:00
56f1e131ec
YARN-9270. Minor cleanup in TestFpgaDiscoverer. Contributed by Peter Bacsko.
2019-03-29 10:58:56 -07:00
a4cd75e09c
YARN-9269. Minor cleanup in FpgaResourceAllocator. Contributed by Peter Bacsko.
2019-03-27 10:08:07 -07:00
5257f50abb
YARN-8967. Change FairScheduler to use PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-25 22:47:24 -07:00
eeda6891e4
YARN-9268. General improvements in FpgaDevice. Contributed by Peter Bacsko.
2019-03-25 13:22:53 -07:00
3c45762a0b
YARN-9391. Fixed node manager environment leaks into Docker containers.
...
Contributed by Jim Brennan
2019-03-25 15:53:24 -04:00
509b20b292
YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent. Contributed by Prabhu Joseph.
2019-03-22 11:45:39 -07:00
ce5eb9cb2e
YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322
...
(Contributed by Zoltan Siegl via Daniel Templeton)
Change-Id: I92d52c0ca630e71afb26b2b7587cbdbe79254a05
2019-03-22 12:28:34 +01:00
548997d6c9
YARN-9402. Opportunistic containers should not be scheduled on Decommissioning nodes. Contributed by Abhishek Modi.
2019-03-21 12:04:05 -07:00
a99eb80659
YARN-9267. General improvements in FpgaResourceHandlerImpl. Contributed by Peter Bacsko.
2019-03-21 11:15:56 -07:00
506502bb83
YARN-9370. Added logging for recovering assigned GPU devices.
...
Contributed by Yesha Vora
2019-03-20 19:12:19 -04:00
f2b862cac6
YARN-9398. Fixed javadoc errors for FPGA related java files.
...
Contributed by Peter Bacsko
2019-03-20 15:45:37 -04:00
b3b0e332e6
YARN-9299. TestTimelineReaderWhitelistAuthorizationFilter ignores Http Errors. Contributed by Prabhu Joseph.
2019-03-20 21:24:31 +05:30
0d24684eee
YARN-9357. Modify HBase Liveness monitor log to debug. Contributed by Prabhu Joseph.
2019-03-20 21:22:54 +05:30
c1a4eeb7c8
YARN-9389. FlowActivity and FlowRun table prefix is wrong. Contributed by Prabhu Joseph.
2019-03-20 21:18:19 +05:30
5d8bd0e5cb
YARN-9392. Handle missing scheduler events in Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-03-19 11:00:21 -07:00
09eabda314
YARN-9364. Remove commons-logging dependency from YARN.
...
Contributed by Prabhu Joseph
2019-03-18 19:58:42 -04:00
5f6e225166
YARN-9363. Replaced debug logging with SLF4J parameterized log message.
...
Contributed by Prabhu Joseph
2019-03-18 13:57:18 -04:00
0e7e9013d4
YARN-9340. [Clean-up] Remove NULL check before instanceof in ResourceRequestSetKey
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I932e29b36f086f7b7c76a250e33b473617ddbda1
2019-03-18 15:08:37 +01:00
2064ca015d
YARN-9349. Changed logging to use slf4j api.
...
Contributed by Prabhu Joseph
2019-03-15 19:20:59 -04:00
03f3c8aed2
YARN-4404. Corrected typo in javadoc.
...
Contributed by Yesha Vora
2019-03-15 18:04:04 -04:00
688b177fc6
YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang
2019-03-14 19:39:00 +00:00
f235a942d5
YARN-9016 DocumentStore as a backend for ATSv2. Contributed by Sushil Ks.
2019-03-13 16:45:23 -07:00
17a3e14d25
YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi.
2019-03-12 21:33:17 -07:00
8e1539eca8
YARN-9266. General improvements in IntelFpgaOpenclPlugin. Contributed by Peter Bacsko.
2019-03-13 02:45:17 +05:30
de15a66d78
YARN-9265. FPGA plugin fails to recognize Intel Processing Accelerator Card. Contributed by Peter Bacsko.
2019-03-08 17:39:22 +05:30
39b4a37e02
YARN-9341. Fixed enentrant lock usage in YARN project.
...
Contributed by Prabhu Joseph
2019-03-07 16:47:45 -05:00
491313ab84
YARN-8218 Add application launch time to ATSV1. Contributed by Abhishek Modi
2019-03-06 21:47:29 -08:00
46045c5cb3
YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth.
2019-03-06 16:01:08 +05:30
7b42e0e32a
YARN-7266. Fixed deadlock in Timeline Server thread initialization.
...
Contributed by Prabhu Joseph
2019-03-05 12:17:01 -05:00
0aefe2846f
YARN-9298. Implement FS placement rules using PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-04 23:49:07 -08:00
e40e2d6ad5
YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-03-05 14:10:08 +09:00
15098df744
Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph."
...
This reverts commit a63c358b78
2019-03-04 16:57:31 +05:30
cab8529ecb
YARN-7904. Privileged, trusted containers should be supported only in ENTRYPOINT mode. Contributed by Eric Yang.
2019-03-01 11:06:09 -08:00
dcaca19871
YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth.
2019-03-01 19:24:35 +05:30
538bb4880d
YARN-9323. FSLeafQueue#computeMaxAMResource does not override zero values for custom resources
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: Id844ccf09488f367c0c7de0a3b2d4aca1bba31cc
2019-02-27 19:59:48 -08:00
7b928f19a4
YARN-9322. Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: I14c12f1265999d62102f2ec5506d90015efeefe8
2019-02-27 19:43:50 -08:00
1779fc57a1
YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph.
2019-02-28 09:56:29 +08:00
ea3cdc60b3
YARN-3841 [atsv2 Storage implementation] Adding retry semantics to HDFS backing storage. Contributed by Abhishek Modi.
2019-02-27 14:55:35 -08:00
0ec962ac8f
YARN-5336 Limit the flow name size & consider cleanup for hex chars. Contributed by Sushil Ks
2019-02-27 14:43:39 -08:00
fbc7bb315f
YARN-9245. Added query docker image command ability to node manager.
...
Contributed by Chandni Singh
2019-02-27 14:57:24 -05:00
8c30114b00
YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie.
2019-02-27 17:29:02 +08:00
6c96f5e4b6
YARN-8378. ApplicationHistoryManagerImpl#getApplications doesn't honor filters. Contributed by Lantao Jin.
2019-02-27 10:32:58 +05:30
8eae260af5
YARN-9311. Fix TestRMRestart hangs. Contributed by Prabhu Joseph.
2019-02-27 10:28:16 +05:30
c6ea28c480
YARN-9331. [YARN-8851] Fix a bug that lacking cgroup initialization when bootstrap DeviceResourceHandlerImpl. Contributed by Zhankun Tang.
2019-02-26 10:05:31 +08:00
95372657fc
YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena.
2019-02-25 11:45:37 -08:00
3e1739d589
YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang.
2019-02-26 00:08:13 +08:00
5e91ebd91a
YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth.
2019-02-25 11:30:46 +05:30
9cd5c5447f
YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph.
2019-02-24 22:42:27 +08:00
50094d7fef
YARN-9300. Lazy preemption should trigger an update on queue preemption metrics for CapacityScheduler. Contributed by Tao Yang.
2019-02-24 22:17:29 +08:00
dddcfa4d9f
YARN-8821. [YARN-8851] GPU hierarchy/topology scheduling support based on pluggable device framework. Contributed by Zhankun Tang.
2019-02-24 14:37:06 +05:30
ed13cf8406
YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph
2019-02-22 21:01:34 +05:30
95fbbfed75
YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth.
2019-02-22 20:22:17 +05:30
9c88695bcd
YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie.
2019-02-22 20:58:33 +08:00
632d5e8a98
YARN-9156. Improve debug message in device plugin method compatibility check of ResourcePluginManager. Contributed by Zhankun Tang.
2019-02-22 11:27:30 +08:00
b17a2602d1
YARN-9319. Fix compilation issue of handling typedef an existing name by gcc compiler. (Zhankun Tang via wangda)
...
Change-Id: Ie8293cc5b710ff355fb6a1ea7201be53ed0385bf
2019-02-21 11:17:32 -08:00
eedcc8e26e
YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph.
2019-02-21 17:21:21 +08:00
a63c358b78
YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph.
2019-02-21 13:29:01 +05:30
b8de78c570
YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T.
2019-02-20 01:19:20 +05:30
db4d1a1e2f
YARN-9060. [YARN-8851] Phase 1 - Support device isolation and use the Nvidia GPU plugin as an example. Contributed by Zhankun Tang.
2019-02-18 15:58:04 +05:30
0f2b65c3da
HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
2019-02-18 15:21:46 +09:00
ba56bc24f5
YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth.
2019-02-17 20:08:20 +05:30
7c1b561e33
YARN-8927. Added support for top level Dockerhub images to trusted registry using library keyword.
...
Contributed by Zhankun Tang
2019-02-15 11:18:07 -05:00
9385ec45d7
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-02-15 18:47:10 +09:00
3dc2523266
YARN-9184. Add a system flag to allow update to latest docker images.
...
Contributed by Zhaohui Xin
2019-02-12 16:16:35 -05:00
7536488bbd
YARN-996. REST API support for node resource configuration. Contributed by Inigo Goiri.
2019-02-11 15:27:43 -08:00
0a1637c750
YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph.
2019-02-11 15:53:50 +08:00
e0ab1bdece
YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri.
2019-02-09 00:26:50 +09:00
d1ca9432dd
YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein
2019-02-07 16:38:11 +00:00
3c96a03dea
YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena.
2019-02-07 13:02:35 +05:30
7fa62e150c
YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi.
2019-02-07 09:58:53 +05:30
1e5e08d83b
YARN-7627. [ATSv2] When passing a non-number as metricslimit, the error message is wrong. Contributed by Charan Hebri.
2019-02-07 09:58:53 +05:30
e3ec18b0c4
YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi.
2019-02-05 13:48:04 +05:30
604b2489a9
YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla.
2019-02-04 12:48:28 +05:30
f20b043a02
YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang.
2019-02-02 23:04:34 +08:00
28ad20a711
YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie.
2019-02-02 23:02:45 +08:00
2c13513040
YARN-8549 Adding a NoOp timeline writer and reader plugin classes for ATSv2. Contributed by Prabha Manepalli.
2019-02-01 14:26:50 -08:00
71c49fa60f
YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth.
2019-01-31 09:25:29 +05:30
a3a9ae3cea
YARN-9251. Build failure for -Dhbase.profile=2.0. Contributed by Rohith Sharma K S.
2019-01-30 05:17:04 -08:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
04105bbfdb
YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph
2019-01-29 17:22:24 +00:00
4f63ffe444
YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang.
2019-01-29 10:34:26 +08:00
2e636dd3c4
YARN-9074. Consolidate docker removal logic in ContainerCleanup.
...
Contributed by Zhaohui Xin
2019-01-28 18:05:53 -05:00
1ab69a9543
YARN-9221. Added flag to disable dynamic auxiliary service feature.
...
Contributed by Billie Rinaldi
2019-01-25 19:05:36 -05:00
6cace58e21
YARN-9222. Print launchTime in ApplicationSummary
2019-01-25 13:17:00 -08:00
a33ef4fd31
YARN-8867. Added resource localization status to YARN service status call.
...
Contributed by Chandni Singh
2019-01-24 18:43:21 -05:00
a4bd64e724
YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu.
2019-01-24 17:53:43 +08:00
721d5c2a5f
YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph.
2019-01-23 17:59:36 +08:00
e3e076dc78
YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang.
2019-01-23 14:14:49 +08:00
2fa9389c2e
YARN-9146. Added REST API to configure auxiliary service.
...
Contributed by Billie Rinaldi
2019-01-22 18:24:43 -05:00
d43df31751
YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang.
2019-01-22 10:23:32 +08:00
abde1e1f58
YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang.
2019-01-21 16:54:14 +08:00
6d7eedfd28
YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda)
...
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
2019-01-17 14:20:10 -08:00
713ded6b15
YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
2019-01-15 21:28:10 -08:00
6be39230a6
YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
2019-01-15 21:25:37 -08:00
614af50625
YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo.
2019-01-15 12:59:59 +09:00
6a923464af
YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R
2019-01-08 16:54:05 -06:00
0921b706f7
YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan.
2019-01-08 14:30:53 +08:00
489411579c
YARN-9169. Add metrics for queued opportunistic and guaranteed containers. Contributed by Abhishek Modi.
2019-01-07 15:16:55 -08:00
0a01d49917
YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda)
...
Change-Id: Id8af27134d3286a7a10d85eda9be25df9689d0e7
2019-01-07 12:07:26 -08:00
944cf87223
YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg.
2019-01-07 15:57:31 +08:00
6e35f7130f
YARN-9166. Fix logging for preemption of Opportunistic containers for Guaranteed containers. Contributed by Abhishek Modi.
2019-01-04 11:16:41 -08:00
8c6978c3ba
YARN-6149. Allow port range to be specified while starting NM Timeline collector manager. Contributed by Abhishek Modi.
2019-01-04 14:07:02 +05:30
f4906ac019
YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang.
2019-01-04 12:10:00 +05:30
dfceffa70d
YARN-9147. Rmove auxiliary services when manifest file is removed.
...
Contributed by Billie Rinaldi
2019-01-03 12:57:21 -05:00
cfe89e6f96
YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie.
2019-01-03 23:56:28 +08:00
128f340f4f
YARN-8894. Improve InMemoryPlan#toString. Contributed by BELUGA BEHR.
2018-12-28 17:43:47 +09:00
15fe1b34dd
YARN-8878. Remove StringBuffer from ManagedParentQueue.java. Contributed by BELUGA BEHR.
2018-12-28 17:29:22 +09:00
091ad974cd
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
657aa433e2
YARN 9108. Fix FederationIntercepter merge home and secondary allocate response typo. Contributed by Abhishek Modi.
2018-12-22 12:41:49 -08:00
f82922dcfa
YARN-5168. Added exposed port information for Docker container.
...
Contributed by Xun Liu
2018-12-21 19:44:07 -05:00
ea724181d6
YARN-9132. Added file permission check for auxiliary services manifest file.
...
Contributed by Billie Rinaldi
2018-12-21 14:56:39 -05:00
f659485ee8
YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang.
2018-12-21 10:56:42 +08:00
a80d321074
YARN-9152. Improved AuxServices REST API output.
...
Contributed by Billie Rinaldi
2018-12-20 19:21:55 -05:00
5df9fb16b9
YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed by Rong Tang.
2018-12-19 18:58:00 -08:00
2e544dc921
YARN-9129. Ensure flush after printing to log plus additional cleanup. Contributed by Eric Yang
2018-12-19 16:45:23 -08:00
e815fd9c49
YARN-9126. Fix container clean up for reinitialization.
...
Contributed by Chandni Singh
2018-12-19 14:55:56 -05:00
c7a5a4435e
YARN-9075. Add ability to register/remove auxiliary service without restart node manager.
...
Contributed by Billie Rinaldi
2018-12-18 17:05:51 -05:00
a6e9d27c29
YARN-9117. Add a check for insecure setup for container terminal. Contributed by Eric Yang
2018-12-18 12:49:10 -08:00
c5c73182e5
YARN-9072. Send exit command to terminate docker exec on connection close. Contributed by Eric Yang
2018-12-18 10:06:33 -08:00
71e0b0d800
YARN-9040. Fixed memory leak in LevelDBCacheTimelineStore and DBIterator.
...
Contributed by Tarun Parimi
2018-12-17 12:04:25 -05:00
4f622337ee
YARN-9091. Improve terminal message when connection is refused. Contributed by Eric Yang
2018-12-14 19:55:50 -08:00
42297f7d99
YARN-8963. Add flag to disable interactive shell. Contributed by Eric Yang
2018-12-14 18:44:36 -08:00
b2d7204ed0
YARN-9125. Fixed Carriage Return detection in Docker container launch command.
...
Contributed by Billie Rinaldi
2018-12-14 17:52:26 -05:00
ca90b2480e
YARN-9032. Support sh shell for interactive container shell at command line. Contributed by Eric Yang
2018-12-14 14:08:22 -08:00
72e7c6a489
YARN-8962. Add ability to use interactive shell with normal yarn container. Contributed by Eric Yang
2018-12-12 18:18:57 -08:00
37eb919c59
YARN-8885. [DevicePlugin] Support NM APIs to query device resource allocation. (Zhankun Tang via wangda)
...
Change-Id: I2a9870709b512af1ac6c09c9701d0b3c0791ff32
2018-12-12 11:45:47 -08:00
61bdcb7b2b
YARN-9015. [DevicePlugin] Add an interface for device plugin to provide customized scheduler. (Zhankun Tang via wangda)
...
Change-Id: Ib2e4ae47a6f29bb3082c1f8520cf5a52ca720979
2018-12-12 11:44:22 -08:00
881230da21
YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth)
2018-12-11 11:41:32 -08:00
ac578c0e82
YARN-9087. Improve logging for initialization of Resource plugins. (Contributed by Szilard Nemeth)
2018-12-10 11:12:54 -08:00
64411a6ff7
YARN-8738. FairScheduler should not parse negative maxResources or minResources values as positive. (Contributed by Szilard Nemeth)
2018-12-10 11:06:50 -08:00
1c09a10e96
YARN-9009. Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs. Contributed by OrDTesters.
2018-12-10 11:55:24 +08:00
154449fbd8
YARN-8914. Add xtermjs to YARN UI2. Contributed by Eric Yang and Akhil PB
2018-12-07 10:56:17 -08:00
cc51607ccd
YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics
2018-12-07 10:45:37 -08:00
Sunil G
Szilard Nemeth
Abhishek Modi
Eric Badger
Eric Yang
Zhankun Tang
Eric E Payne
Mate Szalay-Beko
Jonathan Hung
bibinchundatt
Duo Zhang
HUAN-PING SU
Wangda Tan
Tao Yang
Rohith Sharma K S
Weiwei Yang
Adam Antal
hunshenshi
Haibo Chen
Vidura Mudalige
Akira Ajisaka
Giovanni Matteo Fumarola
Suma Shivaprasad
Ahmed Hussein
Sean Mackrory
Inigo Goiri
Prabhu Joseph
Siyao Meng
Szilard Nemeth
Gabor Bota
Igor Rudenko
Vrushali C
Yufei Gu
Devaraj K
Zoltan Siegl
Shweta Yakkali
Eric Badger
Yufei Gu
Masatake Iwasaki
Vinayakumar B
Jason Lowe
Erik Krogen
Botong Huang
Billie Rinaldi
Jonathan Hung