1845a83cec
YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph
2019-08-15 17:15:38 +02:00
167acd87da
YARN-9683. Remove reapDockerContainerNoPid left behind by YARN-9074 ( #1212 ) Contributed by Kevin Su.
...
Reviewed-by: Eric Yang <eyang@apache.org>
Reviewed-by: Adam Antal <adam.antal@cloudera.com>
2019-08-14 10:42:29 -07:00
c89bdfacc8
YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl… ( #1261 )
...
* YARN-9676. Add DEBUG and TRACE level messages to AppLogAggregatorImpl and connected classes
* Using {} placeholder, and increasing loglevel if log aggregation failed.
2019-08-14 17:35:16 +02:00
3e0410449f
YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko
2019-08-14 17:13:54 +02:00
e5e609384f
YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko
2019-08-14 16:58:22 +02:00
89a53c7eb4
YARN-9747. Reduce additional namenode call by EntityGroupFSTimelineStore#cleanLogs. Contributed by Prabhu Joseph.
2019-08-14 13:46:23 +05:30
2ac029b949
YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan.
2019-08-13 16:21:18 +00:00
b4097b96a3
YARN-9744. RollingLevelDBTimelineStore.getEntityByTime fails with NPE. Contributed by Prabhu Joseph.
2019-08-13 19:04:00 +05:30
201dc667e9
YARN-9719. Fixed YARN service restart bug when application ID no longer exist in RM.
...
Contributed by kyungwan nam
2019-08-12 18:24:00 -04:00
e4b538bbda
YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph
2019-08-12 15:15:43 +02:00
b91099efd6
YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph
2019-08-12 15:05:28 +02:00
e0517fea33
YARN-9134. No test coverage for redefining FPGA / GPU resource types in TestResourceUtils. Contributed by Peter Bacsko
2019-08-12 14:36:07 +02:00
13a5803ccf
YARN-9464. Support pending resource metrics in RM's RESTful API. Contributed by Prabhu Joseph.
2019-08-12 14:31:24 +05:30
8fbf8b2eb0
YARN-9722. PlacementRule logs object ID in place of queue name. Contributed by Prabhu Joseph.
2019-08-12 10:44:46 +05:30
cf5d8957ee
YARN-9657. AbstractLivelinessMonitor add serviceName to PingChecker thread. Contributed by Bilwa S T.
2019-08-11 15:41:56 +05:30
1c5b28659f
YARN-9729. [UI2] Fix error message for logs when ATSv2 is offline. Contributed by Zoltan Siegl.
2019-08-11 11:47:07 +05:30
e69db45e50
Revert "YARN-9729. [UI2] Fix error message for logs without ATSv2. Contributed by Zoltan Siegl."
...
This reverts commit 50cec5a704
2019-08-10 21:10:22 +05:30
50cec5a704
YARN-9729. [UI2] Fix error message for logs without ATSv2. Contributed by Zoltan Siegl.
2019-08-10 21:06:25 +05:30
6ff0453ede
YARN-9527. Prevent rogue Localizer Runner from downloading same file repeatly.
...
Contributed by Jim Brennan
2019-08-09 14:12:17 -04:00
a79564fed0
YARN-9732. yarn.system-metrics-publisher.enabled=false is not honored by RM. Contributed by KWON BYUNGCHANG.
2019-08-09 22:25:30 +05:30
acffec7a92
YARN-9715. [UI2] yarn-container-log URI need to be encoded to avoid potential misuses. Contributed by Akhil PB.
2019-08-09 16:05:02 +05:30
e0c21c6da9
YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak
2019-08-09 10:18:34 +02:00
742e30b473
YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak
2019-08-09 09:59:19 +02:00
72d7e570a7
YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak
2019-08-09 09:49:18 +02:00
9e6519a11a
YARN-9711. Missing spaces in NMClientImpl ( #1177 ) Contributed by Charles Xu.
2019-08-08 06:41:04 -07:00
3b38f2019e
YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang.
2019-08-08 12:37:50 +00:00
22d7d1f8bf
YARN-9601.Potential NPE in ZookeeperFederationStateStore#getPoliciesConfigurations ( #908 ) Contributed by hunshenshi.
2019-08-07 21:26:14 -07:00
f51702d539
YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung)
2019-08-06 13:52:30 -07:00
7c2042a44d
YARN-9678. Addendum: TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 10:21:55 -07:00
b8bf09ba3d
YARN-9678. TestGpuResourceHandler / TestFpgaResourceHandler should be renamed. Contributed by kevin su.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 09:05:53 -07:00
1c53ce0cda
YARN-9410. Typo in documentation: Using FPGA On YARN ( #1220 ) Contributed by kevin su.
2019-08-06 06:29:58 -07:00
d6697da5e8
YARN-9667. Use setbuf with line buffer to reduce fflush complexity in container-executor.
...
Contributed by Peter Bacsko
2019-08-05 13:59:12 -04:00
54ac80176e
Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph
2019-08-02 13:38:06 +02:00
1930a7bf60
YARN-9093. Remove commented code block from the beginning of Tes… ( #444 )
2019-08-02 13:16:19 +02:00
cbcada804d
YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently ( #1121 )
2019-08-02 13:04:34 +02:00
95fc38f2e9
YARN-9375. Use Configured in GpuDiscoverer and FpgaDiscoverer ( #1131 )
...
Contributed by Adam Antal
2019-08-02 11:24:09 +02:00
42683aef1a
YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
2019-07-30 18:58:36 +00:00
9b8b3acb0a
HADOOP-16451. Update jackson-databind to 2.9.9.1. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-24 17:25:25 -07:00
9838a47d44
YARN-6046. Fixed documentation error in YarnApplicationSecurity.
...
Contributed by Yousef Abu-Salah
2019-07-18 12:39:18 -04:00
c34ceb5fde
YARN-9568. Fixed NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover.
...
Contributed by Steve Loughran
2019-07-18 12:30:53 -04:00
85d9111a88
YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam.
2019-07-17 10:28:20 +08:00
5915c902aa
YARN-9646. DistributedShell tests failed to bind to a local host name. (Contributed by Ray Yang)
2019-07-16 17:36:49 -07:00
7a93be0f60
YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T.
2019-07-16 14:03:22 +05:30
5446308360
YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal
2019-07-15 13:28:01 +02:00
18ee1092b4
YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko
2019-07-15 11:59:11 +02:00
91ce09e706
YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko
2019-07-15 10:47:20 +02:00
557056e18e
YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl
2019-07-12 17:40:14 +02:00
61b0c2bb7c
YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal
2019-07-12 17:28:14 +02:00
8b3c6791b1
YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko
2019-07-12 17:20:42 +02:00
c416284bb7
YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal
2019-07-12 16:51:58 +02:00
9b54dd7186
YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonathan Hung)
2019-07-11 13:57:08 -07:00
9cec023186
YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl
2019-07-11 16:11:21 +02:00
ccaa99c923
HADOOP-16381. The JSON License is included in binary tarball via azure-documentdb:1.16.2. Contributed by Sushil Ks.
2019-07-11 13:49:42 +09:00
555dabf4f8
YARN-9660. Update support documentation for Docker on YARN.
...
Contributed by Peter Bacsko
2019-07-10 17:15:33 -04:00
a2a8be18cb
YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal.
2019-07-03 13:45:00 +02:00
15d82fcb75
YARN-9658. Fix UT failures in TestLeafQueue. Contributed by Tao Yang.
2019-07-03 12:08:45 +08:00
e966edd025
YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt.
2019-07-02 12:18:16 +05:30
0c8813f135
YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt.
2019-07-02 11:44:11 +05:30
570eee30e5
YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi.
2019-07-02 09:55:25 +08:00
b1dafc3506
YARN-9661:Fix typo in LocalityMulticastAMRMProxyPolicy.java and AbstractConfigurableFederationPolicy.java ( #1042 )
2019-07-01 10:46:33 -07:00
29465bf169
YARN-9560. Restructure DockerLinuxContainerRuntime to extend OCIContainerRuntime.
...
Contributed by Eric Badger, Jim Brennan, Craig Condit
2019-06-28 17:18:53 -04:00
f02b0e1994
YARN-9581. Add support for get multiple RM webapp URLs.
...
Contributed by Prabhu Joseph
2019-06-28 14:54:57 -04:00
f09c31a97e
Revert "YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023"
...
This reverts commit 5e7caf1287
2019-06-29 00:29:17 +08:00
5e7caf1287
YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. (hunshenshi via wwei) closes apache/hadoop#1023
2019-06-29 00:08:40 +08:00
cbae241320
YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang.
2019-06-28 23:24:53 +08:00
be80334cdf
YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T.
2019-06-27 09:59:44 +05:30
1ac967a6b7
YARN-6055. ContainersMonitorImpl need be adjusted when NM resource changed. Contributed by Inigo Goiri.
2019-06-26 14:01:31 -07:00
062eb605ac
YARN-9477. Implement VE discovery using libudev. Contributed by Peter Bacsko.
2019-06-26 23:53:14 +08:00
b220ec6f61
YARN-9374. Improve Timeline service resilience when HBase is unavailable.
...
Contributed by Prabhu Joseph and Szilard Nemeth
2019-06-24 12:19:14 -04:00
83dcb9d87e
YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
2019-06-21 17:41:05 +08:00
5bfdf62614
YARN-9631. Added ability to select JavaScript test or skip JavaScript tests for YARN application catalog.
...
Contributed by Eric Yang
(cherry picked from commit 6002b0c5c6994965d3f7231330248c093869dba2)
2019-06-19 18:48:16 -04:00
b14f0569bb
YARN-9574. Update hadoop-yarn-applications-mawo artifactId to match directory name.
...
Contributed by Wanqiang Ji
(cherry picked from commit 5f758a69ede8fafd214857a74f5d3b46198094c4)
2019-06-18 13:51:12 -04:00
eb6be4643f
YARN-9630. [UI2] Add a link in docs's top page
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-06-18 14:54:21 +09:00
67414a1a80
YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji.
2019-06-18 12:23:52 +08:00
304a47e22c
YARN-9608. DecommissioningNodesWatcher should get lists of running applications on node from RMNode. Contributed by Abhishek Modi.
2019-06-17 17:09:56 +08:00
cda9f33745
YARN-8499 ATSv2 Generalize TimelineStorageMonitor.
...
Contributed by Prabhu Joseph
2019-06-14 18:59:14 -04:00
3ba090f436
HADOOP-16366. Fixed ProxyUserAuthenticationFilterInitializer for timeline server.
...
Contributed by Prabhu Joseph
2019-06-14 12:54:16 -04:00
bcfd228336
YARN-9599. TestContainerSchedulerQueuing#testQueueShedding fails intermittently. Contributed by Abhishek Modi.
2019-06-13 11:08:35 -07:00
970b0b0c02
YARN-9578. Add limit/actions/summarize options for app activities REST API. Contributed by Tao Yang.
2019-06-13 10:44:47 +08:00
205dd2d8e1
HADOOP-16367. Fixed MiniYarnCluster AuthenticationFilter initialization.
...
Contributed by Prabhu Joseph
2019-06-12 18:03:33 -04:00
2263ead365
YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T.
2019-06-11 23:20:28 +05:30
60c95e9b6a
YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
2019-06-11 23:11:49 +05:30
6d80b9bc3f
YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie.
2019-06-11 22:49:21 +05:30
f7df55f4a8
YARN-9602. Use logger format in Container Executor. Contributed by Abhishek Modi.
2019-06-11 22:29:00 +05:30
9191e08f0a
YARN-9569. Auto-created leaf queues do not honor cluster-wide min/max memory/vcores. Contributed by Craig Condit.
2019-06-10 14:33:24 -07:00
e94e643584
YARN-9471. Cleanup in TestLogAggregationIndexFileController. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 12:18:15 -07:00
cb9bc6e64c
YARN-9581. Fixed yarn logs cli to access RM2.
...
Contributed by Prabhu Joseph
2019-06-06 16:41:58 -04:00
0976392502
YARN-9590. Correct incompatible, incomplete and redundant activities. Contributed by Tao Yang.
2019-06-06 21:59:01 +08:00
649666e118
YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal.
2019-06-06 14:51:55 +05:30
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
433e97cd34
YARN-9600. Support self-adaption width for columns of containers table on app attempt page. Contributed by Tao Yang.
2019-06-05 13:55:30 +08:00
d45669cd3c
YARN-7537. Add ability to load hbase config from distributed file system.
...
Contributed by Prabhu Joseph
2019-06-04 19:26:06 -04:00
606061aa14
YARN-9595. FPGA plugin: NullPointerException in FpgaNodeResourceUpdateHandler.updateConfiguredResource(). Contributed by Peter Bacsko.
2019-06-04 09:56:59 +08:00
bd2590d71b
YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
2019-06-03 22:59:02 +08:00
59719dc560
YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB.
2019-06-03 15:53:23 +05:30
7f46dda513
YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB.
2019-06-03 12:24:35 +05:30
4530f4500d
YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T.
2019-06-03 14:09:37 +08:00
2210897609
YARN-9592. Use Logger format in ContainersMonitorImpl. Contributed by Inigo Goiri.
2019-05-31 17:35:49 -07:00
4cb559ea7b
YARN-9027. Fixed LevelDBCacheTimelineStore initialization.
...
Contributed by Prabhu Joseph
2019-05-31 14:31:44 -04:00
52128e352a
YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B.
2019-05-31 12:29:44 +05:30
e49162f4b3
YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl.
2019-05-31 10:28:09 +05:30
7861a5eb1a
YARN-9033. ResourceHandlerChain#bootstrap is invoked twice during NM start if LinuxContainerExecutor enabled. Contributed by Zhankun Tang.
2019-05-31 10:22:26 +05:30
f1552f6edb
YARN-9553. Fix NPE in EntityGroupFSTimelineStore#getEntityTimelines. Contributed by Prabhu Joseph.
2019-05-30 11:42:27 -07:00
30c6dd92e1
YARN-9452. Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures. Contributed by Prabhu Joseph.
2019-05-30 22:32:41 +05:30
abf76ac371
YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2019-05-29 11:24:08 -05:00
3c63551101
YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph.
2019-05-29 16:05:39 +00:00
544876fe12
YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang.
2019-05-29 16:34:48 +08:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
4a692bc3be
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang.
2019-05-28 16:53:43 +09:00
9078e28a24
YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji.
2019-05-28 15:52:39 +09:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
9f056d905f
YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang.
2019-05-26 09:56:36 -04:00
460ba7fb14
YARN-9558. Fixed LogAggregation test cases.
...
Contributed by Prabhu Joseph
2019-05-23 18:38:47 -04:00
6a0e7dd454
YARN-9482. DistributedShell job with localization fails in unsecure cluster. Contributed by Prabhu Joseph.
2019-05-23 13:27:52 -07:00
e2b79912c3
YARN-9469. Fix typo in YarnConfiguration.
2019-05-23 19:00:38 +02:00
7b03072fd4
YARN-9080. Added clean up of bucket directories.
...
Contributed by Prabhu Joseph, Peter Bacsko, Szilard Nemeth
2019-05-23 12:08:44 -04:00
a85451c8d2
YARN-9575. Fix TestYarnConfigurationFields testcase failing. Contributed by Prabhu Joseph.
2019-05-21 11:25:30 -07:00
24c53e057a
YARN-9546. Add configuration option for YARN Native services AM classpath. Contributed by Gergely Pollak.
2019-05-20 10:53:01 -04:00
12c81610e0
YARN-9505. Add container allocation latency for Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-05-17 12:03:21 -07:00
fab5b80a36
YARN-9554. Fixed TimelineEntity DAO serialization handling.
...
Contributed by Prabhu Joseph
2019-05-16 16:39:50 -04:00
55bd35921c
YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko.
2019-05-15 11:50:46 -07:00
570fa2da20
YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T.
2019-05-15 13:30:09 +05:30
2de1e30658
YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T.
2019-05-15 13:21:39 +05:30
7d831eca64
YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal.
2019-05-14 10:48:08 -07:00
206e633946
YARN-9453. Clean up code long if-else chain in ApplicationCLI#run. Contributed by Wanqiang Ji.
2019-05-13 11:11:20 -07:00
29ff7fb140
YARN-9493. Scheduler Page does not display the right page by query string. Contributed by Wanqiang Ji.
2019-05-13 10:57:12 -07:00
1a47c2b7ae
YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang.
2019-05-12 22:31:39 -07:00
64c7f36ab1
YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl.
2019-05-10 14:27:23 +05:30
1b48100a5e
YARN-9522. AppBlock ignores full qualified class name of PseudoAuthenticationHandler. Contributed by Prabhu Joseph.
2019-05-09 14:02:58 -07:00
ec36126346
YARN-9483. DistributedShell does not release container when failed to localize at launch. Contributed by Prabhu Joseph.
2019-05-09 10:59:31 -07:00
90add05caa
YARN-9489. Support filtering by request-priorities and allocation-request-ids for query results of app activities. Contributed by Tao Yang.
2019-05-09 21:54:09 +08:00
0c5fa2e7d9
YARN-9535. Fixed spell check for container-executor in Docker documentation.
...
Contributed by Charan Hebri
2019-05-08 16:56:26 -04:00
3172f6cbf9
YARN-9513. Addendum patch: Fix ASF License warnings. Contributed by Giovanni Matteo Fumarola.
2019-05-08 14:56:23 +09:00
c336af3847
YARN-9432. Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled. Contributed by Tao Yang.
2019-05-08 09:54:16 +08:00
66c2a4ef89
HADOOP-16270. [JDK 11] Remove unintentional override of the version of Maven Dependency Plugin. Contributed by Xieming Li.
2019-05-08 10:50:07 +09:00
8ecbf61cca
YARN-9513. [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8. Contributed by Adam Antal.
2019-05-07 10:59:02 -07:00
49e1292ea3
YARN-9524. Fixed TestAHSWebService and TestLogsCLI unit tests.
...
Contributed by Prabhu Joseph
2019-05-06 19:48:45 -04:00
597fa47ad1
YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung)
2019-05-06 11:56:22 -07:00
12b7059ddc
YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang.
2019-05-06 20:00:15 +08:00
d331a2a0c2
YARN-9528. Federation RMs starting up at the same time can give duplicate application IDs. Contributed by Young Chen.
2019-05-03 15:05:17 -07:00
b094b94d43
YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein.
2019-05-02 19:39:26 +00:00
accb811e57
YARN-6929. Improved partition algorithm for yarn remote-app-log-dir.
...
Contributed by Prabhu Joseph
2019-04-30 17:04:59 -04:00
7fbaa7d66f
YARN-9476. [YARN-9473] Create unit tests for VE plugin. Contributed by Peter Bacsko.
2019-04-30 11:06:44 +08:00
79d3d35398
YARN-9486. Docker container exited with failure does not get clean up correctly. Contributed by Eric Yang
2019-04-26 01:21:28 +00:00
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
3f2f4186f6
YARN-9424. Change getDeclaredMethods to getMethods in FederationClientInterceptor#invokeConcurrent. Contributed by Shen Yinjie.
2019-04-23 19:58:41 -07:00
9d400627c2
YARN-9081. Update jackson from 1.9.13 to 2.x in hadoop-yarn-services-core.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-04-24 09:50:25 +09:00
fec9bf4b0b
YARN-9501. TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:42:56 -07:00
4a0ba24959
YARN-9491. TestApplicationMasterServiceFair#ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:27:04 -07:00
c504eee0c2
YARN-9339. Apps pending metric incorrect after moving app to a new queue. Contributed by Abhishek Modi.
2019-04-23 12:40:44 -07:00
8a95ea61e1
YARN-9475. [YARN-9473] Create basic VE plugin. Contributed by Peter Bacsko.
2019-04-23 17:33:58 +08:00
1c8046d67e
YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 14:21:13 +08:00
a54c1e3ace
YARN-8551. Project setup for MaWo application.
...
Contributed by Yesha Vora
2019-04-22 16:56:26 -04:00
96e3027e46
YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi.
2019-04-22 09:49:03 -07:00
adefd37105
YARN-9495. Fix findbugs warnings in hadoop-yarn-server-resourcemanager module. Contributed by Tao Yang.
2019-04-20 21:51:00 +08:00
317fcbabe3
YARN-9254. Add support for storing application catalog data to HDFS. Contributed by Eric Yang
2019-04-19 10:29:57 -07:00
aeadb9432f
YARN-9448. Fix Opportunistic Scheduling for node local allocations. Contributed by Abhishek Modi.
2019-04-19 09:41:06 -07:00
ef97a20831
YARN-8622. Fixed container-executor compilation on MacOSX.
...
Contributed by Siyao Meng
2019-04-18 18:59:21 -04:00
df76cdc895
YARN-6695. Fixed NPE in publishing appFinished events to ATSv2.
...
Contributed by Prabhu Joseph
2019-04-18 12:29:37 -04:00
aa4c744aef
YARN-9470. Fix order of actual and expected expression in assert statements
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-18 15:40:37 +09:00
6e4399ea61
YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-17 22:56:57 -07:00
9cf7401794
YARN-9349. Improved log level practices for InvalidStateTransitionException.
...
Contributed by Anuhan Torgonshar
(cherry picked from commit fe2370e039e1ee980d74769ae85d67434e0993cf)
2019-04-16 19:53:45 -04:00
b8086aed86
YARN-9123. Clean up and split testcases in TestNMWebServices for GPU support. Contributed by Szilard Nemeth.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-16 11:06:25 -07:00
2364c7d0bf
YARN-9466. Fixed application catalog navigation bar height in Safari. Contributed by Eric Yang
2019-04-16 10:04:27 -07:00
ad865888a6
YARN-8530. Add SPNEGO filter to application catalog. Contributed by Eric Yang
2019-04-16 09:52:14 -07:00
c65218ee8e
YARN-8943. Upgrade JUnit from 4 to 5 in hadoop-yarn-api.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-04-16 12:13:49 +09:00
5583e1b6fc
YARN-7848 Force removal of docker containers that do not get removed on first try. Contributed by Eric Yang
2019-04-15 20:47:09 +00:00
254efc9358
YARN-9379. Can't specify docker runtime through environment. Contributed by caozhiqiang
2019-04-15 18:24:37 +00:00
7fa73fac26
YARN-9439. Support asynchronized scheduling mode and multi-node lookup mechanism for app activities. Contributed by Tao Yang.
2019-04-16 00:12:43 +08:00
c4c16cae7d
YARN-9481. [JDK 11] Build fails due to hard-coded target version in hadoop-yarn-applications-catalog-webapp. Contributed by Kei Kori.
2019-04-15 13:08:28 +09:00
7a68e7abd5
YARN-9474. Remove hard coded sleep from Opportunistic Scheduler tests. Contributed by Abhishek Modi.
2019-04-14 20:11:20 -07:00
b2cdf809bc
YARN-9281. Add express upgrade button to Appcatalog UI. Contributed by Eric Yang
2019-04-13 18:55:11 +03:00
1943db5571
HADOOP-16237. Fix new findbugs issues after updating guava to 27.0-jre.
...
Author: Gabor Bota <gabor.bota@cloudera.com>
2019-04-12 18:28:38 -07:00
ed3747c1cc
YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi.
2019-04-11 11:49:19 -07:00
8c1bba375b
YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu.
2019-04-10 22:51:28 +08:00
32722d2661
YARN-9433. Remove unused constants in YARN resource manager
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-10 18:37:27 +09:00
cfec455c45
YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri.
2019-04-09 10:59:43 -07:00
fc05b0e70e
YARN-9313. Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities. Contributed by Tao Yang.
2019-04-08 13:40:53 +08:00
ec143cbf67
YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang.
2019-04-06 19:59:36 +08:00
22362c876d
YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi.
2019-04-05 12:06:51 -07:00
27039a29ae
YARN-9382 Publish container killed, paused and resumed events to ATSv2. Contributed by Abhishesk Modi.
2019-04-05 12:02:43 -07:00
2bf82aee82
YARN-9441. Updated YARN app catalog name for consistency.
...
Contributed by Weiwei Yang
2019-04-04 13:21:30 -04:00
8d150067e2
YARN-9396. Fixed duplicated RM Container created event to ATS.
...
Contributed by Prabhu Joseph
2019-04-04 13:01:56 -04:00
945e8c6064
YARN-9394. Use new API of RackResolver to get better performance. Contributed by Lantao Jin.
2019-04-04 18:05:01 +08:00
eb03f7c419
YARN-9303 Username splits won't help timelineservice.app_flow table. Contributed by Prabhu Joseph.
2019-04-03 22:53:05 -07:00
d7979079ea
HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
2019-04-03 12:59:39 -06:00
002dcc4ebf
YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
2019-04-03 18:57:28 +05:30
2f752830ba
YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji.
2019-04-01 20:05:15 -07:00
ab2bda57bd
YARN-9428. Add metrics for paused containers in NodeManager. Contributed by Abhishek Modi.
2019-04-01 14:21:17 -07:00
da7f8c244d
YARN-9431. Fix flaky junit test fair.TestAppRunnability after YARN-8967. Contributed by Wilfred Spiegelenburg.
2019-04-01 11:21:31 -07:00
332cab5518
YARN-9418. ATSV2 /apps//entities/YARN_CONTAINER rest api does not show metrics. Contributed by Prabhu Joseph.
2019-04-01 11:06:51 -07:00
b0d24ef39c
YARN-9227. DistributedShell RelativePath is not removed at end. Contributed by Prabhu Joseph.
2019-04-01 10:59:14 -07:00
35b0a381e7
YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
2019-04-01 09:21:36 -07:00
d9e9e5657d
YARN-9348. Application catalog build system bug fixes. Contributed by Eric Yang
2019-03-29 13:51:21 -07:00
7ee32a2eb1
YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
2019-03-29 13:51:21 -07:00
56f1e131ec
YARN-9270. Minor cleanup in TestFpgaDiscoverer. Contributed by Peter Bacsko.
2019-03-29 10:58:56 -07:00
9cd66198ee
YARN-9411. TestYarnNativeServices fails sporadically with bind address in use. Contributed by Prabhu Joseph.
2019-03-27 10:44:54 -07:00
a4cd75e09c
YARN-9269. Minor cleanup in FpgaResourceAllocator. Contributed by Peter Bacsko.
2019-03-27 10:08:07 -07:00
5257f50abb
YARN-8967. Change FairScheduler to use PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-25 22:47:24 -07:00
710cbc9bd6
YARN-9405. Fixed flaky tests in TestYarnNativeServices.
...
Contributed by Prabhu Joseph
2019-03-25 16:34:04 -04:00
eeda6891e4
YARN-9268. General improvements in FpgaDevice. Contributed by Peter Bacsko.
2019-03-25 13:22:53 -07:00
3c45762a0b
YARN-9391. Fixed node manager environment leaks into Docker containers.
...
Contributed by Jim Brennan
2019-03-25 15:53:24 -04:00
509b20b292
YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent. Contributed by Prabhu Joseph.
2019-03-22 11:45:39 -07:00
ce5eb9cb2e
YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322
...
(Contributed by Zoltan Siegl via Daniel Templeton)
Change-Id: I92d52c0ca630e71afb26b2b7587cbdbe79254a05
2019-03-22 12:28:34 +01:00
548997d6c9
YARN-9402. Opportunistic containers should not be scheduled on Decommissioning nodes. Contributed by Abhishek Modi.
2019-03-21 12:04:05 -07:00
a99eb80659
YARN-9267. General improvements in FpgaResourceHandlerImpl. Contributed by Peter Bacsko.
2019-03-21 11:15:56 -07:00
506502bb83
YARN-9370. Added logging for recovering assigned GPU devices.
...
Contributed by Yesha Vora
2019-03-20 19:12:19 -04:00
f2b862cac6
YARN-9398. Fixed javadoc errors for FPGA related java files.
...
Contributed by Peter Bacsko
2019-03-20 15:45:37 -04:00
b3b0e332e6
YARN-9299. TestTimelineReaderWhitelistAuthorizationFilter ignores Http Errors. Contributed by Prabhu Joseph.
2019-03-20 21:24:31 +05:30
0d24684eee
YARN-9357. Modify HBase Liveness monitor log to debug. Contributed by Prabhu Joseph.
2019-03-20 21:22:54 +05:30
f6e60e966e
YARN-9387. Update document for ATS HBase Custom tablenames (-entityTableName). Contributed by Prabhu Joseph.
2019-03-20 21:20:50 +05:30
c1a4eeb7c8
YARN-9389. FlowActivity and FlowRun table prefix is wrong. Contributed by Prabhu Joseph.
2019-03-20 21:18:19 +05:30
5d8bd0e5cb
YARN-9392. Handle missing scheduler events in Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-03-19 11:00:21 -07:00
c9e50c48de
YARN-9365. Fix table creation command in TimelineServiceV2.md
2019-03-19 12:05:37 +05:30
09eabda314
YARN-9364. Remove commons-logging dependency from YARN.
...
Contributed by Prabhu Joseph
2019-03-18 19:58:42 -04:00
5f6e225166
YARN-9363. Replaced debug logging with SLF4J parameterized log message.
...
Contributed by Prabhu Joseph
2019-03-18 13:57:18 -04:00
19b22c4385
YARN-9385. Fixed ApiServiceClient to use current UGI.
...
Contributed by Eric Yang
2019-03-18 13:16:34 -04:00
0e7e9013d4
YARN-9340. [Clean-up] Remove NULL check before instanceof in ResourceRequestSetKey
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I932e29b36f086f7b7c76a250e33b473617ddbda1
2019-03-18 15:08:37 +01:00
2064ca015d
YARN-9349. Changed logging to use slf4j api.
...
Contributed by Prabhu Joseph
2019-03-15 19:20:59 -04:00
03f3c8aed2
YARN-4404. Corrected typo in javadoc.
...
Contributed by Yesha Vora
2019-03-15 18:04:04 -04:00
688b177fc6
YARN-8376. Separate white list for docker.trusted.registries and docker.privileged-container.registries. Contributed by Eric Yang
2019-03-14 19:39:00 +00:00
f235a942d5
YARN-9016 DocumentStore as a backend for ATSv2. Contributed by Sushil Ks.
2019-03-13 16:45:23 -07:00
17a3e14d25
YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi.
2019-03-12 21:33:17 -07:00
67cc24a7a4
YARN-9381. The yarn-default.xml has two identical property named yarn.timeline-service.http-cross-origin.enabled. Contributed by Abhishek Modi.
2019-03-12 15:21:03 -07:00
8e1539eca8
YARN-9266. General improvements in IntelFpgaOpenclPlugin. Contributed by Peter Bacsko.
2019-03-13 02:45:17 +05:30
ab574ffd72
Revert "YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang"
...
This reverts commit 2576401928
2019-03-12 11:50:16 -07:00
55069bfb09
Revert "YARN-9348. Application catalog build system bug fixes."
...
This reverts commit 01ada40ea4
2019-03-12 11:49:55 -07:00
024b3bae8a
Revert "YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang"
...
This reverts commit aab7b77536
2019-03-12 11:48:55 -07:00
aab7b77536
YARN-9255. Improve recommend applications order and fix findbugs warnings. Contributed by Eric Yang
2019-03-12 10:06:09 -07:00
c24af4b0d6
YARN-9336. JobHistoryServer leaks CLOSE_WAIT tcp connections when using LogAggregationIndexedFileController. Contributed by Tarun Parimi.
2019-03-12 20:57:27 +05:30
fa7a0b269a
YARN-8805. Automatically convert the launch command to the exec form when using entrypoint support
2019-03-11 12:55:01 -05:00
de15a66d78
YARN-9265. FPGA plugin fails to recognize Intel Processing Accelerator Card. Contributed by Peter Bacsko.
2019-03-08 17:39:22 +05:30
373705fcea
YARN-9239. Document docker registry deployment with Ozone CSI driver. Contributed by Eric Yang.
2019-03-08 09:47:20 +08:00
39b4a37e02
YARN-9341. Fixed enentrant lock usage in YARN project.
...
Contributed by Prabhu Joseph
2019-03-07 16:47:45 -05:00
491313ab84
YARN-8218 Add application launch time to ATSV1. Contributed by Abhishek Modi
2019-03-06 21:47:29 -08:00
01ada40ea4
YARN-9348. Application catalog build system bug fixes.
...
Contributed by Eric Yang
2019-03-06 17:55:48 -05:00
c79f139519
YARN-8803. [UI2] Show flow runs in the order of recently created time in graph widgets. Contributed by Akhil PB.
2019-03-06 16:49:08 +05:30
46045c5cb3
YARN-9138. Improve test coverage for nvidia-smi binary execution of GpuDiscoverer. Contributed by Szilard Nemeth.
2019-03-06 16:01:08 +05:30
7b42e0e32a
YARN-7266. Fixed deadlock in Timeline Server thread initialization.
...
Contributed by Prabhu Joseph
2019-03-05 12:17:01 -05:00
2576401928
YARN-7129. Application Catalog for YARN applications. Contributed by Eric Yang
2019-03-05 06:14:14 -08:00
0aefe2846f
YARN-9298. Implement FS placement rules using PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-04 23:49:07 -08:00
e40e2d6ad5
YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-03-05 14:10:08 +09:00
15098df744
Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph."
...
This reverts commit a63c358b78
2019-03-04 16:57:31 +05:30
bd8d299ded
YARN-7477. Moving logging APIs over to slf4j in hadoop-yarn-common. Contributed by Prabhu Joseph.
2019-03-04 16:59:20 +09:00
e20b5ef52c
YARN-9332. RackResolver tool should accept multiple hosts. Contributed by Lantao Jin.
2019-03-04 16:59:20 +09:00
cab8529ecb
YARN-7904. Privileged, trusted containers should be supported only in ENTRYPOINT mode. Contributed by Eric Yang.
2019-03-01 11:06:09 -08:00
dcaca19871
YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth.
2019-03-01 19:24:35 +05:30
538bb4880d
YARN-9323. FSLeafQueue#computeMaxAMResource does not override zero values for custom resources
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: Id844ccf09488f367c0c7de0a3b2d4aca1bba31cc
2019-02-27 19:59:48 -08:00
7b928f19a4
YARN-9322. Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: I14c12f1265999d62102f2ec5506d90015efeefe8
2019-02-27 19:43:50 -08:00
1779fc57a1
YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph.
2019-02-28 09:56:29 +08:00
04b228e43b
YARN-9334. Allow YARN Service client to send SPNEGO challenge header when authentication type is not simple.
...
Contributed by Billie Rinaldi
2019-02-27 18:47:14 -05:00
ea3cdc60b3
YARN-3841 [atsv2 Storage implementation] Adding retry semantics to HDFS backing storage. Contributed by Abhishek Modi.
2019-02-27 14:55:35 -08:00
0ec962ac8f
YARN-5336 Limit the flow name size & consider cleanup for hex chars. Contributed by Sushil Ks
2019-02-27 14:43:39 -08:00
fbc7bb315f
YARN-9245. Added query docker image command ability to node manager.
...
Contributed by Chandni Singh
2019-02-27 14:57:24 -05:00
29e27faf96
YARN-9318. Resources#multiplyAndRoundUp does not consider Resource Types
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: Ia45f528574c2b054f6f764d1d140e592bdb7e217
2019-02-27 10:29:17 -08:00
8c30114b00
YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie.
2019-02-27 17:29:02 +08:00
6c96f5e4b6
YARN-8378. ApplicationHistoryManagerImpl#getApplications doesn't honor filters. Contributed by Lantao Jin.
2019-02-27 10:32:58 +05:30
8eae260af5
YARN-9311. Fix TestRMRestart hangs. Contributed by Prabhu Joseph.
2019-02-27 10:28:16 +05:30
a106d2dc9d
YARN-8783. Improve the documentation for the docker.trusted.registries configuration. Contributed by Eric Yang.
2019-02-26 10:19:06 -08:00
c6ea28c480
YARN-9331. [YARN-8851] Fix a bug that lacking cgroup initialization when bootstrap DeviceResourceHandlerImpl. Contributed by Zhankun Tang.
2019-02-26 10:05:31 +08:00
95372657fc
YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena.
2019-02-25 11:45:37 -08:00
3e1739d589
YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang.
2019-02-26 00:08:13 +08:00
6cec90653d
YARN-9168. DistributedShell client timeout should be -1 by default. Contributed by Zhankun Tang.
2019-02-25 15:28:45 +05:30
5e91ebd91a
YARN-9121. Replace GpuDiscoverer.getInstance() to a readable object for easy access control. Contributed by Szilard Nemeth.
2019-02-25 11:30:46 +05:30
9cd5c5447f
YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph.
2019-02-24 22:42:27 +08:00
50094d7fef
YARN-9300. Lazy preemption should trigger an update on queue preemption metrics for CapacityScheduler. Contributed by Tao Yang.
2019-02-24 22:17:29 +08:00
dddcfa4d9f
YARN-8821. [YARN-8851] GPU hierarchy/topology scheduling support based on pluggable device framework. Contributed by Zhankun Tang.
2019-02-24 14:37:06 +05:30
05bce33d6e
YARN-9244. Document docker registry deployment with direct S3 driver.
...
Contributed by Suma Shivaprasad
2019-02-22 19:13:52 -05:00
ed13cf8406
YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph
2019-02-22 21:01:34 +05:30
95fbbfed75
YARN-9118. Handle exceptions with parsing user defined GPU devices in GpuDiscoverer. Contributed by Szilard Nemeth.
2019-02-22 20:22:17 +05:30
9636fe4114
YARN-8891. Documentation of the pluggable device framework. Contributed by Zhankun Tang.
2019-02-22 20:00:13 +05:30
9c88695bcd
YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie.
2019-02-22 20:58:33 +08:00
632d5e8a98
YARN-9156. Improve debug message in device plugin method compatibility check of ResourcePluginManager. Contributed by Zhankun Tang.
2019-02-22 11:27:30 +08:00
b17a2602d1
YARN-9319. Fix compilation issue of handling typedef an existing name by gcc compiler. (Zhankun Tang via wangda)
...
Change-Id: Ie8293cc5b710ff355fb6a1ea7201be53ed0385bf
2019-02-21 11:17:32 -08:00
6c4ab0312b
YARN-9258. Support to specify allocation tags without constraint in distributed shell CLI. Contributed by Prabhu Joseph.
2019-02-22 00:18:07 +08:00
eedcc8e26e
YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph.
2019-02-21 17:21:21 +08:00
a63c358b78
YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph.
2019-02-21 13:29:01 +05:30
b8de78c570
YARN-9286. [Timeline Server] Sorting based on FinalStatus shows pop-up message. Contributed by Bilwa S T.
2019-02-20 01:19:20 +05:30
db4d1a1e2f
YARN-9060. [YARN-8851] Phase 1 - Support device isolation and use the Nvidia GPU plugin as an example. Contributed by Zhankun Tang.
2019-02-18 15:58:04 +05:30
0f2b65c3da
HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
2019-02-18 15:21:46 +09:00
ba56bc24f5
YARN-9213. RM Web UI v1 does not show custom resource allocations for containers page. Contributed by Szilard Nemeth.
2019-02-17 20:08:20 +05:30
9057aa9fc5
SUBMARINE-1. Move code base of submarine from yarn-applications to top directory. Contributed by Wangda Tan.
2019-02-17 20:03:56 +05:30
7c1b561e33
YARN-8927. Added support for top level Dockerhub images to trusted registry using library keyword.
...
Contributed by Zhankun Tang
2019-02-15 11:18:07 -05:00
9385ec45d7
YARN-9283. Javadoc of LinuxContainerExecutor#addSchedPriorityCommand has a wrong property name as reference
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-02-15 18:47:10 +09:00
5b55f3538c
YARN-8295. [UI2] Improve Resource Usage tab error message when there are no data available. Contributed by Charan Hebri.
2019-02-15 12:42:21 +05:30
3a39d9a2d2
YARN-9284. Fix the unit of yarn.service.am-resource.memory in the document. Contributed by Masahiro Tanaka.
2019-02-15 15:41:53 +09:00
b66d5ae9e2
YARN-9295. [UI2] Fix label typo in Cluster Overview page. Contributed by Charan Hebri.
2019-02-14 23:08:19 +05:30
dfe0f42835
YARN-7824. [UI2] Yarn Component Instance page should include link to container logs. Contributed by Akhil PB.
2019-02-14 15:42:19 +05:30
3dc2523266
YARN-9184. Add a system flag to allow update to latest docker images.
...
Contributed by Zhaohui Xin
2019-02-12 16:16:35 -05:00
7536488bbd
YARN-996. REST API support for node resource configuration. Contributed by Inigo Goiri.
2019-02-11 15:27:43 -08:00
1ce2e91c4b
YARN-9229. Document docker registry deployment with NFS Gateway. Contributed by Eric Yang.
2019-02-11 14:54:30 -08:00
0a1637c750
YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph.
2019-02-11 15:53:50 +08:00
2b7f828d46
YARN-9252. Allocation Tag Namespace support in Distributed Shell. Contributed by Prabhu Joseph.
2019-02-11 14:42:21 +08:00
e0ab1bdece
YARN-9282. Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'. Contributed by Charan Hebri.
2019-02-09 00:26:50 +09:00
d1ca9432dd
YARN-7171: RM UI should sort memory / cores numerically. Contributed by Ahmed Hussein
2019-02-07 16:38:11 +00:00
3c96a03dea
YARN-8498. Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04. Contributed by Ayush Saxena.
2019-02-07 13:02:35 +05:30
7fa62e150c
YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi.
2019-02-07 09:58:53 +05:30
1e5e08d83b
YARN-7627. [ATSv2] When passing a non-number as metricslimit, the error message is wrong. Contributed by Charan Hebri.
2019-02-07 09:58:53 +05:30
711d22f166
YARN-9253. Add UT to verify Placement Constraint in Distributed Shell. Contributed by Prabhu Joseph.
2019-02-06 22:36:51 +08:00
308f3168fa
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
2044967e75
YARN-9246 NPE when executing a command yarn node -status or -states without additional arguments. Contributed by Masahiro Tanaka
2019-02-05 09:39:38 -08:00
e3ec18b0c4
YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi.
2019-02-05 13:48:04 +05:30
ba38db4f5b
YARN-9257. Distributed Shell client throws a NPE for a non-existent queue. Contributed by Charan Hebri.
2019-02-05 13:41:44 +05:30
aa7ce50e55
YARN-9275. Add link to NodeAttributes doc in PlacementConstraints document. Contributed by Masatake Iwasaki.
2019-02-05 16:07:41 +08:00
571838920d
YARN-9149. yarn container -status misses logUrl when integrated with ATSv2. Contributed by Abhishek Modi.
2019-02-05 12:39:10 +05:30
604b2489a9
YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla.
2019-02-04 12:48:28 +05:30
69bcff380f
YARN-9231. TestDistributedShell fix timeout. Contributed by Prabhu Joseph.
2019-02-02 23:27:31 +08:00
f20b043a02
YARN-9263. TestConfigurationNodeAttributesProvider fails after Mockito updated. Contributed by Weiwei Yang.
2019-02-02 23:04:34 +08:00
28ad20a711
YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie.
2019-02-02 23:02:45 +08:00
2c13513040
YARN-8549 Adding a NoOp timeline writer and reader plugin classes for ATSv2. Contributed by Prabha Manepalli.
2019-02-01 14:26:50 -08:00
f738b397ae
YARN-9191. Add cli option in DS to support enforceExecutionType in resource requests. Contributed by Abhishek Modi.
2019-01-31 11:24:15 -08:00
71c49fa60f
YARN-9099. GpuResourceAllocator#getReleasingGpus calculates number of GPUs in a wrong way. Contributed by Szilard Nemeth.
2019-01-31 09:25:29 +05:30
a3a9ae3cea
YARN-9251. Build failure for -Dhbase.profile=2.0. Contributed by Rohith Sharma K S.
2019-01-30 05:17:04 -08:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
04105bbfdb
YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph
2019-01-29 17:22:24 +00:00
085f0e8ae7
YARN-9086. [CSI] Run csi-driver-adaptor as aux service. Contributed by Weiwei Yang.
2019-01-29 14:53:08 +08:00
4f63ffe444
YARN-9237. NM should ignore sending finished apps to RM during RM fail-over. Contributed by Jiandan Yang.
2019-01-29 10:34:26 +08:00
f5a95f7998
YARN-8901. Fixed restart policy NEVER/ON_FAILURE with component dependency.
...
Contributed by Suma Shivaprasad
2019-01-28 18:10:33 -05:00
2e636dd3c4
YARN-9074. Consolidate docker removal logic in ContainerCleanup.
...
Contributed by Zhaohui Xin
2019-01-28 18:05:53 -05:00
1ab69a9543
YARN-9221. Added flag to disable dynamic auxiliary service feature.
...
Contributed by Billie Rinaldi
2019-01-25 19:05:36 -05:00
6cace58e21
YARN-9222. Print launchTime in ApplicationSummary
2019-01-25 13:17:00 -08:00
84bb9808a2
YARN-7761. [UI2] Clicking 'master container log' or 'Link' next to 'log' under application's appAttempt goes to Old UI's Log link. Contributed by Akhil PB.
2019-01-25 14:00:35 +05:30
a33ef4fd31
YARN-8867. Added resource localization status to YARN service status call.
...
Contributed by Chandni Singh
2019-01-24 18:43:21 -05:00
a4bd64e724
YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu.
2019-01-24 17:53:43 +08:00
c726445990
YARN-8961. [UI2] Flow Run End Time shows 'Invalid date'. Contributed by Akhil PB
2019-01-24 15:01:43 +05:30
721d5c2a5f
YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph.
2019-01-23 17:59:36 +08:00
e3e076dc78
YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang.
2019-01-23 14:14:49 +08:00
2fa9389c2e
YARN-9146. Added REST API to configure auxiliary service.
...
Contributed by Billie Rinaldi
2019-01-22 18:24:43 -05:00
d43df31751
YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang.
2019-01-22 10:23:32 +08:00
abde1e1f58
YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang.
2019-01-21 16:54:14 +08:00
27aa6e8899
HADOOP-16046. [JDK 11] Correct the compiler exclusion of org/apache/hadoop/yarn/webapp/hamlet/** classes for >= Java 9. Contributed by Devaraj K.
2019-01-21 10:54:58 +09:00
824dfa3b09
YARN-8489. Support "dominant" component concept in YARN service.
...
Contributed by Zac Zhou
2019-01-18 20:23:04 -05:00
8c7f6b2d4d
YARN-9197. Add safe guard against NPE for component instance failure.
...
Contributed by kyungwan nam
2019-01-18 13:49:56 -05:00
6d7eedfd28
YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda)
...
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
2019-01-17 14:20:10 -08:00
0a46baecd3
YARN-9203. Fix typos in yarn-default.xml.
...
This closes #437
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-01-17 16:18:33 +09:00
104ef5df36
YARN-8747. [UI2] YARN UI2 page loading failed due to js error under some time zone configuration. Contributed by collinma.
2019-01-16 14:36:36 +09:00
713ded6b15
YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
2019-01-15 21:28:10 -08:00
6be39230a6
YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks
2019-01-15 21:25:37 -08:00
614af50625
YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo.
2019-01-15 12:59:59 +09:00
6a923464af
YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R
2019-01-08 16:54:05 -06:00
0921b706f7
YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan.
2019-01-08 14:30:53 +08:00
489411579c
YARN-9169. Add metrics for queued opportunistic and guaranteed containers. Contributed by Abhishek Modi.
2019-01-07 15:16:55 -08:00
0a01d49917
YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda)
...
Change-Id: Id8af27134d3286a7a10d85eda9be25df9689d0e7
2019-01-07 12:07:26 -08:00
5db7c49062
YARN-9162. Fix TestRMAdminCLI#testHelp. Contributed by Ayush Saxena.
2019-01-07 17:32:28 +09:00
944cf87223
YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg.
2019-01-07 15:57:31 +08:00
d3321fbed0
Revert "YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi."
...
This reverts commit 650b9cbe16
2019-01-07 08:58:09 +08:00
2c02aa6ec2
YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda)
...
Change-Id: I115b6710eb11501e21ab3e9e7cd1a87f322e5cf8
2019-01-06 10:25:35 -08:00
8f004fe4e0
YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda)
...
Change-Id: Ib2b17cf7860ddc9188f06498086de3326bc09519
2019-01-06 10:24:29 -08:00
650b9cbe16
YARN-9178. TestRMAdminCli#testHelp is failing in trunk. Contributed by Abhishek Modi.
2019-01-06 11:29:59 +08:00
6e35f7130f
YARN-9166. Fix logging for preemption of Opportunistic containers for Guaranteed containers. Contributed by Abhishek Modi.
2019-01-04 11:16:41 -08:00
Szilard Nemeth
HUAN-PING SU
Adam Antal
bibinchundatt
Eric Badger
Abhishek Modi
Eric Yang
Sunil G
Charles Xu
Eric E Payne
hunshenshi
Haibo Chen
Vidura Mudalige
Siyao Meng
Weiwei Yang
Akira Ajisaka
Giovanni Matteo Fumarola
Zhankun Tang
Wanqiang Ji
Suma Shivaprasad
Ahmed Hussein
Igor Rudenko
Sean Mackrory
Inigo Goiri
Billie Rinaldi
Prabhu Joseph
Szilard Nemeth
Gabor Bota
Igor Rudenko
Vrushali C
Yufei Gu
Devaraj K
Zoltan Siegl
Rohith Sharma K S
Shweta Yakkali
Eric Badger
Yufei Gu
Wangda Tan
Masatake Iwasaki
Vinayakumar B
Jonathan Hung
Takanobu Asanuma
rahul3
Jason Lowe