Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,231 Commits 326 Branches 371 Tags
Commit Graph

3840 Commits

Author SHA1 Message Date
Haibo Chen
895a0391fe YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759f773135f8a27dbaa9731196fac5233)
(cherry picked from commit 8793e45f53607315bf6c99902452ba6cf2e3eb72)
 2018-01-19 10:35:40 -08:00
Miklos Szegedi
86a2ac94b1 YARN-7758. Add an additional check to the validity of container and application ids passed to container-executor. Contributed by Yufei Gu. 2018-01-17 12:56:10 -08:00
Rohith Sharma K S
8985fccbbc YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 
(cherry picked from commit d09058b2fd18803d12f0835fdf78aef5e0b99c90)
 2018-01-16 09:01:02 +05:30
Miklos Szegedi
09428b137f YARN-7705. Create the container log directory with correct sticky bit in C code. Contributed by Yufei Gu. 
(cherry picked from commit 2dcfc18)
 2018-01-12 17:44:31 -08:00
Sunil G
e7a8d6bd1e YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph. 
(cherry picked from commit 128d773a2315fa6baaa3a52b13c53c77e741b69c)
 2018-01-12 15:56:08 +05:30
Miklos Szegedi
35527db157 YARN-7590. Improve container-executor validation check. Contributed by Eric Yang. 
(cherry picked from commit bc285da)
 2018-01-11 10:30:15 -08:00
Akira Ajisaka
68b7908aa8
YARN-7735. Fix typo in YARN documentation. Contributed by Takanobu Asanuma. 
(cherry picked from commit fbbbf59c82658e18dad7e0e256613187b5b75d0f)
 2018-01-11 16:00:23 +09:00
Robert Kanter
ff67c68d3c YARN-7622. Allow fair-scheduler configuration on HDFS (gphillips via rkanter) 2018-01-10 15:03:33 -08:00
bibinchundatt
73478e3bbe YARN-7508. NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode. Contributed by Tao Yang. 
(cherry picked from commit b26e30ab1655a8cdf369862a5512db5c2a88ec46)
 2018-01-10 10:31:46 +05:30
Haibo Chen
a58034cdfd YARN-7716. metricsTimeStart and metricsTimeEnd should be all lower case in the doc. 
(cherry picked from commit f725b9e267c604fbcec09956e3a39caf04798809)
(cherry picked from commit cf816bc751e3b82be1f1931d6d0d5cbd77cb3e17)
 2018-01-09 09:00:23 -08:00
Jason Lowe
c892789896 YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie 
(cherry picked from commit 90c86d6e090d668414681fcb5eb02add5d419b33)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2018-01-09 10:59:33 -06:00
Jason Lowe
f15c7e86e8 YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie 
(cherry picked from commit d2c2d8c913dfc12585394a1168d4a92934a4cfb4)
 2018-01-09 10:01:06 -06:00
Ray Chiang
18c3982851 YARN-4227. Ignore expired containers from removed nodes in FairScheduler. (Wilfred Spiegelenburg via rchiang) 
(cherry picked from commit bc2d67d6c10619716ef7acce263f3269a86c3150)
 2018-01-08 16:23:29 -08:00
Rohith Sharma K S
37a8224101 YARN-7699. queueUsagePercentage is coming as INF for getApp REST api call. Contributed by Sunil G. 
(cherry picked from commit 281350674c70d65233363849a8b60e9b7dc965cf)
 2018-01-08 15:52:41 +05:30
Sunil G
cac8a8616e YARN-7619. Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user. Contributed by Eric Payne. 
(cherry picked from commit a79abbc03e75a5d2982de12587bd3b45169cc884)
 2018-01-05 14:51:29 +05:30
Jason Lowe
42964c2a66 YARN-7678. Ability to enable logging of container memory stats. Contributed by Jim Brennan 2018-01-04 10:21:40 -06:00
Rohith Sharma K S
e46142a8c2 YARN-7692. Skip validating priority acls while recovering applications. Contributed by Sunil G. 
(cherry picked from commit c9bf813c9a6c018d14f2bef49ba086ec0e60c761)
 2018-01-03 18:29:38 +05:30
Rohith Sharma K S
0052353ecd YARN-7674. Update Timeline Reader web app address in UI2. Contributed by Sunil G. 
(cherry picked from commit 13ad7479b0e35a2c2d398e28c676871d9e672dc3)
 2017-12-20 22:30:29 +05:30
Sunil G
320f564bae YARN-7032. [ATSv2] NPE while starting hbase co-processor when HBase authorization is enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit d62932c3b2fcacc81dc1f5048cdeb60fb0d38504)
 2017-12-20 11:39:38 +05:30
Varun Saxena
d72d2440f4 YARN-7662. [ATSv2] Define new set of configurations for reader and collectors to bind (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit c0aeb666a4d43aac196569d9ec6768d62139d2b9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2017-12-19 22:51:56 +05:30
Jason Lowe
31a06baca0 YARN-7661. NodeManager metrics return wrong value after update node resource. Contributed by Yang Wang 
(cherry picked from commit 811fabdebe881248756c0165bf7667bfc22be9bb)
 2017-12-18 15:27:27 -06:00
Jason Lowe
0f8de8cd91 YARN-7595. Container launching code suppresses close exceptions after writes. Contributed by Jim Brennan 2017-12-15 15:54:56 -06:00
Weiwei Yang
a85036aed1 YARN-7642. Add test case to verify context update after container promotion or demotion with or without auto update. Contributed by Weiwei Yang. 
(cherry picked from commit 89b6c482c1720a7f7ac86ce022c403825a086fa0)
(cherry picked from commit 108f8b8fa1dd556eb27f3fd9c2a700947cfe4aa0)
 2017-12-15 12:44:14 +08:00
Subru Krishnan
8d3d7fa1c5 YARN-7630. Fix AMRMToken rollover handling in AMRMProxy. Contributed by Botong Huang. 
(cherry picked from commit 17ba74be29193ac15474f73baaaf4e647a95078b)
 2017-12-14 14:07:44 -08:00
Weiwei Yang
d69b7358b6 YARN-7647. NM print inappropriate error log when node-labels is enabled. Contributed by Yang Wang. 
(cherry picked from commit 7efc4f76885348730728c0201dd0d1a89b213e9c)
 2017-12-13 13:28:48 +08:00
Weiwei Yang
7b3c64aece YARN-7608. Incorrect sTarget column causing DataTable warning on RM application and scheduler web page. Contributed by Gergely Novák. 2017-12-11 13:06:13 +08:00
Wangda Tan
c27a5ba65c YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I67e086d6d35c3166ec0f6305490ea6527bd6d799
(cherry picked from commit 41796ea60ab9dbd6a41dd8fd3c6ff50990bba2f5)
 2017-12-08 21:50:41 -08:00
Subru Krishnan
850bd0ed7c YARN-6704. Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService. (Botong Huang via Subru). 
(cherry picked from commit 670e8d4ec7e71fc3b054cd3b2826f869b649a788)
 2017-12-08 15:44:45 -08:00
Weiwei Yang
029714e3ec YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang. 2017-12-08 13:53:33 +08:00
Robert Kanter
54b5da83ec YARN-5594. Handle old RMDelegationToken format when recovering RM (rkanter) 
(cherry picked from commit d8863fc16fa3cbcdda5b99f79386c43e4fae5917)
 2017-12-04 13:18:04 -08:00
Jason Lowe
3fb83cbeb2 YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719f00a718b484c36a823ec5aa8b099f4)
 2017-12-01 15:51:49 -06:00
Robert Kanter
edf9fd51fb YARN-4813. TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently (grepas via rkanter) 
(cherry picked from commit c83fe4491731c994a4867759d80db31d9c1cab60)
 2017-12-01 12:21:18 -08:00
Junping Du
14f251fcd5 YARN-7558. yarn logs command fails to get logs for running containers if UI authentication is enabled. Contributed by Xuan Gong. 
(cherry picked from commit a409425986fc128bb54f656b05373201545f7213)
 2017-11-30 13:49:45 -08:00
Jason Lowe
ee9ce1d75a YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles 
(cherry picked from commit 5cfaee2e6db8b2ac55708de0968ff5539ee3bd76)
 2017-11-30 12:44:00 -06:00
Wei Yan
5bb6329215 YARN-6851. Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat. 
(cherry picked from commit 333ef303ff0caf9adfd378652a8f966377901768)
 2017-11-29 14:46:47 -08:00
Yufei Gu
de144175a4 YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu) 
(cherry picked from commit d8923cdbf1567aee10a54f144fef734d1465ebed)
(cherry picked from commit c0ab3c8407b110dd9083d775e5ea823cbf73e7d4)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
 2017-11-27 14:54:27 -08:00
Wangda Tan
94f7b0f99f YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
 2017-11-23 20:06:16 -08:00
Haibo Chen
db2651cc9d YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. 2017-11-21 11:08:27 -08:00
Subru Krishnan
ed31091361 YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru). 2017-11-20 14:21:58 -08:00
bibinchundatt
ea8a121423 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 2017-11-21 00:31:15 +05:30
bibinchundatt
5c2986e58d YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 2017-11-20 21:31:50 +05:30
Varun Vasudev
9cf9627e20 YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
 2017-11-17 12:05:35 +05:30
Sunil G
a081d91480 YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 22:42:52 +05:30
Jason Lowe
ec4159c4df YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:33:48 -06:00
Junping Du
a72dcb9cad YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 2017-11-13 18:22:30 -08:00
Arun Suresh
c153bed6e3 YARN-5049. Addendum patch for branch-2. (asuresh) 2017-11-12 10:16:00 -08:00
Eric Yang
90ac8cd5a1 YARN-7412. Fix unit test for docker mount check on ubuntu. (Contributed by Eric Badger) 
(cherry picked from commit 7a49ddfdde2e2a7b407f4a62a42d97bfe456075a)
 2017-11-12 09:59:39 -08:00
Subru Krishnan
0282efab14 YARN-7476. Fix miscellaneous issues in ATSv2 after merge to branch-2. (Varun Saxena via Subru). 2017-11-12 09:19:30 -08:00
Subru Krishnan
3c5b46c2ed YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 2017-11-12 09:18:08 -08:00
Arun Suresh
5cd92ed555 YARN-6627. Addendum patch for branch-2. (Sunil G via asuresh) 2017-11-11 09:10:17 -08:00