Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,046 Commits 326 Branches 371 Tags
Commit Graph

3454 Commits

Author SHA1 Message Date
Eric Yang
0b97dc5869 YARN-8751. Reduce conditions that mark node manager as unhealthy. 
Contributed by Craig Condit

(cherry picked from commit 7d623343879ce9a8f8e64601024d018efc02794c)
 2018-09-07 20:32:11 -04:00
Shane Kumpf
2d68708a1d YARN-8638. Allow linux container runtimes to be pluggable. Contributed by Craig Condit 
(cherry picked from commit dffb7bfe6cd2292162f08ec0bded736bc5194c3f)
 2018-09-05 06:55:25 -06:00
bibinchundatt
e2e0fc26a2 YARN-8535. Fix DistributedShell unit tests. Contributed by Abhishek Modi. 
(cherry picked from commit eed8415dc18fa7415ebd105350bd0532b3b1b6bb)
 2018-09-02 13:37:38 +05:30
Shane Kumpf
b8618556ee YARN-8642. Add support for tmpfs mounts with the Docker runtime. Contributed by Craig Condit 
(cherry picked from commit 73625168c0f29aa646d7a715c9fb15e43d6c7e05)
 2018-08-29 07:11:38 -06:00
Weiwei Yang
f164568b47 YARN-8723. Fix a typo in CS init error message when resource calculator is not correctly set. Contributed by Abhishek Modi. 
(cherry picked from commit 3fa46394214181ed1cc7f06b886282bbdf67a10f)
 2018-08-29 11:15:02 +08:00
Billie Rinaldi
eefd780918 YARN-8675. Remove default hostname for docker containers when net=host. Contributed by Suma Shivaprasad 
(cherry picked from commit 05b2bbeb357d4fa03e71f2bfd5d8eeb0ea6c3f60)
 2018-08-27 11:42:09 -07:00
Haibo Chen
e4282c077b YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 
(cherry picked from commit 93d47a0ed504ee81d4b74d340c1815bdbb3c9b14)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMEmbeddedElector.java
 2018-08-24 13:24:08 -05:00
Jason Lowe
84654451fa YARN-8649. NPE in localizer hearbeat processing if a container is killed while localizing. Contributed by lujie 
(cherry picked from commit 585ebd873a55bedd2a364d256837f08ada8ba032)
 2018-08-23 09:37:43 -05:00
Rohith Sharma K S
63d5214332 YARN-8129. Improve error message for invalid value in fields attribute. Contributed by Abhishek Modi. 
(cherry picked from commit d3fef7a5c5b83d27e87b5e49928254a7d1b935e5)
 2018-08-21 12:11:12 +05:30
Wei-Chiu Chuang
0d155de159 HADOOP-14212. Addendum patch: Expose SecurityEnabled boolean field in JMX for other services besides NameNode. Contributed by Adam Antal. 2018-08-20 14:49:28 -07:00
Wei-Chiu Chuang
78fb14ba49 HADOOP-14212. Expose SecurityEnabled boolean field in JMX for other services besides NameNode. Contributed by Adam Antal. 2018-08-20 14:49:24 -07:00
Jason Lowe
44c4928b64 YARN-8242. YARN NM: OOM error while reading back the state store on recovery. Contributed by Pradeep Ambati and Kanwaljeet Sachdev 
(cherry picked from commit 65e7469712be6cf393e29ef73cc94727eec81227)
 2018-08-20 10:21:57 -05:00
Rohith Sharma K S
a3d4a25bbf YARN-8679. [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked. Contributed by Wangda Tan. 
(cherry picked from commit 4aacbfff605262aaf3dbd926258afcadc86c72c0)
 2018-08-18 11:04:09 +05:30
Eric Yang
5237bdfb5a YARN-8667. Cleanup symlinks when container restarted by NM. 
Contributed by Chandni Singh

(cherry picked from commit d42806160eb95594f08f38bb753cf0306a191a38)
 2018-08-16 18:44:47 -04:00
Jason Lowe
819a2a6f10 YARN-8656. container-executor should not write cgroup tasks files for docker containers. Contributed by Jim Brennan 
(cherry picked from commit cb21eaa026d80a2c9836030d959c0dd7f87c4d6b)
 2018-08-16 10:09:56 -05:00
Jason Lowe
95cd6de5c6 YARN-8640. Restore previous state in container-executor after failure. Contributed by Jim Brennan 
(cherry picked from commit d1d129aa9deecebf42261947fcb0b2ca46dacad5)
 2018-08-14 10:26:21 -05:00
Weiwei Yang
734bc42289 YARN-8575. Avoid committing allocation proposal to unavailable nodes in async scheduling. Contributed by Tao Yang. 
(cherry picked from commit 0a71bf145293adbd3728525ab4c36c08d51377d3)
 2018-08-10 15:10:27 +08:00
Weiwei Yang
991514f7c3 YARN-8521. NPE in AllocationTagsManager when a container is removed more than once. Contributed by Weiwei Yang. 
(cherry picked from commit 08d5060605af81a3d6048044176dc656c0dad56c)
 2018-08-10 08:44:53 +08:00
Wangda Tan
68279fcd65 YARN-8588. Logging improvements for better debuggability. (Suma Shivaprasad via wangda) 
Change-Id: I66aa4b0ec031ae5ce0fae558e2f8cbcbbfebc442
(cherry picked from commit 344c335a920e6f32a35ebace0a118a9dc4a22fb7)
 2018-08-09 12:04:25 -07:00
Weiwei Yang
0ee7e80047 YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 
(cherry picked from commit d352f167ebb865a6486afbbdac8e2a5e97a7bbad)
 2018-08-10 00:43:53 +08:00
Jason Lowe
3dd299a770 YARN-8331. Race condition in NM container launched after done. Contributed by Pradeep Ambati 
(cherry picked from commit cd04e954d2db27f0a15b7d1c492b7cdb656a51db)
 2018-08-09 10:23:02 -05:00
Wangda Tan
450c791ecf YARN-8629. Container cleanup fails while trying to delete Cgroups. (Suma Shivaprasad via wangda) 
Change-Id: I392ef4f8baa84d5d7b1f2e438c560b5426b6d4f2
(cherry picked from commit d4258fcad71eabe2de3cf829cde36840200ab9b6)
 2018-08-07 12:41:55 -07:00
Jason Lowe
619019ccca YARN-8263. DockerClient still touches hadoop.tmp.dir. Contributed by Craig Condit 
(cherry picked from commit 7526815e3234ca352854ecfb142a13f1a188d5bd)
 2018-08-02 10:45:52 -05:00
Sunil G
1f77b20f08 YARN-8593. Add RM web service endpoint to get user information. Contributed by Akhil PB. 
(cherry picked from commit 735b4925569541fb8e65dc0c668ccc2aa2ffb30b)
 2018-08-02 08:35:54 +05:30
Billie Rinaldi
2a94823f32 YARN-8403. Change the log level for fail to download resource from INFO to ERROR. Contributed by Eric Yang 
(cherry picked from commit 67c65da261464a0dccb63dc27668109a52e05714)
 2018-08-01 08:58:15 -07:00
Sunil G
ff35f0c308 YARN-8606. Opportunistic scheduling does not work post RM failover. Contributed by Bibin A Chundatt. 
(cherry picked from commit a48a0cc7fd8e7ac1c07b260e6078077824f27c35)
 2018-08-01 12:17:53 +05:30
Sunil G
cbfd7358d2 YARN-8397. Potential thread leak in ActivitiesManager. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6310c0d17d6422a595f856a55b4f1fb82be43739)
 2018-08-01 08:34:09 +05:30
Eric Yang
7640d62716 YARN-8579. Recover NMToken of previous attempted component data. 
Contributed by Gour Saha
 2018-07-31 18:35:31 -04:00
Wangda Tan
5583711419 Preparing for 3.1.2 release 
Change-Id: If2793e2ed2b5b349a9e1f98f78df43f309dcfcbd
 2018-07-31 13:08:55 -07:00
Wangda Tan
7b552c9d72 YARN-8418. App local logs could leaked if log aggregation fails to initialize for the app. (Bibin A Chundatt via wangda) 
Change-Id: I29a23ca4b219b48c92e7975cd44cddb8b0e04104
(cherry picked from commit 4b540bbfcf02d828052999215c6135603d98f5db)
 2018-07-31 12:13:36 -07:00
Jonathan Hung
b91cf90e1c YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 17:57:25 -07:00
bibinchundatt
8cd2a73777 YARN-8584. Several typos in Log Aggregation related classes. Contributed by Szilard Nemeth. 
(cherry picked from commit 2b39ad26984d641bad57db2cfcc0b7515ef95f46)
 2018-07-31 00:07:08 +05:30
Sunil G
f1eb5777a0 YARN-8591. [ATSv2] NPE while checking for entity acl in non-secure cluster. Contributed by Rohith Sharma K S. 
(cherry picked from commit 63e08ec071852640babea9e39780327a0907712a)
 2018-07-30 14:49:03 +05:30
bibinchundatt
2e7876a725 YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 
(cherry picked from commit 3d586841aba99c7df98b2b4d3e48ec0144bad086)
 2018-07-28 20:56:35 +05:30
Eric Yang
c2c3eee69c YARN-8508. Release GPU resource for killed container. 
Contributed by Chandni Singh

(cherry picked from commit ed9d60e888d0acfd748fda7f66249f5b79a3ed6d)
 2018-07-27 19:36:21 -04:00
Eric Yang
8e3807afe0 YARN-8330. Improved publishing ALLOCATED events to ATS. 
Contributed by Suma Shivaprasad

(cherry picked from commit f93ecf5c1e0b3db27424963814fc01ec43eb76e0)
 2018-07-25 18:51:42 -04:00
Eric E Payne
830ef12af8 YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6e9bb935c3e6ae4da81d70b621781de)
 2018-07-25 16:30:30 +00:00
bibinchundatt
8e65057eb1 YARN-8541. RM startup failure on recovery after user deletion. Contributed by Bibin A Chundatt. 2018-07-25 15:54:32 +05:30
Weiwei Yang
b89624a943 YARN-8546. Resource leak caused by a reserved container being released more than once under async scheduling. Contributed by Tao Yang. 
(Cherry-picked from commit 5be9f4a5d05c9cb99348719fe35626b1de3055db)
 2018-07-25 17:53:40 +08:00
Haibo Chen
7e7792dd7b YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 
(cherry picked from commit 9d3c39e9dd88b8f32223c01328581bb68507d415)
 2018-07-24 12:50:43 -07:00
Sunil G
4488fd8295 YARN-7748. TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events. Contributed by Weiwei Yang. 
(cherry picked from commit 35ce6eb1f526ce3db7e015fb1761eee15604100c)
 2018-07-24 22:21:15 +05:30
bibinchundatt
a684a2efb8 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 
(cherry picked from commit ff7c2eda34c2c40ad71b50df6462a661bd213fbd)
 2018-07-24 16:30:31 +05:30
bibinchundatt
0710107f8d YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 
(cherry picked from commit 84612788339392fcda1aef0e27c43f5c6b2a19e5)
 2018-07-24 13:11:31 +05:30
Eric Yang
23b8546a80 YARN-8380. Support bind propagation options for mounts in docker runtime. 
Contributed by Billie Rinaldi

(cherry picked from commit 8688a0c7f88f2adf1a7fce695e06f3dd1f745080)
 2018-07-23 20:13:41 -04:00
Weiwei Yang
004e1f248e YARN-8528. Final states in ContainerAllocation might be modified externally causing unexpected allocation results. Contributed by Xintong Song. 2018-07-20 22:43:47 +08:00
Eric Yang
76b8beb289 YARN-8501. Reduce complexity of RMWebServices getApps method. 
Contributed by Szilard Nemeth

(cherry picked from commit 5836e0a46bf9793e0a61bb8ec46536f4a67d38d7)
 2018-07-19 12:32:55 -04:00
Robert Kanter
dfa71428ea YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 
(cherry picked from commit 1bc106a738a6ce4f7ed025d556bb44c1ede022e3)
 2018-07-18 16:07:48 -07:00
Robert Kanter
1c7d916347 Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 0838fe833738e04f5e6f6408e97866d77bebbf30)
 2018-07-18 16:07:48 -07:00
Robert Kanter
27e2b4b364 Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 351cf87c92872d90f62c476f85ae4d02e485769c)
 2018-07-18 16:07:48 -07:00
Eric Yang
d82edec3c0 YARN-8538. Fixed memory leaks in container-executor and test cases. 
Contributed by Billie Rinaldi
 2018-07-18 13:44:49 -04:00