Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,377 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

826 Commits

Author SHA1 Message Date
Jason Lowe 13f74a3601 YARN-8054. Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread. Contributed by Jonathan Eagles 
(cherry picked from commit 5aa7052e31)
 2018-03-21 17:40:05 -05:00
Yufei Gu 204674f414 YARN-8039. Clean up log dir configuration in TestLinuxContainerExecutorWithMocks.testStartLocalizer. Contributed by Miklos Szegedi. 
(cherry picked from commit 49c747ab18)
 2018-03-16 16:34:14 -07:00
Jason Lowe 0a3a878428 YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang 
(cherry picked from commit 83798f15f8)
 2018-03-02 10:08:32 -06:00
Billie Rinaldi aa64a50746 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit a72dcb9cad.
 2018-02-07 08:58:28 -08:00
Billie Rinaldi 93128c1904 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362)
 2018-02-06 12:04:52 -08:00
Jian He 0497e09339 YARN-7765. Fixed an issue that kerberos tgt not found when NM posting timeline events. Contributed by Rohith Sharma K S 2018-01-28 13:24:49 -08:00
Zhe Zhang e9a338ff64 YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 
(cherry picked from commit fa8cf4d1b4)
(cherry picked from commit 83a4f92a3f)
 2018-01-24 13:10:35 -08:00
Haibo Chen 895a0391fe YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759)
(cherry picked from commit 8793e45f53)
 2018-01-19 10:35:40 -08:00
Miklos Szegedi 86a2ac94b1 YARN-7758. Add an additional check to the validity of container and application ids passed to container-executor. Contributed by Yufei Gu. 2018-01-17 12:56:10 -08:00
Miklos Szegedi 09428b137f YARN-7705. Create the container log directory with correct sticky bit in C code. Contributed by Yufei Gu. 
(cherry picked from commit 2dcfc18)
 2018-01-12 17:44:31 -08:00
Miklos Szegedi 35527db157 YARN-7590. Improve container-executor validation check. Contributed by Eric Yang. 
(cherry picked from commit bc285da)
 2018-01-11 10:30:15 -08:00
Jason Lowe 42964c2a66 YARN-7678. Ability to enable logging of container memory stats. Contributed by Jim Brennan 2018-01-04 10:21:40 -06:00
Jason Lowe 31a06baca0 YARN-7661. NodeManager metrics return wrong value after update node resource. Contributed by Yang Wang 
(cherry picked from commit 811fabdebe)
 2017-12-18 15:27:27 -06:00
Jason Lowe 0f8de8cd91 YARN-7595. Container launching code suppresses close exceptions after writes. Contributed by Jim Brennan 2017-12-15 15:54:56 -06:00
Weiwei Yang d69b7358b6 YARN-7647. NM print inappropriate error log when node-labels is enabled. Contributed by Yang Wang. 
(cherry picked from commit 7efc4f7688)
 2017-12-13 13:28:48 +08:00
Subru Krishnan 850bd0ed7c YARN-6704. Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService. (Botong Huang via Subru). 
(cherry picked from commit 670e8d4ec7)
 2017-12-08 15:44:45 -08:00
Weiwei Yang 029714e3ec YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang. 2017-12-08 13:53:33 +08:00
Jason Lowe 3fb83cbeb2 YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719)
 2017-12-01 15:51:49 -06:00
Yufei Gu de144175a4 YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu) 
(cherry picked from commit d8923cdbf1)
(cherry picked from commit c0ab3c8407)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
 2017-11-27 14:54:27 -08:00
Subru Krishnan ed31091361 YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru). 2017-11-20 14:21:58 -08:00
Junping Du a72dcb9cad YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 2017-11-13 18:22:30 -08:00
Arun Suresh c153bed6e3 YARN-5049. Addendum patch for branch-2. (asuresh) 2017-11-12 10:16:00 -08:00
Eric Yang 90ac8cd5a1 YARN-7412. Fix unit test for docker mount check on ubuntu. (Contributed by Eric Badger) 
(cherry picked from commit 7a49ddfdde)
 2017-11-12 09:59:39 -08:00
Subru Krishnan 0282efab14 YARN-7476. Fix miscellaneous issues in ATSv2 after merge to branch-2. (Varun Saxena via Subru). 2017-11-12 09:19:30 -08:00
bibinchundatt fd4868bdb4 Add containerId to Localizer failed logs. Contributed by Prabhu Joseph 
(cherry picked from commit dcd99c4b9a)
 2017-11-06 23:09:41 +05:30
Jason Lowe 6ad6882343 YARN-7286. Add support for docker to have no capabilities. Contributed by Eric Badger 
(cherry picked from commit b7dee1f060)
 2017-11-02 09:50:55 -05:00
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Arun Suresh 2654d9973e YARN-7299. Fix TestDistributedScheduler. (asuresh) 
(cherry picked from commit 9c5c68745e)
 2017-10-27 23:11:34 -07:00
Jason Lowe 9093ad6168 YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 
(cherry picked from commit 665bb147aa)
 2017-10-27 17:09:29 -05:00
Eric Yang f6d3c101a6 YARN-7353. Improved volume mount check for directories and unit test compatibility on RHEL7. Contributed by Eric Badger. 2017-10-24 17:01:08 -04:00
Yufei Gu c3946f29e7 YARN-7261. Add debug message for better download latency monitoring. (Yufei Gu) 
(cherry picked from commit 0799fde35e)
 2017-10-20 10:02:35 -07:00
Wangda Tan 2e3b7130c5 YARN-6623. Add support to turn off launching privileged containers in the container-executor. (Varun Vasudev via wangda) 2017-10-19 15:11:05 -07:00
Varun Saxena 56e0873e22 Fix javac, javadoc and checkstyle issues before branch-2 merge. 2017-10-17 23:36:41 +05:30
Haibo Chen 0f51f15097 YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen) 2017-10-17 23:36:40 +05:30
Jian He cbd5e89b24 MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 
(cherry picked from commit c6ce422a3423305ac665f80f039f61294adaa32c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2017-10-17 23:36:40 +05:30
Jian He 53ddf1b15f YARN-7038. Addendum patch. Fix compilation error. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 4e1af68ae1 YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena. 
(cherry picked from commit 387dd3da87759860dd18bd8aa95c60d962b1e6a9)
 2017-10-17 23:36:39 +05:30
Jian He 55e5742394 YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 5a7c272e0a YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-10-17 23:36:39 +05:30
Haibo Chen d53038626e YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 47474fffac)
(cherry picked from commit 8817cb5c8424359b880c6d700e53092f0269c1bb)
 2017-10-17 23:36:38 +05:30
Varun Saxena 16d8df68f6 YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena) 
(cherry picked from commit a4b5aa8493)
 2017-10-17 23:36:37 +05:30
Sangjin Lee bb100d35fd YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 
(cherry picked from commit 73235ab30361b41293846189f3c5fef321ae7cac)
 2017-10-17 23:36:36 +05:30
Sangjin Lee b8cfb4fcb3 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 
(cherry picked from commit f734977b27a514ce0561638c0a6a17b1ef093026)
 2017-10-17 23:36:35 +05:30
Varun Saxena 684c2d149f YARN-3359. Addendum for Recover collector list when RM fails over (Li Lu via Varun Saxena) 
(cherry picked from commit 25b19178dd53bc0d57ca5fd64d3464af7b59e588)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2017-10-17 23:36:35 +05:30
Varun Saxena 6562a354a9 YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena) 
(cherry picked from commit 1c8a57550b)
 2017-10-17 23:36:34 +05:30
Sangjin Lee 853a5da041 YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S. 
(cherry picked from commit 787d5942003a5441ba455a79fa08da28f60d5777)
 2017-10-17 23:36:34 +05:30
Sangjin Lee 4675f22e6a YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu. 
(cherry picked from commit 0c1863144649ea265da65ce25158707cc3a3fb4a)
 2017-10-17 23:36:34 +05:30
Sangjin Lee 51102019f5 YARN-5156. YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state (Vrushali C via sjlee) 
(cherry picked from commit d0a62d8adf86f7cb2481940d212788cc5f327367)
 2017-10-17 23:36:34 +05:30
Sangjin Lee e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Nathan Roberts b876a9314e YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe. 
(cherry picked from commit 8620140a6a3ec0117675ede06d92d830da3da551)
 2017-10-16 16:44:34 -05:00