Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,608 Commits 326 Branches 371 Tags
Commit Graph

1700 Commits

Author SHA1 Message Date
Arun Suresh
707b96fa58 YARN-3736. Add RMStateStore apis to store and load accepted reservations for failover (adhoot via asuresh) 
(cherry picked from commit f271d377357ad680924d19f07e6c8315e7c89bae)
 2015-08-05 12:58:37 -07:00
Xuan
89ccfd9fcc YARN-4004. container-executor should print output of docker logs if the 
docker container exits with non-0 exit status. Contributed by Varun
Vasudev

(cherry picked from commit c3364ca8e75acfb911ab92e19f357b132f128123)
 2015-08-03 18:11:16 -07:00
Xuan
8126fde884 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S

(cherry picked from commit 0306d902f53582320aa5895ca9f5c31f64aaaff6)
 2015-08-03 15:47:21 -07:00
Jason Lowe
b264217bc3 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 
(cherry picked from commit 469cfcd695da979e56c83d9303f9bc1f898c08ce)
 2015-08-03 15:55:04 +00:00
Jonathan Eagles
899df5bce0 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b95224e9d43fd63a4ef9ac5c44f113f710d)
 2015-08-03 10:40:04 -05:00
Jason Lowe
c31e3ba921 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c035487e99df30ce80366446fe09bd6c)
 2015-07-31 17:38:49 +00:00
Zhihai Xu
0e2019fa30 YARN-433. When RM is catching up with node updates then it should not expire acquired containers. Contributed by Xuan Gong 
(cherry picked from commit ab80e277039a586f6d6259b2511ac413e29ea4f8)
 2015-07-30 22:03:04 -07:00
Wangda Tan
53a0653826 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) 
(cherry picked from commit 91b42e7d6e8169feb54dbe3715a3269eea6153a9)
 2015-07-30 10:01:01 -07:00
Karthik Kambatla
8659315062 YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha) 
(cherry picked from commit 5205a330b387d2e133ee790b9fe7d5af3cd8bccc)
 2015-07-29 09:42:57 -07:00
Xuan
2ec008eb38 YARN-3982. container-executor parsing of container-executor.cfg broken 
in trunk and branch-2. Contributed by Varun Vasudev

(cherry picked from commit f17093421521efcbdc813f6f2b8411e45ecc7863)
 2015-07-27 23:47:31 -07:00
Jian He
7c123accda YARN-3846. RM Web UI queue filter is not working for sub queue. Contributed by Mohammad Shahid Khan 
(cherry picked from commit 3572ebd738aa5fa8b0906d75fb12cc6cbb991573)
 2015-07-27 17:12:24 -07:00
Varun Vasudev
9da487e0fd YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit 3e6fce91a471b4a5099de109582e7c6417e8a822)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-07-27 12:03:36 -07:00
Varun Vasudev
ec0f801f52 YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab. 
(cherry picked from commit f36835ff9b878fa20fe58a30f9d1e8c47702d6d2)
 2015-07-27 10:18:06 -07:00
ccurino
26ea045814 YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations. (Jonathan Yaniv and Ishai Menache via curino) 
(cherry picked from commit 156f24ead00436faad5d4aeef327a546392cd265)
 2015-07-25 07:47:11 -07:00
Wangda Tan
4ba2e1ae95 YARN-3973. Recent changes to application priority management break reservation system from YARN-1051 (Carlo Curino via wangda) 
(cherry picked from commit a3bd7b4a59b3664273dc424f240356838213d4e7)
 2015-07-24 16:45:06 -07:00
Jason Lowe
2cf714123a YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a739bb13115167dc661b6a16b2ed2c04)
 2015-07-24 22:15:44 +00:00
Jian He
3233284e87 YARN-3026. Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp. Contributed by Wangda Tan 
(cherry picked from commit 83fe34ac0896cee0918bbfad7bd51231e4aec39b)
 2015-07-24 14:01:36 -07:00
Karthik Kambatla
d2ee02940a YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500. (Anubhav Dhoot via kasha) 
(cherry picked from commit d19d18775368f5aaa254881165acc1299837072b)
 2015-07-24 11:49:46 -07:00
Xuan
98ed4bca3b YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai

(cherry picked from commit fbd6063269221ec25834684477f434e19f0b66af)
 2015-07-24 10:17:05 -07:00
carlo curino
6a71a0a230 YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 
(cherry picked from commit 0fcb4a8cf2add3f112907ff4e833e2f04947b53e)
 2015-07-23 19:36:54 -07:00
Rohith Sharma K S
9d15860e6d YARN-3845. Scheduler page does not render RGBA color combinations in IE11. (Contributed by Mohammad Shahid Khan) 2015-07-24 12:54:07 +05:30
Robert Kanter
ba3b40ae79 YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos that include it (adhoot via rkanter) 
(cherry picked from commit 1d3026e7b3cf2f3a8a544b66ff14783cc590bdac)
 2015-07-23 14:47:28 -07:00
Wangda Tan
6baebcaedc YARN-3941. Proportional Preemption policy should try to avoid sending duplicate PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda) 
(cherry picked from commit 3bba1800513b38a4827f7552f348db87dc47c783)
 2015-07-23 10:08:30 -07:00
Junping Du
6772c3f4dd YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 
(cherry picked from commit ee98d6354bbbcd0832d3e539ee097f837e5d0e31)
 2015-07-22 17:57:16 -07:00
Wangda Tan
3800e25d6e YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda) 
(cherry picked from commit 06e5dd2c84c49460884757b56980b1b9c58af996)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-07-22 12:03:14 -07:00
Wangda Tan
3576b0ac5e YARN-3932. SchedulerApplicationAttempt#getResourceUsageReport and UserInfo should based on total-used-resources. (Bibin A Chundatt via wangda) 
(cherry picked from commit 76ec26de8099dc48ce3812c595b7ab857a600442)
 2015-07-22 11:54:38 -07:00
Wangda Tan
3dd113fa7c YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda) 
(cherry picked from commit c39ca541f498712133890961598bbff50d89d68b)
 2015-07-21 09:57:53 -07:00
Colin Patrick Mccabe
aff955a14a YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit 419c51d233bd124eadb38ff013693576ec02c4f1)
 2015-07-17 12:05:24 -07:00
Jonathan Eagles
b289939514 YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 
(cherry picked from commit 7faae0e6fe027a3886d9f4e290b6a488a2c55b3a)
 2015-07-17 11:03:34 -05:00
Arun Suresh
8018041b49 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 
(cherry picked from commit 9b272ccae78918e7d756d84920a9322187d61eed)
 2015-07-17 04:33:04 -07:00
Wangda Tan
a7de3cde13 YARN-3885. ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level. (Ajith S via wangda) 
(cherry picked from commit 3540d5fe4b1da942ea80c9e7ca1126b1abb8a68a)
 2015-07-16 16:14:07 -07:00
Arun Suresh
5f58be7dd4 YARN-3453. Ensure preemption logic in FairScheduler uses DominantResourceCalculator in DRF queues to prevent unnecessary thrashing. (asuresh) 
(cherry picked from commit ac94ba3e185115b83351e35c610c2b8ff91b1ebc)
 2015-07-14 00:25:57 -07:00
Akira Ajisaka
0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d90e26616accee73b1f7743aab5df692)
 2015-07-13 17:53:27 +09:00
Wangda Tan
87360be089 YARN-3894. RM startup should fail for wrong CS xml NodeLabel capacity configuration. (Bibin A Chundatt via wangda) 
(cherry picked from commit 5ed1fead6b5ec24bb0ce1a3db033c2ee1ede4bb4)
 2015-07-12 21:53:08 -07:00
Wangda Tan
c36090fd3f YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 
(cherry picked from commit 1df39c1efc9ed26d3f1a5887c31c38c873e0b784)
 2015-07-11 10:27:58 -07:00
Zhijie Shen
37a93c2d78 YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 1ea36299a47af302379ae0750b571ec021eb54ad)
 2015-07-10 19:02:13 -07:00
Ming Ma
b169889f01 YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) 
(cherry picked from commit 08244264c0583472b9c4e16591cfde72c6db62a2)
 2015-07-10 08:34:01 -07:00
Xuan
2a01495544 YARN-3888. ApplicationMaster link is broken in RM WebUI when appstate is 
NEW. Contributed by Bibin A Chundatt

(cherry picked from commit 52148767924baf423172d26f2c6d8a4cfc6e143f)
 2015-07-09 21:38:46 -07:00
carlo curino
753d87066f YARN-3800. Reduce storage footprint for ReservationAllocation. Contributed by Anubhav Dhoot. 
(cherry picked from commit 0e602fa3a1529134214452fba10a90307d9c2072)
 2015-07-09 16:52:47 -07:00
Karthik Kambatla
d15f69457b YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 527c40e4d664c721b8f32d7cd8df21b2666fea8a)
 2015-07-09 09:35:39 -07:00
Varun Vasudev
d17a7bbf6e YARN-2194. Addendum patch to fix failing unit test in TestPrivilegedOperationExecutor. Contributed by Sidharta Seethana. 
(cherry picked from ad5b904175a1f89951c7c32f744a4e20fafb51b5)
 2015-07-09 11:52:57 +05:30
Jian He
fa59b224b8 YARN-3892. Fixed NPE on RMStateStore#serviceStop when CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt 
(cherry picked from commit c9dd2cada055c0beffd04bad0ded8324f66ad1b7)
 2015-07-07 14:18:16 -07:00
Varun Vasudev
16ca0fcf99 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan. 
(cherry-picked from commit 7e3ebc0fc59dbd1d7768b5a6c05af18b8c8fea4d)
 2015-07-07 17:05:06 +05:30
Devaraj K
f558429efe YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong 
prefix for auth options. Contributed by Bibin A Chundatt.

(cherry picked from commit af63427c6d7d2fc251eafb1f152b7a90c5bd07e5)
 2015-07-07 12:08:03 +05:30
Devaraj K
e1ca886419 YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id 
properly in log. Contributed by Bibin A Chundatt.

(cherry picked from commit 37d7395773b5bb24aa522db38a2602df9a5ac184)
 2015-07-02 10:21:32 +05:30
Wangda Tan
d61dd10b50 YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) 
(cherry picked from commit 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)
 2015-07-01 17:32:57 -07:00
Jason Lowe
a721b9d812 YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 
(cherry picked from commit b5cdf78e8e6cd6c5c1fb7286207dac72be32c0d6)
 2015-07-01 21:14:35 +00:00
Devaraj K
c343250657 YARN-3830. AbstractYarnScheduler.createReleaseCache may try to clean a 
null attempt. Contributed by nijel.

(cherry picked from commit 80a68d60560e505b5f8e01969dc3c168a1e5a7f3)
 2015-07-01 19:05:27 +05:30
Colin Patrick Mccabe
b1da58d23d YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit d0cc0380b57db5fdeb41775bb9ca42dac65928b8)
 2015-06-30 16:24:44 -07:00
Jian He
0100995c5e YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 
(cherry picked from commit 62e583c7dcbb30d95d8b32a4978fbdb3b98d67cc)
 2015-06-29 13:38:04 -07:00