1459 Commits

Author SHA1 Message Date
Jian He
fa59b224b8 YARN-3892. Fixed NPE on RMStateStore#serviceStop when CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt
(cherry picked from commit c9dd2cada055c0beffd04bad0ded8324f66ad1b7)
2015-07-07 14:18:16 -07:00
Varun Vasudev
16ca0fcf99 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan.
(cherry-picked from commit 7e3ebc0fc59dbd1d7768b5a6c05af18b8c8fea4d)
2015-07-07 17:05:06 +05:30
Devaraj K
f558429efe YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong
prefix for auth options. Contributed by Bibin A Chundatt.

(cherry picked from commit af63427c6d7d2fc251eafb1f152b7a90c5bd07e5)
2015-07-07 12:08:03 +05:30
Devaraj K
e1ca886419 YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id
properly in log. Contributed by Bibin A Chundatt.

(cherry picked from commit 37d7395773b5bb24aa522db38a2602df9a5ac184)
2015-07-02 10:21:32 +05:30
Wangda Tan
d61dd10b50 YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda)
(cherry picked from commit 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)
2015-07-01 17:32:57 -07:00
Jason Lowe
a721b9d812 YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena
(cherry picked from commit b5cdf78e8e6cd6c5c1fb7286207dac72be32c0d6)
2015-07-01 21:14:35 +00:00
Devaraj K
c343250657 YARN-3830. AbstractYarnScheduler.createReleaseCache may try to clean a
null attempt. Contributed by nijel.

(cherry picked from commit 80a68d60560e505b5f8e01969dc3c168a1e5a7f3)
2015-07-01 19:05:27 +05:30
Colin Patrick Mccabe
b1da58d23d YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison via Colin P. McCabe)
(cherry picked from commit d0cc0380b57db5fdeb41775bb9ca42dac65928b8)
2015-06-30 16:24:44 -07:00
Jian He
0100995c5e YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti
(cherry picked from commit 62e583c7dcbb30d95d8b32a4978fbdb3b98d67cc)
2015-06-29 13:38:04 -07:00
Devaraj K
e163c1e0da YARN-3859. LeafQueue doesn't print user properly for application add.
Contributed by Varun Saxena.

(cherry picked from commit b543d1a390a67e5e92fea67d3a2635058c29e9da)
2015-06-28 10:06:08 +05:30
Xuan
1a8d162bc4 YARN-2871. TestRMRestart#testRMRestartGetApplicationList sometime fails
in trunk. Contributed by zhihai xu

(cherry picked from commit fe6c1bd73aee188ed58df4d33bbc2d2fe0779a97)
2015-06-26 19:45:00 -07:00
Jason Lowe
0221d19f4e YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena
(cherry picked from commit 40b256949ad6f6e0dbdd248f2d257b05899f4332)
2015-06-26 15:48:05 +00:00
Devaraj K
3e20f595d4 YARN-3826. Race condition in ResourceTrackerService leads to wrong
diagnostics messages. Contributed by Chengbing Liu.

(cherry picked from commit 57f1a01eda80f44d3ffcbcb93c4ee290e274946a)
2015-06-25 16:15:24 +05:30
Jonathan Eagles
d1f0ff165b YARN-3360. Add JMX metrics to TimelineDataManager (Jason Lowe via jeagles)
(cherry picked from commit 4c659ddbf7629aae92e66a5b54893e9c1c68dfb0)
2015-06-24 16:17:32 -05:00
rohithsharmaks
1392d1ad94 YARN-3790. usedResource from rootQueue metrics may get stale data for FS scheduler after recovering the container (Zhihai Xu via rohithsharmaks) 2015-06-24 23:00:25 +05:30
Jason Lowe
15b1800b12 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula
(cherry picked from commit 8d58512d6e6d9fe93784a9de2af0056bcc316d96)
2015-06-24 16:38:22 +00:00
Jason Lowe
14afa5d53d YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong
(cherry picked from commit 2a20dd9b61ba3833460cbda0e8c3e8b6366fc3ab)
2015-06-24 16:24:43 +00:00
Robert Kanter
5c1e809af5 YARN-3835. hadoop-yarn-server-resourcemanager test package bundles core-site.xml, yarn-site.xml (vamsee via rkanter)
(cherry picked from commit 99271b762129d78c86f3c9733a24c77962b0b3f7)
2015-06-22 18:03:00 -07:00
Karthik Kambatla
9656ee4ee7 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha)
(cherry picked from commit 5ebf2817e58e1be8214dc1916a694a912075aa0a)
2015-06-22 17:48:07 -07:00
Xuan
c3254a63a9 YARN-3834. Scrub debug logging of tokens during resource localization. Contributed by Chris Nauroth
(cherry picked from commit 6c7a9d502a633b5aca75c9798f19ce4a5729014e)
2015-06-21 17:14:59 -07:00
Xuan
86b75ac544 YARN-3802. Two RMNodes for the same NodeId are used in RM sometimes
after NM is reconnected. Contributed by zhihai xu

(cherry picked from commit 5b5bb8dcdc888ba1ebc7e4eba0fa0e7e79edda9a)
2015-06-18 14:38:52 -07:00
Xuan
16d2412a25 YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl. Contributed by Varun Saxena
(cherry picked from commit a826d432f9b45550cc5ab79ef63ca39b176dabb2)
2015-06-17 16:24:50 -07:00
Devaraj K
f0901e703b YARN-3148. Allow CORS related headers to passthrough in
WebAppProxyServlet. Contributed by Varun Saxena.

(cherry picked from commit ebb9a82519c622bb898e1eec5798c2298c726694)
2015-06-17 15:57:45 +05:30
Xuan
fe7f2d40fe YARN-3714. AM proxy filter can not get RM webapp address from
yarn.resourcemanager.hostname.rm-id. Contributed by Masatake Iwasaki

(cherry picked from commit e27d5a13b0623e3eb43ac773eccd082b9d6fa9d0)
2015-06-16 14:06:56 -07:00
Devaraj K
eb8e2c5448 YARN-3789. Improve logs for LeafQueue#activateApplications(). Contributed
by Bibin A Chundatt.

(cherry picked from commit b039e69bb03accef485361af301fa59f03d08d6a)
2015-06-16 14:04:46 +05:30
Devaraj K
3863342ca9 YARN-3794. TestRMEmbeddedElector fails because of ambiguous LOG reference.
Contributed by Chengbing Liu.

(cherry picked from commit d8dcfa98e3ca6a6fea414fd503589bb83b7a9c51)
2015-06-12 13:45:05 +05:30
Xuan
8a6c7d9973 YARN-3785. Support for Resource as an argument during submitApp call in
MockRM test class. Contributed by Sunil G

(cherry picked from commit 5583f88bf7f1852dc0907ce55d0755e4fb22107a)
2015-06-10 21:41:58 -07:00
Zhijie Shen
0dad10d845 YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong.
(cherry picked from commit 8d0ef3163287d811381e94cc0101dbc9847ee2db)
2015-06-08 23:21:35 -07:00
Xuan
869304dc83 YARN-3778. Fix Yarn resourcemanger CLI usage. Contributed by Brahma Reddy Battula
(cherry picked from commit 2b2465dfac1f147b6bb20d878b69a8cc3e85c8ad)
2015-06-08 15:44:25 -07:00
Jian He
a24ead8c6d YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla
(cherry picked from commit 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f)
2015-06-08 14:51:25 -07:00
Devaraj K
8c643e3bf9 YARN-3747. TestLocalDirsHandlerService should delete the created test
directory logDir2. Contributed by David Moore.

(cherry picked from commit 126321eded7dc38c1eef2cfde9365404c924a5cb)
2015-06-08 15:33:38 +05:30
Devaraj K
2a01c01af7 YARN-3780. Should use equals when compare Resource in
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.

(cherry picked from commit c7ee6c151c5771043a6de3b8a951cea13f59dd7b)
2015-06-08 11:56:58 +05:30
Karthik Kambatla
322e7d0d5e YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation and container reservation. (Zhihai Xu via kasha)
(cherry picked from commit bd69ea408f8fdd8293836ce1089fe9b01616f2f7)
2015-06-07 11:38:14 -07:00
Xuan
4d385b48f0 YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong
(cherry picked from commit 3e000a919fede85230fcfb06309a1f1d5e0c479c)
2015-06-05 12:49:51 -07:00
Karthik Kambatla
429754de09 YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha)
(cherry picked from commit 75885852cc19dd6de12e62498b112d5d70ce87f4)
2015-06-05 09:42:12 -07:00
Zhijie Shen
8f04ac9c34 YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong.
(cherry picked from commit 18dd01d6bf67f4d522b947454c1f4347d1cbbc19)
2015-06-04 14:48:05 -07:00
Jian He
2e99210e4e YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran
(cherry picked from commit 1970ca7cbcdb7efa160d0cedc2e3e22c1401fad6)
2015-06-04 11:14:36 -07:00
Jian He
6325e4b7dd YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan
(cherry picked from commit 6ad4e59cfc111a92747fdb1fb99cc6378044832a)
2015-06-04 10:53:24 -07:00
Wangda Tan
78d626fa89 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda)
(cherry picked from commit ebd797c48fe236b404cf3a125ac9d1f7714e291e)
2015-06-04 10:23:33 -07:00
Junping Du
d8c7ee1b53 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K.
(cherry picked from commit d7e7f6aa03c67b6a6ccf664adcb06d90bc963e58)
2015-06-04 05:00:32 -07:00
Xuan
3f0573b059 YARN-3749. We should make a copy of configuration when init
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428f65bb008b5c451f6f09e61e1000300)
2015-06-03 17:21:48 -07:00
Zhijie Shen
2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong."
This reverts commit 4e2ed34d2023a7cd3842e41aba1995f5d0f53cf2.
2015-06-03 14:16:51 -07:00
Karthik Kambatla
62d51b889a YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha)
(cherry picked from commit edb9cd0f7aa1ecaf34afaa120e3d79583e0ec689)
2015-06-03 13:50:02 -07:00
Jason Lowe
752caa95a4 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks
(cherry picked from commit e13b671aa510f553f4a6a232b4694b6a4cce88ae)
2015-06-03 19:45:18 +00:00
Zhijie Shen
aaa567514a YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G.
(cherry picked from commit dbc4f64937ea2b4c941a3ac49afc4eeba3f5b763)
2015-06-03 11:52:39 -07:00
Wangda Tan
a3734f67d3 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda)
(cherry picked from commit 5cc3fced957a8471733e0e9490878bd68429fe24)
2015-05-31 16:33:50 -07:00
Xuan
4563411e04 YARN-2900. Application (Attempt and Container) Not Found in AHS results
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 9686261ecb872ad159fac3ca44f1792143c6d7db)
2015-05-31 15:38:39 -07:00
Xuan
1ebaff0b86 Revert "YARN-2900. Application (Attempt and Container) Not Found in AHS results"
This reverts commit 4fee8b320276bac86278e1ae0a3397592a78aa18.
2015-05-31 00:24:08 -07:00
Xuan
4fee8b3202 YARN-2900. Application (Attempt and Container) Not Found in AHS results
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 06f8e9cabaf3c05cd7d16215cff47265ea773f39)
2015-05-30 21:58:04 -07:00
Zhijie Shen
4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong.
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
2015-05-30 21:06:32 -07:00