Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,593 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2074 Commits

Author SHA1 Message Date
Xuan acb342b77c YARN-1912. ResourceLocalizer started without any jvm memory control. 
Contributed by Masatake Iwasaki

(cherry picked from commit 6471d18bc7)
 2015-05-08 20:02:23 -07:00
Xuan fc980247a6 YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu 
(cherry picked from commit 333f9a896d)
 2015-05-08 17:54:46 -07:00
Zhijie Shen ef096599c3 YARN-2206. Updated document for applications REST API response examples. Contributed by Kenji Kikushima and Brahma Reddy Battula. 
(cherry picked from commit 08f0ae403c)
 2015-05-08 16:41:29 -07:00
Karthik Kambatla a7d8f21630 YARN-3271. FairScheduler: Move tests related to max-runnable-apps from TestFairScheduler to TestAppRunnability. (nijel via kasha) 
(cherry picked from commit 2fb44c8aaf)
 2015-05-08 16:40:36 -07:00
Karthik Kambatla 21a07769d4 YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha) 
(cherry picked from commit 96473bdc2b)
 2015-05-08 16:31:30 -07:00
Robert Kanter 453488cff2 YARN-3473. Fix RM Web UI configuration for some properties (rchiang via rkanter) 
(cherry picked from commit 5658998845)
 2015-05-08 16:23:20 -07:00
Jason Lowe 743ca1a056 YARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith 
(cherry picked from commit 25e2b02122)
 2015-05-08 22:46:34 +00:00
Jian He 5a1c2fed34 YARN-3604. Fixed ZKRMStateStore#removeApplication to also disable watch. Contributed Zhihai Xu 
(cherry picked from commit d0b73287c2)
 2015-05-08 15:13:43 -07:00
Xuan 8d07116266 YARN-2331. Distinguish shutdown during supervision vs. shutdown for 
rolling upgrade. Contributed by Jason Lowe

(cherry picked from commit 088156de43)
 2015-05-08 15:11:55 -07:00
Vinod Kumar Vavilapalli 24c8d3e54b YARN-3018. Unified the default value for the configuration property yarn.scheduler.capacity.node-locality-delay in code and default xml file. Contributed by Nijel SF. 
(cherry picked from commit f7a74d2491)
 2015-05-08 14:45:55 -07:00
Wangda Tan f8807839c6 YARN-3593. Add label-type and Improve "DEFAULT_PARTITION" in Node Labels Page. (Naganarasimha G R via wangda) 
(cherry picked from commit f55d21d894)
 2015-05-08 12:05:25 -07:00
Thomas Graves 638feaaa35 YARN-20. More information for yarn.resourcemanager.webapp.address in yarn-default.xml (Bartosz Ługowski vai tgraves) 
(cherry picked from commit f0f5e3c075)
 2015-05-08 17:22:33 +00:00
Thomas Graves 547b069887 YARN-3600. AM container link is broken (Naganarasimha G R via tgraves 
(cherry picked from commit 5d708a4725)
 2015-05-08 16:37:20 +00:00
Robert (Bobby) Evans 28e0593b96 YARN-644: Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer 
(cherry picked from commit bcf2890502)
 2015-05-08 11:11:53 -05:00
Devaraj K bf8be56292 YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:26:11 +05:30
Jason Lowe bcb863b156 YARN-3554. Default value for maximum nodemanager connect wait time is too high. Contributed by Naganarasimha G R 
(cherry picked from commit 9757864fd6)
 2015-05-08 13:47:48 +00:00
Tsuyoshi Ozawa 5ddc3adb0d YARN-3169. Drop YARN's overview document. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b419c1b2ec)
 2015-05-08 19:58:47 +09:00
Tsuyoshi Ozawa b1f3590b22 YARN-3589. RM and AH web UI display DOCTYPE wrongly. Contbituted by Rohith. 
(cherry picked from commit f26700f287)
 2015-05-08 17:13:56 +09:00
Tsuyoshi Ozawa 1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b)
 2015-05-08 16:00:41 +09:00
Devaraj K 0d286d1041 YARN-3358. Audit log not present while refreshing Service ACLs. 
Contributed by Varun Saxena.

(cherry picked from commit ef3d66d462)
 2015-05-08 12:15:45 +05:30
Devaraj K dce2381dc4 YARN-3592. Fix typos in RMNodeLabelsManager. Contributed by Sunil G. 
(cherry picked from commit a28cd02ba0)
 2015-05-08 10:55:53 +05:30
Akira Ajisaka c7f3dd1eb3 YARN-3572. Correct typos in WritingYarnApplications.md. Contributed by Gabor Liptak. 
(cherry picked from commit a521b50955)
 2015-05-08 13:25:07 +09:00
Akira Ajisaka 170f1b0afd YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo. 
(cherry picked from commit b167fe7605)
 2015-05-08 11:15:28 +09:00
Jian He d817fbb34d YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan 
(cherry picked from commit f489a4ec96)
 2015-05-07 17:36:24 -07:00
Jian He c35114a1e6 YARN-3584. Fixed attempt diagnostics format shown on the UI. Contributed by nijel 
(cherry picked from commit b88700dcd0)
 2015-05-07 13:56:25 -07:00
Zhijie Shen bb035ff087 YARN-3448. Added a rolling time-to-live LevelDB timeline store implementation. Contributed by Jonathan Eagles. 
(cherry picked from commit daf3e4ef8b)
 2015-05-07 10:10:13 -07:00
Junping Du 684a5a6aeb YARN-3523. Cleanup ResourceManagerAdministrationProtocol interface audience. Contributed by Naganarasimha G R 
(cherry picked from commit 8e991f4b1d)
 2015-05-07 05:40:13 -07:00
Akira Ajisaka 67bebabb21 YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 995f1b5831fc33042de90267a689333dc110c4c3)
 2015-05-07 13:33:17 +09:00
Vinod Kumar Vavilapalli d127dd64a7 YARN-3385. Fixed a race-condition in ResourceManager's ZooKeeper based state-store to avoid crashing on duplicate deletes. Contributed by Zhihai Xu. 
(cherry picked from commit 4c7b9b6abe)
 2015-05-06 17:52:13 -07:00
Junping Du 79babc4bd2 YARN-3580. [JDK8] TestClientRMService.testGetLabelsToNodes fails. Contributed by Robert Kanter. 
(cherry picked from commit 31b627b2a8)
 2015-05-06 16:51:56 -07:00
Robert Kanter 28b40aca98 YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter) 
(cherry picked from commit b72507810a)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-05-06 14:20:08 -07:00
Jian He f8488b5aeb YARN-3301. Fixed the format issue of the new RM attempt web page. Contributed by Xuan Gong 
(cherry picked from commit 0d3188fd25)
 2015-05-06 13:51:44 -07:00
Vinod Kumar Vavilapalli da410ea02a YARN-3243. Moving CHANGES.txt entry to the right release. 
(cherry picked from commit 185e63a726)
 2015-05-06 10:44:19 -07:00
Xuan a0445eae5c YARN-3582. NPE in WebAppProxyServlet. Contributed by Jian He 
(cherry picked from commit a583a40693)
 2015-05-05 21:43:35 -07:00
Jian He eb0c6d2ee7 YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate timeout. Contributed by Rohith Sharmaks 
(cherry picked from commit e4c3b52c89)
 2015-05-05 11:34:11 -07:00
Xuan 02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04)
 2015-05-05 10:45:18 -07:00
Junping Du 6ea007ba91 YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 3810242062)
 2015-05-05 10:19:16 -07:00
Jason Lowe 0f30913ea8 YARN-3552. RM Web UI shows -1 running containers for completed apps. Contributed by Rohith 
(cherry picked from commit 9356cf8676)
 2015-05-05 13:43:57 +00:00
Jian He f71c49c5e3 YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore. Contributed by Tsuyoshi Ozawa 
(cherry picked from commit d701acc9c6)
 2015-05-04 16:17:01 -07:00
Wangda Tan 4e77ee2f6f YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda) 
(cherry picked from commit 71f4de220c)
 2015-05-04 16:02:57 -07:00
Allen Wittenauer 02e650248d YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw) 
(cherry picked from commit d4ac6822e1)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-05-04 16:00:20 -07:00
Jason Lowe 99b49a8181 YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne 
(cherry picked from commit 8f65c793f2)
 2015-05-04 10:32:33 -05:00
Tsuyoshi Ozawa f382602de1 YARN-1993. Cross-site scripting vulnerability in TextView.java. Contributed byKenji Kikushima. 
(cherry picked from commit e8d0ee5fc9)
 2015-05-03 10:51:39 +09:00
Junping Du 8068e79989 YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources. Contributed by Xu Yang. 
(cherry picked from commit 57d9a972cb)
 2015-05-02 10:20:30 -07:00
Gera Shegalov 9325dd863c YARN-2893. AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream. (Zhihai Xu via gera) 
(cherry picked from commit f8204e241d)
 2015-05-01 19:04:48 -07:00
Wangda Tan 4151e5729a YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) 
(cherry picked from commit 7d46a806e7)
 2015-05-01 16:44:49 -07:00
Robert Kanter 32b3b8e2ef YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) 
(cherry picked from commit ac7d152901)
 2015-05-01 16:39:59 -07:00
Vinod Kumar Vavilapalli 90f6e8c90f YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. 
(cherry picked from commit 1b3b9e5c31)
 2015-04-30 21:43:22 -07:00
Zhijie Shen c9ee316045 YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 
(cherry picked from commit 21bf2cdcb77f69abc906e6cd401a8fb221f250e9)
 2015-04-30 13:06:46 -07:00
Wangda Tan 100b41add6 YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly. (Jian He via wangda) 
(cherry picked from commit e2e8f77118)
 2015-04-30 11:06:47 -07:00
Jian He a7a5737bc4 YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled. Contributed by Anubhav Dhoot 
(cherry picked from commit 4c1af156ae)
 2015-04-29 14:51:04 -07:00
tgraves 2e13183f60 YARN-3517. RM web ui for dumping scheduler logs should be for admins only (Varun Vasudev via tgraves) 
(cherry picked from commit 2e215484bd)
 2015-04-29 21:27:16 +00:00
Karthik Kambatla 4cc38df7de YARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha) 
(cherry picked from commit 8f82970e0c)
 2015-04-28 21:03:12 -07:00
Wangda Tan 8ee632caa7 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 
(cherry picked from commit db1b674b50)
 2015-04-27 16:42:36 -07:00
Karthik Kambatla d2d8bc243b Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) 
(cherry picked from commit 32cd2c8d42)
 2015-04-27 13:44:06 -07:00
Xuan 0e8af401d7 YARN-3530. ATS throws exception on trying to filter results without 
otherinfo. Contributed by zhijie shen

(cherry picked from commit 7f07c4d810)
 2015-04-27 10:37:40 -07:00
Karthik Kambatla 4045c41afe YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) 
(cherry picked from commit 47279c3228)
 2015-04-26 09:14:12 -07:00
Vinod Kumar Vavilapalli db1a33e956 Moving YARN-3351, YARN-3382, YARN-3472, MAPREDUCE-6238 to the 2.7.1 CHANGES.txt 
sections given the recent merge into branch-2.7.

(cherry picked from commit 2f82ae042a)
 2015-04-24 17:20:02 -07:00
Jian He 9bf09b334d YARN-2498. Respect labels in preemption policy of capacity scheduler for inter-queue preemption. Contributed by Wangda Tan 
(cherry picked from commit d497f6ea2b)
 2015-04-24 17:03:57 -07:00
Tsuyoshi Ozawa 3884948d6c YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 4a3dabd94f)
 2015-04-25 07:17:42 +09:00
Jason Lowe bd750f160b Fix commit version for YARN-3537 
(cherry picked from commit 78fe6e57c7)
 2015-04-24 22:09:01 +00:00
Jason Lowe cf4154676b YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula 
(cherry picked from commit 5e093f0d40)
 2015-04-24 22:04:03 +00:00
Akira Ajisaka 73ba3ebe7c YARN-3444. Fix typo capabililty. Contributed by Gabor Liptak. 
(cherry picked from commit 5ce3a77f3c)
 2015-04-25 06:09:03 +09:00
Jian He 0583c27fb1 YARN-3387. Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time. Contributed by Sandflee 
(cherry picked from commit d03dcb9635)
 2015-04-24 12:14:17 -07:00
Xuan c184466939 YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev 
(cherry picked from commit eee9facbba)
 2015-04-24 09:43:35 -07:00
Xuan 8f6053ae51 YARN-3516. killing ContainerLocalizer action doesn't take effect when 
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu

(cherry picked from commit 0b3f8957a8)
 2015-04-23 16:41:52 -07:00
Vinod Kumar Vavilapalli 0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297)
 2015-04-23 11:25:30 -07:00
Jian He 48d8d88245 YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen 
(cherry picked from commit aa4a192feb)
 2015-04-23 11:07:54 -07:00
Wangda Tan 9692bcc2c8 YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 395205444e)
 2015-04-23 10:48:00 -07:00
tgraves 1cd2fcf25d YARN-3434. Interaction between reservations and userlimit can result in significant ULF violation 
(cherry picked from commit 189a63a719)
 2015-04-23 14:49:24 +00:00
Steve Loughran 63ccb9167e YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:23 +01:00
Vinod Kumar Vavilapalli 04783b0402 YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana. 
(cherry picked from commit a100be685c)
 2015-04-22 17:36:55 -07:00
Junping Du 8de6045842 New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 
(cherry picked from commit fad9d7e85b)
 2015-04-22 10:08:16 -07:00
Akira Ajisaka f469f9af2d YARN-3410. Addendum fix for compilation error. Contributed by Rohith. 
(cherry picked from commit b08908ae5e)
 2015-04-22 17:48:55 +09:00
Jian He 613a783380 YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev 
(cherry picked from commit 674c7ef649)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-04-21 21:06:06 -07:00
Jian He dc4698bb33 YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. Contributed by Rohith Sharmaks 
(cherry picked from commit bdd90110e6)
 2015-04-21 20:34:30 -07:00
Wangda Tan 8a9599fb9e YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) 2015-04-21 17:47:27 -07:00
Tsuyoshi Ozawa 0b87ae27aa YARN-3495. Confusing log generated by FairScheduler. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 105afd5477)
 2015-04-22 05:48:16 +09:00
Vinod Kumar Vavilapalli 27fc4fb998 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:01 -07:00
Wangda Tan d0ea982e64 YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler. (Craig Welch via wangda) 
(cherry picked from commit 44872b76fc)
 2015-04-20 17:13:01 -07:00
Jian He 6b454aacc3 YARN-3497. ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy. Contributed by Jason Lowe 
(cherry picked from commit f967fd2f21)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-04-20 10:40:09 -07:00
Jian He 6f70f1b1af YARN-3136. Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers. Contributed by Sunil G 
(cherry picked from commit 497c86b485)
 2015-04-18 12:47:30 -07:00
Wangda Tan 3316cd4357 YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) 
(cherry picked from commit f47a5763ac)
 2015-04-17 17:23:00 -07:00
Wangda Tan e7cbecddc3 YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed. (Jian He via wangda) 
(cherry picked from commit f65eeb412d)
 2015-04-17 17:15:16 -07:00
Jian He 9827da95bc YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 
(cherry picked from commit 6779467ab6)
 2015-04-17 15:57:36 -07:00
Jian He 57eb07d34b YARN-2696. Queue sorting in CapacityScheduler should consider node label. Contributed by Wangda Tan 
(cherry picked from commit d573f09fb9)
 2015-04-17 13:37:34 -07:00
Junping Du b244701fad YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 
(cherry picked from commit 1db355a875)
 2015-04-17 13:20:27 -07:00
Jian He 42cf8e0554 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 
(cherry picked from commit bb6dde68f1)
 2015-04-16 19:44:32 -07:00
Akira Ajisaka 51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794)
 2015-04-16 10:06:03 -05:00
Jian He 6fed2c2a79 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 
(cherry picked from commit 1b89a3e173)
 2015-04-15 14:03:29 -07:00
Jian He 58f99b740a YARN-3404. Display queue name on application page. Contributed by Ryu Kobayashi 
(cherry picked from commit b2e6cf607f)
 2015-04-15 13:53:24 -07:00
Tsuyoshi Ozawa e977247f3e YARN-3326. Support RESTful API for getLabelsToNodes. Contributed by Naganarasimha G R. 
(cherry picked from commit e48cedc663)
 2015-04-15 14:04:24 -05:00
Wangda Tan 89a7c98436 YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy. (Craig Welch via wangda) 
(cherry picked from commit 5004e75332)
 2015-04-15 09:57:52 -07:00
Harsh J 01af29106a YARN-3462. Patches applied for YARN-2424 are inconsistent between trunk and branch-2. Contributed by Naganarasimha G R. 2015-04-15 09:41:43 +05:30
Tsuyoshi Ozawa 3daca4f040 YARN-3436. Fix URIs in documantion of YARN web service REST APIs. Contributed by Bibin A Chundatt. 
(cherry picked from commit 05007b45e5)
 2015-04-14 14:14:17 -05:00
Jian He 9ebbf1bfce YARN-3361. CapacityScheduler side changes to support non-exclusive node labels. Contributed by Wangda Tan 
(cherry picked from commit 0fefda645b)
 2015-04-14 11:46:35 -07:00
Jian He 81bbee6852 YARN-3266. RMContext#inactiveNodes should have NodeId as map key. Contributed by Chengbing Liu 
(cherry picked from commit b46ee1e7a3)
 2015-04-14 11:31:42 -07:00
Junping Du 7c072bf092 YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. 
(cherry picked from commit 838b06ac87)
 2015-04-13 18:37:39 -07:00
Junping Du a64b8915de YARN-3347. Improve YARN log command to get AMContainer logs as well as running containers logs. Contributed by Xuan Gong. 
(cherry picked from commit a77d628339)
 2015-04-13 17:41:02 -07:00
Jian He 1afea8a15a YARN-3394. Enrich WebApplication proxy documentation. Contributed by Naganarasimha G R 
(cherry picked from commit 7b4671490a)
 2015-04-13 17:05:58 -07:00
Jian He d58f5c8894 YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens. Contributed by Rohith Sharmaks 
(cherry picked from commit a1afbc48b5)
 2015-04-13 14:07:51 -07:00
Karthik Kambatla fa12700379 Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 
(cherry picked from commit 5cda6fffd3)
 2015-04-13 13:52:44 -07:00
Jian He 32b01263d3 YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root queue metrics. Contributed by Rohit Agarwal 
(cherry picked from commit 944a16579f)
 2015-04-13 10:54:15 -07:00
Karthik Kambatla ad99d268e5 YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) 
(cherry picked from commit e516706b89)
 2015-04-10 11:21:30 -07:00
Jian He d3daf9665c YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 10:40:31 -07:00
Junping Du 0891de0712 YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong. 
(cherry picked from commit 92431c9617)
 2015-04-10 08:58:07 -07:00
Xuan f5b49160d9 YARN-3293. Track and display capacity scheduler health metrics in web 
UI. Contributed by Varun Vasudev

(cherry picked from commit afa5d4715a)
 2015-04-09 23:40:02 -07:00
Vinod Kumar Vavilapalli aec1c5ec7d Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1e)
 2015-04-09 13:56:14 -07:00
Vinod Kumar Vavilapalli 1ff3fd33ed YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp. 
(cherry picked from commit 9c5911294e)
 2015-04-09 13:10:06 -07:00
Wangda Tan 73d9e4c77b YARN-3466. Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column. (Jason Lowe via wangda) 
(cherry picked from commit 1885141e90)
 2015-04-09 10:36:10 -07:00
Karthik Kambatla 53e0bf5c17 YARN-3465. Use LinkedHashMap to preserve order of resource requests. (Zhihai Xu via kasha) 
(cherry picked from commit 6495940eae)
 2015-04-09 00:10:23 -07:00
Hitesh Shah 55b794e7fa YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe80)
 2015-04-08 14:14:06 -07:00
Wangda Tan 5f59e621ba YARN-2901 addendum: Fixed findbugs warning caused by previously patch 
(cherry picked from commit ba9ee22ca4)
 2015-04-08 11:02:54 -07:00
Wangda Tan 97244b143c YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender. (Varun Vasudev via wangda) 
(cherry picked from commit 7af086a515)
 2015-04-08 10:58:38 -07:00
Tsuyoshi Ozawa ac32fa187c YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. 
(cherry picked from commit dd852f5b8c)
 2015-04-08 15:56:33 +09:00
Robert Kanter b584ce6c0a YARN-3429. Fix incorrect CHANGES.txt 
(cherry picked from commit 5b8a3ae366)
 2015-04-07 16:16:08 -07:00
Xuan 0522d6970d YARN-3294. Allow dumping of Capacity Scheduler debug logs via web UI for 
a fixed time period. Contributed by Varun Vasudev

(cherry picked from commit d27e9241e8)
 2015-04-07 09:54:29 -07:00
Xuan 7941cc554b YARN-3110. Few issues in ApplicationHistory web ui. Contributed by Naganarasimha G R 
(cherry picked from commit 19a4feaf6f)
 2015-04-07 08:24:09 -07:00
Zhijie Shen 735e4f15ab Move YARN-3273 from 2.8 to 2.7. 
(cherry picked from commit 5e9f4099de4b949c838a6224442f335c67d80936)
 2015-04-06 14:15:23 -07:00
Robert Kanter 354d2c75cf YARN-2429. TestAMRMTokens.testTokenExpiry fails Intermittently with error message:Invalid AMRMToken (zxu via rkanter) 
(cherry picked from commit 99b08a748e)
 2015-04-06 14:11:50 -07:00
Tsuyoshi Ozawa d2b2d76cce TestFairScheduler.testContinuousScheduling fails Intermittently. Contributed by Zhihai Xu. 
(cherry picked from commit 53959e69f7)
 2015-04-06 20:19:47 +09:00
Xuan 1eb6d74967 YARN-3435. AM container to be allocated Appattempt AM container shown as 
null. Contributed by Bibin A Chundatt

(cherry picked from commit 96d72118f5)
 2015-04-05 00:43:22 -07:00
Wangda Tan 405aead879 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 
(cherry picked from commit bad070fe15)
 2015-04-02 17:23:44 -07:00
Vinod Kumar Vavilapalli d8e17c58bc YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana. 
(cherry picked from commit b21c72777a)
 2015-04-02 16:55:00 -07:00
Sandy Ryza 6286cfd711 YARN-3415. Non-AM containers can be counted towards amResourceUsage of a fairscheduler queue (Zhihai Xu via Sandy Ryza) 2015-04-02 13:57:53 -07:00
Zhijie Shen 6d0bebc912 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 
(cherry picked from commit 8366a36ad3)
 2015-04-01 13:49:00 -07:00
Wangda Tan 865be70b02 YARN-3425. NPE from RMNodeLabelsManager.serviceStop when NodeLabelsManager.serviceInit failed. (Bibin A Chundatt via wangda) 
(cherry picked from commit 492239424a)
 2015-04-01 10:15:47 -07:00
Xuan 5e4d457674 YARN-3248. Correct fix version from branch-2.7 to branch-2.8 in the change log. 
(cherry picked from commit 2e79f1c212)
 2015-04-01 04:33:32 -07:00
Xuan e26b6e55e9 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15)
 2015-04-01 04:20:37 -07:00
Tsuyoshi Ozawa 80755edb7a YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. 
(cherry picked from commit c69ba81497)
 2015-04-01 17:44:40 +09:00
Karthik Kambatla 93b29d962d YARN-3428. Debug log resources to be localized for a container. (kasha) 
(cherry picked from commit 2daa478a64)
 2015-03-31 17:35:11 -07:00
Vinod Kumar Vavilapalli c980e34bc5 YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) 
(cherry picked from commit 7610925e90)
 2015-03-31 17:28:48 -07:00
Karthik Kambatla 85c420a4d4 YARN-3412. RM tests should use MockRM where possible. (kasha) 
(cherry picked from commit 79f7f2aabf)
 2015-03-31 09:22:16 -07:00
Tsuyoshi Ozawa 99b8255693 YARN-3258. FairScheduler: Need to add more logging to investigate allocations. Contributed by Anubhav Dhoot. 
(cherry picked from commit b5a22e9838)
 2015-03-31 17:43:07 +09:00
Wangda Tan cba4ed1678 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 
(cherry picked from commit 2a945d24f7)
 2015-03-30 12:05:54 -07:00
Vinod Kumar Vavilapalli c5bc48946d YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f51)
 2015-03-30 10:11:12 -07:00
Ravi Prakash b1b4951452 YARN-3288. Document and fix indentation in the DockerContainerExecutor code 
(cherry picked from commit e0ccea33c9)
 2015-03-28 08:01:26 -07:00
Robert Kanter 6f0e864d61 YARN-3400. [JDK 8] Build Failure due to unreported exceptions in RPCUtil (rkanter) 
(cherry picked from commit 87130bf6b2)
 2015-03-26 11:00:37 -07:00
Haohui Mai 17f17dac4e HADOOP-10670. Allow AuthenticationFilters to load secret from signature secret files. Contributed by Kai Zheng. 2015-03-25 11:13:04 -07:00
Karthik Kambatla 2158cd2bef YARN-3397. yarn rmadmin should skip -failover. (J.Andreina via kasha) 
(cherry picked from commit c906a1de72)
 2015-03-25 07:42:45 -07:00
Xuan 0c4fa872be YARN-2213. Change proxy-user cookie log in AmIpFilter to DEBUG. 
Contributed by Varun Saxena

(cherry picked from commit e556198e71)
 2015-03-25 04:51:55 -07:00
Wangda Tan 29629020c1 YARN-3383. AdminService should use warn instead of info to log exception when operation fails. (Li Lu via wangda) 
(cherry picked from commit 97a7277a2d)
 2015-03-24 10:39:45 -07:00
Harsh J e7702ac6a1 YARN-1880. Cleanup TestApplicationClientProtocolOnHA. Contributed by ozawa. 
(cherry picked from commit 8a9bbf548dad82598354c24fcd4dc80a715da7ac)
 2015-03-24 11:58:16 +05:30
Xuan cbdcdfad6d YARN-3393. Getting application(s) goes wrong when app finishes before 
starting the attempt. Contributed by Zhijie Shen

(cherry picked from commit 9fae455e26)
 2015-03-23 20:34:29 -07:00
Karthik Kambatla 4e0c48703e YARN-2868. FairScheduler: Metric for latency to allocate first container for an application. (Ray Chiang via kasha) 
(cherry picked from commit 972f1f1ab9)
 2015-03-23 14:10:23 -07:00
Karthik Kambatla 75591e4131 YARN-3241. FairScheduler handles invalid queue names inconsistently. (Zhihai Xu via kasha) 
(cherry picked from commit 2bc097cd14)
 2015-03-23 13:24:22 -07:00
cnauroth 342c525eaa YARN-3336. FileSystem memory leak in DelegationTokenRenewer. 
(cherry picked from commit 6ca1f12024)
 2015-03-23 10:46:06 -07:00
Tsuyoshi Ozawa cbacf20755 YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R. 
(cherry picked from commit 82eda771e0)
 2015-03-24 00:25:52 +09:00
Jian He 9c494cedaf YARN-3345. Add non-exclusive node label API. Contributed by Wangda Tan 
(cherry picked from commit e1feb4ea1a)
 2015-03-20 19:05:14 -07:00
Junping Du 9aedb3d688 YARN-3350. YARN RackResolver spams logs with messages at info level. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 7f1e2f9969)
 2015-03-20 18:22:23 -07:00
Jian He a3a155a34e YARN-3356. Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 586348e4cb)
 2015-03-20 13:54:46 -07:00
Junping Du f40f17489c YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 
(cherry picked from commit d81109e588)
 2015-03-20 13:42:31 -07:00
Wangda Tan 8e142d27cb YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda) 
(cherry picked from commit 6bc7710ec7)
 2015-03-20 10:42:51 -07:00
Xuan 7990e1d80d YARN-2777. Mark the end of individual log in aggregated log. Contributed 
by Varun Saxena

(cherry picked from commit 1a4b528691)
 2015-03-20 10:25:26 -07:00
Jian He 3f0c9e5fe3 YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 
(cherry picked from commit 4e886eb9cb)
 2015-03-19 22:27:57 -07:00
Devaraj K a888dac344 YARN-3357. Move TestFifoScheduler to FIFO package. Contributed by Rohith 
Sharmaks.

(cherry picked from commit 93d0f4acc8)
 2015-03-19 12:18:08 +05:30
Karthik Kambatla 05627dd96d YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 
(cherry picked from commit 20b49224eb)
 2015-03-18 16:33:57 -07:00
Jian He 15ebacf03a YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 
(cherry picked from commit 658097d6da)
 2015-03-17 21:31:20 -07:00
Tsuyoshi Ozawa b00b216a91 YARN-3205. FileSystemRMStateStore should disable FileSystem Cache to avoid get a Filesystem with an old configuration. Contributed by Zhihai Xu. 
(cherry picked from commit 3bc72cc16d)
 2015-03-18 11:53:40 +09:00
Jian He ab34e6975b YARN-3305. Normalize AM resource request on app submission. Contributed by Rohith Sharmaks 
(cherry picked from commit 968425e9f7)
 2015-03-17 13:50:44 -07:00
Karthik Kambatla 1e77d92d62 Revert "YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha)" 
This reverts commit c2b185def8.

(cherry picked from commit 32b4330456)
 2015-03-17 12:31:44 -07:00
Jian He 1c601e492f YARN-3243. CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. Contributed by Wangda Tan. 
(cherry picked from commit 487374b7fe)
 2015-03-17 10:25:07 -07:00
Devaraj K 895588b439 YARN-3197. Confusing log generated by CapacityScheduler. Contributed by 
Varun Saxena.

(cherry picked from commit 7179f94f9d)
 2015-03-17 15:59:19 +05:30
Ravi Prakash 9f227ad696 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 
(cherry picked from commit 56085203c4)
 2015-03-16 16:18:42 -07:00
Tsuyoshi Ozawa a20427d339 YARN-3349. Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry. Contributed by Zhihai Xu. 
(cherry picked from commit 7522a643fa)
 2015-03-17 08:10:12 +09:00
Harsh J 2684b0b440 MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration, for debug purposes. Contributed by Plamen Jeliazkov. 
(cherry picked from commit 571f75b468a4412be2178466cb5d3888b05cd076)
 2015-03-17 01:02:21 +05:30
Zhijie Shen fbe811d904 YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. 2015-03-16 10:52:32 -07:00
Tsuyoshi Ozawa d6e3ef2506 YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 
(cherry picked from commit 3da9a97cfb)
 2015-03-16 23:20:17 +09:00
Xuan b6eb36dbdc YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R

(cherry picked from commit 3ff1ba2a7b)
 2015-03-15 20:27:44 -07:00
Jonathan Eagles 44aedad5dd YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 
(cherry picked from commit 8180e676ab)
 2015-03-13 12:05:21 -05:00
Vinod Kumar Vavilapalli 53aa3a4d1f YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 863079bb87)
 2015-03-12 13:33:42 -07:00
Xuan 498d775c9b YARN-3338. Exclude jline dependency from YARN. Contributed by Zhijie 
Shen

(cherry picked from commit 06ce1d9a6c)
 2015-03-12 10:25:55 -07:00
Zhijie Shen 426535007b YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 
(cherry picked from commit 85f6d67fa7)
 2015-03-11 19:39:46 -07:00
Tsuyoshi Ozawa d29cf296cb YARN-3295. Fix documentation nits found in markdown conversion. Contributed by Masatake Iwasaki. 
(cherry picked from commit 30c428a858)
 2015-03-11 15:02:45 +09:00
Jian He 2690c72522 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty 
(cherry picked from commit a380643d20)
 2015-03-10 11:16:51 -07:00
Jian He 4021451746 YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 
(cherry picked from commit c3003eba6f)
 2015-03-09 20:47:33 -07:00
Zhijie Shen a94d23762e YARN-3287. Made TimelineClient put methods do as the correct login context. Contributed by Daryn Sharp and Jonathan Eagles. 
(cherry picked from commit d6e05c5ee2)
 2015-03-09 13:55:51 -07:00
Tsuyoshi Ozawa b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Vinod Kumar Vavilapalli 8e6f8e9a04 Adding 2.8 section in CHANGES.txt 2015-03-08 20:26:57 -07:00
Junping Du b91e1b0feb YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. Contributed by Hitesh Shah 
(cherry picked from commit 7ce3c76353)
 2015-03-08 14:48:42 -07:00
Jason Lowe ef3d9bdf6b YARN-3275. CapacityScheduler: Preemption happening on non-preemptable queues. Contributed by Eric Payne 
(cherry picked from commit 27e8ea820f)
 2015-03-06 22:45:09 +00:00
Jian He 6cef2c16de YARN-2190. Added CPU and memory limit options to the default container executor for Windows containers. Contributed by Chuan Liu 
(cherry picked from commit 21101c01f2)
 2015-03-06 14:18:56 -08:00
Xuan 56c2050ab7 YARN-3227. Timeline renew delegation token fails when RM user's TGT is 
expired. Contributed by Zhijie Shen

(cherry picked from commit d1abc5d4fc)
 2015-03-06 13:33:18 -08:00
Jian He a5f3fb4dc1 YARN-1809. Synchronize RM and TimeLineServer Web-UIs. Contributed by Zhijie Shen and Xuan Gong 
(cherry picked from commit 95bfd087dc)
 2015-03-05 21:27:03 -08:00
Vinod Kumar Vavilapalli 9c8882546a YARN-2786. Created a yarn cluster CLI and seeded with one command for listing node-labels collection. Contributed by Wangda Tan. 2015-03-05 10:56:28 -08:00
Tsuyoshi Ozawa 5680b41597 Update CHANGES.txt for YARN-2616 to fix indentation. 
(cherry picked from commit 28b85a2116)
 2015-03-06 00:53:20 +09:00
Tsuyoshi Ozawa 6660c2f83b YARN-3249. Add a 'kill application' button to Resource Manager's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 1b67209612)
 2015-03-05 19:56:59 +09:00
Karthik Kambatla 0d62e94887 YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client. (Zhihai Xu via kasha) 
(cherry picked from commit 8d88691d16)
 2015-03-04 19:49:21 -08:00
Karthik Kambatla 721d7b5741 YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) 
(cherry picked from commit 22426a1c9f)
 2015-03-04 18:07:33 -08:00
Karthik Kambatla 1749094fab YARN-3122. Metrics for container's actual CPU usage. (Anubhav Dhoot via kasha) 
(cherry picked from commit 53947f37c7)
 2015-03-04 17:39:39 -08:00
Jason Lowe ab397194c3 YARN-3131. YarnClientImpl should check FAILED and KILLED state in submitApplication. Contributed by Chang Li 
(cherry picked from commit 03cc22945e)
 2015-03-04 18:05:20 +00:00
Jian He aafe571398 YARN-3285. (Backport YARN-3168) Convert branch-2 .apt.vm files of YARN to markdown. Contributed by Masatake Iwasaki 2015-03-03 16:44:59 -08:00
Jian He 888a445638 YARN-3222. Fixed NPE on RMNodeImpl#ReconnectNodeTransition when a node is reconnected with a different port. Contributed by Rohith Sharmaks 
(cherry picked from commit b2f1ec312e)
 2015-03-03 16:28:55 -08:00
Wangda Tan 0a502c665b YARN-3272. Surface container locality info in RM web UI (Jian He via wangda) 
(cherry picked from commit e17e5ba9d7)
 2015-03-03 11:49:45 -08:00
Vinod Kumar Vavilapalli 253c78548b YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's available resource-limit from the parent queue. Contributed by Wangda Tan. 
(cherry picked from commit 14dd647c55)
 2015-03-02 17:54:01 -08:00
Wangda Tan 19a8e48230 YARN-3270. Fix node label expression not getting set in ApplicationSubmissionContext (Rohit Agarwal via wangda) 
(cherry picked from commit abac6eb9d5)
 2015-03-02 17:24:55 -08:00
Jian He 76b7283fa1 YARN-3281. Added RMStateStore to StateMachine visualization list. Contributed by Chengbing Liu 
(cherry picked from commit 5d0bae550f)
 2015-03-02 14:43:12 -08:00
Wangda Tan 0b0be0056b YARN-3262. Surface application outstanding resource requests table in RM web UI. (Jian He via wangda) 
(cherry picked from commit edcecedc1c)
 2015-02-27 16:14:35 -08:00
Tsuyoshi Ozawa 79f73f4613 YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail due to IOException. Contributed by Zhihai Xu. 
(cherry picked from commit 01a1621930)
 2015-02-28 00:57:01 +09:00