Commit Graph

1976 Commits

Author SHA1 Message Date
Devaraj K a28cd02ba0 YARN-3592. Fix typos in RMNodeLabelsManager. Contributed by Sunil G. 2015-05-08 10:54:36 +05:30
Akira Ajisaka a521b50955 YARN-3572. Correct typos in WritingYarnApplications.md. Contributed by Gabor Liptak. 2015-05-08 13:24:29 +09:00
Akira Ajisaka b167fe7605 YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo. 2015-05-08 11:14:45 +09:00
Jian He f489a4ec96 YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan 2015-05-07 17:35:41 -07:00
Jian He b88700dcd0 YARN-3584. Fixed attempt diagnostics format shown on the UI. Contributed by nijel 2015-05-07 13:54:07 -07:00
Zhijie Shen daf3e4ef8b YARN-3448. Added a rolling time-to-live LevelDB timeline store implementation. Contributed by Jonathan Eagles. 2015-05-07 10:01:51 -07:00
Junping Du 8e991f4b1d YARN-3523. Cleanup ResourceManagerAdministrationProtocol interface audience. Contributed by Naganarasimha G R 2015-05-07 05:39:04 -07:00
Akira Ajisaka 918af8efff YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 13:33:03 +09:00
Vinod Kumar Vavilapalli 4c7b9b6abe YARN-3385. Fixed a race-condition in ResourceManager's ZooKeeper based state-store to avoid crashing on duplicate deletes. Contributed by Zhihai Xu. 2015-05-06 17:51:17 -07:00
Junping Du 31b627b2a8 YARN-3580. [JDK8] TestClientRMService.testGetLabelsToNodes fails. Contributed by Robert Kanter. 2015-05-06 16:51:05 -07:00
Robert Kanter b72507810a YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter) 2015-05-06 14:19:06 -07:00
Jian He 0d3188fd25 YARN-3301. Fixed the format issue of the new RM attempt web page. Contributed by Xuan Gong 2015-05-06 13:45:00 -07:00
Vinod Kumar Vavilapalli 185e63a726 YARN-3243. Moving CHANGES.txt entry to the right release. 2015-05-06 10:42:50 -07:00
Xuan a583a40693 YARN-3582. NPE in WebAppProxyServlet. Contributed by Jian He 2015-05-05 21:42:19 -07:00
Jian He e4c3b52c89 YARN-3343. Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate timeout. Contributed by Rohith Sharmaks 2015-05-05 11:33:47 -07:00
Xuan b7dd3a4f04 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale.
Contributed by Akira AJISAKA
2015-05-05 10:40:16 -07:00
Junping Du 3810242062 YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula) 2015-05-05 10:18:23 -07:00
Jason Lowe 9356cf8676 YARN-3552. RM Web UI shows -1 running containers for completed apps. Contributed by Rohith 2015-05-05 13:42:32 +00:00
Jian He d701acc9c6 YARN-2725. Added test cases of retrying creating znode in ZKRMStateStore. Contributed by Tsuyoshi Ozawa 2015-05-04 16:13:29 -07:00
Wangda Tan 71f4de220c YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda) 2015-05-04 15:49:19 -07:00
Jason Lowe 8f65c793f2 YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne 2015-05-04 15:31:15 +00:00
Tsuyoshi Ozawa e8d0ee5fc9 YARN-1993. Cross-site scripting vulnerability in TextView.java. Contributed byKenji Kikushima. 2015-05-03 10:51:17 +09:00
Junping Du 57d9a972cb YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources. Contributed by Xu Yang. 2015-05-02 10:18:44 -07:00
Gera Shegalov f8204e241d YARN-2893. AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream. (Zhihai Xu via gera) 2015-05-01 18:18:55 -07:00
Wangda Tan 7d46a806e7 YARN-3006. Improve the error message when attempting manual failover with auto-failover enabled. (Akira AJISAKA via wangda) 2015-05-01 16:40:55 -07:00
Robert Kanter ac7d152901 YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) 2015-05-01 16:39:21 -07:00
Vinod Kumar Vavilapalli 1b3b9e5c31 YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. 2015-04-30 21:41:07 -07:00
Zhijie Shen 7e8639fda4 YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 2015-04-30 13:05:55 -07:00
Wangda Tan e2e8f77118 YARN-3564. Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly. (Jian He via wangda) 2015-04-30 11:03:19 -07:00
Jian He 4c1af156ae YARN-3533. Test: Fix launchAM in MockRM to wait for attempt to be scheduled. Contributed by Anubhav Dhoot 2015-04-29 14:50:01 -07:00
tgraves 2e215484bd YARN-3517. RM web ui for dumping scheduler logs should be for admins only (Varun Vasudev via tgraves) 2015-04-29 21:25:42 +00:00
Karthik Kambatla 8f82970e0c YARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha) 2015-04-28 21:00:35 -07:00
Wangda Tan db1b674b50 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 2015-04-27 16:24:38 -07:00
Karthik Kambatla 32cd2c8d42 Update CHANGES.txt - Pulled in YARN-3465, YARN-3516, and YARN-3464 to branch-2.7 (for 2.7.1) 2015-04-27 13:43:43 -07:00
Xuan 7f07c4d810 YARN-3530. ATS throws exception on trying to filter results without
otherinfo. Contributed by zhijie shen
2015-04-27 10:36:42 -07:00
Karthik Kambatla 47279c3228 YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) 2015-04-26 09:13:46 -07:00
Vinod Kumar Vavilapalli 2f82ae042a Moving YARN-3351, YARN-3382, YARN-3472, MAPREDUCE-6238 to the 2.7.1 CHANGES.txt
sections given the recent merge into branch-2.7.
2015-04-24 17:18:46 -07:00
Jian He d497f6ea2b YARN-2498. Respect labels in preemption policy of capacity scheduler for inter-queue preemption. Contributed by Wangda Tan 2015-04-24 17:03:13 -07:00
Tsuyoshi Ozawa 4a3dabd94f YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi. 2015-04-25 07:17:11 +09:00
Jason Lowe 78fe6e57c7 Fix commit version for YARN-3537 2015-04-24 22:07:53 +00:00
Jason Lowe 5e093f0d40 YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula 2015-04-24 22:02:53 +00:00
Akira Ajisaka 5ce3a77f3c YARN-3444. Fix typo capabililty. Contributed by Gabor Liptak. 2015-04-25 06:08:16 +09:00
Jian He d03dcb9635 YARN-3387. Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time. Contributed by Sandflee 2015-04-24 12:13:29 -07:00
Xuan eee9facbba YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev 2015-04-24 09:41:59 -07:00
Xuan 0b3f8957a8 YARN-3516. killing ContainerLocalizer action doesn't take effect when
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu
2015-04-23 16:40:40 -07:00
Vinod Kumar Vavilapalli f5fe35e297 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 2015-04-23 11:19:55 -07:00
Jian He aa4a192feb YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen 2015-04-23 11:07:26 -07:00
Wangda Tan 395205444e YARN-3319. Implement a FairOrderingPolicy. (Craig Welch via wangda) 2015-04-23 10:47:15 -07:00
tgraves 189a63a719 YARN-3434. Interaction between reservations and userlimit can result in significant ULF violation 2015-04-23 14:39:25 +00:00
Steve Loughran d9bcf99aef YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:48 +01:00
Vinod Kumar Vavilapalli a100be685c YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana. 2015-04-22 17:26:13 -07:00
Junping Du fad9d7e85b New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 2015-04-22 10:07:20 -07:00
Akira Ajisaka b08908ae5e YARN-3410. Addendum fix for compilation error. Contributed by Rohith. 2015-04-22 17:47:00 +09:00
Jian He 674c7ef649 YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev 2015-04-21 20:57:02 -07:00
Jian He bdd90110e6 YARN-3494. Expose AM resource limit and usage in CS QueueMetrics. Contributed by Rohith Sharmaks 2015-04-21 20:06:20 -07:00
Wangda Tan e71d0d87d9 YARN-3410. YARN admin should be able to remove individual application records from RMStateStore. (Rohith Sharmaks via wangda) 2015-04-21 17:51:22 -07:00
Tsuyoshi Ozawa 105afd5477 YARN-3495. Confusing log generated by FairScheduler. Contributed by Brahma Reddy Battula. 2015-04-22 05:47:59 +09:00
Vinod Kumar Vavilapalli d52de61544 Set the release date for 2.7.0 2015-04-20 20:16:58 -07:00
Wangda Tan 44872b76fc YARN-3463. Integrate OrderingPolicy Framework with CapacityScheduler. (Craig Welch via wangda) 2015-04-20 17:12:32 -07:00
Jian He f967fd2f21 YARN-3497. ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy. Contributed by Jason Lowe 2015-04-20 10:38:27 -07:00
Jian He 497c86b485 YARN-3136. Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers. Contributed by Sunil G 2015-04-18 12:46:37 -07:00
Wangda Tan f47a5763ac YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) 2015-04-17 17:22:27 -07:00
Wangda Tan f65eeb412d YARN-3493. RM fails to come up with error "Failed to load/recover state" when mem settings are changed. (Jian He via wangda) 2015-04-17 17:11:22 -07:00
Jian He 6779467ab6 YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 2015-04-17 15:55:34 -07:00
Jian He d573f09fb9 YARN-2696. Queue sorting in CapacityScheduler should consider node label. Contributed by Wangda Tan 2015-04-17 13:36:59 -07:00
Junping Du 1db355a875 YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 2015-04-17 13:18:59 -07:00
Jian He bb6dde68f1 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 2015-04-16 19:43:37 -07:00
Akira Ajisaka 1fa8075794 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 2015-04-16 10:04:09 -05:00
Jian He 1b89a3e173 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 2015-04-15 13:57:06 -07:00
Jian He b2e6cf607f YARN-3404. Display queue name on application page. Contributed by Ryu Kobayashi 2015-04-15 13:52:50 -07:00
Tsuyoshi Ozawa e48cedc663 YARN-3326. Support RESTful API for getLabelsToNodes. Contributed by Naganarasimha G R. 2015-04-15 14:03:55 -05:00
Wangda Tan 5004e75332 YARN-3318. Create Initial OrderingPolicy Framework and FifoOrderingPolicy. (Craig Welch via wangda) 2015-04-15 09:56:32 -07:00
Tsuyoshi Ozawa 05007b45e5 YARN-3436. Fix URIs in documantion of YARN web service REST APIs. Contributed by Bibin A Chundatt. 2015-04-14 14:13:55 -05:00
Jian He 0fefda645b YARN-3361. CapacityScheduler side changes to support non-exclusive node labels. Contributed by Wangda Tan 2015-04-14 11:45:58 -07:00
Jian He b46ee1e7a3 YARN-3266. RMContext#inactiveNodes should have NodeId as map key. Contributed by Chengbing Liu 2015-04-14 10:54:22 -07:00
Junping Du 838b06ac87 YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. 2015-04-13 18:35:56 -07:00
Junping Du a77d628339 YARN-3347. Improve YARN log command to get AMContainer logs as well as running containers logs. Contributed by Xuan Gong. 2015-04-13 17:29:06 -07:00
Jian He 7b4671490a YARN-3394. Enrich WebApplication proxy documentation. Contributed by Naganarasimha G R 2015-04-13 17:05:38 -07:00
Jian He a1afbc48b5 YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens. Contributed by Rohith Sharmaks 2015-04-13 14:07:17 -07:00
Karthik Kambatla 5cda6fffd3 Fix up author name to Jun Gong in CHANGES.txt for YARN-3469 2015-04-13 13:52:28 -07:00
Jian He 944a16579f YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root queue metrics. Contributed by Rohit Agarwal 2015-04-13 10:53:34 -07:00
Karthik Kambatla e516706b89 YARN-3469. ZKRMStateStore: Avoid setting watches that are not required. (Jun Hong via kasha) 2015-04-10 11:20:34 -07:00
Jian He 577d755e4b YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 09:57:17 -07:00
Junping Du 92431c9617 YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong. 2015-04-10 08:56:18 -07:00
Xuan afa5d4715a YARN-3293. Track and display capacity scheduler health metrics in web
UI. Contributed by Varun Vasudev
2015-04-09 23:38:04 -07:00
Vinod Kumar Vavilapalli 623fd46c1e Adding release 2.7.1 to CHANGES.txt 2015-04-09 13:54:20 -07:00
Vinod Kumar Vavilapalli 9c5911294e YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp. 2015-04-09 13:08:53 -07:00
Wangda Tan 1885141e90 YARN-3466. Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column. (Jason Lowe via wangda) 2015-04-09 10:35:12 -07:00
Karthik Kambatla 6495940eae YARN-3465. Use LinkedHashMap to preserve order of resource requests. (Zhihai Xu via kasha) 2015-04-09 00:07:49 -07:00
Hitesh Shah 265ed1fe80 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 2015-04-08 14:13:10 -07:00
Wangda Tan ba9ee22ca4 YARN-2901 addendum: Fixed findbugs warning caused by previously patch 2015-04-08 11:02:06 -07:00
Wangda Tan 7af086a515 YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender. (Varun Vasudev via wangda) 2015-04-08 10:57:48 -07:00
Tsuyoshi Ozawa dd852f5b8c YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. 2015-04-08 15:56:18 +09:00
Robert Kanter 5b8a3ae366 YARN-3429. Fix incorrect CHANGES.txt 2015-04-07 16:15:42 -07:00
Xuan d27e9241e8 YARN-3294. Allow dumping of Capacity Scheduler debug logs via web UI for
a fixed time period. Contributed by Varun Vasudev
2015-04-07 09:52:36 -07:00
Xuan 19a4feaf6f YARN-3110. Few issues in ApplicationHistory web ui. Contributed by Naganarasimha G R 2015-04-07 08:22:39 -07:00
Zhijie Shen 3fb5abfc87 Move YARN-3273 from 2.8 to 2.7. 2015-04-06 14:14:53 -07:00
Robert Kanter 99b08a748e YARN-2429. TestAMRMTokens.testTokenExpiry fails Intermittently with error message:Invalid AMRMToken (zxu via rkanter) 2015-04-06 14:11:20 -07:00
Tsuyoshi Ozawa 53959e69f7 TestFairScheduler.testContinuousScheduling fails Intermittently. Contributed by Zhihai Xu. 2015-04-06 20:19:13 +09:00
Xuan 96d72118f5 YARN-3435. AM container to be allocated Appattempt AM container shown as
null. Contributed by Bibin A Chundatt
2015-04-05 00:40:57 -07:00