Commit Graph

2619 Commits

Author SHA1 Message Date
Wangda Tan 7e8c9beb41 CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 2016-03-16 16:59:59 -07:00
Karthik Kambatla 3ef5500783 YARN-4560. Make scheduler error checking message more user friendly. (Ray Chiang via kasha) 2016-03-15 23:45:01 -07:00
Xuan a888b5bc53 Revert "YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)"
This reverts commit 3a0b695441.
2016-03-15 22:10:39 -07:00
Li Lu 5176a69868 YARN-4817. Change Log Level to DEBUG for putDomain call in ATS 1.5. (Xuan Gong via gtcarrera9) 2016-03-15 11:47:39 -07:00
Li Lu 3a0b695441 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9) 2016-03-15 11:40:54 -07:00
Robert Kanter 22ca176dfe TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 2016-03-15 10:05:10 -07:00
Siddharth Seth eba66a64d2 YARN-4816. Fix incompatible change in SystemClock. 2016-03-14 20:14:33 -07:00
Karthik Kambatla 20d389ce61 YARN-4719. Add a helper library to maintain node state and allows common queries. (kasha) 2016-03-14 14:19:05 -07:00
Junping Du f291d82cd4 YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 2016-03-14 08:28:38 -07:00
Li Lu 017d2c127b YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency.
This commit amends commit d49cfb3504 with a missed test file.
2016-03-10 13:02:28 -08:00
Li Lu d49cfb3504 YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9) 2016-03-10 10:51:55 -08:00
Chris Nauroth 89b16d27e2 HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth. 2016-03-08 21:22:06 -08:00
Wangda Tan 0233d4e0ee YARN-4465. SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled. (Bibin A Chundatt via wangda) 2016-03-08 14:27:03 -08:00
Jian He 3c33158d1c YARN-4764. Application submission fails when submitted queue is not available in scheduler xml. Contributed by Bibin A Chundatt 2016-03-08 13:07:57 -08:00
Akira Ajisaka f86850b544 HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang. 2016-03-08 14:28:47 +09:00
Vinod Kumar Vavilapalli b2661765a5 YARN-4762. Fixed CgroupHandler's creation and usage to avoid NodeManagers crashing when LinuxContainerExecutor is enabled. (Sidharta Seethana via vinodkv) 2016-03-07 11:08:17 -08:00
Jason Lowe 4163e36c2b YARN-4760. proxy redirect to history server uses wrong URL. Contributed by Eric Badger 2016-03-07 15:56:33 +00:00
Jason Lowe 059caf9989 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 2016-03-07 15:40:01 +00:00
Varun Vasudev 8ed2e060e8 YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 2016-03-07 16:18:35 +05:30
Varun Vasudev e51a8c1056 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 2016-03-07 15:26:44 +05:30
Zhihai Xu e1ccc9622b YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee 2016-03-06 19:46:09 -08:00
Rohith Sharma K S 19ee185907 YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks) 2016-03-05 13:02:57 +05:30
Masatake Iwasaki cbd31328a6 HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 2016-03-04 14:11:36 +09:00
Andrew Wang 1bc454615e HADOOP-11792. Remove all of the CHANGES.txt files. 2016-03-03 21:02:31 -08:00
Jian He 5c465df904 YARN-4671. There is no need to acquire CS lock when completing a container. Contributed by Meng Ding 2016-03-01 13:14:12 -08:00
Jian He d93c22ec27 YARN-4748. ApplicationHistoryManagerOnTimelineStore should not swallow exceptions on generateApplicationReport. Contributed by Li Lu 2016-02-29 18:19:09 -08:00
Karthik Kambatla 9dafaaaf0d YARN-4704. TestResourceManager#testResourceAllocation() fails when using FairScheduler. (Yufei Gu via kasha) 2016-02-29 16:10:12 -08:00
Haohui Mai 0fa54d45b1 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 11:41:00 -08:00
Jason Lowe c58a6d53c5 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 2016-02-29 15:24:35 +00:00
Rohith Sharma K S e0b14f26f5 YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 2016-02-29 10:38:14 +08:00
Karthik Kambatla f9692770a5 YARN-4718. Rename variables in SchedulerNode to reduce ambiguity post YARN-1011. (Inigo Goiri via kasha) 2016-02-28 09:35:59 -08:00
Jason Lowe 6b0f813e89 YARN-4723. NodesListManager$UnknownNodeId ClassCastException. Contributed by Kuhu Shukla 2016-02-26 20:24:50 +00:00
Ming Ma 7f3139e54d YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 2016-02-26 08:40:05 -08:00
Robert Kanter d7fdec1e6b YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 2016-02-25 16:36:38 -08:00
Robert Kanter c4d4df8de0 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 2016-02-25 12:48:02 -08:00
Karthik Kambatla c684f2b007 YARN-4729. SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE. (kasha) 2016-02-24 18:33:57 -08:00
Robert Kanter 954dd57043 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 2016-02-24 15:00:24 -08:00
Li Lu 9e0f7b8b69 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via
gtcarrera9)
2016-02-24 13:43:09 -08:00
Sangjin Lee 553b591ba0 YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 2016-02-24 09:29:41 -08:00
Jason Lowe d284e187b8 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 2016-02-23 20:49:09 +00:00
Junping Du 9ed17f181d YARN-3223. Resource update during NM graceful decommission. Contributed by Brook Zhou. 2016-02-23 03:30:26 -08:00
Tsuyoshi Ozawa 0e12114c9c YARN-4648. Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption. Contributed by Kai Sasaki. 2016-02-23 19:50:08 +09:00
Rohith Sharma K S de09f41147 YARN-4651. Document movetoqueue option in 'YARN Commands'. (Takashi Ohnishi via rohithsharmaks) 2016-02-23 18:03:42 +08:00
Varun Vasudev 140cb5d745 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 2016-02-23 12:29:25 +05:30
Zhe Zhang a2fdfff02d HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh.
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
2016-02-22 14:13:54 -08:00
Junping Du 3fab88540f YARN-4386. refreshNodesGracefully() should send recommission event to active RMNodes only. Contributed by Kuhu Shukla. 2016-02-22 07:04:19 -08:00
Akira Ajisaka 5e7d4d51f8 YARN-4707. Remove the extra char (>) from SecureContainer.md. Contributed by Brahma Reddy Battula. 2016-02-22 17:17:40 +09:00
Tsuyoshi Ozawa b68901d7dd YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki. 2016-02-22 07:54:53 +09:00
Rohith Sharma K S c1afac3a98 YARN-4654. Yarn node label CLI should parse "=" correctly when trying to remove all labels on a node. (Naganarasimha G R via rohithsharmaks) 2016-02-18 14:10:37 +05:30
Sangjin Lee 7de70680fe YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee) 2016-02-17 20:55:21 -08:00
Karthik Kambatla a0c95b5fc4 YARN-4569. Remove incorrect part of maxResources in FairScheduler documentation. (Ray Chiang via kasha) 2016-02-17 18:36:12 -08:00
Karthik Kambatla 1c248ea4a8 YARN-4689. Fix up CHANGES.txt 2016-02-17 18:18:46 -08:00
Karthik Kambatla 2ab4c476ed YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable. (Kai Sasaki via kasha) 2016-02-17 18:16:15 -08:00
Jian He dea90c9a86 YARN-4653. Document YARN security model from the perspective of Application Developers. Contributed by Steve Loughran 2016-02-14 17:13:15 +08:00
Junping Du ec12ce8f48 Set the release date for 2.6.4. 2016-02-13 01:42:50 -08:00
Steve Loughran 51fc7f5427 YARN-4682. AMRM client to log when AMRM token updated. (Prabhu Joseph via stevel) 2016-02-12 11:02:08 +00:00
Varun Vasudev 2fb423e195 YARN-4684. TestYarnCLI#testGetContainers failing in CN locale. Contributed by Bibin A Chundatt. 2016-02-12 12:06:55 +05:30
Arun Suresh 23f937e3b7 YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 2016-02-11 10:47:43 -08:00
Varun Vasudev fa00d3e205 YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana. 2016-02-11 12:06:42 +05:30
Varun Vasudev 663a80031c YARN-4628. Display application priority in yarn top. Contributed by Bibin A Chundatt. 2016-02-11 11:56:09 +05:30
Jian He 39a71b606a Move YARN-4519 in CHANGES.txt to 2.8 2016-02-11 10:17:10 +08:00
Jian He d16b17b4d2 YARN-4138. Roll back container resource allocation after resource increase token expires. Contributed by Meng Ding 2016-02-11 10:06:27 +08:00
= cb3a10362d YARN-4420. Amend adding modification to CHANGES.txt 2016-02-10 14:22:01 -08:00
= b706cbc1bc YARN-4420. Add REST API for List Reservations (Sean Po via curino) 2016-02-10 10:19:26 -08:00
Arun Suresh 5cf5c41a89 YARN-4360. Improve GreedyReservationAgent to support "early" allocations, and performance improvements (curino via asuresh) 2016-02-10 09:11:15 -08:00
Steve Loughran e9a622606f YARN-4629. Distributed shell breaks under strong security. (Daniel Templeton via stevel) 2016-02-10 13:01:10 +00:00
Jason Lowe fba6e9f5ef Update CHANGES.txt to reflect recent commits to branch-2.7 and branch-2.6. 2016-02-09 16:54:45 +00:00
Devaraj K 565af873d5 YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing
is configured. Contributed by Naganarasimha G R.
2016-02-08 15:01:54 +05:30
Varun Vasudev 22a2b2231d YARN-4669. Fix logging statements in resource manager's Application class. Contributed by Sidharta Seethana. 2016-02-04 13:51:25 +05:30
Varun Vasudev 308d63f382 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 2016-02-04 13:32:54 +05:30
Xuan 63c63e298c YARN-4662. Document some newly added metrics. Contributed by Jian He 2016-02-03 20:05:22 -08:00
Jason Lowe fa328e2d39 YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 2016-02-03 17:21:12 +00:00
Varun Vasudev 1adb64e09b YARN-4625. Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent. Contributed by Xuan Gong. 2016-02-03 16:26:28 +05:30
Junping Du 414cf07650 Adding release 2.6.5 to CHANGES.txt 2016-02-02 18:06:10 -08:00
Devaraj K db144eb1c5 YARN-4100. Add Documentation for Distributed and Delegated-Centralized
Node Labels feature. Contributed by Naganarasimha G R.
2016-02-02 12:06:51 +05:30
Varun Vasudev 1cd55e0c17 YARN-4649. Add additional logging to some NM state store operations. Contributed by Sidharta Seethana 2016-02-02 09:25:23 +05:30
Wangda Tan 9875325d5c YARN-4340. Add list API to reservation system. (Sean Po via wangda) 2016-02-02 10:17:33 +08:00
Jason Lowe ed55950164 YARN-3102. Decommisioned Nodes not listed in Web UI. Contributed by Kuhu Shukla 2016-02-01 23:15:26 +00:00
Rohith Sharma K S 2673cbaf55 YARN-4615. Fix random test failure in TestAbstractYarnScheduler#testResourceRequestRecoveryToTheRightAppAttempt. (Sunil G via rohithsharmaks) 2016-02-01 10:43:56 +05:30
Jason Lowe 772ea7b41b YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is not available. Contributed by Chang Li 2016-01-29 21:48:54 +00:00
Jian He f4a57d4a53 YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy. Contributed by Rohith Sharma K S 2016-01-29 12:22:23 -08:00
Karthik Kambatla c9a09d6926 YARN-4647. Make RegisterNodeManagerRequestPBImpl thread-safe. (kasha) 2016-01-29 08:12:54 -08:00
Devaraj K a277bdc9ed YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws
IllegalArgumentException. Contributed by Bibin A Chundatt and yarntime.
2016-01-29 13:51:37 +05:30
Rohith Sharma K S ac68666803 YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 2016-01-29 12:29:54 +05:30
Rohith Sharma K S 09d831c95b YARN-4219. addendum patch to fix javadoc errors 2016-01-29 11:51:47 +05:30
Jian He 7f46636495 YARN-4519. Potential deadlock of CapacityScheduler between decrease container and assign containers. Contributed by Meng Ding 2016-01-28 14:51:00 -08:00
Xuan 9fab22b366 YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by
Li Lu
2016-01-28 14:24:22 -08:00
Jason Lowe 61382ff8fa YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 2016-01-28 18:59:35 +00:00
Rohith Sharma K S ef343be82b YARN-4633. Fix random test failure in TestRMRestart#testRMRestartAfterPreemption. (Bibin A Chundatt via rohithsharmaks) 2016-01-28 21:53:45 +05:30
Karthik Kambatla fb238d7e5d YARN-4462. FairScheduler: Disallow preemption from a queue. (Tao Jie via kasha) 2016-01-27 12:29:06 -08:00
Rohith Sharma K S c01bee0108 YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks) 2016-01-27 08:23:02 +05:30
Xuan 4efdf3a979 YARN-4612. Fix rumen and scheduler load simulator handle killed tasks
properly. Contributed by Ming Ma.
2016-01-26 18:17:12 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 2085e60a96 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 2016-01-25 16:19:36 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) ec4d2d9f40 Release process for 2.7.2: Set the release date for 2.7.2 2016-01-25 15:45:57 -08:00
Jian He 992dd2f783 YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:14 -08:00
Akira Ajisaka 56a0c17508 YARN-4592. Remove unused GetContainerStatus proto. Contributed by Chang Li. 2016-01-25 15:20:29 +09:00
rohithsharmaks 10dc2c0493 YARN-4613. Fix test failure in TestClientRMService#testGetClusterNodes. (Takashi Ohnishi via rohithsharmaks) 2016-01-24 23:36:15 +05:30
rohithsharmaks 99829eb221 YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority. (Sunil G via rohithsharmaks) 2016-01-23 07:56:57 +05:30
Xuan 618bfd6ac2 YARN-4496. Improve HA ResourceManager Failover detection on the client.
Contributed by Jian He
2016-01-22 18:20:38 -08:00
Jason Lowe 46e5ea81e0 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 2016-01-22 21:55:01 +00:00