Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,432 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2470 Commits

Author SHA1 Message Date
Jason Lowe bd0f5085e3 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c5)
 2016-02-29 15:26:26 +00:00
Rohith Sharma K S 2a1bb6cb67 YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit e0b14f26f5)
 2016-02-29 10:50:23 +08:00
Karthik Kambatla f3b37d8020 YARN-4718. Rename variables in SchedulerNode to reduce ambiguity post YARN-1011. (Inigo Goiri via kasha) 
(cherry picked from commit f9692770a5)
 2016-02-28 10:01:48 -08:00
Jason Lowe 0bd7ba4ea8 YARN-4723. NodesListManager$UnknownNodeId ClassCastException. Contributed by Kuhu Shukla 
(cherry picked from commit 6b0f813e89)
 2016-02-26 20:25:56 +00:00
Ming Ma 1656bcec5f YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 
(cherry picked from commit 7f3139e54d)
 2016-02-26 08:43:14 -08:00
Robert Kanter 872b8d90a6 YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 
(cherry picked from commit d7fdec1e6b)
 2016-02-25 16:40:05 -08:00
Robert Kanter ec48848711 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 
(cherry picked from commit c4d4df8de0)
 2016-02-25 12:48:29 -08:00
Karthik Kambatla 6a75c5af09 YARN-4729. SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE. (kasha) 
(cherry picked from commit c684f2b007)
 2016-02-24 18:34:21 -08:00
Robert Kanter c2098d2470 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 
(cherry picked from commit 954dd57043)
 2016-02-24 15:00:48 -08:00
Li Lu 51162f9629 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via 
gtcarrera9)
 2016-02-24 13:52:11 -08:00
Sangjin Lee 432a2367ce YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 
(cherry picked from commit 553b591ba0)
 2016-02-24 09:30:37 -08:00
Jason Lowe acffe82353 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 
(cherry picked from commit d284e187b8)
 2016-02-23 20:51:01 +00:00
Junping Du e3ce0ffdc3 YARN-3223. Resource update during NM graceful decommission. Contributed by Brook Zhou. 
(cherry picked from commit 9ed17f181d)
 2016-02-23 03:35:47 -08:00
Tsuyoshi Ozawa 4ee55d0322 YARN-4648. Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption. Contributed by Kai Sasaki. 
(cherry picked from commit 0e12114c9c)
 2016-02-23 19:50:40 +09:00
Rohith Sharma K S 544a2ff7c9 YARN-4651. Document movetoqueue option in 'YARN Commands'. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit de09f41147)
 2016-02-23 18:07:57 +08:00
Varun Vasudev 2c218ca8a8 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 
(cherry picked from commit 140cb5d745)
 2016-02-23 12:32:16 +05:30
Zhe Zhang 56b7f2c2c1 HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh. 
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
 2016-02-22 14:14:39 -08:00
Junping Du 7402448387 YARN-4386. refreshNodesGracefully() should send recommission event to active RMNodes only. Contributed by Kuhu Shukla. 
(cherry picked from commit 3fab88540f)
 2016-02-22 07:38:40 -08:00
Akira Ajisaka 34635f9e37 YARN-4707. Remove the extra char (>) from SecureContainer.md. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5e7d4d51f8)
 2016-02-22 17:18:03 +09:00
Tsuyoshi Ozawa 929e869ae0 YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki. 
(cherry picked from commit b68901d7dd)
 2016-02-22 07:55:35 +09:00
Rohith Sharma K S d4203c9aa2 YARN-4654. Yarn node label CLI should parse "=" correctly when trying to remove all labels on a node. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit c1afac3a98)
 2016-02-18 14:17:04 +05:30
Sangjin Lee 23b5c71729 YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee) 
(cherry picked from commit 7de70680fe)
 2016-02-17 21:05:10 -08:00
Karthik Kambatla 25c2597f9c YARN-4569. Remove incorrect part of maxResources in FairScheduler documentation. (Ray Chiang via kasha) 
(cherry picked from commit a0c95b5fc4)
 2016-02-17 18:43:41 -08:00
Karthik Kambatla 33c1988595 YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable. (Kai Sasaki via kasha) 
(cherry picked from commit 2ab4c476ed)
 2016-02-17 18:28:08 -08:00
Jian He 68d98f70ba YARN-4653. Document YARN security model from the perspective of Application Developers. Contributed by Steve Loughran 2016-02-14 17:14:35 +08:00
Steve Loughran 2ea1913cad YARN-4682. AMRM client to log when AMRM token updated. (Prabhu Joseph via stevel) 2016-02-12 10:58:46 +00:00
Varun Vasudev dcead143fe YARN-4684. TestYarnCLI#testGetContainers failing in CN locale. Contributed by Bibin A Chundatt. 
(cherry picked from commit 74e9d17a813baeb4e610eb5fdd8e289660346782)
 2016-02-12 12:08:57 +05:30
Arun Suresh 6a238e4a6b YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 
(cherry picked from commit 23f937e3b7)
 2016-02-11 10:49:11 -08:00
Varun Vasudev 3a5b39e931 YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana. 
(cherry picked from commit fa00d3e205)
 2016-02-11 12:09:29 +05:30
Varun Vasudev 05b57c87f9 YARN-4628. Display application priority in yarn top. Contributed by Bibin A Chundatt. 
(cherry picked from commit 663a80031c)
 2016-02-11 12:00:32 +05:30
Jian He 358cfcf9ea YARN-4138. Roll back container resource allocation after resource increase token expires. Contributed by Meng Ding 2016-02-11 10:09:04 +08:00
= fc9eb4577c YARN-4420. Add REST API for List Reservations. (Sean Po via curino) 2016-02-10 14:26:38 -08:00
Arun Suresh 17f83c8803 YARN-4360. Improve GreedyReservationAgent to support "early" allocations, and performance improvements (curino via asuresh) 
(cherry picked from commit 5cf5c41a89)
 2016-02-10 09:13:04 -08:00
Steve Loughran 179b36efb5 YARN-4629. Distributed shell breaks under strong security. (Daniel Templeton via stevel) 2016-02-10 13:00:48 +00:00
Devaraj K 329df98f49 YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing 
is configured. Contributed by Naganarasimha G R.

(cherry picked from commit 565af873d5)
 2016-02-08 15:03:11 +05:30
Varun Vasudev be04535f1a YARN-4669. Fix logging statements in resource manager's Application class. Contributed by Sidharta Seethana. 
(cherry picked from commit a4a30c789146c4e824fcc6ebcf10405711120eaa)
 2016-02-04 13:51:08 +05:30
Varun Vasudev 99966b1912 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382)
 2016-02-04 13:33:38 +05:30
Jason Lowe b12f16ab73 YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39)
 2016-02-03 17:22:12 +00:00
Varun Vasudev e211ab777e YARN-4625. Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent. Contributed by Xuan Gong. 
(cherry picked from commit 1adb64e09b)
 2016-02-03 16:28:31 +05:30
Devaraj K 43131af1e2 YARN-4100. Add Documentation for Distributed and Delegated-Centralized 
Node Labels feature. Contributed by Naganarasimha G R.

(cherry picked from commit db144eb1c5)
 2016-02-02 12:07:49 +05:30
Varun Vasudev bbda446889 YARN-4649. Add additional logging to some NM state store operations. Contributed by Sidharta Seethana 
(cherry picked from commit 1cd55e0c17)
 2016-02-02 09:26:22 +05:30
Wangda Tan 9591363ad8 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c)
 2016-02-02 10:19:19 +08:00
Jason Lowe 36aae8050e YARN-3102. Decommisioned Nodes not listed in Web UI. Contributed by Kuhu Shukla 
(cherry picked from commit ed55950164)
 2016-02-01 23:16:18 +00:00
Rohith Sharma K S 676f215595 YARN-4615. Fix random test failure in TestAbstractYarnScheduler#testResourceRequestRecoveryToTheRightAppAttempt. (Sunil G via rohithsharmaks) 
(cherry picked from commit 2673cbaf55)
 2016-02-01 10:45:05 +05:30
Jason Lowe 016c685135 YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is not available. Contributed by Chang Li 
(cherry picked from commit 772ea7b41b)
 2016-01-29 21:54:56 +00:00
Jian He 885c76d85f YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy. Contributed by Rohith Sharma K S 2016-01-29 12:22:44 -08:00
Karthik Kambatla 9d1371d390 YARN-4647. Make RegisterNodeManagerRequestPBImpl thread-safe. (kasha) 
(cherry picked from commit c9a09d6926)
 2016-01-29 08:19:34 -08:00
Devaraj K 123269c07f YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws 
IllegalArgumentException. Contributed by Bibin A Chundatt and yarntime.

(cherry picked from commit a277bdc9ed)
 2016-01-29 13:52:47 +05:30
Rohith Sharma K S b8d5df06f2 YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 
(cherry picked from commit ac68666803)
 2016-01-29 12:30:53 +05:30
Rohith Sharma K S 0466743afb YARN-4219. addendum patch to fix javadoc errors 
(cherry picked from commit 09d831c95b)
 2016-01-29 11:53:06 +05:30
Jian He 36f0e73882 YARN-4519. Potential deadlock of CapacityScheduler between decrease container and assign containers. Contributed by Meng Ding 2016-01-28 14:52:00 -08:00
Xuan 2f00d8d3a7 YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by 
Li Lu

(cherry picked from commit 9fab22b366)
 2016-01-28 14:36:58 -08:00
Jason Lowe 29195c7668 YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 
(cherry picked from commit 61382ff8fa)
 2016-01-28 19:00:51 +00:00
Rohith Sharma K S 6947a5d4be YARN-4633. Fix random test failure in TestRMRestart#testRMRestartAfterPreemption. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit ef343be82b)
 2016-01-28 21:55:12 +05:30
Karthik Kambatla c25154576b YARN-4462. FairScheduler: Disallow preemption from a queue. (Tao Jie via kasha) 
(cherry picked from commit fb238d7e5d)
 2016-01-27 12:29:25 -08:00
Rohith Sharma K S 0b8767e01a YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit c01bee0108)
 2016-01-27 08:24:37 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 47b20d0c74 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 
(cherry picked from commit 2085e60a96)
 2016-01-25 16:22:47 -08:00
Jian He 2d236c9de2 YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:40 -08:00
Akira Ajisaka 02d4a1fce3 YARN-4592. Remove unused GetContainerStatus proto. Contributed by Chang Li. 
(cherry picked from commit 56a0c17508)
 2016-01-25 15:20:45 +09:00
rohithsharmaks 7e66b2f22c YARN-4613. Fix test failure in TestClientRMService#testGetClusterNodes. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit 10dc2c0493)
 2016-01-24 23:37:11 +05:30
rohithsharmaks 280c9b8401 YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority. (Sunil G via rohithsharmaks) 
(cherry picked from commit 99829eb221)
 2016-01-23 07:58:16 +05:30
Xuan 1d19557dda YARN-4496. Improve HA ResourceManager Failover detection on the client. 
Contributed by Jian He

(cherry picked from commit 618bfd6ac2)
 2016-01-22 18:21:50 -08:00
Jason Lowe 635849fa45 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0)
 2016-01-22 21:55:59 +00:00
Jason Lowe ad84f90e88 YARN-4371. "yarn application -kill" should take multiple application ids. Contributed by Sunil G 
(cherry picked from commit e91e8b711c)
 2016-01-22 21:37:08 +00:00
rohithsharmaks 1cec12d95a YARN-4574. Fix random failure in TestAMRMClientOnRMRestart. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit 34a3900773)
 2016-01-22 21:18:42 +05:30
rohithsharmaks 493275b27b YARN-4497. RM might fail to restart when recovering apps whose attempts are missing. (Jun Gong via rohithsharmaks) 
(cherry picked from commit d6258b33a7)
 2016-01-22 20:29:10 +05:30
Akira Ajisaka ddc22bbcb1 YARN-4605. Spelling mistake in the help message of "yarn applicationattempt" command. Contributed by Weiwei Yang. 
(cherry picked from commit 8f58f742ae)
 2016-01-22 19:43:37 +09:00
Varun Vasudev 85561207f9 YARN-4578. Directories that are mounted in docker containers need to be more restrictive/container-specific. Contributed by Sidharta Seethana. 
(cherry picked from commit b41a7e89d1)
 2016-01-22 14:44:22 +05:30
Rohith Sharma K S 844a8e3771 YARN-4584. RM startup failure when AM attempts greater than max-attempts. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit e30668106d)
 2016-01-22 10:20:09 +05:30
Jason Lowe d051200a62 YARN-4610. Reservations continue looking for one app causes other apps to starve. Contributed by Jason Lowe 
(cherry picked from commit 468a53b22f)
 2016-01-21 18:32:47 +00:00
Akira Ajisaka bf4901c609 YARN-4608. Redundant code statement in WritingYarnApplications. Contributed by Kai Sasaki. 
(cherry picked from commit d3224e2d6274daf73a103c0dbe21ba69add7ca26)
 2016-01-22 02:14:53 +09:00
Karthik Kambatla 5c2f495fcc YARN-4603. FairScheduler should mention user requested queuename in error message when failed in queue ACL check. (Tao Jie via kasha) 
(cherry picked from commit 4992398aee)
 2016-01-21 17:43:49 +01:00
Jason Lowe 4e6fc3ca18 YARN-4492. Add documentation for preemption supported in Capacity scheduler. Contributed by Naganarasimha G R 
(cherry picked from commit f858ac2426)
 2016-01-21 15:39:51 +00:00
Wangda Tan 9512866740 YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=*. (Naganarasimha G R via wangda) 
(cherry picked from commit c9fe0371a8ab40c0d3437fadd32788b320788a7a)
 2016-01-21 11:16:58 +08:00
Xuan be3322792f YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1a)
 2016-01-20 14:49:04 -08:00
Wangda Tan 23deeb4230 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)
 2016-01-19 21:28:51 +08:00
Jian He bb2a9a78c0 YARN-4565. Fix a bug that leads to AM resource limit not hornored when sizeBasedWeight enabled for FairOrderingPolicy. Contributed by Wangda Tan 2016-01-18 21:05:03 -08:00
Wangda Tan 704b37d0e1 YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit a44ce3f14f)
 2016-01-19 09:31:13 +08:00
Wangda Tan 1625281ccb Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe5728563.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 150f5ae034)
 2016-01-19 09:31:07 +08:00
Jian He 6db022fc48 YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu 2016-01-18 16:59:04 -08:00
rohithsharmaks da887e4252 YARN-4265 Addendum patch to fix compilation issue caused by project version number 2016-01-18 16:38:41 +05:30
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Wangda Tan 552f15739c YARN-4304. AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics. (Sunil G via wangda) 
(cherry picked from commit b08ecf5c75)
 2016-01-18 11:16:04 +08:00
Wangda Tan 6dab78c71a YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85e)
 2016-01-18 11:15:59 +08:00
Wangda Tan c1193b46d4 YARN-4538. QueueMetrics pending cores and memory metrics wrong. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9523648d57)
 2016-01-18 11:15:53 +08:00
Junping Du 4a30a44b11 YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 
(cherry picked from commit 02f597c5db)
 2016-01-17 17:38:42 -08:00
Colin Patrick Mccabe 48b8c99479 HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe) 
(cherry picked from commit b2c155f810)
 2016-01-15 13:22:14 -08:00
rohithsharmaks d3c08cde68 YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks) 2016-01-15 22:46:21 +05:30
Junping Du e09222306f YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee. 
(cherry picked from commit fc6d3a3b23)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-15 08:43:30 -08:00
Colin Patrick Mccabe cea1c71087 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 
(cherry picked from commit b1ed28fa77)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
 2016-01-14 11:13:13 -08:00
Karthik Kambatla f0923819c3 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c)
 2016-01-14 08:33:52 -08:00
Varun Vasudev 6220a024b1 YARN-4553. Add cgroups support for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 3ddb92bd30)
 2016-01-14 14:30:19 +05:30
Akira Ajisaka ffca0ebfff YARN-4534. Remove the redundant symbol in yarn rmadmin help msg. Contributed by Lin Yiqun. 
(cherry picked from commit 62b0d61513)
 2016-01-14 13:59:58 +09:00
Karthik Kambatla a1ff0b4145 YARN-4551. Address the duplication between StatusUpdateWhenHealthy and StatusUpdateWhenUnhealthy transitions. (Sunil G via kasha) 
(cherry picked from commit 321072ba81)
 2016-01-13 12:09:58 -08:00
Wangda Tan c0b97865e8 YARN-4571. Make app id/name available to the yarn authorizer provider for better auditing. (Jian He via wangda) 
(cherry picked from commit c0537bcd2c)
 2016-01-13 13:19:01 +08:00
Jason Lowe d164cdcda1 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1)
 2016-01-12 15:57:07 +00:00
Akira Ajisaka 93b7ef8aec YARN-4567. javadoc failing on java 8. Contributed by Steve Loughran. This closes #67. 
(cherry picked from commit da1e3e3c57)
 2016-01-12 15:13:38 +09:00
Wangda Tan 15e174fe79 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da014)
 2016-01-12 12:54:09 +08:00
Akira Ajisaka df95343047 HADOOP-12584. Disable browsing the static directory in HttpServer2. Contributed by Robert Kanter. 
(cherry picked from commit 9c89bcd04212543ae279d34938ec2ad319e5ba6d)
 2016-01-12 13:34:44 +09:00
Jian He c4aed35200 Missing file for YARN-4580 2016-01-11 17:05:11 -08:00
Jian He 591e7fb221 YARN-4537. Pull out priority comparison from fifocomparator and use compound comparator for FifoOrdering policy. Contributed by Rohith Sharma K S 
(cherry picked from commit b8942be888)
 2016-01-11 16:45:14 -08:00
Jian He d4ff8fd5d8 YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps. Contributed by Rohith Sharma K S 
(cherry picked from commit 109e528ef5)
 2016-01-08 15:51:33 -08:00
Xuan 2cbbf76c3d YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4b)
 2016-01-07 14:34:14 -08:00
Akira Ajisaka 3905435d3d YARN-4544. All the log messages about rolling monitoring interval are shown with WARN level. Contributed by Takashi Ohnishi. 
(cherry picked from commit 52b77577c4)
 2016-01-08 01:13:30 +09:00
Steve Loughran d9459ffa00 YARN-4550. Some tests in TestContainerLanch fails on non-english locale environment. (Takashi Ohnishi via stevel) 2016-01-07 14:30:07 +00:00
rohithsharmaks 4865642e7c YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 
(cherry picked from commit 791c1639ae)
 2016-01-07 09:39:37 +05:30
Junping Du 1cc001db4c YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe. 
(cherry picked from commit c1462a67ff)
 2016-01-06 05:51:28 -08:00
rohithsharmaks 0492552f30 YARN-4535. Fix checkstyle error in CapacityScheduler.java (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 6da6d87872)
 2016-01-05 12:11:48 +05:30
Rohith Sharma K S ae25e283cf YARN-4530. LocalizedResource trigger a NPE Cause the NodeManager exit. (tangshangwen via rohithsharmaks) 
(cherry picked from commit f9e36dea96)
 2016-01-04 10:04:27 +05:30
Wangda Tan 05fa852d75 YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda) 2015-12-30 15:36:55 -08:00
Wangda Tan 9a7d5f69ec YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda) 
(cherry picked from commit 8310b2e9ff)
 2015-12-30 15:31:11 -08:00
Junping Du 401d1e6e8a YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager to get rid of memory leak. Contributed by Rohith Sharma K S. 
(cherry picked from commit 223ce323bb)
 2015-12-30 07:53:08 -08:00
Jian He 970bb2da15 YARN-4098. Document ApplicationPriority feature. Contributed by Rohith Sharma K S 2015-12-29 16:52:14 -08:00
Jian He 3661d4907a YARN-3480. Remove attempts that are beyond max-attempt limit from state store. Contributed by Jun Gong 
(cherry picked from commit 5273413411)
 2015-12-29 15:59:42 -08:00
Wangda Tan 3efbde5378 YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda) 
(cherry picked from commit 561abb9fee)
 2015-12-29 13:29:19 -08:00
Jian He c7cc9d6bac YARN-4417. Make RM and Timeline-server REST APIs more consistent. Contributed by Wangda Tan 
(cherry picked from commit d0a22bae9b)
 2015-12-28 15:54:14 -08:00
Gera Shegalov 8baf9369bb YARN-2934. Improve handling of container's stderr. (Naganarasimha G R via gera) 
(cherry picked from commit 2c17b81569)
 2015-12-24 23:52:43 -08:00
Karthik Kambatla 2f67645475 YARN-4156. TestAMRestart#testAMBlacklistPreventsRestartOnSameNode assumes CapacityScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit 0af492b4bd)
 2015-12-23 17:52:54 -08:00
Junping Du dbc4c79b44 YARN-4400. AsyncDispatcher.waitForDrained should be final. Contributed by Daniel Templeton. 
(cherry picked from commit bb5df272b9)
 2015-12-23 08:55:55 -08:00
Junping Du 6e97a3c968 YARN-4234. New put APIs in TimelineClient for ats v1.5. Contributed by Xuan Gong. 
(cherry picked from commit 882f2f0464)
 2015-12-23 05:27:47 -08:00
rohithsharmaks 08ddb536eb YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit 8c180a13c8)
 2015-12-23 09:16:58 +05:30
Arun Suresh 31893468b1 YARN-4477. FairScheduler: Handle condition which can result in an infinite loop in attemptScheduling. (Tao Jie via asuresh) 
(cherry picked from commit e88422df45)
 2015-12-21 22:41:49 -08:00
cnauroth fdfcffb3d1 YARN-3458. CPU resource monitoring in Windows. Contributed by Inigo Goiri. 
(cherry picked from commit 114b590955)
 2015-12-21 16:59:17 -08:00
Wangda Tan daf5fae369 YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports information. (Sunil G via wangda) 
(cherry picked from commit a0de702851)
 2015-12-21 11:36:49 -08:00
Wangda Tan f16daa59a4 YARN-4454. NM to nodelabel mapping going wrong after RM restart. (Bibin A Chundatt via wangda) 
(cherry picked from commit bc038b382c)
 2015-12-21 11:30:37 -08:00
Uma Mahesh fc9e130825 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 
(cherry picked from commit 0f82b5d878)
 2015-12-19 23:17:33 -08:00
Jian He 1147b086d9 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2015-12-18 14:14:31 -08:00
Junping Du 6d2914a697 YARN-3226. UI changes for decommissioning node. Contributed by Sunil G. 
(cherry picked from commit 1de56b0448)
 2015-12-17 15:21:42 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) d9b9a75928 YARN-1856. Added cgroups based memory monitoring for containers as another alternative to custom memory-monitoring. Contributed by Varun Vasudev. 
(cherry picked from commit 4e7d32c0db)
 2015-12-17 12:13:56 -08:00
Jason Lowe fad2c5b862 YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne 
(cherry picked from commit 91828fef6b)
 2015-12-16 23:23:19 +00:00
Wangda Tan 5060fd1840 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda) 
(cherry picked from commit 9b856d9787)
 2015-12-16 13:29:08 -08:00
Wangda Tan 2f5435acbd YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 
(cherry picked from commit 7faa406f27)
 2015-12-16 13:29:02 -08:00
Wangda Tan 9f4a6ec663 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83)
 2015-12-16 13:28:56 -08:00
Siddharth Seth 433212ef87 YARN-4207. Add a non-judgemental YARN app completion status. Contributed by Rich Haase. 
(cherry picked from commit 0f708d465f)
 2015-12-16 11:35:38 -08:00
Junping Du 1a2ef845b5 YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d)
 2015-12-16 10:58:40 -08:00
Zhihai Xu be50df687b YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun 
(cherry picked from commit 2aaed10327)
 2015-12-15 00:21:13 -08:00
Xuan 992f65dd8b YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He

(cherry picked from commit d8a45425eb)
 2015-12-14 15:42:07 -08:00
Jian He 72544db3cd YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 14:59:18 -08:00
Jian He 34e86d8d47 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:52:04 -08:00
Wangda Tan c915a658c8 YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 
(cherry picked from commit 07b0fb996a)
 2015-12-14 11:25:00 -08:00
Wangda Tan 9f979107c2 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb09)
 2015-12-14 11:13:51 -08:00
Wangda Tan a1a723fdff YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 
(cherry picked from commit 6cb0af3c39)
 2015-12-14 10:54:20 -08:00
Arun Suresh 0bf328984a YARN-4358 addendum patch to fix javadoc error 
(cherry picked from commit 7fb212e5e6)
 2015-12-12 22:25:59 -08:00
Sangjin Lee 8be667208f YARN-4341. add doc about timeline performance tool usage (Chang Li via sjlee) 
(cherry picked from commit e8964589de)
 2015-12-11 09:53:17 -08:00
Xuan 455c6f39c5 YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml 
(cherry picked from commit 21daa6c68a)
 2015-12-10 10:57:35 -08:00
Junping Du 0b5e76e9ff YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2015-12-10 02:57:46 -08:00
Akira Ajisaka 6451313560 YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. 
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
 2015-12-09 16:32:52 +09:00
rohithsharmaks 81f8a2b008 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks) 
(cherry picked from commit a5e2e1ecb0)
 2015-12-09 11:33:49 +05:30
rohithsharmaks f626d18cc3 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 
(cherry picked from commit 15c3e7ffe3)
 2015-12-09 10:51:57 +05:30
Wangda Tan 7013f9d6cd YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-08 14:30:01 -08:00
Chris Douglas bfe4796802 YARN-4248. Followup patch adding asf-licence exclusions for json test files 
(cherry picked from commit 9f50e13d5d)
 2015-12-08 12:08:35 -08:00
= 4ac1564418 YARN-4248. REST API for submit/update/delete Reservations. (curino) 
(cherry picked from commit c25a635459)
 2015-12-07 13:38:38 -08:00
Jonathan Eagles 7c604c3d06 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 
(cherry picked from commit 4ff973f96a)
 2015-12-07 15:05:15 -06:00
Xuan 7decba83ef YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)
 2015-12-07 12:26:09 -08:00
rohithsharmaks 04e056f4f5 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks) 
(cherry picked from commit 01a641bc44)
 2015-12-07 15:19:59 +05:30
Steve Loughran af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Arun Suresh 78a07e99dd YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 
(cherry picked from commit 742632e346)
 2015-12-05 21:30:48 -08:00
Jian He 8a97ca4b00 Add missing file for YARN-4419 2015-12-03 19:11:07 -08:00
Jian He be404e1680 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 
(cherry picked from commit 755dda8dd8)
 2015-12-03 17:46:09 -08:00
Wangda Tan 51e448d295 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 
(cherry picked from commit a2c3bfc8c1)
 2015-12-03 14:30:25 -08:00
Jian He 23fe1abc02 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)
 2015-12-03 12:49:22 -08:00
Junping Du e76ba91fa5 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 
(cherry picked from commit 62e9348bc1)
 2015-12-03 06:37:31 -08:00
Jian He 48cbde9083 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 2015-12-02 11:08:00 -08:00
Allen Wittenauer 45bb467918 HADOOP-12366. expose calculated paths (aw) 2015-12-01 08:16:20 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Tsuyoshi Ozawa d76b523b02 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83)
 2015-11-26 01:10:41 +09:00
Wangda Tan 7b9f7eaf5c YARN-4384. updateNodeResource CLI should not accept negative values for resource. (Junping Du via wangda) 
(cherry picked from commit 23c625ec57)
 2015-11-24 16:36:30 -08:00
Jason Lowe 3a2f5d6329 YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 
(cherry picked from commit f5acf94eca)
 2015-11-24 23:47:54 +00:00
Jason Lowe 9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 
(cherry picked from commit 4ac6799d4a)
 2015-11-24 22:36:19 +00:00
Tsuyoshi Ozawa 9e458c3e9e YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 
(cherry picked from commit 28dfe721b8)
 2015-11-24 22:10:31 +09:00
Akira Ajisaka ac0ddc4fec YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G. 
(cherry picked from commit 8c0133f3d4)
 2015-11-24 18:40:56 +09:00
Karthik Kambatla 49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20b)
 2015-11-24 13:52:29 +05:30
Jian He b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a1)
 2015-11-23 17:20:24 -08:00
Jason Lowe 9d303edbfb YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 
(cherry picked from commit d36b6e045f)
 2015-11-23 20:31:50 +00:00
Arun Suresh cbbdbe3cdd YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 
(cherry picked from commit da1016365a)
 2015-11-21 10:01:10 -08:00
Wangda Tan cdca460ad4 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 
(cherry picked from commit 2346fa3141)
 2015-11-20 15:56:17 -08:00
Jason Lowe f3bca8bc1a YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 
(cherry picked from commit 060cdcbe5d)
 2015-11-20 23:13:27 +00:00
Xuan be65d319fa YARN-2859.addendum: fix the remaining issue from the previous patch 
(cherry picked from commit f114e728da)
 2015-11-19 10:09:48 -08:00
Steve Loughran a0b3e43a6d YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:24 +00:00
Arun Suresh d8e60080e9 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 
(cherry picked from commit 6a61928fb7)
 2015-11-17 15:51:59 -08:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Junping Du 1d107d8058 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b)
 2015-11-15 04:44:59 -08:00
Wangda Tan 875aec3177 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 
(cherry picked from commit 7ff280fca9)
 2015-11-12 11:50:41 -08:00
Wangda Tan 13f8c7093b YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 
(cherry picked from commit 7f55a18071)
 2015-11-12 11:24:04 -08:00
Wangda Tan 0c1fb15b29 YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 
(cherry picked from commit 796638d9bc)
 2015-11-12 11:12:11 -08:00
Jason Lowe 9050078005 YARN-4345. yarn rmadmin -updateNodeResource doesn't work. Contributed by Junping Du 
(cherry picked from commit 06a70311b9)
 2015-11-12 14:49:55 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) bb5dd7f772 YARN-4183. Reverting the patch to fix behaviour change. 
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit 42bc525acb.
 2015-11-11 10:42:52 -08:00
Akira Ajisaka f6f128c867 YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. 
(cherry picked from commit 23d0db551c)
 2015-11-11 16:13:50 +09:00
Wangda Tan 8a8ac50a5b YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda) 
(cherry picked from commit c99925d6dd)
 2015-11-10 13:17:23 -08:00
Vinod Kumar Vavilapalli b742ea7fe6 HADOOP-12451. Setting HADOOP_HOME explicitly should be allowed. Fixing bugs in 
the first patch committed. (Karthik Kambatla via vinodkv)
 2015-11-10 12:16:52 -08:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
 2015-11-09 11:05:12 -08:00
Wangda Tan 3190446df9 YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 
(cherry picked from commit 0783184f4b)
 2015-11-03 11:36:11 -08:00
Tsuyoshi Ozawa 1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739)
 2015-10-30 18:00:35 +09:00
Jian He 2b197bbffd YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 
(cherry picked from commit e5b1733e04)
 2015-10-29 15:43:26 -07:00
Jonathan Eagles 42bc525acb YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 
(cherry picked from commit c293c58954)
 2015-10-29 16:42:49 -05:00
Arun Suresh 8943abfb08 YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 
(cherry picked from commit 58d1df585c)
 2015-10-29 13:43:43 -07:00
Jason Lowe c75d8b164f YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 
(cherry picked from commit e2267de207)
 2015-10-29 16:39:55 +00:00
Jian He 7d109ef5b5 YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du 
(cherry picked from commit c41699965e)
 2015-10-29 00:01:20 -07:00
Rohith Sharma K S e3fcb711d6 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 
(cherry picked from commit 656c8f9527)
 2015-10-29 12:25:08 +05:30
Xuan 9ce5069d16 YARN-2859. ApplicationHistoryServer binds to default port 8188 in 
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 27414dac66)
 2015-10-28 10:33:56 -07:00