Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,796 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2798 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 47b20d0c74 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 
(cherry picked from commit 2085e60a96)
 2016-01-25 16:22:47 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 97f6cfa94b Release process for 2.7.2: Set the release date for 2.7.2 
(cherry picked from commit ec4d2d9f40)
 2016-01-25 15:47:02 -08:00
Jian He 2d236c9de2 YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:40 -08:00
Akira Ajisaka 02d4a1fce3 YARN-4592. Remove unused GetContainerStatus proto. Contributed by Chang Li. 
(cherry picked from commit 56a0c17508)
 2016-01-25 15:20:45 +09:00
rohithsharmaks 7e66b2f22c YARN-4613. Fix test failure in TestClientRMService#testGetClusterNodes. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit 10dc2c0493)
 2016-01-24 23:37:11 +05:30
rohithsharmaks 280c9b8401 YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority. (Sunil G via rohithsharmaks) 
(cherry picked from commit 99829eb221)
 2016-01-23 07:58:16 +05:30
Xuan 1d19557dda YARN-4496. Improve HA ResourceManager Failover detection on the client. 
Contributed by Jian He

(cherry picked from commit 618bfd6ac2)
 2016-01-22 18:21:50 -08:00
Jason Lowe 635849fa45 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0)
 2016-01-22 21:55:59 +00:00
Jason Lowe ad84f90e88 YARN-4371. "yarn application -kill" should take multiple application ids. Contributed by Sunil G 
(cherry picked from commit e91e8b711c)
 2016-01-22 21:37:08 +00:00
rohithsharmaks 1cec12d95a YARN-4574. Fix random failure in TestAMRMClientOnRMRestart. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit 34a3900773)
 2016-01-22 21:18:42 +05:30
rohithsharmaks 493275b27b YARN-4497. RM might fail to restart when recovering apps whose attempts are missing. (Jun Gong via rohithsharmaks) 
(cherry picked from commit d6258b33a7)
 2016-01-22 20:29:10 +05:30
Akira Ajisaka ddc22bbcb1 YARN-4605. Spelling mistake in the help message of "yarn applicationattempt" command. Contributed by Weiwei Yang. 
(cherry picked from commit 8f58f742ae)
 2016-01-22 19:43:37 +09:00
Varun Vasudev 85561207f9 YARN-4578. Directories that are mounted in docker containers need to be more restrictive/container-specific. Contributed by Sidharta Seethana. 
(cherry picked from commit b41a7e89d1)
 2016-01-22 14:44:22 +05:30
Rohith Sharma K S 844a8e3771 YARN-4584. RM startup failure when AM attempts greater than max-attempts. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit e30668106d)
 2016-01-22 10:20:09 +05:30
Jason Lowe d051200a62 YARN-4610. Reservations continue looking for one app causes other apps to starve. Contributed by Jason Lowe 
(cherry picked from commit 468a53b22f)
 2016-01-21 18:32:47 +00:00
Akira Ajisaka bf4901c609 YARN-4608. Redundant code statement in WritingYarnApplications. Contributed by Kai Sasaki. 
(cherry picked from commit d3224e2d6274daf73a103c0dbe21ba69add7ca26)
 2016-01-22 02:14:53 +09:00
Karthik Kambatla 5c2f495fcc YARN-4603. FairScheduler should mention user requested queuename in error message when failed in queue ACL check. (Tao Jie via kasha) 
(cherry picked from commit 4992398aee)
 2016-01-21 17:43:49 +01:00
Jason Lowe 4e6fc3ca18 YARN-4492. Add documentation for preemption supported in Capacity scheduler. Contributed by Naganarasimha G R 
(cherry picked from commit f858ac2426)
 2016-01-21 15:39:51 +00:00
Wangda Tan 9512866740 YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=*. (Naganarasimha G R via wangda) 
(cherry picked from commit c9fe0371a8ab40c0d3437fadd32788b320788a7a)
 2016-01-21 11:16:58 +08:00
Xuan 66b030f608 YARN-4611. Fix scheduler load simulator to support multi-layer network 
location. Contributed by Ming Ma.

(cherry picked from commit 1708a4cd23)
 2016-01-20 15:43:02 -08:00
Xuan be3322792f YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1a)
 2016-01-20 14:49:04 -08:00
Wangda Tan 23deeb4230 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)
 2016-01-19 21:28:51 +08:00
Jian He bb2a9a78c0 YARN-4565. Fix a bug that leads to AM resource limit not hornored when sizeBasedWeight enabled for FairOrderingPolicy. Contributed by Wangda Tan 2016-01-18 21:05:03 -08:00
Wangda Tan 704b37d0e1 YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit a44ce3f14f)
 2016-01-19 09:31:13 +08:00
Wangda Tan 1625281ccb Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe5728563.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 150f5ae034)
 2016-01-19 09:31:07 +08:00
Jian He 6db022fc48 YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu 2016-01-18 16:59:04 -08:00
rohithsharmaks da887e4252 YARN-4265 Addendum patch to fix compilation issue caused by project version number 2016-01-18 16:38:41 +05:30
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Wangda Tan c56d46307a Move YARN-4524 from 2.9 to 2.8 2016-01-18 17:04:41 +08:00
Wangda Tan 552f15739c YARN-4304. AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics. (Sunil G via wangda) 
(cherry picked from commit b08ecf5c75)
 2016-01-18 11:16:04 +08:00
Wangda Tan 6dab78c71a YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85e)
 2016-01-18 11:15:59 +08:00
Wangda Tan c1193b46d4 YARN-4538. QueueMetrics pending cores and memory metrics wrong. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9523648d57)
 2016-01-18 11:15:53 +08:00
Junping Du 4a30a44b11 YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 
(cherry picked from commit 02f597c5db)
 2016-01-17 17:38:42 -08:00
Colin Patrick Mccabe 48b8c99479 HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe) 
(cherry picked from commit b2c155f810)
 2016-01-15 13:22:14 -08:00
rohithsharmaks d3c08cde68 YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks) 2016-01-15 22:46:21 +05:30
Junping Du e09222306f YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee. 
(cherry picked from commit fc6d3a3b23)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-15 08:43:30 -08:00
Colin Patrick Mccabe cea1c71087 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 
(cherry picked from commit b1ed28fa77)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
 2016-01-14 11:13:13 -08:00
Karthik Kambatla f0923819c3 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c)
 2016-01-14 08:33:52 -08:00
Varun Vasudev 6220a024b1 YARN-4553. Add cgroups support for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 3ddb92bd30)
 2016-01-14 14:30:19 +05:30
Akira Ajisaka ffca0ebfff YARN-4534. Remove the redundant symbol in yarn rmadmin help msg. Contributed by Lin Yiqun. 
(cherry picked from commit 62b0d61513)
 2016-01-14 13:59:58 +09:00
Karthik Kambatla a1ff0b4145 YARN-4551. Address the duplication between StatusUpdateWhenHealthy and StatusUpdateWhenUnhealthy transitions. (Sunil G via kasha) 
(cherry picked from commit 321072ba81)
 2016-01-13 12:09:58 -08:00
Wangda Tan c0b97865e8 YARN-4571. Make app id/name available to the yarn authorizer provider for better auditing. (Jian He via wangda) 
(cherry picked from commit c0537bcd2c)
 2016-01-13 13:19:01 +08:00
Jason Lowe d164cdcda1 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1)
 2016-01-12 15:57:07 +00:00
Jason Lowe 8d34d899e4 Update CHANGES.txt for recent commits to 2.7 and 2.6 
(cherry picked from commit c77c35f83c)
 2016-01-12 15:45:21 +00:00
Akira Ajisaka 93b7ef8aec YARN-4567. javadoc failing on java 8. Contributed by Steve Loughran. This closes #67. 
(cherry picked from commit da1e3e3c57)
 2016-01-12 15:13:38 +09:00
Wangda Tan 15e174fe79 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da014)
 2016-01-12 12:54:09 +08:00
Akira Ajisaka df95343047 HADOOP-12584. Disable browsing the static directory in HttpServer2. Contributed by Robert Kanter. 
(cherry picked from commit 9c89bcd04212543ae279d34938ec2ad319e5ba6d)
 2016-01-12 13:34:44 +09:00
Jian He c4aed35200 Missing file for YARN-4580 2016-01-11 17:05:11 -08:00
Jian He 591e7fb221 YARN-4537. Pull out priority comparison from fifocomparator and use compound comparator for FifoOrdering policy. Contributed by Rohith Sharma K S 
(cherry picked from commit b8942be888)
 2016-01-11 16:45:14 -08:00
Rohith Sharma K S da42b01645 Add YARN-3849 to Release 2.6.4 entry in CHANGES.txt 
(cherry picked from commit 76e7270851)
 2016-01-11 12:18:12 +05:30
Jian He d4ff8fd5d8 YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps. Contributed by Rohith Sharma K S 
(cherry picked from commit 109e528ef5)
 2016-01-08 15:51:33 -08:00
Xuan 2cbbf76c3d YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4b)
 2016-01-07 14:34:14 -08:00
Akira Ajisaka 3905435d3d YARN-4544. All the log messages about rolling monitoring interval are shown with WARN level. Contributed by Takashi Ohnishi. 
(cherry picked from commit 52b77577c4)
 2016-01-08 01:13:30 +09:00
Steve Loughran d9459ffa00 YARN-4550. Some tests in TestContainerLanch fails on non-english locale environment. (Takashi Ohnishi via stevel) 2016-01-07 14:30:07 +00:00
rohithsharmaks 4865642e7c YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 
(cherry picked from commit 791c1639ae)
 2016-01-07 09:39:37 +05:30
Junping Du 818cd0e90d Add YARN-2975, YARN-3893, YARN-2902 and YARN-4354 to Release 2.6.4 entry in CHANGES.txt 
(cherry picked from commit b6c9d3fab9)
 2016-01-06 14:28:43 -08:00
Junping Du 1cc001db4c YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe. 
(cherry picked from commit c1462a67ff)
 2016-01-06 05:51:28 -08:00
Zhihai Xu f41db81d5f update CHANGES.txt to add YARN-3697 to 2.6.4 release 
(cherry picked from commit 96d8f1d6f3)
 2016-01-05 00:42:35 -08:00
rohithsharmaks 0492552f30 YARN-4535. Fix checkstyle error in CapacityScheduler.java (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 6da6d87872)
 2016-01-05 12:11:48 +05:30
Akira Ajisaka 247ab92464 Set release date for 2.6.3 
(cherry picked from commit db99e30f67)
 2016-01-04 16:12:52 +09:00
Rohith Sharma K S ae25e283cf YARN-4530. LocalizedResource trigger a NPE Cause the NodeManager exit. (tangshangwen via rohithsharmaks) 
(cherry picked from commit f9e36dea96)
 2016-01-04 10:04:27 +05:30
Wangda Tan 05fa852d75 YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda) 2015-12-30 15:36:55 -08:00
Wangda Tan 6eefae1b33 YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda) 2015-12-30 15:34:39 -08:00
Wangda Tan 9a7d5f69ec YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda) 
(cherry picked from commit 8310b2e9ff)
 2015-12-30 15:31:11 -08:00
Junping Du 401d1e6e8a YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager to get rid of memory leak. Contributed by Rohith Sharma K S. 
(cherry picked from commit 223ce323bb)
 2015-12-30 07:53:08 -08:00
Jian He 970bb2da15 YARN-4098. Document ApplicationPriority feature. Contributed by Rohith Sharma K S 2015-12-29 16:52:14 -08:00
Jian He 3661d4907a YARN-3480. Remove attempts that are beyond max-attempt limit from state store. Contributed by Jun Gong 
(cherry picked from commit 5273413411)
 2015-12-29 15:59:42 -08:00
Wangda Tan 3efbde5378 YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda) 
(cherry picked from commit 561abb9fee)
 2015-12-29 13:29:19 -08:00
Jian He c7cc9d6bac YARN-4417. Make RM and Timeline-server REST APIs more consistent. Contributed by Wangda Tan 
(cherry picked from commit d0a22bae9b)
 2015-12-28 15:54:14 -08:00
Gera Shegalov 8baf9369bb YARN-2934. Improve handling of container's stderr. (Naganarasimha G R via gera) 
(cherry picked from commit 2c17b81569)
 2015-12-24 23:52:43 -08:00
Karthik Kambatla 2f67645475 YARN-4156. TestAMRestart#testAMBlacklistPreventsRestartOnSameNode assumes CapacityScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit 0af492b4bd)
 2015-12-23 17:52:54 -08:00
Junping Du dbc4c79b44 YARN-4400. AsyncDispatcher.waitForDrained should be final. Contributed by Daniel Templeton. 
(cherry picked from commit bb5df272b9)
 2015-12-23 08:55:55 -08:00
Junping Du 6e97a3c968 YARN-4234. New put APIs in TimelineClient for ats v1.5. Contributed by Xuan Gong. 
(cherry picked from commit 882f2f0464)
 2015-12-23 05:27:47 -08:00
rohithsharmaks 08ddb536eb YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit 8c180a13c8)
 2015-12-23 09:16:58 +05:30
Arun Suresh 31893468b1 YARN-4477. FairScheduler: Handle condition which can result in an infinite loop in attemptScheduling. (Tao Jie via asuresh) 
(cherry picked from commit e88422df45)
 2015-12-21 22:41:49 -08:00
cnauroth fdfcffb3d1 YARN-3458. CPU resource monitoring in Windows. Contributed by Inigo Goiri. 
(cherry picked from commit 114b590955)
 2015-12-21 16:59:17 -08:00
Wangda Tan daf5fae369 YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports information. (Sunil G via wangda) 
(cherry picked from commit a0de702851)
 2015-12-21 11:36:49 -08:00
Wangda Tan f16daa59a4 YARN-4454. NM to nodelabel mapping going wrong after RM restart. (Bibin A Chundatt via wangda) 
(cherry picked from commit bc038b382c)
 2015-12-21 11:30:37 -08:00
Uma Mahesh fc9e130825 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 
(cherry picked from commit 0f82b5d878)
 2015-12-19 23:17:33 -08:00
Jian He 1147b086d9 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2015-12-18 14:14:31 -08:00
Junping Du 6d2914a697 YARN-3226. UI changes for decommissioning node. Contributed by Sunil G. 
(cherry picked from commit 1de56b0448)
 2015-12-17 15:21:42 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) d9b9a75928 YARN-1856. Added cgroups based memory monitoring for containers as another alternative to custom memory-monitoring. Contributed by Varun Vasudev. 
(cherry picked from commit 4e7d32c0db)
 2015-12-17 12:13:56 -08:00
Jason Lowe fad2c5b862 YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne 
(cherry picked from commit 91828fef6b)
 2015-12-16 23:23:19 +00:00
Wangda Tan 5060fd1840 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda) 
(cherry picked from commit 9b856d9787)
 2015-12-16 13:29:08 -08:00
Wangda Tan 2f5435acbd YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 
(cherry picked from commit 7faa406f27)
 2015-12-16 13:29:02 -08:00
Wangda Tan 9f4a6ec663 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83)
 2015-12-16 13:28:56 -08:00
Siddharth Seth 433212ef87 YARN-4207. Add a non-judgemental YARN app completion status. Contributed by Rich Haase. 
(cherry picked from commit 0f708d465f)
 2015-12-16 11:35:38 -08:00
Junping Du 1a2ef845b5 YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d)
 2015-12-16 10:58:40 -08:00
Zhihai Xu 65cbac9de4 Update CHANGES.txt to move MAPREDUCE-6436 from YARN to MAPREDUCE 
(cherry picked from commit 7092d47fc0)
 2015-12-15 21:49:22 -08:00
Zhihai Xu 83c72ce9f6 Update CHANGES.txt to add YARN-3857 and YARN-3535 to branch-2.6 
(cherry picked from commit 0c3a53e5a9)
 2015-12-15 01:51:48 -08:00
Zhihai Xu da7bde60c0 MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d069)
 2015-12-15 01:03:10 -08:00
Zhihai Xu be50df687b YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun 
(cherry picked from commit 2aaed10327)
 2015-12-15 00:21:13 -08:00
Xuan 992f65dd8b YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He

(cherry picked from commit d8a45425eb)
 2015-12-14 15:42:07 -08:00
Jian He 72544db3cd YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 14:59:18 -08:00
Jian He 34e86d8d47 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:52:04 -08:00
Wangda Tan c915a658c8 YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 
(cherry picked from commit 07b0fb996a)
 2015-12-14 11:25:00 -08:00
Wangda Tan 9f979107c2 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb09)
 2015-12-14 11:13:51 -08:00
Wangda Tan a1a723fdff YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 
(cherry picked from commit 6cb0af3c39)
 2015-12-14 10:54:20 -08:00
Arun Suresh 0bf328984a YARN-4358 addendum patch to fix javadoc error 
(cherry picked from commit 7fb212e5e6)
 2015-12-12 22:25:59 -08:00
Sangjin Lee 605914caac YARN-4341 addendum. Moved the changelog entry from 2.8.0 to 2.9.0. 
(cherry picked from commit b7b292367f)
 2015-12-11 09:53:40 -08:00
Sangjin Lee 8be667208f YARN-4341. add doc about timeline performance tool usage (Chang Li via sjlee) 
(cherry picked from commit e8964589de)
 2015-12-11 09:53:17 -08:00
Xuan 455c6f39c5 YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml 
(cherry picked from commit 21daa6c68a)
 2015-12-10 10:57:35 -08:00
Junping Du 0b5e76e9ff YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2015-12-10 02:57:46 -08:00
Akira Ajisaka 6451313560 YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. 
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
 2015-12-09 16:32:52 +09:00
rohithsharmaks 81f8a2b008 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks) 
(cherry picked from commit a5e2e1ecb0)
 2015-12-09 11:33:49 +05:30
rohithsharmaks f626d18cc3 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 
(cherry picked from commit 15c3e7ffe3)
 2015-12-09 10:51:57 +05:30
Wangda Tan 7013f9d6cd YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-08 14:30:01 -08:00
Chris Douglas bfe4796802 YARN-4248. Followup patch adding asf-licence exclusions for json test files 
(cherry picked from commit 9f50e13d5d)
 2015-12-08 12:08:35 -08:00
= 4ac1564418 YARN-4248. REST API for submit/update/delete Reservations. (curino) 
(cherry picked from commit c25a635459)
 2015-12-07 13:38:38 -08:00
Jonathan Eagles 7c604c3d06 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 
(cherry picked from commit 4ff973f96a)
 2015-12-07 15:05:15 -06:00
Xuan 7decba83ef YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)
 2015-12-07 12:26:09 -08:00
rohithsharmaks 04e056f4f5 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks) 
(cherry picked from commit 01a641bc44)
 2015-12-07 15:19:59 +05:30
Steve Loughran af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Arun Suresh 78a07e99dd YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 
(cherry picked from commit 742632e346)
 2015-12-05 21:30:48 -08:00
Jian He 8a97ca4b00 Add missing file for YARN-4419 2015-12-03 19:11:07 -08:00
Jian He be404e1680 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 
(cherry picked from commit 755dda8dd8)
 2015-12-03 17:46:09 -08:00
Wangda Tan 51e448d295 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 
(cherry picked from commit a2c3bfc8c1)
 2015-12-03 14:30:25 -08:00
Jian He 23fe1abc02 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)
 2015-12-03 12:49:22 -08:00
Junping Du e76ba91fa5 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 
(cherry picked from commit 62e9348bc1)
 2015-12-03 06:37:31 -08:00
Jian He 48cbde9083 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 2015-12-02 11:08:00 -08:00
Allen Wittenauer 45bb467918 HADOOP-12366. expose calculated paths (aw) 2015-12-01 08:16:20 -08:00
Junping Du 092c469059 Creating 2.6.4 entries in CHANGES.txt files. 
(cherry picked from commit 028894281b)
 2015-12-01 08:18:57 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Vinod Kumar Vavilapalli 767815f317 Adding release 2.8.0 to CHANGES.txt 
(cherry picked from commit d57fd181c7)
 2015-11-25 17:37:47 -08:00
Tsuyoshi Ozawa d76b523b02 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83)
 2015-11-26 01:10:41 +09:00
Wangda Tan 7b9f7eaf5c YARN-4384. updateNodeResource CLI should not accept negative values for resource. (Junping Du via wangda) 
(cherry picked from commit 23c625ec57)
 2015-11-24 16:36:30 -08:00
Jason Lowe 3a2f5d6329 YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 
(cherry picked from commit f5acf94eca)
 2015-11-24 23:47:54 +00:00
Jason Lowe 9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 
(cherry picked from commit 4ac6799d4a)
 2015-11-24 22:36:19 +00:00
Tsuyoshi Ozawa 9e458c3e9e YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 
(cherry picked from commit 28dfe721b8)
 2015-11-24 22:10:31 +09:00
Akira Ajisaka ac0ddc4fec YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G. 
(cherry picked from commit 8c0133f3d4)
 2015-11-24 18:40:56 +09:00
Karthik Kambatla 49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20b)
 2015-11-24 13:52:29 +05:30
Jian He b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a1)
 2015-11-23 17:20:24 -08:00
Jason Lowe d13b646c61 Update CHANGES.txt for recent commits to branch-2.6. 
(cherry picked from commit aa2009c37b)
 2015-11-23 21:58:57 +00:00
Jason Lowe 9d303edbfb YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 
(cherry picked from commit d36b6e045f)
 2015-11-23 20:31:50 +00:00
Arun Suresh cbbdbe3cdd YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 
(cherry picked from commit da1016365a)
 2015-11-21 10:01:10 -08:00
Wangda Tan 08727287de move YARN-4326/YARN-3849 from 2.8.0 to 2.7.3 2015-11-20 16:42:11 -08:00
Wangda Tan cdca460ad4 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 
(cherry picked from commit 2346fa3141)
 2015-11-20 15:56:17 -08:00
Jason Lowe f3bca8bc1a YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 
(cherry picked from commit 060cdcbe5d)
 2015-11-20 23:13:27 +00:00
Xuan 8347b1048d YARN-2859 addendum: move the entry from 2.6.2 to 2.6.3 in hadoop-yarn/CHANGES.txt. 
(cherry picked from commit 866dce4e8e)
 2015-11-19 10:17:20 -08:00
Xuan be65d319fa YARN-2859.addendum: fix the remaining issue from the previous patch 
(cherry picked from commit f114e728da)
 2015-11-19 10:09:48 -08:00
Wangda Tan 3504d0e795 move fix version of YARN-4287 from 2.8.0 to 2.7.3 2015-11-18 14:01:02 -08:00
Karthik Kambatla d3f9508096 YARN-4367. SLS webapp doesn't load. (kasha) 2015-11-18 13:45:00 -08:00
Steve Loughran a0b3e43a6d YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:24 +00:00
Arun Suresh d8e60080e9 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 
(cherry picked from commit 6a61928fb7)
 2015-11-17 15:51:59 -08:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Junping Du 1d107d8058 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b)
 2015-11-15 04:44:59 -08:00
Wangda Tan 875aec3177 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 
(cherry picked from commit 7ff280fca9)
 2015-11-12 11:50:41 -08:00
Wangda Tan 13f8c7093b YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 
(cherry picked from commit 7f55a18071)
 2015-11-12 11:24:04 -08:00
Wangda Tan 0c1fb15b29 YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 
(cherry picked from commit 796638d9bc)
 2015-11-12 11:12:11 -08:00
Jason Lowe 9050078005 YARN-4345. yarn rmadmin -updateNodeResource doesn't work. Contributed by Junping Du 
(cherry picked from commit 06a70311b9)
 2015-11-12 14:49:55 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) bb5dd7f772 YARN-4183. Reverting the patch to fix behaviour change. 
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit 42bc525acb.
 2015-11-11 10:42:52 -08:00
Akira Ajisaka f6f128c867 YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. 
(cherry picked from commit 23d0db551c)
 2015-11-11 16:13:50 +09:00
Wangda Tan 8a8ac50a5b YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda) 
(cherry picked from commit c99925d6dd)
 2015-11-10 13:17:23 -08:00
Vinod Kumar Vavilapalli b742ea7fe6 HADOOP-12451. Setting HADOOP_HOME explicitly should be allowed. Fixing bugs in 
the first patch committed. (Karthik Kambatla via vinodkv)
 2015-11-10 12:16:52 -08:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
 2015-11-09 11:05:12 -08:00
Jason Lowe 8334d86472 Add 2.7.3 release to CHANGES.txt 
(cherry picked from commit 0383a3973b)
 2015-11-04 16:27:28 +00:00
Wangda Tan 3190446df9 YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 
(cherry picked from commit 0783184f4b)
 2015-11-03 11:36:11 -08:00
Sangjin Lee b2d6ea5e18 Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b4)
 2015-10-30 18:49:21 -07:00
Jason Lowe 8dce482f61 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096)
 2015-10-30 14:52:06 +00:00
Tsuyoshi Ozawa 1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739)
 2015-10-30 18:00:35 +09:00
Tsuyoshi Ozawa dd144035a0 Add an entry of YARN-4312 to CHANGES.txt 
(cherry picked from commit d21214ce33)
 2015-10-30 17:57:29 +09:00
Tsuyoshi Ozawa c522a9ffc2 Move YARN-3580 in CHANGES.txt from 2.8 to 2.7.2. 
(cherry picked from commit d2e01f4ed8)
 2015-10-30 15:51:04 +09:00
Xuan b505bf9e58 YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He

(cherry picked from commit 7412ff48ee)
 2015-10-29 17:37:24 -07:00
Jian He 2b197bbffd YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 
(cherry picked from commit e5b1733e04)
 2015-10-29 15:43:26 -07:00
Jonathan Eagles 42bc525acb YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 
(cherry picked from commit c293c58954)
 2015-10-29 16:42:49 -05:00
Arun Suresh 8943abfb08 YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 
(cherry picked from commit 58d1df585c)
 2015-10-29 13:43:43 -07:00
Jason Lowe c75d8b164f YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 
(cherry picked from commit e2267de207)
 2015-10-29 16:39:55 +00:00
Jian He 7d109ef5b5 YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du 
(cherry picked from commit c41699965e)
 2015-10-29 00:01:20 -07:00
Rohith Sharma K S e3fcb711d6 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 
(cherry picked from commit 656c8f9527)
 2015-10-29 12:25:08 +05:30
Sangjin Lee 16db203a83 Set the release date for 2.6.2 2015-10-28 20:48:49 -07:00
Xuan 9ce5069d16 YARN-2859. ApplicationHistoryServer binds to default port 8188 in 
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 27414dac66)
 2015-10-28 10:33:56 -07:00
Tsuyoshi Ozawa d29bff86c4 YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 9f4dfdf4eb)
 2015-10-28 17:39:26 +09:00
Varun Vasudev c5bf1cb7af YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt. 
(cherry picked from commit c28e16b40c)
 2015-10-27 20:56:45 +05:30
Tsuyoshi Ozawa 3138b43f16 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 96677bef00)
 2015-10-27 12:35:20 +09:00
Akira Ajisaka d9156c1245 YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka) 
(cherry picked from commit 8a68630dd1)
 2015-10-27 10:28:30 +09:00
Wangda Tan 1499af38eb YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda) 
(cherry picked from commit 56e4f6237a)
 2015-10-26 16:45:20 -07:00
Wangda Tan 9bf1e74171 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 
(cherry picked from commit 6f606214e7)
 2015-10-26 16:38:39 -07:00
Wangda Tan baa2998e93 YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 
(cherry picked from commit 3cc73773eb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-10-26 13:12:56 -07:00
Jason Lowe 7d66e2e666 YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee 
(cherry picked from commit 33a03af3c3)
 2015-10-26 19:54:16 +00:00
Rohith Sharma K S 33455fa415 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 5acdde4744)
 2015-10-26 15:45:22 +05:30
Rohith Sharma K S 06ecc37e82 YARN-4223. Fixed findbugs warnings in hadoop-yarn-server-nodemanager project. (varun saxena via rohithsharmaks) 
(cherry picked from commit 092883b34a)
 2015-10-26 15:26:08 +05:30
Tsuyoshi Ozawa e85ebf0825 YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula. 
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java
 2015-10-26 17:06:46 +09:00
Rohith Sharma K S e75799d5fd YARN-4246. NPE while listing app attempt. (Nijel S F via rohithsharmaks) 
(cherry picked from commit b57f08c0d2)
 2015-10-26 12:07:48 +05:30
Allen Wittenauer 599653ad92 YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw) 2015-10-25 21:43:51 -07:00
Arun Suresh f0580dcac0 YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh) 
(cherry picked from commit ab8eb8770c)
 2015-10-24 22:54:45 -07:00
Steve Loughran fc5a433adb YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via stevel) 2015-10-24 12:54:48 +01:00
Steve Loughran 65b928d8c7 YARN-4289. TestDistributedShell failing with bind exception. (Brahma Reddy Battula via stevel) 2015-10-24 12:48:03 +01:00
Akira Ajisaka 356e922632 YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka) 
(cherry picked from commit 7781fe1b9e)
 2015-10-24 11:56:37 +09:00
Jason Lowe 235e84bf3d YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G 
(cherry picked from commit d3a34a4f38)
 2015-10-23 20:57:53 +00:00
Jonathan Eagles 6db7bfbcfd YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 
(cherry picked from commit f8adeb712d)
 2015-10-23 10:43:59 -05:00
Ming Ma 4bb7e68eb6 YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma) 
(cherry picked from commit 934d96a334)
 2015-10-23 08:37:46 -07:00
Junping Du d5145b58f6 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 
(cherry picked from commit 0fce5f9a49)
 2015-10-22 13:42:12 -07:00
Zhihai Xu 9fe7ccc61c YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong 
(cherry picked from commit 960201b79b)
 2015-10-22 12:31:28 -07:00
Anubhav Dhoot c44401f362 YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan. 
(cherry picked from commit 2798723a54)
 2015-10-22 06:52:41 -07:00
Jonathan Eagles cc8093e5ac YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) 
(cherry picked from commit 25f8f801d1)
 2015-10-21 15:38:48 -05:00
Tsuyoshi Ozawa e2f027dddc Moved CHANGES.txt entries to 2.6.2 
YARN-2019, YARN-4092, YARN-4101, YARN-4087, YARN-3727, YARN-4005,
YARN-3780, YARN-3802, YARN-3896, YARN-3554, YARN-3194, YARN-3798

(cherry picked from commit 14f0e2787c)
 2015-10-21 23:25:43 +09:00
Arun Suresh f2009dc89c YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh) 
(cherry picked from commit 506d1b1dbc)
 2015-10-20 16:48:58 -07:00
Arun Suresh acc0e718d6 YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh) 
(cherry picked from commit 7e2837f830)
 2015-10-19 20:02:10 -07:00
Varun Vasudev e4429de86f YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 151eab2a12)
 2015-10-19 20:56:43 +05:30
Varun Vasudev 196d549e2b YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit e39ae0e676)
 2015-10-19 20:21:48 +05:30
Steve Loughran 390f29ab99 YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel) 2015-10-18 12:03:46 +01:00
Sangjin Lee 180efe6677 YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee) 
(cherry picked from commit 58590fef49)
 2015-10-16 16:57:32 -07:00
Jian He 1ba73169c7 YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 
(cherry picked from commit f9da5cdb2b)
 2015-10-16 15:26:46 -07:00
Wangda Tan 204ee6a40f YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda) 
(cherry picked from commit 4337b263aa)
 2015-10-16 15:10:08 -07:00
Jian He 2862057f11 YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena 
(cherry picked from commit cf23f2c2b5)
 2015-10-15 17:13:09 -07:00
Varun Vasudev 342870b70f YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 63020c54c1)
 2015-10-15 13:41:58 +05:30
Steve Loughran 0e4fb329ed HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:33 +01:00
rohithsharmaks 526be695c0 YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks) 
(cherry picked from commit d6c8bad869)
 2015-10-14 16:13:26 +05:30
Varun Vasudev 2dd9c475c4 YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana. 
(cherry picked from commit da1ee078f9)
 2015-10-14 15:27:18 +05:30
Varun Vasudev 0aeedfd572 YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana. 
(cherry picked from commit fb9c1519b3)
 2015-10-14 15:19:19 +05:30
Varun Vasudev a9b03b5afb YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit  8d59293089)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-10-14 15:04:22 +05:30
Steve Loughran 5eab51a24f HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:47:54 +01:00
Varun Vasudev 34b8142802 YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana 
(cherry picked from commit 5b6bae0094)
 2015-10-13 12:50:28 +05:30
Jian He ee1ce393f5 YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding 
(cherry picked from commit 9849c8b386)
 2015-10-12 11:52:04 -07:00
Zhihai Xu 9988b57e71 YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong. 
(cherry picked from commit 049c6e8dc0)
 2015-10-12 00:16:40 -07:00
Devaraj K 28edc7b129 YARN-3964. Support NodeLabelsProvider at Resource Manager side. 
Contributed by Dian Fu.

(cherry picked from commit db93047881)
 2015-10-11 11:24:05 +05:30
Wangda Tan 37642abb83 YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda) 
(cherry picked from commit def374e666)
 2015-10-09 16:40:10 -07:00
Karthik Kambatla 2d8f9e3fd9 MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 
(cherry picked from commit 4aa9b3e75c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-10-09 07:54:47 -07:00
Jason Lowe 0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 
(cherry picked from commit a0bca2b5ad)
 2015-10-09 14:19:41 +00:00
Rohith Sharma K S 21609e8e9a YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks) 
(cherry picked from commit 8f195387a4)
 2015-10-09 10:11:53 +05:30
Jason Lowe ec74b0cd4e YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu 
(cherry picked from commit 8d226225d0)
 2015-10-08 22:26:35 +00:00
Jason Lowe 1caa05de4a Fixup CHANGES.txt to reflect recent commits to 2.7.2 and 2.6.2 
(cherry picked from commit 0841940a21)
 2015-10-08 16:47:30 +00:00
Rohith Sharma K S 99126714a3 YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 3793cbe4c3)
 2015-10-07 10:20:36 +05:30
Rohith Sharma K S 053fcf4c0d YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks) 
(cherry picked from commit 9156fc60c6)
 2015-10-07 09:37:14 +05:30
Wangda Tan 5453a63612 YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda) 
(cherry picked from commit 29a582ada0)
 2015-10-06 11:59:10 -07:00
Wangda Tan d24a3b9a3c YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda) 
(cherry picked from commit 30ac69c6bd)
 2015-10-05 15:48:40 -07:00
Vinod Kumar Vavilapalli b876babdc7 HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some Linux distributions after HADOOP-12317. Contributed by Wangda Tan. 
(cherry picked from commit f1c19b9365)
 2015-10-05 10:57:55 -07:00
Harsh J 21b4ba48ce HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 
(cherry picked from commit c918f7be5e)
 2015-10-03 18:39:22 +05:30
Xuan 53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 
(cherry picked from commit 8f08532bde)
 2015-10-02 18:53:04 -07:00
Jason Lowe 3e3733437f YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 
(cherry picked from commit fdf02d1f26)
 2015-10-02 20:21:32 +00:00
Jason Lowe d645ee1d62 YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 
(cherry picked from commit 854d25b0c3)
 2015-09-30 15:01:01 +00:00
Karthik Kambatla 64a7617eb9 YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) 
(cherry picked from commit a0b5a0a419)
 2015-09-29 08:13:25 -07:00
Anubhav Dhoot 22f2501476 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
 2015-09-28 16:54:06 -07:00
Jason Lowe 600022ae69 YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G 
(cherry picked from commit 9f53a95ff6)
 2015-09-28 22:56:07 +00:00
Anubhav Dhoot 4d8b99423e YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot) 
(cherry picked from commit fb2e525c07)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-28 09:39:52 -07:00
Xuan bf3f08cc63 YARN-3624. ApplicationHistoryServer should not reverse the order of the 
filters it gets. Contributed by Mit Desai

(cherry picked from commit d1b9b85244)
 2015-09-24 15:03:05 -07:00
Jian He ea2b6ee9e9 YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan 
(cherry picked from commit b3f6b641dc)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-24 10:30:49 -07:00
Jian He d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He d592aa115b YARN-3868. Recovery support for container resizing. Contributed by Meng Ding 
(cherry picked from commit c57eac5dfe)
 2015-09-24 10:24:13 -07:00
Jian He b6c594c6ea YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 
(cherry picked from commit c3dc1af072)
 2015-09-24 10:24:04 -07:00
Jian He b8955d81c5 YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan 
(cherry picked from commit c59ae4eeb1)
 2015-09-24 10:23:55 -07:00
Jian He fa0a554ae0 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit 5f5a968d65)
 2015-09-24 10:23:46 -07:00
Jian He afe4afd0c9 YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan 
(cherry picked from commit ffd820c27a)
 2015-09-24 10:23:35 -07:00
Jian He c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10)
 2015-09-24 10:23:25 -07:00
Jian He 89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849f)
 2015-09-24 10:23:13 -07:00
Rohith Sharma K S fbb1d923f4 YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) 2015-09-24 12:17:55 +05:30
Rohith Sharma K S ace4d26936 YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks) 2015-09-24 11:31:47 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) f48af332a7 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd)
 2015-09-23 15:12:30 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 48eb28e7e2 YARN-3248. Moving it to improvements section in CHANGES.txt. 
(cherry picked from commit df52ff92ae)
 2015-09-23 15:02:23 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 23f4cd1c5a CHANGES.txt: Moving YARN-1884, YARN-3171, YARN-3740, YARN-3248, YARN-3544 to 2.6.1 given the backport. 
(cherry picked from commit f3e5bc6766)
 2015-09-23 13:46:00 -07:00
Jason Lowe b7111449a1 YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is enabled. Contributed by Mit Desai 
(cherry picked from commit 692d51c09d)
 2015-09-23 16:39:50 +00:00
Jason Lowe c829be5133 YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu 
(cherry picked from commit c890c51a91)
 2015-09-23 15:44:13 +00:00
Tsuyoshi Ozawa 52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 
(cherry picked from commit dfd807afab)
 2015-09-22 12:54:44 +09:00
Chris Douglas b39b61e356 YARN-4188. Make MoveApplicationAcrossQueues abstract, newInstance static 
Contributed by Giovanni Matteo Fumarola
(cherry picked from commit 8e01b0d97a)
 2015-09-21 17:54:24 -07:00
Wangda Tan fafcb4ac4e YARN-4113. RM should respect retry-interval when uses RetryPolicies.RETRY_FOREVER. (Sunil G via wangda) 
(cherry picked from commit b00392dd9c)
 2015-09-21 11:15:02 -07:00
Rohith Sharma K S 52f343598e YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit c9cb6a5960)
 2015-09-21 10:06:39 +05:30
Arun Suresh b70c7b0e51 YARN-3920. FairScheduler container reservation on a node should be configurable to limit it to large containers (adhoot via asuresh) 
(cherry picked from commit 94dec5a916)
 2015-09-18 14:04:03 -07:00
Wangda Tan 6041fc75d8 MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob. (Junping Du via wangda) 2015-09-18 10:19:32 -07:00
Wangda Tan 4a657e9326 YARN-3212. RMNode State Transition Update with DECOMMISSIONING state. (Junping Du via wangda) 
(cherry picked from commit 9bc913a35c)
 2015-09-18 10:05:04 -07:00
Steve Loughran 22dee29857 YARN-2597 MiniYARNCluster should propagate reason for AHS not starting 2015-09-18 09:45:17 +01:00
Rohith Sharma K S d94c092ad1 YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.(Nijel S F via rohithsharmaks) 
(cherry picked from commit 723c31d45b)
 2015-09-18 08:46:36 +05:30
Jian He 5d2f85021f YARN-4034. Render cluster Max Priority in scheduler metrics in RM web UI. Contributed by Rohith Sharma K S 
(cherry picked from commit 6c6e734f0b)
 2015-09-17 14:56:32 +08:00
Steve Loughran ab09573fcb YARN-3433. Jersey tests failing with Port in Use -again. (Brahma Reddy Battula) 2015-09-16 10:30:49 +01:00
Jian He 9eda3ce3fa YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface. Contributed by Naganarasimha G R 
(cherry picked from commit 452079af8b)
 2015-09-16 15:01:23 +08:00
Xuan f3c0a210b6 YARN-4149. yarn logs -am should provide an option to fetch all the log 
files. Contributed by Varun Vasudev

(cherry picked from commit 34ef1a092b)
 2015-09-15 14:37:59 -07:00
Jason Lowe 181bda0421 YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu 
(cherry picked from commit 8c1cdb17a0)
 2015-09-15 20:22:23 +00:00
Wangda Tan c31fe79979 YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda) 
(cherry picked from commit ae5308fe1d)
 2015-09-15 11:41:57 -07:00
Junping Du df714e25aa YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du) 
(cherry picked from commit 73e3a49eb0)
 2015-09-15 07:58:47 -07:00
Jian He eacc18677a YARN-3635. Refactored current queue mapping implementation in CapacityScheduler to use a generic PlacementManager framework. Contributed by Wangda Tan 
(cherry picked from commit 5468baa80a)
 2015-09-15 19:02:17 +08:00
Wangda Tan e552775545 YARN-4151. Fix findbugs errors in hadoop-yarn-server-common module. (Meng Ding via wangda) 
(cherry picked from commit e2a0270217)
 2015-09-14 16:04:08 -07:00
Jian He b705d9d0ac YARN-4126. RM should not issue delegation tokens in unsecure mode. Contributed by Bibin A Chundatt 
(cherry picked from commit e1b1d7e4ae)
 2015-09-14 14:15:04 +08:00
Karthik Kambatla 1f9f219062 YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 
(cherry picked from commit 332b520a48)
 2015-09-13 18:08:03 -07:00
Karthik Kambatla 11e2fa151c YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha) 
(cherry picked from commit 81df7b586a)
 2015-09-13 17:06:04 -07:00
Steve Loughran 988749e6ea HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:06 +01:00
Robert Kanter 43f051a09a YARN-4145. Make RMHATestBase abstract so its not run when running all tests under that namespace (adhoot via rkanter) 
(cherry picked from commit ea4bb2749f)
 2015-09-11 11:46:35 -07:00
Robert Kanter d07eef8c87 YARN-4115. Reduce loglevel of ContainerManagementProtocolProxy to Debug (adhoot via rkanter) 
(cherry picked from commit b84fb41bb6)
 2015-09-11 11:46:35 -07:00
Varun Vasudev a4288f1262 YARN-4136. LinuxContainerExecutor loses info when forwarding ResourceHandlerException. Contributed by Bibin A Chundatt. 
(cherry picked from commit 486d5cb803)
 2015-09-11 14:39:01 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) cbf90f93f9 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed)
 2015-09-10 14:01:15 -07:00
Wangda Tan 5decd8f9f7 YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) 
(cherry picked from commit 77666105b4)
 2015-09-10 09:30:34 -07:00
Robert Kanter ea64a9b277 YARN-4086. Allow Aggregated Log readers to handle HAR files (rkanter) 
(cherry picked from commit 6dd6ca442a)
 2015-09-09 18:03:47 -07:00
Zhihai Xu 214338bd68 YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe. 
(cherry picked from commit 16b9037dc1)
 2015-09-08 12:33:29 -07:00
Varun Vasudev 1c53cd2b0d YARN-4121. Fix typos in capacity scheduler documentation. Contributed by Kai Sasaki. 
(cherry picked from commit 435f935ba7)
 2015-09-08 12:39:39 +05:30
Jian He f23be93dd1 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 
(cherry picked from commit 6f72f1e600)
 2015-09-08 11:24:07 +08:00
Xuan a0b7ef15d0 YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when 
state-store error occurs. Contributed by Jian He

(cherry picked from commit 9b78e6e33d)
 2015-09-07 17:46:48 -07:00
Xuan d983e55714 Modify the CHANGES.txt to move YARN-2019 Commit log from branch 2.8 to 
branch 2.7.2

(cherry picked from commit 6a50689705)
 2015-09-07 17:30:25 -07:00
Varun Vasudev 70575286b7 YARN-3591. Resource localization on a bad disk causes subsequent containers failure. Contributed by Lavkesh Lahngir. 
(cherry picked from commit 1dbd8e34a7)
 2015-09-07 11:34:14 +05:30
Wangda Tan 9f97b86816 YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda) 
(cherry picked from commit bcc85e3bab)
 2015-09-04 15:14:34 -07:00
Jason Lowe 6a5220cab3 YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li 
(cherry picked from commit 6eaca2e363)
 2015-09-04 15:31:37 +00:00
Varun Vasudev 64c2e8011e YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles. 
(cherry picked from commit 1b2e2ca9104dfc05e9aa7f93954ae2e00b3e62c4)
 2015-09-04 14:32:07 +05:30
Varun Vasudev a53054c853 YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R. 
(cherry picked from commit 223a306da925ad031bc398f2ea5e56f0e6521edf)
 2015-09-03 16:40:42 +05:30
Jian He cd82fa2f83 YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong 
(cherry picked from commit 09c64ba1ba)
 2015-09-02 17:46:04 -07:00
Rohith Sharma K S 6e41aa64a5 YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 7d6687fe76)
 2015-09-02 15:27:33 +05:30
Rohith Sharma K S 7ecdf51441 YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 095ab9ab5f)
 2015-09-02 14:28:50 +05:30
Vinod Kumar Vavilapalli acb3000187 YARN-3222. Added the missing CHANGES.txt entry. 
(cherry picked from commit 4620767156)
 2015-09-01 13:44:10 -07:00
Andrew Wang 7d833a3058 HADOOP-12369. Point hadoop-project/pom.xml java.security.krb5.conf within target folder. 
(cherry picked from commit 0eb9b19325)
 2015-09-01 10:59:39 -07:00
Tsuyoshi Ozawa d3d65f62b3 YARN-2801. Add documentation for node labels feature. Contributed by Wangda Tan and Naganarasimha G R. 
(cherry picked from commit faa38e1aa4)
 2015-09-01 17:55:53 +09:00
Varun Vasudev 855e0f8b00 YARN-4082. Container shouldn't be killed when node's label updated. Contributed by Wangda Tan. 
(cherry picked from commit bf669b6d9f)
 2015-09-01 14:20:54 +05:30
Jian He 21b478e1bf Move YARN-4092 to 2.7.2 
(cherry picked from commit 4eaa7fd3ea)
 2015-08-31 17:44:04 -07:00
Jian He 48f5161cd5 YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong 
(cherry picked from commit a3fd2ccc86)
 2015-08-31 17:34:49 -07:00
Vinod Kumar Vavilapalli 0191dae2ec YARN-2945. Fixing the CHANGES.txt to have the right JIRA number. 
(cherry picked from commit cf831565e8)
 2015-08-30 20:02:36 -07:00
Junping Du 792b9c0eaf YARN-1556. NPE getting application report with a null appId. Contributed by Weiwei Yang. 
(cherry picked from commit beb65c9465)
 2015-08-28 05:58:45 -07:00
Jian He 306c535395 YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit a9c8ea71aa)
 2015-08-27 13:27:34 -07:00
Andrew Wang 7a668981f5 HADOOP-12362. Set hadoop.tmp.dir and hadoop.log.dir in pom. Contributed by Charlie Helin. 
(cherry picked from commit 90fe7bcc3b)
 2015-08-27 10:17:04 -07:00
Jian He 387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit 57c7ae1aff)
 2015-08-24 20:37:18 -07:00
Rohith Sharma K S 6dc732f2f7 YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks) 
(cherry picked from commit feaf034994)
 2015-08-24 11:30:46 +05:30
Xuan 85363ea4bd YARN-221. Addendum patch to compilation issue which is caused by missing 
AllContainerLogAggregationPolicy. Contributed by Xuan Gong

(cherry picked from commit b71c6006f5)
 2015-08-23 16:50:02 -07:00
Xuan 205b98c059 YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma

(cherry picked from commit 37e1c3d82a)
 2015-08-22 16:26:39 -07:00
Rohith Sharma K S 27ced8253b YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface 
(cherry picked from commit 22de7c1dca)
 2015-08-21 10:56:45 +05:30
Vinod Kumar Vavilapalli 9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Vinod Kumar Vavilapalli 45be1240d9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a0)
 2015-08-20 17:23:54 -07:00
Wangda Tan 6452b31760 YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 
(cherry picked from commit fc07464d1a)
 2015-08-20 16:52:57 -07:00
Karthik Kambatla b567aa2b4f YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 13604bd5f1)
 2015-08-20 16:52:33 -07:00
Xuan fc6b176e70 YARN-4028. AppBlock page key update and diagnostics value null on 
recovery. Contributed by Bibin A Chundatt

(cherry picked from commit 22dc5fc209)
 2015-08-18 22:54:01 -07:00
Zhihai Xu 051a622b57 YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong. 
(cherry picked from commit 14215c8ef8)
 2015-08-18 11:47:41 -07:00
Zhihai Xu 8f2eb99d71 YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao. 
(cherry picked from commit 3a76a010b8)
 2015-08-18 10:53:16 -07:00
Karthik Kambatla d2ff763533 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 
(cherry picked from commit def12933b3)
 2015-08-16 06:26:21 -07:00
Jian He b2e869f2a8 YARN-3987. Send AM container completed msg to NM once AM finishes. Contributed by sandflee 
(cherry picked from commit 0a030546e2)
 2015-08-13 16:23:30 -07:00
Jian He 703fa1b141 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfab)
 2015-08-13 16:13:28 -07:00
Jian He 4a5fdb7c23 YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 
(cherry picked from commit 38aed1a94e)
 2015-08-13 14:46:34 -07:00
Robert Kanter fbcfca6f91 YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and WebAppProxyServer (djp via rkanter) 
(cherry picked from commit dc2340c60e)
 2015-08-12 15:48:14 -07:00
Jian He 65d22b3686 YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan 
(cherry picked from commit e5003be907)
 2015-08-12 15:09:46 -07:00
rohithsharmaks c5b20e0db6 YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 
(cherry picked from commit 1c12adb71f)
 2015-08-12 14:47:23 +05:30
Xuan 2ebdf5bfce YARN-3999. RM hangs on draing events. Contributed by Jian He 
(cherry picked from commit 3ae716fa69)
 2015-08-11 18:26:00 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 3dcff42437 Adding release 2.6.1 to CHANGES.txt 
(cherry picked from commit 7c796fd12c)
 2015-08-11 13:21:41 -07:00
Jian He df97e3265b YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G 
(cherry picked from commit fa1d84ae27)
 2015-08-10 20:52:18 -07:00
Wangda Tan 380d401da5 YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy. (Sunil G via wangda) 
(cherry picked from commit cf9d3c9256)
 2015-08-10 14:58:19 -07:00
Wangda Tan cdb61b5fb2 YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) 
(cherry picked from commit 4bc42d76e7)
 2015-08-07 09:53:42 -07:00
Rohith Sharma K S a0da1ec010 YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks) 
(cherry picked from commit b6265d39c5)
 2015-08-07 10:47:08 +05:30
Carlo Curino 8e23a7997f YARN-3974. Refactor the reservation system test cases to use parameterized base test. (subru via curino) 
(cherry picked from commit 8572a5a14b)
 2015-08-02 02:06:06 -07:00
Junping Du 7af531d2e3 YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter. 
(cherry picked from commit cfee02b3bd)
 2015-08-06 06:50:40 -07:00
Arun Suresh 84ea7cd408 YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh) 
(cherry picked from commit 154c9d2e42)
 2015-08-05 23:15:33 -07:00
rohithsharmaks dc76c4b035 YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) 
(cherry picked from commit df9e7280db)
 2015-08-06 10:49:37 +05:30
Jian He 1466772827 YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend container allocation logic. Contributed by Wangda Tan 
(cherry picked from commit ba2313d614)
 2015-08-05 13:51:08 -07:00
Arun Suresh 707b96fa58 YARN-3736. Add RMStateStore apis to store and load accepted reservations for failover (adhoot via asuresh) 
(cherry picked from commit f271d37735)
 2015-08-05 12:58:37 -07:00
Xuan 89ccfd9fcc YARN-4004. container-executor should print output of docker logs if the 
docker container exits with non-0 exit status. Contributed by Varun
Vasudev

(cherry picked from commit c3364ca8e7)
 2015-08-03 18:11:16 -07:00
Xuan 8126fde884 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S

(cherry picked from commit 0306d902f5)
 2015-08-03 15:47:21 -07:00
Jason Lowe b264217bc3 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 
(cherry picked from commit 469cfcd695)
 2015-08-03 15:55:04 +00:00
Jonathan Eagles 899df5bce0 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b9522)
 2015-08-03 10:40:04 -05:00
Jason Lowe c31e3ba921 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c0)
 2015-07-31 17:38:49 +00:00
Zhihai Xu 0e2019fa30 YARN-433. When RM is catching up with node updates then it should not expire acquired containers. Contributed by Xuan Gong 
(cherry picked from commit ab80e27703)
 2015-07-30 22:03:04 -07:00
Wangda Tan 53a0653826 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) 
(cherry picked from commit 91b42e7d6e)
 2015-07-30 10:01:01 -07:00
Wangda Tan a28e23d049 YARN-3963. AddNodeLabel on duplicate label addition shows success. (Bibin A Chundatt via wangda) 
(cherry picked from commit 8acb30b016)
 2015-07-30 09:43:45 -07:00
rohithsharmaks 9c00ae0009 YARN-3919. NPEs' while stopping service after exception during CommonNodeLabelsManager#start. (varun saxena via rohithsharmaks) 2015-07-29 23:44:15 +05:30
Karthik Kambatla 8659315062 YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha) 
(cherry picked from commit 5205a330b3)
 2015-07-29 09:42:57 -07:00
Jason Lowe 6f50a6a456 YARN-3950. Add unique SHELL_ID environment variable to DistributedShell. Contributed by Robert Kanter 
(cherry picked from commit 2b2bd92146)
 2015-07-29 15:18:45 +00:00
Xuan 2ec008eb38 YARN-3982. container-executor parsing of container-executor.cfg broken 
in trunk and branch-2. Contributed by Varun Vasudev

(cherry picked from commit f170934215)
 2015-07-27 23:47:31 -07:00
Jian He 7c123accda YARN-3846. RM Web UI queue filter is not working for sub queue. Contributed by Mohammad Shahid Khan 
(cherry picked from commit 3572ebd738)
 2015-07-27 17:12:24 -07:00
Varun Vasudev 9da487e0fd YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit 3e6fce91a4)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-07-27 12:03:36 -07:00
Varun Vasudev ec0f801f52 YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab. 
(cherry picked from commit f36835ff9b)
 2015-07-27 10:18:06 -07:00
Akira Ajisaka 0e7c88347e YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api module. Contributed by Varun Saxena. 
(cherry picked from commit 42d4e0ae99)
 2015-07-27 11:45:41 +09:00
ccurino 26ea045814 YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations. (Jonathan Yaniv and Ishai Menache via curino) 
(cherry picked from commit 156f24ead0)
 2015-07-25 07:47:11 -07:00
Wangda Tan 4ba2e1ae95 YARN-3973. Recent changes to application priority management break reservation system from YARN-1051 (Carlo Curino via wangda) 
(cherry picked from commit a3bd7b4a59)
 2015-07-24 16:45:06 -07:00
Jason Lowe 2cf714123a YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a7)
 2015-07-24 22:15:44 +00:00
Jian He 3233284e87 YARN-3026. Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp. Contributed by Wangda Tan 
(cherry picked from commit 83fe34ac08)
 2015-07-24 14:01:36 -07:00
carlo curino bddc48ed86 YARN-3969. Updating CHANGES.txt to reflect the correct set of branches where this is committed 
(cherry picked from commit fc42fa8ae3)
 2015-07-24 13:40:14 -07:00
Karthik Kambatla d2ee02940a YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500. (Anubhav Dhoot via kasha) 
(cherry picked from commit d19d187753)
 2015-07-24 11:49:46 -07:00
Xuan 98ed4bca3b YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai

(cherry picked from commit fbd6063269)
 2015-07-24 10:17:05 -07:00
carlo curino 6a71a0a230 YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 
(cherry picked from commit 0fcb4a8cf2)
 2015-07-23 19:36:54 -07:00
Rohith Sharma K S 9d15860e6d YARN-3845. Scheduler page does not render RGBA color combinations in IE11. (Contributed by Mohammad Shahid Khan) 2015-07-24 12:54:07 +05:30
Robert Kanter ba3b40ae79 YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos that include it (adhoot via rkanter) 
(cherry picked from commit 1d3026e7b3)
 2015-07-23 14:47:28 -07:00
Wangda Tan 6baebcaedc YARN-3941. Proportional Preemption policy should try to avoid sending duplicate PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda) 
(cherry picked from commit 3bba180051)
 2015-07-23 10:08:30 -07:00
Junping Du 6772c3f4dd YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 
(cherry picked from commit ee98d6354b)
 2015-07-22 17:57:16 -07:00
rohithsharmaks be2334ba3a YARN-3954. Fix TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml. (varun saxena via rohithsharmaks) 2015-07-23 02:10:45 +05:30
Wangda Tan 3800e25d6e YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda) 
(cherry picked from commit 06e5dd2c84)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-07-22 12:03:14 -07:00
Wangda Tan 3576b0ac5e YARN-3932. SchedulerApplicationAttempt#getResourceUsageReport and UserInfo should based on total-used-resources. (Bibin A Chundatt via wangda) 
(cherry picked from commit 76ec26de80)
 2015-07-22 11:54:38 -07:00
Jian He 11cb00f7a0 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 
(cherry picked from commit 393fe71771)
 2015-07-21 15:06:16 -07:00
Wangda Tan 3dd113fa7c YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda) 
(cherry picked from commit c39ca541f4)
 2015-07-21 09:57:53 -07:00
Arun Suresh eddb326669 Pulling in YARN-3535 to branch 2.7.x 
(cherry picked from commit 176131f12b)
 2015-07-18 10:11:29 -07:00
Colin Patrick Mccabe aff955a14a YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe) 
(cherry picked from commit 419c51d233)
 2015-07-17 12:05:24 -07:00
Jonathan Eagles b289939514 YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 
(cherry picked from commit 7faae0e6fe)
 2015-07-17 11:03:34 -05:00
Arun Suresh 8018041b49 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 
(cherry picked from commit 9b272ccae7)
 2015-07-17 04:33:04 -07:00
Wangda Tan a7de3cde13 YARN-3885. ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level. (Ajith S via wangda) 
(cherry picked from commit 3540d5fe4b)
 2015-07-16 16:14:07 -07:00
Wangda Tan 7b2540770e YARN-3930. FileSystemNodeLabelsStore should make sure edit log file closed when exception is thrown. (Dian Fu via wangda) 
(cherry picked from commit fa2b63ed16)
 2015-07-16 16:08:04 -07:00
Tsuyoshi Ozawa e97a14ae6f YARN-3805. Update the documentation of Disk Checker based on YARN-90. Contributed by Masatake Iwasaki. 
(cherry picked from commit 1ba2986dee)
 2015-07-16 17:52:58 +09:00
Tsuyoshi Ozawa f95e3c3091 YARN-3174. Consolidate the NodeManager and NodeManagerRestart documentation into one. Contributed by Masatake Iwasaki. 
(cherry picked from commit f02dd146f5)
 2015-07-16 15:22:57 +09:00
Tsuyoshi Ozawa bfbc805a13 YARN-3170. YARN architecture document needs updating. Contirubted by Brahma Reddy Battula. 
(cherry picked from commit edcaae44c1)
 2015-07-15 15:43:09 +09:00
Arun Suresh 5f58be7dd4 YARN-3453. Ensure preemption logic in FairScheduler uses DominantResourceCalculator in DRF queues to prevent unnecessary thrashing. (asuresh) 
(cherry picked from commit ac94ba3e18)
 2015-07-14 00:25:57 -07:00
Chris Douglas c6b0bbabc7 HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl 
(cherry picked from commit a431ed9075)
 2015-07-13 15:36:25 -07:00
Jian He 27366dd7c3 Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa4.
(cherry picked from commit 2466460d4c)
 2015-07-13 14:31:55 -07:00
Akira Ajisaka 0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d9)
 2015-07-13 17:53:27 +09:00
Wangda Tan 87360be089 YARN-3894. RM startup should fail for wrong CS xml NodeLabel capacity configuration. (Bibin A Chundatt via wangda) 
(cherry picked from commit 5ed1fead6b)
 2015-07-12 21:53:08 -07:00
Akira Ajisaka ad6787aa43 YARN-3069. Document missing properties in yarn-default.xml. Contributed by Ray Chiang. 
(cherry picked from commit d6675606dc)
 2015-07-13 12:57:29 +09:00
Gera Shegalov 896b8c6551 YARN-3917. getResourceCalculatorPlugin for the default should intercept all exceptions. (gera) 
(cherry picked from commit d7319dee37)
 2015-07-11 22:41:04 -07:00
Wangda Tan c36090fd3f YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 
(cherry picked from commit 1df39c1efc)
 2015-07-11 10:27:58 -07:00
Zhijie Shen 37a93c2d78 YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 1ea36299a4)
 2015-07-10 19:02:13 -07:00
Ming Ma b169889f01 YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) 
(cherry picked from commit 08244264c0)
 2015-07-10 08:34:01 -07:00
Xuan 2a01495544 YARN-3888. ApplicationMaster link is broken in RM WebUI when appstate is 
NEW. Contributed by Bibin A Chundatt

(cherry picked from commit 5214876792)
 2015-07-09 21:38:46 -07:00
Chris Douglas 66419d3672 HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl 
(cherry picked from commit 1a0752d85a)
 2015-07-09 17:48:56 -07:00
carlo curino 753d87066f YARN-3800. Reduce storage footprint for ReservationAllocation. Contributed by Anubhav Dhoot. 
(cherry picked from commit 0e602fa3a1)
 2015-07-09 16:52:47 -07:00
Karthik Kambatla fc989ebe16 HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) 
(cherry picked from commit ac6048372a)
 2015-07-09 09:59:02 -07:00
Karthik Kambatla ccf18705f7 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 
(cherry picked from commit aa067c6aa4)
 2015-07-09 09:48:46 -07:00
Karthik Kambatla d15f69457b YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 527c40e4d6)
 2015-07-09 09:35:39 -07:00
Varun Vasudev d17a7bbf6e YARN-2194. Addendum patch to fix failing unit test in TestPrivilegedOperationExecutor. Contributed by Sidharta Seethana. 
(cherry picked from ad5b904175a1f89951c7c32f744a4e20fafb51b5)
 2015-07-09 11:52:57 +05:30
Akira Ajisaka d260478d3a YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d6325745e2)
 2015-07-08 15:43:04 +09:00
Jian He fa59b224b8 YARN-3892. Fixed NPE on RMStateStore#serviceStop when CapacityScheduler#serviceInit fails. Contributed by Bibin A Chundatt 
(cherry picked from commit c9dd2cada0)
 2015-07-07 14:18:16 -07:00
Varun Vasudev 16ca0fcf99 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan. 
(cherry-picked from commit 7e3ebc0fc59dbd1d7768b5a6c05af18b8c8fea4d)
 2015-07-07 17:05:06 +05:30
Devaraj K f558429efe YARN-3837. javadocs of TimelineAuthenticationFilterInitializer give wrong 
prefix for auth options. Contributed by Bibin A Chundatt.

(cherry picked from commit af63427c6d)
 2015-07-07 12:08:03 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) a8bf34cad8 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8)
 2015-07-06 16:41:00 -07:00
Xuan 09639ac6b4 YARN-3882. AggregatedLogFormat should close aclScanner and ownerScanner 
after create them. Contributed by zhihai xu

(cherry picked from commit 688617d6d7)
 2015-07-04 21:54:22 -07:00
Devaraj K e1ca886419 YARN-3875. FSSchedulerNode#reserveResource() doesn't print Application Id 
properly in log. Contributed by Bibin A Chundatt.

(cherry picked from commit 37d7395773)
 2015-07-02 10:21:32 +05:30