Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,748 Commits 326 Branches 371 Tags
Commit Graph

1882 Commits

Author SHA1 Message Date
Wangda Tan
b74391ce14 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda) 
(cherry picked from commit 9b856d9787be5ec88ef34574b9b98755d7b669ea)
(cherry picked from commit 5060fd1840ba2962cdfd7d91e2f405d7bdcfd169)
 2015-12-18 16:34:45 -08:00
Wangda Tan
c4b1386b4e YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 
(cherry picked from commit 7faa406f27f687844c941346f59a27a375af3233)
(cherry picked from commit 2f5435acbda53470724d833cfa1c58dc3c3c9bb1)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan
9ed9eb265f YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83e981ae74cb8b58ffcf7907b7612ad4)
(cherry picked from commit 9f4a6ec6630f3776859f8d677532370095a58d0c)
 2015-12-18 16:34:45 -08:00
Zhihai Xu
e4f638d815 YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun 
(cherry picked from commit 2aaed1032719e978660d1f14a8ed299e83b5f65e)
(cherry picked from commit be50df687bf800d35f418128d0c3f05ab6c429c8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan
1d1c6c1f8d YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 
(cherry picked from commit 07b0fb996a32020678bd2ce482b672f0434651f0)
(cherry picked from commit c915a658c8a5481ca88d8b2f6ce9a72f2e1f5b6a)
 2015-12-18 16:34:44 -08:00
Wangda Tan
c7e10d70de YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb0963c89c0455d41223427165b9f9e537)
(cherry picked from commit 9f979107c22f365633811a38363d0dbd34573374)
 2015-12-18 16:34:44 -08:00
Wangda Tan
8ee83765f4 YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 
(cherry picked from commit 6cb0af3c39a5d49cb2f7911ee21363a9542ca2d7)
(cherry picked from commit a1a723fdffee44be9a5d2a1ced896442a5318375)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:44 -08:00
Wangda Tan
a4ba6c8525 YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d31ac889fba26d453feedcebb11fc70)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cda88e72a839b1c55757615b55101beb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Jonathan Eagles
c4ae007fd4 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 
(cherry picked from commit 4ff973f96ae7f77cda3b52b38427e2991819ad31)
(cherry picked from commit 7c604c3d065ab15927ef7e14be6acf7772f6077f)
 2015-12-18 16:34:43 -08:00
Xuan
46cb47c03e YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b6762c18ba150d80a8976eb51a8290c)
(cherry picked from commit 7decba83ef541b1f89745ad040c6137d24b187b0)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Jian He
f355645ca4 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 
(cherry picked from commit 755dda8dd8bb23864abc752bad506f223fcac010)

(cherry picked from commit be404e168060c03a33a35c8d30925e317a64cad7)
 2015-12-18 16:34:42 -08:00
Wangda Tan
15b3ff1971 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 
(cherry picked from commit a2c3bfc8c1349102a7f2bc4ea96b80b429ac227b)
(cherry picked from commit 51e448d295f6d2214c1d6455446a8e05e8a967b1)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Jian He
9c22910570 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 
(cherry picked from commit 48cbde908319775b82ae30f262cca0503866e88e)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Junping Du
8812be3dc1 YARN-3226. UI changes for decommissioning node. Contributed by Sunil G. 
(cherry picked from commit 1de56b0448d332717c8316c621b4f6af542a85cc)
(cherry picked from commit 6d2914a69791d66862630516142573e02b9d55d7)
 2015-12-17 15:22:37 -08:00
Junping Du
92de57bdca YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d63d4c80dc1e7bf4024bfaf42cf4416)
(cherry picked from commit 1a2ef845b54166194b133e524ad9533cc259aed2)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java
(cherry picked from commit 0f9dd488428ae65a17043f3a976c6ef778806328)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-17 06:16:08 -08:00
Jason Lowe
d939c23cc1 YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne 
(cherry picked from commit 91828fef6b9314f72d1f973f00e81404dc6bba91)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-16 23:24:51 +00:00
Xuan
f22d0046de YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He

(cherry picked from commit d8a45425eba372cdebef3be50436b6ddf1c4e192)
(cherry picked from commit 992f65dd8b53ee31b6b7347e2cfe59a78d031268)
 2015-12-14 17:16:07 -08:00
Jian He
bc7603e393 YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 15:06:13 -08:00
Jian He
16f7ac48d4 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:52:33 -08:00
Arun Suresh
c2e788a7b5 YARN-4358 addendum patch to fix javadoc error 
(cherry picked from commit 7fb212e5e672b3f285493f6c7b2598ef7e61406b)
 2015-12-12 22:26:35 -08:00
rohithsharmaks
5bc33c5ae4 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks) 
(cherry picked from commit a5e2e1ecb06a3942903cb79f61f0f4bb02480f19)
 2015-12-09 11:35:02 +05:30
rohithsharmaks
6a6bbc51c4 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 2015-12-09 11:06:31 +05:30
Chris Douglas
3d100163de YARN-4248. Followup patch adding asf-licence exclusions for json test files 
(cherry picked from commit 9f50e13d5dc329c3a6df7f9bcaf2f29b35adc52b)
(cherry picked from commit bfe4796802c8f706fdc4c393324ddb9f24194677)
 2015-12-08 12:08:53 -08:00
=
3205fe3da8 YARN-4248. REST API for submit/update/delete Reservations. (curino) 
(cherry picked from commit c25a6354598ec855bec7f695a7c3eed8794cd381)
(cherry picked from commit 4ac1564418597e2ebbaa80600eb5f002b1431281)
 2015-12-07 13:46:37 -08:00
rohithsharmaks
03e615ba55 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks) 
(cherry picked from commit 01a641bc447c464b2830d58addd482f47dbd92ae)
 2015-12-07 15:20:50 +05:30
Arun Suresh
db69c5f744 YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 
(cherry picked from commit 742632e346604fd2b263bd42367165638fcf2416)
 2015-12-05 21:35:32 -08:00
Junping Du
6af5255b02 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 
(cherry picked from commit 62e9348bc10bb97a5fcb4281f7996a09d8e69c60)
(cherry picked from commit e76ba91fa58dae6750f608c0f8b86e9a43e80005)
 2015-12-03 06:39:07 -08:00
Tsuyoshi Ozawa
d76b523b02 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83af6a48a8d8d0e37cdf1f813124f366)
 2015-11-26 01:10:41 +09:00
Jason Lowe
9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 
(cherry picked from commit 4ac6799d4a8b071e0d367c2d709e84d8ea06942d)
 2015-11-24 22:36:19 +00:00
Tsuyoshi Ozawa
9e458c3e9e YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 
(cherry picked from commit 28dfe721b86ccbaf2ddcfb7e709b226ac766803a)
 2015-11-24 22:10:31 +09:00
Karthik Kambatla
49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20bd1446164df1d3920c46c96dad750ae)
 2015-11-24 13:52:29 +05:30
Jian He
b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a12165ae5a8b80a2a4596c133471ebc1)
 2015-11-23 17:20:24 -08:00
Jason Lowe
9d303edbfb YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 
(cherry picked from commit d36b6e045f317c94e97cb41a163aa974d161a404)
 2015-11-23 20:31:50 +00:00
Arun Suresh
cbbdbe3cdd YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 
(cherry picked from commit da1016365aba1cee9c06771ab142d077379f27af)
 2015-11-21 10:01:10 -08:00
Wangda Tan
cdca460ad4 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 
(cherry picked from commit 2346fa3141bf28f25a90b6a426a1d3a3982e464f)
 2015-11-20 15:56:17 -08:00
Jason Lowe
f3bca8bc1a YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 
(cherry picked from commit 060cdcbe5db56383e74e282ac61eedff8248c11b)
 2015-11-20 23:13:27 +00:00
Xuan
be65d319fa YARN-2859.addendum: fix the remaining issue from the previous patch 
(cherry picked from commit f114e728da6e19f3d35ff0cfef9fceea26aa5d46)
 2015-11-19 10:09:48 -08:00
Arun Suresh
d8e60080e9 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 
(cherry picked from commit 6a61928fb790d553c942641761ea963b5e996ef6)
 2015-11-17 15:51:59 -08:00
Jian He
2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200cf02a49ab81fa3c71cb707f6d2fee1.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Junping Du
1d107d8058 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b6115e2cfbd97a94d6c1a3ddf0e94bb)
 2015-11-15 04:44:59 -08:00
Wangda Tan
13f8c7093b YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 
(cherry picked from commit 7f55a18071ebe1c5591ff8a410b227c855c3f428)
 2015-11-12 11:24:04 -08:00
Wangda Tan
0c1fb15b29 YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 
(cherry picked from commit 796638d9bc86235b9f3e5d1a3a9a25bbf5c04d1c)
 2015-11-12 11:12:11 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
bb5dd7f772 YARN-4183. Reverting the patch to fix behaviour change. 
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit 42bc525acb78088ed952e92aa49408413937de86.
 2015-11-11 10:42:52 -08:00
Jian He
a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7f7b665f6f55af54c8ebf330118ff37)
 2015-11-09 11:05:12 -08:00
Tsuyoshi Ozawa
1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739512804da38cf87e0ce1059e3128da3)
 2015-10-30 18:00:35 +09:00
Jian He
2b197bbffd YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 
(cherry picked from commit e5b1733e049dc0f1859b93618354e049a0efdc4a)
 2015-10-29 15:43:26 -07:00
Jonathan Eagles
42bc525acb YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 
(cherry picked from commit c293c58954cdab25c8c69418b0e839883b563fa4)
 2015-10-29 16:42:49 -05:00
Arun Suresh
8943abfb08 YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 
(cherry picked from commit 58d1df585c2b8a4d2f78ae30726f07b4c6a94731)
 2015-10-29 13:43:43 -07:00
Jason Lowe
c75d8b164f YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 
(cherry picked from commit e2267de2076245bd9857f6a30e3c731df017fef8)
 2015-10-29 16:39:55 +00:00
Rohith Sharma K S
e3fcb711d6 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 
(cherry picked from commit 656c8f952738f469766ede729dc0f25a3ce4214e)
 2015-10-29 12:25:08 +05:30