Commit Graph

2419 Commits

Author SHA1 Message Date
Jian He 9b11e7bd6d YARN-4098. Document ApplicationPriority feature. Contributed by Rohith Sharma K S 2015-12-29 16:52:30 -08:00
Wangda Tan c7fe0bac77 YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda)
(cherry picked from commit 561abb9fee)
(cherry picked from commit 3efbde5378)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-29 13:31:43 -08:00
Junping Du ce74b17028 YARN-4400. AsyncDispatcher.waitForDrained should be final. Contributed by Daniel Templeton.
(cherry picked from commit bb5df272b9)
(cherry picked from commit dbc4c79b44)
2015-12-23 08:56:37 -08:00
Junping Du 630b637ff3 YARN-4234. New put APIs in TimelineClient for ats v1.5. Contributed by Xuan Gong.
(cherry picked from commit 882f2f0464)
(cherry picked from commit 6e97a3c968)
2015-12-23 05:31:16 -08:00
Arun Suresh 546178a5de YARN-4477. FairScheduler: Handle condition which can result in an infinite loop in attemptScheduling. (Tao Jie via asuresh)
(cherry picked from commit e88422df45)
2015-12-21 22:42:42 -08:00
cnauroth eb5f46e34f YARN-3458. CPU resource monitoring in Windows. Contributed by Inigo Goiri.
(cherry picked from commit 114b590955)
(cherry picked from commit fdfcffb3d1)
2015-12-21 16:59:24 -08:00
Wangda Tan e507ed1f90 YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports information. (Sunil G via wangda)
(cherry picked from commit a0de702851)
2015-12-21 11:38:15 -08:00
Wangda Tan 01c813d210 YARN-4454. NM to nodelabel mapping going wrong after RM restart. (Bibin A Chundatt via wangda)
(cherry picked from commit bc038b382c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-21 11:31:54 -08:00
Uma Mahesh 6f30919336 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh)
(cherry picked from commit 0f82b5d878)
2015-12-19 23:28:51 -08:00
Wangda Tan b74391ce14 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda)
(cherry picked from commit 9b856d9787)
(cherry picked from commit 5060fd1840)
2015-12-18 16:34:45 -08:00
Wangda Tan c4b1386b4e YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda)
(cherry picked from commit 7faa406f27)
(cherry picked from commit 2f5435acbd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-18 16:34:45 -08:00
Wangda Tan 9ed9eb265f YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda)
(cherry picked from commit 79c41b1d83)
(cherry picked from commit 9f4a6ec663)
2015-12-18 16:34:45 -08:00
Zhihai Xu e4f638d815 YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun
(cherry picked from commit 2aaed10327)
(cherry picked from commit be50df687b)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-18 16:34:45 -08:00
Wangda Tan 1d1c6c1f8d YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda)
(cherry picked from commit 07b0fb996a)
(cherry picked from commit c915a658c8)
2015-12-18 16:34:44 -08:00
Wangda Tan c7e10d70de YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda)
(cherry picked from commit dfcbbddb09)
(cherry picked from commit 9f979107c2)
2015-12-18 16:34:44 -08:00
Wangda Tan 8ee83765f4 YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda)
(cherry picked from commit 6cb0af3c39)
(cherry picked from commit a1a723fdff)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-18 16:34:44 -08:00
Xuan 41986d5373 YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml
(cherry picked from commit 21daa6c68a)
(cherry picked from commit 455c6f39c5)
2015-12-18 16:34:43 -08:00
Wangda Tan a4ba6c8525 YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda)
(cherry picked from commit 7e4715186d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-18 16:34:43 -08:00
Jonathan Eagles c4ae007fd4 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles)
(cherry picked from commit 4ff973f96a)
(cherry picked from commit 7c604c3d06)
2015-12-18 16:34:43 -08:00
Xuan 46cb47c03e YARN-4392. ApplicationCreatedEvent event time resets after RM
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)
(cherry picked from commit 7decba83ef)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-18 16:34:43 -08:00
Jian He 12772f5a45 Add missing file for YARN-4419
(cherry picked from commit 8a97ca4b00)
2015-12-18 16:34:42 -08:00
Jian He f355645ca4 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan
(cherry picked from commit 755dda8dd8)

(cherry picked from commit be404e1680)
2015-12-18 16:34:42 -08:00
Wangda Tan 15b3ff1971 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda)
(cherry picked from commit a2c3bfc8c1)
(cherry picked from commit 51e448d295)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-18 16:33:53 -08:00
Jian He 9c22910570 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding
(cherry picked from commit 48cbde9083)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-18 16:33:53 -08:00
Junping Du 8812be3dc1 YARN-3226. UI changes for decommissioning node. Contributed by Sunil G.
(cherry picked from commit 1de56b0448)
(cherry picked from commit 6d2914a697)
2015-12-17 15:22:37 -08:00
Junping Du 92de57bdca YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R.
(cherry picked from commit 50bd067e1d)
(cherry picked from commit 1a2ef845b5)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java
(cherry picked from commit 0f9dd48842)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2015-12-17 06:16:08 -08:00
Jason Lowe d939c23cc1 YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne
(cherry picked from commit 91828fef6b)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2015-12-16 23:24:51 +00:00
Siddharth Seth 7415870d13 YARN-4207. Add a non-judgemental YARN app completion status. Contributed by Rich Haase.
(cherry picked from commit 0f708d465f)
(cherry picked from commit 433212ef87)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-16 11:39:37 -08:00
Xuan f22d0046de YARN-4439. Clarify NMContainerStatus#toString method. Contributed by
Jian He

(cherry picked from commit d8a45425eb)
(cherry picked from commit 992f65dd8b)
2015-12-14 17:16:07 -08:00
Jian He bc7603e393 YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 15:06:13 -08:00
Jian He 16f7ac48d4 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:52:33 -08:00
Arun Suresh c2e788a7b5 YARN-4358 addendum patch to fix javadoc error
(cherry picked from commit 7fb212e5e6)
2015-12-12 22:26:35 -08:00
Junping Du 5adf7d851c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong.
(cherry picked from commit f910e4f639)
(cherry picked from commit 0b5e76e9ff)
2015-12-10 02:58:28 -08:00
Akira Ajisaka 59c892683f YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang.
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
(cherry picked from commit 925bb94d87db7e916ed380d4512225e796f9e03d)
2015-12-09 16:33:12 +09:00
rohithsharmaks 5bc33c5ae4 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks)
(cherry picked from commit a5e2e1ecb0)
2015-12-09 11:35:02 +05:30
rohithsharmaks 6a6bbc51c4 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 2015-12-09 11:06:31 +05:30
Chris Douglas 3d100163de YARN-4248. Followup patch adding asf-licence exclusions for json test files
(cherry picked from commit 9f50e13d5d)
(cherry picked from commit bfe4796802)
2015-12-08 12:08:53 -08:00
= 3205fe3da8 YARN-4248. REST API for submit/update/delete Reservations. (curino)
(cherry picked from commit c25a635459)
(cherry picked from commit 4ac1564418)
2015-12-07 13:46:37 -08:00
rohithsharmaks 03e615ba55 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks)
(cherry picked from commit 01a641bc44)
2015-12-07 15:20:50 +05:30
Arun Suresh db69c5f744 YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh)
(cherry picked from commit 742632e346)
2015-12-05 21:35:32 -08:00
Junping Du 6af5255b02 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter.
(cherry picked from commit 62e9348bc1)
(cherry picked from commit e76ba91fa5)
2015-12-03 06:39:07 -08:00
Junping Du 12fb8b4cc5 Add 2.6.4 as release entry to CHANGES.txt 2015-12-02 07:38:03 -08:00
Allen Wittenauer 81a7ad45c9 HADOOP-12366. expose calculated paths (aw) 2015-12-01 08:15:57 -08:00
Vinod Kumar Vavilapalli efaef45a24 Reverting previous commit which added 2.9.0 entries. Don't need that in branch-2.8.
Revert "Adding release 2.8.0 to CHANGES.txt"

This reverts commit 767815f317.
2015-11-25 17:49:48 -08:00
Vinod Kumar Vavilapalli 767815f317 Adding release 2.8.0 to CHANGES.txt
(cherry picked from commit d57fd181c7)
2015-11-25 17:37:47 -08:00
Tsuyoshi Ozawa d76b523b02 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena.
(cherry picked from commit 0656d2dc83)
2015-11-26 01:10:41 +09:00
Wangda Tan 7b9f7eaf5c YARN-4384. updateNodeResource CLI should not accept negative values for resource. (Junping Du via wangda)
(cherry picked from commit 23c625ec57)
2015-11-24 16:36:30 -08:00
Jason Lowe 3a2f5d6329 YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla
(cherry picked from commit f5acf94eca)
2015-11-24 23:47:54 +00:00
Jason Lowe 9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li
(cherry picked from commit 4ac6799d4a)
2015-11-24 22:36:19 +00:00
Tsuyoshi Ozawa 9e458c3e9e YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang.
(cherry picked from commit 28dfe721b8)
2015-11-24 22:10:31 +09:00
Akira Ajisaka ac0ddc4fec YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G.
(cherry picked from commit 8c0133f3d4)
2015-11-24 18:40:56 +09:00
Karthik Kambatla 49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha)
(cherry picked from commit 52948bb20b)
2015-11-24 13:52:29 +05:30
Jian He b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan
(cherry picked from commit 8676a118a1)
2015-11-23 17:20:24 -08:00
Jason Lowe d13b646c61 Update CHANGES.txt for recent commits to branch-2.6.
(cherry picked from commit aa2009c37b)
2015-11-23 21:58:57 +00:00
Jason Lowe 9d303edbfb YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev
(cherry picked from commit d36b6e045f)
2015-11-23 20:31:50 +00:00
Arun Suresh cbbdbe3cdd YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh)
(cherry picked from commit da1016365a)
2015-11-21 10:01:10 -08:00
Wangda Tan 08727287de move YARN-4326/YARN-3849 from 2.8.0 to 2.7.3 2015-11-20 16:42:11 -08:00
Wangda Tan cdca460ad4 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda)
(cherry picked from commit 2346fa3141)
2015-11-20 15:56:17 -08:00
Jason Lowe f3bca8bc1a YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li
(cherry picked from commit 060cdcbe5d)
2015-11-20 23:13:27 +00:00
Xuan 8347b1048d YARN-2859 addendum: move the entry from 2.6.2 to 2.6.3 in hadoop-yarn/CHANGES.txt.
(cherry picked from commit 866dce4e8e)
2015-11-19 10:17:20 -08:00
Xuan be65d319fa YARN-2859.addendum: fix the remaining issue from the previous patch
(cherry picked from commit f114e728da)
2015-11-19 10:09:48 -08:00
Wangda Tan 3504d0e795 move fix version of YARN-4287 from 2.8.0 to 2.7.3 2015-11-18 14:01:02 -08:00
Karthik Kambatla d3f9508096 YARN-4367. SLS webapp doesn't load. (kasha) 2015-11-18 13:45:00 -08:00
Steve Loughran a0b3e43a6d YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:24 +00:00
Arun Suresh d8e60080e9 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh)
(cherry picked from commit 6a61928fb7)
2015-11-17 15:51:59 -08:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan"
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-11-16 20:19:31 -08:00
Junping Du 1d107d8058 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe.
(cherry picked from commit 855d52927b)
2015-11-15 04:44:59 -08:00
Wangda Tan 875aec3177 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda)
(cherry picked from commit 7ff280fca9)
2015-11-12 11:50:41 -08:00
Wangda Tan 13f8c7093b YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda)
(cherry picked from commit 7f55a18071)
2015-11-12 11:24:04 -08:00
Wangda Tan 0c1fb15b29 YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda)
(cherry picked from commit 796638d9bc)
2015-11-12 11:12:11 -08:00
Jason Lowe 9050078005 YARN-4345. yarn rmadmin -updateNodeResource doesn't work. Contributed by Junping Du
(cherry picked from commit 06a70311b9)
2015-11-12 14:49:55 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) bb5dd7f772 YARN-4183. Reverting the patch to fix behaviour change.
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit 42bc525acb.
2015-11-11 10:42:52 -08:00
Akira Ajisaka f6f128c867 YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas.
(cherry picked from commit 23d0db551c)
2015-11-11 16:13:50 +09:00
Wangda Tan 8a8ac50a5b YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda)
(cherry picked from commit c99925d6dd)
2015-11-10 13:17:23 -08:00
Vinod Kumar Vavilapalli b742ea7fe6 HADOOP-12451. Setting HADOOP_HOME explicitly should be allowed. Fixing bugs in
the first patch committed. (Karthik Kambatla via vinodkv)
2015-11-10 12:16:52 -08:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan
(cherry picked from commit 8fbea531d7)
2015-11-09 11:05:12 -08:00
Jason Lowe 8334d86472 Add 2.7.3 release to CHANGES.txt
(cherry picked from commit 0383a3973b)
2015-11-04 16:27:28 +00:00
Wangda Tan 3190446df9 YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda)
(cherry picked from commit 0783184f4b)
2015-11-03 11:36:11 -08:00
Sangjin Lee b2d6ea5e18 Updated the 2.6.2 final release date.
(cherry picked from commit a4a6b5b4b4)
2015-10-30 18:49:21 -07:00
Jason Lowe 8dce482f61 Creating 2.6.3 entries in CHANGES.txt files.
(cherry picked from commit eadf7b3096)
2015-10-30 14:52:06 +00:00
Tsuyoshi Ozawa 1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena.
(cherry picked from commit ce31b22739)
2015-10-30 18:00:35 +09:00
Tsuyoshi Ozawa dd144035a0 Add an entry of YARN-4312 to CHANGES.txt
(cherry picked from commit d21214ce33)
2015-10-30 17:57:29 +09:00
Tsuyoshi Ozawa c522a9ffc2 Move YARN-3580 in CHANGES.txt from 2.8 to 2.7.2.
(cherry picked from commit d2e01f4ed8)
2015-10-30 15:51:04 +09:00
Xuan b505bf9e58 YARN-4313. Race condition in MiniMRYarnCluster when getting history
server address. Contributed by Jian He

(cherry picked from commit 7412ff48ee)
2015-10-29 17:37:24 -07:00
Jian He 2b197bbffd YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena
(cherry picked from commit e5b1733e04)
2015-10-29 15:43:26 -07:00
Jonathan Eagles 42bc525acb YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)
(cherry picked from commit c293c58954)
2015-10-29 16:42:49 -05:00
Arun Suresh 8943abfb08 YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh)
(cherry picked from commit 58d1df585c)
2015-10-29 13:43:43 -07:00
Jason Lowe c75d8b164f YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena
(cherry picked from commit e2267de207)
2015-10-29 16:39:55 +00:00
Jian He 7d109ef5b5 YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du
(cherry picked from commit c41699965e)
2015-10-29 00:01:20 -07:00
Rohith Sharma K S e3fcb711d6 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks)
(cherry picked from commit 656c8f9527)
2015-10-29 12:25:08 +05:30
Sangjin Lee 16db203a83 Set the release date for 2.6.2 2015-10-28 20:48:49 -07:00
Xuan 9ce5069d16 YARN-2859. ApplicationHistoryServer binds to default port 8188 in
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 27414dac66)
2015-10-28 10:33:56 -07:00
Tsuyoshi Ozawa d29bff86c4 YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing. Contributed by Brahma Reddy Battula.
(cherry picked from commit 9f4dfdf4eb)
2015-10-28 17:39:26 +09:00
Varun Vasudev c5bf1cb7af YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt.
(cherry picked from commit c28e16b40c)
2015-10-27 20:56:45 +05:30
Tsuyoshi Ozawa 3138b43f16 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula.
(cherry picked from commit 96677bef00)
2015-10-27 12:35:20 +09:00
Akira Ajisaka d9156c1245 YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka)
(cherry picked from commit 8a68630dd1)
2015-10-27 10:28:30 +09:00
Wangda Tan 1499af38eb YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda)
(cherry picked from commit 56e4f6237a)
2015-10-26 16:45:20 -07:00
Wangda Tan 9bf1e74171 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda)
(cherry picked from commit 6f606214e7)
2015-10-26 16:38:39 -07:00
Wangda Tan baa2998e93 YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda)
(cherry picked from commit 3cc73773eb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-10-26 13:12:56 -07:00
Jason Lowe 7d66e2e666 YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee
(cherry picked from commit 33a03af3c3)
2015-10-26 19:54:16 +00:00