Commit Graph

2560 Commits

Author SHA1 Message Date
Steve Loughran 65f395226b HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:43:35 +00:00
Arun Suresh 742632e346 YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 2015-12-05 21:26:16 -08:00
Jian He e84d6ca2df Add missing file for YARN-4419 2015-12-03 19:07:28 -08:00
Jian He 755dda8dd8 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 2015-12-03 17:45:31 -08:00
Wangda Tan a2c3bfc8c1 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 2015-12-03 14:28:32 -08:00
Jian He 9f77ccad73 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2015-12-03 12:48:50 -08:00
Junping Du 62e9348bc1 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 2015-12-03 06:36:37 -08:00
Jian He 6b9a5beb2b YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 2015-12-02 11:07:18 -08:00
Junping Du 028894281b Creating 2.6.4 entries in CHANGES.txt files. 2015-12-01 08:16:39 -08:00
Vinod Kumar Vavilapalli d57fd181c7 Adding release 2.8.0 to CHANGES.txt 2015-11-25 17:33:26 -08:00
Tsuyoshi Ozawa 0656d2dc83 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 2015-11-26 01:10:02 +09:00
Wangda Tan 23c625ec57 YARN-4384. updateNodeResource CLI should not accept negative values for resource. (Junping Du via wangda) 2015-11-24 16:35:56 -08:00
Jason Lowe f5acf94eca YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 2015-11-24 23:47:10 +00:00
Jason Lowe 4ac6799d4a YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 2015-11-24 22:35:37 +00:00
Tsuyoshi Ozawa 28dfe721b8 YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 2015-11-24 19:24:01 +09:00
Akira Ajisaka 8c0133f3d4 YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G. 2015-11-24 18:40:33 +09:00
Tsuyoshi Ozawa fb0f09e46b CHANGES.txt: add YARN-4367 to 2.8.0. 2015-11-24 17:39:34 +09:00
Karthik Kambatla 52948bb20b YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 2015-11-24 13:47:17 +05:30
Jian He 8676a118a1 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 2015-11-23 17:19:48 -08:00
Jason Lowe aa2009c37b Update CHANGES.txt for recent commits to branch-2.6. 2015-11-23 21:57:18 +00:00
Jason Lowe d36b6e045f YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 2015-11-23 20:30:26 +00:00
Arun Suresh da1016365a YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 2015-11-21 09:59:41 -08:00
Wangda Tan a30eccb38c move YARN-4326/YARN-3849 from 2.8.0 to 2.7.3
(cherry picked from commit 08727287de)
2015-11-20 16:42:42 -08:00
Wangda Tan 2346fa3141 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 2015-11-20 15:55:50 -08:00
Jason Lowe 060cdcbe5d YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 2015-11-20 23:12:29 +00:00
Xuan 866dce4e8e YARN-2859 addendum: move the entry from 2.6.2 to 2.6.3 in hadoop-yarn/CHANGES.txt. 2015-11-19 10:14:18 -08:00
Xuan f114e728da YARN-2859.addendum: fix the remaining issue from the previous patch 2015-11-19 10:08:38 -08:00
Wangda Tan 23a130abd7 move fix version of YARN-4287 from 2.8.0 to 2.7.3
(cherry picked from commit 3504d0e795)
2015-11-18 14:01:49 -08:00
Steve Loughran 253e0404f3 YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:39 +00:00
Arun Suresh 6a61928fb7 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 2015-11-17 15:50:34 -08:00
Jian He fcd7888029 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan"
This reverts commit 8fbea531d7.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-11-16 20:18:44 -08:00
Junping Du 855d52927b YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 2015-11-15 04:43:57 -08:00
Wangda Tan 7ff280fca9 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 2015-11-12 11:50:00 -08:00
Wangda Tan 7f55a18071 YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 2015-11-12 11:23:40 -08:00
Wangda Tan 796638d9bc YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 2015-11-12 11:09:37 -08:00
Jason Lowe 06a70311b9 YARN-4345. yarn rmadmin -updateNodeResource doesn't work. Contributed by Junping Du 2015-11-12 14:49:06 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 6351d3fa63 YARN-4183. Reverting the patch to fix behaviour change.
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit c293c58954.
2015-11-11 10:40:43 -08:00
Akira Ajisaka 23d0db551c YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. 2015-11-11 16:13:08 +09:00
Wangda Tan c99925d6dd YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda) 2015-11-10 11:45:46 -08:00
Jian He 8fbea531d7 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 2015-11-09 10:43:45 -08:00
Allen Wittenauer b76b0ce51e HADOOP-12366. expose calculated paths (aw) 2015-11-07 08:32:56 -08:00
Jason Lowe 0383a3973b Add 2.7.3 release to CHANGES.txt 2015-11-04 16:26:14 +00:00
Varun Vasudev 73b9c7b82b HADOOP-10787. Rename/remove non-HADOOP_*, etc from the shell scripts. Contributed by Allen Wittenauer. 2015-11-04 15:56:17 +05:30
Wangda Tan 0783184f4b YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 2015-11-03 11:18:34 -08:00
Sangjin Lee a4a6b5b4b4 Updated the 2.6.2 final release date. 2015-10-30 18:47:16 -07:00
Jason Lowe eadf7b3096 Creating 2.6.3 entries in CHANGES.txt files. 2015-10-30 14:50:50 +00:00
Tsuyoshi Ozawa ce31b22739 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 2015-10-30 18:00:20 +09:00
Tsuyoshi Ozawa d21214ce33 Add an entry of YARN-4312 to CHANGES.txt 2015-10-30 17:56:59 +09:00
Tsuyoshi Ozawa d2e01f4ed8 Move YARN-3580 in CHANGES.txt from 2.8 to 2.7.2. 2015-10-30 15:49:06 +09:00
Xuan 7412ff48ee YARN-4313. Race condition in MiniMRYarnCluster when getting history
server address. Contributed by Jian He
2015-10-29 17:36:36 -07:00
Jian He e5b1733e04 YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 2015-10-29 15:42:57 -07:00
Jonathan Eagles c293c58954 YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 2015-10-29 16:41:10 -05:00
Arun Suresh 58d1df585c YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 2015-10-29 13:42:09 -07:00
Jason Lowe e2267de207 YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 2015-10-29 16:34:25 +00:00
Jian He c41699965e YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du 2015-10-29 00:00:48 -07:00
Rohith Sharma K S 656c8f9527 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 2015-10-29 12:22:44 +05:30
Sangjin Lee b0c818b60d Set the release date for 2.6.2 2015-10-28 20:43:35 -07:00
Xuan 27414dac66 YARN-2859. ApplicationHistoryServer binds to default port 8188 in
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli
2015-10-28 10:32:16 -07:00
Tsuyoshi Ozawa 9f4dfdf4eb YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing. Contributed by Brahma Reddy Battula. 2015-10-28 17:39:12 +09:00
Varun Vasudev c28e16b40c YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt. 2015-10-27 20:56:00 +05:30
Tsuyoshi Ozawa 96677bef00 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula. 2015-10-27 12:35:01 +09:00
Akira Ajisaka 8a68630dd1 YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka) 2015-10-27 10:28:01 +09:00
Wangda Tan 56e4f6237a YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda) 2015-10-26 16:44:39 -07:00
Wangda Tan 6f606214e7 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 2015-10-26 16:36:34 -07:00
Wangda Tan 3cc73773eb YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 2015-10-26 13:07:39 -07:00
Jason Lowe 33a03af3c3 YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee 2015-10-26 19:53:03 +00:00
Rohith Sharma K S 5acdde4744 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 2015-10-26 15:42:42 +05:30
Rohith Sharma K S 092883b34a YARN-4223. Fixed findbugs warnings in hadoop-yarn-server-nodemanager project. (varun saxena via rohithsharmaks) 2015-10-26 15:23:51 +05:30
Tsuyoshi Ozawa ce60b4fc8b YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula. 2015-10-26 16:45:11 +09:00
Rohith Sharma K S b57f08c0d2 YARN-4246. NPE while listing app attempt. (Nijel S F via rohithsharmaks) 2015-10-26 11:56:36 +05:30
Allen Wittenauer 1aa735c188 YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw) 2015-10-25 21:43:23 -07:00
Arun Suresh ab8eb8770c YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh) 2015-10-24 22:53:10 -07:00
Steve Loughran 446212a39e YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via stevel) 2015-10-24 12:55:36 +01:00
Steve Loughran 14f8dd0423 YARN-4289. TestDistributedShell failing with bind exception. (Brahma Reddy Battula via stevel) 2015-10-24 12:48:15 +01:00
Akira Ajisaka 7781fe1b9e YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka) 2015-10-24 11:54:42 +09:00
Jason Lowe d3a34a4f38 YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G 2015-10-23 20:57:01 +00:00
Ming Ma 934d96a334 YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma) 2015-10-23 08:36:33 -07:00
Jonathan Eagles f8adeb712d YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 2015-10-23 10:34:08 -05:00
Junping Du 0fce5f9a49 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 2015-10-22 13:41:09 -07:00
Zhihai Xu 960201b79b YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong 2015-10-22 12:28:03 -07:00
Anubhav Dhoot 2798723a54 YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan. 2015-10-22 06:51:00 -07:00
Jonathan Eagles 25f8f801d1 YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) 2015-10-21 15:38:08 -05:00
Tsuyoshi Ozawa 14f0e2787c Moved CHANGES.txt entries to 2.6.2
YARN-2019, YARN-4092, YARN-4101, YARN-4087, YARN-3727, YARN-4005,
YARN-3780, YARN-3802, YARN-3896, YARN-3554, YARN-3194, YARN-3798
2015-10-21 23:23:07 +09:00
Arun Suresh 506d1b1dbc YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh) 2015-10-20 16:46:14 -07:00
Arun Suresh 7e2837f830 YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh) 2015-10-19 20:00:38 -07:00
Varun Vasudev 151eab2a12 YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana. 2015-10-19 20:56:01 +05:30
Varun Vasudev e39ae0e676 YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 2015-10-19 20:19:22 +05:30
Steve Loughran e2d59e2c7b YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel) 2015-10-18 12:03:57 +01:00
Sangjin Lee 58590fef49 YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee) 2015-10-16 16:53:20 -07:00
Jian He f9da5cdb2b YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 2015-10-16 15:26:27 -07:00
Wangda Tan 4337b263aa YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda) 2015-10-16 15:06:28 -07:00
Jian He cf23f2c2b5 YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena 2015-10-15 17:12:46 -07:00
Varun Vasudev 63020c54c1 YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 2015-10-15 13:41:20 +05:30
Steve Loughran dfa7848463 HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:47 +01:00
rohithsharmaks d6c8bad869 YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks) 2015-10-14 16:11:34 +05:30
Varun Vasudev da1ee078f9 YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana. 2015-10-14 15:25:21 +05:30
Varun Vasudev fb9c1519b3 YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana. 2015-10-14 15:18:08 +05:30
Varun Vasudev 8d59293089 YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana. 2015-10-14 14:58:10 +05:30
Steve Loughran c59af2fdf8 HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:49:28 +01:00
Varun Vasudev 5b6bae0094 YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana 2015-10-13 12:48:30 +05:30
Jian He 9849c8b386 YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding 2015-10-12 11:51:33 -07:00
Zhihai Xu 049c6e8dc0 YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong. 2015-10-12 00:14:25 -07:00
Devaraj K db93047881 YARN-3964. Support NodeLabelsProvider at Resource Manager side.
Contributed by Dian Fu.
2015-10-11 11:21:29 +05:30
Wangda Tan def374e666 YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda) 2015-10-09 16:38:59 -07:00
Karthik Kambatla 4aa9b3e75c MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 2015-10-09 07:37:39 -07:00
Jason Lowe a0bca2b5ad YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 2015-10-09 14:17:38 +00:00
Rohith Sharma K S 8f195387a4 YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks) 2015-10-09 10:09:26 +05:30
Jason Lowe 8d226225d0 YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu 2015-10-08 22:25:34 +00:00
Jason Lowe 0841940a21 Fixup CHANGES.txt to reflect recent commits to 2.7.2 and 2.6.2 2015-10-08 16:45:29 +00:00
Rohith Sharma K S 3793cbe4c3 YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks) 2015-10-07 10:12:14 +05:30
Rohith Sharma K S 9156fc60c6 YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks) 2015-10-07 09:34:59 +05:30
Wangda Tan 29a582ada0 YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda) 2015-10-06 11:56:04 -07:00
Wangda Tan 30ac69c6bd YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda) 2015-10-05 15:47:21 -07:00
Vinod Kumar Vavilapalli f1c19b9365 HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some Linux distributions after HADOOP-12317. Contributed by Wangda Tan. 2015-10-05 10:56:55 -07:00
Harsh J c918f7be5e HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 2015-10-03 18:37:58 +05:30
Xuan 8f08532bde YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 2015-10-02 18:50:47 -07:00
Jason Lowe fdf02d1f26 YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 2015-10-02 20:20:31 +00:00
Jason Lowe 854d25b0c3 YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 2015-09-30 14:59:44 +00:00
Karthik Kambatla a0b5a0a419 YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) 2015-09-29 07:55:34 -07:00
Anubhav Dhoot 9735afe967 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 2015-09-28 16:13:41 -07:00
Jason Lowe 9f53a95ff6 YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G 2015-09-28 22:55:20 +00:00
Anubhav Dhoot fb2e525c07 YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot) 2015-09-28 09:05:45 -07:00
Xuan d1b9b85244 YARN-3624. ApplicationHistoryServer should not reverse the order of the
filters it gets. Contributed by Mit Desai
2015-09-24 15:01:17 -07:00
Rohith Sharma K S a9aafad12b YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) 2015-09-24 12:13:22 +05:30
Rohith Sharma K S 8ed0d4b744 YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks) 2015-09-24 11:24:14 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 1f707ecffd Release process for 2.6.1: Set the release date for 2.6.1 2015-09-23 15:10:50 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) df52ff92ae YARN-3248. Moving it to improvements section in CHANGES.txt. 2015-09-23 15:01:29 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) f3e5bc6766 CHANGES.txt: Moving YARN-1884, YARN-3171, YARN-3740, YARN-3248, YARN-3544 to 2.6.1 given the backport. 2015-09-23 13:40:37 -07:00
Jian He b3f6b641dc YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan 2015-09-23 13:29:38 -07:00
Jian He 89cab1ba5f YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 2015-09-23 13:29:38 -07:00
Jian He c57eac5dfe YARN-3868. Recovery support for container resizing. Contributed by Meng Ding 2015-09-23 13:29:38 -07:00
Jian He c3dc1af072 YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:37 -07:00
Jian He c59ae4eeb1 YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan 2015-09-23 13:29:37 -07:00
Jian He 5f5a968d65 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:37 -07:00
Jian He ffd820c27a YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan 2015-09-23 13:29:37 -07:00
Jian He 83a18add10 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 2015-09-23 13:29:36 -07:00
Jian He dfe2cb849f YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:36 -07:00
Jason Lowe 692d51c09d YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is enabled. Contributed by Mit Desai 2015-09-23 16:38:55 +00:00
Jason Lowe c890c51a91 YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu 2015-09-23 15:42:01 +00:00
Tsuyoshi Ozawa dfd807afab HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 2015-09-22 12:54:29 +09:00
Chris Douglas 8e01b0d97a YARN-4188. Make MoveApplicationAcrossQueues abstract, newInstance static
Contributed by Giovanni Matteo Fumarola
2015-09-21 17:51:48 -07:00
Wangda Tan b00392dd9c YARN-4113. RM should respect retry-interval when uses RetryPolicies.RETRY_FOREVER. (Sunil G via wangda) 2015-09-21 11:04:03 -07:00
Rohith Sharma K S c9cb6a5960 YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks) 2015-09-21 09:59:30 +05:30
Arun Suresh 94dec5a916 YARN-3920. FairScheduler container reservation on a node should be configurable to limit it to large containers (adhoot via asuresh) 2015-09-18 14:02:55 -07:00
Wangda Tan 372ad270a0 MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob. (Junping Du via wangda) 2015-09-18 10:10:32 -07:00
Wangda Tan 9bc913a35c YARN-3212. RMNode State Transition Update with DECOMMISSIONING state. (Junping Du via wangda) 2015-09-18 10:04:17 -07:00
Steve Loughran a7201d635f YARN-2597 MiniYARNCluster should propagate reason for AHS not starting 2015-09-18 09:45:35 +01:00
Rohith Sharma K S 723c31d45b YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.(Nijel S F via rohithsharmaks) 2015-09-18 08:44:10 +05:30
Jian He 6c6e734f0b YARN-4034. Render cluster Max Priority in scheduler metrics in RM web UI. Contributed by Rohith Sharma K S 2015-09-17 14:55:50 +08:00
Steve Loughran bf2f2b4fc4 YARN-3433. Jersey tests failing with Port in Use -again. (Brahma Reddy Battula) 2015-09-16 10:31:02 +01:00
Jian He 452079af8b YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface. Contributed by Naganarasimha G R 2015-09-16 14:59:20 +08:00
Xuan 34ef1a092b YARN-4149. yarn logs -am should provide an option to fetch all the log
files. Contributed by Varun Vasudev
2015-09-15 14:36:30 -07:00
Jason Lowe 8c1cdb17a0 YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu 2015-09-15 20:21:33 +00:00
Wangda Tan ae5308fe1d YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda) 2015-09-15 11:40:50 -07:00
Junping Du 73e3a49eb0 YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du) 2015-09-15 07:56:47 -07:00
Jian He 5468baa80a YARN-3635. Refactored current queue mapping implementation in CapacityScheduler to use a generic PlacementManager framework. Contributed by Wangda Tan 2015-09-15 15:39:20 +08:00
Wangda Tan e2a0270217 YARN-4151. Fix findbugs errors in hadoop-yarn-server-common module. (Meng Ding via wangda) 2015-09-14 16:00:43 -07:00
Jian He e1b1d7e4ae YARN-4126. RM should not issue delegation tokens in unsecure mode. Contributed by Bibin A Chundatt 2015-09-14 14:09:19 +08:00
Karthik Kambatla 332b520a48 YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 2015-09-13 18:07:43 -07:00
Karthik Kambatla 81df7b586a YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha) 2015-09-13 17:03:15 -07:00
Steve Loughran 7269906254 HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:26 +01:00
Robert Kanter ea4bb2749f YARN-4145. Make RMHATestBase abstract so its not run when running all tests under that namespace (adhoot via rkanter) 2015-09-11 11:46:10 -07:00
Robert Kanter b84fb41bb6 YARN-4115. Reduce loglevel of ContainerManagementProtocolProxy to Debug (adhoot via rkanter) 2015-09-11 11:46:10 -07:00
Varun Vasudev 486d5cb803 YARN-4136. LinuxContainerExecutor loses info when forwarding ResourceHandlerException. Contributed by Bibin A Chundatt. 2015-09-11 14:37:48 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) fbbb7ff1ed Updating all CHANGES.txt files to move entires from future releases into 2.6.1
section given the large number of backports to 2.6.1.
2015-09-10 13:45:57 -07:00
Wangda Tan 77666105b4 YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) 2015-09-10 09:30:09 -07:00
Robert Kanter 6dd6ca442a YARN-4086. Allow Aggregated Log readers to handle HAR files (rkanter) 2015-09-09 18:03:04 -07:00
Zhihai Xu 16b9037dc1 YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe. 2015-09-08 12:29:54 -07:00
Varun Vasudev 435f935ba7 YARN-4121. Fix typos in capacity scheduler documentation. Contributed by Kai Sasaki. 2015-09-08 12:33:43 +05:30
Jian He 6f72f1e600 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 2015-09-08 09:35:46 +08:00
Xuan 9b78e6e33d YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when
state-store error occurs. Contributed by Jian He
2015-09-07 17:45:47 -07:00
Xuan 6a50689705 Modify the CHANGES.txt to move YARN-2019 Commit log from branch 2.8 to
branch 2.7.2
2015-09-07 17:27:53 -07:00
Varun Vasudev 1dbd8e34a7 YARN-3591. Resource localization on a bad disk causes subsequent containers failure. Contributed by Lavkesh Lahngir. 2015-09-07 11:32:12 +05:30
Wangda Tan bcc85e3bab YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda) 2015-09-04 15:13:53 -07:00
Jason Lowe 6eaca2e363 YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li 2015-09-04 15:30:53 +00:00
Varun Vasudev 40d222e862 YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles. 2015-09-04 14:31:51 +05:30
Varun Vasudev b469ac531a YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R. 2015-09-03 16:40:10 +05:30
Jian He 09c64ba1ba YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong 2015-09-02 17:45:23 -07:00
Rohith Sharma K S 7d6687fe76 YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks) 2015-09-02 15:22:48 +05:30
Rohith Sharma K S 095ab9ab5f YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks) 2015-09-02 14:13:33 +05:30
Vinod Kumar Vavilapalli 4620767156 YARN-3222. Added the missing CHANGES.txt entry. 2015-09-01 13:43:10 -07:00
Andrew Wang 0eb9b19325 HADOOP-12369. Point hadoop-project/pom.xml java.security.krb5.conf within target folder. 2015-09-01 10:57:32 -07:00
Tsuyoshi Ozawa faa38e1aa4 YARN-2801. Add documentation for node labels feature. Contributed by Wangda Tan and Naganarasimha G R. 2015-09-01 17:54:49 +09:00
Varun Vasudev bf669b6d9f YARN-4082. Container shouldn't be killed when node's label updated. Contributed by Wangda Tan. 2015-09-01 14:19:11 +05:30
Jian He 4eaa7fd3ea Move YARN-4092 to 2.7.2 2015-08-31 17:43:36 -07:00
Jian He a3fd2ccc86 YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong 2015-08-31 17:33:24 -07:00
Vinod Kumar Vavilapalli cf831565e8 YARN-2945. Fixing the CHANGES.txt to have the right JIRA number. 2015-08-30 20:01:47 -07:00
Junping Du beb65c9465 YARN-1556. NPE getting application report with a null appId. Contributed by Weiwei Yang. 2015-08-28 05:57:34 -07:00
Jian He a9c8ea71aa YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S 2015-08-27 13:25:53 -07:00
Andrew Wang 90fe7bcc3b HADOOP-12362. Set hadoop.tmp.dir and hadoop.log.dir in pom. Contributed by Charlie Helin. 2015-08-27 10:17:00 -07:00
Jian He 57c7ae1aff YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 2015-08-24 20:36:44 -07:00
Rohith Sharma K S feaf034994 YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks) 2015-08-24 11:25:07 +05:30
Xuan b71c6006f5 YARN-221. Addendum patch to compilation issue which is caused by missing
AllContainerLogAggregationPolicy. Contributed by Xuan Gong
2015-08-23 16:46:30 -07:00
Xuan 37e1c3d82a YARN-221. NM should provide a way for AM to tell it not to aggregate
logs. Contributed by Ming Ma
2015-08-22 16:25:24 -07:00
Rohith Sharma K S 22de7c1dca YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface 2015-08-21 10:51:11 +05:30
Vinod Kumar Vavilapalli 33474939a0 Creating 2.6.2 entries in CHANGES.txt files. 2015-08-20 17:21:06 -07:00
Wangda Tan fc07464d1a YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 2015-08-20 11:51:03 -07:00
Xuan 22dc5fc209 YARN-4028. AppBlock page key update and diagnostics value null on
recovery. Contributed by Bibin A Chundatt
2015-08-18 22:53:03 -07:00
Zhihai Xu 14215c8ef8 YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong. 2015-08-18 11:36:02 -07:00
Zhihai Xu 3a76a010b8 YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao. 2015-08-18 10:36:40 -07:00
Karthik Kambatla 13604bd5f1 YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha) 2015-08-16 15:08:53 -07:00
Karthik Kambatla def12933b3 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 2015-08-16 06:24:16 -07:00
Jian He 0a030546e2 YARN-3987. Send AM container completed msg to NM once AM finishes. Contributed by sandflee 2015-08-13 16:22:53 -07:00
Jian He 7a445fcfab YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 2015-08-13 16:02:57 -07:00
Jian He 38aed1a94e YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 2015-08-13 14:46:08 -07:00
Robert Kanter dc2340c60e YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and WebAppProxyServer (djp via rkanter) 2015-08-12 15:47:47 -07:00
Jian He e5003be907 YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan 2015-08-12 15:07:50 -07:00
rohithsharmaks 1c12adb71f YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 2015-08-12 14:45:41 +05:30
Xuan 3ae716fa69 YARN-3999. RM hangs on draing events. Contributed by Jian He 2015-08-11 18:25:11 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 7c796fd12c Adding release 2.6.1 to CHANGES.txt 2015-08-11 13:20:23 -07:00
Jian He fa1d84ae27 YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G 2015-08-10 20:51:54 -07:00
Wangda Tan cf9d3c9256 YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy. (Sunil G via wangda) 2015-08-10 14:54:55 -07:00
Wangda Tan 4bc42d76e7 YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) 2015-08-07 09:46:57 -07:00
Rohith Sharma K S b6265d39c5 YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks) 2015-08-07 10:43:41 +05:30
Carlo Curino 8572a5a14b YARN-3974. Refactor the reservation system test cases to use parameterized base test. (subru via curino) 2015-08-02 01:55:31 -07:00
Junping Du cfee02b3bd YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter. 2015-08-06 06:49:45 -07:00
Arun Suresh 154c9d2e42 YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh) 2015-08-05 23:14:14 -07:00
rohithsharmaks df9e7280db YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) 2015-08-06 10:43:37 +05:30
Jian He ba2313d614 YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend container allocation logic. Contributed by Wangda Tan 2015-08-05 13:47:40 -07:00
Arun Suresh f271d37735 YARN-3736. Add RMStateStore apis to store and load accepted reservations for failover (adhoot via asuresh) 2015-08-05 12:57:12 -07:00
Xuan c3364ca8e7 YARN-4004. container-executor should print output of docker logs if the
docker container exits with non-0 exit status. Contributed by Varun
Vasudev
2015-08-03 18:10:11 -07:00
Xuan 0306d902f5 YARN-3543. ApplicationReport should be able to tell whether the
Application is AM managed or not. Contributed by Rohith Sharma K S
2015-08-03 15:46:00 -07:00
Jason Lowe 469cfcd695 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 2015-08-03 15:53:32 +00:00
Jonathan Eagles 3cd02b9522 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 2015-08-03 10:38:05 -05:00
Allen Wittenauer 666cafca8d HADOOP-12249. pull argument parsing into a function (aw) 2015-07-31 14:32:21 -07:00
Jason Lowe 32e490b6c0 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 2015-07-31 17:37:24 +00:00
Zhihai Xu ab80e27703 YARN-433. When RM is catching up with node updates then it should not expire acquired containers. Contributed by Xuan Gong 2015-07-30 21:57:11 -07:00
Wangda Tan 91b42e7d6e YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) 2015-07-30 10:00:31 -07:00
Wangda Tan 8acb30b016 YARN-3963. AddNodeLabel on duplicate label addition shows success. (Bibin A Chundatt via wangda) 2015-07-30 09:42:55 -07:00
rohithsharmaks c020b62cf8 YARN-3919. NPEs' while stopping service after exception during CommonNodeLabelsManager#start. (varun saxena via rohithsharmaks) 2015-07-29 23:30:47 +05:30
Karthik Kambatla 5205a330b3 YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha) 2015-07-29 09:42:32 -07:00
Jason Lowe 2b2bd92146 YARN-3950. Add unique SHELL_ID environment variable to DistributedShell. Contributed by Robert Kanter 2015-07-29 15:16:40 +00:00
Xuan f170934215 YARN-3982. container-executor parsing of container-executor.cfg broken
in trunk and branch-2. Contributed by Varun Vasudev
2015-07-27 23:45:58 -07:00
Jian He 3572ebd738 YARN-3846. RM Web UI queue filter is not working for sub queue. Contributed by Mohammad Shahid Khan 2015-07-27 17:12:05 -07:00
Varun Vasudev 3e6fce91a4 YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 2015-07-27 11:57:40 -07:00
Varun Vasudev f36835ff9b YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab. 2015-07-27 10:14:51 -07:00
Akira Ajisaka 42d4e0ae99 YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api module. Contributed by Varun Saxena. 2015-07-27 11:43:25 +09:00
ccurino 156f24ead0 YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations. (Jonathan Yaniv and Ishai Menache via curino) 2015-07-25 07:39:47 -07:00
Wangda Tan a3bd7b4a59 YARN-3973. Recent changes to application priority management break reservation system from YARN-1051 (Carlo Curino via wangda) 2015-07-24 16:44:18 -07:00
Jason Lowe ff9c13e0a7 YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 2015-07-24 22:14:39 +00:00
Jian He 83fe34ac08 YARN-3026. Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp. Contributed by Wangda Tan 2015-07-24 14:00:25 -07:00
carlo curino fc42fa8ae3 YARN-3969. Updating CHANGES.txt to reflect the correct set of branches where this is committed 2015-07-24 13:38:44 -07:00
Karthik Kambatla d19d187753 YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500. (Anubhav Dhoot via kasha) 2015-07-24 11:44:37 -07:00
Xuan fbd6063269 YARN-3967. Fetch the application report from the AHS if the RM does not
know about it. Contributed by Mit Desai
2015-07-24 10:15:54 -07:00
carlo curino 0fcb4a8cf2 YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 2015-07-23 19:33:59 -07:00
Rohith Sharma K S e202efaf93 YARN-3845. Scheduler page does not render RGBA color combinations in IE11. (Contributed by Mohammad Shahid Khan) 2015-07-24 12:43:06 +05:30
Robert Kanter 1d3026e7b3 YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos that include it (adhoot via rkanter) 2015-07-23 14:46:54 -07:00
Wangda Tan 3bba180051 YARN-3941. Proportional Preemption policy should try to avoid sending duplicate PREEMPT_CONTAINER event to scheduler. (Sunil G via wangda) 2015-07-23 10:07:57 -07:00
Junping Du ee98d6354b YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 2015-07-22 17:52:35 -07:00
Wangda Tan 06e5dd2c84 YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda) 2015-07-22 12:01:41 -07:00